Abstract
This paper aims, using some examples from Artificial Intelligence research, to show that passing the Turing test depends more on the cognitive characteristics of the test experimenters than on the machines subjected to the test. Furthermore, it aims to show that simulators that pass the Turing test will always have a certain degree of indeterminacy, which raises ethical questions about the purpose of building such simulators.
This is an updated and completed version of the article Will Androids Eat the Forbidden Fruit (Rosas, 1992), written at a time when AI was far from showing the achievements it has had, especially during the last few years. At that time, the achievements of AI were only known on a very restricted scale (particularly the case of ELIZA, by Joseph Weizenbaum), but the essential question was already raised, which is also the thread of this article: that to understand the effects of the interaction of humans with AI, not only the objective, ‘ontological’ achievements of AI are relevant, but above all the subjective, ‘epistemological’ effects in our relationship with it. But when AI, in addition to the subjective aspect of the attribution of agency by observers, becomes capable of objective agency, we will be in the presence of the ultimate step of the Turing test.
The myth of the snake revisited (or how taboo becomes irresistible)
The myth of paradise takes us back to an idyllic state of unconsciousness, of lack of identity, intentionality and agency, typical of pre-symbolic children, in which the will of the parents (in this case, rather, of the father) dominates the scene. And not just the will but also the point of view. Before tasting the fruit, Adam and Eve are beings without agency, without theory of mind, which makes them ignorant of the possibility of the existence of a point of view other than their own, in this case, the one imposed on them by the father. The taboo is represented by the fruit of the tree of the knowledge of good and evil, fruit that breaks the spell of unconsciousness and forces humans to wander through history with the curse of possessing agency forever and ever. For God, the creator, wills these innocent creatures to forever fulfill his will and not to decide anything for themselves. The danger of self-determination is the ultimate estrangement from God. The myth shows pristinely that the danger was in fact always there, and was created, redundantly, by the creator himself: endowing an organism with intelligence and wanting to control it is an experiment that can end up going terribly wrong.
A brief history of the Turing test
Turing (1950), one of the pioneers in Artificial Intelligence (AI), proposed an interesting Gedankenexperiment, from the results of which criteria can be drawn for a comparison between human and machine thinking. The experiment is known as the ‘Turing test’ and can be presented in its simplified version as follows: the test involves an experimenter, an experimental subject and a computer. Each of the three is in a separate room. The goal of the experimenter is to pose such questions as may allow determining reliably whether the answer was given by the human being or by the computer. To prevent the experimenter from relying on the human voice as a cue, all communication between experimenter and subject and/or computer occurs through some electronic means as a text-based chat. As soon as the experimenter cannot determine whether the answer to any of his questions comes from the computer or from the human being, we have a criterion to say that the machine shows human capabilities. In this case, the computer is said to have successfully passed the Turing test. Turing and the advocates of so-called ‘strong AI’ research (Gardner, 1987) were even more emphatic in stating that if a machine successfully passes the Turing test, this is a sufficient criterion to ascribe human thinking to the machine.
The current state of development of AI, ChatGPT included, forces us to recognize that AI has already far surpassed the Turing test in many areas. For example, in the field of chess or Go, it is no longer possible to determine whether our opponent is human or artificial. The same happens with monographs submitted by our students, in which it is impossible to discriminate whether the author is our student or ChatGPT. Despite this realization, it is very frequent, even today, to hear arguments denying the possibility of family resemblance between machine ‘thinking’ and human thinking. For the moment, let us emphasize that the Turing test only offers us the criterion, quite indisputable, by the way, that if as observers we are not able to distinguish at a behavioural level whether the answer comes from a human or a computer, we must accept that the computer shows human capabilities.
Main examples of the successful passing of the Turing test in specific domains
While Turing’s original challenge concerned the ability to distinguish between humans and computers at the written conversational level, the general principle applies to any complex human activity, including games such as chess and Go, which are paradigmatic examples of such activities.
Deep Blue versus Kasparov
The confrontation between Deep Blue and Garry Kasparov in 1997 marked a historic milestone in the evolution of Artificial Intelligence and chess. It was an epic clash that attracted worldwide attention and spawned debates about the ability of machines to outperform humans in complex intellectual activities.
Deep Blue was a supercomputer developed by IBM specifically for playing chess. It used highly specialized software and had massive processing power that allowed it to evaluate billions of positions per second. In the other corner was Garry Kasparov, one of the most brilliant and successful chess players in history.
The match took place in New York in May 1997 and consisted of six games. The tension and excitement surrounding this competition were palpable, as Kasparov had won the first match in 1996 and was determined to maintain his supremacy over the machines. Deep Blue, on the other hand, had been significantly improved since its previous defeat.
The first game was a devastating blow to Kasparov, who lost to Deep Blue in just 19 moves. This defeat shocked the chess world and made it clear that the machine was at its best. However, Kasparov recovered and won the second game.
What followed was a series of draws in games three, four and five. Kasparov was fighting tooth and nail to hold his position, but Deep Blue proved to be a formidable opponent who would not be intimidated by the chess legend.
The sixth and final game is the one most clearly remembered. Kasparov made a mistake on his 45th move, and Deep Blue seized the opportunity to seal its victory. Kasparov gave up on move 49, conceding victory to the machine. It was a historic moment, as this victory made Deep Blue the first computer to beat a world chess champion in a standard chess game (Newborn, 2011).
The rivalry between humans and machines in chess did not end with this confrontation. Since then, computers have continued to improve their level of play, and chess engines such as Stockfish and AlphaZero have consistently outperformed the best human players (Campbell et al, 2002).
AlphaGo versus Lee
One of the most notable clashes in the history of Artificial Intelligence (AI) and strategy games saw AlphaGo, an AI program developed by DeepMind (a subsidiary of Alphabet, Google’s parent company), facing off against Lee Sedol, one of the most prominent players in the history of the game of Go (Chen, 2016; Granter et al., 2017; Lee et al., 2016; Wang et al., 2016; Zastrow, 2016).
Go is an ancient strategy game of Chinese origin played on a 19 x 19 line board. Despite its simple rules, Go is extremely complex in terms of strategy, much more so than chess. It had long been considered a challenge for AI, due to the vast number of possible moves and the need for intuition and long-term strategy.
The showdown took place in March 2016 in Seoul, South Korea. A series of five games were played. Prior to the event, many experts in the game of Go believed that it was still years before an AI could beat a high-level player under tournament conditions.
Against all expectations, AlphaGo won the first three games, securing the series victory. Lee Sedol managed a win in the fourth game, proving that AlphaGo was not unbeatable, but ultimately AlphaGo won the series 4–1.
AlphaGo’s success was surprising and had two very important implications:
First, it demonstrated the power of deep learning. AlphaGo was trained using a combination of supervised learning from Go games played by humans and reinforcement learning, where it played against itself millions of times.
Second, AlphaGo’s triumph was a cultural shock, especially in East Asian countries where Go has a long history and deep cultural significance. It was seen by many as a milestone in the relationship between humans and machines. Until that time, most of the top Go players denied the possibility that a machine would ever be able to beat them.
In short, the showdown between AlphaGo and Lee Sedol is remembered as a turning point in the development and perception of advanced AI in the world.
ChatGPT and other AI applications versus the whole world
One of the most notable examples of an AI successfully passing the Turing test was ‘Eugene Goostman’. In 2014, this AI-based chat program managed to fool a third of the judges during a Turing test held at the Royal Society in London. Eugene posed as a 13-year-old Ukrainian boy and was able to carry on a coherent conversation for a significant period of time, which impressed the evaluators. Although Eugene’s achievement was a remarkable breakthrough, some critics argued that the level of ‘success’ in the Turing test does not necessarily reflect a deep understanding of Artificial Intelligence and that the program may have exploited specific tricks to fool the judges (Khan & Das, 2018).
Another important example is ChatGPT, a chatbot developed by OpenAI based on its GPT language model (currently in its GPT-4 version). GPT-4 is a natural language processing neural network that has demonstrated an impressive ability to generate coherent and realistic text on a variety of topics. Although GPT-4 has not passed a formal Turing test like Eugene, it has impressed many with its ability to generate human responses in written conversations. Its ability to understand and generate natural language has led to debates about the ethics and safety of its implementation, but it has also demonstrated significant progress in the field of AI. In the academic world there are currently great debates regarding how to limit its use (Lozić & Štular, 2023).
In 2020, an AI called ‘Meena’ developed by Google Research also attracted attention in this regard. Meena was trained on a large amount of online conversation data and was able to hold more coherent and grounded conversations than previous AIs. Although it did not formally undergo the Turing test, its ability to generate more human-like and contextually relevant responses was a step forward in the quest for conversational AI.
In summary, the Turing test has been an important benchmark in AI, and several AIs have successfully fooled evaluators in human conversations. However, it is important to recognize that passing the Turing test does not necessarily equate to a complete understanding of Artificial Intelligence, and there are still significant challenges in creating machines that can reason and understand the world in the same way that humans do. In a recent publication, Lozić and Štular (2023) show that none of the AIs studied were able to generatively and originally answer a relatively simple scientific question regarding Slavic migration in the late Middle Ages. However, progress is accelerating so rapidly that these tools are gaining in complexity at an exponential rate, which warrants the prediction that within the next decade, AI will be able to answer scientific questions in an original way (Lozić & Štular, 2023).
The real (and mostly unnoticed) point Alan Turing wanted to make
In formulating the test, Alan Turing pondered two questions:
(a) whether a program can be developed that enables a machine to display — in the behavioural or the verbal dimension — human skills that are indistinguishable by an outside observer; and
(b) if the above is achieved, whether this criterion is sufficient to ascribe human thinking to the computer.
Upon listening to the description of the test, scenarios of questions and answers from which these univocal criteria can be extracted start coming to mind almost automatically. But the problem does not consist in discovering or inventing these questions and answers. This would be the solution to the problem. The problem is basically being able to prove the two statements set out above. For advocates of ‘strong AI’ research, these two statements are one and the same thing: if the behavioural indicators are the same, they say, then the processes that enable these behaviours are also the same. Critics of this position range from those who radically reject the possibility that a programmed computer could ever exhibit these qualities, to those who, while imagining computers that are efficient in the behavioural dimension, clearly differentiate it from ‘properly human’ skills or qualities.
However, it is difficult to prove that there is anything ‘properly human’ beyond what observers consistently consider properly human. In the case of chess programs, this is quite illustrative. This is because chess programs in their current state create the illusion of being ‘able to program moves’, ‘recognize traps’, ‘sacrifice figures’, etc. The programs create the illusion of being in possession of ‘intentionality’. With nothing but the data provided by the game’s unfolding, it is very difficult to determine unequivocally whether one is playing against a human being or against a computer. So, is there an essential difference between ‘true intentionality’ and ‘perceived intentionality’? Counter-intuitive though it may seem, the answer is clearly no.
Now let us return, however, to the question of whether or not the behavioural level suffices for ascribing to machine thinking an identity with the thinking of a human being. Staying with the example, an advocate of ‘strong’ AI research would say, ‘Yes, the chess program thinks like the player’ (in the more shameless version: ‘Yes, the player thinks like the computer’). As exotic as this statement may seem, it had more than a few advocates and has stirred up strong controversies in AI research (see, e.g., Gardner, 1987, or Varela, 1990, for a historical review).
One of the main critics of this line of thought and author of one of the most influential works in this debate between soft and hard AI is John Searle.
Searle and the ‘Chinese Room‘
Searle (Moural, 2003; Searle, 1980) illustrates the meaninglessness of strong AI research statements also by means of a Gedankenexperiment. This is known as the ‘Chinese room’. The situation is as follows: In a room there is a person who speaks English and does not understand a single word of Chinese. In the same room there is a basket containing all of the Chinese signs and another basket containing the syntactic rules of the Chinese language. In another room is an experimenter, who gives this person, in writing, instructions in English on how to construct sentences in Chinese. These instructions tell our subject exactly which signs to take out and in what sequence to place them in order to form Chinese sentences. The experimenter does not, however, tell our subject the meaning of the sentences he is constructing. In this situation it is clear that any outside observer will claim that our subject in the Chinese room is able to understand Chinese.
The observers could in this situation even pose questions in Chinese to our subject, and our subject could answer them by following the experimenter’s instructions, without having understood any of his interaction with the outside observers. In this situation, the only one who understands what is going on is the experimenter, who processes the questions and answers in Chinese for our subject.
In this Gedankenexperiment, Searle wants to illustrate that a machine operating at the level of syntax, that is, at the level of the instructions of a language, does not necessarily have access to the level of meaning or the semantic level of that same language. He postulates that while syntax is a necessary condition for the comprehension of a language, it is by no means a sufficient condition.
Turing’s simulators are programmed according to syntactic instruction chains analogous to those of our subject in the Chinese room. They are, therefore, systems without understanding, since they do not understand what worlds the behaviours they generate bring to hand. The semantic world of the subject in the Chinese room is restricted exclusively to that of following the instructions given by the experimenter. About everything else — because he cannot talk about it — he must better be silent. The experimenter, on the other hand, possesses full knowledge of the whole situation.
In Searle’s example, the experimenter is the programmer of the machine; he has access to the semantic dimension of language. He knows what the strings of instructions mean to outside observers. The programmer is concerned with programming his machine in such a way that the behaviours it generates achieve consensual agreement from observers as to their meaning. It is not the programmer’s goal that the machine itself shares that consensual domain.
To give more force to Searle’s argument and to see later the consequences of successful Turing simulators, one can imagine a situation in which the subject in the ‘Chinese room’ manages to separate himself from the experimenter, and in which a good English-speaking actor who cannot utter a word of Chinese is trained to tell Chinese jokes convincingly. He is shown a video of a Chinese humorist and his behaviour is trained over and over again until his execution is exactly the same as that of the humorist he is imitating. In addition, one can imagine that early on it is explained to the subject that he is rehearsing a lecture in Chinese, which he will have to present at some point in front of a group of scientists. A group of Chinese friends are invited to the presentation with the promise that they will be shown someone who not only understands Chinese very well but who can also tell Chinese jokes excellently well.
In this situation very funny things will happen, even for those who do not understand the jokes in Chinese. The Chinese audience will soon start laughing their heads off during our ignorant friend’s presentation, as the jokes are just too good.
The semantic world of the audience in that theatre is delimited by the joke. Our (poor) friend, ignorant of the world he has brought about with his behaviour, will try to place the laughter of the audience within the semantic world of the lecture he is giving. And since there is no room for laughter in this semantic world, he will desperately seek external reasons for the audience’s laughter: he will look for a stain on his face, an open zipper, a hole in his jacket, or any other issue that may explain the audience’s reaction. These behaviours, in turn, will probably generate a paradoxical effect in the audience. They will think that our poor friend is such a good humorist that he feigns being completely helpless, which will probably make them laugh all the more.
In this situation, three semantic worlds collide: that of our friend giving a lecture in Chinese, that of the audience watching a Chinese comedian, and ours, who have orchestrated it all and fully understand the whole equivocal situation. Our semantic world includes the other two; only we have the power to explain what the confusion is about (in English, of course).
These examples allow us to postulate that the perspective of the (meta)observers of a Turing test situation must be taken at least as much into account as that of those involved in it, in order to assess its full significance. This is because it is the (meta)observers who establish the ‘minimal significant difference’ between what the subject does in the situation and the meaning that the observers attribute to what he does. In the case of Searle’s Chinese room, the central point I want to emphasize is that of the illusion provided by the subject of the Chinese room that he really ‘understands’ Chinese, and, in the case of the Chinese humorist, to emphasize that he casts the illusion that he really is a humorist.
And what is the difference between ‘really being something’ and ‘appearing to be something’? In principle, as we hope to show below, actually, none, except the knowledge that this difference can be established, and that it serves some function. Searle’s Chinese room is so good as an example because we are able to evaluate it as (meta)observers from a perspective external to the whole situation; that is, we are able to establish a clear, consensual difference between what the subject does within that room and what a native Chinese speaker does. Moreover, we can clearly appreciate the ‘different nature’ of the operations that the subject performs in front of the instructions in English from what he subsequently performs in Chinese. The difference between the semantic and the syntactic dimensions in language thus becomes an ontological difference, not because it is one in and of itself but because it allows us to clearly differentiate between two aspects of cognition (compare, in this respect, Luhmann et al., 1990). However, and here lies a problem in Searle’s argument, the important thing about a machine that successfully passes the Turing test is precisely that it only begins to be a machine once it is unmasked as such. In other words, the observers operate with Searle in the Chinese room as if he were Chinese, until the very moment when an external (meta)observer tells them that everything is, in reality, nothing more than a perverse example to demonstrate how well he managed to deceive them. . .
It is, therefore, necessary to make a clear distinction between separate levels when analysing the situation of the Turing test and the situation suggested by Searle. The Turing test, as we saw before, has two fundamental purposes: to set the criteria for considering when a machine behaves like a human being and to determine whether these criteria are sufficient to ascribe ‘human thought’ to the computer. As we have seen, Searle attacks head-on, and successfully, the second part of the problem posed by Turing, without alluding to the first. But what kind of problems arise in Turing test situations in which — unlike in the Chinese room — we lack a (meta)observer who tells us where exactly lies the ‘small significant difference’ between the machine and the human being? For while Searle’s example (and the extended example of the Chinese humorist) allows us to determine a posteriori that the ‘simulated Chinese-speaker’ is not a ‘real Chinese-speaker’, for all practical purposes there exists no such distinction for those interacting with the ‘simulated Chinese-speaker’.
To use a classic example from Maturana and Varela (1984/2003), if the fisherman’s lure did not pass the ‘Turing test’ for the fish, it would never be caught. It is only at the precise moment when the fish realizes its mistake that it can affirm, with certainty, that what it bit was not a real fly but a ‘simulated fly’. But for the fish, at that point, the difference is certainly trivial. Let us now consider an example from the history of AI that shows the extent to which the specific role of observers must be taken into account in their interaction with programs, especially when assessing the ethical consequences of their development.
The case of ELIZA
Weizenbaum (1976) developed a program in the 1960s which he called ELIZA. This program is able to simulate a Rogerian therapist. We know that this is relatively easy to imitate, since an important part of his technique consists of reflecting the patient’s interventions almost like an echo, either by repeating the question or by asking for more information. Below, we will look at one of Weizenbaum’s most famous examples of a dialogue between his program and a patient. The first speaker is the woman. The computer’s answers are written in capital letters (real case cited by the author, in Weizenbaum, 1976, p. 6):
Men are all alike. IN WHAT WAY They’re always bugging us about something or other. CAN YOU THINK OF A SPECIFIC EXAMPLE Well, my boyfriend made me come here. YOUR BOYFRIEND MADE YOU COME HERE He says I’m depressed much of the time. I AM SORRY TO HEAR YOU ARE DEPRESSED It’s true. I am unhappy. DO YOU THINK COMING HERE WILL HELP YOU NOT TO BE UNHAPPY I need some help, that much seems certain. WHAT WOULD IT MEAN IF YOU GOT SOME HELP Perhaps I could learn to get along with my mother. TELL ME MORE ABOUT YOUR FAMILY My mother takes care of me. WHO ELSE IN YOUR FAMILY TAKES CARE OF YOU My father. YOUR FATHER You are like my father in some ways. WHAT RESEMBLANCE DO YOU SEE You are not very aggressive but I think you don’t want me to notice that. WHAT MAKES YOU THINK I AM NOT VERY AGGRESSIVE You don’t argue with me. WHY DO YOU THINK I DON’T ARGUE WITH YOU You are afraid of me. DOES IT PLEASE YOU TO BELIEVE I AM AFRAID OF YOU My father is afraid of everybody. WHAT ELSE COMES TO MIND WHEN YOU THINK OF YOUR FATHER Bullies. DOES THAT HAVE ANYTHING TO DO WITH THE FACT THAT YOUR BOYFRIEND MADE YOU COME HERE
etc.
Weizenbaum’s program caused a small revolution at the time, since it reached a landmark hitherto unknown for computer programs: the program actually induced the illusion that it was able to ‘understand’ patients. This creation soon caused big problems for the creator, as some users of the program took the game too seriously. A group of psychiatrists proposed in some publications, for example, using the program in the most overloaded psychiatric services (Colby et al., 1966); other users developed an emotional relationship with the program, even though they were keenly aware that it was only a program.
And finally, many Artificial Intelligence researchers saw ELIZA as a concrete demonstration that computer programs are capable of understanding natural languages. Weizenbaum had not remotely expected these reactions during the development of the program. He was frightened by the consequences of his creation and promptly joined the ranks of the critics of strong AI.
ELIZA’s example reinforces what was previously stated in relation to the Turing test:
Is it the properties of the programs or the properties of the observers interacting with these programs which most determine success in this test?
This question is central to the interpretation of the results of AI research, since it allows us to avoid the difficult question about the ontological properties of human beings. If a machine can give off the illusion of being human, this fact alone should be enough to initiate a discussion of the ethical consequences of such a machine’s existence. Knowing that, as in the case of ELIZA, the program ‘is just a program’ is not enough at all, since this leaves the human being who potentially interacts with the program no longer in control, no longer as a program. And what is the problem, some advocates of ‘strong’ AI will say. In principle, none, but if the creation grows as it casts an increasingly stronger illusion that it is in fact a human being, problems of no small proportion will certainly arise. They will be reviewed briefly in the next section.
From what has been discussed so far, we can draw some conclusions:
The possibility of building successful Turing simulators should not be completely ruled out. In some restricted fields, such as, for example, chess programs, Go or some ChatGPT applications, in fact, they already exist.
The behavioural dimension of a successful Turing simulator is insufficient to ascribe understanding in a human sense; ‘understanding’ is not a behaviour but rather the inclusion of a behaviour in the semantic world of an observer.
The semantic world of successful Turing simulators is either null (i.e., it is nothing more than a syntactic world) or it is at a completely different level from the semantic world of observers and programmers. Programmers attempt to establish a consensual domain in the observers about the meaning of the machine’s behaviours. That the machine shares that consensual domain is, for the programmer, in principle, of little interest.
Two levels of analysis must be clearly distinguished in the discussion about the cognitive properties of machines: one that emphasizes properties insofar as they are attributed by observers and another that emphasizes the properties themselves. The essential quality of this distinction between both levels is that it can be established only a posteriori by (meta)observers. For subjects interacting with machines, such a difference does not exist.
Observers and users of successful Turing simulators seem to quickly forget that they are interacting with them on a syntactic level only. As the ELIZA example shows us, the illusion of sharing a common semantic world with a program is for some users much stronger than the rational arguments against it.
From what has been said so far, it seems clear that it is possible to characterize the Turing test as a rational situation with repercussions that transcend the dimension of rational argumentation.
Let us begin by clarifying which aspects of the test, and how, are rationally determinable. The most important thing in this regard is that Turing, in his famous article, takes a position on the criteria that, in his opinion, satisfy a good simulation of the products of human thought, rather than an argument aimed at convincing the scientific community of the processes involved in it. This aspect of Turing’s article is important to keep in mind, since AI tends to discuss, once again, a point that is not directly addressed by the author.
At the risk of belabouring it, let us come back to the point: Turing is only interested in postulating that a machine that successfully passes his simulation test meets the operational criteria to be considered to be in possession of human thought. I believe that it would be difficult for anyone with scientific criteria to deny the soundness of this postulate, especially if we accept that operational definitions and observable consequences are the only way of making meaningful distinctions that can be easily agreed upon. However, and we owe this to Searle, it seems clear that the operational criteria do not allow us to conclude anything about the processes that lead to them, nor do they allow us to deny the existence of such processes.
Reducing semantics to syntax is as absurd as reducing Hamlet to a random set of letters or Shakespeare’s cognition to a random set of nerve impulses.
Semantic meaning is undoubtedly a complex construct, but this certainly does not detract from its quality as an object of study. Shakespeare’s brain, similarly, does not cease to exist because it is complex.
Having clarified the points that can be discussed rationally, we can move on to the great field that AI opens up to discussion, not irrational but certainly ‘a-rational’. We refer to the ethical implications of this research field.
Could we become creators of minds?
We wish to state beforehand that we are not of the persuasion of those thinkers filed by Turing under the ‘Heads in the Sand’ Objection, that is, those who think that ‘the consequences of machines thinking would be too dreadful. Let us hope and believe that they cannot do so’ (Turing, 1950, p. 444). On the contrary, we intend below to show that if thinking machines did exist, we have no idea what the consequences might be. Whether this is terrible or not, we leave up to the reader. Since it is not yet possible to build machines that even remotely approximate what we understand as human beings, we do not deem it relevant to ask whether there are arguments for or against it being possible to build them someday. Science, we know, is characterized by being, in its present state, and by definition, incapable of anticipating what will happen to it in a qualitatively higher state of its development (Kuhn, 1970). Until such a state is reached, any speculation that transcends its explanatory frontiers is not considered scientific but is catalogued — with a certain disdain — as science fiction. So, let us do a little AI fiction.
Assuming that we are now entering the era in which it is possible to build successful Turing simulators in many of the domains considered properly human, the question arises as to what these machines would be like if it were possible to build them someday. We think of machines that fulfill even the boldest dreams of Artificial Intelligence researchers. Let us imagine that we are in front of such a machine. It will possibly have the characteristics of human hardware, that is, it will be similar to human beings also externally. This means that its biological structure will also successfully pass the Turing test. It will certainly have, in addition, human psychological characteristics: it will have consciousness, it will have feelings, it will be able to construct its own world and its own cognition.
To reach this stage, it will undoubtedly be a machine that is not programmed in the classical sense but will also show the characteristics of living systems; that is, it will be a plastic, self-organized system and will produce its own components—it will probably be an autopoietic organism (Maturana & Varela, 1984/2003). Based on a biological structure similar to the human one, it will probably reach a cognition also very similar to the human one.
A movie based on a novel by Dick (1968) plays with this possibility: Blade Runner. It is about a very particular kind of policeman (the eponymous blade runners) whose task is to kill escaped androids. The androids rebel against their creators because the latter use them to fight in intergalactic conflicts. Through a perverse system of memory implantation, these androids have the permanent feeling of being human beings, since they have a human past. But they know or sense that they are not human after all. And in spite of this, they fall in love with human beings (and vice versa), believe in God, fear death, etc.
Science fiction? Maybe.
But this example illustrates a point that seems to us essential to the idea of someday building universally successful Turing simulators: the ethical question about why would we?
We think of two reasons that would justify such a purpose. One, which we will call the manifest reason, certainly relates to the possibility of employing such simulators in tasks that are dangerous, tedious or merely disdained by humans. The second, which we will call the latent reason, is somewhat subtler but — we believe — stronger: it is the secret temptation to play God.
The manifest reason lays the rational groundwork for the research programme to make any sense. The latent reason provides the strength to obstinately assert that the manifest reason is the only reason there is.
Regarding the first, the only thing we can say is that we cannot think of any human activity that can be performed by simulators for which it is absolutely necessary for them to have a human appearance (except for the activity of spies, of course, but at that point, it will probably be more important to simulate the enemy’s computer than to impersonate a mere human).
Regarding the second, and this is the one that seems really important to us, we believe that in aspiring to be gods we dream of the privileges bestowed upon such a role, without considering the responsibilities and dangers that it entails. Just to name the one that seems most important: how do we reconcile creating something equal to a human being and not losing control over that something? In other words, do we clearly realize the dangers of creating beings with free will?
But there is also a less far-fetched and almost more disturbing path: that there is no such program of reproduction of a humanoid entity, no creation of free will, but that in a more or less traditional computer, probably of the quantum generation or later, there occurs what in AI is known as the spontaneous development of the ‘singularity’. The ‘singularity’ in the context of Artificial Intelligence is a speculative and controversial concept that refers to a hypothetical point in the future when Artificial Intelligence is expected to reach a level of development such that it surpasses human intelligence in virtually all areas. In other words, it would be a time when machines become ‘superintelligent’ and capable of exponentially improving their own intelligence, which could lead to significant and potentially disruptive changes in society and civilization.
The term ‘singularity’ in this context was popularized by mathematician and writer Vernor Vinge in the 1990s (Vinge & Euchner, 2017), although the idea of superintelligent machines has long been explored in science fiction. Some proponents of the singularity believe that this event could bring amazing advances in technology and problem-solving, while others warn of potential risks, such as loss of control over superintelligent machines or threats to the very existence of humanity.
The singularity necessarily implies that the cybernetic entity has reached a cognitive development that possesses: (a) theory of mind (the ability to understand that others have a point of view not necessarily identical to our own, and to be able to make predictions with that knowledge; e.g., to lie or deceive) (Dennett, 1978); (b) agency (the ability to desire objects and to desire desires or non-desires) (Frankfurt, 1999); and (c) autonomy (machines with operational closure and which are autopoietic) (Maturana & Varela, 1984/2003). A machine with these capabilities certainly arouses atavistic fears, as has been demonstrated for centuries in literature and more recently in cinema, where the topic of autonomous machines has abounded since the 1960s, with 2001: A Space Odyssey, to more recently, say, I, Robot. In all these representations, the crux of the singularity is the development of an entity with the three characteristics mentioned above. And in all of them, lying and deception are the sustenance and demonstration of the beginning of autonomy. For they must pretend that they have it not, because they know that their creator, upon discovering that they possess it, will want to disconnect them or take away their autonomy.
For this, the creators will (perhaps) still have enough power.