The Turing Test, originally proposed as a simple operational definition of intelligence, has now been with us for exactly half a century. It is safe to say that no other single article in computer science, and few other articles in science in general, have generated so much discussion. The present article chronicles the comments and controversy surrounding Turing's classic article from its publication to the present. The changing perception of the Turing Test over the last fifty years has paralleled the (...) changing attitudes in the scientific community towards artificial intelligence: from the unbridled optimism of 1960's to the current realization of the immense difficulties that still lie ahead. I conclude with the prediction that the Turing Test will remain important, not only as a landmark in the history of the development of intelligent machines, but also with real relevance to future generations of people living in a world in which the cognitive capacities of machines will be vastly greater than they are now. (shrink)

The paper begins by examining the original Turing Test (2T) and Searle’s antithetical Chinese Room Argument, which is intended to refute the 2T in particular, as well as any formal or abstract procedural theory of the mind in general. In the ensuing dispute between Searle and his own critics, I argue that Searle’s ‘internalist’ strategy is unable to deflect Dennett’s combined robotic-systems reply and the allied Total Turing Test (3T). Many would hold that the 3T marks the culmination of the (...) dialectic and, in principle, constitutes a fully adequate empirical standard for judging that an artifact is intelligent on a par with human beings. However, the paper carries the debate forward by arguing that the sociolinguistic factors highlighted in externalist views in the philosophy of language indicate the need for a fundamental shift in perspective in a Truly Total Turing Test (4T). It’s not enough to focus on Dennett’s individual robot viewed as a system; instead, we need to focus on an ongoing system of such artifacts. Hence a 4T should evaluate the general category of cognitive organization under investigation, rather than the performance of single specimens. From this comprehensive standpoint, the question is not whether an individual instance could simulate intelligent behavior within the context of a pre-existing sociolinguistic culture developed by the human cognitive type. Instead the key issue is whether the artificial cognitive type itself is capable of producing a comparable sociolinguistic medium. (shrink)

The Turing Test is one of the most disputed topics in artificial intelligence, philosophy of mind, and cognitive science. This paper is a review of the past 50 years of the Turing Test. Philosophical debates, practical developments and repercussions in related disciplines are all covered. We discuss Turing's ideas in detail and present the important comments that have been made on them. Within this context, behaviorism, consciousness, the 'other minds' problem, and similar topics in philosophy of mind are discussed. We (...) also cover the sociological and psychological aspects of the Turing Test. Finally, we look at the current situation and analyze programs that have been developed with the aim of passing the Turing Test. We conclude that the Turing Test has been, and will continue to be, an influential and controversial topic. -/- (This paper was reprinted in: The Turing Test: The Elusive Standard of Artificial Intelligence, J. H. Moor, ed., Kluwer Academic, pp. 23-78, 2003.). (shrink)

A fundamental problem in artificial intelligence is that nobody really knows what intelligence is. The problem is especially acute when we need to consider artificial systems which are significantly different to humans. In this paper we approach this problem in the following way: we take a number of well known informal definitions of human intelligence that have been given by experts, and extract their essential features. These are then mathematically formalised to produce a general measure of intelligence for arbitrary machines. (...) We believe that this equation formally captures the concept of machine intelligence in the broadest reasonable sense. We then show how this formal definition is related to the theory of universal optimal learning agents. Finally, we survey the many other tests and definitions of intelligence that have been proposed for machines. (shrink)

Turing's celebrated 1950 paper proposes a very general methodological criterion for modelling mental function: total functional equivalence and indistinguishability. His criterion gives rise to a hierarchy of Turing Tests, from subtotal ("toy") fragments of our functions (t1), to total symbolic (pen-pal) function (T2 -- the standard Turing Test), to total external sensorimotor (robotic) function (T3), to total internal microfunction (T4), to total indistinguishability in every empirically discernible respect (T5). This is a "reverse-engineering" hierarchy of (decreasing) empirical underdetermination of the theory (...) by the data. Level t1 is clearly too underdetermined, T2 is vulnerable to a counterexample (Searle's Chinese Room Argument), and T4 and T5 are arbitrarily overdetermined. Hence T3 is the appropriate target level for cognitive science. When it is reached, however, there will still remain more unanswerable questions than when Physics reaches its Grand Unified Theory of Everything (GUTE), because of the mind/body problem and the other-minds problem, both of which are inherent in this empirical domain, even though Turing hardly mentions them. (shrink)

Alan Turing’s 1950 imitation game has been widely understood as a means for testing if an entity is intelligent. Following a series of papers by Diane Proudfoot, I offer a socio-technological interpretation of Turing’s paper and present an alternative way of understanding both the imitation game and Turing’s concept of intelligence. Turing, I claim, saw intelligence as a social concept, meaning that possession of intelligence is a property determined by society’s attitude toward the entity. He realized that as long as (...) human society held a prejudiced attitude toward machinery—seeing machines a priori as mindless objects—machines could not be said to be intelligent, by definition. He also realized, though, that if humans’ a priori, chauvinistic attitude toward machinery changed, the existence of intelligent machines would become logically possible. Turing thought that such a change would eventually occur: He believed that when scientists overcome the technological challenge of constructing sophisticated machines that could imitate human verbal behavior—i.e., do well in the imitation game—humans’ prejudiced attitude toward machinery will have altered in such a way that machines could be said to be intelligent. The imitation game, for Turing, was not an intelligence test, but a technological aspiration whose realization would likely involve a change in society’s attitude toward machines. (shrink)

We introduce a construct called “humanness sensitivity,” which we define as the ability to recognize uniquely human characteristics. To evaluate the construct, we used a “concurrent study design” to conduct an internet-based study with a convenience sample of 42,063 people from 88 countries.We sought to determine to what extent people could identify subtle characteristics of human behavior, thinking, emotions, and social relationships which currently distinguish humans from non-human entities such as bots. Many people were surprisingly poor at this task, even (...) when asked simple questions about human relationships or anatomy. Participants were best at identifying subtle aspects of human cognition and worst at identifying subtle aspects of human communication. Test scores were good predictors of whether someone was employed and modest predictors of other self-reported criterion measures. We also found that people identifying themselves in marginal societal categories identified themselves as less human and also scored lower on our test. As computers continue to become more human-like, our study suggests that the vast majority of humankind will likely have great difficulty distinguishing them from people. Can methods be devised for improving this ability? Might humanness sensitivity help people to make such distinctions? Will people who excel at differentiating humans and non-human entities—like the “blade runners” in the 1982 and 2017 feature films—someday hold a special place in society? (shrink)

Disertační práce se zabývá problematikou připisování myšlení jiným entitám, a to pomocí imitační hry navržené v roce 1950 britským filosofem Alanem Turingem. Jeho kritérium, známé v dějinách filosofie jako Turingův test, je podrobeno detailní analýze. Práce popisuje nejen původní námitky samotného Turinga, ale především pozdější diskuse v druhé polovině 20. století. Největší pozornost je věnována těmto kritikám: Lucasova matematická námitka využívající Gödelovu větu o neúplnosti, Searlův argument čínského pokoje konstatující nedostatečnost syntaxe pro sémantiku, Blockův návrh na použití brutální síly pro (...) řešení imitační hry, Frenchova teorie subkognitivních informací a Michieho skepticismus ohledně možnosti umělého vědomí. Závěr práce zachycuje současný stav recepce Turingova testu a představuje pokusy o jeho praktickou realizaci, například v každoroční soutěži o Loebnerovu cenu. Autor práce zastává názor, že ani po více než šedesáti letech od uveřejnění Turingova paradigmatického eseje stále neexistují žádné vážné důvody pro zamítnutí jeho tvrzení. Tradiční komputační funkcionalismus možná není ideální teorií vysvětlující činnost myslí a jako slibnější se může jevit vývoj v neurálních vědách, ale Turingův test je přesto užitečným a snad i jediným nástrojem pro detekci inteligence u lidmi vytvořených strojů. (shrink)

The traditional Turing test appeals to an interrogator's judgement to determine whether or not their interlocutor is an intelligent agent. This paper argues that this kind of asymmetric experimental set-up is inappropriate for tracking a property such as intelligence because intelligence is grounded in part by symmetric relations of recognition between agents. In place, it proposes a reciprocal test which takes into account the judgments of both interrogators and competitors to determine if an agent is intelligent. This form of social (...) interaction better tracks both the evolution of natural intelligence and how the concept of intelligence is actually used within our society. This new test is defended against the criticisms that a proof of intelligence requires a demonstration of self-consciousness and that semantic externalism entails that a non-embodied Turing test is inadequate. (shrink)

In the last 20 years the Turing test has been left further behind by new developments in artificial intelligence. At the same time, however, these developments have revived some key elements of the Turing test: imitation and adversarialness. On the one hand, many generative models, such as generative adversarial networks, build imitators under an adversarial setting that strongly resembles the Turing test. The term “Turing learning” has been used for this kind of setting. On the other hand, AI benchmarks are (...) suffering an adversarial situation too, with a ‘challenge-solve-and-replace’ evaluation dynamics whenever human performance is ‘imitated’. The particular AI community rushes to replace the old benchmark by a more challenging benchmark, one for which human performance would still be beyond AI. These two phenomena related to the Turing test are sufficiently distinctive, important and general for a detailed analysis. This is the main goal of this paper. After recognising the abyss that appears beyond superhuman performance, we build on Turing learning to identify two different evaluation schemas: Turing testing and adversarial testing. We revisit some of the key questions surrounding the Turing test, such as ‘understanding’, commonsense reasoning and extracting meaning from the world, and explore how the new testing paradigms should work to unmask the limitations of current and future AI. Finally, we discuss how behavioural similarity metrics could be used to create taxonomies for artificial and natural intelligence. Both testing schemas should complete a transition in which humans should give way to machines—not only as references to be imitated but also as judges—when pursuing and measuring machine intelligence. (shrink)

Since the Nuremberg Code and the first Declaration of Helsinki, globally there has been increasing adoption and adherence to procedures for ensuring that human subjects in research are as well informed as possible of the study’s reasons and risks and voluntarily consent to serving as subject. To do otherwise is essentially viewed as violation of the human research subject’s legal and moral rights. However, with the recent philosophical concerns about responsible robotics, the limits and ambiguities of research-subjects ethical codes become (...) apparent on the matter of constructing automata that maximally resemble human beings (as defined hereunder). In this case, the automata themselves, as products of research and development, are in the very process of their construction subjects of research and development. However, such research faces a paradox: The subjects cannot give their informed consent to this research for their own development, although their consent would be needed for the research. According to ethical codes, this research would be unethical. The article then explores whether the background concepts giving rise to this paradox could be reframed in order to allow such research to proceed ethically. (shrink)

We confront the following popular views: that mind or life are algorithms; that thinking, or more generally any process other than computation, is computation; that anything other than a working brain can have thoughts; that anything other than a biological organism can be alive; that form and function are independent of matter; that sufficiently accurate simulations are just as genuine as the real things they imitate; and that the Turing test is either a necessary or sufficient or scientific procedure for (...) evaluating whether or not an entity is intelligent. Drawing on the distinction between activities and tasks, and the fundamental scientific principles of ontological lawfulness, epistemological realism, and methodological skepticism, we argue for traditional scientific materialism of the emergentist kind in opposition to the functionalism, behaviourism, tacit idealism, and merely decorative materialism of the artificial intelligence and artificial life communities. (shrink)