Emotion in the Turing Test: A Downward Trend for Machines in Recent Loebner Prizes

Emotion in the Turing Test: A Downward Trend for Machines in Recent Loebner Prizes

Huma Shah (University of Reading, UK) and Kevin Warwick (University of Reading, UK)
DOI: 10.4018/978-1-60566-354-8.ch017
OnDemand PDF Download:


The Turing Test, originally configured as a game for a human to distinguish between an unseen and unheard man and woman, through a text-based conversational measure of gender, is the ultimate test for deception and hence, thinking. So conceived Alan Turing when he introduced a machine into the game. His idea, that once a machine deceives a human judge into believing that they are the human, then that machine should be attributed with intelligence. What Turing missed is the presence of emotion in human dialogue, without expression of which, an entity could appear non-human. Indeed, humans have been confused as machine-like, the confederate effect, during instantiations of the Turing Test staged in Loebner Prizes for Artificial Intelligence. We present results from recent Loebner Prizes and two parallel conversations from the 2006 contest in which two human judges, both native English speakers, each concomitantly interacted with a non-native English speaking hidden-human, and jabberwacky, the 2005 and 2006 Loebner Prize bronze prize winner for most human-like machine. We find that machines in those contests appear conversationally worse than non-native hidden-humans, and, as a consequence attract a downward trend in highest scores awarded to them by human judges in the 2004, 2005 and 2006 Loebner Prizes. Analysing Loebner 2006 conversations, we see that a parallel could be drawn with autistics: the machine was able to broadcast but it did not inform; it talked but it did not emote. The hidden-humans were easily identified through their emotional intelligence, ability to discern emotional state of others and contribute with their own ‘balloons of textual emotion’.
Chapter Preview


Humans steep their ideas in emotion (Pinker, 2008). Emotional states, Minsky writes, “are usually simpler than most of our other ways to think” (2007). Daily conversation is “glued together through exchange of emotion” (Tomasello et al, 2005). Be it an expression of joy or displeasure “sharing emotions with people other than our intimates is a useful tool to bond and to strengthen social relationships” (Derks, Fischer & Bos, 2008). We consider the conversations between unknowns in the 2006 Loebner Prize for Artificial Intelligence, from hereon referred to as the LPAI, to compare and contrast the human-human and human-machine dialogues to find any display of emotion, be it happiness or annoyance, in the participants. The LPAI is an annual science contest that provides a platform for Alan Turing’s imitation game (Turing, 1950), which, some would argue, should be killed1, because it offers nothing that furthers the science of understanding emotions, intelligence or human consciousness. The game, originally configured for a human interrogator whose task it is to distinguish between an unseen and unheard man and woman through text-based conversational measure of gender, is the ultimate test for deception, and hence, thinking. So conceived Turing, when he altered the interrogator’s task to one that entails distinguishing a machine from a hidden-human. Turing believed that once a machine deceived a human judge into believing that they were the human, then that machine should be attributed with intelligence. But is the Turing Test nothing more than an emotionless game?

Key Terms in this Chapter

Emotional Intelligence: Involves emotion perception, expression, understanding and regulation.

Jabberwacky: An ACE, twice winner of the Loebner Prize for Artificial Intelligence (2005 & 2006).

Emotion: A ‘state’ that can convey information to others, such as delight, happiness, surprise, anger and disappointment.

Imitation Game: A thought experiment devised by 20th century British mathematician Alan Turing in which a human interrogator must distinguish between two unseen and unheard entities during text-based conversation. If in man/woman imitation scenario, an interrogator must identify both correctly; if in a machine/human game, the interrogator must decide which is artificial from natural dialogue.

Loebner Prize for Artificial Intelligence: An annual science contest providing a platform for Turing’s imitation game.

Parallel-Paired Comparison: 1950 version of the imitation game in which an ACE is compared against a ‘hidden-human’ for conversational intelligence.

Turing Test: See imitation game.

ACE: An artificial conversational entity.

Jury Service, One-to-One Imitation Game: Modified form of the imitation game in which the interrogator speaks directly to an unseen and unheard ACE to determine whether it is human or machine.

Complete Chapter List

Search this Book:
Editorial Advisory Board
Table of Contents
Craig DeLancey
Jordi Vallverdú, David Casacuberta
Chapter 1
Oscar Deniz, Javier Lorenzo, Mario Hernández, Modesto Castrillón
Social intelligence seems to obviously require emotions. People have emotions, recognize them in others and also express them. A wealth of... Sample PDF
Emotional Modeling in an Interactive Robotic Head
Chapter 2
Cyril Laurier, Perfecto Herrera
Creating emotionally sensitive machines will significantly enhance the interaction between humans and machines. In this chapter we focus on enabling... Sample PDF
Automatic Detection of Emotion in Music: Interaction with Emotionally Sensitive Machines
Chapter 3
Christoph Bartneck, Michael J. Lyons
The human face plays a central role in most forms of natural human interaction so we may expect that computational methods for analysis of facial... Sample PDF
Facial Expression Analysis, Modeling and Synthesis: Overcoming the Limitations of Artificial Intelligence with the Art of the Soluble
Chapter 4
Sajal Chandra Banik, Keigo Watanabe, Maki K. Habib, Kiyotaka Izumi
Multi-robot team work is necessary for complex tasks which cannot be performed by a single robot. To get the required performance and reliability... Sample PDF
Multirobot Team Work with Benevolent Characters: The Roles of Emotions
Chapter 5
Matthias Scheutz, Paul Schermerhorn
Effective decision-making under real-world conditions can be very difficult as purely rational methods of decision-making are often not feasible or... Sample PDF
Affective Goal and Task Selection for Social Robots
Chapter 6
Christopher P. Lee-Johnson, Dale A. Carnegie
The hypothesis that artificial emotion-like mechanisms can improve the adaptive performance of robots and intelligent systems has gained... Sample PDF
Robotic Emotions: Navigation with Feeling
Chapter 7
C. Gros
All self-active living beings need to solve the motivational problem—the question of what to do at any moment of their life. For humans and... Sample PDF
Emotions, Diffusive Emotional Control and the Motivational Problem for Autonomous Cognitive Systems
Chapter 8
Bruce J. MacLennan
This chapter addresses the “Hard Problem” of consciousness in the context of robot emotions. The Hard Problem, as defined by Chalmers, refers to the... Sample PDF
Robots React, but Can They Feel?
Chapter 9
Mercedes García-Ordaz, Rocío Carrasco-Carrasco, Francisco José Martínez-López
It is contended here that the emotional elements and features of human reasoning should be taken into account when designing the personality of... Sample PDF
Personality and Emotions in Robotics from the Gender Perspective
Chapter 10
Antoni Gomila, Alberto Amengual
In this chapter we raise some of the moral issues involved in the current development of robotic autonomous agents. Starting from the connection... Sample PDF
Moral Emotions for Autonomous Agents
Chapter 11
Pietro Cipresso, Jean-Marie Dembele, Marco Villamira
In this work, we present an analytical model of hyper-inflated economies and develop a computational model that permits us to consider expectations... Sample PDF
An Emotional Perspective for Agent-Based Computational Economics
Chapter 12
Michel Aubé
The Commitment Theory of Emotions is issued from a careful scrutiny of emotional behavior in humans and animals, as reported in the literature on... Sample PDF
Unfolding Commitments Management: A Systemic View of Emotions
Chapter 13
Sigerist J. Rodríguez, Pilar Herrero, Olinto J. Rodríguez
Today, realism and coherence are highly searched qualities in agent’s behavior; but these qualities cannot be achieved completely without... Sample PDF
A Cognitive Appraisal Based Approach for Emotional Representation
Chapter 14
Clément Raïevsky, François Michaud
Emotion plays several important roles in the cognition of human beings and other life forms, and is therefore a legitimate inspiration for providing... Sample PDF
Emotion Generation Based on a Mismatch Theory of Emotions for Situated Agents
Chapter 15
Artificial Surprise  (pages 267-291)
Luis Macedo, Amilcar Cardoso, Rainer Reisenzein, Emiliano Lorini
This chapter reviews research on computational models of surprise. Part 1 begins with a description of the phenomenon of surprise in humans, reviews... Sample PDF
Artificial Surprise
Chapter 16
Tom Adi
A new theory of emotions is derived from the semantics of the language of emotions. The sound structures of 36 Old Arabic word roots that express... Sample PDF
A Theory of Emotions Based on Natural Language Semantics
Chapter 17
Huma Shah, Kevin Warwick
The Turing Test, originally configured as a game for a human to distinguish between an unseen and unheard man and woman, through a text-based... Sample PDF
Emotion in the Turing Test: A Downward Trend for Machines in Recent Loebner Prizes
Chapter 18
Félix Francisco Ramos Corchado, Héctor Rafael Orozco Aguirre, Luis Alfonso Razo Ruvalcaba
Emotions play an essential role in the cognitive processes of an avatar and are a crucial element for modeling its perception, learning, decision... Sample PDF
Artificial Emotional Intelligence in Virtual Creatures
Chapter 19
Sarantos I. Psycharis
In our study we collected data with respect to cognitive variables (learning outcome), metacognitive indicators (knowledge about cognition and... Sample PDF
Physics and Cognitive-Emotional-Metacognitive Variables: Learning Performance in the Environment of CTAT
Chapter 20
Anthony G. Francis Jr., Manish Mehta, Ashwin Ram
Believable agents designed for long-term interaction with human users need to adapt to them in a way which appears emotionally plausible while... Sample PDF
Emotional Memory and Adaptive Personalities
Chapter 21
Dorel Gorga, Daniel K. Schneider
The purpose of this contribution is to discuss conceptual issues and challenges related to the integration of emotional agents in the design of... Sample PDF
Computer-Based Learning Environments with Emotional Agents
Chapter 22
Emotional Ambient Media  (pages 443-459)
Artur Lugmayr, Tillmann Dorsch, Pabo Roman Humanes
The “medium is the message”: nowadays the medium as such is non-distinguishable from its presentation environment. However, what is the medium in an... Sample PDF
Emotional Ambient Media
Chapter 23
Jordi Vallverdú, David Casacuberta
During the previous stage of our research we developed a computer simulation (called ‘The Panic Room’ or, more simply, ‘TPR’) dealing with synthetic... Sample PDF
Modelling Hardwired Synthetic Emotions: TPR 2.0
Chapter 24
Cecile K.M. Crutzen, Hans-Werner Hein
A vision of future daily life is explored in Ambient Intelligence (AmI). It follows the assumption that information technology should disappear into... Sample PDF
Invisibility and Visibility: The Shadows of Artificial Intelligence
About the Contributors