Abstract
Eye-gaze plays an important role in face-to-face communication. This chapter presents research on exploiting the rich information contained in human eye-gaze for two types of applications. The first is to enhance computer mediated human-human communication by overlaying eye-gaze movement onto the shared visual spatial discussion material such as a map. The second is to manage multimodal human-computer dialogue by tracking the user’s eye-gaze pattern as an indicator of user’s interest. The authors briefly review related literature and summarize results from two research projects on human-human and human-computer communication.
The eyes of men converse as much as their tongues, with the advantage that the ocular dialect needs no dictionary, but is understood all the world over.
—Ralph Waldo Emerson, 1860
TopGaze And Conversation
In face-to-face conversation, much can be intuitively felt from the conversational partners’ eye-gaze—whether they are interested or bored, attentive or preoccupied, engaged or unmindful, in doubt or in agreement, wanting to continue or trying to finish the conversation. Indeed, research has confirmed that eye-gaze plays an important role in face-to-face conversation. It enables us to assess a conversational partner’s understanding, what he or she is looking at, and his or her feelings (Argyle & Cook, 1976).
Gaze plays a particularly important role in face-to-face communication when it comes to regulating the turn-taking behavior in a conversation. Gaze is used to signal if the speaker is about to hand over the turn, if he or she will continue after a pause, or if the speaker expects some feedback from the listener (Bavelas, Coates, & Johnson, 2002; Kendon, 1967). When two people attempt to take the turn simultaneously, gazes are used to resolve who should have the turn (Duncan & Niederehe, 1974). Beyond turn-taking, gaze is also used for emphasizing particular words or phrases, and aversion of gaze indicates lack of interest or disapproval (Argyle & Cook, 1976). The use of gaze is also related to the content of speech. Cassell, Torres, and Prevost (1999) has showed that when the speaker starts a new topic with a new utterance, he or she looks at the listener. When the speaker is pursuing an old topic, the speaker looks away at the beginning of the turn. When the utterance is a request, gaze is used to make sure that the addressee understands that he or she is supposed to listen (Goodwin, 1980, 1981). Similar pattern have been found when a person gives commands to an interactive object (Maglio, Matlock, Campbell, Zhai, & Smith, 2000).
Key Terms in this Chapter
Eye-Gaze Overlay: a visualization of a person’s eye-gaze overlaid on visual spatial information shared by two or more persons.
Eye-Gaze Pattern: a pattern of eye-gaze fixations and saccades that can be used to infer a person’s intention or goal within a particular context.
Deictic Reference: the use of gestures or other means of pointing to specify an ambiguous utterance, for instance pointing at a place in a map and saying “here.”
Eye Tracker: equipment for tracking where a person is looking.
Multimodal Dialogue or Interaction: Combing information from different sources, for instance vision and speech, for human-computer interaction or dialogue.
Fixation and Saccade: Two basic elements of eye movement. A saccade is the rapid eye movement between fixations to move the eye-gaze from one point to another. A fixation is the point between two saccades, during which the eyes are relatively stationary and virtually all visual input occurs.