Nonverbal communication is the main channel through which we experience inner life of others, including their emotions, feelings, moods, social attitudes, etc. This attracts the interest of the computing community because nonverbal communication is based on cues like facial expressions, vocalizations, gestures, postures, etc. that we can perceive with our senses and can be (and often are) detected, analyzed and synthesized with automatic approaches. In other words, nonverbal communication can be used as a viable interface between computers and some of the most important aspects of human psychology such as emotions and social attitudes. As a result, a new computing domain seems to emerge that we can define “technology of nonverbal communication”. This chapter outlines some of the most salient aspects of such a potentially new domain and outlines some of its most important perspectives for the future.
TopIntroduction
Nonverbal communication is one of the most pervasive phenomena of our everyday life. On one hand, just because we have a body and we are alive, we constantly display a large number of nonverbal behavioral cues like facial expressions, vocalizations, postures, gestures, appearance, etc. (Knapp & Hall, 1972; Richmond & McCroskey, 1995). On the other hand, just because we sense and perceive the cues others display, we cannot avoid interpreting and understanding them (often outside conscious awareness) in terms of feelings, emotions, attitudes, intentions, etc. (Kunda, 1999; Poggi, 2007). Thus, “We cannot not communicate” (Watzlawick et al., 1967) even when we sleep and still display our feelings (of which we are unaware) through movements, facial expressions, etc., or when we make it clear that we do not want to communicate:
If two humans come together it is virtually inevitable that they will communicate something to each other [...] even if they do not speak, messages will pass between them. By their looks, expressions and body movement each will tell the other something, even if it is only, “I don't wish to know you: keep your distance”; “I assure you the feeling is mutual. I'll keep clear if you do”. (Argyle, 1979).
As nonverbal communication is such a salient and ubiquitous aspect of our life, it is not surprising to observe that computing technology, expected to integrate our daily life seamlessly and naturally like no one else, identifies automatic understanding and synthesis of nonverbal communication as a key step towards human-centered computers, i.e. computers adept to our natural modes of operating and communicating (Pantic et al., 2007; Pantic et al., 2008). This is the case of Affective Computing, where the aim is automatic understanding and synthesis of emotional states (Picard, 2000), of certain trends in Human-Computer Interaction, where the goal is to interface machines with the psychology of users (Reeves & Nass, 1996; Nass & Brave, 2005), of research in Embodied Conversational Agents, where the goal is to simulate credible human behavior with synthetic characters or robots (Bickmore & Cassell, 2005), and of the emerging field of Social Signal Processing, where the target is to understand mutual relational attitudes (social signals) of people involved in social interactions (Vinciarelli et al., 2008; Vinciarelli et al., 2009).
This list of domains is by no means complete, but it is sufficient to show how a nonverbal communication technology is actually developing in the computing community. Its main strength is an intense cross-fertilization between machine intelligence (e.g., speech processing, computer vision and machine learning) and human sciences (e.g., psychology, anthropology and sociology) and its main targets are artificial forms of social, emotional and affective intelligence (Albrecht, 2005; Goleman, 2005). Furthermore, social and psychological research increasingly relies on technologies related to nonverbal communication to develop insights about human-human interactions, like in the case of large scale social networks (Lazer et al., 2009), organizational behavior (Olguin et al., 2009), and communication in mobile spaces (Raento et al., 2009).