This chapter addresses recent advances in computer vision for facial expression classification. The authors present the different processing steps of the problem of automatic facial expression recognition. They describe the advances of each stage of the problem and review the future challenges towards the application of such systems to everyday life situations. The authors also introduce the importance of taking advantage of the human strategy by reviewing advances of research in psychology towards multidisciplinary approach for facial expression classification. Finally, the authors describe one contribution which aims at dealing with some of the discussed challenges.
TopIntroduction
Nowadays, computers become more and more presented in everyday life of the general population. Involved in professional as well as personal activities to perform tasks more and more complex, computer accessibility needs to be greatly improved. Notably a human-computer interaction can become more effective if the communication between computers and users mimic as much as possible the way human beings interact between each other. One way to achieve this is to provide them the ability to analyze the behavior of the user in order to give them access to other dimensions of the social communication that are unattainable through the keyboard instructions. Computers may then be able to better answer the user’s expectations.
The human face is an important and complex communication channel and it is considered as one of the most ecologically relevant entities for visual perception. Expressive changes in the face are a rich source of cues about intra and interpersonal indicators and functions of emotion. Facial expressions communicate information from which we can quickly infer the state of mind of our peers, and adjust our behavior accordingly (Darwin, 1872). Facial expressions are a visible manifestation of the emotional state, cognitive activity, intention, personality, and psychopathology of a person (Donato et al., 1999). It is said that 55% of the communicating feeling is conveyed by the facial expressions while only 7% by the linguistic language and 38% by the paralanguage (Mehrabian, 1968). This implies that facial expressions play an important role in human communication. Human-computer interaction will then definitively benefit from automated facial expression recognition.
In this chapter, we address the issue of automatic recognition of facial expression. The development of such a system constitutes a significant challenge because of the many constraints that are imposed by its application in a real world context (Pantic & Bartlett, 2007). In particular, this system should be able to provide relevant information with great accuracy and work under harsh conditions without demanding too many interventions from the user. Based on these considerations, there have been major advances in computer vision over the past 15 years applying advanced techniques of image and video processing for the recognition of facial expressions.
This chapter introduces advances in computer vision for facial expression classification. First, we describe the most comment and useful representation of facial expressions. Then, after briefly presenting the different approaches and limits of facial feature extraction methods, we survey the state of the art of automatic facial expression systems, discussing their achievements and their limitations. In order to go beyond the computational approach, we make an original introduction to the psychology of visual perception, focusing on the most recent available data of facial expression recognition by humans and we show how research in facial expression classification may favorably take advantage of a multidisciplinary approach. Finally, we describe one of the most recently proposed contributions for the recognition of the six basic facial expressions, which deals with some of the current challenges and limitations faced by the automatic facial expression community.