Facial expression recognition is a challenging task. A facial expression is formed by contracting or relaxing different facial muscles on human face that results in temporally deformed facial features like wide-open mouth, raising eyebrows or etc. The challenges of such system have to address with some issues. For instances, lighting condition is a very difficult problem to constraint and regulate. On the other hand, real-time processing is also a challenging problem since there are so many facial features to be extracted and processed and sometimes, conventional classifiers are not even effective in handling those features and produce good classification performance. This chapter discusses the issues on how the advanced feature selection techniques together with good classifiers can play a vital important role of real-time facial expression recognition. Several feature selection methods and classifiers are discussed and their evaluations for real-time facial expression recognition are presented in this chapter. The content of this chapter is a way to open-up a discussion about building a real-time system to read and respond to the emotions of people from facial expressions.
TopIntroduction
Given the significant role of the face in our emotional and social lives, it is not surprising that the potential benefits from efforts to automate the analysis of facial signals, in particular rapid facial signals, are varied and numerous (Ekman et al., 1993), especially when it comes to computer science and technologies brought to bear on these issues (Pantic, 2006). As far as natural interfaces between humans and computers are concerned, facial expressions provide a way to communicate basic information about needs and demands to the machine. In fact, automatic analysis of facial signals seem to have a natural place in various vision sub-systems, including automated tools for tracking gaze and focus of attention, lip reading, bimodal speech processing, face/visual speech synthesis, and face-based command issuing.
Facial Expression Analysis is a challenging task. A facial expression is formed by contracting or relaxing different facial muscles on human face that results in temporally deformed facial features like wide-open mouth, raising eyebrows or etc. The challenges of such system have to address the following issues:
Lighting conditions is a very difficult problem to constraint and regulate. The strength of the light depends on the light source (see Figure 1).
The direction of the subjects face is not always ideal which may pose difficulties when the system is implemented live that captures moving subjects’ facial expression (see Figure 2).
Another difficulty is the way image is acquired by the image acquisition system. The characteristics of the image acquisition system can affect the quality of the images or videos captured.
Occlusion of subject face may tumble the hit rate of many established approaches. The experiments being carried out by most researchers do not take occlusion into account (see Figure 3).
Because of the above challenges, this chapter is going to introduce the recent advances in feature selection and classification methodologies for facial expression analysis. It first describes the background of different techniques used for facial expression analysis. Then it introduces the ideas of an automatic facial expression recognition system proposed by the authors which includes feature extraction, feature selection and classification methods. Finally, some of the future trends in terms of scientific and engineering challenges are discussed and recommendations for achieving a better facial expression technology are outlined.