Most of the works on Facial Expression Recognition (FER) have worked on 2D images or videos. However, researchers are now increasingly utilizing 3D information for FER. As a contribution, this chapter zooms in on 3D based approaches while introducing FER. Prominent works are reviewed briefly, and some of the issues involved in 3D FER are discussed along with the future research directions. In most of the FER approaches, there is a need for having a neutral (expressionless) face of the subject which might not always be practical. This chapter also presents a novel technique of feature extraction which does not require any neutral face of the test subject. A proposition has been verified experimentally that motion of a set of landmark points on the face, in exhibiting a particular facial expression, is similar in different persons. The presented approach shows promising results using Support Vector Machine (SVM) as the classifier.
TopIntroduction
Computers have now become a part of our day to day lives as they are being widely used for running industries, account keeping, entertainment, health, shopping, communication and so on. It is believed that in the near future the environment around us will embed intelligent devices which will assist us in our day to day activities. E.g. when you come back tired from the office work, your music player will play your favorite musical program and if you are drowsy and want to take a nap, lights will dim automatically.
To make the above scenario a reality, computing devices need to become socially intelligent as well, apart from their computational intelligence (Vinciarelli et. al., 2009). Computers need to understand day to day needs of humans, which may be physical, mental, emotional and so on. This is motivating the researchers to delve into the field of Human Computer Interaction (HCI).
An important avenue in HCI is human emotion recognition which aims at automatically understanding human emotions and present desired responses. Emotion is much of an internal state and sometimes even human beings find it difficult to understand internal feelings of a person. This makes human emotion recognition even much more difficult for computers. From the perspective of computers, there are different modalities reflecting emotions of a person such as facial expression, voice, spoken words, hand and body gestures etc. Out of these modalities, it has been found that facial expressions of a speaker account for about 55 percent of the effect conveyed in human communication, while 38 percent of the rest is conveyed by voice intonation and 7 percent by spoken words (Pantic and Rothkrantz, 2000). Considering the importance of facial expressions in conveying emotions, automatic Facial Expression Recognition (FER) is developing as an important area of research.
Applications of Facial Expression Recognition
FER enables automation of services that require a good appreciation of the emotional state of the user. For example, if we understand the emotion of customers, a system can recommend products that they may be interested in. Similarly, there are many other areas where FER finds useful applications.
Facial expressions have been widely used in clinical research to study schizophrenia, which is a neuropsychiatric disorder in which patients have difficulty in recognizing and expressing emotions. Techniques of facial expression recognition have been used to analyze such abnormalities. Research has also been performed to recognize the facial expressions of epileptic patients while they undergo seizures. This helps in understanding the cerebral organization when seizures take place.
In academia, FER is used for understanding receptivity of students towards an automated tutoring system. Facial expressions of the students can reflect whether he finds the lecture interesting or not. Based on this feedback, the tutoring system adjusts the speed of instruction; slowing down if the student is bored and speeding up if student is grasping well.
In the field of advertising, FER is used for understanding the emotional responses of consumers towards television advertisements or towards different consumer products while they are shopping on the internet. Recent application of FER has come up in the form of development of technologies such as targeted advertisement where the advertisement on the billboards adapts to the facial expressions of the viewer.
FER techniques are helpful in making robots more social. Recently robots are being designed in such that they are able to interact socially with each other and also with humans. The capability of FER is very helpful in facilitating robots to communicate with humans because for humans, the face to face communication is a real-time process operating in times of the order of 40 milliseconds which is very fast for robots (Bartlett et al., 2003)
As far as surveillance is concerned, it has been performed mainly using face recognition rather than facial ‘expression’ recognition. Generally face recognition algorithms are effective only as long as the subject doesn’t exhibit any expression. This is not easy to achieve especially in camera surveillance systems where people are unaware of being tracked. FER is very useful in these situations. FER systems have been developed for such real-time situations. Another security application is to hide the identity of an internet user during a chat without hiding his expressions. In such a case, the person’s expressions are communicated using an animated avatar.