Methods for the Recognition of Human Emotions Based on Physiological Response: Facial Expressions

1. Introduction

Human emotions provide a glimpse into the inner thoughts of a person. Humans are capricious and their emotions fluctuate and depend upon the situation and the circumstances. It is to be noted that even the intensity of a particular emotion varies from situation to situation. An accurate assessment of emotions provides a fair idea of one's interest or disinterest in a particular product or activity and behavior in general. This knowledge may be utilized by business organizations to streamline their strategies or institutions to prepare robust policies. And, it is also possible to predict one’s next course of action by real-time detection of emotions. This helps in the prevention of potential suicides or attacks and thus, is advantageous in saving human lives. Researchers face a difficult task when it comes to the detection of human emotions. Human sentiments are generally recognizable through facial expressions, but this is not the case all the time. Thus, parameters like a heartbeat, brain waves, pupil size, facial warmth, and others may be captured/measured for the determination of human emotions. Humans often produce distinctive facial gestures for a particular mood; the intensity may vary, and these can be easily ascertained. Certain human emotions, on the other hand, may have similar physical morphology but represent different moods and intensities (Singh, 2012). Detection of human emotions is a key area of research. The findings of the research are utilized by various industries for business purposes and by civic bodies for administration purposes.

A few areas, in which human emotion detection is utilized, are listed below:

• •

  Robotics: To create intelligent and interactive playful toy robots which can behave in a particular way in response to a certain human gesture? An example will be the Sony AIBO Dog robot.

• •

  Marketing: To add some dynamic and immersive effects to advertisement boards following the emotional state of a consumer to reach out to the consumer in a targeted manner.

• •

  Education: To enhance the knowledge transfer and the teaching-learning process by assessing the learners ‘perception from their emotions.

• •

  Entertainment: To provide selected content to the audience based on their mood.

• •

  Surveillance: To monitor suspicious activities and combat potential threats by reading the emotions of a crowd.

• •

  Suicide Prevention: To prevent possible suicide attempts by assessing the state of mind of a person out in the public.

In most cases, detection of human emotions at a large scale is required. Hence, it is impractical to do the same manually. Therefore, automated systems for the detection of human emotions are required. Recent advancements in the automation industry have made it possible to build such automated systems with state-of-the-art sensors (Dzedzickis et al., 2020; Feidakis, et al., 2011) state that there are a total of 66 human emotions as per the fundamental taxonomy. Out of these, 10 emotions are deemed to be the fundamental ones and the rest are considered to be the secondary ones. Figure 1 depicts the fundamental emotions as described in (Feidakis et al., 2011) Detection of such a large number of human emotions is a difficult endeavor even with an automated system. Also, a captured/measured parameter is likely a result of mixed emotion, or the same physiological or facial parameter may point to more than one emotion. To avoid such ambiguities, a large number of emotion detection studies have been carried out with consideration of some other/further classification of human emotions (Feidakis et al., 2011). This explores the motional dimensions like activation and arousal, in which it is observed that whether a certain emotional element alleviates or aggravates one’s mood (Russell, 1980; Csikszentmihaly, 2014).