Facial Expression Analysis Using 3D Range Images

Introduction

Face recognition is an important issue today in the field of computer vision. Face recognition relies mainly on the quality of data resource. A face can be 2D or 3D, and static or dynamic. From the implementation task point of view, face data is used for face recognition, occlusion detection, expression recognition and similar other uses. Over the past few years, many related works have been based on expression recognition and authors have made efforts to build face recognition systems which can detect facial expressions like pain and mood, as well as of more subtle emotions such as embarrassment, amusement and shame. It is, therefore, indeed necessary, to develop a robust expression recognition system.

With the advancement of 3D technologies, compared to 2D faces, 3D faces have been used for face information analysis. The problem which lies with the majority of existing expression recognition systems is that, majority of existing recognition systems are based on 2D images. More specifically, advanced research has been conducted to model the temporal relationships of different expressions. But, none of the prevalent systems aims to use the nature of facial expressions and use this for recognition purpose. Of late, facial recognition systems have been developed that, explicitly recognize and model the temporal segments of either full expressions or components of expression such as facial action units. These systems make use of 2D face images, they also find the kind of motion between frames using feature based or appearance based methods to perform classification and modeling. Unfortunately, these systems present in history are sensitive to illumination, pose and occlusions like makeup, sunglasses, etc.In most cases, when 2D faces are used, it is necessary to maintain a frontal facial pose, in order to achieve a better recognition. Facial pose, hence, can simultaneously reduce the recognition rate of the existing systems. So, in order to address the challenges of accuracy and pose, extreme poses in case of 3D systems must be employed. Today, with the advances in 3D structured light scanning, stereo photogrammetric, the acquisition of 3D facial structure and capturing of facial pose is now, an easy task.

There have been previous works using the concept of 2Dface images for the reconstruction of 3D models, in order to extract 3D features that could be used for classification of the facial expression, but these methods are dominated and corrupted by problems of illumination and pose inherent to all 2D methods.

Our main contribution in this proposed work is to exploit, the geometry of 3D faces which deal with expressions e.g. sad, happy, exclamation, surprise, with an exclusive combination of expression and pose.

In this context, a review of the majority of work done in the field of 3D expression analysis using Hidden Markov’s model, PCA, LDA, etc. will also be done.

Also, we shall propose a new system where we shall consider a combination of expression and pose, and we shall propose methods for recognizing 3D faces with expressions.

Normally, there is a devoid of systems in the field of 3D face recognition, which works with a combination of expression and pose. In analyzing facial expressions, it is indeed necessary to understand which facial features change under expressions and pose. For example, the nose region does not change under facial expressions but, the region surrounding mouth changes vigorously. Hence, it is needed to extract the areas surrounding the mouth, in order to analyze the expression.

In case of expression, faces become ill-posed, and so it is also necessary for 3D faces to be registered because the recognition rate diminishes if the 3D faces are not in frontal pose. Also, expression faces provide a rich set of features. By extracting those features, it is possible to demonstrate the efficiency of recognition rate.

In this present work, a framework for statistical shape analysis of facial surfaces for 3D expression faces has been proposed. Methods that make it appropriate for 3D face recognition in non-cooperative scenarios will also be discussed in this work. So, two major issues have been handled in this proposed work:

• 1.

  Firstly, to handle expressions with poses.

• 2.

  Secondly, to recognize faces in the presence of expressions.

• 3.

  Finally, to reconstruct registered neutral faces from expression faces.