Automated Face Analysis: Emerging Technologies and Research

Automated Face Analysis: Emerging Technologies and Research

Daijin Kim (Pohang University of Science & Technology, Korea) and Jaewon Sung (LG Electronics, Korea)
Indexed In: SCOPUS View 1 More Indices
Release Date: March, 2009|Copyright: © 2009 |Pages: 448
ISBN13: 9781605662169|ISBN10: 160566216X|EISBN13: 9781605662176|DOI: 10.4018/978-1-60566-216-9


Since research on face recognition began in the 1960's, the field has rapidly widened to automated face analysis including face detection, facial gesture recognition, and facial expression recognition.

Automated Face Analysis: Emerging Technologies and Research provides theoretical background to understand the overall configuration and challenging problem of automated face analysis systems, featuring a comprehensive review of recent research for the practical implementation of the analysis system. A must-read for practitioners and students in the field, this book provides understanding by systematically dividing the subject into several subproblems such as detection, modeling, and tracking of the face.

Topics Covered

The many academic areas covered in this publication include, but are not limited to:

  • AdaBoost learning
  • Automated face analysis
  • Eye detection
  • Face certainty map
  • Face disguise discrimination
  • MCT-based feature correlation
  • Modified census transform
  • Robust eye detection
  • Role of face in human communication

Reviews and Testimonials

This book combines all related subjects such as face and eye detection, face modeling, face tracking, face recognition, facial expression recognition, and facial gesture recognition into one book, as well as, provides the introductory theoretical backgrounds for each subject.

– Daijin Kim, Pohang University of Science & Technology, Korea

This book surveys recent research in areas of the field such as face and eye detection, face modeling, and facial expression recognition, as well as related subjects such as hand-gesture and body-gesture recognition.

– Book News Inc. (June 2009)

Table of Contents and List of Contributors

Search this Book:


According to neuropsychology, the human face is believed to be the most important means for human communication among all social communication instruments. Of the information that the human face carries, the identity information is the most valuable. By recognizing the idenity of a person, we feel comfortable with their familiar face, sometimes uncomfortable with unfamiliar ones as when a baby cries when a strange face shows up, and recall our memories what we have talked with the person, which brings rich backgrounds and context information for smooth conversation.

Facial expressions provide an important means of non-verbal comunication in human communcation. They reflect the internal emotional state of the individual including a smile (happy), a frown (worry), a glare (anger), a snarl (anger), a stare (curiosity), a grimace (pain), ecstasy, anger, fear, doubt, confidence and threat. Besides these, facial gestures provide other non-verbal communication in human communications. By their nodding, shaking, dozzing, or winking, we can understand the intention or status of the other person. Sometimes, we express a sad or frustrated feeling by just looking down or looking up at the sky without moving the face for a while.

Since Takeo Kanade began to develop a face recognition algorithm 30 years ago, many researchers have performed research of automated face analysis extensively and achieved a great number of remarkable accomplishments. Thanks to their endeavour, the automated face analysis technique has been widely applied to many fields such as biometircs, security and surveillance systems, clinical psychology and psychiatry, biomedical applications, human computer interaction (HCI), human robot interaction (HRI), lip reading (to assist the speech recognition in noisy environments), low bit rate video coding of the face for telecommunication, avatar systems, and entertainment.

This book was written to meet three primary requirements. First, we want to put together all related subjects such as face and eye detection, face modeling, face tracking, face recognition, facial expression recognition, and facial gesture recognition into this book. Second, we provide the introductory theoretical backgrounds for each subject, which help the reader’s understanding greatly. Third, beyond the faces, we add some other subjects such as the human detection, hand gesture recognition, and body gesture recognition, which are closely related to the automated face analysis.

The book is intended for students and practioners who plan to work in the automated face analysis field or who want to become familiar with state-of-the-art automated face analysis. This book provides plenty of references for scientists and engineers working in related research fields such as image processing, computer vision, biometrics, security, computer graphics, human-computer interaction (HCI), human-robots interaction (HRI), or the computer game industry. We have also provided the summary of many current databases and demonstration systems; of face, eye, facial expression, and gesture, in Appnedix A and B, respectively, which are essential for doing the related researches. The material fits a variety of categories such as advanced tutorial, guide to the current technology, and state-of-the-art survey.

The book consists of eight chapters, covering all the major components and subareas required to understand the emerging technologies and research in automated face analysis. Each chapter focuses on a specific part of the automated face analysis, introduces the required theoretical background, reviews the related up-to-date techniques, presents the experimental results, and points out challenges and future research directions.

Chapter I presents the roles of the face in human communication and the goals of the automated face analysis.

Chapter II presents the face and detection including the theorectial background such as the AdaBoost learning technique and the modified census transform (MCT). We emphasize how to improve the face and eye detection performance by introducing the face certainty map and the MCT. We also introduce the face disguise discrimination technique using the AdaBoost learning of eye/non-eye and/or mouth/non-mouth.

Chapter III presents face modeling including the theoretical background such as the active shape models (ASMs) and the active appearance models (AAMs). We emphasize how to make the face modeling robust by introducing stereo-based AAM, view-basee AAM, and a unified gradient-based approach for combining ASM into AAM.

Chapter IV presents face tracking including the theoretical background such as particle filters, the cylindrical head model (CHM), and the incremental PCA (IPCA). We emphasize how to make the face tracking robust to the changes of background, pose, illumination, and moving speed by introducing: background-robust face tracking using AAM and ACM, pose-robust face tracking using AAM and CHM, illumination-robust face tracking using AAM and IPCA, and fast and robust face tracking using AAM in particle filter framework.

Chapter V presents face recognition including the theoretical background such as the mixture models, the embedded hidden Markov models (HMMs), the local feature analysis (LFA), the tensor analysis, and the 3D morphable models (3D MM). We present a variety of face recognition methods using the mixture model, the embedded HMM, the LFA, the tensor-based AAM, and the 3D MM, and compare the pros and cons of each face recognition method.

Chapter VI presents the facial expression recognition including the theoretical background such as the generalized discriminant analysis (GDA), the bilinear models, and the relative expression image. We present the facial expression recognition using the AAM features and the GDA, and the natural facial expression recognition using the differential-AAM and the manifold learning. Also, we present the facial expression synthesis using the AAM and the bilinear model.

Chapter VII presents the facial gesture recognition including the theoretical background such as the hidden Markov model (HMM). We present how to recognize facial gestures like nodding, denying, and blinking by combining CHM and HMM and apply it to control a TV set.

Chapter VIII presents the human detection, the hand gesture recognition, and the body gesture recognition beyond the automated face analysis, including the theoretical background such as the scale adaptive filters (SAFs) and the iterative closest point (ICP) algorithm. We present a variety of the human motion analysis techniques such as the human detection using the pose-robust SAFs, the hand gesture rcognition using MEI and MHI, the 2D body gesture recognition using the forward spotting accumulative HMMs, and the 3D body gesture recognition using 3D articulated human body model.

Author(s)/Editor(s) Biography

Daijin Kim received the BS degree in electronic engineering from Yonsei University, Seoul, Korea, in 1981, and the MS degree in electrical engineering from the Korea Advanced Institute of Science and Technology(KAIST), Taejon, 1984. In 1991, he received the PhD degree in electrical and computer engineering from Syracuse University, Syracuse, NY. During 1992–1999, he was an Associate Professor in the Department of Computer Engineering at DongA University, Pusan, Korea. He is currently a Professor in the Department of Computer Science and Engineering at POSTECH, Pohang, Korea. His research interests include intelligent systems, biometrics, and Human Computer Interaction.
Jaewon Sung received his BS degree in computer engineering from Chungnam National University, Taejeon, Korea (2000) and his MS and PhD degrees in computer science and engineering from Pohang University of Science and Technology (POSTECH), Pohang, Korea (2002 and 2007, respectively). He was a postdoctoral researcher at POSTECH in 2007. He is currently a senior research engineer of LG electronics, Korea. His research interests include pattern recognition, machine vision, and video compression and coding.