Learning 3D Face Deformation Model: Methods and Applications

Learning 3D Face Deformation Model: Methods and Applications

Zhen Wen (University of Illinois at Urbana Chapmaign, USA), Pengyu Hong (Harvard University, USA), Jilin Tu (University of Illinois at Urbana Champaign, USA) and Thomas S. Huang (University of Illinois at Urbana Champaign, USA)
DOI: 10.4018/978-1-59140-299-2.ch010
OnDemand PDF Download:


This chapter presents a unified framework for machine-learning-based facial deformation modeling, analysis and synthesis. It enables flexible, robust face motion analysis and natural synthesis, based on a compact face motion model learned from motion capture data. This model, called Motion Units (Muss), captures the characteristics of real facial motion. The MU space can be used to constrain noisy low-level motion estimation for robust facial motion analysis. For synthesis, a face model can be deformed by adjusting the weights of Mus. The weights can also be used as visual features to learn audio-to-visual mapping using neural networks for real-time, speech-driven, 3D face animation. Moreover, the framework includes parts-based MUs because of the local facial motion and an interpolation scheme to adapt MUs to arbitrary face geometry and mesh topology. Experiments show we can achieve natural face animation and robust non-rigid face tracking in our framework.

Complete Chapter List

Search this Book:
Table of Contents
Chapter 1
Angel Sappa, Sotiris Malassiotis
This chapter presents a survey of the most recent vision-based human body modeling techniques. It includes sections covering the topics of 3D human... Sample PDF
Advances in Vision-Based Human Body Modeling
Chapter 2
Marius Preda, Ioan A. Salomie, Françoise Preteux, Gauthier Lafruit
Besides being one of the well-known audio/video coding techniques, MPEG-4 provides additional coding tools dedicated to virtual character animation.... Sample PDF
Virtual Character Definition and Animation within the MPEG-4 Standard
Chapter 3
B. J. Lei, E. A. Hendriks, Aggelos K. Katsaggelos
This chapter presents an extensive overview of passive camera calibration techniques. Starting with a detailed introduction and mathematical... Sample PDF
Camera Calibration for 3D Reconstruction and View Transformation
Chapter 4
Burak Ozer, Tiehan Lv, Wayne Wolf
This chapter focuses on real-time processing techniques for the reconstruction of visual information from multiple views and its analysis for human... Sample PDF
Real-Time Analysis of Human Body Parts and Gesture-Activity Recognition in 3D
Chapter 5
Kostas Karpouzis, Amaryllis Raouzaiou, Athanasios Drosopoulos, Spiros Ioannou, Themis Balomenos, Nicolas Tsapatsoulis
This chapter presents a holistic approach to emotion modeling and analysis and their applications in Man-Machine Interaction applications. Beginning... Sample PDF
Facial Expression and Gesture Analysis for Emotionally-Rich Man-Machine Interaction
Chapter 6
Ana C. Andres del Valle, Jean-Luc Dugelay
This chapter presents a state-of-the-art compilation on facial motion and expression analysis. The core of the chapter includes the description and... Sample PDF
Techniques for Face Motion & Expression Analysis on Monocular Images
Chapter 7
Peter Eisert
In this chapter, the state-of-the-art in facial animation and expression analysis is reviewed and new techniques for the estimation of 3-D human... Sample PDF
Analysis and Synthesis of Facial Expressions
Chapter 8
Gregor A. Kalberer, Pascal Müller, Luc Van Gool
The problem of realistic face animation is a difficult one. This is hampering a further breakthrough of some high-tech domains, such as special... Sample PDF
Modeling and Synthesis of Realistic Visual Speech in 3D
Chapter 9
Markus Kampmann, Liang Zhang
This chapter introduces a complete framework for automatic adaptation of a 3D face model to a human face for visual communication applications like... Sample PDF
Automatic 3D Face Model Adaptation with Two Complexity Modes for Visual Communication
Chapter 10
Zhen Wen, Pengyu Hong, Jilin Tu, Thomas S. Huang
This chapter presents a unified framework for machine-learning-based facial deformation modeling, analysis and synthesis. It enables flexible... Sample PDF
Learning 3D Face Deformation Model: Methods and Applications
Chapter 11
Nikos Karatzoulis, Costas T. Davarakis, Dimitrios Tzovaras
This chapter presents a number of promising applications and provides an overview of recent developments and techniques in the area of analysis and... Sample PDF
Synthesis and Analysis Techniques for the Human Body: R&D Projects
About the Authors