Three-Dimensional Face Shape by Local Fitting to a Single Reference Model

Introduction

Shape from shading is usually presented as an ill-posed problem, and although attempts to present the problem as well-posed exist (Prados & Faugeras, 2005), it is necessary to employ constraints of some kind in order to find a proper solution. A class of such constraints is obtained by applying dimensionality reduction techniques to large samples of images, usually faces, and their corresponding surfaces. In this context it is considered that there is a low ratio between the sample size and the number of variables needed to apply these techniques (Castelán, Puerto-Souza & Van Horebeek, 2009). Among such techniques are principal component analysis (PCA) applied to depth images (Atick, Redlich & Griffin, 1996), PCA applied to depth and texture images separately (Blanz & Vetter, 1999), PCA applied to coupled intensity and depth images (Castelán, Smith, & Hancock, 2007), searching for latent variables by partial least squares (Castelán & Van Horebeek, 2008) and canonical correlation (Reiter et al., 2006; Castelán et al., 2009).

From another point of view, the face molding method (Kemelmacher & Basri, 2006; Kemelmacher-Shlizerman & Basri, 2011) uses the surface and albedo of a single reference model to find a solution that simultaneously satisfies the reflectance equation for the input image and its similarity to the surface and albedo of the reference model. It does so by applying two regularization terms, one for the difference between the predicted surface and the reference surface, and another one for the difference between the predicted albedo and reference albedo, formulated as a Laplacian of a Gaussian to assure that the difference between the prediction and the model is smooth. To obtain the image surface, which acts as the unknown term in the model, it appeals to the Lambertian reflectance equation using the reference model albedo and a vector of estimated illumination of the image in the form of spherical harmonic coefficients. By using a first order approximation of four spherical harmonics (Kemelmacher & Basri, 2006) at least 87.5% of the variability of energy in the image is captured (Basri & Jacobs, 2003). With a second order approximation of nine spherical harmonics (Kemelmacher-Shlizerman & Basri, 2011) at least 99.2% of the variability of energy in the image is captured (Basri & Jacobs, 2003). Similar reconstruction results are obtained with both approaches, although the first order one of four coefficients is faster and accurate enough, with an average error in illumination estimation of 11.3 ° (Kemelmacher & Basri, 2006). The four coefficients of the first order approximation are estimated by least squares from the set of linear equations that relate the surface normals and albedo of the reference model to the test image. It is equivalent to a description of the illumination by a point source at infinity plus a homogeneous ambient lighting. A drawback of this method is the need for an appropriate reference model for the test image. Kemelmacher and Basri (2011) compared the average of the entire sample of subjects as a reference model to using each of the individual subjects as a reference model, obtaining better results for the average model compared to individual models in the form of mean error.