3D facial reconstruction is an emerging and interesting application in the field of computer graphics and computer vision. It is difficult and challenging to reconstruct the 3D facial model from a single photo because of arbitrary poses, non-uniform illumination, expressions, and occlusions. Detailed 3D facial models are difficult to reconstruct because every algorithm has some limitations related to profile view, fine detail, accuracy, and speed. The major problem is to develop 3D face with texture of large poses, wild faces, large training data, and occluded faces. Mostly algorithms use convolution neural networks and deep learning frameworks to create facial model. 3D face reconstruction algorithms used for application such as 3D printing, 3D VR games and facial recognition. Different issues, problems and their proposed solutions are discussed. Different facial dataset and facial 3DMM used for 3D face reconstructing from a single photo are explained. The recent state of art 3D facial reconstruction and 3D face learning methods developed in 2019 is briefly explained.
TopIntroduction
Mostly algorithms for 3D facial shape is used only for small poses, medium poses, and uniform illumination but challenge comes when there is profile view and occluded view. Current algorithms and system are using multiple images as an input and getting more fine details as compared to using single image as an input. The computational cost of multi-view algorithm is always high. Constructing the 3D facial model from a single photo is still a problem and researchers are going to solve that problem. Existing algorithms of facial reconstruction are designed that are based on variety of models such as landmark-based face model and 3D Morphable Models (3DMM). This paper focuses on 3D facial reconstruction from single image using deep neural network architecture. Single photo face reconstruction is rarely employed by modern face recognition systems. 3DMM is the most popular method for reconstructing the full 3D facial model from a single photo. CNN has applications in the field of image classification, object detection and extract meaningful information from photos. Currently, there are four popular 3D facial deformation models which are publicly available. Bolkart et al. (Brunton, 2014) developed the first 3D face model that has neutral different subjects, Imperial College LSFM (Booth, 2018) built the large-scale face model built by, which contains faces of different races and ages. The BFM (Paysan, 2009) model proposed by the University of Basel. University of Surrey, UK (Huber, 2016) developed the multi-resolution 3D face model. There are following 3D facial reconstruction techniques where 3D Morphable Models created from High-quality Scans. 1) Monocular 3D Reconstruction. 2) 3D Reconstruction via Photo-collections. 3) Multi-frame 3D Reconstruction.
CNN-based methods (Jourabloo, Liu, 2016) (Zhu, 2016) (Liu, 2016) (Liu, 2017) (Tran, 2017) (Tran, 2018) are expensive because they need a lot of training data with labeled the 3D faces which is very expensive to collect, They generally lacking in face appearance, expression, occlusions and environment conditions, and limiting the generalization performance of resulted 3D estimated facial models. Some recently algorithms used to reconstruct the 3D facial model by bypass the 3DMM coefficient. They use image to image (Feng, 2018) and image to volume (Jackson, 2017) strategy but they are lack in ground truths. The proposed algorithms (Richardson, 2016) (Richardson, 2017) reconstructed the 3D facial model with using CNN cascaded structure that consume too much time because of their CNN has multi stages. These methods (Dou, 2017) (Tan, 2017) (Jourabloo, Liu, 2015), used the holistic approach to regress the 3DMM parameters. Jackson et al. (Jackson, 2017) proposed an algorithm that is used to reconstruct the 3D face model by mapping the image pixels to a voxel representation. This method does not need a 3DMM but needs a complex structure takes a lot of time to get voxel information. Feng et al. (Feng, 2018) trained the image to image CNN approach that stored the 3D model into UV position map and reconstruct the 3D facial from a single image. 3D facial reconstruction from a photo collection (Kemelmacher-Shlizerman, 2013)(Liang, 2016)(Suwajanakorn, 2014) does not 3DMM face models.