As the first step of many visual speech recognition and visual speaker authentication systems, robust and accurate lip region segmentation is of vital importance for lip image analysis. However, most of the current techniques break down when dealing with lip images with complex and inhomogeneous background region such as mustaches and beards. In order to solve this problem, a Multi-class, Shapeguided FCM (MS-FCM) clustering algorithm is proposed in this chapter. In the proposed approach, one cluster is set for the lip region and a combination of multiple clusters for the background which generally includes the skin region, lip shadow or beards. With the spatial distribution of the lip cluster, a spatial penalty term considering the spatial location information is introduced and incorporated into the objective function such that pixels having similar color but located in different regions can be differentiated. Experimental results show that the proposed algorithm provides accurate lip-background partition even for the images with complex background features.
TopIntroduction
Visual speech recognition has aroused the interest of many researchers (Chan 2001, Kaynak et al. 2001, Zhang et al 2001). The visual information of lip movement can help enhancing the accuracy of automatic speech recognition systems especially in noisy environments (Petajan 1985, Bregler et al. 1993). Accurate and robust lip region segmentation, as the first step of most lip extraction systems, is of key importance for subsequent processing.
Lip region segmentation aims to classify all the pixels in an image into two categories: the lip pixels and the background (non-lip) ones, and various techniques have been proposed to address this problem. In recent years, segmentation of color lip images has gained more popularity than segmentation from gray-scale images due to the availability of low-cost hardware and increasing computing power. Color can provide additional information that is not available in gray-scale images and thus it enhances the robustness of the lip segmentation algorithm. In addition, it is also easier for detecting the teeth and tongue, which are important for extracting the lip region accurately.
Various lip image segmentation methods have been proposed in the literature. Color space analyses such as preset color filtering (Wark et al. 1998) and color transformation (Eveno et al. 2001) have been used to enlarge the color difference between the lip and skin. Nevertheless, this kind of methods will result in large segmentation error if the color distribution of lip region overlaps with that of background region. Edge detection algorithms (Hennecke et al 1994, Caplier 2001) can produce accurate result if prominent and consistent intensity changes around the boundary exist. However, this condition may not be easily satisfied for people with low color contrast between the lip and skin. Spatial continuity has also been exploited in Markov random field based techniques to improve the robustness of segmentation (Lieven and Luthon 1999, Zhang and Mersereau 2000). These algorithms can reduce the segmentation error caused by “pepper” noise. Fuzzy c-means (FCM) clustering is another kind of widely used image segmentation techniques (Bezdek 1981). In FCM-based methods, neither prior assumption about the underlying feature distribution nor training is needed.
The methods mentioned above all produce satisfactory results to a certain extent for lip image without mustache or beards. However, most of them fail to provide accurate lip segmentation for lip images with beards. We have previously proposed a fuzzy clustering based algorithm that takes into consideration the lip shape, i.e., fuzzy c-means with shape function (FCMS) (Leung et al 2004), to segment the lip region. The FCMS exploits both the shape information and the color information to provide accurate segmentation results even for lip images with low contrast. However, it still fails for image with a complex background due to the insufficient background modeling.