Article Preview
TopIntroduction
Speech comprises of a mixture of audio frequencies, and every speech sound belongs to one of the two main classes known as vowels and consonants. Vowels and consonants belong to the basic linguistic units known as phonemes, which can be mapped to visible mouth shapes called visemes. Visemes and phonemes can be used as the basic units of visible articulatory mouth shapes (Waters et al., 1993). A phoneme is an abstract representation of a sound, and the set of phonemes in a language is defined as the minimum number of symbols required to represent every word in that language (Breen et al., 1996).
The set of visemes in a language is often defined as the number of visibly different phonemes in that language. A simple definition of viseme would be that a viseme could be generated from a set of archetypal sounds in a language based on the phonemes of that language (Breen et al., 1996). Möttönen et al. (2000) define a viseme set as phoneme realizations that are visually inextricable from each other. In lip reading studies viseme categories can be defined as clustering response distributions to observed phoneme articulations. These clusters are then used to find those phoneme articulations, which are perceived to be similar. Typically a cluster is considered as a viseme category if the proportion of within cluster responses is at least 70% of the total amount of responses to the phoneme articulations that are included in the cluster (Möttönen et al., 2000).
To date, there has been no precise definition for the term, but in general it has come to refer to a speech segment that is visually contrastive from another (Möttönen et al., 2000; Ezzat et al., 1999). In their work, they define a viseme to be a static lip shape image that is visually contrastive from another.
It is also important to point out that that the map from phonemes to visemes is also one-to-many: the same phoneme can have many different visual forms. This phenomenon is termed coarticulation, and it occurs because the neighboring phonemic context in which a sound is uttered influences the lip shape for that sound (Ezzat et al., 2000).
However, current literature on viseme research indicates that the mapping between phonemes and visemes is many-to-one: there are many phonemes which look alike visually, and hence they fall into the same visemic category. This is particularly true, for example, in cases where two sounds are identical in manner and place of articulation, but differ only in voicing characteristics (Ezzat et al., 2000).
Several standards for visemes (Tiddeman et al., 2002) exist but they were developed for English language. Since each language comprises of a different phonetic set (sounds) therefore, visemes have to be identified separately for each language. Möttönen et al. (2000) developed a Finnish talking head, which had to identify Finnish visemes for each Finnish phoneme (Rafay et al., 2003; Ezzat et al., 1999).
Bastanfard et al. (2009) proposed a novel method adopting an image-based approach for grouping visemes in Persian language considering coarticulation effect. In their work, visemes can be recognized in two ways: Model-based, and image-based. In model based approach, lip is identified by its parameters, in that various geometrical factors are specified to lip shape. Model-based approach has low accuracy and high execution time while the iteration of this algorithm may cause local minimum. In image-based approach, lip images are considered, where all work is done on lip pixels (Bastanfard et al., 2009).