Visual Speech Perception, Optical Phonetics, and Synthetic Speech

Impediments To Knowledge About Optical Phonetics And Visual Speech Perception

One impediment to knowledge about visible speech is the presupposition that, “A relatively small proportion of the information in speech is visually available” (Kuhl & Meltzoff, 1988, p. 240). We would not be surprised if researchers were not drawn to study a signal with little intrinsic information value: The impoverished visual speech stimulus might not deserve the same rigorous approach as that which has been applied to the acoustic speech signal. Indeed, in general, visible speech signals afford inherently fewer phonetic cues than audible signals, particularly, when listening conditions are favorable. Many of the activities of the vocal tract (Catford, 1977) that contribute to the acoustic speech signal (Stevens, 1998) are hidden from view. Paradoxically, the listener has greater access to vocal tract shape and activity than does the viewer, because the acoustic waveform is affected by all of the vocal tract settings.

For example, the hidden actions of the velum are critical to the control of nasality (e.g., the distinction between /b/ with the velum raised versus /m/ with the velum lowered). The hidden actions of the glottis contribute to voicing distinctions (e.g., in pre-vocalic /b/, for which glottal vibration is initiated earlier, versus /p/). The degree to which the tongue causes air flow restriction is only partly visible and affects manner of articulation (e.g., glide consonants, such as /w/, have less restricted air flow versus stop consonants, such as /b/). The position and shape of the tongue are also responsible for the different vowels (Catford, 1977). Nevertheless, research has shown that the talking face is a rich source for phonetic information.