Improving Interaction with TV-Based Applications through Adaptive Multimodal Fission

Adaptive Multimodal Systems

Dumas, Lalanne, and Oviatt (2009) define multimodal systems as “computer systems endowed with multimodal capabilities for human-computer interaction and able to interpret information from various sensory and communication channels”. These systems offer users a set modalities to allow them to interact with machines and “are expected to be easier to learn and use, and are preferred by users for many applications” (Oviatt, 2003).

Adaptive multimodal systems enable a more effective interaction by adapting to different situations and different users according to their skills, physical or cognitive abilities.

The flexibility of these systems allows them to adapt not only to users but also to the environment (context awareness). For example, the system can use speech to warn or present information in an eyes-busy situation. Combining adaptation capabilities with multimodal interaction can result in an interface even more flexible and natural to the user. In the past, several works have tried to explore this combination as a way to improve interaction. Most of these have targeted adaptation within specific modalities, instead of adaptation across modalities (Duarte & Carriço, 2006).