Improving Interaction with TV-Based Applications through Adaptive Multimodal Fission

Improving Interaction with TV-Based Applications through Adaptive Multimodal Fission

David Costa (University of Lisbon, Portugal) and Carlos Duarte (University of Lisbon, Portugal)
DOI: 10.4018/978-1-4666-4623-0.ch003
OnDemand PDF Download:
List Price: $37.50


In this chapter, the authors describe the design and implementation of an adaptive multimodal fission component integrated in the multimodal GUIDE framework. This component is able to adapt any HTML-based application’s UI to a specific user’s characteristics, making it possible for elderly and impaired users to interact by offering several output modalities that try to overcome possible interaction difficulties. They also present an evaluation of the proposed solution, conducted with more than 50 participants, that shows the efficiency of multimodal adaptation in increasing task perception and task execution.
Chapter Preview

Adaptive Multimodal Systems

Dumas, Lalanne, and Oviatt (2009) define multimodal systems as “computer systems endowed with multimodal capabilities for human-computer interaction and able to interpret information from various sensory and communication channels”. These systems offer users a set modalities to allow them to interact with machines and “are expected to be easier to learn and use, and are preferred by users for many applications” (Oviatt, 2003).

Adaptive multimodal systems enable a more effective interaction by adapting to different situations and different users according to their skills, physical or cognitive abilities.

The flexibility of these systems allows them to adapt not only to users but also to the environment (context awareness). For example, the system can use speech to warn or present information in an eyes-busy situation. Combining adaptation capabilities with multimodal interaction can result in an interface even more flexible and natural to the user. In the past, several works have tried to explore this combination as a way to improve interaction. Most of these have targeted adaptation within specific modalities, instead of adaptation across modalities (Duarte & Carriço, 2006).

Complete Chapter List

Search this Book: