Ubiquitous User Interfaces: Multimodal Adaptive Interaction for Smart Environments

Ubiquitous User Interfaces: Multimodal Adaptive Interaction for Smart Environments

Marco Blumendorf (Technische Universität Berlin, Germany), Grzegorz Lehmann (Technische Universität Berlin, Germany), Dirk Roscher (Technische Universität Berlin, Germany) and Sahin Albayrak (Technische Universität Berlin, Germany)
DOI: 10.4018/978-1-60566-978-6.ch002
OnDemand PDF Download:


The widespread use of computing technology raises the need for interactive systems that adapt to user, device and environment. Multimodal user interfaces provide the means to support the user in various situations and to adapt the interaction to the user’s needs. In this chapter we present a system utilizing design-time user interface models at runtime to provide flexible multimodal user interfaces. The server-based system allows the combination and integration of multiple devices to support multimodal interaction and the adaptation of the user interface to the used devices, the user and the environment. The utilization of the user interface models at runtime allows exploiting the design information for advanced adaptation possibilities. An implementation of the system has been successfully deployed in a smart home environment throughout the Service Centric Home project (www.sercho.de).
Chapter Preview


Computer technology is currently changing our lives and the way we handle technology. The computer moves from a business machine dedicated to specific tasks in a well defined environment to a universal problem solver in all areas of live. Powerful mobile devices that are always online, and the ongoing paradigm shift towards ubiquitous computing concepts (Weiser, 1993) provide increasingly complex functionality and allow remote access to additional services and information. Wireless ad-hoc network technologies and the upcoming Internet of Things (ITU, 2005) drive the trend to local networks and smart environments. This poses challenges to applications and their user interfaces that now have to support various situations instead of the well known scenario of the user sitting in front of his desk. The widespread use of computers in all areas of life also continuously affects new groups of users. As their number grows, so does their diversity, with each user having different personal preferences, different experience levels and different capabilities.

Smart environments confront user interfaces with a variety of available (mobile) interaction resources supporting diverse modalities, and heterogeneous users with different capabilities and preferences. A user interface supporting smart environments requires a high degree of adaptability to innumerable contexts of use. Unfortunately, today’s user interfaces do not sufficiently support the creation of ubiquitous systems and smart environments and a significant improvement of the communication, interaction and adaptation capabilities is required. At the same time the user must be given the power of understanding and controlling her smart environment in a flexible and comprehensible way. We therefore see the need for Ubiquitous User Interfaces (UUIs) addressing the challenges of the ubiquitous computing paradigm within the following dimensions:

  • multi-situation – support of multiple UI layouts for different usage contexts;

  • multi-device – support for the usage of multiple devices simultaneously (or sequentially);

  • multi-modal – support for multiple interaction modalities according to the needs of the interaction;

  • multi-user – support to share applications, information and interaction devices between multiple users;

  • multi-application – support to use multiple applications per user and device simultaneously and sequentially.

Based on these five features we define Ubiquitous User Interfaces as interfaces that are shapeable, distributable, multimodal, shareable and mergeable. In the remainder of this chapter we focus mainly on the aspects related to multi-situation, multi-device, and multi-modal. Furthermore, we address the alteration of the user interface configuration at runtime, often denoted as adaptation. We thus address:

  • The adaptation of the user interface to the used device(s), the user’s needs and the environment either directly by the user or automatically by the application.

  • The ability of the user interface to be distributed across multiple devices and modalities.

  • The capability of the user interface to change the currently used devices by migrating the whole UI or parts of it to a different device.

Complete Chapter List

Search this Book: