Context-Aware Mobile and Wearable Device Interfaces

Introduction

Mobile devices in the form of Smartphones have become a part of our everyday life. We expect them to perform a variety of tasks that cover a much broader field than just making phone calls. Equipped with applications for many tasks, these small computer systems are meant to adapt to our needs and give us access to many functions that are relevant to our current situation. While many people find using these devices easy their interfaces are very different from what is present in typical desktop computing systems. Compared to normal desktop computing environments or notebooks, a mobile device has many limitations in terms of computing power, information density and input mechanisms. By using a different approach in the design of mobile applications these limitations can be overcome or mitigated. While the first mobile digital assistants still tried to resemble small versions of desktop applications with windows, icons, menus and pointer metaphors, it became quickly clear that this approach did not really fit the circumstances. Nowadays application design reflects the state of mind of the user and incorporates dynamic sources of information. Location information, software generated data or sensors adapt the interface.

Devices will become more and more invisible and integrated into clothing of the user or worn by the user. While mobile applications still require a form of active use, future devices will provide a different kind of supporting functionality. To achieve this goal, the mobile interface paradigm has to be generalized into something different to mechanisms for information display and user input as used so far. For a wearable application interface the use of implicit information sources will be a central design aspect where the user’s needs are anticipated and only a minimum of explicit input is required.

Before diving deeper into the topic, a short definition and history of wearable computing is needed to understand how the user interaction in this field demands changes to the traditional concept of assessing a user interface (UI).

Wearable computing describes body worn computing devices that provide useful information relevant to a physical real world task. Unlike mobile devices, the focus is on the task and the device should not unnecessarily distract from it. To fulfill this expectation the wearable device cannot depend on direct interaction with the wearer but needs to process information from the environment to infer the next helpful action to be taken. This relevant information is the context of a wearable computing application. There are many sources for contextual information: One can use sensors that measure physical properties like location, lighting conditions, the presence of digital markers to name a few. Other possibilities are known aspects of the work-flow and the resulting mental model of the user to take pro-active actions, software generated information from databases, current time, simulations, or implicit interaction by analyzing the actions of the wearer or co-workers.

The first wearable computing system, built with the intention of supporting a real world task, can be found in the experiments on Roulette wheel prediction by Thorp (Thorp, 1998) as early as 1955. While the core idea is quite simple, it shows some general needs of wearable applications. To predict the outcome of Roulette wheels, a shoe-integrated computer was used where timing information about the current game was input with the toes. The result of the computation was transmitted to another device¹ worn by a (potentially different) person and mediated by audible signals. Analyzing the needed information here can be quickly done. The computer system needs timing information that is abstracted by keeping track of the time when a button is pressed. The transmitted result is a single symbol that either specifies where to place the bet or not to bet at all. While the setup in this early experiment is technically very direct, it can be logically abstracted. On the one side is an action from the user such as pressing a button that is not used directly as a means of control but indirectly used to setup a timespan, very similar to the control scheme of a stop watch. On the other side, an abstract output symbol is computed that needs to be conveyed to the wearer. While this example is very simple, it shows one important property. Neither the actual physical source of the input, nor the actual physical representation of the output are important for the application and can be replaced by other means. This has actually been done with the Eudaemons' shoe (The Eudaemons' Shoe, 1998), a project directly related to the work of Thorp. In this project, instead of an audio output, a tactile feedback device was used to convey the computed symbol to the wearer.