Abstract
This chapter introduces an approach for user interaction on mobile devices, focusing on camera-enabled mobile phones. A user interacts with an application by moving their device, and the captured camera video is used to estimate phone motion or interact with the real world. We first survey technical issues, recent research results, and then present a prototype implementation and discuss various ways how phone motion can be used for different tasks, such as navigating through large number of media files, and phone motion and shake detection for gaming. The results and discussion may guide interface designers when targeting camera-based user interfaces.
Key Terms in this Chapter
Augmented Reality: A field of computer research which deals with the combination of the real world with computer generated data.
Ubiquitous Computing: A computing paradigm, where computation is integrated into the environment, rather than having computers as distinct objects. One of the goals of the field is to embed computation into the environment, and allow everyday objects to be used for interaction.
3-D Interaction: A type of user interaction, where the output is displayed in 3-D (on stereoscopic or 2-D displays), and user input is received through 3-D interaction devices.
Zoom Control: Ability to control the zoom level in a user interface or an image or a document.
Camera-Based User Interaction: A type of interaction, where the user interacts with an application by moving their device, and the captured camera video is used to estimate phone motion or interact with the real world.
Gestural Interaction: A type of user interaction, where the user interacts with the computing device through a set of well-defined gestures. Gestures can originate from any bodily motion or state but commonly originate from the face or hand.
Computer Vision: The analysis of image sequences, concerned with computer processing of images from the real world. Computer vision typically requires a combination of low-level image processing and high-level pattern recognition and image understanding to recognize important features in the image.