MagiThings: Gestural Interaction with Mobile Devices Based on Using Embedded Compass (Magnetic Field) Sensor

MagiThings: Gestural Interaction with Mobile Devices Based on Using Embedded Compass (Magnetic Field) Sensor

Hamed Ketabdar (Telekom Innovation Labs, Deutsche Telekom Laboratories, TU Berlin, Berlin, Germany), Amin Haji-Abolhassani (Centre for Intelligent Machines, McGill University, Montreal, Canada) and Mehran Roshandel (Telekom Innovation Labs, Deutsche Telekom Laboratories, Berlin, Germany)
Copyright: © 2013 |Pages: 19
DOI: 10.4018/jmhci.2013070102
OnDemand PDF Download:
No Current Special Offers


The theory of around device interaction (ADI) has recently gained a lot of attention in the field of human computer interaction (HCI). As an alternative to the classic data entry methods, such as keypads and touch screens interaction, ADI proposes a touchless user interface that extends beyond the peripheral area of a device. In this paper, the authors propose a new approach for around mobile device interaction based on magnetic field. Our new approach, which we call it “MagiThings”, takes the advantage of digital compass (a magnetometer) embedded in new generation of mobile devices such as Apple’s iPhone 3GS/4G, and Google’s Nexus. The user movements of a properly shaped magnet around the device deform the original magnetic field. The magnet is taken or worn around the fingers. The changes made in the magnetic field pattern around the device constitute a new way of interacting with the device. Thus, the magnetic field encompassing the device plays the role of a communication channel and encodes the hand/finger movement patterns into temporal changes sensed by the compass sensor. The mobile device samples momentary status of the field. The field changes, caused by hand (finger) gesture, is used as a basis for sending interaction commands to the device. The pattern of change is matched against pre-recorded templates or trained models to recognize a gesture. The proposed methodology has been successfully tested for a variety of applications such as interaction with user interface of a mobile device, character (digit) entry, user authentication, gaming, and touchless mobile music synthesis. The experimental results show high accuracy in recognizing simple or complex gestures in a wide range of applications. The proposed method provides a practical and simple framework for touchless interaction with mobile devices relying only on an internally embedded sensor and a magnet.
Article Preview

1. Introduction

Compass, a human made navigational tool, has been widely employed to facilitate the navigation difficulties in the past centuries. An ordinary compass, by itself, is nothing more than a magnetized needle that pivots on an axis and tends to stay aligned with the earth’s north-south magnetic field. Recent developments in electronics, has introduced compact, cheaper and high-performing electronic devices such as magnetometer, gyroscope, and accelerometer. In recent years, digital compass, along with other genre of sensors such as GPS, accelerometer and dual camera have been embedded within the cell phones to enhance the functionalities of the phone. Digital Compass along with GPS has been used to provide navigation to the user. We however show that the usability of the digital compass can be extended beyond navigational applications, providing a new user interaction approach with mobile devices (Ketabdar et al., 2010b).

The electronic magnetic sensor in a mobile device acts like a regular compass. Any slight displacement of the device with respect to the earth’s magnetic field is sensed and registered by the device. A similar type of influence can be imposed upon the magnetic field of the sensor if we slide a permanent magnet around the device. Specifically, a small magnet that moves around the device affects the magnetic field around the sensor, and therefore generates a temporal pattern which changes along the x, y and z axes depending on the movement of the magnet. This pattern can be used to establish a touchless interaction framework as a mean of interaction between the user and the device (Figure 1). In other words, the user generates a specific gesture while moving the magnet, which creates a temporal pattern of change in the magnetic field sensed by the compass sensor. This pattern can then be compared against the pre-recorded templates or pre-trained models in order to recognize the gesture and interpret it as a command.

Figure 1.

Gestural interaction with a mobile device by a magnet taken (or worn) around a finger, based on using embedded compass sensor


This touchless input method addresses some of the limitations commonly associated with traditional input methods such as keypads or touch screens interaction. One of the main restrictions in designing miniature electronic devices is the size of the user input interface that needs to be large enough to comply with the human physical characteristics. A small properly shaped magnet, e.g. in shape of a rod, ring or pen though, can freely move in the 3D space around the device which is considerably broader than the surface of hand held device screen. By properly shaped magnet, we mean a magnetic material that can be taken or worn in finger comfortably and naturally. This can make design of very small handheld devices yet with a proper user interface mechanism feasible. Moreover, the 3D characteristic of the proposed method opens new door for augmented and virtual reality applications on mobile phones. Additionally, since the magnetic field can penetrate through occluding objects, it allows for interactions, even though the device is concealed by other objects, or while the device is inside the user pocket or handbag. For instance, the user may be able to dial a number, enter a pin code, or select a music album without taking the mobile device out of his pocket/bag. Additionally, by the same reason, space at back of the device can be also freely used for interaction (Figure 2). This is in contrary with touch screens where interactions are only possible when the device is in a direct contact with the user.

Figure 2.

Back of device interaction based on magnetic interaction framework. The magnetic field can pass through many covering fabrics, allowing interaction even when the hand is not in the line of sight of the device (Ketabdar et al., 2010b).


Complete Article List

Search this Journal:
Open Access Articles
Volume 13: 4 Issues (2021): Forthcoming, Available for Pre-Order
Volume 12: 4 Issues (2020): 3 Released, 1 Forthcoming
Volume 11: 4 Issues (2019)
Volume 10: 4 Issues (2018)
Volume 9: 4 Issues (2017)
Volume 8: 4 Issues (2016)
Volume 7: 4 Issues (2015)
Volume 6: 4 Issues (2014)
Volume 5: 4 Issues (2013)
Volume 4: 4 Issues (2012)
Volume 3: 4 Issues (2011)
Volume 2: 4 Issues (2010)
Volume 1: 4 Issues (2009)
View Complete Journal Contents Listing