The theory of around device interaction (ADI) has recently gained a lot of attention in the field of human computer interaction (HCI). As an alternative to the classic data entry methods, such as keypads and touch screens, ADI founds a 3D user interface that extends to the peripheral area of a device. In this chapter, the authors introduce a revolutionary interaction framework that is based on the idea of ADI. The proposed method constitutes a touchless data entry system that is based on the interaction between the magnetic fields around a device and a properly shaped magnet. The magnetic field that surrounds the device is generated by a magnetic sensor (compass) that is embedded in the new generation of mobile phones such as Apple’s iPhone 3GS and 4G, and Google’s Nexus one. The user movements of the properly shaped magnet in front of the device, then, deforms the sensor’s original magnetic field pattern whereby we can constitute a new means of communication between the user and the device. Thus, the magnetic field encompassing the device plays the role of a communication channel and encodes the hand-movement patterns of the user into temporal changes of the sensor’s magnetic field. In the back-end of the communication, an engine samples the momentary status of the field during a trial and recognizes the user’s pattern by matching it against some pre-recorded templates. The proposed method has been tested in a variety of applications such as handwriting recognition, user authentication, gesture recognition, and some entertainment applications. The experimental results show that the proposed interface not only elevates the convenience of user-device interactions, but also shows very promising accuracies in a wide range of applications requiring user interactions.
TopIntroduction
Compass, a human made navigational tool, has been widely employed to facilitate the piloting difficulties in the past centuries. An ordinary compass, by itself, is nothing more than a magnetized needle that pivots on an axis and tends to stay aligned with the earth’s north-south magnetic field. Being in telecommunication era, we are facing a rapid growth of electronic mobile devices that integrate more and more facilities every day. In the very recent years, electronic magnetic sensors have been combined with the cell phones to enhance the functionality of the phone’s built-in navigation system, which in typically the GPS. We found, however, that the usability of the compass is beyond navigational applications and can be extended to the context of human computer interaction (HCI).
The electronic magnetic sensor in a mobile device acts like a regular compass. Namely, the constant influence of the earth’s magnetic field on the sensor causes any slight displacement of the device with respect to the earth to be sensed and registered by the device. A similar type of influence can be imposed upon the magnetic field of the sensor if we slide a permanent magnet around the device. Specifically, a small moving magnet that travels in the 3D space around the device influences the effect of the earth’s magnetic field on the sensor and accordingly generates a temporal pattern in its x, y and z axes. This pattern can be used to build up a touchless interaction framework as a means of communication between the user and the device (Figure 1). In other words, the user can generate a specific gesture which in turn creates a temporal pattern in the magnetic field axes of the sensor. This pattern, then, can be compared against the pre-recorded templates to be labeled as an observation of one of them.
Figure 1. Using a properly shaped magnet as a user entry medium
This touchless input method addresses some of the limitations of common input methods such as keypads or touch screens. One of the main restrictions in designing tiny electronic devices is the size of the user input interface that needs to be big enough to comply with the human physical specifications. A magnet in shape of a rod, ring or pen though, can freely move in the 3D space around the device which is considerably broader than the surface of a typical touch screen and thereby can lead to designing of adequately small devices. Moreover, the 3D characteristic of the proposed method opens new door to the application of mobile phones in augmented and virtual reality. Additionally, since the magnetic field can penetrate through occluding objects, it allows for interactions, even though the device is concealed by other objects. This is in contrary with touch screens where interactions are only possible if the user and the device are in a direct contact. This feature of the proposed touchless technique will allow interactions through occluding objects such as the fabric between a user and a device in a pocket or the protecting cover of the phone itself. For instance, the user may be able to dial a number, enter a pin code, or select an album without taking the mobile device out of his pocket/bag.
Acquiring this utility does not impose any major change in the physical specifications of a device which is a notable advantage in small mobile devices. Replacing keypads or touch screens with such data entry technique in small devices allows saving cost, complexity and physical space in design. Compared to keypad or touch screen, a magnetic sensor can be much simpler, smaller and cheaper, and can be internally embedded. We believe that using magnetic sensors in mobile devices can potentially lead to a significant change in design and usability of tangible and wearable devices in the upcoming future.
Magnetic interaction (MagiTact) is not bound to a one on one interaction between a magnet and a sensor. Multiple magnets with different shapes can be distinguished in the same magnetic field produced by a sensor which results in a concurrent multi-user interaction between a device and users.