Using a Hands-Free System to Manage Common Devices in Constrained Conditions

Introduction

Definitions of Human-Computer Interaction (HCI) are easy to find. The majority of those definitions say that Human–computer interaction involves the study, planning, design and uses of the interaction between users and computers. The HCI, term introduced more than three decades ago in works like (Card, Moran, & Newell, 1980; Carlisle, 1976), is often regarded as the intersection of computer science, behavioral sciences, design, media studies, among other fields of study. It is therefore quite easy to recognize that the boundaries of what is HCI are relatively fuzzy, including a wide-range of practice fields. Historically, in a non-exhaustive way, HCI evolved from sets of switches, punched cards, monitors, keyboards, mouse pointers, touch-screens, motion detection devices, to bionic interfaces. HCI is changing fast. Dialogs between the computer HAL 9000 and humans in the 1968 film “2001: A Space Odyssey” (Kubrick, 1968) or the ones with the artificially intelligent, Vox 114, library host hologram in “Time Machine” movie, which communicates and interacts naturally with a time traveler (Wells, 2002), the invasive neural interfaces presented in “The Matrix” (Wachowski & Wachowski, 1999) or many of the interacting technologies presented in “Minority Report” (Spielberg, 2002) were once science fiction, but in the global sense not any more.

The truth is that, as Jonathan Grudin from the Microsoft Corporation claims, HCI is a moving target (Grudin, 2012). The future of HCI is expected to be supported on the ubiquitous and continuous presence of devices where computers communicate to give universal access to data and computational services. The future user expects high functional systems where, among other things, accessing those functionalities is natural, with mass availability of computer graphics, high-bandwidth interaction, and wide variety of displays (e.g., on common surfaces, with flexibility, large and thin).

Most computers and mobile devices have already the computational capacity and are equipped with devices to mimic humans’ senses like sight and hearing. With the appropriate sensors other “feelings” can be easily achieved like temperature, taste, smell, touch, balance or measure acceleration (Bhowmik, 2014). The input devices of these computers can be used to control machines in a natural and intuitive way, giving a new dimension to the traditional users’ interfaces.

There is an extensive list of works that study “human senses” devices to interact with computers. In Breen et al. (2014) a survey of the key concepts and underlying technologies is presented for voice and language understanding (including voice recognition, hardware optimization, speech synthesis, natural language understanding, etc.). A multimodal human-computer interaction focused on body, gesture, gaze, and affective interaction is presented in (Jaimes & Sebe, 2007). Senses like smell and taste are starting to be more effectively achieved. For instance, in (Villarreal & Gordillo, 2013) the results of a sensor model of aspiration and the design of a smell system device inspired in biological process is presented. In (Halder et al., 2012) the development of a polymer membrane is addressed, based in potentiometric taste sensors with efficient selectivity and sensitivity to mimic the mammalian tongue for measurement of basic tastes like saltiness, sourness, bitterness, sweetness and umami or savory.

Besides the usual keyboards/mouse or touchscreen devices, driven among others by the game industry, one of the more developed type of interactions are the ones based on non-touching gesture recognition. For this purpose, several types of sensors can be used, such as: embedded cameras or mobile 3D sensors, such as the Structure Sensor (Struture sensor, 2015), the Leap Motion (Leap Motion, 2015), the Kinetic (Kinect, 2015) or the zSense (Withana, Peiris, Samarasekara, & Nanayakkara, 2015).