Multisensory Experiences in Virtual Reality and Augmented Reality Interaction Paradigms

Multisensory Experiences in Virtual Reality and Augmented Reality Interaction Paradigms

Inma García-Pereira, Lucía Vera, Manuel Pérez Aixendri, Cristina Portalés, Sergio Casas
Copyright: © 2020 |Pages: 23
DOI: 10.4018/978-1-7998-2112-0.ch014
(Individual Chapters)
No Current Special Offers


Multisensory stimuli can be integrated in systems that make use of different paradigms, such as Virtual Reality (VR), Augmented Reality (AR) or, in a wider sense, Mixed Reality (MR), enhancing user experiences within the virtual content. However, despite the many technological solutions that exist (both hardware and software), only visual and sonic stimuli can be considered as highly integrated in consumer-grade applications. This chapter addresses the current state of the art in multisensory experiences, taking also in consideration the aforementioned interaction paradigms, and brings the benefits and challenges. As an example, authors introduce ROMOT, a RObotic 3D-MOvie Theatre, that supports and integrates various types of displays and interactive applications, providing users with multisensory experiences.
Chapter Preview


Smart systems incorporate functions of sensing, actuation and control to describe and analyze a given situation, in order to make smart decisions. Interactive technologies are key in smart systems. They involve different paradigms and a variety of sensors and displays that make it possible for a user to have enriched experiences through his/her senses. Nowadays, we are experiencing an evolution of interactive technologies that are even embedded in small devices, such as tablets and smartphones, reaching wide audiences. Most interactive systems make use of sight (e.g. 3D models) and hearing (e.g. music, narrator, sonic effects), but also other senses can take part of these systems, including touch (e.g. feeling the shape and texture of virtual objects), smell (e.g. the aroma of food) and vestibular (e.g. travelling inside a computer-generated world while feeling the movements and vibrations). Of course, the last cases are less spread over consumer-grade devices, on the one hand, because of the increased costs and on the other hand, because of the needed hardware. However, multisensory experiences have been researched for many years. One of the earliest immersive, multisensory machines is the Sensorama, that was patented back in 1962 by Morton Heilig (United States Patent No. US3050870A, 1962). The technology integrated in the Sensorama allowed a single person to see a stereoscopic film enhanced with seat motion, vibration, stereo sound, wind and aromas, which were triggered during the film. This was a visionary system, and has been referred to as “the cinema of the future” (Heilig, 1992; Robinett, 1994), although at the time it was invented, it did not attract wide audiences, maybe because the technology was too incipient to be widely exploited.

In the last few years, the rapid technological advancements have allowed the development of commercial solutions that integrate a variety of multimodal displays in movie theatres, such as the 4DX (“4DX | Absolute Cinema Experience,” n.d.) or the Pix 5D cinema (“Pix 5D Cinema,” n.d.). Some claim that this technology shifts the cinema experience from “watching the movie to almost living it” (Yecies, 2016), also enhancing the cinematic experience while creating a new and contemporary version of storytelling, which can be conceptualised as a “reboot cinema” (Tryon, 2013).

Multisensory experiences can be integrated in systems that make use of different interaction paradigms, such as Virtual Reality (VR), Augmented Reality (AR) or, in a wider sense, Mixed Reality (MR). Examples of these paradigms to create cinema-related experiences can be found in (Portalés, Viñals, Alonso-Monasterio, & Morant, 2010; Vosmeer & Schouten, 2014), to name some. While VR substitutes the real world by a synthetic one, AR enriches the real world by means of virtual stimuli, without (completely) replacing the real world. On the other hand, MR is understood as the result of merging the real and virtual worlds at some point along the “real virtuality continuum”, which connects completely real and virtual environments. Although the MR concept was defined for the first time by Milgram (Milgram & Kishino, 1994), it has become very popular in the last years because of the emergence of cutting-edge technologies in this field, such as more sophisticated headsets and glasses among others.

Key Terms in this Chapter

Interaction: A kind of action that occurs as two or more objects have an effect upon one another.

Multisensory: Involving or using more than one of the senses when interacting with the real or synthetic worlds.

Tangible Assets: Assets that have a physical form, including buildings, places, monuments, tools, artifacts, etc.

Three-Dimensional (3D) Data: Data that is realized in a three dimensional space, a geometric setting in which three values are required to determine the position of an element.

Augmented Reality (AR): The technology that simultaneously combines real and virtual objects that are interactive in real-time and are registered in a three-dimensional space.

Virtual Reality (VR): The technology by which a user, stimulated with computer-generated perceptual cues, experiences an alternative reality that is different from the one he/she actually lives in.

Digital Representation of Data: Reproduction of data by means of digital forms, as enabled by computers.

Mixed Reality: The result of blending the physical world with a synthetic one, including the paradigms of Augmented Reality and Augmented Virtuality.

Complete Chapter List

Search this Book: