Exploiting Multimodality for Intelligent Mobile Access to Pervasive Services in Cultural Heritage Sites

Exploiting Multimodality for Intelligent Mobile Access to Pervasive Services in Cultural Heritage Sites

Antonio Gentile (Università di Palermo, Italy), Antonella Santangelo (Università di Palermo, Italy), Salvatore Sorce (Università di Palermo, Italy), Agnese Augello (Università di Palermo, Italy), Giovanni Pilato (Consiglio Nazionale delle Ricerche, Italy), Alessandro Genco (Università di Palermo, Italy) and Salvatore Gaglio (Consiglio Nazionale delle Ricerche, Italy)
DOI: 10.4018/978-1-60566-978-6.ch009
OnDemand PDF Download:


In this chapter the role of multimodality in intelligent, mobile guides for cultural heritage environments is discussed. Multimodal access to information contents enables the creation of systems with a higher degree of accessibility and usability. A multimodal interaction may involve several human interaction modes, such as sight, touch and voice to navigate contents, or gestures to activate controls. We first start our discussion by presenting a timeline of cultural heritage system evolution, spanning from 2001 to 2008, which highlights design issues such as intelligence and context-awareness in providing information. Then, multimodal access to contents is discussed, along with problems and corresponding solutions; an evaluation of several reviewed systems is also presented. Lastly, a case study multimodal framework termed MAGA is described, which combines intelligent conversational agents with speech recognition/ synthesis technology in a framework employing RFID based location and Wi-Fi based data exchange.
Chapter Preview


Imagine your last visit at a popular art museum. You approach the cashier to buy your ticket and the lady offers, at an extra cost, to rent an audio guide. You take it and move to the entry gate where a guard check your ticket and let you in. Your visit then begins, and here you are, fiddling with the buttons to find the beginning of your story. As you wade between halls and paintings you keep punching buttons, following the numbered path, and hearing a fascinating voice narrating the wonder behind each piece of art. “What did she just say?” Not a problem, you simply push that button again and here is she untiringly repeating the story. ”So she is really referring to that other painting I saw two or three halls ago” – you find yourself thinking – “what was the name of the artist?” You can’t remember and look for a clue in your keypad to only find anonymous numbers. You end up putting your question away and continue the visit. Can technology advances help us envision better scenarios in the next future? We believe the answer is yes, and multimodal mobile devices will play a key role. As this chapter will attempt to show, in a non-distant future the same you will now enter the museum with your smart-phone in your pocket and wearing a headset. The entry gate will now open as you pass through, activated via a Bluetooth exchange that checks your e-ticket, activating your virtual guide. Your visit begins, and you move through the halls following a path in the museum map, which you previously set while planning your visit from home. A wi-fi connection updates your screen as you wade, and the your guide, as before, is telling about what you see. This time, when you ask, an answer is given and further details materialize on your screen if so you desire…Seems too far-away? Well all of the technologies involved are already here, as we will see in the rest of the chapter.

Such technologies, both hardware and software, made the Mark Weiser’s vision of Ubiquitous Computing real and more and more available for users in their in everyday life. The Ubiquitous Computing paradigm relies on a framework of smart devices that are thoroughly integrated into common objects and activities. Such a framework implements what is otherwise called a pervasive system, which main goal is to provide people with useful services for everyday activities.

As a consequence, the environment in which a pervasive system is operating becomes augmented, that is enriched by the possibility to access additional information and/or resources on a per-needed basis. Augmented environments can be seen as the composition of two parts: a visible part populated by animated (visitors, operators) or inanimate (artificial intelligence) entities interacting through digital devices in a real landscape, and an invisible part made of software objects performing specific tasks within a underlying framework. People would perceive the system as a whole entity in which personal mobile devices are used as human-environment adaptable interfaces.

The exponential diffusion of small and mobile devices, third-generation wireless communication devices, as well as location technologies, has led to a growing interest towards the development of pervasive and context-aware services. There are many domains where pervasive systems are suitably exploited. One of the most recent and interesting applications of pervasive technology is the provision of advanced information services within public places, such as cultural heritage sites or schools and university campuses. In such contexts, concurrent technologies exploited in smart mobile devices can be used to satisfy the mobility need of users allowing them to access relevant resources in a context-dependent manner. Of course, most of the constraints to be taken into account when designing a pervasive information providing system are given by the actual domain where they are deployed.

Cultural Heritage applications pose tremendous challenges to designers under different aspects. Firstly, because of the large variety of visitors they have to deal with, each with specific needs and expectations about the visit. Secondly, no two sites are the same, and pretty much you need a framework that can easily produce a new installation given the site characteristics (indoor versus outdoor, distributed versus centralized, individual centered versus group centered, etc.). Lastly, the technologies involved must be robust to failures, redundant and, above all, easy and intuitive to use. These are the reasons why there are several research groups that are focusing their attention on this applicative domain. It is a good test bed to validate almost all models and design choices.

Complete Chapter List

Search this Book: