Intangible Heritage Management and Multimodal Navigation

Intangible Heritage Management and Multimodal Navigation

Gianluigi Ciocca (Università degli Studi di Milano-Bicocca, Italy), Alessandro Colombo (Università degli Studi di Milano-Bicocca, Italy), Raimondo Schettini (Università degli Studi di Milano-Bicocca, Italy), Maria Teresa Artese (Consiglio Nazionale delle Ricerche, Italy) and Isabella Gagliardi (Consiglio Nazionale delle Ricerche, Italy)
DOI: 10.4018/978-1-60960-044-0.ch005
OnDemand PDF Download:
$30.00
List Price: $37.50

Abstract

This chapter presents tools and methods which have been developed in order to manage and consult multimedia ethnographic archives for texts, images, audios (both music and speech), and videos. The system offers the user several retrieval strategies for querying the multimedia archive database by exploiting alphanumeric relational query, audio similarity query, clustering, and image and video similarity. Once a subset of information meeting the user’s needs has been identified, this can be displayed in a 3D virtual exhibition which can be visited interactively. The system presented is actually exploited to manage and multimodally navigate the Archive of Ethnography and Social History of the Italian Lombardy Region with some 18,000 oral documents, 3,000 textual transcriptions, 2,000 musical transcriptions, 5,000 MP3 audio files, 10,000 photos, and 500 videos.
Chapter Preview
Top

Introduction

According to the 2003 Convention for the Safeguarding of the Intangible Cultural Heritage of Unesco (Unesco, 2003), the intangible cultural heritage (ICH) – or living heritage – is defined as the practices, representations, expressions, as well as the knowledge and skills, that communities, groups and, in some cases, individuals recognize as part of their cultural heritage. The Convention states that the ICH is manifested, among others, in the following domains:

  • 1.

    Oral traditions and expressions including language as a vehicle of the intangible cultural heritage;

  • 2.

    Performing arts (such as traditional music, dance and theatre);

  • 3.

    Social practices, rituals and festive events;

  • 4.

    Knowledge and practices concerning nature and the universe;

  • 5.

    Traditional craftsmanship.

The Archive of Ethnography and Social History of the Italian Lombardy Region (AESS) (AESS, 2009) (Renata Meazza, 2008) was founded to preserve, study, and enhance the value of documents and images of the life, social transformations, literature, oral history, material culture, and anthropic landscapes of the Lombard territory.

The archive is composed of 18,000 oral documents, 3,000 textual transcriptions, 2,000 musical transcriptions, 5,000 audio files in MP3 format, and 10,000 photographic documents, that are related to photographs, and 500 videos. It is managed through a database which integrates the catalogue cards with multimedia objects of different types: audio files, images, digital videos, textual transcriptions, musical scores, etc.

On-line museums and archives call for tools and methods to navigate their contents and to provide the facilities of the search, browsing, clustering and visualization of multimedia data.

To support the search of information, digital archives generally employ three different retrieval modes, considered alternative, the alphanumeric relational query; the content-based query, exploiting automatically computed low-level image features (such as colour and texture); and the textual similarity query, exploiting any textual notations attached to database items (such as captions or textual cards…).

The text-based approach, although effective for the expression of high-level features of the documents, may imply a considerable manual task in indexing and is clearly not suitable for the description of purely visual contents, such as texture, or contour or of audio content. On the other hand, the low-level features may not precisely model the user’s perception: for example the system’s inference of texture may be quite different from the user’s. Consequently all these indexes must be integrated to provide a complete and accurate description of the documents, and, since the objective of the user’s queries can not be defined in advance, it is evident that user feedback must be considered in the retrieval.

In designing a system able to manage multimedia data, as in the case of the AESS database, these main problems have therefore to be addressed:

  • what navigation paradigms should be offered to the users?

  • what information should be used for the retrieval purposes?

  • how does the system evaluate the match between the query and the database items ?

  • how can non-expert users interact with the system in an intuitive way ?

In this chapter we shall address all these issues together with the solution we propose and the tools and methods which have been developed and integrated into a single system, in order to manage and consult multimedia ethnographic archives.

Our system is able to manage all the three types of retrieval modes to navigate the AESS ethnographic multimedia data: besides the standard textual query, we propose to compute similarities among images, by comparing the terms of their textual descriptions (QuickLink) (Gagliardi et al., 2001) (Gagliardi et al., 2001); similarities among images, by comparing significant visual features of an image against the features of the images contained in a database (QuickLook) (Ciocca et al., 1999b) (Ciocca et al., 2001b) and finally, similarities among audios, through the identification of shared acoustic features (AudioCluster) (Gagliardi et al., 2005).

Key Terms in this Chapter

Web Site Design: Is the process to create usable, accessible, effective, and efficient web sites according to design methodologies.

Information Retrieval: Is the science of searching for objects (documents, information within documents, and metadata about documents, as well as that of searching relational databases and the World Wide Web) that match a search query.

Audio Retrieval: Searching for audio files in large databases similar to the audio query: the search will analyze the actual contents of the audio rather than the metadata such as keywords, tags, and/or descriptions associated with the audio.

Content-Based Image Retrieval: Searching for digital images in large databases similar to the image query. “Content-based” means that the search will analyze the actual contents of the image rather than the metadata such as keywords, tags, and/or descriptions associated with the image.

Lombardy Region: Lombardy is one of the 20 regions of Italy, and the Administrative Regional Government

Multimedia Database: A collection of information (data) which is arranged in individual records and is searchable. In our case information is composed of text, images, audio and video.

Ethnographic Archive: Archive related to ethnography (“ethnographical data”), that is the branch of anthropology that provides scientific description of individual human societies.

Archivio di Etnografia e Storia Sociale (AESS): Is the department of Lombardy Region that was founded to preserve, study, and enhance the value of documents and images of the life, social transformations, literature, oral history, material culture, and anthropic landscapes of the Lombard territory.

Video Analysis and Summarization: Is the process in which video sequences are analyzed and summarized (according to their visual contents) into still images.

Similarity-Based Textual Retrieval: Searching for files in large databases similar to the query (in our case, image): the search will compare the terms of their textual descriptions, and retrieve the most similar objects according to their description;

Complete Chapter List

Search this Book:
Reset