Information Engagement through Interactive Sonification Design

Introduction

While data mining focuses on methods for interrogating datasets, it could be said that information visualization and sonification (auditory representation of data, or auditory graphing) attends to the aesthetics, interactivity, clarity and flexibility of data representation to enhance the experience of understanding, evaluating, and comparing information, usually by human users. Hence human-centered customization of the interface, and interaction that allows multiple iterations or “views” of the data, aids human interaction with content (such as the ability to navigate, zoom and reorganize data) and these techniques can contribute to user control of auditory and graphical representations.

This chapter looks at information design, a subset of content in design, and at ways of leveraging the interface, e.g. with physical computing interaction and social interplay, in order to increase the satisfaction and engagement with the digital experience. Physical interaction and social interaction will be discussed in relation to data visualization and sonification but key concepts can be applied to other forms of representation and interaction design. Features such as the persuasiveness of the data become metrics of the performance of a representation. Increasingly, the goal of contemporary data representations often extends beyond informativeness to include social and non-expert engagement with the data representation through the media and popular forms of dissemination. Due to the efficiency of a graphical and auditory representation, information visualization is one of the most effective methods for conveying data in a penetrable, available form. User-centered interfaces allow lay people to interrogate, manipulate, compare and perform simple transformations of complex data sets that are otherwise only accessible to expert analysts when the data is in its raw form.

Visualization and sonification are becoming evermore important and ubiquitous in our understanding of complex data and for monitoring data events. Technologies such as web-cameras and sensors provide constant and potentially massive streams of information about the environment, information transactions, Internet activity, etc. and sensors and captors are increasingly widespread in urban, domestic, climatic and operational contexts. Less well explored are ways to effectively and ergonomically interact with live data utilizing the advantages of real-time, instantaneous responsiveness. While systems for computational analysis and software data interpretation are quite well established, more flexible and interactive systems for human access and understanding of information requires a different design approach. The way in which humans interact with digital information is also rapidly transforming as pervasive and physical, tangible, gestural interfaces for interaction are increasingly usual, offering the advantage that intuitive natural actions can be used to control computational processes in a myriad of physical locations. Therefore, interactive interfaces and designing the interaction experience is inseparable from the quality of the “experience” of negotiating information representations. This chapter examines these two major influences (the interaction experience and the information representation) in information engagement.

The chapter will draw on recent research in gestural, tangible, sensing, real-time interaction designs in the author's research lab¹, cases from pedagogical interaction design studios and work examining innovative real-time and interactive representation of information (i.e. information sonification and visualization).