Sound for Fantasy and Freedom

Introduction

At the Interactive Institute, Sonic Studio in Piteå, Sweden, we do research on sound and auditory perception in order to find new ways to use sound, new contexts where sound can be utilized, and new applications for sound in general. Of special interest to us is how sound resembles and differs from other sensory stimuli and how this can be put to play. In our work we use perspectives and methods from art, science, and technology and we utilize digital technology as a vehicle for our ideas and experiments.

In a series of projects we have explored intuitive, emotional, imaginative, and liberating properties of sound. These projects have resulted in new insights and knowledge as well as in new and innovative applications for sound, audio, and technology. In this chapter I will describe our perspective on a number of sound properties and how we have put these to work in various ways. The projects are based on and inspired by an ecologic and everyday-listening approach to sound, like the ones proposed by R. Murray Shafer, William Gaver, and their followers.

As human beings, we are good at interpreting the soundscape constantly surrounding us. When we hear a sound we can make relatively accurate judgments about the objects involved in generating the sound, their weight, the materials they are made of, the type of event or series of events that caused the sound, the distance and direction to the sound source, and the environment surrounding the sound source and the listener, for example. Much of the existing research on sound and auditory perception is about how to convey clear and unambiguous information through sound. In computer games, however, the aim is also to create other effects, effects that have as much to do with emotions, the subconscious, intuition, and immersion as they do with clear and unambiguous messages.

This article describes a couple of projects in which we have worked with the balance between eye and ear, between ambiguity and un-ambiguity, between cognition and intuition and between body and mind. The aim has been to create experiences built on a multitude of human abilities and affordances, mediated by new media technology.

In a traditional computer game setting, the TV screen or computer monitor is the center of attention. The screen depicts the virtual game world and the player uses some kind of input device, such as a game pad, a mouse, a keyboard, or a Wiimote, to remotely control the virtual gameworld and objects and creatures in it. The action takes place in the virtual world and the player is naturally detached from the game action by the gap between the player’s physical world and the virtual world of the game. Much work has to be done and complex technology used in order to bridge that gap and to have the player experience a sense of presence in the virtual gameworld. The aim is to make the player feel as immersed as possible in the game experience and to make her suspend her natural disbelief. To achieve this, the computer game industry must build broader and broader bridges over the reality gap to make the virtual game reality more immersive. The traditional way to increase immersion and suspension of disbelief has primarily been to increase graphics capability and, today we can enjoy near photo-realistic, 3D-graphics in real time. But there might be alternative ways to tackle the problem. Potentially, computer games could be more engaging and immersive without having to build long and broad bridges over the reality gap. What about narrowing the gap instead of building broader bridges over it?