3D Sound Simulation over Headphones

Introduction

Using physical modelling synthesis techniques it is possible to simulate as accurately as possible an acoustic guitar: of course, this is useful in terms of the opportunities made available to musicians to use an instrument they cannot in reality play, and in terms of acoustical studies of the instrument itself. But: Is that it? Once created, a guitar mathematical model can be altered as far as the imagination can extend: an acoustic guitar made of gold, with 40 strings, each of 100 metres in length, could virtually be created and played with a one-square-metre plectrum! Therefore, the potential can be seen both in the simulation of that which in nature already exists, and in the creation of that which in nature cannot exist.

Another kind of example will be discussed later in the chapter: instead of toying with the simulation of musical instruments, we shall try to create virtual acoustical environments through the simulation of three-dimensional (henceforth: 3D) soundscapes. The binaural spatialization technique will be used in order to achieve this goal; multi-dimensional soundscapes will be simulated not by placing real sound sources, such as loudspeakers, within the three dimensions, but by simulating the behaviour of our outer ear in terms of directional modifications brought to the sound input into the hearing system.

The mechanisms of spatial hearing will be investigated and analyzed, and three localization cues will be characterized and simulated. These are the Interaural Level Differences (ILDs), the Interaural Time Differences (ITDs), and the Direction-Dependent Filtering (DDF). Within the simulation of a real environment, these parameters would all be coherent with the position of the sound source: for example, for a sound source placed at 60° of azimuth, the sound would reach first the right ear and then the left (ITDs). Furthermore, it would be more intense at the right ear (ILDs) than at the left, and the sound would be filtered depending on the particular resonances of our outer hearing systems for that specific sound source location.

It must, however, be asked what could happen if the three localization cues were incoherent with the real position of the sound source. Of course, this is impossible in nature, and equally so in a standard soundscape simulation, when loudspeakers are placed in a 3D space. Still, achieving such incoherence is not impossible in a system based on headphones, where the signals sent to the hearing system are much more controllable, thus the whole reproduction system results may be much more flexible.

In this case, the binaural spatialization technique is useful not only to simulate a real 3D soundscape, but also to create new soundscapes, i.e., environments that are impossible to find in the real world. This seems to be one of the amazing new options offered by computer science: while it could indeed be considered inessential to simulate a feature that already exists in nature, it is particularly interesting to create a feature that as yet has no existence in the real world.

To appreciate this new ‘digital feature’ fully, it may help to think about a voyage into multiple dimensions; the results may appear similar to Abbott’s graphic achievements (seeAbbott, 1999) when he wrote Flatland (frequent reference will be made to this book later in the chapter). A monophonic diotic signal (the same at both ears) could be perceived as a point sound source located in the middle of the head: zero dimensions, or the point. By introducing intensity and content differences between the two channels (ILDs) and creating a dichotic signal (different for each of the ears), it is possible to obtain a standard headphone stereo signal, with multiple sound sources located along a line between the ears (always inside the head): one dimension, or the line. Through introducing time differences between the two channels (ITDs), it is possible to obtain the sensation of the sound coming from out of the head, and with multiple sound sources located in a plane: two dimensions, or the square. Then, upon introducing a simulation of the DDF, with different frequency filtering for each virtual sound source, the perception reaches the third dimension, the cube. The auditory passage between these steps could be visualized as the graphic perception of a point that becomes a line, then a square and at the end a cube. What, then, about the fourth dimension?