This chapter discusses the progression of visual music and related audio-visual artworks through the 20th Century and considers the next steps for this field of research. The principles of visual music are described, with reference to the films of early pioneers such as John Whitney. A further exploration of the wider spectrum of subsequent work in various audio-visual art forms is then given. These include visualisations, light synthesizers, VJ performances, digital audio-visual artworks, projection mapping artworks, and interactive visual music artworks. Through consideration of visual music as a continuum of related work, the authors consider the Holophonor, a fictional audio-visual instrument, as an example of the ideal visual music instrument of the future. They conclude by proposing that a device such as the Holophonor could be constructed in the near future by utilising inter-disciplinary approaches from the fields of HCI and affective computing.
TopIntroduction
The Holophonor is a musical instrument of the 31st Century; it is best described as a combination of an Oboe and a Holographic Projector.
The notes played by its user triggers the projector to show holographic images that relate to the mood of the notes. Due to its complicated nature, it requires a great amount of skill to play. According to Leela, only a few people possess the skill to play the instrument - and they are not very good at it. –Holophonor: Futurama Wiki (n.d.).
The Holophonor is a fictional audio-visual performance instrument, as seen in the science fiction TV show Futurama (Groening et al., 2001; Groening et al., 2003), created by Matt Groening. It is an example of the type of new instrument that could result from research or commercial developments in the fields of visual music or interactive audio-visual artworks. In many ways, the Holophonor is the ideal visual music instrument: it operates in real-time, is classically musical, responds expressively to the performer, is portable and creates spectacular, unique visuals that integrate perfectly with the music. As such, the Holophonor provides an excellent lens through which to identify some of the challenges that we might seek to address in order to create the visual music instruments of the future.
This chapter commences with a contextual review of visual music, and the increasingly large sphere of associated audio-visual art, including light synths, visualisations, light shows, VJ performances, music videos, electroacoustic audio-visual compositions, projection-mapped artworks and real-time audio-visual installations. This provides a necessary background to the field in which inventions like the Holophonor would be situated. The main features of the Holophonor are reviewed, establishing the creative and computing challenges for research in this area.
TopVisual Music
“There is geometry in the humming of the strings, there is music in the spacing of the spheres.” – Pythagoras (569-475 B.C.).
‘Visual music’ as a 20th Century art form consists of moving visual images or animations, which are organised in a way that the composer considers to be musical. Works may include an original musical soundtrack, or may use an existing piece of music to provide a soundtrack. Others may not use a soundtrack at all, but are considered musical through the structure and arrangement of visual materials. Notable pioneers, as archived by the Centre for Visual Music (2013) include John Whitney, Oskar Fischinger, Jordan Belson, Mary Ellen Bute and Charles Dockum. The work of direct animation (a process where materials are applied directly to film without the use of a camera) film-makers such as Len Lye, Norman McLaren, Harry Smith and Stan Brakhage may also be associated with the visual music label. In a broader context, visual music can be seen as part of the avant-garde artistic practices and experimental film making movements of the early 20th Century (Russett & Starr, 1976).