Emotion, Content, and Context in Sound and Music

Introduction

Music and sound stimulate one of the five human senses: hearing. Any form of stimulation is subject to psychological interpretation by the individual and a cause-and-effect relationship occurs. Whilst this relationship is unique to each individual up to a point, it is safe to assume that broad, often shared, experiences occur across multiple listeners. It can be argued that the emotional reaction and response of a listener to a sound or piece of music is the single most important event resulting from that experience.

The goal of this chapter is to explore the relationship between sound stimuli and human emotion. In particular, this chapter examines the role sound plays in conveying emotional information, even from sources that may be visual in origin. Equally, the chapter seeks to demonstrate how human emotion is able to flip this paradigm and influence music and sound selection, based on emotional state and consideration of the context of the user.

The content being represented digitally provides the opportunity to gain a greater understanding of the information present in a data set. Information being stored often has a number of characteristic features and structural elements that can be identified automatically. For example, music generally contains an identifiable structure, which might consist of several movements, parts, or, more commonly, verses and choruses. However, such structure can almost be considered fractal, in that there are microscopic and macroscopic levels of organisation and also repetition, ranging from musical beats, bars, verses and choruses to the level of the song itself.

Contextual data provides additional information about factors that contribute to making the user interaction experience much more relevant and effective by acquiring knowledge of the external factors that influence decision making and the emotion of the user.

The conceptual diagram of Figure 1 shows an idealised situation in which a large database of audio media is presented to the user through a suitable application (such as a computer game). In this scenario, the user’s emotion and context are analysed and compared against analysis of appropriate media content. This provides selection of the ‘best fit’ media that will further stimulate and engage the user in the most effective way.

Figure 1.

Idealised Role of Emotion, Content and Context in a Computer Application

The chapter explains the fundamentals of emotional stimulation using sounds and music, whilst retaining relevance to the audiologist. We demonstrate that by analysing the information present within media and considering its applications, significant advantages can be gained which improve user experiences, reduce overheads, and aid in the development of more suitable, efficient applications: whether they be computer games or other audio tools.

Emotion

Emotion is a key factor to consider in computer applications given that almost all applications will have some form of Human Computer Interface (HCI). Humans are emotional beings and the interaction with the machine will have some emotional effect on them to a greater or lesser extent. The computer, therefore, has an ability to invoke an emotional response in the user. The user may bring their own emotions with them to an interactive experience which has been affected by external factors in the environment around them (Dix, Finlay, Abowd, & Beale, 2003). The quality and resultant experience that a user has with a machine is important and this is also true when we consider the frequent interaction that we have with entertainment media and computer games.