This chapter treats computer game playing as an affective activity, largely guided by the audio-visual aesthetics of game content (of which, here, we concentrate on the role of sound) and the pleasure of gameplay. To understand the aesthetic impact of game sound on player experience, definitions of emotions are briefly discussed and framed in the game context. This leads to an introduction of empirical methods for assessing physiological and psychological effects of play, such as the affective impact of sonic player-game interaction. The psychological methodology presented is largely based on subjective interpretation of experience, while psychophysiological methodology is based on measurable bodily changes, such as context-dependent, physiological experience. As a means to illustrate both the potential and the difficulties inherent in such methodology we discuss the results of some experiments that investigate game sound and music effects and, finally, we close with a discussion of possible research directions based on a speculative assessment of the future of player-game interaction through affective sound.
TopIntroduction
Digital games have grown to be among the favourite leisure activities of many people around the world. Today, digital gaming battles for a share of your individual leisure time with other traditional activities like reading books, watching movies, listening to music, surfing the internet, or playing sports. Games also impose new research challenges to many scientific disciplines – old and new – as they have been hailed as drivers of cloud computing and innovation in computer science (von Ahn & Dabbish, 2008), promoters of mental health (Miller & Robertson, 2009; Pulman, 2007), tools for training cognitive and motor abilities (Dorval & Pepin, 1986; Pillay, 2002), and as providers of highly immersive and emotional environments for their players (Ravaja, Turpeinen, Saari, Puttonen, & Keltikangas-Järvinen, 2008; Ryan, Rigby, & Przybylski, 2006). Gaming is a joyful and affective activity that provides emotional experiences and these experiences may guide how we process information.
Regarding emotions, Norman’s (2004) definition is that emotion works through neurochemical transmitters which influence areas of our brain and successively guide our behaviour and modify how we perceive information and make decisions. While Norman makes a fine distinction between affect and cognition, he also suggests that both are information-processing systems with different functionality. Cognition refers to making sense of the information that we are presented with, whereas affect refers to the immediate “gut reaction” or feeling that is triggered by an object, a situation, or even a thought. Humans strive to maximize their knowledge by accumulating novel, but also interpretative information. Experiencing novel information and being able to interpret it may be a cause of neurophysiological pleasure (Biedermann & Vessel, 2006). Cognitive processing of novel information activates endorphins in the brain, which moderate the sensation of pleasure. Thus, presenting novel cues in a game environment will affect and mediate player experience and in-game learning. This is an excellent example of how cognition and affect mutually influence each other, which is in line with modern emotion theories (Damasio, 1994; LeDoux, 1998; Norman, 2004). Norman (2004) proceeds to define emotion as consciously experienced affect, which allows us to identify, who (or what) caused our affective response and why. The problem of not making a clear distinction between emotion and affect is further addressed by Bentley, Johnston, & von Baggo (2005), who recall Plutchik’s (2001) view on emotion as an accumulated feeling which is influenced by context, experience, personality, affective state, and cognitive interpretation. They also explain that user experience for desktop software or office-based systems is more dependent on performance factors while, for digital games, user experience depends much more on affective factors. Affect is defined as a discrete, conscious, subjective feeling that contributes to, and influences, an individual’s emotion (Bentley, et al., 2005; Damasio, 1994; Russell, 1980). We will revisit this notion later in the text.
In addition, Moffat (1980) introduced an interesting notion about the relationships between personality and emotion, which are distinguished along the two dimensions: duration (brief and permanent) and focus (focused and global). For example, an emotion might develop from brief affection into a long-term sentiment or a mood that occurs steadily might become a personality trait. The two dimensions can be plausibly identified at a cognitive level, making a strong case for the relation between emotion, cognition, and personality both at the surface and at a deep, structural level.