Effects of Selective and Divided Attention on Audiovisual Interaction

Introduction

The concept of “attention” is at once intuitive and enigmatic. Attention is not the same as seeing or perceiving, but on deeper reflection, it is clear that attention involves something more than sensation and perception (Gazzaniga, Ivry, & Mangun, 2002). For example, when we are reading a magazine on a bus the sound of car engines, music from vehicles’ CD players, etc can be distracting. When those noises grab our attention, our eyes can inadvertently skip a line, or even stop reading. So, why is it so hard to concentrate on reading when we are surrounded with loud sounds? In order to focus on relevant information and ignore what is irrelevant, the human brain is equipped with a selection mechanism accomplished by the cognitive function of attention. This attention system allows us to dynamically select and enhance the processing of objects and events that are the most relevant at each moment. The brain can then combine the task-relevant information from anatomically different sensory pathways to form unified percepts.

Audiovisual interaction plays an important role in performance and perception. The interactions of auditory and visual systems can occur at the level of multisensory integration (Stein & Stanford, 2008). A typical example of the audiovisual interaction is the McGurk effect, which was first described in a paper by McGurk and MacDonald in 1976. There have been numerous studies of the mechanisms of multisensory integration at the single neuron level in animals such as cats (Gordon, 1973; Meredith M. A & Stein, 1983), monkeys (Jay & Sparks, 1984) and guinea pigs, mice, hamsters, and rats (Wallace T. M, Wilkinson L. K, & Stein, 1996). Many previous studies have also investigated multisensory audiovisual integration using divided attention tasks in behavioral and event-related potential (ERP) measures in humans (Fort, Delpuech, Pernier, & Giard, 2002; Giard & Peronnet, 1999; Molholm et al., 2002; Teder-Sälejärvi, McDonald, Di Russo, & Hillyard, 2002; Teder-Sälejärvi, Russo, McDonald, & Hillyard, 2005; Vidal, Giard, Roux, Barthélémy, & Bruneau, 2008). Behavioral results have shown that responses to audiovisual stimuli are more rapid and accurate than the responses to either a unimodal visual or auditory stimulus (Giard & Peronnet, 1999; Molholm, et al., 2002; Teder-Sälejärvi, et al., 2002; Teder-Sälejärvi, et al., 2005). The ERP results from those studies were limited in that their analysis of ERPs was of the first 200 ms after stimulus presentation, and few assessed the brain activity after this point. Talsma and Woldorff (2005) have reported that multisensory integration processes can be modulated by attention in a divided-attention task. Talsma et al. (2007) studied the interactions between attention and audiovisual integration when auditory and visual stimuli were presented centrally. They reported that multisensory integration effects depended on the attention method; early audiovisual integration only occurred in divided-attention tasks. In visual attention tasks in which only visual information was task-relevant and required attention, the visual and auditory stimuli interacted with each other after 200 ms of presentation of the stimulus (Talsma, Doty, & Woldorff, 2007).

This paper reviews studies that have investigated the effects of attention on audiovisual interaction. In general, two aspects of attention can be distinguished; selective and divided attention. The first section describes that the effects of selective attention on visual and auditory modalities meet and integrate. The second section introduces the brain mechanisms of audiovisual interaction in divided attention. The final section looks at controversial issues between attention and audiovisual interaction, and future research directions.