Crossmodal Interactions in Visual Competition

Introduction

Visual competition is the phenomenon that occurs when two or more interpretations of a visual figure are created, even though the visual input itself is kept constant. This phenomenon casted a doubt on the traditional view that the mind sees a mirror image of the external world, and it has attracted attention of vision scientists, cognitive psychologists, neuroscientists, and philosophers (Blake & Logothetis, 2002; Kim & Blake, 2005; Leopold & Logothetis, 1999; Sterzer, Kleinschmidt, & Rees, 2009; Tong, Meng, & Black, 2006).

Figure 1 illustrates some example illusions that produce visual competition. In general, visual competition can be categorized into two forms, including binocular rivalry and stimulus rivalry. In binocular rivalry, different visual images simultaneously but separately projected to the left and right eye can produce dominant perception alterations between each eye image. In stimulus rivalry, the visual image is ambiguous (Figure 1A-C), which leads to dominant perception changes between two or more possible interpretations. Not all of the features of binocular and stimulus rivalry overlap (Meng & Tong, 2004); however, we do not address these differences in this chapter.

Figure 1.

A–C: Some example figures of stimulus rivalry. A: The Necker cube illusion. B: The Rubin’s vase illusion. C: The Quartet dot illusion used in Takahashi & Watanabe (2010, 2011). D: The stream/bounce illusion (Sekuler, Dekuler, and Lau, 1997). E: The haptic bias on ambiguous visual figures (Blake, Sobel, & James, 2004).

During visual competition, the stochastic changes in the dominant perception of an image can be characterized in terms of “what” and “when.” The aspect of “what” simply refers to which interpretation is currently observed. In practice, the “what” aspect can also include, for example, which interpretation initially appears or which interpretation appears for longer duration. On the other hand, the “when” aspect of visual competition refers to the timing and frequency of the perceptual alternations.

While the spontaneity and stochasticity are distinctive features of visual competition, various external factors also affect the “what” and “when” in visual competition. A number of studies suggest that visual competition is susceptible to such factors as attention and intention, action, task-irrelevant visual stimulation and stimulus configuration, perceptual history, eye movement, and transcranial magnetic stimulation to visual areas (see Takahashi & Watanabe, 2010, 2011 for the list of references). Moreover, visual competition is also susceptible to nonvisual input. In this chapter, we summarize the recent studies that have investigated the nonvisual modulation of the “what” and “when” aspects of visual competition.

The Effects Of Nonvisual Bias On The Dominant Perceptions Of Visual Competition

The sensory, perceptual and cognitive processes of visual and nonvisual modalities are not completely independent (Alais, Newell, & Mamassian, 2010; Shimojo & Shams, 2001). Classical phenomena, such as the McGurk effect (McGurk & MacDonald, 1976) or ventriloquism effect (Alais & Burr, 2004), suggest that auditory (or other nonvisual) perception is biased and distorted by simultaneous visual inputs. In addition, nonvisual inputs can distort visual perceptions by resolving the ambiguity of the visual world. A striking example of this phenomenon is known as the stream/bounce illusion (Sekular, Sekular, & Lau, 1997, Figure 1D). This illusion, which consists of two dots moving towards each other, can be perceived either as bouncing or as streaming; however, the illusion is preferentially perceived as bouncing dots if their collision is accompanied by a brief clicking sound. Along with the other studies showing nonvisual modulation on visual perception (Shams & Kim, 2010), these data suggest that nonvisual inputs can significantly modulate the visual process for ambiguous visual motion.