Crossmodal Interactions in Visual Competition

Crossmodal Interactions in Visual Competition

Kohske Takahashi (The University of Tokyo, Japan & Japan Society for the Promotion of Science, Japan) and Katsumi Watanabe (The University of Tokyo, Japan & Japan Science and Technology Agency, Japan)
DOI: 10.4018/978-1-4666-2113-8.ch007
OnDemand PDF Download:
No Current Special Offers


Visual competition is one of the long-standing mysteries in vision science. The image that arises from a person’s visual awareness of a constant visual input can spontaneously and stochastically changed between two or more possible interpretations. Visual competition is largely defined by the actual visual experience. However, recent studies have suggested that the process of resolving visual ambiguity is not limited to the domain of vision. Rather, the process is likely susceptible to various types of nonvisual modulation (e.g., auditory and haptic/tactile). Here, the authors review the recent studies that investigate the crossmodal interactions found in visual competition. These current studies highlight the significant crossmodal effects in visual competition, including the bias toward visual interpretations that are congruent with other modalities and the temporal synchronization of the transition between two (or more) visual interpretations with nonvisual events. These nonvisual modulations of visual competition reveal that visual perception is built upon several levels of crossmodal synchronization.
Chapter Preview


Visual competition is the phenomenon that occurs when two or more interpretations of a visual figure are created, even though the visual input itself is kept constant. This phenomenon casted a doubt on the traditional view that the mind sees a mirror image of the external world, and it has attracted attention of vision scientists, cognitive psychologists, neuroscientists, and philosophers (Blake & Logothetis, 2002; Kim & Blake, 2005; Leopold & Logothetis, 1999; Sterzer, Kleinschmidt, & Rees, 2009; Tong, Meng, & Black, 2006).

Figure 1 illustrates some example illusions that produce visual competition. In general, visual competition can be categorized into two forms, including binocular rivalry and stimulus rivalry. In binocular rivalry, different visual images simultaneously but separately projected to the left and right eye can produce dominant perception alterations between each eye image. In stimulus rivalry, the visual image is ambiguous (Figure 1A-C), which leads to dominant perception changes between two or more possible interpretations. Not all of the features of binocular and stimulus rivalry overlap (Meng & Tong, 2004); however, we do not address these differences in this chapter.

Figure 1.

A–C: Some example figures of stimulus rivalry. A: The Necker cube illusion. B: The Rubin’s vase illusion. C: The Quartet dot illusion used in Takahashi & Watanabe (2010, 2011). D: The stream/bounce illusion (Sekuler, Dekuler, and Lau, 1997). E: The haptic bias on ambiguous visual figures (Blake, Sobel, & James, 2004).


During visual competition, the stochastic changes in the dominant perception of an image can be characterized in terms of “what” and “when.” The aspect of “what” simply refers to which interpretation is currently observed. In practice, the “what” aspect can also include, for example, which interpretation initially appears or which interpretation appears for longer duration. On the other hand, the “when” aspect of visual competition refers to the timing and frequency of the perceptual alternations.

While the spontaneity and stochasticity are distinctive features of visual competition, various external factors also affect the “what” and “when” in visual competition. A number of studies suggest that visual competition is susceptible to such factors as attention and intention, action, task-irrelevant visual stimulation and stimulus configuration, perceptual history, eye movement, and transcranial magnetic stimulation to visual areas (see Takahashi & Watanabe, 2010, 2011 for the list of references). Moreover, visual competition is also susceptible to nonvisual input. In this chapter, we summarize the recent studies that have investigated the nonvisual modulation of the “what” and “when” aspects of visual competition.


The Effects Of Nonvisual Bias On The Dominant Perceptions Of Visual Competition

The sensory, perceptual and cognitive processes of visual and nonvisual modalities are not completely independent (Alais, Newell, & Mamassian, 2010; Shimojo & Shams, 2001). Classical phenomena, such as the McGurk effect (McGurk & MacDonald, 1976) or ventriloquism effect (Alais & Burr, 2004), suggest that auditory (or other nonvisual) perception is biased and distorted by simultaneous visual inputs. In addition, nonvisual inputs can distort visual perceptions by resolving the ambiguity of the visual world. A striking example of this phenomenon is known as the stream/bounce illusion (Sekular, Sekular, & Lau, 1997, Figure 1D). This illusion, which consists of two dots moving towards each other, can be perceived either as bouncing or as streaming; however, the illusion is preferentially perceived as bouncing dots if their collision is accompanied by a brief clicking sound. Along with the other studies showing nonvisual modulation on visual perception (Shams & Kim, 2010), these data suggest that nonvisual inputs can significantly modulate the visual process for ambiguous visual motion.

Complete Chapter List

Search this Book: