To perceive our world, we make full use of multiple sources of sensory information derived from different modalities which include five basic sensory systems; visual, auditory, tactile, olfactory, and gustatory. In the real world, we normally simultaneously acquire information from different sensory receptors. Therefore, multisensory integration in the brain plays an important role in performance and perception. This review focuses on the crossmodal between the visual and tactile. Many previous studies have indicated that visual information effects tactile perception and in return, tactile perception is also active in the MT, the main visual motion information processing area. However, few studies have explored how information of the crossmodal between the visual and tactile is processed. Here, the authors highlight the processing mechanism of the crossmodal in the brain. They show that integration between the visual and tactile has two stages: combination and integration.
TopBackground
To perceive the world around us, we need to get more useful information about our environment from various sensory channels. Different sensory modalities provide redundant information: to obtain the object information (size, shape, curvature, orientation) we should see and touch the object and then make a decision. In general, most of our senses are stimulated simultaneously, and information about the environment is acquired by multiple modalities at the same time. Therefore, our perception is often driven by multiple sensory channels.
Recently a growing number of research studies have focused on the crossmodal. For example, tactile events can influence auditory evoked activity at the earliest stage of cortical processing (Foxe & Schroeder, 2005; Schurmann, Caetano, Hlushchuk, Jousmaki, & Hari, 2006). The middle temporal visual cortex (MT) responds both to tactile and auditory motion (Blake, Sobel, & James, 2004; Kayser, Logothetis, & Panzeri, 2010). For the integration between the visual and auditory, we have good knowledge about the crossmodal between the visual and auditory. Behavioral studies have showed that responses to auditovisual stimuli are more rapid and more accurate than unimodal visual or auditory stimuli (Giard & Peronnet, 1999; Molholm et al., 2002; Teder-Sälejärvi, McDonald, Di Russo, & Hillyard, 2002; Teder-Sälejärvi, Russo, McDonald, & Hillyard, 2005). The primary visual cortex (V1) responds to input from the auditory (Foxe et al., 2000; Martuzzi et al., 2007; Romei, Murray, Merabet, & Thut, 2007; Shams, Iwaki, Chawla, & Bhattacharya, 2005), and in return, the primary auditory cortex can be affected by the visual (Romei, et al., 2007). In addition, some studies have suggested that the auditory modality is dominant in the temporal domain and the visual modality is dominant in the spatial domain (Slutsky & Recanzone, 2001). Multisensory integration between the visual and auditory takes place across multiple levels, including sub-cortical areas like the superior colliculus (Meredith, Nemitz, & Stein, 1987; Meredith & Stein, 1996), early cortical areas like the primary visual and auditory cortices (Romei, et al., 2007; Shams, et al., 2005), and higher cortical areas like the superior temporal sulcus and intraparietal areas (Barraclough, Xiao, Baker, Oram, & Perrett, 2005; Benevento, Fallon, Davis, & Rezak, 1977; Lewis & Van Essen, 2000; Linden, Grunewald, & Andersen, 1999). For instance, Barraclough, et al., 2005showed that matching sights and sounds of actions, such as ripping a sheet of paper, integrate in the superior temporal cortex. Illusions like the McGurk effect (McGurk & MacDonald, 1976) most probably occur at higher areas because of the more complex nature of the information.
However, we are not familiar with the integration between the visual and tactile, especially integration of motion. In this review, we will focus on the specific case of visual-tactile interactions regarding motion perception. This particular case is very useful because of the close relationship between the tactile and visual. In fact, we have good knowledge on tactile and visual perception (see next section).