The author of this chapter describes a binocular robotic vision system that was designed to emulate the neural images of cortical cells under vergence eye movements. The robotic vision system is constructed by employing a combinational strategy of neuromorphic engineering and conventional digital technology. The system consists of two silicon retinas and a field programmable gate array (FPGA). The silicon retinas carry out Laplacian-Gaussian-like spatial filtering, mimicking the response properties of the vertebrate retina. The outputs of the silicon retina chips on the left and right cameras are transmitted to the FPGA. The FPGA receives the outputs from the two simple cell chips and calculates the responses of complex cells based on the disparity energy model. This system provides complex cell outputs tuned to five different disparities in real-time. The vergence control signal is obtained by pooling these multiple complex cell responses. The system is useful for predicting the neural images of the complex cells and for evaluating the functional roles of cortical cells in real situations.
TopIntroduction
Vision is one of the most important sensory systems used to perceive the external world for both robots and humans. In the visual system in the brain, an external image is received by the left and right retina, and is sent to the primary visual cortex (V1). To understand the functional roles of visual neurons in such a visual pathway, computational models are useful. However, most of the computational models simulating the response properties of visual cortical cells were constructed from physiological experiments in which the electrical responses to simple visual stimuli, e.g., randomly modulated dots or slits of light, were recorded from anesthetized animals. To verify the function of these models in visual computation, one needs to study how the models respond to natural scenes in real-time. In a real situation, the visual system of the brain receives natural images continuously during the eye movements. Therefore, the responses of cortical cell networks, namely, the neural images generated in the real environments with consideration of the eye movements, need to be studied to understand the functional role of cortical cells.
In this paper, we describe a binocular robotic vision system that is designed to emulate the neural images of cortical cells during the vergence eye movement, which is used to change fixation and align the left and right eyes on the same object. Figure 1 shows the geometry of stereoscopic vision. The left and right eyes are fixed on point F. Here we define the binocular disparity, which is the difference between the positions of an object in the left and right retinal images, as xR-xL. The fixation point F projects onto the corresponding points on the left and right retinas and corresponds to zero disparity. The far point A and the near point B have negative and positive disparities, respectively. Vergence eye movements allow the left and right eyes to change fixation by rotating inward or outward. Here, to fix both eyes on the object located at point A, they should be rotated outwards. It is known that one of the important cues in this process is binocular disparity (Busettini, Miles, Krauzlis, 1996). In physiological experiments, it has been shown that step changes in the horizontal disparity of a large pattern induce a vergence response. A crossed (positive) disparity increases convergence, while an uncrossed (negative) disparity decreases it.
Figure 1. Schematic of stereopsis and vergence eye movement
In the following sections, the model for vergence eye movement, the construction of the robot vision system, and experiments for emulating the cortical cell responses using the robot vision system are described.
TopComputational Model For Vergence Eye Movement Control
Figure 2 shows the model for vergence eye movement control. The first stage of the model is the computation of binocular disparity. We employ a disparity energy model as a physiologically plausible computational model to detect the binocular disparity. The disparity energy model explains the response properties of the binocular complex cells in V1 (Ohzawa, DeAngelis, Freeman, 1990). This model adequately predicts the shape of the binocular receptive fields of complex cells in cats. The validity of the disparity energy model has also been confirmed in monkeys (Cumming, Parker, 1997).
Figure 2. Block diagram of the model for vergence eye movement control. The left block of the model is a disparity energy model (Ohzawa, DeAngelis, Freeman, 1990), which is well known as the model of disparity selective complex cells in V1. The right block generates a vergence eye movement control signal based on the responses from multiple complex cells tuned to different disparities.