Methodologies for Evaluating Disparity Estimation Algorithms

Introduction

A 3D scene can be described as a set of points in space. It can be captured, from different viewpoints, by a stereo camera system generating a stereo image pair. The conjugate projections of a common scene point are captured into image planes in different image coordinates. These projections are corresponding points, and the shift or displacement between them is termed disparity (Hartley & Zisserman, 2003). Disparity is, in essence, a vector related corresponding points. Disparity values are used to recover the 3D information of the captured scene. If the disparity of a point is known, its original 3D position can be recovered by a triangulation process. In fact, the 3D information recovery process from a stereo image is an inverse and ill-posed problem, due to depth ambiguity, data instability and lack of information. A stereo image pair can be modified in such a way that the displacement between corresponding points becomes horizontal (Fusiello et al., 2000). Such modified stereo image pair is the input to disparity estimation algorithms, which produce as output an estimated disparity map. Nevertheless, even in horizontal parallax, there is still a lack of information about the correspondences between points. Such lack of information gives rise to the stereo correspondence problem. Moreover, the stereo correspondence problem involves two inherent problems: ambiguity and occlusion. The projections into image planes are ambiguous by nature, since different objects, at different depths, can generate equal projections into image planes. Moreover, multiple objects captured into a stereo image pair may have the same appearance. On the other hand, a point may lack of a corresponding point – be occluded – in the conjugate image plane. In fact, occlusion phenomena arise naturally in stereo images, but it is not known beforehand.

The assessment of disparity estimation is important since a small inaccuracy in an estimated disparity may produce a large error in the recovered 3D. The performance of disparity estimation algorithms can be assessed by following an evaluation methodology. This assessment has several applications such as comparing algorithms (Szeliski, 1999; Szeliski & Zabih, 2000; Scharstein & Szeliski, 2002; Tombari et al., 2010; Cabezas & Trujillo, 2011), comparing methods and procedures (Tombari et al. 2008; Hirschmuller & Scharstein, 2009; Bleyer & Chambon, 2010), tuning algorithm’s parameters within a particular context (Hoyos et al., 2011), and identifying algorithm’s advantages or drawbacks (Kostlivá et al., 2008), among others.