Automated Evaluation of Interest Point Detectors

Automated Evaluation of Interest Point Detectors

Simon R. Lang (School of Computer Science, Engineering and Mathematics, Flinders University, Adelaide, Australia), Martin H. Luerssen (School of Computer Science, Engineering and Mathematics, Flinders University, Adelaide, Australia) and David M. W. Powers (School of Computer Science, Engineering and Mathematics, Flinders University, Adelaide, Australia)
Copyright: © 2014 |Pages: 20
DOI: 10.4018/ijsi.2014010107


Interest point detectors are important components in a variety of computer vision systems. This paper demonstrates an automated virtual 3D environment for controlling and measuring detected interest points on 2D images in an accurate and rapid manner. Real-time affine transform tools enable easy implementation and full automation of complex scene evaluations without the time-cost of a manual setup. Nine detectors are tested and compared using evaluation and testing methods based on Schmid, Mohr, and Bauckhage (2000). Each detector is tested on the BSDS500 image set and 34 3D scanned, and manmade models using rotation in the X, Y, and Z axis as well as scale in the X,Y axis. Varying degrees of noise on the models are also tested. Results demonstrate the differing performance and behaviour of each detector across the evaluated transformations, which may assist computer vision practitioners in choosing the right detector for their application.
Article Preview


Interest points are pixel coordinates highlighting a location that contains a simple localised feature. These often serve as a means for tracking a scene's camera motion, or an object's motion over time, and are therefore highly relevant to augmented reality applications (Gauglitz, Höllerer, & Turk, 2011; Gil, Mozos, Ballesta, & Reinoso, 2010). In Schmid's classic work (2000), the performance of interest point detectors was evaluated by manipulating a camera and a backlit projector. Such a manual approach to image capture and homography estimation has since been used in a number of related studies (Gauglitz, Höllerer, & Turk, 2011; Gil, Mozos, Ballesta, & Reinoso, 2010; Guillaume Galés, 2010; Olague & Trujillo, 2011) and typically relies on marked-up images or predictable, physical transformations of real scenes to measure the performance of a detector. While this strategy implies a higher chance of inaccuracies and smaller data sets, it has remained largely unchanged, although its application has expanded over the years. Our paper therefore aims to revisit Schmid's work and recreate the same experimental conditions in a much more powerful and versatile virtual 3D environment. This way we can utilise complex and precise transformations that would have been time consuming or impossible to perform in a real world environment. The position of interest points on 3D models can thus be quickly and accurately determined so that their repeatability—and by extension the performance of the detector that found these points—can be properly measured, as illustrated in Figure 2.

Figure 2.

Illustration of 3D model being rotated from 0° to 50°. 3D correspondences are determined via inverse projection and rotated by 50°, so that each point in can be mapped (and vice versa, not shown). Repeatability can now be measured easily by t testing for points in close neighbourhood.


The most common method of evaluation of interest points in a scene is based on repeatability. By measuring the repeatability of interest points in two slightly different scenes and , we can determine how well a given detector performs (Schmid, Mohr, & Bauckhage, 2000). Consider that we compare detected interest points and from and . The distance of any two given interest points can be determined by using a homography of an interest point in two different images of a scene. The repeatability of interest points is gauged by its locality within a radial threshold . Any interest points that appear in both scenes (denoted and ) and are within the threshold distance are considered repeated (as shown in Figure 1). This determines a set of “repeated” points as:

Figure 1.

Schmid repeatability: the points and are the projections of a 3D point onto images and . A detected point is repeated if is detected in the neighbourhood of

Complete Article List

Search this Journal:
Open Access Articles: Forthcoming
Volume 7: 4 Issues (2019): 1 Released, 3 Forthcoming
Volume 6: 4 Issues (2018)
Volume 5: 4 Issues (2017)
Volume 4: 4 Issues (2016)
Volume 3: 4 Issues (2015)
Volume 2: 4 Issues (2014)
Volume 1: 4 Issues (2013)
View Complete Journal Contents Listing