From Classification to Retrieval: Exploiting Pattern Classifiers in Semantic Image Indexing and Retrieval

Abstract

Users query images by using semantics. Though low-level features can be easily extracted from images, they are inconsistent with human visual perception. Hence, low-level features cannot provide sufficient information for retrieval. High-level semantic information is useful and effective in retrieval. However, semantic information is heavily dependent upon semantic image regions and beyond, which are difficult to obtain themselves. Bridging this semantic gap between computed visual features and user query expectation poses a key research challenge in managing multimedia semantics. As a spin-off from pattern recognition and computer vision research more than a decade ago, content-based image retrieval research focuses on a different problem from pattern classification though they are closely related. When the patterns concerned are images, pattern classification could become an image classification problem or an object recognition problem. While the former deals with the entire image as a pattern, the latter attempts to extract useful local semantics, in the form of objects, in the image to enhance image understanding. In this chapter, we review the role of pattern classifiers in state-of-the-art content-based image retrieval systems and discuss their limitations. We present three new indexing schemes that exploit pattern classifiers for semantic image indexing, and illustrate the usefulness of these schemes on the retrieval of 2,400 unconstrained consumer images.