On Hierarchical Content-Based Image Retrieval by Dynamic Indexing and Guided Search

Introduction

Cognitive informatics is a multidisciplinary research area, which studies not only how human brains and minds internally process information but also their engineering applications. It covers the Object-Attribute-Relation (OAR) model of information representation in the brain (Wang, 2007b; Wang & Wang, 2006), the cognitive processes of formal inferences (Wang, 2007a), the formal knowledge system (Wang, 2006a), and so on.

Image retrieval is an important part of multimedia systems and requires an integration of research in image understanding, artificial intelligence and databases to address the problem of information retrieval from large collections of images and video frames. The developments in this area are summarized in Antani, Kasturi, and Jain (2002) and Rui, Huang, and Chang (1999). In general, image retrieval approaches fall into two categories: attribute-based methods and content-based methods. Examples of such methods include the Kodak Picture Exchange System (KPX) (Larish, 1995) and the PressLink library (Martucci, 1995). Examples of content-based image retrieval systems include QBIC (Flickner et al., 1995), Virage, Photobook (Pentland, Picard, & Sclaroff, 1996), CANDID (Kelley, Cannon, & Hush, 1995) and Garlic (Cody et al., 1995). The current content-based and object-oriented methods cannot process abstract queries, and multiple image features are not integrated for similarity measures. In addition, they are computationally expensive and somewhat domain dependent.

Applications on the World-Wide Web require a multimedia system with scalability, flexibility and efficiency. Although some existing systems, such as WebSeek and C-BIRD (Smith & Chang, 1997), support both attribute-based queries and content-based queries, they employ limited visual features for retrieval and they lack a general approach to deal with multiple features for indexing and query. Moreover, these systems cannot handle audio data for a comprehensive multimedia system. The MARS (Multimedia Analysis and Retrieval System) project (Huang & Naphade, 2000) aims to bridge the gap between the low-level features and the high-level semantic concepts. However, the challenges faced by new techniques for processing and analyzing multimedia information remain untackled, and the traditional analysis of image data alone cannot provide satisfactory solutions to these problems. Most of the existing techniques for multimedia information retrieval are based on the use of conventional database structures to handle large collections of high-dimensional multimedia data. Although the research on multimedia database systems (Flickner et al., 1995; Antani, Kasturi, & Jain, 2002) has made advances in the creation of large multimedia databases with effective facilities for query processing, that work has mainly focused on data modeling and structuring.

The main contributions of this paper may be summarized as follows: this paper proposed an image data warehouse structure and a new image data model to support multiple image feature extraction and integration, which allows dynamic query processing and hierarchical content-based image retrieval. The proposed data warehousing model will allow users to combine multiple image features in a top-down manner to facilitate image representation, indexing and search. Instead of using a fixed similarity measurement to search for the best matching, this paper adopted a statistically based feature selection scheme that generates the matching criteria in accordance with the type of query and the level of search. In addition, this paper introduced a feature component code to guide the coarse-to-fine search in a hierarchical structure. To demonstrate the feasibility and the advantages of our approach, the proposed algorithms are applied to several case studies.