Solving the Small and Asymmetric Sampling Problem in the Context of Image Retrieval

Solving the Small and Asymmetric Sampling Problem in the Context of Image Retrieval

Ruofei Zhang (Yahoo!, Inc., USA) and Zhongfei (Mark) Zhang (SUNY Binghamton, USA)
DOI: 10.4018/978-1-60566-174-2.ch009
OnDemand PDF Download:


This chapter studies the user relevance feedback in image retrieval. We take this problem as a standard two-class pattern classification problem aiming at refining the retrieval precision by learning through the user relevance feedback data. However, we have investigated the problem by noting two important unique characteristics of the problem: small sample collection and asymmetric sample distributions between positive and negative samples. We have developed a novel approach to empirical Bayesian learning to solve for this problem by explicitly exploiting the two unique characteristics, which is the methodology of BAyesian Learning in ASymmetric and Small sample collections, thus called BALAS. In BALAS different learning strategies are used for positive and negative sample collections, respectively, based on the two unique characteristics. By defining the relevancy confidence as the relevant posterior probability, we have developed an integrated ranking scheme in BALAS which complementarily combines the subjective relevancy confidence and the objective similarity measure to capture the overall retrieval semantics. The experimental evaluations have confirmed the rationale of the proposed ranking scheme, and have also demonstrated that BALAS is superior to an existing relevance feedback method in the current literature in capturing the overall retrieval semantics.
Chapter Preview


Very large collections of images have become ever more common than before. From stock photo collections and proprietary databases to the World Wide Web, these collections are diverse and often poorly indexed; unfortunately, image retrieval systems have not kept pace with the collections they are searching. How to effectively index and retrieve semantically relevant images according to users’ queries is a challenging task. Most existing image retrieval systems, such as image search engines in Yahoo! (Yahoo! Search website) and Google (Google search website), are textual based. The images are searched by using the surrounding text, captions, keywords, etc. Although the search and retrieval techniques based on textual features can be easily automated, they have several inherent drawbacks. First, textual description is not capable of capturing the visual contents of an image accurately and in many circumstances the textual annotations are not available. Second, different people may describe the content of an image in different ways, which limits the recall performance of textual-based image retrieval systems. Third, for some images there is something that no words can convey. Try to imagine an editor taking in pictures without seeing them or a radiologist deciding on a verbal description. The content of images is beyond words. They have to be seen and searched as pictures: by objects, by style, by purpose.

To resolve these problems, Content-Based Image Retrieval (CBIR) has attracted significant research attention (Marsicoi et al., 1997; Ratan & Grimson, 1997; Sivic & Zisserman, 2003; Liu et al., 1998). In CBIR, a query image (an image to which a user tries to find similar ones) is imposed to the image retrieval system to obtain the semantically relevant images. The similarity between the query image and the indexed images in the image database is determined by their visual contents, instead of the textual information. Early research of CBIR focused on finding the “best” representation for image features, e. g., color, texture, shape, and spatial relationships. The similarity between two images is typically determined by the distances of individual low-level features and the retrieval process is performed by a k-nn search in the feature space (Del Bimbo, 1999). In this context, high level concepts and user’s perception subjectivity cannot be well modeled. Recent approaches introduce more advanced human-computer interaction (HCI) into CBIR. The retrieval procedure incorporates user’s interaction into the loop, which consists of several iterations. In each iteration, the user cast positive samples (relevant images) as well as negative samples (irrelevant images) for the returned results from the previous iteration. Based on user’s feedback, the retrieval system is able to adaptively customize the search results to the user’s query preference. This interaction mechanism is called relevance feedback, which allows a user to continuously refine his/her querying information after submitting a coarse initial query to the image retrieval system. This approach greatly reduces the labor required to precisely compose a query and easily captures the user’s subjective retrieval preference.

However, most approaches to relevance feedback, e. g., (Rui et al., 1998; Picard et al., 1996; Porkaew et al., 1999; Zhang & Zhang, 2004), are based on heuristic formulation of empirical parameter adjustment, which is typically ad hoc and not systematic, and thus cannot be substantiated well. Some of the recent work (Wu et al., 2000; MacArthur et al., 2000; Tieu & Viola, 2000; Tong & Chan, 2001; Tao & Tang, 2004) formulates the relevance feedback problem as a classification or learning problem. Without further exploiting the unique characteristics of the training samples in the relevance feedback for image retrieval, it is difficult to map the image retrieval problem to a general two-class (i.e., relevance vs. irrelevance) classification problem in realistic applications.

Complete Chapter List

Search this Book:
Table of Contents
Zongmin Ma
Chapter 1
Danilo Avola, Fernando Ferri, Patrizia Grifoni
The novel technologies used in different application domains allow obtaining digital images with a high complex informative content, which can be... Sample PDF
Genetic Algorithms and Other Approaches in Image Feature Extraction and Representation
Chapter 2
Dany Gebara, Reda Alhajj
This chapter presents a novel approach for content-fbased image retrieval and demonstrates its applicability on non-texture images. The process... Sample PDF
Improving Image Retrieval by Clustering
Chapter 3
Gang Zhang, Z. M. Ma, Li Yan
Texture feature extraction and description is one of the important research contents in content-based medical image retrieval. The chapter first... Sample PDF
Review on Texture Feature Extraction and Description Methods in Content-Based Medical Image Retrieval
Chapter 4
Jafar M. Ali
Advances in data storage and image acquisition technologies have enabled the creation of large image datasets. Thus, it is necessary to develop... Sample PDF
Content-Based Image Classification and Retrieval: A Rule-Based System Using Rough Sets Framework
Chapter 5
David García Pérez, Antonio Mosquera, Stefano Berretti, Alberto Del Bimbo
Content-based image retrieval has been an active research area in past years. Many different solutions have been proposed to improve performance of... Sample PDF
Content Based Image Retrieval Using Active-Nets
Chapter 6
Ming Zhang, Reda Alhajj
Content-Based Image Retrieval (CBIR) aims to search images that are perceptually similar to the querybased on visual content of the images without... Sample PDF
Content-Based Image Retrieval: From the Object Detection/Recognition Point of View
Chapter 7
Chotirat “Ann” Ratanamahatana, Eamonn Keogh, Vit Niennattrakul
After the generation of multimedia data turning digital, an explosion of interest in their data storage, retrieval, and processing, has drastically... Sample PDF
Making Image Retrieval and Classification More Accurate Using Time Series and Learned Constraints
Chapter 8
Hakim Hacid, Abdelkader Djamel Zighed
A multimedia index makes it possible to group data according to similarity criteria. Traditional index structures are based on trees and use the... Sample PDF
A Machine Learning-Based Model for Content-Based Image Retrieval
Chapter 9
Ruofei Zhang, Zhongfei (Mark) Zhang
This chapter studies the user relevance feedback in image retrieval. We take this problem as a standard two-class pattern classification problem... Sample PDF
Solving the Small and Asymmetric Sampling Problem in the Context of Image Retrieval
Chapter 10
Chia-Hung Wei, Chang-Tsun Li
An image is a symbolic representation; people interpret an image and associate semantics with it based on their subjective perceptions, which... Sample PDF
Content Analysis from User's Relevance Feedback for Content-Based Image Retrieval
Chapter 11
Pawel Rotter, Andrzej M.J. Skulimowski
In this chapter, we describe two new approaches to content-based image retrieval (CBIR) based on preference information provided by the user... Sample PDF
Preference Extraction in Image Retrieval
Chapter 12
Iker Gondra
In content-based image retrieval (CBIR), a set of low-level features are extracted from an image to represent its visual content. Retrieval is... Sample PDF
Personalized Content-Based Image Retrieval
Chapter 13
Zhiping Shi, Qingyong Li, Qing He, Zhongzhi Shi
Semantics-based retrieval is a trend of the Content-Based Multimedia Retrieval (CBMR). Typically, in multimedia databases, there exist two kinds of... Sample PDF
A Semantics Sensitive Framework of Organization and Retrieval for Multimedia Databases
Chapter 14
Chia-Hung Wei, Chang-Tsun Li, Yue Li
As distributed mammogram databases at hospitals and breast screening centers are connected together through PACS, a mammogram retrieval system is... Sample PDF
Content-Based Retrieval for Mammograms
Chapter 15
Ying-li Tian, Arun Hampapur, Lisa Brown, Rogerio Feris, Max Lu, Andrew Senior
Video surveillance automation is used in two key modes: watching for known threats in real-time and searching for events of interest after the fact.... Sample PDF
Event Detection, Query, and Retrieval for Video Surveillance
Chapter 16
Min Chen, Shu-Ching Chen
This chapter introduces an advanced content-based image retrieval (CBIR) system, MMIR, where Markov model mediator (MMM) and multiple instance... Sample PDF
MMIR: An Advanced Content-Based Image Retrieval System Using a Hierarchical Learning Framework
About the Contributors