Content Analysis from User's Relevance Feedback for Content-Based Image Retrieval

Content Analysis from User's Relevance Feedback for Content-Based Image Retrieval

Chia-Hung Wei (Ching Yun University, Taiwan) and Chang-Tsun Li (University of Warwick, UK)
DOI: 10.4018/978-1-60566-174-2.ch010
OnDemand PDF Download:


An image is a symbolic representation; people interpret an image and associate semantics with it based on their subjective perceptions, which involves the user’s knowledge, cultural background, personal feelings and so on. Content-based image retrieval (CBIR) systems must be able to interact with users and discover the current user’s information needs. An interactive search paradigm that has been developed for image retrieval is machine learning with a user-in-the-loop, guided by relevance feedback, which refers to the notion of relevance of the individual image based on the current user’s subjective judgment. Relevance feedback serves as an information carrier to convey the user’s information needs / preferences to the retrieval system. This chapter not only provides the fundamentals of CBIR systems and relevance feedback for understanding and incorporating relevance feedback into CBIR systems, but also discusses several approaches to analyzing and learning relevance feedback.
Chapter Preview

1. Introduction

The rapid growth in the amount of digital images has highlighted the importance of effective retrieval approaches in order to facilitate the searching and browsing of large image databases. Although the design of content-based image retrieval (CBIR) systems is based on the nature of the underlying images and the system's purposes, one of the common purposes of all image retrieval systems is to satisfy human information needs and support human activities in an efficient and effective way. The development of an image retrieval system has to take human factors into account. Among human factors, subjective perception is one of the most challenging issues. An image is a symbolic representation; people interpret an image and associate semantics with it based on their subjective perceptions, which involves the user’s knowledge, cultural background, personal feelings and so on (Jaimes, 2006b).

An important assumption in image retrieval is that each user’s information need is different and time varying (Zhou & Huang, 2003). This assumption indicates that humans exhibiting subjective perceptions when interpreting images can be classified as different information seekers or same information seekers (Jaimes, 2006a). Different information seekers normally have different interpretations for the same image based on their individual perceptions. As a result, when different information seekers provide the same query example, they will have different satisfactory degrees for the same search results; even the same information seekers have different subjective perceptions as time evolves.

Another challenging issue arises from the difference between two descriptions of an object by high-level semantics and representations of low-level pixel data (Liu, Zhang, Lu, & Ma, 2007; Vasconcelos, 2007). The difference, known as the semantic gap, exists because low-level features are more easily computed in the system design process, but high-level queries are used as the starting point of the retrieval process. The semantic gap involves not only the conversion between low-level features and high-level semantics, but also the understanding of contextual meaning of the query involving human knowledge and emotion. Figure 1 shows that visual similarity mismatches human similarity judgments, resulting in a semantic gap between the user and the CBIR system. The “riding bicycle” query contains color gradients and two circular shapes that might guide a CBIR system, which utilizes shape and color as features for discriminating images, to associate it with objects, such as earphones, glasses, binoculars and two coins, with similar low-level features. However, the user actually looks for “riding bicycle” images, correlating the query example with high-level semantic context.

Figure 1.

Visual similarity does not match with human similarity judgments, resulting in a semantic gap. (a) A query example. (b) Visual feature matches. (c) Semantic matches.

Complete Chapter List

Search this Book:
Table of Contents
Zongmin Ma
Chapter 1
Danilo Avola, Fernando Ferri, Patrizia Grifoni
The novel technologies used in different application domains allow obtaining digital images with a high complex informative content, which can be... Sample PDF
Genetic Algorithms and Other Approaches in Image Feature Extraction and Representation
Chapter 2
Dany Gebara, Reda Alhajj
This chapter presents a novel approach for content-fbased image retrieval and demonstrates its applicability on non-texture images. The process... Sample PDF
Improving Image Retrieval by Clustering
Chapter 3
Gang Zhang, Z. M. Ma, Li Yan
Texture feature extraction and description is one of the important research contents in content-based medical image retrieval. The chapter first... Sample PDF
Review on Texture Feature Extraction and Description Methods in Content-Based Medical Image Retrieval
Chapter 4
Jafar M. Ali
Advances in data storage and image acquisition technologies have enabled the creation of large image datasets. Thus, it is necessary to develop... Sample PDF
Content-Based Image Classification and Retrieval: A Rule-Based System Using Rough Sets Framework
Chapter 5
David García Pérez, Antonio Mosquera, Stefano Berretti, Alberto Del Bimbo
Content-based image retrieval has been an active research area in past years. Many different solutions have been proposed to improve performance of... Sample PDF
Content Based Image Retrieval Using Active-Nets
Chapter 6
Ming Zhang, Reda Alhajj
Content-Based Image Retrieval (CBIR) aims to search images that are perceptually similar to the querybased on visual content of the images without... Sample PDF
Content-Based Image Retrieval: From the Object Detection/Recognition Point of View
Chapter 7
Chotirat “Ann” Ratanamahatana, Eamonn Keogh, Vit Niennattrakul
After the generation of multimedia data turning digital, an explosion of interest in their data storage, retrieval, and processing, has drastically... Sample PDF
Making Image Retrieval and Classification More Accurate Using Time Series and Learned Constraints
Chapter 8
Hakim Hacid, Abdelkader Djamel Zighed
A multimedia index makes it possible to group data according to similarity criteria. Traditional index structures are based on trees and use the... Sample PDF
A Machine Learning-Based Model for Content-Based Image Retrieval
Chapter 9
Ruofei Zhang, Zhongfei (Mark) Zhang
This chapter studies the user relevance feedback in image retrieval. We take this problem as a standard two-class pattern classification problem... Sample PDF
Solving the Small and Asymmetric Sampling Problem in the Context of Image Retrieval
Chapter 10
Chia-Hung Wei, Chang-Tsun Li
An image is a symbolic representation; people interpret an image and associate semantics with it based on their subjective perceptions, which... Sample PDF
Content Analysis from User's Relevance Feedback for Content-Based Image Retrieval
Chapter 11
Pawel Rotter, Andrzej M.J. Skulimowski
In this chapter, we describe two new approaches to content-based image retrieval (CBIR) based on preference information provided by the user... Sample PDF
Preference Extraction in Image Retrieval
Chapter 12
Iker Gondra
In content-based image retrieval (CBIR), a set of low-level features are extracted from an image to represent its visual content. Retrieval is... Sample PDF
Personalized Content-Based Image Retrieval
Chapter 13
Zhiping Shi, Qingyong Li, Qing He, Zhongzhi Shi
Semantics-based retrieval is a trend of the Content-Based Multimedia Retrieval (CBMR). Typically, in multimedia databases, there exist two kinds of... Sample PDF
A Semantics Sensitive Framework of Organization and Retrieval for Multimedia Databases
Chapter 14
Chia-Hung Wei, Chang-Tsun Li, Yue Li
As distributed mammogram databases at hospitals and breast screening centers are connected together through PACS, a mammogram retrieval system is... Sample PDF
Content-Based Retrieval for Mammograms
Chapter 15
Ying-li Tian, Arun Hampapur, Lisa Brown, Rogerio Feris, Max Lu, Andrew Senior
Video surveillance automation is used in two key modes: watching for known threats in real-time and searching for events of interest after the fact.... Sample PDF
Event Detection, Query, and Retrieval for Video Surveillance
Chapter 16
Min Chen, Shu-Ching Chen
This chapter introduces an advanced content-based image retrieval (CBIR) system, MMIR, where Markov model mediator (MMM) and multiple instance... Sample PDF
MMIR: An Advanced Content-Based Image Retrieval System Using a Hierarchical Learning Framework
About the Contributors