A Semantics Sensitive Framework of Organization and Retrieval for Multimedia Databases

A Semantics Sensitive Framework of Organization and Retrieval for Multimedia Databases

Zhiping Shi (Institute of Computing Technology, Chinese Academy of Sciences, China), Qingyong Li (Beijing Jiaotong University, China), Qing He (Institute of Computing Technology, Chinese Academy of Sciences, China) and Zhongzhi Shi (Institute of Computing Technology, Chinese Academy of Sciences, China)
DOI: 10.4018/978-1-60566-174-2.ch013
OnDemand PDF Download:


Semantics-based retrieval is a trend of the Content-Based Multimedia Retrieval (CBMR). Typically, in multimedia databases, there exist two kinds of clues for query: perceptive features and semantic classes. In this chapter, we proposed a novel framework for multimedia database organization and retrieval, integrating the perceptive features and semantic classes. Thereunto, a semantics supervised cluster-based index organization approach (briefly as SSCI) was developed: the entire data set is divided hierarchically into many clusters until the objects within a cluster are not only close in the perceptive feature space, but also within the same semantic class; then an index entry is built for each cluster. Especially, the perceptive feature vectors in a cluster are organized adjacently in disk. Furthermore, the SSCI supports a relevance feedback approach: users sign the positive and negative examples regarded a cluster as unit rather than a single object. Our experiments show that the proposed framework can improve the retrieval speed and precision of the CBMR systems significantly.
Chapter Preview


The advances in the data capturing, storage, and communication technologies have made vast amounts of multimedia data be available to consumer and enterprise applications (Smeulders, 2002). To find needed data from multimedia databases, the initial method is the multimedia data are categorized and labeled according to human semantic understanding, then retrieved with the labeled keywords matching. It is an efficient method to organize a data collection by semantic classification according to people’s custom. However, it is a difficult and expensive manual task to label a large data set with semantic concepts, and the labeling process is subjective, inaccurate and incomplete. Moreover, the amount of the data in one class is too large to looking up. So the researchers proposed a CBMR technology. In the CBMR system, multimedia objects are usually represented by high-dimensional perceptive feature vectors, for example, an image is represented by a visual perceptive feature vector with some number of dimensions, and the similarity between two objects is defined by a distance function, e.g., Euclidean distance, between the corresponding perceptive feature vectors. CBMR is the similarity query. Similarity query is usually implemented by finding k feature vectors most similar to the feature vector of the query example, namely k-nearest neighbor (k-NN) search. Now CBMR has gained a degree of succeed, and a number of techniques extracting low-level perceptive features of multimedia automatically have been brought out. However, one side, there is no efficient index methods for large-scale perceptive features data that is represented by high-dimensional vectors. On the other hand, users of multimedia search engines are generally interested in retrieving data based on semantics, such as a video clip for “shoot events in football games” and so on. But the perceptive features of some data with relevant semantics may not be located very close in the perceptive feature space, or vice versa, the objects with similar perceptive features may come from different semantic classes. The difficulty in supporting semantics lies in the gap between perceptive features and semantic concepts, the so-called semantic gap (Smeulders, 2002). Thus, indexing multimedia data based only on perceptive features sometimes could not provide satisfied solutions.

Typically, there exist usually two kinds of clues for query in a large-scale multimedia database: 1) semantic classes, 2) perceptive features. Intuitively, it is reasonable to develop techniques that combine the advantages of both semantics and perceptive feature index.

In this chapter, we propose a semantics supervised cluster based index approach (briefly as SSCI) to achieve the target. We model the relationship between semantic classes and perceptive feature distributions of the data set with the Gaussian mixture model (GMM). The SSCI method proceeds as follows: the entire data set is divided hierarchically by a modified clustering technique into many clusters until the objects within a cluster not only are close in the perceptive feature space but also are within the same semantic class and the cluster here is called as index cluster, in particular, the perceptive feature vectors in an index cluster are organized adjacently in disk; an index entry (cluster index) including semantic clue and perceptive feature clue is built for each index cluster.

Based on the SSCI, we develop our approximate nearest neighbor (NN) searching technique that consists two phases: the first phase computes the distances between the query example and each cluster index and returns the clusters with the smallest distances, the so-called candidate clusters; then the second phase retrieves the original feature vectors within the candidate clusters to gain the approximate nearest neighbors. The main character of our technique is that it distinctly improves the speed and the semantic precision of CBMR.

Complete Chapter List

Search this Book:
Table of Contents
Zongmin Ma
Chapter 1
Danilo Avola, Fernando Ferri, Patrizia Grifoni
The novel technologies used in different application domains allow obtaining digital images with a high complex informative content, which can be... Sample PDF
Genetic Algorithms and Other Approaches in Image Feature Extraction and Representation
Chapter 2
Dany Gebara, Reda Alhajj
This chapter presents a novel approach for content-fbased image retrieval and demonstrates its applicability on non-texture images. The process... Sample PDF
Improving Image Retrieval by Clustering
Chapter 3
Gang Zhang, Z. M. Ma, Li Yan
Texture feature extraction and description is one of the important research contents in content-based medical image retrieval. The chapter first... Sample PDF
Review on Texture Feature Extraction and Description Methods in Content-Based Medical Image Retrieval
Chapter 4
Jafar M. Ali
Advances in data storage and image acquisition technologies have enabled the creation of large image datasets. Thus, it is necessary to develop... Sample PDF
Content-Based Image Classification and Retrieval: A Rule-Based System Using Rough Sets Framework
Chapter 5
David García Pérez, Antonio Mosquera, Stefano Berretti, Alberto Del Bimbo
Content-based image retrieval has been an active research area in past years. Many different solutions have been proposed to improve performance of... Sample PDF
Content Based Image Retrieval Using Active-Nets
Chapter 6
Ming Zhang, Reda Alhajj
Content-Based Image Retrieval (CBIR) aims to search images that are perceptually similar to the querybased on visual content of the images without... Sample PDF
Content-Based Image Retrieval: From the Object Detection/Recognition Point of View
Chapter 7
Chotirat “Ann” Ratanamahatana, Eamonn Keogh, Vit Niennattrakul
After the generation of multimedia data turning digital, an explosion of interest in their data storage, retrieval, and processing, has drastically... Sample PDF
Making Image Retrieval and Classification More Accurate Using Time Series and Learned Constraints
Chapter 8
Hakim Hacid, Abdelkader Djamel Zighed
A multimedia index makes it possible to group data according to similarity criteria. Traditional index structures are based on trees and use the... Sample PDF
A Machine Learning-Based Model for Content-Based Image Retrieval
Chapter 9
Ruofei Zhang, Zhongfei (Mark) Zhang
This chapter studies the user relevance feedback in image retrieval. We take this problem as a standard two-class pattern classification problem... Sample PDF
Solving the Small and Asymmetric Sampling Problem in the Context of Image Retrieval
Chapter 10
Chia-Hung Wei, Chang-Tsun Li
An image is a symbolic representation; people interpret an image and associate semantics with it based on their subjective perceptions, which... Sample PDF
Content Analysis from User's Relevance Feedback for Content-Based Image Retrieval
Chapter 11
Pawel Rotter, Andrzej M.J. Skulimowski
In this chapter, we describe two new approaches to content-based image retrieval (CBIR) based on preference information provided by the user... Sample PDF
Preference Extraction in Image Retrieval
Chapter 12
Iker Gondra
In content-based image retrieval (CBIR), a set of low-level features are extracted from an image to represent its visual content. Retrieval is... Sample PDF
Personalized Content-Based Image Retrieval
Chapter 13
Zhiping Shi, Qingyong Li, Qing He, Zhongzhi Shi
Semantics-based retrieval is a trend of the Content-Based Multimedia Retrieval (CBMR). Typically, in multimedia databases, there exist two kinds of... Sample PDF
A Semantics Sensitive Framework of Organization and Retrieval for Multimedia Databases
Chapter 14
Chia-Hung Wei, Chang-Tsun Li, Yue Li
As distributed mammogram databases at hospitals and breast screening centers are connected together through PACS, a mammogram retrieval system is... Sample PDF
Content-Based Retrieval for Mammograms
Chapter 15
Ying-li Tian, Arun Hampapur, Lisa Brown, Rogerio Feris, Max Lu, Andrew Senior
Video surveillance automation is used in two key modes: watching for known threats in real-time and searching for events of interest after the fact.... Sample PDF
Event Detection, Query, and Retrieval for Video Surveillance
Chapter 16
Min Chen, Shu-Ching Chen
This chapter introduces an advanced content-based image retrieval (CBIR) system, MMIR, where Markov model mediator (MMM) and multiple instance... Sample PDF
MMIR: An Advanced Content-Based Image Retrieval System Using a Hierarchical Learning Framework
About the Contributors