Techniques for Content-Based Multimedia Retrieval

Techniques for Content-Based Multimedia Retrieval

Chia-Hung Wei (Ching Yun University, Taiwan) and Sherry Y. Chen (Brunel University, UK)
DOI: 10.4018/978-1-61692-859-9.ch001


As the size of multimedia database grows, it becomes impractical to manually annotate all contents and attributes of the media, and the difficulty in finding desired information increases. To copy with these challenges, content-based multimedia retrieval systems have been developed for various applications. The chapter not only provides a conceptual architecture for the design of content-based retrieval system, but also discusses essential components of retrieval system and their research issues, including feature extraction and representation, dimension reduction of feature vector, indexing, and query specifications. As content-based multimedia retrieval is a young research field and there exists many challenging research problems, this chapter also addresses several research issues for the future research.
Chapter Preview


In the past decade there has been rapid growth in the use of digital media, such as images, video, and audio. As the use of digital media increases, retrieval and management techniques become more important in order to facilitate the effective searching and browsing of large multimedia databases. Before the emergence of content-based retrieval, media was annotated with text, allowing the media to be accessed by text-based searching. Through textual description, media is managed and retrieved based on the classification of subject or semantics. This hierarchical structure, like yellow pages, allows users to easily navigate and browse, or search using standard Boolean queries. However, with the emergence of massive multimedia databases, the traditional text-based search suffers from the following limitations (Djeraba, 2003; Shah et al., 2004):

  • Manual annotations require too much time and are expensive to implement. As the number of media in a database grows, the difficulty in finding desired information increases. It becomes infeasible to manually annotate all attributes of the media content. Annotating a sixty-minute video, containing more than 100,000 images, consumes a vast amount of time and expense.

  • Manual annotations fail to deal with the discrepancy of subjective perception. The phrase, “an image says more than a thousand words,” implies that the textual description is sufficient for depicting subjective perception. To capture all concepts, thoughts, and feelings for the content of any media is almost impossible.

  • Some media contents are difficult to concretely describe in words. For example, a piece of melody without lyric or irregular organic shape cannot easily be expressed in textual form, but people expect to search media with similar contents based on examples they provided.

In an attempt to address these limitations, content-based retrieval employs content information to automatically index data with minimal human intervention.



Content-based retrieval has been proposed by different communities for various applications. These applications includes

  • Medical Diagnosis: The medical community is currently developing picture archiving and communication systems (PACS), which integrate imaging modalities and interfaces with hospital and departmental information systems in order to manage the storage and distribution of images to radiologists, physicians, specialists, clinics, and imaging centers. A crucial requirement in PACS is to provide an efficient search function to access desired images. As images with the similar pathology-bearing regions can be found and interpreted, those images can be applied to aid diagnosis for image-based reasoning. For example, Wei & Li (2006) proposed a content-based retrieval system for locating mammograms with similar pathological characteristics.

  • Intellectual Property: Trademark image registration has applied content-based retrieval techniques to compare a new candidate mark with existing marks to ensure that there is no repetition. Copyright protection can also benefit from content-based retrieval as copyright owners are able to search and identify unauthorized copies of images on the Internet. For example, Jiang et al. (2006) developed a content-based system using adaptive selection of visual features for trademark image retrieval.

  • Broadcasting Archives: Every day broadcasting companies produce a lot of audio-visual data. To deal with these large archives, which can contain millions of hours of video and audio data, content-based retrieval techniques are used to annotate their contents and summarize the audio-visual data to drastically reduce the volume of raw footage. For example, Lopez & Chen (2006) developed a content-based video retrieval system to support news and sports retrieval.

  • Multimedia Searching on the Internet: Although a large amount of multimedia has been made available on the Internet for retrieval, existing search engines mainly perform text-based retrieval. To access the various media on the Internet, content-based search engines can assist users in searching the media with the most similar contents based on queries. For example, Khan (2007) designed a framework for image annotation and used ontology to enable content-based image retrieval on the Internet.

Complete Chapter List

Search this Book: