Putting the Content Into Context: Features and Gaps in Image Retrieval

Putting the Content Into Context: Features and Gaps in Image Retrieval

Henning Müller (University and Hospitals of Geneva and University of Applied Sciences, Switzerland) and Jayashree Kalpathy-Cramer (Oregon Health and Science University, USA)
Copyright: © 2011 |Pages: 11
DOI: 10.4018/978-1-60960-780-7.ch006
OnDemand PDF Download:
No Current Special Offers


Digital management of medical images is becoming increasingly important as the number of images being created in medical settings everyday is growing rapidly. Content-based image retrieval or techniques based on the query-by-example paradigm have been studied extensively in computer vision. However, the global, low level visual features automatically extracted by these algorithms do not always correspond to high level concepts that a user has in his mind for searching. The role of image retrieval in diagnostic medicine can be quite complex, making it difficult for the user to express his/her information needs appropriately. Image retrieval in medicine needs to evolve from purely visual retrieval to a more holistic, case-based approach that incorporates various multimedia data sources. These include multiple images, free text, structured data, as well as external knowledge sources and ontologies.
Chapter Preview


Digital multimodal archives have become ubiquitous with the rapid growth of the Internet, available computing power, and other technological advances, leading to immense amounts of digital multimodal data generation in the information society. Most common forms of such data include structured data, free text, audio, images, and videos, and of course combinations of all these. The need for semi- or fully-automatic means of organizing massive databases containing structured and unstructured components in this multimodal environment has exploded with the generation speed of such databases greatly exceeding the anticipated rates.

Images in Clinical Practice

Medical images have become a significant component of clinical practice and research (Bui, Taira, Dionisio, Aberle, El-Saden, & Kangarloo, 2002). Due to advances in medical imaging technology, vast quantities of medical images covering a large variety of conditions are produced and stored. This variety is steadily growing with new imaging technologies developing (new contrast agents, higher resolutions, and thinner slices) and combinations of modalities such as PET/CTs making it even harder for clinicians to really understand all available information sources. Combining all the available information sources for a single patient is even harder as psychological literature shows clearly that humans can only integrate a fairly small number of information sources, from 3-7 depending on the tests (Miller, 1956; Cowan, 2001). The accessibility of these data in the electronic patient record for all clinicians makes the situation even worse as not only specialists access the data but all clinicians (Haux, 2006). Undoubtedly, the effective management of such visual data, including x-ray images, computed tomography (CT) scans, magnetic resonance imaging (MRI), and non-radiology imaging sources, is imperative to maximize the utility of the collected images and to maximize the accuracy and efficiency of the health services. Images convey more information to the medical researcher or practitioner than can be abstracted in a brief report or annotation. Critical diagnostic and interventional decisions are based on the digital images acquired from a particular patient and often assessed in comparison with historical cases that are individually or institutionally accumulated such as in the Casimage1system (Rosset, et al., 2004).

An effective medical image retrieval system can not only play a crucial role for clinical care, but it can also contribute greatly to medical research by allowing scientists to identify images of relevant cases more accurately and efficiently. It can prove to be extremely beneficial for medical students, as well as for patients and the general public to identify information relevant to their health related search. However, only a few studies (Müller, et al., 2006) have looked at the user-behavior of image retrieval system users. This study noted that many clinicians store reference images from past cases, often on their personal computers, and also that most often images are searched for by pathology and not anatomic region or modality that are often implemented for image classification.

Complete Chapter List

Search this Book: