The Mathematics of Perception: Statistical Models and Algorithms for Image Annotation and Retrieval

Abstract

This chapter summarizes the work on Mathematics of Perception performed by my research team between 2000 and 2005. To support personalization, a search engine must comprehend users’ query concepts (or perceptions), which are subjective and complicated to model. Traditionally, such query-concept comprehension has been performed through a process called “relevance feedback.” Our work formulates relevance feedback as a machine-learning problem when used with a small, biased training dataset. The problem arises because traditional machine learning algorithms cannot effectively learn a target concept when the training dataset is small and biased. My team has pioneered in developing a method of query-concept learning as the learning of a binary classifier to separate what a user wants from what she or he does not want, sorted out in a projected space. We have developed and published several algorithms to reduce data dimensions, to maximize the usefulness of selected training instances, to conduct learning on unbalanced datasets, to accurately account for perceptual similarity, to conduct indexing and learning in a non-metric, high-dimensional space, and to integrate perceptual features with keywords and contextual information. The technology of mathematics of perception encompasses an array of algorithms, and has been licensed by major companies for solving their image annotation, retrieval, and filtering problems.