Displaying a large image in a small screen of a handheld gadget is a challenging task. Simple down-scaling of the image may reduce some objects too small to be perceptible. This gives rise to content-aware retargeting of the image. Important contents are allotted more screen space as compared to relatively less important contents of the image. Various types of content-aware image retargeting approaches have been proposed in a span of just over a decade. Another challenging area is to estimate importance of importance of the contents. Lot of researches has been carried out in this direction too to identify the important contents in the context of image retargeting. Equally important aspect is evaluation of these retargeting methods. This article contains a brief survey of related research in all of these aspects.
TopIntroduction
Twenty first century has witnessed, so far, breakthrough technological advances in digital handheld gadgets. Usage of tablet PCs and mobile phones has increased significantly due to their availability in affordable prices. These devices have good quality displays to provide the users a pleasant experience of viewing personal photos, music videos, and even movies. Moreover, availability of high-resolution cameras with these devices and revolutionary ways of connecting with people through social media have added a new dimension in how people express themselves through images and videos.
As these images and videos are viewed using variety of gadgets having wide range of display resolutions, adaptation of these images/videos to each individual display is essential for good viewing experience. If an image is captured with a high resolution camera and the image is viewed in a smaller display, then certainly some mechanism is needed to fit the image with the display size. This has opened a new area of research in image processing, namely image retargeting for fitting the image in the display size.
Simple down-scaling an image to fit into the smaller display is the rudimentary solution. This approach shrinks every object in the image equally. But down-scaling makes a small object even smaller and, in certain cases, viewers may find it difficult to recognize those objects. Researchers have found a solution to this problem based on importance of each individual content in the image. Idea, here, is that the important contents are allotted more space in the target area as compared to the unimportant contents. A wide spectrum of content-aware solutions is available in literature, like cropping, seam carving, warping, growth and shrinkage of rectangles, etc.
Researchers, hence, have faced another challenge in identifying important contents in an image, which is the first step of content-aware image retargeting as depicted in Figure 1. In this context, visual attention/saliency models have played a crucial role to signify importance of the contents in the image. Apart from visual saliency models, use of various energy estimation functions (gradient-based or entropy-based) can be found in literature. Often, face and/or text recognition techniques have been used to identify human faces and texts to capture semantically important contents.
Figure 1.
Components of content-aware image retargeting
This article attempts to compile a significant amount of these research efforts. It provides a comprehensive view of the works which have been carried out in this field. At first, this article focuses on how importance of the contents in an image is measured. Then, various retargeting approaches have been summarized. Key aspects of each category of these approaches have been highlighted. Evaluating a method is also an integral component of proposing any new method. Therefore, this article discusses various evaluating strategies of image retargeting approaches. Though majority of the researchers resorted to subjective evaluation, few recent researches demonstrate the objective evaluation of these methods. Thus, this article provides a good overview of the current status in the field of content-aware image retargeting. It is to be noted that a comparative study of various image retargeting approaches can also be found in (Rubinstein et al, 2010).
TopEstimating Importance
Content-aware image retargeting approaches suggest discriminative treatments to the contents of an image based on their importance. While reducing the size of the entire image to fit it into a smaller display, these approaches allot a bigger space in the display for the important contents as compared to the space allotted to the unimportant contents. Thus, important contents maintain their recognizability. It helps the image to correctly convey its main theme and also leads to pleasant viewing experience for the viewer.
Visual attention models have been extensively used to find out important contents in an image. Extensive experiments, reported in (Elazary & Itti, 2008), also indicate that saliency leads to interesting objects in a scene.