Abstract
Imagery must be adequately processed and adapted to be used in mobile environments. This contribution shows that this can be reached by a rigorous combination of all stages of the image communication pipeline to the task at hand. To achieve this, the modern imaging standard JPEG2000 serves as a foundation for compression and streaming. Based on the concept of regions of interest (ROI) and levels of detail (LOD), powerful generic strategies for content handling are proposed. The task is represented by one of three common problems in mobile environments—image browsing, viewer guidance, and content exchange. Besides new ideas to accomplish these tasks on application level, appropriate strategies to combine the respective visual representation with compression and streaming are introduced. In addition to a much better content representation, the achieved results show that the consumed resources can be strongly decreased by the proposed new concepts for mobile image communication using JPEG2000.
TopIntroduction
The enthusiasm for mobile computing is still unbroken. With the ability to access information every time and everywhere, mobility allows for completely novel and out breaking solutions, applications, and services. Due to their form factors and the application environment, however, mobile devices are still restricted by low computing power and bandwidth. As imagery is one of the most important but also most resource-demanding media within multimedia it must be adequately processed and adapted to be used in such environments (Rosenbaum, Tominski, & Schumann, 2006). How to achieve this by efficient JPEG2000-based image communication strategies designed to solve a task at hand will be the focus of this contribution. To be able to provide the big picture on the topic, of main interest are the features and performance gains provided by the strategies rather than a detailed technical description of their components. However, all underlying technology will be introduced and carefully referenced.
A meaningful image communication strategy requires the integration of the two disparate research fields, “multimedia” and “communication,” and consists of the stages compression, streaming, and representation (cf., Figure 1). Although numerous, often quite sophisticated techniques for each stage have been proposed, an arbitrary combination of single solutions does usually not lead to an appropriate result. A high-performing system can only be guaranteed if all involved techniques can be tightly coupled (Rauschenbach, 1999; Rosenbaum & Schumann, 2005). Founded on an appropriate visual representation at the mobile client, this should lead to an image communication strategy that ensures that only data contributing to the current representation is handled and transmitted. All ideas proposed in this contribution are founded on this principle, significantly reducing the required system resources at all communication stages. Although similar ideas have already been proposed, they are either based on proprietary solutions (Owen et al., 2001) or discuss a specific task only (Deshpande & Zeng, 2001; Ortiz, Ruiz, & Garcia, 2004). The ideas proposed in this publication are fully compliant to the international standard JPEG2000 (ISO, 2002), which allows for an easy migration of the introduced strategies into existing systems, and founded on a generic description of current demands by regions of interest (RoI) and levels of detail (LoD) for a broad applicability. Thereby, the adopted principle of a progressive refinement avoids the redundant data transfer in case demands change during interaction.
Figure 1. The image communication pipeline
Although the applicability of the proposed JPEG2000-based compression and streaming technology extends far beyond the discussed tasks, this publication will focus on three common problems in mobile environments: image browsing, viewer guidance, and content exchange. Beside the proposal of new ideas for appropriate visual representations and the underlying data handling, of major interest is their coupling with compression and streaming to high-performance communication systems and the resulting decrease of the consumed resources.
Key Terms in this Chapter
Image Browsing: Stepwise navigation within a 2D image in order to explore its contents. It is founded on an appropriate user interface providing—besides the visual representation of the image contents—also different means for an interactive determination of the user’s current interest. A single technique can be classified by regions of interest and level of detail.
Image Communication: Umbrella term to describe the handling and processing of image data in remote environments. It consists of the main stages: image compression, image streaming, and visual representation.
JPEG2000: Modern image compression standard published by ISO (ISO/IEC 15444-1:2000). It is founded on the discrete wavelet transform (DWT) and embedded block coding with optimized truncation (EBCOT). The standard provides—beside a superb performance for lossy as well as lossless compression—numerous features beneficial for image streaming.
Image Streaming: Continuous transmission of the image data from a sender with simultaneous interpretation and rendering of the parts already received by the viewer. The transmission can often be flexibly adapted to current demands. To allow for independent handling and decoding, the encoded image data is usually organized in individual packets.
Region of Interest (RoI): Selected spatial subset of samples within a dataset identified for a particular purpose. With regard to image compression and image streaming: region within an image, which is of certain importance. The importance can be expressed by the concept of level of detail. Related terms: area of interest (AoI), window of interest (WoI).
Level of Detail (LoD): Concept that expresses the support of multiple versions of a visual representation in order to reduce resource consumption. A certain version is selected dependent on current demands. The LoD of digital imagery usually consists of the attributes resolution, accuracy, and color.
Image compression: Reduction of redundancy within the image in order to be able to handle the data in a more compact and efficient form. The original data can be either fully reconstructed (lossless) or restored in such a way so as the loss of some data is not visible (lossy). To achieve this principles of data compression are applied to digital imagery.
Viewer Guidance: Visual highlighting that intuitively directs the attention of a viewer to pre-defined parts of large information spaces. To achieve this for digital imagery, the representation of the contents must be adapted in such a way so as to allow for an accentuation of important regions with clear visual distinction to unimportant regions.
Content Exchange: Replacement of content of a destination image by content from one or multiple sources images. This task is easy to accomplish in spatial domain by substituting single pixel values, but requires much resources if the image content is only available in compressed representation.