Image Analysis and Understanding Based on Information Theoretical Region Merging Approaches for Segmentation and Cooperative Fusion

Introduction

So easy, yet so difficult. Image analysis and understanding is an immediate and almost innate task for humans, but a challenging and surprisingly difficult task for artificial vision systems and computers (Levine, 1985). The lack of a detailed and complete understanding of the neurological and cognitive processes involved in human vision makes not only high-level processes but also the lowest and fundamental stages of visual processing a stimulating and widely open problem. Nevertheless, psychological studies (Sternberg, 2003; Bruce, 1996) agree on some key principles, such as the importance of perceptual grouping as a preliminary stage into the visual processing chain to scene recognition and understanding.

Perceptual grouping (Lowe, 1985) refers to the human visual ability to extract significant image relations from lower-level primitive image features, without any knowledge of the image content, and to group them to obtain meaningful higher-level structures. Research in perceptual grouping was started in 1920's by Gestalt psychologists, whose goal was to discover the underlying principle that would unify the various grouping phenomena of human perception. Gestalt psychologists observed the tendency of the human visual system to perceive configurational wholes, with rules that govern the uniformity of psychological grouping for perception and recognition, as opposed to recognition by analysis of discrete primitive image features (Geisler, 2000).

Inspired by this perceptual foundation (Desolneux, 2004), the image processing community has tirelessly worked for more than four decades into the image segmentation field. Image segmentation refers to the process of partitioning a digital image into a disjoint set of connected pixels, known as regions. The benefit of this region-based image representation or image partition representation is twofold. From a semantic point of view, image segmentation is a first level of abstraction providing an image representation closer to the object representation than the set of pixels (Ballard, 1982). Ideally, we expect objects to be formed by a single region or by a union of regions (adjacent or not). From a practical point of view, a region-based representation of the image reduces the number of elementary primitives and allows a more robust estimation of parameters and descriptors, especially when compared with a direct pixel-based representation, and facilitates later processing, storage, and retrieval. In other words, segmentation simplifies the image into some primitives more semantically meaningful and easier to analyze (Forsyth, 2003). However, it is a critical step since the results of the segmentation will have considerable influence over all the subsequent processes of image analysis (Zhang, 1995), such as object representation and description, feature measurements, and even the following higher level tasks, such as object classification or scene interpretation.

Apart from the intrinsic difficulty of discovering and emulating the perceptual grouping principles and mechanisms of human vision, the image segmentation problem is an ill-posed problem in a double sense¹. The first type of ill-posedness refers to the fact that, in a large number of cases, a unique solution to the image segmentation problem does not exist. Instead of a single optimal partition, it is possible to find different region-based explanations of an image at different level of analysis or detail (Bertero, 1988). Hence, the desired segmentation result depends on the image itself and on the particular application or purpose of the analysis to be performed. For instance, consider the example shown in Figure 1. If we are interested in counting the number of tigers in the image or in detecting an animal presence in the scene, the partition in Figure 1(b) would be appropriate. However, if our purpose is to recognize the animal or to count the number of stripes of the tiger, we would find the partition in Figure 1(c) a more valuable segmentation.

Figure 1.

Non-uniqueness of the solution to the image segmentation problem: counting tigers or stripes? (a) Original image extracted from the Berkeley Segmentation Dataset² (Martin, 2001). (b) Image partition into 2 regions. (c) Image partition into 100 regions. Both partitions were computed using one of the techniques presented in this chapter.