Computer-aided foliage image retrieval systems have the potential to dramatically speed up the process of plant species identification. Despite previous research, this problem remains challenging due to the large intra-class variability and inter-class similarity of leaves. This is particularly true when a large number of species are involved. In this chapter, the authors present a shape-based approach, the inner-distance shape context, as a robust and reliable solution. The authors show that this approach naturally captures part structures and is appropriate to the shape of leaves. Furthermore, they show that this approach can be easily extended to include texture information arising from the veins of leaves. They also describe a real electronic field guide system that uses our approach. The effectiveness of the proposed method is demonstrated in experiments on two leaf databases involving more than 100 species and 1,000 leaves.
TopIntroduction
Plant species identification is critical to the discovery of new plant species, as well as in monitoring changing patterns of species distribution due to development and climate change. However, biologists are currently hampered by the shortage of expert taxonomists, and the time consuming nature of species identification even for trained botanists. Computer-aided foliage identification has the potential to speed up expert identification and improve the accuracy with which non-experts can identify plants. While recent advances in user interface hardware and software make such a system potentially affordable and available for use in the field, a reliable and efficient computer vision recognition algorithm is needed to allow users to access such a system with a simple, general interface. In this chapter we will describe our recent work using computer vision techniques for this task.
Due to the reasons we have mentioned, foliage image retrieval has recently started attracting research efforts in computer vision and related areas (Agarwal, et al. (2006), Mokhtarian & Abbasi 2004, Weiss & Ray 2005, Im, Hishida, & Kunii 1998, Saitoh & Kaneko 2000, Soderkvist 2001, Yahiaoui, Herve, & Boujemaa 2005). Leaf images are very challenging for retrieval tasks due to their high inter-class similarity and large intra-class deformations. In addition, occlusion and self-folding often damage leaf shape. Furthermore, some species have very similar shape but different texture, which therefore makes the combination of shape and texture desirable. In summary, the challenges mainly come from several reasons:
- •
The between class similarity is great (see the first row in Figure. 1).
- •
Self occlusion happens for some species, especially for composite leaves (see the second row in Figure. 1).
- •
Some species have large intra class deformations. For example, composite leaves often have large articulations (see the second row in Figure. 1).
- •
In practice, leaves are often damaged due to folding, erosion, etc. (see the third row in Figure. 1).
- •
Usually the hundreds, if not thousands of species are present in a region. So that we can begin to address problems of this scale, one of the databases in our test contains leaves from about 100 species.
Figure 1.
Example of challenging leaves. First row: Three leaves from three different species (from the Swedish leaf database). Second row: Self occlusions due to overlapping leaflets and deformation of composite leaves. The left two leaves come from the same species; so do the right two leaves. Third row: damaged leaves.
The shapes of leaves are one of the key features used in their identification, and are also relatively easy to determine automatically from images. This makes them especially useful in species identification. Variation in leaf shape also provides an interesting test domain for general work on shape comparison (Felzenszwalb 2005, Mokhtarian & Abbasi 2004, Sclaroff & Liu 2001).
Part structure plays a very important role in classifying complex shapes in both human vision and computer vision (Biederman 1987, Hoffman & Richards 1985, Kimia, Tannenbaum, & Zucker 1995, etc). However, capturing part structure is not a trivial task, especially considering articulations, which are nonlinear transformations between shapes. To make things worse, sometimes shapes can have ambiguous parts (e.g. [4]). Unlike many previous methods that deal with part structure explicitly, we propose an implicit approach to this task.