Article Preview
TopIntroduction
Several initiatives such as the Million Book Project and Google Print Library Project have already archived several million books in digital format, and it is believed that within a few years a significant fraction of world’s books will be online (Herwig, 2007). As Kevin Kelly recently noted, “the real magic will come in the second act, as each word in each book is cross-linked, clustered, cited, extracted, indexed, analyzed, annotated, remixed, reassembled and woven deeper into the culture than ever before” (Kelly, 2006). While this quotation explicitly singles out text, a similar argument can be made for images. Clearly the majority of the data gleaned from scanned books will be text, but there will also be tens of millions of pages of images. Many of these images will defy automation annotation for the foreseeable future; however a considerable fraction of the images may be amiable to automatic annotation by algorithms that can link the historical image with a modern contemporary, with its attendant meta tags (Agosti et al., 2007). As a concrete example, consider Figure 1.
Figure 1. A page from a scientific text published in 1849(D’Orbigny, 1849). The heavily stylized script is difficult to read even at full resolution, however we have independently confirmed that the three insects are (left to right) Zygaena filipendulae, Acherontia atropos and Syntomis phegea.
In this image the text annotations will surely defy even the state-of-the-art handwriting recognizers (Adamek et al., 2007), and humans, particularly those without experience in reading cursive script are also unlikely to be able to parse these words. Suppose that we segment out the individual insects and search for the most similar images on the web (for the moment, we will gloss over the technical details of how this is done). In fact we have done this, and discovered the image in Figure 2.
Figure 2. An image of Acherontia atropos, also known as the Death's-head Hawkmoth)
The image is uncannily like the query image, we can confidently assume it is the same (or closely related) species and therefore we can link the historical image to its modern counterpart to provide context and annotations to the digital archive. In this example the shape and color provided the necessary clues to the identity of unknown object. More generally, different sets of features may be useful depending on the application. For example, most Diatoms (eukaryotic algae) are colorless when viewed at the microscopic scale, but are often richly textured, as in Figure 3.
Figure 3. Left) Plate 5, fig. 44 of a Smith’s British Diatomaceae (1853), Right) An image of Triceratium favum, a type of Algae
Here (rotation invariant) shape and texture allow us to link this object to a modern image, and learn of the latest research on this intriguing life form.