Segmentation-Free Word Spotting in Handwritten Documents Using Scale Space Co-HoG Feature Descriptors

Introduction

There is a huge amount of information in libraries and institutions all over the world in the form of books, documents and in other conventional methods. We need to be digitized in order to preserve and for efficient searching and browsing of information for different applications. In order to create digital libraries, thousands of digitized documents have to be transcribed (George, N, et al., 2006). Optical Character Recognition (OCR) is first used to transcribe documents where image-based documents are converted into ASCII format through automatic recognition. The automatic recognition by OCR system achieves best performance for modern high-quality printed documents with simple layouts and known fonts. The performance of OCR is very poor for handwritten text due to various challenges posed by handwritten text such as unconstrained writing styles, open vocabulary and paper degradation such as stains, ancient fonts, and faded ink.

To overcome the aforementioned limitations of OCR, the Document Image Analysis (DIA) community has developed a technique called as word spotting. Word spotting is a technique for recognition and retrieval of words in any form of document images. Word spotting can be defined as process aimed at locating and retrieving a particular word from a document image collection. The main objective of word spotting systems is to propose methods that show high accuracy, high speed and work on any language with minimum preprocessing steps. A word spotting method requires a collection of documents/document corpus and an input element is a query word. The output of word spotting method is spotting and retrieval of documents or sub images that are similar to the query word. Figure 1 illustrates a general architecture of word spotting method where the whole procedure is divided in an offline and an online phase. In the offline stage, a set of features are extracted from either word images, or text lines or whole document pages which are then represented by feature vectors. In the online phase, a user formulates a query either by selecting an actual example from the collection or by typing an ASCII text word. Then matching process is applied to these representations in order to obtain a similarity score which yields a ranking list of results according to their similarity with the query.

Figure 1.

General architecture of word spotting (Courtesy: Giotis et al., 2017)

Challenges Posed By Word Spotting Problem

The word spotting in handwritten documents is not completely solved due to various challenges posed by handwritten documents and the challenges involved in handwritten documents are:

• •

  Either historical or modern Handwritten documents suffer from variability in writing style, not only for different authors but also for documents of the same writer.

• •

  The handwritten words may be skewed, characters may be slanted, non-text content such as symbols may be present and letters may be broken or connected in a cursive manner

• •

  Degradations such as missing data, non-stationary noise due to illumination changes, low contrast, and warping effects, which directly affect the segmentation and feature extraction stages of a word spotting method.

Applications Of Word Spotting

There are a variety of applications of word spotting in handwritten documents such as:

• •

  Searching and browsing historical handwritten documents collections written by a single or several authors. Retrieval of documents with a given word in company/organization files. Retrieval of keywords in hospital care reports.

• •

  Helps human transcribers in identifying words in degraded documents

• •

  Sorting of mails based on significant words like urgent, cancellation and complain

• •

  Identification of figures and their corresponding captions. Word spotting in geographical maps.