Boosting of Deep Convolutional Architectures for Arabic Handwriting Recognition

Boosting of Deep Convolutional Architectures for Arabic Handwriting Recognition

Mohamed Elleuch (National School of Computer Science (ENSI), University of Manouba, Manouba, Tunisia) and Monji Kherallah (Faculty of Sciences, University of Sfax, Sfax, Tunisia)
DOI: 10.4018/IJMDEM.2019100102

Abstract

In recent years, deep learning (DL) based systems have become very popular for constructing hierarchical representations from unlabeled data. Moreover, DL approaches have been shown to exceed foregoing state of the art machine learning models in various areas, by pattern recognition being one of the more important cases. This paper applies Convolutional Deep Belief Networks (CDBN) to textual image data containing Arabic handwritten script (AHS) and evaluated it on two different databases characterized by the low/high-dimension property. In addition to the benefits provided by deep networks, the system is protected against over-fitting. Experimentally, the authors demonstrated that the extracted features are effective for handwritten character recognition and show very good performance comparable to the state of the art on handwritten text recognition. Yet using Dropout, the proposed CDBN architectures achieved a promising accuracy rates of 91.55% and 98.86% when applied to IFN/ENIT and HACDB databases, respectively.
Article Preview
Top

1. Introduction

The techniques related to the information processing know currently a very active development in connection with data processing. It presents an increasingly important potential in the field of the human-computer interaction. In addition, machine simulation of human reading has been the subject of intensive research for the last years. The recognition of the writing comes under the wider domain which is the pattern recognition. It seeks to develop a system which is nearest to the human capacity of reading.

Automatic classification / recognition of handwritten script is a challenging task due to the great variability of handwriting stylus and its cursive nature. In the last five decades, Arabic Handwritten Script (AHS) recognition has been regarded as the topic of intensive research thanks to its broad applicability in several engineering technological areas (Mota & Scott, 2014). Many surveys have been being carried out to recognize Arabic handwritten characters exploiting unsupervised feature learning and hand-designed features (Porwal, Shi, & Setlur, 2013; Elleuch, Hani, & Kherallah, 2017).

Until the last decade, feature extraction represents one of the most interesting steps in the computer vision field and especially in handwritten recognition, but developing appropriate characteristics/features from the image depicts difficult and intricate task. It needs both knowledge and experience (know-how) in the area of feature extraction methods (Mel Frequency Cepstral Coefficient (MFCC) features in speech area, Gabor/ Scale-invariant Feature Transform (SIFT) / Histogram of Oriented Gradients (HOG) features in computer vision, etc.). The selection and the quality of these hand-designed features identify the effectiveness of frameworks used for recognition and classification (Hidden Markov Model, Multi-layer Perceptron, K-means, Support Vector Machine, etc.). Employing raw data or unlabeled data in training developed handwriting systems has been targeted by a lot of researchers because it constitutes the easiest manner to treat with huge data.

As a consequence, the first challenge of this study is to develop automatic features extraction system richer than the one obtained using heuristics signal processing based on knowledge domain. This approach is based on the concept of learning in a deep way a representation of Arabic handwriting from the image signal. To do this, the uses of supervised and unsupervised learning methods have shown some potential. Learning such representations can be applied in many handwriting recognition tasks.

In recent years, new research concerning the training strategies of Deep Networks (Deep Neural Network, Deep Belief Network, etc.) have allowed to improve performance in the machine learning area and specifically in pattern recognition problems. Deep Networks (DN) learn a hierarchy of non-linear feature detectors that better capture complex data models (Bengio, Lamblin, Popovici, & Larochelle, 2007). Figure 1a illustrates a conventional structure of off-line text recognition systems consists essentially of four important stages: pretreatment, segmentation, feature extraction, and training/recognition stage. As for Figure 1b, it showed a deep machine learning approach exploiting convolutional deep belief network (CDBN) for extraction features stage.

Compared to shallow learning, the pros of deep learning (DL) is that deep structures can be constructing to learn more abstract details. However, the great number of parameters provided may also conduct to other problem: over-fitting. The major contributions of this paper are to take advantage of the DL approach to resolve the Arabic handwritten text recognition issue. To achieve this goal, we study the potential benefits of our proposed hybrid CDBN/SVM architecture (Elleuch, Tagougui, & Kherallah, 2015); this model took the CDBN as an automatic feature extractor and it allowed SVM to be the output predictor. On the other hand, to boost the performance of CDBN/SVM model, regularization methods can add in defense of over-fitting as Dropout and Dropconnect techniques. In this paper, we mainly study the effect of the dropout technique on convolutional DBN (CDBN).

Complete Article List

Search this Journal:
Reset
Open Access Articles: Forthcoming
Volume 11: 4 Issues (2020): 1 Released, 3 Forthcoming
Volume 10: 4 Issues (2019)
Volume 9: 4 Issues (2018)
Volume 8: 4 Issues (2017)
Volume 7: 4 Issues (2016)
Volume 6: 4 Issues (2015)
Volume 5: 4 Issues (2014)
Volume 4: 4 Issues (2013)
Volume 3: 4 Issues (2012)
Volume 2: 4 Issues (2011)
Volume 1: 4 Issues (2010)
View Complete Journal Contents Listing