Article Preview
Top1. Introduction
The techniques related to the information processing know currently a very active development in connection with data processing. It presents an increasingly important potential in the field of the human-computer interaction. In addition, machine simulation of human reading has been the subject of intensive research for the last years. The recognition of the writing comes under the wider domain which is the pattern recognition. It seeks to develop a system which is nearest to the human capacity of reading.
Automatic classification / recognition of handwritten script is a challenging task due to the great variability of handwriting stylus and its cursive nature. In the last five decades, Arabic Handwritten Script (AHS) recognition has been regarded as the topic of intensive research thanks to its broad applicability in several engineering technological areas (Mota & Scott, 2014). Many surveys have been being carried out to recognize Arabic handwritten characters exploiting unsupervised feature learning and hand-designed features (Porwal, Shi, & Setlur, 2013; Elleuch, Hani, & Kherallah, 2017).
Until the last decade, feature extraction represents one of the most interesting steps in the computer vision field and especially in handwritten recognition, but developing appropriate characteristics/features from the image depicts difficult and intricate task. It needs both knowledge and experience (know-how) in the area of feature extraction methods (Mel Frequency Cepstral Coefficient (MFCC) features in speech area, Gabor/ Scale-invariant Feature Transform (SIFT) / Histogram of Oriented Gradients (HOG) features in computer vision, etc.). The selection and the quality of these hand-designed features identify the effectiveness of frameworks used for recognition and classification (Hidden Markov Model, Multi-layer Perceptron, K-means, Support Vector Machine, etc.). Employing raw data or unlabeled data in training developed handwriting systems has been targeted by a lot of researchers because it constitutes the easiest manner to treat with huge data.
As a consequence, the first challenge of this study is to develop automatic features extraction system richer than the one obtained using heuristics signal processing based on knowledge domain. This approach is based on the concept of learning in a deep way a representation of Arabic handwriting from the image signal. To do this, the uses of supervised and unsupervised learning methods have shown some potential. Learning such representations can be applied in many handwriting recognition tasks.
In recent years, new research concerning the training strategies of Deep Networks (Deep Neural Network, Deep Belief Network, etc.) have allowed to improve performance in the machine learning area and specifically in pattern recognition problems. Deep Networks (DN) learn a hierarchy of non-linear feature detectors that better capture complex data models (Bengio, Lamblin, Popovici, & Larochelle, 2007). Figure 1a illustrates a conventional structure of off-line text recognition systems consists essentially of four important stages: pretreatment, segmentation, feature extraction, and training/recognition stage. As for Figure 1b, it showed a deep machine learning approach exploiting convolutional deep belief network (CDBN) for extraction features stage.
Compared to shallow learning, the pros of deep learning (DL) is that deep structures can be constructing to learn more abstract details. However, the great number of parameters provided may also conduct to other problem: over-fitting. The major contributions of this paper are to take advantage of the DL approach to resolve the Arabic handwritten text recognition issue. To achieve this goal, we study the potential benefits of our proposed hybrid CDBN/SVM architecture (Elleuch, Tagougui, & Kherallah, 2015); this model took the CDBN as an automatic feature extractor and it allowed SVM to be the output predictor. On the other hand, to boost the performance of CDBN/SVM model, regularization methods can add in defense of over-fitting as Dropout and Dropconnect techniques. In this paper, we mainly study the effect of the dropout technique on convolutional DBN (CDBN).