Palmprint And Dorsal Hand Vein Multi-Modal Biometric Fusion Using Deep Learning

Palmprint And Dorsal Hand Vein Multi-Modal Biometric Fusion Using Deep Learning

Norah Abdullah Al-johani (Computer Science Department, Faculty of Computing and Information Technology, King Abdulaziz University, Jeddah, Saudi Arabia) and Lamiaa A. Elrefaei (Computer Science Department, Faculty of Computing and Information Technology, King Abdulaziz University, Jeddah, Saudi Arabia & Electrical Engineering Department, Faculty of Engineering at Shoubra, Benha University, Cairo, Egypt)
DOI: 10.4018/IJAIML.2020070102


Advancements in biometrics have attained relatively high recognition rates. However, the need for a biometric system that is reliable, robust, and convenient remains. Systems that use palmprints (PP) for verification have a number of benefits including stable line features, reduced distortion and simple self-positioning. Dorsal hand veins (DHVs) are distinctive for every person, such that even identical twins have different DHVs. DHVs appear to maintain stability over time. In the past, different features algorithms were used to implement palmprint (PP) and dorsal hand vein (DHV) systems. Previous systems relied on handcrafted algorithms. The advancements of deep learning (DL) in the features learned by the convolutional neural network (CNN) has led to its application in PP and DHV recognition systems. In this article, a multimodal biometric system based on PP and DHV using (VGG16, VGG19 and AlexNet) CNN models is proposed. The proposed system is uses two approaches: feature level fusion (FLF) and Score level fusion (SLF). In the first approach, the features from PP and DHV are extracted with CNN models. These extracted features are then fused using serial or parallel fusion and used to train error-correcting output codes (ECOC) with a support vector machine (SVM) for classification. In the second approach, the fusion at score level is done with sum, max, and product methods by applying two strategies: Transfer learning that uses CNN models for features extraction and classification for PP and DHV, then score level fusion. For the second strategy, features are extracted with CNN models for PP and DHV and used to train ECOC with SVM for classification, then score level fusion. The system was tested using two DHV databases and one PP database. The multimodal system is tested two times by repeating PP database for each DHV database. The system achieved very high accuracy rate.
Article Preview

1. Introduction

Biometric systems have gained popularity for authentication purposes due to the dire need for protecting personal identity. These types of systems are deemed the best in terms of security (Orság & Drahanský, 2003). Their advancement in technology allows for unique and effective identification in automated systems; which is a great replacement to conventional approaches such as passwords. Users prefer biometrics, since they provide better security, as opposed to using password protection (Haghighat, Zonouz, & Abdel-Mottaleb, 2015).

Biometric systems rely either on physical (e.g., fingerprint, facial features, palm) or behavioral features (e.g., voice, movements, handwriting) (Bolle & Pankanti, 1998).

The palm print, fingerprint, hand vein, and ear canal are four biometric modalities that possess each of the following seven biometric characteristics: universality (i.e., it must be present in everyone), uniqueness (i.e., it must be distinct), permanence (i.e., it must be consistent over long periods of time), measurability (i.e., it must be measurable), performance (i.e., its efficiency in correct identification), acceptability (i.e., people need to be inclined to provide the feature), and circumvention (i.e., the ease to which it can be duplicated or imitated) (Bolle & Pankanti, 1998).

Palmprint recognition, which is the process of using PPs in the verification or identification process, has been the focus of numerous researches. Researchers have found that using the palm for identification purposes is more advantageous than using fingerprints or the iris (Jia, Hu, Lei, Zhao, & Gui, 2013; Ding, Zhuang, & Wang, 2005). Such advantages include stable line features, decreased distortion, that it is easier to manually position, and its ability to reach a higher recognition rate quicker (Jia, Hu, Lei, Zhao, & Gui, 2013). Hand veins are gaining popularity among recognition systems since their patterns possess the following four biometric characteristics: universality, uniqueness, permanence, and circumvention. The vein patterns are hard to copy, or forge given that they are beneath the skin and are generally not visible. The intricate hand-vascular patterns provide an effective feature set for identification (Sanchit, Ramalho, Correia, & Soares, 2011).

Biometrics are applied in several areas, the most prevalent of which are the following areas: logical/physical access control, tracking time/attendance, law enforcement, and surveillance (Orság & Drahanský, 2003). As shown in Figure 1, some applications that use the DHV and PP for recognition are ATM machines, bank transactions, computers, and entry systems (e.g., home, school, hospital, airport).

Figure 1.

Example technologies that use dorsal hand vein and palmprint patterns


Several features from previous researches were hand crafted to handle occlusion, scale variation and lighting. Designing hand crafted features usually entails identifying the right compromise between accuracy and computational efficiency. The use of deep learning (DL) allows for complex networks to be created using convolutional neural networks (CNNs). The deeper layers within the network serve as feature extractors (LeCun, Bengio, & Hinton, 2015). This implies that DL is able to obtain a set of features just by observing the input images (Bora, Chowdhury, Mahanta, Kundu, & Das, 2016), which may have been pre-processed with pyramidal method (Han, Lei, & Chen, 2016). This method aims at identifying numerous representation levels so the semantics of the data may be represented by higher-level features. This subsequently may result in increased robustness to intraclass variance (Chan, et al., 2015). While feature extraction in CNNs requires DL, hand crafted features are designed in advance by humans for extracting a specific set of select characteristics; as shown in Figure 2.

Complete Article List

Search this Journal:
Open Access Articles: Forthcoming
Volume 11: 2 Issues (2021): Forthcoming, Available for Pre-Order
Volume 10: 2 Issues (2020)
Volume 9: 2 Issues (2019)
View Complete Journal Contents Listing