Optimization-Driven Kernel and Deep Convolutional Neural Network for Multi-View Face Video Super Resolution

Optimization-Driven Kernel and Deep Convolutional Neural Network for Multi-View Face Video Super Resolution

Amar B. Deshmukh (Vignan University, Guntur, India) and N. Usha Rani (Vignan University, Guntur, India)
Copyright: © 2020 |Pages: 19
DOI: 10.4018/IJDCF.2020070106
Article PDF Download
Open access articles are freely available for download


One of the major challenges faced by video surveillance is recognition from low-resolution videos or person identification. Image enhancement methods play a significant role in enhancing the resolution of the video. This article introduces a technique for face super resolution based on a deep convolutional neural network (Deep CNN). At first, the video frames are extracted from the input video and the face detection is performed using the Viola-Jones algorithm. The detected face image and the scaling factors are fed into the Fractional-Grey Wolf Optimizer (FGWO)-based kernel weighted regression model and the proposed Deep CNN separately. Finally, the results obtained from both the techniques are integrated using a fuzzy logic system, offering a face image with enhanced resolution. Experimentation is carried out using the UCSD face video dataset, and the effectiveness of the proposed Deep CNN is checked depending on the block size and the upscaling factor values and is evaluated to be the best when compared to other existing techniques with an improved SDME value of 80.888.
Article Preview

1. Introduction

Video surveillance is considered as an interesting field of research due to the increasing demand in video surveillance for determining human behaviors and identifying objects. The visual surveillance aims not only to install cameras in the place of human eyes, but also, for achieving complete surveillance attention (Savitha & Ramesh, 2018). Video surveillance is broadly applied in various areas, such as inventory control for retail stores, equipment monitoring for factories, monitoring for intersections, and traffic control security operations for campus, and security surveillance for houses (Porikli et al., 2013; Wang et al., 2018). Tracking and object detection in video surveillance systems mainly depend on background subtraction. Nowadays, video surveillance system uses video compression technology to store images from a total number of cameras to save devices, such as discs, video tapes (Chavda & Dhamecha, 2017). Also, intelligent surveillance systems are utilized in several areas including protection and the security networks. In various areas, video surveillance face video images are provided for identifying humans. Moreover, the user interest is so far away from the camera, in which the face resolution in the picture is too small for providing information. Due to constrained imaging conditions, it is difficult to obtain high-definition face images. From the generated results, the face images that are captured lose so many meticulous facial features, which are recognized by the users (Qu et al., 2014).

One of the major techniques to improve the resolution of images is super-resolution (SR). SR indicates a class of digital image processing approach, which improve the resolution of an imaging system (Huang et al., 2015). SR combines various low resolution (LR) to generate high resolution (HR) image with best optical resolution. The high frequency content is improved and the degradations produced by the image acquisition are minimized. The LR images are somewhat different, so they consist of dissimilar information about the same scene. SR methods are classified into frequency domain approach, statistical approaches, non-uniform interpolation approach in the spatial domain, and other approaches (Yang et al., 2018). Interpolation approaches based on a single image are sometimes considered as closely related to SR. These techniques lead to a bigger picture size but failed to consider any additional information. In contrast to SR, the high frequency content cannot be recovered. Hence, image interpolation methods are not considered as SR techniques (Dong et al., 2016).

The HR videos are generated from original LR videos and this process is termed as video SR (Huang et al., 2015). Video SR has received considerable attention from both industry and academia. Several HR video devices are developed for storing, producing and transmitting HR videos (Yang et al., 2018). The main aim of SR is to recover HR video or image from its LR finding direct applications that ranges from medical imaging into satellite imaging and also the facilitating tasks like face recognition. Reconstructing HR data from LR input is moreover an extremely ill-posed issue and additional constraint is required for solving (Caballero et al., 2017). The inter-frame temporal relation and the intra-frame spatial relation are the two kinds of relations, which are used for video SR (Liu et al., 2017). In the last two decades, wide variety of SR techniques has been analyzed. The sparse representation (Yang et al., 2010) and self-similar based approaches are used in SR method to reconstruct HR images. Sparse representation techniques are established to evaluate important defect in image processing, often on super resolution and denoising, in which the aim is not for obtaining a compact high-fidelity representation of the observed image, but also for extracting semantic information (Barzigar et al., 2012). One of the representative external example-based SR methods is Sparse-Coding-based (SC) technique (Dong et al., 2016). Depending on cascade of Convolutional Neural Networks (CNNs), Laplacian Pyramid Super-Resolution Network (LapSNR) provides an LR image as input to predict the sub-band residuals in a coarse-to-fine fashion (Lai et al., 2017).

Complete Article List

Search this Journal:
Volume 14: 1 Issue (2022): Forthcoming, Available for Pre-Order
Volume 13: 6 Issues (2021): 5 Released, 1 Forthcoming
Volume 12: 4 Issues (2020)
Volume 11: 4 Issues (2019)
Volume 10: 4 Issues (2018)
Volume 9: 4 Issues (2017)
Volume 8: 4 Issues (2016)
Volume 7: 4 Issues (2015)
Volume 6: 4 Issues (2014)
Volume 5: 4 Issues (2013)
Volume 4: 4 Issues (2012)
Volume 3: 4 Issues (2011)
Volume 2: 4 Issues (2010)
Volume 1: 4 Issues (2009)
View Complete Journal Contents Listing