Object classification problem is essential in many applications nowadays. Human can easily classify objects in unconstrained environments easily. Classical classification techniques were far away from human performance. Thus, researchers try to mimic the human visual system till they reached the deep neural networks. This chapter gives a review and analysis in the field of the deep convolutional neural network usage in object classification under constrained and unconstrained environment. The chapter gives a brief review on the classical techniques of object classification and the development of bio-inspired computational models from neuroscience till the creation of deep neural networks. A review is given on the constrained environment issues: the hardware computing resources and memory, the object appearance and background, and the training and processing time. Datasets that are used to test the performance are analyzed according to the images environmental conditions, besides the dataset biasing is discussed.
TopIntroduction
Object classification is an increasingly important topic in computer vision. It is defined as the classification of an object existing lonely in an image. For multiple objects images, object localization or detection is required first. The objects to be classified in an image can be in a constrained environment or unconstrained environment. Constrains of the environment could be the background; plain or real, hardware resources required for computation, processing time in real time applications and object appearance. Object classification is used in wide range of applications; such, mobile applications (Xiong, Kim, & Hedau, 2019), tablets and other smart appliances, sorting and quality control in food industry (Naranjo-Torres, et al., 2020).
Classical machine learning techniques were popular, for many decades, for object classification. The popular technique was to extract features from the input images, to form a feature vector, which is used in learning the patterns from the input images, and training a model. Along with being time-consuming, feature extraction depends on the image type. Thus, the accuracy of the system depends on designing the right set of features, which may need an expert for some types of tasks. Humans can identify thousands of object categories in strewn scenes, despite the variations in the object posture, changes in its illumination and occlusions.
Visual cortex models appeared as a solution for bridging the gap between the performance of the human vision and the computational models. Cortical models are bio-inspired systems based on the available neuroscience research to imitate the human visual performance. Neuroscience experiments provide sufficient information to understand how the visual cortex works. Many frameworks appeared as a result of advances and cooperation between several fields like brain science, cognitive science, and computer vision.
Deep learning techniques, especially deep convolutional neural networks (DCNN), are also bio-inspired from the hierarchical way of the visual cortex to approach the performance of the human visual system. DCNN is powerful in extracting features from the input data directly. Identifying the object using the extracted features in unconstrained environment is done by the convolution and pooling layers (Haq, et al., 2020). Some developing applications begin with constrains such as the image background like in the case of LeafSnap mobile application (Kang & Oh, 2018) that classifies the types of the trees from their leaves. The old version of the application requires that the leaf to be pictured on a white paper, then the application is developed now to picture in unconstrained environment. The availability of different software platforms for building DCNNs such as: Caffe, Theano and TensorFlow facilities the use of deep neural network (DNN) in many applications.
The issue of discussing the object classification using DCNN in constrained and unconstrained environment on general is not discussed explicitly in the published papers. This issue is discussed on specific object type such as fish species classification (Khalifa, Taha, & Hassanien, 2018), and gender classification (Huang et al., 2017).
The objective of this chapter is to highlight the superiority of the DCNN over classical techniques in object classification that came from mimicking the human visual system, analysis of the different architectures, challenges present of constrains of the object classification using DCNN and their presented solutions. Constrains include the computing and storage resources, object appearance, object background, and training and processing time. Also, the datasets types and their biasing effect are discussed. Finally, the future trends in this area are given.