Deep learning (DL) is a special type of machine learning that attains great potency and flexibility by learning to represent input raw data as a nested hierarchy of essences and representations. DL consists of more layers than conventional machine learning that permit higher levels of abstractions and improved prediction from data. More abstract representations computed in terms of less abstract ones. The goal of this chapter is to present an intensive survey of existing literature on DL techniques over the last years especially in the medical imaging analysis field. All these techniques and algorithms have their points of interest and constraints. Thus, analysis of various techniques and transformations, submitted prior in writing, for plan and utilization of DL methods from medical image analysis prospective will be discussed. The authors provide future research directions in DL area and set trends and identify challenges in the medical imaging field. Furthermore, as quantity of medicinal application demands increase, an extended study and investigation in DL area becomes very significant.
TopIntroduction
Deep learning (Bengio et al., 2012), is a developed direction in general data analysis, and has been entitled one of the year 2013 ten-breakthrough technologies (Greenspan, Hayit, & Bram van Ginneken, 2016; Wang 2016). It is a significant subfield of Artificial Intelligence (AI) that linked various topics like Machine Learning, Neural Networks, and Classification. Figure 1 represents the connections between the diverse AI fields. This discipline has advanced significantly over the years due to the huge works of scientists like Andrew Ng, Geoff Hinton, Yann LeCun, Adam Gibson, and Andrej Karpathy (Hinton et al., 2015; LeCun et al., 2015; Ng, 2014; Ng et al., 2015).
For decades, attaining and extracting the well-founded and distinguishable features is always a pivotal step to complete the task of image recognition and computer vision. Engineering expertise is required in the domain to model a feature extractor that mutated raw data (i.e., such as pixels in the image) into a suitable internal representation or feature vector from which the classical machine learning subsystem could complete its work either classification, prediction, or clustering of the input. An interest and research venture in feature extraction is a key concern in a lot of researches in various applications (Elnemr et al., 2016). Representation learning is a set of procedures that provides a machine to be fed with raw data and to automatically determine the portrayals needed for recognition or grouping (LeCun et al., 2015). Deep learning techniques are representation learning techniques with multiple levels of portrayal, prevailed by composing simple but non-linear modules that each transmuted the representation at one level (starting with the raw input) into a representation at a higher, slightly more abstract level (LeCun et al., 2015). With the composition of enough such transmutations, very complex functions can be learned. Figure 2 demonstrates stream outlines for the disparate parts of an AI system relate to each other within different AI disciplines. Shaded boxes indicate components that are able to learn from data (Bengio et al., 2012; Goodfellow et al., 2016).
A deep learning framework is a multilayer heap of simple modules, all (or most) of which are liable to learning, and many of which calculate non-linear input–yield mappings. Each module in the load transforms its input to elevate both the selectivity and the invariance of the representation. With multiple non-linear layers, a framework can implement extremely sophisticated functions of its inputs that are altogether sensitive to minute details and unkind to large unrelated variations such as the background, pose, lighting and surrounding objects. The hidden layers of a deep multilayer neural network learn to represent the network’s inputs in a way that makes it easy to predict the target outputs.
Deep learning is known of sundry hundred years as modern neural networks with multiple hidden layers (Hinton, 2014; Ng et al., 2015; Deeplearning4j Development Team, 2016). Back in the nineteen eightieth, the researchers were excited about the prospect of using neural networks with many layers of hidden neurons, but it was hard to get networks to learn well using standard techniques (i.e. Backpropagation “especially with the problem of gradient descent over the loss function” more details are discussed in sub-section 2.1). Neural nets fell out of favor as other machine learning approaches with comparable better results stemming from statistical machine learning in 1990s and 2000s (Alpaydın, 2012). In the previous couple of years the deep learning has become more useful and gained the attention as the development of the methods of training networks (i.e. good weight regularization/dropout), amount of available training data has increased, its models have grown in size over time as computer hardware (i.e. compute power of the parallel hardware graphical processing units (GPUs), computer clusters) and software infrastructure has improved (Le, 2014; Schmidhuber, 2015).
Figure 1. The relationship between the different AI fields
Figure 2. Shows flow charts for the disparate parts of an AI system relate to each other within disparate AI fields; shaded boxes indicate components that are able to learn from data.