In this chapter, the authors evaluate several basic image processing and advanced image pattern recognition techniques for automatically analyzing bioimages, with the aim of designing different ensembles of canonical and deep classifiers for breast lesion classification in ultrasound images. The analysis starts from convolutional neural networks (CNNs) in a square matrix that is used to feed other CNNs. The novel ensemble, named TakhisisNet, is the combination by sum rule of the whole set of the modified CNNs and the original one. Moreover, the performance of the system is further improved by combining it with some handcrafted features. Experimental results obtained on the well-known OASBUD breast cancer dataset (i.e., the open access series of breast ultrasonic data) and on a large set of bioimage classification problems show that TakhisisNet obtains very valuable results and outperforms other approaches previously tested in the same datasets.
TopIntroduction
Breast cancer is one of the main causes of death for women living in western countries (Wild & Stewart, 2014). The diagnosis and the detection of a breast lesion relies upon ultrasound images. This technique is safe, low cost and let radiologists discriminate between benignant and malignant lesions with a very high accuracy. However, the need of an expert doctor to read ultrasound images increases the cost of screening and makes the diagnosis process operator-dependent (Byra, 2018).
In order to support radiologists, the recent innovation in the analysis of digital images led researchers to propose automatic classifiers with the purpose of discriminating between benignant and malignant tumors. E.g. automatic systems based on manually engineered features extracted from ultrasound images and fed into machine learning classifiers like Support Vector Machines (SVMs) are proposed in (Cheng, Shan, Ju, Guo, & Zhang, 2010). However, the recent rise of deep learning lead to the first attempts to use convolutional neural networks to recognize and classify malignant lesions in medical images (Saikia, Bora, Mahanta, & Das, 2019).
Convolutional neural networks (CNNs) are a class of neural networks designed to perform image classification, image segmentation and object recognition (Krizhevsky, Sutskever, & Hinton, 2012). One of the first successful attempt to use CNNs for image classification can be found in (Krizhevsky et al., 2012), where the authors designed a CNN able to outperform any previous classifier on the ImageNet challenge 2012. Since then, every winner of the ImageNet challenge was a CNN. Nowadays, modern CNNs obtain superhuman accuracies on ImageNet (He, Zhang, Ren, & Sun, 2015).
CNNs have already been used on several medical datasets reaching very high performance. In (Pereira, Pinto, Alves, & Silva, 2016) authors used deep CNNs with very small kernels to perform brain tumor segmentation from MRI images; in (Esteva et al., 2017) it is shown that a single CNN could detect keratinocyte carcinomas and malignant melanomas with the same accuracy as expert dermatologists; in (Chi et al., 2017) a fine-tuned version of GoogleNet (Szegedy et al., 2015) is proposed trained to classify thyroid nodules from ultrasound images; the work in (Lakhani & Sundaram, 2017) presented a system to detect pulmonary tuberculosis from radiographies using AlexNet (Krizhevsky et al., 2012) and GoogleNet (Szegedy et al., 2015). In (Byra, 2018) authors proposed a deep learning based method to classify breast cancer from ultrasound images. They used the OASBUD (Open Access Series of Breast Ultrasonic Dataset), a publicly available dataset containing 52 benignant lesions a 48 malignant lesions (Piotrzkowska-Wróblewska, Dobruch-Sobczak, & Byra Michałand Nowicki, 2017). Their approach consisted in combining Fisher Linear Discriminant Analysis (Welling, 2005) and neural style transfer (Gatys, Ecker, & Bethge, 2016).
In this chapter we propose a novel method for the digital classification of breast cancer based on CNNs. The architecture of CNN consists of several convolutional and pooling layers stacked at the beginning of the network. These layers are usually followed by one or more fully connected layers. While the first layers of a CNN can be interpreted as trained feature extractors, the fully connected layers can be thought as a classifier. In particular, the last fully connected layer, after a softmax normalization, returns a probability distribution of the predicted label over the set of the possible classes. Our idea consists in replacing a pooling layer with other CNNs architectures. To be more precise, we add a reshape layer that takes the output of a pooling layer and randomly puts its entries in a square matrix. If the number of features is not a perfect square, the matrix can be padded with some zeros. After this reshape layer, we feed other CNNs with the new matrix. The CNNs on top are trained end-to-end with the first CNN and each of them return its own set of probability distributions. The final ensemble, resulting from the sum rule of such probability distribution is named TakhisisNet, since it is a multi-head convolutional neural network.
Finally, to improve the performance of the system we have experimented the combination by sum rule of TakhisisNet with a set of Support Vector Machine (SVM) trained on handcrafted features.