This concept appears in several machine learning models. In neural network models, the learning rate is the fraction of the gradient vector that the weight vectors descend. The learning rate affects the speed and convergence of the training process. In gradient boosting, the learning rate decides how quickly or slowly the next tree corrects the error of the current tree. A learning rate that is too small might lead to a long training time, while a learning rate that is too large might not lead to convergence.
Published in Chapter:
Comparing Deep Neural Networks and Gradient Boosting for Pneumonia Detection Using Chest X-Rays
Son Nguyen (Bryant University, USA), Matthew Quinn (Harvard University, USA), Alan Olinsky (Bryant University, USA), and John Quinn (Bryant University, USA)
Copyright: © 2022
|Pages: 22
DOI: 10.4018/978-1-7998-8455-2.ch003
Abstract
In recent years, with the development of computational power and the explosion of data available for analysis, deep neural networks, particularly convolutional neural networks, have emerged as one of the default models for image classification, outperforming most of the classical machine learning models in this task. On the other hand, gradient boosting, a classical model, has been widely used for tabular structure data and leading data competitions, such as those from Kaggle. In this study, the authors compare the performance of deep neural networks with gradient boosting models for detecting pneumonia using chest x-rays. The authors implement several popular architectures of deep neural networks, such as Resnet50, InceptionV3, Xception, and MobileNetV3, and variants of a gradient boosting model. The authors then evaluate these two classes of models in terms of prediction accuracy. The computation in this study is done using cloud computing services offered by Google Colab Pro.