How to Train Multilayer Perceptrons Efficiently With Large Data Sets

How to Train Multilayer Perceptrons Efficiently With Large Data Sets

Hyeyoung Park (Brain Science Institute, Japan)
Copyright: © 2002 |Pages: 21
DOI: 10.4018/978-1-930708-26-6.ch010
OnDemand PDF Download:


Feed forward neural networks or multilayer perceptrons have been successfully applied to a number of difficult and diverse applications by using the gradient descent learning method known as the error backpropagation algorithm. However, it is known that the backpropagation method is extremely slow in many cases mainly due to plateaus. In data mining, the data set is usually large and the slow learning speed of neural networks is a critical defect. In this chapter, we present an efficient on-line learning method called adaptive natural gradient learning. It can solve the plateau problems, and can be successfully applied to the learning associated with large data sets. We compare the presented method with various popular learning algorithms with the aim of improving the learning speed and discuss briefly the merits and defects of each method so that one can get some guidance as to the choice of the proper method for a given application. In addition, we also give a number of technical tips, which can be easily implemented with low computational cost and can sometimes make a remarkable improvement in the learning speed.

Complete Chapter List

Search this Book:
Table of Contents
Ruhul Sarker, Hussein A. Abbass, Charles S. Newton
Chapter 1
R. Sarker, H. Abbass, C. Newton
The terms Data Mining (DM) and Knowledge Discovery in Databases (KDD) have been used interchangeably in practice. Strictly speaking, KDD is the... Sample PDF
Introducing Data Mining and Knowledge Discovery
Chapter 2
A. M. Bagirov, A. M. Rubinov, J. Yearwood
The feature selection problem involves the selection of a subset of features that will be sufficient for the determination of structures or clusters... Sample PDF
A Heuristic Algorithm for Feature Selection Based on Optimization Techniques
Chapter 3
Kai Ming Ting
This chapter reports results obtained from a series of studies on costsensitive classification using decision trees, boosting algorithms, and... Sample PDF
Cost-Sensitive Classification Using Decision Trees, Boosting and MetaCost
Chapter 4
Agapito Ledezma, Ricardo Aler, Daniel Borrajo
Currently, the combination of several classifiers is one of the most active fields within inductive learning. Examples of such techniques are... Sample PDF
Heuristic Search-Based Stacking of Classifiers
Chapter 5
Craig M. Howard
The overall size of software packages has grown considerably over recent years. Modular programming, object-oriented design and the use of static... Sample PDF
Designing Component-Based Heuristic Search Engines for Knowledge Discovery
Chapter 6
Jose Ruiz-Shulcloper, Guillermo Sanchez-Diaz, Mongi A. Abidi
In this chapter, we expose the possibilities of the Logical Combinatorial Pattern Recognition (LCPR) tools for Clustering Large and Very Large Mixed... Sample PDF
Clustering Mixed Incomplete Data
Chapter 7
Bayesian Learning  (pages 108-121)
Paula Macrossan, Kerrie Mengersen
Learning from the Bayesian perspective can be described simply as the modification of opinion based on experience. This is in contrast to the... Sample PDF
Bayesian Learning
Chapter 8
Paul D. Scott
This chapter addresses the question of how to decide how large a sample is necessary in order to apply a particular data mining procedure to a given... Sample PDF
How Size Matters: The Role of Sampling in Data Mining
Chapter 9
The Gamma Test  (pages 142-167)
Antonia J. Jones, Dafydd Evans, Steve Margetts, Peter J. Durrant
The Gamma Test is a non-linear modelling analysis tool that allows us to quantify the extent to which a numerical input/output data set can be... Sample PDF
The Gamma Test
Chapter 10
Denny Meyer, Andrew Balemi, Chris Wearing
Neural networks are commonly used for prediction and classification when data sets are large. They have a big advantage over conventional... Sample PDF
Neural Networks - Their Use and Abuse for Small Data Sets
Chapter 11
Hyeyoung Park
Feed forward neural networks or multilayer perceptrons have been successfully applied to a number of difficult and diverse applications by using the... Sample PDF
How to Train Multilayer Perceptrons Efficiently With Large Data Sets
Chapter 12
Kevin E. Voges, Nigel K.L. Pope, Mark R. Brown
Cluster analysis is a common market segmentation technique, usually using k-means clustering. Techniques based on developments in computational... Sample PDF
Cluster Analysis of Marketing Data Examining On-line Shopping Orientation: A Comparison of K-Means and Rough Clustering Approaches
Chapter 13
Susan E. George
This chapter presents a survey of medical data mining focusing upon the use of heuristic techniques. We observe that medical mining has some unique... Sample PDF
Heuristics in Medical Data Mining
Chapter 14
A. de Carvalho, A. P. Braga, S. O. Rezende, E. Martineli, T. Ludermir
In the last few years, a large number of companies are starting to realize the value of their databases. These databases, which usually cover... Sample PDF
Understanding Credit Card User's Behaviour: A Data Mining Approach
Chapter 15
Alina Lazar
The goal of this research is to investigate and develop heuristic tools in order to extract meaningful knowledge from archeological large-scale data... Sample PDF
Heuristic Knowledge Discovery for Archaeological Data Using Genetic Algorithms and Rough Sets
About the Authors