Heuristic Approach Performances for Artificial Neural Networks Training

Heuristic Approach Performances for Artificial Neural Networks Training

Kerim Kürşat Çevik (Akdeniz University, Turkey)
DOI: 10.4018/978-1-7998-2742-9.ch019
OnDemand PDF Download:
No Current Special Offers


This chapter aimed to evaluate heuristic approach performances for artificial neural networks (ANN) training. For this purpose, software that can perform ANN training application was developed using four different algorithms. First of all, training system was developed via back propagation (BP) algorithm, which is the most commonly used method for ANN training in the literature. Then, in order to compare the performance of this method with the heuristic methods, software that performs ANN training with genetic algorithm (GA), particle swarm optimization (PSO), and artificial immunity (AI) methods were designed. These designed software programs were tested on the breast cancer dataset taken from UCI (University of California, Irvine) database. When the test results were evaluated, it was seen that the most important difference between heuristic algorithms and BP algorithm occurred during the training period. When the training-test durations and performance rates were examined, the optimal algorithm for ANN training was determined as GA.
Chapter Preview


Various methods have been used for training of ANNs. These methods are widely classified into two sets as supervised and unsupervised learning. In training ANNs with supervised learning, both the input data and the output data are provided. In unsupervised learning, only the input data are provided in training ANNs and it is expected to estimate outputs (Nabiyev, 2003). The Backpropagation training algorithm (or generalized delta rule) technique, which is a gradient-descent method, is one of the most popular training algorithms in the domain of neural networks (Rumelhart, Hinton, & Williams, 1988). The back propagation algorithm is a family of methods used to efficiently train artificial neural networks by following a gradient-based optimization algorithm using the chain rule (Kahramanli & Allahverdi, 2008). As ANNs generate complex error surfaces with multiple local minima, BP tends to converging into local minima rather than global minima (Gupta & Sexton, 1999; Valian, Mohanna, & Tavakoli, 2011). Many advanced learning algorithms have been proposed in recent years in order to overcome the shortcomings of gradient based techniques (Valian et al., 2011). These algorithms include direct optimization method using a polytope algorithm (Curry & Morgan, 1997), Evolutionary Algorithms (EA), a class of general search technique (Salchenberger, Cinar, & Lash, 1992) and genetic algorithm (GA) (Sexton, Dorsey, & Johnson, 1998). Other techniques, such as EA, have been applied to the ANN problem in the past (Cantú-Paz, 2003; Cotta, Alba, Sagarna, & Larrañaga, 2002) and they have tried to avoid the local minima in the error that usually occurs in complex problems.

Many researchers have preferred meta-heuristic optimization algorithm since conventional numerical methods have some computational drawbacks in solving complex optimization problems. In recent years, various meta-heuristic algorithms have been successfully applied to various engineering optimization problems. When compared to conventional numerical methods, EA has provided better solutions for most complicated real-world optimization problems (Valian et al., 2011). In meta-heuristic algorithms, many rules and randomness are combined to imitate natural phenomena. These phenomena include the biological evolutionary processes such as the Genetic Algorithm (GA) (Goldberg & Holland, 1988; Holland, 1992), Evolutionary Algorithm (EA) (Fogel, 1998), Artificial Immunity (Hofmeyr & Forrest, 2000) and Differential Evolution (DE) (Storn, 1996) animal behavior such as Ant Colony Algorithm (ACA) (Dorigo & Di Caro, 1999) and Particle Swarm Optimization (PSO) (Shi & Eberhart, 1999), human’s intuition such as Tabu Search Algorithm (Glover, 1977); and physical annealing processes, such as simulated annealing (SA) (Valian et al., 2011).

Complete Chapter List

Search this Book: