Abstract
Artificial neural networks (ANNs) are computational models, loosely inspired by biological neural networks, consisting of interconnected groups of artificial neurons which process information using a connectionist approach. ANNs are widely applied to problems like pattern recognition, classification, and time series analysis. The success of an ANN application usually requires a high number of experiments. Moreover, several parameters of an ANN can affect the accuracy of solutions. A particular type of evolving system, namely neuro-genetic systems, have become a very important research topic in ANN design. They make up the so-called Evolutionary Artificial Neural Networks (EANNs), i.e., biologicallyinspired computational models that use evolutionary algorithms (EAs) in conjunction with ANNs. Evolutionary algorithms and state-of-the-art design of EANN were introduced first in the milestone survey by Xin Yao (1999), and, more recently, by Abraham (2004), by Cantu-Paz and Kamath (2005), and then by Castellani (2006). The aim of this article is to present the main evolutionary techniques used to optimize the ANN design, providing a description of the topics related to neural network design and corresponding issues, and then, some of the most recent developments of EANNs found in the literature. Finally a brief summary is given, with a few concluding remarks.
TopEvolutionary Artificial Neural Networks
There are several approaches to evolve ANNs, that usually fall into two broad categories: problem-independent and problem-dependent representation of EAs. The former are based on a general representation, independent of the type and structure of the ANN sought for, and require the definition of an encoding scheme suitable for Genetic Algorithms (GAs). They can include mapping between ANNs and binary representation, taking care of decoders or repair algorithms, but this task is not usually easy.
Key Terms in this Chapter
Artificial Neural Networks: Models inspired by the working of the brain, considered as a combination of neurons and synaptic connections, which are capable of transmitting data through multiple layers, giving a system able to solve different problems like pattern recognition and classification.
Search Space: Set of all possible situations of the problem that we want to solve could ever be in.
Evolutionary Computation: In computer science it is a subfield of artificial intelligence (more particularly computational intelligence) involving combinatorial optimization problems. Evolutionary computation defines the quite young field of the study of computational systems based on the idea of natural evolution and adaptation.
Evolutionary Artificial Neural Networks: Special class of artificial neural networks in which evolution is another fundamental form of adaptation in addition to learning. They are represented by biologically inspired computational models that use evolutionary algorithms in conjunction with neural networks to solve problems.
Error Backpropagation: Essentially a search procedure that attempts to minimize a whole network error function such as the sum of the squared error of the network output over a set of training input/output pairs.
Adaptive System: System able to adapt its behavior according to changes in its environment or in parts of the system itself.
Backpropagation algorithm: A supervised learning technique used for training ANNs. It is based on a set of recursive formulas for computing the gradient vector of the error function, that can be used in a first-order method like gradient descent.
Multi-Layer Perceptrons (MLPs): Class of neural networks that consists of a feed-forward fully connected network with an input layer of neurons, one or more hidden layers and an output layer. The output value is obtained through the sequence of activation functions defined in each hidden layer. Usually, in this kind of network, the supervised learning process is the backpropagation algorithm
Evolutionary Algorithms: Algorithms based on models that consider ‘artificial’ or ‘simulated’ genetic evolution of individuals in a defined environment. They are a broad class of stochastic optimization algorithms, inspired by biology and in particular by those biological processes that allow populations of organisms to adapt to their surrounding environment: genetic inheritance and survival of the fittest.