Swarm-Based Nature-Inspired Metaheuristics for Neural Network Optimization

Introduction

Swarm based Optimization is a strategy that considers several agents collectively working to intelligently achieve a goal in the most optimal manner. Nature inspired techniques might include the consideration of these agents to be a flock of birds, a school of fish, a swarm of bees etc. Metaheuristic approaches are framed to form an analogy between the nature and computational systems and hence implement a relevant behavior as a paradigm to perform the required task. Since 1990, several nature inspired meta-heuristic techniques have been proposed. There exist several applications areas where the metaheuristics like swarm based or evolutionary optimization algorithms play vital role. Several NP-hard optimization problems like Traveling Salesman Problem, Quadratic Assignment Problem, Graph problems are also solved using nature inspired techniques.

Classification is the task of assigning an object to a pre-defined class or group (Duda, 1973). Classifier can be considered as a mapping function of the form ), where f(.) is the classifier that maps the object X_i to class Y_i based on parameters that are related attributes of object X_i. Classification is widely used in business, science, industry, and medicine and addresses many real world problems such as bankrupt prediction, credit scoring, medical diagnosis, handwritten character recognition, and speech recognition.

In traditional statistical classifiers, classification decision depends on posterior probability which is derived based on the assumptions on underlying probability model. Prior knowledge required on data properties and model capabilities limits the scope of statistical classifiers in many real world problems. Emergence of Neural Network, a non-linear model that models real world complex problems provides solution for the conventional statistical classifiers. The advantage of neural networks lies in the following theoretical aspects. First, neural networks are data driven self-adaptive methods in that they can adjust themselves to the data without any explicit specification of functional or distributional form for the underlying model. Second, they are universal functional approximators in that neural networks can approximate any function with arbitrary accuracy (Cybenko 1989; Hornik, 1991; Hornik et al, 1989).

Since any classification procedure seeks a functional relationship between the group membership and the attributes of the object, accurate identification of this underlying function is doubtlessly important. Third, neural networks are nonlinear models, which makes them flexible in modelling real world complex relationships. Finally, neural networks are able to estimate the posterior probabilities, which provide the basis for establishing classification rule and performing statistical analysis.Neural networks are considered as data driven self-adaptive methods and universal functional approximators that estimates posterior probability with arbitrary accuracy.

To improve the performance of the neural networks by optimizing its parameters, the authors (Werbos, 1990, 1994; Williams et al.,1986; Gupta & Sexton, 1999; Wilamowski, 2002) have suggested that back propagation using gradient descent methods is the most widely used neural network training method to optimize the neural network parameters in supervised learning strategy. In recent years, many improved learning algorithms have been developed that aim to remove the shortcomings of the gradient descent based systems.