Particle Swarm Optimization: A Tutorial

Introduction

Swarm optimization techniques are recent techniques used to solve many optimization problems. They are employed in image processing (Mostafa et al., 2015; Ali et al., 2016), machine learning (Yamany et al., 2015a; Tharwat et al.,2015c; Ibrahim & Tharwat, 2014; Tharwat et al., 2015b, Tharwat et al., 2016a, Tharwat et al., 2016e), power electronics (Yoshida et al., 2000), and numerical problems (Vesterstrom & Thomsen, 2004), and mechanical problems (dos Santos Coelho, 2010). The optimization can be defined as a mechanism through which the maximum or minimum value of a given function or process can be found. The optimization is usually used in different fields such as economics, physics, and engineering where the main aim is to achieve maximum production, efficiency, or some other measure. In addition, there are many scientific, social, and commercial problems which have various parameters which if they have been adjusted can produce a more desirable outcome. Generally, the optimization could be used to achieve maximization or minimization where the maximization of a given function f is equivalent to the minimization of this function opposite, -f (Van Den Bergh, 2006; Gaber et al., 2016, Tharwat et al., 2016e, f).

In the past years, a wide range of optimization techniques was proposed to solve such optimization problems. These techniques generally make use of two main kinds of learning techniques in the literature: stochastic or random versus deterministic. First, in deterministic techniques, the algorithm results in the same accuracy if the algorithm uses the same initial values. On the contrary, stochastic methods employ stochastic optimization algorithms and it achieved different results due to randomness. There are two main problems with the stochastic algorithms due to randomness. First, the goodness of the obtained solution depends mainly on the initial random solution. Hence, different initialization may lead to different solutions. Second, there exist greater probabilities of local optima problem. This problem is common in almost all optimization algorithms (Ho et al., 2007; Tharwat A., 2016b, Tharwat A., 2016c).

The optimization algorithms have two main phases: exploration and exploitation. In the exploration phase, the goal is to explore the search space to search for the optimal or near optimal solutions. Hence, exploration may lead to new search regions that may contain better solutions. In the exploitation phase, the aim is to search locally around the current good solution(s) this is so-called exploitation. The optimization algorithms should make a balance between a random selection and greedy selection to bias the search towards better solutions, i.e. exploitation, while exploring the search space to find different solutions, i.e. exploration (Yamany et al., 2015a, 2015b; Tharwat et al., 2015c, 2015a, Tharwat et al., 2016d).

One of the widely used swarm-based optimization techniques is the Particle Swarm Optimization (PSO). PSO has a small number of parameters which control the movements of the particles inside the search space. PSO has been used in many applications such as machine learning (Subasi, 2013), image processing (Maitra & Chatterjee, 2008), and power electronics (Miyatake et al., 2011). In general, PSO algorithm lends itself strongly to exploitation phase which may lead to local optima problem. Hence, in some cases, PSO fails to find the global optimal solution.