Particle Swarm Optimization: A Tutorial

Particle Swarm Optimization: A Tutorial

Alaa Tharwat (Suez Canal University, Egypt & Cairo University, Egypt & Scientific Research Group in Egypt (SRGE), Egypt), Tarek Gaber (University of Manchester, UK and Suez Canal University, Egypt), Aboul Ella Hassanien (Cairo University, Egypt) and Basem E. Elnaghi (Suez Canal University, Egypt)
Copyright: © 2017 |Pages: 22
DOI: 10.4018/978-1-5225-2229-4.ch026
OnDemand PDF Download:
List Price: $37.50


Optimization algorithms are necessary to solve many problems such as parameter tuning. Particle Swarm Optimization (PSO) is one of these optimization algorithms. The aim of PSO is to search for the optimal solution in the search space. This paper highlights the basic background needed to understand and implement the PSO algorithm. This paper starts with basic definitions of the PSO algorithm and how the particles are moved in the search space to find the optimal or near optimal solution. Moreover, a numerical example is illustrated to show how the particles are moved in a convex optimization problem. Another numerical example is illustrated to show how the PSO trapped in a local minima problem. Two experiments are conducted to show how the PSO searches for the optimal parameters in one-dimensional and two-dimensional spaces to solve machine learning problems.
Chapter Preview


Swarm optimization techniques are recent techniques used to solve many optimization problems. They are employed in image processing (Mostafa et al., 2015; Ali et al., 2016), machine learning (Yamany et al., 2015a; Tharwat et al.,2015c; Ibrahim & Tharwat, 2014; Tharwat et al., 2015b, Tharwat et al., 2016a, Tharwat et al., 2016e), power electronics (Yoshida et al., 2000), and numerical problems (Vesterstrom & Thomsen, 2004), and mechanical problems (dos Santos Coelho, 2010). The optimization can be defined as a mechanism through which the maximum or minimum value of a given function or process can be found. The optimization is usually used in different fields such as economics, physics, and engineering where the main aim is to achieve maximum production, efficiency, or some other measure. In addition, there are many scientific, social, and commercial problems which have various parameters which if they have been adjusted can produce a more desirable outcome. Generally, the optimization could be used to achieve maximization or minimization where the maximization of a given function f is equivalent to the minimization of this function opposite, -f (Van Den Bergh, 2006; Gaber et al., 2016, Tharwat et al., 2016e, f).

In the past years, a wide range of optimization techniques was proposed to solve such optimization problems. These techniques generally make use of two main kinds of learning techniques in the literature: stochastic or random versus deterministic. First, in deterministic techniques, the algorithm results in the same accuracy if the algorithm uses the same initial values. On the contrary, stochastic methods employ stochastic optimization algorithms and it achieved different results due to randomness. There are two main problems with the stochastic algorithms due to randomness. First, the goodness of the obtained solution depends mainly on the initial random solution. Hence, different initialization may lead to different solutions. Second, there exist greater probabilities of local optima problem. This problem is common in almost all optimization algorithms (Ho et al., 2007; Tharwat A., 2016b, Tharwat A., 2016c).

The optimization algorithms have two main phases: exploration and exploitation. In the exploration phase, the goal is to explore the search space to search for the optimal or near optimal solutions. Hence, exploration may lead to new search regions that may contain better solutions. In the exploitation phase, the aim is to search locally around the current good solution(s) this is so-called exploitation. The optimization algorithms should make a balance between a random selection and greedy selection to bias the search towards better solutions, i.e. exploitation, while exploring the search space to find different solutions, i.e. exploration (Yamany et al., 2015a, 2015b; Tharwat et al., 2015c, 2015a, Tharwat et al., 2016d).

One of the widely used swarm-based optimization techniques is the Particle Swarm Optimization (PSO). PSO has a small number of parameters which control the movements of the particles inside the search space. PSO has been used in many applications such as machine learning (Subasi, 2013), image processing (Maitra & Chatterjee, 2008), and power electronics (Miyatake et al., 2011). In general, PSO algorithm lends itself strongly to exploitation phase which may lead to local optima problem. Hence, in some cases, PSO fails to find the global optimal solution.

Complete Chapter List

Search this Book: