Particle swarm optimization with crossover: a review and empirical analysis

Abstract

Since its inception in 1995, many improvements to the original particle swarm optimization (PSO) algorithm have been developed. This paper reviews one class of such PSO variations, i.e. PSO algorithms that make use of crossover operators. The review is supplemented with a more extensive sensitivity analysis of the crossover PSO algorithms than provided in the original publications. Two adaptations of a parent-centric crossover PSO algorithm are provided, resulting in improvements with respect to solution accuracy compared to the original parent-centric PSO algorithms. The paper then provides an extensive empirical analysis on a large benchmark of minimization problems, with the objective to identify those crossover PSO algorithms that perform best with respect to accuracy, success rate, and efficiency.

Appendix: Benchmark functions and performance tables

This appendix provides detail on the functions included in the benchmark set and provides the detailed performance tables.

Table 15 summarizes the characteristics of these functions. A function, $f_l$, that is shifted, rotated, or shifted and rotated, is respectively referred to as $f_l^{Sh}, f_l^R$ and $f_l^{ShR}$, where l is the index of the function.

Table 15 Characteristics of benchmark functions

Table 16 Comparison of solution accuracy wins and losses for crossover PSO algorithms (results are given as wins/losses/sum of wins and losses)

Table 17 Comparison of success rate wins and losses for crossover PSO algorithms (results are given as wins/losses/sum of wins and losses)

Table 18 Comparison of efficiency wins and losses for crossover PSO algorithms (results are given as wins/losses/sum of wins and losses)

A function, $f_l$, was shifted using

$$\begin{aligned} f_l^{Sh}({\mathbf x}) = f_l({\mathbf z}) + \beta \end{aligned}$$

where ${\mathbf z} = {\mathbf x} - \gamma$; $\gamma$ and $\beta$ are constants. Table 15 lists the values by which each function was shifted. Two approaches to rotation were implemented: Where an entry in the rotation column of Table 15 indicates “ortho”, the function $f_l$ was rotated by a randomly generated orthonormal rotation matrix. The second approach, indicated by “linear” rotates the function using a linear transformation matrix. For both approaches the condition number is given in the table in parentheses, and rotation was done using Salomon’s method Salomon (1996). A new rotation matrix was computed for each of the 50 independent runs of the algorithm. The rotated function, referred to as $f_l^R$, was computed by multiplying the decision vector ${\mathbf x}$ with the transpose of the rotation matrix.

Some functions were both shifted and rotated, with the resulting function referred to as $f_l^{ShR}$. Noisy functions were generated by multiplying each decision variable, $x_j$, by zero-mean noise sampled from a Gaussian distribution with deviation of one. These functions are referred to as $f_l^N$. Functions that are shifted and noisy are referred to as $f_l^{ShN}$.

Note that $f_{27}$ is indicated as a non-separable function, even though it is separable near the optimum.

Rotations for functions $f_{26}$–$f_{37}$ and function $f_{15}^{ShRE}$ were by linear transformation matrix, with condition numbers that differ for each component function. Also the severity of shifts differ for the different component functions. For the detail of these parameters, the reader is referred to Suganthan et al. (2005). For these functions, the entry “yes” simply indicates if a transformation of the expanded or composition function was done. An entry of “CEC05” refers the reader to the definition of the function as in Suganthan et al. (2005).

The definitions of the benchmark functions used in this study is provided below together with detail on the domain of each function. Many of the functions below are equivalent to functions defined in the IEEE Congress on Evolutionary Computation (CEC) 2005 competition benchmark set Suganthan et al. (2005). Such equivalencies are indicated by giving the CEC 2005 benchmark number with the superscript CEC (Tables 16, 17, 18).

$f_1$,:

the absolute value function, defined as

$$\begin{aligned} f_1({\mathbf x})=\sum _{j=1}^{n_x}|x_i| \end{aligned}$$

(7)

with each $x_j \in [-100,100]$.

$f_2$,:

the ackley function, defined as

$$\begin{aligned} f_2({\mathbf x})=-20e^{-0.2\sqrt{\frac{1}{n}\sum _{j=1}^{n_x}x_j^2}} -e^{\frac{1}{n}\sum _{j=1}^{n_x}\cos (2\pi x_j)}+20+e \end{aligned}$$

(8)

with each $x_{j} \in [-32.768,32.768]$. Shifted, rotated, and rotated and shifted verions of the ackley function were used, respectively referred to as $f_2^{Sh}, f_2^R$ and $f_2^{ShR}$. Function $f_2^{ShR}$ is equivalent to function $f_8^{CEC}$.

$f_3$,:

the alpine function, defined as

$$\begin{aligned} f_3({\mathbf x})=\left( \prod _{j=1}^{n_x}\sin (x_j)\right) \sqrt{\prod _{j=1}^{n_x}x_j} \end{aligned}$$

(9)

with each $x_j \in [-10,10]$.

$f_4$,:

the egg holder function, defined as

$$\begin{aligned} f_4({\mathbf x})= & {} \sum _{j=1}^{n_x-1}\left( -(x_{j+1}+47) \sin \left( \sqrt{|x_{j+1}+x_j/2+47|}\right) \right. \nonumber \\&\left. +\sin \left( \sqrt{|x_j-(x_{j+1}+47)|}\right) (-x_j)\right) \end{aligned}$$

(10)

with each $x_j\in [-512,512]$.

$f_5$,:

the elliptic function, defined as

$$\begin{aligned} f_5({\mathbf x}) = \sum _{j=1}^{n_x}(10^6)^{\frac{j-1}{n_x-1}} \end{aligned}$$

(11)

with each $x_{j} \in [-100,100]$. Shifted, rotated, and rotated and shifted verions of the elliptic function were used, respectively referred to as $f_5^{Sh}, f_5^R$ and $f_5^{ShR}$. Function $f_5^{ShR}$ is equivalent to function $f_3^{CEC}$.

$f_6$,:

the griewank function, defined as

$$\begin{aligned} f_6({\mathbf x})=1+\frac{1}{4000}\sum _{j=1}^{n_x}x_j^2- \prod _{j=1}^{n_x}\cos \left( \frac{x_j}{\sqrt{j}}\right) \end{aligned}$$

(12)

with each $x_j \in [-600,600]$. Shifted, rotated, and rotated and shifted verions of the elliptic function were used, respectively referred to as $f_6^{Sh}, f_6^R$ and $f_6^{ShR}$. For function $f_6^{ShR}$, each $x_j \in [0,600]$, which means that the global minimum is outside of the bounds. Function $f_6^{ShR}$ is equivalent to function $f_7^{CEC}$. with bounds as given above.

$f_7$,:

the hyperellipsoid function, defined as

$$\begin{aligned} f_7({\mathbf x})=\sum _{j=1}^{n_x}jx_j^2 \end{aligned}$$

(13)

with each $x_j \in [-5.12,5.12]$.

$f_8$,:

the michalewicz function, defined as

$$\begin{aligned} f_8({\mathbf x}) = -\sum _{j=1}^{n_x}\sin (x_j)\left( \sin \left( \frac{jx_j^2}{\pi }\right) \right) ^{2m} \end{aligned}$$

(14)

with each $x_j \in [0,\pi ]$ and $m=10$.

$f_9$,:

the norwegian function, defined as

$$\begin{aligned} f_9({\mathbf x}) = \prod _{j=1}^{n_x}\left( \cos \left( \pi x_j^3\right) \left( \frac{99+x_j}{100}\right) \right) \end{aligned}$$

(15)

with each $x_j \in [-1.1,1.1]$.

$f_{10}$,:

the quadric function, defined as

$$\begin{aligned} f_{10}({\mathbf x})=\sum _{i=1}^{n_x}\left( \sum _{j=1}^ix_j\right) ^2 \end{aligned}$$

(16)

with each $x_j \in [-100,100]$.

$f_{11}$,:

the quartic function, defined as

$$\begin{aligned} f_{11}({\mathbf x}) = \sum _{j=1}^{n_x}jx_j^4 \end{aligned}$$

(17)

with each $x_j \in [-1.28,1.28]$. A noisy version of the quartic function, referred to as de jong’s f4 function, was generated as follows:

$$\begin{aligned} f_{11}^N({\mathbf x}) = \sum _{j=1}^{n_x}\left( jx_j^4+N(0,1)\right) \end{aligned}$$

(18)

The domain was the same as that of the quartic function.

$f_{12}$,:

the rastrigin function, defined as

$$\begin{aligned} f_{12}({\mathbf x}) = 10n_x + \sum _{j=1}^{n_x}\left( x_j^2-10\cos (2\pi x_j)\right) \end{aligned}$$

(19)

with $x_j \in [-5.12,5.12]$. Shifted, rotated, and rotated and shifted verions of the rastrigin function were used, respectively referred to as $f_{12}^{Sh}, f_{12}^R$ and $f_{12}^{ShR}$. Function $f_{12}^{ShR}$ is equivalent to function $f_{10}^{CEC}$.

$f_{13}$,:

the rosenbrock function, defined as

$$\begin{aligned} f_{13}({\mathbf x}) = \sum _{j=1}^{n_x-1}\left( 100\left( x_{j+1}-x_j^2\right) ^2+(x_j-1)^2\right) \end{aligned}$$

(20)

with $x_j \in [-30,30]$. Shifted, rotated, and rotated and shifted verions of the rosenbrock function were used, respectively referred to as $f_{13}^{Sh}, f_{13}^R$ and $f_{13}^{ShR}$. Function $f_{13}^{Sh}$’s domain was $[-100,100]$ to be equivalent to $f_6^{CEC}$. Function $f_{13}^R$’s domain was also set to $[-100,100]$. $f_{10}^{CEC}$.

$f_{14}$,:

the salomon function, defined as

$$\begin{aligned} f_{14}({\mathbf x}) = -\cos \left( 2\pi \sum _{j=1}^{n_x}x_j^2\right) +0.1\sqrt{\sum _{j=1}^{n_x}x_j^2} + 1 \end{aligned}$$

(21)

with $x_j \in [-100,100]$.

$f_{15}$,:

the schaffer 6 function, defined as

$$\begin{aligned} f_{15}({\mathbf x})=\sum _{j=1}^{n_x-1}\left( 0.5+\frac{sin^2\left( x_j^2+x_{j+1}^2\right) -0.5}{\left( 1+0.001\left( x_j^2+x_{j+1}^2\right) \right) ^2}\right) \end{aligned}$$

(22)

with each $x_j \in [-100,100]$. A shifted and rotated, expanded version of the schaffer 6 function was used, referred to as $f_{15}^{ShRE}$, which is equivalent to $f_{14}^{CEC}$.

$f_{16}$,:

the schwefel 1.2 function, defined as

$$\begin{aligned} f_{16}({\mathbf x}) = \sum _{j=1}^{n_x}\left( \sum _{k=1}^jx_k\right) ^2 \end{aligned}$$

(23)

with $x_j \in [-100,100]$. Shifted, rotated, and noisy shifted verions of the schwefel 1.2 function were used, respectively referred to as $f_{16}^{Sh}, f_{16}^R$ and $f_{16}^{ShN}$. Function $f_{16}^{Sh}$ is equivalent to $f_2^{CEC}$, and $f_{16}^{ShN}$ is equivalent to $f_4^{CEC}$, defined as

$$\begin{aligned} f_{16}({\mathbf x}) = \sum _{j=1}^{n_x}\left( \sum _{k=1}^jx_k\right) ^2(1+0.4|N(0,1)|) \end{aligned}$$

(24)

$f_{17}$,:

the schwefel 2.6 function, defined as

$$\begin{aligned} f_{17}({\mathbf x}) = \max _{j}\{|{\mathbf A}_j{\mathbf x}-{\mathbf B}_j|\} \end{aligned}$$

(25)

with $x_j \in [-100,100]$, $a_{ji} \in {\mathbf A}$ is uniformly sampled from $U(-500,500)$ such that $\text{ det }({\mathbf A}) \ne 0$, and each ${\mathbf B}_j = {\mathbf A}_j{\mathbf x}$. Function $f_{17}$ is equivalent to $f_5^{CEC}$. A shifted version of schwefel 2.6 was also implemented, referred to as $f_{17}^{Sh}$.

$f_{18}$,:

the schwefel 2.13 function, defined as

$$\begin{aligned} f_{18}({\mathbf x}) = \sum _{j=1}^{n_x}({\mathbf A}_j - {\mathbf B}_j({\mathbf x}))^2 \end{aligned}$$

(26)

with each $x_j \in [-\pi ,\pi ]$, and

$$\begin{aligned} {\mathbf A}_j= & {} \sum _{i=1}^{n_x}(a_{ji}\sin \alpha _i + b_{ji}\cos \alpha _i)\\ {\mathbf B}_j({\mathbf x})= & {} \sum _{i=1}^{n_x}(a_{ji}\sin x_i + b_{ji}\cos x_i \end{aligned}$$

where $a_{ji} \in {\mathbf A}$ and $b_{ji} \in {\mathbf B}$ with $a_{ji}, b_{ji} \sim U(-100,100)$, and $\alpha _i \sim U(-\pi ,\pi )$. This function is equivalent to $f_{12}^{CEC}$. A shifted version of schwefel 2.13 was also implemented, referred to as $f_{18}^{Sh}$.

$f_{19}$,:

the schwefel 2.21 function, defined as

$$\begin{aligned} f_{19}({\mathbf x}) = \max _j\{|x_j|, 1\le j\le n_x\} \end{aligned}$$

(27)

with each $x_j \in [-100,100]$.

$f_{20}$,:

the schwefel 2.22 function, defined as

$$\begin{aligned} f_{20}({\mathbf x}) = \sum _{j=1}^{n_x}|x_j| + \prod _{j=1}^{n_x}|x_j| \end{aligned}$$

(28)

with each $x_j \in [-10,10]$.

$f_{21}$,:

the shubert function, defined as

$$\begin{aligned} f_{21}({\mathbf x}) = \prod _{j=1}^{n_x}\left( \sum _{i=1}^5(i\cos ((i+1)x_j+i)\right) \end{aligned}$$

(29)

with each $x_j \in [-10,10]$.

$f_{22}$,:

the spherical function, defined as

$$\begin{aligned} f_{22}({\mathbf x}) = \sum _{j=1}^{n_x}x_i^2 \end{aligned}$$

(30)

with each $x_j \in [-5.12,5.12]$. A shifted version was implemented, referred to as $f_{22}^{Sh}$ and equivalent to $f_{1}^{CEC}$.

$f_{23}$,:

the step function, defined as

$$\begin{aligned} f_{23}({\mathbf x}) = \sum _{j=1}^{n_x}\left( \lfloor x_j+0.5\rfloor \right) ^2 \end{aligned}$$

(31)

with each $x_j \in [-100,100]$.

$f_{24}$,:

the vincent function, defined as

$$\begin{aligned} f_{24}({\mathbf x}) = -\left( 1+\sum _{j=1}^{n_x}\sin (10\sqrt{x_j})\right) \end{aligned}$$

(32)

with each $x_j \in [0.25,10]$.

$f_{25}$,:

the weierstrass function, defined as

$$\begin{aligned} f_{25}({\mathbf x})= & {} \sum _{j=1}^{n_x}\left( \sum _{i}^{20}(a^i\cos (2\pi b^i(x_j+0.5)))\right) \nonumber \\&-n_x\sum _{i=1}^{20}(a^i\cos (\pi b^i)) \end{aligned}$$

(33)

with each $x_j \in [-0.5,0.5]$, $a=0.5$ and $b=3$. A shifted and rotated version of the weierstrass function was implemented, referred to as $f_{25}^{ShR}$, equivalent to $f_{11}^{CEC}$.

$f_{26}$,:

a shifted expansion of the griewank and rosenbrock functions ($f_{6}$ and $f_{13}$ respectively), equivalent to $f_{13}^{CEC}$. Each $x_j \in [-3,1]$.

$f_{27}-f_{37}$,:

which are all composition functions, respectively equivalent to $f_{15}^{CEC}$ to $f_{25}^{CEC}$. All functions have $x_j \in [-5,5]$, except $f_{37}$ for which $x_j \in [2,5]$.