A Novel and Fast Memory Perturbation Method to Increase Exploration in Particle Swarm Optimization Algorithm

Particle Swarm Optimization (PSO), one of the versatile nature-inspired optimization algorithm, continue to suffer from premature convergence despite the numerous amount of research trying to improve this algorithm. Many research had tried to address this issue but often use a complex algorithm which tax on computational time and complexity. This research introduced a novel perturbation method to mitigate premature convergence / to increase exploration while keeping the computational cost at a minimum. The particles’ memories (i.e the position of personal and global best) are modified by a random multiplier which in turn will ‘perturb’ the particles’ velocity. The implementation of this novel perturbation method in early iterations had resulted in 100% success rate in finding global optima in multimodal benchmark tests including the Rastrigin problem - whereas the original PSO failed in all benchmark tests - without adding a significant amount of computational complexity and time.


Introduction
Particle Swarm Optimization (PSO) is a versatile optimization algorithm used in several areas. The original PSO was introduced in 1995 by Eberhart and Kennedy, and had gained popularity in research to further improve the algorithm, customized it to particular problems, and combined it with other algorithms.
Some of the PSO enhancement strategies include the use of multi-sub-populations [1], particle restart [2], combining PSO with other search techniques [1], using different neighborhood topologies [3], and by having a better initial population [4]. Despite the rigorous research in PSO, one of the main problems which still remains for PSO is premature convergence, especially in problem with many local optima.
Many research had addressed this issue by incorporating perturbation to increase diversification/exploration. This kind of modification often incorporate complex computation and thus increase its computational and time cost. This research proposed a novel and simple perturbation

The Original PSO
PSO algorithm is inspired by how birds or fish flock for food. Individual / candidate solutions in PSO are called 'particles'. Each particle will move randomly in the search space to find the best solution. In each iteration, particles will move closer to the best solution by learning from the experience of the particle itself (cognitive learning, c1, applied to personal best, x*) and from other particle experiences (social learning, c2, applied to global best, xg). The c1 and c2 are positive constants which usually set to = 2. The formula of particle movement or "velocity update" vi at t+1 is shown in Equation 1.
The search capability of PSO in finding optimum solution lies in the tuning of particle's exploration and intensification in their movements [2], [5]. Exploration is when the particles explore a wider range of possibility, while intensification is when the particles move 'slowly' to find a better solution near a good solution (i.e the particles' memory of global best / personal best).

PSO problem: premature convergence
Research had shown that PSO tends to converge quickly (prematurely) [6], [7]. This characteristic of PSO will consequently cause a failure in finding an optimal solution, especially in a multimodal problem -a problem with multiple "good solutions" and one optimum solution. In a multimodal problem, PSO will tend to prematurely converge to / "trapped" in the "good solution" / local optima. This problem with the original PSO is called premature convergence [3], [4], [7]- [11]. Analysis showed that the premature convergence and the ability to find best solution / global optimum of PSO are both probabilistic [7], [12]. There had been numerous research which attempted to alleviate this issue such as [8], [9]. Some of the common strategies include the use of inertia weight and perturbation (explained later). These research often use a complex algorithm. The persistent premature convergence problem might mean that PSO needs exploration in general. It had been noted that the performance of the PSO mostly relies upon inertia weight and optimal parameter setting [10], [13], [14]. These parameters will affect the exploration and intensification ability of particles in PSO, thus research in solving premature convergence will focus on these parameter tuning.

Solving The Premature Convergence Problem: Exploration and Intensification Tuning 2.3.1. The Inertia Weight Strategy
The first attempt to tune the exploration and intensification is to incorporate inertia weight (IW) [15], which is not present in the original PSO. The inertia weight (α or w) refers to the amount of the contribution of the previous particle velocity to its current velocity. Inertia weight is intended to balance the exploitation and exploration when the particles are searching for optimal solution in a search space [13]. [15] used a randomized IW and in 1998, Shi and Eberhart used a linearly decreasing inertia weight (LDIW) (α) into the algorithm [16].
Where the value of α gradually decreases from α max to α min throughout the iterations: When w = 0, particle speed will only be affected by the personal best and global best position. This means that the particles will immediately change their position to the best position once the best position is known. A small value of w will increase intensification (local search). In contrast, when w is high, the current velocity of the particle will be affected by the previous velocity (inertia). Such particles maintain the previous speed even though a better position is known. A high value of w will increase diversification (global search). To successfully find the optimum values (maximum or 3 minimum), high diversification is needed at the beginning of the iteration, while intensification is needed later in the iteration [17]. This can be achieved simply by using a linearly decreasing inertia weight. However, research had pointed out that LDIW strategy will cause PSO to prematurely converge to local optimum in multimodal problems [18], [19]. Other research also used an adaptive selection of inertia weight in which the value of inertia weight will be set according to certain criteria within the process of finding an optimum solution, such as in [20]. However, research also showed that LDIW is still a competitive strategy compared to other IW strategies if it's parameters are set properly [18].  [19] Linearly decreasing Best LDIW runs from 0.9 to 0.4 Arasomwan, 2013 [18] Linearly decreasing LDIW is still competitive against other variants Farooq et al., 2017 [4] Linearly decreasing LDIW in [19] applied twice Nobile et al., 2018 [14] Self-tuning Fuzzy logic Agrawal, 2018 [13] Adaptive Cumulative binomial probability A. Agrawal and S. Tripathi, 2019 [15] Adaptive Binomial probability distribution

The Perturbation Strategy
Another attempt to alleviate premature convergence is by the use of 'perturbation'. A perturbation is a way to 'randomize' the particles' movement so that they can explore a more diverse candidate solution in the attempt to find the global optimum. The list of research in PSO perturbation strategy and the position of this research are shown in Table 2.

Method
In the standard PSO method, the "perturbation" is only applied to the difference between personal best and global best with the current particle position. The disadvantage of this method is that it limits the exploration capability of particles which can actually be greater. (Yang et. Al., 2007) proposed a new way to maintain diversification by applying direct 'perturbation' to the (current) personal best and global best in each iteration [25]. It uses two random number, r3 and r4, whinge range from -2 to 2, r [-2,2]. The formula for particles' velocity in [25] is shown in Equation 2.
Inspired by [25], we propose a perturbation k*r which are applied to personal best and global best as shown in Equation 3. This modification will cause the particle to have a probabilistic movement within the enclosed area shown in Figure 2. With r is a random number between 0 to 1, r [0,1]. When k is set to 2, the (k*r) will produce perturbation that ranges from 0 to 2 and is applied as a multiplier for the personal best and global best. The perturbation is applied to both personal and global best as an attempt to further increase the exploration capability of the particles. We also experiment with a combination of IW proposed by Y. Shi and R.C. Eberhart [16] and use several range values of linear-decreasing iw. Here, we experiment with a w range that is somewhat wider (1-0) to see if the performance will get better. Experiments were done using Java programming language run in Netbeans. For all experiments, the parameters are set as follows: population= 50; max iteration = 100; xmax = 100; xmin = -100, vmax = 100, vmin= -100, C1 = 2, C2 = 2, k1 = 2, k2 = 2, and r[0,1]. IW is set with αmax = 1 and αmin = 0. Each experiment setting/type is repeated 1000 times. Then the success rate, min value and SD of min value for each test cases are calculated and compared.

Result
Based on 50 tests of each of the five function tests, the results of the combination of the Linearly-Decreasing Inertia-Weight (LDIW) and Memory Perturbation (MP) methods produced a 100% success rate in finding the minimum global for all test functions. This perturbation is only needed several times at the beginning of the iteration. Based on testing, the initial 10 iterations have given satisfactory results. The use of perturbation that is too long will actually weaken the power of LDIW's intensification. With 'nip' is the amount of perturbation we want at the beginning of the iteration (number of initial perturbation).  . This failure in the Rastrigin function is caused by the characteristics of the (standard) PSO where which the particles are "trapped" in a solution which appears to be the global minimum but is actually a local optimum.
To increase success in the Rastrigin function, an additional method is needed to maintain diversification so that particles can further explore the search space before determining the area for intensive searches. By maintaining diversification for some time, it is expected that particles can explore a more diverse search space and hopefully can visit an area near the minimum global -the intensification will proceed the digging into the minimum value.
The result showed that the best use of this perturbation method is to be applied only several times (10 times) in the initial iterations. Prolonged use of this perturbation will only result in particles moving randomly and failed to find global optimum, which is not what we want.

Conclusion
Despite a large amount of research, PSO still suffers from premature convergence where the particles are trapped in a sub-optimal solution in a multimodal problem. This research aims to explore a new strategy to increase the particles exploration capacity of particles in PSO in order to increase its capability to find the global optimum. Two perturbation factors are applied to global best and personal best to increase the exploration of search space in the hope of that the particle will stop at a point near the global optimum and continue to dig into the minimum value. The experimental result showed that this applying this perturbation method in early iteration, and combined it with LDIW can achieve 100% success rate in finding global optimum in five benchmark problems, i.e the Sphere function, Schwefel Problem 2.22, Rosenbrock, Rastrigin, and Ackley. Compared to other research, this research proposes a simple perturbation method which did not add a significant amount of computation complexity and time. The limitation of this study is that this method was only benchmarked against 2D problems. Further researches are open to experimenting on problems with a higher dimension.