Mohammad Reza Bonyadi
ABSTRACT
In a particle swarm optimization algorithm (PSO) it is essential to guarantee convergence of particles to a point in the search space (this property is called stability of particles). It is also important that the PSO algorithm converges to a local optimum (this is called the local convergence property). Further, it is usually expected that the performance of the PSO algorithm is not affected by rotating the search space (this property is called the rotation sensitivity). In this paper, these three properties, i.e. stability of particles, local convergence, and rotation sensitivity are investigated for a variant of PSO called Standard PSO2011 (SPSO2011). We experimentally define boundaries for the parameters of this algorithm in such a way that if the parameters are selected in these boundaries, the particles are stable, i.e. particles converge to a point in the search space. Also, we show that, unlike earlier versions of PSO, these boundaries are dependent on the number of dimensions of the problem. Moreover, we show that the algorithm is not locally convergent in general case. Finally, we provide a proof and experimental evidence that the algorithm is rotation invariant.
- J. Kennedy and R. Eberhart, "Particle swarm optimization," in International Conference on Neural Networks, IEEE, 1995, pp. 1942--1948.Google Scholar
- A. P. Engelbrecht, Fundamentals of computational swarm intelligence. Hoboken, NJ: Wiley, 2005. Google ScholarDigital Library
- R. Poli, "Analysis of the publications on the applications of particle swarm optimisation," Journal of Artifficial Evolution and Application, pp. 1--10, 2008 Google ScholarCross Ref
- D. Bratton and J. Kennedy, "Defining a standard for particle swarm optimization," in Swarm Intelligence Symposium, IEEE, 2007, pp. 120--127.Google Scholar
- M. Clerc. (2011). Standard Particle Swarm Optimization Available: http://hal.archives-ouvertes.fr/hal-00764996Google Scholar
- M. Zambrano-Bigiarini, M. Clerc, and R. Rojas, "Standard Particle Swarm Optimisation 2011 at CEC-2013: A baseline for future PSO improvements," in Congress on Evolutionary Computation, IEEE, 2013, pp. 2337--2344.Google Scholar
- M. Clerc and J. Kennedy, "The particle swarm - Explosion, stability, and convergence in a multidimensional complex space," IEEE Transactions on Evolutionary Computation, vol. 6, pp. 58--73, Feb 2002 Google ScholarDigital Library
- I. C. Trelea, "The particle swarm optimization algorithm: convergence analysis and parameter selection," Information Processing Letters, vol. 85, pp. 317--325, Mar 31 2003 Google ScholarDigital Library
- P. K. Lehre and C. Witt, "Finite First Hitting Time versus Stochastic Convergence in Particle Swarm Optimisation," CoRR, vol. abs/1105.5540, 2011Google Scholar
- F. Van den Bergh and A. P. Engelbrecht, "A Convergence Proof for the Particle Swarm Optimiser," Fundamenta Informaticae, vol. 105, pp. 341--374, 2010 Google ScholarCross Ref
- D. N. Wilke, S. Kok, and A. A. Groenwold, "Comparison of linear and classical velocity update rules in particle swarm optimization: Notes on scale and frame invariance," International journal for numerical methods in engineering, vol. 70, pp. 985--1008, 2007Google Scholar
- R. Eberhart and J. Kennedy, "A new optimizer using particle swarm theory," in International Symposium on Micro Machine and Human Science, IEEE, 1995, pp. 39--43.Google Scholar
- M. A. M. de Oca, T. Stützle, M. Birattari, and M. Dorigo, "Frankenstein's PSO: A Composite Particle Swarm Optimization Algorithm," IEEE Transactions on Evolutionary Computation, vol. 13, pp. 1120--1132, Oct 2009 Google ScholarDigital Library
- M. Clerc, Particle swarm optimization: Wiley-ISTE, 2006.Google Scholar
- Y. Shi and R. Eberhart, "A modified particle swarm optimizer," in World Congress on Computational Intelligence, IEEE, 1998, pp. 69--73.Google Scholar
- R. Poli, J. Kennedy, and T. Blackwell, "Particle swarm optimization an overview," Swarm Intelligence, vol. 1, pp. 33--57, 2007Google ScholarCross Ref
- J. Matyas, "Random optimization," Automation and Remote Control, vol. 26, pp. 246--253, 1965Google Scholar
- F. J. Solis and R. J.-B. Wets, "Minimization by random search techniques," Mathematics of Operations Research, vol. 6, pp. 19--30, 1981Google ScholarDigital Library
- N. Baba, "Convergence of a random optimization method for constrained optimization problems," Journal of Optimization Theory and Applications, vol. 33, pp. 451--461, 1981Google ScholarDigital Library
- R. Poli, "Mean and Variance of the Sampling Distribution of Particle Swarm Optimizers During Stagnation," IEEE Transactions on Evolutionary Computation, vol. 13, pp. 712--721, Aug 2009 Google ScholarDigital Library
- M. Jiang, Y. P. Luo, and S. Y. Yang, "Stochastic convergence analysis and parameter selection of the standard particle swarm optimization algorithm," Information Processing Letters, vol. 102, pp. 8--16, Apr 15 2007 Google ScholarDigital Library
- F. Van den Bergh and A. P. Engelbrecht, "A study of particle swarm optimization particle trajectories," Information Sciences, vol. 176, pp. 937--971, 2006 Google ScholarDigital Library
- M. Jiang, Y. Luo, and S. Yang, "Particle swarm optimization-stochastic trajectory analysis and parameter selection," Swarm intelligence focus on ant and particle swarm optimization. I-TECH Education and Publishing, Wien, 2007Google Scholar
- M. Schmitt and R. Wanka, "Particle Swarm Optimization Almost Surely Finds Local Optima," in Genetic and Evolutionary Computation Conference, ACM, Amsterdam, The Netherlands, 2013, pp. 1629--1636. Google ScholarDigital Library
- D. Wilke, "Analysis of the particle swarm optimization algorithm," Master Thesis, University of Pretoria, 2005.Google Scholar
- M. R. Bonyadi, X. Li, and Z. Michalewicz, "A hybrid particle swarm with velocity mutation for constraint optimization problems," in Genetic and Evolutionary Computation Conference, ACM, Amsterdam, The Netherlands, 2013, pp. 1--8. Google ScholarDigital Library
- N. Hansen, R. Ros, N. Mauny, M. Schoenausr, and A. Auger, "PSO facing non-separable and ill-conditioned problems," INRIA, 2008Google Scholar
- N. Hansen and A. Ostermeier, "Adapting arbitrary normal mutation distributions in evolution strategies: The covariance matrix adaptation," in Congress on Evolutionary Computation, IEEE, 1996, pp. 312--317.Google Scholar
- R. Storn and K. Price, "Differential evolution-a simple and efficient heuristic for global optimization over continuous spaces," Journal of Global Optimization, vol. 11, pp. 341--359, 1997 Google ScholarDigital Library
- R. Salomon, "Reevaluating Genetic Algorithm Performance under Coordinate Rotation of Benchmark Functions - A survey of some theoretical and practical aspects of genetic algorithms," BioSystems, vol. 39, pp. 263--278, 1995Google ScholarCross Ref
- A. Auger, N. Hansen, J. P. Zerpa, R. Ros, and M. Schoenauer, "Empirical comparisons of several derivative free optimization algorithms," in Acte du 9ieme colloque national en calcul des structures (Proceedings of the 9th National Symposium in calculation of structures), 2009.Google Scholar
- M. R. Spiegel, Theory and problems of vector analysis: Schaum's Outline Series, 1959.Google Scholar
- W. M. Spears, D. T. Green, and D. F. Spears, "Biases in particle swarm optimization," International Journal of Swarm Intelligence Research, vol. 1, pp. 34--57, 2010 Google ScholarDigital Library
Index Terms
- SPSO 2011: analysis of stability; local convergence; and rotation sensitivity
Recommendations
Book review: particle swarm optimization for single objective continuous space problems: A review
This paper reviews recent studies on the Particle Swarm Optimization PSO algorithm. The review has been focused on high impact recent articles that have analyzed and/or modified PSO algorithms. This paper also presents some potential areas for future ...
Particle swarm optimization with adaptive population size and its application
A particle swarm optimization (PSO) that uses an adaptive variable population size and periodic partial increasing or declining individuals in the form of ladder function is proposed in the paper. The aim is to enhance the overall performance of PSO. ...
Following the leader - particle dynamics in constricted PSO
ICCCI'11: Proceedings of the Third international conference on Computational collective intelligence: technologies and applications - Volume Part IIWe consider particle swarm optimization algorithm with a constriction coefficient and investigate particle dynamics without stagnation assumptions. We propose differential models of a particle following the swarm leader, while global best and personal ...
Comments