ESTIMATION OF PARAMETERS IN PARTICLE SWARM OPTIMIZATION

: Nowadays the heuristic techniques are becoming one of the most useful tools in solving optimization problems. One of these techniques is Particle Swarm Optimization, PSO algorithm. From the numerical analysis perspective this is a successful method, but many issues are still to be considered regarding the convergence of algorithm. In this paper we deal with the problem of the evaluation of the parameters of the algorithm that assure its convergence. In the previous work we presented some restriction on the parameters of the perturbated dynamical system, that modeled the PSO algorithm. These restrictions are necessary to guarantee the stability of the system. In this paper we present some other restrictions needed to ensure the stability of the system and to advance in the research of the convergence of PSO.


INTRODUCTION
The particle swarm optimization is an algorithm based on the behavior of a flock of birds, fish, ants, more generally a group of individuals, and considers their behavior to control the area for their necessity, like feeding. There is no surprise that its first origin comes from Social Psychology as a simulation of social and cognitive processes in order to model abstract concepts as intelligence and learning. The model was invented by Kennedy and Eberhart in 1995. The main idea of the algorithm is that the particles play the role of the members of the flock, and the algorithm imitates the behavior of the member (particle) locally and globally which means the flock as a whole. Since the algorithm is based in the behavior of a particle and the interaction of each particle with the ones that are in the neighborhood, then it converges to the global solution. [1][2][3]. There are many other algorithms which operate in a similar way, known as population-based evolutionary algorithms, however, PSO algorithm is motivated by the simulation of social behavior instead of the best performance made by the strongest individual (natural selection). The challenge in the optimization methods we use for optimization problems is to find the global solution because they mostly converge to the local solution. The heuristic algorithms try to converge to the global solution, which applies even for the method we study, PSO. The PSO algorithm is very efficient numerically but still lacks theoretical foundations. In this work we present a study on the stability of the algorithm. First, we present the basic PSO algorithm and then we consider the dynamical system where this algorithm is modeled. In sections 4 and 5 we introduce the parameters , , , , that allow us to operate within intervals where stability holds. Then in section 6 we tested these values on well-known functions and arrived at an optimal solution. Three cases are considered, two of which have the inertia weight vary at each iteration and in one case it is kept constant with value one, the ideal case. Finally, in the conclusion section we give a resume of the evaluation of the parameters theoretically and then the results obtained by applying the restrictions on the parameters in some test functions.

PARTICLE SWARM OPTIMIZATION
As it is known the particle swarm optimization (PSO) is an algorithm that finds optimal solution through the interaction of individuals in a population of particles. The algorithm searches a space by adjusting the trajectories of individual vectors, named particles. Each particle is aimed to move toward the positions of their own best position and the best position of their neighbors. More specifically, a particle is identified by its current speed and its position, the most optimal position of each particle and the most optimal position of the neighborhood. We focus our work in the study of the behavior of one particle, of one individual. The speed and the velocity of a particle is given by the formula [3] (1) +1 = represents the speed of the particle i in the k time and x represents the d-dimension quantity of its position or more simply the current position of the particle represents the d dimension quantity of the particle i at its most optimal position at its k times and is the d-dimension quantity of the swarm at its most optimal position. The speed is always between two boundaries, − and , in order to avoid the wandering of the point away from the operational searching space. 1 , 2 are non-negative constants, called cognitive learning rate or the acceleration coefficients and play an important role in the algorithm process. They represent the particle stochastic acceleration weight toward the personal best (pbest) and the global best (gbest). We should avoid small accelerate constant since the particle can move away from the goal area and large accelerate constant values since the particle can move very quickly to the goal area and leave it totally. How do we determine these acceleration coefficients? In previous work of Kennedy and Eberhart, also Clerc later was stated that high values of the cognitive component coefficients. All of this discussion is based more heuristically because still there is no mathematical assurance on the convergence of PSO. In this paper we present some values for the acceleration coefficients based on the perturbated dynamical system presented in the work of Clerc and Kennedy.

MODELLING THE PROBLEM IN A DYNAMICAL SYSTEM
First, we denote 1 = 1 1 , 2 = 2 2 , and = 1 + 2 . Then we redefine The dynamical system obtained by (1) and (2) (3) To estimate the parameters 1 , 2 for = 0 and We continue our study of convergence taking the inertia weight = 1, and perturbate the dynamical system introducing the parameters , . , , as follows where ∈ + * , ∀ ∈ , ( , ) ∈ 2 In this case we operate with matrix theory as the eigenvalues of the coefficients of a linear dynamical system determine the solution of that system as well.

= [ − − ]
If we denote the eigenvalues of the matrix ̅ 1 ̅ 2 then the solution of (4) is As in the previous section we evaluate the parameters 1 , 2 for the initial time = 0 And also from (*) for (5) and (6) guarantee that the system is stable.
Since the parameters must be real-valued then another restriction is the following We can compute the discriminant and obtain From the other hand, even if the condition (7) is satisfied one of the values of the eigenvalues ̅ 1 , where t non integer can be complex. So, we add a stronger condition for the eigenvalues, we require to be positive ̅ 1 > 0, ̅ 2 > 0 which is equivalent from (5) that From operation on the second inequality we obtain What we can deduce directly from (8) Table 1. Test function used, dimension and boundaries of the respective variables.
We have tested three different cases; in the first and third case we consider the inertia weight variable at each iteration based in the formulas shown in the respective cases. In the second case the inertia weight is kept constant, which is considered an ideal case. In the second case we consider the inertia weight equal to one, = 1 , and the acceleration coefficients 1, 2 equal, more specifically 2: 1 = 2 = 2.05, = 1 In the third case we use a different formula to evaluate the inertia weight, and the acceleration coefficients are kept constant.   In the figures the black color represents the curve of case 1, blue color the curve of case 2, and the red color the curve of case 3. For each test function we have presented in figure1, the result of the optimal values which we obtain from the algorithm until we arrive at the best step. From tables 2 we concluded that in case 1 and case 3 we have the best values which is noticed and emphasized also by figure 1. So, we can conclude that the convergence is faster in case 1 and case 3, when the inertia weight changes in each iteration of algorithm.

CONCLUSIONS
First, we explained thoroughly how the parameters , , , , are chosen to guarantee the convergence of the proposed variants of PSO. This was a very important issue of our work and it was based in the perturbated dynamical systems where we operated with analytic tools. The importance is shown in the request that this system must be stable. Then, for each of the proposed cases we took a group of functions considered as test functions. The results obtained are reasonable and satisfactory, emphasizing that case 1 and case 3 give the best results. This occurs due to the fact that the inertia weight changes in each iteration assuring a faster convergence then the cases when = 1.