Hybrid Form of Particle Swarm Optimization and Genetic Algorithm For Optimal Path Planning in Coverage Mission by Cooperated Unmanned Aerial Vehicles

In this paper, a new form of open traveling salesman problem (OTSP) is used for path planning for optimal coverage of a wide area by cooperated unmanned aerial vehicles (UAVs). A hybrid form of particle swarm optimization (PSO) and genetic algorithm (GA) is developed for the current path planning problem of multiple UAVs in the coverage mission. Three path-planning approaches are introduced through a group of the waypoints in a mission area: PSO, genetic algorithm, and a hybrid form of parallel PSO-genetic algorithm. The proposed hybrid optimization tries to integrate the advantages of the PSO, i.e. coming out from local minimal, and genetic algorithm, i.e. better quality solutions within a reasonable computational time. These three approached are compared in many scenarios with different levels of difficulty. Statistical analyses reveal that the hybrid algorithm is a more effective strategy than others for the mentioned problem.


INTRODUCTION
trajectories by GA under different scenarios and the considered objectives and constraints are closely related to the cooperation among multiple UAVs, etc. All of these have a common weakness based on genetic resolution: when the size of the problem grows up, genetic methods get stuck in the local maxima.
Although GA has proven to be a useful method of optimization for difficult and discontinuous multidimensional engineering problems, another evolutionary method of optimization, the particle swarm optimization (PSO), can accomplish the same goal as GA optimization in a different way (Kennedy and Spears 1998). When the problem size has been bigger in terms of waypoint or the number of UAVs, most of the optimization methods are encountered in the clogged situation.
The PSO aims to come out from local minimal, while the GA can return better quality solutions within a reasonable computational time. Khanesar et al. (2007) presented a useful and applicable model to solve discrete waypoint traveling problems by the PSO framework that can be expanded for multiagent problems. Particle velocity pattern in cellular and string base problem is discussed and several supplementary rules, operators, and discrete deduction. Jingjin and LaValle (2013) calculated the shortest path and developed a method for multiagent systems with an innovative search space division model. In this research, the traveling problems were solved in a discrete form with bit and string base pattern by PSO.
The hybrid form of these two evolutionary algorithms is widely used in different applications such as formation (Duan et al. 2013), robot path planning (Wang et al. 2016b), communication (Robinson et al. 2002), and map regeneration (Ghamisi and Benediktsson 2015). Some methods are not real hybrid methods where the output of one method is used for the input of another method, while the methods are completely independent of each other (Robinson et al. 2002). To overcome the irregularity form of mentioned papers in multiagent path planning, a novel form of the parallel formulation was planned to create a hybrid form of PSO-genetic algorithms (HPSOGA) for multiple UAVs coverage mission.
The main contributions of the approach can be summarized as follows. First, design a parallel formulation to build up a hybrid form of solution algorithm for multiple UAVs to coverage a group of important areas as waypoints. Second, plan a way to increase the speed of convergence rate due to the removal of the local maximum answers. The performance of the considered hybrid algorithm, compared to other algorithms, is presented in several scenarios of cooperated patrolling. The proposed hybrid architecture is presented after a brief review of multi-UAV patrolling and related mathematical model and the coding system. Simulation results and discussion will be presented in the next sections. Finally, the conclusion presents the key results of our research.

MULTI-UAV PATROLLING
Optimal waypoint traveling systems in aerial fields depend on vehicle capabilities and environmental limitations. The use of multiple UAVs can reduce the search space per agent, but make the difficult problem that cannot be solved by deterministic methods. Multi-UAV patrolling patterns must provide N distinct trajectories for N-number of UAV through specified waypoints. The generated trajectories are usually constrained by UAV capabilities. The mathematical structure of this problem is a discrete optimization form and population-based evolutionary algorithms efficiently can handle it.
The considered population method is based on the statistical impression of all initial populations where the stronger paths impress weaker ones and improve all answers simultaneously. This approach is the initial idea for solving the main section of this problem. In genetic structure, after the formation of initial chromosomes (as initial paths), different functions and operators upgrade existing trajectories. In the PSO structure, each path (chromosomes) are impressed by the best neighboring path and best global path. Neighbors are defined as the generated paths in one chromosome at different iterations. Population-based methods as well as hybrid evolutionary algorithms due to faster computation time can be applied in a real and-time agent-based pattern that combines the advantages of each algorithm in the integrated format (Khakmardan et al. 2011).

MATHEMATICAL MODEL
The patrolling process in the aerial vehicles is performed by waypoints, therefore, these waypoints can be embedded in a string with location bits. Traveling through the desired waypoints need enough flight time and fuel capacity. To formulate the two-dimensional multiagent patrolling problem, it is necessary to define the searchable areas by a set of waypoints. These waypoints are indexed respectively from 1 to n in a vector to facilitate the discrete optimization problem. Due to the discrete nature of this problem, the initial n-dimensional possible vector for each population is randomly generated as a permutation X i (t) = x 1 (t),x 2 (t),...,x n (t), where X(t) denotes the set of trajectories for n agents in the t-th iteration for the i-th parallel response.
As formulated, in a multiagent patrolling problem due to the given start and endpoints, the path strings are divided into some sections called substrings. Each substring starts with a predefined start point and terminates with the known endpoint. If n is the number of UAVs, subscripts B and E denote the location of "begin" and "end" waypoint, the sampling trajectory for the k-th UAV is presented in Eq. 1: (1) All trajectories can be presented in one string called X(t), which is defined in Eq. 2: (2) Therefore, the number of trajectory string member of the i-th parallel answer is calculated as follow. The number of members in one sampling trajectory is the same as all waypoints numbers (m), given by Eq. 3: ( 3) where the number of waypoints (m) is equal to m = r 1 + r 2 + ... + r n + 2n, the space of possible answers for I parallels answers has I × m members. Hence the length of one trajectory is deferent from others, the location of the starting bit of each agent trajectory vector can be stored in a separate matrix. The matrix is as follows (Eq. 4): (4) As shown in Eq. 3 the location of a starting waypoint for the first agent is 0 and the second agent starts its travel from location m li and so on. Because of the denoted structure, the length of the matrix M X is I × n, where I is the number of parallel population of trajectories and n is a number of UAVs. So, for simplicity of calculation, the symmetric distance matrix is defined, which indicates the distance between all waypoints in indoor environments. R is the matrix of Euclidean distance is shown in Eq. 5: is Euclidean distance between two waypoints x i , x j , and (X,Y) denote related Cartesian coordination.

CODING SYSTEM
According to the bitwise path variable X and population number i in iteration t, it is necessary to use a coding system to avoid the complicated computations and engage with complex issues around this problem. In this case for simplicity, the elements of X variable (x 1 , x 2 , ..., x n ) are marked by magnitude 1 to n respectively and every change takes place due to noted magnitudes. Figure 1 shows the initialization and Eq. 6 denotes related path string.
In parallel processing, this method is distributed to the matrix in size of I rows. Equations 7 and 8 denote one possible answer for two agents parallel coding system over 10 waypoints with three parallel populations, where Fig. 2 shows the result trajectories according to X and N variables. Three sample paths for two agents through ten waypoints numbered 1 to 10, modelized by the coding system Eq. (7) and (8).

HYBRID PSO-GENETIC ALGORITHM (HPSOGA)
The solution of the discrete problem by population methods requires a useful pattern for the optimization process. The current paper deals with a hybrid method that combines PSO and genetic operators to achieve the best performance of waypoint permutation and dividing manner. The considered operators are in three general categories based on the mutation, inverse, and crossover along the path. The fitness function evaluates the quality of the generated path and upgrades the local and global best neighbors.

PSO-MUTATION OPERATOR
The mutation operator changes the location of trajectory string nodes based on discrete PSO. In this operator, the transfer length is obtained from local and global best terms. In PSO calculations, some improvement has been carried out for bit analysis and especially for multiagent traveling problems. Equation 9 represents the PSO equation based on environmental changes (Zhong et al. 2007). This equation must change due to bit analysis and discrete form of PSO.
where ω, α, β are empirical constant-coefficient, V denotes bit velocity, P ibest and P gbest are the local and global optimal location for a considered bit. The mutation bit must be imagined between 0 to m due to string length so it has to change in Eq. 10. This equation guarantees that the resulting answer belongs to [0,n].
In Eq. 10, the operator [ ] denotes the ceiling function. At the final step, the mutation direction must be determined by Eq. 11. (11)

PSO-INVERSE OPERATOR
This operator inverses two or more waypoints bit location with themselves. The swap method can be obtained by the PSO approach. The changeable waypoints are selected in one sub-string for certain UAVs and, after tending to the local best answer, these points can be selected in two different substrings. This operator is presented in Eq. 12.
In a multipoint inversion, all the points between the selected locations are inverted. Equation 12 shows this operator in multipoint inversion.

PSO-CROSSOVER OPERATOR
This operator combines the trajectories with the calculated local or global optimal path, which is found at this moment of the solution. For population, i in iteration t this operator is written as follow (Eq. 13): where coefficients (c1 × r1) and (c2 × r2) are random variables and denote related crossover length from P ibest and P gbest . The sign ⊕ represents a crossover operator and ◊ denotes operator for bit selection in local-global best answers. For one selected path Xi where k is a random number and belongs to the range [1, m-1] and r is r = cl × rl.

COST FUNCTION
The patrolling cost function is defined as the minimum distance of trajectories. The multiagent distance processing is different in application. The overall optimal distance is calculated by a min-max function. In this approach, the min-max function selects the best answer from minimum magnitude among all maximum distance that each population has experienced. Equation 15 represents the min-max function where k is a number of parallel populations.
In considered evolutionary algorithm, n is a number of UAVs and k is population count, so the location of the agents start point in the string X(t) is saved in other vector or matrix that mark by N Xi = [0, m li (t), m 2i (t), ..., m (n-1)i (t)]. As a result, in the multiagent case, algorithms make and promote these two matrixes. The elements of these matrixes are shown in the integrated form of the multiagent case as follows (Eqs. 16 and 17): where r l is the number of waypoints assign to the first agent and so on; k populations are updated in each iteration and upgrade P ibest and P gbest simultaneously. The main purpose of populations is rapid optimizing the starting points and the number of the assigned waypoint to each agent. Whereas the number of assigned waypoints to each agent was saved in the matrix N Xi , two new variables N ibest and N gbest are created. These two variables saved the local best number of assigned waypoints and global ones, respectively. They also upgrade in each iteration and are determined by the random process. Finally, after generating the new data by mention operators in each iteration, four update process will be done according to the cost function value or fitness check. These variables are P ibest , P gbest , N ibest and N gbest Pseudocode 1 represents the updating process for the new path string and the mention variables.

Pseudocode 1:
The resulted path and related flight time for one airplan' e might be longer than the airplane's endurance; in this case, the count of assigned waypoints must be reduced. The traveled path string for population i and agent j in iteration t is d i j (t); so, due to flight time, this path can be accepted in such a situation as the following pseudocode:

Pseudocode 2:
The UAVs flight in the same altitude with different velocities and no constraint/limitation on starting/ending waypoint as well as flight time or ending time. The model for calculating dynamic parameters and distances is based on point-mass 3D equations. After each operation, the fitness function measures the quality of the result and updates the previous parameters. Two-point crossover, mutation, and crossover with P ibest and P gbest act on path string matrix (X i k (t)) respectively and random permutation change the array of path location matrix (N i (t)). These iterative steps continue until the algorithm achieves the best results. The flowchart of the algorithm is present in Fig. 3.
Data Selection

RESULTS AND DISCUSSION
For algorithm evaluation, the initial random waypoint was used and the performance in the traveling process was measured with two, three, and four UAVs. The initial waypoint random pattern is based on the uniform distribution in generalized ranges 0 to 1. The starting point for all UAVs is the origin of the coordinate system and the ending point in the dimensionless scale is [1,1]; therefore, in no waypoints condition the minimum distance between these two points is square diagonal that its value is 2=1.4142 . Consequently, the average traveling distance for all agents compares with straight traveling distance. Figures 4 to 9 show three scenarios on the waypoints division and travel results. In the first scenario, two UAVs pass over the waypoints. In Fig. 4, the first iteration of waypoints is determined; and the optimal result for two similar UAVs that traveling over 20 waypoints is shown in Fig. 5. In this case, the average traveling distance is 2.3447 due to normalized length. Figures 6  to 9 show similar scenarios for 3 and 4 UAVs and represent the reduction of average traveling distance to 1.6264 for 4 UAVs in multiagent over 20 waypoints patrolling.   The proposed algorithms use beneficial items of PSO and genetic algorithm together. Table 1 denotes the results of PSO, GA, and HPSOGA approach for several cases. According to these results, the performance of GA is reduced by increasing the number of agents and cause a sharp increase in the computation time during this process. HPSOGA becomes more efficient when the number of agents increases. Figure 10 represents a comparison of average convergence curves and Table 2 compares results.

CONCLUSIONS
The multiagent traveling process for aerial agents is a complex problem that cannot be solved by the deterministic method; therefore, the evolutionary algorithms are applied. The hybrid form of discrete PSO-GA with some heuristic promotions prepares the fast computation and optimal framework to solve these problems. This paper deals with the new discrete method and related coding system for aerial multi-agent UAVs. It can stimulate new and optimal paths by evolutionary algorithms. Considered operators and evaluation system can combine waypoints and separate the number of the optimal waypoint assigned to each UAV simultaneously. This method is simple to implement and can also compute in real-time situations.
The final results show that by increasing the number of UAVs, the average traveling distance is reduced by passing over the optimal paths. This fact can open the field to replace the low-cost and limited-range UAVs by high-cost UAVs in traveling and search missions.

FUNDING
There are no funders to report for this submission.