Maintaining Complex Formations and Avoiding Obstacles for Multi-agents

This paper researches some problems in complex formation for multi-agents, in which two matrices are proposed to record the formation. The pattern matrix is used to describe the pattern of the formation; meanwhile, the location matrix is used to record the location of each agent. Thus, all desired positions of each agent will be obtained by geometrical relationship on the basis of two matrices above. In addition a self-adaptation flocking algorithm is proposed to control all agents to form a desired formation and avoid obstacles. The main idea is as follows: agents will form a desired formation through the method of formation control when far away from obstacles; otherwise, agents will freely fly to pass through the area of obstacles. In the simulation, three scenarios are designed to verify the effectiveness of our method. The results show that our method also can be applied in three dimensions. All agents will form a stable formation and keep the same velocity at last.


Introduction
Flocking is a form of collective behavior of a large number of interacting agents with a common group objective to complete certain tasks [Tanner, Jadbabaie and Pappas (2007); Meng, Jia, Du et al. (2014)]. Formation control is one of the most actively studied topics within the realm of multi-agent systems and their coordination control has attracted large amounts of research interest. Based on different sensing capability and the topology of multi-agents [Oh, Park and Ahn (2015)], formation control problems can be divided into three parts: position-based control, displacement-based control, and distance-based control. According to the fundamental ideas in control methods, formation control can be divided into four parts: leader-following, behavior-based, virtual structure, and consensus [Benzerrouk, Adouane and Martinet (2014) ;Lü, Chen and Chen (2016) ;Ge, Han, Ding et al. (2018) ;Essghaier, Beji, El Kamel et al. (2011); Xie and Wang (2007)]. In 2006, Olfati-Saber [Olfati-Saber (2006] proposed a theoretical framework for design and analysis of distributed flocking algorithms. It primarily discussed potentials to control agents for free flocking and obstacle avoidance. However, this method cannot control agents to form a desired formation. Su et al. [Su, Wang and Lin (2009)] proposed a method to track a virtual leader based on the artificial potential field method. By the influence of the potential field between neighboring robots, the organization of the swarm is stable and robust. Obstacle avoidance is driven by a repulsive potential field. The method of potential functions with leader-follower has mainly been researched by many scholars [Vilca, Adouane and Mezouar (2016); Yan, Guan, Luo et al. (2017); Shi, Zhao and Zhang (2019); Park and Yoo (2015); Mas and Kitts (2010) ;Brandao, Barbosa, Mendoza et al. (2014)] for formation control and obstacles avoidance. For example, an application of the multi-layer control scheme to guide a formation of three unmanned aerial vehicles (UAV) in trajectory tracking missions was presented [Mas and Kitts (2010)]. The centralized and distributed architecture of the lead-follower method was presented under the cluster perspective [Brandao, Barbosa, Mendoza et al. (2014)]. The key idea of this combination method is to use leader-follower to maintain a desired formation, and to use potential function to avoid obstacles. The differences between those papers are the various application backgrounds or special potential functions for obstacles avoidance. However, the velocity of leader will change dramatically when near by obstacles, which has a severe impact on formation control. Further, agents are probably trapped into a region with a large number of obstacles for an inappropriate method of obstacle avoidance. This paper aims to address the following issues. Firstly, the graph of formation is simple in most papers. A new model of formation needs to be constructed when the number of agents increases or decreases. This paper aims to provide a unitized method to record the formation graph by using the method of pattern matrix. Secondly, many papers often omit the obstacle environment or pass around obstacles with a fixed formation. However, the jitter of the leader will impose a significant effect on formation control when agents are close to obstacles. More seriously, agents will be trapped in the region with many obstacles. Hence, a self-adaptation distributed flocking algorithm is introduced to provide a strategy for multi-agents formation control and obstacle avoidance. The paper is structured as follows. Section 2 introduces some preliminaries and problem description. Section 3 discusses the models of matrices to describe a complex formation. Furthermore, the control protocol of our method is proposed in Section 4. A simulation of multi-agent system is presented in Section 5, and the simulation result is analyzed. The last section gives brief conclusions and discusses future works.

Graph theory
A graph G is a pair ( , ) ν ε that consists of a set of vertices ⊆ ; otherwise, it is directed. Consider a group of n agents in a graph G , and the adjacency matrix is defined to describe the position relation for agent i and its neighbors, specified as The elements in the adjacent matrix are other than just 0 or 1. Furthermore, the set of neighbors of agent i is defined as Meanwhile, all agents have the ability to sense their neighbors. Let r be the communication range, and r′ be the recognition range for obstacles. Let i p and j p be the positions of agents i and j , respectively. The adjacency matrix and neighbors can be rewritten as where || || ⋅ is the Euclidean norm. It means that adjacency matrix =0 ij a if the distance between agent i and j is outside the scope of communication range r ; otherwise =1 ij a .
Meanwhile, the neighbors of obstacles of agent i can be described as where ˆi k q is the location of each obstacle, and k O is the set of obstacles. It means that agent i is the neighbor of obstacle if their distance is within the obstacle recognition range r′ . It is important that agent would not need to know all the information from other agents or environment, i.e., agents just need to communicate with their neighbors, regardless of other agents outside the scope of communication range r . If all agents have the same interaction range, the proximity net G becomes an undirected graph. This paper contains some assumptions as follows.
(1) All agents have the same spherical sensing range.
(2) The delay of communication is omitted.

Problem description
n agents will form a desired formation, in which agent 1 is called "leader", and others numbered by 2,3, , n  are "followers". The leader has an independent motion; while followers will be influenced by the leader and other followers [Olfati-Saber (2006)]. The kinematic model of agents is defined as  f v t t is the input vector of dynamics of leader. A potential function which is influenced by the neighbors of all agents, the control protocol i u can be specified as where 0, 0 p v k k > > are the parameters of position and velocity. i V is the total potential of agent i , which is denoted by ij V is a differentiable, nonnegative, function of the distance ij p between agents i and j . Thus, the problem of formation can be described as where d i p is the desired position of agent i which is relative to the leader. 1 v is the velocity of the leader. The whole formation control problem can be regarded that all agents will reach to their desired positions and have the same velocity with leader agent over time.

Control scheme
In Fig. 1(a), the formation control of multi-agent system consists of a guidance law as an outer loop and a flight control as an inner loop, i.e., outer loop makes decisions and inner loop executes actuators. Hence, a closed control loop will be constructed by using the strategy of outer and inner loop for sampling the state with time T. It should be known that this paper mainly discusses outer loop for making decisions of multi-agents. Here, as shown in Fig. 1(b), the control processes of multi-agent system are detailed as follows. Firstly, the initial parameters of each agent (position, velocity, acceleration and so on), global object point, and obstacles will be given to construct the Environment as an initial state of multi-agents. For outer loop, there are two parts of Preprocessing and Control protocol, specified as follows. (1) The preprocessing of formation control will be executed. Thus, the desired positions of each agent will be obtained based on the pattern matrix F and conversion function ( ) T N in location matrix.
(2) Three modules of Interaction, Navigation, and Obstacle Avoiding are used to accomplish control protocol, in which two strategies of free-flying and self-adaptation flocking algorithm are proposed to obtain control signal to make agents form a desired formation from pattern matrix F . Further, in the inner loop, new states of multi-agents will be obtained in actuators among individual agents for sampling the state ( , , ) p v u with T . It is important to know that this control protocol is a distributed strategy. Each agent will execute their individual control signal based on the interactions with their surrounding agents, obstacles, and desired positions from different modules. The control protocol will continue to control agents to move until agents reach to target states 1 ( , ) d i p v of each agent. Further, as new mission arrives, multi-agents will move again through control scheme above.

Pattern matrix
Pattern matrix is the fundamental matrix to record the graphical information for formation. Furthermore, the pattern matrix can be formally represented by Any complex formation can be presented by an n n × pattern matrices. Furthermore, consider that a pattern matrix contains n agents, in which all agents are numbered from left to right and sequentially incremented for every row.  In an 8 8 × pattern matrices, a number '1' formation contains 17 agents. The pattern matrices of 1 F ("1") is denoted by Fig. 2(a). Further, the number of 1 F is shown in Fig. 2 Meanwhile, an appropriate dimension of pattern matrix should be considered to avoid the explosion for the complexity of the matrix. However, this problem is not the key issue for this paper, and it will not be further studied here.

Location matrix
Location matrix is introduced to record the location information in a formation. Firstly, a conversion function ( ) T N is defined to convert the number of agents in pattern matrix to the information in location matrix, specified as, where N is the number of each agent in pattern matrices. r and c are the row and column in location matrix, respectively. Meanwhile, a notation [ ] , N r c is utilized to simplify the conversion function ( ) T N , i.e., the agent of number N is located at rows r and columns c . For example, in Fig. 2(b), the agent of number 5 can be denoted by [ ] 5 2,3 . It means that agent 5 is located at row 2 and column 3 in location matrix.

Calculation of desired position
It is clear that the desired position of agent j can be computed by the actual position of leader 1 1 1 ( , , ) x y z = p through geometrical relations based on pattern matrix F and conversion function ( ) T N . Additionally, a relative coordinate system x oy ′ ′ (the altitude of z is constant) is shown in Fig. 3.
where j c and j r are the rows and columns in location matrices based on conversion function ( ) T N . Furthermore, the desired position ( , , ) where θ is the angle of velocity 1 v with x axis in global coordinate system xoy . j x′ and j y′ are the relative desired position.

Control protocol
Olfati-Saber [Olfati-Saber (2006)] proposed a distributed flocking algorithm, where a systematic method is provided for the construction of collective potentials for flocking. Meanwhile, according to the different constructions in collective potentials, those agents in flocking can be divided into three kinds. Firstly, some agents are composed of flocking members, called α -agent. Secondly, some are all kinds of obstacles, called β -agent. Thirdly, the others are virtual agents associated with respective object point, called γ -agent. A comparison of the distributed flocking algorithm and our method is shown in Fig. 4. In Fig. 4(a), all α -agents will arrive at the equilibrium point in the effect of potentials. β -agent will generate repulsion to make all agents get away from obstacles. Meanwhile, the γ -agent will generate navigation information to guide all agents to the object point. However, the distributed flocking algorithm is not capable of completing a complicated task for forming a specific formation from scatter to order. In Fig. 4(b), the method of this paper is specified as follows. Firstly, α -agent has the same function with Fig. 4(a). It can be used to remain a certain distance and avoid collision with other agents. Secondly, β -agent has the same function with Fig. 4(a), in which the repulsion will make all agents away from obstacles. However, the velocity direction of the leader will jitter continuously when near to obstacles. It is clear that the jitter will impose a significant effect on formation control. Consider that agents will prefer to avoid obstacles when reaching to the region of obstacles. Thus, a free-flying strategy is used to control all agents to pass through the region of obstacles, while, a formation control strategy is used to control all agents to form a desired formation when all agents are far away from obstacles. Thirdly, γ -agent is different from Fig. 4(a). It contains two functions: leader and followers. The γ -agent of leader is located at the global objective point, which will guide all α -agents to move to the objective collectively. The rest γ -agents of followers are located at their desired location. Thus, all agents will construct an intended formation under the whole virtual agents.

Formation control for the absence of obstacles
The control protocol i u of i th agent can be specified as, where i α u is the control feedback of i th agent from surrounding α -agents. And i γ u is the navigation information of i th agent from its corresponding γ -agent. As the Euclidean norm || z || is not differentiable at 0 z = . Thus, a new σ norm is defined to ensure the differentiability of potential at any point. || z σ || specifies as where ε is a positive parameter. Thus, for any point, the gradients || z ∇ σ || is defined as Hence, it is clear that || z σ || is differentiable at 0 z = . Meanwhile, a term of action functions ( ) α φ ⋅ is designed to ensure the smoothness about the potential of attraction and repulsion. The forces will disappear for outside the range of communication r and the forces are minimum for a desired distance d . ( ) α φ ⋅ is specified as, is a piecewise function, specified as is a smoothness function, and there exists ( ) 0 h z ρ′ = for any [1, ) z ∈ ∞ . Further, a smooth potential function ( ) V p is defined as where ( ) z α ψ is a smooth attractive/repulsive potential, specified as . As agent 1 is the leader, the other agents 2, , N  are followers. The γ -agent of leader is located at global object point. The leader will guide all followers to move to the desired destination. Similarly, the γ -agent of followers is located at their desired location, which will control all followers to form a desired formation. The navigation output γ i u can be described as where the former can be considered as an attractive force from agent i . And the latter is the damping force from velocities matching. 1 c γ and 2 c γ are positive parameters. As the navigation outputs are different from leader and followers, γ i u can be divided into two kinds: leader and followers. Leader will receive the global object point and velocity. 1 where γ p and γ v are global object point and velocity vectors, i.e., agent will reach to γ p with velocity γ v . The object points of follower are their respective desired locations, and desired velocities of followers are the velocity of leader. The desired locations and velocities of followers will be changed for the motion of leader. i γ p and i γ v are obtained by (25)

Free-flying strategy for the presence of obstacles
The obstacles avoidance should take precedence over formation control for the presence of obstacles. The free-flying strategy will be used to control all agents to pass through the region of obstacles. The control output o i u of obstacles can be described as where i α u is the same with formula (22). It is used to maintain a certain distance between every agent. i β u is the control output which is used to maintain a safe distance d ′ between agents and obstacles. The control output of neighbors of obstacles for agent i can be computed as . Agents will receive the information of global object point and velocity to pass through the region of obstacles preferentially. The navigation feedback o i γ u of each agent generated by global γ -agents can be computed as where i p and i v are the actual location and velocity vector. γ p and γ v are location and velocity vector of global object point.

Stability analysis
In order to analyze the stability of multi-agents, a coordinate system is established from the origin of leader. The position vector of agent i can be described by i r , and its velocity is A moving frame can be obtained on the basis of Eq. (3) in coordinate system above, denoted by, For the relative position vector ij i j = − r r r , the control output i u of agent i can be described as Generally, the Lyapunov function is selected as Take the time derivative to Eq. (31), we have, Φ is strictly positive for every non-zero , ,

Simulation and result analysis
This section designs two scenarios to verify the efficiency of our method. Firstly, a delta formation of 6 agents is designed in three dimensions for the absence of obstacles. Secondly, a delta formation of 28 agents is designed in two-dimensional plane for the presence of 7 obstacles.
(1) A delta formation in three dimensions (2) Fig. 5 represents the performance of delta formation for four time periods during simulation (0 s, 42 s, 84 s, and 105 s). Agents will form a delta formation on the basis of the strategy of formation control. Furthermore, the positions and velocities of agents are shown in Fig. 6(a) and Fig. 6(b) in each iteration. It is clear that six agents will keep a stabilized distance; while followers will remain the same altitudes and velocity with leader rapidly.  The velocity oscillations are shown at the beginning of simulation in Fig. 6(b). The reason is represented as follows. Considering the control scheme of multi-agent system, all agents are constantly adjusting their motions to complete formation control based on relatively simple interactions among individuals or environments until the system reaches a steady state. The control scheme with interactions is low cost and simple to realize, and possesses some practical value in engineering. Meanwhile, the velocities become smooth and steady as time passes, which also verify the stability of this control scheme.
(3) A delta formation and obstacles avoidance This scenario contains 28 agents and 7 obstacles. All agents will form a delta formation and avoid obstacles through the strategy of self-adaptation flocking algorithm. The positions and radius of all obstacles are specified as   Fig. 8(a) and Fig. 8(b). In Fig. 7, agents will form a desired formation from the initial position during iteration 0 to 280; while agents will avoid obstacles based on self-adaptation flocking algorithm after iteration 280 to 480; finally, agents will form the desired formation again after passing through obstacles. Further, all agents will form a desired formation on the basis of the strategy of formation control quickly with a stabilized distance and velocity when all agents are far away from obstacles; otherwise, agents will freely fly to pass through the area of obstacles without any collisions. Furthermore, the traditional potential method is compared with self-adaptation flocking algorithm under the same conditions. One agent is trapped into the region of obstacles through traditional potential method, shown in Fig. 9. The main reason is probably that the repulsion force of β -agent takes the attractive force of γ -agent out. Additionally, the velocity error ( ) in each iteration of our method and traditional potential method is depicted in Fig. 10. It is clear that all agents will complete velocities matching through self-adaptation flocking algorithm quickly when agents are far away from obstacles; while potential method will be significantly affected by obstacles, in which all agents cannot complete velocities matching.

Conclusion
The methods of matrices are proposed to describe complex formation for multi-agents. Firstly, pattern matrices are used to record the graphical information for a desired formation. Secondly, location matrix is used to record the desired location for each agent. Furthermore, the desired positions of whole agents can be obtained through those two matrices in a global coordinate system. In order to form a desired formation and avoid obstacles, a self-adaptation flocking algorithm is proposed for multi-agents. Agents will form a desired formation through the strategy of formation control when far away from obstacles; and agents will freely fly when near by the area of obstacles.