Nonlinear Uncertainties Canceling in Multi-Agent Systems Enabled by Cooperative Adaptation

This paper dealswith uncertainties problem inmulti-agent systemswith novel cooperative adaptation approach. Since uncertainties in multi-agent systems are interconnected, local agent often faces uncertainties not only from itself but also from neighbors. The proposed approach is that a local agent estimates uncertainties from itself and neighboring agents and then changes control strategy. The uncertainties or the equivalences of neighbors can be estimated based on their available outputs; thus, the local agent can adapt to them to cancel out these effects. Stability analysis is also derived that characterizes the transient and steady state performance of multi-agent system. The simulation presents the details of the proposed cooperative adaptation mechanism by compared typical cooperative control.


Introduction
The cooperative control in multi-agent system has attracted compelling attention in recent years from various applications including scheduling satellite formation, cooperative unmanned air vehicles, connected autonomous vehicles, and multiple underwater vehicle coordination.In these applications, the agents are required to collaborate with each other in order to finish cooperative mission.Consensus in multiagent systems is a typical example that agents have common interest and then make negotiations to reach consensus by appropriate algorithmic methods.Consensus problems with various dynamic systems have already been successfully addressed in literature.The consensus in multi-agent systems with single-integrator and double-integrator kinematics is studied in [1][2][3].The general linear dynamics of consensus in multi-agent systems is also discussed in [4][5][6][7][8][9][10].In [11][12][13][14][15][16], the authors further extended to nonlinear dynamics for multiagent systems.
In practical applications, the models of agents are difficult to obtain because of sensing or measurement limitation.Also, disturbances often exist in interconnected network systems, such as channel noise and communication noise.Thus, it is necessary to consider uncertainty effects in multi-agent systems.The typical direct and indirect adaptive control [17] are two nonlinear schemes explicitly designed for signal system uncertainties.In [18], the model reference adaptive control approach is used to ensure asymptotic tracking of a defined reference model for a class of multi-agent systems with constant unknown parameters.Utilizing the universal approximation capability of artificial neural network [19], adaptive control could deal with nonlinear functions instead of linear parameters in multi-agent systems, and this leads to the neural network controller design.Consensus problem of second-order systems with Lipschitz nonlinearity is addressed [16].The uncertainties of nonlinear multi-agent systems under jointly connected directed switching networks are also addressed in [20,21].The results in [9,18] deal with high-order system with Lipschitz nonlinearity.In these literatures, the results are restricted to local stability and certain type of network structure.
This paper proposed a novel cooperative adaptation approach to deal with uncertainties issue of network consensus.The cooperative law drives agents to collaborate and then cooperative adaptation law eliminates uncertainties from network.The idea is that the local agent adjusts his own control strategies based on the uncertainty estimation of itself 2 Journal of Control Science and Engineering and neighbors.The uncertainties from other agents can be treated as external disturbances and are not available to the local agent.Thus, other agents uncertainties can be estimated based on their available output and then local agent cancels out these uncertainties by adapting to them.This paper also provided transient and steady state performance analysis in the presence of uncertainties for multi-agent systems.
The article is organized as follows.The problem formulation is given in Section 2. The adaptive cooperation control scheme is introduced in Section 3. In Section 4, the stability analysis and performance bounds for multi-agent systems are presented in Section 4. The simulation results are presented in Section 5, while Section 6 concludes this paper.

Problem Formulation
There are  agents moving in  dimensional Euclidean space, where the dynamics of each agent is described by where   ∈ R  is the state vector, such as position and velocity vector of agent ,   ∈ R  are the control input, and   : R  × R  ×R  → R  represents the unknown, nonlinear, and timevarying uncertainties.
The objective is to design decentralized control inputs that ensure multi-agent systems with uncertainties achieve consensus.Consensus in this paper means all agents converge to common attitude or speed.Each agent is subjected to individual uncertainties, such as dynamics uncertainty, affected from neighborhood agents.The decentralized control input plays the role not only cooperating but also canceling uncertainty among agents.The network connectivity has a fairly general assumption that creates a spanning tree in the network connection.The uncertainties are restricted to being Lipschitz and allow less restriction on structure of nonlinearity.
The connection among agents are specified by a graph G = (V, E) which consists of a set of vertices denoted by V = {1, 2, . . ., } and a set of edges denoted by E ⊆ {(, ) : ,  ∈ V,  ̸ = }.A vertex represents an agent, and each edge represents a connection.The adjacency matrix A with elements   from the graph denotes the connections, where   = 1 means a path from agent  to agent ; otherwise   = 0.The Laplacian matrix L is defined as   = ∑

Uncertainties Canceling Algorithm
In this section, we introduce a novel cooperative adaptation law to drive multi-agent systems as (1) achieve global objective,   toward neighbor agents   .The algorithm first deals with the agent's collaboration by the cooperative law that is based on local neighbors' relative information.The algorithm further handles uncertainties issue among network through cooperative adaptation law.The overall control law is represented as below: where    is cooperative control law and    is cooperative adaptation law.First, we introduced the state transformation to describe the relation between individual agent and network as follows: The network dynamics between local agent and neighbors is described as The overall network transformation follows Cooperative Law.In order to achieve the network consensus, the cooperative control law design is moving to the average state of neighbors.In other words, it uses neighbors' relative information as feedback and is presented as where the selection of  will make sure to stabilize the overall network system that is described as where , and   =  ⊗ 1  .Thus,   is Hurwitz matrix by  selection.
Cooperative Adaptation Law.We further design the decentralized cooperative adaptation law to deal with uncertainty issue among network.This disturbance rejection includes state predictor, adaptive law, and control as below.Since uncertainty for dynamical systems is made in real time, all uncertainties from network can be estimated based on their available output.Hence, a local agent can adjust its own control law based on the estimation of network uncertainty.Thus, the local agent estimates these uncertainties through transferred dynamics.With network dynamics from ( 9), we consider the following state predictor: where ξ ∈ R  is the estimated state vector of  in ( 9) and σ is the estimated uncertainty of .These parameters vectors are updated by adaptive law below.Fast adaptation mechanism is applied in here as a highgain feedback for state predictor.It is noted that both state predictor and adaptation work together and there is no time-delay or at most one integration time-step in the loop.Here, the predicted state, ξ , is used to generate the adaptive parameter, σ , based on the network error dynamics.The piece-wise constant adaptive law is used in here.In order to drive ξ to track ξ.The update law is given by σ () = −Φ −1 () e    ξ () ,  ∈ [, ( + 1) ] (11) where Φ() = ∫  0    (−) ,  > 0 is the adaptation step size, and ξ = ξ−.The adaptive law drives ξ to be small and as a result, σ will approach .The information of σ will be used in the control law design.The error dynamics for network system is described as To prevent aggressive signals in real application, a lowpass filtering mechanism is added in control law, which filters away aggressive signals and recovers robustness.The adaptive control law design is where () is low-pass filter to filter the control signal.The introduction of low-pass filter at this point in the control loop allows us to decouple control and estimation by canceling the overall network uncertainty.Furthermore, the bandwidth of filter can be chosen to attenuate frequencies based on the variation of uncertainties.The closed-loop network system from ( 1) and ( 13) becomes ξ =    −  ()   σ +    (14 Remark 4. The objective of cooperative adaptation law is to make the adaptive parameter σ() approach the network disturbance ().The effects of the control law   () cancel the effects of () and as a result ξ() in (12) will become very small.Thus, the agents can achieve consensus without uncertainties.

Network Stability Analysis
The analysis first verifies the network stability by introducing a reference network system.The reference system will asymptotically track desired system when the bandwidth of low-pass filter is large enough.The network convergence analysis also proved the stability conditions.Further, the real network system is proven to track the reference system and its stability condition is provided.Finally, the estimation errors between real network and predict network system are shown to be bounded and the uncertainties canceling is achieved by decreasing the size of time-step.

Nonadaptive Reference System.
We consider a reference network dynamics for (9) as where and the reference control law The closed-loop reference network system from ( 15) and ( 16) becomes Lemma 5.For the closed-loop network reference system in (15) and ( 16) subject to the L 1 gain upper bound where Then where   is introduced in ( 18) and ( 19) where The stability condition in (18), together with (28), implies that ‖    ‖ ∞ <   , which clearly contradicts assumption in (24) since there is no   to make assumption true.This proves (21).From ( 16), the network control law is which proves (22) and concludes the proof.
Lemma 6.Consider the reference system in (17).If the bandwidth of low-pass filter () is large enough, the control law in (16) can make   asymptotically track the desired network system described by where Proof.For a given network, if the bandwidth of low-pass filter () = /( + ) can be chosen large enough, the disturbance effect in (17) will be canceled and (17) become as This further implies We further obtain that lim Thus, the reference system will track the desired network system, which concludes the proof.

Network Convergence Analysis.
Since network G is connected, it follows from Assumption 3 that 0 is a simple eigenvalue of L and the other eigenvalues have positive real part.Thus, there exists a nonsingular matrix  such that  −1 L is in the Jordan canonical form [9].Because the left and right eigenvectors of L corresponding to zero eigenvalue are  and 1, respectively, the following can be chosen: where  ∈ C ×(−1) and  ∈ C ×(−1) such that where Δ ∈ C ×(−1) is an upper-triangular matrix, where the diagonal entries are the nonzero eigenvalues of L. Let   = ( −1 ⊗   )  .Then, (31) can be presented as Lemma 7. Suppose Assumption 3 holds.The distributed control law in (16) with agents in (15) solves consensus problem if and only if all the matrices  −   ,  = 2, . . ., , are Hurwitz, where   are the nonzero eigenvalues of Laplacian matrix L.
Proof.The elements of the state matrix in (37) are either block diagonal or block upper-triangular.Thus,   ,  = 2, . . ., , converge asymptotically to zero if and only if the  − 1 subsystems along the diagonal as are asymptotically stable.The stability of closed-loop system in (32) and ( 31) is equal to the system in (37) that converges asymptotically to zero.The system consensus is achieved.This concludes the proof.
Proof.The desired network dynamics (31) can transform as where   = L  .The solution of (40) with Lemma 7 can be obtained as From Lemma 7,   ⊗  − Δ ⊗  is Hurwitz.Thus, Thus, the final consensus value is lim This concludes the proof.

Network Transient Performance.
We discussed the relation between reference network system and desired network system and then perform the network convergence analysis in last section.We further prove the real network system can track reference network system in this section.Before this proof, we will establish the error bounds between the network state predictor and real network system and between the adaptive law and the signal it estimates.
Lemma 9. Given the network system in ( 9) with L 1 adaptive control law in ( 13) and the reference system in (15) with (16) subject to stability conditions in (18) and (19), if ‖  0 ‖ <   and we choose  in (11) and low-pass filter C(s) in (13) The solution of network error dynamics ( 12) over the interval [, ( + 1)] is described as where Φ() =     , ( + 1) <   and  is a dummy variable.At  = ( + 1), it becomes ξ (( + 1) ) = Φ () ξ () We can choose σ such that its effects on the system derive x to zero at the next time-step.Then, we have where  1 ,  2 , and  3 are defined in (50), (51), and (52), respectively.Then, for all  ∈ [0,   ], following from (44), we have The results clearly contradict assumption in (53) since there is no   to make assumption true.This proves (46).Next, we will drive the bounds for the estimation error of adaptive law, σ = σ − , which are used to further obtain the bounds of ‖ −   ‖.We derive the bound for σ (, ) first and then get From ( 14), we have To establish the bound for this signal, we need to derive a bound for ().Using ( 11) and ( 53) for all  <  1 , then we have Using (64), we can now write the Then, control signal upper bound Note that the adaptive law of ( 11) is designed such that For the time  ∈ [( − 1), ], we also have Following from the definition of adaptive law, we obtain Thus, (70) and (71) give us which imply there exist a time,   ∈ [( − 1), ], such that Let σ = σ − ; then we have Next, we derive the bounds of ‖  −    ‖.From network control law in ( 8) and ( 13), we have From (29), we have Thus, apply (76) and ( 77 The solution of (80) is Substituting ( 79) and ( 55) to (81), we further get The stability condition in (18), together with (84), implies that ‖ −   ‖ ∞ <   , which clearly contradicts assumption in (55) since there is no   to make assumption true.This proves (47).If we choose  = 0 and the bandwidth of low-pass filters () large enough, (84) with the condition become ‖   ‖ ∞ <   .Thus, it always exists   to satisfy (84).This completes the proof of Lemma 9.
Lemma 10.The error bounds from Lemma 9 depend on the time-step .When T approaches 0, the error approaches 0 as well.Proof.The inside of integration in (61) is a bounded function; so when  → 0, then  ξ → 0. Since (61) holds, the first term in (74) goes to 0 as  → 0 and the second term clearly goes to 0 as well.Thus,   → 0 when  → 0. The last statement is   .Because (74) holds, we can clearly see from (48) that when  → 0,   → 0. This completes the proof.

Simulation
The simulation will show the details of proposed adaptive cooperation algorithm by compared with typical cooperative control.The network systems are included uncertainties and disturbances in these examples.The agent dynamics under consideration is a vertical motion of a quadrotor.There are 10 quadrotors that try to maintain in the specified altitude and velocity for cooperation observation in simulation scenarios.
The dynamics described by a second-order state space as where   = [    ]  are vertical position and velocity,  = 1 is the mass of quadrotor,   is the thrust generated by rotors, and   (  , ) is the force produced by wind on quadrotor.
The first example shows the cooperative control algorithm from (8).The integration step in simulation is set as  = 10 −2 seconds; the initial altitude and velocity (0)  are chosen randomly within the range 0 ∼ 10 and −1 ∼ 1/ individually.The control gain is chosen as  = [1 1].The communication links are considered as undirected ring network topology as Figure 1.The Laplacian matrix is given by The eigenvalues of L only have one zero eigenvalues and other eigenvalues are positive that means Assumption 3 is satisfied.The uncertainties considered in agent dynamics are globally Lipschitz as Although the cooperative control tries to drive quadrotors to common position, the uncertainties affect the effectiveness of algorithm (8).The cooperative control signals from Figure 3 show the fluctuation and diverge gradually.Thus, the consensus from Figure 2 cannot be achieved by cooperative control (8) for the group of quadrotors.
The second example shows the proposed adaptive cooperation algorithm from ( 8) and (13).The integration step in simulation is set as  = 10 −2 seconds.The initial altitude and velocity (0)  are chosen randomly within the range 0 ∼ 10 and −1 ∼ 1/ individually.The filter of adaptive cooperation controller for agents is chosen as () = 10/( + 10).The communication links are also considered as undirected ring network topology as Figure 1.
Figure 4 shows the consensus achieved with the same position and velocity for the group of quadrotors, and the uncertainties in network are also canceled in Figure 6. Figure 5 shows the adaptive cooperation control signal of quadrotors.Although network systems create uncertainties, the transient performance is still maintained well by (8) and (13).The simulation results with Figures 4 and 5 demonstrate Lemmas 6-8. Figure 6 shows the mismatched dynamics that means the state predictor tracks the real state well and adapts to uncertainty quickly.The results in Figure 6

Conclusion
This paper presents a novel cooperative adaptation approach to deal with uncertainty of multi-agent systems.The typical cooperative control was used to achieve collaboration in multi-agent systems but it also causes interconnected uncertainties among network.Then, the cooperative adaptation approach was proposed to cancel uncertainties in multiagent systems.The cooperative adaptation algorithm features as fast adaptation scheme and can be treated as high-gain approach.To prevent aggressive changing in real control  signal, a low-pass filtering mechanism is added in control law, which filters away aggressive signals and recovers robustness.The stability condition and tracking performance for closedloop network system are presented.The estimation errors between predicted states and real state are proved to be arbitrarily small.Then, the network stability and performance bounds between real system and reference system can be made arbitrarily small by decreasing the integration timestep.The simulation results show the details of proposed scheme.This approach can be further extended to deal with uncertainties issue in multi-unmanned vehicle systems.
thus demonstrate Lemma 10.Further, the results in Figures4-6