A Suboptimality Approach to Distributed H2 Control by Dynamic Output Feedback

This paper deals with suboptimal distributed H2 control by dynamic output feedback for homogeneous linear multi-agent systems. Given a linear multi-agent system, together with an associated H2 cost functional, the objective is to design dynamic output feedback protocols that guarantee the associated cost to be smaller than an a priori given upper bound while synchronizing the controlled network. A design method is provided to compute such protocols. The computation of the two local gains in these protocols involves two Riccati inequalities, each of dimension equal to the dimension of the state space of the agents. The largest and smallest nonzero eigenvalue of the Laplacian matrix of the network graph are also used in the computation of one of the two local gains.A simulation example is provided to illustrate the performance of the proposed protocols.


Introduction
The design of distributed protocols for networked multiagent systems has been one of the most active research topics in the field of systems and control over the last two decades, see e.g. [22] or [6]. This is partly due to the broad range of applications of multi-agent systems, e.g. smart grids [7], formation control [21], [31], and intelligent transportation systems [3]. One of the challenging problems in the context of linear multi-agent systems is the problem of developing distributed protocols to minimize given quadratic cost criteria while the agents reach a common goal, e.g., synchronization. Due to the structural constraints that are imposed on the control laws by the communication topology, such optimal control problems are difficult to solve. These structural constraints make distributed optimal control problems non-convex, and it is unclear under what conditions optimal solutions exist in general.
In the existing literature, many efforts have been devoted to addressing distributed linear quadratic optimal control problems. In [4], suboptimal distributed stabilizing controllers were computed to stabilize multi-agent networks with identical agent dynamics subject to a global linear quadratic cost functional. For a network of agents with single integrator dynamics, an explicit expression for the optimal gain was given in [5], see also [13]. In [19] and [33], a distributed linear quadratic control problem was dealt with using an inverse optimality approach. This approach was further employed in [20] to design reduced order controllers. Recently, also in [12], the suboptimal distributed LQ problem was considered. In parallel to the above, much work has been put into the problem of distributed H 2 optimal control. Given a particular global H 2 cost functional, [16] and [15] proposed suboptimal distributed stabilizing protocols involving static state feedback for multi-agent systems with undirected graphs. Later on, in [29] these results were generalized to directed graphs. For a given H 2 cost criterion that penalizes the weighted differences between the outputs of the communicating agents, in [11] a suboptimal distributed synchronizing protocol based on static relative state feedback was established.
In the past, also the design of structured controllers for large-scale systems has attracted much attention. In [23], the notion of quadratic invariance was adopted to develop decentralized controllers that minimize the performance of the feedback system with constraints on the controller structure. In [17], the so called alternating direction method of multipliers was adopted to design sparse feedback gains that minimize an H 2 performance. In [8], conditions were provided under which, for a given optimal centralized controller, a suboptimal distributed controller exists so that the resulting closed loop state and input trajectories are close in a certain sense.
The distributed H 2 optimal control problem for multiagent systems by dynamic output feedback is to find an optimal distributed dynamic protocol that achieves synchronization for the controlled network and that minimizes the H 2 cost functional. This problem, however, is a non-convex optimization problem, and therefore it is unclear whether such optimal protocol exists, or whether a closed form solution can be given. Therefore, in the present paper, we look at an alternative version of this problem that requires only suboptimality. More precisely, we extend our preliminary results from [11] on static relative state feedback to the general case of dynamic protocols using relative measurement outputs. The main contributions of this paper are the following.

1)
We solve the open problem of finding, for a single continuous-time linear system, a separation principle based H 2 suboptimal dynamic output feedback controller. This result extends the recent result in [9] on the separation principle in suboptimal H 2 control for discrete-time systems.
2) Based on the above result, we provide a method for computing H 2 suboptimal distributed dynamic output feedback protocols for linear multi-agent systems.
The outline of this paper is as follows. In Section 2, we will provide some notation and graph theory used throughout this paper. In Section 3, we will formulate the suboptimal distributed H 2 control problem by dynamic output feedback for linear multi-agent systems. In order to solve this problem, in Section 4, we will first study suboptimal H 2 control by dynamic output feedback for a single linear system. In Section 5 we will then treat the problem introduced in Section 3. To illustrate our method, a simulation example is provided in Section 6. Finally, Section 7 concludes this paper.

Notation
In this paper, the field of real numbers is denoted by R and the space of n dimensional real vectors is denoted by R n . We denote by 1 n ∈ R n the vector with all its entries equal to 1 and we denote by I n the identity matrix of dimension n × n. For a symmetric matrix P , we denote P > 0 if P is positive definite and P < 0 if P is negative definite. The trace of a square matrix A is denoted by tr(A). A matrix is called Hurwitz if all its eigenvalues have negative real parts. We denote by diag(d 1 , d 2 , . . . , d n ) the n × n diagonal matrix with d 1 , d 2 , . . . , d n on the diagonal. For given matrices M 1 , M 2 , . . . , M n , we denote by blockdiag(M 1 , M 2 , . . . , M n ) the block diagonal matrix with diagonal blocks M i . The Kronecker product of two matrices A and B is denoted by A ⊗ B.

Graph Theory
A directed weighted graph is denoted by G = (V, E, A) with node set V = {1, 2, . . . , N } and edge set E = {e 1 , e 2 , . . . , e M } satisfying E ⊂ V × V, and where A = [a ij ] is the adjacency matrix with nonnegative elements a ij , called the edge weights. If (i, j) ∈ E we have a ji > 0. If (i, j) ∈ E we have a ji = 0.
A graph is called undirected if a ij = a ji for all i, j. It is called simple if a ii = 0 for all i. A simple undirected graph is called connected if for each pair of nodes i and j there exists a path from i to j. Given a simple undirected weighted graph G, the degree matrix of G is the diagonal matrix, given by The Laplacian matrix is defined as L := D − A. The Laplacian matrix of an undirected graph is symmetric and has only real nonnegative eigenvalues. A simple undirected weighted graph is connected if and only if its Laplacian matrix L has a simple eigenvalue at 0. In that case there exists an orthogonal matrix U such that U ⊤ LU = Λ = diag(0, λ 2 , . . . , λ N ) with 0 = λ 1 < λ 2 ≤ · · · ≤ λ N . Throughout this paper, it will be a standing assumption that the communication among the agents of the network is represented by a connected, simple undirected weighted graph.
A simple undirected weighted graph obviously has an even number of edges M . Define K := 1 2 M . For such graph, an associated incidence matrix R ∈ R N ×K is defined as a matrix R = (r 1 , r 2 , . . . , r K ) with columns r k ∈ R N . Each column r k corresponds to exactly one pair of edges e k = {(i, j), (j, i)}, and the ith and jth entry of r k are equal to ±1, while they do not take the same value. The remaining entries of e k are equal to 0. We also define the matrix as the K × K diagonal matrix, where w k is the weight on each of the edges in e k for k = 1, 2, . . . , K. The relation between the Laplacian matrix and the incidence matrix is captured by L = RW R ⊤ [18].

Problem Formulation
In this paper, we consider a homogeneous multi-agent system consisting of N identical agents, where the underlying network graph is a connected, simple undirected weighted graph with associated adjacency matrix A and Laplacian matrix L. The dynamics of the ith agent is represented by a finite-dimensional linear time-invariant systemẋ where x i ∈ R n is the state, u i ∈ R m is the coupling input, d i ∈ R q is an unknown external disturbance, y i ∈ R r is the measured output and z i ∈ R p is the output to be controlled. The matrices A, B, C 1 , D 1 , C 2 , D 2 and E are of compatible dimensions. Throughout this paper we assume that the pair (A, B) is stabilizable and the pair (C 1 , A) is detectable. The agents (2) are to be interconnected by 2 means of a dynamic output feedback protocol. Following [28] and [32], we consider observer based dynamic protocols of the forṁ where G ∈ R n×r and F ∈ R m×n are local gains to be designed. We briefly explain the structure of this protocol. Each local controller of the protocol (3) observes the weighted sum of the relative input signals N j=1 a ij (u i −u j ) and the weighted sum of the disagreements between the measured output signals N j=1 a ij (y i − y j ). The first equation in (3) in fact represents an asymptotic observer for the weighted sum of the relative states of agent i, and the state of this observer is an estimate of this value. Note that, for the error . An estimate of the weighted sum of the relative states of each agent is then fed back to this agent using a static gain.
Denote by x = (x ⊤ 1 , x ⊤ 2 , . . . , x ⊤ N ) ⊤ the aggregate state vector and likewise define u, y, z, d and w. The multiagent system (2) can then be written in compact form aṡ and the dynamic protocol (3) is represented bẏ By interconnecting the network (4) using the dynamic protocol (5), we obtain the controlled network Foremost, we want the dynamic protocol (5) to achieve synchronization for the network.
Definition 1. The protocol (5) is said to synchronize the network if, whenever the external disturbances of all agents are equal to zero, i.e. d = 0, we have The distributed H 2 optimal control problem by dynamic output feedback is to minimize a given global H 2 cost functional over all dynamic protocols of the form (5) that achieve synchronization for the controlled network. In the context of distributed control for multi-agent systems, we are interested in the differences of the state and output values of the agents in the controlled network, see e.g. [14], [18]. Note that these differences are captured by the incidence matrix R of the underlying graph. Therefore, we introduce a new output variable as ζ = (W where W is the weight matrix of the underlying graph, as defined in (1). Thus, the output ζ is the vector of weighted disagreements between the outputs of the agents, in which the weights are given by the square roots of the edge weights connecting these agents. Subsequently, we consider the network (6) with this new output: Denote The impulse response matrix from the external disturbance d to the output ζ is then equal to Next, the associated global H 2 cost functional is defined to be the squared L 2 -norm of the closed loop impulse response, and is given by The distributed H 2 optimal control problem by dynamic output feedback is the problem of minimizing (10) over all dynamic protocols of the form (5) that achieve synchronization for the network. Unfortunately, due to the particular form of the protocol (5), this optimization problem is, in general, non-convex and difficult to solve, and a closed form solution has not been provided in the literature up to now. Therefore, instead of trying to find an optimal solution, in this paper we will address a suboptimality version of the problem. More specifically, we will design synchronizing dynamic protocols (5) that guarantee the associated cost (10) to be smaller than an a priori given upper bound. More concretely, the problem that we will address is the following: Problem 1. Let γ > 0 be a given tolerance. Design local gains F ∈ R m×n and G ∈ R n×r such that the dynamic protocol (5) achieves J(F, G) < γ and synchronizes the network.
Before we address Problem 1, we will first study the suboptimal H 2 control problem by dynamic output feedback for a single linear system. In that way, we will collect the required preliminary results to treat the actual suboptimal distributed H 2 control problem for multi-agent systems.

Suboptimal H 2 Control by Dynamic Output Feedback for Linear Systems
In this section, we will discuss the suboptimal H 2 control problem by dynamic output feedback for a single linear system. This problem has been dealt with before, see e.g. [25], [24], [26] or [9]. In particular, in [9], the separation principle for suboptimal H 2 control for discrete-time linear systems was established. Here, we will establish the analogue of that result for the continuous-time case.
Consider the linear systeṁ x =Āx +Bu +Ēd, where x ∈ R n is the state, u ∈ R m the control input, d ∈ R q an unknown external disturbance, y ∈ R r the measured output, and z ∈ R p the output to be controlled. The matricesĀ,B,C 1 ,D 1 ,C 2 ,D 2 andĒ have compatible dimensions. In this section, we assume that the pair (Ā,B) is stabilizable and that the pair (C 1 ,Ā) is detectable. Moreover, we consider dynamic output feedback controllers of the formẇ =Āw +Bu + G y −C 1 w , where w ∈ R n is the state of the controller, and F ∈ R m×n and G ∈ R n×r are gain matrices to be designed. By interconnecting the controller (12) and the system (11), we obtain the controlled system Then the impulse response matrix from the disturbance d to the output z is given by T F,G (t) = C a e Aat E a . Next, we introduce the associated H 2 cost functional, given by We are interested in the problem of finding a controller of the form (12) such that the controlled system (13) is internally stable and the associated cost (14) is smaller than an a priori given upper bound.
Before we proceed, we first review a well-known result that provides necessary and sufficient conditions such that a closed loop system is H 2 suboptimal, see e.g. [25, Proposition 3.13].
(ii) there exists X a > 0 such that The following lemma is an extension of Theorem 6 in [9]. It provides conditions under which the controller (12) with gain matrices F and G = QC ⊤ 1 is suboptimal for the continuous-time system (11), where Q is a particular real symmetric solution of a given Riccati inequality. The result shows that the separation principle is also applicable in the context of suboptimal H 2 control for continuoustime systems.
Lemma 3. Let γ > 0 be a given tolerance. Assume that If, moreover, the inequality holds, then the controller (12) with the gains F and G = QC ⊤ 1 yields an internally stable closed loop system (13), and it is suboptimal, i.e. J(F, G) < γ.
Proof. Let Q > 0 satisfy (18) and gain matrix F be given. Note that (19) is equivalent to According to cases (ii) and (iii) in Proposition 2, there exists P > 0 satisfying (17) and (20) if and only if there exists ∆ > 0 satisfying and On the other hand, by applying the state transformation w e = 0 I n −I n I n x w .

4
The system (13) then becomes (23) Clearly, the system (13) is internally stable if and only if A +BF andĀ − GC 1 are Hurwitz. Thus, what remains to show is that the controller (12) with the gains F and G = QC ⊤ 1 internally stabilizes the system (11) and that J(F, G) < γ.
We are now ready to deal with the suboptimal distributed H 2 control problem by dynamic output feedback for multi-agent systems.

Suboptimal Distributed H 2 Control for Multi-Agent Systems by Dynamic Output Feedback
In this section, we will address Problem 1. For the multiagent system (2), we will establish a design method for local gains F and G such that the protocol (3) achieves J(F, G) < γ and synchronizes the network (6).
Let U be an orthogonal matrix such that U ⊤ LU = Λ = diag(0, λ 2 , . . . , λ N ) with 0 = λ 1 < λ 2 ≤ · · · ≤ λ N the eigenvalues of the Laplacian matrix. We apply the state transformation Then the controlled network (6) with the associated output (8) is also represented by Denotē Obviously, the impulse response matrix T F,G (t) given by (9) is then equal toC e eĀ etĒ e . In order to proceed, we now introduce the N −1 auxiliary linear systemsξ and associated dynamic output feedback controllerṡ with gain matrices F and G. By interconnecting (34) and (33), we obtain the N − 1 closed loop systems for i = 2, 3, . . . , N . The impulse response matrix of (35) from the disturbance δ i to the output η i is equal to Furthermore, for each system (33) the associated H 2 cost functional is given by (37) Then we have the following lemma: Lemma 5. Let F ∈ R m×n and G ∈ R n×r . Then the dynamic protocol (3) with gain matrices F and G achieves synchronization for the network (6) if and only if for each i = 2, 3, . . . , N the controller (34) with gain matrices F and G internally stabilizes the system (33). Moreover, we have (39) We now analyze the matrix functionC e eĀ e tẼ e appearing in (39). By applying suitable permutations of the blocks appearing in the matricesC e ,Ẽ e and A e , it is straightforward to show thatC e eĀ e tẼ e = blockdiag 0, C 2 e A2t E 2 , . . . , C N e AN t E N , where It is easily seen that for i = 2, 3, . . . , N the systems (A i , E i , C i ) and (Ā i ,Ē i ,C i ) are isomorphic. Hence they have the same impulse response T i,F,G (t), which is given by (36), see e.g., [27,Theorem 3.10]. As a consequence we obtain thatC e eĀ e tẼ e = blockdiag (0, T 2,F,G (t), . . . , T N,F,G (t)) . Thus we find that The claim (38) then follows immediately.
By applying Lemma 5, we have transformed the suboptimal distributed H 2 control problem by dynamic output feedback for the multi-agent network (6) into suboptimal H 2 control problems for the N − 1 linear systems (33) using controllers (34) with the same gain matrices F and G. Next, we establish conditions under which the N − 1 systems (33) are internally stabilized by their corresponding controllers (34) for i = 2, 3, . . . , N , while achieving N i=2 J i (F, G) < γ. Lemma 6. Let γ > 0 be a given tolerance. Assume that For i = 2, 3, . . . , N , let F , P i > 0, and Q > 0 be such that the inequalities hold. Then for each i = 2, 3, . . . , N , the controller (34) with gain matrices F and G = QC ⊤ 1 internally stabilizes the system (33), and, moreover, Proof. By (42), for ǫ i > 0 sufficiently small, we have Lemma 3, it follows that the controller (34) internally stabilizes the system (33) and Again, we note that the four conditions D 1 E ⊤ = 0, D ⊤ 2 C 2 = 0, D 1 D ⊤ 1 = I r and D ⊤ 2 D 2 = I m are made here to simplify notation, and can be replaced by the regularity conditions D 1 D ⊤ 1 > 0 and D ⊤ 2 D 2 > 0 alone. By combining Lemma 5 and Lemma 6 we have established sufficient conditions for given gain matrices F and G to synchronize the network (6) and to be suboptimal, i.e. J(F, G) < γ. In fact, G is taken to be equal to QC ⊤ 1 , with Q > 0 a solution to the Riccati inequality (41). However, no design method has yet been provided to compute a suitable matrix F . In the following theorem, we will establish a design method for computing such gain matrix F . Together with G given above, this will lead to a distributed suboptimal protocol for multi-agent system (2) with associated cost functional (10).
Theorem 7. Let γ > 0 be a given tolerance. Assume that Let c be any real number such that 0 < c < 2 λ 2 N . We distinguish two cases: then there exists P > 0 satisfying then there exists P > 0 satisfying In both cases, if in addition P and Q satisfy Then the protocol (3) with F := −cB ⊤ P and G := QC ⊤ 1 synchronizes the network (6) and it is suboptimal, i.e. J(F, G) < γ.
Proof. We will only provide the proof for case (i) above.
Obviously, the smaller S(P, Q), the smaller the feasible upper bound γ. It can be shown that, unfortunately, the problem of minimizing S(P, Q) over all P, Q > 0 that satisfy (43) and (45) is a nonconvex optimization problem. However, since smaller Q leads to smaller tr C 2 QC ⊤ 2 and smaller P and Q leads to smaller tr C 1 QP QC ⊤ 1 and, consequently, smaller feasible γ, we could therefore try to find P and Q as small as possible. In fact, one can find Q = Q(ǫ) > 0 to (43) by solving with ǫ > 0 arbitrary. By using a standard argument, it can be shown that Q(ǫ) decreases as ǫ decreases, so ǫ should be taken close to 0 in order to get small Q. Similarly, one can find P = P (c, σ) > 0 satisfying (45) by solving where c is chosen as in (44) and σ > 0 arbitrary. Again, it can be shown that P (c, σ) decreases with decreasing σ and c. Therefore, small P is obtained by choosing σ > 0 close to 0 and c = 2 Similarly, if c satisfies (46) corresponding to case (ii), it can be shown that if we choose ǫ > 0 and σ > 0 very close to 0 and c > 0 very close to 2 λ 2 2 +λ2λN +λ 2 N , we find small solutions to the Riccati inequalities (43) and (47) in the sense as explained above for case (i).
Remark 9. In Theorem 7, exact knowledge of the largest and the smallest nonzero eigenvalue of the Laplacian matrix is used to compute the local control gains F and G. We want to remark that our results can be extended to the case that only lower and upper bounds for these eigenvalues are known. In the literature, algorithms are given to estimate λ 2 in a distributed way, yielding lower and upper bounds, see e.g. [2]. Also, an upper bound for λ N can be obtained in terms of the maximal node degree of the graph, see e.g. [1]. Using these lower and upper bounds on the largest and the smallest nonzero eigenvalue of the Laplacian matrix, results similar to Theorem 7 can be formulated, see e.g., [12] or [10].

Simulation Example
In this section, we will give a simulation example to illustrate our design method. Consider a network of N = 6 identical agents with dynamics (2), where A = −2 2 −1 1 , The pair (A, B) is stabilizable and the pair (C 1 , A) is detectable. We also have D 1 E ⊤ = 0 0 , D ⊤ 2 C 2 = 0 0 and D 1 D ⊤ 1 = 1, D ⊤ 2 D 2 = 1. We assume that the communication among the six agents is represented by the undirected cycle graph. For this graph, the smallest non-zero and largest eigenvalue of the Laplacian are λ 2 = 1 and λ 6 = 4. Our goal is to design a distributed dynamic output feedback protocol of the form (3) that synchronizes the controlled network and guarantees the associated cost (10) to satisfy J(F, G) < γ. Let the desired upper bound for the cost be γ = 17.
We adopt the design method given in case (i) of Theorem 7. First we compute a positive definite solution P to (45) by solving the Riccati equation A ⊤ P + P A + (c 2 λ 3 6 − 2cλ 6 )P BB ⊤ P + λ 6 C ⊤ 2 C 2 + σI 2 = 0 (54) with σ = 0.001. Moreover, we choose c =  As an example, we take the initial states of the agents to be x 10 = 1 −2 ⊤ , x 20 = 2 −5 ⊤ , x 30 = 3 1 ⊤ , x 40 = 4 2 ⊤ , x 50 = −1 2 ⊤ and x 60 = −3 1 ⊤ , and we take the initial states of the protocol to be zero. In Figure 1, we have plotted the controlled state trajectories of the agents. It can be seen that the designed protocol indeed synchronizes the network. The plots of the protocol states are shown in Figure 2. For each i, the state w i of the local controller is an estimate of the weighted sum of the relative states of agent i, it is seen that the protocol states converge to zero. Moreover, we compute 5 tr C 1 QP QC ⊤ 1 + λ 6 tr C 2 QC ⊤ 2 = 16.6509, which is indeed smaller than the desired tolerance γ = 17.

Conclusion
In this paper, we have studied the suboptimal distributed H 2 control problem by dynamic output feedback for linear multi-agent systems. The interconnection structure between the agents is given by a connected undirected graph. Given a linear multi-agent system with identical agent dynamics and an associated global H 2 cost functional, we have provided a design method for computing distributed protocols that guarantee the associated cost to be smaller than a given tolerance while synchronizing the controlled network. The local gains are given in terms of solutions of two Riccati inequalities, each of dimension equal to that of the agent dynamics. One these Riccati inequalities involves the largest and smallest nonzero eigenvalue of the Laplacian matrix of the network graph.