Measures for the Importance of Information Exchange in Linear Quadratic Differential Games Over Networks

The objective of this article is to study and characterize the role and the importance of <italic>information</italic> in achieving a feedback (Nash) equilibrium strategy in linear quadratic (LQ) differential games whenever the underlying players are distributed over a (physical or logic) network. It is assumed that each player should achieve a desired goal, quantified by an individual cost functional, in a competitive dynamic environment—captured by an <italic>interconnection network</italic>—by relying only on the information and data exchanged with other players according to a prescribed <italic>information network:</italic> The objective of this article is to establish the <italic>value</italic> of such an information exchange pattern toward achieving a more favorable (social) equilibrium. Interestingly, it is not assumed that the <italic>interconnection network</italic> and the <italic>information network</italic> coincide. Moreover, since the ability to achieve a certain Nash equilibrium strategy may be lost even by removing a single communication link in the network—thus partially limiting the use of the metrics discussed above—in the second part of this article, we also consider the value of the information in forming approximate Nash equilibrium strategies, namely, the so-called <inline-formula><tex-math notation="LaTeX">$\varepsilon$</tex-math></inline-formula>-Nash equilibria. Finally, the newly defined metrics are corroborated—together with a few suggested constructive results to characterize such values—by means of numerical examples.

Measures for the Importance of Information Exchange in Linear Quadratic Differential Games Over Networks Corrado Possieri and Mario Sassano , Senior Member, IEEE Abstract-The objective of this article is to study and characterize the role and the importance of information in achieving a feedback (Nash) equilibrium strategy in linear quadratic (LQ) differential games whenever the underlying players are distributed over a (physical or logic) network.It is assumed that each player should achieve a desired goal, quantified by an individual cost functional, in a competitive dynamic environment-captured by an interconnection network-by relying only on the information and data exchanged with other players according to a prescribed information network: The objective of this article is to establish the value of such an information exchange pattern toward achieving a more favorable (social) equilibrium.Interestingly, it is not assumed that the interconnection network and the information network coincide.Moreover, since the ability to achieve a certain Nash equilibrium strategy may be lost even by removing a single communication link in the network-thus partially limiting the use of the metrics discussed above-in the second part of this article, we also consider the value of the information in forming approximate Nash equilibrium strategies, namely, the so-called ε-Nash equilibria.Finally, the newly defined metrics are corroborated-together with a few suggested constructive results to characterize such values-by means of numerical examples.
Index Terms-Communication networks, dynamical games, linear systems, optimal control.

I. INTRODUCTION
T HE simultaneous presence of several independent agents interacting with each other-possibly in a competitive and noncooperative environment-and of a typically large-scale communication network capturing the information exchange pattern is becoming unavoidable as they are ubiquitous features of most modern applications [1], [2], [3], [4], [5], [6], [7], [8], [9].As far as the former aspect is concerned, since the individual payoffs/costs of each player inevitably depend also on the choices and actions of all the remaining agents in the network, it is not surprising that the concept of Nash equilibrium strategy naturally arises [10], [11], [12], [13], [14], [15], [16].However, despite its crucial importance in the aforementioned scenario, the role played by information toward the ability (or even the possibility) of the agents to settle for and form an equilibrium strategy has not been extensively dealt with so far in the literature, see, e.g., [17] for static games.
Along similar lines, interesting results have been proposed in [18] and [19].The former paper, on one hand, deals with identical dynamically decoupled plants and the focus lies essentially in the stabilization task via a distributed optimal control linear quadratic regulator (LQR) problem.The latter, on the other hand, considers the synchronization problem of multiagent models under the assumption that the individual cost functionals are influenced only by the effect of neighboring agents.The idea of providing metrics to capture the importance of the lack of a central authority to settle for an equilibrium strategy has been explored in the literature (See [20] and [21], where the prices of anarchy and stability have been introduced).
The main contribution of this article consists of the attempt of extending the current understanding of linear quadratic (LQ) dynamic games distributed on networks in several directions.Toward this end, different from most results currently available in the literature, the analysis is carried out under rather mild and generic standing assumptions, e.g., by allowing for the dependence of the cost functionals on the state of the entire network, instead of only that of neighbors, and for the presence of natural (unforced) interactions between the agents in addition to that eventually imposed by the choice of the decentralized control actions.The former aspect implies that the information structure does not necessarily coincide with the topology summarizing the (direct or indirect) influence among the players, whereas the latter point leads to an unstructured (not necessarily block-diagonal) dynamic matrix.Then, the role of information is quantified by introducing suitable metrics that associate a value/cost to each communication link among the agents.This is achieved by comparing the outcome of Nash equilibrium strategies obtained with a certain network topology with similar strategies achievable with an all-to-all data exchange pattern.Necessary and sufficient conditions are provided to characterize the set of all Nash equilibria that can be generated for a given network topology, namely, even in the presence of partial information.One of the main drawbacks of using this approach to evaluate the importance of information is that several games may not admit any achievable Nash equilibrium strategy that is compatible with a prescribed information network.Therefore, in the second part of this article, alternative metrics are proposed to evaluate the importance of information in forming approximate equilibrium strategies.Namely, the concept of ε α -Nash equilibrium is exploited to quantify the social utility that can be gathered if agents admit to sacrifice a portion of their selfish objective compared to the amount of objectives that each agent has to sacrifice toward the construction of an equilibrium.
Preliminary results concerning the loss of performance in dynamical games over networks due to limited information exchange are presented in [22].Compared to this work, herein we provide techniques to evaluate the proposed metrics, we report the proofs of all the main results, and we introduce new metrics to evaluate the importance of information for games that do not admit classical Nash strategies.
Notation: Given a symmetric matrix M = M , the notation M 0 (M 0) specifies that M is positive definite (semidefinite).Let M ∈ R n×n be a positive-semidefinite matrix and v ∈ R n a vector, then

II. LQ DYNAMICAL GAMES OVER NETWORKS
The main objective of this section is twofold.First, we recall the formulation of the decision-making process of individuals or agents interconnected by a physical or a communication network within the framework of LQ differential game theory.Then, we thoroughly examine the importance (value) and the role of information exchange patterns and of the underlying informative content in the process of achieving an equilibrium among the players.Toward this end, consider the scenario in which N heterogeneous players seek to optimize individual, potentially conflicting, objectives in a competitive (noncooperative) environment.To achieve such a task, the players exchange data over a certain communication network (information network) and influence each other according to a possibly different topology (interaction network).In particular, as discussed ahead in detail, the former network prescribes the set of (measured) state variables of other players that each agent can employ in forming its own feedback control law, whereas the latter network dictates the pattern of dynamic interconnection among agents, namely at the level of the underlying differential equations modeling the behavior of the players.It is worth stressing that the two networks may be different, i.e., it is possible that one agent is dynamically influenced by a certain player, although the former cannot use the information derived from the state of the latter to design its own equilibrium strategy.On the contrary, it may also be possible that one agent can measure the output of an agent although the latter is not physically interconnected to the former.
To make the above discussion more precise, each agent is assumed to be completely characterized by the state variable x i (t) ∈ R n i , whose dynamics are given by where is the aggregate state of all the agents, u j (t) ∈ R m j denotes the control input associated with the jth agent, A i,i ∈ R n i ×n i , A i,j ∈ R n i ×n j , and B i,j ∈ R n i ×m j .In particular, the state x i (t) denotes the set of variables of interest to effectively characterize the evolution of the corresponding player, encompassing macroeconomic or physical quantities, as well as more abstract variables such as time histories of measured quantities in sensors.On the other hand, the matrices A i,j and B i,j , j = i, represent the dynamical interaction between the current values of the state and the control input of the jth agent and the state of the ith one (possibly, A i,j = 0 and B i,j = 0 if there is no direct interaction between the jth and the ith node).It is worth observing that the nonzero elements A i,j and B i,j implicitly characterize the topology of the interconnection network among the players.Therefore, (1) summarizes, in compact and implicit form, the space of all the possible behaviors-described by specific time histories of the state x i (•)-that can be generated and pursued by the ith player.
and denoting the solution of system (1) at time t with input u = [u 1 • • • u N ] and initial condition x(0) = x 0 as φ(t, u; x 0 ), the desired individual objective of each agent consists in minimizing the cost functional where It is worth observing that, while it appears that the right-hand side of (2) depends explicitly only on u i , the cost J i is in fact simultaneously influenced also by the control strategies of all the remaining players, hence u −i , via the dynamic evolution of the state x in (1), which appears in J i and which is steered by the combined action of all players.The information network instead is captured by means of a directed graph described by the pair G = (V, E), where V = {1, . . ., N} is the node set and E ⊂ V × V is the edge set.In particular, the ith agent is assumed to be able to measure the state x j (t) of the jth agent if and only if (j, i) ∈ E. Let N (i) denote the in-neighborhood of the ith node, i.e., N (i) = {j ∈ V : (j, i) ∈ E}.Then, let {k i,1 , . . ., k i,s i } denote the set of indexes of neighboring agents to the ith player, namely {k i,1 , . . ., k i,s i } = N (i).The value s i characterizes the cardinality of the set of neighbors of the ith player and {k i,1 , . . ., k i,s i } ⊂ {1, . .., N} is a selection of a subset of the indexes of all players.The available information to the ith agent is then The main standing assumption is that ith agent must solve its own decision-making process by relying on (i.e., on the basis only of) the knowledge acquired from the current (instantaneous) value of y i .Therefore, define so that y i = Π i x, where p i = s i j=1 n k i,j is the number of states that can be measured by the ith agent.The concept of admissible policy in this context is then provided in the following statement.Roughly speaking, similarly to the interaction network that is captured by nonzero blocks A i,j in (1), the topology of the information network is implicitly prescribed by nonzero blocks of the matrices Π i .
Definition 1 (LQ admissible policy): A feedback policy (u 1 , . . ., u N ) is admissible for the game (1), (2) on the network G if there exist matrices K i with the property that

, N, and the closed-loop matrix
is Hurwitz, i.e., all its eigenvalues have a negative real part, with It is straightforward to note that if a policy (u 1 , . . ., u N ) satisfies (4), then each agent would be capable of generating the corresponding admissible equilibrium strategy by relying only on the knowledge of the current value of the available information y i .If, additionally, the matrix Ãcl is Hurwitz, then the zero equilibrium of the closed-loop system is globally asymptotically stable, hence ensuring boundedness of all the individual costs J i [23].This suggests that the players must cooperate at least in order to maintain the stability of the overall interconnected system, as it is common for dynamic games over the infinite horizon.Thus, determining a solution to the differential game (1), (2) on the network G consists in determining a stabilizing Nash equilibrium that satisfies the feedback constraints given in (4).
Remark 1: It is worth stressing that the majority of existing results in the literature about (static or dynamic) games on networks are based on the assumption that the cost functions depend only on the set of variables associated with neighboring nodes, see e.g., [17], [19].On the contrary here, the focus lies on the study of the task of forming a decision (equilibrium strategy) for each player in the presence of partial or incomplete information concerning phenomena that potentially have a direct influence on the individual costs of each agent independently from the given communication topology.This objective permits then the characterization of the importance of available information toward the possibility of achieving an equilibrium.In other words, the cost functional J i or the dynamics given in (1), i = 1, . .., N, may, indeed, explicitly depend on x j (t), with j / ∈ N (i).Therefore, it is worth stressing that the interaction network, summarized by the structurally nonzero matrices in (1), may be different from the information network defined by the graph G.
The definition of a solution to the differential game (1), (2) on the network G is provided ahead.
Definition 2 (Nash equilibrium): An admissible policy u = (u 1 , . . ., u N ) is a feedback solution to the differential game (1), (2) on the network G if for all admissible (u i , u −i ).Then, the policy u constitutes a Nash equilibrium strategy of the game (1), (2) on G.
In the scenario of Definition 2, each agent acts spontaneously and selfishly aiming at minimizing its own individual cost functional J i , without a centralized authority that monitors and potentially regulates the control inputs (u 1 , . . ., u N ) in order to achieve some "social optimum."It is worth noticing that, in this context, the feasible set of each player depends on the decisions taken by the other players, since all together they need to ensure that the closed-loop system is asymptotically stable.Therefore, determining u satisfying Definition 2 corresponds to solve a generalized Nash equilibrium problem.
Two measures of the loss of performance due to the lack of a supervisory authority that envisions and designs all the control inputs are the price of anarchy [20] and the price of stability [21], which are formally recalled in the following definitions.To provide concise statements of such definitions, let W G denote the set of all the solutions to the differential game ( 1), ( 2) on the network G, according to Definition 2. [20] and price of stability [21]): Consider the dynamic game (1), (2) and the aggregate cost

Definition 3 (Price of anarchy
Let C be the complete (all-to-all) graph with N nodes.The price of anarchy of the game (1), ( 2) is 1PoA whereas the price of stability of the game (1), ( 2) is The two metrics provided above in Definition 3 allow us to quantify the loss of performance due to the selfishness of the agents.In fact, when each agent has access to the state of all the others, the PoA and PoS measure, in the worst case and best case scenarios, respectively, the loss of performance due to the lack of a central authority that tunes all the control inputs toward the common optimization of J, rather than allowing each agent to selfishly optimize the individual cost J i .These metrics can be easily evaluated by means of classical tools, as highlighted in the following remark.(5).By [24] and [25], the set W C can be determined by computing symmetric, positive-semidefinite solutions i = 1, . .., N. Namely, the control strategies Nash equilibrium for the differential game ( 1), ( 2), provided the zero-equilibrium of the closed-loop system is asymptotically stable, and the corresponding value of the cost functional J i is x 0 P i x 0 , where x 0 ∈ R n denotes the initial aggregate state.On the other hand, let Under the assumptions that the pair ( Ã, B) is stabilizable and the pair ( Ã, Q) is detectable, letting P be the unique positive-semidefinite solution to by classical optimal control arguments [23], one has that inf u J(u) = x 0 P x 0 .Therefore, letting P C be the set of all the N -tuples of matrices in R n×n solving (9) and satisfying the stability requirement (which can be determined, e.g., using the technique given in [25]) and letting x 0 be the initial condition of the system, the PoA and PoS can be computed as PoA = sup These metrics depend on the initial aggregate state x 0 .
Although the PoA and PoS allow us to quantify the loss of social performance due to the lack of cooperation among agents, since in both metrics the all-to-all graph C is considered as baseline (hence assuming that each agent is capable of accessing the state of all the other players), such measures do not suitably capture the influence of a certain underlying communication network topology to the achievable outcomes of the game.Since one of the main objectives of this article consists in characterizing the role played by the available information in the ability to form an equilibrium strategy for each player, we provide ahead two novel metrics that allow us to quantify also the loss of performance due to the information structure of the game.
Definition 4 (Price of Information): Consider the dynamic game ( 1), ( 2) on the network G.The price of information of the game (1), (2) inf u∈W C J(u) .

Definition 5 (Price of Measurement):
Consider the dynamic game ( 1), (2) on the network G.The price of measurement of the game ( 1), (2) Remark 3: The rationale behind the meaning of PoI and PoM may be summarized as follows.Intuitively, the notion of PoI quantifies the loss of social performance [measured by the aggregate cost functional (7) evaluated at an equilibrium strategy] due to incomplete exchange of information, by comparing the aggregate outcome of the most favorable Nash equilibrium under the given communication network with the outcome of the most favorable Nash equilibrium in the presence of complete information (all-to-all information network).The PoM, instead, measures the (largest) cost that the community must be willing a priori to potentially pay for not measuring some of the data, by comparing the outcome of the least favorable Nash equilibrium with the given communication network with the outcome of the most favorable Nash equilibrium in the presence of complete information.
Remark 4: The loss of social performance due to the selfishness of the agents and the information structure has been studied in [26].However, while the definition of PoA given herein is essentially the same as the one given in [21], the concept of PoI introduced in this article is different.The notion of PoI in [26] refers to the ratio between two possible information patterns only, namely open-loop or feedback Nash equilibria: By relying on such a metric, one could compare the benefit (for all the players simultaneously) of measuring the current value of the state with respect to the knowledge of the initial condition alone.Herein instead, while limiting the focus to the class of feedback Nash equilibria, the notion of PoI allows us to assess the relative importance of the exchange of information among the players according to a certain topology.This permits the modeling of a plethora of diverse scenarios with partial and asymmetric information available to each individual player.
Similar to what has been pursued in the discussion of Remark 2, the results in the following section provide necessary and sufficient conditions for the computation of Nash equilibrium strategies that are, indeed, admissible for a given communication topology.Such a problem is instrumental for the computation of the metrics defined by PoI and PoM and it is, in fact, an interesting problem per se, which has not been solved hitherto in the literature, to the best of our knowledge.

III. ACHIEVABLE NASH EQUILIBRIA OF LQ DYNAMIC GAMES ON NETWORKS
The main contribution of this section consists of the complete description of all the Nash equilibrium strategies that can be generated under the constraint imposed by a certain communication network, by stating necessary and sufficient conditions for the existence of a solution to the differential game (1), (2) on the network G.This analysis is clearly instrumental for a discussion regarding the characterization of the price of information or, equivalently, of measurement, in the underlying game.To provide a concise statement of the main result, define the matrices Ã and Bi as in (5), so that the dynamics of the overall system in a closed loop with u i given by (4) are compactly described by ẋ = Ãx + N i=1 Bi K i Π i x, in which the communication topology is already captured by the definition of the matrices Π i .
The proof of the next theorem is proposed in Appendix A.
Theorem 1: Consider the LQ dynamic game (1), ( 2) on the network G. Suppose that N (i) = ∅ for i = 1, . . ., N, so that Π i = 0.Then, there exists an achievable Nash equilibrium strategy if and only if there exists a solution (P 1 , . . ., P N ) to the coupled Riccati equation ( 9) such that S) the following matrix is Hurwitz: F) for any i = 1, . .., N, one has Moreover, the value of the game is and the equilibrium strategies are given by The rest of this section is devoted to comments and further insights about the above statement.To begin with, it is worth stressing that despite the fact that the claim of Theorem 1 merely refers to the existence of a feasible Nash equilibrium strategy, it, indeed, implicitly characterizes the entire set of admissible Nash equilibrium strategies, as explicitly carried out in the following result.
Corollary 1: Consider the LQ dynamic game (1), (2) on the network G.The control policies (u 1 , . .., u N ) with with P i satisfying (9) together with the requirement (S).Remark 5: While the stability requirement (S) is rather classical in this context, the feasibility requirement (F) in Theorem 1 summarizes the somewhat intuitive, but yet to be proved in the literature, conclusion that a Nash equilibrium strategy can be achieved in the presence of limited and partial information only provided that the corresponding unconstrained equilibrium can be generated by relying only on available information.Note that this does not imply that the PoI is 1 since the coupled Riccati equation ( 9) may admit multiple solutions [24].Only some of such solutions satisfy also the stability requirement (S) and the feasibility requirement (F).It does not appear evident how to draw general conclusions about the satisfiability of (F), since the latter depends not only on the network topology but also on the specific data in ( 1) and ( 2).On the other hand, item (F) may instead suggest a somewhat negative conclusion: Since it involves a certain subspace inclusion, it constitutes in fact a fragile property, in the sense that it may be lost even for small variations on the data of the problem.Such a consideration is, indeed, the motivation for generalizing the concept of solution to the notion of -Nash equilibria, as carried out in Section IV.Therein, it is shown in fact that, while (17d) remains a subspace inclusion, the crucial difference with Theorem 1 consists of the fact that the matrix P i should satisfy a matrix inequality (17), in place of an equality (9), thus considerably increasing the set of matrices that should further satisfy item (F).Finally, note that, by hinging upon similar arguments as above, item (F) is not necessarily related to observability properties, of individual players or of the entire network, via the matrices Π i .
Remark 6: The PoI and PoM can be determined by using tools similar to those recalled in Remark 2. Namely, letting P C be the set of all the N -tuples of matrices in R n×n solving (9) and satisfying the stability requirement (S), which can be computed as detailed in Remark 2, the set W G can be determined by retaining only those N -tuples of matrices that additionally satisfy the feasibility requirement (F).Hence, letting P G be such matrices and letting x 0 be the initial aggregate state, the PoI and PoM can be computed as As the PoA and the PoS, these metrics depend on x 0 .Remark 7: According to Definition 1, the set of admissible policies is restricted to the functional space of static state feedback control laws.The motivation for limiting the admissible set to static feedback policies-compared for instance to dynamic control actions-is twofold.First, the introduction of additional dynamics would hinder the rationale behind the concept of equilibrium strategy, differently from what happens, e.g., in (single-objective) optimal control problems, in which such auxiliary dynamics typically contribute to additional costs.Then, on a more abstract note, the use of individual observers to reconstruct the state of the entire network would implicitly compromise the assessment of the role played by available information, which is the main objective of this article while also introducing additional complexity in the network.

A. Illustrative Example
The aim of this section consists in substantiating the above definitions and results by means of a numerical example.Let N = 3 and let x 1 (t) ∈ R, x 2 (t) ∈ R, and x 3 (t) ∈ R denote the state of the first, second, and third agents, respectively.Assume that the dynamics of the agents are completely characterized by the linear, time-invariant system (1) with Authorized licensed use limited to the terms of the applicable license agreement with IEEE.Restrictions apply.
A 3,1 = −0.5, By inspection of the (nonzero) terms A i,j in (1), it appears evident that the interaction network consists of fact in the all-to-all, complete, graph.The objective of the three agents is to minimize the following cost functionals for i = 1, 2, 3: By using the homotopy continuation method given in [27], one obtains that the coupled Riccati equation ( 9) admits 130 solutions, 3 of which, in the following denoted (P i 1 , P i 2 , P i 3 ), i = 1, 2, 3, also satisfy the stability requirement (S): It is then straightforward to realize that the solution (P 3 1 , P 3 2 , P 3 3 ) satisfies the feasibility requirement (F) for any of the communication topologies G i , i = 1, . . ., 64, obtained by removing one, some, or all the dashed lines from the graph depicted in Fig. 1(b).Note that the self-loops cannot be removed from these topologies, otherwise the feasibility requirement (F) is not met.On the contrary, the other two solutions satisfy the feasibility requirement (F) only for the communication topology represented by the (complete) graph C depicted in Fig. 1(a).
Thus, letting the aggregate cost be defined as J = J 1 + J 2 + J 3 , if x 0 = [1 0 1] , then the game is characterized by the metrics reported in Table I, depending on the communication topology, which summarizes the information available to each of the two agents.
Note that the PoA and the PoS are independent from the communication topology, since they are defined with respect to the complete information graph (they essentially encode the loss of performance due to the selfishness of the agents).On the other hand, the PoI and the PoM encode the loss of performance due to the limitations imposed by the communication topology, and hence, as expected, they are strongly related to the information network itself.
In many applications, it is of interest to highlight the loss of performance due to the fact that the specific link between agent i and agent j has been removed, thus characterizing the social cost of each individual communication arc.Such information can be encoded on the complete communication graph, by labeling each edge (i, j) with the PoI of the game over the network where (only) the edge (i, j) has been removed [see Fig. 2(a)].
It is possible to iterate such a procedure, by removing one edge that is not labeled with ∞ at a time, and reporting on the remaining edges the PoI of the game over the network where such an edge has been removed until all the edges are labeled with ∞; see Fig. 2(b).This representation allows one to easily determine the maximum number of edges that can be removed from the complete communication graph while still guaranteeing the existence of at least one achievable Nash equilibrium strategy (in the reported example, 6 edges), together with the loss of performance induced the removal of the corresponding edges.The label thus defined for each communication link provides the social price paid for removing that specific arc from the network topology.
By inspecting the labels of each arc in Fig. 2(a) and (b), one immediately recognizes a potential practical limitation of the previous theoretical analysis: Since the feasibility condition (F) involves the inclusion of certain matrices into specific subspaces, it is intrinsically fragile and generically 2 not satisfied; it seems that the typical scenario to expect comprises several arcs with infinite price since they cannot be removed without completely comprising the ability to compute any admissible Nash equilibrium strategy.To somewhat circumvent such a drawback of the conceptually correct definitions of PoI and PoM, one may be in fact willing to trade the computation of an equilibrium strategy of the originally given game-settling instead for an approximate policy, in the sense specified ahead-to retain, on the other hand, the possibility of thoroughly computing the relative importance of each individual communication link in the topology.This direction is pursued in the following section.

IV. ε α -NASH EQUILIBRIA OF LQ DYNAMIC GAMES ON NETWORKS
As anticipated in the discussion before, one of the intrinsic drawbacks of using Nash equilibria to evaluate the role of information in LQ dynamical games on networks is that several games may not admit any achievable Nash equilibrium strategy, compatible with a prescribed information network.In fact, by Theorem 1, there is an achievable Nash equilibrium strategy if and only if there exists a stabilizing solution to the coupled Riccati equation ( 9) that also meets the feasibility requirement (F).Since the coupled Riccati equation ( 9) does not directly account for the information matrices Π 1 , . . ., Π N , there may not be any of such solutions, thus leading to an infinite price of information.Therefore, in this section, we resort to a relaxed notion of Nash equilibrium strategy, namely the concept of ε α -Nash equilibrium [16], with the aim of providing a more relevant and practical metric to evaluate the role of information in the process of forming, or approximating, a Nash equilibrium.Toward this objective, consider the following definition.
Intuitively, the requirement of Definition 6 entails that the matrix Ãcl should be Hurwitz and, in addition, with all the 2 A certain property is generically satisfied if it holds with probability one for a random selection of the problem data.In the specific context of this article, one can expect that for a random choice of the involved matrices the requirement (F) would hold only for the complete graph C. Clearly, this does not exclude the presence of specially structured cases in which the conditions are, indeed, verified (see also the illustrative example).
eigenvalues possessing real parts to the left of the value −α.We can now formalize the concept of ε α -Nash equilibrium.
Definition 7 (ε α -Nash equilibrium): An admissible policy u * = (u * 1 , . . ., u * N ) is an ε α -Nash equilibrium of the differential game ( 1), ( 2) on the network G if there exists a nonnegative constant ε α,x 0 , parameterized with respect to the initial aggregate state x 0 and the constant α > 0, such that for all α-admissible (u i , u * −i ).Although the concept of ε α -Nash equilibrium clearly constitutes a relaxation of the stricter notion of solution to the differential game ( 1), ( 2), it is of interest in practical cases since its computation may be significantly easier than that of a classical Nash equilibrium [15], [28].The following theorem, whose proof is postponed to Appendix B, shows how to determine a feasible ε α -Nash equilibrium of the differential game ( 1), ( 2) on the network G.
Theorem 2: Consider the differential game ( 1), ( 2) on the network G. Suppose that there exist symmetric, positivesemidefinite matrices i = 1, . . ., N, subject to the constraints Then, the policy is admissible and constitutes an ε α -Nash equilibrium of the differential game ( 1), ( 2) on the network G for any α > 0.
Note that the matrices on the left-hand sides of (17b) and (17c) are at least positive semidefinite by the definition of Q i and the construction of Si in (8).While the subspace inclusion (17d) essentially coincides with (11), the most relevant difference between the conditions leading to Nash or ε α -Nash equilibria, respectively, consists of the fact that the equality in ( 9) is replaced (relaxed) to the inequality in (17).As it can be easily understood, this aspect significantly increases the chances of determining a solution to the latter [compared to (9)] with the additional property that the feasibility inclusion ( 11) is satisfied.The following corollary, whose proof follows from the same reasoning used in Appendix B, shows how to relax some of the constraints given in (17) at the expense of introducing another auxiliary matrix variable.It is worth observing that (17b)-(17c) and the subsequent inequality ( 19) are both sufficient conditions Authorized licensed use limited to the terms of the applicable license agreement with IEEE.Restrictions apply.
ensuring the asymptotic stability of the system in a closed loop with the equilibrium policies.
The rest of this section is devoted to comments and further insights about the above statements and about how they can be used to measure the role of information in LQ dynamical games over networks.
To begin with, note that although Theorem 2 and Corollary 2 refer to the existence of a feasible ε α -Nash equilibrium strategy, it, indeed, implicitly characterizes the entire set of admissible ε α -Nash equilibrium strategies that are also classical Nash equilibria of an augmented game, as explicitly carried out in the following result, whose proof is postponed to Appendix C.
We can now formalize a new metric based on ε α -Nash equilibria so to characterize the role played by the available information in the ability to form an approximate equilibrium strategy for each player.The main advantage of this new metric with respect to the PoI is that it is based on a relaxed notion of equilibrium and, hence, may be capable of quantifying the loss of performance due to limited communication among the agents even in the case that a classical Nash equilibrium strategy does not exist.The consequence of such an approach is to prevent the presence of structurally unavoidable communication links, which would necessarily possess infinite price, and it allows us to quantify instead the cost of every single arc.We refer to this metric as the Price of Deal since, under the considered admissible policies, each agent admits to settle-hence stipulating a deal with the other players toward maintaining a reasonable level of social outcome-for approximate performance with respect to its selfish objective (due to the fact that ε α -Nash equilibria are accounted for rather than classical Nash strategies) toward the construction of an equilibrium.

Definition 8 (Price of Deal):
Consider the dynamic game (1), (2) on the network G. Let V G denote the set of all the ε α -Nash equilibrium strategies of the game (1), (2) on the network G that are also classical Nash equilibria of an augmented game with costs defined as in (20).The Price of Deal of the game (1), ( 2) The price of deal is a generalization of the price of information to ε α -Nash equilibrium strategies.Although also the price of measurement can be extended to such a context, the resulting metric may not be as useful as the price of deal since the set of all the matrices (P 1 , . . ., P N ) that satisfies the coupled Riccati inequalities (17) need not be compact.
While Definition 8 characterizes the price of deal for the entire given topology, a procedure identical to the one discussed in the illustrative example given in Section III-A can be carried out to compute the price of each individual arc as well.The following remark details how to compute the PoD of a dynamic game on a network.
Remark 8: By following the results stated in Theorem 2 and Corollary 3, let x 0 be the initial aggregate state of the network and consider the optimization problem with quadratic matrix inequality constraints defined as follows: The semidefinite program (21) can be solved either using freely available solvers, such as PENLAB [29], or commercial solvers, such as PENBMI [30], interfaced with general purpose optimization software, such as Yalmip [31] and fminsdp [32].Hence, letting c be the solution to such a problem one has that the PoD can be computed as where P C is defined as in Remark 6.In fact, by classical LQ optimal control arguments [23], letting H be the solution to the (21e), the aggregate cost corresponding to the policy ( 18) is x 0 Hx 0 , and, by Theorem 2 and Corollary 3, the set V G of all the ε α -Nash equilibrium strategies of the game ( 1), ( 2) on the network G that are also classical Nash equilibria of an augmented game with costs defined as in ( 20) is completely characterized by means of (21b)-(21d).
The following remark provides further insights on the PoD.Remark 9: Since W G ⊂ V G due to the fact that classical Nash strategies are also ε α -Nash ones, we have that PoD ≤ PoI for all differential games of the form (1), ( 2) and all initial aggregate states x 0 ∈ R n .In particular, since the underlying assumption behind ε α -Nash equilibria is that agents sacrifice a portion of their selfish utility toward the construction of a socially efficient equilibrium, differently from the metrics introduced so far, the PoD can be greater than, equal to, or smaller than 1.In fact, from a social perspective, it may be more convenient to pursue an ε α -Nash equilibrium rather than a classical full information one.
The next example illustrates the computation of the PoD.
Example 1: Consider again the differential game given in Section III-A and let the communication topology be the one corresponding to the graph depicted in Fig. 3 .
As shown in Section III-A, such an information structure admits a classical Nash equilibrium and the price of information, given the initial aggregate state x 0 = [1 0 1] is PoI = 1.06483.By computing the solution to the semidefinite program (21) via the optimization toolbox fminsdp one obtains that PoD = 0.837332.Note that the PoD is much smaller than the PoI since the underlying assumption behind its definition is that the agents accept to reach a compromise toward the construction of a socially efficient equilibrium.
As discussed in detail in the previous comments and remark, the Price of Deal remains intimately related to a certain notion of social performance, since the matrix H in (21) quantifies the outcome of the aggregate cost functional, given by the sum of the individual objectives of the players.It is then evident that such a notion does not provide any information about the loss of performance, in terms of the achievement of an approximate ε α -Nash equilibrium strategy, incurred by each player in the network on its own.Therefore, the newly defined metrics introduced in this article to quantify the role of exchanged information in LQ differential games are completed by what we refer to as the Price of Compromise, which is discussed ahead.To this end, consider a variation of the optimization problem (21) described by the following task: Si P i ≺ 0 (22e) Let * denote the solution to the problem ( 22) for a given information network captured by the graph G.Then, the following definition can be given.While the spirit of Definition 9 consists in characterizing a (worst case) distance from a Nash equilibrium common to all players, it may be possible to envision also the notion of individual PoC as a vector [ 1 • • • N ] , whose ith entry is the amount of selfish objective that the ith agent has to sacrifice in order to form an equilibrium.In fact, note that such a modification is already contained in the constructions carried out in the proof of Theorem 2, in which it appears evident that such coefficients are strongly related to the matrices Υ i , i = 1, . .., N. Following the discussion above, the PoC allows us to precisely quantify the amount of selfish objective that agents must admit to sacrifice toward the construction of an equilibrium.In fact, note that if there exists an achievable Nash equilibrium strategy for the dynamical game on the network, then the PoC is 0 due to the fact that (22b) and (22c) with = 0 imply (9).On the other hand, if there is no achievable Nash equilibrium strategy, then the PoC is greater than zero and quantifies the amount of selfish objective sacrificed by the agents toward the construction of an equilibrium.In fact, following the construction given in Appendices B and C, the PoC equals the smallest value of such that there exists an ε α -Nash equilibrium of the differential game (1), (2) on the network G for any α > 0 that is also a Nash equilibrium for system (1) and the modified cost functionals (20) with Υ i I, i = 1, . . ., N.
The next example shows how to use the PoC to evaluate the importance of information in games with interconnection and information networks represented by path graphs.
Example 2: Let x i (t) ∈ R, denote the state of the ith agent and assume that its dynamics are completely characterized by the linear, time-invariant system (1) with Authorized licensed use limited to the terms of the applicable license agreement with IEEE.Restrictions apply.whereas all the other entries of the matrices Ã and B1 , . . ., BN are zero, i = 1, . . ., N. Suppose that the objective of the ith agent is to minimize the cost functional Assuming that the interaction topology equals the communication one, the main objective of this section is to characterize the importance of each single communication arc.As for the PoI, such information can be encoded on the communication graph, by labeling each edge (i, j) with the PoC of the game over the network where the edge (i, j) has been removed.Fig. 4 depicts this graphical representation considering games involving a different number of agents.
As shown by such a figure, the PoC allows us to quantify the importance of information exchange in dynamical games over networks.In fact, higher values of the PoC indicate that the agents have to sacrifice a larger amount of their objective toward the construction of an equilibrium strategy.In particular, for all the considered information structure, the agents have to measure their own state to guarantee the existence of an ε α -Nash equilibrium strategy since the corresponding value of the PoC is ∞.

V. CONCLUSION
The importance of information exchange in LQ differential games distributed over a network has been characterized through different metrics.By providing necessary and sufficient conditions for the existence of a Nash equilibrium strategy that is compatible with the information structure of the game, two metrics (namely, the PoI and the PoM) have been given to evaluate the social utility of the best and the worst Nash equilibrium that is compatible with a given information network.Since several games may not admit any achievable Nash equilibrium strategy that is compatible with a prescribed information structure, the value of information has also been evaluated toward the formation of approximate Nash equilibrium strategies.Namely, the social utility that can be gathered if agents admit to sacrifice a portion of their objective and the amount of selfishness that each agent has to sacrifice toward the construction of an equilibrium have been both characterized by means of the PoD and the PoC.Computational techniques have been given to practically evaluate all the proposed metrics and several numerical examples have been reported to corroborate the theoretical results.Furthermore, while it appears intuitive that the resulting metrics would not be uniquely determined by the underlying network topology but would heavily depend also on the specific dynamics exhibited by each node of the network, it would be interesting to establish deeper connections between the topology itself and the expected measures, i.e., by identifying patterns or limiting cases.Similarly, the feasibility requirement (F) constitutes a condition in which the dynamics, the relative cost functionals and the topology are all intertwined factors.Therefore, it would be of some interest to identify such interconnected contributions toward its satisfaction.
Finally, the problem of designing the most desirable topology according to the metrics defined by the PoI and PoM (which may not be uniquely defined, as entailed by the example in Section III-A) is worth investigating.

A. Proof of Theorem 1 and Corollary 1
To begin with, we recall two technical lemmas taken from the work in [33].To provide a concise statement of the following results, let E m×p j,k denote the matrix of dimension m × p with all the elements equal to zero except the entry of position (j, k), which is equal to one.
Lemma 1 (see [33]) : The identity [33]): Consider two matrices A ∈ R m×n and B ∈ R p×n , and suppose that B = 0.Then, it follows that Axx B = 0 for all x ∈ R n if and only if A = 0.
By relying on the statements of Lemmas 1 and 2, we are now in the position of providing the proof of Theorem 1, which states necessary and sufficient conditions for the existence of a solution to the game (1), (2) on the network G.
1) Sufficiency: In order to show the sufficiency of the stated conditions, define the quadratic functions V i = x P i x, i = 1, . .., N, substitute the information-constrained control laws (4) in (1), and consider the Hamilton-Jacobi-Isaacs partial differential equations, focusing to the case of player ith and assuming that all the other agents are implementing the equilibrium strategies K j , namely Authorized licensed use limited to the terms of the applicable license agreement with IEEE.Restrictions apply.
which should hold for any x ∈ R n .By following arguments similar to the results in [14], then there exists a solution to the game (1), (2) on G, provided there exist matrices P i that solve such equations for i = 1, . . ., N. Therefore, define the matrix-valued function . By borrowing the tools discussed in [34], one obtains that The only terms dependent on K i in the expression above are Furthermore, by Lemma 1, it follows that (I Consider now the Hessian matrix H of the (scalar) function x M (K i )x with respect to vec(K i ) By the reasoning given earlier, it results that vec(N which is positive semidefinite due to the positive semidefiniteness of Π i xx Π i and R i .Hence, the scalar function x M (K i )x is convex with respect to vec(K i ).Therefore, by classical results about convex functions [35], the matrix for all x ∈ R n , i.e., equivalently, (K i Π i + R −1 i B i P i )xx Π i = 0. Thus, by Lemma 2, one has that K i must be such that (14) holds.Thus, if (14) holds, by substituting K i Π i into the Hamilton-Jacobi-Isaacs equation ( 24), one obtains that 0 for all x ∈ R n , i.e., P i must satisfy the coupled Riccati equation (9).Therefore, if there exist P 1 , . . ., P N that solve (9), with K i satisfying (14), then (K 1 x, . . ., K N x) constitutes a Nash equilibrium of the game (1), (2) over the network G, provided that the matrix given in (10) is Hurwitz.
2) Necessity: In order to prove necessity, assume that there exists a solution (K 1 , . . ., K N ) to the game (1), (2) over the network G, but ( 9) and ( 14) do not hold.Define the analytic functions V i (x) = J i (x), which must satisfy, for all x ∈ R n 0 = ∂V i ∂x By considering the Taylor series expansion of V i about the origin x = 0, one has that V i = ≥1 p i, , where p is a homogeneous polynomial in x of degree .Therefore, since ∂ ∂x V i = ≥1 ∂ ∂x p i, and p i, is homogeneous of degree with respect to the standard dilation [36], one has that ∂p i, ∂x ( Ã + Bi K i Π i + N j=1,j =i Bj K j Π j )x is still homogeneous of degree .Thus, letting A = Ã + Bi K i Π i + N j=1,j =i Bj K j Π j , the expression given in ( 25) can be equivalently rewritten as ∂x A x = 0, > 2. Thus, by rewriting p i,2 as p i,2 = x P i x, for some symmetric P i ∈ R n×n , one has that (24) must hold, thus leading to a contradiction by the reasoning given to prove sufficiency.

B. Proof of Theorem 2
By (17b), the function V = x ( N i=1 P i )x is positive definite.Moreover, if the matrices P i , i = 1, . . ., N, satisfy (17), then there exist symmetric and positive-semidefinite matrices Q i + Υ i + P i Si P i = 0 (28) thus implying that V = −x ( N i=1 (Q i + Υ i + P i Si P i ))x.Hence, by (17c) and (17d), the policy ( 18) is admissible.
Authorized licensed use limited to the terms of the applicable license agreement with IEEE.Restrictions apply.
Furthermore, by inspecting the coupled Riccati equation ( 26), the control policy (18) constitutes a Nash equilibrium for system (1) and the modified cost functionals Since Υ i 0, i = 1, . . ., N, implies that J i (u i , u −i ) ≤ Ji (u i , u −i ), and, by the definition of Nash equilibrium, we have Ji (u * i , u * −i ) ≤ Ji (u i , u * −i ), for all admissible (u i , u * −i ), it results that J i (u * i , u * −i ) ≤ J i (u i , u * −i ) + ∞ 0 x (t)Υ i x(t)dt, where x(t) is the solution to system (1) with inputs u i and u * −i , for all admissible (u i , u * −i ).Hence, by considering that, by classical linear arguments [37], for each α > 0, there exists a constant c α > 0 such that the state x(t) of the closed-loop system satisfies x(t) ≤ c α exp(−αt) x 0 for all t ≥ 0 and for any α-admissible policy (u i , u * −i ), it results that x 0 2 .Therefore, the policy given in (18) satisfies for all α-admissible policies (u i , u * −i ), that is (u * i , u * −i ) constitutes an ε α -Nash equilibrium of the differential game (1), (2) on the network G for any α > 0.

C. Proof of Corollary 3
Since the policy u * = (u * 1 , . . ., u * N ) is admissible and hence the closed-loop system with such a control input is asymptotically stable, there exists a symmetric, positive-definite matrix P = P ∈ R n×n , P 0, that satisfies (19).
Furthermore, since u * = (u * 1 , . . ., u * N ) is a Nash equilibrium of the differential game (1), (20) on the network G, by Theorem 1 and Corollary 1, we have that u * i = −R −1 i B i P i x, where the matrices P i are symmetric and positive semidefinite, solve the coupled Riccati equation (26), and also satisfy the feasibility requirement (17d), i = 1, . . ., N. Hence, by considering that the matrices Υ i are positive semidefinite, the coupled Riccati inequalities (17a) hold.

Remark 2 :
In order to streamline the exposition, define

Definition 9 (
Price of Compromise): Consider the dynamic game (1), (2) on the network G.The Price of Compromise of the game (1), (2) over the network G is PoC * .