An asynchronous distributed and scalable generalized Nash equilibrium seeking algorithm for strongly monotone games

In this paper, we present three distributed algorithms to solve a class of generalized Nash equilibrium (GNE) seeking problems in strongly monotone games. The first one (SD-GENO) is based on synchronous updates of the agents, while the second and the third (AD-GEED and AD-GENO) represent asynchronous solutions that are robust to communication delays. AD-GENO can be seen as a refinement of AD-GEED, since it only requires node auxiliary variables, enhancing the scalability of the algorithm. Our main contribution is to prove converge to a variational GNE of the game via an operator-theoretic approach. Finally, we apply the algorithms to network Cournot games and show how different activation sequences and delays affect convergence. We also compare the proposed algorithms to the only other in the literature (ADAGNES), and observe that AD-GENO outperforms the alternative.


Introduction
In modern society, multi-agent network systems arise in several areas, leading to increasing research activities. When selfinterested agents interact between each other, one of the best mathematical tools to study the emerging collective behavior is noncooperative game theory over networks. In fact, networked games emerges in several application domains, such as smart grids [8,12] , social networks [10,19,20] and distributed robotics [6] . In a game setup, the players (or agents) aim at minimizing a local and private cost function which represents their individual interest, and, at the same time, satisfy local and global constraints, limiting the possible decisions (or strategies/actions). The cost function and constraints of a single player are influenced by the behavior of a fraction of the others, called "neighbors". Thus, each decision is influenced by some local information, which is typically exchanged with the neighbors. One popular notion of solution for these games is the GNE, where no player benefits from unilaterally changing its strategy, see [16] .
In [3,20,30] , the authors focused on developing synchronous and distributed equilibrium seeking algorithms for noncoopera-affine coupling constraints. This work generalizes and extends the current literature on the topic in the following ways.
• We tackle the case of a game subject to inequality coupling constraint (rather than only equality constraint). This drastically broaden the type of problems that can be solved by the proposed approach. For example in signal processing [29] and smart grids [12] inequality constraints arise naturally. This extension cannot be achieved via an extension of the results currently available due to the different control structure considered. • The algorithms that we develop rely on node variables only, rather than edge variables as in [31] . This, apparently subtle difference, leads to a solution that adopts (almost always) a lower number of variables. So, it is lighter from a computational point of view, requires less memory and involves lighter communication between agents. All these features make the proposed solution achieve overall better performances than [31] .
We conclude the paper comparing the proposed algorithms to that in [31] , for the case of a Cournot game, showing that our algorithms achieve faster convergence. A preliminary and partial version of these results were presented in [7] .

Basic notation
The set of real, positive, and non-negative numbers are denoted by R , R > 0 , R ≥0 , respectively; R := R ∪ {∞} . The set of natural numbers is N . For a square matrix A ∈ R n ×n , its transpose is A , [ A ] i is the i th row of the matrix and [ A ] ij represents the element in i th row and j th column. A 0 ( A 0) stands for a positive definite (semidefinite) matrix. A B is the Kronecker product of the matrices A and B . The identity matrix is denoted by I n ∈ R n ×n . 0 (resp. 1 ) is the vector/matrix with only 0 (resp. 1) elements. For x 1 , . . . , x N ∈ R n , the collective vector is denoted by x := col ((x i ) i ∈ (1 , ... ,N) (1 , ... ,N) ) describes a block-diagonal matrix with A 1 , . . . , A N on the main diagonal.

Operator-theoretic notation
The identity operator is denoted by Id( · ). The set valued mapping N C : R n ⇒ R n denotes the normal cone to the set C ⊆ R n , that > )0 holds true, and maximally monotone if it does not exist a monotone operator with a graph that strictly contains gra (F ) . 2 for all x, y ∈ R n ; F is β-cocoercive if βF is 1 2 -averaged, i.e. firmly nonexpansive (FNE). The resolvent of an op-

Mathematical formulation
We consider a noncooperative game between N agents (or players) subject to affine coupling constraints. We define the game as the triplet , where its elements are respectively: the collective feasible decision set, the players' local cost functions and the graph describing the communication network. In the following subsections, each one of them is introduced.

Feasible strategy set
Every agent i ∈ N := { 1 , . . . , N} has a local decision variable (or strategy) x i belonging to its private decision set i ⊂ R n i , namely the set of all those strategies that satisfy the local constraints of player i . The collective vector of all the strategies, or strategy profile of the game, is denoted as where n := i ∈N n i . Then, all the decision variables of all the players other than i are represented via the compact notation x −i := col (x 1 , . . . , x i −1 , x i +1 , . . . , x N ) . We assume that the agents are subject to m affine coupling constraints described by the affine function x → A x + b, where A ∈ R m ×n and b ∈ R m . Thus, the collective feasible decision set can be written as is the Cartesian product of the local constraints sets i 's. Accordingly, the set of all the feasible strategies of each agent i ∈ N reads as The choice of affine coupling constraints is widely spread in the literature of noncooperative games, see e.g., [10,24,30] . Moreover, in [20] , Remark 3, it is highlighted that separable and convex coupling constraints can always be rewritten in an affine form. Finally, we introduce some blanket assumptions on this set of feasible strategy, standard in the literature [9,10,16,30,31] .
Standing Assumption 1 (Convex constraint sets) . For each player i ∈ N , the set i is convex, nonempty and compact. The collective feasible set X satisfies Slater's constraint qualification.

Cost functions
The coupling between the players appears not only in the constraints but also in the cost function, due to the dependency on both x i and x −i . Next, we assume some properties for these functions that are extensively used in the literature [16,30] .
Standing Assumption 2 (Convex and differentiable cost func-

Communication network
The communication between agents is described by an undirected and connected graph G = (N , E ) where E ⊆ N × N is the set of edges. Given two agents i, j ∈ N , the couple ( i , j ) belongs to E, if agent i shares information with agent j and vice versa. Then we say that j is a neighbour of i , i.e., j ∈ N i where N i is the neighbourhood of i . The number of edges in the graph is denoted by E := |E| .
To define the incidence matrix V ∈ R E×N associated to G, let us label the edges as e l , for l ∈ { 1 , . . . , E} . We define the entry [ V ] li := 1 (resp. −1 ) if e l = (i, ·) (resp. e l = (·, i ) ) and 0 otherwise. The decision of which of the two agents composing an edge is the sink and which the source is arbitrary. By construction, V 1 N = 0 N . Then, we define E out i (resp. E in i ) as the set of all the indexes l of the edges e l that start from (resp. end in) node i , and hence E i = E out i ∪ E in i . The node Laplacian L ∈ R N×N of an undirected graph is a symmetric matrix defined by L := V V . Another important property of L , used in the remainder, is L 1 N = 0 N .

Generalized Nash equilibrium
In summary, the considered generalized game is described by the following set of inter-dependent optimization problems: The most popular equilibrium concept considered for noncooperative games with coupling constraints is the generalized Nash equilibrium , thus the configuration in which all the relations in (2) simultaneously hold.
Definition 1 (Generalized Nash Equilibrium) . A collective strategy x * ∈ X is a generalized Nash equilibrium (GNE) if, for each player i , it holds In this work, we focus on a subset of GNE, the so called variational GNE (vGNE), a class of equilibria that is considered in many other works throughout the literature -see [3,16,21,22] . The name of these equilibria derives from the fact that they can be formulated as the solutions to a variational inequality (VI). An important property of these equilibria is that each agent faces the same penalty to fulfill the coupling constraints, which is particularly useful to represent a "fair" competition between agents [16] . Variational GNE can be seen as a particular case of the concept of normalized equilibrium points , firstly introduced by Rosen in [28] and further studied in [10,25] .
To properly characterize this set, we define the pseudo-gradient mapping (or game mapping) of (2) , as The pseudo-gradient gathers in a collective vector form the gradients of the cost functions each w.r.t. the local decision variable. Next, we introduce some standard technical assumptions, e.g., [2,13] .
When Standing assumption 2 holds true, the mapping F is single valued and the set of vGNE of the game in (2) corresponds to the solution to VI( F , X ), namely the problem of finding a vector x * ∈ X such that The continuity of F ( Assumption 2 ) and compactness of X ( Assumption 1 ) imply the existence of a solution to VI( F , X ), while the strong monotonicity ( Assumption 3 ) entails uniqueness, see [15] , Th. 2.3.3.
Next, let us define the KKT conditions associated to the game in (2) . The strong duality of the problem ( Assumptions 1 and 2 ) implies that, if x * is a GNE of (2) , then there exist N dual variables ≥0 , for all i ∈ N , such that the following inclusions are satisfied: Instead of looking for the solution of the general case where λ * 1 , . . . , λ * N may be different, we examine the special case when It follows from [17] , Th. 3.1(ii), that the KKT inclusions in (6) correspond to the solution set to VI( F , X ). Thus, every solution x * to VI( F , X ) is also a GNE of the game in (2) , [17 ,Th. 3.1(i)]. Since the solution set to VI( F , X ) is a singleton, we conclude that there exists a unique vGNE of the game (2) .

Synchronous distributed GNE seeking algorithm
We first introduce the synchronous counterpart of AD-GENO, i.e., the Synchronous Distributed GNE Seeking Algorithm with Node variables (SD-GENO). The derivation of the algorithm is based on an operator splitting approach to solve the KKT system in (6) . A similar approach was also adopted in [3,30] in the contest of GNE finding problems.

Algorithm design
The KKT conditions of each agent i in (5) where ρ ∈ R > 0 . In (7) , the term L λ accelerates the convergence of the dual variables to consensus.
A solution = col ( x * , σ * , λ * ) of the above inclusions can be equivalently recast as a zero of the sum of two mappings A and B defined as In fact, * ∈ zer (A + B) if and only if ϖ * satisfies (7) .
Next, we show that the zeros of A + B characterize the vGNE of the original game. (8) . Then the following hold:

Proposition 1. Let A and B be as in
The proof is attained by exploiting the property that ker (V ) = ker (L ) , for the graph described in Section 3.1.3 . The steps are similar to those in [30 ,Th. 2]. We omit them here for brevity reasons.
Several researchers have analyzed the problem of finding a zero of the sum of two monotone operators. The so called splitting methods represent one of the most popular approach developed to attain an iterative algorithm to solve this class of problem -see [14] , [1 ,Ch. 26]. (8) are maximally monotone.

Lemma 1. The mappings A and B in
The properties of the operators proved above drive us to select the preconditioned forward-backward splitting (PFB) to derive a distributed and iterative algorithm seeking zer (A + B) . This approach was previously adopted by other researchers, e.g., [30] .
The PFB splitting operator reads as The so-called preconditioning matrix is defined by The update rule of the algorithm is obtained by including a relaxation step, i.e., It comes from (9) see [1 ,Th. 26.14].
In the remainder of this section, we provide the complete derivation of SD-GENO, obtained directly from (11) . In the following, we denote ϖ := ϖ( k ), + := (k + 1) and ˜ := ˜ (k ) to simplify the notation. Consider ˜ = T . From (9) it holds that The update rule of each components of ϖ is attained by analyzing the row blocks of (12) . The first reads as 0

. By solving this inclusion by ˜
x , we attain the update rule for the primal variables: Similarly, from the second row block of (12) , we attain the update for ˜ σ, i.e., ˜ σ = σ + δρV λ. (14) Finally, the third row block of (12 (14) = proj R mN Note that, the update of ˜ λ depends only on the aggregate information V σ. We can exploit this feature to replace the edge auxiliary variables σ l 's, with a single variable for each agent i defined by L , we compute the update rule of these new variables and replace (14) by (16) Consequently, (15) is modified accordingly as To ensure that this change of variables does not affect the equilibrium of the game, we introduce the following result proving that an equilibrium point of the new set of equations is indeed a vGNE of (2) .

Remark 1.
In [31] , the algorithm SYDNEY achieves convergence to the vGNE of the game (2) , when this is subject to equality coupling constraints only. This solution relies on edge auxiliary variables to enforce the consensus of the λ i 's. Therefore, the number of variables that each agent has to store is O(N) .
The change of "variables", from σ to z , is convenient when the edges outnumber the nodes, which is almost always the case. In fact, a lower number of variables leads to an overall increment in the algorithmic efficiency and to a fixed memory requirement for each player that does not increase with N . Furthermore, if SYDNEY in [31] is modified to address inequality constraints, it would require an additional round of communication between the agents, making it more demanding and slower than SD-GENO.

Synchronous, distributed algorithm with node variables (SD-GENO)
The complete formulation of the algorithm is obtained by gathering together all the update rules introduced in the previous section, i.e., (13), (16), (17) and adding a relaxation step. The algorithm in compact form is expressed as while the local updates and the initial condition of SD-GENO are provided in Algorithm 1 . It is composed of two main phases: (19), while η ∈ (0 , 1) and ρ ∈ (0 , 1] .

Iteration k :
Communication: each i ∈ N gathers λ j (k ) from the neighbors and updates the disagreement vector the communication with the neighbors and the local update. First, each agent gathers the information about the strategies and the dual variables of the neighbors. Next, the local update is performed, based on a gradient descent and dual ascend structure. It is worth noticing that only one round of communication is required at each iteration of SD-GENO. The convergence of SD-GENO to the vGNE of the game in (2) is proven in the following theorem. Theorem 2. Set the step sizes ε i , δ, τ i , for all i ∈ N , and ϑ ∈ R such with χ as in Lemma 1 and η ∈ 0 , 4 χϑ−1 2 χϑ . Then, the sequence ( x (k )) k ∈ N generated by SD-GENO ( Algorithm 1 ) converges to the vGNE of the game in (2) .

Asynchronous distributed algorithm with edge variables (AD-GEED)
In the case of heterogeneous agents with very different update rates, SD-GENO can converge slowly, due to its synchronous structure. To overcome this limitation, we introduce here the Asynchronous Distributed GNE Seeking Algorithm with Edge variables (AD-GEED). It uses edge auxiliary variables { σ l } l∈{ 1 ... E} and an asynchronous update to compute the vGNE of the game in (2) . As discussed in the previous section, the use of edge-based auxiliary variables may lead to a limited scalability of the final algorithm. In Section 6 , we use AD-GEED as a starting point to develop an algorithm relying on node variables only. From a technical point of view, the asynchronicity is achieved by exploiting an asynchronous framework for fixed-point iterations, the so called "ARock" framework, developed in [26] .
We assume that each agent i is equipped with public and private memory, the former is used by the neighbors to write their strategies (and dual/auxiliary variables) when they complete an update. The latter instead is used by i to store a copy of the public memory, when it is performing a local update. This memory is not accessible to the neighbors, so it ensures the consistency of the local updates, refer also to [26] . If an agent j ∈ N i concludes its update while agent i is still computing its future strategy during iteration k , then the value of the strategy of j , which agent i is using, becomes outdated. We denote the vector of possibly outdated strategy used for the update during iteration k as ˆ (k ) . All the variables updated by an agent i , i.e., x i , λ i and { σ l } l∈E out i , share the same delay ϕ i (k ) ∈ N , since they are written at the same moment in the neighbors' public memories of its neighbors. Technically, the components of ˆ (k ) associated to agent j = i used during the k -th iteration by agent i for the According to this, the final formulation of the update rule (20) becomes The only assumption that we impose over the delay, is boundedness, as formalized next.
Assumption 4 (Bounded maximum delay) . The delays are uniformly upper bounded, i.e. there exists φ > 0 such that The local update rules of AD-GEED are presented in Algorithm 2 and they are achieved via steps similar to those introduced in Section 4.1 for SD-GENO. To ease the notation, for each agent j ∈ N , we define ˆ and ˆ σ l := σ l (k − ϕ j (k )) , for all l ∈ E out j , and furthermore that each agent has always access to the most recent value of its variables, i.e., ϕ i (k ) = 0 for every agent i ∈ N .
The following convergence theorem is achived by exploiting the results in [26] for a Krasnosel'ski ȋ asynchronous iteration. The structure of AD-GEED is similar to that of ADAGNES in [31 , Algorithm 1], where edge auxiliary variables are used to achieve consensus over the dual variables. However, unlike ADAGNES, our algorithm can handle inequality coupling constraints. Moreover, it has better performances, in terms of convergence time, according to our numerical experience, see Figure 3 . Input: k = 0 , x 0 ∈ R n , λ 0 ∈ R mN , σ 0 = 0 mM , chose δ, ε i , τ i satisfying ( ?? ) and η ∈ (0 , 1) .
Iteration k : Select the agent i k with probability Reading: Agent i k copies in its private memory the current values of the public memory, i.e. ˆ x j , ˆ λ j , ∀ j ∈ N i k and ˆ σ l , ∀ l ∈ E in i k and l ∈ E out j .

Asynchronous, distributed algorithm with node variables (AD-GENO)
This section presents the main result of the paper, namely, we use AD-GEED as a backbone to design an algorithm converging in the same number of iteration, but relying on node auxiliary variables only, and therefore intrinsically lighter from a computational point of view. We name it Asynchronous Distributed GNE Seeking Algorithm with Node variables (AD-GENO). It is based on an idea akin to the one used to develop SD-GENO. In fact, the local update of λ i in AD-GEED requires only the aggregate quantity ([ V ] i I m ) σ. We introduce a variable z i to capture the variation of this aggregate quantity and show that it does not affect the dynamics of the pair ( x i , λ i ), thus preserving the convergence proved in Theorem 3 . Unlike the synchronous case, we cannot directly define z = V σ, due to the different update frequencies of { σ l } l∈E i and z i that would affect the dynamics of λ. This mismatch is clarified via the following example.

Example 2.
Consider the communication network in Example 1 and assume that in the first three time instances, agent 2 updates twice and then 3 updates once, i.e., i 0 = i 1 = 2 and i 2 = 3 . For k = 1 , according to Algorithm 2 it holds where ∝ is used to describe dependency. Next, for k = 2 only λ 3 is updated, then If we substitute the edge variables σ 1 , 2 , 3 , and apply the same activation sequence, it leads to From the comparison of (24) and (25) , it is clear that the value of λ 3 (3) would be different in the two cases. This is explained by the fact that σ 2 is updated twice, while z 3 only once.
To bridge the gap between σ and z , we introduce an extra variable μ i ∈ R m for each node i . The role of μ i is to store the changes of the neighbors dual variable λ j , during the time between the last update of i and the next one.
In Algorithm 3 we present the local update rules of AD-GENO.
Iteration k : Select the agent i k with probability Reading: Agent i k copies its public memory in the private one, i.e., the values ˆ x j , ˆ λ j , ∀ j ∈ N i k , and μ i k . Reset the public values of μ i k to 0 m .

Update:
Writing: in the public memory of each j ∈ N i k The convergence of AD-GENO is proven by the following theorem.
Essentially, we show that introducing z and μ does not affect the dynamics of ( x , λ).

Remark 3.
Only one extra scalar variable μ i is used for every agent i ∈ N , and hence the benefits of adopting only node variables, discussed in Remark 1 , hold also in this asynchronous counterpart. Furthermore, the number of required communication rounds between agents does not increase, since the variable μ i is updated by the neighbors of agent i during their writing phase.

Simulations
We conclude by proposing two sets of simulations to validate the theoretical results in the previous sections. First, we apply AD-GENO on a network Cournot game and study how delays and different activation sequences affect the convergence. Then, we compare the total computation time required by AD-GENO, AD-GEED

AD-GENO convergence
In a network Cournot game, N firms compete over m markets and the coupling constraints arise from the maximum markets capacities. We consider a smilar formulation to that proposed in [30] . Here, we considered N = 8 firms, with the possibility to act over m = 3 markets, i.e., x i ∈ R 3 , for all i ∈ N . The local production is bounded in 0 ≤ x i ≤ x i , where each component of x i ∈ R 3 is randomly drawn from [10,45]. In Fig. 1 a, the interaction of each firm with the markets is shown, where an edge is drawn between a firm and a market if one of former's strategies is applied to the latter. Two firms are neighbors if they compete over the same market, therefore the communication network between the firms is the one in Fig. 1 0 and it is applied to market j . Each nonzero element in A is randomly chosen from [0. 6,1], this value can be seen as the efficiency of a strategy on a market. The components of b ∈ R 3 are the capacities of the markets, randomly drawn from [20,100]. The local cost function is defined as and it describes the cost of opting for a certain strategy, while P ( Ax ) is the reward attained.
The price is assumed linear in its argument P (z) = P − Dz, where P ∈ R 3 and D ∈ R 3 ×3 is a diagonal matrix, their non zero components are randomly chosen from [250,500] and [1,5] respectively.
In order to explore different setups we simulate three different cases: (A) The communication is delay free, i.e., φ = 0 , and the activation sequence is alphabetic, and hence P [ ζ (k ) The activation sequence is still alphabetic, but the communication may be delayed of 3 time instants at most, i.e., φ = 3 . (C) The communication has no delay, but the probability of update is different between agents, half of them have p i = 1 6 , while the rest p i = 1 12 .
The outcome of these scenarios are presented in Fig. 2 . The main difference from case (A) can be noticed if there is a nonuniform update probability, i.e., case (C). In fact, we notice that a skewer probability implies slower converge. From simulations, we noticed that the convergence of the dual variables is often the bottleneck to high convergence performances. In all our algorithms, we mitigated this effect by an appropriate tuning of ρ.

Comparison between algorithms
Next, we compare the performance of AD-GENO with respect to AD-GEED and ADAGNES, from a computational time point of  view. For the comparison with ADAGNES, we consider a modified version of the Nash-Cournot game presented in Section 7.1 with coupling equality constraints, i.e., only A x = b. Here, we consider N = 40 firms, each with at most n i = 2 products. To provide an extensive comparison, we considered many instances of this game varying the communication between agents, from a complete to a sparse graph. The other quantities in the games are chosen as in the previous section. We compared the algorithms over 160 different graphs. The computational time required to obtain convergence is compared in the three cases. 1 The results of the simulations are presented in Fig. 3 . As expected AD-GENO always outperforms AD-GEED, since it achieves the same dynamics of ( x , λ) with fewer auxiliary variables. As expected, the gap between the two algorithms shrinks for a sparse graph while it increases for a dense one, from ∼ 3% to ∼ 20%. A similar behavior arises when AD-GENO is compared to ADAGNES, due to the increment of auxiliary variables for highly connected graphs. In particular, the advantage in using AD-GENO starts from ∼ 20% when the graph has an average degree of 3 and becomes ∼ 60% when the graph is complete.

Conclusion
The AD-GENO algorithm developed in this work solves GNE seeking problems in strongly monotone games via an asynchronous update scheme. It adopts node variables only, and ensures resilience to delayed information. In our numerical experience, AD-GENO outperforms the available solutions in the literature, both in terms of computational time and number of variables required.
Unfortunately, the "ARock" framework does not ensure robustness to lossy communication. This is currently an open problem that is left to future research. Another interesting topic is the generalization of the algorithm to the case of time-varying communication networks, as the independence from the edge variables makes the structure of AD-GENO more suitable to address this problem.

Declaration of Competing Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper. The proof follows similar steps to that of [31 ,Lem. 5], hence we omit it.

(C.4)
From the definition given in Algorithm 3 of μ i , we attain that , therefore (C.1) hold.
Induction step: Suppose that (C.1) holds for some k > 0 that corresponds to the latest iteration in which agent i performed the update, i.e. z i ( k ) = 0 .
Consider the next iteration k in which agent i updates, k > k .
Here, S j (k ) is defined as above, but for time indexes ( k , k ] . Following similar reasoning in the previous case, we obtain where we used the fact that l ∈ E out i is updated at the same time of i . Furthermore, from the induction assumption, where the last step holds because in the reading phase of Algorithm 3 , we reset to zero the values of μ i , every time that i starts an update. Therefore, (C.1) holds for k .
Finally, the convergence of ( x (k )) k ∈ N to the vGNE of the game (2) follows from Theorem 3 .