Distributed Nash equilibrium seeking design for energy consumption games of HVAC systems over digraphs

A novel distributed Nash equilibrium seeking algorithm is proposed to solve the energy consumption problem of heating, ventilation, and air conditioning systems over directed graphs Abstract: The energy consumption problem of heating, ventilation, and air conditioning systems over general directed graphs is investigated. The considered problem is firstly reformulated as a Nash equilibrium seeking problem, and a distributed consensus-based algorithm is then proposed to solve it. To address the challenge arising from general directed graphs, a distributed estimation algorithm is embedded such that the explicit dependence on the left eigenvector associated with the eigenvalue zero of the Laplacian matrix can be avoided. Then, the exponential convergence of the proposed distributed Nash equilibrium seeking algorithm is established under a standing assumption. A numerical example is finally provided to verify the effectiveness of the proposed algorithm.


Introduction
In the past few years, the energy consumption problem has attracted considerable attention due to the growing demand for energy [1−4] . It is worth noting that heating, ventilation, and air conditioning (HVAC) systems are responsible for a large proportion of energy consumption in practice [5,6] . A few centralized approaches have been developed to solve the energy consumption problem of HVAC systems, see, for example, Refs. [7−9]. A limitation of such centralized scheme is its resulting heavy computational burden on the centralized location [10] .
More recently, motivated by the investigation of the distributed control and optimization (see, for instance, Refs. [11−14] and references therein), several distributed algorithms have been proposed to solve the energy consumption problem of HVAC systems. In particular, Refs. [15−18] proposed distributed optimization schemes for HVAC systems to minimize the total energy consumption cost while to maintain the zone thermal comfort. Ref. [19] designed a distributed real-time pricing feedback algorithm for HVAC systems in a noncooperative game scheme. Further, Ref.
[20] developed a distributed Nash equilibrium seeking strategy based on average consensus protocol for the energy consumption game of HVAC systems over undirected graphs.
It should be pointed out that the existing works [16,17,20] only considered the energy consumption problem of HVAC systems for undirected graphs. Nevertheless, it is of much more theoretical and practical significance to consider directed graphs because the information exchange between agents may be unidirectional due to limited bandwidth or other constraints. Recently, by utilizing the left eigenvector associated with the eigenvalue zero of the Laplacian matrix, several distributed algorithms were developed in Refs. [21−23] to cope with directed graphs from various aspects. However, since the left eigenvector is some kind of global information, the abovementioned algorithms may not be applicable when the left eigenvector is not known a priori. This brings difficulties in both design and convergence analysis of the distributed algorithms.
In this paper, the energy consumption problem of HVAC systems over directed graphs is considered. The problem is firstly reformulated by a game model similar to that in Ref. [20]. Then, a novel distributed Nash equilibrium seeking algorithm is proposed in case that the left eigenvector associated with the eigenvalue zero of the Laplacian matrix is unavailable. Exponential convergence of the proposed algorithm is also established under a standing assumption. Main contributions of this paper compared with relevant literatures can be summarized as follows.
(Ⅰ) In contrast with the existing work [20] for undirected graphs, this work considers more general and thus more challenging directed graphs. A new distributed Nash equilibrium seeking algorithm is developed to address such issue of general directed graphs.
(Ⅱ) Different from the distributed design in Refs. [21−23] requiring the exact value of the left eigenvector associated with the eigenvalue zero of Laplacian matrix, our result does not rely on the exact value of the left eigenvector. In fact, it is a challenge to obtain the left eigenvector, especially in largescale networks. Thus, our result gives rise to a promising design in implementation.
The remainder of this paper is organized as follows. Some preliminaries are presented and the energy consumption game is formulated in Section Ⅱ, which is followed by designing a distributed Nash equilibrium seeking algorithm in Section Ⅲ. Then, the effectiveness of the proposed algorithm is validated via a numerical example in Section Ⅳ. Finally, the conclu- sion is stated in Section Ⅴ.
Notations: ‖· ‖ denotes the Euclidean norm. Let , n and n×m denote the sets of real numbers, n-dimensional real vectors and n×m-dimensional real matrices, respectively. x T and A T refer to the transpose of vector x and matrix A, respectively. Let 1 n =[1, 1, …, 1] T ∈ n and 0 n =[0, 0, …, 0] T ∈ n denote the all-zeros vector and the all-ones vector, respectively. I n ∈ n×n refers to the identity matrix. The matrix diag(x 1 , x 2 , …, x n ) denotes the diagonal matrix with the ith diagonal entry being x i . λ min (B) and λ max (B) represent the smallest eigenvalue and the largest eigenvalue of matrix B, respectively. represents the Kronecker product.
represents the gradient vector of f(x).

Preliminary and formulation
In this section, some preliminaries on convex analysis and graph theory are presented, and the energy consumption game of HVAC systems is given.

Preliminary
for all x, y∈ n and 0≤λ≤1 [24] . A function f: n → n is globally Lipschitz on n if there exists a positive constant L such that ‖ f(x)−f(y) ‖≤L‖ x−y‖ for all x, y∈ n . A function f: n → n is u-strongly monotone if there exists a positive constant u such that (x−y) T (f(x)−f(y))≥u‖x−y‖ 2 for all x, y∈ n .
We use a directed graph to describe the information communication among agents. A directed graph is expressed as ={ , , }, where ={1, 2, …, N} is the node set, × is the edge set and is the adjacency matrix. If the ith agent can receive information from the jth agent, there is an edge (j, i)∈ .
A path from node x 0 to node x r in graph is a sequence of edges with the form (x 0 , x 1 ), (x 1 , x 2 ), …, (x r−1 , x r )∈ . If there exists at least a directed path between any two nodes in graph , the directed graph is strongly connected. L=D in − is the Laplacian matrix, where .
It can be verified that L1 N =0 N . For more details of graph theory, please see Ref. [25].

Problem formulation
Consider the energy consumption game for a network of N (N > 3) electricity users been equipped with HVAC systems. Three elements including player, strategy and cost function of the energy consumption game are described as follows. The electricity users are the players, the strategy of the ith player is its energy consumption, and the cost function of the ith player is defined as follows [19] : where represents the load curtailment cost, v i and ξ i are thermal coefficients satisfying v i ξ 2 i > 0. l i is the strategy of the ith player. is the energy needed for the ith user to maintain the indoor temperature. P(σ(l))l i denotes the billing payment for the energy consumption l i .
The pricing function is defined as where is the aggregate strategy of all the players, p 0 is a basic price for energy consumption and a is a positive parameter to implement elastic pricing with 0 < a≤ min 1 ≤ i ≤ N 2v i ξ 2 i being satisfied [20] . [ It can be verified that the cost function C i (l i , σ(l)) is continuously differentiable in l and convex in l i for every fixed Moreover, is strongly monotone. Therefore, the existence and uniqueness of the NE can be guaranteed by Theorem 3 in Ref. [26].
The Nash equilibrium of the energy consumption game is defined as follows.

Remark 2.1
For the ith player, the objective is to minimize its cost function [27] . If the strategy of the ith player is the ith element of the NE solution l * , no player can reduce its cost function value by unilaterally changing its individual strategy. Thus, all players have no motivation to change their strategies at the NE.
In this paper, we aim to solve the following defined problem.
Problem 2.1 For the energy consumption game over directed graphs with cost functions described in Eq. (1), design a distributed algorithm for each player so that the strategy l(t)=[l 1 (t), l 2 (t), …, l N (t)] T converges to the NE l * .
An assumption is needed for designing the distributed Nash equilibrium seeking algorithm. is satisfied [28] .

Distributed Nash equilibrium seeking design for HVAC systems
In this section, we propose a new distributed NE seeking algorithm to solve Problem 2.1. Its design diagram is shown in Fig. 1. The proposed algorithm includes three parts: gradient, average consensus and the distributed estimation of a positive left eigenvector of L associated with the zero eigenvalue. We analyse the convergence of the proposed algorithm by two steps. First, some preliminary results are given. Then, the convergence analysis of the proposed algorithm is presented. In this subsection, a distributed NE seeking algorithm is designed. To proceed, define

Algorithm design
Then we have The distributed NE seeking algorithm is designed aṡ where α 1 , α 2 and α 3 are positive constants to be determined later, τ Define l = [l 1 , l 2 , …, l N ] T , p = [p 1 , p 2 , …, p N ] T , q = [q 1 , q 2 , …, q N ] T , τ = [τ 1 , τ 2 , …, τ N ] T , = diag (τ 1 1 , τ 2 2 , …, τ N N ) and Ξ = diag (υ 1 , υ 2 , …, υ N ). Then the distributed algorithm (4) can be rewritten in the following compact form,l Under Assumption 2.1, we obtain that exp(−Lt)(t > 0) is a nonnegative matrix with positive diagonal entries and [28] . By referring to the initial condition (5), it can be deduced that Remark 3.1 In algorithm (4), H i (l i , p i ) is the gradient information for seeking the NE. The dynamics of p i are an average consensus for estimating the aggregate variable σ(l). The q i dynamics are designed for eliminating the average consensus error. The τ i dynamics are needed to cancel the imbalance arising from the directed graphs, and the requirement on the left eigenvector of the Laplacian matrix associated with the zero eigenvalue thus can be removed.τ

Remark 3.2 Since exp(−Lt)(t > 0) is a nonnegative matrix
with positive diagonal entries, it can be proved that τ i i (t) > 0 for t ≥ 0 by noting the initial condition (5). Thus, is invertible. The Eq. (7) implies that and = Ξ −1 .

Preliminary resultω
To facilitate the convergence analysis, some lemmas are presented in this subsection. Define . Then the dynamics of can be described as follows, The following lemma shows the optimality analysis of the concerned system.
Proof Let be the equilibrium point of the system . Then, it is satisfied that According to Eq.(8c), we obtain that . Note that υ is the positive left eigenvector of L associated qq with the zero eigenvalue. Left multiplying by υ T yields υ T = 0. This implies that Then multiplying both sides of Eq.(8b) by υ T , we have υ T L = 0 T N and υ T Ξ -1 = 1 T N . One then has , . Thus, is satisfied by using Eq.(8a). This implies that , in other words, is the NE of the considered game by Lemma 2 in Ref. [29]. This completes the proof.
To proceed, introduce the coordinate transformations and . Then referring to Eq. (6), we can obtain the following system, To show the stability of Eq.(10), we establish an auxillary stability result on the truncated systeṁ ω = D 1 (t, ω) (11) which is summarized in the following lemma. Lemma 3.2 Under Assumption 2.1, there exist positive constants α 1 , α 2 and α 3 such that the origin is a globally exponentially stable equilibrium point of the system (11).
Proof The dynamics of , and satisfẏ Consider the following Lyapunov function candidate, By referring to Eq. (3), we obtain thaṫ , v i∈V where =min {2v i ξ 2 i }. Moreover, we havė T Ξq. With the obtained facts, one haṡ By Lemma 2.2 in Ref. [11], the eigenvalues of can be ordered as 0=λ 1 It follows from the Eq. (9) that T Ξ1 N =0. Then, we havē Choose the positive constants α 1 , α 2 , α 3 , δ 1 and δ 2 such that the following inequalities are satisfied, This reveals that the origin is a globally exponentially stable equilibrium point of the system by applying Theorem 4.10 in Ref. [30]. The proof is thus completed. w = G 1 (t,w) Remark 3.3 Lemma 3.2 indicates that if υ is available in advance, we can design a distributed NE seeking algorithm of the form such that l(t) can exponentially converge to the desired NE l * .

Main result
We show the convergence of algorithm (6) in this subsection. The main result of this paper is given below. Theorem 3.1 Under Assumption 2.1, there exist positive constants α 1 , α 2 and α 3 such that the trajectory of l(t) generated by algorithm (6) exponentially converges to the NE l * of the considered aggregative game.
ω Proof First, recall the dynamics of in Eq. (10). According to Lemma 3.2, there exist positive constants α 1 , α 2 and α 3 such that the origin is a globally exponentially stable equilibrium point of the system (11).
Second, note that D 2 (t, 0 3N )=0 3N . Moreover, it is satisfied that It then follows from Corollary 9.1 and Lemma 9.5 in Ref. [30] that the origin is a globally exponentially stable equilibrium point of the system . Thus, according to Theorem 4.14 in Ref. [30], there is a function satisfying the inequalities ≤ , for some positive constants d 1 , d 2 , d 3 and d 4 .
Finally, exponential stability of system (10) is presented. It then follows from the proof of Theorem 2 in Ref. [14] that there exist two positive constants c 1 and such that γ(t)≤c 1 e -c 2 t . Thus, it can be verified that ‖ D 3 (t) ‖≤c 3 e -c 2 t , where . Therefore, the derivative of along the trajectories of the system (10) can be described aṡ and . By the comparison lemma in Ref. [30], it can be obtained that where c 4 =min{d 5 , c 2 } and . This implies that converges to zero exponentially. One can further claim that converges to zero exponentially. It then follows that l(t) converges to l * exponentially, and the proof is thus completed.

Remark 3.4
The related work [20] mainly focuses on the energy consumption problem of HVAC systems over undirected graphs. On the contrary, this paper considers the more general and also more challenging scenarios in directed graphs. It is worth noting that the selection of the control gains α 2 and α 3 in algorithm (6) relies on the second smallest eigenvalue of and the minimum value of the left eigenvector υ. Some distributed algorithms [31,32] can be utilized to estimate them in advance.
Remark 3.5 A relevant result can be found in Ref. [33], where a distributed algorithm was developed to address the Nash equilibrium seeking problem. However, the following two significant differences should be noted. Firstly, this work ensures that the trajectory of l(t) generated by the designed algorithm converges to the desired NE exponentially, while the algorithm in Ref. [33] can only achieve asymptotic convergence according to Theorem 2 in Ref. [33]. Secondly, this work focuses on a more practical scenario in HVAC systems, while it is not considered in Ref. [33].

An example
In this section, a numerical example is given to illustrate the effectiveness of the proposed algorithm.
Consider an energy consumption game of eight electricity users that are equipped with HVAC systems. The cost function for each electricity user i is given as follows.