Frequency-driven market mechanisms for optimal dispatch in power networks

This paper studies real-time bidding mechanisms for economic dispatch and frequency regulation in electrical power networks. We consider a market administered by an independent system operator (ISO) where a group of strategic generators participate in a Bertrand game of competition. Generators bid prices at which they are willing to produce electricity. Each generator aims to maximize their profit, while the ISO seeks to minimize the total generation cost and to regulate the frequency of the system. We consider a continuous-time bidding process coupled with the swing dynamics of the network through the use of frequency as a feedback signal for the negotiation process. We analyze the stability of the resulting interconnected system, establishing frequency regulation and the convergence to a Nash equilibrium and optimal generation levels. The results are verified in the IEEE 14-bus benchmark case.


I. INTRODUCTION
Power generation dispatch is typically done in a hierarchical fashion, where the different layers are separated according to their time scales. Broadly, at the top layer economic efficiency is ensured via market clearing and at the bottom layer frequency control and regulation is achieved via primary and secondary controllers. However, the intermittent and uncertain nature of distributed energy resources (DERs) and their integration into the power grid represents a major challenge to the current design. Of particular concern is the need to maintain both frequency regulation and cost efficiency of regulation reserves in the face of increasing fluctuations in renewables. To this end, we propose an integrated dynamic market mechanism which combines the real-time market and frequency regulation, allowing competitive market players, including renewable generation, to negotiate electricity prices while using the most recent information on the grid frequency.
Literature review: The combination of economic dispatch and frequency regulation has received increasing attention in recent years. Various works have sought to move beyond the traditional and compartmentalized hierarchical control layers to instead simultaneously achieve frequency stabilization and economic dispatch in power networks [2], [3], [4] and microgrids [5], [6]. Along this line of research, the various agents involved work cooperatively towards the satisfaction of a common goal. An alternative body of research has investigated the use of price-based incentives for economic generation-and demand-side management and frequency regulation [7], [8], [9]. To achieve these goals, these works consider dynamic pricing mechanisms in conjunction with system dynamics of the power network. We also adopt this approach, with the important distinction that here we allow generators to bid in the market (hence, they are price-setters rather than pricetakers). This viewpoint results in a Bertrand game of competition among the generators. Our previous work [10], [11] studied this type of games established that iterative bidding can achieve convergence to an optimal allocation of power generation, without considering the effects on the dynamics of the power network. The underlying assumption was that generation setpoints could be commanded after convergence, which in practice poses a limitation, considering the fast timescales at which DERs operate. Instead, this paper proposes an online bidding scheme where the setpoints are updated continuously throughout time to better cope with fast changes in the network. In this way, we tackle simultaneously both frequency regulation, optimal power dispatch and the competitive aspect among the generators.
Statement of contributions: We consider an electrical power network consisting of an independent system operator (ISO) and a group of competitive generators. Each generator seeks to maximize its individual profit, while the ISO aims to solve the economic dispatch problem and regulate the frequency. Since the generators are not willing to share their cost functions, the ISO is unable to solve the economic dispatch problem. Instead, it has the generators compete in a bidding market where they submit bids to the ISO in the form of a price at which they are willing to produce electricity. In return, the ISO determines the power generations levels the generators have to meet. We analyze the underlying Bertrand game among the generators and characterize the Nash equilibria that correspond to optimal power dispatch termed efficient Nash equilibria. In particular, we establish the existence of such efficient Nash equilibria and provide a sufficient condition for its uniqueness. We also propose a Nash equilibrium seeking scheme in the form of a continuous-time bidding process that captures the interaction between the generators and the ISO. In this scheme, the generators adjust their bid based on their current bid and the production level that the ISO requests from them with the aim to maximize their profit. At the same time, the ISO adjusts the generation setpoints to minimize the total payment to the generators while taking the power balance and frequency deviation into account. Moreover, along the execution of the algorithm the nonnegativity constraints on the bids and power generation quantities are satisfied. The use of the local frequency error as a feedback signal in the negotiation process couples the ISO-generator coordination scheme with the swing dynamics of the power network. We show that each equilibrium of the interconnected system corresponds to an efficient Nash equilibrium, optimal generation levels and zero frequency regulation. We furthermore establish local convergence to such an equilibrium by invoking a suitable invariance principle for the closed-loop projected dynamical system. Finally, the numerical results on the IEEE 14-bus benchmark show fast convergence of the closed-loop system to an optimal equilibrium, even under sudden changes of the load and the cost functions.
Notation: Let R, R ≥0 , R >0 be the set of real, nonnegative real, and positive real numbers, respectively. We write the set {1, . . . , n} compactly as [n]. We denote by 1 ∈ R n the vector whose elements are equal to 1. Given a twice differentiable function f : R n → R, its gradient and its Hessian evaluated at x is written as ∇f (x) and ∇ 2 f (x), respectively. A twice continuously differentiable function f : R n → R is strongly convex on S ⊂ R n if it is convex and, for some µ > 0, its For vectors a, b ∈ R n , [a] + b denotes the vector whose i-th element is given by Given a set of numbers v 1 , v 2 , . . . , v n ∈ R, col(v 1 , . . . , v n ) denotes the column vector v 1 , . . . , v n T and likewise diag(v 1 , . . . , v n ) denotes the n × n diagonal matrix with entries v 1 , . . . , v n on the diagonal. For u, v ∈ R n we write u ⊥ v if u T v = 0. We use the compact notational form 0 ≤ u ⊥ v ≥ 0 to denote the complementarity conditions Each bus i represents a control area and is assumed to have one generator and a load P di . The dynamics at the buses is assumed to be governed by the swing equations [12], given byδ = ω with P d = col(P d1 , . . . , P dn ). Here Γ = diag(γ 1 , . . . , γ m ), corresponds to the edge between nodes i and j. Table I presents a list of symbols employed in the model (2).
δ ∈ R n voltage phase angle ω ∈ R n frequency deviation w.r.t. the nominal frequency Pg ∈ R n ≥0 power generation diagonal matrix of asynchronous damping constants negative of the susceptance of transmission line (i, j) For the stability analysis carried out later, it is convenient to work with the voltage phase angle differences ϕ = D T t δ ∈ R n−1 . Here D t ∈ R n×(n−1) is the incidence matrix of an arbitrary tree graph on the set of buses [n] (e.g., a spanning tree of the physical network). Furthermore, let denotes the Moore-Penrose pseudo-inverse of D t . Then the physical system (2) in the (ϕ, ω)-coordinates takes the forṁ where we observe that D t D † t D = (I − 1 n 11 T )D = D.

III. PROBLEM DESCRIPTION
In this section we formulate the problem statement, introduce the necessary game-theoretic tools and discuss the goals of the paper.

A. ISO-generator coordination
Taking as starting point the electrical power network model described in Section II, here we outline the elements of the ISO-generator coordination problem following the exposition of [10], [11]. Let C i : R ≥0 → R ≥0 be the cost incurred by generator i ∈ [n] in producing P gi units of power. We assume C i is strongly convex on the domain R ≥0 and satisfies ∇C i (0) ≥ 0. Given the total network cost and a power load P d , the ISO seeks to solve the economic dispatch (ED) problem and, at the same time, to regulate the frequency of the physical power network. We assume the total load to be positive, i.e., 1 T P d > 0 such that (5) is feasible. Since the constraints (5b) (5c) are affine, Slater's condition holds implying that (5) has zero duality gap. We can also show that its primaldual optimizer (P * g , λ * , µ * ) is unique by exploiting strong convexity of C. We assume that for the power injection P g = P * g , there exists an equilibrium (φ,ω) of (3) that satisfies D T D †T tφ ∈ (−π/2, π/2) m . The latter assumption is standard and is referred to as the security constraint in the power systems literature [12].
We note that the ISO cannot determine the optimizer of the ED problem (5) because generators are strategic and they do not reveal their cost functions to anyone. Instead, the ISO operates a market where each generator i ∈ [n] submit a bid b i ∈ R ≥0 in the form of a price at which it is willing to provide power. Based on these bids, the ISO aims to find the power allocation that meets the load and minimizes the total payment to the generators. Thus instead of solving the ED problem (5) directly, the ISO considers, given a bid b ∈ R n ≥0 , the convex optimization problem (6c) A fundamental difference between (5) and (6) is that the latter optimization is linear and may in general have multiple solutions. Let P opt g (b) be the optimizer of (6) the ISO selects given bids b and note that this might not be unique. Knowing the ISO's strategy, each generator i bids a quantity b i ≥ 0 to maximize its payoff where P opt gi (b) is the i-th component of the optimizer P opt g (b). Note that this function is not continuous in the bid b. Since each generator is strategic, we analyze the market clearing, and hence the dispatch process explained above using tools from game theory [13], [14].

B. Inelastic electricity market game
We define the inelastic electricity market game as • Players: the set of generators [n].
• Action: for each player i, the bid b i ∈ R ≥0 . • Payoff: for each player i, the payoff Π i defined in (7). In the sequel we interchangeably use the notation b ∈ R n represents the bids of all players except i. We note that the payoff of generator i not only depends on the bids of the other players but also on the optimizer P opt g (b) the ISO selects. Therefore, the concept of a Nash equilibrium is defined slightly differently compared to the usual one.
Definition III.1 (Nash equilibrium [11]). A bid profile b * ∈ R n ≥0 is a Nash equilibrium of the inelastic electricity market game if there exists an optimizer P opt g (b * ) of (6) such that for each i ∈ [n], . We are particularly interested in bid profiles for which the optimizer of (5) is also a solution to (6). This is captured in the following definition.
Definition III.2 (Efficient bid and efficient Nash equilibrium). An efficient bid of the inelastic electricity market is a bid b * ∈ R n ≥0 for which the optimizer P * g of (5) is also an optimizer of (6) given bids b = b * and is an efficient Nash equilibrium of the inelastic electricity market game if it is an efficient bid and a Nash equilibrium.
At the efficient Nash equilibrium, the optimizer of the ED problem coincides with the production levels that maximize the individual profits (7) of the generators. This justifies studying the efficient Nash equilibria.

C. Paper objectives
Given the problem setup described above, neither the ISO nor the individual strategic generators are able to determine the efficient Nash equilibrium a priori. As a first objective, we are interested in designing a Nash equilibrium seeking mechanism in the form of a bidding process where the generators coordinate with the ISO to dynamically update their bids and production levels, while respecting the nonnegativity constraints throughout its execution. Our second objective is the characterization of the stability properties of the interconnection of the bidding process with the physical dynamics of the power network.

IV. EXISTENCE AND UNIQUENESS OF NASH EQUILIBRIA
In this section we establish existence of an efficient Nash equilibrium and also provide a condition for its uniqueness. While [11] has established the existence of one specific efficient Nash equilibrium, we provide in the following result a characterization of all efficient Nash equilibria.
Suppose P * gi > 0 for at least two distinct generators. Then, is an efficient Nash equilibrium of the inelastic electricity market game.
Proof. Let (P * g , λ * , µ * ) satisfy (9), then in particular 1λ * ≤ ∇C(P * g ). Fix any bid b * ∈ R n ≥0 satisfying 1λ * ≤ b * ≤ ∇C(P * g ). We will now prove that b * is efficient. Define ν * := b * − 1λ * and note that (P * g , λ * , ν * ) satisfies b * = 1λ * + ν * , We note that Slater's condition holds for (6) and its KKT conditions are given by (10). Consequently, P * g is a primal optimizer of (6). In addition, the bid b * satisfies This is true as for each i ∈ [n], the following optimality conditions Thus, we have established that b * is efficient. In the remainder of the proof we show that b * is a Nash equilibrium. Suppose generator i deviates from the bid b * i . We distinguish between two cases. Suppose first that (6) and checking the optimality conditions, we obtain P opt gi (b i , b * −i ) = 0 as, by assumption, there is at least one other generator j such that b * j = λ * < b i . Without loss of generality assume that where the second inequality follows from (11) as b * is efficient. Hence, each generator i has no incentive to deviate from bid b * i given b * −i . We conclude that b * is an efficient Nash equilibrium of the inelastic electricity market game.
The proof of Proposition IV.1 shows that if P * gi > 0, then generator i's efficient Nash equilibrium bid b * i is equal to the (unique) Lagrange multiplier λ * associated to the power balance (5b). In the other case that P * gi = 0, generator i's Nash equilibrium bid is larger than or equal to λ * . This represents the case that generator i's marginal costs at zero power production is larger than or equal to the market clearing price, and hence generator i is not willing to produce any electricity in that case. The underlying assumption in Proposition IV.1 is that at least two generators have a positive production at the optimal generation levels. We assume this condition holds for the remainder of the paper unless stated otherwise.
The previous observations lead to the identification of the same sufficient condition as in [11] to guarantee the uniqueness of the efficient Nash equilibrium, which we state here for completeness.
Corollary IV.2 (Uniqueness of the efficient Nash equilibrium [11]). Let (P * g , λ * , µ * ) be the primal-dual optimizer of (5) and suppose that P * g > 0, then b * = ∇C(P * g ) = 1λ * is the unique efficient Nash equilibrium of the inelastic electricity market game. Remark IV.3 (Any efficient Nash equilibrium is positive). We observe from the optimality conditions (9) that, since 1 T P d > 0, and P * g ≥ 0, we must have that P * gi > 0 and µ * i = 0 for some i ∈ [n]. As ∇C i (P * gi ) > 0 by the strict convexity of C i and the assumption ∇C i (0) ≥ 0, this implies that λ * > 0 and therefore also b * > 0. •

V. INTERCONNECTION OF BID UPDATE SCHEME WITH POWER NETWORK DYNAMICS
In this section we introduce a Nash equilibrium seeking mechanism between the generators and the ISO. Each generator dynamically updates its bid based on the power generation setpoint received from the ISO, while the ISO changes the power generation setpoints depending on the generator bids and the frequency of the network. This update mechanism of the bids and the setpoints is written as a continuous-time dynamical system. We assume that each generator can only communicate with the ISO and is not aware of the number of other generators participating, their respective cost functions, or the load at its own bus. We study the interconnection of the online bidding process with the power system dynamics and establish local convergence to an efficient Nash equilibrium, optimal power dispatch, and zero frequency deviation.

A. Price-bidding mechanism
In our design, each generator i ∈ [n] changes its bid b i ≥ 0 according to the projected dynamical system with gain τ bi > 0. The projection operator in the above dynamics ensures that trajectories starting in the nonnegative orthant remain there. The map C * i : R ≥0 → R ≥0 denotes the convex conjugate of the cost function C i and is defined as Using tools from convex analysis [15, Section I.6], one can deduce that C * i is convex and continuously differentiable on the domain R ≥0 and strictly convex on the domain The motivation behind the update law (12a) is as follows. Given the bid b i > 0, generator i seeks to produce power that maximizes its profit, which is given by However, if the ISO requests more power from the generator compared to its desired quantity, i.e., P gi > P des gi , then i will increase its bid to increase its profit. On the other hand if P gi < P des gi , then i will decrease its bid. For the ISO we also provide an update law which depends on the generator bids and the network frequency. This involves seeking a primal-dual optimizer of (6) or, equivalently, finding a saddle-point of the augmented Lagrangian with parameter ρ > 0. By writing the associated projected saddle-point dynamics (see e.g., [16], [17]), the ISO dynamics takes the form with design parameters σ, τ λ ∈ R >0 and diagonal positive definite matrix τ g ∈ R n×n . Bearing in mind the ISO's second objective of driving the frequency deviation to zero, we add the feedback signal −σ 2 ω to adjust the generation based on the frequency deviation in the grid. The dynamics (12b) can be interpreted as follows. If generator i bids higher than the Lagrange multiplier λ (which can be interpreted as a price) associated with the power balance constraint (6b), then the power generation setpoint at node i is decreased, and vice versa. The terms ρ11 T (P g − P d ) and −σ 2 ω in (12b) help to compensate for the supply-demand mismatch in the network.
In the following, we analyze the equilibria and the stability of the interconnection of the physical power network dynamics (3) with the bidding process (12). We assume that the bids and power generations are initialized within the feasible domain, i.e., b(0) ≥ 0, P g (0) ≥ 0.

B. Equilibrium analysis of the interconnected system
The closed-loop system composed of the ISO-generator bidding scheme (12) and the power network dynamics (3) is described bẏ . . , τ bn ) ∈ R n×n . We investigate the equilibria of (13). In particular, we are interested in equilibria that correspond simultaneously to an efficient Nash equilibrium, economic dispatch and frequency regulation, as specified next.
The next result shows that all equilibria of (13) are efficient.

C. Convergence analysis
In this section we establish the local asymptotic convergence of (13) to an efficient equilibrium.
Then X is locally asymptotically stable under (13). Moreover, the convergence is to a point.
Proof. Our proof strategy to show local convergence to X is based on applying Theorem A.1, which is a special case of the invariance principle stated in [18] adapted for complementarity systems. To this end, we rewrite the projected dynamical system (13) as the equivalent complementarity system (17), see also [19,Theorem 1] for more details, where µ b , µ g ∈ R n . We can equivalently write (17) in the compact formẋ with x = col(ϕ, ω, b, P g , λ), Λ = col(µ b , µ g ), and Note that F is Lipschitz continuous 1 . For the equivalence of the projected dynamical system (13) and the complementarity system (17) to hold, we consider absolutely continuous solutions t → x(t) that satisfy (17) almost everywhere (in time) in the sense of Lebesgue measure. In addition, we consider (unique) solutions of (18) that are slow. That is, at each time t, Λ satisfies (18b) and is such thatẋ(t) is of minimal norm, see also [19]. Letx ∈ X be arbitrary and fixed for the remainder of the proof. For aesthetic reasons we first consider the case where σ = 1 in (13d) or (17d) and later we explain how to generalize the convergence result. Consider the function V defined by Consequently, there exists a compact level set Ψ of V aroundx. We show now that the two conditions of Theorem A.1 are satisfied.
Condition (I): For C given in (19b) and d = 0 the polyhedron (24) takes the form Consequently, for all x ∈ ∂K ∩ Ψ we have

Condition (II):
Sincex ∈ X there existsΛ such that F (x)+ C TΛ = 0. As a result, for each x ∈ K we have 1 Here we observe that, since C is continuously differentiable and µ-strongly convex on R ≥0 , C * is 1 µ -Lipschitz continuous on R ≥0 . Each edge in the graph represents a transmission line. Red nodes represent loads. All the other nodes represent synchronous generators, with different colors that match the ones used in Figure 2. The physical dynamics are modeled by (2).
where the inequality holds because C * is convex,b Tμ b = 0,P T gμg = 0 andμ b ,μ g , b, P g ≥ 0. Hence, the second condition of Theorem A.1 is satisfied.
Invariance of Ψ: We note that (21) does not necessarily imply that Ψ is forward invariant. We show this next. Observe that for each x, Λ satisfying 0 ≤ Cx ⊥ Λ ≥ 0 we have Hence, the V is non-increasing along trajectories initialized in K ∩ Ψ. Since Ψ is a level set of V , this implies that Ψ is forward invariant. Largest invariant set: Define and denote the largest invariant subset of E by M. By (21) we note that each x ∈ M satisfies ω = 0, 1 T (P d − P g ) = 0 and, b i =b i > 0 (otherwise, ifb i = 0, then 0 =P gi − ∇C * i (b i ) + µ bi =P gi +μ bi > 0, which results in a contradiction) for each i ∈ [n] withP gi > 0 as C * i is strictly convex around suchb i . For these i, P gi =P gi > 0 by (13c) and b i = λ =λ by (13d). For each x ∈ M and i ∈ [n] withP gi = 0, we have ∇C * i (b i ) = ∇C * i (b i ) = 0 by the convexity of C i and thus P gi =P gi = µ bi = 0 and thus b i = λ + µ gi . Hence, M ⊂ X and therefore each trajectory initialized in Ψ converges to X . Moreover, from (22), we deduce thatx is stable. Since this equilibrium has been chosen arbitrarily, we conclude that every point in X is Lyapunov stable, implying that convergence of the trajectories is to a point.
The proof for the case σ > 0, σ = 1 proceeds in the same way as before except that we appropriately scale the Lyapunov function. Specifically, we define the Lyapunov function V as in (20) but with τ = blockdiag(0, M, στ b , στ g , στ λ ) > 0.

VI. SIMULATIONS
We simulate the closed-loop dynamics (13) for the modified IEEE 14-bus benchmark model illustrated in Figure 1. We assume quadratic costs at each node i ∈ [14] of the form with q i > and c i ≥ 0. and with P gi = 0 for all other nodes. Figure 2 shows the evolution of the system in the case when σ = 300 and Figure  3 in the case when σ = 0. Note that in the latter case, there is no frequency signal fed back into the bidding process, so the dynamics (13) effectively becomes a cascaded system (where the bidding process drives the physical dynamics of the power network). At t = 1 s the load at node 3 is increased from 80 MW to 94.2 MW and the trajectories converge to a new efficient equilibrium with optimal power generation level (P g1 , P g2 ) = (211, 48) and P gi = 0 for all other nodes. Furthermore, at steady state generators 1, 2 bid equal to the Lagrange multiplier while generators 3, 6, 8 bid their marginal cost at zero production (b i = c i , for i = 3, 6, 8) and thus, by Proposition IV.1, we know that this corresponds to an efficient Nash equilibrium. At t = 15 s the cost of producing electricity is decreased in areas 3, 6, 8 by setting (q 3 , q 6 , q 8 ) = (60, 75, 68) and (c 3 , c 6 , c 8 ) = (38, 45, 23). This allows these generators to make profit by participating in the bidding process and results in a reduction of the total cost of the generation from 9711 $/h to 8540 $/h. As illustrated in both Figures 2 and 3, the power generations converge to the new optimal steady state given by (P g1 , P g2 , P g3 , P g6 , P g8 ) = (161, 29, 21, 7, 41).
In addition, we observe that after each change of either the load or the cost function, the frequency is stabilized and the bids converge to a new efficient Nash equilibrium. The fact that the frequency transients are better in Figure 2 than in Figure 3 is consistent, since in the latter case there is no frequency feedback in the bidding process.

VII. CONCLUSIONS
We have studied a market-based power dispatch scheme and its interconnection with the swing dynamic of the physical network. From the market perspective, we have considered a continuous-time bidding scheme that describes the negotiation process between the independent system operator and a group of competitive generators. Using the frequency as a feedback signal in the bidding dynamics, we have shown that the interconnected projected dynamical system provably converges to an efficient Nash equilibrium (where generation levels minimize the total cost) and to zero frequency deviation. In this way, competitive generators are enabled to participate in the real-time electricity market without compromising efficiency and stability of the power system. Future work will investigate finite-horizon scenarios and incorporate generator bounds and power flow constraints in the economic dispatch formulation.

APPENDIX
Theorem A.1 (Invariance principle for complementarity systems [18]). Consider the systeṁ with Lipschitz continuous F and let K be the polyhedron Let Ψ ⊂ R n be a compact set and V : R n → R be a continuous differentiable function such that for all x ∈ ∂K ∩ Ψ, (II) ∇V (x), F (x) ≤ 0, for all x ∈ K ∩ Ψ.
Let E ⊂ R n be given by   Figure 2(c), the convergence is faster in this scenario because the bidding process does not take into account its impact on the dynamics of the power network. Figure 3: Simulations of the interconnection (13) between the ISO-generation bidding mechanism and the power network dynamics for the case of σ = 0, i.e., when there is no frequency feedback signal in the bidding process. The scenario is the same as in Figure 2. As illustrated, the closed-loop system also converges in this case to an efficient equilibrium.