Real-Time Pricing-Based Scheduling Strategy in Smart Grids: A Hierarchical Game Approach

This paper proposes a scheduling strategy based on real-time pricing in smart grids. A hierarchical game is employed to analyze the decision-making process of generators and consumers. We prove the existence and uniqueness of Nash equilibrium and utilize a backward induction method to obtain the generation and consumption strategies. Then, we propose two dynamic algorithms for the generators and consumers to search for the equilibrium in a distributed fashion. Simulation results demonstrate that the proposed scheduling strategy can match supply with demand and shift load away from peak time.


Introduction
Matching supply with demand has been an active topic in power systems.Traditionally, we have to provide enough generation capacity to meet peak demand [1].This requires substantial infrastructures to be idle for all but a few days a year.In recent years, demand response (DR) has been proposed to reduce the peak demand by managing the consumption of consumers [2].DR can be implemented by various pricing strategies, such as time of use (TOU), critical peak pricing (CPP), extreme day CPP (ED-CPP), extreme day pricing (EDP), and real-time pricing (RTP) [3].For the RTP strategy, the electricity price is published on a rolling basis.The price for a given time period is determined before the start of the period.With the development of smart grids, RTP can be implemented within a shorter period.
Recently, game theory has been applied to the DR programs [4].For instance, the congestion game was applied to control the electricity consumption by dynamic pricing [5], the noncooperative game was utilized to find the pricing condition for demand response [6] and minimize the electricity cost to consumers [7,8], and the Stackelberg game was applied to residential power scheduling [9], energy exchange between electric vehicles and smart grids [10], and demand response with one energy provider [11] and multiple energy providers [12].Nevertheless, few papers are devoted to the DR program with the integration of generation scheduling.In this study, we propose an integrated scheduling strategy for the generators and the consumers based on a hierarchical game approach and formulate the competitions among the generators into a noncooperative game at the leader level.The main contributions are as follows.
(i) The generation and consumption scheduling problem is formulated as a hierarchical game, where the generators act as the leaders and the consumers act as the followers.
(ii) The existence and uniqueness of Nash equilibrium are proved, and the optimal generation and consumption strategies are obtained by a backward induction method.
(iii) Two dynamic algorithms are proposed to search for the optimal generation and consumption strategies in a distributed fashion.
The rest of the paper is organized as follows.The scheduling problem is formulated as a hierarchical game in Section 2. We give the game analysis by the backward induction method in Section 3 and propose two dynamic algorithms to search for the equilibrium in Section 4. Section 5 shows the numerical results.Finally, conclusions are summarized in Section 6.

Problem Formulation
As shown in Figure 1, we consider smart grids consisting of generators, consumers, and an Independent Electricity System Operator (IESO).The IESO is a nonprofit organization, which is responsible for operating the grids and directing the operation of the participants by pricing.Each consumer installs a smart meter to implement two-way communications with the IESO.The intended operation cycle is divided into  time slots indexed by  ( ∈ {1, 2, . . ., }).In each time slot, the generators determine the generation, and then the consumers manage the consumption according to the real-time price published by the IESO.

Consumer Level Formulation.
Let   = {1, 2, . . ., } denote the set of consumers and x  = (  1 , . . .,    , . . .,    ) denote the consumption profile, where    is the consumption of consumer  ( ∈   ) in time slot .To describe the profits of consumers, we consider the following utility functions with linear decreasing marginal benefit [13]: where    (   > 0) represents the willingness to increase the consumption in time slot , and    / denotes the maximal demand for consumer .For instance, the utility functions with different    are shown in Figure 2, from which we see that the utility function   (⋅) has the following properties: where   is the electricity price, and  is a positive parameter.
Each consumer manages the consumption to maximize its payoff, which can be described as the following optimization problems: 2.2.Generator Level Formulation.The generators aim to maximize the profit without too much generation cost.Thus, the payoff of generator  is defined as where      and (   ) are the profit and the cost to generator , respectively.As shown in Figure 3, the generation cost is defined as a quadratic function [14]: where    denotes the generation of generator  in time slot .The generation profile is denoted as L  = (  1 , . . .,    , . . .,    ).The parameters   ,   , and   are positive.Then, the optimal generation can be obtained from the following optimization problems: where   = {1, 2, . . ., } denotes the set of generators.

IESO Pricing Model.
The IESO operates the wholesale markets and sets the electricity price to balance the supply and the demand.Following the principle of locational marginal pricing (LMP), the price needs to reflect the value of electricity at a specific location at the time that it is delivered.
To this end, we construct a real-time pricing function based on the difference between the demand and the supply:  where  0 ( 0 > 0) is a basic price.The parameter  ( > 0) is to implement the elastic pricing such that the supply can meet the demand.

Analysis of Hierarchical Game
In smart grids, the IESO can acquire information from the consumers based on an advanced metering infrastructure (AMI).Thus, the generation scheduling and the consumption scheduling are performed with information asymmetry.Specifically, the generators determine the generation first, and then the consumers manage the consumption in response.This multistage decision process can be formulated as a hierarchical game consisting of a generator level subgame and a consumer level subgame.In particular, the generators act as the leaders, and the consumers act as the followers.Next, we utilize the backward induction method to analyze the hierarchical game and omit  for convenience in the following sections.

Consumer Level Subgame.
We first give the definitions of the consumer level subgame and the Nash equilibrium in the game [15].Definition 1.A consumer level subgame is defined as a triple  = {  , (  ) ∈  , (   (x)) ∈  }, where   is the set of active consumers participating in the game,   = {  | 0 ≤   ≤  max  } is the set of possible strategies that consumer  can take, and is the payoff function.It is noted that the Nash equilibrium is a set of strategies where no one has an incentive to change unilaterally.Differentiating    (x) with respect to   , we have According to Definition 2, we know that the Nash equilibrium can be obtained from ℎ  (x) = 0, which gives the following equations: Next, we derive the Nash equilibrium in the following theorem.
Theorem 3. The consumer level subgame  has a unique Nash equilibrium if and the equilibrium is given as Proof.The constraints (11) can be obtained directly from the pricing condition in [6], and the consumer level subgame has a unique Nash equilibrium.Next, we will give the explicit expression of the unique equilibrium.From (10), we have Adding the two sides of (13) from 1 to , we obtain Then, the optimal consumption ( 12) is obtained by substituting ( 14) into (13).

Journal of Applied Mathematics
The objective of the IESO is to achieve the balance between supply and demand: Combining with (14), we calculate the pricing parameter  as Substituting ( 15) and ( 16) into (7), we obtain the pricing function: with which, the balance between supply and demand is achieved.

Generator Level Subgame.
In this section, we consider the profit allocation among the generators with the knowledge of pricing function in (17).Substituting ( 17) into (4), we obtain the payoff of generator : Then, the generator level subgame can be defined as follows.
Definition 4. A generator level subgame is defined as a triple  = {  , (  ) ∈  , (   (L)) ∈  }, where   is the set of active generators participating in the game,   is the set of possible strategies that generator  can take, and the payoff is denoted by (18).
Next, we need to study the existence and uniqueness of the Nash equilibrium for the generator level subgame , which requires the following lemma.
Remark 7. To implement RTP in smart grids, we choose the linear pricing function.The reasons are as follows.First, according to the technical report from U. S. Department of Energy [19], the electricity price is almost linearly increased with the total load.Second, a pricing condition in [6] shows that the linear pricing function is sufficient to make the demand response stable.Last but not least, the linear pricing function is easy to ensure the existence of a unique Nash equilibrium and derive the explicit expression of the Nash equilibrium in the consumer level subgame, which is necessary to the backward induction method in hierarchical model.In practice, the pricing function ( 7) is not necessary to be linear.For example, we can also choose the quadratic pricing function.
Substituting (24) into the payoff function ( 2), we obtain Then, we can obtain the second derivative of the payoff function (25) with respect to   : We can choose sufficient small  to make the second derivative (26) negative, that is, the existence of Nash equilibrium.However, it is infeasible to find the conditions of  to ensure (26) always to be negative.Further, we cannot obtain the explicit expression of the Nash equilibrium from    (x)/  = 0.
Remark 8.For the LMP in wholesale electricity markets, we need to set different prices for different areas in the power system.In fact, the proposed pricing function (7) can be seen as the electricity price at a specific area in one time slot, and (∑ ∈    − ∑ ∈    ) describes the power usage in the area.For a predefined pricing parameter , we see that the price is increased with the difference between the total load and the total supply, which consists with the roles of LMP.
Remark 9.The scheduling model proposed in this work only deals with the power balance in one area.In practice, we need to consider the scheduling with operating constraints, for example, power flow between different areas, which will introduce the voltage phase angles to the pricing function.In that case, the pricing function is defined as where  and  are the indices of different areas in power system,   is the susceptance of bus −, and   and   are the voltage phase angles of area  and , respectively.In this study,   and   are neglected because we only consider the electricity scheduling in one area.For electricity scheduling in the whole power system, we also need to consider the transmission congestion on the transmission line, which requires additional pricing models.The pricing method to solve the transmission congestion is out of the scope of this study.

Distributed Algorithms and Implementations
4.1.Iterative Algorithms.In smart grids, the generators and the consumers need to search for the optimal generation and consumption in a distributed fashion.We propose two distributed iterative algorithms to search for the optimal generation and consumption.The generation scheduling algorithm for generator  is defined as where () is the price to be announced to the generators for generation scheduling: The consumption scheduling algorithm for consumer  is defined as where () is the price published to the consumers for consumption scheduling: first search for the optimal generation  *  by algorithm (28), and then the consumers search for the optimal consumption by algorithm (30).The pricing function ( 29) is equal to the pricing function (31) at the Nash equilibrium because of the balance between supply and demand.Since the price is known to the generators and the consumers, the iterative algorithms (28) and (30) in a distributed fashion.Next, we give the convergence conditions of the algorithms.Proposition 10.Suppose the generator level subgame  has a unique inner Nash equilibrium L * .The dynamic algorithm (28) converges to the unique Nash equilibrium if Proposition 11.Suppose the consumer level subgame  has a unique inner Nash equilibrium x * .The dynamic algorithm (30) converges to the unique Nash equilibrium if The proofs are omitted here.Interesting readers can find the details in [6].

Implementations.
In this section, we provide distributed implementations for the scheduling strategy in smart grids.The programming pseudocode of the scheduling strategy is given in Algorithm 1. Specifically, the generators compete to maximize the payoff (18) and search for the optimal generation by algorithm (28).Then, the consumers decide the optimal consumption by algorithm (30).Both of the prices (29) and (31) are announced by the IESO.

Numerical Results
In the simulations, the entire time cycle is divided into 24 time slots representing 24 hours in a day.The numbers of consumers and generators are set to be 100 and 5, respectively.Before giving the numerical results, we define the daily average price as To characterize the peak load shifting, we adopt the peak-toaverage ratio (PAR) defined in [7].
The balance between the total electricity consumption and the supply is shown in Figure 4 for the RTP-based scheduling strategy.In comparison, the difference between the total electricity consumption and the supply is large for the flat pricing-based scheduling strategy.From Figure 5, we see that the RTP-based scheduling strategy can implement peak load shifting by flexible pricing.Specifically, the IESO increases the price at the peak time and decreases the price at the non-peak time.We show the convergence of algorithms (28) and (30) in Figures 6 and 7, respectively.The results indicate that the generators and the consumers can both search for the optimal strategies within 30 iterations, which demonstrate the rapid convergence speed.
We compare the average price, the PAR, and the daily load in Table 1.The results demonstrate that the RTP-based scheduling strategy lowers the average price compared with the flat pricing-based scheduling strategy and also reduces       the PAR.Furthermore, the daily load under the RTP-based scheduling strategy is smaller than that under the flat pricingbased scheduling strategy, which indicates the advantage of electricity saving.

Conclusions
In this paper, we propose a RTP-based scheduling strategy based on the hierarchical game in smart grids, where the generators act as the leaders and the consumers act as the followers.The distributed scheduling algorithms are proposed to be implemented by the generators and the consumers
Total electricity consumption and supply (kW) TEC (RTP) TEC (Flat pricing) Supply 1 am 4 am 7 am 10 am 1 pm 4 pm 7 pm 10 pm

Figure 4 :1
Figure 4: Total electricity consumption (TEC) and supply in a day.

Figure 5 :
Figure 5: RTP and flat pricing in a day.
Definition 2. An energy consumption profile x * is a Nash equilibrium in the consumer level subgame if    ( *  ,  * − ) ≥    (  ,  * − ), for all ,   ∈   , where  − is an energy consumption profile of all consumers except for consumer , and    ( *  ,  * − ) denotes the payoff function of consumer  given the energy consumption of the other consumers.
(L)) ∈  }, if for all  ∈   , (1)   is a nonempty, convex, and compact subset of some Euclidean space R  ; is concave in   .According to Lemma 5, we conclude that the generator level subgame  has at least one or more Nash equilibriums.

Table 1 :
Comparison of the scheduling strategies with RTP and flat pricing.