Bilevel Aggregator-Prosumers' Optimization Problem in Real-Time: A Convex Optimization Approach

This paper proposes a Real-Time Market (RTM) platform for an aggregator and its corresponding prosumers to participate in the electricity wholesale market. The proposed energy market platform is modeled as a bilevel optimization problem where the aggregator and the prosumers are considered as self-interest agents. The current state-of-the-art Mathematical Programming with Equilibrium Constraints (MPEC) and Mixed-Integer Programming (MIP) based approaches to solve bilevel optimization problems are not satisfactory for real-time applications. The computation time for such approaches grows exponentially as the number of prosumers and decision variables increase. This paper presents a convex optimization problem which can capture a subset of the set of global optima of the bilevel problem as its optimal solution.

Both the MPEC and MIP based methods are computationally expensive. One of the main challenges to implement an RTM is the computational efficiency. For an RTM the time intervals are in the order of a few minutes [Vlachos and Biskas, 2013]. Therefore, new computational tools are needed for the aggregator's real-time control over the prosumers and its participation in the RTM. Ghamkhari et al. 2016 has addressed the computational efficiency of the dominant firm's strategic offering by introducing a convex relaxation for the bilevel optimization problem and has found a close to optimal solution. However, to the best of our knowledge, no study has been done on finding the global optimum of a bilevel optimization problem by solving a convex one in the field of prosumers integration in the wholesale energy markets.
In this paper, we define the problem of economic optimization of an aggregator and its corresponding prosumers for participation in an RTM over a time horizon as a bilevel optimization problem. The aggregator represents the prosumers to participate in the wholesale market in a real-time scenario. This problem, in general, is nonconvex [Luo et al., 1996]. We show that a subset of the set of global minimizers for the nonconvex problem can be obtained as the solution of a certain convex optimization problem. The convex problem has two main advantages. On the one hand, a convex formulation is attractive in real-time applications since the computation time is linear in the number of variables. On the other hand, off-the-shelf software packages can be used to solve the problem. In addition, replacing a bilevel optimization problem by a convex one is a key step toward decentralized or distributed algorithms [Bertsekas, 1999]. This work is a continuation of the preliminary study by the authors [Shomalzadeh et al., 2020] which dealt with a simple static model for balancing markets.
The paper is organized as follows. In Section 2, we define the aggregator and prosumers problems as a bilevel optimization problem. The results toward introducing a convex optimization problem for the bilevel one comes in Section 3. Finally, the paper closes with the conclusions in Section 4.

Notation
We denote the set of real numbers by R, n-vectors by R n and m × n matrices by R m×n . Throughout the paper, the inequalities for vectors are meant entrywise. The n-vectors of ones is denoted by 1 n . For vectors x i ∈ R ni with i = 1, 2, . . . , k, we write col(x 1 , x 2 , . . . , x k ) to denote the vector x T 1 x T 2 · · · x T k T . The m × m identity matrix is denoted by I m . For a matrix M ∈ R m×n and index sets α ⊆ {1, 2, . . . , m}, β ⊆ {1, 2, . . . , n}, the notation M αβ denotes the matrix (M ij ) i∈α,j∈β . If α = {1, 2, . . . , m}, then we write M •β and if β = {1, 2, . . . , n}, then we write M α• . A symmetric matrix M = M T ∈ R m×m is said to be positive semidefinite if x T M x ≥ 0 for all x ∈ R m and positive definite if x T M x > 0 for all 0 = x ∈ R m . The symmetric square root of a positive definite matrix M is denoted by M 1 2 . For a vector v ∈ R n , we write diag(v) for the diagonal matrix with diagonal entries v 1 , v 2 , . . . , v n .
Let f : R n → R and S ⊆ R n . Consider the optimization problem OP : min We say thatx is feasible for OP ifx ∈ S. Also, we define the set of global optima for OP as 2 Problem Statement In this section, we define a market model and platform for an aggregator and the prosumers under its contract to participate in an RTM with the grid, i.e., the wholesale market. Here, the role of the aggregator is to act as an intermediary agent between the prosumers and the grid to facilitate the energy transactions. We consider the case where each prosumer can generate energy through some RESs with zero cost. Example of such energy sources are solar panels and wind turbines. Moreover, each prosumer's demand is elastic at each time-step. The aggregator goal is to propose the prosumers with a personalized price to deal their surplus or shortage energy with the grid in an optimal way. The advantages of a personalized price over a unique price have been addressed in many recent research (see e.g., Tushar et al. 2014, Yang et al. 2018). Next, we explain the problem setting and market structure in detail.

Prosumer's Problem
The main source of energy supply for a prosumer is its renewable energy units. Due to uncertain and uncontrollable nature of RESs, there might be a mismatch between supply and demand at each time-step. Each prosumer has two options to cancel this mismatch. One is to trade with the wholesale market through the aggregator. The other option is to use its demand elasticity. Therefore, the prosumer needs to find a trade-off between these two possible options for its optimal strategy. Before providing a mathematical formulation for the prosumer, we elaborate on demand elasticity.
We say that the demand of each prosumer is elastic if: 1. Each prosumer has a preference for its demand at each time-step.
2. Altering the demand from its preferred value causes dissatisfaction for the prosumer. Here, we model this dissatisfaction using a quadratic function.
3. Each prosumer has a lower bound and an upper bound for its demand at each time-step.
4. Total demand of each prosumer in a specific time period is constant.
As explained before, the prosumer goal is to find a trade-off between two possible options to minimize its cost and maximize its comfort. We model this problem as an optimization problem. We define the set of prosumers by {1, 2, . . . , n} and the set of time-steps by {1, 2, . . . , K}. Then, prosumer i ∈ {1, 2, . . . , n} at time-step k ∈ {1, 2, . . . , K} has three decision variables: its demand h i (k), the energy it sells to (buy form) the grid y + i (k) (y − i (k)). For the ith prosumer, we consider the following optimization problem: ) is the proposed price by the aggregator to buy energy from (sell energy to) the prosumer at time-step k, s i (k) ≥ 0 is the generated energy by the prosumer at time-step k, which assumed to be known, and q i (k) > 0 is the dissatisfaction parameter for the prosumer. Moreover, h 0 i (k) ≥ 0,h i (k) ≥ 0 andh i (k) ≥ 0 are the preferred value, lower bound and upper bound for the demand h i (k), respectively. The parameter h tot i is the total demand for the prosumer over the period k = 1 to k = K.
In (3a), the first term models the dissatisfaction the prosumer experiences by changing its demand from the preferred value. The second term is the cost of buying energy from the grid through the aggregator and the third term is the revenue the prosumer can obtain by selling energy through the aggregator. The constraint (3b) indicates that the total demand should be equal to the total supply for each prosumer at each time-step. The constraints (3c) and (3d) specify the lower bound and upper bound for the decision variables. Finally, (3e) captures the assumption that the total demand over a period is constant.
(5) Thus, the variable h i (k) can be eliminated from the problem PP i and we can rewrite it as the following optimization problem: . Moreover, since the optimization problems PP i s are independent, we can add them and rewrite them in a vector form. To do so, we define the following vectors: q = col(q 1 (1), q 1 (2), . . . , q n (K)), c = col(c 1 (1), c 1 (2), . . . , c n (K)), s = col(s 1 (1), s 1 (2), . . . , s n (K)), Then, the vector form can be written as

PP : min
where we have the following parameters: Note that ⊗ denotes the Kronecker product.
The prices x + i (k) and x − i (k) are proposed by the aggregator. In this work, the aggregator acts as a self-interest agent which has the ability to anticipate the reaction of the prosumers. Therefore, knowing the reaction of the prosumers, the aggregator sets the prices to maximize its revenue as an intermediary player between the grid and the prosumers. In the next subsection, we elaborate on the aggregator's problem as a bilevel optimization problem.

Aggregator's Problem
The aggregator receives two prices from the grid for each time-step. The price p + (k) is the price for selling energy to grid and the price p − (k) is the price for buying energy from the grid at kth time-step. Having these prices and the ability of the aggregator to anticipate the reaction of the prosumers allow the aggregator to propose prices x + i (k) and x − i (k) to the prosumers in an optimal way. The bilevel optimization below models this problem for the aggregator.
AP : max ). The first term in (11a) corresponds to aggregator's revenue from selling energy to the grid. The second term models the aggregator's cost for buying energy from the grid.
In this paper, we consider a scenario where p + = −p − = p and the aggregator proposes prices x + and x − such that Therefore, we can rewrite the optimization problems AP and PP based on the new decision variables x and y = y + − y − as the minimization problems BLP and LLP, respectively.

BLP : min
x,y Here the decision vector x ∈ R m is the proposed prices of the aggregator and the parameter vector p ∈ R m is the prices of selling to and buying from the grid. The prosumers' reactions y to the proposed prices are the solution of the optimization problem LLP.

LLP : min
The vector y ∈ R m is the decision variable for LLP. The vectors and matrices c, , u ∈ R m , d ∈ R n , Q ∈ R m×m , E ∈ R n×m are parameters for LLP as defined in (9) and (10). Moreover, x ∈ R m is the decision variable for the aggregator and the prosumers has no control over it. It should be noted that m = nK and rank E = n ≤ m. We assume that there existsȳ which satisfies (13b)-(13c). Since Q = diag(q) is positive definite, LLP is a strictly convex quadratic optimization problem and hence has always a unique optimal solution, i.e., the set MIN(LLP) is a singleton.
Bilevel optimization problems are in general nonconvex and have combinatorial nature. Many algorithms and approaches have been developed to solve different classes of bilevel problems. Recent surveys on bilevel optimization can be found in [Dempe and Zemkoho, 2020] and [Luo et al., 1996]. In contrast to existing methods that deal with rather more general bilevel optimization problems, our focus here is to exploit the particular structure of (12) in order to introduce a convex optimization problem which has the same global optimum as the bilevel one. The next section investigates the conditions under which the global optimal solution of the optimization (12) can be found by solving a convex problem.

Main Results
In this section, we will show that the set of global optima for a specific convex optimization problem is a subset of the set of global optima for the optimization BLP, under some assumptions on the parameters of the problem. Before dealing with the the optimization problem BLP, we consider two variations of this optimization problem. First, we only consider a lower bound on y in (13b). Later, we will consider an upper bound on y. Then, we come back to BLP (12) to introduce a convex optimization problem which can be used to find a subset of MIN(BLP). Finally, we comment on the restrictions of the proposed convex optimization for the RTM platform.

Lower Bound on y
Consider the following bilevel optimization problem for which the decision variable y has only a lower bound: BLP1 : min and LLP1 is as Here R ∈ R m×m is positive definite and not necessarily diagonal and F ∈ R n×m has full row rank. Assume that there existsȳ satisfying (16b)-(16c) Since LLP1 is a convex optimization problem, we can write the following necessary and sufficient KKT conditions to characterize (14c): where µ ∈ R m and λ ∈ R n are dual variables for the constraints (16b) and (16c), respectively. The dual variable λ can be eliminated from KKT conditions (17)-(19). First, we solve y from (17) as and then substitute y in (18): Since F has full row rank and R is positive definite, F R −1 F T is nonsingular. Therefore, we obtain  Proof. Clearly M is the Schur complement of [Zhang, 2006, Theorem 1.12] that M is also positive semidefinite.
(29) Since neither φ nor S 1 is convex, the optimization problem SLP1 is a nonconvex one. Nevertheless, a subset of the global minimizers of SLP1 can be captured by a convex optimization problem.

Upper Bound on y
Next, we consider the following bilevel problem for which the decision variable y has only an upper bound: where LLP2 is given by LLP2 : min Assume that there existsȳ which satisfies (40b)-(40c). In a procedure similar to what we had for BLP1, we can show that BLP2 can be written as the following optimization problem using the KKT conditions:

SLP2
: min and M and r are as in (25) and (26). Note that ν is the dual variable for the constraint (40b).
Again here we want to find sufficient conditions such that at least a global optimum of SLP2 can be found using a convex optimization problem. The main assumption here is on the structure of M . The following definitions elaborate on this specific structure. Definition 1. A matrix N ∈ R k×k is called • a Z-matrix if its off-diagonal entries are nonpositive.
• an M-matrix if it is an Z-matrix and the real part of its eigenvalues are nonnegative. Remark 1. In particular, a positive semidefinite matrix is an M-matrix if its off-diagonal entries are nonpositive. Now, we can state the main result concerning the the optimization problem SLP2. Theorem 2. Suppose that M is an M-matrix, u ≥ 0 and u > r. Consider the following optimization problem:
The following results are needed to prove Theorem 2. Lemma 3. Consider the following optimization problem:
The set S 2 is a nonconvex set due to complementarity terms. In what follows, we will show that under some conditions on M, u and r, the set S 2 is equal to the polyhedral set C 2 in CVX2.
ii) There exists z ≥ 0 such that A T z ≥ 0 and b T z < 0.
Proof. Let α = {1, 2, . . . , m}. The alternative system for (59) is as which clearly have no solutions since u − r is positive. Therefore, it follows from Lemma 5 that (59) has at least one solution.
Now let α = {1, 2, . . . , m} which meansᾱ is a nonempty set. The alternative system for (59) in this case is given by We argue that this system has at least one solution. To see this, take z α = 0. This leads to Since u > r and hence uᾱ > rᾱ, (62) has a solution if and only if zᾱ ≤ 0 and zᾱ = 0. Consequently, (62) has a solution if and only if the following has a solution: Since M is positive semidefinite due to Lemma 1, it follows form Lemma 4 that there exists a nonzero zᾱ such that −Mᾱᾱzᾱ ≥ 0 and zᾱ ≤ 0. As M is an M-matrix, M αᾱ is nonpositive. Therefore, we have that M αᾱ zᾱ ≥ 0 which concludes that (61) has at least one solution and hence (59) is infeasible for α = {1, 2, . . . , m}.
After these preparations, we are in a position to prove Theorem 2.

On M being an M-matrix:
One of the assumptions in Theorem 2 is on the structure of the matrix M . We know that M is the Schur complement of matrix X as mentioned in the proof of Lemma 1 and (27). Here, we discuss when M as in (25) is an M-matrix. The following lemma shows that the Schur complement of an M-matrix is also an M-matrix. Lemma 7 (Fiedler 2008, Theorem 5.13). Suppose that N is an M-matrix. Then, Schur complement of N with respect to a positive definite submatrix of N is also an M-matrix.
The following theorem provides sufficient conditions for M to be an M-matrix. Theorem 3. Suppose that F is a nonpositive matrix and rows of F are orthogonal, i.e., F i• (F j• ) T = 0 for all i = j. Then, M is an M-matrix if R is diagonal.
Proof. The matrix M is the Schur complement of matrix X given by (27). Since F is a matrix with orthogonal rows and R is positive definite and diagonal, F R −1 F T and hence (F R −1 F T ) −1 are also positive definite diagonal matrices. Moreover, F R −1 is nonpositive which makes X an M-matrix. Consequently, M is also an M-matrix based on Lemma 7.

General Case
Here, we consider the main problem, i.e. the optimization problem (12). We rewrite BLP by characterizing lower level problem based on KKT conditions as SLP : min for some µ ≥ 0 and ν ≥ 0}. (68) Note that µ and ν are dual variables for ≤ y and y ≤ u in (13b), respectively. Moreover, the dual variable for the constraint (13c) has been eliminated from KKT conditions in a similar way to (17)-(24). Also, M and r are as in (25) and (26), respectively with R = Q and F = E. As a result, M is an M-matrix based on Theorem 3.
The theorem below indicates that there exists a convex optimization problem which can capture a subset of the set of global optima for SLP.

Interpretation of Theorem 4 for RTM
Theorem 4 has certain hypotheses on the parameters , u, and r. Here, we discuss the implications of these hypotheses for the proposed RTM platform. It follows from Theorem 4 that the vectors = s −h and u = s −h should be nonpositive and nonnegative, respectively. The vector s is the generated energy by RESs of the prosumers, the vectorh is the lower bound of the prosumers' demand and the vectorh is the upper bound for their demand. To have ≤ 0 and u ≥ 0, the aggregator should ask the prosumers to set the upper bound of their demandsh greater than or equal to their RESs' capacity and also the lower bound for their demandsh equal to zero. Moreover, Theorem 4 states that u should be strictly greater than r where r is defined as in (26) with R = Q and F = E. Considering Assumption 1, we can show that u > r ⇐⇒ h 0 >h. (78) Therefore, to have u > r, the aggregator should ask the prosumers to set their preferred values h 0 greater than the lower bound for their demandh, i.e., h 0 > 0.

Conclusions
The problem of participation of the prosumers in the wholesale market through the aggregator has been widely studied in the literature. To represent the intrinsic hierarchy of this problem, we developed a market platform based on a bilevel optimization problem. Bilevel optimization are generally highly nonconvex and current approaches to deal with these problems are computationally expensive. To implement this market platform in real-time, we proposed a specific convex optimization problem and showed that each global minimizer of this convex problem are also a global minimizer for the original bilevel problem under some assumptions on the parameters.
While the proposed convex approach can reduce the computational time significantly in contrast to the state-of-the-art methods (e.g., MIP), the assumption that the aggregator has a centralized control over the prosumers may limit the applicability of the proposed method to large scale networks. An interesting important area of future research could be design of a decentralized or distributed control mechanism using the convex problem to tackle this issue.