A decentralised bi‐level control approach to wind power regulation via thermostatically controlled loads

This paper studies the wind power regulation problem by controlling a population of thermostatically controlled loads (TCLs) in smart grids. A decentralised bi-level control approach is proposed, where the first-level controller achieves load following based on wind power forecast, while the second-level controller deals with the forecast error effectively using the frequency deviation information. Specifically, to realise the fair dispatch of wind power in a users’ comfort sense, the wind power following problem is formulated as a quadratic optimisation problem, which is solved by the first-level controller. The second-level adopts a local proportional controller to handle the frequency deviation caused by the forecast errors of wind power. The proposed algorithm converges fast and is decentralised in the sense that the TCLs conduct local computation and keep the parameters’ privacy from the aggregator. Simulations are given to show the performance of the proposed approach.


Introduction
Wind power has been intensively researched by scientific communities due to its environmental and economic benefits [1]. However, due to the volatile nature of wind, direct integration of wind power will pose a threat to the stable operation of power systems, which increases the system-scale operational costs in an indirect fashion [2]. To alleviate the uncertainties and fluctuations of wind power, an effective way is to control flexible loads for ancillary services including load following and frequency regulation, such that they can cooperatively achieve wind power regulation, which is usually referred to as demand-side management [3]. Specifically, load following refers to the service that can track hourly or sub-hourly (e.g., 5 min) changes in the aggregated load and generation outputs based on forecast data, while frequency regulation deals with uncertainties and forecast errors on a second-to-second basis, such that system frequency is stabilised [4].
Due to the insight that thermal mass can be treated as generalised energy storage and considering the large-scale installment of thermostatically controlled loads (TCLs), TCLs are deemed as a power participant of demand-side management, and the modelling and control of aggregated TCLs have drawn great attention from researchers. To be best of the authors' knowledge, the idea of real-time wind power regulation via TCLs was originated by [5] and followed up by [6][7][8], etc. The above works have proved that TCL populations have great potential and flexibility in providing wind power regulation services.
Motivated by the necessity of wind power regulation service and inspired by [5], in this paper we study the wind power regulation problem via TCLs and propose a decentralised bi-level control approach. The proposed control approach is bi-level in the following senses: (1) on a sub-hourly basis the aggregated TCLs under coordination can cooperatively track the time-varying prediction of wind power output, (2) the wind power prediction errors could be eliminated via local feedback controllers on a second-to-second basis. In addition, on the premise that the aggregated TCLs cooperatively consume the given wind power supply, fair dispatch is achieved in the sense that their relative temperature deviations (RTDs) can reach consensus. For this purpose, with each TCL endowed with a discomfort cost associated with its RTD, the first level load following problem is formulated as a quadratic optimisation problem, of which the objective is to minimise the total discomfort costs of TCLs, and the optimal solution will be taken as the control signal of power consumption for each TCL. On a second-to-second basis, the forecast errors of wind power will lead to frequency deviations, for which a feedback controller is designed to adjust each TCL's power consumption in real time.
Since traditional centralised control paradigm may lack feasibility and scalability due to lack of global information or centralised computing capability when dealing with large scale integration of TCLs, this paper aims at proposing a decentralised control approach to the wind power regulation problem. Basically, decentralised approaches are deemed as an effective solution to large-scale problems, with the advantages of scalability, enhanced robustness, reduced communication and computation complexity, plug-and-play feature, etc [9][10][11]. For the first level quadratic optimisation problem, a decentralised bisection method is adopted, which is originated by the authors' earlier work [12]. Specifically, the above method is decentralised in the sense that each TCL conducts local computation and communicates with an aggregator, e.g., regional power company or a low voltage transformer, while the aggregator guides each TCL's power consumption by broadcasting 'price' signals such that the power balance constraint is guaranteed. As for the frequency regulation problem, it is assumed that the frequency deviations are the same throughout the network and can be sensed by each TCL. Then a local proportional controller is proposed, which is naturally distributed.
The remaining parts of this paper are organised as follows. Section 2 presents some preliminaries, including the single TCL model and a brief review of wind power forecast. We present the problem formulation of wind power regulation via TCLs in Section 3. The decentralised bi-level control approach is proposed in Section 4. Simulation results are given in Section 5 to illustrate the performance of the proposed method. At last, we conclude our paper in Section 6.

Modelling of a single TCL
Although this paper considers a population of TCLs, in this paper, we only present the model of a single TCL instead of an aggregated model of TCL populations. Because the decentralised control approach to be proposed in Section 4 allows us to control each TCL in an explicit and direct fashion while still achieving the aggregated behaviour of TCLs for wind power regulation. we introduce the model of a single TCL.
Let us consider the cooling process (the results of this paper also apply to the heating process, with the sign of power consumption reversed). With the total number of TCLs in the network denoted by n, the dynamics of the ith TCL can be described by the following linear discrete-time model: where T i [k] and x i [k] are the indoor temperature and power consumption of the ith TCL at time k, respectively; T a,i is the ambient temperature; a i is the thermal characteristic that is associated with the thermal capacity and thermal resistance R i ; the rate of energy exchange between the thermal mass and the ambient temperature is denoted by η i . The parameters T a,i , R i , a i , and η i are all assumed to be constant. The sampling interval for the dynamics of TCLs is Δk (e.g., 5 min). It is further assumed that the TCLs can work at any power during 0 and the rated power. This is a rational assumption, because variable-frequency TCLs have an overwhelming trend of high penetration nowadays, while the constant-frequency TCLs can emulate variable-frequency ones using the idea of pulse width modulation.

Wind power forecast
The uncertainty and intermittency in wind power outputs are a very crucial challenge in integrating wind energy as a highly reliable electricity power source [13]. Advanced wind power forecasting plays a key role in handling this issue. Since in this paper we focus on the wind power regulation using TCLs, the wind power forecast shall be very-short-term forecast, in which the time horizon is only several seconds to 30 minutes ahead. Commonly used methods for very short-term wind power forecasting are summarised below: • Persistence method [14]. In this method, we take the wind power measurement at time k to be the wind power forecast at time k + 1. The persistence method is the most widely used in wind power industry, as it is easy to implement while achieving high accuracy. • Statistical methods, including time-series based methods and artificial neural-network based methods [13]. This type of methods uses historical data to tune the wind model parameters. But due to the lack of accuracies in prediction using historical data, statistically methods are not widely used in practice. • In the recent work [15], a hybrid forecasting approach is proposed which is based on grey relational analysis and wind speed distribution features. According to the authors, this forecasting model in [15] can obtain accurate wind power forecast with the mean absolute percentage error (MAPE) and root mean square error (RMSE) as little as 2.37% and 3.79%, respectively.
Given the recent progress in wind power forecasting, in this paper, we assume that the wind power forecast is sufficiently reliable such that on a minute-to-minute basis, we can achieve load following based on wind power forecast. Nevertheless, forecast errors are dealt with on a second-to-second basis via distributed feedback controller.

Problem formulation
In this section, we temporarily assume that the wind power forecast is 100% accurate and therefore the wind power regulation is simply reduced to the pure load following problem (wind forecast errors will be taken into consideration in Section 4). Given the wind power forecast P[k], the load following problem aims at minimising the total discomfort cost of TCLs by coordinating each TCL's power consumption, on the premise that the total power consumption of TCLs equals to the wind power forecast. Note that the wind power supply is assumed to be constant as P[k] between time k and k + 1.
Since the TCLs are manipulated to provide ancillary services, their temperatures are inevitably deviated from their desired set point T s,i . It is further assumed that each TCL is able to set their own limits of temperature deviation ΔT i , and the control scheme enforced on each TCL shall not drive its temperature out of the , the RTD is given by: We further define local cost function associated with each TCL to capture their degree of discomfort from their set points: We assume that line losses are negligible and the distribution network has enough redundancy such that security constraints can be neglected as well. Then the first-level load following problem is formulated as follows: where the x i max is the rated power of the ith TCL. It can be easily verified that problem (1) is convex with quadratic cost function and linear constraints. Similar structure could be found in optimal resource allocation and economic power dispatch problems [16,17].

Decentralised bi-level control
In this section, we propose a decentralised bi-level control approach which achieves load following of wind power by solving the optimisation problem (1) and deals with the real-time forecast errors of wind power based on the second-level feedback controller. Firstly, to facilitate the implementation of the proposed decentralised approach, we first give the sufficient communication network condition, and then propose the decentralised bisection method and distributed proportional controller for the load following and frequency regulation subproblems, respectively.

Assumptions
To coordinate the TCLs in a decentralised fashion such that they can cooperatively achieve wind power regulation, an aggregator is needed. In practice, the aggregator may correspond to a lowvoltage transformer of a residential area, or a regional power company. A communication network of star topology is established, where the aggregator is the hub node while the TCLs are leaf nodes. The star topology is reasonable as it resembles the radial structure of distribution networks. Every node in the communication network is able to conduct local computation, and through the communication network, the aggregator can conduct We remark that although an aggregator is assumed in this paper, it is not equivalent to the control centre in the traditional centralised control paradigm. As we will show, the aggregator only generates signals with a small amount of computation, to influence each TCL, while each TCL still conducts local computation to make their own decisions. Furthermore, only limited data are exchanged in the network and no parametric information of TCLs is known by the aggregator. Therefore, the methodology in this paper is still decentralised, featuring scalability and privacy preservation.

First-level controller for load following
To solve the first-level subproblem in a decentralised fashion, we adopt the decentralised bisection method which was firstly proposed in our previous work [12].
In order to solve the optimisation problem (1) in the distributed fashion, one must exploit its separable feature. It could be seen that the local cost functions f i (x i [k]) of each TCL are independent of each other and the maximum/minimum power constraints are also local. Therefore, the system balance constraint is the only source of correlation in Problem (1). Considering the assumption that the aggregator is the only node who knows the wind power forecast P[k], it would be natural to let the aggregator takes care of the system balance constraint, while each TCL seeks optima in a parallel fashion. The Lagrange multiplier λ associated with the system balance constraint, from an economic perspective, represents the incremental cost of consuming wind power, and in this case, is monotonically increasing with the boundary value P [k].
Based on the above analysis, we propose a decentralised method where the aggregator finds the optimal Lagrange multiplier in a bisection fashion, while the TCLs react to the evolving λ to reach their own optima.
Let s = 0,1,2… be index of bisection steps and denote by λ(s), λ(s), and λ(s) the Lagrange multiplier and the upper and lower bounds of the interval at the sth bisection step, which are maintained by the aggregator. Initialise the interval [λ(0), λ(0)] such that the optimal Lagrange multiplier lies within it, which can be achieved by setting λ(0) = 1 and λ(0) = − 1, otherwise at least one of the TCLs' RTD is beyond ± 100% at optima.
For s = 0,1,2…, the aggregator computes and transmits λ(s) to each TCL. From an economic perspective, the signal λ(s) is the 'price' paid to each TCL for consuming the wind power. Therefore, each TCL aims at maximising their 'profit' associated with the current λ(s), i.e., The local subproblem (3) is a single-variable quadratic optimisation problem with a simple interval constraint, and its optimal solution could be given by the following closed-form equation: where P i denotes the projection on to [0, x i max ]. With the time index k omitted, denote by x i (s) the optimal solution of the ith TCL at the sth bisection step. The aggregator then collects all the optimal solution of each TCL and computes Note that by counting the data packages from the TCLs, the aggregator is able to check whether all the TCLs have sent their optimal solutions. Since the aggregator is in charge of the system balance constraint (also referred to as 'market clearing' by economists), at step s + 1 the aggregator updates the bisection interval by comparing X(s) with the wind power P according to the following rule: Since the bisection method converges asymptotically, the aggregator can stop the algorithm when the following stopping criterion is met: where ɛ > 0 is the error tolerance. Or in practice, one can predetermine the maximum bisection step, e.g. 20 steps, as the bisection process converges linearly. The pseudo code of the first-level controller is presented in Fig. 1.
Note that communication delay, packet loss, and temporalasynchrony tend to bring about a big challenge in implementing decentralised control algorithms [26]. But our algorithm does not require synchronised clocks, a global clock, or time-stamped data packages. The proposed algorithm can be viewed as eventtriggered. On one hand, the aggregator executes the bisection updates only when the aggregator receives all x i from the TCLs. On the other hand, each TCL updates x i according to (3) only when it receives λ from the aggregator. So, the iteration performed by each TCL is synchronised at every step by the event of receiving the information from the aggregator, even if the TCLs do not execute the computation (3) at the same time. Moreover, in our algorithm, communication delays only make the execution of each step take longer time, but do not affect the convergence rate.

Second-level controller for frequency regulation
The first-level controller which utilises the forecast of wind power on a sub-hourly basis, does not take into consideration the uncertainties and errors of the forecast data. Although wind power forecast has attracted lots of attention from researchers and many advanced forecast technologies have been invented, at this point, it is unrealistic to neglect the forecast errors. The forecast errors tend to cause the imbalance between power demand and supply, driving the system frequency away from its nominal value and therefore compromising system stability [18]. To cope with the forecast errors and to maintain system balance, we propose the second-level controller. Since the forecast errors are higher-frequency oscillations on a second-to-second basis, we introduce a new time index t with the sampling time denoted by Δt. In this paper, we take Δt = 1 s. Therefore, between time k and k + 1, there are N minor time steps, where N = Δτ/Δt. It is further assumed that each TCL is outfitted with a metering device that can measure the frequency deviation from the nominal value and the measurements at each TCL are identical. The information of frequency deviation works as a feedback signal to the second-level controller, which is a local proportional controller at each TCL. Defining Δω [k,t] as the frequency deviation measured at time [k,t], the power assignments regulated by the second-level controller is given by where P i denotes the projection on to [0, x i max ] and the K p is the proportional gain. A sketch of the second-level controller is shown in Fig. 2.
The choice of the proportional gain affects the performance of the second-level controller. A smaller gain leads to smaller adjustments of power assignments, which might be insufficient to counter the frequency deviation caused by forecast errors. On the other hand, a larger gain may lead to overcorrection of power assignments and undesirable overshoots, or even rendering the performance worse than that of the unregulated case. The proper choice of K p is subject to the network size N and the equivalent inertia M. A heuristic candidate of K p would be:

Overall control scheme
In this subsection, we combine the results in sections 4.2 and 4.3, and present the pseudo code of the overall control scheme in Fig. 3.
In this section, we use simulations to illustrate the performance of the proposed decentralised bi-level control approach. We consider a network of 10 TCLs with heterogeneous parameters adopted from [19]. The wind power forecast and the actual wind power are depicted in Fig. 4.
We first neglect the wind power forecast error and simple investigate the load following performance of the first-level controller. For simplicity, we set the total number of bisection steps of each time k to be 20. The gap of wind power supply P[k] and the total power consumption of TCLs ∑ i = 1 n x i [k] is kept within ± 0.08%, as shown by Fig. 5. In addition, Fig. 6 shows that the RTDs of the TCLs reach convergence in about 3.7 h, which means fair wind power dispatch is achieved. We then take into consideration the wind power forecast errors and test the performance of the bi-level controller. We assume that the actual wind power is with random errors of ± 10%. We neglect the impact of frequency changes on the other motor loads, and set M = 325 kW/(rad•s). According to the proposed heuristic gain, we take K p = 65. Fig. 7 shows that although the power assignments are readjusted by the second-level controller, the fair dispatch is still achieved and the RTDs, in this case, stay almost the same as the case without forecast errors. And Fig. 8 shows the performance of the second-level controller. From the unregulated (blue) curve, we can see that even wind power forecast with such small errors (≤ ±  10%) can lead a frequency deviation up to ± 0.1 Hz, which might compromise the stability of the power system. When the secondlevel controller is applied, the frequency deviation is well kept to zero.

Conclusion
In this paper, we study the procurement of ancillary services via aggregated TCLs for the purpose of wind power regulation. A decentralised bi-level controller is proposed, where the first-level controller achieves load following while the second-level controller maintains system balance and stability. The proposed algorithm converges fast and is decentralised in the sense that the TCLs conduct local computation and keep the parameters' privacy from the aggregator. Through simulation, we show the performance of our algorithm. Future work would be the extension of the proposed algorithm to other types of loads and energy storage devices, such as refrigerators and plug-in electric vehicles.

Acknowledgments
This work was supported by the National Natural Science Foundation of China (grant no. U1766201).