Combined time-varying forecast based on the proper scoring approach for wind power generation

: Compared with traditional point forecasts, combined forecast have been proposed as an effective method to provide more accurate forecasts than individual model. However, the literature and research focus on wind-power combined forecasts are relatively limited. Here, based on forecasting error distribution, a proper scoring approach is applied to combine plausible models to form an overall time-varying model for the next day forecasts, rather than weights-based combination. To validate the effectiveness of the proposed method, real data of 3 years were used for testing. Simulation results demonstrate that the proposed method improves the accuracy of overall forecasts, even compared with a numerical weather prediction.


Introduction
Wind power forecasting is a critical technology to increase wind power penetration in an economical manner. Therefore, there are many researches focus on how to improve the accuracy of point forecast models [1]. In traditional, forecasting techniques can be totally classified into four categories [2][3][4]: the reference forecast model, physical model, statistical model, and hybrid forecast model. More details can be found in the reference of the state of wind power forecasts [5][6][7]. However, it is considered as a low accuracy forecast method. It also lacks of uncertainty information for generation scheduling. As an alternative forecasting method, probabilistic forecasts [8][9][10] are proposed to improve forecast accuracy, which can provide more valuable uncertainty information of wind generation. Probabilistic forecasts are indicated as a range of probabilities, for example, probabilistic interval 10-20%. Furthermore, about how to measure the accuracy of these forecast models, Mitchell and Ferro proposed a scoring rule method [11] which assigns scores to each possible outcome of the event and each probabilistic forecast. The literature [12,13] also applied scoring approach to measure historical forecast performances with all information at hand.
Overall, the forecasting accuracy of wind power forecasts has not been effectively improved for decades. Normally, the normalised average absolute error (NMAE) of wind power forecast is 10-20% [3,4]. On the other hand, forecasted wind power points are regarded as essential basic data for unit commitments and electricity market. The low forecast accuracy strengthen the price waving and the uncertainty of power generation, in a large wind power injected power grid. It need purchase a huge volume of power reserve to keep the energy balance of the system in a day-ahead market.
In recent years, a novel combined method was proposed and obtained widely focus already, which combined sister forecasts together to get more accurate short-term forecasts using weighting algorithms [14][15][16][17]. Combined forecasting method [15] is regarded as an effective method to provide more accurate forecast than individual forecast models, due to its capability of integrating different types of advantage methods together.
According to Jakub [18], the sister forecasts are generated from a family of models, which have a similar model structure but are built by different parameters. Therefore, this paper proposed an improved combined forecast, which uses a modified proper scoring approach. It embraces two improved aspects: (i) do not limit the component forecasts are generated from a similar model structure. (ii) Assume that the time-varying probabilistic interval range or probabilistic distribution represent forecast capability of models, so the proper scores approach is applied to select the most accurate component models on each time intervals to consist a final one model for next 24 h forecasts.
The rest of the paper is organised as follows: the proper scores approach is introduced in Section 2; in Section 3, based on the proposed method, three widely used models are used to combining the time-varying point forecasts for next 24 h; in Section 4, a comprehensive study on improvement of forecast accuracy is carried out. Section 5 concludes this paper.

Scores approach
Wind is a physical phenomenon of a bulk air movement. Continuous wind power curve has a persistent characteristics, which can be forecasted. In traditional, forecast accuracy is defined as the average degree of correspondence errors between forecasts and measurements. So, there are many scoring rules proposed and used to calculate a forecaster's accuracy, such as mean absolute error (MAE) of (3), which was used in this paper.

Variables and criteria definition
Variable y is forecast wind power, f (·) is the probability density function (PDF), and h(·) is a forecast model. f (1) expresses uncertainty around forecast error 1, which is analysed from historical data and information in hand. Furthermore, y t denotes a point forecast issued at time t, its parameters w t , and the information set V t gathering the available information on the process up to time t Let forecast error 1 at time t + k is The domain of 1 t is [0, P cap ]. Three precision metrics [19][20][21] are used in this paper: MAE of (3), NMAE of (4), and root mean square error (RMSE) (% of the installed capacity) of (5).
MAE is defined as: NMAE is given as follows: where P measure is the measured wind power data, P forecast the forecast result, P cap the total installed capacity, and π the calculation period.

Scores
A scoring approach assigns a numerical score S[ · ] to each pair is the probabilistic distribution of wind power forecast error which belongs to forecast model h and Q v [ R is the verification value. In this paper, Q v = {v 1 , . . . , v 24 } is defined as the average error of last 2 weeks (t days) as shown below

Proper scores
A proper scoring rule is designed such that truth telling and quoting the true distribution as the forecast distribution [1]. Mathematically, a score is proper if for any two probability densities f (·) and f ′ (·), it is written as: where z is a random variable. The scoring rule S[·] is said to be strictly proper if (7) holds with equality if and only if f ′ · ( ) = f · ( ). In other words, the minimum of the left-hand side over all possible choices of f ′ (·) obtained if f ′ (·) = f (·) for all z.
Assume that the error Q 1 = {1 1,h , . . . , 1 24,h } of combined forecast is consisted by different forecast models h t,h (·) on each time interval of the next 24 h. h t,h (·) is that a model which has a lowest score on each time interval t [ [1, 24 h].
According to (6) and (7), the score of combined forecast f C (·) can be written as: f C (·) is PDF of the most accurate combined forecast error Q 1, and f ′ C (·) is the error PDF of the other combinations. The scoring rule S[·] of (8) is said to be strictly proper if (9) holds with equality if and only if f ′ C (·) = f C (·).

Combined forecasts
In the literature [22][23][24], based on continuous forecast error curves, different forecast models are combined to one accurate time-varying model, using look-ahead time. It was found that models always performance accurate at one particular forecast time intervals, but bad at other intervals. Even the simplest persistence model (PM) performances better than numerical weather prediction (NWP) model, in the first few hours [4,12,25]. Therefore, three basic forecasting models are used as component models to combination, which include PM, ARMA model, and NWP model, due to their wide application and high accuracy.

Component forecast models
3.1.1 Persistence model: PM is a simple but widely used timeseries model. It can surpass many other models in very short-term prediction. In this paper, the PM is not only used as a component model, but also as a benchmark with which the proposed forecast method is compared. The persistence forecast [4] can be written as: where y(t + k|t) is the wind power forecast for time t + k made at time origin t, k the prediction horizon, T the prediction interval length (here T = k), x(t − j · Dt) the measured wind power for time t and the previous i time steps within T, and Dt the time step length of the measured time series ( T = l · Dt). The delay k describes the time gap, when the forecast is done and the beginning of T. Therefore, the PM model is utilised in the study as a component forecast model h 1 and a benchmark reference also.
3.1.2 ARMA model: As a powerful, well-known time-series technique, ARMA model has been widely used to forecasting or hybrid with other models to forecasting for >50 years. More details about the forecast performance description and application can be found in the state-of-the-art [4]. Therefore, the ARMA model is utilised in the study as a component forecasting model h 2 . The ARMA (p, q) model [12] is written as: where y t denotes the hourly forecast wind power at hour t, a is the parameter, p the order of the autoregressive part of the model, q the order of the moving average part of the model, w i the ith autoregressive parameter, u j the jth moving average parameter, and 1 t the error term at time t. Before ARMA models are used to forecast, a Box-Jenkins methodology is used to establish the parameters of models which best fit the wind power data. In the phase of parameters estimation, tools of the sample autocorrelation function (ACF) and the sample partial ACF are used to identify the parameters ( p and q) of the ARMA model. The detailed process can be found in [2].

NWP model:
The NWP uses mathematical models of the atmosphere to moulding weather condition using radiosondes, weather satellites, and other observing systems. When use NWP model to forecast wind power, it includes wind speed forecasted from the local meteorological service and transformed to wind power by wind turbine's power curve. In practical application, the NWP model is the most accurate forecast method. However, it is based on complex calculation, which requires super computers to get solutions. An NWP model WPFS Ver1.0, which belongs a combination of physical and statistical approach, is used in this paper. WPFS Ver1.0 is the first wind power forecasting model developed by Chinese electric power science institute and has been used in Jiangsu Provincial power grid. More details of this mature commercial forecast model can be found in [26]. The NWP model is utilised in the study as a component forecast model h 3 .

Standard modelling procedure
According to the above derivation, the procedures of constructing the combined forecasting model and how to use it to forecast for the next 24 h is summarised as follows: Step 1: Calculating historical forecast error distribution f t,h (·) of each hour t [ [1,24]. f t,h (·) belongs to different forecast models h h (·), h [ [1,3], which corresponds to PM, ARMA, and NWP models. It is a time-varying probabilistic distribution.
Step 2: Using forecast errors of recent days to consist verification value Q v = {v 1 , . . . , v 24 } for each forecast. In this study, Q v is defined as the average error of last 2 weeks as shown in (5).
Step 3: Using the time-varying historical error distribution f t,h (·) and the rolling verification value Q v to build the score function S[ · ] which is described in (8) and (9).

Model analysis
A single wind farm and two probabilistic forecast models, which have a similar accuracy, are chosen to have a further mechanism analysis of the proposed method. Nineteen months data (July 2013-January 2015) collected from a single Long Yuan wind farm is used to analysis, which has an installed capacity of 400.5 MW and located in the Yan Cheng of Jiang Su province of China. Two kind of ARMA forecast methods are used as component models, including ARMA-based direct multi-step-ahead forecast (ARMA-DMS) and ARMA-based indirect multi-step-ahead forecast (ARMA-IMS) which can be found in [12]. The forecast error distributions are presented in Fig. 1. As shown in Fig. 1a, the ARMA-DMS has an MAE of 72 MW, and Fig. 1b presents the ARMA-DMS which has an MAE of 67 MW. The two forecast method has a similar accuracy due to using the same ARMA as a basis model. It will be convenient for a description of the proposed model. Fig. 2 illustrates the generation process of the proposed combined method. According to the function of (12), the 24 h forecast length is divided into four intervals: (i) interval 1 is from 0:00 to 9:00, in which ARMA-IMS method has a best prediction accuracy; (ii) interval 2 is from 9:00 to 13:00, in which ARMA-DMS method has a good prediction; (iii) interval 3 is from 13:00 to 15:00, in which ARMA-IMS method does better; (iv) interval 4 is from 15:00 to 24:00, in which ARMA-DMS method has a good prediction, as shown in Figs. 1 and 2. Therefore, the final combined forecast model is written as   Based on the analysis of the behaviours (probabilistic distribution) of the two methods shown in Fig. 1, ARMA-IMS is used to forecast on interval 1 and interval 3 , due to more a accurate forecast on the two time intervals, as shown in Fig. 2. ARMA-DMS is used to forecast on interval 2 and interval 4 covered by slash lines. Finally, the combination of these forecasts is used as the overall forecast (grey arrow) for the next day, as shown in Fig. 2.

Case study
A 3-year case is used to test, which is carried out in Nantong which is located in the east coast of China. Nantong is an interesting region, given that it already has a substantial amount of installed capacity of wind power at 1.33 GW by 2015, the wind capacity penetration (WCP) is 24.19% when considering the annual maximum load of 5.58 GW at 6 August 2015 WCP = Installed wind power capacity Peak load (16) The data is collected from seven operating wind farms. Table 1 summarises the details of these. The total installed power capacity of these seven wind farms is 1.33 GW and the geographical conditions of the studied wind farms (marked in blue) are shown in Fig. 3. Their power outputs and forecasts from June 2012 to January 2015 with 1 h resolution are chosen for the analysis. Data are continuously acquired over this period with the only unavailability occurred for few days due to continuous faults of data acquisition system. The availability of wind power output data is 83.2%. The benefit gained by using the proposed model is measured as the accuracy improvement, when compared with the reference model. It is written as: where error r is the evaluation criterion (i.e. MAE or RMSE) of the reference model and error p is of the proposed model. Table 2 summarises the forecast error by the proposed model, the NWP model, the ARMA model, and the PM. It shows that the proposed forecast method has a better performance. The NMAE of the proposed method is 9.30%, and the NWP model is 9.75%. The accuracy improvement of the proposed method is 4.58% when compared with the NWP model, 37.42% compared with the ARMA model, and 51.49% compared with the PM for 24 h in advance. It also shows that the average RMSE of the proposed method is 43.79 MW, and the NWP model is 45.11 MW. Then it has an improvement of 2.93%, when compared with the NWP.

Conclusion
Combining individual forecast models to build a more accurate model is an effective method to improve point forecast accuracy and it is easily to be understand and calculate. Therefore, there is a lot of research focus on the combination method in recent decade. This paper proposes a time-varying combined forecasts method, which provides a more accurate forecast including two aspects: (i) the proper scoring approach is applied to measure the model's performance, which is presented by probabilistic distribution, rather than using single or simple assessment metrics. (ii) A time-varying combining frame is proposed to build the overall forecast rather than weights-based combination in traditional. To validate the advantageous performance of the proposed method, a long term of 3-year period study is carried out. Results show that the proposed method is accurate and effective even compared with the commercial NWP model.