Probabilistic energy forecasting using the nearest neighbors quantile filter and quantile regression

https://doi.org/10.1016/j.ijforecast.2019.06.003Get rights and content

Abstract

Parametric quantile regression is a useful tool for obtaining probabilistic energy forecasts. Nonetheless, traditional quantile regressions may be complicated to obtain using complex data mining techniques (e.g., artificial neural networks), since they are trained using a non-differentiable cost function. This article presents a method that uses a new nearest neighbors quantile filter to obtain quantile regressions independently of the data mining technique utilized and without the non-differentiable cost function. This method is subsequently validated using the dataset from the 2014 Global Energy Forecasting Competition. The results show that the method presented here is able to solve the competition’s task with a similar accuracy to the competition’s winner and in a similar timeframe, but requiring a much less powerful computer. This property may be relevant in an online forecasting service for which the fast computation of probabilistic forecasts using less powerful machines is required.

Introduction

The integration of volatile renewable power systems (e.g., wind and photovoltaic (PV) power plants) into the electrical grid has complicated the necessary balancing of electricity demand and supply  (Gottwalt et al., 2017, Ludwig et al., 2017). As a result, it has now become necessary to use forecasts in order to plan and schedule the electrical grid correctly (Dannecker, 2015).

Time series forecasting models are tools that are used for estimating the future development of values whose change over time is of interest (e.g., renewable generation and load; Hyndman & Athanasopoulos, 2014). Most of the forecasting models that are described in the literature can be classified as point forecasting models; in other words, they deliver a single value at a given forecast horizon (Gneiting, 2011) but are unable to quantify their own forecast uncertainty. In contrast, probabilistic forecasting models are able to quantify such uncertainty by delivering, for example, intervals with a given probability of a future value lying within them, or probability distribution functions of a forecast time series value (Gneiting & Katzfuss, 2014). Thus, probabilistic forecasts have become an important decision-making tool (Gneiting & Katzfuss, 2014), since quantifying the forecast uncertainty may lead to better decisions. The current development of the “Smart Grid” (Fang, Misra, Xue, & Yang, 2012) has found probabilistic forecasts to be both useful and necessary, as they are able to describe the inherent uncertainty of the future renewable power generation and load (i.e., the energy demand); see Hong et al., 2016.

Probabilistic forecasting models can be divided largely into parametric and non-parametric models (González Ordiano, Waczowicz, Hagenmeyer, & Mikut, 2018). While the former assume that the uncertainty will follow a given probability distribution (e.g. Gaussian), the latter do not. In the context of energy forecasting, quantile regression is one of the most commonly used approaches for obtaining non-parametric probabilistic forecasts, especially in wind power forecasting. Quantile regression is able to estimate a quantile of a future time series value given the regression’s input (Fahrmeir, Kneib, Lang, & Marx, 2013), and hence, its use allows us to obtain probabilistic forecasts, such as interval forecasts. Some examples of quantile regression applied to wind power forecasting are provided by Bremnes (2004), Haque, Nehrir, and Mandal (2014), and Nielsen, Madsen, and Nielsen (2006). The first of these studies utilizes local linear quantile regressions, while the others obtain quantile regressions based on linear combinations of basis functions. In the case of the load, Gaillard, Goude, and Nedellec (2016) create probabilistic forecasts using linear quantile regressions with non-linear functions of the used features as covariates, while Liu, Nowotarski, Hong, and Weron (2017) train linear quantile regressions using the forecasts of several point forecasting models as independent variables. Similarly, quantile regression has also been found to be useful in the case of PV power. For example, Nagy, Barta, Kazi, Borbély, and Simon (2016) use a combination of a quantile regression forest and a stacked random forest to estimate the forecast uncertainty. It is important to mention that quantile regression is not the only approach for describing the forecast uncertainty. Other techniques are shown in the works of Juban, Siebert, and Kariniotakis (2007), Xie and Hong (2016), and Zhang and Wang (2015), for example. In the first work, PV power is forecast using a traditional k-nearest neighbors regression and a kernel density estimator; in the second, wind power probabilistic forecasts are obtained again with a kernel density estimator; and in the third, a scenario-based probabilistic forecast together with a postprocessing step is used to obtain probabilistic load forecasts. A more in-depth description of these and other methods is outside the scope of the present contribution, and therefore interested readers are referred to the survey articles presented by Antonanzas et al. (2016), Hong and Fan (2016), and Zhang, Wang, and Wang (2014) for more information regarding PV power, load, and wind power probabilistic forecasting respectively.

Traditional parametric quantile regression models are trained by minimizing the non-differentiable sum of pinball losses, a procedure that complicates the problem of obtaining them using more complex data mining techniques (e.g. ANNs and support vector regressions (SVRs)). One of the reasons for this is that the lack of differentiability may lead, firstly, to problems when using training algorithms based on gradient-based optimization (Cannon, 2011), and, secondly, to higher computation times. Furthermore, this minimization of the sum of pinball losses makes it impossible to utilize “out of the box” regression training algorithms (i.e., already implemented algorithms found in typical statistic/machine learning libraries), since they normally minimize other cost functions. Thus, training a traditional parametric quantile regression requires the additional effort of either modifying an “out of the box” training algorithm (as shown by Cannon, 2011, Shim et al., 2016, Taylor, 2000) or programming a new training algorithm from scratch.

The present contribution offers a generic approach that simplifies the training of linear and non-linear parametric quantile regression models. The approach presented here expands a preliminary concept (González Ordiano et al., 2016) that has been applied for renewable energy scheduling (Appino, González Ordiano, Mikut, Faulwasser, & Hagenmeyer, 2018). The new method is based on a newly-developed nearest neighbors quantile filter (NNQF) that modifies the used training set. This eliminates the need to minimize the sum of pinball losses, hence allowing a parametric quantile regression to be trained using any regression data mining technique and their “out of the box” training algorithms. We then validate the developed method using the data from the solar track of the Global Energy Forecasting Competition of 2014 (GEFCom14) as a benchmark (Hong et al., 2016).

In addition, the interest in obtaining parametric quantile regressions without needing to minimize the sum of pinball losses is motivated further by the desire to keep the computational effort for both their training and their application as low as possible. This is to allow these quantile regressions to be deployed as part of an online forecasting service, which may require the rapid computation of probabilistic forecasts, such as the service planned for the Energy Lab 2.0 (Düpmeier et al., 2015, Hagenmeyer et al., 2016).

The present contribution is organized as follows: Section 2 briefly presents background information on both traditional parametric quantile regression and time series forecasting. Section 3 gives a description of the new method that we present, after which Section 4 describes the validation procedure conducted. Section 5 shows the results obtained, and, finally, Section 6 offers the conclusion and outlook of this work.

Section snippets

Traditional parametric quantile regression

Regression can be defined as a supervised learning approach that uses data mining techniques to obtain a data-driven model that is able to estimate an output value y given an input vector x (Fayyad, Piatetsky-Shapiro, & Smyth, 1996). Parametric regression models, which are the ones that are relevant in the present contribution, can be defined as yˆ=f(x;θˆ),where f() represents the regression model, yˆ the estimate of the desired output, and θˆ the estimated regression parameters. The values

Quantile regression based on the nearest neighbors quantile filter

As Fig. 1 shows, the training of a single quantile regression consists of modifying the available training set and using it to train a regression model with a given data mining technique and its unmodified training algorithm. The specifics of each step in Fig. 1 are described thoroughly in the following paragraphs. In addition, the differences between the present method and the k-nearest neighbors quantile regression approach are explained.

Benchmark data

The present contribution uses as a benchmark the data provided for the solar track of the Global Energy Forecasting Competition of 2014 (GEFCom14; see Hong et al., 2016). This dataset consists of photovoltaic (PV) power time series {P(k);k=1,,K}, with values normalized to be between zero and one, of three different PV power plants, as well as several corresponding weather forecast time series. The weather forecast time series that are relevant in the present contribution are: the surface solar

Results and discussion

Table 3 contains the average pinball loss values (averaged over all relevant tasks) of all data mining techniques tested and all benchmarks used, including the winner of GEFCom14.

The low pinball loss values obtained, ranging from 1.51% to 1.94%, confirm that the method presented here allows quantile regressions to be trained independently of the data mining technique utilized. Furthermore, the differences between the pinball loss values of the GEFCom14 winner (i.e., 1.21%) and the NNQF-based

Conclusion and outlook

The present work describes the new nearest neighbors quantile filter (NNQF), i.e., a procedure for obtaining a modified training set that allows parametric quantile regressions to be trained without minimizing the non-differentiable sum of pinball losses.

The use of the NNQF not only eliminates the risks of training quantile regressions with gradient-based optimization, but also opens up the possibility of training quantile regressions using “out-of-the-box” training algorithms (i.e., training

Acknowledgments

The present contribution is supported by the Helmholtz Association under the Joint Initiative “Energy System 2050 — A Contribution of the Research Field Energy”. With thanks to Katherine Quinlan-Flatter for proofreading the article.

Jorge Ángel González Ordiano received his Ph.D. degree in mechanical engineering from the Karlsruhe Institute of Technology (KIT), Germany, in 2019. He currently works as a Postdoctoral Fellow at the Colorado State University within the Lab of Prof. Steven Simske. He is an expert in the fields of data mining, machine learning, time series analysis, and time series forecasting.

References (46)

  • NagyG.I. et al.

    GEFCom2014: Probabilistic solar and wind power forecasting using a generalized additive tree ensemble approach

    International Journal of Forecasting

    (2016)
  • StulpF. et al.

    Many regression algorithms, one unified model: A review

    Neural Networks

    (2015)
  • XieJ. et al.

    GEFCom2014 probabilistic electric load forecasting: An integrated solution with forecast combination and residual simulation

    International Journal of Forecasting

    (2016)
  • ZhangY. et al.

    GEFCom2014 probabilistic solar power forecasting based on k-nearest neighbor and kernel density estimator

  • ZhangY. et al.

    Review on probabilistic forecasting of wind power generation

    Renewable and Sustainable Energy Reviews

    (2014)
  • BremnesJ.B.

    Probabilistic wind power forecasts using local quantile regression

    Wind Energy

    (2004)
  • BrockwellP.J. et al.
  • DanneckerL.

    Energy time series forecasting: efficient and accurate forecasting of evolving time series from the energy domain

    (2015)
  • DüpmeierC. et al.

    A concept for the control, monitoring and visualization center in Energy Lab 2.0

  • FahrmeirL. et al.

    Regression: models, methods and applications

    (2013)
  • FangX. et al.

    Smart grid — the new and improved power grid: A survey

    IEEE Communications Surveys Tutorials

    (2012)
  • FayyadU. et al.

    From data mining to knowledge discovery in databases

    AI Magazine

    (1996)
  • GneitingT. et al.

    Probabilistic forecasts, calibration sharpness

    Journal of the Royal Statistical Society Series B (Statistical Methodology)

    (2007)
  • Cited by (20)

    • Adaptive quantile control for stochastic system

      2022, ISA Transactions
      Citation Excerpt :

      Moreover, robust control methods consider the worst-case disturbance in systems, which induce quite conservative optimal control strategies. Since the random noise with sharp and thick-tailed characteristics can be described by Asymmetric Laplace Distribution (ALD) [33–37], the single quantile regression is introduced to estimate model parameters for a system with ALD noise [38–40]. Nevertheless, it is difficult to choose the quantile in single quantile regression when the distribution of system noise is unknown.

    • A deep generative model for probabilistic energy forecasting in power systems: normalizing flows

      2022, Applied Energy
      Citation Excerpt :

      They both make probabilistic forecasts in the form of Monte Carlo samples that can be used to compute consistent quantile estimates for all sub-ranges in the prediction horizon. Thus, they cannot suffer from the issue raised by Ordiano et al. [25] on the non-differentiable quantile loss function. Note that generative models such as GANs and VAEs allow generating scenarios of the variable of interest directly.

    • Probabilistic forecasts of the distribution grid state using data-driven forecasts and probabilistic power flow

      2021, Applied Energy
      Citation Excerpt :

      The new method we present combines non-parametric probabilistic forecasts of the active power injection of all nodes with probabilistic power flow (ppf) [12]. We compute non-parametric probabilistic forecasts of the active power at each node using quantile regressions [13] obtained via the method described in [14]. Afterwards, we propagate the power uncertainty to the remaining variables by solving the ppf problem.

    • Very short-term probabilistic wind power prediction using sparse machine learning and nonparametric density estimation algorithms

      2021, Renewable Energy
      Citation Excerpt :

      A novel joint quantile model is developed in Ref. [34] to facilitate the simultaneously evaluation of uncertainties. Recently [35], proposes a novel quantile regression model that is equipped with a novel nearest neighbors quantile filter. By leveraging a modified training data set, computational superiority is gained.

    View all citing articles on Scopus

    Jorge Ángel González Ordiano received his Ph.D. degree in mechanical engineering from the Karlsruhe Institute of Technology (KIT), Germany, in 2019. He currently works as a Postdoctoral Fellow at the Colorado State University within the Lab of Prof. Steven Simske. He is an expert in the fields of data mining, machine learning, time series analysis, and time series forecasting.

    Lutz Gröll received the Dipl.-Ing. and Dr.-Ing. degrees from the Technical University Dresden, Dresden, Germany, in 1985 and 1995, both in electrical engineering. Since 2001 he has been the head of the research group “Systems Theory and Control Engineering” at the Institute for Automation and Applied Informatics, Karlsruhe Institute of Technology (KIT), Karlsruhe, Germany. His research interests are in parameter identification, nonlinear control, and optimization theory.

    Ralf Mikut received the diploma degree in Automatic Control from the University of Technology, Dresden, Germany, in 1994, and the Ph.D. and Habilitation degree in mechanical engineering from the University of Karlsruhe, Karlsruhe, Germany, in 1999 resp. 2007. Since 2011, he is Associate Professor at the Faculty of Mechanical Engineering and Head of the Research Group “Automated Image and Data Analysis” at the Institute for Automation and Applied Informatics of the Karlsruhe Institute of Technology (KIT), Germany. His current research interests include data mining, image processing, big data, computational intelligence, life science applications, and smart grids.

    Veit Hagenmeyer studied engineering cybernetics in Stuttgart, Germany, and Berkeley, CA, USA. He wrote his dissertation in Paris, France, in the area of differential flatness and electrical drives. After Postdoctoral stays in Paris and Stuttgart, he worked in several positions at BASF SE, Ludwigshafen, Germany, mainly in the fields of Automation Technology, Advanced Process Control and Verbund Simulation. In his final position at BASF SE, he was responsible for the three power plants and energy grids at Ludwigshafen site. Now he is a Professor for energy informatics and a Research Director at the Karlsruhe Institute of Technology (KIT), Germany. In this position, together with his team, he is responsible for the Smart Energy System Simulation and Control Center, the ICT-part of the Energy Lab 2.0, and of the research field “Tools” of the Energy 2050 Initiative of the Helmholtz Association.

    View full text