A novel hybrid artificial neural network - Parametric scheme for postprocessing medium-range precipitation forecasts
Introduction
Statistical postprocessing techniques are increasingly used to improve the reliability and skill of real time probabilistic quantitative precipitation forecasts (PQPFs) produced by numerical weather prediction (NWP) models. Broadly speaking, these techniques can be categorized as nonparametric and parametric ones. A prominent example of the former is the Analog approach (Hamill and Whitaker, 2006; Hamill et al., 2015). The parametric techniques rely on prescribed parametric forms of conditional (predictive), joint and marginal distributions, and employ various techniques ranging from regression to the method of moments, and their variants, for estimating distributional parameters. Many of the modern parametric approaches fall under the broad umbrella of Ensemble Model Output Statistics (EMOS; Gneiting et al., 2005), also known as nonhomogeneous regression. As the name implies, the EMOS approaches use prescribed predictive distributions and relate distributional parameters to ensemble statistics through a set of regression equations (Scheuerer and Hamill, 2015; Zhang et al., 2017; Stauffer et al., 2017).
The extent to which postprocessing techniques have improved forecast skill has varied in practice (Li et al., 2017; Wilks, 2018; Vannitsem et al., 2020). There are several common limitations in postprocessing methods adopted to date. Among the frequently cited are the inflexible and subjective way of selecting predictors, structural rigidity that makes it difficult to integrate ancillary predictors, and the ad hoc way of determining spatial-temporal training domains (see related discussions in Rasp and Lerch (2018)). The advent of machine learning techniques offers many new opportunities to address these limitations. Relative to the parametric approaches, EMOS techniques included, some of the recent machine learning techniques offer flexibility in identifying predictors, in integrating ancillary information, and in capturing complex, nonlinear predictor-predictand relationships that are difficult to characterize parametrically (see, e.g., Taillardat et al., 2019). Particularly promising are the various artificial neural networks (ANNs) which have been known for their ability to model nonlinear dependencies. Recent years have seen an explosion of ANN-based prediction paradigms (Liu et al., 2016; Brenowitz and Bretherton, 2018; Gentine et al., 2018; Rasp et al., 2018; Chapman et al., 2019; Cloud et al., 2019; Gagne et al., 2019; Lagerquist et al., 2019). Yet, the use of these techniques in the context of postprocessing remains relatively limited. Rasp and Lerch (2018) is perhaps the first attempt of this nature. The authors explored a hybrid scheme that retains a parametric form of the predictive distribution of 2-m temperature but relies on ANNs to estimate the distribution parameters from the ensemble statistics of 2-m temperature as well as ancillary variables. Scheuerer et al. (2020), in a similar vein, developed an ANN-based scheme for producing 7-day accumulated PQPFs at subseasonal range (2‒4 weeks) from NWP ensemble forecasts, and showed that the PQPFs thus generated broadly outperforms climatology. Other studies of note include Bremnes (2020) wherein ANN was used for postprocessing wind speed forecasts. Collectively, these studies indicate that embedding local information and incorporating ancillary forecast variables can lead to clear, discernible improvements in forecast skills. They further suggest that ANN models, contrary to the common perception of being black boxes, can help uncover, and offer physical insights to the meteorological processes that underpin the links between predictors and predictands.
Inspired by the successes of recent ANN-based postprocessing approaches, and motivated by the broader need for improving the skill of PQPF while circumventing limitations inherent in existing EMOS schemes, we propose a hybrid ANN-nonhomogeneous regression-based scheme capable of postprocessing precipitation forecasts at multiple lead times and seasons in a unified way. The proposed scheme retains the parametric form of the predictive distribution of precipitation proposed by Scheuerer and Hamill (2015) and Baran and Nemoda (2016), but departs from the conventional EMOS by using ANNs to relate NWP forecasts to the distributional parameters. The potential advantages of the proposed scheme, which we will henceforth refer to as ANN-CSGD are three-fold. First, this scheme does not require an explicit prescription of predictor-predictand relationships as is currently done in EMOS models - it can discover and integrate arbitrary nonlinear relationships through training. Second, the training of the model can be done using the entire data archive and thereby obviate the need for explicit treatment of lead time-based and seasonally varying NWP forecast errors. Third, it can account for seasonal variations in the interaction between NWP forecasts and temporal predictors.
In this paper we describe and evaluate the proposed scheme which relies only on the ensemble mean of NWP forecasts as the major predictor. The evaluation is conducted for sub-basins within three selected river basins in California. The proposed scheme is applied to postprocess Global Ensemble Forecast System (GEFS; Hamill et al., 2013) precipitation reforecasts along with two benchmark schemes. The first is the single predictor version of the censored, shifted gamma distribution (CSGD; Scheuerer and Hamill, 2015). The second is the Mixed-type Mata-Gaussian Distribution (MMGD; Wu et al., 2011), which has been the standard method in the U.S. National Weather Service (NWS) Hydrologic Ensemble Forecast Service (HEFS; Demargne et al., 2014). Our overarching hypothesis is that the flexibility accorded by the ANN-based model in establishing complex predictor-distributional parameter relationships, in determining temporal training windows, and in lumping forecasts for different lead times, will help the proposed scheme attain superior predictive performance relative to the benchmarks.
The reminder of this paper is organized as follows. Section 2 describes the proposed ANN-CSGD scheme as well as the benchmark methods, data, and experimental setup. Section 3 presents the outcomes of the experiments and section 4 summarizes the findings and discusses future possible extensions.
Section snippets
Proposed model
The censored, shifted gamma distribution (CSGD) introduced by Scheuerer and Hamill (2015), has been a popular choice to represent the right skewed, mixed-type dichotomous-continuous nature of the predictive distribution of precipitation (Scheuerer and Hamill, 2015; Baran and Nemoda, 2016; Zhang et al., 2017; Scheuerer et al., 2020). Let Fk,θ denote the cumulative distribution function (CDF) of the gamma distribution with shape parameter k > 0 and scale parameter θ > 0. The CDF at realized
Results
In this section we present verification results using different metrics (see Appendix B for mathematical definitions and details). We first use the continuous ranked probability skill score (CRPSS) to assess the overall predictive performance of PQPFs from ANN-CSGD relative to those from the benchmark models with different training scenarios. Subsequently, we analyze ANN-CSGD's performance relative to the benchmark models with a 61-day training window, using Brier skill score (BSS), reliability
Discussion and conclusions
We propose a unified, univariate, hybrid neural network-parametric PQPF postprocessing scheme capable of producing postprocessed forecasts for lead times at least up to 7 days (medium-range). This scheme retains the use of parametric predictive distribution, but employs ANN to estimate distribution parameters from forecast-observation pairs. The predictors explored in this study include ensemble mean forecast, forecast lead time, and month of the year, whereas the predictands are three
CRediT authorship contribution statement
Mohammadvaghef Ghazvinian: Conceptualization, Data curation, Formal analysis, Methodology, Software, Validation, Visualization, Writing - original draft. Yu Zhang: Conceptualization, Funding acquisition, Investigation, Methodology, Project administration, Resources, Software, Supervision, Writing - review & editing. Dong-Jun Seo: Conceptualization, Methodology, Funding acquisition, Writing - review & editing. Minxue He: Conceptualization, Methodology, Writing - review & editing. Nelun Fernando:
Declaration of Competing Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Acknowledgements
The authors thank the editor and reviewers for their valuable comments that helped improve the article. The first author was financially supported by the faculty startup fund for Dr. Yu Zhang provided by UT Arlington, NOAA Grant NA18OAR4590370-01, Texas Water Development Board Contract No. 1800012276, and NSF grant 1909367. These supports are duly acknowledged here. The authors would also like to thank Michael Scheuerer at Norwegian Computing Center (NR) whose comments and suggestions led to
References (70)
- et al.
Combining predictive distributions for statistical post-processing of ensemble forecasts
Int. J. Forecast.
(2018) - et al.
Verification of temperature, precipitation, and streamflow forecasts from the NOAA/NWS hydrologic ensemble forecast service (HEFS): 1. Experimental design and forcing verification
Hydrol
(2014) - et al.
Verification of temperature, precipitation, and streamflow forecasts from the NOAA/NWS hydrologic ensemble forecast service (HEFS): 2. Streamflow verification
Hydrol
(2014) Neural networks for probabilistic environmental prediction: conditional density estimation network creation and evaluation (CaDENCE) in R
Comput. Geosci.
(2012)Verification of ensemble water supply forecasts for Sierra Nevada watersheds
Hydrology
(2016)- et al.
Generic probability distribution of rainfall in space: the bivariate model
J. Hydrol.
(2005) Univariate ensemble postprocessing
- et al.
Generation of ensemble precipitation forecast from single-valued quantitative precipitation forecast for hydrologic ensemble prediction
J. Hydrol.
(2011) - et al.
Tensorflow: a system for largescale machine learning
- Ba, J.L., J.R. Kiros, and G.E. Hinton, 2016: Layer normalization. arXiv preprint arXiv:1607.06450....
Censored and shifted gamma distribution based EMOS model for probabilistic quantitative precipitation forecasting
Environmetrics
Mixture EMOS model for calibrating ensemble forecasts of wind speed
Environmetrics
Controlling the false discovery rate: a practical and powerful approach to multiple testing
J. Roy. Stat. Soc. B
Ensemble postprocessing using quantile function regression based on neural networks and Bernstein polynomials
Mon. Wea. Rev.
Prognostic validation of a neural network unified physics parameterization
Geophys. Res. Lett.
Verification of forecasts expressed in terms of probability
Mon. Wea. Rev.
Increasing the reliability of reliability diagrams
Wea. Forecast.
Verification of precipitation forecasts from NCEP's short-range ensemble forecast (SREF) system with reference to ensemble streamflow prediction using lumped hydrologic models
J. Hydrometeor.
Improving atmospheric river forecasts with machine learning
Geophys. Res. Lett.
Fast and accurate deep network learning by exponential linear units (ELUs)
A feed forward neural network based on model output statistics for short-term hurricane intensity prediction
Wea. Forecast.
Natural language processing (almost) from scratch
J. Mach. Learn. Res.
Physiographically sensitive mapping of climatological temperature and precipitation across the conterminous United States
Int. J. Climatol.
The science of NOAA's operational hydrologic ensemble forecast service
Bull. Amer. Meteor. Soc.
Simulated hydrologic responses to climate variations and change in the Merced, Carson, and American river basins, Sierra Nevada, California, 1900–2099
Clim. Change
Comparing predictive accuracy
J. Bus. Econ. Stat.
Interpretable deep learning for spatial analysis of severe hailstorms
Mon. Wea. Rev.
Could machine learning break the convection parameterization deadlock?
Geo-Phys. Res. Lett.
A nonhomogeneous regression-based statistical postprocessing scheme for generating probabilistic quantitative precipitation forecast
J. Hydrometeor.
Improving medium-range probabilistic quantitative precipitation forecast for heavy-to-extreme events through the conditional bias-penalized regression
Calibrated probabilistic forecasting using ensemble model output statistics and minimum CRPS estimation
Mon. Wea. Rev.
Probabilistic forecasts, calibration and sharpness
J. Roy. Stat. Soc.
Deep Learning
Cited by (38)
Modeling transient natural convection in heterogeneous porous media with Convolutional Neural Networks
2024, International Journal of Heat and Mass TransferDeep-learning post-processing of short-term station precipitation based on NWP forecasts
2023, Atmospheric ResearchMulti-step ahead probabilistic forecasting of multiple hydrological variables for multiple stations
2023, Journal of HydrologyEnhancing short-term forecasting of daily precipitation using numerical weather prediction bias correcting with XGBoost in different regions of China
2023, Engineering Applications of Artificial IntelligenceCitation Excerpt :In summary, the results showed a decreasing pattern of forecast accuracy from 1–8 d lead times for three methods in all seven climate zones. This is also accorded with the law obtained by predecessors (Ghazvinian et al., 2021; Liu et al., 2019). The climate influence factors will gradually increase as the lead time increases.
Convolutional neural network-based statistical post-processing of ensemble precipitation forecasts
2022, Journal of HydrologyCitation Excerpt :Therefore, adding nearby spatial predictors can improve forecast skills. As mentioned in the introduction, there are post-processing methods that incorporate machine learning methods to provide probabilistic meteorological forecasts for predictand such as temperature (e.g., Peng et al., 2020; Rasp and Lerch, 2018) and precipitation (Ghazvinian et al., 2021). Our work extends the existing FCN-based models to CNN-based models for precipitation forecasts, which enables the model to make use of multi-scale spatial information and multiple auxiliary predictors.