Skip to main content
SpringerLink
Log in

Ensemble Bayesian model averaging using Markov Chain Monte Carlo sampling

  • Original Article
  • Published:
Environmental Fluid Mechanics Aims and scope Submit manuscript

Abstract

Bayesian model averaging (BMA) has recently been proposed as a statistical method to calibrate forecast ensembles from numerical weather models. Successful implementation of BMA however, requires accurate estimates of the weights and variances of the individual competing models in the ensemble. In their seminal paper (Raftery et al. Mon Weather Rev 133:1155–1174, 2005) has recommended the Expectation–Maximization (EM) algorithm for BMA model training, even though global convergence of this algorithm cannot be guaranteed. In this paper, we compare the performance of the EM algorithm and the recently developed DiffeRential Evolution Adaptive Metropolis (DREAM) Markov Chain Monte Carlo (MCMC) algorithm for estimating the BMA weights and variances. Simulation experiments using 48-hour ensemble data of surface temperature and multi-model streamflow forecasts show that both methods produce similar results, and that their performance is unaffected by the length of the training data set. However, MCMC simulation with DREAM is capable of efficiently handling a wide variety of BMA predictive distributions, and provides useful information about the uncertainty associated with the estimated BMA weights and variances.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Ajami NK, Duan Q, Sorooshian S (2007) An integrated hydrologic Bayesian multimodel combination framework: confronting input, parameter, and model structural uncertainty in hydrologic prediction. Water Resour Res 43: W01403. doi:10.1029/2005WR004745

    Article  Google Scholar 

  2. Barnston AG, Mason SJ, Goddard L, DeWitt DF, Zebiak SE (2003) Multimodel ensembling in seasonal climate forecasting at IRI. Bull Am Meteorl Soc 84: 1783–1796. doi:10.1175/BAMS-84-12-1783

    Article  Google Scholar 

  3. Dempster AP, Laird NM, Rubin DB (1977) Maximum likelihood from incomplete data via the EM algorithm. J R Stat Soc 39: 1–39

    Google Scholar 

  4. Doblas-Reyes FJ, Hagedorn R, Palmer TN (2005) The rationale behind the success of multi-model ensembles in seasonal forecasting—II. Calibration and combination. Tellus 57: 234–252

    Article  Google Scholar 

  5. Duan Q, Sorooshian S, Gupta V (1992) Effective and efficient global optimization for conceptual rainfall-runoff models. Water Resour Res 28(4): 1015–1031

    Article  Google Scholar 

  6. Gelman A, Rubin DR (1992) Inference from iterative simulation using multiple sequences. Stat Sci 7: 457–472

    Article  Google Scholar 

  7. Georgekakos KP, Seo DJ, Gupta H, Schaake J, Butts MB (2004) Characterizing streamflow simulation uncertainty through multi-model ensembles. J Hydrol 298(1–4): 222–241

    Article  Google Scholar 

  8. Gneiting T, Raftery AE (2006) Weather forecasting with ensemble methods. Science 310: 248–249

    Article  Google Scholar 

  9. Gneiting T, Raftery AE, Westerveld AH III, Goldman T (2005) Calibrated probabilistic forecasting using ensemble model output statistics and minimum CRPS estimation. Mon Weather Rev 133: 1098–1118

    Article  Google Scholar 

  10. Grimitt EP, Mass CF (2002) Initial results of a mesoscale short-range ensemble forecasting system over the Pacific Northwest. Wea Forecast 17: 192–205

    Article  Google Scholar 

  11. Hamill TM (2007) Comments on “Calibrated surface temperature forecasts from the Canadian ensemble prediction system using Bayesian model averaging”. Mon Weather Rev 135: 4226–4230. doi:10.1175/2007MWR1963.1

    Article  Google Scholar 

  12. Hamill TM, Colucci SJ (1997) Verification of Eta-RSM short range ensemble forecasts. Mon Weather Rev 125: 1312–1327

    Article  Google Scholar 

  13. Krishnamurti TN, Kishtawal CM, LaRow TE, Bachiochi D, Zhang Z, Williford CE, Gadgil S, Surendan S (1999) Improved weather and seasonal climate forecasts from multimodel superensembles. Science 258: 1548–1550

    Article  Google Scholar 

  14. McLachlan GJ, Krishnan T (1997) The EM algorithm and extensions. Wiley, New York, p 274

    Google Scholar 

  15. Min S, Hense A (2006) A Bayesian approach to climate model evaluation and multi-model averaging with an application to global mean surface temperatures from IPCC AR4 coupled climate models. Geophys Res Lett 33: 1–5

    Article  Google Scholar 

  16. Molteni F, Buizza R, Palmer TN, Petroliagis T (1996) The ECWMF ensemble prediction system: methodology and validation. Q J R Meteorol Soc 122: 73–119

    Article  Google Scholar 

  17. Neuman SP (2003) Maximum likelihood Bayesian averaging of uncertain model predictions. Stoch Environ Res Risk Assess 17: 291–305. doi:10.1007/800477-003-0151-7

    Article  Google Scholar 

  18. Palmer TN, Alessandri A, Andersen U, Cantelaube P, Davey M, Délécluse P, Déqué M, Diez E, Doblas-Reyes J, Feddersen H, Graham R, Gualdi S, Guérémy J-F, Hagedorn R, Hoshen M, Keenlyside N, Latif M, Lazar A, Maisonnave E, Marletto V, Morse AP, Orfila B, Rogel P, Terres J-M, Thomson MC (2004) Development of a European Multi-model ensemble system for seasonal-to-interannual prediction (DEMETER). Bull Am Meteorol Soc 85: 853–872. doi:10.1175/BAMS-85-6-853

    Article  Google Scholar 

  19. Raftery AE, Madigan D, Hoeting JA (1997) Bayesian model averaging for linear regression models. J Am Stat Assoc 92: 179–191

    Article  Google Scholar 

  20. Raftery AE, Gneiting T, Balabdaoui F, Polakowski M (2005) Using Bayesian model averaging to calibrate forecast ensembles. Mon Weather Rev 133: 1155–1174

    Article  Google Scholar 

  21. Rajagopalan B, Lall U, Zebiak SE (2002) Categorical climate forecasts through regularization and optimal combination of multiple GCM ensembles. Mon Weather Rev 130: 1792–1811

    Article  Google Scholar 

  22. Richardson DS (2001) Measures of skill and value of ensemble prediction systems, their interrelationship and the effect of sample size. Q J R Meteorol Soc 127: 2473–2489

    Article  Google Scholar 

  23. Sloughter JM, Raftery AE, Gneiting T (2006) Probabilistic quantitative precipitation forecasting using Bayesian model averaging. University of Washington, Department of Statistics, Technical Report 496, Seattle, WA, 20 pp

  24. Vrugt JA, Robinson BA (2006) Treatment of uncertainty using ensemble methods: comparison of sequential data assimilation and Bayesian model averaging. Water Resour Res W01411. doi:10.1029/2005WR004838

  25. Vrugt JA, Gupta HV, Bouten W, Sorooshian S (2003) A Shuffled Complex Evolution Metropolis algorithm for optimization and uncertainty assessment of hydrologic model parameters. Water Resour Res 39(8): 1201. doi:10.1029/2002WR001642

    Article  Google Scholar 

  26. Vrugt JA, Clark MP, Diks CGH, Duan Q, Robinson BA (2006) Multi-objective calibration of forecast ensembles using Bayesian model averaging. Geophys Res Lett 33: L19817. doi:10.1029/2006GL027126

    Article  Google Scholar 

  27. Vrugt JA, ter Braak CJF, Diks CGH, Robinson BA, Hyman JM, Higdon D (2008a) Accelerating Markov chain Monte Carlo simulation by self-adaptive differential evolution with randomized subspace sampling. Int J Nonlinear Sci Numer Simul (in press)

  28. Vrugt JA, ter Braak CJF, Clark MP, Hyman JM, Robinson BA (2008b) Treatment of input uncertainty in hydrologic modeling: doing hydrology backwards with Markov Chain Monte Carlo simulation. Water Resour Res doi:10.1029/2007WR006720

  29. Wöhling T, Vrugt JA (2008) Combining multi-objective optimization and Bayesian model averaging to calibrate forecast ensembles of soil hydraulic models. Water Resour Res doi:10.1029/2008WR007154

  30. Ye M, Neuman SP, Meyer PD (2004) Maximum likelihood Bayesian averaging of spatially variability models in unsaturated fractured tuff. Water Resour Res 40: W05113. doi:10.1029/2003WR002557

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Los Alamos National Laboratory (LANL), Center for NonLinear Studies (CNLS), Mail Stop B258, Los Alamos, NM, 87545, USA

    Jasper A. Vrugt

  2. Center for Nonlinear Dynamics in Economics and Finance, University of Amsterdam, Amsterdam, The Netherlands

    Cees G. H. Diks

  3. NIWA, P.O. Box 8602, Riccarton, Christchurch, New Zealand

    Martyn P. Clark

Authors
  1. Jasper A. Vrugt
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Cees G. H. Diks
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Martyn P. Clark
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jasper A. Vrugt.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Vrugt, J.A., Diks, C.G.H. & Clark, M.P. Ensemble Bayesian model averaging using Markov Chain Monte Carlo sampling. Environ Fluid Mech 8, 579–595 (2008). https://doi.org/10.1007/s10652-008-9106-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10652-008-9106-3

Keywords

Search

Navigation