Skip to main content

Advertisement

SpringerLink
Log in

A framework for testing the ability of models to project climate change and its impacts

  • Published:
Climatic Change Aims and scope Submit manuscript

Abstract

Models used for climate change impact projections are typically not tested for simulation beyond current climate conditions. Since we have no data truly reflecting future conditions, a key challenge in this respect is to rigorously test models using proxies of future conditions. This paper presents a validation framework and guiding principles applicable across earth science disciplines for testing the capability of models to project future climate change and its impacts. Model test schemes comprising split-sample tests, differential split-sample tests and proxy site tests are discussed in relation to their application for projections by use of single models, ensemble modelling and space-time-substitution and in relation to use of different data from historical time series, paleo data and controlled experiments. We recommend that differential-split sample tests should be performed with best available proxy data in order to build further confidence in model projections.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

References

  • Bates BC, Kundzewicz ZW, Wu S, Paulikof JF (eds) (2008) Climate change and water. Technical paper of the intergovernmental panel on climate change. IPCC Secretariat, Geneva

    Google Scholar 

  • Boberg F, Christensen JH (2012) Overestimation of Mediterranean summer temperature projections due to model deficiencies. Nat Clim Chang 2:433–436

    Article  Google Scholar 

  • Bommer JJ, Scherbaum F (2008) The use and misuse of logic trees in probabilistic seismic hazards analysis. Earthquake Spectra 24(4):997–1099, Earthquake Engineering Research Institute

    Article  Google Scholar 

  • Challinor AJ, Ewert F, Arnold S, Simelton E, Fraser E (2009) Crops and climate change: progress, trends and challenges in simulating impacts and informing adaptation. J Exp Bot 60:2775–2789

    Article  Google Scholar 

  • Christensen JH, Kjellström E, Giorgi F, Lenderink G, Rummukainen M (2010) Weight assignment in regional climate models. Clim Res 44(2–3):179–194

    Article  Google Scholar 

  • Coron L, Andréassian V, Perrin C, Lerat J, Vaxe J, Bourqui M, Hendrickx (2012) Crash testing hydrological models in contrasted climate conditions: An experiment on 216 Australian catchments. Water Resour Res 48, W05552

    Google Scholar 

  • Eitzinger J, Thaler S, Schmid E, Strauss F, Ferrise R, Moriondo M, Bindi M, Palosuo T, Rötter R, Kersebaum C, Olesen JE, Patil RH, Saylan L, Caldag B, Caylak O (2012) Sensitivities of crop models to extreme weather conditions during flowering period demonstrated for maize and winter wheat in Austria. J Agric Sci (in press)

  • Elsgaard L, Børgesen CD, Olesen JE, Siebert S, Ewert F, Peltonen-Sainio P, Rötter RP, Skjelvåg AO (2012) Shifts in comparative advantages for maize, oat, and wheat cropping under climate change in Europe. Food Addit Contam 29:1514–1526

    Article  Google Scholar 

  • Gleckler PJ, Taylor KE, Doutriaux C (2008) Performance metrics for climate models. J Geophys Res 113

  • Jensen HS, Andersen FØ (1992) Importance of temperature, nitrate and pH for phosphorus from aerobic sediments of four shallow, eutrophic lakes. Limnol Oceanogr 37:577–589

    Article  Google Scholar 

  • Jentsch A, Kreyling J, Beierkuhnlein C (2007) A new generation of climate-change experiments: events, not trends. Front Ecol Environ 5(7):365–374

    Article  Google Scholar 

  • Jørgensen SE (1995) State of the art of ecological modelling in limnology. Ecol Model 78:101–115

    Article  Google Scholar 

  • Karlsson IB, Sonnenborg TO, Jensen KH, Refsgaard JC (2013) Evaluating the influence of long term historical climate change on catchment hydrology—using drought and flood indices. Hydrol Earth Syst Sci Discuss 10:2373–2428

    Article  Google Scholar 

  • Klemes V (1986) Operational testing of hydrological simulation models. Hydrol Sci J 31:13–24

    Article  Google Scholar 

  • Liboriussen L, Lauridsen TL, Søndergaard M, Landkildehus F, Larsen SE, Jeppesen E (2011) Effects of warming and nutrients on sediment community respiration in shallow lakes: an outdoor mesocosm experiment. Freshw Biol 56(3):437–447

    Article  Google Scholar 

  • Madsen H (2000) Automatic calibration of a conceptual rainfall-runoff model using multiple objectives. J Hydrol 235:276–288

    Article  Google Scholar 

  • Meerhoff M, Teixeira-de Mello F, Kruk C, Alonso C, González-Bergonzoni I, Pacheco JP, Lacerot G, Arim M, Beklioğlu M, Bruce S, Goyenola G, Iglesias C, Mazzeo N, Kosten S, Jeppesen E (2012) Environmental warming in shallow lakes: a review of potential changes in community structure as evidenced from space-for-time substitution approaches. Adv Ecol Res 46:259–349

    Article  Google Scholar 

  • Murphy JM, Booth BBB, Collins M, Harris GR, Sexton DMH et al (2007) A methodology for probabilistic predictions of regional climate change from perturbed physics ensembles. Phil Trans R Soc A Math Phys Eng Sci 365:1993–2028

    Article  Google Scholar 

  • Olsen J, Anderson NJ, Knudsen MF (2012) Variability of the North Atlantic Oscillation over the past 5,200 years. Nat Geosci 5:808–812

    Article  Google Scholar 

  • Oreskes N, Shrader-Frechette K, Belitz K (1994) Verification, validation and confirmation of numerical models in the earth sciences. Science 264:641–646

    Article  Google Scholar 

  • Palosuo T, Kersebaum KC, Angulo C, Hlavinka P, Moriondo M, Olesen JE, Patil R, Ruget F, Rumbaur C, Takáč J, Trnka M, Bindi M, Caldag B, Ewert F, Ferrise R, Mirschel W, Saylan L, Šiška B, Rötter R (2011) Simulation of winter wheat yield and its variability in different climates of Europe. A comparison of eight crop growth models. Eur J Agron 35:103–114

    Article  Google Scholar 

  • Popper KR (1959) The logic of scientific discovery. Hutchingson & Co, London

  • Refsgaard JC, Henriksen HJ (2004) Modelling guidelines—terminology and guiding principles. Adv Water Resour 27(1):71–82

    Article  Google Scholar 

  • Refsgaard JC, Knudsen J (1996) Operational validation and intercomparison of different types of hydrological models. Water Resour Res 32(7):2189–2202

    Article  Google Scholar 

  • Refsgaard JC, Arnbjerg-Nielsen K, Drews M, Halsnæs K, Jeppesen E, Madsen H, Markandya A, Olesen JE, Porter JR, Christensen JH (2013) The role of uncertainty in climate change adaptation strategies—a Danish water management example. Mitig Adapt Strateg Glob Chang 18:337–359

    Article  Google Scholar 

  • Rötter R, Carter TR, Olesen JE, Porter JR (2011) Crop-climate models need an overhaul. Nat Clim Chang 1:175–177

    Article  Google Scholar 

  • Schlesinger S, Crosbie RE, Gagné RE, Innis GS, Lalwani CS, Loch J, Sylvester J, Wright RD, Kheir N, Bartos D (1979) Terminology for model credibility. SCS Technical Committee on Model Credibility. Simulation 32(3):103–104

    Article  Google Scholar 

  • Seibert J (2003) Reliability of model predictions outside calibration conditions. Nord Hydrol 34(5):477–492

    Google Scholar 

  • Singh R, Wagener T, van Werkhoven K, Mann ME, Crane R (2011) A trading-space-for-time approach to probabilistic continuous streamflow predictions in a changing climate—accounting for changing watershed behaviour. Hydrol Earth Syst Sci 15(11):3591–3601

    Article  Google Scholar 

  • Sunyer MA, Madsen H, Rosbjerg D, Arnbjerg-Nielsen K (2013) Regional interdependency of precipitation indices across Denmark in two ensembles of high resolution RCMs. J Clim. doi:10.1175/JCLI-D-12-00707.1

    Google Scholar 

  • Swayne D, Lam D, MacKay M, Rouse W, Schertzer W (2005) Assessment of the interaction between the Canadian Regional Climate Model and lake thermal-hydrodynamic models. Environ Model Softw 20:1505–1513

    Article  Google Scholar 

  • Taylor KE, Stouffer RJ, Meehl GA (2012) An overview of CMIP5 and the experiment design. Bull Am Meteorol Soc 93:485–498

    Article  Google Scholar 

  • Tebaldi C, Smith R, Nychka D, Mearns L (2005) Quantifying uncertainty in projections of regional climate change: a Bayesian approach to the analysis of multi-model ensembles. J Clim 18:1524–1540

    Article  Google Scholar 

  • Teutschbein C, Seibert J (2012) Is bias correction of Regional Climate Model (RCM) simulations possible for non-stationary conditions? Hydrol Earth Syst Sci Discuss 9:12765–12795

    Article  Google Scholar 

  • Trolle D, Hamilton DP, Pilditch CA, Duggan IC, Jeppesen E (2011) Predicting the effects of climate change on trophic status of three morphologically varying lakes: Implications for lake restoration and management. Environ Model Softw 26:354–370

    Article  Google Scholar 

  • Van Der Linden P, Mitchell JFB (eds) (2009) ENSEMBLES: climate change and its impacts: summary of research and results from the ENSEMBLES project. Met Office Hadley Centre, Exeter

    Google Scholar 

  • Van Steenbergen N, Willems P (2012) Method for testing the accuracy of rainfall-runoff models in predicting peak flow changes due to rainfall changes, in a climate changing context. J Hydrol 414–415:425–434

    Article  Google Scholar 

  • Velázquez JA, Schmid J, Ricard S, Muerth MJ, Gauvin St-Denis B, Minville M, Chaumont D, Caya D, Ludwig R, Turcotte R (2012) An ensemble approach to assess hydrological models' contribution to uncertainties in the analysis of climate change impact on water resources. HESSD, 9, 7441–7474

  • Wagener T, Sivapalan M, Troch PA, McGlynn BL, Harman CJ, Gupta HV, Kumar P, Rao PSC, Basu NB, Wilson JS (2010) The future of hydrology: an evolving science for a changing world. Water Resour Res 46, W05301

    Google Scholar 

  • Wilby RL, Harris I (2006) A framework for assessing uncertainties in climate change impacts: Low-flow scenarios for the River Thames, UK. Water Resour Res 42:W02419

    Google Scholar 

  • Wolf J, Evans LG, Semenov MA, Eckersteen H, Iglesias A (1996) Comparison of wheat simulation models under climate change. I. Model calibration and sensitivity analyses. Clim Res 7:253–270

    Article  Google Scholar 

Download references

Acknowledgments

The present study was funded by a grant from the Danish Council for Strategic Research for the project Centre for Regional Change in the Earth System (CRES—www.cres-centre.dk) under contract no: DSF-EnMi 09-066868.

Author information

Authors and Affiliations

  1. Geological Survey of Denmark and Greenland (GEUS), Øster Voldgade 10, 1350, Copenhagen K, Denmark

    J. C. Refsgaard & T. O. Sonnenborg

  2. DHI, Agern Alle 5, 2970, Hørsholm, Denmark

    H. Madsen

  3. IRSTEA, Antony, France

    V. Andréassian

  4. Technical University of Denmark, 2800, Lyngby, Denmark

    K. Arnbjerg-Nielsen

  5. Aarhus University, Vejlsøvej 25, 8600, Silkeborg, Denmark

    T. A. Davidson

  6. Technical University of Denmark, Risø, 4000, Roskilde, Denmark

    M. Drews

  7. Environmental Research Institute, University of Waikato, Hamilton, New Zealand

    D. P. Hamilton

  8. Department of Bioscience, Aarhus University, Vejlsøvej 25, 8600, Silkeborg, Denmark

    E. Jeppesen & D. Trolle

  9. Swedish Meteorological and Hydrological Institute, Norrköping, Sweden

    E. Kjellström

  10. Aarhus University, 8830, Tjele, Denmark

    J. E. Olesen

  11. KU Leuven, Leuven, Belgium

    P. Willems

  12. Danish Meteorological Institute, Lyngbyvej 100, 2100, Copenhagen Ø, Denmark

    J. H. Christensen

Authors
  1. J. C. Refsgaard
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. H. Madsen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. V. Andréassian
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. K. Arnbjerg-Nielsen
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. T. A. Davidson
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. M. Drews
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. D. P. Hamilton
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. E. Jeppesen
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. E. Kjellström
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. J. E. Olesen
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. T. O. Sonnenborg
    View author publications

    You can also search for this author in PubMed Google Scholar

  12. D. Trolle
    View author publications

    You can also search for this author in PubMed Google Scholar

  13. P. Willems
    View author publications

    You can also search for this author in PubMed Google Scholar

  14. J. H. Christensen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to J. C. Refsgaard.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Refsgaard, J.C., Madsen, H., Andréassian, V. et al. A framework for testing the ability of models to project climate change and its impacts. Climatic Change 122, 271–282 (2014). https://doi.org/10.1007/s10584-013-0990-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10584-013-0990-2

Keywords

Search

Navigation