Skip to main content
SpringerLink
Log in

Background error covariance iterative updating with invariant observation measures for data assimilation

  • Original Paper
  • Published:
Stochastic Environmental Research and Risk Assessment Aims and scope Submit manuscript

Abstract

In order to leverage the information embedded in the background state and observations, covariance matrices modelling is a pivotal point in data assimilation algorithms. These matrices are often estimated from an ensemble of observations or forecast differences. Nevertheless, for many industrial applications the modelling still remains empirical based on some form of expertise and physical constraints enforcement in the absence of historical observations or predictions. We have developed two novel robust adaptive assimilation methods named Covariance Updating iTerativE and Partially Updating BLUE. These two non-parametric methods are based on different optimization objectives, both capable of sequentially adapting background error covariance matrices in order to improve assimilation results under the assumption of a good knowledge of the observation error covariances . We have compared these two methods with the standard approach using a misspecified background matrix in a shallow water twin experiments framework with a linear observation operator. Numerical experiments have shown that the proposed methods bear a real advantage both in terms of posterior error correlation identification and assimilation accuracy.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

Notes

  1. Here, by the term “exact”, we refer to the covariance truly corresponding to the remaining errors present in the analysed state, no matter the level of optimality of the chosen assimilation scheme.

References

  • Argaud J-P, Bouriquet B, Courtois M, Le Roux J-C (2016) Reconstruction by data assimilation of the inner temperature field from outer measurements in a thick pipe. In: Pressure vessels and piping conference, British Columbia, Canada, July 17–21. ASME

  • Asch M, Bocquet M, Maëlle N (2016) Data assimilation: methods, algorithms, and applications. Fundamentals of algorithms. SIAM, Philadelphia

    Book  Google Scholar 

  • Bishop CH (2019) Data assimilation strategies for state dependent observation error variances. Q J R Meteorol Soc 145:217–227

    Article  Google Scholar 

  • Bouttier F, Courtier P (1999) Data assimilation, concepts and methods. Meteorological training course lecture series. ECMWF, European Center for Medium range Weather Forecast, Reading, UK

  • Carrassi A, Bocquet M, Bertino L, Evensen G (2018) Data assimilation in the geosciences: an overview of methods, issues, and perspectives. Wiley Interdiscip Rev Clim Change 9(5):e535

    Article  Google Scholar 

  • Chabot V, Berre L, Desroziers G (2017) Diagnosis and normalization of gridpoint background-error variances induced by a block-diagonal wavelet covariance matrix. Q J R Meteorol Soc 143(704):1268–1279

    Article  Google Scholar 

  • Chapnik B, Desroziers G, Rabier F, Talagrand O (2006) Diagnosis and tuning of observational error in a quasi-operational data assimilation setting. Q J R Meteorol Soc 132(615):543–565

    Article  Google Scholar 

  • Cherian A, Sra S, Banerjee A, Papanikolopoulos N (2011) Efficient similarity search for covariance matrices via the Jensen–Bregman LogDet divergence. In: 2011 international conference on computer vision. IEEE, pp 2399–2406

  • Cioaca A, Sandu A (2014) Low-rank approximations for computing observation impact in 4d-var data assimilation. Comput Math Appl 67(12):2112–2126

    Article  Google Scholar 

  • Clayton AM, Lorenc AC, Barker DM (2012) Operational implementation of a hybrid ensemble/4D-VAR global data assimilation system at the MET office. Q J R Meteorol Soc 139(675):1445–1461

    Article  Google Scholar 

  • Courtier P, Andersson E, Heckley W, Vasiljevic D, Hamrud M, Hollingsworth A, Rabier F, Fisher M, Pailleux J (1998) The ECMWF implementation of three-dimensional variational assimilation (3D-VAR). I: formulation. Q J R Meteorol Soc 124(550):1783–1807

    Google Scholar 

  • Desroziers G, Ivanov S (2001) Diagnosis and adaptive tuning of observation-error parameters in a variational assimilation. Q J R Meteorol Soc 127(574):1433–1452

    Article  Google Scholar 

  • Desroziers G, Berre L, Chapnik B, Poli P (2005) Diagnosis of observation, background and analysis-error statistics in observation space. Q J R Meteorol Soc 131(613):3385–3396

    Article  Google Scholar 

  • Dreano D, Tandeo P, Pulido M, Ait-El-Fquih B, Chonavel T, Hoteit I (2017) Estimating model error covariances in nonlinear state-space models using Kalman smoothing and the expectation–maximisation algorithm. Q J R Meteorol Soc 143(705):1877–1885

    Article  Google Scholar 

  • Eyre JR, Hilton FI (2013) Sensitivity of analysis error covariance to the mis-specification of background error covariance. Q J R Meteorol Soc 139(671):524–533

    Article  Google Scholar 

  • Fisher M (2003) Background error covariance modelling. In: Seminar on recent developments in data assimilation for atmosphere and ocean, Shinfield Park, Reading, 8–12 September. ECMWF

  • Fisher M, Leutbecher M, Kelly GA (2005) On the equivalence between Kalman smoothing and weak-constraint four-dimensional variational data assimilation. Q J R Meteorol Soc 131(613):3235–3246

    Article  Google Scholar 

  • Fitt A, Norbury J, Ockendon H, Eddie Wilson R (2010) Progress in industrial mathematics at ECMI 2008. In: Proceedings of the 15th European conference on mathematics for industry, vol 15, London, UK, June 30–July 4, 2008

  • Gaspari G, Cohn ES (1999) Construction of correlation functions in two and three dimensions. Q J R Meteorol Soc 125(554):723–757

    Article  Google Scholar 

  • Goeury C, Ponçot A, Argaud J-P, Zaoui F, Ata R, Audouin Y (2017) Optimal calibration of TELEMAC-2D models based on a data assimilation algorithm. In: the 14th TELEMAC-MASCARET user conference, 17–20 October 2017, Graz University of Technology, Graz, Austria

  • Hodges J, Reich B (2010) Adding spatially-correlated errors can mess up the fixed effect you love. Am Stat 64(4):325–334

    Article  Google Scholar 

  • Hollingsworth A, Lönnberg P (1989) The verification of objective analyses: diagnostics of analysis system performance. Meteorol Atmos Phys 40:3–27

    Article  Google Scholar 

  • Hunt B, Kostelich E, Szunyogh I (2005) Efficient data assimilation for spatiotemporal chaos: a local ensemble transform Kalman filter. Phys D 230(1):112–126

    Google Scholar 

  • Ishibashi T (2015) Tensor formulation of ensemble-based background error covariance matrix factorization. Mon Weather Rev 143(12):4963–4973

    Article  Google Scholar 

  • Janjić T, Bormann N, Bocquet M, Carton JA, Cohn SE, Dance SL, Losa SN, Nichols NK, Potthast R, Waller JA, Weston P (2018) On the representation error in data assimilation. Q J R Meteorol Soc 144(713):1257–1278

    Article  Google Scholar 

  • Kalnay E, Yang S-C (2010) Accelerating the spin-up of Ensemble Kalman Filtering. Q J R Meteorol Soc 136(651):1644–1651

    Article  Google Scholar 

  • Leisenring M, Moradkhani H (2011) Snow water equivalent prediction using Bayesian data assimilation methods. Stoch Environ Res Risk Assess 25(2):253–270

    Article  Google Scholar 

  • Li W, Sankarasubramanian A, Ranjithan RS, Sinha T (2016) Role of multimodel combination and data assimilation in improving streamflow prediction over multiple time scales. Stoch Environ Res Risk Assess 30(8):2255–2269

    Article  Google Scholar 

  • Liu C, Xue M (2016) Relationships among four-dimensional hybrid ensemble-variational data assimilation algorithms with full and approximate ensemble covariance localization. Mon Weather Rev 144(2):591–606

    Article  Google Scholar 

  • Lucor D, Le Maître OP (2018) Cardiovascular modeling with adapted parametric inference. ESAIM: ProcS 62:91–107

    Article  Google Scholar 

  • Mirouze I (2010) Régularisation de problèmes inverses à l’aide de l’équation de diffusion, avec application à l’assimilation variationnelle de données océaniques. Ph.D. thesis, Université de Toulouse

  • Parrish DF, Derber JC (1992) The National Meteorological Center’s spectral statistical-interpolation analysis system. Mon Weather Rev 120(8):1747–1763

    Article  Google Scholar 

  • Pennec X, Fillard P, Ayache N (2006) A Riemannian framework for tensor computing. Int J Comput Vis 66(1):41–66

    Article  Google Scholar 

  • Ponçot A, Argaud J-P, Bouriquet B, Erhard P, Gratton S, Thual O (2013) Variational assimilation for xenon dynamical forecasts in neutronic using advanced background error covariance matrix. Ann Nucl Energy 60:39–50 10

    Article  CAS  Google Scholar 

  • Rabier F (2005) Overview of global data assimilation developments in numerical weather-prediction centres. Q J R Meteorol Soc 131(613):3215–3233

    Article  Google Scholar 

  • Rochoux M, Collin A, Zhang C, Trouvé A, Lucor D, Moireau P (2018) Front shape similarity measure for shape-oriented sensitivity analysis and data assimilation for Eikonal equation. ESAIM: ProcS 63:258–279

    Article  Google Scholar 

  • Saint-Venant AB (1871) Théorie du mouvement non permanent des eaux, avec application aux crues des rivières et a l’introduction de marées dans leurs lits. Comptes rendus de l’Académie des Sciences 73:147–154

    Google Scholar 

  • Sénégas J, Wackernagel H, Rosenthal W, Wolf T (2001) Error covariance modeling in sequential data assimilation. Stoch Environ Res Risk Assess 15(1):65–86

    Article  Google Scholar 

  • Singh K, Jardak M, Sandu A, Bowman K, Lee M, Jones D (2011) Construction of non-diagonal background error covariance matrices for global chemical data assimilation. Geosci Model Dev 4(2):299–316

    Article  Google Scholar 

  • Sinsbeck M, Tartakovsky D (2015) Impact of data assimilation on cost-accuracy tradeoff in multifidelity models. SIAM/ASA J Uncertain Quantif 3(1):954–968

    Article  Google Scholar 

  • Sørensen JVT, Madsen H (2004) Data assimilation in hydrodynamic modelling: on the treatment of non-linearity and bias. Stoch Environ Res Risk Assess 18(4):228–244

    Article  Google Scholar 

  • Stewart LM, Dance SL, Nichols NK (2013) Data assimilation with correlated observation errors: experiments with a 1-D shallow water model. Tellus A: Dyn Meteorol Oceanogr 65(1):19546

    Article  Google Scholar 

  • Talagrand O (1999) A posteriori evaluation and verification of analysis and assimilation algorithms. In: Workshop on diagnosis of data assimilation systems, Shinfield Park, Reading, ECMWF, pp 17–28

  • Waller JA, Simonin D, Dance SL, Nichols NK, Ballard SP (2016) Diagnosing observation error correlations for doppler radar radial winds in the Met office UKV model using observation-minus-background and observation-minus-analysis statistics. Mon Weather Rev 144(10):3533–3551

    Article  Google Scholar 

  • Weaver AT, Mirouze I (2013) On the diffusion equation and its application to isotropic and anisotropic correlation modelling in variational assimilation. Q J R Meteorol Soc 139(670):242–260

    Article  Google Scholar 

  • Xu M, Yuan B, Wang L, Zhang L (2017) Data assimilation for Fukushima nuclear accident assessments. In: International conference on nuclear engineering, vol 4, Shanghai, China. ASME

Download references

Author information

Authors and Affiliations

  1. LIMSI, CNRS, Université Paris-Saclay, Campus Universitaire bâtiment 507, Rue John Von Neumann, 91400, Orsay, France

    Sibo Cheng & Didier Lucor

  2. Institut de Mathématiques de Toulouse, Université Paul Sabatier, Toulouse, France

    Bertrand Iooss

  3. EDF R&D, EDF Lab Paris-Saclay, 7 Boulevard Gaspard Monge, 91120, Palaiseau, France

    Sibo Cheng, Jean-Philippe Argaud & Angélique Ponçot

  4. EDF R&D, EDF Lab Chatou, 6 quai Watier, 78400, Chatou, France

    Bertrand Iooss

Authors
  1. Sibo Cheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jean-Philippe Argaud
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Bertrand Iooss
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Didier Lucor
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Angélique Ponçot
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Didier Lucor.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Cheng, S., Argaud, JP., Iooss, B. et al. Background error covariance iterative updating with invariant observation measures for data assimilation. Stoch Environ Res Risk Assess 33, 2033–2051 (2019). https://doi.org/10.1007/s00477-019-01743-6

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00477-019-01743-6

Keywords

Search

Navigation