Skip to main content
SpringerLink
Log in

Efficient methods for computing observation impact in 4D-Var data assimilation

  • ORIGINAL PAPER
  • Published:
Computational Geosciences Aims and scope Submit manuscript

Abstract

This paper presents a practical computational approach to quantify the effect of individual observations in estimating the state of a system. Such a methodology can be used for pruning redundant measurements and for designing future sensor networks. The mathematical approach is based on computing the sensitivity of the analyzed model states (unconstrained optimization solution) with respect to the data. The computational cost is dominated by the solution of a linear system, whose matrix is the Hessian of the cost function, and is only available in operator form. The right-hand side is the gradient of a scalar cost function that quantifies the forecast error of the numerical model. The use of adjoint models to obtain the necessary first- and second-order derivatives is discussed. We study various strategies to accelerate the computation, including matrix-free iterative solvers, preconditioners, and an in-house multigrid solver. Experiments are conducted on both a small-size shallow-water equations model and on a large-scale numerical weather prediction model, in order to illustrate the capabilities of the new methodology.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Palmer, T., Gelaro, R., Barkmeijer, J., Buizza, R.: Singular, vectors, metrics, and adaptive observations. J. Atmos. Sci. 55(4), 633–653 (1998)

    Article  Google Scholar 

  2. Liao, W., Sandu, A., Carmichael, G., et al.: Total energy singular vector analysis for atmospheric chemical transport models. Mon. Weather Rev. 134(9), 2443–2465 (2006)

    Article  Google Scholar 

  3. Bishop, C.H., Toth, Z.: Ensemble transformation and adaptive observations. J. Atmos. Sci. 56(11), 1748–1765 (1999)

    Article  Google Scholar 

  4. Bishop, C.H., Etherton, B.J., Majumdar, S.J.: Adaptive sampling with the ensemble transform Kalman filter. Mon. Weather Rev. 129(3), 420–436 (2001)

    Article  Google Scholar 

  5. Bergot, T., Doerenbecher, A.: A study on the optimization of the deployment of targeted observations using adjoint-based methods. Q. J. R. Meteorol. Soc. 128(583), 1689–1712 (2002)

    Article  Google Scholar 

  6. Fourrie, N., Doerenbecher, A., Bergot, T., Joly, A.: Adjoint sensitivity of the forecast to TOVS observations. Q. J. R. Meteorol. Soc. 128(586), 2759–2777 (2002)

    Article  Google Scholar 

  7. Joly, A., Jorgensen, D.: The fronts and Atlantic storm-track experiment (FASTEX): Scientific objectives and experimental design. Bull. Am. Meteorol. Soc. 78(9), 1917–1941 (1997)

    Article  Google Scholar 

  8. Langland, R.H., Toth, Z., Gelaro, R., et al.: The North Pacific Experiment (NORPEX-98): Targeted observations for improved North American weather forecasts. Bull. Am. Meteorol. Soc. 80(7), 1363–1384 (1998)

    Article  Google Scholar 

  9. Fourrie, N., Marchal, D., Rabier, F., et al.: Impact study of the 2003 North Atlantic THORPEX regional campaign. Q. J. R. Meteorol. Soc. 132(615), 275–295 (2006)

    Article  Google Scholar 

  10. Liu, J., Kalnay, E.: Estimating observation impact without adjoint model in an ensemble Kalman filter. Q. J. R. Meteorol. Soc. 134(634), 1327–1335 (2008)

    Article  Google Scholar 

  11. Baker, N.L., Daley, R.: Observation and background adjoint sensitivity in the adaptive observation-targeting problem. Q. J. R. Meteorol. Soc. 126(565), 1431–1454 (2000)

    Article  Google Scholar 

  12. Langland, R.H., Baker, N.L.: Estimation of observation impact using the NRL atmospheric variational data assimilation adjoint system. Tellus A 56(3), 189–201 (2004)

    Article  Google Scholar 

  13. Daescu, D.N., Navon, I.M.: Adaptive observations in the context of 4D-Var data assimilation. Met. Atmos. Phys 85(4), 205–226 (2004)

    Article  Google Scholar 

  14. Tremolet, Y.: Computation of observation sensitivity and observation impact in incremental variational data assimilation. Tellus A 60(5), 964–978 (2008)

    Article  Google Scholar 

  15. Berliner, L.M., Lu, Z.Q., Snyder, C.: Statistical design for adaptive weather observations. J. Atmos. Sci. 56(15), 2536–2552 (1999)

    Article  Google Scholar 

  16. Zupanski, D., Hou, A.Y., Zhang, S.Q., Zupanski, M., Kummerow, C.D., Cheung, S.H.: Applications of information theory in ensemble data assimilation. Q. J. R. Meteorol. Soc. 133(627), 1533–1545 (2007)

    Article  Google Scholar 

  17. Singh, K., Sandu, A., Jardak, M., et al.: Information theoretic metrics to characterize observations in variational data assimilation. Proc. Int. Conf. Comp. Sci. 9(1), 1047–1055 (2012)

    Google Scholar 

  18. LeDimet, F.X., Ngodock, H.E., et al.: Sensitivity analysis in variational data assimilation. J. Meteorol. Soc. Japan 75(1), 245–255 (1997)

    Google Scholar 

  19. Daescu, D.N.: On the sensitivity equations of four-dimensional variational (4D-Var) data assimilation. Mon. Weather Rev. 136(8), 3050–3065 (2008)

    Article  Google Scholar 

  20. Daescu, D.N., Todling, R.: Adjoint sensitivity of the model forecast to data assimilation system error covariance parameters. Q. J. R. Meteorol. Soc. 136(653), 2000–2012 (2010)

    Article  Google Scholar 

  21. Daley, R.: Atmospheric Data Analysis. Cambridge University Press, Cambridge (1991)

    Google Scholar 

  22. Kalnay, E.: Atmospheric Modeling, Data Assimilation and Predictability. Cambridge University Press, Cambridge (2002)

    Book  Google Scholar 

  23. Cacuci, D.G.: Sensitivity theory for nonlinear systems. I. nonlinear functional analysis approach. J. Math. Phys. 22(12), 2794–2802 (1981)

    Article  Google Scholar 

  24. Sandu, A., Daescu, D.N., Carmichael, G.R., Chai, T.: Adjoint sensitivity analysis of regional air quality models. J. Comput. Phys. 204(1), 222–252 (2005)

    Article  Google Scholar 

  25. Cioaca, A., Sandu, A., Alexe, M.: Second-order adjoints for solving PDE-constrained optimization problems. Optim. Meth. Softw. 27(4–5), 625–653 (2011)

    Google Scholar 

  26. Gejadze, I.: On analysis error covariances in variational data assimilation. SIAM J. Sci. Comput 4(30), 1847–1874 (2008)

    Article  Google Scholar 

  27. Gejadze, I., LeDimet, F.X., Shutyaev, V.: On error covariances in variational data assimilation. Russ. J. Numer. Anal. Math. Model. 22(2), 163–175 (2008)

    Google Scholar 

  28. Liska, R., Wendroff, B.: Composite schemes for conservation laws. SIAM J. Numer. Anal. 35(6), 2250–2271 (1998)

    Article  Google Scholar 

  29. Wang, Z., Navon, I.M., LeDimet, F.X., Zou, X.: The second order adjoint analysis: Theory and applications. Met. Atm. Phys. 50(1–3), 3–20 (1992)

    Article  Google Scholar 

  30. Sandu, A., Zhang, L.: Discrete second order adjoints in atmospheric chemical transport modeling. J. Comput. Phys. 227(12), 5949–5983 (2008)

    Article  Google Scholar 

  31. Griewank, A.: On automatic differentiation. Mathematical programming: Recent developments and applications 6, 83–107 (1989)

    Google Scholar 

  32. Giering, R.: Tangent linear and adjoint model compiler, user manual, TAMC version 4

  33. Giering, R., Kaminski, T.: Recipes for adjoint code construction. ACM Trans. Math. Softw. 24(4), 437–474 (1998)

    Article  Google Scholar 

  34. Zhu, C., Gelaro, R.: Observation sensitivity calculations using the adjoint of the Gridpoint Statistical Interpolation (GSI) analysis system. Mon. Weather Rev. 136(1), 335–351 (2008)

    Article  Google Scholar 

  35. Courtier, P., Thepaut, J.N., Hollingsworth, A.: A strategy for operational implementation of 4D-Var using an incremental approach. Q. J. R. Meteorol. Soc. 120(519), 1367–1387 (1994)

    Article  Google Scholar 

  36. Tremolet, Y.: Incremental 4D-Var convergence study. Tellus A 59(5), 706–718 (2007)

    Article  Google Scholar 

  37. Ng, M.: Iterative Methods for Toeplitz Systems. Oxford University Press (2004)

  38. Vorst, H.A.: Iterative Krylov Methods for Large Linear Systems. Cambridge University Press, Cambridge (2003)

    Book  Google Scholar 

  39. Saad, Y.: Iterative Methods for Sparse Linear Systems. PWS-Kent, Boston (1996)

    Google Scholar 

  40. Balay, S., Brown, J., Buschelman, K., Gropp, W.D., Kaushik, D., Knepley, M.G., McInnes, L.C., Smith, B.F., Zhang, H.: PETSc web page (2012). http://www.mcs.anl.gov/petsc

  41. Forsythe, G.E., Strauss, E.G.: On best conditioned matrices. Proc. Amer. Math. Soc. 6(3), 340–345 (1955)

    Article  Google Scholar 

  42. Zupanski, M.: A preconditioning algorithm for large-scale minimization problems. Tellus A 45(5), 478–492 (1993)

    Article  Google Scholar 

  43. Yang, W., Navon, I.M., Courtier, F.: A new Hessian preconditioning method applied to variational data assimilation experiments using NASA general circulation models. Mon. Weather. Rev. 124(5), 1000–1017 (1996)

    Article  Google Scholar 

  44. Martinsson, P.G., Liberty, E., Woolfe, F., Rokhlin, V., Tygert, M.: Randomized algorithms for the low-rank approximation of matrices. Proc. Natl. Acad. Sci. USA 104(51), 20167–20172 (2007)

    Article  Google Scholar 

  45. Fedorenko, R.P.: A relaxation scheme for the solution of elliptic differential equations. UdSSR Comput. Math. Phys. 1(5), 1092–1096 (1961)

    Google Scholar 

  46. Hackbusch, W.: Multigrid Methods and Applications. Springer, Berlin (2003)

    Google Scholar 

  47. Skamarock, W.C., Klemp, J.B., Dudhia, J., Gill, D.D., et al. Co-authors: A description of the advanced research wrf version 3, Tech. Rep (2008)

  48. Xiao, Q., Ma, Z., Huang, W., Huang, X.Y., Barker, D., Kuo, Y.H., Michalakes, J.J.: Development of the WRF tangent linear and adjoint models: Nonlinear and linear evolution of initial perturbations and adjoint sensitivity analysis at high southern latitudes. In: WRF/MM5 Users Workshop, Boulder, Colorado, pp. 27–29 (2005)

  49. Schwartz, C.S., Liu, Z., Chen, Y., Huang, X.Y.: Impact of assimilating microwave radiances with a limited-area ensemble data assimilation system on forecasts of Typhoon Morakot. Weather Forecast. 27(2), 424–437 (2012)

    Article  Google Scholar 

  50. Cioaca, A., Zavala, V., Constantinescu, E.: Adjoint sensitivity analysis for numerical weather prediction: Applications to power grid optimization. In: Proceedings of the First International Workshop on High-Performance Computing, Networking and Analytics for the Power Grid, HiPCNA-PG ’11, ACM, New York, pp. 35–42 (2011). doi:10.1145/2096123.2096133

  51. Tshimanga, J., Gratton, S., Weaver, A.T., Sartenaer, A.: Limited-memory preconditioners, with application to incremental four-dimensional variational data assimilation. Q. J. R. Meteorol. Soc. 134(632), 751–769 (2008)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Computational Science Laboratory, Departmentof Computer Science, Virginia Tech, Blacksburg, USA

    Alexandru Cioaca, Adrian Sandu & Eric de Sturler

Authors
  1. Alexandru Cioaca
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Adrian Sandu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Eric de Sturler
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Alexandru Cioaca.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Cioaca, A., Sandu, A. & de Sturler, E. Efficient methods for computing observation impact in 4D-Var data assimilation. Comput Geosci 17, 975–990 (2013). https://doi.org/10.1007/s10596-013-9370-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10596-013-9370-2

Keywords

Search

Navigation