Skip to main content
SpringerLink
Log in

Predicting shallow water dynamics using echo-state networks with transfer learning

  • Original Paper
  • Published:
GEM - International Journal on Geomathematics Aims and scope Submit manuscript

Abstract

In this paper we demonstrate that reservoir computing can be used to learn the dynamics of the shallow-water equations. In particular, while most previous applications of reservoir computing have required training on a particular trajectory to further predict the evolution along that trajectory alone, we show the capability of reservoir computing to predict trajectories of the shallow-water equations with initial conditions not seen in the training process. However, in this setting, we find that the performance of the network deteriorates for initial conditions with ambient conditions (such as total water height and average velocity) that are different from those in the training dataset. To circumvent this deficiency, we introduce a transfer learning approach wherein a small additional training step with the relevant ambient conditions is used to improve the predictions.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  • Arcomano, T., Szunyogh, I., Pathak, J., Wikner, A., Hunt, B.R., Ott, E.: A machine learning-based global atmospheric forecast model. Geophys. Res. Lett. 47(9), 2020–087776 (2020)

    Article  Google Scholar 

  • Audusse, E., Chalons, C., Ung, P.: A simple well-balanced and positive numerical scheme for the shallow-water system. Commun. Math. Sci. 13(5), 1317–1332 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  • Berg, J., Nyström, K.: Data-driven discovery of PDEs in complex datasets. J. Comput. Phys. 384, 239–252 (2019)

    Article  MathSciNet  MATH  Google Scholar 

  • Bollt, E.: On explaining the surprising success of reservoir computing forecaster of chaos? the universal machine learning dynamical system with contrast to VAR and DMD. Chaos Interdiscip. J. Nonlinear Sci. 31(1), 013108 (2021). https://doi.org/10.1063/5.0024890

    Article  MathSciNet  MATH  Google Scholar 

  • Buehner, M., Young, P.: A tighter bound for the echo state property. IEEE Trans. Neural Netw. 17(3), 820–824 (2006)

    Article  Google Scholar 

  • Caluwaerts, K., Wyffels, F., Dieleman, S., Schrauwen, B.: The spectral radius remains a valid indicator of the echo state property for large reservoirs. In: The 2013 International Joint Conference on Neural Networks (IJCNN), pp. 1–6 (2013). https://doi.org/10.1109/IJCNN.2013.6706899

  • Champion, K., Lusch, B., Kutz, J.N., Brunton, S.L.: Data-driven discovery of coordinates and governing equations. Proc. Natl. Acad. Sci. 116(45), 22445–22451 (2019)

    Article  MathSciNet  MATH  Google Scholar 

  • Chattopadhyay, A., Hassanzadeh, P., Subramanian, D.: Data-driven predictions of a multiscale Lorenz 96 chaotic system using machine-learning methods: reservoir computing, artificial neural network, and long short-term memory network. Nonlinear Process. Geophys. 27(3), 373–389 (2020)

    Article  Google Scholar 

  • Chen, X.: Effective data-driven models for chaotic and turbulent dynamics. Ph.D. thesis, University of Houston (2021)

  • DeGennaro, A.M., Urban, N.M., Nadiga, B.T., Haut, T.: Model structural inference using local dynamic operators. Int. J. Uncertain. Quantif. 9(1), 59–83 (2019)

    Article  MathSciNet  Google Scholar 

  • Gandhi, M., Jaeger, H.: Echo state property linked to an input: exploring a fundamental characteristic of recurrent neural networks. Neural Comput. 25, 671–696 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  • Golub, G.H., Hansen, P.C., O’Leary, D.P.: Tikhonov regularization and total least squares. SIAM J. Matrix Anal. Appl. 21(1), 185–194 (1999)

    Article  MathSciNet  MATH  Google Scholar 

  • Goutal, N.: Proceedings of the 2nd Workshop on Dam-break Wave Simulation. Department Laboratoire National d’Hydraulique, Groupe Hydraulique Fluviale (1997)

  • Inubushi, M., Goto, S.: Transferring reservoir computing: formulation and application to fluid physics. In: International Conference on Artificial Neural Networks, pp. 193–199. Springer (2019)

  • Inubushi, M., Goto, S.: Transfer learning for nonlinear dynamics and its application to fluid turbulence. Phys. Rev. E 102(4), 043301 (2020)

    Article  Google Scholar 

  • Jaeger, H.: The “echo state” approach to analysing and training recurrent neural networks-with an erratum note. German National Research Center for Information Technology GMD Technical Report, Bonn, Germany vol. 148, no. 34, p. 13 (2001)

  • Jaeger, H.: Echo state network. Scholarpedia 2(9), 2330 (2007)

    Article  Google Scholar 

  • Jaeger, H., Haas, H.: Harnessing nonlinearity: predicting chaotic systems and saving energy in wireless communication. Science 304(5667), 78–80 (2004)

    Article  Google Scholar 

  • Jaeger, H., Lukoševičius, M., Popovici, D., Siewert, U.: Optimization and applications of echo state networks with leaky-integrator neurons. Neural Netw. 20(3), 335–352 (2007)

    Article  MATH  Google Scholar 

  • Li, J., Sun, G., Zhao, G., Li-wei, H.L.: Robust low-rank discovery of data-driven partial differential equations. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, pp. 767–774 (2020)

  • Long, Z., Lu, Y., Ma, X., Dong, B.: PDE-Net: Learning PDEs from data. In: International Conference on Machine Learning, pp. 3208–3216. PMLR (2018)

  • Maass, W., Natschläger, T., Markram, H.: Real-time computing without stable states: A new framework for neural computation based on perturbations. Neural Comput. 14(11), 2531–2560 (2002)

    Article  MATH  Google Scholar 

  • Manjunath, G., Jaeger, H.: Echo state property linked to an input: exploring a fundamental characteristic of recurrent neural networks. Neural Comput. 25(3), 671–696 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  • Maslyaev, M., Hvatov, A., Kalyuzhnaya, A.: Data-driven partial differential equations discovery approach for the noised multi-dimensional data. In: International Conference on Computational Science, pp. 86–100. Springer (2020)

  • Nadiga, B.: Reservoir computing as a tool for climate predictability studies. J. Adv. Model. Earth Syst. 13, 2020–002290 (2021)

    Article  Google Scholar 

  • Pandey, S., Schumacher, J.: Reservoir computing model of two-dimensional turbulent convection. Phys. Rev. Fluids 5, 113506 (2020). https://doi.org/10.1103/PhysRevFluids.5.113506

    Article  Google Scholar 

  • Pathak, J., Hunt, B., Girvan, M., Lu, Z., Ott, E.: Model-free prediction of large spatiotemporally chaotic systems from data: a reservoir computing approach. Phys. Rev. Lett. 120(2), 024102 (2018)

    Article  Google Scholar 

  • Pyle, R., Jovanovic, N., Subramanian, D., Palem, K.V., Patel, A.B.: Domain-driven models yield better predictions at lower cost than reservoir computers in Lorenz systems. Phil. Trans. R. Soc. A 379(2194), 20200246 (2021)

    Article  MathSciNet  Google Scholar 

  • Reinbold, P.A., Grigoriev, R.O.: Data-driven discovery of partial differential equation models with latent variables. Phys. Rev. E 100(2), 022219 (2019)

    Article  MathSciNet  Google Scholar 

  • Rudy, S.H., Brunton, S.L., Proctor, J.L., Kutz, J.N.: Data-driven discovery of partial differential equations. Sci. Adv. 3(4), 1602614 (2017)

    Article  Google Scholar 

  • Vlachas, P.R., Pathak, J., Hunt, B.R., Sapsis, T.P., Girvan, M., Ott, E., Koumoutsakos, P.: Backpropagation algorithms and reservoir computing in recurrent neural networks for the forecasting of complex spatiotemporal dynamics. Neural Netw. 126, 191–217 (2020)

    Article  Google Scholar 

  • Xu, H., Chang, H., Zhang, D.: DL-PDE: deep-learning based data-driven discovery of partial differential equations from discrete and noisy data. arXiv preprint arXiv:1908.04463 (2019)

  • Yildiz, I.B., Jaeger, H., Kiebel, S.J.: Re-visiting the echo state property. Neural Netw. 35, 1–9 (2012). https://doi.org/10.1016/j.neunet.2012.07.005

    Article  MATH  Google Scholar 

  • Zhang, Z., Liu, Y.: Robust data-driven discovery of partial differential equations under uncertainties. arXiv preprint arXiv:2102.06504 (2021)

Download references

Acknowledgements

Research of I. Timofeyev and X. Chen has been partially supported by Grants NSF DMS-1620278 and ONR N00014-17-1-2845. Nadiga was supported under DOE’s SciDAC program under SciDAC4 Project “Non-Hydrostatic Dynamics with Multi-Moment Characteristic Discontinuous Galerkin Methods (NH-MMCDG).” The model code and data for this project are publicly available online at github in the repository Timofeyev/SWE_ESN and on zenodo under the title “Predicting Shallow Water Dynamics using Echo-State Networks with Transfer Learning” or DOI: https://doi.org/10.5281/zenodo.6828772.

Author information

Authors and Affiliations

  1. Department of Mathematics, University of Houston, Houston, TX, USA

    Xiaoqian Chen & Ilya Timofeyev

  2. Los Alamos National Laboratory, Los Alamos, NM, USA

    Balasubramanya T. Nadiga

Authors
  1. Xiaoqian Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Balasubramanya T. Nadiga
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ilya Timofeyev
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ilya Timofeyev.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chen, X., Nadiga, B.T. & Timofeyev, I. Predicting shallow water dynamics using echo-state networks with transfer learning. Int J Geomath 13, 20 (2022). https://doi.org/10.1007/s13137-022-00210-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s13137-022-00210-9

Keywords

Mathematics Subject Classification

Search

Navigation