Abstract
We propose a method for inference on moderately high-dimensional, nonlinear, non-Gaussian, partially observed Markov process models for which the transition density is not analytically tractable. Markov processes with intractable transition densities arise in models defined implicitly by simulation algorithms. Widely used particle filter methods are applicable to nonlinear, non-Gaussian models but suffer from the curse of dimensionality. Improved scalability is provided by ensemble Kalman filter methods, but these are inappropriate for highly nonlinear and non-Gaussian models. We propose a particle filter method having improved practical and theoretical scalability with respect to the model dimension. This method is applicable to implicitly defined models having analytically intractable transition densities. Our method is developed based on the assumption that the latent process is defined in continuous time and that a simulator of this latent process is available. In this method, particles are propagated at intermediate time intervals between observations and are resampled based on a forecast likelihood of future observations. We combine this particle filter with parameter estimation methodology to enable likelihood-based inference for highly nonlinear spatiotemporal systems. We demonstrate our methodology on a stochastic Lorenz 96 model and a model for the population dynamics of infectious diseases in a network of linked regions.
Similar content being viewed by others
References
Acevedo, W., de Wiljes, J., Reich, S.: Second-order accurate ensemble transform particle filters. SIAM J. Sci. Comput. 39, A1834–A1850 (2017)
Ades, M., Van Leeuwen, P.J.: The equivalent-weights particle filter in a high-dimensional system. Q. J. R. Meteorol. Soc. 141, 484–503 (2015)
Andrieu, C., Doucet, A., Holenstein, R.: Particle Markov chain Monte Carlo methods. J. R. Stat. Soc. Ser. B (Stat. Methodol.) 72, 269–342 (2010)
Bakker, K.M., Martinez-Bakker, M.E., Helm, B., Stevenson, T.J.: Digital epidemiology reveals global childhood disease seasonality and the effects of immunization. Proc. Natl. Acad. Sci. 113, 6689–6694 (2016)
Becker, A.D., Birger, R.B., Teillant, A., Gastanaduy, P.A., Wallace, G.S., Grenfell, B.T.: Estimating enhanced prevaccination measles transmission hotspots in the context of cross-scale dynamics. Proc. Natl. Acad. Sci. 113, 14595–14600 (2016)
Bengtsson, T., Bickel, P., Li, B.: Curse-of-dimensionality revisited: collapse of the particle filter in very large scale systems. In: Probability and Statistics: Essays in Honor of David A. Freedman, pp. 316–334. Institute of Mathematical Statistics (2008)
Beskos, A., Crisan, D.O., Jasra, A., Whiteley, N.: Error bounds and normalising constants for sequential Monte Carlo samplers in high dimensions. Adv. Appl. Probab. 46, 279–306 (2014a)
Beskos, A., Crisan, D., Jasra, A.: On the stability of sequential Monte Carlo methods in high dimensions. Ann. Appl. Probab. 24, 1396–1445 (2014b)
Beskos, A., Crisan, D., Jasra, A., Kamatani, K., Zhou, Y.: A stable particle filter for a class of high-dimensional state-space models. Adv. Appl. Probab. 49, 24–48 (2017)
Bickel, P.J., Doksum, K.A.: Mathematical Statistics: Basic Ideas and Selected Topics, vol. 117. CRC Press, Boca Raton (2015)
Billingsley, P.: Convergence of Probability Measures. Wiley Series in Probability and Statistics. Wiley, New York (1999)
Bjørnstad, O.N., Grenfell, B.T.: Noisy clockwork: time series analysis of population fluctuations in animals. Science 293, 638–643 (2001)
Blackwood, J.C., Streicker, D.G., Altizer, S., Rohani, P.: Resolving the roles of immunity, pathogenesis, and immigration for rabies persistence in vampire bats. Proc. Natl. Acad. Sci. 110, 20837–20842 (2013)
Blake, I.M., Martin, R., Goel, A., Khetsuriani, N., Everts, J., Wolff, C., Wassilak, S., Aylward, R.B., Grassly, N.C.: The role of older children and adults in wild poliovirus transmission. Proc. Natl. Acad. Sci. 111, 10604–10609 (2014)
Bloem-Reddy, B., Orbanz, P.: Random-walk models of network formation and sequential Monte Carlo methods for graphs. J. R. Stat. Soc. Ser. B (Stat. Methodol.) 80, 871–898 (2018)
Bretó, C., He, D., Ionides, E.L., King, A.A.: Time series analysis via mechanistic models. Ann. Appl. Stat. 3, 319–348 (2009)
Bunch, P., Godsill, S.: Approximations of the optimal importance density using Gaussian particle flow importance sampling. J. Am. Stat. Assoc. 111, 748–762 (2016)
Cappé, O., Godsill, S.J., Moulines, E.: An overview of existing methods and recent advances in sequential Monte Carlo. Proc. IEEE 95, 899–924 (2007)
Chen, R., Wang, X., Liu, J.S.: Adaptive joint detection and decoding in flat-fading channels via mixture Kalman filtering. IEEE Trans. Inf. Theory 46, 2079–2094 (2000)
Cheng, Y., Reich, S.: Assimilating data into scientific models: an optimal coupling perspective. In: Van Leeuwen, P.J., Cheng, Y., Reich, S. (eds.) Nonlinear Data Assimilation, Frontiers in Applied Dynamical Systems: Reviews and Tutorials, vol. 2, pp. 75–118. Springer, Cham (2015)
Chopin, N.: Central limit theorem for sequential Monte Carlo methods and its application to Bayesian inference. Ann. Stat. 32, 2385–2411 (2004)
Chopin, N., Jacob, P.E., Papaspiliopoulos, O.: SMC\({}^2\): an efficient algorithm for sequential analysis of state space models. J. R. Stat. Soc. Ser. B (Stat. Methodol.) 75, 397–426 (2013)
Chorin, A.J., Tu, X.: Implicit sampling for particle filters. Proc. Natl. Acad. Sci. 106, 17249–17254 (2009)
Chorin, A.J., Morzfeld, M., Tu, X.: A survey of implicit particle filters for data assimilation. In: Zeng, Y., Wu, S. (eds.) State-Space Models, pp. 63–88. Springer, Berlin (2013)
Clapp, T., Godsill, S.: Fixed-lag smoothing using sequential importance sampling. In: Bayesian statistics 6: Proceeding of the Sixth Valencia International Meeting, vol. 6, pp. 743–752 (1999)
Cleveland, W.S., Grosse, E., Shyu, W.M.: Local regression models. In: Chambers, J., Hastie, T. (eds.) Statistical Models in S, pp. 309–376. Chapman and Hall, London (1992)
Dalziel, B.D., Bjørnstad, O.N., van Panhuis, W.G., Burke, D.S., Metcalf, C.J.E., Grenfell, B.T.: Persistent chaos of measles epidemics in the prevaccination United States caused by a small change in seasonal transmission patterns. PLoS Comput. Biol. 12, e1004655 (2016)
Del Moral, P.: Feynman-Kac Formulae: Genealogical and Interacting Particle Systems with Applications. Springer, New York (2004)
Del Moral, P., Guionnet, A.: On the stability of interacting processes with applications to filtering and genetic algorithms. Ann. l’Inst. Henri Poincaré (B) Probab. Stat. 37, 155–194 (2001)
Del Moral, P., Jacod, J.: Interacting particle filtering with discrete observations. In: Doucet, A., de Freitas, N., Gordon, N. (eds.) Sequential Monte Carlo Methods in Practice, pp. 43–75. Springer, Berlin (2001)
Del Moral, P., Murray, L.M.: Sequential Monte Carlo with highly informative observations. SIAM/ASA J. Uncertain. Quantif. 3, 969–997 (2015)
Diggle, P.J., Gratton, R.J.: Monte Carlo methods of inference for implicit statistical models. J. R. Stat. Soc. Ser. B (Methodol.) 46, 193–212 (1984)
Douc, R., Cappé, O., Moulines, E.: Comparison of resampling schemes for particle filtering. In: Proceedings of the 4th International Symposium on Image and Signal Processing and Analysis, 2005, pp. 64–69. IEEE (2005)
Doucet, A., Johansen, A.M.: A tutorial on particle filtering and smoothing: Fifteen years later. In: Crisan, D., Rozovskii, B. (eds.) Oxford Handbook of Nonlinear Filtering. Oxford University Press, Oxford (2011)
Doucet, A., Godsill, S., Andrieu, C.: On sequential Monte Carlo sampling methods for Bayesian filtering. Stat. Comput. 10, 197–208 (2000)
Doucet, A., De Freitas, N., Gordon, N.: Sequential Monte Carlo Methods in Practice. Springer, Berlin (2001)
Doucet, A., Briers, M., Sénécal, S.: Efficient block sampling strategies for sequential Monte Carlo methods. J. Comput. Graph. Stat. 15, 693–711 (2006)
Doucet, A., Pitt, M., Deligiannidis, G., Kohn, R.: Efficient implementation of Markov chain Monte Carlo when using an unbiased likelihood estimator. Biometrika 102, 295–313 (2015)
Eggo, R.M., Cauchemez, S., Ferguson, N.M.: Spatial dynamics of the 1918 influenza pandemic in England, Wales and the United States. J. R. Soc. Interface 8, 233–243 (2010)
Evensen, G.: Sequential data assimilation with a nonlinear quasi-geostrophic model using Monte Carlo methods to forecast error statistics. J. Geophys. Res. Oceans 99, 10143–10162 (1994)
Farchi, A., Bocquet, M.: Comparison of local particle filters and new implementations. Nonlinear Process. Geophys. 25, 765–807 (2018)
Fasiolo, M., Pya, N., Wood, S.N.: A comparison of inferential methods for highly nonlinear state space models in ecology and epidemiology. Stat. Sci. 31, 96–118 (2016)
Giraud, F., Del Moral, P.: Nonasymptotic analysis of adaptive and annealed Feynman-Kac particle models. Bernoulli 23, 670–709 (2017)
Gordon, N.J., Salmond, D.J., Smith, A.F.: Novel approach to nonlinear/non-Gaussian Bayesian state estimation. IEE Proc. F (Radar Signal Process.) 140, 107–113 (1993)
Guarniero, P., Johansen, A.M., Lee, A.: The iterated auxiliary particle filter. J. Am. Stat. Assoc. 112, 1636–1647 (2017)
He, D., Ionides, E.L., King, A.A.: Plug-and-play inference for disease dynamics: measles in large and small populations as a case study. J. R. Soc. Interface 7, 271–283 (2009)
Houtekamer, P.L., Mitchell, H.L.: A sequential ensemble Kalman filter for atmospheric data assimilation. Mon. Weather Rev. 129, 123–137 (2001)
Hunt, B.R., Kostelich, E.J., Szunyogh, I.: Efficient data assimilation for spatiotemporal chaos: a local ensemble transform Kalman filter. Phys. D Nonlinear Phenomena 230, 112–126 (2007)
Ionides, E.L., Nguyen, D., Atchadé, Y., Stoev, S., King, A.A.: Inference for dynamic and latent variable models via iterated, perturbed Bayes maps. Proc. Natl. Acad. Sci. 112, 719–724 (2015)
Ionides, E.L., Breto, C., Park, J., Smith, R.A., King, A.A.: Monte Carlo profile confidence intervals for dynamic systems. J. R. Soc. Interface 14, 20170126 (2017)
Johansen, A.M.: On blocks, tempering and particle MCMC for systems identification. In: Proceedings of 17th IFAC Symposium on System Identification, pp. 969–974 (2015)
Kevrekidis, I.G., Gear, C.W., Hummer, G.: Equation-free: the computer-aided analysis of complex multiscale systems. AIChE J. 50, 1346–1355 (2004)
King, A.A., Nguyen, D., Ionides, E.L.: Statistical inference for partially observed Markov processes via the R package pomp. J. Stat. Softw. 69, 1–43 (2016)
King, A.A., Ionides, E.L., Bretó, C.M., Ellner, S.P., Ferrari, M.J., Kendall, B.E., Lavine, M., Nguyen, D., Reuman, D.C., Wearing, H., Wood, S.N.: pomp: Statistical inference for partially observed Markov processes. R package, version 2.4 (2019). https://kingaa.github.io/pomp/. Accessed 10 Mar 2020
Kitano, H.: Computational systems biology. Nature 420, 206 (2002)
Kong, A., Liu, J.S., Wong, W.H.: Sequential imputations and Bayesian missing data problems. J. Am. Stat. Assoc. 89, 278–288 (1994)
Le Gland, F., Oudjane, N.: Stability and uniform approximation of nonlinear filters using the Hilbert metric and application to particle filters. Ann. Appl. Probab. 14, 144–187 (2004)
Lei, J., Bickel, P., Snyder, C.: Comparison of ensemble Kalman filters under non-Gaussianity. Mon. Weather Rev. 138, 1293–1306 (2010)
Lin, M., Chen, R., Mykland, P.: On generating Monte Carlo samples of continuous diffusion bridges. J. Am. Stat. Assoc. 105, 820–838 (2010)
Lin, M., Chen, R., Liu, J.S.: Lookahead strategies for sequential Monte Carlo. Stat. Sci. 28, 69–94 (2013)
Liu, J.S., Chen, R.: Blind deconvolution via sequential imputations. J. Am. Stat. Assoc. 90, 567–576 (1995)
Lorenz, E.N.: Predictability: a problem partly solved. Proc. Seminar Predict. 1, 1–18 (1996)
Marjoram, P., Molitor, J., Plagnol, V., Tavaré, S.: Markov chain Monte Carlo without likelihoods. Proc. Natl. Acad. Sci. 100, 15324–15328 (2003)
Miller, R.N., Carter, E.F., Blue, S.T.: Data assimilation into nonlinear stochastic models. Tellus A Dyn. Meteorol. Oceanogr. 51, 167–194 (1999)
Morzfeld, M., Tu, X., Atkins, E., Chorin, A.J.: A random map implementation of implicit filters. J. Comput. Phys. 231, 2049–2066 (2012)
Neal, R.M.: Annealed importance sampling. Stat. Comput. 11, 125–139 (2001)
Owen, J., Wilkinson, D.J., Gillespie, C.S.: Scalable inference for Markov processes with intractable likelihoods. Stat. Comput. 25, 145–156 (2015)
Palmer, T.N.: Towards the probabilistic Earth-system simulator: a vision for the future of climate and weather prediction. Q. J. R. Meteorol. Soc. 138, 841–861 (2012)
Papadakis, N., Mémin, É., Cuzol, A., Gengembre, N.: Data assimilation with the weighted ensemble Kalman filter. Tellus A Dyn. Meteorol. Oceanogr. 62, 673–697 (2010)
Pitt, M.K., Shephard, N.: Filtering via simulation: auxiliary particle filters. J. Am. Stat. Assoc. 94, 590–599 (1999)
Pons-Salort, M., Grassly, N.C.: Serotype-specific immunity explains the incidence of diseases caused by human enteroviruses. Science 361, 800–803 (2018)
Ranjeva, S.L., Baskerville, E.B., Dukic, V., Villa, L.L., Lazcano-Ponce, E., Giuliano, A.R., Dwyer, G., Cobey, S.: Recurring infection with ecologically distinct HPV types can explain high prevalence and diversity. Proc. Natl. Acad. Sci. 114, 13573–13578 (2017)
Rebeschini, P., Van Handel, R.: Can local particle filters beat the curse of dimensionality? Ann. Appl. Probab. 25, 2809–2866 (2015)
Reich, S.: A nonparametric ensemble transform method for Bayesian inference. SIAM J. Sci. Comput. 35, A2013–A2024 (2013)
Sisson, S.A., Fan, Y., Tanaka, M.M.: Sequential Monte Carlo without likelihoods. Proc. Natl. Acad. Sci. 104, 1760–1765 (2007)
Snyder, C., Bengtsson, T., Bickel, P., Anderson, J.: Obstacles to high-dimensional particle filtering. Mon. Weather Rev. 136, 4629–4640 (2008)
Snyder, C., Bengtsson, T., Morzfeld, M.: Performance bounds for particle filters using the optimal proposal. Mon. Weather Rev. 143, 4750–4761 (2015)
Van Leeuwen, P.J.: Nonlinear data assimilation in geosciences: an extremely efficient particle filter. Q. J. R. Meteorol. Soc. 136, 1991–1999 (2010)
Vergé, C., Dubarry, C., Del Moral, P., Moulines, E.: On parallel implementation of sequential Monte Carlo methods: the island particle model. Stat. Comput. 25, 243–260 (2015)
Whiteley, N.: Stability properties of some particle filters. Ann. Appl. Probab. 23, 2500–2537 (2013)
Whiteley, N., Lee, A.: Twisted particle filters. Ann. Stat. 42, 115–141 (2014)
Wilks, D.S.: Effects of stochastic parametrizations in the Lorenz 96 system. Q. J. R. Meteorol. Soc. 131, 389–407 (2005)
Xia, Y., Bjørnstad, O.N., Grenfell, B.T.: Measles metapopulation dynamics: a gravity model for epidemiological coupling and dynamics. Am. Nat. 164, 267–281 (2004)
Xiu, D., Kevrekidis, I.G., Ghanem, R.: An equation-free, multiscale approach to uncertainty quantification. Comput. Sci. Eng. 7, 16 (2005)
Acknowledgements
The authors thank Aaron King for the discussions motivating this research and for insightful feedback. Comments on the manuscript by Kidus Asfaw, Yves Atchadé, and two anonymous referees have led to improvements. This work was supported by National Science Foundation Grants DMS-1308919, DMS-1761603, and DMS-1513040, and National Institutes of Health Grants 1-U54-GM111274 and 1-U01-GM110712.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
About this article
Cite this article
Park, J., Ionides, E.L. Inference on high-dimensional implicit dynamic models using a guided intermediate resampling filter. Stat Comput 30, 1497–1522 (2020). https://doi.org/10.1007/s11222-020-09957-3
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11222-020-09957-3