Abstract
Standard methods for maximum likelihood parameter estimation in latent variable models rely on the Expectation-Maximization algorithm and its Monte Carlo variants. Our approach is different and motivated by similar considerations to simulated annealing; that is we build a sequence of artificial distributions whose support concentrates itself on the set of maximum likelihood estimates. We sample from these distributions using a sequential Monte Carlo approach. We demonstrate state-of-the-art performance for several applications of the proposed approach.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Amzal, B., Bois, F.Y., Parent, E., Robert, C.P.: Bayesian optimal design via interacting particle systems. J. Am. Stat. Assoc. 101(474), 773–785 (2006)
Chopin, N.: Central limit theorem for sequential Monte Carlo methods and its applications to Bayesian inference. Ann. Stat. 32(6), 2385–2411 (2004)
Del Moral, P.: Feynman-Kac Formulae: Genealogical and Interacting Particle Systems with Applications. Probability and its Applications. Springer, New York (2004)
Del Moral, P., Doucet, A., Jasra, A.: Sequential Monte Carlo samplers. J. R. Stat. Soc. B 63(3), 411–436 (2006)
Dempster, A.P., Laird, N.M., Rubin, D.B.: Maximum likelihood from incomplete data via the EM Algorithm. J. R. Stat. Soc. B 39, 2–38 (1977)
Doucet, A., de Freitas, N., Gordon, N. (eds.): Sequential Monte Carlo Methods in Practice. Statistics for Engineering and Information Science. Springer, New York (2001)
Doucet, A., Godsill, S.J., Robert, C.P.: Marginal maximum a posteriori estimation using Markov chain Monte Carlo. Stat. Comput. 12, 77–84 (2002)
Doucet, A., Briers, M., Sénécal, S.: Efficient block sampling strategies for sequential Monte Carlo methods. J. Comput. Graph. Stat. 15(3), 693–711 (2006)
Escobar, M.D., West, M.: Bayesian density estimation and inference using mixtures. J. Am. Stat. Assoc. 90(430), 577–588 (1995)
Gaetan, C., Yao, J.-F.: A multiple-imputation Metropolis version of the EM algorithm. Biometrika 90(3), 643–654 (2003)
Hwang, C.-R.: Laplace’s method revisited: weak convergence of probability measures. Ann. Probab. 8(6), 1177–1182 (1980)
Jacquier, E., Johannes, M., Polson, N.: MCMC maximum likelihood for latent state models. J. Econom. 137(2), 615–640 (2007)
Johansen, A.M.: Some non-standard sequential Monte Carlo methods with applications, Ph.D. thesis. University of Cambridge, Department of Engineering (2006)
Liu, J.S., Chen, R.: Sequential Monte Carlo methods for dynamic systems. J. Am. Stat. Assoc. 93(443), 1032–1044 (1998)
Müller, P., Sansó, B., de Iorio, M.: Optimum Bayesian design by inhomogeneous Markov chain simulation. J. Am. Stat. Assoc. 99(467), 788–798 (2004)
Robert, C.P., Casella, G.: Monte Carlo Statistical Methods, 2nd edn. Springer, New York (2004)
Robert, C.P., Titterington, D.M.: Reparameterization strategies for hidden Markov models and Bayesian approaches to maximum likelihood estimation. Stat. Comput. 8, 145–158 (1998)
Roeder, K.: Density estimation with cofidence sets exemplified by superclusters and voids in galaxies. J. Am. Stat. Assoc. 85(411), 617–624 (1990)
Roeder, K., Wasserman, L.: Practical Bayesian density estimation using mixtures of normals. J. Am. Stat. Assoc. 92(439), 894–902 (1997)
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Johansen, A.M., Doucet, A. & Davy, M. Particle methods for maximum likelihood estimation in latent variable models. Stat Comput 18, 47–57 (2008). https://doi.org/10.1007/s11222-007-9037-8
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11222-007-9037-8