Skip to main content
SpringerLink
Log in

Finite-State Markov Chains

  • Chapter
  • First Online:
Numerical Analysis for Statisticians

Part of the book series: Statistics and Computing ((SCO))

Abstract

Applied probability and statistics thrive on models. Markov chains are one of the richest sources of good models for capturing dynamical behavior with a large stochastic component [2, 3, 7, 9, 13, 18, 19, 21]. Certainly, every research statistician should be comfortable formulating and manipulating Markov chains. In this chapter we give a quick overview of some of the relevant theory of Markov chains in the simple context of finite-state chains. We cover both discrete-time and continuous-time chains in what we hope is a lively blend of applied probability, graph theory, linear algebra, and differential equations. Since this may be a first account for many readers, we stress intuitive explanations and computational techniques rather than mathematical rigor.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 89.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 119.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Baum LE (1972) An inequality and associated maximization technique in statistical estimation for probabilistic functions of Markov processes. Inequalities 3:1-8

    Google Scholar 

  2. Bhattacharya RN, Waymire EC (1990) Stochastic Processes with Applications. Wiley, New York

    Google Scholar 

  3. Billingsley P (1986) Probability and Measure, 2nd ed. Wiley, New York

    MATH  Google Scholar 

  4. Cao Y, Gillespie DT, Petzold LR (2006) Efficient leap-size selection for accelerated stochastic simulation. J Phys Chem 124:1-11

    Google Scholar 

  5. Devijver PA (1985) Baum’s forward-backward algorithm revisited. Pattern Recognition Letters 3:369-373

    Article  MATH  Google Scholar 

  6. Diaconis P (1988) Group Representations in Probability and Statistics. Institute of Mathematical Statistics, Hayward, CA

    MATH  Google Scholar 

  7. Doyle PG, Snell JL (1984) Random Walks and Electrical Networks. The Mathematical Association of America

    Google Scholar 

  8. Durbin R, Eddy S, Krogh A, Mitchison G (1998) Biological Sequence Analysis: Probabilistic Models of Proteins and Nucleic Acids. Cambridge University Press, Cambridge

    MATH  Google Scholar 

  9. Feller W (1968) An Introduction to Probability Theory and Its Applications, Volume 1, 3rd ed. Wiley, New York

    Google Scholar 

  10. Fredkin DR, Rice JA (1992) Bayesian restoration of single-channel patch clamp recordings. Biometrics 48:427-448

    Article  Google Scholar 

  11. Gillespie DT (1977) Exact stochastic simulation of coupled chemical reactions. J Phys Chem 81:2340-2361

    Article  Google Scholar 

  12. Gillespie DT (2001) Approximate accelerated stochastic simulation of chemically reacting systems. J Chem Phys 115:1716-1733

    Article  Google Scholar 

  13. Grimmett GR, Stirzaker DR (1992) Probability and Random Processes, 2nd ed. Oxford University Press, Oxford

    Google Scholar 

  14. Hastings WK (1970) Monte Carlo sampling methods using Markov chains and their applications. Biometrika 57:97-109

    Article  MATH  Google Scholar 

  15. Higham DJ (2008) Modeling and simulating chemical reactions. SIAM Review 50:347-368

    Article  MATH  MathSciNet  Google Scholar 

  16. Higham NJ (2009) The scaling and squaring method for matrix exponentiation. SIAM Review 51:747-764

    Article  MATH  MathSciNet  Google Scholar 

  17. Hirsch MW, Smale S (1974) Differential Equations, Dynamical Systems, and Linear Algebra. Academic Press, New York

    MATH  Google Scholar 

  18. Karlin S, Taylor HM (1975) A First Course in Stochastic Processes, 2nd ed. Academic Press, New York

    MATH  Google Scholar 

  19. Karlin S, Taylor HM (1981) A Second Course in Stochastic Processes. Academic Press, New York

    MATH  Google Scholar 

  20. Kelly FP (1979) Reversibility and Stochastic Networks. Wiley, New York

    MATH  Google Scholar 

  21. Lamperti J (1977) Stochastic Processes: A Survey of the Mathematical Theory. Springer, New York

    MATH  Google Scholar 

  22. Lange K (2002) Mathematical and Statistical Methods for Genetic Analysis, 2nd ed. Springer, New York

    MATH  Google Scholar 

  23. Lange K, Boehnke M, Cox DR, Lunetta KL (1995) Statistical methods for polyploid radiation hybrid mapping. Genome Res 5:136-150

    Article  Google Scholar 

  24. Lazzeroni LC, Lange K (1997) Markov chains for Monte Carlo tests of genetic equilibrium in multidimensional contingency tables. Ann Stat 25:138-168

    Article  MATH  MathSciNet  Google Scholar 

  25. Moler C, Van Loan C (1978) Nineteen dubious ways to compute the exponential of a matrix. SIAM Review 20:801-836

    Article  MATH  MathSciNet  Google Scholar 

  26. Nachbin L (1965) The Haar Integral. Van Nostrand, Princeton, NJ

    MATH  Google Scholar 

  27. Rabiner L (1989) A tutorial on hidden Markov models and selected applications in speech recognition. Proc IEEE 77:257-285

    Article  Google Scholar 

  28. Rubinow SI (1975) Introduction to Mathematical Biology. Wiley, New York

    MATH  Google Scholar 

  29. Sehl ME, Alexseyenko AV, Lange KL (2009) Accurate stochastic simulation via the step anticipation (SAL) algorithm. J Comp Biol 16:1195-1208

    Article  Google Scholar 

  30. Stewart WJ (1994) Introduction to the Numerical Solution of Markov Chains. Princeton University Press, Princeton, NJ

    MATH  Google Scholar 

  31. Waterman MS (1995) Introduction to Computational Biology: Maps, Sequences, and Genomes. Chapman & Hall, London

    MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Departments of Biomathematics, Human Genetics, and Statistics David Geffen School of Medicine, University of California, Los Angeles, Le Conte Ave. 10833, Los Angeles, CA, 90095-1766, USA

    Kenneth Lange

Authors
  1. Kenneth Lange
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Kenneth Lange .

Rights and permissions

Reprints and permissions

Copyright information

© 2010 Springer New York

About this chapter

Cite this chapter

Lange, K. (2010). Finite-State Markov Chains. In: Numerical Analysis for Statisticians. Statistics and Computing. Springer, New York, NY. https://doi.org/10.1007/978-1-4419-5945-4_25

Download citation

Publish with us

Policies and ethics

Search

Navigation