Abstract
One of the central considerations in the theory of Markov chains is their convergence to an equilibrium. Coefficients of ergodicity provide an efficient method for such an analysis. Besides giving sufficient and sometimes necessary conditions for convergence, they additionally measure its rate. In this paper we explore coefficients of ergodicity for the case of imprecise Markov chains. The latter provide a convenient way of modelling dynamical systems where parameters are not determined precisely. In such cases a tool for measuring the rate of convergence is even more important than in the case of precisely determined Markov chains, since most of the existing methods of estimating the limit distributions are iterative. We define a new coefficient of ergodicity that provides necessary and sufficient conditions for convergence of the most commonly used class of imprecise Markov chains. This so-called weak coefficient of ergodicity is defined through an endowment of the structure of a metric space to the class of imprecise probabilities. Therefore we first make a detailed analysis of the metric properties of imprecise probabilities.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Beer G (1993) Topologies on closed and closed convex sets. Kluwer, Dordrecht
Birkhoff G (1957) Extensions of Jentzsch’s theorem. Trans Am Math Soc 85(1): 219–227
Crossman RJ, Škulj D (2010) Imprecise Markov chains with absorption. Int J Approx Reason 51: 1085–1099. doi:10.1016/j.ijar.2010.08.008
Crossman RJ, Coolen-Schrijner P, Coolen FPA (2009a) Time-homogeneous birth-death processes with probability intervals and absorbing state. J Stat Theory Practice 3(1): 103–118
Crossman RJ, Coolen-Schrijner P, Škulj D, Coolen FPA (2009b) Imprecise Markov chains with an absorbing state. In: Augustin T, Coolen FPA, Moral S, Troffaes MCM (eds) ISIPTA’09: proceedings of the sixth international symposium on imprecise probability: theories and applications, SIPTA, Durham, UK, pp 119–128
Darroch JN, Seneta E (1965) On quasi-stationary distributions in absorbing discrete-time finite Markov chains. J Appl Probab 2(1):88–100, http://www.jstor.org/stable/3211876
de Campos LD, Huete J, Moral S (1994) Probability intervals: a tool for uncertain reasoning. Int J Uncertain Fuzz Knowl Based Syst 2(2): 167–196
De Cooman G, Hermans F, Quaeghebeur E (2009) Imprecise Markov chains and their limit behavior. Probab Eng Inform Sci 23(4): 597–635. doi:10.1017/S0269964809990039
Dobrushin R (1956) Central limit theorem for non-stationary Markov chains, I, II. Theory Probab Appl 1(4): 329–383
Dunford N, Schwartz J (1988) Linear operators. Part I: general theory. Wiley, New York
Fan K (1953) Minimax theorems. Proc Natl Acad Sci USA 39: 42–47
Hable R (2009) Data-based decisions under complex uncertainty. PhD thesis, Ludwig-Maximilians-Universität (LMU) Munich, http://edoc.ub.uni-muenchen.de/9874/
Hable R (2010) Minimum distance estimation in imprecise probability models. J Stat Plan Inference 140: 461–479
Harmanec D (2002) Generalizing Markov decision processes to imprecise probabilities. J Stat Plan Inference 105: 199–213
Hartfiel D (1998) Markov set-chains. Springer, Berlin
Hartfiel D, Rothblum U (1998) Convergence of inhomogenous products of matrices and coefficients of ergodicity. Linear Algebra Appl 277: 1–9
Hartfiel D, Seneta E (1994) On the theory of Markov set-chains. Adv Appl Probab 26(4): 947–964
Holmes RB (1975) Geometric functional analysis and its applications. Springer, Berlin
Itoh H, Nakamura K (2007) Partially observable Markov decision processes with imprecise parameters. Artif Intell 171(8–9): 453–490
Kozine I, Utkin L (2002) Interval-valued finite Markov chains. Reliable Comput 8(2): 97–113
Nilim A, Ghaoui LE (2005) Robust control of Markov decision processes with uncertain transition matrices. Oper Res 53: 780–798
Paz A (1970) Ergodic theorems for infinite probabilistic tables. Ann Math Stat 41(2): 539–550
Satia J, Lave R (1973) Markovian decision processes with uncertain transition probabilities. Oper Res 21(3): 728–740
Seneta E (1979) Coefficients of ergodicity—structure and applications. Adv Appl Probab 11(2): 270–271
Seneta E (2006) Non-negative matrices and Markov chains. Springer, Berlin
Škulj D (2006) Finite discrete time Markov chains with interval probabilities. In: Lawry J, Miranda E, Bugarín A, Li S, Gil MA, Grzegorzewski P, Hryniewicz O (eds) SMPS. Advances in soft computing. Springer, Berlin, vol 37, pp 299–306
Škulj D (2007) Regular finite Markov chains with interval probabilities. In: De Cooman G, Zaffalon M, Vejnarová J (eds) ISIPTA’07—proceedings of the fifth international symposium on imprecise probability: theories and applications, SIPTA, pp 405–413
Škulj D (2009) Discrete time Markov chains with interval probabilities. Int J Approx Reason 50(8): 1314–1329. doi:10.1016/j.ijar.2009.06.007
Škulj D, Hable R (2009) Coefficients of ergodicity for imprecise Markov chains. In: Augustin T, Coolen FPA, Moral S, Troffaes MCM (eds) ISIPTA’09: proceedings of the sixth international symposium on imprecise probability: theories and applications, SIPTA, Durham, UK, pp 377–386
Walley P (1991) Statistical reasoning with imprecise probabilities. Chapman and Hall, London
Walley P (2000) Towards a unified theory of imprecise probability. Int J Approx Reason 24: 125–148
Weichselberger K (2001) Elementare Grundbegriffe einer allgemeineren Wahrscheinlichkeitsrechnung. I: Intervallwahrscheinlichkeit als umfassendes Konzept. Physica-Verlag, Heidelberg
White C, Eldeib H (1994) Markov decision processes with imprecise transition probabilities. Oper Res 42(4): 739–749
Author information
Authors and Affiliations
Corresponding author
Additional information
An earlier version of this paper, Škulj and Hable (2009), was presented at the ISIPTA’09 conference in Durham, UK, 2009.
Rights and permissions
About this article
Cite this article
Škulj, D., Hable, R. Coefficients of ergodicity for Markov chains with uncertain parameters. Metrika 76, 107–133 (2013). https://doi.org/10.1007/s00184-011-0378-0
Received:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00184-011-0378-0