Countable Markov Chains

Abstract

In this chapter we consider a time-homogeneous MC = {fit} on acountablespaceX.For notational convenience, and without loss of generality, we assume thatX ={0, 1, 2,… with the discrete topology. In this caseBis the o-algebra of all the subsets ofX.The corresponding one-step t.p.f.Pis an infinite matrix {P(i, j)} where As in §2.2, the n-step t.p.f. is denoted byP“and it can be obtained recursively asP” = PP“ ^-1=Pn^-1Pfor all n = 1, 2 with Po =Ithe identity matrix.