Equilibrium states and invariant measures for random dynamical systems

Random dynamical systems with countably many maps which admit countable Markov partitions on complete metric spaces such that the resulting Markov systems are uniformly continuous and contractive are considered. A non-degeneracy and a consistency conditions for such systems, which admit some proper Markov partitions of connected spaces, are introduced, and further sufficient conditions for them are provided. It is shown that every uniformly continuous Markov system associated with a continuous random dynamical system is consistent if it has a dominating Markov chain. A necessary and sufficient condition for the existence of an invariant Borel probability measure for such a non-degenerate system with a dominating Markov chain and a finite (16) is given. The condition is also sufficient if the non-degeneracy is weakened with the consistency condition. A further sufficient condition for the existence of an invariant measure for such a consistent system which involves only the properties of the dominating Markov chain is provided. In particular, it implies that every such a consistent system with a finite Markov partition and a finite (16) has an invariant Borel probability measure. A bijective map between these measures and equilibrium states associated with such a system is established in the non-degenerate case. Some properties of the map and the measures are given.


Introduction
The purpose of this note is to show the existence of invariant measures for some random dynamical systems introduced in [25] as contractive Markov systems. To avoid some confusion, let us stress that the word 'Markov' in the name was used to indicate a Markovian topological structure of the random dynamical system, which naturally generalizes a weighted directed graph, but the dependence structure of random processes which can be generated by it on a code space, in contrast to directed graphs, can be far beyond Markovian. They certainly can generate any stationary process with values in a discrete state space. At the same time, the algorithm for generating a process by such a system is not much different to that for generating a process by a weighted directed graph. Such random dynamical systems find more and more applications in modern sciences, e.g. [20], [1], [7], and provide new challenging and illuminating examples for mathematical theories, e.g. [27], [28], [29]. However, in contrast to weighted directed graphs, the behaviour of contractive Markov systems is still not fully understood.
The existence of stationary states for such systems was shown on some locally compact spaces in [25]. This was proved under the condition that the partition of the Markov system consists of open sets. Though, this was sufficient to cover finite Markov chains and g-measures [11] with the theory, it clearly poses a severe restriction on the applicability of it. In particular, the removal of the condition admits the usage of Markov partitions for random dynamical systems, which reduces the latter to Markov systems, the behaviour of which is more transparent [31]. The proof which was given in [25] went along the lines of that which had been given by M. Barnsley et al. for iterated function systems with place-dependent probabilities [2] and [3]. The result then was extended by K. Horbacz and T. Szarek [8] on Polish spaces, through application of some results which had been obtained by the second author for Markov operators satisfying some non-expansiveness and concentration conditions on Polish spaces, using a lower bound technique [22]. Unfortunately, the condition of the openness of the partition has been left in place.
In this article, we close the gap by introducing a non-degeneracy and a consistency conditions and providing further sufficient conditions for them. These conditions admit some proper Markov partitions of connected spaces and allow us to prove the existence of invariant measures for the random dynamical systems which exhibit the continuity and the contraction on average properties only on the atoms of their Markov partitions. In particular, the consistency condition includes all uniformly continuous Markov systems which are associated with random dynamical systems with continuous maps and probability functions and have a dominating Markov chain (Theorem 4). Furthermore, every countable refinement of a uniformly continuous, positive and consistent Markov system with contractive maps is again consistent if it has a dominating Markov chain (Proposition 2).
The presented proof is self-contained, does not require any special knowledge and works for countable Markov systems. Moreover, it is shown that the separability of the space is not needed in this case. The existence of the invariant measures is deduced from the existence of equilibrium states or, in general, asymptotic states on the code space associated with such a system, via a coding map. This method is easier because the code space is either a compact metrizable space, as in the case with finitely many maps, or can be easily extended to such a space in the case of countably many maps. In particular, one can take advantage of the weak-star compactness of the set of all Borel probability measures on it.
The existence of the equilibrium states for energy functions associated with such systems has been already shown in [27], but it has been deduced from the existence of the invariant measures for such systems with the open partition on locally compact spaces. The main message of [27] was that the current thermodynamic formalism is not applicable to such systems because it fails even to predict the existence of equilibrium states for such energy functions, not to mention the construction of them.
Recently, a construction of such equilibrium states has been proposed in [28] and [29]. It seems to require the existence of an equilibrium state for such a system for the proof that the constructed measure is not zero.
An other related existence result is a recent proof in the particular case of gmeasures by A. Johansson et al. [10]. However, it does not intersect much with the present result as the g-functions associated with our systems are not continuous, even in the case of the openness of the Markov partition (see [27]) or the case of contractive maps on a compact metric space, but without the openness condition on the Markov partition (e.g. see Example 7 below).
The present result also establishes a bijection between the equilibrium states and the invariant Borel probability measures of such systems in the non-degenerate case. In particular, this generalizes a theorem by F. Ledrappier [12], Theorem 2.1 in [23].
The article is organized as follows. Section 1 collects all the necessary definitions and notations. Section 2 presents the main results. Finally, Section 3 provides, in particular, some simple examples to which the technique of Markov partitions can be applied. As far as the author is aware, some of the examples have not been accessible by the theory before.
It was pointed out by an anonymous reviewer that it might be appropriate to cite the works [15], [21], [16], [17], despite the fact that the authors of the works seem to indicate that their works are not related to contractive Markov systems, by not citing any previous works on contractive Markov systems. It is a remarkable coincidence that the 'graph directed constructions' [13] evolved into the 'graph directed Markov systems' [15] in literature, even on the costs of making the new name somewhat tautological, shortly after the author had introduced the 'contractive Markov systems' with contractive maps on compact metric spaces in his diploma thesis [24] (certainly, aware of [13], but mostly influenced by the work of J. Elton [6], see [31]). (As a matter of fact, the diploma thesis was then developed further to his Ph.D. thesis at the University of St Andrews in 2003, but for some reason the university allowed to defend the thesis only after 1 year of waiting, in November 2004.) Notable also is that the unusual direction of the arrows of the directed graphs in [13] evolved also into that of Markov systems. (Curiously, the anonymous reviewer pointed out that a term 'conformal graph directed Markov systems' had already appeared in [14], but apparently only in the first two sentences as a name for a future theory towards which the authors direct their efforts without giving a definition for such objects yet. In fact, the rest of the article seems to be completely detached from the information contained in these two sentences. However, the date of that publication and the date when the author was allowed to officially submit the diploma thesis make the coincidence even more remarkable.) The author leaves it to the reader to judge on how the structures studied in the cited works relate to contractive Markov systems and on their scientific motivation.
For some criteria for the uniqueness of the invariant probability measures, the existence of which is proved in this article, the reader is referred to [30].

Definitions and notation
Let B(X) denote Borel σ-algebra on a topological space X and P (X) denote the set of all Borel probability measures on it. Let L B (X) denote the set of all real-valued, non-negative, Borel measurable functions on X. For B ∈ B(X), let P (B) denote the set of all ν ∈ P (X) such that ν(B) = 1.
Let (K, d) be a complete metric space. A family D R := (K, w e , p e ) e∈E is called a random dynamical system on K iff E is at most countable, w e : K −→ K and p e : K −→ [0, 1] are Borel-measurable for all e ∈ E such that e∈E p e (x) = 1 for all x ∈ K. w e 's are called maps and p e 's are called probability functions.
With D R is associated a Markov operator U defined on L B (K) by for all f ∈ L B (K). Let U * denote its adjoint operator acting on ν ∈ P (K) by U * ν(f ) := U f dν for all bounded f ∈ L B (K). µ ∈ P (K) is called an invariant measure for the random dynamical system iff U * µ = µ. Observe that each w e needs to be defined only on the set {x ∈ K| p e (x) > 0} for the definitions of U and U * , it can be then extended on the whole space arbitrarily. A random dynamical system is called a Markov system if and only if it has the form (K i(e) , w e , p e ) e∈E ′ where E ′ is a set such that there exists a partition of K into non-empty Borel subsets (K j ) j∈N , N ⊂ N with 1 ∈ N (case where the size of N is 1 is not excluded), and a surjective map i : E ′ −→ N and t : E ′ −→ N such that for every e ∈ E ′ there exist Borel measurable w e : K i(e) −→ K t(e) and p e : K i(e) −→ [0, 1] such that there exists x e ∈ K i(e) with p e (x e ) > 0, and e∈E ′ ,i(e)=j p e (y) = 1 for all y ∈ K j and j ∈ N . K j 's are called the vertex sets of the Markov system. The Markov system is called countable iff N and E are at most countable. Clearly, a countable Markov system defines a random dynamical system on K by extending p e 's on K by zero and w e 's arbitrarily. Such extensions define the actions of the Markov system on functions and measures through operators U and U * and will be always assumed.
We say that a random dynamical system has a Markov partition iff there exists a partition of K into non-empty Borel subsets such that the restrictions of its maps and probability functions on the atoms of the partition (after a possible re-indexation) form a Markov system.
A Markov system (K i(e) , w e , p e ) e∈E is called contractive with a contraction rate 0 < a < 1 iff e∈E,i(e)=j p e (x)d(w e x, w e y) < ad(x, y) for all x, y ∈ K j and j ∈ N. (1) The condition was introduced by R. Isaac in [9] for the case of N = {1}.
We say that a Markov system (K i(e) , w e , p e ) e∈E is (uniformly) continuous iff w e | K i(e) and p e | K i(e) are (uniformly) continuous for each e ∈ E, where the notation f | A means the restriction of a function f on a set A. We call the Markov system positive iff p e | K i(e) > 0 for all e ∈ E.
A sequence (e 1 , ..., e n ) of e i ∈ E for all 1 ≤ i ≤ n is called a path of the Markov system iff i(e i+1 ) = t(e i ) for all i. We will denote by δ x ∈ P (K) the Dirac probability measure concentrated at x ∈ K, by B α (x) the closed ball of radius α and centre x, by 1 A the indicator function of a set A, byĀ the topological closure of a set A and byf the continuous extension of a uniformly continuous function f on the closure of the domain of its definition. For a measurable map between measure spaces f : (X, A, µ) −→ (Y, B), f (µ) will denote the measure on (Y, B) given by f (µ)(B) := µ(f −1 (B)) for all B ∈ B, and f −1 (B) will denote the σ-algebra {f −1 (B)| B ∈ B}. As usual, ≪ will denote the absolute continuity relation for measures.
Let (X, B, Λ) be a probability space and I be an at most countable set. A family (A i ) i∈I ⊂ B is called a partition of (X, B, Λ) iff its members are pairwise disjoint and Λ(X \ i∈I A i ) = 0. For a partition α of (X, B, Λ) and a sub-σalgebra C ⊂ B, H Λ (α|C) will denote the conditional entropy of α conditioned on C with respect to Λ, which is given by with the usual definition 0 log 0 := 0, where E Λ (1 A |C) denotes the conditional expectation of the indicator function 1 A conditioned on C with respect to Λ.
We will use the usual notion of the tightness. A set of Borel measures {Λ i | i ∈ I} on a topological space X is called (uniformly) tight iff for every ǫ > 0 there exists a compact C ⊂ X such that Λ i (X \ C) < ǫ for all i ∈ I.

Results
Let D R be a random dynamical system on a complete metric space (K, d) which has a Markov partition (K j ) j∈N such that the resulting Markov system M := (K i(e) , w e , p e ) e∈E is countable. Set Let E and N be provided with the discrete topologies. SetĒ := E ∪ {∞} endowed with Alexandrov's one-point compactification topology, i.e. the topology consists of all subsets of E and sets of the formĒ \ C where C ⊂ E is finite. Note that the topology has a countable base (the axiom of choice is assumed in this paper). LetĒ be equipped with the Borel σ-algebra. Note that the Borel σ-algebra still consist of all subsets. We can write D R = (K, w e , p e ) e∈Ē where w ∞ := id and p ∞ := 0, as such an extension does not change the action of D R on functions and measures by its operators. Set i(∞) := 1 and t(∞) := 1.
Then we also can write M = (K i(e) , w e , p e ) e∈Ē in the above sense. Now, set Σ := {σ := (..., σ −1 , σ 0 , σ 1 , ...)| σ i ∈Ē for all i ∈ Z} provided with the product topology (a similar compactification has been used in [10]). HenceΣ is Hausdorff and, by the Tikhonov Theorem, compact. Moreover, the topology ofΣ has a countable base, sinceĒ does, and it is regular, sinceĒ is, and therefore, it is metrizable, by the Urysohn Metrization Theorem.Σ is called the code space of the Markov system. Note that, since the topology ofĒ has a countable base, the Borel σ-algebra onΣ coincides with the product σ-algebra. Let S :Σ −→Σ be the left shift map given by (Sσ) i−1 = σ i for all i ∈ Z and σ ∈Σ. Let P S (Σ) denote the space of all shift invariant Borel probability measures onΣ equipped with the weak-star topology. Recall that, since the topology ofΣ has a countable base, the Banach space of all continuous functions on it is separable, and therefore, the weak-star topology on the unit ball of the dual space is metrizable. Furthermore, by the Riesz Representation Theorem and the Alaoglu Theorem, P S (Σ) is compact and metrizable in the weak-star topology, as a closed subset of the unit ball. LetΣ + := {(σ 1 , σ 2 , ...)| σ i ∈Ē for all i ∈ N} provided with the product topology and the product σ-algebra, and let B(K) ⊗ B(Σ + ) denote the product σalgebra of the Borel σ-algebra on K and that onΣ + . Let m [e m , ..., e n ] + ⊂Σ + , m > 0, denote a cylinder set. For x ∈ K, let P x denote the Borel probability measure onΣ + given by Set We call Σ G the path space associate with M. One easily checks that the topology on Σ G which is induced fromΣ coincides with that given by d ′ .
Remark 1 In the following, often implicitly, the following fact will be used, which might be useful to observe before. For every m [e m , ..., e n ] ∈ A m , x ∈ K and m ∈ Z, P m x ( m [e m , ..., e n ]) > 0 implies that (e m , ..., e n ) is a path of the Markov system and x ∈ K i(em) . This follows from the definition that the probability functions are zero outside their vertex sets. Note that, in this paper, they are allowed to be zero also on their vertex sets, whereas in [25] it was required that p e | K i(e) > 0 for all e ∈ E. The latter is necessary if one wants to prove that the process started at any x ∈ K i , for a fixed i ∈ N , converges to the same stationary state [31]. However, in this article, we are concerned only with the question on the existence of the stationary states.
Moreover, observe that, since i is surjective, for all x ∈ K, m < n and k ≥ 0.

Refinement of a Markov system
The study of a random dynamical system via an associated Markov system has some flexibility. Often one can choose several Markov systems associated with a given random dynamical system, e.g. see Examples 3 and 4. In such a case, choosing a finer Markov partition can help to obtain a Markov system with desired properties. Also, sometimes one can obtain some proprieties of a Markov system from those which refine it, e.g. see Example 1 in [30], and vice versa, e.g. see Proposition 2. In this subsection, we provide some tools which enable one to exploit this flexibility.

Definition 1
We call a Markov system M r := (K r i(e) , w r e , p r e ) e∈E r a refinement of M if and only if partition {K r i(e) } e∈E r refines partition {K i(e) } e∈E (i.e. each K i is a union of some K r j 's) and there is a surjective map r : E r −→ E such that w r(e) | K r i(e) = w r e | K r i(e) and p r(e) | K r i(e) = p r e | K r i(e) for all e ∈ E r (we use the same letters for maps i, t : E r −→ N r ). We call r the refinement map. Let r be extended on the one-point compactification by r(∞) := ∞. Then r defines a Borel-Borel-measurable surjective map whereΣ r denotes the compact code space associated with M r , and in the same way ψ r :Σ r+ −→Σ + . We will denote most objects associated with M r with the same letters as for those associated with M and a superscript r or subscript r (e.g. Σ r G denotes the path space of M r ).
Note that a measure is invariant for M if and only if it is invariant for M r , as with both Markov systems is associated the same Markov operator U .
is open for every U from the subbase of the topology onΣ.
for some e ∈ E and i ∈ Z. Then, by the definition of Ψ r , Ψ −1 r (U ) = e ′ ∈E r ,r(e ′ )=e i [e ′ ], and therefore, it is open. Now, let U = e∈Ē\C i [e] for some finite C ⊂ E and i ∈ Z. Then Hence, by the hypothesis, is open for any of the above cases, but this is possible only if r −1 (C) is finite (since there exists finite (ii) Let σ ′ ∈ Σ r G and σ := Ψ r (σ ′ ). Let n ∈ Z and j := i(σ n ). Observe that . Hence i(σ n+1 ) = t(σ n ). The assertion follows.
(iii) Let σ ∈ Σ G . Since Ψ r is surjective, there exists σ ′ ∈Σ r such that Ψ r (σ ′ ) = σ. Suppose there exists j ∈ Z such that t(σ ′ j ) = i(σ ′ j+1 ). Choose such j with the smallest absolute value. In the case that there are two such j, do what follows first for the non-positive one and then iterate the choice of such j with the smallest absolute value again. By the definition of M r , Since M is positive, there exists e ∈ E r such that i(e) = i and r(e) = σ j+1 . Set σ ′ j+1 := e. Then t(σ ′ j ) = i(σ ′ j+1 ) and Ψ r (σ ′ ) = σ. Iterate the procedure until j exceeds the maximal absolute value encountered so far and set σ 1 := σ ′ . By iterating the procedure, we obtain a sequence (σ n ) n∈N ⊂Σ r such that there exists an increasing sequence (m n ) n∈N ⊂ N ∪ {0} such that (σ n −mn , ..., σ n mn ) is a path and Ψ r (σ n ) = σ for all n ∈ N. By the compactness and the metrizability ofΣ r , there exists a subsequence (σ n k ) k∈N and σ ′′ ∈Σ r such that σ n k → σ ′′ as k → ∞. Hence, since Ψ r is continuous, Ψ r (σ ′′ ) = σ. In particular, by the , it contains infinitely many of σ n , but this contradicts to their construction. Thus σ ′′ ∈ Σ r G . Together with (ii), this completes the proof of (iii).
(iv) Let x ∈ K and 1 [e 1 , ..., e n ] + ⊂Σ + . Then, by the definition of ψ r , Therefore, Now, observe that, by the definition of p r e ′ 's, there exists at most one e ′ 1 ∈ E r with r(e ′ 1 ) = e 1 and p r If there are no such e ′ 1 , then the right hand side of (3) is zero and Now, by repeating the argument for Thus, the claim follows, since the class of the cylinder sets generates the σalgebra, is ∩-stable and has a countable subset coveringΣ r+ .

Equilibrium states
Now, we are going to define the main objects on the code space which are useful not only for a description of the invariant measures, but also, combined with an other object which will be introduced in subsection 3.3.1, allow to control the asymptotic behaviour of the system, most importantly at the boundaries of the atoms of the Markov partition, where the continuity of the system may not be available.
Next, set We call the members of E(M) the equilibrium states of M.
It will be shown in subsection 3.2.1 that the definition of E(M) naturally extends the notion of equilibrium states in the thermodynamic sense. Also, an anonymous reviewer pointed out that the condition for members in E(M) is related to the 'conformality', as in [5].
Now, we show that the property of E(M) is transferable under the refinement in some cases.
Definition 2 Let M r := (K r i(e) , w r e , p r e ) e∈E r be a refinement of M. Define D r and the coding map F r associated with M r as above by choosing, for every If all w e | K i(e) 's are contractions, then, obviously, D and F | D do not depend on the choice of x i 's.
Lemma 2 Suppose D and F | D do not depend on the choice of x i 's, and M r is countable. Then the following holds true.
. Then the following limits exist, and, by the hypothesis, This shows (i) and (iii).

Thermodynamic equilibrium states
Now, we are going to show that the members of E(M) with finite entropy which can be computed according to Kolmogorov-Sinai Theorem are exactly the equilibrium states in the thermodynamic sense, which minimise the free energy of the system, for the following energy function. Set for all σ ∈Σ with the definition log(0) := −∞. u is called the energy function of the Markov system.
Recall that, by Kolmogorov-Sinai Theorem, Let E(u) ⊂ P S (Σ) denote the set of all equilibrium states for u. Then and the equality holds if and only if Λ ∈ E(M).
log p e • F dΛ That is This completes the proof. ✷ Conversely, for every Λ 0 ∈ E(u), by the hypothesis and Lemma 3, Thus, by Lemma 3, Λ 0 ∈ E(M). This completes the proof. ✷ Theorem 1 and Example 2 from Section 4 seem to indicate that Shannon-Kolmogorov-Sinai entropy might be not the best choice of the entropy for a satisfactory thermodynamic description of such systems.

Uniformly continuous Markov system
In this subsection, we develop a general theory on the relation of the equilibrium states and the invariant measures of M if it is uniformly continuous.
Let e ∈ E. If w e | K i(e) is uniformly continuous, letw e denote the continuous extension of w e | K i(e) onK i(e) , which then can be considered to be extended on K arbitrarily.
Then, by the shift invariance of M , for all A ∈ A m and m ≤ 0. Observe that, by the invariance of µ, φ m (µ)'s are consistent for all m ≤ 0 (e.g. see [28]). Let Φ(µ) ∈ P S (Σ) denote the measure which uniquely extends φ m (µ)'s on the Borel σ-algebra, e.g. by the Kolmogorov Consistency Theorem. This defines a map Φ : It is not difficult to check that, for every ν ∈ P (K) and Ω ∈ B(K) ⊗ B(Σ + ), defines a probability measure on product σ-algebra for everyφ(ν)-integrable function f : and .
(iv) For F (M )-a.e x 0 ∈ K, the sequence of probability measures (α x0 n ) n∈N on N given by α x0 n ({j}) := 1/n n k=1 U * k δ x0 (K j ) for all j ∈ N and n ∈ N converges in total variation.

The non-degeneracy condition
Now, we are going to specify the case where the association of the directed graph with the topological structure of M does not degenerate asymptotically almost surely, i.e. F (σ) ∈ K t(σ0) for almost every σ ∈ Σ G with respect to every asymptotic state, which now will be defined also.
Definition 4 Set T j := {σ ∈ Σ G | t(σ 0 ) = j} for all j ∈ N . Then, obviously, T j ∩ T j ′ = ∅ for all j = j ′ and j∈N T j = Σ G . Suppose p e | K i(e) is uniformly continuous for all e ∈ E. For each e ∈ E, letp e denote the continuous extension of p e | K i(e) onK i(e) which is extended further on K by zero. LetẼ(M) denote the set We call the members ofẼ(M) the asymptotic states of M.
Below it will be shown that every equilibrium state is an asymptotic state and that the converse is true only in the case which we are going to specify now. Proof. (i) Let i ∈ N and A ∈ F . Then, since Λ(Σ G ) = 1, Therefore, (ii) Let Λ ∈ E ⊥ (M), i ∈ N and A ∈ F . First, observe that, by the Fatou Lemma, Hence Λ(F −1 (K i ) ∩ T i ) = 0, and therefore, ) for all n ≥ k, and therefore, both M G -a.e.. Hence, since M G (G ∩ D) = 1,v e = v e M G -a.e., and therefore, This completes the proof of (i).
Thus, the decomposition is unique.

A sufficient condition for the non-degeneracy
The following lemma will be used to show the non-degeneracy of most of the examples in this article.  Proof. Let n ∈ N. Using the shift-invariance of Λ and (8), one easily checks that, for all e 1 , ..., e n ∈ E, Therefore, for every B ∈ B(K), That is, 1 ≤ R n 1 F (Λ)-a.e.. The assertion follows. Proof. The assertion follows immediately from Lemma 7 and Theorem 2. ✷

The consistency condition
Example 4 (below) shows that the non-degeneracy is not a necessary condition for the existences of an invariant measure for a finite, contractive and uniformly continuous Markov system. Now, we are going to weaken the non-degeneracy condition, in order to include the case where a degenerate system still has an invariant measure because every degeneracy with respect to an asymptotic state transpires in a consistent way, so that no loss of measure occurs. For example, this can happen because of some continuities at some boundaries of some atoms of the partition, or even, as in Example 4, because the degenerate system is actually a refinement of a non-degenerate one. Of course, in the latter case, one might want to consider instead the non-degenerate Markov system associated with the random dynamical system, but it might be not favourable because of the loss of some other nice properties, such as the contraction on average. The theorem below can be used in a degenerate case. By Theorem 2 and Proposition 1, every uniformly continuous, non-degenerate Markov system is consistent.

Condition 1 M is uniformly continuous and
and all bounded f ∈ L B (K).
Obviously, the condition is satisfied if F −1 (Ω) is empty, which, by Lemma 8, implies the non-degeneracy. In general, it implies the consistency, as the next theorem shows. However, it is not a necessary condition for it (see Example 4). (ii) Let Λ ∈ E ⊥ (M) and f ∈ L B (K) be bounded. Then, by the hypothesis and Lemma 7, ).
Since f was arbitrary, it follows that F (Λ) ∈ P (M). ✷

Lemma 9
Suppose D R is continuous, and M is uniformly continuous.
(i) Let f ∈ L B (K) be bounded. Then Proof. (i) Let D R = (K, w ′ e , p ′ e ) e∈E ′ with w ′ e : K −→ K and p ′ e : K −→ [0, 1] both continuous for all e ∈ E ′ and c : E −→ E ′ be given by w ′ c(e) | K i(e) = w e | K i(e) and p ′ c(e) | K i(e) = p e | K i(e) for all e ∈ E. Note that, for each j ∈ N , for all e ∈ c −1 ({c(e 0 )}). Therefore, This completes the proof of (i).
(ii) The assertion follows immediately from (i) by Theorem 3 (ii) and the hypothesis. ✷

Remark 2
The consistency condition is also not a necessary condition for the existences of an invariant measure for a finite, contractive and uniformly continuous Markov system. For example, an inconsistent system still can have an invariant measure because there exists M ∈Ẽ(M) such that F (M ) ∈ P (M), e.g. see Example 5. However, this situation can be often reduced, as in Example 5, to the consistency of a subsystem, see Corollary 4.

The dominating Markov chain
The next object arises naturally from the following condition, which will be used for several purposes in the case of an infinite set {e ∈ E| i(e) = j}, j ∈ N .
Definition 7 We say that M has a dominating Markov chain iff for every i ∈ N , In this case, set for all for all i, j ∈ N , and Proof. Let x ∈ K. If x / ∈K j , then, clearly, e∈E,i(e)=jp e (x) = 0. Otherwise, there exists a sequence (x n ) n∈N ⊂ K j such that lim n→∞ x n = x. Clearly, e∈E,i(e)=jp e (x n ) = e∈E,i(e)=j p e (x n ) = 1 for all n ∈ N. Since lim n→∞ p e (x n ) =p e (x) for all e ∈ E with i(e) = j, and M has a dominating Markov chain, it follows, by Lebesgue's Dominated Convergence Theorem, that for all e ∈ E. Then, by the continuity of D R , for each e ∈ E, ∂p e = p e 1K i(e) \K i(e) = 1K i(e) \K i(e) p ′ c(e) andw e |K i(e) = w ′ c(e) |K i(e) . Note that, by Lemma 10, e∈E,i(e)=j ∂p e = 1K j \Kj for all j ∈ N . Therefore, as in the proof of Theorem 3 (ii), Hence, f dF (Λ) ≤ f dU * F (Λ). Since f was arbitrary, it follows that U * F (Λ) = F (Λ). Thus, the assertion follows by Theorem 3 (i). Proof. By Lemma 6 and Lemma 2 (iv), it is sufficient to show that for all e ′ ∈ E r with r(e ′ ) = e, and, since M r has a dominating Markov chain, for all e ′ ∈ E r . Let A := n [e n , ..., e 0 ] ∈ F . Then, obviously, Ψ −1 r (A) ∈ F r , and therefore, Now, observe that, since M is positive and Λ r (Σ r G ) = 1, by Lemma 1 (ii),

A recurrence condition
It is well known from the theory of discrete homogeneous Markov chains that the existence of an invariant probability measure requires the positive recurrence of the process. The following condition serves the same purpose for our generalization (see [30] for more elaboration on that). The next lemma gives then a sufficient condition for it in terms of the dominating Markov chain. (It was pointed out by an anonymous reviewer of [30] that the condition might be related to the notion of positive recurrence for countable Markov shifts introduced by O. Sarig, see [18] and [19].) Definition 8 For x ∈ K, let (α x n ) n∈N denote the sequence of probability measures on N given by for all j ∈ N and n ∈ N.

Condition 2
There exists x 0 ∈ K such that (α x0 n ) n∈N is uniformly tight, i.e. for every ǫ > 0 the exists a finite V ⊂ N such that α x0 n (N \ V ) < ǫ for all n ∈ N.
Lemma 12 Condition 2 is satisfied for all x 0 ∈ K if M has a dominating Markov chain such that c < ∞.
Proof. By the hypothesis, ξ i < ∞ for all i ∈ N , as in (14). Let k > 0. Then for every x ∈ K and j ∈ N , where (q ij ) i,j∈N is the transition matrix of the dominating Markov chain. Hence, Fix x 0 ∈ K. Let i 0 ∈ N such that x 0 ∈ K i0 . Let ǫ > 0. By the hypothesis, there exists a finite V ǫ ⊂ N such that j∈N \Vǫ sup i∈N ξ i q ij < ǫ/2. Let n 0 ∈ N such that ξ i0 q i0j /n 0 < ǫ/2. Then

Contractive, uniformly continuous Markov system
In this subsection, we are going to apply the theory developed so far to the case when M is contractive.
Lemma 13 Suppose M is contractive with a contraction rate 0 < a < 1.

Lemma 14
Suppose M is contractive with a contraction rate 0 < a < 1. Then Proof. Let µ ∈ P (M). Clearly, the inequality is true if b = ∞. Now, suppose b < ∞. For f ∈ L B (K) and k ∈ N, set f ∧ k := min{f, k}. Observe that U (f ∧ k) ≤ k ∧ U (f ). Hence, by induction, U n (f ∧ k) ≤ k ∧ U n (f ) for all n ∈ N. Therefore, by (17), for all n ∈ N. Since, for every k ≥ 0, the functions k ∧ (a n L + b/(1 − a)) are integrable and converge monotonously to k ∧ (b/(1 − a)), as n → ∞, by the Monotone Convergence Theorem, for all k ≥ 0. Applying the Monotone Convergence Theorem again, as k → ∞, implies the assertion. (iii) There exists a sequence of Borel sets Q 1 ⊂ Q 2 ⊂ ... ⊂ Σ G with k≥n Φ(µ)(Σ\ Q k ) ≤ 1/(1 − √ a)a n/2 for all µ ∈ P (M) and n ∈ N such that for each k ∈ N for all m ∈ Z and n ≥ 1. Then each φ n m is clearly a measure on A m . Recall that Σ + is a compact metrizable space. Clearly, the set of all pre-images of cylinder sets in B(Σ + ) under π is exactly the set of all cylinder sets in A 1 . Therefore, since both σ-algebras are generated by their cylinder sets, π −1 (B(Σ + )) = A 1 , i.e. the induced set mapπ −1 : B(Σ + ) −→ A 1 is bijective. Since the set of all Borel probability measures onΣ + is sequentially compact in the weakly-star topology, there exists a subsequence (π(φ n k 1 )) k∈N and a probability measure φ + on B(Σ + ) such that π(φ n k 1 ) converges to φ + weak-star as k → ∞. Observe that, by the definition of φ n 1 's, φ + is invariant with respect to the left shift map on Σ + , as the shift maps commute with π. Set M (π −1 (B)) := φ + (B) for all B ∈ B(Σ + ).
Then this defines a shift-invariant measure M on A 1 . Furthermore, by the shift-invariance of M , this gives consistent measures on A m for all m ∈ Z, by M (S m−1 A), which we will also denote by M . In particular, M defines consistent measures on all finite dimensional sub-σ-algebras of the product σ-algebra on Σ. Let e 1 , ..., e n ∈ E. Observe that the image of 1 [e 1 , ..., e n ] ⊂Σ under π is a cylinder set which is open and closed inΣ + . Observe that φ n m = φ n 1 • S m−1 for all m ≤ 1 and n ≥ 1. Therefore, for all m ≤ 1. Furthermore, observe that for every open set O ∈ A m and m ≤ −1, as π is open. Now, for every m ≤ 0, k ≥ 1 and finite C ⊂ E, Let ǫ > 0. By the hypothesis, there exists a finite V ǫ ⊂ N such that α x0 n (N \V ǫ ) < ǫ/2 for all n ∈ N. Thus, by (21) Therefore, by the hypothesis, there exists a finite C ⊂ E such that This means that each one-dimensional measure M has an approximating compact class. Therefore, M extends uniquely to a shift-invariant Borel probability measure onΣ, which we will also denote by M , e.g. by Kolmogorov Consistency Theorem [4]. (AsĒ is a Polish space, the existence of a compact approximating class is actually automatic. However, (22) is still needed for the next step.) Now, set By (22), for every m ∈ Z there exists a finite C m ⊂ E such that As Ω ∞ ⊂ m∈Z e∈Ē\Cm m [e], it follows that M (Ω ∞ ) ≤ 3/4ǫ < ǫ. Since ǫ was arbitrary, we conclude that M (Ω ∞ ) = 0.
Next, we show that Σ G has the full measure. First, observe that every σ ∈Σ\Σ G is either in Ω ∞ , or there exists e 1 , ..., e n ∈ E such that (e 1 , ..., e n ) is not a path and σ ∈ m [e 1 , ..., e n ] for some m ∈ Z. By Remark 1 and (20) Then, by the compactness ofΣ, there exist finitely many σ 1 , ..., Since ǫ was arbitrary, we conclude that Now, we are going to show that M (D) = 1. For x ∈ K and m ≤ 0, set Then, by (19), for all x ∈ K, m ≤ 0 and n ≥ 1. Hence for all m ≤ −1 and n ≥ 1. Therefore, by (23) and (21) for all m ≤ −1.

Now, set
Then for all l ≤ −1.
Now, observer that for every σ ∈ Σ G \ A x0 , sequence (X m (σ)) m≤0 is Cauchy. Hence, by the completeness of (K, d), Σ G \ A x0 ⊂ D. Therefore, where r mn,x is a signed measure on F m given by for all A ∈ F m . Hence, as every member of F m can be written as a countable disjoint union of cylinder sets, for all A ∈ F m and m < 0 and n ∈ N. This implies, by (20) and (23), that for all A which are finite unions of cylinder sets from F m and m < 0. Now, for y ∈ K and m ≤ 0, set and β e (t) := sup Then, by (18), for all y ∈ K and m ≤ 0. Therefore, for all A ∈ F m , m ≤ 0 and k ∈ N. Set ρ := b/(1 − a) + L(x 0 ) and for all α ≥ 0. Then, by (17), for all k ∈ N. Now, set C ǫ := 2ρ/ǫ + b/(1 − a). Then C(y) ≤ C ǫ for all y ∈ B(2ρ/ǫ). Therefore, by (29), for all A ∈ F m , m ≤ 0 and k ∈ N. Thus, by (28), for all A which are finite unions of cylinder sets from F m and m < 0, and, since every member of F m can be written as a countable union of cylinder sets, by Lebesgue's Dominated Convergence Theorem, it holds true for all A ∈ F m and m < 0. That is and, obviously, the same inequality holds true also withΣ \ A − m in place of A − m for all m < 0. Hence Therefore, by Lebesgue's Dominated Convergence Theorem, p e • X m (σ) converges top e • F (σ)1 K i(e) (x t(σ0) ) in L 1 (M ). Therefore, by the triangle inequality and (31), Since ǫ was arbitrary, we conclude that  (30), Hence, there exists a path (e m0 , ..., e 0 , e) such that The integration of (19) with respect to µ implies that for all m ≤ 0. By Lemma 14, C(x)µ(x) ≤ 2b/(1 − a), therefore we can define for all m ≤ −1, and the same way as for (26), this implies that for all A ∈ F m . Furthermore, the integration of (18) gives for all m ≤ 0. Hence, the same way as for (29), it follows that for all A ∈ F m . Therefore, as in the proof of (i), (34) implies that for all e ∈ E. Thus Φ(µ) ∈Ẽ(M), as desired.
Next, we show that F (Φ(µ)) = µ. It is sufficient to show that the measures agree on all bounded uniformly continuous non-negative functions on K (as this set of functions is closed under multiplication and generates Borel σ-algebra).
Let f ∈ L B (K) be bounded and uniformly continuous. Observe that, for each m ≤ 0, where Since f is uniformly continuous and bounded, one sees, by the contraction condition, the same way as for (36), that |R m | → 0 as m → −∞. Therefore, since Φ(µ)(D) = 1, by the already shown, (37) implies by Lebesgue's Dominated Convergence Theorem that as desired.
(iii) Now, set for all k ∈ N. By (33), k≥n Φ(µ)(Σ \ Q k ) ≤ 1/(1 − √ a)a n/2 for all µ ∈ P (M) and n ∈ N. Since Φ(µ)(D) = 1 for all µ ∈ P (M), we can assume Q k ⊂ D for all k. The proof that F | Q k is locally Hölder continuous is the same as that of Lemma 3 (iii) in [26]. We give it for completeness here. Let σ, σ ′ ∈ Q l for some l ∈ N. Then, by the triangle inequality, for all m ≤ −l.

Remark 3
An anonymous reviewer pointed out that it might be possible to replace the compactification with the tightness argument for obtaining measure M in the proof of Theorem 5 and restrict the consideration to Σ G . Also, it might be possible to replace other mertizable compactness arguments in the article with the completeness and the separability arguments. However, the author thinks that it would be short-sighted to discard the powerful information of working on a compact metrizable space from the set-up of the theory, in particular, because it gives the access to the results of the well-developed ergodic theory on the standard topological space, e.g. the author refers to it in the follow-up paper [30], which uses many results from this paper. Note that, in contrast to the mainstream thermodynamic formalism on countable Markov shifts, we do not have the continuity of the potential on Σ G , and therefore, we have nothing to lose by working on the standard topological space of the ergodic theory, and sometimes taking advantage of it. Furthermore, with not having the openness of the Markov partition, and therefore, working with possibly overlapping closures of the atoms of the partition, it seems to be unreasonable to restrict a priori the consideration only to Σ G .
Also, the reviewer found that 'The proof of theorem 5 could be simplified (and significantly shortened) by using tightness to prove the existence of an invariant measure, Borel-Cantelli to prove M (D) = 1 and Radon-Nikodym derivatives instead of conditional expectations.', though no proof was presented.
Corollary 1 Suppose M is contractive, uniformly continuous and non-degenerate such that b < ∞. Then the following holds true. (ii) The assertion follows by Theorem 5 (iii), Theorem 2 and Lemma 4 (i). ✷

Invariant measures
Corollary 2 Suppose M is contractive and uniformly continuous such that b < ∞. Then the following holds true. (v) If E is finite, then P (M) is uniformly tight.
Proof. (i) The assertion follows by Corollary 1 and Proposition 1.
(ii) The assertion follows by Theorem 5 (i).
is open inΣ, and d ′ generates exactly the topology on Σ G which is induced fromΣ, . By the hypothesis, for every n ∈ N, Σ G ∩ 0 [e] can be written as a finite union of sets Σ G ∩ −n [e −n , ..., e n ] where each (e −n , ..., e n ) is a path. Since the sets are open balls in (Σ G , d ′ ), choosing σ kn from the ball containing infinite number of σ k 's for each n ∈ N gives a Cauchy subsequence, which, by the completeness of (Σ G , d ′ ) converges to some σ ∈ Σ G . Since is compact in (Σ G , d ′ ). Now, letQ be the closure of Q in (Σ G , d ′ ), andF be the is compact in (Σ G , d ′ ). Let E c := {e ∈ E| i(e) ∈ c}. Then, by the hypothesis, E c is finite, and therefore, is compact in (Σ G , d ′ ). Hence,C :=F (B) is compact in (K, d), and, by Theorem 5 (ii), for every µ ∈ P (M), This completes the proof of (iv).
(v) follows immediately from (iv). ✷ Corollary 3 Suppose M is consistent, contractive, uniformly continuous with b < ∞ and has a dominating Markov chain such that c < ∞. Then P (M) is not empty.
Proof. The assertion follow by Lemma 12 and Corollary 2 (ii). ✷ Finally, we remark that the study of a random dynamical system via an equivalent Markov system has another flexibility. As the following simple lemma shows, the problem of determining whether a Markov systems has an invariant measure can be reduced to that on a subsystem, which might be easier, as, obviously, any Borel probability measure on i∈S K i can be uniquely identified with a member of P ( i∈S K i ). For example, as Examples 4, 5, 7 and 8 show, a non-empty Ω sometimes contains a closed Markov subsystem. We cover this situation by the following simple corollary.

Definition 9
We say that M contains a Markov subsystem iff there exists S ⊂ N such that (K i(e) , w e , p e ) e∈i −1 (S) is a Markov system on i∈S K i . We call the Markov subsystem closed iff i∈S K i is closed in K.

Lemma 15
Suppose M has a Markov subsystem (K i(e) , w e , p e ) e∈i −1 (S) , for some S ⊂ N , which has an invariant µ ∈ P ( i∈S K i ). Then µ ∈ P (M).
Observe that, in this case, F is nothing else but the natural projection Σ G −→ Σ − G and Φ is the natural extension of a g-measure. Moreover, in this example, Corollary 2 (v) is obvious, Theorem 5 (iii) can be strengthened to globally Hölder continuous F , and Proposition 1, Corollary 1 (ii) and Theorem 1 reduce to Theorem 2.1 in [23] as follows. Let B denote the Borel σ-algebra on Σ − G and P T (Σ − G ) denote the set of all T -invariant members of P (Σ − G ).
Thus m is an equilibrium state for log g. This proves the implication from (ii) to (iii).
for the invitations to give several talks on the subject at the Ergodic Theory and Statistical Mechanics Seminar at Lomonosov Moscow State University and also the other organizers and participants of the seminar for the questions which helped to improve the article, in particular Sergey A. Pirogov whose question on the dominating Markov chain condition, the author strongly suspects, was an indirect way to point out that the formulation of the condition was unnecessarily strong.