On the $f$-Norm Ergodicity of Markov Processes in Continuous Time

Consider a Markov process $\{\Phi(t) : t\geq 0\}$ evolving on a Polish space ${\sf X}$. A version of the $f$-Norm Ergodic Theorem is obtained: Suppose that the process is $\psi$-irreducible and aperiodic. For a given function $f\colon{\sf X}:\to[1,\infty)$, under suitable conditions on the process the following are equivalent: \begin{enumerate} \item[(i)] There is a unique invariant probability measure $\pi$ satisfying $\int f\,d\pi<\infty$. \item[(ii)] There is a closed set $C$ satisfying $\psi(C)>0$ that is ``self $f$-regular.'' \item There is a function $V\colon{\sf X} \to (0,\infty]$ that is finite on at least one point in ${\sf X}$, for which the following Lyapunov drift condition is satisfied, \[ {\cal D} V\leq - f+b\field{I}_C\, , \eqno{\hbox{(V3)}} \] where $C$ is a closed small set and ${\cal D}$ is the extended generator of the process. \end{enumerate} For discrete-time chains the result is well-known. Moreover, in that case, the ergodicity of $\bfPhi$ under a suitable norm is also obtained: For each initial condition $x\in{\sf X}$ satisfying $V(x)<\infty$, and any function $g\colon{\sf X}\to\Re$ for which $|g|$ is bounded by $f$, \[ \lim_{t\to\infty} {\sf E}_x[g(\Phi(t))] = \int g\,d\pi. \] Possible approaches are explored for establishing appropriate versions of corresponding results in continuous time, under appropriate assumptions on the process $\{\Phi(t)\}$ or on the function $g$.


Introduction
Consider a Markov process Φ = {Φ(t) : t ≥ 0} in continuous time, evolving on a Polish space X, equipped with its Borel σ-field B. Assume it is a nonexplosive Borel right process: It satisfies the strong Markov property and has right-continuous sample paths [1,9].
A set C is called small if there is probability measure ν on (X, B), a time T > 0, and a constant ε > 0 such that, for every A ∈ B.
It is assumed that the process is ψ-irreducible and aperiodic, where ψ is a probability measure on (X, B). This means that for each set A ∈ B satisfying ψ(A) > 0, and each x ∈ X, P t (x, A) > 0, for all t sufficiently large.
It follows that there is a countable covering of the state space by small sets [8,Prop. 3.4]. The Lyapunov theory considered in this paper and in our previous work [5,9] is based on the extended generator of Φ, denoted D. A function h : X → R is in the domain of D if there exists a function g : X → R such that the stochastic process defined by, is a local martingale, for each initial condition Φ(0) [1,13]. We then write g = Dh.
For example, consider a diffusion on X = R d , namely, the solution of the stochastic differential equation, dΦ(t) = u(Φ(t))dt + M (Φ(t))dB(t), t ≥ 0, Φ(0) = x, where u = (u 1 , u 2 , . . . , u d ) T : X → R d and M : R d → R d × R k are Lipschitz, and B = {B(t) : t ≥ 0} is k-dimensional standard Brownian motion. If the function h : X → R is C 2 then we can write [13], The Lyapunov condition considered in this paper is Condition (V3) of [9]: For a function V : X → (0, ∞] which is finite for at least one x ∈ X, a function f : X → [1, ∞), a constant b < ∞, and a closed, small set C ∈ B, It is entirely analogous to its discrete-time counterpart [11], in which the extended generator is replaced by a difference operator D = P − I, where P is the transition kernel of the discretetime chain and I is the identity operator. The lower bound f ≥ 1 is imposed in (V3) because this function is used to define two norms: One on measurable functions g : X → R via, and a second norm on signed measures µ on (X, B): Our main goal is to establish the erodicity of Φ in terms of this norm: There is an invariant measure π for the semi-group {P t } satisfying, The following result is a partial extension of the f -Norm Ergodic Theorem of [11] to the continuous time setting.
Theorem 1.1. Suppose that the Markov process Φ is ψ-irreducible and aperiodic, and let f ≥ 1 be a function on X. Then the following conditions are equivalent: (i) The semi-group admits an invariant probability measure π satisfying: (ii) There exists a closed, small set C ∈ B such that, where τ C (1) := inf{t ≥ 1 : Φ(t) ∈ C} and E x denotes the expectation operator under X 0 = x.
(iii) There exists a closed, small set C and and an extended-valued non-negative function V satisfying V (x 0 ) < ∞ for some x 0 ∈ X, such that Condition (V3) holds.
Moreover, if (iii) holds then there exists a constant b f such that, where V and C satisfy the conditions of (iii). The set S V = {x : V (x) < ∞} is absorbing (P t (x, S V ) = 1 for each x ∈ S V and all t ≥ 0), and also full (π(S V ) = 1).
Proof. Theorem 1.2 (b) of [10] gives the equivalence of (i) and (ii). Theorem 4.3 of [10] gives the implication (iii) ⇒ (ii), along with the bound (5). Conversely, if (ii) holds then we can define, We show in Proposition 2.2 that this is a solution to (V3) and that it is uniformly bounded on C.
f -Norm Ergodicity December 3, 2015 3 The function V in (6) has the following interpretation. Let T denote an exponential random variable that is independent of Φ, and denote, We then have, where now the expectation is over both Φ and T . Consequently, this construction is similar to the converse theorems found in [11] for discrete-time models. Theorem 1.1 is almost identical to the f -Norm Ergodic Theorem of [11], except that it leaves out the implications to ergodicity of the process. This brings us to two open problems: Under the conditions of Theorem 1.1: Q1 Can we conclude that (3) holds for any initial condition x ∈ S V ?
Q2 Assume in addition that π(V ) < ∞. Can we conclude that there exists a finite constant In discrete time, questions Q1 and Q2 are answered in the affirmative by the f -Norm Ergodic Theorem of [11], with the integral replaced by a sum in (8). Q2 is resolved in the affirmative in this paper by an application of the discrete-time counterpart: Theorem 1.2. Suppose that the Markov process Φ is ψ-irreducible and aperiodic, and that there is a solution to (V3) with V everywhere finite. Then there is a constant B 0 f such that for each x, y ∈ X, If in addition π(V ) < ∞, then (8) also holds for some constant B f and all x.
Although the full resolution of Q1 remains open, in Section 3 we discuss how (3) can be established under additional conditions on the process Φ.
We begin, in the following section, with the proof of the implication (ii) ⇒ (iii), which is based on theory of generalized resolvents and f -regularity [8]. Following this result, it is shown in Proposition 2.3 that f -regularity of the process is equivalent to f ∆ -regularity for the sampled process, where ∆ is the sampling interval, and, This is the basis of the proof of Theorem 1.2 that is contained in Section 3. Acknowledgment. The work reported in this note was prompted by a question of Yuanyuan Liu who, in a private communication, pointed out to us that some results in our earlier work [3] were stated inaccurately. Specifically: (1.) The implication (ii) ⇒ (iii) in Theorem 2.2 of [4], which is the same as the corresponding result in our present Theorem 1.1, was stated there without proof; and (2.) The convergence in (3) was stated as a consequence of any of the three equivalent conditions (i)-(iii), again without proof. This note attempts to address and correct these omissions, although the relevant statements in [3] were only discussed as background material and do not affect any of the subsequent results in that paper.

f -Regularity
Following [8], we denote for each r ≥ 0 and B ∈ B, where The following result, given here without proof, is a simple consequence of Lemma 4.1 and Prop. 4

.3 of [8]:
Proposition 2.1. Suppose that the set C is closed and small, and that the following selfregularity property holds: There exists r 0 > 0 such that sup x∈C G C (x, f ; r 0 ) < ∞. Then: (i) There is b C < ∞ such that G C (x, f ; r) < G C (x, f ; r 0 ) + b C r for each x and r.
(ii) For each B ∈ B satisfying ψ(B) > 0, for each r ≥ 0, and for each x ∈ X, Consequently, the process is f -regular if G C (x, f ; r 0 ) < ∞ for each x.
We next show that the function V in (6) is finite-valued on {x ∈ X : G C (x, f ; r 0 ) < ∞}. We show that V is in the domain of the extended generator, and obtain an expression for DV .
Consider the generalized resolvent developed in [8,12]: For a function h : X → R + , A ∈ B, and x ∈ X, denote, With the usual interpretation of P t , or any kernel Q(x, dy), as a lineal operator, g → Qg = g(y)Q(·, dy), it is shown in [12] that the following resolvent equation holds: For any functions g ≥ h ≥ 0, where, for any function g, I g denotes the (operator induced by the) kernel I g (x, dy) = g(x)δ x (dy).

f -Norm Ergodicity
December 3, 2015 5 When h ≡ α is constant, we obtain the usual resolvent, In the case α = 1 we write R := R 1 = ∞ 0 e −t P t dt, and call R "the" resolvent kernel. For any non-negative function g : X → R + for which Rg is finite valued, the function γ = Rg is in the domain of the extended generator, with, Proposition 2.2. Suppose that the assumptions of Theorem 1.1 (ii) hold: There is a closed, small set C ∈ B such that, sup x∈C G C (x, f ; r 0 ) < ∞ with r 0 = 1. Then the function V defined in (7) is finite on the full set S V ⊂ X and (V3) holds with this function V and this closed set C.
Proof. Proposition 4.3 (ii) of [8] implies that the set of x for which G C (x, f ; 1) < ∞ is a full set. This result combined with Proposition 4.4 (ii) of [8] implies that V is bounded on C.
For arbitrary x we haveτ C > τ C = min{t ≥ 0 : Φ(t) ∈ C}. Consequently, by the strong Markov property and the representation (7), Hence V (x) is finite whenever G C (x, f ; 1) is finite.
To establish (V3), first observe that the function V in (7) can be expressed, Taking g ≡ 1, the resolvent equation gives, where, for any set B and kernel Q, I B Q denotes the kernel I B (x)Q(x, dy). Combining the representation of V above with (14) we obtain, The second equation can be decomposed as follows, Substitution then gives, This establishes (V3) with b = sup x∈C V (x).

f -Norm Ergodicity
December 3, 2015 6 The final results in this section concern the ∆-skeleton chain. This is the discrete-time Markov chain with transition kernel P ∆ , where ∆ ≥ 1 is given. It can be realized by sampling the Markov process with sampling interval ∆. The sampled process is denoted, In prior work, the skeleton chain is used to translate ergodicity results for discrete-time Markov chains to the continuous time setting. For example, Theorem 6.1 of [9] implies that a weak version of the ergodic convergence (3) holds for an f -regular Markov process: The proof consists of two ingredients: (i) The corresponding ergodicity result holds for the ∆-skeleton chain, and (ii) the error P t (x, · ) − π( · ) 1 is non-increasing in t.
In the next section we use a similar approach to address question Q2. The f ∆ norm is considered, where the function f ∆ is defined in (10). Denote, for every x ∈ X and every B ∈ B satisfying ψ(B) > 0.
Proposition 2.3. If the process Φ is f -regular, then each ∆-skeleton is f ∆ -regular. Moreover, there is a closed f -regular set C such that: (i) For a finite-valued function V ∆ : X → (0, ∞] and a finite constant b, and (ii) For every x ∈ X and every B ∈ B satisfying ψ(B) > 0, there is a constant c B < ∞ such that, Proof. It is enough to establish (i). Theorem 14.2.3 of [11] then implies that for every Let C denote any closed f -regular set for the process, satisfying ψ(C) > 0. For V 0 (x) = G C (x, f ) we obtain a bound similar to (17) through the following steps. First write, The integral can be expressed as a sum, By the strong Markov property, Consequently, To eliminate the function s in (19) we establish the following bound: For some ε 0 > 0 and k 0 ≥ 1, The proof is again by the strong Markov property: where ε(k) = inf{P k 0 ∆−r (y, C) : y ∈ C, 0 ≤ r ≤ ∆}. This is strictly positive for sufficiently large k because (16) holds. This establishes (20). The Lyapunov function can now be specified as, , f ), and the second term is uniformly bounded: f -Norm Ergodicity December 3, 2015 8 Consequently, from familiar arguments, This establishes (17) with b = b 0 ε −1 0 (k 0 + 1).

f -Norm Ergodicity
In this section we consider the implications to the ergodicity of the process. We assume that (V3) holds for a finite-valued function V : X → (0, ∞), so that the process is f -regular.
Q1. f -norm ergodicity. The ergodicity of Φ in terms of the f -norm as in (3) has only been established under special conditions. Theorem 5.3 of [10] implies that (3) will hold if f is subject to this additional bound: For some β ≥ 0, This holds for example if f ≡ 1 and β = 1. It is likely that the application of coupling bounds will lead to a more general theory. Under stronger conditions on the process, such a coupling time was obtained in [6], and it was used in [7] to obtain rates of convergence in the law of large numbers. However, to construct the coupling time, it is assumed in this prior work that the semi-group {P t } admits a density for each t. No such assumptions are required in the discrete-time setting, so the full answer to Q1 remains open.
Q2. Proof of Theorem Theorem 1.2. The copmplete resolution of Q2 is possible by applying Proposition 2.3, which implies that the skeleton chain {X(i) = Φ(i∆) : i ≥ 0} is f ∆ -regular. The bound (18) is the main ingredient in the proof of Theorem 1.2, but we also require the following relationship between a norm for the process and a norm for the sampled chain.
Proof. We first consider the right-hand side. Consider the signed measure Γ on [0, ∆] × X where the supremum is over all g satisfying |g(t, y)| ≤ f ∆ (t, y) for all t, y. It is shown next that the norm can be expressed, The Jordan decomposition theorem [2] implies that there is a minimal decomposition, Γ = Γ + − Γ − , in which the two measures on the right-hand side are non-negative, with disjoint supports denoted S + , S − , resoectively. Hence |Γ| := Γ + + Γ − is a non-negative measure. In this notation the norm is expressed, For each t, the measure on (X, B) defined by I S + (t, y) − I S − (t, y) µP t (dy) is the marginal of |Γ|, and is hence a non-negative measure for a.e. t. It follows that for such t, which gives (21). Consider next the left-hand side of the inequality in the lemma. Letting µ = µ + − µ − denote the Jordan decomposition for the signed measure µ, and |µ| = µ + + µ − , we have, µ f ∆ = f ∆ (x)|µ|(dx) = ∆ t=0 x∈X |µ|(dx)P t (x, dy)f (y).