Quantitative bounds for convergence rates of continuous time Markov processes

. We develop quantitative bounds on rates of convergence for continuous-time Markov processes on general state spaces. Our methods involve coupling and shift-coupling, and make use of minorization and drift conditions. In particular, we use auxiliary coupling to establish the existence of small (or pseudo-small) sets. We apply our method to some di(cid:11)usion examples. We are motivated by interest in the use of Langevin di(cid:11)usions for Monte Carlo simulation.


Introduction.
This paper concerns quantitative bounds on the time to stationarity of continuous time Markov processes, in particular diffusion processes.
Quantitative bounds for discrete time Markov chains have recently been studied by several authors, using drift conditions and minorization conditions (i.e. small sets) to establish couplings or shift-couplings for the chains. Bounding the distance of L(X t ) to stationarity for a fixed t has been considered by Meyn and Tweedie (1994), Rosenthal (1995), and Baxendale (1994). In Roberts and Rosenthal (1994), periodicity issues are avoided by instead bounding the distance of ergodic average laws t k=1 L(X k ) to stationarity. In each of these cases, quantitative bounds are obtained in terms of drift and minorization conditions for the chain.
In this paper we extend these results to honest (i.e. stochastic) continuous time Markov processes. We derive general bounds, using coupling and shift-coupling, which are similar to the discrete case (Section 2). Drift conditions are defined in terms of infinitesimal generators, and are fairly straightforward. However, the task of establishing minorization conditions is rather less clear, and this is the heart of our paper. We approach this problem for both one-dimensional (Section 3) and multi-dimensional (Section 4) diffusions, by producing an auxiliary coupling of certain processes started at different points of the proposed small set, which has probability > 0 of being successful by some fixed time t 0 . This implies (Theorem 7) the existence of a corresponding minorization condition.
Our construction relies on the use of "medium sets" on which the drifts of the diffusions remain bounded. It makes use of the Bachelier-Lévy formula (Lévy, 1965;Lerche, 1986) to lower-bound the probability of coupling.
For one-dimensional diffusions, we are able to simultaneously couple an entire collection of processes, started from each point of the proposed small set. However, for multidimensional diffusions, this is not possible. Instead, we couple the processes pairwise, thus establishing the existence of a pseudo-small set (Section 4) rather than an actual small set. We show that we can still use such a construction to obtain a coupling, even though there is no longer a regenerative structure. This suggests a certain advantage to studying minorization conditions through the use of coupling (e.g. Nummelin, 1992, Section III.10; Rosenthal, 1995) rather than the use of regeneration times as is often done (e.g. Athreya and Ney, 1978;Nummelin, 1984;Asmussen, 1987).
We apply our results to some examples of Langevin diffusions in Section 5.
In discrete time, studies of quantitative convergence rates have been motivated largely by Markov chain Monte Carlo algorithms (see Gelfand and Smith, 1990;Smith and Roberts, 1993). The current study is motivated partially by recent work (Grenander and Miller, 1994;Philips and Smith, 1994;Roberts and Tweedie, 1995;Roberts and Rosenthal, 1995) considering the use of Langevin diffusions for Monte Carlo simulations.

Bounds involving minorization and drift conditions.
We begin with some general results related to convergence rates of positive recurrent, continuous time Markov processes to stationarity.
Let P t (x, ·) be the transition probabilities for a Markov process on a general state space X . We say that a subset C ⊆ X is (t, )-small, for a positive time t and > 0, if there exists a probability measure Q(·) on X , satisfying the minorization condition The subset C is small if it is (t, )-small for some positive t and . (For background on small sets and the related notion of Harris chains, see Nummelin, 1978;Meyn and Tweedie, 1993b;Asmussen, 1987, pp. 150-158;Lindvall, 1992, pp. 91-92.) The advantage of small sets, for our purposes, is that they can be used (together with information about the return times to C) to establish couplings and shift-couplings of Markov processes, leading to bounds on convergence rates of processes to their stationary distributions. This is a well-studied idea; for extensive background, see Nummelin (1992, pp. 91-98).
For simplicity, we begin with general results related to shift-coupling, which provides for bounds on the ergodic averages of distances to stationary distributions. (Corresponding results for ordinary coupling are considered in Theorem 3 and Corollary 4.) These results are analogous to the discrete time results of Roberts and Rosenthal (1994). They bound the total variation distance between various probability measures, defined by The proofs are deferred until after the statements of all of the initial results.
Suppose further that there is δ > 0 and a non-negative function V : where τ C is the first hitting time of C. Set A = sup x∈C E x (V (X t * )), and assume that A < ∞.
Then for t > 0, and for any 0 < r < 1/t * with Ae δt * −δ/r < 1, It can be difficult to directly establish bounds on return times such as (2). It is often easier to establish drift conditions. Thus, we now consider using drift conditions and generators to imply (2). We require some notation. We let A be the weak generator of our Markov process, and let D be the "one-sided extended domain" consisting of all functions U : X → R which satisfy the one-sided Dynkin's formula This formula holds with equality if U is in the domain of the strong generator; furthermore, it holds with inequality if U is in the domains of certain stopped versions of the diffusion.
For background and discussion see, e.g., Meyn and Tweedie (1993a). For smooth functions in concrete examples, Dynkin's formula can be verified directly using Itô's formula and the dominated convergence theorem, as we do in Section 5.
Corollary 2. Suppose (1) holds, and in addition there is a function U ∈ D with U(x) ≥ 1 for all x ∈ X , and δ > 0 and Λ < ∞, such that Then (2) holds for this δ, with V = U, and hence the conclusion of Theorem 1 holds with Remark. Under the hypothesis of Corollary 2 (or similarly of Corollary 4 below), we can also derive a bound on E π (U). Indeed, we have that Integrating both sides with respect to π(dx) and using the stationarity of π, we obtain that 0 ≤ −δE π (U) + Λ, so that This may be of help in cases where the stationary distribution π(·) is too complicated for direct computation, as is often the case in the Markov chain Monte Carlo context.
We now consider results which directly bound L(X t ) − π(·) , rather than bounding ergodic averages. The statements are similar, except that the return-time conditions are more involved since we now need two copies of the process to simultaneously return to C.
These results are analogous to the discrete-time results of Rosenthal (1995).
Suppose further that there is δ > 0 and a non-negative function h : where τ C×C is the first hitting time of C × C.
where {X t } and {Y t } are defined jointly as described in the proof, and assume that A < ∞.
Then for t > 0, and for any 0 < r < 1/t * , As before, it can be hard to establish (3) directly, and it is thus desirable to relate this bound to a drift condition.
Corollary 4. Suppose (1) holds, and in addition there is a function U ∈ D with U(x) ≥ 1 for all x ∈ X , and λ > 0 and Λ < ∞, such that Hence the conclusion of Theorem 3 holds with these values. Furthermore, we We now proceed to the proofs of these results. Recall (see e.g. Lindvall, 1992) that T is a coupling time for {X t } and {Y t } if the two processes can be defined jointly so that Similarly, following Aldous and Thorisson (1993), we define T and T to be shift- To make use of shift-coupling, we use the following elementary result. (For a proof, and additional background on shiftcoupling, see Thorisson, 1992, equation 10.2;Thorisson, 1993;Thorisson, 1994;Roberts and Rosenthal, 1994.) Proposition 5. Let {X t }, {Y t } be continuous-time Markov processes, each with transition probabilities P (x, ·), and let T and T be shift-coupling epochs for {X t } and {Y t }.
Then the total variation distance between ergodic averages of L({X t }) and L({Y t }) satisfies The key computation for the proofs of Theorems 1 and 3 is contained in the following lemma, which is a straightforward consequence of regeneration theory (cf. Nummelin, 1992, p. 92; see also Athreya and Ney, 1978;Asmussen, 1987;Meyn and Tweedie, 1994;Rosenthal, 1995). Lemma 6. With notation and assumptions as in Theorem 1, there is a random stopping time T with L(X T ) = Q(·), such that for any 0 < r < 1/t * and s > 0,

Proof.
We construct {X t } as follows. We begin by choosing q ∼ Q(·), and letting Let τ 1 be the first time the process {X t } is in the set C.
). In either case, we then fill in X t for τ 1 < t < τ 1 + t * from the appropriate conditional distributions. Similarly, for i ≥ 2, we let τ i be the first time ≥ τ i−1 + t * at which the process {X t } is in the set C, and again if ). This defines the process {X t } for all times t ≥ 0, such that {X t } follows the transition probabilities P t (x, ·).
To proceed, we define N t = max{i; τ i ≤ t}. Now, for any 0 < r < 1 t * , However, setting D 1 = τ 1 and D i = τ i − τ i−1 for i ≥ 2, for any positive integer j, we have from Markov's inequality that Moreover, from (2), as in Rosenthal (1995), we have Hence, as required.
Proof of Theorem 1.
We augment the process {X t } with a second process {Y t }, also following the transition probabilities P t (x, ·), but with L(Y 0 ) = π(·), so that P (Y s ∈ ·) = π(·) for all times s ≥ 0. From the lemma, there are times T and T with L(X T ) = L(Y T ) = Q(·). We define the two processes jointly in the obvious way so that X T = Y T .
We complete the construction by re-defining Y s for s > T , by the formula Y T +t = X T +t for t > 0. It is easily seen that {Y s } still follows the transition probabilities P t (x, ·). The times T and T are then shift-coupling epochs as defined above. The lemma gives upper bounds on P (T > s) and P (T > s). Hence, integrating, with a similar formula for P (T > s)ds. The result then follows from Proposition 5.
Proof of Corollary 2. The statement about (2) follows directly by a standard martingale argument, as in Rosenthal (1995). Specifically, e δ(t∧τ C ) U(X t∧τ C ) is a non-negative local supermartingale, so that For the statement about A, let E(t, x) = E x (U(X t )). Then Therefore, E(t, x) ≤ W (t, x) for all t and x, where with W (0, x) = E(0, x) for all x ∈ X . Hence, solving, we have The result follows by taking t = t * and taking supremum over x ∈ X .
Proof of Theorem 3. We construct the processes {X t } and {Y t } jointly as follows.
We begin by choosing q ∼ Q(·). We further set t 0 = inf{t ≥ 0 ; (X t , Y t ) ∈ C × C}, and t n = inf{t ≥ t n−1 + t * ; (X t , Y t ) ∈ C × C} for n ≥ 1. Then, for each time t i , if we have not yet coupled, then we proceed as follows. With probability we set X t i +t * = Y t i +t * = q, set T = t i + t * , and declare the processes to have coupled. Otherwise (with probability 1 − , conditionally independently. In either case, we then fill in the values X s and Y s for t i < s < t i + t * conditionally independently, using the correct conditional distributions given Finally, if {X t } and {Y t } have already coupled, then we let them proceed conditionally independently. It is easily seen that {X t } and {Y t } each marginally follow the transition probabilities P t (·, ·). Furthermore, T is a coupling time. We bound P (T > s) as in Lemma 6 above.
The result then follows from the coupling inequality.
Proof of Corollary 4. Setting h(x, y) = 1 2 (U(x) + U(y)), we compute that Statement (3), and also the statement about A, then follow just as in Corollary 2.

Computing minorizations for one-dimensional diffusions.
To use Theorem 1 or Theorem 3, it is necessary to establish drift and minorization conditions. Consideration of the action of the generator A on test functions U provides a method of establishing drift conditions. However, finding a minorization condition appears to be more difficult. In this section, we present a method for doing so.
We shall have considerable occasion to estimate first hitting time distributions of one-dimensional diffusions. A key quantity is provided by the Bachelier-Lévy formula (Lévy, 1965;Lerche, 1986): Suppose {Z t } is a Brownian motion with drift µ and diffusion coefficient σ, defined by dZ t = µdt + σdB t , started at Z 0 = 0. Then if T x is the first time that {Z t } hits a point x > 0, then Assume that µ(·) is C 1 , and that there exist positive constants a, b, N such that sgn(x) µ(x) ≤ a|x| + b whenever |x| ≥ N. This implies (see for example Roberts and Tweedie, 1995, Theorem 2.1) that the diffusion is non-explosive. Furthermore, it is straightforward to check that this diffusion has a unique strong solution.
The set S is medium if it is [c, d]-medium for some c < d. Note that if µ(·) is a continuous function, then all compact sets are medium. On medium sets we have some control over the drift of the process; this will allow us to get bounds to help establish minorization conditions.
Theorem 7. Let {X t } be a one-dimensional diffusion process defined by dX t = µ(X t )dt+ dB t . Suppose C = [α, β] is a finite interval. Suppose further that S = [a, b] is an interval containing C, which is [c, d]-medium. Then for any t > 0, there exists an > 0 such that C is (t, )-small. Moreover, given any t 0 > 0, for all t ≥ t 0 , we have that C is (t, )-small Proof. Given a standard Brownian motion {B t }, we simultaneously construct, for each x ∈ C, a version {X x t } of X t started at x, and defined by where τ Set r = P(τ α β ≤ t 0 ), and for t ≥ t 0 let Q t (·) = P(X α t ∈ · | τ α β ≤ t) .
Then it follows by construction that P t (x, ·) ≥ r Q t (·) , x∈ C , as desired.
It remains to compute a lower bound for r. We let Z t = X β t − X α t , and let U t satisfy the S.D.E.
for the same Brownian motion {B t } as before, with U 0 = β − α. We let T 0 be the first Furthermore, defining processes {V 1 t } and {V 2 t } by The result now follows from applying the Bachelier-Lévy formula to each of these three probabilities.

Multi-dimensional diffusions and pseudo-small sets.
In this section we consider k-dimensional diffusions {X t } defined by dX t = µ(X t )dt + dB t . Here B is standard k-dimensional Brownian motion, and X t , µ(·) ∈ R k . As before, we assume that µ(·) is C 1 , and that there exist positive constants a, b, N such that n x · µ(x) ≤ a x 2 + b whenever x 2 ≥ N. This again implies (see for example Roberts and Tweedie, 1995, Theorem 2.1) that the diffusion is non-explosive; and again the diffusion has a unique strong solution.
It may appear that the method of the previous section is quite specific to onedimension. There, diffusions were jointly constructed so that they had probability > 0 of coupling by time t * , and this produced the required minorization condition. However, in multi-dimensions, it may be possible for the two diffusions to "miss" each other, so that bounding the probability of their coupling would not be easy.
It is possible to get around this difficulty by considering what might be called "rotating axes". Specifically, we shall break up the two processes into components parallel and perpendicular to the direction of their difference. We shall define them so that they proceed identically in the perpendicular direction, but are perfectly negatively correlated in the parallel direction. Thus, they will have a useful positive probability of coupling in finite time t * . (For other approaches to coupling multi-dimensional processes in various contexts, see Lindvall and Rogers, 1986;Davies, 1986;Chen and Li, 1989;Lindvall, 1992, Chapter VI.) However, it is no longer possible to use such a construction to simultaneously couple all of the processes started from different points of a proposed small set. This is because the parallel and perpendicular axes are different for different pairs of processes. Instead, we shall couple the processes pairwise only. This does not establish the existence of a small set, however it does establish the existence of a pseudo-small set as we define now.

Definition.
A subset S ⊆ X is (t, )-pseudo-small for a Markov process P t (·, ·) on a state space X if, for all x, y ∈ S, there exists a probability measure Q xy (·) on X , such that P t (x, ·) ≥ Q xy (·) and P t (y, ·) ≥ Q xy (·) .
That pseudo-small sets are useful for our purposes is given by the following theorem, whose statement and proof are very similar to Theorem 3, but which substitutes pseudosmallness for smallness.
Theorem 8. Given a Markov process with transition probabilities P t (x, ·) and stationary distribution π(·), suppose C ⊆ X is (t * , )-pseudo-small, for some positive time t * , and > 0. Suppose further that there is δ > 0 and a non-negative function h : X × X → R with h(x, y) ≥ 1 for all x ∈ X , such that where {X t } and {Y t } are defined jointly as described in the proof, and assume that A < ∞.

Proof.
We construct the processes {X t } and {Y t } jointly as follows. We first set T = ∞. Then, each time (X t , Y t ) ∈ C × C and T = ∞, with probability we choose ·) and set T = t + t * , while with probability 1 − we choose and Y t+t * ∼ P t * (Y t , ·), conditionally independently.
Having chosen X t+t * and Y t+t * , we then fill in the values of X s and Y s for t < s < t+t * conditionally independently, with the correct conditional distributions given X t , Y t , X t+t * , and Y t+t * .
It is easily seen that {X t } and {Y t } each marginally follow the transition probabilities P t (·, ·). Furthermore, T is a coupling time. We bound P (T > s) as in Lemma 6 above.
The result then follows from the coupling inequality.
We now turn our attention to the multi-dimensional diffusions. As in the onedimensional case, it is necessary to restrict attention to events where the processes remain in some medium set on which their drifts are bounded. Here, for c, d ∈ R k , we say a set [α i , β i ], and let D = sup x,y∈C x − y 2 be the . Then for any t > 0, there exists an > 0 such that C is (t, )-pseudo-small. Moreover, given any t 0 > 0, for all The above estimates are clearly based again upon the Bachelier-Lévy formula; in fact many aspects of this proof are similar to those of Theorem 7. In some special cases the estimates can be improved upon considerably; the method of proof will indicate where improvements might be possible.
Proof. Given a standard k-dimensional Brownian motion {B s }, and a pair of points x, y ∈ C, we define diffusions X x and X y simultaneously by X x 0 = x, X y 0 = y, dX y s = µ(X y s )ds + dB s , and dX x s = µ(X x s )ds + dB s − 2(n s · dB s )1 τ ≥s n s , where τ denotes the first time that X x and X y coincide, and n s denotes the unit vector in the direction from X x s to X y s . This has the effect that X x and X y proceed identically tangentially to the axis joining them, but are perfectly negatively correlated in the direction parallel to this axis. The processes X x and X y coalesce on coincidence. (Note that, unlike in the one-dimensional case, this intricate construction is necessary to ensure that coupling is achieved with positive probability.) Hence, if r = P(τ ≤ t 0 ), then for t ≥ t 0 we have P t (x, ·) ≥ r Q t (·) and P t (y, ·) ≥ r Q t (·), exactly as in Theorem 7. We now proceed to lower-bound r.
If we set Z s = X y s − X x s , then by a simple application of Itô's formula, and at least up until the first time one of the X x processes exits S, we have that where U s is the one-dimensional diffusion defined by U 0 = D and dU s = 2 dB s + sgn(U s ) L ds , whereB s = B s · n s is a standard one-dimensional Brownian motion. Now note that, if A = {X x s ∈ S and X y s ∈ S, for 0 ≤ s ≤ τ }, then We compute the first of these two probabilities using the Bachelier-Lévy formula, exactly as in Theorem 7. We bound the second by writing P s,i ∈ S and X y s,i ∈ S, for 0 ≤ s ≤ τ } (here X x s,i is the i th component of X x s ), and computing each P (A i ) similarly to Theorem 7 (after first redefining µ(x) = c for x ∈ S, so that we can assume c i ≤ µ i (x) ≤ d i for all x ∈ X ). (The extra factor of 2 is required because either of X x and X y can now escape from either side; orderings like X x s,i ≤ X y s,i need not be preserved.) The result follows.

Examples: Langevin diffusions.
In this section, we consider applying our results to some simple examples of Langevin diffusions. A Langevin diffusion requires a probability distribution π(·) on R k , having C 1 density f(·) with respect to Lebesgue measure. It is defined by the S.D.E.
Such a diffusion is reversible with respect to π(·), which is thus a stationary distribution (intuitively, the diffusion is more likely to proceed in the direction where f is increasing).
This fact has been exploited to use Langevin diffusions for Monte Carlo simulation (see Grenander and Miller, 1994;Philips and Smith, 1994;Roberts and Tweedie, 1995;Roberts and Rosenthal, 1995).
This is L 2 -bounded on compact time intervals, so by the dominated convergence theorem, M(t) is in fact a martingale. Thus U ∈ D.
The main point is that Theorem 7 and Corollary 4 provide us with concrete, quantitative bounds on the distance of L(X t ) to the stationary distribution.
Remark. Multivariate normal distributions. In R k , if f(·) is the density for a multivariate normal distribution, then we can get similar bounds without any additional work.
Indeed, an appropriate linear transformation reduces this case to that of a standard multivariate normal. From there, we inductively use the inequality µ 1 × µ 2 − ν 1 × ν 2 ≤ µ 1 − ν 1 + µ 2 − ν 2 , to reduce this to the one-dimensional case. Thus, we obtain bounds identical to the above, except multiplied by a global factor of k.
We choose the function U(x, y) = 1 + x 2 + y 2 + x 2 y 2 , and compute that AU(x, y) = 2 + x 2 + y 2 − 2x 2 (1 + y 2 ) 2 − 2y 2 (1 + x 2 ) 2 . As in the previous example, we set Again, by Itô's lemma, M(t) is a local martingale. Furthermore, by comparison to two independent Ornstein-Uhlenbeck processes, we see that M(t) is again L 2 -bounded on compact time intervals, so that by dominated convergence M(t) is again a martingale. Hence again U ∈ D.
We let C be the square with corners (±β, ±β) and let S be the square with corners (±b, ±b). For fixed λ > 0 (to be chosen later to optimize the results), we choose β = 2+λ 1−λ