Convergence rates of the random scan Gibbs sampler under the Dobrushin’s uniqueness condition

In this paper, under the Dobrushin’s uniqueness condition, we obtain explicit esti-mates of the geometrical convergence rate for the random scan Gibbs sampler in the Wasserstein metric.

In order to approximate µ via iterations of the one-dimensional conditional distributions µ i , i = 1, · · · , N, the various scan Gibbs samplers are often used (see [4]). In [6], Wu and the author studied systematic scan Gibbs sampler by Dobrushin's uniqueness conditions. In this paper, we will study the random scan Gibbs sampler.
The scheme of the random scan Gibbs sampler approximating µ is that, in each iteration, one randomly chooses one coordinate to update according to the one-dimensional conditional distributions µ i , i = 1, · · · , N. It is described as follows. Given any initial value X 0 = (X 1 0 , · · · , X N 0 ) ∈ E N , independently draw an index σ from the uniform distribution on the index set {1, · · · , N }, then draw X σ 1 from µ σ (·|x) and take X i 1 = X i 0 , i = σ, completing one iteration of the scheme. After m such iterations, we obtain X m = (X 1 m , · · · , X N m ). Thus this scan Gibbs sampler is exactly the time-homogeneous Markov chain {X m : m = 0, 1, · · · } with invariant distribution µ, whose one step transition probability P (x, dy) = 1 Our objective is to study the convergence rate of the m−step transition probability P m to µ under Dobrushin's uniqueness conditions as m tends to ∞. To this end, by coupling, our main idea is to establish some contractive properties in the sense of the maximum or sum distance (respectively, see the two lemmas in Section 3). Although the coupling is similar to [6], to prove the contractive properties is very different because of this scan Gibbs sampler with the random index σ instead of the systematic scan in [6]. This paper is organized as follows. We present the main results in Section 2, and then prove them in Section 3.

Main results
Throughout the paper E is a Polish space with the Borel σ-field B, and d is a metric on E such that d(·, ·) is lower semi-continuous on E 2 . On the product space E N , we consider the l N 1 -metric The product space E N is always endowed with the d l N 1 -metric unless otherwise stated.
Let M 1 (E) be the space of Borel probability measures on E, and (Here x 0 ∈ E is some fixed point, but the definition above does not depend on x 0 by the triangle inequality). Given ν 1 , ν 2 ∈ M d 1 (E), the L 1 -Wasserstein distance between ν 1 , ν 2 is given by d(x, y)π(dx, dy), where the infimum is taken over all probability measures π on E × E such that its marginal distributions are respectively ν 1 and ν 2 (called a coupling of ν 1 and ν 2 ).
Recall the Kantorovich-Rubinstein duality relation ( [5]) Throughout the paper we assume that Define the matrix of the d-Dobrushin interdependence coefficients C := (c ij ) i,j=1,··· ,N as Obviously c ii = 0. By the triangle inequality of the metric W 1,d , for any x, y ∈ E N , Then the well known Dobrushin uniqueness condition (see [1,2] or [6]) is read as Notice that r ∞ (or r 1 ) coincides with the operator norm of the N by N matrix C acting as an operator from l N ∞ (or l N 1 respectively) to itself.

Remark 2.4. As indicated by a referee, it is especially relevant for statistical applications to investigate exponential concentration inequalities for the convergence of empirical means 1
n n i=1 f (X i ) to E N f dµ. But regretfully unlike the systematic scan sampler of [6], because the current sampler has random index σ, we don't succeed in establishing those concentration inequalities under the Dobrushin's uniqueness condition.

Proofs of the main results
Given any two initial distributions ν 1 and ν 2 on E N , we construct our coupled homogeneous Markov chain (X m , Y m ) m≥0 , which is quite close to the coupling by K. Marton [3] (see also [6]).
Then we have: where the last inequality above holds because of (2.4). And thus Because the inequalities hold for any i ≥ 1, m ≥ 1, and by induction, (a) For any Lipschitzian function f : E N → R, by Lemma 3.1, δ i (f ).