Quasi stationary distributions and Fleming-Viot processes in countable spaces

We consider an irreducible pure jump Markov process with rates Q=(q(x,y)) on \Lambda\cup\{0\} with \Lambda countable and 0 an absorbing state. A quasi-stationary distribution (qsd) is a probability measure \nu on \Lambda that satisfies: starting with \nu, the conditional distribution at time t, given that at time t the process has not been absorbed, is still \nu. That is, \nu(x) = \nu P_t(x)/(\sum_{y\in\Lambda}\nu P_t(y)), with P_t the transition probabilities for the process with rates Q. A Fleming-Viot (fv) process is a system of N particles moving in \Lambda. Each particle moves independently with rates Q until it hits the absorbing state 0; but then instantaneously chooses one of the N-1 particles remaining in \Lambda and jumps to its position. Between absorptions each particle moves with rates Q independently. Under the condition \alpha:=\sum_x\inf Q(\cdot,x)>\sup Q(\cdot,0):=C we prove existence of qsd for Q; uniqueness has been proven by Jacka and Roberts. When \alpha>0 the {\fv} process is ergodic for each N. Under \alpha>C the mean normalized densities of the fv unique stationary measure converge to the qsd of Q, as N \to \infty; in this limit the variances vanish.


Introduction
Let Λ be a countable set and Z t be a pure jump regular Markov process on Λ ∪ {0} with transition rates matrix Q = (q(x, y)), transition probabilities P t (x, y) and with absorbing state 0; that is q(0, x) = 0 for all x ∈ Λ. Assume that the exit rates are uniformly bounded above:q := sup x y∈{0}∪Λ\{x} q(x, y) < ∞, that P t (x, y) > 0 for all x, y ∈ Λ and t > 0 and that the absorption time is almost surely finite for any initial state. The process Z t is ergodic with a unique invariant measure δ 0 , the measure concentrating mass in the state 0. Let µ be a probability on Λ. The law of the process at time t starting with µ conditioned to non absorption until time t is given by ϕ µ t (x) = y∈Λ µ(y)P t (y, x) 1 − y∈Λ µ(y)P t (y, 0) , x ∈ Λ. (1.1) A quasi stationary distribution (qsd) is a probability measure ν on Λ satisfying ϕ ν t = ν. Since P t is honest and satisfies the forward Kolmogorov equations we can use an equivalent definition of qsd, according Nair and Pollett [12]. Namely, a qsd ( and only a qsd) is a left eigenvector ν for the restriction of the matrix Q to Λ with eigenvalue − y∈Λ ν(y)q(y, 0): ν must satisfy the system y∈Λ ν(y) [q(y, x) + q(y, 0)ν(x)] = 0, ∀x ∈ Λ.
When Λ is finite, Darroch and Seneta (1967) prove that there exists a unique qsd ν for Q and that the Yaglom limit converges to ν independently of the initial distribution. When Λ is infinite the situation is more complex. Neither existence nor uniqueness of qsd are guaranteed. An example is the asymmetric random walk p = q(i, i + 1) = 1 − q(i, i − 1), for i ≥ 0. In this case there are infinitely many qsd when p < 1/2 and none when p ≥ 1/2 (see Cavender [2] and Ferrari, Martinez and Picco [6] for birth and death more general examples). For Λ = N under the condition lim x→∞ P(R > t|Z 0 = x) = 0, where R is the absorption time of Z t , Ferrari, Kesten, Martínez and Picco [5] prove that the existence of qsd is equivalent to the existence of a positive exponential moment for R, i.e. Ee θR < ∞ for some θ > 0. When the Yaglom limit exists, it is known it is a qsd, but existence of the limit is not known in general for infinite state space. Phil Pollett maintains an updated bibliography on qsd in the site http://www.maths.uq.edu.au/˜pkp/papers/qsds/qsds.html.
Define the ergodicity coefficient of the chain Q by Since the chain is absorbed with probability one, C > 0. On the other hand, C <q, the maximal rate.
Jacka and Roberts [9] proved that if there exists a Doeblin state z ∈ Λ such that α(z) > C and if there exists a qsd ν for Q, then ν is the unique qsd for Q and the Yaglom limit converges to ν for any initial measure µ; their proof also works under the weaker assumption α > C. We show that α > C is a sufficient condition for the existence of a qsd for Q. Theorem 1.1. If α > C then there exists a unique qsd ν for Q and the Yaglom limit converges to ν for any initial measure µ.
The condition α > C is disjoint to the condition lim x P(R > t|Z 0 = x) = 0, under which [5] show existence of qsd. On the other hand, α > 0 implies that R has a positive exponential moment.
The Fleming-Viot process (fv). Let N be a positive integer and consider a system of N particles evolving on Λ. The particles move independently, each of them governed by the transition rates Q until absorption. At most one particle is absorbed at any given time. When a particle is absorbed, it returns instantaneously to a state in Λ chosen with the empirical distribution of the particles remaining in Λ. In other words, it chooses one of the other particles uniformly and jumps to its position. Between absorption times the particles move independently governed by Q. This process has been introduced by Fleming and Viot [7] and studied by Burdzy, Holyst and March [1], Grigorescu and Kang [8] and Löbus [11] in a Brownian motion setting. The generator acts on functions f : Λ (1,...,N ) → R as follows where ξ i,y (j) = y for j = i and ξ i,y (j) = ξ(j) otherwise and We call ξ t the process in Λ (1,...,N ) with generator (1.6) and η t = η(ξ t , ·) the corresponding unlabeled process on N Λ ; η t (x) counts the number of ξ particles in state x at time t. For µ a measure on Λ, we denote ξ N,µ t the process starting with independent identically µdistributed random variables (ξ N,µ 0 (i), i = 1, . . . , N); the corresponding variables η N,µ 0 (x) have multinomial law with parameters N and (µ(x), x ∈ Λ). The profile of the fv process at time t converges as N → ∞ to the conditioned evolution of the chain Z t : Theorem 1.2. Let µ be a probability measure on Λ. Assume that (ξ N,µ 0 (i), i = 1, . . . , N) are iid with law µ. Then, for t > 0 and x ∈ Λ, The convergence in probability has been proven for Brownian motions in a compact domain in [1]. Extensions of this result and the process induced in the boundary have been studied in [8] and [11].
When Λ is finite, the fv process is an irreducible pure-jump Markov process on a finite state space. Hence it is ergodic (that is, there exists a unique stationary measure for the process and starting from any measure, the process converges to the stationary measure). When Λ is infinite, general conditions for ergodicity are not still established. We prove the following result Theorem 1.3. If α > 0, then for each N the fv process with N particles is ergodic.
Assume α > 0. Let η N be a random configuration distributed with the unique invariant measure for the fv process with N particles. Our next result says that the empiric profile of the invariant measure for the fv process converges in L 2 to the unique qsd for Q. Theorem 1.4. Assume α > C. Then there exists a probability measure ν on Λ such that for all x ∈ Λ, Furthermore ν is the unique qsd for Q.
Sketch of proofs The existence part of Theorem 1.1 is a corollary of Theorem 1.4. The rest is a consequence of Jacka and Robert's theorem (stated later as Theorem 5.1). Theorem 1.3 is proven by constructing a stationary version of the process "from the past" as in perfect simulation. We do it in Section 2.
From a generator computation, taking f (ξ) = η(ξ, x) in (1.6), If solutions of (1.11) converge along subsequences as N → ∞, then the limits equal the unique solution of (1.10). In fact, we prove in Proposition 3.1 that for x, y ∈ Λ, This argument shows the convergence of the means Eη N,µ t (x)/N to ϕ µ t (x). Since the variances ((1.12), with x = y) divided by N 2 go to zero, the L 2 convergence follows.
The stationary case is proven analogously. If η N is distributed with the invariant measure for the fv process, from (1.11), Under the hypothesis α > C we show a result for η N analogous to (1.12) to conclude that solutions of (1.13) converge to the unique solution of (1.2).
To show that the limits are probability measures it is necessary to show that the families of measures ( 1 N Eη N,µ t , N ∈ N) and ( 1 N Eη N , N ∈ N) are tight; we do it in Section 4.
Comments One interesting point of the Fleming-Viot approach is that it permits to show the existence of a qsd in the α > C case, a new result as far as we know.
Compared with the results for Brownian motion in a bounded region with absorbing boundary (Burdzy, Holyst and March [1], Grigorescu and Kang [8] and Löbus [11] and other related works), we do not have trouble with the existence of the fv process, it is immediate here. On the other hand those works prove the convergence in probability without computing the correlations. We prove that the fact that the correlations vanish asymptotically is sufficient to show convergence in probability. For the moment we are able to show that the correlations vanish for the stationary state under the hypothesis α > C.
To prove tightness we have classified the ξ particles in types. This already appears in Burdzy, Holyst and March [1] to show the convergence result. Our application here is somehow simpler. Curiously our tightness proof needs the same condition (α > C) as the vanishing correlations proof.

Construction of fv process
In this section we perform the graphic construction of the fv process ξ N t . Recall C < ∞ and α ≥ 0. Recall α(z) = inf x∈Λ\{z} q(x, z).
For each i = 1, . . . , N, we define independent stationary marked Poisson processes (PP's) on R: • Regeneration times. PP rate α: The marks are independent of the PP's and mutually independent. The denominations will be transparent later. The marginal laws of the marks are: Denote (Ω, F , P) the space where the product of the marked Poisson processes has been constructed. Discard the null event corresponding to two simultaneous events at any given time.
We construct the process in an arbitrary time interval [s, t]. Given the mark config- as a function of the Poisson times and its respective marks and the initial configuration ξ at time s. The relation of this notation with the one in Theorem 1.2 is the following: (2.14) where X = (X 1 , . . . , X N ) is a random vector with iid coordinates, each distributed according to µ on Λ. That is, for any function f : Λ N → R, Since for each particle i there are three Poisson processes with rates C, α andq − α, the number of events in the interval [s, t] is Poisson with mean N(C +q). So the events can be ordered from the earliest to the latest.
At time s the initial configuration is ξ. Then, proceed event by event following the order as follows: The configuration does not change between Poisson events.
At each regeneration time a i n particle i jumps to state A i n regardless the current configuration.
If at the internal time b i n − the state of particle i is x, then at time b i n particle i jumps to state B i n (x) regardless the position of the other particles. If at the voter time c i n − the state of particle i is x and F i n (x) = 1, then at time c i n particle i jumps to the state of particle C i n ; if F i n (x) = 0, then particle i does not jump. The configuration obtained after using all events is ξ N,ξ [s,t] . The denominations are now clear. At regeneration times a particle jumps to a new state independently of the current configuration. At voter times a particle either jumps to the state of another particle chosen at random or does not jump. At internal times the particle jumps are indifferent to the position of the other particles. Proof Follows from the Markov properties of the Poisson processes; the rate for particle i to jump from x to y is the sum of three terms: (a) α α(y) α (the rate of a regeneration event times the probability that the corresponding mark takes the value y), (b) (q − α) q(x,y)−α(y) q−α (the maximal rate of internal events times the probability that the corresponding mark takes the value y) and (c) C q(x,0) C j =i 1{ξ(j) = y} 1 N −1 (the maximal absorption rate times the probability the absorption rate from state x divided by the maximal absorption rate times the empirical probability of state y for the particles different from i). The sum of these three rates is the rate indicated by the generator (the square brackets in (1.6) with ξ(i) = x). ,ω (i). We drop the label ω in the notation. Initially Ψ i [t, t] = {i} and look backwards in time for the more recent i-Poisson event at some time τ in the past of t but more recent than s. If τ is a regeneration event, then we don't need to go further in the past to know the state of the i particle, so we erase the i particle from Ψ i [τ −, t]. If τ is the voter event c i n , its C i n mark pointing to particle j, say, then we need to know the state of the particle i at time τ − to see which F i n will be used to decide if the i particle effectively takes the value of particle j or not. Hence, we need to follow backwards particles i and j and we add the j particle to Ψ i [τ −, t]. Then continue this procedure starting from each of the particles in Ψ i [τ −, t]. The process backwards finishes if Ψ i [r, t] is empty for some r smaller than s or if we have processed all marks involving i in the time interval [s, t]. More rigorously: We construct Ψ i [s, t] backwards in time. Changes occur at Poisson events and Ψ i [s, t] is constant between two Poisson events. The construction of Ψ i [s, t] depends only on the regeneration and voter events. It ignores the internal events.
Assume Ψ i [r ′ , t] has been constructed for all r ′ ∈ [τ ′ , t]. Let τ be the time of the latest Poisson event before τ ′ .
If not, proceed as follows.
If τ is a regeneration event involving particle j (that is, τ = a j n for some n), then set If τ is a voter event whose mark points to particle j (that is, τ = c j ′ n for some j ′ and n and C j ′ n = j), then set Ψ i [τ, t] = Ψ i [τ ′ , t] ∪ {j}. This ends the iterative step of the construction. is well defined with probability one and does not depend on ξ. By construction (ξ N t , t ∈ R) is a stationary Markov process with generator (1.6). Since the law at time t does not depend on the initial configuration ξ, the process admits a unique invariant measure, the law of ξ N t . See [4] for more details about this argument. The number of points in ω i [−∞, t] is finite if and only if for some finite s < t, Ψ i [s, t] = ∅. But since there are 3N stationary finite-intensity Poisson processes, with probability one, for almost all ω there is an interval [s(ω), s(ω) + 1] in the past of t such that there is at least one regeneration mark for all particle k and there are no voter marks in that interval. We have used here that the regeneration rate α > 0. This guarantees that Ψ

Particle correlations in the fv process
In this section we show that the particle-particle correlations in the fv process with N particles is of the order of 1/N.

.20)
Assume α > C. Let η N be distributed according to the unique invariant measure for the fv process with N particles. Then We introduce a 4-fold coupling and such that the marginal process (Ψ i [s, t],Ψ j [s, t]) have the same law as two independent processes with the same marginals as (Ψ i [s, t], Ψ j [s, t]). The construction is analogous to the one in Fernández, Ferrari and Garcia [4].
We use two independent families of marked Poisson processes each with the same law as the Poisson family used in the graphic construction; the marked events are called red and green. We augment the probability space and continue using P and E for the probability and the expectation with respect to the product space generated by the red and green events. With these marked events we construct simultaneously the processes (Ψ i [s, t], Ψ j [s, t],Ψ i [s, t],Ψ j [s, t]) and a new process I[s, t] as follows.
Go backwards in time as in the construction of Ψ i in Section 2 proceeding event by event as follows. andΨ i [s, t] respectively. Take two independent random vectors X and Y with the same distribution as in (2.14), that is, iid coordinates with law µ. Denote the initial particles defined as in (2.17

25)
Proof: At time s the process I[s, t] jumps from 0 to 1 at a rate depending onΨ i [s, t] and Ψ j [s, t] which is bounded above by Dominating the indicator function by one: where F [s,t] is the sigma field generated by ((Ψ i [s ′ , t],Ψ j [s ′ , t]), s < s ′ < t). From (3.26), using 1 − e −a ≤ a and taking expectations, On the other hand,Ψ i [s ′ , t] is dominated by the position at time t − s of a random walk that grows by one with rate C and decreases by one with rate α. Hence its expectation is bounded above by e (t−s ′ )(C−α) . Substituting this bound in (3.27), which gives (3.25).
If α > C, η N,η [s,t] converges as s → −∞ to η N t a configuration distributed with the unique invariant measure, as in Theorem 1.3, see (2.19) for the corresponding statement for ξ N t . Hence the left hand side of (3.21) is bounded above by P(I[−∞, t] = 1). Taking s = −∞ in (3.25) we get (3.21).

Tightness
In this section we prove tightness for the mean densities as probability measures in Λ, indexed by N.  Assume α > 0 and define the probability measure µ α on Λ by The matrix R λ represents the semigroup P t evaluated at a random time T λ exponentially distributed with rate λ independent of (Z t ). R λ (z, x) is the probability the process (Z z t ) be in x at time T λ . The matrix R is substochastic: x∈Λ R λ (z, x) is just the probability of non absorption of (Z z t ) at the random time T λ . Proposition 4.2. Assume α > C and let ρ N (x) be the mean proportion of particles in state x under the unique invariant measure for the fv process with N particles. Then for x ∈ Λ, As a consequence, the family of measures (ρ N , N ∈ N) is tight.
Types To prove the propositions we introduce the concept of types. We say that particle i is type 0 at time t if it has not been absorbed in the time interval [0, t]. Particles may change type only at absorption times. If at absorption time s particle i jumps over particle j which has type k, then at time s particle i changes its type to k + 1. Hence, at time t a particle has type k if at its last absorbing time it jumped over a particle of type k − 1. We write type(i, t):= type of particle i at time t.
The marginal law of ξ N,µ t (i)1{type(i, t) = 0} is the law of the process Z µ t : Proof of Proposition 4.1 Since We proceed by induction. By (4.4) the statement is true for k = 0. Assume (4.5) holds for some k ≥ 0. We prove it holds for k + 1. Time is partitioned according to the last absorption time s of the ith particle. The absorption occurs at rate bounded above by C.
The particle jumps at time s to a particle j with probability 1/(N − 1), this particle has type k and state y. Then it must go from y to x in the time interval [s, t] without being absorbed. Using the Markov property, we get: The symmetry of the particles allows to cancel the sum over j with (N −1) −1 . The recursive hypothesis (4.5) implies that (4.7) equals by Chapman-Kolmogorov. This completes the induction step.
Proof of Proposition 4.2 If ξ N is distributed according to the unique invariant measure for the fv process then ρ N = P(ξ N (i) = x). Since α > 0 we can construct a version of the stationary process ξ N s such that P(ξ N (i) = x) = P(ξ N s (i) = x), ∀s. We analyze the marginal law of the particle distribution for each type, as in the proof of Proposition 4.1. Define the types as before, but when a particle meets a regeneration mark, then the particle type is reset to 0. In the construction, at that time the state of the particle is chosen with law µ α .
Under the hypothesis α > C the process is Markovian and can be constructed in a stationary way as ξ N t . Hence does not depend on s.
The regeneration marks follow a Poisson process of rate α and the last regeneration mark of particle i before time s happened at time s − T i α , where T i α is exponential of rate α. Then, (4.10) A reasoning similar to (4.6)-(4.7) implies We interpret R k λ (z, x) as the expectation of P τ k (z, x), where τ k is a sum of k independent random variables with exponential distribution of rate λ. Summing (4.11), and multiplying and dividing by (α − C), The sum can be interpreted as the expectation of µ α R K α , where K is a geometric random variable with parameter p = 1 − (C/α). Since an independent geometric(p) number of independent exponentials(α) is exponential(αp), we get (4.14)

Proofs of theorems
In this section we prove Theorems 1.3 and 1.4. We start deriving the forward equations for ϕ µ t and show they have a unique solution.
Lemma 5.1. The Kolmogorov forwards equations for ϕ µ t are given by d dt These equations have a unique solution.
Sum and subtract y∈Λ q(y, 0)E( By Proposition 4.1, the family (Eη N,µ t (x)/N, n ∈ N) is tight. Call ρ µ t the limit along a convergent subsequence. Use q(x, y) <q, q(y, 0) < C and (4.1) to bound the summands in (5.7) and (5.8) by (q + C)e Ct µP t (x) and Ce Ct µP t (x), respectively and use dominated convergence to take the limits inside the sums. Proposition 3.1 implies (5.8) converges to zero as N goes to infinity for any subsequence. We conclude that the limit along a convergent subsequence must satisfy If F is a bounded function twice continuously differentiable and with uniformly bounded derivatives, then, is a martingale (see Kipnis and Landim (1999), for instance). We choose F (η N t ) = From (5.9) we conclude that any limit ρ µ t must satisfy ρ µ t (x) = ρ µ 0 (x) + t 0 y∈Λ ρ µ s (y)[q(y, x) + q(y, 0)ρ µ s (x)]dt. (5.12) which implies ρ µ t must satisfy (1.10), the forward equations for ϕ µ t . Since there is a unique solution for this equation, the limit exists and it is ϕ µ t . Taking y = x in (3.20), the variances asymptotically vanish: This concludes the proof.
Uniqueness and the Yaglom limit convergence of Theorem 1.4 is a consequence of the next Theorem.
Theorem 5.1 (Jacka & Roberts). If there exists an x ∈ Λ such that α > C and there exists a qsd ν for Q, then ν is the unique qsd for Q and the Yaglom limit (1.3) converges to ν for any initial distribution µ.
Jacka and Roberts [9] use the stronger hypothesis inf y∈Λ q(y, x) > C for some x ∈ Λ but the proof works under the hypothesis α > C.
Proof of Theorem 1.4 Since α > 0, the fv process governed by Q is ergodic by Theorem 1.3. Call η N a random configuration chosen with the unique invariant measure. Since EL N η N (x) = 0, summing and subtracting y∈Λ q(y, 0)ρ N (x)ρ N (y), we get 0 = y∈Λ ρ N (y)[q(y, x) + q(y, 0)ρ N (x)] which holds for any N and x ∈ Λ. By Proposition 4.2, (ρ N , N ∈ N) is tight and by (4.3) dominated uniformly in N by a summable sequence. Bounding η N (x) N by one, the covariances of (5.14) are bounded by ρ N (y). Hence we can interchange limit with integral in (5.14) and use (3.21) to show that the second term in (5.14) vanishes as N goes to infinity. Then any limit ρ by subsequences must satisfy the qsd equation (1.2). Since by Theorem 5.1 the solution is unique, the limit lim N ρ N exists and equals the unique qsd ν.