Excited Random Walk in a Markovian Environment

One dimensional excited random walk has been extensively studied for bounded, i.i.d. cookie environments. In this case, many important properties of the walk including transience or recurrence, positivity or non-positivity of the speed, and the limiting distribution of the position of the walker are all characterized by a single parameter $\delta$, the total expected drift per site. In the more general case of stationary ergodic environments, things are not so well understood. If all cookies are positive then the same threshold for transience vs. recurrence holds, even if the cookie stacks are unbounded. However, it is unknown if the threshold for transience vs. recurrence extends to the case when cookies may be negative (even for bounded stacks), and moreover there are simple counterexamples to show that the threshold for positivity of the speed does not. It is thus natural to study the behavior of the model in the case of Markovian environments, which are intermediate between the i.i.d. and stationary ergodic cases. We show here that many of the important results from the i.i.d. setting, including the thresholds for transience and positivity of the speed, as well as the limiting distribution of the position of the walker, extend to a large class of Markovian environments. No assumptions are made about the positivity of the cookies.

(1.6) (iv) If δ = 4 then there is some b > 0 such that X n − vn n log(n) → Z 2,b in distribution, as n → ∞. (1.7) (v) If δ > 4 then there is some b > 0 such that X n − vn √ n → Z 2,b in distribution, as n → ∞. example). The final example, which we present here for the first time, is a modification of the second. It shows that if one considers Markovian environments as in Assumption (A) with bounded and elliptic cookie stacks, but assumes only that the Markov chain (S k ) k∈Z is ergodic (rather than uniformly ergodic), then again the threshold of Theorem 1.4 for ballisticity is not in general valid 1 . Thus, our uniformly ergodic assumption on the Markov chains (S k ) and (R k ), or at least some assumption beyond ergodicity, is necessary for the (IID) results to translate completely.
The following definition and lemma on monotonicity of the speed will be needed for our examples. Definition 1. 16. If ω 1 , ω 2 ∈ Ω are two cookie environments, we say that ω 1 dominates ω 2 if ω 1 (x, i) ≥ ω 2 (x, i), for all x ∈ Z and i ∈ N. If P 1 and P 2 are two probability measures on Ω, we say that P 1 dominates P 2 if there exists some joint probability measure P on Ω × Ω with marginals P 1 and P 2 such that P {(ω 1 , ω 2 ) ∈ Ω 2 : ω 1 dominates ω 2 } = 1.  [11]). Let P 1 and P 2 be two (SE) and (ELL) probability measures on Ω such that P 1 dominates P 2 , and let v 1 and v 2 be the corresponding velocities of the associated ERWs (as in Theorem 1.2). Then v 1 ≥ v 2 . Example 1. Define cookie stacks s 0 and s 1 by s 0 (i) = 1 for all i ∈ N, s 1 (i) = 1/2 for all i ∈ N. That is, s 0 is an infinite stack of completely right biased cookies, and s 1 is an infinite stack of "placebo cookies" which induce no bias on the random walker. Let S = {s 0 , s 1 }, and let (S k ) k∈Z be a stationary and ergodic process taking values in S such that the intervals between consecutive occurrences of s 0 in the process (S k ) are i.i.d. Define the random environment ω ∈ Ω by (1.11). Also, define τ 1 = inf{k > 0 : S k = s 0 } and τ i+1 = inf{k > τ i : S k = s 0 }, for i ≥ 1, and let T i = inf{n > 0 : X n = τ i }. That is, T i is the time at which the walker first reaches the i-th position to the right of 0 which has an s 0 stack. Assume that the distribution of the random variable (τ 2 − τ 1 ), i.e. the distribution of the distance between consecutive occurrences of stack s 0 in the process (S k ), is such that E(τ 2 − τ 1 ) < ∞, but E (τ 2 − τ 1 ) 2 = ∞. Then the following all hold.
Excited random walk in a Markovian environment Now, define a new process ( S k ) k∈Z from the process (S k ) k∈Z by the following projection: S k =s j , where j = k − and = sup{m ≤ k : S m = s 0 }.
Moreover, the state spaceS of this Markov chain has only bounded, elliptic stacks, and the Markov chain ( S k ) itself is stationary (since (S k ) is), irreducible and aperiodic (since (τ 2 − τ 1 ) has support on all of N), and positive recurrent (since E(τ 2 − τ 1 ) < ∞). Finally, for the probability measure P 3 on environments ω constructed from the ( S k ) process and associated parameter δ the following hold: 1. P 3 is dominated by the probability measure P 2 from Example 2. Hence, by Lemma 1.17, v 3 ≤ v 2 = 0, where v 3 and v 2 are associated velocities.
2. Let δ(s j ) = ∞ i=1 (2s j (i) − 1) be the net drift in stacks j , and let π = (π j ) j≥0 be the stationary distribution of the Markov chain ( S k ). Then δ(s 0 ) = M (2p − 1) and −2/3 = δ(s 1 ) < δ(s 2 ) < δ(s 3 ) < . . ., so δ = ∞ j=0 π j δ(s j ) > π 0 δ(s 0 ) + ∞ j=1 π j δ(s 1 ) In summary, the stack sequence ( S k ) k∈Z (hence also the reversed sequence ( R k ) k∈Z ) are stationary and ergodic Markov chains, and δ > 2, but the velocity v 3 ≤ 0. So, the ballisticity threshold of Theorem 1.12 does not extend to the case when the sequence of cookies stacks is simply an ergodic Markov chain rather than a uniformly ergodic one. Indeed, to the best of our knowledge, it is not even known for this example whether the walk is transient or recurrent since the environment does not satisfy the (POS) condition (and, thus, Theorem 1.3-(ii) is not applicable).

Outline of paper
An outline of the remainder of the paper is as follows. In Section 1.5 we introduce some basic notation and conventions that will be used throughout. In Section 2 we review a well known connection between ERW and certain branching processes, called the forward branching process and backward branching process. We also introduce some related processes, which are easier to analyze, and give some concentration estimates and expectation and variance calculations for these related processes. In Section 3 we prove Theorem 1.11. The proof is based on the connection between the ERW and forward branching process, and follows the general approach used in [13]. In Section 4 we prove Theorems 1.12 and 1.13. The proofs are based on a connection between the ERW and backward branching process and follow the general approach used in [8], [9]. Central to these arguments is a diffusion approximation limit for the backward branching process introduced in [8]. The proofs of several technical results are deferred to the appendices.

Notation
The positive integers are denoted by N (as above), and the non-negative integers by N 0 . The infimum of the empty set is defined to be ∞, and k i=j z i ≡ 0, for any j > k and sequence (z i ). For a stochastic process Z = (Z n ) n≥0 , τ Z x ≡ inf{n > 0 : Z n = x}. The same notation is also used for a continuous time process (Z(t)) t≥0 with continuous sample paths. For sequences of real numbers (a n ), (b n ) we write a n ∼ b n if lim n→∞ a n /b n = 1. Similarly, f (x) ∼ g(x) means lim x→∞ f (x)/g(x) = 1, for real valued functions f and g. Constants of the form c i are assumed to carry over between the various propositions and lemmas throughout. Other constants C, c, C i , K i , ... etc. are particular only to the specific lemma or proposition where they are introduced.
Unless otherwise specified it is assumed in the remainder of the paper that Assumption (A) holds for the probability measure P on cookie environments. The stack height M , state set S ⊂ S * M , and transition matrix K for the Markov chain (S k ) k∈Z are all assumed to be fixed. The marginal distribution of S 0 (according to P) is denoted by φ, and the stationary distribution of the Markov chain (S k ) is denoted by π. Also, δ is given by (1.12). By a slight abuse of notation, we will use P as the probability measure for the Markov chain (S k ) itself, as well as the environment ω derived from it according to (1.11). The probability measures P s , s ∈ S, and P π are the modified probability measures for the Markov chain (S k ) (or equivalently for the environment ω) when S 0 = s or S 0 ∼ π: P s (·) ≡ P(·|S 0 = s) and P π (·) ≡ s∈S π(s)P s (·).
Expectations with respect to P s and P π are denoted by E s and E π , respectively, and the corresponding averaged measures for the random walk (X n ) started from position x are P x,s (·) ≡ E s [P ω x (·)] and P x,π (·) ≡ E π [P ω x (·)]. The probability measure P and corresponding expectation operator E will be used generically for auxiliary random variables living on outside probability spaces, separate from those of the environment ω and random walk (X n ).

Branching processes
In this section we introduce our main tool in the analysis of the ERW, which is a connection with two related branching processes known as the forward branching process and backward branching process. The definition of the forward branching process is given in Section 2.1, and the definition of the backward branching process in Section 2.2. Some related branching processes which are easier to analyze are introduced in Section 2.3. Various concentration estimates and expectation and variance calculations for some of the related branching processes are given in Section 2.4.

The forward branching process
The construction of both the forward and backward branching processes is based on the coin tossing construction of the ERW introduced in [10]. For a fixed environment ω ∈ Ω, we initially flip an infinite sequence of coins at each site k, where the i-th coin at site k has probability ω(k, i) of landing heads. The walker begins its walk at some given site x, and if it ever reaches site k for the i-th time, then it jumps right if the i-th coin toss at site k was heads and left otherwise. More formally, let ξ k i , k ∈ Z and i ∈ N, be independent random variables such that ξ k i has Bernoulli distribution with parameter p = ω(k, i). Then the random walk (X n ) n≥0 started from position x in the given environment ω can be constructed from the ξ k i 's as follows: X 0 = x and X n+1 = (2ξ Xn In − 1) + X n (2.1) where I n = |{0 ≤ m ≤ n : X m = X n }|. We will say that ξ k i is a success if ξ k i = 1 (i.e. heads) and a failure if ξ k i = 0 (i.e. tails). The forward branching process (U k ) k≥0 started from level u 0 ∈ N 0 is defined by U 0 = u 0 and U k+1 = inf m : That is, U k+1 is the number of successes in the sequence (ξ k+1 i ) i∈N before the U k -th failure 2 . If we define G k i to be the number of successes in the sequence (ξ k j ) j∈N between the (i − 1)-th and i-th failures then we have, for each k ≥ 0, Thus, the process (U k ) k≥0 may be seen as a type of branching process with a time dependent migration term. More precisely, the k-th step of the process may be interpreted as a combination of the following 3 things: • First, U k ∧ M individuals emigrate out of the population before reproducing.
• Then, all remaining individuals (if any) have a Geo(1/2) number of offspring independently. • Finally, i individuals immigrate into the population after reproduction. Now, this construction for the process U = (U k ) k≥0 has been for a fixed environment ω, but one can also consider the same process when the environment ω is first chosen randomly according to some probability measure. We will denote by P U,ω u0 the probability measure for the process U started from level u 0 in a fixed environment ω, as constructed above, and by P U u0,s the probability measure for the joint process (U k , S k ) k≥0 when U 0 = u 0 and S 0 = s. That is, we first sample (S k ) k∈Z according to P s to get an environment ω = (ω(k, i)) k∈Z,i∈N = (S k (i)) k∈Z,i∈N , and then we sample (U k ) k≥0 according to P U,ω u0 . This two step procedure gives a joint measure on 3 (U k , S k ) k≥0 which is the measure P U u0,s . The measures P U u0,π and P U u0 are the averaged measures when S 0 is distributed according to π or φ, respectively: Under any of these measures P U u0,s , P U u0,π , and P U u0 the joint process (U k , S k ) k≥0 is a time-homogeneous Markov chain with transition probabilities p U (u,r)(u ,r ) ≡ Prob(U k+1 = u , S k+1 = r |U k = u, S k = r) given by where K is the transition matrix for the Markov chain (S k ) and ω r is the deterministic environment with stack r at each site: ω r (x, i) = r (i), for all x ∈ Z and i ∈ N.
The main interest in the forward branching process is its connection to a related process (U k ) k≥1 defined by Clearly, survival of the process (U k ), i.e. occurrence of the event {U k > 0, ∀k > 0}, is closely related to right transience of the random walk (X n ). The following lemma is standard, but we will provide a proof for the convenience of the reader. 2 In the case U k = ∞ (when (2.2) is no longer directly meaningful) we will extend this interpretation, so that U k+1 is then defined to be the total number of successes in the sequence (ξ k+1 i ) i∈N . 3 Note that the process (U k ) k≥0 depends only on S k (i) = ω(k, i), for i, k ≥ 1. So, we do not need to completely specify ω to construct (U k ) k≥0 . It is sufficient to consider (S k ) k≥0 . Lemma 2.1. Assume the process (U k ) k≥0 is started from level u 0 = 1 and that (X n ) n≥0 is started from position X 0 = 1. Then, for any realization of the random variables (ξ k i ) k∈Z,i∈N , U k ≤ U k for all k ≥ 1. Moreover, for any realization of the random variables (ξ k i ) k∈Z,i∈N such that τ X 0 < ∞, U k = U k for all k ≥ 1.

Note:
The lemma does not specify anything about the probability measure on the environment ω. The relation between U k and U k is a deterministic function of the values of (ξ k i ) k∈Z,i∈N , from which both processes (U k ) and (U k ) are constructed.
Proof. First fix a realization (ξ k i ) k∈Z,i∈N such that τ X 0 < ∞. By definition U 1 is the number of right jumps from site 1 before time τ X 0 , which (if τ X 0 < ∞) is simply the number of right jumps from site 1 before the first left jump from this site, or equivalently the number of successes in the sequence (ξ 1 i ) i∈N before the first failure. Since we assume u 0 = 1, the latter quantity is exactly U 1 , so we have U 1 = U 1 . Now suppose that U k = U k = m ≥ 0, for some k ≥ 1. Then, by the definition of the (U j ) process, the random walk (X n ) must jump right from site k exactly m times prior to time τ X 0 . Thus, the walk must jump left from site k + 1 exactly m times prior to time τ X 0 . Thus, the number of right jumps from site k + 1 prior to time τ X 0 is exactly the number of right jumps from site k + 1 before there are m left jumps from it, or equivalently the number of successes in the sequence (ξ k+1 i ) i∈N before the m-th failure. Since U k = m, this shows that U k+1 = U k+1 . It follows, by induction, that U k = U k for all k ≥ 1. Now, fix a realization (ξ k i ) k∈Z,i∈N such that τ X 0 = ∞. Then, for each k ≥ 1, U k is simply the total number of right jumps from site k by the random walk (X n ). Because the walk never jumps left from site 1, the total number of right jumps from site 1 is at most the number of successes in the sequence (ξ 1 i ) i∈N before the first failure. So, we have U 1 ≤ U 1 . Now suppose that U k = and U k = m for some 0 ≤ m ≤ ≤ ∞ and k ≥ 1. If m = 0, then the walk never jumps right from site k, so it never reaches site k + 1, so U k+1 = 0, so U k+1 ≤ U k+1 . If 1 ≤ m < ∞, then the walk jumps right from site k exactly m times total, so either it jumps left from site k + 1 exactly (m − 1) times or it jumps left from site k + 1 exactly m times and never returns to site k + 1 after the m-th left jump. Either way, the total number of right jumps from site k + 1 can be at most the number of successes in (ξ k+1 i ) i∈N before the m-th failure, which is at most the number of successes in (ξ k+1 i ) i∈N before the -th failure. Hence, again, U k+1 ≤ U k+1 . Finally, if m = ∞ then the walk must jump right from site k infinitely many times, so it must jump left from site k + 1 infinitely many times, so it must visit site k + 1 infinitely often, so the total number of right jumps from site k + 1 is simply the total number of successes in the sequence (ξ k+1 i ) i∈N , which is equal to U k+1 (since = ∞, with m = ∞ and ≥ m). Thus, in all possible cases U k+1 ≤ U k+1 , so it follows, by induction, that U k ≤ U k for all k ≥ 1.
In Appendix A we will prove the following basic fact using a finite modification argument. Lemma 2.2. Define A + = {X n > 0, ∀n > 0 and lim n→∞ X n = +∞}. Then, for each x ∈ N and s ∈ S, P x,s (A + ) > 0 if and only if P x,s (X n → +∞) > 0.
Using this fact along with Lemma 2.1 and Theorem 1.1, we now establish an explicit criteria relating transience/recurrence of the random walk (X n ) to the forward branching process (U k ). This criteria will be used to prove Theorem 1.11 in Section 3. For the statement of the lemma recall that P U u0,s (·) is the joint probability measure for (U k , S k ) k≥0 when U 0 = u 0 and S 0 = s. If ∃s ∈ S such that P U 1,s (U k > 0, ∀k > 0) > 0, then P 0 (X n → +∞) = 1. If ∃s ∈ S such that P U 1,s (U k > 0, ∀k > 0) = 0, then P 0 (X n → +∞) = 0.
We consider these two cases separately.

Case (a):
In this case, it follows from Lemma 2.2 that P 1,s (A + ) > 0, for each s ∈ S. Hence, P 1,s (U k survives) > 0, for each s ∈ S. By Lemma 2.1 this implies P U 1,s (U k survives) > 0, for each s ∈ S.

Case (b):
Since ω(x, i) = 1/2 for each i > M and x ∈ Z, P s a.s., we have P 1,s (lim inf n→∞ X n = x) = 0, for each x ∈ Z. Thus, if P 1,s (X n → +∞) = 0 for each s ∈ S, then P 1,s (lim inf n→∞ X n = −∞) = 1 for each s ∈ S, and in particular, P 1,s (τ X 0 < ∞) = 1 for each s ∈ S. By Lemma 2.1 and the definition of the (U k ) process this implies P U 1,s (U k survives) = 0 for each s ∈ S.
Thus, we have established the following dichotomy: Either (a') P U 1,s (U k survives) > 0 for each s ∈ S and (a) holds, or (b') P U 1,s (U k survives) = 0 for each s ∈ S and (b) holds.

The backward branching process
Let the random variables (ξ k i ) k∈Z,i∈N be as above in Section 2.1. We continue to assume the random walk (X n ) is constructed from these random variables using (2.1). Also, we recall that (R k ) k∈Z is the spatial reversal of the stack sequence (S k ) k∈Z : The backward branching process (V k ) k≥0 started from level v 0 ∈ N 0 is defined by That is, V k+1 is the number of failures in the sequence (ξ −(k+1) i ) i∈N (i.e. at stack R k+1 ) before there are V k + 1 successes. If we let H k i be the number of failures in the sequence (ξ −k j ) j∈N between the (i − 1)-th and i-th successes then Thus, by similar reasoning as for the forward branching process (U k ) k≥0 , this process (V k ) k≥0 may also be seen as a type of branching process with a time dependent migration term.
In fact, the definition of the backward branching process (V k ) is almost exactly symmetric to the definition of the forward branching process (U k ) with successes replaced by failures and S k replaced by R k , but there is one notable difference: In the backward process we count failures until V k + 1 successes, whereas in the forward process we only count successes until U k failures. This "+1" is important, because it means that 0 is not an absorbing state for the process (V k ), as it is for the process (U k ).
Our interest in the backward branching process stems from the following lemma about down crossings. The analog in the case of (IID) environments is well known. Lemma 2.4. Assume that δ > 1 and X 0 = V 0 = 0. For n ∈ N and k ≤ n, let D n,k = |{0 ≤ m < τ X n : X m = k, X m+1 = k − 1}| be the number of down crossings of the edge (k, k − 1) by the random walk (X m ) up to time τ X n . Then (V 0 , V 1 , . . . , V n ) and (D n,n , D n,n−1 , . . . , D n,0 ) have the same distribution if the environment ω is chosen according to the stationary measure P π .
Proof. Since we assume that δ > 1, it follows from Theorem 1.11 that τ X n is P 0,π a.s. finite 4 for each n. Fix n ∈ N and any realization of the random variables (ξ k i ) k∈Z,i∈N such that τ X n is finite. Then, define ξ k i = ξ k+n i , for k ∈ Z and i ∈ N. Let D n,k , 0 ≤ k ≤ n, be as in the statement of the lemma when the random walk (X m ) is generated according to the specific fixed values of (ξ k i ) k∈Z,i∈N , and let ( V k ) k≥0 be defined as in (2.6) The proof is by induction on k. For k = 0 we have V 0 = 0, by assumption, and D n,n = 0, since the walk cannot down cross the edge (n, n − 1) before first hitting site n. Now assume D n,n−k = V k = for some 0 ≤ k < n and ≥ 0. Then the walk (X m ) must jump left from site n − k exactly times prior to time τ X n . Thus, the walk must jump right from site n − (k + 1) exactly + 1 times prior to τ X n . Thus, the number of left jumps from site n − (k + 1) prior to time τ X n is exactly the number of left jumps from site n − (k + 1) before the ( + 1) − th right jump. So, we have: D n,n−(k+1) = # left jumps from site n − (k + 1) prior to time τ X n = # left jumps from site n − (k + 1) before ( + 1)-th right jump This completes the proof that D n,n−k = V k , for each 0 ≤ k ≤ n, using the specific fixed values of (ξ k i ) k∈Z,i∈N and associated values of ξ k i = ξ k+n i . The lemma now follows since (S k ) k∈Z is stationary under P π , so the stochastic process (ξ k i ) k∈Z,i∈N has the same distribution as the process (ξ k+n i ) k∈Z,i∈N .
The importance of the down crossings is their relation to hitting times for the random walk (X n ). If X 0 = 0, then for each n ∈ N τ X n = n + 2 k≤n D n,k = n + 2 n k=0 D n,k + 2 k<0 D n,k .
(2.8) 4 Theorem 1.11 will not be proved till later in Section 3, but the proof uses only the forward branching process described above, and is thus independent of the development in this section. The theorem is stated when ω is chosen according to P ≡ P φ , rather than Pπ, but φ is allowed to be any initial distribution on S in the theorem. So, in particular, the conclusion is valid when φ = π.
If δ > 1, so that the random walk is right transient, then lim n→∞ k<0 D n,k is a.s. finite, and thus the asymptotic distribution of τ X n is determined by the asymptotic distribution of n k=0 D n,k . By Lemma 2.4, the latter sum has the same distribution as n k=0 V k (assuming that φ = π). The general proof strategy for Theorems 1.12 and 1.13 is to analyze asymptotic properties of n k=0 V k , relate these to asymptotic properties of the hitting times τ X n , and then relate those to asymptotic properties of the random walk (X n ) itself. This basic approach has been employed many times before in the study of excited random walks, e.g. [10], [3], [8], [9]. (See also [7], [20], [21], and [18] for similar uses of branching processes in analyzing one dimensional self-interacting random walks and random walk in random environment.) In the sequel we will use the following notation for the backward branching process V = (V k ) k≥0 , similar to that for the forward branching process (U k ) k≥0 . P V,ω v0 is the probability measure for (V k ) k≥0 started from level v 0 in a fixed environment ω, and, for s ∈ S, P V v0,s is the probability measure for the joint process (V k , R k ) k≥0 when V 0 = v 0 and R 0 = s. The measures P V v0,π and P V v0 are defined by Under any of these measures P V v0,s , P V v0,π , and P V v0 the joint process (V k , R k ) k≥0 is a time-homogeneous Markov chain with transition probabilities p where K is the transition matrix for the Markov chain (R k ) given by K(r, r ) = K(r , r) · π(r ) π(r) and ω r is the deterministic environment with stack r at each site: ω r (x, i) = r (i), for all x ∈ Z and i ∈ N.

Related processes
The branching processes (U k ) and (V k ) are difficult to analyze directly because their transition probabilities depend on the underlying environment ω, and therefore these processes are not Markovian when ω is chosen randomly according to P (or P π or P s ), and are not time-homogeneous in a fixed environment ω. In this section we introduce some simpler related processes, which are both Markovian and time-homogeneous and, thus, easier to analyze.

The processes
Throughout this section and the remainder of the paper s ∈ S is an arbitrary but fixed stack. We define stopping times (τ k ) k≥0 and (τ k ) k≥0 by τ 0 = inf{j ≥ 0 : R j = s} and τ k+1 = inf{j > τ k : R j = s} , k ≥ 0; (2.10) τ 0 = inf{j ≥ 0 : S j = s} and τ k+1 = inf{j > τ k : S j = s} , k ≥ 0. Then we define processes ( V k ) k≥0 and ( U k ) k≥0 by V k = V τ k and U k = U τ k . (2.12) In other words, the processes ( V k ) and ( U k ) are constructed from (V k ) and (U k ) by observing the latter only at times j when R j = s or S j = s, respectively.
Since the process (U k , S k ) k≥0 is a time-homogeneous Markov chain (under P U x , P U x,π , and P U x,r , r ∈ S) and the process (V k , R k ) k≥0 is a time-homogeneous Markov chain (under P V x , P V x,π , and P V x,r ), the processes ( U k ) k≥0 and ( V k ) k≥0 are also time homogeneous Markov chains (under these same measures) with transition probabilities 5 for x, y, z, k ∈ N 0 , where (in accordance with our conventions in Section 1.5) In words, the probability of transitioning from z to y for the Markov chain ( U k ) is the probability the process (U k ) transitions from level z to level y during the time period that the process (S k ) makes one excursion from state s. Similarly, the probability of transitioning from z to y for the Markov chain ( V k ) is the probability the process (V k ) transitions from level z to level y during the time period that the process (R k ) makes one excursion from state s. Although the transition probabilities for these Markov chains are complicated because they depend on the random return times τ S s and τ R s , we will see in Section 2.4 that they can be analyzed reasonably well. By contrast, trying to analyze the processes (U k ) and (V k ) directly, under any of the above averaged measures, appears difficult, because they are not Markovian.

The dominating processes
To analyze the transition probabilities for the processes ( U k ) and ( V k ) it will be helpful to introduce some additional auxiliary processes, which dominate the processes (U k ) and (V k ), from both above and below. Recall that the forward branching process (U k ) k≥0 and the backward branching process (V k ) k≥0 , started from level x, are defined in terms of the random variables (ξ k i ) k∈Z,i∈N by That is, U k+1 is the number of successes in the sequence (ξ k+1 i ) i∈N prior to the U k -th failure, and V k+1 is the number of failures in the sequence (ξ −(k+1) i ) i∈N prior to the (V k + 1)-th success. Let us define modified processes (U + k ) k≥0 , (V + k ) k≥0 , (U − k ) k≥0 , and 5 Note that we extend here the probability measures P U x , P U x,π , P U x,r for the joint process (U k , S k ) k≥0 to the process ( U k ) k≥0 derived from it. The initial value x is still for U 0 . This is not, in general, the same as the initial value U 0 , except under P U x,s where S 0 = s deterministically. Similar remarks apply to the process ( V k ) with respect to the probability measures P V x , P V x,π , P V x,r .
(V − k ) k≥0 , all started from level x as follows 6 : In words, U + k+1 is the number of successes in the sequence (ξ k+1 i ) i∈N before the (U + k + 1)th failure, when we condition that ξ k+1 if all processes are started from the same level x. Also, since (ξ k i ) k∈Z,i>M are i.i.d. Ber(1/2) random variables, for any values of the cookie stacks (S k ) k∈Z , each of the These statements hold for any initial values U − 0 , U + 0 , V − 0 , V + 0 and any marginal distribution ρ on S 0 (including φ, π, or a point mass at r ∈ S). For the same reason (i.e. that (ξ k i ) k∈Z,i>M are i.i.d. Ber(1/2)), we also have when all processes are started from the same level x (again for any marginal distribution on S 0 ). To analyze the processes (U ± k ) and (V ± k ) it will be helpful to represent them in a form similar to (2.3) and (2.7) for the processes (U k ) and (V k ). Define G k i to be the number of successes in the sequence (ξ k j ) j>M between the (i − 1)-th and i-th failures, and define H k i to be the number of failures in the sequence (ξ −k j ) j>M between the (i − 1)-th and i-th successes. Then where (G k i ) k∈Z,i∈N are i.i.d. Geo(1/2) random variables, and where (H k i ) k∈Z,i∈N are i.i.d. Geo(1/2) random variables.

Expectation, variance, and concentration estimates
In this section we use the dominating processes (U ± k ) and (V ± k ) to analyze the transition probabilities (2.13) and (2.14) for the processes ( U k ) and ( V k ). We also prove, slightly more generally, concentration estimates for the processes (U k ) and (V k ) up to the random stopping times τ S s and τ R s , when (S k ) and (R k ), respectively, are started from an arbitrary initial state r ∈ S, rather than s. Finally, we prove a type of "overshoot lemma" for the processes ( U k ) and ( V k ) analogous to Lemma 5.1 of [8].
Throughout it is assumed, when not otherwise specified, that the processes (U k ), (U + k ), (U − k ) are all started from the same level x, and the processes (V k ), (V + k ), (V − k ) are all started from the same level x. The probability measure P U x,r will be used for all these "U -processes" started from level x when S 0 = r, and the probability measure P V x,r will be used for all these "V -processes" started from level x when R 0 = r. The following general fact will be needed in our analysis of the "U -processes" and "V -processes" below, as well as in several other parts of the paper. Lemma 2.5. Let Z be a random variable with mean µ and exponential tails, and let Z 1 , Z 2 , . . . be i.i.d. random variables distributed as Z. Then for any 0 ∈ (0, ∞) there exist constants C 1 ( 0 ), C 2 ( 0 ) > 0 such that the empirical means Z n ≡ 1 n n i=1 Z i satisfy: , for all 0 < ≤ 0 and n ∈ N. (2.20) , for all ≥ 0 and n ∈ N. (2.21) Proof. The exponential tails condition on the random variable Z implies there exist some positive constants b, c such that for all λ ∈ [−b, b], E(e λ(Z−µ) ) ≤ e cλ 2 and E(e λ(µ−Z) ) ≤ e cλ 2 . Thus, the lemma is a consequence of [17,Theorem III.15].
Using Lemma 2.5 along with (2.18) and (2.19) and a small bit of analysis one may obtain the following concentration estimates for the differences (U ± k − U ± k−1 ) and (V ± k − V ± k−1 ). Lemma 2.6. For each 0 ∈ (0, ∞), there exist constants c 1 ( 0 ), c 2 ( 0 ) > 0 such that the following hold for all r ∈ S, x, y ∈ N 0 , and k ∈ N : Moreover, the equivalent statements also hold for (V + k ) and (V − k ) with the same constants c 1 , c 2 . Remark 2.7. Note that since (U + k ) and (U − k ) are each time-homogeneous Markov chains independent of (S k ) k∈Z the probabilities on the left hand side of these equations do not depend on x, r, or k. Similar statements also apply for the processes (V + k ) and (V − k ).
We wish now to extend these concentration estimates for the single time step differences in the processes (U ± k ) and (V ± k ) to concentration estimates for these processes up to the random stopping times τ S s and τ R s . This is where we will need the uniform ergodicity hypothesis on the Markov chains (S k ) and (R k ). Due to Lemma 2.8 below and uniform ergodicity of (S k ) and (R k ) there exist some constants c 3 , c 4 > 0 such that: , for all r ∈ S and t ∈ [0, ∞). (2.26) Here, as described in Section 1.5, P r is the probability measure for the process (S k ) itself (equivalently the process (R k )) when S 0 = R 0 = r. Lemma 2.8. Let (Z k ) be a uniformly ergodic Markov chain on a countable state space Z, and let P x (·) be the probability measure for the Markov chain (Z k ) started from Z 0 = x. Then, for each z ∈ Z, there exist constants C > 0 and 0 < α < 1 such that Proof. Fix z ∈ Z. Let ρ = (ρ(x)) x∈Z denote the stationary distribution of the Markov chain (Z k ), and let M = {M(x, y)} x,y∈Z be its transition matrix. Also, let = ρ(z). By uniform ergodicity of (Z k ), there is some ∈ N such that M (y, ·) − ρ(·) T V ≤ /2, for all y ∈ Z. This implies P y (Z = z) = M (y, z) ≥ /2, for all y ∈ Z. Thus, starting from any initial state x we have

Lemma 2.9.
There exist constants c 5 , c 6 > 0 such that the following hold for each r ∈ S : , for all x ∈ N 0 and ≥ 1.
Moreover, the equivalent statements (with τ S s replaced by τ R s ) also hold for the processes with the same constants c 5 , c 6 . All statements are trivially true if x = 0 (taking any c 5 ≥ 1 and any c 6 > 0), so we will assume x ≥ 1. We will prove (2.27) and (2.28). The proof of (2.29) is identical to that of (2.27) with U − k replaced by U + k line by line, and the proof of (2.30) is almost identical to that of (2.28) with U − k replaced by U + k line by line 7 . The equivalent statements to (2.27)-(2.30) for the processes (V − k ) and (V + k ) are also proved exactly the same way; all the proofs use is Lemma 2.6 and (2.26), and these estimates are the same for (V ± k ) and (U ± k ) and for τ R s and τ S s . The proofs of (2.27) and (2.28) will be given separately and the constants c 5 , c 6 obtained in the two cases will not be the same. To find a single c 5 and c 6 that hold in both cases simply take c 5 to be the maximum of the c 5 's from the two proofs and c 6 to be the minimum of the c 6 's from the two proofs. 7 In the derivation of (2.44) the case z = 0 must be considered separately for both proofs. For the process as noted in the proof of (2.28) below. For the process (U + k ) this is not the case. However, if we start with U + 0 = x ≥ 1, as we assume, then it is actually impossible that U + k is 0 for any k ≥ 0 (indeed, U + k ≥ M for all k ≥ 1). So we do not have this problem to deal with.
Proof of Lemma 2.9, Equation . Fix such an n and define Also, by Lemma 2.6, there exist some constants c 1 , c 2 > 0 such that , for each k, z and 0 < ≤ 2. (2.35) Hence, Since, (2.37) is valid for each n ≤ 1 2 2/3 x 1/3 it follows from (2.31) and (2.32) that Excited random walk in a Markovian environment Remark 2.10. If we define the deterministic time t ,x = 1 2 2/3 x 1/3 , for 0 < ≤ 1 and x ∈ N, then the same exact steps used in the derivation of (2.37) for an arbitrary n ≤ t ,x show that We isolate this observation, as it will be needed later in the proof of Lemma 2.12.
Thus, for each 2/3 x 2/3 ≤ z ≤ 2 x and 1 ≤ k ≤ n, and, for each z < 2/3 x 2/3 and 1 ≤ k ≤ n, Note that the inequality (2.44) remains valid when z = 0 (even though the derivation above has an issue dividing by 0), since in this case Thus, Since (2.45) is valid for each n < 1/3 x 1/3 it follows from (2.39) and (2.40) that Using (2.15) the concentration estimates for the processes (U ± k ) and (V ± k ) proven above in Lemma 2.9 yield the following concentration estimates for the processes (U k ) and (V k ). Lemma 2.11. Let c 7 = 2c 5 . Then the following hold for each r ∈ S : , for all x ∈ N 0 and 0 < ≤ 1.
, for all x ∈ N 0 and ≥ 1. (2.49) The next two lemmas give estimates for the expectation and variance of U τ S s and V τ R s in the case that S 0 = R 0 = s. For these lemmas, and the remainder of this section, E U to be the mean return time to state s for the Markov chain (S k ), or equivalently for the Markov Proof. We will prove the statement about the expectation of U τ S s . The proof of the analogous claim for the expectation of V τ R s is very similar.
Fix a realization (s k ) k∈Z of the random variables (S k ) k∈Z with s 0 = s, and let ω = (ω(k, i)) k∈Z,i∈N be the corresponding cookie environment defined by ω(k, i) = s k (i). Also, let t S s = inf{k > 0 : s k = s} be the corresponding realization of the random variable τ S s .
We will consider first the process (U k ) k≥0 started from level x in this fixed environment ω.
to be the net drift induced by consuming all cookies in stack s k . Also, we denote by E U,ω x expectation with respect to the probability measure P U,ω x for the process (U k ) k≥0 in this fixed environment ω. Thus, Plugging into (2.50) gives So far our analysis has been for a fixed environment ω. To prove the lemma we will need to take expectations with respect to the probability measure P s on environments.
Recall that, for r ∈ S, In step (*) we have used Fubini's Theorem to interchange the sum with the expectation; (2.54) We define n 0 = (1/2) 5/3 x 1/3 . Splitting the sum at n 0 , the right hand side of (2.54) may be bounded as follows: (2.55) (2.56) Also, using (2.38) with = 1/2 shows that, for all x > 2M and 0 ≤ k ≤ n 0 , We will prove the statement about the variance of U τ S s . The proof of the analogous statement for the variance of V τ R s is again very similar. A central element of the proof is the following claim.
Claim: There exists a non-negative random variable ∆ with finite variance (defined on some outside probability space, separate from the (U + k ) and (U − k ) processes) such that The proof of the claim will be given after the main proof of the lemma. The basic idea for the proof of the lemma is to approximate the process (U k ) k≥0 by the process (U * k ) k≥0 defined by where the random variables G k i are as in (2.18). This process (U * k ) k≥0 is a standard Galton-Watson branching process with Geo(1/2) offspring distribution, independent of (S k ), and satisfies (2.60) By (2.15) and (2.59), along with the claim (2.58), | ∆| stoch ≤ ∆, for a random variable ∆ with finite variance (not depending on x). The first term U * τ S s can be analyzed exactly.
for any fixed k > 0. Thus, since the stopping time τ S s is independent of the process (U * k ) and a.s. finite, (2.63) By the calculation above, the first term on the right hand side of (2.63) is exactly equal to 2xµ s . The second two terms may be bounded as follows: Proof of Claim: Fix any x ∈ N 0 and assume throughout that be a standard Galton-Watson branching processes with Geo(1/2) offspring distribution, started from B 0 = 1. Also, let (β k,i ) k≥0,i∈N and (β k,i ) k≥0,i∈N all be independent random variables such that β k,i law =β k,i law = B k . Finally, let T be a random time with the same distribution as τ S s (under P U x,s ) which is defined on the same probability space as the β andβ random variables, but independently of them. We will denote the probability measure for this probability space by P, and the corresponding expectation operator by E. We claim that Direct computations using E(β k,i ) = E(β k,i ) = 1 and Var(β k,i ) = Var(β k,i ) = 2k, along with independence, give E(∆ n ) = (2M + 1)n , Var(∆ n ) = 2M n 2 + n 2 + n , E(∆ 2 n ) = (2M + 1)(2M + 2)n 2 + n.
Excited random walk in a Markovian environment Thus, since T has an exponential tail and the random variables ∆ n are independent of T , So, it remains only to show (2.66).
To this end, recall again that the processes (U + k ) k≥0 and (U − k ) k≥0 may be represented in terms of the independent Geo(1/2) random variables G k i according to (2.18). Let us define a new process (Z k ) k≥0 as follows: To prove (2.67) observe that W 0 = 0 and, for all k ≥ 0, Thus, the process (W k ) k≥0 has the same distribution as the process ( W k ) k≥0 defined by Geo(1/2) random variables (living on some probability space). To see this note that, for each u = (u 0 , . . . , u n ) ∈ N n+1 Thus, the conditional law of the random variables G k i such that 1 ≤ k ≤ n and i > (u k−1 − M ) + on the event E( u) is still i.i.d. Geo(1/2). Thus, conditional on the event E( u), (W k ) n k=0 has the same law as ( W k ) n k=0 . And since that holds for all n ∈ N and u = (u 0 , . . . , u n ) with P U x,s (E( u)) > 0, it follows that the entire process (W k ) k≥0 has the same (unconditional) law as ( W k ) k≥0 . Now, the process ( W k ) defined by (2.68) may be understood as a type of branching process with migration where, at each step k, in addition to all the "regular children" created by the branching mechanism we also introduce M + 1 immigrants prior to the reproduction stage (the M + 1 in the upper limit of the sum) along with M additional immigrants after the reproduction stage (the M in front of the sum). It follows that, for each n, Our final result of this section is an "overshoot lemma" which gives concentration estimates for ( V k ) when exiting certain intervals. Analogous statements also hold for the process ( U k ), but we will not need these.
Then there exist constants c 8 , c 9 > 0 and N ∈ N such that for all x ≥ N the following hold: Before proceeding to the main proof we isolate an important piece as its own lemma.
For the statement of this lemma recall that P V,ω v0 is the probability measure for the backward branching process (V k ) k≥0 defined by (2.6) in the particular environment ω ∈ Ω, started from V 0 = v 0 .

Lemma 2.15.
There exist constants C, c > 0 such that for any environment ω satisfying ω(k, i) = 1/2 for all k ∈ Z and i > M , Proof. This is implicit in the proof of Lemma 5.1 in [8]. That lemma is stated for the process (V k ) under the averaged measure on environments in an (IID) and (BD) setting, rather than for a fixed environment ω. But the proof of the equivalent statement to (2.70) in this setting (given within the proof of Lemma 5.1 in [8]) uses only the fact that all cookies stacks are of bounded height M .
Proof of Lemma 2.14. We will prove (i) under the assumption that y ≥ 12M and x ≥ N ≡ 2M + 1. By increasing the constant c 8 if necessary, the desired inequalities in (i) will then hold for all y ≥ 0. A similar proof gives (ii) with some different values of the constants.
Under the measure P V z,s we have V 0 = V 0 = z < x, so τ V x + > 0. Define a random variable k 0 as follows: Given that τ V x + = j + 1, for some j ∈ N 0 , let k 0 = inf{k > τ j : V k ≥ x}. In other words, (j + 1) is the first time that the process ( V ) reaches level x or above, and k 0 is the first time k > τ j that the process (V k ) reaches level x or above. Since the process (V k , R k ) is Markovian under P V z,s it follows that, for each y ∈ N, Thus, it will suffice to show the following two claims to establish the lemma.
Excited random walk in a Markovian environment Claim 1: There exist C 1 , C 2 > 0 such that for all y ≥ 12M , x ≥ 2M + 1, 0 ≤ w < x, and r ∈ S Claim 2: There exists C 3 > 0 such that for each x ≥ 2M + 1, 0 ≤ w < x, and r ∈ S Fix any x ≥ 2M + 1, 0 ≤ w < x, and r ∈ S. For each y ≥ 12M we have . Define w 0 = x + y/2 . By monotonicity of (V k ) with respect to its initial condition and Lemma 2.11, Now, for x ≤ y we have 1 8 y < (y/2) 2 w −1 0 < y, and for x ≥ y we have 1 8 y 2 /x < (y/2) 2 w −1 0 < y 2 /x. So, it follows that is a Markov chain independent of (R k ) k≥0 it follows from (2.15) and (2.19) that, for any (r 0 , . . . , r m ) ∈ S m+1 and n ≥ x ≥ 2M + 1, To complete the proof we note that by (2.26) there exist some m 0 ∈ N and p > 0 such that P r (τ R s ≤ m 0 ) ≥ p, for each r ∈ S. Therefore, for any x ≥ 2M + 1, 0 ≤ w < x, and r ∈ S,

Proof of Theorem 1.11
The proof of Theorem 1.11 relies on the following lemma concerning transience vs. recurrence of integer-valued Markov chains, which will be proved in Appendix B.
Lemma 3.1. Let (Z k ) k≥0 be an irreducible, time-homogeneous Markov chain on state space N 0 . Let P x be the probability measure for (Z k ) started from Z 0 = x, and let E x be the corresponding expectation operator. Assume that there exist constants C 1 , C 2 > 0 and x 0 ∈ N such that: and assume also that lim inf x→∞ ν(x) > 0. Then the following hold.
(i) If there exists some a ∈ (1, ∞) such that θ(x) ≤ 1 + 1 a ln(x) for all sufficiently large x, then the Markov chain (Z k ) is recurrent.
for all sufficiently large x, then the Markov chain (Z k ) is transient.  Then the following hold.
(ii) If there exists some a ∈ (1, ∞) such that θ(x) ≥ 1 + 2a ln(x) for all sufficiently large x, then P z (Z k > 0, ∀k > 0) > 0, for each z ≥ 1. Remark 3.3. Note that we do not assume in the corollary that the Markov chain (Z k ) is irreducible. State 0 may be absorbing, and in fact when we apply the corollary in the proof of Theorem 1.11 below it will be for the Markov chain ( U k ), where state 0 is absorbing.
Proof of Corollary 3.2. The Markov chain (Z k ) k≥0 is not necessarily irreducible since state 0 may be absorbing, but the modified Markov chain ( Z k ) k≥0 with transition probabilities P 0 ( Z 1 = 1) = 1 and P x ( Z 1 = y) = P x (Z 1 = y), x ≥ 1 and y ≥ 0 is certainly irreducible. Due to (3.3), this modified chain ( Z k ), like the original chain (Z k ), has positive transition probabilities from each state x ≥ 1 to each state y ≥ 0. Thus, if ( Z k ) is transient it has positive probability never to hit state 0 started from any state z ≥ 1. Applying Lemma 3.1 shows that (i) and (ii) of the corollary hold for the Markov chain ( Z k ). This implies (i) and (ii) also hold for (Z k ), since (Z k ) and ( Z k ) have the same transition probabilities from all non-zero states.
Before proceeding to the proof of Theorem 1.11 we isolate one important observation in the following remark. This observation will also be necessary for the proof of Theorem 1.12 in Section 4.1.

Remark 3.4 (Invariance of Assumption (A) under interchange of spatial directions).
Assume that Assumption (A) is satisfied for the probability measure P, and let (S k ) be the associated stack sequence and R k = S −k , k ∈ Z. Construct the spatially reversed probability measure P on environments ω by the following coupling: ω(k, i) = S k (i), for k ∈ Z and i ∈ N, where the process ( S k ) is given by S k (i) = 1 − S −k (i) = 1 − R k (i). Thus, the probability of jumping right (left) on the i-th visit to site k for the random walk ( X n ) in the spatially reversed environment ω is the same as the probability of jumping left (right) on the i-th visit to site −k for the original random walk (X n ). With this construction there is also a natural coupling between the random walks ( X n ) and (X n ), when both are started from site 0, such that ( X n ) jumps right whenever (X n ) jumps left, and vice versa. This implies X n = −X n , for all n, and so we may consider ( X n ) as the spatially reversed version of the random walk (X n ).
Moreover, since the original model with probability measure P is assumed to satisfy Assumption (A), we know that (R k ) is a uniformly ergodic Markov chain, which implies ( S k ) is also a uniformly ergodic Markov chain. Similarly, since the original model is assumed to satisfy Assumption (A), we know that (S k ) is a uniformly ergodic Markov chain, which implies ( R k ) is a uniformly ergodic Markov chain (where R k ≡ S −k ). Thus, the spatially reversed probability measure P also satisfies Assumption (A). Finally, note that δ = −δ, where δ and δ are the drift parameters for the probability measures P and P on environments.
Since the probability measure π places positive probability on each states r ∈ S we must have the same trichotomy under the measure P 0 . That is, either Thus, lim inf x→∞ ν(x) > 0, as required by the corollary, and θ(x) = 2ρ(x)/ν(x) = δ + O(x −1/2 ). Applying the corollary with z = 1 we have Since 0 is an absorbing state for the process (U k ) k≥0 it follows also that Hence, by Lemma 2.3, which establishes the claim about (a').

Proof of Theorems 1.12 and 1.13
The proofs of Theorems 1.12 and 1.13 are based on an analysis of the backward branching process (V k ) k≥0 , and follow the general strategy used in [8]. However, there is an additional complication due to our different probability measure P on environments.
If P is (IID), as considered in [8], then the process (V k ) k≥0 is Markovian. However, if P is Markovian, as we consider, then the process (V k ) k≥0 is not. Thus, we will instead analyze the modified process ( V k ) k≥0 , which is Markovian, and then translate the results from the ( V k ) process back to the (V k ) process. The outline for our proofs and the remainder of this section is described below.
Step 1 : In Appendix C we will establish the following two propositions which are analogous to Theorems 2.1 and 2.2 of [8]. The general methodology is very similar to that in [8], so we will provide only a general outline and reprove a few key lemmas from which everything else follows just as in [8].
Step 2 : Define σ V 0 = inf{k > 0 : V k = 0 and R k = s}. In Appendix D we will translate the above results for the process ( V k ) to the process (V k ) proving the following propositions.
Step 3 : In Section 4.1 we will use Propositions 4.3 and 4.4 to prove Theorem 1.12.
Step 4 : In Section 4.2 we will use Propositions 4.3 and 4.4 to prove Theorem 1.13.
For future reference we observe the following simple corollary of Proposition 4.3.
Proof. The process (V k , R k ) k≥0 is a time-homogeneous Markov chain under P V x,r , for any x, r. Furthermore since the Markov chain (R k ) is irreducible and the cookie stacks in S are elliptic this Markov chain on pairs (V k , R k ) is also irreducible. By Propostion 4.3 the pair (0, s) is a recurrent state for the Markov chain (V k , R k ), so the Markov chain itself is recurrent, so the hitting time of state (0, s) is a.s. finite starting from any initial state (x, r).

Proof of Theorem 1.12
Define stopping times (σ i ) i≥0 by (4.1) We denote the mean of the Q i 's by µ Q and the mean of the ∆ σ,i 's by µ σ .

Proof of Theorem 1.13
Throughout Section 4.2 the random variables Z α,b are as in (1.3), the random variables Q i , σ i , ∆ σ,i and i n are as in Section 4.1, and m n ≡ n/µ σ . The general proof strategy for Theorem 1.13 will be to first prove a limiting distribution for n k=0 V k , then translate to a limiting distribution for the hitting times τ X n using (2.8) and Lemma 2.4, then translate to a limiting distribution for the walk (X n ) itself. This basic approach has been used before in [3,8,9], and our methods will be quite similar to these works, but the details differ a bit because the process (V k ) is not Markovian when the environment is not (IID). Thus, we must consider renewal times (σ i ) i≥0 , rather than simply successive times at which V k = 0. Also, we have the additional minor complication of the Q 0 and σ 0 terms to deal with (which would be 0 in the (IID) case).
Unless other specified it is assumed throughout that V 0 = 0 and the probability measure on environments is P π (rather than P). Everything will be proved initially under the stationary measure P π , and then at the very end after proving Theorem 1.13 under the stationary measure P π we will translate the result to the given measure P.
To state our first lemma for the limiting distribution of n k=0 V k we first need to introduce a little notation. Let µ Q (t) be the truncated expectation of the random variables is expectation with respect to the probability measure P V 0,π . Also, let Z 1,b,c be a random variable with characteristic function E[e itZ 1,b,c ] = exp itc − b|t| 1 + 2i π log |t|sgn(t) , = Z α,ba α , for all α ∈ (0, 1) ∪ (1, 2], b > 0, a > 0.
Also, we note that µ σ is finite for all δ > 1 by Propositions 4.3, and µ Q is finite for all δ > 2 by Proposition 4.4.

Lemma 4.6.
Under the probability measure P V 0,π for the process (V k ) k≥0 the following hold: (i) If δ ∈ (1, 2) then there is some b > 0 such that n k=0 V k n 2/δ d.
To prove Lemma 4.6 we will need two general results about the limiting distributions of sums of i.i.d. random variables. The first result is a particular case of convergence to stable distributions for sums of i.i.d. random variables with regularly varying tails. The second result concerns sums of a random number of i.i.d. random variables with finite variance.
In addition to these two theorems we will also need the following lemma for the proof of part (v) of Lemma 4.6. Lemma 4.9. For any δ > 1 the following hold: (ii) There exists some constant c 14 > 0 such that: Proof of Lemma 4.9 (i). Since ∆ σ,i , i ≥ 1, are i.i.d., the times (σ i ) i≥0 are the renewal times for a (delayed) renewal process. By definition i n is the number of renewals up to time n (excluding σ 0 ). So, by [19, Proposition 3.5.1] i n /n → 1/µ σ a.s.
Proof of Lemma 4.9 (ii). Since the Markov chain (R k ) is uniformly ergodic it is aperiodic, which implies the Markov chain on pairs (V k , R k ) is aperiodic (due to ellipticity of the cookie stacks), which implies the discrete renewal process with renewal times (σ i ) i≥0 is itself aperiodic. Let A m,k be the event that there are no renewal times σ i in {m, m+1, . . . , m+k−1} and let ∆ σ be a random variable (on some probability space) with the common distribution of the random variables ∆ σ,i , i ≥ 1. It follows from aperiodicity and [19,Example 4 Proof of Lemma 4.6. The proofs of the various parts of the lemma will be given separately, but we begin with one important general observation that is necessary for the proof of several parts. If 1 ≤ ≤ m n and |σ mn − n| ≤ , then since the random variables V k are nonnegative and σ mn− ≤ σ mn − < σ mn + ≤ σ mn+ . Thus, if (a n ) and (b n ) are any sequences of positive real numbers such that b n → ∞ and a n → ∞ with a n = o(n), then lim sup In the last line we have used the fact that the random variables (Q i ) i≥1 are i.i.d., and also the fact that Q 0 is P V 0,π a.s. finite (due to Corollary 4.5), which implies Proof of (i): With δ ∈ (1, 2) it follows from Proposition 4.3 and Theorem 4.7-(iii) that −→ Z δ,B1 , as m → ∞, for some B 1 > 0. Also, since σ 0 is a.s. finite σ0+(mnµσ−n) m 1/δ n → 0 a.s, as n → ∞. Hence, −→ Z δ,B1 , as n → ∞.
So it will suffice to show that (II) p.
−→ 0. To do this, fix an arbitrary > 0. Applying (4.8) with a n = n p , p ∈ (1/δ, 1), and b n = n 2/δ gives lim sup The first term on the right hand side is 0 by (4.9), and the second term on the right hand side is 0 by (4.10). Since > 0 is arbitrary, it follows that (II) p.
−→ 0. To do this, fix an arbitrary > 0. Applying (4.8) with a n = n p , p ∈ (1/2, 1), and b n = n gives lim sup The first term on the right hand side is 0 by (4.12), and the second term on the right hand side is 0 by (4.11) and (4.13). Since > 0 is arbitrary, it follows that (III) p.

−→ 0.
To do this, fix an arbitrary > 0. Applying (4.8) with a n = n 1/2 (log n) 1/4 and b n = d(n) The first term on the right hand side is 0 by (4.14), and the second term on the right hand side is 0 by (4.15). Since > 0 is arbitrary, it follows that (III) p.
(ii) If δ = 2 then there are constants b > 0 and c ∈ R such that Proof. Since (X n ) is P 0,π a.s. right transient with any δ > 1, lim sup n→∞ k<0 D n,k is P 0,π a.s. finite. So, k<0 D n,k /n α p. The proof of Theorem 1.13 below will be based on Lemma 4.10, but first we will need one final lemma about backtracking probabilities. The same statement has been given in [16,Lemma 6.1] for the case of (IID) environments, and our proof will be very similar, but must be adjusted slightly to use the renewal times (σ i ) i≥0 , instead of the successive times at which V k = 0. Lemma 4.11. Assume that δ > 1 and let c 14 be as in Lemma 4.9. Then P 0,π inf m≥τ X n+k X m ≤ n ≤ c 14 · k 1−δ , for all k, n ∈ N.
Proof. As in the proof of Lemma 4.9 we consider the renewal times σ i and note that, for each fixed k, Using this along with Lemma 2.4 gives Proof of Theorem 1.13. Recall that for the probability measure P on environments, the marginal distribution of S 0 is φ. We will first prove the theorem in the case that φ = π is the stationary distribution. We will then extend to the case of general φ.
Case 1: φ = π. Let X + n = sup i≤n X i and X − n = inf i≥n X i . Since X − n ≤ X n ≤ X + n , for all n, it will suffice to show that Now observe that, for any n, m, k ∈ N, (4.16) and We proceed now to the proof of (a) for part (ii) of the theorem. Assume δ = 2 and define D(t) = c + 1 +   For t > 0, let Γ(t) = inf{s > 0 : sD(s) ≥ t}. Note that D(t) ∼ 1 a log(t) =⇒ Γ(t) ∼ at/ log(t).
Further we claim that Γ(t)D(Γ(t)) = t + o(Γ(t)), as t → ∞. (4.20) To see this note that the function g(s) = sD(s) is right continuous and strictly increasing for all sufficiently large s. Moreover, jump discontinuities in this function g(s) can occur only at s = kµ σ for integer k, and at such an s the size of the jump discontinuity is . It follows from these observations and the definition of Γ(t) that for all sufficiently large t. Now, since P V 0,π (Q i > x) ∼ c 13 x −1 , we have xP V 0,π (Q i = x) → 0, as x → ∞. So, the right hand side of (4.21) is o(Γ(t)), which proves (4.20). Now, for x ∈ R and n ∈ N, let k n,x = max{ Γ(n) + xn (log n) 2 , 0}. Note that k n,x ∼ Γ(n) as n → ∞, since Γ(n) ∼ an/ log(n). Using this fact along with (4.19) and (4.20) and, again, the tail asymptotics for Γ(n) shows that, for any fixed x,  Further, since X + n takes only integer values it follows from (4.16) that, for all sufficiently large n, Taking the limit of both sides as n → ∞ and using (4.18) and (4.22) shows that lim n→∞ P V 0,π Case 2: General φ. We will extend from Case 1 using a coupling argument. Let (S φ k ) k∈Z and (S π k ) k∈Z denote the stack sequences when S 0 has marginal distribution φ and π respectively. Couple these processes as follows. First sample (S φ k ) k≤0 and (S π k ) k≤0 independently. Then run the Markov chains (S φ k ) and (S π k ) forward in time independently, starting from the given values of S φ 0 and S π 0 , until the first time N > 0 such that S φ N = S π N (due to the uniform ergodicity hypothesis N is a.s. finite). After the chains first meet at time N , run them forward together, so that S φ k = S π k , for all k ≥ N . Now, let ω φ and ω π be the corresponding environments given by ω φ (k, i) = S φ k (i) and ω π (k, i) = S π k (i), as in (1.11), and couple the random walks (X φ n ) and (X π n ) in these environments as follows: • Let (θ k,i ) k∈Z,i∈N be i.i.d. Uniform([0,1]) random variables.
With this construction both walks (X π n ) and (X φ n ) have the correct averaged laws. Moreover, due to the coupling between environments ω π (k, i) = ω φ (k, i), for all k ≥ N and i ∈ N. So, both walks (X π n ) and (X φ n ) have the same theoretical "jump sequence" at each site k ≥ N . That is, both walks will jump the same direction from any site k ≥ N on their i-th visit to that site, if such an i-th visit occurs. Now, since it is assumed that δ > 1 in all cases of the theorem, we know both walks are right transient. So, with probability 1, each walk eventually reaches site N and also returns to this site after any leftward excursion from it. Combining this with the previous observation about matching jump sequences at all sites k ≥ N shows that lim sup n→∞ |X φ n − X π n | < ∞ a.s.

A Proof of Lemma 2.2
Proof of Lemma 2.2. Fix x ∈ N. Clearly, P x,s (A + ) = 0 if P x,s (X n → +∞) = 0, so we need only show that P x,s (A + ) > 0 if P x,s (X n → +∞) > 0. By definition, P x,s (A + ) = E s [P ω x (A + )] and P x,s (X n → +∞) = E s [P ω x (X n → +∞)], so it will suffice to show the following claim.
Proof of Claim: For m ∈ N and any nearest neighbor path ζ = (x 0 , . . . , x m ) ∈ Z m+1 with x 0 = x, define the events A m and A ζ by If P ω x (X n → +∞) > 0, then there must be some finite path ζ = (x 0 , . . . , x m ) such that We construct from ζ = (x 0 , . . . , x m ) the reduced path ζ = ( x 0 , . . . , x m ) by setting x 0 = x 0 = x, and then removing from the tail (x 1 , . . . , x m ) all steps in any leftward excursions from site x. For example, (where we denote the removed steps in bold for visual clarity). By construction, x m = x m = x+1 and the number of visits to each site k ≥ x+1 up to time m if (X 0 , . . . , X m ) = ζ is exactly the same as the number of visits to each site k ≥ x + 1 up to time m if (X 0 , . . . , X m ) = ζ. Thus, (Note that P ω x (X 0 = x 0 , . . . , X m = x m ) > 0, since we assume ω(k, i) ∈ (0, 1), ∀k ∈ Z, i ∈ N.)

B Proof of Lemma 3.1
The proof of Lemma 3.1 is based on the following much more general, but less explicit, condition for transience vs. recurrence of Markov chains on the nonnegative integers given in [13].
Lemma B.1 (Theorem A.1 of [13]). Let (Z k ) k≥0 be an irreducible, time-homogenous Markov chain on state space N 0 . Let P x (·) be the probability measure for the Markov chain started from Z 0 = x, and let E x (·) be the corresponding expectation operator. The function F (x) is called a Lyapunov function. Our proof of Lemma 3.1 using Lemma B.1 follows closely the proof of Theorem 1.3 in [13]. In particular, we will use the same choice of Lyapunov functions F (x) and Taylor expand in the same fashion.
However, controlling the error term in the Taylor expansion becomes somewhat more involved, because of the weaker concentration condition we assume for the transition probabilities.
By Taylor's Theorem with remainder where ξ is some (random, depending on Z 1 ) number between Z 1 and x. Thus, for all positive integer x > 3, So, Therefore, in light of Lemma B.1 and the assumption that lim inf x→∞ ν(x) > 0, it will suffice to show . Now, ≡ (I) + (II) + (III).
Bound on (II) : By (B.1), for all x > 6, We define Z 1 = Z 1 · 1 {x/2<Z1≤2x} + x · 1 {Z1 ∈(x/2,2x]} , and bound the expectation on the right hand side as follows: We define Z 1 = Z 1 · 1 {Z1>2x} + x · 1 {Z1≤2x} and write the expectation on the right hand side as The first integral in the brackets on the right hand side of (B.7) is easily bounded using (3.1): x 0 P x (Z 1 − x > x)y 2 dy ≤ The second integral in the brackets on the right hand side of (B.7) is bounded as follows: , for x > 2. Then for all x > 2 , .
Thus, by Taylor's Theorem with remainder, for all positive integer x > 2 where ξ is some (random) number between Z 1 and x. So, Therefore, in light of Lemma B.1 and the assumption that lim inf x→∞ ν(x) > 0, it will suffice to show .
Combining these estimates on (I), (II), and (III) shows that E

C Proof of Propositions 4.1 and 4.2
Our proof of Propositions 4.1 and 4.2 is based on the following slightly more general proposition, which we isolate in this form because it seems it may be applicable for analyzing other critical branching (or branching-type) processes.
Proposition C.1. Let (Z k ) k≥0 be an irreducible, time-homogeneous Markov chain on state space N 0 , and denote by P Z x the probability measure for the Markov chain (Z k ) started from Z 0 = x. Also, let E Z x and Var Z x denote, respectively, expectation and variance with respect to the measure P Z x . Assume that the following four conditions are satisfied.
(D) Exit Probability Concentration Estimates: There exist constants C 3 , C 4 > 0 and N ∈ N such that for all x ≥ N the following hold: Here, as usual, τ Z x = inf{k > 0 : Z k = x}, and τ Z x + , τ Z x − are defined by τ Z x + = inf{k > 0 : Z k ≥ x}, τ Z x − = inf{k > 0 : Z k ≤ x}. 8 Note that this condition (along with the Markov property) implies one can couple together versions of the process (Z k ) starting from two different initial conditions x > y in such a way that Z k , for all k ≥ 0.

Excited random walk in a Markovian environment
Then there exist constants C 5 , C 6 > 0 such that, as t → ∞, Proof of Propositions 4.1 and 4.2. We simply apply Proposition C.1 to the (irreducible, time-homogeneous) Markov chain ( V k ) k≥0 with transition probabilities give by (2.14). We consider this Markov chain under the family of measure P V x,s , x ∈ N 0 , so that V 0 = V 0 = x deterministically. Thus, the measure P V x,s for ( V k ) is the equivalent of the measure P Z x for the Markov chain (Z k ) in Proposition C.1. By construction the Markov chain ( V k ) satisfies the monotonicity condition (A) of Proposition C.1. Also, by Lemmas 2.12 and 2.13, condition (B) is satisfied with α = µ s > 0 and β = (1 − δ) < 0. Finally, condition (C) is satisfied due to Lemma 2.11 and condition (D) is satisfied due to Lemma 2.14. Thus, the proposition is applicable and Proof of Proposition C.1 (Sketch). Our proof of Proposition C.1 follows very closely the approach used in [8] to prove Theorems 2.1 and 2.2, so we will provide only a rough sketch. Throughout we will use italics when referring to all theorems, lemmas, sections... etc. from [8], to distinguish from the corresponding items in our paper. For the sake of comparison we restate Theorems 2.1 and 2.2 below explicitly.
Theorem C.2 (Theorems 2.1 and 2.2 of [8]). Let P be a probability measure on cookie environments satisfying (IID), (BD), and (ELL), and let ( V k ) k≥0 be the associated backward branching process for ERW in this environment. Assume δ > 0 (where δ is given by (1.2)). Then there exists constants C 1 , C 2 > 0 such that The Markov chain (Z k ) k≥0 of our proposition is the equivalent of the process ( V k ) k≥0 , under the correspondence β = 1−δ. More precisely, if δ > 1 then ( V k ) satisfies conditions (A)-(D) of the proposition with β = 1 − δ and α = 1, and the decay rates in (C.6) and (C.7) are the same with β = 1 − δ. The value of α does not effect the decay rates in (C.6), only the constants C 5 and C 6 . along with the fact that the process ( V k ) is an irreducible Markov chain on state space N 0 that is monotonic in the sense of (C.1). Essentially the goal of these other sections is to show that the discrete process ( V k ) has the same type of scaling properties as the limiting diffusion ( Y t ), and to do this requires some technical work, using the estimates of Section 5.
Now, let us compare to our situation for the process (Z k ). Lemma C.3, given below in Section C.1, is the analog of Lemma 3.1, and Lemma C.4 in Section C.1 is the analog of Lemmas 3.2, 3.3, and 3.5. Also, Condition (D), which we assume to hold for the process (Z k ), is the analog of Lemma 5.1 in [8], where the corresponding property of the process ( V k ) is proven. Finally, Lemmas C.7, C.8, and C.9 in Section C.2 are, respectively, the analogs of Lemma 5.2, Lemma 5.3, and Corollary 5.5 in [8].
With the analogous results to the lemmas in Section 3 and Section 5 of [8] established, the proof or our Proposition C.1 for the process (Z k ) proceeds almost the same way, line by line, as the proof of Theorems 2.1 and 2.2 for the process ( V k ). So, we will not repeat it. However, for the sake of completeness, let us point out the few small differences in our analog lemmas from the originals.
1. The bound of exp(−a n/10 ) in our Lemma C.7-(i) is instead exp(−a n/4 ) in Lemma 5.2 -(i). This is irrelevant for how the lemma is applied; a bound of exp(−a cn ), for any c > 0, would be sufficient.
2. The concentration bounds (C.4) and (C.5) in our condition (D) are instead the following in Lemma 5.1 : The latter bounds are slightly stronger than ours. However, when x is large, with either (C.8) and (C.9) or with (C.4) and (C.5), the right hand sides become small only when y x 1/2 . In [8] the inequalities (C.8) and (C.9) are applied either when y is of order x or larger, or in other instances when y is of order x 2/3 , giving respectively bounds on the right hand side which are O(e −cy ) or O(e −cx 1/3 ). If instead (C.4) and (C.5) are used these bounds reduce to, respectively, O(e −y 1/4 ) and O(e −x 1/10 ). But, again, this is irrelevant for how the estimates are applied; any sort of stretched exponential decay would be sufficient. With our weaker estimates slightly larger error terms arise in the proofs, but they always remain negligible in comparison with all other terms.

C.1 Diffusion approximation lemmas
In the statement of the following lemma (Z k ) k≥0 is an irreducible, time-homogeneous Markov chain on state space N 0 satisfying (A)-(D), as in Proposition C.1. Lemma C.3 (Diffusion Approximation, Analog of Lemma 3.1 from [8]). Fix any 0 < < y < ∞, and let (Y (t)) t≥0 be the solution of where (B(t)) t≥0 is a standard 1-dimensional Brownian motion 9 .
Also, let Y (t) = Y (t ∧ τ Y ). For each n ∈ N, let (Z n,k ) k≥0 be a process with the distribution of (Z k ) k≥0 when Z 0 = yn , and define Y ,n (t) = Z n, nt ∧κ ,n n , where κ ,n = inf{k ≥ 0 : Z n,k ≤ n}.
In addition, let τ ,n = κ ,n /n. Then: −→ denotes convergence in distribution with respect to the Skorokhod J 1 topology.
The following properties of the diffusion Y (t) are established in [8] for the case α = 1, with β ≡ 1 − δ, but generalize to any α > 0. Indeed note that if Y α is the solution to (C.10) with a given value α and Y 1 is the solution to (C.10) with α = 1 (both with the same value of β and initial value y) then (Y α (t)) t≥0 d.
= (Y 1 (αt)) t≥0 . All properties of Y α follow from the corresponding properties for Y 1 and this relation.
In the remainder of Section C.1 we will use the generic probability measure P and corresponding expectation operator E for all random variables, including the Markov chains (Z n,k ) k≥0 of Lemma C.3. The proof of Lemma C.3 is based on the following result from [9].
where (B(t)) t≥0 is a standard 1-dimensional Brownian motion. Let (Z n,k ) k≥0 , n ∈ N, be integer-valued (time-homogenous) Markov chains such that Z n,0 = ny n , where y n → y as n → ∞, and such that conditions (i) and (ii) below are satisfied. 0 as x → ∞, such that: We will also need the following lemma, which is a minor extension of Lemma 3.3 in [12] where the same result is stated in the case D = 2.
Proof of Lemma C.3. For concreteness take N n = n 1/2 and define, for n ∈ N, integervalued (time homogeneous) Markov chains Z n ≡ (Z n,k ) k≥0 by Z n,0 = yn and P(Z n,k+1 = x + z|Z n,k = x) = P(Z n,k+1 = x + z|Z n,k = x), z ∈ Z and x ≥ N n P(Z n,k+1 = N n + z|Z n,k = N n ), z ∈ Z and x < N n . (C.12) Thus, Z n,0 = Z n,0 ≡ yn , and if the Markov chain (Z n,k ) k≥0 is currently at level x ≥ N n , then it has the same transition probabilities as the Markov chain (Z n,k ) k≥0 . On the other hand, if the Markov chain (Z n,k ) k≥0 is currently at some level x < N n , then the difference (Z n,k+1 − Z n,k ) between the current value and next value has the same law as the difference (Z n,k+1 − Z n,k ) when Z n,k = N n . With this construction the chains (Z n,k ) k≥0 and (Z n,k ) k≥0 can be naturally coupled until the first time k that they fall below level n (for all n large enough that N n = n 1/2 < n). Thus, by Lemma C.6 and the continuous mapping theorem it will suffice to show the following claim to establish the proposition.

Proof of Claim:
We apply Lemma C.5 with b = αβ and D = 2α. By definition (Z n,k ) k≥0 has the distribution of (Z k ) k≥0 when Z 0 = yn , where the Markov chain (Z k ) k≥0 is originally The first integral on the right hand side of (C.13) is at most n 3/2 . Using the definition (C.12) along with condition (C) in the statement of Proposition C.1 and the union bound estimate P(M n > x 1/2 ) ≤ T n max m≤an P |Z n,k − Z n,k−1 | > x 1/2 Z n,k−1 = m one finds the second integral on the right hand side of (C.13) is o(1), as n → ∞. This establishes (ii) of Lemma C.5, and, hence, the claim.

C.2 Exit probability lemmas
In the statement of the following lemmas (Z k ) k≥0 is an irreducible, time-homogeneous Markov chain on state space N 0 satisfying (A)-(D), as in Proposition C.1.
Lemma C.8 (Analog of Lemma 5.3 from [8]). For each a ∈ (1, 2] there is some a ∈ N such that if , m, u, x ∈ N satisfy a ≤ < m < u and x ∈ I m (where I m is as in Lemma Moreover, for each > 0 there exists c = c( ) > 0 such that We will prove Lemma C.7 below. The proof is similar to the proof of Lemma 5.2 in [8], but the remainder term r n k in (C.17) must be bounded differently, because we do not have the same explicit form for the transition probabilities of the Markov chain (Z k ). The proofs of Lemmas C.8 and C.9 are essentially the same as the proofs of their counterparts in [8], and are therefore omitted.
Proof of Lemma C.7. Part (i) follows directly from condition (D) in the statement of Proposition C.1 where the Markov chain (Z k ) is defined. To prove (ii), fix a ∈ (1, 2] and let g ∈ C ∞ c ([0, ∞)) be any non-negative function such that g(t) = t 1−β for t ∈ ( 2 3a , 3a 2 ). Then, for each n ∈ N, define a process W n ≡ (W n k ) k≥0 by W n k = g Z k∧γn a n .
Let F k = σ(Z 0 , . . . , Z k ) ⊇ σ(W n 0 , . . . , W n k ). At the end of the main proof we will establish the following two claims.
Claim 1: There exists some B 1 = B 1 (a) > 0 such that E Z x (γ n ) ≤ B 1 a n , for each n ∈ N and x ∈ I n .
Claim 2: The process (W n k ) k≥0 is "close" to being a martingale, when n is large, in the following sense: There exists some B 2 = B 2 (a) > 0 such that 3 n a.s., for each k ∈ N 0 , n ∈ N, and x ∈ I n .
We now show how these two claims can be used to prove the lemma. Assume Z 0 = x ∈ I n and define a process (R n k ) k≥0 by Observe that (W n k − R n k ) k≥0 is a martingale with initial value W n 0 . Moreover, by Claims 1 and 2, , for all k, this shows that (R n k ) k≥0 is uniformly integrable, and (W n k ) k≥0 is also uniformly integrable, since |W n k | ≤ g ∞ with probability 1. Thus, the martingale (W n k − R n k ) k≥0 is itself uniformly integrable, so we may apply the optional stopping theorem to conclude Combining (C.14) and (C.15) and using the fact that g(t) is equal to t 1−β on ( 2 3a , 3a 2 ) shows that Using part (i) and the fact that W n γn is bounded by g ∞ gives uniformly in the initial value Z 0 = x ∈ I n , where p ≡ P Z x (Z γn ≤ a n−1 ). Now, using the definition W n 0 = g(Z 0 /a n ) = g(x/a n ) shows also that W n 0 = 1 + O(a − 1 3 n ), uniformly in x ∈ I n . So, we have uniformly in x ∈ I n . Solving for p gives p = a 1−β /(1 + a 1−β ) + O(a − 1 3 n ), which implies (ii).
Proof of Claim 1: It will suffice to prove the claim for all sufficiently large n. Assume n is large enough that I n ⊂ (a n−1 , a n+1 ), and define γ − n = inf{k ≥ 0 : Z k ≤ a n−1 }. By monotonicity of the process (Z k ) k≥0 with respect to its initial condition P Z z (γ n < a n ) ≥ P Z z (γ − n < a n ) ≥ P Z a n+1 (γ − n < a n ) , for all z ∈ (a n−1 , a n+1 ).

D Proof of Propositions 4.3 and 4.4
In this section we use Propositions 4.1 and 4.2 to prove Propositions 4.3 and 4.4.
It is assumed throughout, if not otherwise specified, that R 0 = s and V 0 = 0. Thus, τ 0 = 0, V 0 = 0, and σ V 0 = τ τ V 0 (where (τ k ) k≥0 is as in (2.10)). The details of the proofs are somewhat technical, but the general ideas are fairly simple, so we will present these first before proceeding to the formal proofs. First, for Proposition 4.3, observe that if τ V 0 = m, for some large m, then So, by Proposition 4.1, for large x, Next, for Proposition 4.4, note that if k=0 V k is large, generally it will be the case that σ V 0 is large as well, and also that V k will be relatively large for most times k between 0 and σ V 0 (because the V k process is unlikely to remain close to 0 very long without hitting 0). Thus, by Lemma 2.11, the V j process, and also the V k process, will not fluctuate too much too rapidly, relative to their current values. So, very roughly speaking, we should expect that when either the sum on the right hand side or left hand side (equivalently both sums) are large. Therefore, by Proposition 4.2, we should expect that, for large x, Proof of Proposition 4.3. Fix any > 0. It will suffice to show that lim sup Proof of (D.1): Choose ρ > 0 sufficiently small that (1 + ρ) δ µ δ s (c 10 + ρ) + ρ ≤ c 10 µ δ s + = c 12 + . For n ∈ N, let m = m(n) = (1 + ρ)µ s n (it is not assumed that m is an integer).
• i max = max{i ≥ 0 : T i ≤ σ V 0 }, k max = T imax , and j max is the unique j such that τ jmax = k max .
• j max i = max{j : j ∈ J i } and j min i = min{j : j ∈ J i }.
Also, denote V max = max{V k : 0 ≤ k ≤ σ V 0 } and ∆ τ,j = τ j − τ j−1 , for j ≥ 1, and define the following events: At the end of the main proof we will establish the following claim.
Claim 1: For all sufficiently large n, By the triangle inequality,  Step (a) follows from the fact that G ⊂ A c 3 , step (b) follows from the fact that G ⊂ A c 1 , and step (c) follows from the fact that 2 < 1 .
Bound on (II) and (IV): First note that (IV ) ≡ µ s · τ V 0 j=jmax V j ≤ µ s · σ V 0 k=kmax V k ≡ µ s ·(II), so it will suffice to bound (II). Now, on the event G, Step (a) follows from the fact that G ⊂ A c 2 and G ⊂ A c 5 , and step (b) follows from the fact that 2 < 1 .
Bound on (III): On the event G, Step (a) follows from the fact that G ⊂ A c 2 , G ⊂ A c 3 , and G ⊂ A c 4 .
Step (b) follows from the relations j max ≤ τ V 0 and i max ≤ k max / n 1 ≤ σ V 0 / n 1 .
Step (c) follows from the inequality τ V 0 ≤ σ V 0 and the fact that G ⊂ A c 1 . Finally, Step (d) follows from the fact that 2 < 1 .
Using this estimate along with Claim 1 and Proposition 4.2 we can now establish the proposition.
Then for each n ≥ N 0 ≡ max{N 1 , N 2 , N 3 } we have Since ∈ (0, 1) was arbitrary it follows that lim n→∞ n δ/2 · P V 0,s This concludes the main proof of the proposition, and it remains only now to show Claim 1. To do this, though, we will first need to establish some auxiliary claims that will be used in its proof.
Proof. By Lemma 2.11, we have that for all sufficiently large n , 0 ≤ x ≤ 2n .
So, it will suffice to show that, for all sufficiently large n,  Thus, all the lemmas in Appendix C which hold for a Markov chain (Z k ) satisfying these properties apply to ( V k ). We will use Lemma C.9. By this lemma there exists some constant C = C(0) such that P V 0,s ( V max > n) = P V 0,s (τ V (n+1) + < τ V 0 ) ≤ C(n + 1) −δ , for all n ∈ N.