On recurrence of the multidimensional Lindley process

A Lindley process arises from classical studies in queueing theory and it usually reflects waiting times of customers in single server models. In this note we study recurrence of its higher dimensional counterpart under some mild assumptions on the tail behaviour of the underlying random walk. There are several links between the Lindley process and the associated random walk and we build upon such relations. We apply a method related to discrete subordination for random walks on the integer lattice together with various facts from the theory of fluctuations of random walks.


Introduction
Let (Y n ) n≥1 be a sequence of independent and identically distributed random variables with common distribution µ. A Lindley process (LP) is a discrete time stochastic process (W n ) n≥0 defined recursively by W 0 = w 0 ≥ 0 and W n = max{W n−1 − Y n , 0}, for n ≥ 1. (1) and we exploit them repeatedly in our paper. The main aim of the article is to present sufficient criteria for recurrence of the multidimensional counterpart of the LP.
We briefly state the well-known facts about recurrence of the LP in the one-dimensional case. Recall that an essential class for a Markov chain is a subset of the state space which is irreducible and absorbing. Given P(Y 1 > 0) > 0 there is only one essential class for (W n ) and it contains all the states that can be visited after the process reached 0. Thus to study its recurrence it suffices to concentrate on the behaviour at the origin.
We recall from Feller [14, Ch. XII, Sec. 2, Theorem 1] that there are three types of random walks: (S n ) is either oscillating, then lim inf n→∞ S n = −∞ and lim sup n→∞ S n = ∞; or it has a positive drift with lim n→∞ S n = ∞; or it has a negative drift meaning that lim n→∞ S n = −∞. In the first two cases we have P(τ (1) < ∞) = 1, whereas in the negative drift case È(τ(1) < ∞) < 1. By the correspondence between the ladder epochs of (S n ) and the return times of (W n ), we conclude that (W n ) is recurrent if and only if È(τ(1) < ∞) = 1. Therefore (W n ) is recurrent if and only if (S n ) is oscillating or if it has a positive drift and the following dichotomy holds true: 1) The process (W n ) is null recurrent if and only if (S n ) is oscillating. Thenτ =τ (1) has infinite first moment, cf. Gut [17,Theorem 9.1]. It happens if Y 1 = 0 or if µ is symmetric.
2) The process (W n ) is positive recurrent if and only if (S n ) has a positive drift. In this case τ is finite and W n converges weakly to the random variable M ∞ = sup{S 0 , S 1 , . . .} which is finite a.s. This holds in particular if |Y 1 | < ∞ and Y 1 > 0.
We observe that for a general distribution µ on R and the associated LP with an arbitrary initial random variable W 0 ≥ 0 which is independent of (Y n ) we have equality in law W n = max{M n−1 , W 0 + S n }, for n ≥ 1. This in turn implies that, given (Y 1 ) > 0, W n → M ∞ in law and thus the distribution of M ∞ is the unique stationary measure for (W n ), cf. also Diaconis and Freedman [9,Theorem 4.1].
As already mentioned, the LP comes up naturally in the framework of single server queues and thus it was extensively studied over the past decades, see e.g. the seminal paper by Kendall [18] with references therein and cf. also the books by Feller [14], Borovkov [7] and Asmussen [2]. Lindley [23] was the first who investigated the limit behaviour of (W n ) and discovered its connections with the Wiener-Hopf integral equations. More recently, asymptotics of the return probabilities of (W n ) were computed by Essifi, Peigné and Raschel [13].
The LP may be also viewed as a random walk with a certain barrier at zero and in this spirit we mention the reflected random walk (RRW), denoted by (X x n ) n≥0 , which is defined analogously to (W n ) but instead of the maximum function in (1) one sets X x 0 = x ≥ 0 and X x n = |X x n−1 − Y n |, for n ≥ 1. There is an obvious and striking resemblance between the two processes and in this note we take advantage of this aspect. In particular, the question of recurrence of RRW received much attention in the literature, see Peigné and Woess [24] with references therein and Kloas and Woess [20] for a treatment of the multidimensional case. From that perspective, one can use powerful methods related to stochastic dynamical systems in order to study recurrence of various processes. We partially apply such techniques to obtain a result concerning positive recurrence of the multidimensional LP in the final section.
The multidimensional counterpart of the LP arises from the studies on many server queueing models which were initiated by Kiefer and Wolfowitz [19]. In this note we aim at finding sufficient conditions for recurrence of the multidimensional LP as well as for a process of which some coordinates are Lindley processes and the other are ordinary random walks. We focus mainly on the two-dimensional lattice case but we also present a satisfactory result for higher dimensions. More precisely, the paper is organized as follows: Section 2 is devoted to the study of the asymptotic behaviour of a given random walk on integers which is evaluated at some random stopping times that are ladder epochs of a second independent random walk. Further, we take advantage of the result and construct a pair of examples of random walks with infinite second moment and investigate their recurrence. In section 3 we treat the two-dimensional LP in the lattice quadrant and investigate its recurrence under various assumptions on the tail behaviour of the underlying random walk. Among other methods, we apply the asymptotics obtained in Section 2. In the last paragraph we use a technique of local contractivity, which is related to stochastic dynamical systems, to study positive recurrence of the LP in higher dimensions.
Notation. We use the standard notation:

Subordination tools for random walks
In this section we focus on the local asymptotic behaviour of time-changed (subordinated) random walks on the integer numbers. We consider an increasing random walk which is responsible for a random change of time. We usually assume that this random walk has heavy tails and in our studies on the LP it is supposed to coincide with a sequence of ladder epochs of some other random walk. We then use the theory of regular variation to find exact asymptotics for the tails of the accordingly time-changed random walk. This enables us to present a vast class of examples of subordinated random walks and to handle the question concerning their recurrence. Although this topic is interesting in itself, our primary goal is to apply results of this section to find some criteria for the recurrence of the two-dimensional process of which one coordinate is a LP whereas the second is a random walk, cf. Theorem 3.5.
Let S n = Y 1 + . . . + Y n be an oscillating random walk such that S 0 = 0. We always assume that the distribution µ of the increment Y is supported by Z. Since (S n ) is oscillating, the first strict ascending ladder epoch τ = τ (1) = min{n ≥ 1 : S n > 0} is well-defined. Following Vatutin and Wachtel [31], for α, β ∈ R we consider the set For (α, β) ∈ A we write Y ∈ D(α, β) if the distribution of Y belongs to the domain of attraction of the stable law with characteristic function for c > 0. If 1 < α ≤ 2 we assume that (Y ) = 0. It is known by Doney [11] that if Y ∈ D(α, β) then where the parameter ρ is given by Moreover, condition (3) is equivalent to the existence of a slowly varying (at infinity) function ℓ such that , n → ∞.
Recall that a function f is regularly varying of index γ at infinity if lim x→∞ f (λx)/f (x) = λ γ , for all λ > 0, and f is called slowly varying if γ = 0. Equation (5) means that τ belongs to the domain of attraction of the one-sided stable law of index ρ. According to [31,Theorem 3] and [32, Theorem 10] for α = 1 or β = −1 we also have the following local result with the same slowly varying function ℓ as in (5).
We study the local asymptotic behaviour of a random walk which is evaluated at ladder epochs of the random walk (S n ). More precisely, we consider a finite range and centered random walk (Z n ) on Z (i.e. the support of the law of Z 1 is bounded and Z 1 = 0) and we look more closely at the tail decay of the random variable Z τ , where τ is the first strictly ascending ladder epoch of (S n ). The proof of the theorem below is based on the similar result obtained in Bendikov and Cygan [4] for the Green function of the subordinated random walk in Z d but it requires numerous improvements and adjustments to the present setting. To our best knowledge, this is the first result of this type in the centred but not necessarily symmetric case.
We emphasise that the scope of the theorem is wider than it is stated. One can consider an arbitrary increasing random walk (η n ) on non-negative integers and then a new subordinated random walk (Z ηn ). The result is applicable given that the increments of (η n ) behave locally as in (6). We obtain the local behaviour of the subordinated random walk without any assumption on the structure of the distribution of η 1 , cf. Bendikov, Cygan and Trojan [5] for the detailed discussion on the asymptotic behaviour of subordinated random walks under the assumption that the Laplace transform of η 1 is governed by a Bernstein function.
Theorem 2.1. Suppose that (S n ) is an oscillating random walk such that its increment Y ∈ D(α, β). Let τ be the first strict ascending ladder epoch of (S n ). Assume that (S n ) is independent of (Z n ), then , as |x| → ∞, where ℓ is the slowly varying function from (6) and Let p n (x) = ( √ 2πnσ) −1 e −|x| 2 /(2σ 2 n) and E(n, x) = p n (x) − p n (x). Applying Lawler and Limic [21, Theorem 2.1.1] (see the discussion following Proposition 2.1.2), we get that for a centered irreducible and aperiodic random walk in Z d with finite third moment there is some C > 0 such that We decompose I 2 (x) into two parts and first we establish that I 22 (x) = o (|x| −2ρ−1 /ℓ(|x| 2 )). Our assumptions combined with (6) and (9) for d = 1 imply that for some C > 0 dt, as |x| → ∞.
By Bingham, Goldie and Teugels [6, Proposition 1.5.10], we have , as |x| → ∞ and whence, for |x| large enough, We show that the right hand side of the last inequality tends to 0. To do so we find an upper bound for the last fraction with the slowly varying function. For that we apply Potter bounds [6, Theorem 1.5.6] which assure that for any ε > 0 there exists X ≥ 0 such that If we choose ε = 1 we obtain that ℓ(|x| 2 ) ≤ 2|x| 1/3 ℓ |x| 5/3 , for |x| large enough, and therefore as desired. Next, with I 21 (x) we proceed as follows. For |x| → ∞, We choose an arbitrary ε ∈ (0, (2ρ + 1)/2). By Potter bounds we get that for |x| big enough, With this estimate we are allowed to apply the dominated convergence theorem to the last integral in (10) which thus converges to Γ(ρ + 1/2). Hence , as |x| → ∞.
We are left to show that ). Here we use the assumption that the random walk (Z n ) has finite range. The Gaussian upper bound of Alexopoulos [1, Theorem 1.8] yields that there is c > 0 such that p n (x) ≤ cn −1/2 e −cx 2 /n , for all n ∈ N and x ∈ Z. We have and therefore we obtain Observe that x −ν ℓ(|x| 2 ) tends to 0 for any ν > 0. We conclude that dt.
Hence, by [6, Prop. 1.5.10], 1 − F (x) ∼ C(ρ)/(2ρ x 2ρ ℓ(x 2 )) at infinity. Asymptotics (7) are symmetric in x and whence one easily shows that F (−x)/(1 − F (x)) tends to 1 as x goes to infinity. We conclude that The conditions of Gnedenko and Kolmogorov [16, §35, Thm. 2] are fulfilled and we obtain that Z τ belongs to the domain of attraction of the stable law of index 2ρ. Since ) tends to 1/2 as x goes to infinity, the skewness parameter β equals 0.
We next present a variety of examples of random walks on Z which are constructed according to the discussed procedure of the random change of time. For that reason we consider a sequence of strict ascending ladder times τ (n) which are defined via τ (0) = 0, τ (k + 1) = inf{n > τ (k) : S n > S τ (k) }.
In the (critical) case ρ = 1 2 we give an example of a recurrent random walk (Z τ (n) ) with increments that have no finite first moment. First we recall an important notion of α-conjugate pairs from the theory of regular variation which we extract from Doney [10].
The function ℓ * α is called the α-conjugate of the function ℓ. The way to remember the meaning of ℓ * α is that y ∼ x α ℓ(x), when x goes to infinity, if and only if x ∼ y 1/α ℓ * α (y), as y goes to infinity. One easily checks that if This holds for many slowly varying functions, for example log x, log log x etc.
Example 2.4. Let (S n ) be an oscillating random walk such that its increment Y belongs to the domain of attraction of the normal distribution. It is known from Feller [14, Ch. XVII, Sec. 5, Thm. 1a] that it holds if and only if the truncated variance of Y is slowly varying, that is , as y → ∞, for some slowly varying function ℓ. We additionally assume that (Y 2 ) = ∞ and (S τ ) < ∞.
Next, if we take a symmetric random walk (Z n ) then Theorem 2.1 gives us , as x → ∞.
As in the proof of Proposition 2.3, , as x → ∞, , as t → 0.
We end this section with a result concerning Theorem 2.1 when the increments of the random walk (S n ) have finite second moment. Proposition 2.5. Let (S n ) be an oscillating random walk with the increment Y having finite second moment and let (Z n ) be a centred and finite range random walk on Z independent of (S n ). Then there is some C > 0 such that lim y→∞ y 2 È (Z τ = y) = C, (14) and in this case (|Z τ |) = ∞. Equation (14) holds also when Y is symmetric and has a density.
Proof. The proof is similar to that of Theorem 2.1, but in place of formula (6) one uses the result byÉppel' [12], We prove that (|Z τ |) = ∞. We set F (x) = È(Z τ ≤ x) and by (14) we get that, for some which means that 1−F (n) is regularly varying at infinity of index −1. In view of symmetry this implies that Z τ is in the domain of attraction of the Cauchy law. Therefore, if C is the distribution function of the Cauchy law, then there are sequences b n > 0 and a n > 0 such that F * n (b n x + b n a n ) → C(x), for all x as n goes to infinity. We find the asymptotic behaviour of the normalizing sequence (b n ). It is known from [16] that (b n ) satisfies 1 − F (b n ) ∼ C 2 n at infinity and, by (15), we obtain that b n ∼ C 3 n at infinity. Finally, by Tucker [29], the integral |x|dF (x) is finite if and only if n≥1 n −2 b n < ∞ and the proof is finished.

Multidimensional Lindley process
We proceed to study recurrence of the LP in higher dimensions. We start by discussing the two-dimensional case for which we apply various probabilistic methods including arguments from renewal theory as well as Theorem 2.1 from Section 2. In the last subsection we give a result about positive recurrence of the LP. To prove it we use a technique of local contractivity which has its roots in stochastic dynamical systems.

Two-dimensional LP.
Let (W i n ), i = 1, 2, be two Lindley processes as defined in (1) with the underlying random walks (S i n ) with increments Y i which have distributions µ i supported in Z. We consider a process (W 1 n , W 2 n ) in the lattice quadrant N 0 × N 0 and we assume that Then the probability to reach (0, 0) from an arbitrary state after finitely many steps is positive. Thus, the origin and all the states that can be reached from it build a unique essential class. Without our assumption it may happen that some states will never be reached by the process even when gcd(supp µ i ) = 1, see the following example. We also emphasize that a precise description of essential classes in a general case is a very hard task. . Then gcd(supp µ i ) = 1 and clearly the two coordinates of (W 1 n , W 2 n ) are transient and whence also the two dimensional process is transient. In this case every point in N 0 × N 0 will be visited at most one time a.s. On the other hand, setting µ = 1 4 δ (−1,1) + δ (−1,2) + δ (1,−1) + δ (2,−1) we also have gcd(supp µ i ) = 1 with positive recurrent coordinates and the two-dimensional LP (W 1 n , W 2 n ) will never reach (0, 0) in this case. We notice however that (W 1 n , W 2 n ) is positive recurrent in its essential class, cf. Theorem 3.7.
We begin our discussion on recurrence with a very simple but fruitful lemma. Lemma 3.2. Letτ 1 (n) be the n-th non-strict ascending ladder epoch of (S 1 n ). Assume that the first coordinate process (W 1 n ) and the projected process (0, W 2 τ 1 (n) ) are recurrent then the two-dimensional process (W 1 n , W 2 n ) is recurrent. If W 1 n and W 2 τ 1 (n) are positive recurrent then (W 1 n , W 2 n ) is positive recurrent.
Proof. Let T and T be the first return times to the point (0, 0) of (W 1 n , W 2 n ) and (0, W 2 τ 1 (n) ) respectively. By the assumption, T is a.s. finite. We claim that T =τ 1 ( T ). Indeed, we have where we used the fact that (W 1 n , W 2 n ) attains the value (0, 0) only if n ∈ {τ 1 (k) : k ≥ 0}. This in turn implies that T is a.s. finite and we get the first part of the result.
For the positive recurrent case, we consider a random walkτ 1 (n) = ξ 1 + . . . + ξ n with independent increments ξ i =τ 1 (i) −τ 1 (i − 1) which have the same law asτ 1 (1). We build a new filtration {F n } n≥1 given by and notice that, since { T ≤ n} ∈ F n , T is a stopping time with respect to the filtration {F n } n≥1 . Moreover, the increments of the random walk (τ 1 (n)) have the form and therefore ξ n is independent of F n−1 . This allows us to apply Wald's identity in the form τ 1 ( T ) = τ 1 (1) T < ∞ which implies positive recurrence of (W 1 n , W 2 n ). In the next proposition we apply Lemma 3.2 and combine it with an argument which comes from renewal theory. Proposition 3.3. Suppose that the random walks (S i n ), i = 1, 2, are independent and oscillating with increments Y i ∈ D(α, β) satisfying ρ 1 + ρ 2 > 1, where ρ i are defined in (4). Then the process (W 1 n , W 2 n ) is null recurrent. Proof. Since (S 1 n ) is oscillating, (W 1 n ) is null recurrent. Let τ 1 (n) denote the n-th strict ladder epoch of (S 1 n ). We show that the Green function G(0, 0) of the process (0, W 2 τ 1 (n) ) is infinite and whence it is a recurrent Markov chain. Evidently, this implies recurrence of the process (0, W 2 τ 1 (n) ) which in view of Lemma 3.2 forces that (W 1 n , W 2 n ) is recurrent. Moreover, since (W 1 n ) is null recurrent, the two-dimensional process is also null recurrent as claimed.
We also present a positive result in the case when ρ 1 = ρ 2 = 1/2. Proposition 3.4. If the random walks (S i n ), i = 1, 2, are independent, centered and with finite second moment then (W 1 n , W 2 n ) is null recurrent. Proof. The proof is similar as that of Proposition 3.3 but instead of asymptotics (5) we use the result byÉppel' [12,Theorem 1], that is This allows us to show that G(0, 0) is infinite and we again get the result.
Our next result concerns recurrence of the two-dimensional process (W n , Z n ), where in the first coordinate (W n ) is a LP with the underlying random walk S n = Y 1 + . . . + Y n and the second coordinate (Z n ) is a random walk on Z with increments V 1 , V 2 , . . .. Theorem 3.5. The two-dimensional process (W n , Z n ) is recurrent in each of the following cases. 1. If (W n ) is positive recurrent and (Z n ) is a centered random walk. 2. If (S n ) is oscillating with the increment Y ∈ D(α, β) such that 1/2 < ρ < 1, and (Z n ) is a symmetric finite range random walk independent of (S n ). 3. If (W n ) is null recurrent with (|Y | 2 ) < ∞ and independent of (Z n ) which we assume to be a symmetric random walk of finite range.
Proof. Recall that the k-th return time of the first coordinate (W n ) to 0 is equal to the k-th non-strict ascending ladder epochτ (k) of the underlying random walk (S n ). Thus, the return times to the origin of (W n , Z n ) are the same as for the induced random walk (Zτ (n) ). Moreover, the process (Zτ (n) ) is recurrent if the random walk (Z τ (n) ) is recurrent, where τ (n) is the n-th strict ascending ladder epoch of (S n ).
To prove the first assertion we notice that as W n is positive recurrent we known that τ < ∞, where τ = τ (1). We also notice that τ is a stopping time for the two-dimensional random walk (S n , Z n ) which implies that the event {τ ≤ n} is independent of V n+1 and whence we are allowed to apply the Wald's identity in the form Z τ = Z 1 τ = 0. Therefore (Z τ (n) ) is recurrent.
The second claim is a direct consequence of Proposition 2.3. In the last case we have (Y ) = 0 and, as follows by Proposition 2.5, the following asymptotic relation holds Since (Z τ (n) ) is a symmetric random walk, we conclude, for instance by Spitzer [28, Sec. 8, E2], that it is recurrent.
LP in higher dimensions.
We consider the multidimensional Lindley process (W 1 n , . . . , W d n ) in N 0 × · · · × N 0 . The underlying random walks (S i n ) are governed by distributions µ i which are supported by Z and such that È(Y i > 0) > 0.
The following theorem treats positive recurrence of the multidimensional LP. Its proof uses elements from the theory of stochastic dynamical systems and thus we briefly present necessary definitions and facts, see Peigné and Woess [25] and [26] for more detailed description where in particular the substantial PhD work of Benda [3] is outlined.
Let (X, d) be a proper metric space and denote by C = C(X) the monoid of all continuous functions from X to X equipped with the topology of uniform convergence on compact sets. Fix a probability space (Ω, P) and consider a sequence (F n ) n≥1 of independent and identically distributed C-valued random functions with a common distributioñ µ. The corresponding stochastic dynamical system ω → X x n (ω) is given by x ∈ X, n ≥ 1. For a LP on N 0 × · · · × N 0 we have F n (x) = max{x − Y n , 0} and these mappings are contractions so that we can restrict our attention to the set L ⊂ C of all Lipschitz mappings with Lipschitz constants ≤ 1. Notice that if µ is the distribution of Y n thenμ is the image of µ under y → f y , f y (x) = max{x − y, 0}. A stochastic dynamical system is called conservative if P lim inf n→∞ d(X x n , x) < ∞ = 1, for every x ∈ X, and it is locally contractive if for every x ∈ X and every compact set K ⊂ X, P (d(X x n , X y n )1 K (X x n ) → 0, for all y ∈ X) = 1. For ω ∈ Ω we consider the set L x (ω) of all accumulation points of the sequence (X x n (ω)) n≥0 in X. The following lemma allows us to show that there is only one essential class for the multidimensional LP, see [25,Lemma (2.5)].
Lemma 3.6. For a conservative and locally contractive stochastic dynamical system, there exists a set L ⊂ X such that P (L x (·) = L, for all x ∈ X) = 1.
We now prove the main theorem of this section. . The stationary measure is the distribution of a random variable U which is the limit of the backward process where F n (x) = max{x − Y n , 0}. In particular, the multidimensional process (W 1 n , . . . , W d n ) is positive recurrent in its unique essential class. Proof. We start by showing that the LP (W i n ) is locally contractive in each coordinate. We set f y (x) = max{x − y, 0} and consider random contractions F i n = f Y i n with lawμ i which is the image of µ i under the mapping f y . Let S i be the closed sub-semigroup of L generated by supp(μ i ). Our aim is to show that there is a constant function in S i and this, in view of [25,Corollary 4.4], will force local contractivity. The claim follows by the assumption P(Y i > 0) > 0. Indeed, there is y > 0 such that for any x ∈ R there is N x > 1 such that for all n ≥ N x we obtain that the n-fold composition f y • · · · • f y (x) ≡ 0, and thus the null-function lies in S i as desired.
Next, by positive recurrence, each (W i n ) has a unique invariant probability measure, say ν i . This together with local contractivity imply that for any starting point x i we have the a.s.-convergence of the backward process F i 1 • · · · • F i n (x i ) → U i , where U i is a random variable with distribution ν i . This goes back to Leguesdron [22], compare with [20,Prop. 2.6]. Since we have convergence of all coordinates, we get (16). Applying Furstenberg's contraction principle, see [25,Prop. 1.3], we conclude that the distribution ν of the random vector (U 1 , U 2 , . . . , U d ) is the unique invariant probability measure for (W 1 n , . . . , W d n ) which is equivalent to positive recurrence. It is left to show that there is only one essential class. Indeed, by the very definition, local contractivity of the coordinates implies that the multidimensional process (W 1 n , . . . , W d n ) is locally contractive as well. Since we have proved it is recurrent, it must be conservative. In our case, the Lindley process lives on the grid and thus the deterministic set L ⊂ N 0 × · · · × N 0 from Lemma 3.6 is such that, independently of the starting point, P (W 1 n , . . . , W d n ) = l for infinitely many n = 1, for every l ∈ L. We clearly conclude that L is the unique essential class of (W 1 n , . . . , W d n ). Remark 3.8. To prove the existence of positive recurrent states of (W 1 n , . . . , W d n ) there is a simple argument which was presented to us by Nina Gantert and mentioned already in the context of RRW in [20,Remark (4.10)]. However, this argument yields no understanding of the number of essential classes and their absorption properties. The use of local contractivity leads to an answer and additional insight, namely a.s. convergence of the backward process.