Stability of a Stochastic Ring Network

In this paper we establish a necessary and sufficient stability condition for a stochastic ring network. Such networks naturally appear in a variety of applications within communication, computer, and road traffic systems. They typically involve multiple customer types and some form of priority structure to decide which customer receives service. These two system features tend to complicate the issue of identifying a stability condition, but we demonstrate how the ring topology can be leveraged to solve the problem.

Notation. Unless otherwise specified, all random variables and processes are defined on a common probability space ( Ω, F, P ). We write Z + for the non-negative integers, N for the natural numbers (i.e., N ≡ Z + \ {0}), R for the real numbers, and R + for the set of non-negative reals. For any dimension d, | · | denotes the L 1 -norm on elements of R d . If a and b are real numbers, then a ∧ b is the minimum and a ∨ b the maximum of a and b.

Model description
As we mentioned in the introduction, the model we consider is a stochastic ring network that has a variety of application domains, but was originally introduced as a model for a roundabout in [29]. In this section, to describe the model, we stay close to the original formulation as a roundabout model and use the related (road traffic) terminology. For technical reasons, which we will explain in Section 6, we consider a version of the model that differs slightly from the roundabout model in [29]. For the model we consider, we identify and prove a necessary and sufficient condition for stability. This result also identifies the global stability regions for the model in [29] and the slotted-ring model in [31], as we formally show in Section 6.
The roundabout model is a slotted ring consisting of L cells, with on-ramp queues in front of each cell. The cells and queues are indexed by i ∈ {1, . . . , L}, with cell 1 adjacent to cell L; using this cyclic structure, we allow ourselves to use the index i + L to refer to cell/queue i. The presence of vehicles on the roundabout is modeled by the state of the cells. Each cell can either be empty, or contain a vehicle that has entered the roundabout at some cell j. In the first case we say the state of cell i is 0 and in the second case we say that its state is j; we will also say that a cell is occupied by a vehicle of type j when the state of the cell is j. The queues model vehicles waiting to enter the roundabout, and their state is a number in Z + .
The model has discrete-time dynamics. The main idea is that vehicles arrive via the queues to the roundabout, traverse a number of cells to an off-ramp, and depart the roundabout. At each time t ∈ Z + , if there are vehicles in queue i and cell i is empty, one vehicle from the queue will enter the roundabout and occupy cell i + 1 at time t + 1. If cell i is occupied by a vehicle of type j, then the vehicle will either depart from the system with probability q ij ∈ [0, 1], or occupy cell i + 1 at the next time step. We emphasize that the probability q ij depends on j to reflect that the cell at which a vehicle leaves the system may depend on where it entered. To model arriving vehicles, we impose that at every time step, a new vehicle will arrive to queue i with probability p i ∈ [0, 1]. We assume that all decisions whether a vehicle arrives at a cell i and whether a vehicle of type j will leave cell i (if present), are made independently of each other and of the current state and history of the process (see Appendix A for an explicit construction of the process). Note that on-ramps and off-ramps can be removed by setting arrival or departure probabilities equal to zero.
We now provide a more precise account of these dynamics by formulating update rules for the system (as is customary for cellular automata). These rules are local in the sense that we only need to know the current joint states of queue i and cell i, and can disregard the remainder of the system, to determine the new states of queue i and cell i + 1 (in accordance with the cellular automata paradigm). It turns out that we need to distinguish three cases: Case 1: cell i and queue i are both empty. In this case, no vehicle can enter the roundabout from queue i and cell i + 1 will be empty the next time step. If a new vehicle arrives to queue i (which happens with probability p i ), then queue i will have length 1 at the next time step. Otherwise, it will have length zero.
Case 2: cell i is empty and queue i is not empty. Then, the vehicle at the front of queue i will enter the roundabout and move on to cell i + 1 in the process. Thus, cell i + 1 will be in state i one unit of time later. Queue i either remains at its current length (if a new vehicle arrives to queue i), or its length decreases by one.
Case 3: cell i is occupied by a type-j vehicle. In this case, queue i is blocked, hence its length will grow by one if a new vehicle arrives, or will stay the same otherwise. The vehicle in cell i will either leave the system with probability q ij , in which case cell i + 1 will be empty one unit of time later, or the vehicle remains in the system, so that cell i + 1 will be in state j at the next time step.
Having specified the precise dynamics of the model, we observe that if for type-j vehicles, q ij = 0 for each i ∈ {1, . . . , L}, then such vehicles cannot leave the network. We therefore require that L ℓ=1 (1 − q ℓj ) < 1 for each j ∈ {1, . . . , L}, so that any vehicle eventually leaves the network. Additionally, note that when p i = 1 for some i ∈ {1, . . . , L}, then queue i can never decrease in length. Since our focus is on stability, we impose p i ∈ [0, 1) for all i ∈ {1, . . . , L}.

Stability condition and main result
The model under consideration is a discrete-time Markov chain, which we denote by X 1 (·) = {X 1 (t) : t ∈ Z + }. The associated state space is S 1 := (Z + × {0, . . . , L}) L , the set of vectors in R 2L that describe the state of each cell and queue. We observe that X 1 (·) is irreducible on the set of states that can be reached from the empty state, since from any state, with strictly positive probability, we can empty the system in a finite number of steps. Note that specific choices of the p i and q ij can make it impossible to reach all states in the state space. In addition, X 1 (·) is aperiodic as we can remain in the empty state for an arbitrary (finite) number of time steps, with strictly positive probability.
Following [7,13,23], we say that X 1 (·) is stable when it is positive recurrent. In Section 3.1, we will derive an explicit expression for the marginal stationary distribution of the states of the cells under the assumption of stability. This result was already obtained in [29], but here we use a different method based on the system's offered load. To be more precise, we show that when the system is stable, the marginal stationary probability that cell i is in state j is equal to the expected number of times that a vehicle of type j will occupy cell i during the time it spends on the roundabout. This connection leads us to formulate a necessary and sufficient condition for stability in Section 3.2, which is our main result. Moreover, this connection plays an important role in our proof that the condition is sufficient in Section 5. We prove the necessity of the condition for stability at the end of Section 3.2.

Marginal stationary distribution
In our formulation of the model given above, we update the system at each time step by checking for external arrivals, and deciding for each vehicle on the roundabout whether or not it leaves. As a consequence of the dynamics, the time spent on the roundabout by each vehicle that enters the roundabout at cell j is an independent copy of a generic random variable T j , the distribution of which is completely determined by the parameters q ij . We use this fact in this subsection to derive the marginal stationary distribution of the states of the cells when the system is stable, and again in Section 5 to prove our main result.
To be precise, T j is a random variable with distribution given by where we adopt the convention that the empty product is equal to 1, and define q ℓj for ℓ > L by setting q ℓj := q ij whenever ℓ ≡ i (mod L). Observe that the model dynamics determine the distribution of the T j and that different distributions could be considered as well. Such model extensions do not significantly impact our analysis and are discussed in Section 6.3. Using the 4 above definition of T j we will now show that, if the model is stable, the marginal stationary probability that a cell i is in state j is given by and Proposition 3.1 (Marginal stationary distribution). If the model is stable, then the marginal stationary probability that cell i is in state j is given by (1)-(2).
Proof. Assume that the model is stable. Consider the case that i and j satisfy 1 ≤ j < i ≤ L.
Then the probability that a vehicle that enters the roundabout at cell j will exit at cell i is given by .
We conclude that this expression times p j is the rate at which vehicles arrive that are of type j and intend to leave the roundabout at cell i. But since the model is stable, this rate must be equal to the rate at which vehicles of type j leave the system at cell i. By the ergodic theorem, the latter rate equals π ij q ij , where π ij denotes the marginal stationary probability that cell i contains a vehicle of type j. This proves (1) when j < i. The proof in the case 1 ≤ i ≤ j ≤ L is similar.
We will now show that the π ij can also be interpreted as the fraction of time cell i would be occupied by a type-j vehicle if arriving vehicles entered the roundabout at cell j at the rate p j . To this end, we define N ij as the number of times a vehicle of type j that spends time T j on the roundabout will occupy cell i before leaving. Specifically, for i, j ∈ {1, 2, . . . , L}, we define and set b ij := EN ij . Proof. As before, first consider the case 1 ≤ j < i ≤ L. From the definition of N ij it follows that for n ≥ 0, P(N ij ≥ n + 1) = P(T j ≥ i − j + nL), and by induction in k, it follows from the definition of T j that (1 − q ℓj ) for all k ≥ 1.
Combining these observations, we obtain hence by (1), The proof in the case 1 ≤ i ≤ j ≤ L is similar.
Remark 3.3. Lemma 3.2 tells us that if the roundabout manages to process all vehicles arriving at cell j at rate p j , then π ij = p j b ij is the expected fraction of time cell i is occupied by a type-j vehicle. We do not need stability for this quantity to be meaningful, since π ij is only a measure of the load that is imposed on the system. The question of stability deals with whether or not all the vehicles actually manage to get onto the roundabout. If the system is unstable, then the π ij still describe the load that is offered to the system, but we cannot use ergodicity to conclude that the fraction of time cell i will be occupied by a vehicle of type j is given by π ij . Our second characterization of the π ij in terms of EN ij is going to play an important role in proving the main theorem (Theorem 3.4 below).

Condition for stability
Under the assumption that the system is stable, the Markov chain X 1 (·) has a stationary distribution π. In the previous section we have shown that the vector {π i0 , π i1 , . . . , π iL } given by (1) and (2) is then the marginal stationary distribution of cell i, for i = 1, 2, . . . , L. In particular, π i0 is the stationary rate at which cell i is empty when the system is stable. Since vehicles arrive at cell i at rate p i and can only enter onto the roundabout when the cell is empty, it seems reasonable to believe that the system cannot be stable if p i > π i0 for some cell i. Conversely, one would suspect that if p i < π i0 for all cells i, the cells will be vacant often enough to prevent the queues from blowing up. It turns out that the latter condition is in fact not only sufficient, but also necessary for the system to be stable: Theorem 3.4 (Main result). A necessary and sufficient condition for the roundabout model to be stable (by which we mean that the Markov chain X 1 (·) is positive recurrent) is that for all i ∈ {1, 2, . . . , L}.
The hard part of this theorem is to prove that the stated condition is sufficient for the stability of the system. We close this section with a proof of the necessity of the stability condition; Sections 4 and 5 are devoted to proving the sufficiency.
Proposition 3.5. The condition that p i < π i0 for all i ∈ {1, 2, . . . , L} is a necessary condition for the stability of the roundabout model.
Proof. We know that the Markov chain X 1 (·) is aperiodic and irreducible. Assume furthermore that the system is stable, that is, that X 1 (·) is also positive recurrent. Then X 1 (·) has a stationary distribution π, and since the Markov chain is irreducible, the state in which every cell and every queue is empty must have a strictly positive probability, which we denote by π ∅ , under the stationary distribution.
Suppose that we start our Markov chain from the stationary distribution π. Let C i (t) denote the state of cell i at time t, and let Q i (t) be the length of queue i at time t. Write A i (t) for the event that a new vehicle arrives at cell i at time t. Then the queue length process satisfies We now want to take n → ∞ in (3). On the one hand, since we assumed the model is stable, it is clear that we must have that On the other hand, the first term on the right hand side in (3) clearly converges almost surely to 0 as n → ∞, whereas the second term converges a.s. to p i by the strong law of large numbers, and the third term converges a.s. to −π i0 by the ergodic theorem. As for the last term in (3), we observe that the event that the system is completely empty is a subset of the event {Q i (t) = C i (t) = 0}, so that by the ergodic theorem we can conclude that Combining these observations, it follows that almost surely, Hence, p i must be strictly smaller than π i0 for every cell i if the roundabout model is stable.

Multiclass queueing network formulation
To prove sufficiency of the stability condition in Theorem 3.4, we formulate in this section a multiclass network (explained below) that has essentially the same dynamics as the roundabout model from Section 2. That is, for each choice of parameters p i and q ij , i, j ∈ {1, . . . , L}, for the roundabout model we define an analogous multiclass network, using the same parameters to define the appropriate exogenous arrival and routing processes in the network. We shall refer to a choice of these parameters as a parameter setting. The two model formulations can be coupled on a sample-path level, formalized in Lemma 4.1 below. This allows us in Section 5 to utilize the powerful fluid model framework for multiclass queueing networks to prove stability of the multiclass-network formulation of our model, hence proving it for the model in the original formulation as well.
A multiclass queueing network, or simply multiclass network, is a network consisting of a finite collection of (single server) stations serving customers from a finite number of customer classes. Each customer class has its own queue at one of the stations, with its own exogenous arrival process and service time distribution. After service completion, customers are either routed to another queue (and associated customer class), or depart from the network. In addition, a multiclass network can have other characteristics relating to specific model applications, such as customer class priorities in stations and blocking features.
Fix a parameter setting. To map the roundabout model to a multiclass network, we identify each cell i with a station i in the multiclass network. Vehicles that arrive from outside to the on-ramp of cell i become customers of class (i, 0) in the multiclass network, and vehicles of type j that occupy cell i become class (i, j) customers. Thus, there are L stations and L(L + 1) = L 2 + L customer classes in the multiclass network. The set of customer classes is and station i serves the customers of all classes (i, j) with j ∈ {0, . . . , L}. We will now define the exogenous arrival processes, service time distributions, routing policy, and priority structure that will make the multiclass network mimic the dynamics of the roundabout model. First, to account for the external arrivals, the exogenous arrival process for class (i, 0) customers clearly must have independent, geometrically distributed inter-arrival times with parameter p i . All other customer classes (i, j) with i, j ∈ {1, . . . , L} have no exogenous arrivals; such customers can appear in the network only because they arrive as a class (j, 0) customer and are subsequently routed in the network to become a class (i, j) customer. This corresponds in the roundabout model to a vehicle entering onto the roundabout at cell j, and then traversing a number of cells to reach cell i.
To mimic this behavior correctly in the multiclass network, we impose that the service times are exactly one unit of time for each class of customer. Furthermore, after completing service at station i, a class (i, 0) customer is routed to the next station and becomes a customer of class ((i mod L) + 1, i). Likewise, a class (i, j) customer is either routed to the next station with probability 1 − q ij , or leaves the network. This routing policy is captured by the routing matrix P . The rows and columns of this matrix are indexed by the set of customer classes K, and the entry P ij,kℓ defines the probability that a class (i, j) customer is routed to become a class (k, ℓ) customer after service completion, and is therefore given by Next, we need to account for the fact that in the roundabout model, any vehicle that occupies a cell blocks the corresponding on-ramp and is guaranteed to move out of that cell after one unit of time. We therefore impose the following priority structure on the multiclass network: for each i ∈ {1, . . . , L}, customers of classes (i, j) with j ∈ {1, . . . , L} have priority over class (i, 0) customers. This means that class (i, 0) customers only receive service at station i if the queues associated with class (i, j) customers with j = 0 are all empty.
Finally, we need to consider the restriction in the roundabout model that each cell can be occupied by at most one vehicle at a time. For the multiclass network this means that we have to guarantee that, for each station i, the combined number of customers of classes (i, j) with j = 0 is at most 1 at all times. But we can achieve this by simply imposing this condition on the initial state of the network: if at time 0, for each station i, the combined number of customers of classes (i, j) with j ∈ {1, . . . , L} is at most 1, then the same is automatically true for all times t ∈ Z + , because all service times are exactly 1, class (i, j) customers with j = 0 have priority over class (i, 0) customers, and customers can only be routed to one of the queues at station i by the previous station in the network. Henceforth, we only consider the multiclass network under this condition on the initial state.
It is well known (see, e.g., [13, Section 2] or [7, Section 4.1]) that a multiclass network can be represented as a Markov chain. In our case, due to the geometric inter-arrival times, deterministic service times and discrete-time nature of the network, we can represent it as a discrete-time Markov chain with a discrete state space. We denote this Markov chain by X 2 (·), and take as our state space Thus, each state is a vector in R L 2 +L that tells us, for every customer class, how many of those customers are present in the network. Because of the way we constructed the multiclass network, there is a natural bijection between the two state spaces S 1 and S 2 , and it is possible to couple the two Markov chains X 1 (·) and X 2 (·) in such a way that they follow the same sample paths up to this bijection. We formalize this claim in the following lemma, the proof of which is given for completeness in Appendix A: Lemma 4.1. For any fixed parameter setting, there exists a coupling of X 1 (·) and X 2 (·) and a bijection f : S 1 → S 2 such that for every ω ∈ Ω, if f X 1 (ω, 0) = X 2 (ω, 0), then for all t ∈ Z + . Lemma 4.1 allows us to complete the proof of Theorem 3.4 by proving that the stability condition implies stability of X 2 (·). For this we can rely on the theory of fluid limits and Foster-Lyapunov functions, cf. [7,8,13,21].

Proof of the main result
This section of the paper is dedicated to proving that X 2 (·) is stable if the stability condition in Theorem 3.4 is satisfied. By Lemma 4.1 and Proposition 3.5, this completes the proof of Theorem 3.4. Nowadays there is a standardized approach for proving stability of multiclass queueing networks, which utilizes the powerful framework of fluid limits and stability of the associated fluid model. For a complete exposition of this technique we refer to [7], which is based on work that goes back to [26,13,6]. We begin our proof in Section 5.1 by writing down the set of queueing equations that describe the dynamics of the multiclass network in terms of simple stochastic processes, for which we derive the associated fluid model in Section 5.2. We then complete the proof of stability in Section 5.3 by making use of the fluid model.

The queueing equations
For i in {1, . . . , L}, let A i0 (·) := {A i0 (t) : t ∈ Z + } be the stochastic process that counts the cumulative number of exogenous class (i, 0) arrivals up to time t. For i, j in {1, . . . , L} we set A ij (·) ≡ 0, reflecting that the customer class (i, j) has no exogenous arrivals. Let Q x ij (t) denote the length of the queue containing class (i, j) customers at time t ∈ Z + , where x denotes the initial state of the system. This defines the queue length processes Q x ij (·) := {Q x ij (t) : t ∈ Z + }. Note that they can be combined into a vector-valued process that has the same law as the Markov chain X 2 (·) starting from the state x. We set T is the cumulative amount of service time that has been spent by station i on class (i, j) customers up to time t. As the service times of each customer class are precisely one unit of time, T x ij (t) is also equal to the total number of class (i, j) service completions up to time t.
Recall that K is the set of customer classes. For each class (i, j) we introduce the random vector φ ij (n) with components φ ij kl (n), (k, l) ∈ K, where φ ij kl (n) = 1 if the nth class (i, j) customer served by station i is routed to the class (k, l) queue, and φ ij kl (n) = 0 otherwise. Observe that φ ij (n) is an (L 2 + L)-dimensional random vector with expectation (P ij ) ⊤ , where P ij is the row of the routing matrix P corresponding to the index (i, j). For each class (i, j) we thus have an i.i.d. sequence of routing vectors {φ ij (n) : n ∈ N}. Define the cumulative routing process as Φ ij (·) := {Φ ij (n) : n ∈ Z + }, where Φ ij (0) := 0 and Φ ij (n) := n k=1 φ ij (k) for n ≥ 1. With this definition, Φ ij kl (n) counts the number of class (i, j) customers that have been routed to the queue of class (k, l) customers after the first n service completions of class (i, j) customers.
For each class (i, j), the stochastic processes A ij (·) and Φ kl ij (·), T x kl (·) for all (k, l) ∈ K completely determine the paths of the queue length process Q x ij (·). To be precise, for each time t we can calculate Q x ij (t) by adding the exogenous arrivals and the arrivals via routing to the initial queue length, and subtracting the departures. Thus we have for all The multiclass network has the work-conserving property, which means that a station will not idle whenever at least one of its queues is non-empty. To express this property in a formula, for i ∈ {1, . . . , L} and t ∈ Z + , let I x i (t) denote the cumulative amount of time that station i has been idle up to time t, and write ∆I x i (t) := I x i (t + 1) − I x i (t). Then Similarly to the way in which (9) captures the work-conserving property, we can also account for the priorities in the system. Recall that we imposed that customers of classes (i, j) with j = 0 have priority over class (i, 0) customers. If we let I x i0 (t) := t − L j=1 T x ij (t) be the total service time available for class (i, 0) customers up to time t at station i, and write ∆I x i0 (t) := I x i0 (t + 1) − I x i0 (t), then the priority structure is captured by the priority equation For each sample path of the multiclass network, the associated processes Q x ij (·) and T x ij (·) have to satisfy equations (5)- (10). We refer to this set of equations as the queueing equations or the queue length process representation. The evolution of the sample paths clearly depends on the parameter setting, i.e., these equations are parameterized by the collection of p i and q ij .

The fluid model
We are now ready to introduce the fluid model equations, or fluid model for short. They are continuous-time analogs of the queueing equations that arise naturally by taking a scaling limit of solutions to the queueing equations. To introduce these fluid model equations, it is convenient to first extend all the discrete-time processes introduced in Section 5.1 to continuous time by linear interpolation. That is, we define Q x ij (t) for non-integer values of t ∈ R + by interpolating linearly between Q x ij ⌊t⌋ and Q x ij ⌊t⌋ + 1 , and we extend the other processes introduced in Section 5.1 to continuous time in the same way. 1 Furthermore, we switch to a vector notation by defining Q x (·) := {Q x (t) : t ∈ R + } as the vector-valued process with components Q x ij (·), (i, j) ∈ K. Likewise, we define T x (·) as the process with components T x ij (·). All vectors are column vectors, and (in)equalities between vectors have to be read component-wise.
To obtain the fluid model, we scale the queue length and service time processes by the norm of the initial state x. That is, we introduce the scaled processes SQ x (·) and ST x (·) by setting for each t ∈ R + . We claim that for any sequence of initial states with norm tending to ∞, these scaled processes converge at almost every sample point along a subsequence to limit processes Q(·) andT (·) which we refer to as a fluid limit. Moreover, this fluid limit has to satisfy fluid model analogs of the queueing equations (5)-(10). The first three of those equations arē Q(t) ≥ 0 and each component ofQ(·) is Lipschitz-1, T (t) is non-decreasing in t, each component ofT (·) is Lipschitz-1, andT (0) = 0, where I denotes the identity matrix of dimension L 2 +L and p is the vector with components p ij for (i, j) ∈ K defined by p i0 := p i and p ij := 0 if j = 0. Next we have the two equations where e is the L-dimensional vector of ones and C is the incidence matrix linking each station to the customer classes served by that station; it has dimension L × (L 2 + L) and entries defined for i ∈ {1, . . . , L} and (j, k) ∈ K by C i,jk = 1 if j = i and C i,jk = 0 otherwise. Finally, We refer to equations (11)-(16) as the fluid model equations or fluid model. Every pair of functionsQ(·),T (·) that is a solution to these equations is called a fluid model solution. Our claims about convergence to a fluid limit which is a fluid model solution are made precise in the following theorem. A general version of this result has been proven in [13] for continuous-time multiclass networks. Because of the discrete-time nature of our processes, the proof of the theorem requires some small modifications, and is therefore given in Appendix B.

Stability of the fluid model
In this section we complete the proof of Theorem 3.4, with an appeal to Lemma 4.1, by proving that the multiclass network defined in Section 4 is stable under the stability condition of the theorem. The key is to prove stability of the fluid model, which is defined as follows: Definition 5.2. We say that the fluid model (11)-(16) is stable if there exists a constant δ > 0 that depends only on p and P such that any fluid model solutionQ(·),T (·) with |Q(0)| = 1 satisfiesQ(t) = 0 for all t ≥ δ.
It is well-known that in general, stability of the fluid model is a sufficient condition for stability of X 2 (·). This result has been proven for Markov chains with general state spaces under an additional condition, e.g., see [13,Theorem 4.2] or [7,Theorem 4.16]. In the setting of a countable state space, a considerably shorter argument is given in [8]. For completeness, we formulate this key result in the following proposition, a proof of which is included in Appendix B. So all that remains is to prove that, under the stability condition in Theorem 3.4, the fluid model is stable. A more general version of this result is proven in [14,Section 6], but here we present a more transparent proof based on [30]. In our proof, we consider the residual amount of time stations are going to be busy in the fluid model, given the amount of fluid mass that is present in the network. In essence, the goal of the proof is to show that this quantity must become zero, and hence every fluid model solution becomes zero, before a fixed deterministic time that is determined by the parameter setting.
It is customary in the theory of multiclass queueing networks to consider the quantities λ := I − P ⊤ −1 p and ρ := Cλ.
The quantity λ is known as the solution of the traffic equations [7, Section 1.2], and λ ij can be interpreted as the effective arrival rate to the queue of class (i, j) customers. By summing over j we obtain the effective arrival rate at station i, given by ρ i = (Cλ) i = L j=0 λ ij . In any multiclass network, ρ i represents the amount of service time that arrives to the network per unit time and is required from station i in the future. One can prove that the multiclass network cannot be stable when ρ < 1 fails to hold, using an argument similar to the one we used to prove Proposition 3.5. In many cases ρ < 1 is sufficient for stability, although this is not always true (see [7,Chapter 3] for counter-examples). The next lemma establishes that the condition for stability in Theorem 3.4 is equivalent to the condition that ρ < 1. Proof. Let i ∈ {1, . . . , L}. We will show that ρ i = p i + L j=1 π ij , where π ij is given by (1). The result then follows immediately from (2). We start by observing that Hence, using the definition (4) of the routing matrix P and the fact that p i0 = p i and p ij = 0 for j = 0, we obtain Here, (P m ) j0,ij is the probability that a class (j, 0) customer is routed to become a class (i, j) customer in m steps, which is the same as the probability in the roundabout model that a vehicle that enters the roundabout at cell j visits cell i after having stayed on the roundabout for m − 1 time steps. Hence, we see that ρ i is equal to p i + L j=1 p j b ij , using that b ij = EN ij with N ij the random variable introduced in Section 3.1, and the result follows by Lemma 3.2.
The proof that the fluid model is stable hinges on another crucial lemma concerning the residual amount of work for each station i at a given time. We claim that this quantity is described by the processR(·) := R (t) : t ∈ R + defined through To see that this process describes the residual work in the system, note first of all the analogy with the definition of ρ as C I − P ⊤ −1 p. Since we have both p ij = 0 and, by (16),Q ij (t) = 0 for j = 0, we can repeat the steps in the proof of Lemma 5.4 withQ(t) in place of p to see that As we have seen in Lemma 3.2, b ij is the mean number of times a customer who arrives at station j in the multiclass network (i.e., a class (j, 0) customer) intends to visit station i (as a class (i, j) customer) in the future. It follows that we can indeed interpretR i (t) as the residual amount of work for station i that is present in the system at time t. This quantity has an important property, which is the fluid equivalent of the statement that in the multiclass network, when the queue of class (i, 0) customers is empty, all customers who are present in the system and will visit station i, must also visit the preceding station i − 1. This crucially reflects the ring topology in our system, and enables us to control the decay of the amount of fluid in the fluid model. We formulate this key result in the following lemma: where i − 1 has to be read as L when i = 1.
Proof. Let t ≥ 0 be arbitrary and assumeQ i0 (t) = 0. Let k := L − (L + 1 − i mod L) denote the station preceding station i in the network. Then, using (19), we havē Since P k0,ik = 1 and P j0,ij = 0 for j = k, the first two terms on the right cancel each other. Furthermore, since for m ≥ 1, a class (j, 0) customer who is routed to become a class (i, j) customer in m + 1 steps must first be routed to become a class (k, j) customer after m steps, each term under the sum over m is non-negative. Hence,R k (t) ≥R i (t).
We are now ready to state and prove the main result of this section, the stability of the fluid model under the condition ρ < 1 from Lemma 5.4, which also completes the proof of Theorem 3.4. Proof. Set B := max ij b ij ,ρ := max i ρ i and δ := (1 + B)(1 −ρ) −1 . Assume thatρ < 1, and let the pairQ(·),T (·) be a solution of the fluid model equations with |Q(0)| = 1. Now suppose thatQ i0 (t) = ε for some i ∈ {1, . . . , L}, t > 0 and ε > 0. We will show that this implies t < δ, which proves that the fluid model satisfies Definition 5.2 of stability with δ as defined above.
To get us started, we set i 0 := i, so that i 0 is the index of the station where we have assumed the external queue length is ε at time t. We then recursively define i n for n ≥ 1 as the index of the station that directly precedes station i n−1 along the circle. This means that station i n is located n steps before station i 0 = i on the circle, which formally corresponds to setting i n := L − (L + n − i mod L).
The motivation for this definition is as follows. Starting from time t 0 = t, we go backwards in time. Whenever t n is non-zero and the class (i n , 0) queue is not empty at time t n (as is the case for n = 0), we take (t n+1 , t n ] to be the interval of maximal length with right endpoint t n during which this queue is continuously non-empty. We will show below that this implies that, going backwards in time, the residual work process of station i n must be increasing on the interval (t n+1 , t n ] at a minimum rate 1 −ρ. In all other cases, we take t n+1 equal to t n . At every non-zero time t n+1 we can then use Lemma 5.5 to switch from station i n to station i n+1 : the class (i n , 0) queue is by definition empty at time t n+1 , so we know that station i n+1 has at least as much residual work as station i n at this point. Thus, by arguing backwards in time along the sequence {t n }, we can show that as long as t n is non-zero, the residual work for station i n at time t n+1 must be larger than (1 −ρ)(t − t n+1 ). Showing this is the first step of the proof. We now fill in the details. Combining (18) and (11) givesR(t) =R(0) + ρ t − CT (t), hence by (14) we can expressR in (t) in terms of the cumulative idle time process of station i n as As we observed above, for all n ≥ 0 we either have t n+1 = t n or elseQ in0 (s) > 0 for every s in the interval (t n+1 , t n ]. By the work-conserving property (15), we obtainĪ in (t n+1 ) =Ī in (t n ) in either case, and sinceρ ≥ ρ in , it follows that for all n ≥ 0, Furthermore, if n ≥ 1 and t n > 0, thenQ i n−1 0 (t n ) = 0 and Lemma 5.5 yields (19). Using the derived inequalities and the fact that t n ≥ t m−1 if n ≤ m − 1, we conclude that for all m ≥ 1 for which t m−1 > 0, Our next step is to show that this result in fact holds all the way down to time 0, by showing that the t n cannot be all strictly positive. To see this, set τ := ε/(1 + LB), and suppose that we have t n > 0 for some n ≥ 1. Then t n−1 > 0, soR i n−1 (t n ) ≥ ε by the inequality we have just derived, and it follows from (19) that we must have thatQ j0 (t n ) ≥ τ for some station j. But by (11), using the fact that no customers are ever routed to the class (j, 0) queue, for all s ∈ (0, t n ]. So sinceT j (·) is non-decreasing by (13), p j ≤ρ by Lemma 5.4, andρ < 1, which shows thatQ j0 (s) > 0 for all s ∈ (t n − τ, t n ]. This implies that if we take k to be the smallest integer strictly larger than n for which i k−1 = j, then k is at most n + L, and by the definition of t k , it must be the case that t k ≤ t n − τ unless t k = 0. Hence we can conclude that there has to exist an m ∈ N such that t m−1 > 0 but t m = 0, and we know that for this m, But |Q(0)| = 1 impliesR i m−1 (0) ≤ 1 + B, so it follows that t < (1 + B)(1 −ρ) −1 = δ.

Related models
At the start of Section 2, we briefly mentioned why we consider the roundabout model instead of the slotted-ring model from [31]. In the first part of this section, we explain this decision by arguing that Theorem 3.4 also applies to the slotted-ring model. In the second part, we discuss in detail how the model we introduced in Section 2 differs from the roundabout model in [29], and provide a rigorous argument to show that this difference does not alter the global stability region. In the last part, we present ways in which the routing can be modified without affecting the validity of the proof of stability. In particular, we explain how our setting can also be used to cover formulations of multiclass networks with fixed routes.

Slotted-ring model
Regarding the issue of stability, the slotted-ring model in [31] can be seen as a slightly less general variant of the roundabout model. To clarify this, we highlight the small differences between the two models and explain why they are not relevant for the issue of stability.
The main differences are that time in the slotted-ring model is continuous and the (independent) inter-arrival times have exponential instead of geometric distributions, so that arrivals occur according to Poisson processes. Additionally, the processing times in the cells are deterministic with length τ /L, for some real number τ > 0, instead of deterministic with length one. Finally, the routing in [31] imposes that packets never make more than one loop around the ring, whereas the roundabout model allows parameter settings in which they can make arbitrarily many loops. However, it is not difficult to see that the parameters q ij in the roundabout model can be chosen in such a way that the routing in the slotted-ring model is reproduced exactly (note that this requires setting certain q ij equal to 1). So the roundabout model is more general in terms of the routing, which is why we focus on that model in the paper.
The other differences between the two models can be dealt with as follows. If we observe the slotted-ring model in [31] at times t τ /L, with t ∈ Z + , we obtain a discrete-time model that is a special case of the roundabout model, except in one aspect: whereas in the roundabout model one can have at most one arrival to an on-ramp queue in each time unit, it is possible in the slotted-ring model to have two or more arrivals to a queue between times t τ /L and (t + 1) τ /L. However, because of the strong law of large numbers, this discrepancy does not change the condition for stability. To be precise, suppose that the arrival process to a queue in the slottedring model is a Poisson process with rate λ. Then, after translating the model to discrete time as explained above, in the fluid limit this arrival process will converge uniformly on each set [0, m] to the function t → p t, where p = λ τ /L. Therefore, as a consequence of Theorem 5.1, both models have the same fluid model, and hence are stable under the same conditions.

Roundabout model
We now explain our reasons for considering a slightly different model than the one in [29]. In the roundabout model in [29], a vehicle that arrives to an empty queue and cell at time t ∈ Z + enters the roundabout immediately, and is therefore not in the queue at time t or time t + 1. This poses a problem for the multiclass network formulation of the model: in the multiclass network, a vehicle must have been in a queue for one unit of time in order to receive service.
Since our proof relies on coupling the roundabout model to a multiclass network, we have slightly modified the model here so that an arriving vehicle enters a queue first, and is allowed to enter the roundabout no sooner than one time unit later. We emphasize that this model alteration does not affect the stability condition: Theorem 3.4 also applies to the model in [29]. This follows from the fact that the two models can be coupled in such a way that their sample paths stay close together at all times. The precise result is stated in the proposition below, the proof of which is presented in Appendix A. We conclude that the Markov chain X 1 (·) is positive recurrent if and only if the Markov chain for the model in [29] is positive recurrent. Proposition 6.1. LetX(·) := {X(t) : t ∈ Z + } denote the discrete-time Markov chain associated with the model in [29]. There exists a coupling between X 1 (·) andX(·) in which at all times the states of all cells are the same for the two processes, while the difference in queue length is at most 1 for each queue.

Models with other types of routing
We designed the multiclass network in Section 4 to mimic the routing of vehicles in the roundabout model. We will refer to this type of routing as 'probabilistic routing'. This type of routing enables us to identify customers classes by customers' location and type.
A frequently used alternative in queueing networks is to fix a set of (deterministic) routes that customers can take through a network, and identify each customer class with one of these routes (see, e.g., [8]). These fixed routes are generally not included in the possible set of routes obtained from probabilistic routing, but our setting can be modified to handle this alternative type of routing. This is possible because in our setting, a route is determined by the location of the queue where the customer arrives together with the time the customer intends to spend in the network. That is, to replace probabilistic routing by fixed routes, we only need to change the distributions of the variables T j we introduced in Section 3. This of course leads to different formulas for the marginal stationary probabilities π ij , but Lemma 3.2 remains valid. To establish the pathwise coupling with a multiclass network, we need that only finitely many customer classes are required to describe all routes. In that case, the statement of Theorem 3.4 and its proof go through, with a different expression for the π ij , leading to a rigorous derivation of the global stability region.
Both formulations of the routing (and associated customer classes) have their advantages. On the one hand, fixed routes allow for customer behavior that is impossible with probabilistic routing. For instance, using fixed routes, we can have a class of customers who arrive to queue 1 and always complete exactly two full circles before leaving the roundabout at cell/station 2. This is not possible with probabilistic routing. On the other hand, with probabilistic routing, the time that customers can spend in the system is unbounded. With fixed (deterministic) routes, this time is necessarily bounded, since we can only handle a finite number of customer classes (i.e., routes of finite length) in the multiclass network.
A further model generalization is to combine probabilistic routing with fixed routes. That is, we could allow customers arriving at a station j to first follow a fixed route through the network (chosen from a finite set), and then either leave the network (with a certain probability), or continue according to probabilistic routing with parameters q ij . Again, this generalization only changes the distribution of the variable T j , so our methods still apply. But because the model formulation in Section 2 already covers a broad range of applications in communication systems and transportation networks, we have chosen not to work with this more general setting.

Concluding remarks
In this paper, we have considered a ring-topology stochastic network with queues, involving different customer classes and priority structures. A careful consideration of its marginal stationary distribution gave us a condition for stability, which we have proven to be necessary and sufficient. In our proof, we coupled the sample paths of our model to those of a multiclass queueing network, enabling us to appeal to fluid model techniques to prove sufficiency of the stability condition. The approach we used explicitly exploited the relationship between the condition for stability and the rate at which different segments of the ring are occupied in case the model is stable.
With our main theorem on the model's stability region, we can precisely quantify the capacity of associated communication or transportation systems. In practice, however, scenarios can occur in which the system is not stable, often during certain periods, e.g., rush hours for road traffic applications, or specific busy hours for communication networks. In that case, one is interested in determining for instance which queues will be stable, and at which rates the lengths of unstable queues grow.
To address this problem, letπ i0 be the stationary probability at which cell i is empty when the system is unstable, assuming ergodic behavior in the occupation of the cells. Then the ergodic rate at which type-j vehicles enter the ring will be the minimum of p j andπ j0 depending on whether queue j is stable or unstable, respectively. Following the arguments of Proposition 3.1 and Lemma 3.2, the marginal stationary probability that a cell i will be occupied by a type-j vehicle will then be given byπ ij = b ij (π j0 ∧ p j ). Sinceπ i0 + · · · +π iL = 1 for every i, it follows that theπ i0 must satisfy the system of equations π i0 = 1 − L j=1 b ij (π j0 ∧ p j ), i = 1, . . . , L.
Therefore, solving this system of equations identifies the possible candidates for the ergodic behavior of the system in the unstable regime. We study the solution set of the system (20) and its implications for the ergodic behavior of the queues in a forthcoming paper [19].