Skip to main content
Log in

Approximation of the exit probability of a stable Markov modulated constrained random walk

  • S.I.: QueueStochMod2019
  • Published:
Annals of Operations Research Aims and scope Submit manuscript

Abstract

Let X be the constrained random walk on \({\mathbb {Z}}_+^2\) having increments (1, 0), \((-\,1,1)\), \((0,-\,1)\) with jump probabilities \(\lambda (M_k)\), \(\mu _1(M_k)\), and \(\mu _2(M_k)\) where M is an irreducible aperiodic finite state Markov chain. The process X represents the lengths of two tandem queues with arrival rate \(\lambda (M_k)\), and service rates \(\mu _1(M_k)\), and \(\mu _2(M_k)\); the process M represents the random environment within which the system operates. We assume that the average arrival rate with respect to the stationary measure of M is less than the average service rates, i.e., X is assumed stable. Let \(\tau _n\) be the first time when the sum of the components of X equals n for the first time. Let Y be the random walk on \({{\mathbb {Z}}} \times {{\mathbb {Z}}}_+\) having increments \((-\,1,0)\), (1, 1), \((0,-\,1)\) with probabilities \(\lambda (M_k)\), \(\mu _1(M_k)\), and \(\mu _2(M_k)\). Supposing that the queues share a joint buffer of size n, \(p_n =P_{(x_n,m)}(\tau _n < \tau _0)\) is the probability that this buffer overflows during a busy cycle of the system. To the best of our knowledge, the only methods currently available for the approximation of \(p_n\) are classical large deviations analysis giving the exponential decay rate of \(p_n\) and rare event simulation. Let \(\tau \) be the first time the components of Y are equal. For \(x \in {{\mathbb {R}}}_+^2\), \(x(1) + x(2) < 1\), \(x(1) > 0\), and \(x_n = \lfloor nx \rfloor \), we show that \(P_{(n-x_n(1),x_n(2),m)}( \tau < \infty )\) approximates \(P_{(x_n,m)}(\tau _n < \tau _0)\) with exponentially vanishing relative error as \(n\rightarrow \infty \). For the analysis we define a characteristic matrix in terms of the jump probabilities of (XM). The 0-level set of the characteristic polynomial of this matrix defines the characteristic surface; conjugate points on this surface and the associated eigenvectors of the characteristic matrix are used to define (sub/super) harmonic functions which play a fundamental role both in our analysis and the computation/approximation of \(P_{(y,m)}(\tau < \infty )\).

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Notes

  1. Note that \(\log ([(\beta ,\alpha ), y ]) = \langle (\log (\beta ),\log (\alpha )), ((y(1)-y(2)),y(2)) \rangle \) where \(\langle \cdot ,\cdot \rangle \) denotes the standard inner product in \({{\mathbb {R}}}^2\); the notation \([(\beta ,\alpha ), y ]\) was chosen with this connection to the inner product in mind. It also leads to much easier to read formulas because multiindex variables will be substituted for \(\beta \) and \(\alpha \), see, e.g., display (92).

References

  • Berman, A., & Plemmons, R. J. (1994). Nonnegative matrices in the mathematical sciences (Vol. 9). Philadelphia: SIAM.

    Book  Google Scholar 

  • Blanchet, J. (2013). Optimal sampling of overflow paths in Jackson networks. Mathematics of Operations Research, 38(4), 698–719.

    Article  Google Scholar 

  • Blanchet, J. H., Leder, K., & Glynn, P. W. (2008). Efficient simulation of light-tailed sums: An old folk song sung to a faster new tune. In Monte Carlo and Quasi-Monte Carlo Methods (Vol. 2009, pp. 227–248).

  • Borovkov, A. A., & Mogul’skii, A. A. (2001). Large deviations for Markov chains in the positive quadrant. Russian Mathematical Surveys, 56(5), 803–916.

    Article  Google Scholar 

  • Comets, F., Delarue, F., & Schott, R. (2009). Large deviations analysis for distributed algorithms in an ergodic Markovian environment. Applied Mathematics and Optimization, 60(3), 341–396.

    Article  Google Scholar 

  • De Boer, P.-T. (2006). Analysis of state-independent importance-sampling measures for the two-node tandem queue. ACM Transactions on Modeling and Computer Simulation (TOMACS), 16(3), 225–250.

    Article  Google Scholar 

  • De Boer, P.-T., Kroese, D. P., & Rubenstein, R. Y. (2004). A fast cross-entropy method for estimating buffer overflows in queueing networks. Management Science, 50(7), 883–895.

    Article  Google Scholar 

  • Dean, T., & Dupuis, P. (2009). Splitting for rare event simulation: A large deviation approach to design and analysis. Stochastic Processes and Their applications, 119(2), 562–587.

    Article  Google Scholar 

  • Dean, T., & Dupuis, P. (2011). The design and analysis of a generalized RESTART/DPR algorithm for rare event simulation. Annals of Operations Research, 189(1), 63–102.

    Article  Google Scholar 

  • Dupuis, P., Leder, K., & Wang, H. (2007). Importance sampling for sums of random variables with regularly varying tails. ACM Transactions on Modeling and Computer Simulation (TOMACS), 17(3), 14.

    Article  Google Scholar 

  • Dupuis, P., Sezer, A. D., & Wang, H. (2007). Dynamic importance sampling for queueing networks. Annals of Applied Probability, 17(4), 1306–1346.

    Article  Google Scholar 

  • Dupuis, P., & Wang, H. (2004). Importance sampling, large deviations and differential games. Stochastics : An International Journal of Probability and Stochastic Processes, 76(6), 481–508.

    Google Scholar 

  • Dupuis, P., & Wang, H. (2009). Importance sampling for Jackson networks. Queueing Systems, 62(1–2), 113–157.

    Article  Google Scholar 

  • Durrett, R. (2010). Probability: Theory and examples (4th ed.). Cambridge: Cambridge University Press.

    Book  Google Scholar 

  • Flajolet, P. (1986). The evolution of two stacks in bounded space and random walks in a triangle. In International symposium on mathematical foundations of computer science (pp. 325–340). Springer.

  • Foley, R. D., & McDonald, D. R. (2005). Large deviations of a modified Jackson network: Stability and rough asymptotics, The Annals of Applied Probability, 15(1B), 519–541.

    Google Scholar 

  • Frater, M. R., Lennon, T. M., & Anderson, B. D. O. (1991). Optimally efficient estimation of the statistics of rare events in queueing networks. IEEE Transactions on Automatic Control, 36(12), 1395–1405.

    Article  Google Scholar 

  • Freidlin, M. I., & Wentzell, A. D. (1998). Random perturbation of dynamical systems (2nd ed.). New York: Springer-Verlag Telos.

    Book  Google Scholar 

  • Glasserman, P., & Kou, S.-G. (1995). Analysis of an importance sampling estimator for tandem queues. ACM Transactions on Modeling and Computer Simulation (TOMACS), 5(1), 22–42.

    Article  Google Scholar 

  • Ignatiouk-Robert, I. (2000). Large deviations of Jackson networks. Annals of Applied Probability, 10, 962–1001.

    Article  Google Scholar 

  • Ignatiouk-Robert, I., & Loree, C. (2010). Martin boundary of a killed random walk on a quadrant. The Annals of Probability, 38(3), 1106–1142.

    Article  Google Scholar 

  • Ignatyuk, I. A., Malyshev, V. A., & Scherbakov, V. V. (1994). Boundary effects in large deviation problems. Russian Mathematical Surveys, 49(2), 41–99.

    Article  Google Scholar 

  • Juneja, S., & Nicola, V. (2005). Efficient simulation of buffer overflow probabilities in Jackson networks with feedback. ACM Transcations on Modeling and Computer Simulation (TOMACS), 15(4), 281–315.

    Article  Google Scholar 

  • Kroese, D. P., & Nicola, V. (2002). Efficient simulation of a tandem Jackson network. ACM Transactions on Modeling and Computer Simulation (TOMACS), 12(2), 119–141.

    Article  Google Scholar 

  • Kurkova, I. A., & Malyshev, V. A. (1998). Martin boundary and elliptic curves. Markov Process and Related Fields, 4(2), 203–272.

    Google Scholar 

  • Lax, P. D. (1996). Linear algebra. Pure and applied mathematics. Hoboken: Wiley-Interscience.

    Google Scholar 

  • Louchard, G., & Schott, R. (1991). Probabilistic analysis of some distributed algorithms. Random Structures & Algorithms, 2(2), 151–186.

    Article  Google Scholar 

  • Maier, R. S. (1991). Colliding stacks: A large deviations analysis. Random Structures & Algorithms, 2(4), 379–420.

    Article  Google Scholar 

  • Maier, R. S. (1993). Large fluctuations in stochastically perturbed nonlinear systems: Applications in computing. In Proceedings 1992 Complex Systems Summer School (pp. 501–517). Addison-Wesley.

  • McDonald, D. R. (1999). Asymptotics of first passage times for random walk in an orthant. Annals of Applied Probability, 9, 110–145.

    Article  Google Scholar 

  • Nicola, V., & Zaburnenko, T. (2007). Efficient importance sampling heuristics for the simulation of population overflow in Jackson networks. ACM Transactions on Modeling and Computer Simulation (TOMACS), 17(2), 10.

    Article  Google Scholar 

  • Parekh, S., & Walrand, J. (1989). A quick simulation method for excessive backlogs in networks of queues. IEEE Transactions on Automatic Control, 34(1), 54–66.

    Article  Google Scholar 

  • Prabhu, N. U., & Zhu, Y. (1989). Markov-modulated queueing systems. Queueing Systems, 5(1–3), 215–245.

    Article  Google Scholar 

  • Ridder, A. (2009). Importance sampling algorithms for first passage time probabilities in the infinite server queue. European Journal of Operational Research, 199(1), 176–186.

    Article  Google Scholar 

  • Serre, D. (2010). Matrices: Theory and applications. Graduate texts in mathematics (2nd ed., Vol. 216). New York: Springer.

    Book  Google Scholar 

  • Sezer, A. D. (2005). Dynamic importance sampling for queueing networks. Ph.D. thesis, Brown University Division of Applied Mathematics.

  • Sezer, A. D. (2007). Asymptotically optimal importance sampling for Jackson networks with a tree topology. Preprint http://arxiv.org/abs/0708.3260.

  • Sezer, A. D. (2009). Importance sampling for a Markov modulated queuing network. Stochastic Processes and their Applications, 119(2), 491–517.

    Article  Google Scholar 

  • Sezer, A. D. (2010). Asymptotically optimal importance sampling for Jackson networks with a tree topology. Queueing Systems, 6(2), 103–117.

    Article  Google Scholar 

  • Sezer, A. D. (2015). Exit probabilities and balayage of constrained random walks. https://arxiv.org/abs/1506.08674.

  • Sezer, A. D. (2018). Approximation of excessive backlog probabilities of two tandem queues. Journal of Applied Probability, 55(3), 968–997.

    Article  Google Scholar 

  • Ünlü, K. D., & Sezer, A. D. (2019). Excessive backlog probabilities of two parallel queues. Annals of Operations Research. https://doi.org/10.1007/s10479-019-03324-w.

  • Yao, A. C. (1981). An analysis of a memory allocation scheme for implementing stacks. SIAM Journal on Computing, 10(2), 398–403.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Ali Devin Sezer.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Two lemmas

Two lemmas

For a square matrix \({\varvec{G}}\), let \({\varvec{G}}^{i,j}\) denote the matrix obtained by removing the ith row and jth column of \({\varvec{G}}\).

Lemma A.1

For \(n_0 \in \{2,3,\ldots \}\), suppose \({\varvec{G}}\) is an \(n_0 \times n_0\) irreducible and aperiodic matrix with nonnegative entries. Then \(\det \left( (\Lambda _1({\varvec{G}}) {\varvec{I}} - {\varvec{G}})^{i,i}\right) > 0\) for all \(i \in \{1,2,\ldots ,n_0\}\), where \({\varvec{I}}\) is the \(n_0 \times n_0\) identity matrix.

Proof

The argument is the same for all \( i \in \{1,2,\ldots ,n_0\}\); so it suffices to argue for \(i=1\). Suppose the claim is not true and

$$\begin{aligned} \det \left( (\Lambda _1({\varvec{G}}){\varvec{I}} - {\varvec{G}})^{1,1}\right) \le 0. \end{aligned}$$
(101)

Consider the function \(u \mapsto g(u) = \det \left( ( u{\varvec{I}} -{\varvec{G}})^{1,1} \right) \), \(u \ge 0\). The multilinearity and continuity of \(\det \) implies \(\lim _{u\nearrow \infty } g(u) = \infty \). This implies that if (101) is true there must be \(u_0 \ge \Lambda _1({\varvec{G}})\) such that

$$\begin{aligned} \det \left( (u_0 {\varvec{I}} - {\varvec{G}})^{1,1} \right) = 0. \end{aligned}$$
(102)

The matrix \({\varvec{G}}^{1,1}\) is nonnegative, therefore, it has a largest eigenvalue \(\Lambda _1({\varvec{G}}^{1,1})\) with an eigenvector \({\varvec{v}}_1 \ge 0\). The equality (102) implies

$$\begin{aligned} \Lambda _1({\varvec{G}}^{1,1}) \ge u_0 \ge \Lambda _1({\varvec{G}}). \end{aligned}$$
(103)

That \({\varvec{G}}\) is irreducible and aperiodic implies that \({\varvec{G}}^{n_0}\) is strictly positive; its largest eigenvalue is

$$\begin{aligned} \Lambda _1({\varvec{G}}^{n_0}) = \Lambda _1({\varvec{G}})^{n_0}. \end{aligned}$$

The matrix \(({\varvec{G}}^{n_0})^{1,1}\) has strictly positive entries and therefore its largest eigenvalue \(\Lambda _1( ({\varvec{G}}^{n_0})^{1,1})\) has an eigenvalue \({\varvec{v}}_2\) with strictly positive entries. For two vectors \(x,y \in {{\mathbb {R}}}^d\), let \(x \ge y\) and \(x > y\) denote componentwise comparison. The inequality

$$\begin{aligned} ({\varvec{G}}^{n_0})^{1,1} \ge ({\varvec{G}}^{1,1})^{n_0} \end{aligned}$$

implies

$$\begin{aligned} ({\varvec{G}}^{n_0})^{1,1} {\varvec{v}}_1 \ge \Lambda _1({\varvec{G}}^{1,1})^{n_0} {\varvec{v}}_1. \end{aligned}$$
(104)

On the other hand

$$\begin{aligned} \Lambda _1(({\varvec{G}}^{n_0})^{1,1}) = \sup \{ c: \exists x \in {\mathbb R}^{n_0-1}_+, ({\varvec{G}}^{n_0})^{1,1}x \ge cx \}, \end{aligned}$$
(105)

(see Lax 1996, Proof of Theorem 1, Chapter 16). This and (104) imply

$$\begin{aligned} \Lambda _1(({\varvec{G}}^{n_0})^{1,1}) \ge \Lambda _1({\varvec{G}}^{1,1})^{n_0}. \end{aligned}$$
(106)

Define \({\varvec{v}}_3 = [ 1; {\varvec{v}}_2] \in {{\mathbb {R}}}^{n_0}\); it follows from \(({\varvec{G}}^{n_0})^{1,1} {\varvec{v}}_2 = \Lambda _1(({\varvec{G}}^{n_0}){1,1}) {\varvec{v}}_2\), the strict positivity of the components of \({\varvec{G}}^{n_0}\) and \({\varvec{v}}_2\) that one can choose \(\delta > 0\) small enough so that

$$\begin{aligned} {\varvec{G}}^{n_0} {\varvec{v}}_3 > \left( \Lambda _1(({\varvec{G}}^{n_0})^{1,1}+ \delta \right) {\varvec{v}}_3; \end{aligned}$$

This and

$$\begin{aligned} \Lambda _1({\varvec{G}}^{n_0})= \sup \{ c: \exists x \in {\mathbb R}^{n_0}_+, {\varvec{G}}^{n_0}x \ge cx \} \end{aligned}$$

imply

$$\begin{aligned} \Lambda _1({\varvec{G}}^{n_0}) > \Lambda _1\left( ({\varvec{G}}^{n_0})^{1,1}\right) . \end{aligned}$$

The last inequality, (106) and (103) imply

$$\begin{aligned} \Lambda _1({\varvec{G}})^{n_0}= \Lambda _1({\varvec{G}}^{n_0}) > \Lambda _1(({\varvec{G}}^{n_0})^{1,1}) \ge \Lambda _1({\varvec{G}}^{1,1})^{n_0} \ge \Lambda _1({\varvec{G}})^{n_0}, \end{aligned}$$

which is a contradiction. \(\square \)

In our analysis we need the following fact from Sezer (2009); its proof is elementary and follows from the multilinearity of the determinant function and the previous lemma.

Lemma A.2

Let \({\varvec{G}}\) be an aperiodic and irreducible transition matrix. Then the row vector whose ith component equals \(\det \left( ({\varvec{I}} - {\varvec{G}})^{i,i}\right) \) is the unique (upto scaling by a positive number) left eigenvector associated with the eigenvalue 1 of \({\varvec{G}}\).

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Başoğlu Kabran, F., Sezer, A.D. Approximation of the exit probability of a stable Markov modulated constrained random walk. Ann Oper Res 310, 431–475 (2022). https://doi.org/10.1007/s10479-020-03693-7

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10479-020-03693-7

Keywords

Mathematics Subject Classification

Navigation