Abstract
We apply the external uniformization (EU) method for the transient study of continuous-time Markov chain models in which the transition rates could be either constant or time-dependent. In the method, we run an exponential clock for watching the Markov chain. Each realization epoch of the random clock is considered as an absorption epoch. The approximate transient probability that the Markov chain is found at a particular state is then found as the probability of absorption from that state. The convergence of the method to the analytical value has been proved. In the case of infinite/huge dimensional Markov chains, implementation of the method needs truncation of the state-space into a finite window of states. A comparison of the results obtained using the EU method with some of the existing analytical results shows that the described method produces highly accurate results on numerical implementation. Several examples considered also reveal that the EU method required much less computational time when compared to the uniformization/randomization (U/R) method.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Aiewsakun P, Katzourakis A (2016) Time-dependent rate phenomenon in viruses. J Virol 90(16):7184–7195. https://doi.org/10.1128/jvi.00593-16
Angus JE (1992) Some bounds on the error in approximating transition probabilities in continuous-time Markov processes. SIAM Rev 34(1):110–113. https://doi.org/10.1137/1034005
Biswas S, Ebraheem NA (1988) On an application of the diffusion approximation of the generalized birth and death process for stochastic population projection. Commun Stat Theory Methods 17(3):779–789. https://doi.org/10.1080/03610928808829654
Brown FLH (2003) Single-molecule kinetics with time-dependent rates: a generating function approach. Phys Rev Lett 90(2):028302. https://doi.org/10.1103/PhysRevLett.90.028302
Browne S, Whitt W (1995) Piecewise-linear diffusion processes. CRC Press, Boca Raton, pp 463–480
Crane MA, Iglehart DL (1974) Simulating stable stochastic systems, I: general multiserver queues. J ACM 21(1):103–113. https://doi.org/10.1145/321796.321805
Fichthorn KA, Weinberg WH (1992) Influence of time-dependent rates of mass transfer on the kinetics of domain growth. Phys Rev Lett 68(5):604–607
Grassmann WK (1977) Transient solutions in markovian queueing systems. Comput Oper Res 4(1):47–53. https://doi.org/10.1016/0305-0548(77)90007-7
Green LV, Kolesar PJ, Soares J (2001) Improving the sipp approach for staffing service systems that have cyclic demands. Oper Res 49(4):549–564. https://doi.org/10.1287/opre.49.4.549.11228
Green LV, Kolesar PJ, Whitt W (2007) Coping with time-varying demand when setting staffing requirements for a service system. Prod Oper Manag 16:13–29. https://doi.org/10.1111/j.1937-5956.2007.tb00164.x
Henzinger TA, Mateescu M, Wolf V (2009) Sliding window abstraction for infinite markov chains. In: Lecture notes in computer science (including subseries lecture notes in artificial intelligence and lecture notes in bioinformatics), vol 5643 LNCS, pp 337–352. https://doi.org/10.1007/978-3-642-02658-4_27
Ho SY, Lanfear R, Bromham L et al (2011) Time-dependent rates of molecular evolution. Mol Ecol 20(15):3087–3101. https://doi.org/10.1111/j.1365-294X.2011.05178.x
Ingolfsson A, Akhmetshina E, Budge S, Li Y, Wu X (2007) A survey and experimental comparison of service-level-approximation methods for nonstationary M(t)/M/s(t) queueing systems with exhaustive discipline. INFORMS J Comput 19(2):201–214. https://doi.org/10.1287/ijoc.1050.0157
Jensen A (1953) Markov chains as an aid in the study of Markov processes. Skand Aktuarietidskrift 3:87–91. https://doi.org/10.1080/03461238.1953.10419459
Kaczynski WH, Leemis LM, Drew JH (2012) Transient queueing analysis. INFORMS J Comput 24(1):10–28. https://doi.org/10.1287/ijoc.1110.0452
Karlin S, Taylor HM (1981) A second course in stochastic processes. Academic Press, Cambridge
Kelton W, Law A (1985) The transient behavior of the M/M/s queue, with implications for steady-state simulation. Oper Res 33(2):378–396. https://doi.org/10.1287/opre.33.2.378
Kendall DG (1948) On the generalized birth-and-death process. Ann Math Stat 19(1):1–15. https://projecteuclid.org/euclid.aoms/1177730285
Koops DT, Boxma OJ, Mandjes MRH (2017) Networks of ·/G/∞ queues with shot-noise-driven arrival intensities. Queueing Syst 86(3–4):301–325. https://doi.org/10.1007/s11134-017-9520-7
Lyche M (2020) Numerical linear algebra and matrix factorizations. Springer, Berlin
Margolius BH (1999) A sample path analysis of the Mt/Mt/c queue. Queueing Syst 31(1–2):59–93
Margolius BH (2007) Transient and periodic solution to the time-inhomogeneous quasi-birth death process. Queueing Syst 56(3–4):183–194. https://doi.org/10.1007/s11134-007-9027-8
Margolius BH (2008) The matrices R and G of matrix analytic methods and the time-inhomogeneous periodic quasi-birth-and-death process. Queueing Syst 60(1–2):131–151
Maruyama G (1955) Continuous Markov processes and stochastic equations. Rendi Circ Mat Palermo 4(1):48–90. https://doi.org/10.1007/BF02846028
MATLAB (2019) Version (R2019b Update 3). The MathWorks Inc, Natick
McClure JP, Wong R (1976) Infinite systems of differential equations. Can J Math 28(6):1132–1145. https://doi.org/10.4153/CJM-1976-111-4
Newell GF (1968) Queues with time-dependent arrival rates I: the transition through saturation. J Appl Probab 5(2):436–451. https://doi.org/10.2307/3212264
Novozhilov AS, Karev GP, Koonin EV (2006) Biological applications of the theory of birth-and-death processes. Brief Bioinform. https://doi.org/10.1093/bib/bbk006
Pawlikowski K (1990) Steady-state simulation of queueing processes: survey of problems and solutions. ACM Comput Surv 22(2):123–170. https://doi.org/10.1145/78919.78921
Reibman A, Trivedi K (1988) Numerical transient analysis of markov models. Comput Oper Res 15(1):19–36. https://doi.org/10.1016/0305-0548(88)90026-3
Ross SM (1987) Approximating transition probabilities and mean occupation times in continuous-time Markov chains. Probab Eng Inform Sci 1(3):251–264. https://doi.org/10.1017/S0269964800000036
Sauer T (2012) Numerical solution of stochastic differential equations in finance. Handbook of computational finance. Springer, Berlin, pp 529–550. https://doi.org/10.1007/978-3-642-17254-0_19
Schwarz M, Sauer C, Daduna H, Kulik R, Szekli R (2006) M/M/1 queueing systems with inventory. Queueing Syst 54(1):55–78. https://doi.org/10.1007/s11134-006-8710-5
Schwarz JA, Selinka G, Stolletz R (2016) Performance analysis of time-dependent queueing systems: survey and classification. Omgea 63:170–189. https://doi.org/10.1016/j.omega.2015.10.013
Shortle JF, Thompson JM, Gross D, Harris CM (2017) Fundamentals of queueing theory: fifth edition. Wiley, Hoboken, pp 1–548. https://doi.org/10.1002/9781119453765
Stolletz R (2008) Approximation of the non-stationary M(t)/M(t)/c(t)-queue using stationary queueing models: the stationary backlog-carryover approach. Eur J Oper Res 190(2):478–493. https://doi.org/10.1016/j.ejor.2007.06.036
Thanh VH, Priami C (2015) Simulation of biochemical reactions with time-dependent rates by the rejection-based algorithm J. Chem Phys 143(5):054104. https://doi.org/10.1063/1.4927916
Tietz C, Schuler S, Speck T et al (2006) Measurement of stochastic entropy production. Phys Rev Lett 97(5):050602. https://doi.org/10.1103/PhysRevLett.97.050602
Tirdad A, Grassmann WK, Tavakoli J (2016) Optimal policies of M(t)/M/c/c queues with two different levels of servers. Eur J Oper Res 249(3):1124–1130. https://doi.org/10.1016/j.ejor.2015.10.040
van Der Weij W, van Dijk NM, van Der Mei RD (2012) Product-form results for two-station networks with shared resources. Perform Eval 69(12):662–683. https://doi.org/10.1016/j.peva.2012.08.002
van Dijk NM (1988) A note on error bounds for approximating transition probabilities in continuous-time Markov chains. Probab Eng Inform Sci 2(4):471–474. https://doi.org/10.1017/S0269964800000176
van Dijk NM, van Brummelen SPJ, Boucherie RJ (2018) Uniformization: basics, extensions and applications. Perform Eval 118:8–32. https://doi.org/10.1016/j.peva.2017.09.008
Vilk O, Assaf M (2018) Population extinction under bursty reproduction in a time modulated environment. Phys Rev E 97:062114. https://doi.org/10.1103/PhysRevE.97.062114
Walker DM (1998) The expected time until absorption when absorption is not certain. J Appl Probab 35(4):812–823. https://doi.org/10.1239/jap/1032438377
Whitt W (2004) A diffusion approximation for the G/GI/n/m queue. Oper Res 52(6):922–941. https://doi.org/10.1287/opre.1040.0136
Whitt W (2017) Queues with time-varying arrival rates: a bibliography. https://www.columbia.edu/~ww2040/TV_bibliography_091016.pdf
Whitt W (2018) Time-varying queues. Queueing Models Serv Manag 1(2):79–164. https://www.columbia.edu/~ww2040/TVQ_QMSM_062618.pdf
Zhang J, Watson LT, Cao Y (2010) A modified uniformization method for the solution of the chemical master equation. Comput Math Appl 59(1):573–584. https://doi.org/10.1016/j.camwa.2009.04.021
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
About this article
Cite this article
Viswanath, N.C. Transient study of Markov models with time-dependent transition rates. Oper Res Int J 22, 2209–2243 (2022). https://doi.org/10.1007/s12351-020-00613-2
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12351-020-00613-2