Numerical transient analysis of markov models

Abstract

We consider the numerical evaluation of Markov model transient behavior. Our research is motivated primarily by computer system dependability modeling. Other application areas include finitecapacity queueing models, closed queueing networks and inventory models. We focus our attention on the general problem of finding the state probability vector of a large, continuous-time, discrete-state Markov chain. Two computational approaches are examined in detail: uniformization and numerical linear multistep methods for ordinary differential equation solution. In general, uniformization provides greater accuracy but deals poorly with stiffness. A special stable ordinary differential equation solver deals well with stiffness, but it provides increased accuracy only at much greater cost. Examples are presented to illustrate the behavior of the techniques discussed as a function of model size, model stiffness, increased accuracy requirements and mission time.