Skip to main content
SpringerLink
Log in

Estimation for Discrete-time Semi-Markov Reward Processes: Analysis and Inference

  • Published:
Methodology and Computing in Applied Probability Aims and scope Submit manuscript

Abstract

In this paper, the moments of a class of reward processes defined on a discrete-time semi-Markov process and the asymptotic behaviors of the corresponding empirical estimators have been investigated. Some known results concerning the asymptotic distribution and properties of semi-Markov kernel have been obtained by a different approach. By using the empirical estimator of the semi-Markov kernel and the mentioned approach, the estimators for the moments of the reward process have been introduced and their asymptotic properties have been established. As a consequence of the strong consistency and asymptotic normality, the confidence intervals have also been obtained. A numerical example illustrates the results.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Similar content being viewed by others

References

  • Anderson TW, Goodman LA (1957) Statistical inference about Markov chain. Ann math Statist 28:89–110

    Article  MathSciNet  MATH  Google Scholar 

  • Barbu V, Boussmart M, Limnios N (2004) Discrete time semi-Markov model for reliability and survival analysis. Communications in Statistics – Theory and Methods 33(11):2833–2868

    Article  MathSciNet  MATH  Google Scholar 

  • Barbu V, Limnios N (2006) Empirical estimator for discrete-time semi-Markov processes with applications in reliability. Journal of Nonparametric statistics 18:483–498

    Article  MathSciNet  MATH  Google Scholar 

  • Cinlar E (1969) Markov renewal theory. Adv Appl Prob 1:123–187

    Article  MathSciNet  MATH  Google Scholar 

  • D’Amico G (2009) Nonparametric estimation of the accumulated reward for semi-Markov chains. SORT 33(2):159–170

    MathSciNet  MATH  Google Scholar 

  • D’Amico G (2010) Measuring the quality of life through Markov reward process: Analysis and inference. Environmetrics 21:208–220

    MathSciNet  Google Scholar 

  • D’Amico G, Guillen M, Manca R (2013) Semi-Markov Disability Models. Communications in Statistics: Theory and Methods 42(16):2172–2188

    MathSciNet  MATH  Google Scholar 

  • D’Amico G, Janssen J, Manca R (a 2011) A Non-Homogeneous Semi-Markov Reward Model for the Credit Spread Computation. Int J Theo Appl Finance 14 (2):221–238

    Article  MathSciNet  MATH  Google Scholar 

  • D’Amico G, Manca R, Salvi G (2013) A Semi-Markov Modulated Interest Rate Model. Statistics and Probability Letters 83:2094–2102

    Article  MathSciNet  MATH  Google Scholar 

  • Janssen J, Manca R (2006) Applied Semi-Markov processes. Springer, New York

    MATH  Google Scholar 

  • Khorshidian K, Soltani AR (2002) Asymptotic behavior of multivariate reward processes with nonlinear reward functions. Bulletin of the Iranian Mathematical society 28(2):1–17

    MathSciNet  MATH  Google Scholar 

  • Khorshidian K (2008) Strong law large numbers of semi-Markov reward processes. Asian Journal of Mathematics and Statistics 1(1):57–62

    Article  MathSciNet  Google Scholar 

  • Khorshidian K (2009) Central Limit Theorem for Nonlinear Semi-Markov Reward processes. Stoch Anal Appl 27(4):656–670

    Article  MathSciNet  MATH  Google Scholar 

  • Limnios N (2004) A functional central limit theorem for the empirical estimator of a semi-Markov kernel. J Nonparametric statistics 16(1–2):13–18

    Article  MathSciNet  MATH  Google Scholar 

  • Ouhbi B, Limnios N (1999) Nonparametric estimation for Markov processes based on its hazard rate. Stat Infer Stochat Proc 2(2):151–173

    Article  MathSciNet  MATH  Google Scholar 

  • Ouhbi B, Limnios N (2003) Nonparametric reliability estimation of semi-Markov processes. J Statist Plann Infer 109(1–2):155–165

    Article  MathSciNet  MATH  Google Scholar 

  • Pyke R (1961a) Markov renewal process: definitions and preliminary properties. Ann Math Statist 32:1231–1242

    Article  MathSciNet  MATH  Google Scholar 

  • Sadek A, Limnios N (2002) Asymptotic properties for maximum likelihood estimators for reliability and failure rates of Markov chains. Communication in Statistics-Theory and Methods 31(10):1837–1861

    Article  MathSciNet  MATH  Google Scholar 

  • Soltani AR (1996) Reward processes with nonlinear reward function. J Appl Prob 33:1101–1017

    Article  MathSciNet  MATH  Google Scholar 

  • Soltani AR, Khorshidian K (1998) Reward processes for semi-Markov processes: Asymptotic behavior. J Appl Prob 35:833–842

    Article  MathSciNet  MATH  Google Scholar 

  • Soltani AR, Khorshidian K, Ghafaripour A (2010) Prediction for reward processes. Stoch Model 26:242–255

    Article  MathSciNet  MATH  Google Scholar 

  • Van Der Vart AW (2000). Asymptotic statistics, Cambridge series in statistical and probability mathematics, 3, Cambridge University Press

Download references

Author information

Authors and Affiliations

  1. Department of Statistics, Shiraz University, Shiraz, Iran

    K. Khorshidian & F. Negahdari

  2. Department of Mathematics, Yasouj University, Yasouj, Iran

    H. A. Mardnifard

Authors
  1. K. Khorshidian
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. F. Negahdari
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. H. A. Mardnifard
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to K. Khorshidian.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Khorshidian, K., Negahdari, F. & Mardnifard, H.A. Estimation for Discrete-time Semi-Markov Reward Processes: Analysis and Inference. Methodol Comput Appl Probab 18, 885–900 (2016). https://doi.org/10.1007/s11009-015-9468-1

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11009-015-9468-1

Keywords

Mathematics Subject Classification (2010)

Search

Navigation