Skip to main content
SpringerLink
Log in

Nonstationary Policies and Average Optimality in Multichain Markov Decision Processes with a General Action Space

  • Published:
Journal of Mathematical Sciences Aims and scope Submit manuscript

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

REFERENCES

  1. A. Arapostathis, V. S. Borkar, E. Fernandez-Gaucherand, M. K. Ghosh, and S. I. Markus, "Discrete-time controlled Markov processes with average cost criterion," SIAM J. Contr. Optim., 31, 282-344 (1993).

    Google Scholar 

  2. D. P. Bertsekas, Dynamic Programming: Deterministic and Stochastic Models, Prentice-Hall, Englewood Cliffs, New Jersey (1987).

    Google Scholar 

  3. E. V. Denardo and B. Fox, "Multichain Markov renewal programs," SIAM J. Appl. Math., 16, 468-487 (1968).

    Google Scholar 

  4. E. B. Dynkin and A. A. Yushkevich, Controlled Markov Processes, Springer, New York (1979).

    Google Scholar 

  5. E. A. Feinberg, "The existence of a stationary "εoptimal policy for a finite Markov chain," Teor. Veroyatn. Primen., 23, 297-313 (1978).

    Google Scholar 

  6. B. Fox, "Existence of stationary optimal policies for some Markov renewal programs," SIAM Rev., 9, 573-576 (1967).

    Google Scholar 

  7. A. Y. Golubin, "A note on the convergence of policy iteration in Markov decision processes with compact action spaces," Math. Oper. Res., 28 (2003) (to appear).

  8. A. Hordijk, Dynamic Programming and Markov Potential Theory, Math. Centre, Amsterdam (1974).

    Google Scholar 

  9. R. A. Howard, Dynamic Programming and Markov Processes, Wiley, New York-London (1960).

    Google Scholar 

  10. S. S. Lavenberg and M. Reiser, "Mean value analysis of closed multichain queueing networks," J.A.C.M., 27, 313-322 (1980).

    Google Scholar 

  11. S. Stidham Jr. and R. R. Weber, "Control of service rates in networks of queues," Adv. Appl. Probab., 19, 202-218 (1987).

    Google Scholar 

  12. D. H. Wagner, "Survey of measurable selection theorems," SIAM J. Contr. Optim., 16, 859-903 (1977).

    Google Scholar 

  13. H. Zijm, "The optimality equations in multichain denumerable Markov decision processes with average cost criterion: The bounded cost case," Statist. Decisions, 3, 143-165 (1985).

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Institute of Electronics and Mathematics, B. Trekhsvyatitelskii 3/12, 109028, Moscow, Russia

    A. Y. Golubin

Authors
  1. A. Y. Golubin
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Golubin, A.Y. Nonstationary Policies and Average Optimality in Multichain Markov Decision Processes with a General Action Space. Journal of Mathematical Sciences 123, 3733–3740 (2004). https://doi.org/10.1023/B:JOTH.0000036314.29733.3d

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1023/B:JOTH.0000036314.29733.3d

Keywords

Search

Navigation