Skip to main content
SpringerLink
Log in

Perfect information two-person zero-sum markov games with imprecise transition probabilities

  • Original Article
  • Published:
Mathematical Methods of Operations Research Aims and scope Submit manuscript

Abstract

Based on an extension of the controlled Markov set-chain model by Kurano et al. (in J Appl Prob 35:293–302, 1998) into competitive two-player game setting, we provide a model of perfect information two-person zero-sum Markov games with imprecise transition probabilities. We define an equilibrium value for the games formulated with the model in terms of a partial order and then establish the existence of an equilibrium policy pair that achieves the equilibrium value. We further analyze finite-approximation error bounds obtained from a value iteration-type algorithm and discuss some applications of the model.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Altman E, Feinberg EA, Shwartz A (2000) Weighted discounted stochastic games with perfect information. Annals of Dynamic Games, vol 5, Birkhauser, V. Gaitsgory, J. Filar and K. Mizukami, editors, pp. 303–323

  • Bes C, Lasserre JB (1986) An on-line procedure in discounted infinite-horizon stochastic optimal control. J Optim Theory Appl 50(1):61–67

    Article  MATH  MathSciNet  Google Scholar 

  • Chang HS, Marcus SI (2003) Two-person zero-sum Markov games: receding horizon approach. IEEE Trans Automat Control 48(11):1951–1961

    Article  MathSciNet  Google Scholar 

  • Condon A (1992) The complexity of stochastic games. Inform Comput 96(2):101–150

    Article  MathSciNet  Google Scholar 

  • Filar J, Vrieze K (1996) Competitive Markov decision processes. Springer, Berlin Heidelberg New York

    Google Scholar 

  • Givan R, Leach S, Dean T (2000) Bounded-parameter Markov decision processes. Artif Intell 122:71–109

    Article  MATH  MathSciNet  Google Scholar 

  • Hartfiel DJ (1998) Markov Set-Chains. Springer, Berlin Heidelberg New York

    MATH  Google Scholar 

  • Littman M (1996) Algorithms for sequential decision making. Ph.D Thesis, Department of Computer Science, Brown University

  • Kurano M, Song J, Hosaka M, Huang Y (1998) Controlled Markov set-chains with discounting. J Appl Prob 35:293–302

    Article  MATH  MathSciNet  Google Scholar 

  • Lasserre JB, Bes C (1984) Infinite horizon nonstationary stochastic optimal control problems: a planning horizon result. IEEE Trans Automat Control AC-29:836–837

    Article  MATH  MathSciNet  Google Scholar 

  • Puterman ML (1994) Markov decision processes: discrete stochastic dynamic programming. Wiley, New York

    MATH  Google Scholar 

  • Raghavan TES, Syed Z (2003) A policy improvement type algorithm for solving zero-sum two-person stochastic games of perfect information. Math Programm 95(3):513–532

    Article  MATH  MathSciNet  Google Scholar 

  • Satia JK, Lave RE (1973) Markovian decision processes with uncertain transition probabilities. Oper Res 21:728–740

    MATH  MathSciNet  Google Scholar 

  • Shapley L (1953) Stochastic games. Proc Natl Acad Sci USA 39:1095–1100

    Article  MATH  MathSciNet  Google Scholar 

  • White CC, Eldeib HK (1994) Markov decision processes with imprecise transition probabilities. Oper Res 43:739–749

    Article  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science and Engineering, Sogang University, Seoul, South Korea

    Hyeong Soo Chang

Authors
  1. Hyeong Soo Chang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hyeong Soo Chang.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Chang, H.S. Perfect information two-person zero-sum markov games with imprecise transition probabilities. Math Meth Oper Res 64, 335–351 (2006). https://doi.org/10.1007/s00186-006-0081-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00186-006-0081-5

Keywords

Search

Navigation