Skip to main content
Log in

On stationary equilibria of a single-controller stochastic game

  • Published:
Mathematical Programming Submit manuscript

Abstract

We consider a two-person, general-sum, rational-data, undiscounted stochastic game in which one player (player II) controls the transition probabilities. We show that the set of stationary equilibrium points is the union of a finite number of sets such that, every element of each of these sets can be constructed from a finite number of extreme equilibrium strategies for player I and from a finite number of pseudo-extreme equilibrium strategies for player II. These extreme and pseudo-extreme strategies can themselves be constructed by finite (but inefficient) algorithms. Analogous results can also be established in the more straightforward case of discounted single-controller games.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. M.L. Balinski, “An algorithm for finding all vertices of convex polyhedral sets”,SIAM Journal on Applied Mathematics 9 (1961) 72–88.

    Article  MATH  MathSciNet  Google Scholar 

  2. T. Bewley and E. Kohlberg, “On stochastic games with stationary optimal strategies”,Mathematics of Operations Research 3 (1978) 104–125.

    MATH  MathSciNet  Google Scholar 

  3. D. Blackwell and T.S. Ferguson, “The big match”,Annals of Mathematical Statistics 39 (1968) 159–163.

    MathSciNet  MATH  Google Scholar 

  4. E.V. Denardo and B.L. Fox, “Multichain Markov renewal programs”,SIAM Journal on Applied Mathemàtics 16 (1968) 468–487.

    Article  MATH  MathSciNet  Google Scholar 

  5. J.A. Filar, “Algorithms for solving some undiscounted stochastic games”, Ph.D. Dissertation, University of Illinois at Chicago Circle (Chicago, IL, 1980).

    Google Scholar 

  6. J.A. Filar and T.E.S. Raghavan, “An algorithm for solving an undiscounted stochastic game in which one player controls transitions”, Research Memorandum, University of Illinois at Chicago Circle (Chicago, IL, 1979).

    Google Scholar 

  7. J.A. Filar and T.E.S. Raghavan, “A matrix game solution to a single-controller stochastic game”,Mathematics of Operations Research 9 (1984) 356–362.

    MATH  MathSciNet  Google Scholar 

  8. J.A. Filar, “The travelling inspector model”, Technical Report #374, Department of Mathematical Sciences, The Johns Hopkins University (Baltimore, Maryland, 1983).

    Google Scholar 

  9. A.J. Goldman, “Resolution and separation theorems for polyhedral convex sets”, in: H.W. Kuhn and A.W. Tucker, eds.,Linear inequalities and related systems (Princeton University Press, Princeton, NJ, 1956) pp. 41–71.

    Google Scholar 

  10. D. Gillette, “Stochastic games with zero-stop probabilities”, Annals of Mathematics Studies, 39 (Princeton University Press, Princeton, NJ, 1957) pp. 179–187.

    Google Scholar 

  11. A.J. Hoffman and R.M. Karp, “On non-terminating stochastic games”,Management Science 12 (1966) 359–370.

    Article  MathSciNet  Google Scholar 

  12. A. Hordijk and L.C.M. Kallenberg, “Linear programming and Markov decision chains”,Management Science 25 (1979) 352–362.

    MATH  MathSciNet  Google Scholar 

  13. A. Hordijk and L.C.M. Kallenberg, “Linear programming and Markov games II”, in: O. Moeschlin and D. Pallaschke, eds.,Game theory and mathematical economics (North-Holland, Amsterdam, 1981) pp. 307–320.

    Google Scholar 

  14. L.C.M. Kallenberg, “Linear programming and finite Markovian control problems”, Mathematical Centre Tracts 148, Amsterdam (1983).

  15. E. Kohlberg, “On repeated games with absorbing states” Annals of Statistics 2 (1974) pp. 724–738.

    MATH  MathSciNet  Google Scholar 

  16. H.W. Kuhn, “An algorithm for equilibrium points in bimatrix games”,Proceedings of the National Academy of Sciences 47 (1961) 1657–1662.

    Article  MATH  MathSciNet  Google Scholar 

  17. J.F. Mertens and A. Neyman, “Stochastic games have a value”,International Journal of Game Theory 10 (1981) 53–66.

    Article  MATH  MathSciNet  Google Scholar 

  18. C.A. Monash, “Stochastic games, the minimax theorem”, Ph.D. Thesis, Harvard University (Cambridge, Massachusetts, 1979).

    Google Scholar 

  19. T. Parthasarathy and T.E.S. Raghavan, “An orderfield property for stochastic games when one player controls transition probabilities”,Journal of Optimization Theory and Applications 33 (1981) 375–392.

    Article  MATH  MathSciNet  Google Scholar 

  20. L.S. Shapley, “Stochastic games”, Proceedings of the National Academy of Science, U.S.A. 39 (1953) 1095–1100.

    Article  MATH  MathSciNet  Google Scholar 

  21. M.A. Stern, “On stochastic games with limiting average payoff”, Ph.D. Dissertation, University of Illinois at Chicago Circle (Chicago, IL, 1975).

    Google Scholar 

  22. N.N. Vorob’ev, “Equilibrium points in bimatrix games”, Theoriya Veroyatnostej i ee Primeneniya 3 (1958) 318–331. [English translation in:Theory of Probability and Its Applications 3 (1958) 297–309.]

    MATH  MathSciNet  Google Scholar 

  23. O.J. Vrieze, “Linear programming and undiscounted stochastic games in which one player controls transitions”,Operations Research Spektrum 3 (1981) 29–35.

    Article  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Rights and permissions

Reprints and permissions

About this article

Cite this article

Filar, J.A. On stationary equilibria of a single-controller stochastic game. Mathematical Programming 30, 313–325 (1984). https://doi.org/10.1007/BF02591936

Download citation

  • Received:

  • Revised:

  • Issue Date:

  • DOI: https://doi.org/10.1007/BF02591936

Key words

Navigation