Abstract
We consider a two-person, general-sum, rational-data, undiscounted stochastic game in which one player (player II) controls the transition probabilities. We show that the set of stationary equilibrium points is the union of a finite number of sets such that, every element of each of these sets can be constructed from a finite number of extreme equilibrium strategies for player I and from a finite number of pseudo-extreme equilibrium strategies for player II. These extreme and pseudo-extreme strategies can themselves be constructed by finite (but inefficient) algorithms. Analogous results can also be established in the more straightforward case of discounted single-controller games.
Similar content being viewed by others
References
M.L. Balinski, “An algorithm for finding all vertices of convex polyhedral sets”,SIAM Journal on Applied Mathematics 9 (1961) 72–88.
T. Bewley and E. Kohlberg, “On stochastic games with stationary optimal strategies”,Mathematics of Operations Research 3 (1978) 104–125.
D. Blackwell and T.S. Ferguson, “The big match”,Annals of Mathematical Statistics 39 (1968) 159–163.
E.V. Denardo and B.L. Fox, “Multichain Markov renewal programs”,SIAM Journal on Applied Mathemàtics 16 (1968) 468–487.
J.A. Filar, “Algorithms for solving some undiscounted stochastic games”, Ph.D. Dissertation, University of Illinois at Chicago Circle (Chicago, IL, 1980).
J.A. Filar and T.E.S. Raghavan, “An algorithm for solving an undiscounted stochastic game in which one player controls transitions”, Research Memorandum, University of Illinois at Chicago Circle (Chicago, IL, 1979).
J.A. Filar and T.E.S. Raghavan, “A matrix game solution to a single-controller stochastic game”,Mathematics of Operations Research 9 (1984) 356–362.
J.A. Filar, “The travelling inspector model”, Technical Report #374, Department of Mathematical Sciences, The Johns Hopkins University (Baltimore, Maryland, 1983).
A.J. Goldman, “Resolution and separation theorems for polyhedral convex sets”, in: H.W. Kuhn and A.W. Tucker, eds.,Linear inequalities and related systems (Princeton University Press, Princeton, NJ, 1956) pp. 41–71.
D. Gillette, “Stochastic games with zero-stop probabilities”, Annals of Mathematics Studies, 39 (Princeton University Press, Princeton, NJ, 1957) pp. 179–187.
A.J. Hoffman and R.M. Karp, “On non-terminating stochastic games”,Management Science 12 (1966) 359–370.
A. Hordijk and L.C.M. Kallenberg, “Linear programming and Markov decision chains”,Management Science 25 (1979) 352–362.
A. Hordijk and L.C.M. Kallenberg, “Linear programming and Markov games II”, in: O. Moeschlin and D. Pallaschke, eds.,Game theory and mathematical economics (North-Holland, Amsterdam, 1981) pp. 307–320.
L.C.M. Kallenberg, “Linear programming and finite Markovian control problems”, Mathematical Centre Tracts 148, Amsterdam (1983).
E. Kohlberg, “On repeated games with absorbing states” Annals of Statistics 2 (1974) pp. 724–738.
H.W. Kuhn, “An algorithm for equilibrium points in bimatrix games”,Proceedings of the National Academy of Sciences 47 (1961) 1657–1662.
J.F. Mertens and A. Neyman, “Stochastic games have a value”,International Journal of Game Theory 10 (1981) 53–66.
C.A. Monash, “Stochastic games, the minimax theorem”, Ph.D. Thesis, Harvard University (Cambridge, Massachusetts, 1979).
T. Parthasarathy and T.E.S. Raghavan, “An orderfield property for stochastic games when one player controls transition probabilities”,Journal of Optimization Theory and Applications 33 (1981) 375–392.
L.S. Shapley, “Stochastic games”, Proceedings of the National Academy of Science, U.S.A. 39 (1953) 1095–1100.
M.A. Stern, “On stochastic games with limiting average payoff”, Ph.D. Dissertation, University of Illinois at Chicago Circle (Chicago, IL, 1975).
N.N. Vorob’ev, “Equilibrium points in bimatrix games”, Theoriya Veroyatnostej i ee Primeneniya 3 (1958) 318–331. [English translation in:Theory of Probability and Its Applications 3 (1958) 297–309.]
O.J. Vrieze, “Linear programming and undiscounted stochastic games in which one player controls transitions”,Operations Research Spektrum 3 (1981) 29–35.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Filar, J.A. On stationary equilibria of a single-controller stochastic game. Mathematical Programming 30, 313–325 (1984). https://doi.org/10.1007/BF02591936
Received:
Revised:
Issue Date:
DOI: https://doi.org/10.1007/BF02591936