Abstract
One-step algorithms are presented for two classes of structured stochastic games, namely, those with additive rewards and transitions and those which have switching controllers. Solutions to such classes of games under the average reward criterion can be derived from optimal solutions to appropriate bilinear programs. The validity of using bilinear programming as a solution method follows from two preliminary theorems, the first of which is a complete classification of undiscounted stochastic games with optimal stationary strategies. The second of these preliminary theorems relaxes the conditions of the classification theorem for certain classes of stochastic games and provides the basis for the bilinear programming results. Analogous results hold for the discounted stochastic games with the above special structures.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Shapley, L. S.,Stochastic Games, Proceedings of the National Academy of Sciences, Vol. 39, pp. 1095–1100, 1953.
Gillete, D.,Stochastic Games with Zero Stop Probabilities, Contributions to the Theory of Games, Edited by M. Dresher, A. W. Tucker, and P. Wolfe, Princeton University Press, Princeton, New Jersey, pp. 179–188, 1957.
Hoffman, A. J., andKarp, R. M.,On Nonterminating Stochastic Games, Management Science, Vol. 12, pp. 359–370, 1966.
Kohlberg, E.,On Repeated Games with Absorbing States, Annals of Statistics, Vol. 2, pp. 724–738, 1974.
Stern, M.,On Stochastic Games with Limiting Average Payoff, PhD Thesis, University of Chicago at Chicago Circle, Illinois, 1975.
Parthasarathy, T., andRaghavan, T. E. S.,An Order Field Property for Stochastic Games when One Player Controls Transitions, Journal of Optimization Theory and Applications, Vol. 33, pp. 375–392, 1981.
Bewley, T., andKohlberg, E.,On Stochastic Games with Stationary Strategies, Mathematics of Operations Research, Vol. 3, pp. 104–125, 1978.
Filar, J. A.,Ordered Field Property for Stochastic Games when the Player Who Controls Changes from State to State, Journal of Optimization Theory and Applications, Vol. 34, pp. 503–515, 1981.
Monash, C. A.,Stochastic Games: The Minimax Theorem, PhD Thesis, Harvard University, Cambridge, Massachusetts, 1979.
Mertens, J. F., andNeyman, A.,Stochastic Games Have a Value, International Journal of Game Theory, Vol. 10, pp. 53–66, 1981.
Blackwell, D., andFerguson, T. S.,The Big Match, Annals of Mathematical Statistics, Vol. 39, pp. 159–163, 1968.
Bewley, T., andKohlberg, E.,The Asymptotic Solution of a Recursion Equation in Stochastic Games, Mathematics of Operations Research, Vol. 1, pp. 321–336, 1976.
Bewley, T., andKohlberg, E.,The Asymptotic Theory of Stochastic Games, Mathematics of Operations Research, Vol. 1, pp. 197–208, 1976.
Van der Wal, J.,Discounted Markov Games: Successive Approximations and Stopping Times, International Journal of Game Theory, Vol. 6, pp. 11–22, 1977.
Federgruen, A.,Successive Approximation Methods in Undiscounted Stochastic Games, Operations Research, Vol. 28, pp. 794–810, 1980.
Hordijk, A., andKallenberg, L. C. M.,Linear Programming and Markov Games, I, Game Theory and Mathematical Economics, Edited by O. Moeschlin and D. Pallaschke, North-Holland Publishing Company, Amsterdam, Holland, pp. 291–305, 1981.
Hordijk, A., andKallenberg, L. C. M.,Linear Programming and Markov Games, II, Game Theory and Mathematical Economics, Edited by O. Moeschlin and D. Pallaschke, North-Holland Publishing Company, Amsterdam, Holland, pp. 307–319, 1981.
Vrieze, O. J.,Linear Programming and Undiscounted Games in Which One Player Controls Transitions, OR Spektrum, Vol. 3, pp. 29–35, 1981.
Filar, J. A., andRaghavan, T. E. S.,A Matrix Game Solution of the Single Controller Stochastic Game, Mathematics of Operations Research, Vol. 9, pp. 356–362, 1984.
Vrieze, O. J., Tijs, S. H., Raghavan, T. E. S., andFilar, J. A.,A Finite Algorithm for the Switching Controller Stochastic Game, OR Spektrum, Vol. 5, pp. 15–24, 1983.
Vrieze, O. J.,Stochastic Games with Finite State and Action Spaces, PhD Thesis, Free University of Amsterdam, Amsterdam, Holland, 1983.
Parthasarathy, T., Tijs, S. H., andVrieze, O. J.,Stochastic Games with State Independent Transitions and Separable Rewards, Selected Topics in Operations Research, Edited by G. Hammer and D. Pallaschke, Springer-Verlag, Berlin, Germany, 1984.
Raghavan, T. E. S., Tijs, S. H., andVrieze, O. J.,On Stochastic Games with Additive Reward and Transition Structure, Journal of Optimization Theory and Applications, Vol. 47, pp. 451–464, 1985.
Sobel, M. J.,Noncooperative Stochastic Games, Annals of Mathematical Statistics, Vol. 42, pp. 1930–1935, 1971.
Filar, J. A., andSchultz, T.,Nonlinear Programming and Stationary Strategies in Stochastic Games, Mathematical Programming, Vol. 35, pp. 243–247, 1986.
Kallenberg, L. C. M.,Linear Programming and Finite Markovian Control Problems, Mathematical Center, Amsterdam, Holland, 1983.
Blackwell, D.,Discrete Dynamic Programming, Annals of Mathematical Statistics, Vol. 33, pp. 104–125, 1962.
Sherali, H. N., andShetty, C. M.,A Finitely Convergent Algorithm for Bilinear Programming Problems, Mathematical Programming, Vol. 19, pp. 14–31, 1980.
Gallo, G., andUlkucu, A.,Bilinear Programming: An Exact Algorithm, Mathematical Programming, Vol. 12, pp. 173–194, 1977.
Faiz, A., andFalk, J.,Jointly Constrained Biconvex Programming, Mathematics of Operations Research, Vol. 8, pp. 273–286, 1983.
Author information
Authors and Affiliations
Additional information
Communicated by G. Leitmann
This research was supported in part by the Air Force Office of Scientific Research and by the National Science Foundation under Grant No. ECS-850-3440.
Rights and permissions
About this article
Cite this article
Filar, J.A., Schultz, T.A. Bilinear programming and structured stochastic games. J Optim Theory Appl 53, 85–104 (1987). https://doi.org/10.1007/BF00938818
Issue Date:
DOI: https://doi.org/10.1007/BF00938818