Definition of the Subject
Stochastic games, first introduced by Shapley [60], model dynamic interactions in which theenvironment changes in response to the behavior of the players. Formally, a stochastic game is a tuple \( { G = \langle N, S, (\mathcal{A}_i, A_i, u_i)_{i \in N}, q) } \) where
N is a set of players .
S is a state space. If S is uncountable, it is supplemented with a σ‑algebra of measurable sets.
For every player \( { i \in N } \), \( { \mathcal{A}_i } \) is a set of actions for that player, and \( { A_i \colon S \to \mathcal{A}_i } \) is a set‐valued (measurable) function that assigns to each state \( { s \in S } \) the set of actions \( { A_i(s) } \) that are available to player i in state s. If \( { \mathcal{A}_i } \) is uncountable, it is supplemented with a σ‑algebra of measurable sets. Denote \( SA = \{(s,a) \colon s \in S, a=(a_i)_{i \in N}, a_i\in A_i(s) \enskip \forall i \in N\} \). This is the set of all possible action profiles.
For every player \( { i \in N }...
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
That is, at each stage player 2 plays L with probability \( { \frac{1}{2} } \) and R with probability \( { \frac{1}{2} } \).
- 2.
Mertens and Neyman's [38]result actually holds in every stochastic game that satisfies a proper condition, which is always satisfied when the state and action spaces arefinite.
Abbreviations
- A stochastic game:
-
A repeated interaction between several participants in which the underlying state of the environment changes stochastically, and it depends on the decisions of the participants.
- A strategy:
-
A rule that dictates how a participant in an interaction makes his decisions as a function of the observed behavior of the other participants and of the evolution of the environment.
- Evaluation of stage payoffs:
-
The way that a participant in a repeated interaction evaluates the stream of stage payoffs that he receives (or stage costs that he pays) along the interaction.
- An equilibrium:
-
A collection of strategies, one for each player, such that each player maximizes (or minimizes, in case of stage costs) his evaluation of stage payoffs given the strategies of the other players.
- A correlated equilibrium:
-
An equilibrium in an extended game in which at the outset of the game each player receives a private signal, and the vector of private signals is chosen according to a known joint probability distribution. In the extended game, a strategy of a player depends, in addition to past play, on the signal he received.
Bibliography
Primary Literature
Altman E (2005) Applications of dynamic games in queues. Adv Dyn Games 7:309–342
Altman E, Gaitsgory VA (1995) A hybrid (differential‐stochastic) zero-sum game with fast stochastic part. Ann Int Soc Dyn Games 3:47–59
Altman E, Solan E (2007) Games with constraints with networking applications. Preprint
Altman E, Avrachenkov K, Marquez R, Miller G (2005) Zero-sum constrained stochastic games with independent state processes. Math Methods Oper Res 62:375–386
Altman E, Avrachenkov K, Bonneau N, Debbah M, El‐Azouzi R, Sadoc Menasche D (2008) Constrained cost‐coupled stochastic games with independent state processes. Oper Res Lett 36:160–164
Amir R (1996) Continuous stochastic games of capital accumulation with convex transitions. Games Econ Behav 15:111–131
Aumann RJ (1974) Subjectivity and correlation in randomized strategies. J Math Econ 1:67–96
Aumann RJ (1987) Correlated equilibrium as an expression of bayesian rationality. Econometrica 55:1–18
Başar T, Olsder GJ (1995) Dynamic noncooperative game theory. Academic Press, New York
Bewley T, Kohlberg E (1976) The asymptotic theory of stochastic games. Math Oper Res 1:197–208
Blackwell D (1956) An analog of the minimax theorem for vector payoffs. Pac J Math 6:1–8
Blackwell D, Ferguson TS (1968) The big match. Ann Math Stat 39:159–163
Chari V, Kehoe P (1990) Sustainable plans. J Political Econ 98:783–802
Chatterjee K, Majumdar R, Henzinger TA (2008) Stochastic limit‐average games are in EXPTIME. Int J Game Theory 37:219–234
Coulomb JM (2003) Absorbing games with a signalling structure. In: Neyman A, Sorin S (eds) Stochastic games and applications. NATO Science Series. Kluwer, Dordrecht, pp 335–355
Coulomb JM (2003) Games with a recursive structure. In: Neyman A, Sorin S (eds) Stochastic games and applications. NATO Science Series. Kluwer, Dordrecht, pp 427–442
Dutta P, Sundaram RK (1992) Markovian equilibrium in a class of stochastic games: Existence theorems for discounted and undiscounted models. Econ Theory 2:197–214
Dutta P, Sundaram RK (1993) The tragedy of the commons? Econ Theory 3:413–426
Filar JA, Vrieze K (1996) Competitive Markov decision processes. Springer
Fink AM (1964) Equilibrium in a stochastic n‑person game. J Sci Hiroshima Univ 28:89–93
Flesch J, Thuijsman F, Vrieze K (1997) Cyclic Markov equilibria in stochastic games. Int J Game Th 26:303–314
Flesch J, Thuijsman F, Vrieze OJ (2003) Stochastic games with non‐observable actions. Math Meth Oper Res 58:459–475
Flesch J, Schoenmakers G, Vrieze K (2008) Stochastic games on a product state space. Math Oper Res 33:403–420
Flesch J, Thuijsman F, Vrieze OJ (2007) Stochastic games with additive transitions. Europ J Oper Res 179:483–497
Forges F (1990) Universal mechanisms. Econometrica 58:1341–1364
Fortnow L, Kimmel P (1998) Beating a finite automaton in the big match. In: Proceedings of the 7th conference on theoretical aspects of rationality and knowledge. Morgan Kaufmann, San Francisco, pp 225–234
Gillette D (1957) Stochastic games with zero stop probabilities, contributions to the theory of games, vol 3. Princeton University Press, Princeton
Herings JJP, Peeters RJAP (2004) Stationary equilibria in stochastic games: Structure, selection, and computation. J Econ Theory 118:32–60
Jaskiewicz A, Nowak AS (2006) Zero-sum ergodic stochastic games with Feller transition probabilities. SIAM J Control Optim 45:773–789
Kakutani S (1941) A generalization of Brouwer's fixed point theorem. Duke Math J 8:457–459
Kohlberg E (1974) Repeated games with absorbing states. Ann Stat 2:724–738
Krausz A, Rieder U (1997) Markov games with incomplete information. Math Meth Oper Res 46:263–279
Levhari D, Mirman L (1980) The great fish war: An example using a dynamic Cournot–Nash solution. Bell J Econ 11(1):322–334
Maitra A, Sudderth W (1998) Finitely additive stochastic games with Borel measurable payoffs. Int J Game Theory 27:257–267
Martin DA (1998) The determinacy of Blackwell games. J Symb Logic 63:1565–1581
Mertens JF (1987) Repeated games. In: Proceedings of the international congress of mathematicians, American Mathematical Society, Berkeley, California, pp 1528–1577
Mertens JF, Neyman A (1981) Stochastic games. Int J Game Th 10:53–66
Mertens JF, Parthasarathy T (1987) Equilibria for discounted stochastic games, CORE Discussion Paper No. 8750. (Also published in Stochastic Games and Applications, Neyman A, Sorin S (eds), NATO Science Series, Kluwer, 131–172)
Mertens JF, Sorin S, Zamir S (1994) Repeated games, CORE Discussion Paper 9420-9422
Milman E (2006) Approachable sets of vector payoffs in stochastic games. Games Econ Behav 56:135–147
Neyman A, Sorin S (2003) Stochastic games and applications. NATO Science Series. Kluwer
Nowak AS (1985) Existence of equilibrium stationary strategies in discounted noncooperative stochastic games with uncountable state space. J Optim Theory Appl 45:591–620
Nowak AS (2003) N‑person stochastic games: Extensions of the finite state space case and correlation. In: Neyman A, Sorin S (eds) Stochastic games and applications. NATO Science Series. Kluwer, Dordrecht, pp 93–106
Nowak AS (2003) On a new class of nonzero‐sum discounted stochastic games having stationary Nash equilibrium points. Int J Game Theory 32:121–132
Nowak AS (2003) Zero-sum stochastic games with Borel state spaces. In: Neyman A, Sorin S (eds) Stochastic games and applications. NATO Science Series. Kluwer, Dordrecht, pp 77–91
Nowak AS, Raghavan TES (1991) Existence of stationary correlated equilibria with symmetric information for discounted stochastic games. Math Oper Res 17:519–526
Phelan C, Stacchetti E (2001) Sequential equilibria in a Ramsey tax model. Econometrica 69:1491–1518
Puterman ML (1994) Markov decision processes: Discrete stochastic dynamic programming. Wiley, Hoboken
Raghavan TES, Syed Z (2002) Computing stationary Nash equilibria of undiscounted single‐controller stochastic games. Math Oper Res 27:384–400
Raghavan TES, Syed Z (2003) A policy improvement type algorithm for solving zero-sum two‐person stochastic games of perfect information. Math Program Ser A 95:513–532
Renault J (2006) The value of Markov chain games with lack of information on one side. Math Oper Res 31:490–512
Renault J (2007) The value of repeated games with an informed controller. Preprint
Renault J (2007) Uniform value in dynamic programming. Preprint
Rosenberg D, Solan E, Vieille N (2002) Blackwell optimality in Markov decision processes with partial observation. Ann Statists 30:1178–1193
Rosenberg D, Solan E, Vieille N (2004) Stochastic games with a single controller and incomplete information. SIAM J Control Optim 43:86–110
Rosenberg D, Solan E, Vieille N (2006) Protocol with no acknowledgement. Oper Res, forthcoming
Sagduyu YE, Ephremides A (2003) Power control and rate adaptation as stochastic games for random access. Proc 42nd IEEE Conf Decis Control 4:4202–4207
Savani R, von Stengel B (2004) Exponentially many steps for finding a Nash equilibrium in a bimatrix game. Proc 45th Ann IEEE Symp Found Comput Sci 2004:258–267
Shapley LS (1953) Stochastic games. Proc Nat Acad Sci USA 39:1095–1100
Simon RS (2003) The structure of non-zero-sum stochastic games. Adv Appl Math 38:1–26
Solan E (1998) Discounted stochastic games. Math Oper Res 23:1010–1021
Solan E (1999) Three‐person absorbing games. Math Oper Res 24:669–698
Solan E (2001) Characterization of correlated equilibria in stochastic games. Int J Game Theory 30:259–277
Solan E, Vieille N (2001) Quitting games. Math Oper Res 26:265–285
Solan E, Vieille N (2002) Correlated equilibrium in stochastic games. Games Econ Behav 38:362–399
Solan E, Vieille N (2007) Calculating uniform optimal strategies and equilibria in two‐player stochastic games. Preprint
Solan E, Vohra R (2002) Correlated equilibrium payoffs and public signalling in absorbing games. Int J Game Theory 31:91–122
Sorin S (1984) Big match with lack of information on one side (part 1). Int J Game Theory 13:201–255
Sorin S (1985) Big match with lack of information on one side (part 2). Int J Game Theory 14:173–204
Sorin S (1986) Asymptotic properties of a non‐zerosum stochastic games. Int J Game Theory 15:101–107
Sorin S (2002) A first course on zero-sum repeated games. Mathématiques et Applications, vol 37. Springer
Sorin S (2003) Stochastic gameswith incomplete information. In: Neyman A, Sorin S (eds) Stochastic Games and Applications. NATO Science Series. Kluwer, Berlin, pp 375–395
Sorin S, Zamir S (1991) Big match with lack of information on one side (part 3). In: Raghavan TES et al (eds) Stochastic games and related topics. Kluwer, pp 101–112
Takahashi M (1962) Stochastic games with infinitely many strategies. J Sci Hiroshima Univ Ser A-I 26:123–134
Thuijsman F, Raghavan TES (1997) Perfect information stochastic games and related classes. Int J Game Theory 26:403–408
Vieille N (2000) Equilibrium in 2‑person stochastic games I: A Reduction. Israel J Math 119:55–91
Vieille N (2000) Equilibrium in 2‑person stochastic games II: The case of recursive games. Israel J Math 119:93–126
Vrieze OJ, Thuijsman F (1989) On equilibria in repeated games with absorbing states. Int J Game Theory 18:293–310
Vrieze OJ, Tijs SH (1982) Fictitious play applied to sequences of games and discounted stochastic games. Int J Game Theory 12:71–85
Whitt W (1980) Representation and approximation of noncooperative sequential games. SIAM J Control Optim 18:33–48
Books and Reviews
Başar T, Olsder GJ (1995) Dynamic noncooperative game theory. Academic
Filar JA, Vrieze K (1996) Competitive Markov decision processes. Springer
Maitra AP, Sudderth WD (1996) Discrete gambling and stochastic games. Springer
Mertens JF (2002) Stochastic games. In: Aumann RJ, Hart S (eds) Handbook of game theory with economic applications, vol 3. Elsevier, pp 1809–1832
Mertens JF, Sorin S, Zamir S (1994) Repeated games. CORE Discussion Paper 9420-9422
Raghavan TES, Shapley LS (1991) Stochastic games and related topics: In honor of Professor L.S. Shapley. Springer
Vieille N (2002) Stochastic games: Recent results. In: Aumann RJ, Hart S (eds) Handbook of game theory with economic applications, vol 3. Elsevier, pp 1833–1850
Acknowledgments
I thank Eitan Altman, János Flesch, Yuval Heller, Jean‐Jacques Herings, AyalaMashiach‐Yakovi, Andrzej Nowak, Ronald Peeters, T.E.S. Raghavan, Jérôme Renault, Nahum Shimkin, Robert Simon, Sylvain Sorin, WilliamSudderth, and Frank Thuijmsman, for their comments on an earlier version of the entry.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2009 Springer-Verlag
About this entry
Cite this entry
Solan, E. (2009). Stochastic Games. In: Meyers, R. (eds) Encyclopedia of Complexity and Systems Science. Springer, New York, NY. https://doi.org/10.1007/978-0-387-30440-3_522
Download citation
DOI: https://doi.org/10.1007/978-0-387-30440-3_522
Publisher Name: Springer, New York, NY
Print ISBN: 978-0-387-75888-6
Online ISBN: 978-0-387-30440-3
eBook Packages: Physics and AstronomyReference Module Physical and Materials ScienceReference Module Chemistry, Materials and Physics