Abstract
We report on a series of computational experiments with artificial agents learning in the context of games. Two kinds of learning are investigated: (1) a simple form of associative learning, called Q-learning, which occurs in state space, and (2) a simple form of learning, which we introduce here, that occurs in policy space. We compare the two methods on a number of repeated 2×2 games. We conclude that learning in policy space is an effective and promising method for learning in games.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Colin F. Camerer, Behavioral game theory: Experiments in strategic interaction, Russell Sage Foundation and Princeton University Press, New York, NY and Princeton, NJ, 2003.
D. Fundenberg and E. Maskin, The folk theorem with discounting and with incomplete information, Econometrica 54 (1986), 533–554.
Steven O. Kimbrough and Ming Lu, A note on Q-learning in the Cournot game, WeB 2003: Proceedings of the Second Workshop in e-Business (Seattle, WA), December 13–14, 2003, Available at http://opimsun.wharton.upenn.edu/~sok/sokpapers/2004/cournot-rl-note-final.doc.
—, Simple reinforcement learning agents: Pareto beats Nash in an algorithmic game theory study, Information Systems and e-Business (forthcoming 2004).
Leslie Pack Kaelbling, Michael L. Littman, and Andrew W. Moore, Reinforcement learning: A survey, Journal of Artificial Intelligence Research 4 (1996), 237–285.
Anatol Rapoport, Melvin J. Guyer, and David G. Gordon, The 2×2 game, The University of Michigan Press, Ann Arbor, MI, 1976.
Richar S. Sutton and Andrew G. Barto, Reinforcement learning: An introduction, The MIT Press, Cambridge, MA, 1998.
T. Sandholm and R. Crites, Multiagent reinforcement learning in iterated prisoner’s dilemma, Biosystems 37 (1995), 147–166, Special Issue on the Prisoner’s Dilemma.
Brian Skyrms, The stag hunt, World Wide Web, 2001, Proceedings and Addresses of the American Philosophical Association. http://www.lps.uci.edu/home/fac-staff/faculty/skyrms/StagHunt.pdf. Accessed September 2004.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2005 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Kimbrough, S.O., Lu, M., Kuo, A. (2005). A Note on Strategic Learning in Policy Space. In: Kimbrough, S.O., Wu, D. (eds) Formal Modelling in Electronic Commerce. International Handbooks on Information Systems. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-26989-4_18
Download citation
DOI: https://doi.org/10.1007/3-540-26989-4_18
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-21431-1
Online ISBN: 978-3-540-26989-2
eBook Packages: Business and EconomicsBusiness and Management (R0)