Skip to main content
SpringerLink
Log in

A Framework for Computational Strategic Analysis: Applications to Iterated Interdependent Security Games

  • Published:
Computational Economics Aims and scope Submit manuscript

Abstract

Past work on tournaments in iterated prisoner’s dilemma and the evolution of cooperation spawned by Axelrod has contributed insights about achieving cooperation in social dilemmas, as well as a framework for strategic analysis in such settings. We present a broader, more extensive framework for strategic analysis in general games, which we illustrate in the context of a particular social dilemma encountered in interdependent security settings. Our framework is fully quantitative and computational, allowing one to measure the quality of strategic alternatives across a series of measures, and as a function of relevant game parameters. Our special focus on performing analysis over a parametric landscape is motivated by public policy considerations, where possible interventions are modeled as affecting particular parameters of the game. Our findings qualify the touted efficacy of the Tit-for-Tat strategy, demonstrate the importance of monitoring, and exhibit a phase transition in cooperative behavior in response to a manipulation of policy-relevant parameters of the game.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Notes

  1. Following publication of the tournaments there ensued a flurry of studies pointing to shortcomings in Tit-for-Tat and offering alternatives, e.g., (Nowak and Sigmund 1993).

  2. We do not deal with the mixed strategy equilibria of the stage game here, since pure strategy equilibria always exist in our setting.

  3. We are indebted for this idea to Walsh et al. (2002). It assumes that replicator dynamics converges, which it did in every instance we had observed.

  4. The NetLogo implementation used for our data can be found at http://opim.wharton.upenn.edu/~sok/netlogo/IDS-experiments.nlogo. An updated version can be found at http://opim.wharton.upenn.edu/~sok/AGEbook/nlogo/IDS-2x2-Tournaments.nlogo.

  5. We remind the reader that although IDS games have stochastic payoffs, and the behavioral experiments have shown that this matters to players in laboratory studies, our discussion here proceeds in terms of the estimated expected values realized by our computational experiments.

  6. Note that the observed phase transitions are not immediate from stage game analysis, since the phase transition points do not correspond to the stage game transitions between equilibria.

References

  • Axelrod, R. (1980a). Effective choice in the prisoner’s dilemma. Journal of Conflict Resolution, 24(1), 3–25.

    Google Scholar 

  • Axelrod, R. (1980b). More effective choice in the prisoner’s dilemma. Journal of Conflict Resolution, 24(3), 379–403.

    Article  Google Scholar 

  • Axelrod, R. (1984). The Evolution of Cooperation. New York, NY: Basic Books Inc.

    Google Scholar 

  • Axelrod, R., & Hamilton, W. D. (1981). The evolution of cooperation. Science, 211, 1390–1396.

    Article  Google Scholar 

  • Bartholdi, J. J., Butler, C. A., & Trick, M. A. (1986). More on evolution of cooperation. Journal of Conflict Resolution, 30(1), 129–140.

    Article  Google Scholar 

  • Bendor, J. (1993). Uncertainty and the evolution of cooperation. Journal of Conflict Resolution, 37(4):709–734. doi:10.1177/0022002793037004007. URL http://jcr.sagepub.com/cgi/content/abstract/37/4/709.

  • Bendor, J., Kramer, R. M., Stout S. (1991). When in doubt... Journal of Conflict Resolution, 35(4):691–719. doi:10.1177/0022002791035004007. URL http://jcr.sagepub.com/cgi/content/abstract/35/4/691.

  • Bereby-Meyer, Y., Roth, A. E. (2006). The speed of learning in noisy games: Partial reinforcement and the sustainability of cooperation. The American Economic Review, 96(4), 1029–1042. ISSN 00028282. URL http://www.jstor.org/stable/30034329.

  • Bowling, M. (2004). Convergence and no-regret in multiagent learning. In Neural Information Processing Systems.

  • Bowling, M., & Veloso, M. (2002). Multiagent learning using a variable learning rate. Artificial Intelligence, 136, 215–250.

    Article  Google Scholar 

  • Camerer, C., & Kunreuther, H. (1989). Decision processes for low probability events: Policy implications. Journal of Policy Analysis and Management, 8(4), 565–592.

    Article  Google Scholar 

  • Friedman, D. (1991). Evolutionary games in economics. Econometrica, 59(3), 637–666.

    Article  Google Scholar 

  • Fudenberg, D., & Maskin, E. (1990). Evolution and cooperation in noisy repeated games. The American Economic Review, 80(2), 274–279.

    Google Scholar 

  • Fudenberg, D., & Tirole, J. (1991). Game theory. Cambridge, MA: The MIT Press.

    Google Scholar 

  • Fudenberg, D., Rand, D. G., Dreber, A. (2010). Slow to anger and fast to forgive: Cooperation in an uncertain world. Working paper, Harvard University, 26 May 2010.

  • Greenwald, A., Hall, K. (2003). Correlated-q learning. In International Conference on Machine Learning, pp. 242–249

  • Harsanyi, J. C., & Selten, R. (1988). A general theory of equilibrium selection in games. Cambridge, MA: MIT Press.

    Google Scholar 

  • Heal, G., & Kunreuther, H. (2005a). You only die once: Interdependent security in an uncertain world. In H. W. Richardson, P. Gordon, & J. E. Moore II (Eds.), The economic impact of terrorist attacks (pp. 35–56). Cheltenham: Edward Elgar.

    Google Scholar 

  • Heal, G., & Kunreuther, H. (2005b). IDS models of airline security. Journal of Conflict Resolution, 49(2), 201–217.

    Article  Google Scholar 

  • Kimbrough, S. O. (2012). Agents, games, and evolution: Strategies at work and play. Boca Raton, FL: CRC Press.

    Google Scholar 

  • Kunreuther, H., & Heal, G. (2003). Interdependent security. The Journal of Risk and Uncertainty, 26(2/3), 231–249.

    Article  Google Scholar 

  • Kunreuther, H., Silvasi, G., Bradlow, E., & Small, D. (2009). Bayesian analysis of deterministic and stochastic prisoner’s dilemma games. Judgment and Decision Making, 4(5), 363–384.

    Google Scholar 

  • Littman, M. (1994). Markov games as a framework for multi-agent reinforcement learning. In International Conference on Machine Learning, pp. 157–163.

  • Marinoff, L. (1992). Maximizing expected utilities in the prisoner’s dilemma. Journal of Conflict Resolution, 36(1):183–216. doi:10.1177/0022002792036001007. URL http://jcr.sagepub.com/cgi/content/abstract/36/1/183.

  • Nowak, M., & Sigmund, K. (1993). A strategy of win-stay lose-shift that outperforms tit-for-tat in the prisoner’s dilemma game. Nature, 364, 56–58.

    Article  Google Scholar 

  • Ostrom, E. (2009). Beyond markets and states: Polycentric governance of complex economic systems. In Nobel Prize Lecture, pp. 408–444.

  • Pelc, A., Pelc, K. J. (2009). Same game, new tricks. Journal of Conflict Resolution, 53(5):774–793. doi:10.1177/0022002709339045. URL http://jcr.sagepub.com/cgi/content/abstract/53/5/774.

  • Powers, R., Shoham, Y. (2005). New criteria and a new algorithm for learning in multi-agent systems. In Neural Information Processing Systems, pp. 1089–1096.

  • Rendell, L., Boyd, R., Cownden, D., Enquist, M., Eriksson, K., Feldman, M. W., et al. (2010). Why copy others? Insights from the social learning strategies tournament. Science, 328, 208–213.

    Article  Google Scholar 

  • Rogers, A., Dash, R. K., Ramchurn, S. D., Vytellngum, P., & Jennings, N. R. (2007). Coordinating team players within a noisy Iterated Prisoner’s Dilemma tournament. Theoretical Computer Science, 377, 243–259.

    Article  Google Scholar 

  • Rogers, E. M. (2003). Diffusion of innovations (5th ed.). Florence, MA: Free Press.

    Google Scholar 

  • Roughgarden, J. (2009). The genial gene: Deconstructing Darwinian selfishness. Berkeley, CA: The University of California Press.

    Google Scholar 

  • Skyrms, B. (2010). Signals: Evolution, learning & information. Oxford: Oxford University Press.

    Book  Google Scholar 

  • Thomas, E. A. C., & Feldman, M. W. (1988). Behavior-dependent contexts for repeated plays of the Prisoner’s Dilemma. Journal of Conflict Resolution, 32(4), 699–726.

    Article  Google Scholar 

  • Walsh, W. E., Das, R., Tesauro, G., Kephart, J. O. (2002). Analyzing complex strategic interactions in multi-agent systems. In Fourth Workshop on Game Theoretic and Decision Theoretic Agents.

  • Wu, J., Axelrod, R. (1995). How to cope with noise in the iterated prisoner’s dilemma. Journal of Conflict Resolution, 39(1):183–189. doi:10.1177/0022002795039001008. URL http://jcr.sagepub.com/cgi/content/abstract/39/1/183.

  • Xiao, E., Kunreuther, H. (2010). Punishment and cooperation in stochastic social dilemmas. Working paper, Wharton School, University of Pennsylvania, 14 July 2010.

  • Zawadzki, E., Lipson, A., Leyton-Brown, K. (2008). Empirically evaluating multiagent learning algorithms. Working paper.

Download references

Acknowledgments

This work was supported in part by the Climate Decision Making Center (CDMC) located in the Department of Engineering and Public Policy (Cooperative Agreement between the NSF (SES-0345798) and Carnegie Mellon University), CREATE (National Center for Risk and Economic Analysis of Terrorism Events, funded by the U.S. Department of Homeland Security, award number 2007-ST-061-000001), the Center for Research on Environmental Decisions (CRED; NSF Cooperative Agreement SES-0345840 to Columbia University), Wharton Risk Management and Decision Processes Center, and Sandia National Laboratories. Sandia National Laboratories is a multi-program laboratory managed and operated by Sandia Corporation, a wholly owned subsidiary of Lockheed Martin Corporation, for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-AC04-94AL85000.

Author information

Authors and Affiliations

  1. Vanderbilt University, Nashville, TN, USA

    Yevgeniy Vorobeychik

  2. University of Pennsylvania, Philadelphia, PA, USA

    Steven Kimbrough & Howard Kunreuther

Authors
  1. Yevgeniy Vorobeychik
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Steven Kimbrough
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Howard Kunreuther
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yevgeniy Vorobeychik.

Appendix

Appendix

1.1 Description of Strategies

1.1.1 Full Feedback

Our consideration set of strategies in the full information context is

  1. 1.

    Prob(I)=0.7: play Invest with probability 0.7 (approximately the probability of Invest in early rounds of the human subject experiments)

  2. 2.

    Prob(I)=0.2: play Invest with probability 0.2 (approximately the probability of Invest in later rounds of the human subject experiments)

  3. 3.

    AlwaysInvest: Invest no matter what the opponent does

  4. 4.

    NeverInvest: Don’t Invest no matter what the opponent does

  5. 5.

    TFT: classic Tit-for-Tat strategy

  6. 6.

    InvestAfterLoss: Invest after experiencing a loss

  7. 7.

    InvestNAfterLoss: A player using InvestNAfterLoss does not invest on the first round, and continues to not invest, except for the N rounds immediately following a loss, whether direct or indirect. N is set to 3 for these experiments.

  8. 8.

    DontInvestAfterLoss: Don’t Invest after experiencing a loss

  9. 9.

    1TitFor2Tats: same as Tit-for-Tat except wait until the counterpart plays Don’t Invest for two rounds in a row before responding with Don’t Invest

  10. 10.

    2TitsFor1Tat: same as Tit-for-Tat except respond with two consecutive rounds of Don’t Invest to any Don’t Invest decision by the counterpart

  11. 11.

    FictiousPlay: plays a best response to the observed (empirical) mixed strategy of the counterpart

1.1.2 Partial Feedback

The set of policies used in partial feedback games is

  1. 1.

    Prob(I)=0.7: same as above

  2. 2.

    Prob(I)=0.2: same as above

  3. 3.

    AlwaysInvest: same as above

  4. 4.

    NeverInvest: same as above

  5. 5.

    InvestAfterLoss: same as above

  6. 6.

    InvestNAfterLoss: same as above

  7. 7.

    DontInvestAfterLoss: same as above

  8. 8.

    TitForTatPlusLossInvest: partial feedback analog of Tit-for-Tat, where a player responds only when the Don’t Invest decision by the opponent is inferred (i.e., when he experiences the indirect loss); in addition, Invest after experiencing a loss

  9. 9.

    TitForTatPlusLossNotInvest: partial feedback analog of Tit-for-Tat, where a player responds only when the Don’t Invest decision by the opponent is inferred (i.e., when he experiences the indirect loss); in addition, Don’t Invest after experiencing a loss

  10. 10.

    TitForTatPlusSticky: Under full feedback a player knows whether the counterpart has invested in security during the previous rounds of play. In this strategy, the player plays a tempered form of Tit-for-Tat. The player cooperates until the the Don’t Invest decision by the opponent is inferred (i.e., when he experiences the indirect loss), then defects and continues to defect until the counterpart has cooperated N = 3 times in a row.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Vorobeychik, Y., Kimbrough, S. & Kunreuther, H. A Framework for Computational Strategic Analysis: Applications to Iterated Interdependent Security Games. Comput Econ 45, 469–500 (2015). https://doi.org/10.1007/s10614-014-9431-1

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10614-014-9431-1

Keywords

Search

Navigation