Evolutionary dynamics of finite populations in games with polymorphic fitness equilibria
Introduction
The standard replicator dynamics (e.g., Taylor and Jonker, 1978, Hofbauer and Sigmund, 1998) are deterministic processes that operate on infinite populations. Here we examine two stochastic replicator dynamics that operate on small, well-mixed finite populations of fixed size N. These two replicator dynamics can be described as frequency-dependent Wright–Fisher (Wright, 1931, Fisher, 1930) and Moran (Moran, 1958) processes, respectively, operating on haploid populations. A population consists of pure-strategist agents; these agents play a symmetric variable-sum game. We are particularly interested in games, such as hawk–dove (HD) (Maynard Smith, 1982), that have a polymorphic fitness equilibrium (PFE).
In the games we study, when the population state is away from the PFE, then the under-represented strategy (relative to the strategy proportions at the PFE) is always favored by selection. Thus, selection always acts to move the population state towards the PFE. Given standard replicator dynamics operating deterministically on an infinite population, this action of selection causes the PFE to be a point attractor (Hofbauer and Sigmund, 1998). Given a stochastic finite-population system, however, the role of the PFE in the system's behavior is less clear. Indeed, if our system lacks mutation, then we know that sampling error will ultimately cause the population to enter one of the two monomorphic absorbing states; but, the pre-absorption transient can be very long-lived—how does the PFE shape the dynamics of the population before absorption occurs? If, instead, the system includes mutation, then it will have a unique steady-state distribution over the possible population states; does the PFE correspond to the expected population state at the system's steady state?
In an empirical investigation of finite-population dynamics using agent-based computer simulation, Fogel et al. (1997) observe that the mean population state obtained under a stochastic replication process diverges from the PFE with statistical significance. We provide theoretical explication of this observation. We show that deviation away from the PFE occurs when the selection pressures that surround the PFE are asymmetric. Game payoffs determine the magnitude and shape of this asymmetry, but the amount of asymmetry to which the system is actually exposed is determined by the population's size; smaller populations are more exposed to selection-pressure asymmetry and so diverge more from the PFE.
Further, we prove with Markov-chain analysis that the finite-population process generates a distribution over population states that equilibrates asymmetries in selection pressure; the mean of the distribution is generally not the PFE. More simply put, the finite populations we study equilibrate selection pressure, not fitness.
This article is organized as follows. Section 2 reviews related work. Section 3 details the finite-population models we examine; this section specifies the class of games that we consider, discusses the calculation of the PFE, and describes the replicator dynamics that we analyze. Section 4 presents four example games that we examine in detail. Section 5 gives empirical results on these games which indicate that finite-population behavior generally does not correspond to the PFE. Section 6 proposes the hypothesis that asymmetry in selection pressure causes actual behavior to diverge from the PFE, and Section 7 formalizes this intuition. 8 Discussion, 9 Conclusion provide further discussion and concluding remarks. Appendix A details our agent-based simulation methods. Appendix B describes how we construct our Markov models, and Appendix C gives our central proof. Appendix D discusses the special case of very small populations in the absence of mutation. Appendix E contrasts our equation to compute fitness equilibrium in a finite population with the equation given by Schaffer (1988) to compute an evolutionarily stable strategy (ESS) in a finite population.
Section snippets
Related work
Our focus on fixed-size populations of pure-strategists and PFE stands in contrast to much other research in finite-population dynamics. For example, in their studies of ESS, Schaffer (1988), Vickery, 1987, Vickery, 1988, and Maynard Smith (1988) require agents to use mixed strategies. Schaffer (1988) points out that a population of N pure-strategists cannot represent an arbitrary distribution over a game's pure strategies. This is certainly true for a static population; but, when acted upon
symmetric games
A generic payoff matrix for a symmetric two-player game of two pure strategies X and Y is given by By convention, the payoffs are for the row player.
There exist exactly two payoff structures for such a game that create a PFE between X- and Y-strategists. Case 1 has and . In games with this payoff structure, selection pressure always points away from the PFE, making it dynamically unstable. Case 2 has and . As we discuss below, games with this second payoff structure
Example games
We will examine the following four symmetric games (we use for all four games): Given a finite population with self-play (or an infinite population), all four games have their PFE at . Given a finite population without self-play, we know from (4) that, as N increases, will approach from above for and from below for ; games and have fitness equilibrium at regardless of population
Agent-based simulations and Markov-chain models
We now examine how well the PFE curves in Fig. 1 predict the behavior of stochastic agent-based simulations and Markov-chain models. We investigate Wright–Fisher and Moran dynamics, with and without agent self-play. We choose population sizes N such that is a whole number of agents. For all experiments, we initialize the population to be at the PFE.
We use 19 different population sizes in the range ; when played without self-play, game requires an additional agent to obtain
Selection-pressure asymmetry
We now present our hypothesis that asymmetry in selection pressure can explain the empirical results detailed above. The intuitions we develop here will be formalized in Section 7.
Let be the expected change in population state when the replicator function acts upon a population in state p, that is,When selection favors X-strategists, the expected change in population state is positive and ; similarly, when selection favors Y-strategists, . Thus, the sign of
Selection equilibrium
In this section, we make our intuitions about selection-pressure asymmetry concrete. Fig. 7 shows for game under the Wright–Fisher process without self-play (7) acting on a population of size . Here, fitness equilibrium is achieved at (15 X-strategists and 9 Y-strategists). The dashed curve indicates the binomial distribution produced by the replicator process when the population state is ; the expected value of this distribution is also and is indicated by the
Main findings
In Section 5, we demonstrate that different games can induce different behaviors in a finite population despite sharing the same infinite- or finite-population fitness equilibrium. Three of the four games we study cause the mean population state to diverge with statistical significance from the finite-population fitness equilibrium ; depending upon the game, the mean population state is either too high or too low. With the fourth game, does not diverge appreciably from . These
Conclusion
This article considers simple variable-sum games that have PFE attractors under standard replicator dynamics when played by an infinite population of pure-strategists. When these games are played by finite, fixed-size populations of pure-strategists, we show that the expected population state will likely diverge from the PFE; data for Wright–Fisher and frequency-dependent Moran replication processes are given. We then show that this deviation occurs when the selection pressures that surround
Acknowledgments
The authors thank Bruce Boghosian, David Fogel, David Haig, Lorens Imhof, Martin Nowak, Shivakumar Viswanathan, John Wakeley, Daniel Weinreich, L. Darrell Whitley, and the anonymous reviewers for their helpful feedback on this work. This work was supported in part by the US Department of Energy (DOE) under Grant DE-FG02-01ER45901.
References (29)
- et al.
On the instability of evolutionary stable states
BioSystems
(1997) - et al.
On the instability of evolutionary stable strategies in small populations
Ecol. Modelling
(1998) Long-term stability from fixation probabilities in finite populations: new perspectives for ESS theory
Theor. Popul. Biol.
(2005)Can a mixed strategy be stable in a finite population?
J. Theor. Biol.
(1988)Evolutionary equilibrium strategies
J. Theor. Biol.
(1979)Evolutionarily stable strategies for a finite population and a variable contest size
J. Theor. Biol.
(1988)- et al.
Evolutionary game dynamics in finite populations
Bull. Math. Biol.
(2004) - et al.
Evolutionarily stable strategies and game dynamics
Math. Biosci.
(1978) How to cheat against a simple mixed strategy ESS
J. Theor. Biol.
(1987)Reply to Maynard Smith
J. Theor. Biol.
(1988)
The many-demes limit for selection and drift in a subdivided population
Theor. Popul. Biol.
Generalized urn models of evolutionary processes
Ann. Appl. Probab.
On the evolution of behavioral heterogeneity in individuals and populations
Biol. Philos.
Mathematical Population Genetics
Cited by (18)
The impact of truncation selection and diffusion on cooperation in spatial games
2019, Journal of Theoretical BiologyCitation Excerpt :Note that reproduction is not dependent upon the degree to which a replicator is above the threshold for survival. Theoretical models of truncation selection show significant differences from the replicator equation such as the presence of chaos and cooperation where neither are present in the replicator equation employing identical games (Ficici and Pollack, 2007; Fogel et al., 1998; Fogel and Fogel, 2011; Morsky and Bauch, 2016). There are several examples from biology where truncation selection may be appropriate.
Stochastic dynamics in the fitness-based process which can be on behalf of the standard Moran, local and Wright-Fisher processes
2019, Journal of Theoretical BiologyTruncation selection and payoff distributions applied to the replicator equation
2016, Journal of Theoretical BiologyCitation Excerpt :The difference between these results and the ESS are due to selection at the extreme lower ends of the payoff distributions of each replicator (caused by stochastic elements of the agent-based models). Thus, the discrepancy is due to asymmetric selection pressures on either side of the ESS (Ficici et al., 2005; Ficici and Pollack, 2007). The primary objective of our paper is to explore how relaxing simplifying assumptions of the classic replicator equation — namely, relaxing mean payoffs and proportional selection in favour of payoff distributions and truncation selection, respectively — influence replicator dynamics and the corresponding evolutionary stable states (ESSes).
Multigame effect in finite populations induces strategy linkage between two games
2014, Journal of Theoretical BiologyCitation Excerpt :Repeating the process N-times is regarded as the unit time. These models have enabled many analyses of evolutionary processes and have provided considerable insight into the stochastic effects in evolutionary game dynamics (Nowak et al., 2004; Taylor et al., 2004; Fudenberg et al., 2003; Traulsen et al., 2005; Ficici and Pollack, 2007; Ohtsuki et al., 2007). However, most of the previous studies have only focused on a single game with a maximum of two to three strategies.
Emergence of cooperative linkages by random intensity of selection on a network
2011, BioSystemsCitation Excerpt :For example, Fogel et al. (1997) constructed an agent-based evolutionary game model, in which finite populations and random payoffs were allowed, and showed that the results were deviated from an evolutionary stable strategy. Ficici and Pollack (2007) provided theoretical implications of such phenomena. Fudenberg et al. (2006) introduced frequency dependent selection and mutation based on the Moran process in finite populations and showed similar results.