Spatial Evolutionary Games with small selection coefficients

Here we will use results of Cox, Durrett, and Perkins for voter model perturbations to study spatial evolutionary games on $Z^d$, $d\ge 3$ when the interaction kernel is finite range, symmetric, and has covariance matrix $\sigma^2I$. The games we consider have payoff matrices of the form ${\bf 1} + wG$ where ${\bf 1}$ is matrix of all 1's and $w$ is small and positive. Since our population size $N=\infty$, we call our selection small rather than weak which usually means $w =O(1/N)$. The key to studying these games is the fact that when the dynamics are suitably rescaled in space and time they convergence to solutions of a reaction diffusion equation (RDE). Inspired by work of Ohtsuki and Nowak and Tarnita et al we show that the reaction term is the replicator equation for a modified game matrix and the modifications of the game matrix depend on the interaction kernel only through the values of two or three simple probabilities for an associated coalescing random walk. Two strategy games lead to an RDE with a cubic nonlinearity, so we can describe the phase diagram completely. Three strategy games lead to a pair of coupled RDE, but using an idea from our earlier work, we are able to show that if there is a repelling function for the replicator equation for the modified game, then there is coexistence in the spatial game when selection is small. This enables us to prove coexistence in the spatial model in a wide variety of examples where the replicator equation of the modified game has an attracting equilibrium with all components positive. Using this result we are able to analyze the behavior of four evolutionary games that have recently been used in cancer modeling.


Introduction
Game theory was invented by John von Neumann and Oscar Morgenstern [1] to study strategic and economic decisions of humans. Maynard Smith and Price [2], introduced the concept into ecology in order to explain why conflicts over territory between male animals of the same species are usually of the "limited war" type and do not cause serious damage. As formulated in John Maynard Smith's classic book, [3], this is the Example 1.1. Hawks-Doves game. When a confrontation occurs, Hawks escalate and continue until injured or until an opponent retreats, while Doves make a display of force but retreat at once if the opponent escalates. The payoff matrix is Here V is the value of the resource, which two doves split, and C is the cost of competition.
There are two conventions in the literature for the interpretation of the payoff matrix. Here we are using the one in which the first row gives the payoffs for the Hawk strategy based on the opponent's strategy choice.
If we suppose that C > V then a frequency p = V /C of Hawks in the population represents an equilibrium, since in this case each strategy has the same payoff. The equilibrium is stable. If the frequency of the Hawk strategy rises to p > V /C then the Hawk strategy has a worse payoff than the Dove strategy so its frequency will decrease. Axelrod and Hamilton [4], see also [5], studied the evolution of cooperation by investigating Example 1.2. Prisoner's dilemma. Rather than formulate this in terms of prisoners deciding whether to confess or not, we consider the following game between cooperators C and defectors D. Here c is the cost that cooperators pay to provide a benefit b to the other player.
If b > c > 0 then the defection dominates cooperation, and, as we will see, altruistic cooperators are destined to die out in a homogeneously mixing population. This is unfortunate since the (D, D) payoff is worse than (C, C) payoff. Nowak and May [11,12] found an escape from this dilemma by showing that spatial structure allowed persistence of cooperators. Huberman and Glance [13] suggested that this was an artifact of the synchronous deterministic update rule, but later work [14,15] showed that the same phenomena occurred for asynchronous stochastic updates.
There are many examples of 2 × 2 games with names like Harmony, the Snowdrift Game, and the Battle of the Sexes, see e.g., [21], but as we will see in Section 6 there are really only 3 (or 4) examples. To complete the set of possibilities, we now consider Example 1.3. Stag Hunt. The story of this game was briefly described by Rousseau in his 1755 book A Discourse on Inequality. If two hunters cooperate to hunt stag (an adult male deer) then they will bring home a lot of food, but there is practically no chance of bagging a stag by oneself. If both hunters go after rabbit they split what they kill. An example of a game of this type is: In this case if the two strategies have frequency p = 1/2 in the population, then the two strategies have equal payoffs. If the frequency of the stag strategy rises to p > 1/2 then it has the better payoff and will continue to increase so this is an unstable equilibrium.
In the forty years since the pioneering work of Maynard Smith and Price [2] evolutionary game theory has been used to study many biological problems. For surveys see [6]- [10]. This is natural because evolutionary game theory provides a general framework for the study of frequency dependent selection. In the references we have listed representative samples of work of this type, [16]- [37]. There are literally hundreds of references, so we have restricted our attention to those that are the most relevant to our investigations.
In recent years, evolutionary game theory has been applied to study cancer. This provides an important motivation for our work, so we will now consider four examples beginning with one first studied by Tomlinson [48]. Example 1.4. Three species chemical competition. In this system, there are cells of three types. 1. Ones that Produce a toxic substance.
2. Others that are Resistant to the toxin, but do not produce it. 3. Wild type cells that are Sensitive to the toxin but do not produce it.
Based on the verbal description, Tomlinson wrote down the following game matrix.
Here, and in what follows it is sometimes convenient to number the strategies 1 = P , 2 = R and 3 = S. For example, this makes it easier to say that G ij is the payoff to a player who plays strategy i against an opponent playing strategy j. Taking the rows in reverse order, z is the baseline fitness while f is the cost to a sensitive cell due to the presence of the toxin. The cost of resistance to the toxin is h. In top row e is the cost of producing the toxin, and g is advantage to a producer when it interacts with a sensitive cell. It is interesting to note that in the same year [48] was published, Durrett and Levin [61] used a spatial model to model the competition two strains of E. coli, whose behaviors correspond to strategies P , R, and S above. In their model, there are also empty cells (denoted by 0). Thinking of a petri dish, their system takes place on the two dimensional lattice with the following dynamics.
• Individuals of type i > 0 give birth at rate β i with their offspring sent to a site chosen at random from the four nearest neighbors of x. If the site is occupied nothing happens.
• Each species dies at rate δ i due to natural causes, while type 3's die at an additional rate γ times the fraction of neighbors of type 1 due to the effect of colicin.
Example 1.5. Glycolytic phenotype. It has been known for some time that cancer cells commonly switch to glycolysis for energy production. Glycolysis is less efficient than the citrate cycle in terms of energy, but allows cell survival in hypoxic environments. In addition, cells with glycolytic metabolism can change the pH of the local microenvironment to their advantage. The prevalence of glycolytic cells in invasive tumor suggests that their presence could benefit the emergence of invasive phenotypes. To investigate this using evolutionary game theory, Basanta et al [45] considered a three strategy game in which cells are initially characterized as having autonomous growth (AG), but could switch to glycolysis for energy production (GLY ), or become increasing motile and invasive (IN V ). The payoff matrix for this game, which is the transpose of the one in their Table 1, is: Here c is the cost of motility, k is the cost to switch to glycolysis, and n is the detriment for nonglycolytic cell in glycolytic environment, which is equal to the bonus for a glycolytic cell. Recently this system has been extended to a four player game by adding an invasiveglycolytic strategy, see [43].
Example 1.6. Tumor-Stroma Interactions. Tumors are made up of a mixed population of different types of cells that include normal structures as well as ones associated with malignancy, and there are multiple interactions between the malignant cells and the local microenvironment. These intercellular interactions effect tumor progression. In prostate cancer it has been observed that one can have three different outcomes: (i) The tumor remains well differentiated and relatively benign. In this case the local stromal cells (also called connective tissue) may serve to restrain the growth of the cancer.
(ii) Early in its genesis the tumor acquires a highly malignant phenotype, growing rapidly and displacing the stromal population (often called small cell prostate cancer).
(iii) The tumor co-opts the local stroma to aid in its growth.
To understand the origin of these behaviors, Basanta et al [44] formulated a game with three types of players S = stromal cells, I = cells that have become independent of the microenvironment, and D = cells that remain dependent on the microenvironmnet. The payoff matrix is: Again this is the transpose of their matrix. Here γ is the cost of being environmentally independent, β cost of extracting resources from the micro-environment, α is the benefit derived from cooperation between S and D, and ρ benefit to D from paracrine growth factors produced by I. For more on the biology, see Dingli et al. [46].
A number of other examples have been studied. Tomlinson and Bodmer [49] studied games motivated by angiogenesis and apototsis. See Basanta and Deutsch [40] for a survey of this and other early work. Swierniak and Krzeslak's survey [47] contains the four examples we have covered there as well as a number of others.

Overview
In a homogeneously mixing population, x i = the frequency of players using strategy i follows the replicator equation is the fitness of strategy i andF = i x i F i is the average fitness. Twenty years ago, Durrett and Levin [16] studied evolutionary games and formulated rules for predicting the behavior of spatial models from the properties of the mean-field differential equations obtained by supposing that adjacent sites are independent. See [59] for an overview of this approach. Our main motivation for revisiting this question is that the recent work of Cox, Durrett, and Perkins [56] allows us to turn the heuristic principles of [16] into rigorous results for evolutionary games with matrices of the formḠ = 1 + wG, where w > 0 is small, and 1 is a matrix that consists of all 1's.
We will study these games on Z d where d ≥ 3 and the interactions between an individual and its neighbors are given by an irreducible probability kernel p(x) on Z d that is finite range, symmetric p(x) = p(−x), and has covariance matrix σ 2 I. To describe the dynamics we let ξ t (x) be the strategy used by the individual at x at time t and be the fitness of x at time t. In Birth-Death dynamics, site x gives birth at rate ψ t (x) and sends its offspring to replace the individual at y with probability p(y − x). In Death-Birth dynamics, the individual at x dies at rate 1, and replaces itself by a copy of the one at y with probability proportional to p(y − x)ψ t (y).
When w = 0 either version of the dynamics reduces to the voter model, a system in which each site at rate 1 changes its state to that of a randomly chosen neighbor. As we will explain in Section 3, when w is small, our spatial evolutionary game is a voter model perturbation in the sense of Cox, Durrett, and Perkins [56]. Section 4 describes the duality between the voter model and coalescing random walk, which is the key to study of the voter model and its perturbations. More details are given in Section 10. Section 4 also introduces the coalescence probabilities that are the key to our analysis and states some identities between them that are proved in Section 11. The next step is to show that when w → 0 and space and time are rescaled appropriately then frequencies of strategies in the evolutionary game converge to a limiting PDE. Section 5 states the results, while Section 12 shows how these conclusions follow from results in [56]. While this is a "known result," there are two important new features here. The reaction term is identified as the replicator equation for a modified game, and it is shown that the modifications of the game matrix depend only on the effective number of neighbors κ (defined in (1) below) and two probabilities for the coalescing random walk with jump kernel p. The first fact allows us to make use of the theory that has been developed for replicator equations, see [7].
Two strategy games are studied in Section 6. A complete description of the phase diagram for Death-Birth and Birth-Death dynamics is possible because the limiting RDE's have cubic reaction terms f (u) with f (0) = f (1) = 0, so we can make use of work of [50,51,65,66]. As a corollary of this analysis, we are able to prove that Tarnita's formula for two strategy games with weak selection holds in our setting. They say that a strategy in a k strategy game is "favored by selection" if its frequency in equilibrium is > 1/k when w is small. Tarnita et al [35] showed that this holds for strategy 1 in a 2 by 2 game if and only if where σ is a constant that depends only on the dynamics. In our setting σ = 1 for Birth-Death dynamics while σ = (κ + 1)/(κ − 1) for Death-Birth dynamics where is the "effective number of neighbors." To explain the name note that if p is uniform on a symmetric set S of size m, i.e., x ∈ S implies −x ∈ S, and 0 = S then κ = m. In Section 7 we begin our study of three strategy games by analyzing the behavior of their replicator equations. This is done using the approach of [58]. The first step is to analyze the three 2 × 2 subgames. In 1, 2 subgame there are four possibilities: • strategy 1 dominates 2, 1 2; • strategy 2 dominates 1, 2 1; • there is a stable mixed strategy equilibrium which is the limit starting from any point on the interior of the edge; • there is an unstable mixed strategy which separates the sets of points that converge to the two endpoints.
In the case that there is a mixed strategy equilibrium, we also have to see if it can be invaded by the strategy not in the subgame, i.e., its frequency will increase when rare. Our method for analyzing the spatial game is to prove the existence of a repelling function (i.e., a convex Lyapunov function that blows up near the boundary and satisfies some mild technical conditions). Unfortunately, a repelling function cannot exist when there is an unstable fixed point on some edge, but this leaves a large number of possibilities. Section 8.1 introduces the notion of the repelling function and while Sections 8.2 and 8.3 prove that they exist for four classes of examples identified in Section 7 that have attracting interior fixed points.
Our method for constructing repelling functions is to construct for i = 1, 2, 3 a function h i that works near the side u i = 0, and then define where M is chosen large enough. This repelling function is easy to construct and allows us to prove coexistence of the three types, but it does not allow us to obtain very precise information about the frequencies of the three types in the equilibrium. An exception is the set of games considered in Section 8.4 that are almost constant sum. In that case we have a repelling function that is decreasing everywhere except at the interior fixed point ρ, so Theorem 1.4 of [56] allows us to conclude that when w is small any nontrivial stationary distribution for the spatial model has frequencies near ρ. We use this result in Section 8.5 to show that when there are k ≥ 3 strategies Tarnita's formula for the case of weak selection, does not hold in our setting. Their condition for a strategy to be favored by selection when selection is weak is linear in the coefficients, while the right condition in our setting for almost constant sum games is quadratic.
In Section 9, we turn our attention to the three strategy examples from the introduction, Examples 1.4-1.7. Using the results from the Sections 7 and 8, we are able to analyze these games in considerable detail. In particular, we can prove coexistence in the spatial game in these examples whenever there is an interior fixed point in replicator for the modified game that is attracting. While we are able to demonstrate coexistence in these four examples, there are also some large gaps in our knowledge.
The most important open problem is that we cannot handle the case in which the replicator equation is bistable, i.e., there are two locally attracting fixed points u * and v * . See Examples 7.2B and 7.3B. By analogy with the work of Durrett and Levin [16], we expect that the winner in the spatial game is dictated by the direction of movement of the traveling wave solution that tends to u * at −∞ and v * at ∞. However, we are not aware of results that prove the existence of traveling waves solutions.
A second open problem is prove "dying out results" which show that one strategy takes over the system (see Examples 7.1A and 7.3A) or that one strategy dies out leaving an equilibrium that consists of a mixture of the other two strategies (see Examples 7.2A, 7.3C and 7.3D). Given results in Section 8, it is sufficient to find suitable convex Lyapunov functions. However, in contrast to the repelling functions used to prove coexistence, these functions must be decreasing everywhere except at the point which is the limiting value of the replicator equation.

Voter model perturbations
The state of the system at time t is ξ t : Z d → S where S is a finite set of sates that are called strategies or opinions. ξ t (x) gives the state of the individual at x at time t. To formulate this class of models, let f i (x, ξ) = y p(y − x)1(ξ(y) = i) be the fraction of neighbors of x in state i. In the voter model, the rate at which the voter at x changes its opinion from i to j is The voter model perturbations that we consider have flip rates The perturbation functions h ε ij may be negative (and will be for games with negative entries) but in order for the analysis in [56] to work, there must be a law q of (Y 1 , . . . Y m ) ∈ (Z d ) m and functions g ε i,j ≥ 0, which converge to limits g i,j as ε → 0, so that for some γ < ∞, we have for ε ≤ ε 0 In words, we can make the perturbation positive by adding a positive multiple of the voter flip rates. This is needed so that [56] can use g ε i,j to define jump rates of a Markov process. As we will see, the condition is harmless in our examples, and all our calculations can be done using the original perturbation The next result is the key to the analysis of voter model perturbation. The result was proved in [56] for systems with two opinions but generalizes easily to more than two opinions. Theorem 3.1. If we rescale space to εZ d and speed up time by ε −2 then in d ≥ 3 converges to the solution of the system of partial differential equations: and the brackets are expected value with respect to the voter model stationary distribution ν u in which the densities are given by the vector u.
Intuitively, since on the fast time scale the voter model runs at rate ε −2 versus the perturbation at rate 1, the process is always close to the voter equilibrium for the current density vector u. Thus, we can compute the rate of change of u i by assuming the nearby sites are in that voter model equilibrium. The restriction to dimensions d ≥ 3 is due to the fact that the voter model does not have nontrivial stationary distributions in d ≤ 2. For readers not familiar with the voter model, we recall the relevant facts in Section 10 We will consider two versions of spatial evolutionary game dynamics.
Birth-Death dynamics. In this version of the model, a site x gives birth at a rate equal to its fitness y p(y − x)Ḡ(ξ(x), ξ(y)) and the offspring replaces a "randomly chosen neighbor of x." Here and in what follows, the phrase in quotes means z is chosen with probability p(z − x). If we let r i,j (0, ξ) be the rate at which the state of 0 flips from i to j, then setting w = ε 2 and using symmetry p(x) = p(−x) where f To see that this satisfies the technical condition (2) note that if γ = max i,j G − i,j where x − = max{−x, 0}, and we let g i,j (k, ) = γ + 1 (k=j) G k, when ξ(0) = i and 0 otherwise then where Y 1 is a randomly chosen neighbor of 0 and Y 2 is a randomly chosen neighbor of Y 1 .
Death-Birth Dynamics. In this case, each site dies at rate one and is replaced by a neighbor chosen with probability proportional to its fitness. Using the notation in (3) the rate at which ξ(0) = i jumps to state j is The new perturbation, which depends on ε, is It is not hard to see that it also satisfies the technical condition (2).
There are a number of other updates rules. In Fermi updating, a site x and a neighbor y are chosen at random. Then x adopts y's strategy with probability The main reason for interest in this rule is the Ising model like phase transition that occurs, for example in Prisoner's Dilemma games as β is increased, see [19,24,25].
In Imitate the best one adopts the strategy of the neighbor with the largest fitness. All of the matricesḠ = 1 + wG have the same dynamics, so this is not a voter model perturbation. In discrete time (i.e., with synchronous updates) the process is deterministic. See Evilsizor and Lanchier [38] for recent work on this version.

Voter model duality
In this section we set w = 0, soḠ = 1 and the system becomes the voter model. Let ξ t (x) be the state of the voter at x at time t. The key to the study of the voter model is that we can define for each x and t, random walks ζ x,t s , 0 ≤ s ≤ t that move independently until they hit and then coalesce to 1 so that Intuitively, the ζ x,t s are genealogies that trace the origin of the opinion at x at time t. See Section 10 for more details about this and other facts about the voter model we cite in this section.
Consider now the case of two opinions. A consequence of this duality relation is that if we let p(0|x) be the probability that two continuous time random walks with jump distribution p, one starting at the origin 0, and one starting at x never hit then To prove this, we recall that the stationary distribution ν u is the limit in distribution as t → ∞ of ξ u t , the voter model starting with sites that are independent and = 1 with probability u, and then observe that (7) implies Letting t → ∞ gives the desired identity.
To extend this reasoning to three sites, let p(0|x|y) be the probability that the three random walks never hit and let p(0|x, y) be the probability that the walks starting from x and y coalesce, but they do not hit the one starting at 0. Considering the possibilities that the walks starting from x and y may or may not coalesce: Let v 1 , v 2 , v 3 be independent and chosen according to the distribution p and let κ = 1/P (v 1 + v 2 = 0) be the "effective number of neighbors" defined in (1). The coalescence probabilities satisfy some remarkable identities that will be useful for simplifying formulas later on. Since the v i have the same distribution as steps in the random walk, simple arguments given in Section 11 show that It is easy to see that for any x, y, z coalescence probabilities must satisfy Combining this with the identities for two sites leads to From (10) and (11) it follows that where x, y, z is any ordering of 0, v 1 , v 1 +v 2 . Later, we will be interested in p 1 = p(0|v 1 |v 1 +v 2 ) and p 2 = p(0|v 1 , v 1 + v 2 ). In this case, (13) implies Similar reasoning to that used for (13) gives Later, we will be interested inp 1 = p(v 1 |v 2 |v 2 + v 3 ) andp 2 = p(v 1 |v 2 , v 2 + v 3 ). In this case, (15) implies that We will also need the following consequence of (15) and (10) Work of Tarnita et al. [35,36] has shown that when selection is weak (i.e., w 1/N where N is the population size) one can determine whether a strategy in an k-strategy game is favored by selection (i.e., has frequency > 1/k) by using an inequality that is linear in the entries of the game matrix that involves one (k = 2) or two (k ≥ 3) constants that depend on the spatial structure. Our analysis will show that on Z d , d ≥ 3, the only aspects of the spatial structure relevant for a complete analysis of the game with small selection are p(0|v 1 ) and p(0|v 1 |v 1 + v 2 ) for Birth-Death updating and κ, p(v 1 |v 2 ) and p(v 1 |v 2 |v 2 + v 3 ) for Death-Birth updating.
The coalescence probabilities p(0|v 1 ) = p(v 1 |v 2 ) are easily calculated since the difference between the positions of the two walkers is a random walk. Let S n be the discrete time random walk that has jumps according to p and let be its characteristic function (a.k.a Fourier transform). The inversion formula implies For more details see pages 200-201 in [60]. Since the number of visits to 0 has a geometric distribution with success probability p(0|v 1 ) it follows that In the three dimensional nearest neighbor case it is know that χ = 1.561386 . . . so we have To evaluate p(0|v 1 |v 1 +v 2 ) and p(v 1 |v 2 |v 2 +v 3 ) we have to turn to simulation. Simulations of Yuan Zhang suggest that p(0|v 1 |v 1 + v 2 ) ∈ [0.32, 0.33] and p(v 1 |v 2 |v 2 + v 3 ) ∈ [0.34, 0.35].

PDE limit
In a homogeneously mixing population the frequencies of the strategies in an evolutionary game follow the replicator equation, see e.g., Hofbauer and Sigmund's book [7]: Birth-Death dynamics. Let In this case the limiting PDE in Theorem 3.1 is Formula (13) implies that so it is enough to know the two probabilities on the right-hand side. If coalescence is impossible then p 1 = 1 and p 2 = 0 and the new equation reduces to the old one. There is a second more useful connection to the replicator equation. Let This matrix is skew symmetric A i,j = −A j,i so i,j u i A i,j u j = 0 and it follows that φ i B (u) is p 1 times the RHS of the replicator equation for the game matrix A + G. This observation is due to Ohtsuki and Nowak [28] who studied the limiting ODE that arises from the nonrigorous pair approximation. In their case, the perturbing matrix, see their (14), is To connect the two formulas note if space is a tree in which each site has κ neighbors then . Under the pair approximation, the coalescence of 0 and v 1 is assumed independent of the coalescence of v 1 and v 1 + v 2 , so Death-Birth Dynamics. Let Note that in comparison with p 1 and p 2 , 0 has been replaced by v 1 and then the other two v i have been renumbered. In this case the limiting PDE is The first two terms are the ones in (21). The similarity is not surprising since the numerators of the flip rates in (5) are the flip rates in (3). The third term comes from the denominator in (5). Formula (13) implies that so it is enough to know the two probabilities on the right-hand side. As in the Birth-Death case, if we let then φ D i (u) isp 1 times the RHS of the replicator equation forĀ + G. Again, Ohtsuki and Nowak [28] have a similar result for the ODE resulting from the pair approximation. In their case the perturbing matrix, see their (23), is This time the connection is not exact, since under the pair approximation

Two strategy games
We now consider the special case of a 2 × 2 games.
Let u denote the frequency of strategy 1. In a homogeneously mixing population, u evolves according to the replicator equation (20): Using results from the previous section gives the following.
Birth-Death dynamics. The limiting PDE is ∂u Death-Birth dynamics. The limiting PDE is ∂u

Analysis of 2 × 2 games
Suppose that the limiting PDE is ∂u/∂t = (1/2d)∆u + φ(u) where φ is a cubic with roots at 0 and 1. There are four possibilities To see this, we draw a picture. For convenience, we have drawn the cubic as a piecewise linear function.
= ∞} be the configurations with infinitely many 1's and infinitely many 0's. We say that coexistence occurs if there is a stationary distribution ν for the spatial model with ν(Ω 0 ) = 1.
Theorem 6.1. If ε < ε 0 (G), then: In case S 3 , 2's take over. In case S 4 , 1's take over. In case S 2 , 1's take over ifū < 1/2, and 2's take over ifū > 1/2. In case S 1 , coexistence occurs. Furthermore, if δ > 0 and ε < ε 0 (G, δ) then any stationary distribution with ν(Ω 0 ) = 1 has To begin to apply this result we note that if φ is the RHS of the replicator equation for the game matrix in (23) then writing i j for strategy i dominates strategy j the cases are: To check S 1 we draw a picture.
@ @ @ @ @ @ @ @ @ @ @ @ 0 1 u β δ γ α When the frequency of strategy 1 is u ≈ 0 then strategy 1 has fitness ≈ β and strategy 2 has fitness ≈ δ, so u will increase. The condition α < γ implies that when u ≈ 1 it will decrease and the fixed point is attracting. When both inequalities are reversed in S 2 , the fixed point exists but is unstable. Finally the second strategy dominates the first in S 3 , and the first strategy dominates the second in S 4 .

Phase diagram
At this point, we can analyze the spatial version of any two strategy game. In the literature on 2 × 2 games it is common to use the following notation for payoffs, which was introduced in the classic paper by Axelrod and Hamilton [4].

C D C R S D T P
Here T = temptation, S = sucker payoff, R = reward for cooperation, P = punishment for defection. If we assume, without loss of generality, that R > P then there are 12 possible orderings for the payoffs. However, from the viewpoint of Theorem 6.1, there are only four cases.
Hauert simulates spatial games with R = 1 and P = 0 for a large number of values of S and T . He considers three update rules: (a) switch to the strategy of the best neighbor, (b) pick a neighbor's strategy with probability proportional to the difference in scores, and (c) pick a neighbor's strategy with probability proportional to its fitness, or in our terms Death-Birth updating. He considers discrete and continuous time updates using the von Neumann neighborhood (four nearest neighbors) and the Moore neighborhood (which also includes the diagonally adjacent neighbors). The picture most relevant to our investigation here is the graph in the lower left corner of his Figure 5, reproduced here as Figure 1, which shows equilibrium frequencies of the two strategies in continuous time for update rule (c) on the von Neumann neighborhood. Similar pictures can be found in the work of Roca, Cuesta, and Sanchez [33,34].
Our situation is different from his, since the games we consider are small perturbations of the voter model game 1, but as we will see, the qualitative features of the phase diagrams are the same. Under either update the game matrix changes to Birth-Death updating. In this case We will now find the boundaries between the four cases using (26).
Repeating this calculation shows that β = δ when This leads to the four regions drawn in Figure 2. Note that the coexistence region is smaller than in the homogeneously mixing case.
In the coexistence region, the equilibrium is Plugging in the value of θ from (27) this becomes Note that in the coexistence region,ū is constant on lines through (S, T ) = (P, R).
In the lower left region where there is bistability, 1's win ifū < 1/2 or what is the same if strategy 1 is better than strategy 2 when u = 1/2, that is, Writing this as R + S > T + P , we see that the population converges to strategy 1 when it is "risk dominant", a term introduced by Harsanyi and Selten [6]. Note that bistablity in the replicator equation disappears in the spatial model, an observation that goes back to Durrett and Levin [16].
Death-Birth updating. The phase diagram is similar to that for Birth-Death updating but the regions are shifted over in space. Since the algebra in the derivation is messier, we state the result here and hide the details away in Section 13. If we let µ =p 2 /p 1 , ν = p(v 1 |v 2 )/κp 1 , and let λ = µ − ν, then the two lines α = γ and β = δ can be written as This leads to the four regions drawn in Figure 3. In the coexistence region, the equilibrium u is constant on lines through (S, T ) = (R * , P * ). In the lower left region where there is bistability, 1's win if Even though Hauert's games do not have weak selection, there are many similarities with Figure 1. The equilibrium frequencies are linear in the coexistence region, and in the lower left, the equilibrium state goes from all 1's to all 2's over a very small distance.

Tarnita's formula
Tarnita et al. [35] say that strategy C is favored over D in a structured population, and write C > D, if the frequency of C in equilibrium is > 1/2 in the gameḠ = 1 + wG when 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2  w is small. Assuming that (i) the transition probabilities are differentiable at w = 0, (ii) the update rule is symmetric for the two strategies, and (iii) in the game given by the matrix with β = 1, α = γ = δ = 0, strategy A is not disfavored, they argued that I. C > D is equivalent to σR + S > T + σP where σ is a constant that only depends on the spatial structure and update rule.
By using results for the phase diagram given above, we can show that Theorem 6.2. I holds for the Birth-Death updating with σ = 1 and for the Death-Birth updating with σ = (κ + 1)/(κ − 1).
Proof. For Birth-Death updating it follows from (32) that this is the correct condition in the bistable quadrant. By (31), in the coexistence quadrant, Cross-multiplying we see thatū > 1/2 when we have Thus in both quadrants the condition is R + S > T + P . The proof of the formula for Death-Birth updating is similar but requires more algebra, so again we hide the details away in Section 13. Since the derivation of the formula from the phase diagram in the Death-Birth case is messy, we also give a simple self-contained proof of Theorem 6.2 in this case.

Concrete examples
In this, we present calculations for concrete examples to complement the general conclusions from the phase diagram. Before we begin, recall that the original and modified games are for Death-Birth updating.
Example 6.1. Prisoner's Dilemma. As formulated in Example 1.2 Under either updating the matrix changes to In the Birth-Death case, Again Γ = 0 so the victor is determined by the sign of Identity (15) implies that 2p 2 +p 1 = p(v 1 |v 2 )(1 + 1/κ) so cooperators will persist if Since p(v 1 |v 2 ) > 0 the condition is just b/c > κ giving a proof of the result of Ohtsuki et al. [29], which has already appeared in Cox, Durrett, and Perkins [56].
Example 6.2. Nowak and May [11] considered the "weak" Prisoner's Dilemma game with payoff matrix: As you can see from Figure 3, if Death-Birth updating is used, these games will show a dramatic departure from the homogeneously mixing case. Nowak and May used "imitate the best dynamics" so the process was deterministic and there are only finitely many different evolutions. See Section 2.1 of [12] for locations of the transitions and pictures of the various cases. When 1.8 < b < 2, if the process starts with a single D in a sea of C's, and color the sites based on the values of (ξ n−1 (x), ξ n (x)) a kaleidoscope of Persian carpet style patterns results. As Huberman and Glance [13] have pointed out, these patterns disappear if asynchronous updating is used. However, work of Nowak, Bonhoeffer and May [14,15] showed that their conclusion that spatial structure enhanced cooperation remained true with stochastic updating or when games were played on random lattices. Example 6.3. The Harmony game has P < S and T < R. In this game strategy 1 dominates strategy 2, but in contrast to Prisoner's dilemma the payoff for the outcome (C, C) is the largest in the matrix. Licht [74] used this game to explain the proliferation of MOUs (memoranda of understanding) between securities agencies involved in international antifraud regulation. From Figures 2 and 3, we see that in the spatial model cooperators take over the system. Example 6.4. Snowdrift game. In this game, two motorists are stuck in their cars on opposite sides of a snowdrift. They can get out of their car and start shoveling (C) or do nothing (D). The payoff matrix is That is, if both players shovel then the work is cut in half, but if one player cooperates and the other defects then the C player gains the benefit of sleeping in his bed rather than in his car.
The story behind the game makes it sound frivolous, however, in a paper published in Nature [32], the snowdrift game has been used to study "facultative cheating in yeast." For yeast to grow on sucrose, a disaccharide, the sugar has to be hydrolyzed, but when a yeast cell does this, most of the resulting monosaccharide diffuses away. None the less, due to the fact that the hydrolyzing cell reaps some benefit, cooperators can invade a population of cheaters.
If b > c then the game has a mixed strategy equilibrium, which by (25) is Under either update rule the modified payoff becomes and using (25) again the equilibrium changes tō assuming that this stays in (0, 1). If thisū > 1, then 1 becomes an attracting fixed point; if u < 0, then 0 is attracting. If θ > 0 then spatial structure enhances cooperation. If we use Birth-Death updating: If we use Death-Birth updating: Hauert and Doebeli [23] have used simulation to show that spatial structure can inhibit the evolution of cooperation in the snowdrfit game. One of their examples has R = 1, S = 0.38, T = 1.62, and P = 0 in which case θ < 0 for both update rules. At the end of their article they conclude that "space should benefit cooperation for low cost to benefit ratios," which is consistent with our calculation. For more discussion of the contrasting effects on cooperation in Prisoner's Dilemma and Snowdrift games, see the review by Doebeli and Hauert [26].
Example 6.5. Hawk-dove game. As formulated in Example 1.1 the payoff matrix is Killingback and Doebli [17] studied the spatial version of the game and set V = 2, β = C/2 to arrive at the payoff matrix Hawk Dove Hawk 1 − β 2 Dove 0 1 We will assume β > 1. In this case, (25) implies that the equilibrium mixed strategy plays Hawk with probabilityū = 1/β. To put this game into our phase diagram we need to label the Dove strategy as Cooperate and the Hawk strategy as Defect: Under either of our update rules the payoff matrix changes to for Death-Birth updating. In one of Killingback and Doebli's favorite cases, β = 2.2, both of these terms are negative, so by (77) the frequency of Hawks in equilibrium is reduced, in agreement with their simulations. While the conclusions may be similar, the updates used in the models are very different. In [17], a discrete time dynamic ("synchronous updating") was used in which the state of a cell at time t + 1 is that of the eight Moore neighbors with the best payoff. As in the pioneering work of Nowak and May [11] this makes the system deterministic and there are only finitely many different behaviors as β is varied with changes at β passes through 9/7, 5/3, 2, and 7/3. Figure 1 in [17] shows spatial chaos, i.e., the dynamics show a sensitive dependence on initial conditions. For more on this see [18]. In continuous time with small selection our results predict that as long as the mixed strategy equilibrium is preserved in the perturbed game we will get coexistence of Hawks and Doves in an equilibrium with density of Hawks and Doves close to that predicted by the perturbed game matrix.
Example 6.6. The Battle of the Sexes is another game that leads to an attracting fixed point in the replicator equation. The story is that the man wants to go to a sporting event while the woman wants to go to the opera. In an age before cell phones they make their choices without communicating with each other. If C is the choice to go to the other person's favorite and D is go to your own then the game matrix might be In Hauert's scheme this case is defined by T > S > R > P in contrast to the inequalities T > R > S > P for the snowdrift game, and Hawks-Doves.
Despite the difference in the inequalities the results are very similar. Again in either case the modified payoffs in this particular example are Under Birth-Death updating θ = 0 since R + S − T − P = 0, while for Death-Birth updating Since the equilibrium changes toū = S − P + θ S − P + T − R spatial structure inhibits cooperation. In Hauert's scheme this case is defined by the inequalities R > T > P > S. Since R > T and P > S, we are in the bistable situation. Returning to the general situation in either case the modified payoffs in this particular example are If R + S > T + P then θ BD > 0 while for Death-Birth updating So the 1's win out. If R + S < T + P then θ BD < 0 so the 2's win, but θ DB may be positive or negative. Under Birth-Death updating the winner is always the risk dominant strategy, and under Death-Birth updating it often is. This is consistent with results in the economics literature. See Kandori, Mailath, and Rob [73], Ellison [64] and Blume [52]. Blume uses a spatially explicit model with a log-linear strategy revision, which turns the system into an Ising model.

ODEs for the three strategy games
In this section we will prove results for the replicator equation in order to prepare for analyzing examples in Section 9. For simplicity, we will assume the game is written with zeros on the diagonal. For the replicator equation and for the spatial model with Birth-Death updating, this entails no loss of generality.
Here we have numbered the entries by the strategy that is left out in the corresponding 2 × 2 game. It would be simpler to put the α's above the diagonal and the β's below but (i) this scheme simplifies the statement of Theorem 7.5 and (ii) this pattern of α's and β's is unchanged by a cyclic permutation of the strategies It is not hard to check that in general if the game G in (33) has an interior fixed point it must be: where D is the sum of the three numerators. See Section 14 for details.

Special properties of replicator equations
To study replicator equations it is useful to know some of the existing theory. To keep our treatment self-contained we will prove many of the results we need.
This comes from Exercise 7.1.3 in [7]. We will use this in the proof of Theorem 7.5 to transform the game so that α i + β i constant. Another common application is to choose where the ρ i are the coordinates of the interior equilibrium in order to make the equilibrium uniform.
Proof. To prove this note that The second terms on the two lines cancel leaving us with The factor k u k m k is a time change, so we have proved the desired result.

Reduction to Lotka-Volterra systems
We begin with the "quotient rule" d dt Proof. Using the quotient rule of calculus, d dt which proves the desired result.
Proof. By subtracting G n,j from the jth column, we can suppose without loss of generality that the last row is 0. By the quotient rule (35) and the fact that The factor u n corresponds to a time change so the desired result follows.
Theorem 7.2 allows us to reduce the study of the replicator equation for three strategy games to the study of two species Lotka-Volterra equation: If we suppose that bf − ce = 0 then the right-hand side is 0 when Suppose for the moment that x * , y * > 0. If we have an ODE with a fixed point at (x * , y * ) then linearizing around the fixed point gives and setting X = x − x * and Y = y − y * gives In the case of Lotka-Volterra systems this is: In ecological competitions it is natural to assume that b < 0 and f < 0 so that the population does not explode when the other species is absent. In this case the trace of the linearization, which is the sum of the eigenvalues is x * b + y * f < 0. The determinant, which is the product of the eigenvalues is x * y * (bf − ce). so the fixed point will be locally attracting if bf − ec > 0. The next result was proved by Goh [67] in 1976, but as Harrison [71] explains, it has been proved many times, and was known to Volterra in 1931.
is a Lyapunov function, i.e., it is decreasing along solutions, and hence x * , y * is an attracting fixed point.

Proof. A little calculus gives
since a = −bx * − cy * and d = −ex * − f y * . If c and e have different signs then we can choose A/B = −e/c, so that Ac + Be = 0 and Ab and Bf are negative so V is a Lyapunov function.
To deal with the case ce > 0, we write (38) as We would like to arrange things so that the symmetric matrix Q is negative definite, i.e., both eigenvalues are negative. The trace of Q, which is the sum of the eigenvalues is Ab + Bf < 0. The determinant, which is the product of the eigenvalues, is To conclude both eigenvalues are negative, we want to show that the determinant is positive. We have assumed bf − ce > 0. To deal with the second term we choose A, B so that Ac − Be = 0. Finally, we have to consider the situation ce = 0. We can suppose without loss of generality that e = 0. In this case the determinant is A[4Bbf − Ac 2 ] with bf > 0 so if A/B is small this is positive.
It follows from Theorem 7.2 that is a Lyapunov function for the replicator equation. Unfortunately for our purposes, it is not convex near the boundary.

Classifying three strategy games
To investigate the asymptotic behavior of the replicator equation in three strategy games, we will use a technique we learned from mathematical ecologists [58]. We begin by investigating the three two-strategy sub-games. When the game is written with 0's on the diagonal it is easy to determine the behavior of the 1 vs. 2 subgame. The mixed strategy equilibrium, if it exists, is given by In equilibrium both types have fitness α 3 β 3 /(α 3 + β 3 ). There are four cases: • α 3 , β 3 > 0, attracting (stable) mixed strategy equilibrium.
Our main technique for proving coexistence in the spatial game is show the existence of a repelling function for replicator equation associated with the (modified) game. (A repelling function is a convex function that blows up on the boundary and satisfies a couple of technical conditions, see Section 8.1.) A repelling function will not exist if on one of the edges there is an unstable mixed strategy equilibrium, so we will ignore games that have them. As the reader will soon see, if an edge has a stable mixed strategy equilibrium, we will also need to determine if the third strategy can invade: i.e., will increase when rare. Note that here the word stable refers only to the behavior of the replicator equation on the edge. Combining the possibilities for the three edges, we have 4 3 = 64 cases, but the number of cases can be reduced to about a dozen by using symmetry. Bomze [53] drew 46 phase portraits to illustrate the possible behaviors of the replicator equation. A follow up paper twelve years later, [54], corrected the examples associated with five of the cases and add two more pictures. Bomze considered situations in which an entire edge consists of fixed points or there was a line of fixed points connecting two edge equilibria. Here, we will restrict our attention to "generic" cases that do not have these properties. We approach the enumeration of possibilities by considering the number of stable edge fixed points, which can be 3, 2, 1, or 0, and then the number of these fixed points that can be invaded.

Three edge fixed points
Here, and in the next three subsections, we begin with the example with an attracting interior equilibrium. In this and the next two subsections this is the case in which all edge fixed points can be invaded.
Each strategy has fitness α 2 β 2 /(α 2 + β 2 ) at this point, so 2's can invade this equilibrium if which we can rewrite as The invadability condition (43) implies that the numerator of ρ 2 given in (34) is positive. The 1 vs. 2 subgame has a mixed strategy equilibrium at Each strategy has fitness α 3 β 3 /(α 3 + β 3 ) at this point, so 3's can invade this equilibrium if which we can rewrite as The invadability condition (41) implies that the numerator of ρ 3 in (34) is positive. Combining the last three results we see that there is an interior equilibrium. This game transforms into the Lotka-Volterra equation: Clearly, b, f < 0. To check the other condition we note that by (45) bf which is the condition for ρ 3 > 0.
For future reference note that if bf − ce > 0 and there is interior fixed point then (37) implies dc − f a > 0 and ed − bd > 0 Plugging in the coefficients, these conditions become i.e., the numerators of ρ 1 and ρ 2 are positive.
Example 7.1A. If only TWO of the edge fixed points are invadable, then the numerator of one of the ρ i will be negative while the other two are positive, so there is no interior equilibrium. Bomze #35 shows that the system will converge to the noninvadable fixed point. We can prove this by transforming to a Lotka-Volterra equation. Since that argument will also cover Example 7.2A, we will wait until then to give the details. Proof. Suppose that 1 cannot invade the 2, 3 equilibrium and that 2 cannot invade the 1, 3 equilibrium. By making a projective transformation, see Theorem 7.1, we can suppose the α i = 1. The failure of the two invadabilities implies that the numerators of ρ 1 and ρ 2 are negative: Since the β i > 0 the second equation implies β 2 > 1 and hence the first inequality is impossible.

Two edge fixed points
Example 7.2. Two stable invadable edge fixed points. (Bomze #9) If we suppose α 2 , β 2 > 0, α 1 , β 1 > 0 and 1 2 (α 3 > 0 > β 3 ) then there are two stable edge equilibria. In words all entries are positive except for β 3 . Again, we assume that the two edge equilibria can be invaded. 1 By the reasoning in Example 7.1, if the 2's can invade the 1, 3 equilibrium then the numerator of ρ 2 given in (34) is positive., and if 1's can invade the 2, 3 equilibrium then the numerator of ρ 1 in (34) is positive. In the numerator of ρ 3 , α 3 α 2 > 0 and −α 3 β 3 > 0, but β 3 β 1 < 0. It does not seem to be possible to prove algebraically that In Section 8.3, we will show that in this class of examples there is a repelling function. This will imply that trajectories cannot reach the boundary, so the existence of a fixed point follows from Theorem 7.2 and Theorem 5.
Once we know that the fixed point exists it is easy to conclude that it is attracting. As in Example 7.1 in the associated Lotka-Volterra equation b, f < 0 and the condition bf −ec > 0 follows from the fact that ρ 3 > 0.  Proof.
To prove the claim about the limit behavior of this system and of Example 7.1A, we transform to a Lotka-Volterra system. Example 7.2B. If NEITHER fixed point is invadable, see Bomze #10, there is an interior saddle point separating the domains of attraction of the two edge equilibria. As we mentioned in the overview, by analogy with results of Durrett and Levin [16], we expect that the winner in the spatial game is dictated by the direction of movement of the traveling wave connecting the two boundary fixed points, but we do not even know how to prove that the traveling wave exists.

One edge fixed point
There are several possibilities based on the orientation of the edges on the other two sides.
We begin with the one that leads to coexistence.  In words, all entries are positive except for α 1 and α 3 . As in the previous two examples, the fact that 2's can invade the 1,3 equilibrium implies that the numerator of ρ 2 in (34) is positive. The numerator of ρ 1 is positive since In the numerator of ρ 3 β 3 β 1 > 0, α 3 α 2 < 0, −α 3 β 3 > 0 so again we have to resort to the existence of a repelling function proved in Section 8.3 and Theorem 7.4 to prove that this is positive. As in the previous two cases, the game transforms into the Lotka-Volterra equation given in (46). It has b, f < 0, and the positivity of bf − ce follows from that of the numerator of ρ 3 , so the fixed point is globally attracting.
Example 7.3A. Suppose now we reverse the direction of the 2, 3 edge. Proof. We will convert the system to Lotka-Volterra, so it is convenient to relabel things so that the edge with the stable equilibrium is 1, 2 The behavior of the edges implies that the sign pattern of the matrix 1 2 3 1 0 + − 2 + 0 − 3 + + 0 When we convert to a Lotka-Volterra system, we subtract the last row of the matrix from the other two, and the last column gives us the constants.
dx/dt = x(a + bx + cy) dy/dt = y(d + ex + f y) The sign pattern of the game matrix tells us that a, d, b, f < 0, while c and e can take either sign. If either c or e is negative the system converges to (0, 0).

' '
• dy/dt = 0 k c c Since there are no boundary equilibria, dx/dt = 0 cannot intersect the x axis and dy/dt = 0 cannot intersect the y axis, so the line dx/dt = 0 must lie above dy/dt = 0. A little thought reveals that in the region dx/dt dy/dt Consulting the picture we see that if the trajectory starts in A or B then it will enter C, and cannot re-enter A or B so it must converge to (0, 0).
Example 7.3B. At first glance it may seem that 2 1 and 2 3 imply that 2 can invade the 1, 3 equilibrium. However, 2 3, 2 1, and the existence of a stable fixed point on the 1, 3 edge implies only that α 1 , α 3 < 0 while the other six entries are positive. The example shows it is possible that 2 is not invadable (Bomze #14). Since 2 is always locally attracting (look at the middle column of the matrix) there is an interior fixed point, which is a saddle point. The system is bistable. See the discussion of Example 7.2B for the conjectured behavior of the spatial game.  (Bomze #42), and the replicator equation will converge to the 1, 3 equilibrium. The proof of this claim is the same as for the next example, so we will wait until then. Let (p 13 , 0, q 13 ) be the boundary equilibrium. The proof of Lemma 8.1 will show that if ε is small enough is a convex Lyapunov function, so using Theorem 1.4 in [56] we can show that any equilibrium has frequencies close to the boundary equilibrium. It should be possible to show that the 2's die out, but as in [56] this is much harder than proving coexistence. Example 7.3D. Suppose now that 1, 3 cannot be invaded by 2's. We have already considered this possibility in Examples 7.3B and 7.3C so we can suppose that the arrows are in opposite directions. If we transform this example or the previous one to a Lotka-Volterra system by dropping the third strategy, then we can use Lemma 5.2 in [58] to construct a convex Lyapunov function. As in the case of Example 7.3A, it is unfortunate that (i) this does not pull back to be a convex function for the replicator equation, and (ii) we cannot generalize the Lyapunov function from the previous example to cover. Thus we leave it to some reader to show that in the spatial model 2's will die out leaving an equilibrium consisting of 1's and 3's.

No edge fixed points
There are 2 3 = 8 possible orientations for the arrows on the three edges. Two lead to an interesting situation.
In the situation drawn we have β i > 0 > α i so the denominator of ρ i and there is an interior fixed point. Theorem 7.7.2 in Hofbauer and Sigmund [7] describes the asymptotic behavior of the game. Bomze #17 is the case ∆ > 0, while #16 is the case ∆ = 0. He does not give an example of ∆ < 0 because in his classification he does not distinguish a flow from its reversal.
Proof. By using the projective transformation in Lemma 7.1 we can make the game constant sum, i.e., b i − a i+1 = γ, where a 4 = a 1 . In the constant sum case, if ρ i is the interior equilibrium, V (u) = i u ρ i i and u(t) is a solution of the replicator equation then In the constant sum case, the sign of γ is the same as the sign of ∆. The gradient V has a local maximum on {(u 1 , u 2 , u 3 ) : Example 7.4A. The other six orientations lead to one strategy dominating the other two (Bomze #43) and taking over the system. Consider now, without loss of generality the case in which 1 2, 1 3, and 2 3.
In this case the matrix has the form We have written the matrix this way so we can use Theorem 7.2 to transform into a Lotka-Volterra equation with where x = u 2 /u 1 and y = u 3 /u 1 . From the equation it is clear that x(t) ↓ 0. If a ≥ c, y(t) ↓ 0. If a < c then when x < d/(c − a), y is decreasing and will converge to 0.

Spatial three strategy games
Recall that we are supposing the game is written in the form.
Since the diagonal entries G i,i = 0, the reaction terms for both updates have the form where p = p 1 and θ = p 2 /p 1 for Birth-Death updating and for Death-Birth updating p =p 1 and Thus the perturbation matrix A ij = θ(G i,j − G j,i ), the limiting reaction-diffusion equation of interest has the form where φ H is the right-hand side of the replicator equation for the matrix More compactly, the diagonal entries are 0. If i = j then H i,j = (1 + θ)G i,j − θG j,i .
Here we are interested in how the system behaves in general, so we will forget about the transformation that turned G into H and assume that H has entries given in (47). However, we will consistently use H for the game matrix to remind ourselves that we are working with the transformed game, and the results we prove are for 1 + ε 2 G where G is the original game.

Repelling functions
The key to our study of spatial games is a result stated in the introduction of [58] as Proposition 3. Given is an ODE A continuous function from Γ = {(u 1 , . . . u n ) : u i ≥ 0, u 1 + · · · + u n = 1} to [0, ∞) is said to be a repelling function if (ii) for each δ > 0 there is a c δ > 0 so that dφ(u(t))/dt ≤ −c δ when M + δ < φ < ∞, Consider now the PDE du i /dt = δu i + f i (u).
This result implies that Assumption 1 in Section 1.2 of Cox, Durrett, and Perkins [56] so using their Theorem 1.4 stated a few lines later, we get the result what we will use to prove coexistence in spatial games.
Theorem 8.2. Suppose a repelling function exists for the replicator equation for the modified game H. If ε < ε 0 (G) then there is a nontrivial stationary distribution for the spatial game with matrix 1 + ε 2 G that concentrates on Ω 0 , the configurations with infinitely many 1's, 2's, and 3's.

Boundary lemmas
To construct our repelling function, we will define for each i define h i near each edge u i = 0. The desired repelling function is is chosen so that {h > M } is inside the union of strips where h i is known to be decreasing. Here, and in what follows, we set h i = 0 when u i ≥ δ i The first case to consider is an edge with an attracting fixed point.
The function and the constants have subscript 2 because we are looking at the 1, 3 edge where u 2 = 0. To explain the definition of h 2 , we note that has a minimum when u = p 13 , so it will be decreasing along solutions of the ODE on the boundary. The logarithmic divergence of the function near u = 0 and u = 1 keeps dh 2 /dt bounded away from 0 when |u 1 − p 1,3 |, |u 3 − q 1,3 | ≥ η > 0. The ε in front of log(u 2 ) implies that the impact of this term will be felt only near the boundary fixed point, where the invadability condition will imply h 2 (u(t)) is decreasing.
Proof. To begin the calculation, we recall that the replicator equation is where From this it follows that The terms that do not have u 2 are Using the fact that p 13 = β 2 /(α 2 + β 2 ) and q 13 = α 2 /(α 2 + β 2 ) we can write the above as The sum of the first two terms is bounded away from 0, when |u 1 − p 13 |, |u 3 − q 13 | ≥ η 2 , and u 2 ≤ 1/2. When u 1 = p 13 and u 2 = q 13 the term in square brackets is > 0 by the invadability condition (42): From this we see that if we choose η 2 small then the term in square brackets will be > 0 when |u 1 − p 13 |, |u 3 − q 13 | ≤ η 2 . Hence if ε 2 is small and ε ≤ ε 2 then the expression in (52) is bounded away from 0 when u 2 ≤ 1/2. The terms that have u 2 are The absolute value of the last expression is ≤ C 2 u 2 and the desired result follows.
The next case to consider is an edge where one strategy dominates the other (in their 2 × 2 subgame).
The assumption α 2 > 0 implies that on the 1, 3 edge, 1 is an unstable fixed point. The assumption β 1 > 0 implies that on the 2, 3 edge, 2 is an unstable fixed point. To see these conclusions look at the matrices If α 2 > 0, u 1 = 1 − δ and u 2 = δ then for small δ the fitness of 3 is ≈ α 2 while the fitness of 1 is ≈ 0. Similar reasoning gives the second conclusion.
Proof. Using (50) we have Consulting (51), we see that the terms that do not involve u 3 are: If ε is chosen small enough then (here we use α 3 < 0 < β 3 ) 1]. In (53) this term is multiplied by u 1 u 2 which vanishes when u 1 or u 2 is zero Using α 2 , β 1 > 0, the terms that do not involve u 3 are when u 3 ≤ 1/2. The terms that involve u 3 are The absolute value of the last expression is ≤ C G u 3 where and the desired result follows.
In the next result we reverse the condition α 2 > 0. In this case h 2 (u(t)) fails to be decreasing near (1, 0, 0), but this can be counteracted by using a function from Lemma 8.1 or Lemma 8.2 which has no exceptional set. The precise bounds are need for rock-paperscissors where we have to combine three of these functions and choose the δ i at the same time. . Given δ 1 , δ 2 , δ 3 > 0, we can pick ε 2 so that if ε ≤ ε 2 and u 3 ≤ δ 3 , then dh 3 (u(t))/dt is bounded by where C G is the constant given in (55).
The first two bounds do not require the conditions listed beside them, but the conditions indicate when we will use them.
Proof. As in Lemma 8.2, the terms with u 3 are ≤ C G δ 3 while the terms that do not involve u 3 are: To bound the first term we note that for u 1 ∈ [0, 1], which gives the first result. Since α 3 < 0 < β 3 , if we pick ε 3 small then for ε ≤ ε 2 , we have (1 − u 1 )α 3 − β 3 u 1 + C G ε ≤ 0. From this, we see that the terms that do not involve u 3 are ≤ −ε(α 2 u 1 + β 1 u 2 ) To get the second bound note that implies that (56) ≤ −εα 2 . When u 1 ≤ δ 1 and u 3 ≤ δ 3 then and we have the desired result.
Remark. By changing the indices we can apply this lemmas to other edges. To save the reader the headache of translating them and to make sure we did it correctly we do this now Lemma 8.4. Suppose 3 2 (α 1 < 0 < β 1 ), β 2 > 0 and α 3 ≤ 0. Let u(t) be a positive solution of the replicator equation and let h 1 (u) = u 2 − ε log(u 1 ). Given δ 1 , δ 2 , δ 3 > 0, we can pick ε 1 so that if ε ≤ ε 1 and u 1 ≤ δ 1 , then dh 1 (u(t))/dt is bounded by To check the indices note that using (50) we have In the first round of analysis in the proof, we do not look at terms with u 1 , so β 3 u 1 drops out, while u t Hu gives us term u 2 u 3 (α 1 + β 1 ).
Lemma 8.5. Suppose 1 3 (α 2 < 0 < β 2 ), β 3 > 0 and α 1 ≤ 0. Let u(t) be a positive solution of the replicator equation and let h 2 (u) = u 3 − ε log(u 2 ). Given δ 1 , δ 2 , δ 3 > 0, we can pick ε 2 so that if ε ≤ ε 2 and u 2 ≤ δ 2 , then dh 2 (u(t))/dt is bounded by To check the indices note that using (50) we have and reason as in the previous lemma. As a final check, note that as we cycle through the three results we see each index i exactly once in each position. It is for this reason we wrote the terms (1 − δ i − δ j ) in the third case in a special order. Example 7.2. We can apply Lemma 8.1 to the sides u 1 = 0 and u 2 = 0, and Lemma 8.2 to the side u 3 = 0 in order to construct the h i . Example 7.3. This time it is useful to have the picture to look at.

Results for four classes of examples
We can apply Lemma 8.1 to the side u 2 = 0 and Lemma 8.2 to the side u 3 = 0. These functions decrease without exceptions. That is not true for the function that Lemma 8.4 gives us for the side u 1 = 0. dh 1 /dt < 0 fails when u 3 is small. however, if we let δ 3 be the constant that comes from Lemma 8.2 and choose ε small then dh 3 /dt ≤ −γ 1 < 0 and when u 3 ≤ δ 3 , dh 3 /dt ≤ −εα 3 and −εα 3 − γ 3 < 0. With these choices made h 1 + h 2 + h 3 is always decreasing.
Example 7.4. In this example there are no stable boundary fixed points, so we have to use Lemmas 8.3, 8.4, and 8.5. At each corner we will need to use the good part of one function to cancel the bad part of another one. To do this we will need the condition β 1 β 2 β 3 + α 1 α 2 α 3 > 0. In the next figure we have labeled the corners with the inequalities that dictate the behavior there.
Here u 1 ≈ 1 means u 2 ≤ δ 2 , u 3 ≤ δ 3 , etc. To control the behavior near the corners, it is enough to pick the A i so that for then if we pick the δ i smaller the good bound plus the bad bound is always negative. To do this we set A 1 = 1 then pick At this point we are out of choices so it is fortunate that ∆ = β 1 β 2 β 3 + α 1 α 2 α 3 > 0 implies Having taken care of the corners the rest is routine. Lemma 8.3 implies that if ε is small enough dψ(u(t)/dt < 0 when u 1 ≥ δ 1 , u 2 ≥ δ 2 and u 3 ≤ δ 3 . So applying Lemma 8.4 and 8.5 controls the other regions next to the boundary and the proof is complete.

Almost constant sum games
The results in the previous section prove the existence of a stationary distribution but does not provide much information about the frequencies of the three strategies. If we can find a Lyapunov function that is strictly decreasing over the whole domain then we can use Theorem 1.4 of [56] to conclude that when ε is small the frequencies of strategies in the spatial game are almost the same as those of the equilibrium in the modified replicator equation. Generalizing the calculation from the proof of Theorem 7.5 we can prove: Theorem 8.3. Suppose that the three strategy game H has (i) zeros on the diagonal, (ii) an interior equilibrium ρ, and that H is almost constant sum: H ij + H ji = γ + η ij with γ > 0 and max i,j |η i,j | < γ/2. Then V (u) = i u i − ρ i log u i is a repelling function with M = 0, i.e., it is always decreasing not just near the boundary.
Note that due to the formula for the perturbation (49) G is almost constant sum if and only if H is.
Proof. If u(t) evolves according to replicator equation for H then To bound the remaining piece we note that |ab| ≤ (a 2 + b 2 )/2 so so if max ij |η ij | < γ/2 then V is a convex Lyapunov function.It is not hard to checking that the other conditions to be a repelling function are satisfied.

Tarnita's formula
Tarnita, Wage and Nowak [36] have generalized Theorem 6.2 to games with n strategies. They say that a strategy is favored by selection if, in the presence of weak selection, its frequency is > 1/n. Their main result is that strategy k is favored by selection if (σ 1 a k,k +ā k, * −ā * ,k − σ 1ā * , * ) + σ 2 (ā k, * −ā) > 0 The parameters σ 1 and σ 2 depend on the population structure, the update rule and the mutation rate, but they do not depend on the number of strategies or on the entries a ij of the payoff matrix. The coefficients arē a * , * = (1/n) In our three strategy games the diagonal is 0, so a k,k = 0, a * , * = 0 and the structural constant σ 1 is not important. When k = 1, (57) is linear in the coefficients. The almost constant sum games in the previous section are an open set of games for which we can show that in the small selection limit, the equilibrium frequencies of the spatial game converges to those of the equilibrium frequencies of the replicator equation for the modified game. In this situation strategy 1 will be favored by selection if when we use (34) on the modified game This quantity involves our structural coefficient θ which p 2 /p 1 for Birth-Death updating and for Death-Birth updating but due to (34) the condition (58) is quadratic in the entries in the game matrix.
9 Analysis of three strategy cancer games

Multiple myeloma
We begin with this example since the analysis of the original game is fairly simple and space can allow coexistence, which is not possible in the replicator equation for original game. As explained in Example 1.7, the players are osteoclasts (OC), osteoblasts (OB), and multiple myeloma (M M ) cells, and the payoff matrix is To study the properties of the game we begin with the two strategy games it contains. Invadability. As in Section 7, to determine the behavior of the three strategy game, we will see when the third strategy can invade the other two in equilibrium. Here and what follows it will be convenient to also refer to strategies by number: 1 = OC, 2 = OB, 3 = M M .
In the OC, M M equilibrium, the fitnesses F 1 = F 3 = bc/(b + c), while F 2 = (eb − dc)/(b + c), so OB can invade if eb − dc > bc. Letting δ = dc/be, we can write this as 1 − dc be > c e or β + δ < 1 Since δ > 0, these two conditions β > 1 and β + δ < 1 cannot be satisfied at the same time. Let x 1 = (a/(a + e), e/(a + e), 0) Thus we have three cases Case 1. β > 1, x 1 can be invaded, while x 2 cannot. This is Example 7.2A, so the replicator equation converges to x 1 .
Case 2. β < 1, β + δ > 1: Neither x 1 nor x 2 can be invaded. This is Example 7.2B, so there is an interior fixed point that is a saddle point, and the replicator equation exhibits bistability. We leave it to the reader to verify that the interior equilibrium is: where D is the sum of the numerators.
Case 3. β + δ < 1, x 1 cannot be invaded but x 2 can. This is Example 7.2A, so the replicator equation converges to x 2 .
The modified game has entries The modification of the game does not change the sign of G 2,3 but it puts a positive entry in G 3,2 . It may also change the signs of one or two of the other four non-zero entries. Noting that If one of these two entries is negative the other one is positive. The same holds for G 13 and G 31 To simplify formulas, we let We always have D, F > 0. Suppose to begin that A, B, C, E > 0. The condition for OB to invade the OC, M M equilibrium, when written in the new notation, is unchanged In the OC, OB equilibrium, i.e., (C − E)A + F E > 0 or C/E + F E > 1, which is no longer inconsistent with C/E < 1 − DC/BE, so we have a new possibility.
We are in the situation of Example 7.2 so there is coexistence in the spatial game.

Three species chemical competition
As explained in Example 1.4, the payoff matrix is Here P's produce a toxin, R's are resistant to it, S's are sensitive wild-type cells, and we have supposed g > e i.e., the benefit of producing the toxin outweighs the cost. Subtracting a constant from each column to make the diagonal entries 0.
To simplify the algebra we will consider a more general game thinking primarily about the special case Here c, d > 0 since g > e and h > 0, but a and b can have either sign.
To investigate the game we begin by considering the two strategy subgames it contains.
R vs. S. d > 0 so S dominates R.
P vs. S. c > 0, so P dominates S. P vs. R. All four cases can occur. We now consider them one at a time.
Case 1. If a > 0 and b < 0 then P dominates R. Since P also dominates S, this is Example 7.4A. P 's take over in the replicator equation. We conjecture but do not know how to prove that this happens in the spatial model. Case 2. If a < 0 and b > 0, then R dominates P and we have a cyclic relationship between the competitors. We are in the situation of Example 7.4. Let If ∆ > 0 solutions of the replicator equation will spiral into a fixed point, so we will have coexistence in the spatial game.
If neither of the two strategies P and R dominates the other, we will have a mixed strategy equilibrium with Here and in what follows p ij and q ij denote the equilibrium frequencies of the first and second strategies in the i, j subgame. Using the facts about 2 × 2 games recalled right after (39): Case 3. If a > 0 and b > 0 then (p 12 , q 12 ) is attracting on the P, R edge. Case 3A. The interior equilibrium will attracting and we are in the situation of Example 7.3, and there will be coexistence in the spatial game.
Case 3B. The fixed point on the R, P edge will attracting, and we are in the situation of Example 7.3D. The S's should die out in the spatial game and we should have an equilibrium consisting only of R's and P 's but we do not know how to prove that.
so the numerator of ρ 3 is negative. If all three numerators were negative then there would be an equilibrium but the numerator of ρ 2 > 0, so there is not. As a check of our computation of the fixed point ρ, we note that when a = h − e, b = f − g + e − h, c = g − e, and d = h we have so D = gf and the interior fixed point is in agreement with page 1496 of [48].
In Case 4A, S can invade the P, R equilibrium, so looking at (60), we see that the numerator of ρ 3 is negative. The numerator of ρ 1 is positive so there can be no equilibrium.
In Case 4B, S cannot invade so the numerator of ρ 3 is positive. Since a < 0 and c, d > 0 the numerator of ρ 1 is positive. In general we might have b + c + d < 0 but in our special case b + c + d = (1 + 2θ)g + (1 + θ)(f − g) = θg + (1 + θ)f In this case, there is an interior equilibrium, but it is unstable. Since P is locally attracting, we should have convergence to P in the spatial game but cannot prove it.

Glycolytic phenotype
As indicated in Example 1.5, the three strategies are AG = autonomous growth, IN V = invasive, and GLY = glycolytic phenotype, while the payoff matrix is: As noted earlier, if we subtract a constant from each column to make the diagonal zero, this will not change the properties of the replicator equation.

Tumor-stroma interactions
As explained in Example 1.6 there are three strategies S = stromal cells, I = cells that have become independent of the micro-environment, and D = cells that remain dependent on the microenviornmnet. The payoff matrix is: Recall that we assume β < 1, γ < 1. Subtracting a constant from the second and third columns to make the diagonals 0 the game becomes Generalizing we have where the constants are We assume β < 1 so a, b > 0 when θ = 0, and there will an attracting fixed point (a.f.p.) on the S, D edge for any θ ≥ 0, c > 0. We assume γ < 1 so I S. Almost anything is possible for the signs of d and e. When θ = 0 we have There are a large number of possible cases, so we only consider the ones that lead to coexistence.
Case 1. If the S, D and D, I fixed points exist and can be invaded then we are in the situation of Example 7.2 and there will be coexistence in the spatial model.

Voter model duality: details
In the degenerate case of an evolutionary game in which G i,j ≡ 1, the system reduces to the voter model which was introduced in the mid 1970s independently by Clifford and Sudbury [55] and Holley and Liggett [72] on the d-dimensional integer lattice. It is a very simple model for the spread of an opinion and has been investigated in great detail, see Liggett [75] for a survey. To define the model we let p(y) be an irreducible probability kernel p on Z d that is finite range, symmetric p(y) = p(−y), and has covariance matrix σ 2 I.
In the voter model, each site x has an opinion ξ t (x) and at the times of a rate 1 Poisson process decides to change its opinion, imitating the voter at x + y with probability p(y).
To study the voter model, it is useful to give an explicit construction called the graphical representation, see Harris [70] and Griffeath [69]. For each x ∈ Z d and y with p(y) > 0 let T x,y n , n ≥ 1 be the arrival times of a Poisson process with rate p(y). At the times T x,y n , n ≥ 1, x decides to change its opinion to match the one at x + y. To indicate this, we draw an arrow from (x, T x,y n ) to (x + y, T x,y n ).
To define the dual process ζ x,t s , which we think of as the motion of a particle, we start with ζ x,t 0 = x. To define the dynamics we start at x at time t and work down the graphical representation. The process stays at x until the first time t − r that it encounters the tail of an arrow x. At this time, the particle jumps to the site x + y at the head of the arrow, i.e., ζ x,t r = x+y. The particle stays at x+y until the next time the tail of an arrow is encountered and then jumps to the head of the arrow etc. The definition of the dual guarantees that if ξ t (x) is the opinion of x at time t in the voter model then we have the result called (7) in the introduction ξ t (x) = ξ t−s (ζ x,t s ) For fixed x and t, ζ x,t s is a random walk that jumps at rate 1 and by an amount with distribution p(y). It should be clear from the definition that if ζ x,t s = ζ y,t s for some s then the two random walks will stay together at later times. For these reasons the ζ x,t s are called coalescing random walks.
Having a dual random walk starting from x for each t is useful for the derivation of (7). However for some computations it is more convenient to have a single set valued dual process ζ B t that starts with a particle at each point of B, particles move as independent random walks until they hit, at which time they coalesce to 1. Extending the reasoning that lead to (7) P (ξ t (x) = 1 for all x ∈ B) = P (ξ 0 (y) = 1 for all y ∈ ζ B t ) Holley and Liggett (1975) have shown Theorem 10.1. In d ≤ 2 the voter model approaches complete consensus, i.e., P (ξ t (x) = ξ t (y)) → 1. In d ≥ 3 if we start from product measure with density u (i.e., we assign opinions 1 and 0 independently to sites with probabilities u and 1 − u) then as t → ∞, ξ u t converges in distribution to ξ u ∞ , a stationary distribution, ν u , in which a fraction u of the sites have opinion 1.
Proof. We will give the proof here since it is short and we think it will be instructive for readers not familiar with duality. Our assumptions imply that p(y) has mean 0 and finite variance. Well known results for random walk imply that in d ≤ 2 the associated random walk is recurrent. This implies that two independent random walks will eventually meet and hence the probability of disagreement between two sites in the voter model To prove the second result it suffices to show that ξ u t converges in distribution to a limit ξ u ∞ , for then standard results for Markov processes will imply that the distribution of ξ u ∞ is stationary. To do this, we use (62) to conclude

Proofs of the coalescence identities
Let v 1 , v 2 , v 3 be independent and have distribution p. In this section we prove the coalescence identities stated in Section 4 Proof. Let S n be the discrete time random walk with jumps distributed according to p and let h(x) = P x (S n = 0 for all n ≥ 0) be the probability the walk never returns to 0 starting from x. By considering what happens on the first step The two results follow since p(x|y) = h(y − x).
which proves the desired result From the two particle identities we easily get some for three particles. The starting point is to note that considering the possibilities for y when x and z don't coalesce we have a result we earlier called (10) p(x|z) = p(x|y|z) + p(x, y|z) + p(x|y, z) Combining this identity with one for another pair that shares a site in common leads to identities that relate the three ways three particles can coalesce to give two.
where x, y, z is any ordering of 0, v 1 , v 1 + v 2 .
Proof. Using (10) Since p(0|v 1 + v 2 ) = p(0|v 1 ) by Lemma 8 the first result follows. Translation invariance implies p(0|v 1 ) = p(0|v 2 ) = p(v 1 |v 1 + v 2 ) so comparing the first and third lines gives the second result. The displayed identity follows from the first two and first and third in the proof.

Derivation of the limiting PDE
In this section we will compute the functions φ i B (u) and φ i D (u) that appear in the limiting PDE. To do this it is useful to note that since j u i = 1 we can write the replicator equation (20) as where the restriction to j = i comes from noting that the when i = j the G's cancel.

Birth-Death dynamics
By (4) the perturbation is h i,j (0, ξ) = k f (2) j,k (0, ξ)G j,k In the multi-strategy case the rate of change of the frequency of strategy i in the voter model equilibrium is where q(a, b, c) = P (ξ(0) = a, ξ(v 1 ) = b, ξ(v 1 + v 2 ) = c) and v 1 , v 2 are independent random choices from N . If a = b then q(a, b, c) = p(0|v 1 |v 1 + v 2 )u a u b u c (65) + 1 (c=a) p(0, v 1 + v 2 |v 1 )u a u b + 1 (c=b) p(0|v 1 , v 1 + v 2 )u a u b Combining the last two equations, and comparing with (63) we see that Lemma 11 implies that p(0, v 1 +v 2 |v 1 ) = p(0|v 1 , v 1 +v 2 ) so the last two terms can be combined into one.
which is the formula given in (21)

Proof of Tarnita's formula
Lemma 13.1. Suppose the φ in the limiting PDE is cubic. The limiting frequency of 1's in the PDE is > 1/2 (and hence also in the particle system with small ε) if and only if φ(1/2) > 0.
Proof. As explained in the proof of Theorem 6.1 given in [56] the behavior of the PDE is related to the shape of φ in the following way.