Fixation times in evolutionary games under weak selection

In evolutionary game dynamics, reproductive success increases with the performance in an evolutionary game. If strategy $A$ performs better than strategy $B$, strategy $A$ will spread in the population. Under stochastic dynamics, a single mutant will sooner or later take over the entire population or go extinct. We analyze the mean exit times (or average fixation times) associated with this process. We show analytically that these times depend on the payoff matrix of the game in an amazingly simple way under weak selection, ie strong stochasticity: The payoff difference $\Delta \pi$ is a linear function of the number of $A$ individuals $i$, $\Delta \pi = u i + v$. The unconditional mean exit time depends only on the constant term $v$. Given that a single $A$ mutant takes over the population, the corresponding conditional mean exit time depends only on the density dependent term $u$. We demonstrate this finding for two commonly applied microscopic evolutionary processes.


Introduction
Systems in which successful strategies spread by imitation or genetic reproduction can be described by evolutionary game theory. Such models are routinely analyzed in evolutionary biology, sociology, anthropology and economics. Recently, the application of methods from statistical physics to these systems has lead to many important insights [1,2,3,4,5].
Traditionally, the dynamics is described by the replicator equations, where the growth rate of a strategy is associated with its relative success compared with the population average [6,7].
In the past years, research has focused on stochastic evolutionary game dynamics in nite populations [8,9,10,11,12,13,14,15,16,17,18,19,20,21,22]. In this context, a connection to the weak selection limit of population genetics has been established [8]. Weak selection means that the payo dierences based on dierent strategic behavior in interactions represent only a small correction to otherwise random dynamics, similar to high temperature expansions in physics. Weak selection is considered as a relevant limit in biology, as most evolutionary changes are driven by small tness dierences [23]. Moreover, it allows analytical approximations that are often impossible when selective dierences in payos are large [8,24,25].
Most of the recent work that uses the weak selection approximation has been focusing on the probability that a certain strategy takes over. The time associated with this process has been calculated [26], but it received considerably less attention so far. Here, we present the weak selection corrections to the conditional and unconditional mean exit or xation times in evolutionary 2×2 games with N players.
The conditional average time to xation t A 1 is the expected time a single mutant needs to take over the population, given that such a takeover occurs at all. The unconditional average time of xation t 1 is the expectation value for the time until the population is homogenous again after the arrival of a single mutant. This is regardless of wether the mutant type takes over the population or becomes extinct. Equivalently, the average xation times for such one dimensional random walks can also be interpreted as mean rst passage times or mean exit times [27,28,29].
Throughout this paper, we use the payo matrix An A player interacting with another A receives a. If it interacts with B, it obtains b. Similarly, B receives c from A and d from other B's. Thus, the average payos are A quantity that is of particular interest is the dierence between the average payos, We show that under weak selection, the conditional time (t A 1 ) during which a single mutant takes over the whole population depends only on u (and, of course, on the population size). The unconditional time (t 1 ) during which the mutant either takes over the population or reaches extinction depends only on v (and the population size). See Figure 1 for an illustration of the relevant quantities.
Our manuscript is organized as follows: In Section 2, we introduce a particular evolutionary process for our analysis. Although our results are valid for a broader class of processes, we only present the full calculation for this evolutionary process. In Section 3, we recall the general form of xation probabilities and times. We discuss neutral selection in Section 4 as a prerequisite to the weak selection expansion, which we explore in Section 5. In Section 6 we address the frequency dependent Moran process to underline the generality of our ndings. The consequences of our analytical results are discussed in Section 7.  Figure 1: Illustration of the most relevant quantities. We are interested in the evolutionary fate of a single A player. All quantities depend on the intensity of selection β and the population size N . The payo dierence between A and B players is given by ∆π = u i + v, with i as the number of A players. Both the transition probabilities T + i and T − i and the probability that a single A player takes over the population φ 1 depend on u and v. But for weak selection, β 1, the conditional time t A 1 during which a single A player takes over a population of B players only depends on u, whereas the unconditional time t 1 until either A or B has taken over the population only depends on v.

Fermi process
In a nite population of size N with two possible strategies A and B, the state of the system is characterized by the number of type A individuals i. In general, the dynamics is stochastic. In each time step, a randomly chosen individual evaluates its sucess. It compares this payo with a second, randomly chosen individual. If this second individual has a higher payo, the rst one switches strategies with probability p > 1 2 . Otherwise, it switches with p < 1 2 . We assume that the switching probability is given by the Fermi distribution. Its shape is controlled by the intensity of selection β, which can be interpreted as an inverse temperature, In previous work [30,31,32], there is a dierent strategy update procedure. The rst individual switches to the second's strategy with probability p ± i . The second individual can also switch to the rst individual's strategy with probability 1 − p ± i . This yields a factor 2 in the transition probabilities (and, as we will become clear later, a factor 1 2 in the xation times). This process also has a proper strong selection limit, i. e. it is possible to examine β → ∞. In this latter case we have The population size is constant in time, in each time step the state of the system can at most change by one, i. e. from i to i−1 or to i+1. The transition probabilities The probability to stay in the current state is i . An important measure of where the system is more likely to move is their ratio, This is a quantity that describes the tendency to move from the state i to i ∓ 1, depending on whether γ i ≷ 1. Of course, T + i > 0 is required, which follows from β < ∞. The T ± i and thus the γ i are invariant under adding a value to each of the payos given in (1), whereas multiplying the payo matrix with a factor λ results in a change in the intensity of selectionβ = β λ.
Let us now focus on weak selection, β 1. In this case we have Weak selection corresponds to high temperature in Fermi statistics. A Taylor expansion of the γ i up to rst order in β yields γ i ≈ 1 − β ∆π(i). In this case, the probability to move from i to i + 1 is very similar to the probability to move from i to i − 1. Weak selection links the Fermi process to a variety of birth death processes, cf. [8,33].

Fixation probabilities and xation times
From equation (8) it follows that the two pure states all A or all B are absorbing, In a nite population, we can calculate the probability φ i that the system will xate to the pure state all A, starting with the mixed state i. Obviously, we have φ 0 = 0 and φ N = 1. For 0 < i < N , there is a balance equation for the xation probabilities, This recursion leads to an expression for the xation probabilities in terms of the γ i [34,35,36], which is valid for any birth death process.
For the Fermi process, the exact equation (9) simplies matters in an elegant way because the products in equation (11) can be solved, Hence, equation (11) simplies to For large N , the sums in equation (13) can be approximated by integrals, which yields a closed expression for the probabilities φ i [33,37]. General expressions for the unconditional and conditional mean exit times or average times of xation, t 1 and t A 1 , are well known, especially for simple, translational invariant random walks [26,27,38]. A complete derivation for the average times of xation in nite systems without translational invariance can be found in [26,35,39].
In the following, we will focus on the xation of a single A mutant in a population of B. Accordingly, the unconditional and conditional xation times read and respectively. Time is measured in elementary time steps here. Thus, in each time step one reproductive event occurs. In biological contexts, it is often more convenient to measure time in generations, such that each individual reproduces once per generation on average. Time in generations is obtained by dividing the number of time steps by the population size N . It is well known that the variance of the exit times under weak selection can be large [39], which has important biomedical implications [40]. Nonetheless, here we concentrate on the expectation values and do not address the distribution of the exit times.

Neutral selection
An important reference case is neutral selection, which results from vanishing selection intensity β = 0 [41]. Neutral selection is a very general limit, which is typically not aected by the details of the evolutionary process. For neutral selection This is a dierence to the simple random walk in one dimension, which is invariant with respect to translation [29].
For the Fermi process, the neutral transition probabilities are We (11), it is thus clear that the probability of xation to A is given by the initial abundance of A, For the neutral unconditional time of xation t 1 we get Details for this calculation can be found in Appendix A. We introduced the shorthand notation for the harmonic numbers H N −1 = N −1 l=1 1 l , which diverge logarithmically with N . In the same way we can solve For neutral selection, the conditional average time of xation of a single mutant diverges quadratically with the system size.

Weak Selection
In this section we will calculate the linear corrections of the mean exit times or xation times t 1 , and t A 1 under weak selection, β 1. Of course, all weak selection approximations are valid only if the term linear in β is small compared to the constant term.
The xation probabilities for small β are which has been derived for a variety of evolutionary processes before [8,10,12,35,36,42]. Next, we address the weak selection approximation of the xation times. The expectation value of the unconditional xation time of a single A mutant in a population of B is in general given by the exact equation (14). With the the transition and xation probabilities of the Fermi process, the unconditional xation time of absorption at any boundary simplies to The weak selection approximation takes the remarkably simple form (see Appendix B for details) with v given in (6). Thus, t 1 depends only on the constant term of the payo dierence. For large N , this yields v ≈ b − d. That is, for large populations under weak selection the linear correction of the average xation time only depends on the advantage (or disadvantage) of the A mutants in the resident population. For b > d, invasion of A mutants is likely and slows down the time until the population is homogeneous again. For d > b, it is dicult for A to invade a B population and extinction of the mutants is faster than in the neutral case. Note that the payo entries a and c have no inuence on the unconditional xation time under weak selection corrections. Since xation is unlikely for weak selection (the probability of xation of a single A mutant is approximately N −1 ), the unconditional xation time is dominated by the xation to B. In this case, it is enough to discuss the invasion of A mutants. Next, we address the average time to xation given that the A mutant takes over the population. With the general result (15) the Fermi processes conditional xation time to all A reads Its linear approximation turns out to be dependent on the payos in a very simple way as well, The detailed calculation can be found in Appendix B. Since during the xation process all payos are of importance, it is obvious that they all enter here. For example, when it is easy to invade because few mutants have an advantage (b > d), but dicult to reach xation because mutants are disadvantageous once they are frequent (c > a), we have u < 0 and the conditional time to xation is larger than neutral. In the last section, we discuss special classes of games to show that, under weak selection, the conditional mean exit times of xation (or absorption) do not always follow the intuition based on the payo matrix (1).

Frequency dependent Moran process
In this section we address the generality of the previous ndings discussing an alternative evolutionary process. The rst model that connects payos from a 2 × 2 game to reproductive tness using a weak selection approach in nite populations is the frequency dependent Moran process [8,9]. In this process, an individual is chosen for reproduction with probability proportional to its tness f (i). The ospring replaces a randomly chosen individual. The average payos (2) and (3) are mapped to the tness such that probabilities of the standard Moran process read Although these transition probabilities are dierent from those of the Fermi process, they also yield γ i ≈ 1 − ∆π(i) and k Thus, the weak selection approximations of the xation probabilities φ l of the Moran process and the Fermi process are identical, see equation (20). But the weak selection approximation of the transition probabilities are not identical, which leads consequently to dierent mean exit times. Nevertheless, the results have the same, remarkably simple connection to the payo matrix (1). The mean exit times or xation times of the frequency dependent Moran process are Qualitatively, the dependence on the payo matrix via u and v is the same as for the Fermi process. Their calculation is analogous to the ndings of the previous section, details can be found in Appendix B. Note that, comparing with the Fermi process, there is a factor of 2 missing in the neutral terms. However, this can be avoided by rescaling the transition probabilities, without changing the properties of the dierent processes.

Discussion
Finally, let us discuss the implications of our results for general 2 × 2 games. While we concentrate on the Fermi process here, the discussion is equally valid for the frequency dependent Moran process. An important question is whether the linear correction for weak selection is compatible with the general features of the game and the known asymptotic behavior for large N of the mean exit or xation times derived by Antal and Scheuring [26]. Clearly, this depends on the payo matrix of the 2 × 2 game, as the payos enter the rst exit times of absorption linearly. To analyze the dierence to the neutral case we consider the rescaled average times of xation, The rescaled unconditional xation time reads Accordingly, the rescaled conditional xation time for absorption at all A is Note that for population sizes N > 2 and suciently small β, we always have t 1 (0) < t A 1 (0). In other words, the average time until the A individual has reached xation or gone extinct is smaller than the conditional average time until the A individual has reached xation. For β → ∞, the process follows deterministically the intensity of selection and thus both xation times may coincide, This ordering of the xation times is blurred by our rescaling, as we focus only on the change relative to the neutral case.
In the following, we discuss these two expressions for the three generic types of 2 × 2 games, namely dominance of A (a > c and b > d), coexistence of A and B (a < b and c > d) and a coordination game (a > c and b < d).

Dominance of A.
Consider a game where strategy A is always dominant, i. e. it obtains a larger payo than B, regardless of the fraction of A in the population. This is the case for a > c and b > d. One special case is the Prisoner's Dilemma with b > d > a > c. The interesting feature of this game is that the social optimum d is not the Nash equilibrium, which is a. For neutral selection, a single A individual goes extinct with probability 1 − N −1 . Thus, the unconditional xation time τ 1 is dominated by the extinction of A. Since strategy A is favored by selection, increasing the intensity of selection decreases the probability of the extinction of A. Since xation takes at least N − 1 time steps, τ 1 increases with increasing intensity of selection β. For large N , this is obvious from our equation (30), because in this case the quantity N (b−d)−a+d is positive. However, once extinction of A becomes unlikely, increasing β further will lead to a decrease of τ 1 .
The discussion of the conditional xation time τ A 1 is not as straightforward, because the sign of a − b − c + d can be positive or negative. The sign of this quantity is also decisive for the evolutionary dynamics in other contexts, see e.g. [43]. When the advantage of an A individual is initially large and decreases with the abundance of A (a − c > b − d > 0), then the sign of a − b − c + d is positive and τ A 1 decreases with increasing intensity of selection. But when the advantage of strategy A decreases with the number of A individuals (b − d > a − c > 0), then τ A 1 increases with increasing intensity of selection. However, this apparently counterintuitive phenomenon (after all, A dominates B) can only be observed for weak selection. For strong selection, τ A 1 decreases again. These results are compatible with the observation that the conditional xation time scales as N ln N for large N [26]. In Figure 2 (a) we show a numerical example for the rescaled average times. We include averages from numerical simulations of the evolutionary process, our linear approximation as well as the exact result that can be obtained from dividing equation (14) by (18) and equation (15) by (19), respectively. The payo matrix is chosen such that a + d > b + c, which means that with increasing intensity of selection τ A 1 decreases and τ 1 increases.

Coexistence of A and B.
As a second class, we consider games in which B is the best reply to A (c > a), but A is the best reply to B (b > d). Important examples for such games are the Hawk-Dove game [44] or the Snowdrift game [45]. For innite populations, the replicator dynamics predicts a stable coexistence of A and B. In nite populations, the system typically uctuates around that point until eventually, uctuations lead to absorption in one the boundaries [46,47]. Consequently, the conditional xation times increase exponentially with the population size [26]. Since a − b − c + d is negative, we also have an increase of τ A 1 with the selection intensity for weak selection. Further, N (b − d) − a + d is positive in large populations, such that also τ 1 increases with the selection intensity. Figure 2 (b) shows that the divergence of the exact results is faster than the linear approximation even for weak selection.

Coordination games.
Finally, let us discuss coordination games in which a > c and b < d. In these games, A is the best reply to A and B is the best reply to B. The replicator equation of such systems exhibits a bistability: If the fraction of A individuals is suciently high in the beginning, the A individuals will reach xation. Otherwise, B individuals will take over the system. The stronger the intensity of selection, the less likely it is that a single A individual can take over a B population. Consequently, τ 1 should decrease with β. This also follows from our weak selection approximation: In large populations, N (b − d) − a + d is negative and thus τ 1 decreases with the intensity of selection, see equation (30). Perhaps less intuitive, also τ A 1 decreases with β, which results from a − b − c + d > 0, cf. (30). However, this is again consistent with the observation that τ A 1 scales as N ln N in large populations. Although the xation probability of a single A decreases with β, if such an event occurs, it is faster than in the neutral case. A numerical example for this behavior is shown in Figure 2 (c).
The numerical examples indicate that the convergence radius of our weak selection expansion is of the order of N −1 , which is also known for many systems in population genetics. Although N −1 might appear small, this kind of weak selection is the most relevant limit in evolutionary biology, as evolutionary change is typically only connected with small selective dierences. We stress that we have made no assumptions on the population size, such that our results are valid for arbitrary N .
Our approach shows under which circumstances the general features of the game are reected in the xation times under weak selection. Although the weak selection expansion of the mean exit or xation times is technically rather tedious, the resulting asymptotic behavior shows remarkable simplicity.

Acknowledgment
Financial support by the Emmy-Noether program of the DFG is gratefully acknowledged.   Figure 1:

Appendix A. Finite double sums
Here, we collect some helpful calculations for double sums as they appear in the mean exit times. An important observation is for any function f l < ∞ and l = 1, . . . , N − 1. This can be seen by writing the left hand side term by term, i. e.
For the case i = 1 the result is especially simple, since the rst sum of the right hand side of equation (A.1) vanishes. This case is of special interest for the computation of t A 1 under neutral selection with f l = 1 and for t 1 with f l = 1/l. Another nding for double sums with M ∈ N and two bounded functions f k and g l is Here, we calculate the linear corrections of the mean exit times t 1 and t A 1 for the Fermi process in detail, compare equations (21) and (23). We aim at nding these times for weak selection, e.g.
The rst term follows directly from the calculation in Appendix A, see equation (18). Our goal here is to compute the linear term ∂ For the xation probability under weak selection and with ∆π(l) = u l + v, we have The weak selection approximation of the inverse of the transition probability T + l , compare equation (8), yields ∂ ∂β Thisleads to While the rst two double sums can be solved with the help of Appendix A, the third term is more complicated. For this more tedious calculation, we refer to Appendix C. Eventually, the solution of the double and triple sums leads to where the last step is elementary. Combining this with equation (18) The only dierence compared to the unconditional xation time, equation (B.2), is the xation probability φ l instead of φ 1 . The linear term of the weak selection expansion of φ l is given in equation (B.3). This yields Again, the rst two double sums can be solved using the results from Appendix A. The third term follows from a calculation which is similar to Appendix C, but simpler. This last term reduces to Finally, combining the three terms again results in (B.10) In combination with equation (19), this results in the conditional mean exit time under weak selection, equation (24). For completeness, we briey repeat this calculation for the mean exit times of the frequency dependent Moran process. With the transition probabilities (25) and (26), the xation probabilities under weak selection are identical to those of the Fermi process, see equation (20). However, the inverse transition probability is dierent in the weak selection regime, i. e. the linear correction is (N − 1)((N + 1)u + 3v) 6N which diers from equation (B.5) only in the second double sum. With the previous ndings for the Fermi processes times the required calculation is straightforward and results in That is, this linear correction has a dierent dependence on the system size N . For the conditional mean exit time the situation is similar. In dierence to equation (B.8), the linear correction reads (B.14) This the leads to for the linear correction of the conditional mean exit times of the frequency dependent Moran process.

Appendix C. Finite triple sum
Here, we calculate the triple sum from Appendix B, that require some additional steps. Our goal is to solve  (C.5) The second last term, K 3 , is a sum over a linear function and can be treated with any table of elementary sums, e. g. [48], (C.6) The remaining two terms require more eort. Both terms, K 1 and K 2 have the same structure regarding functions of k and l. Using equation (A.3), we have (C.10) For K 1 , we obtain Summing up the terms, σ = (K 1 + K 2 − K 3 − K 4 )/N , nally yields the result σ = N − 1 6 (((N + 1)u + 3v)H N −1 − 3(u + v)) .

(C.11)
Again, H n = n l=1 1/l are the harmonic numbers. In equation (B.9), the reasoning is very similar, but only terms of the structure of K 2 and K 4 appear.