Fitness-based models and pairwise comparison models of evolutionary games are typically different—even in unstructured populations

The modeling of evolutionary game dynamics in finite populations requires microscopic processes that determine how strategies spread. The exact details of these processes are often chosen without much further consideration. Different types of microscopic models, including in particular fitness-based selection rules and pairwise comparison dynamics, are often used as if they were interchangeable. We challenge this view and investigate how robust these choices on the micro-level really are. We focus on a key macroscopic quantity, the probability for a single mutant to take over a population of wild-type individuals. We show that even in unstructured populations there is only one pair of a fitness-based process and a pairwise comparison process leading to identical outcomes for arbitrary games and for all intensities of selection. This strong restriction is not relaxed even when the class of pairwise comparison processes is broadened. This highlights the perils of making arbitrary choices at the micro-level without regard of the consequences at the macro-level.


Introduction
Evolutionary game theory is a powerful framework to model biological and social evolution when the success of an individual depends on the presence or absence of other strategies [1][2][3][4]. In this context, the payoff from a game between individuals is translated into reproductive fitness. Methods from statistical physics have been applied extensively since the field moved from mostly deterministic models based on rate equations to stochastic individual-based models [5][6][7][8][9][10]. These more sophisticated models use a microscopic process as a starting point to determine how successful strategies spread. Tools and ideas from statistical physics are key to making the connection between the assumptions on the micro-scale, and effective descriptions on the macro-scale.
Two classes of microscopic processes have been used extensively: (i) fitness-based processes in which an individual chosen proportional to fitness reproduces and the offspring replaces a randomly chosen individual [11]; (ii) pairwise comparison processes in which a pair of individuals is chosen, and where subsequently one of these individuals may adopt the strategy of the other. This adoption occurs with a probability that depends on the payoff of both individuals, such that better players are more likely to be imitated than those who do worse [12,13]. The payoff can be taken as the growth rate, often referred to as Malthusian fitness [14], and the fitness can be taken as the average number of offspring, often called Wrightian fitness [14]. Thus the payoff-to-fitness mapping used in the context of fitness-based processes can be interpreted as a transformation between Malthusian fitness and Wrightian fitness [14,15].
Fitness-based processes are typically applied to model biological evolution. Also pairwise comparison processes can describe biological evolution by reinterpreting the process: an individual is randomly selected and it produces an identical offspring to replace another randomly chosen individual. The probability of the reproduction and the replacement depends on the payoffs of both individuals. The first individual will be more likely to reproduce and replace the second one if it has a higher payoff. It is noteworthy that the reproduction and replacement is a single event in pairwise comparison processes. In fitness-based processes, these are two independent events.
In both types of processes, the relative influence of the game is controlled by an external parameter, the socalled intensity of selection β. This parameter has strong parallels to the inverse temperature in statistical mechanics [16]. In populations of size N the dynamics is dominated by the evolutionary game for strong selection, β ≫ N 1, with demographic noise only affecting the outcome weakly. For weak selection, β ≪ N 1, the dynamics is largely stochastic, with only a small influence of the game on the evolution of the system. The outcome of evolutionary game dynamics thus depends on the interplay between selection and noise, both changing with the relative abundance of the types of individuals in the population.
In well-mixed populations and on some special networks (e.g., on a ring), the evolutionary dynamics between two types of strategies, wild-type and mutant, can be described by simple birth-death processes. In such processes, the state of the system is characterized by the number of mutants alone, and the mutants change by at most one in number for each time step. A quantity that is of particular interest in evolutionary biology is the fixation probability, which is the likelihood that a mutant type takes over the entire population [17,18]. It is the basis of the definition of evolutionary stable strategies in finite populations [11]. It also features in the leadingorder of the stationary distribution for small mutation rates, which serves as a powerful analytical description when multiple strategies are present in the population [19,20].
The choice of a fitness-based process versus a pairwise comparison process is typically not further justified in the literature [21,22]. Often the type of model employed is chosen arbitrarily. This is usually no cause for concern, as many results do not seem to depend on the particular choice of the microscopic process. In particular, a wide class of microscopic processes leads to similar results under weak selection [23,24]. This equivalence is, however, only partial, and in some cases the outcome on the macro-scale can crucially depend on the specific choices made at the microscopic level [25]. Here, we show the choice of a fitness-based versus a pairwise comparison process is restricted to a unique pair if we require that for an arbitrary game, the two processes lead to identical fixation probabilities for all intensities of selection β. This indicates that the choice of the microscopic process can make a difference even in unstructured populations.

Two evolutionary process classes and their non-equivalence in fixation probability
We consider well-mixed populations with fixed size N. Each individual can be of one of two types, A and B. The state of the population is thus characterized by the number i of individuals of type A. The interaction between the two types of individuals is described by the functions π i A and π i B . These indicate the expected payoff for two types in a population in state i. The interaction can be generated from a two-player matrix game which leads to payoffs linear in i [26], but we keep the formalism general to include games played between an arbitrary number of players, which leads to payoffs that depend on i in a polynomial way [27][28][29][30]. In fact, our results hold for an arbitrary dependence of the payoffs on i.
A discrete-time birth-death process on the set of states = … i N 0, , is characterized by the transition probabilities ± T i that the system moves to state ± i 1 in the next step, when it is currently in state i. . The population can never escape from homogenous states. In biology this corresponds to the absence of mutation, where extinct types cannot be re-introduced.

Fitness-based processes and pairwise comparison processes
We will now characterize fitness-based processes and pairwise comparison processes in more detail. For a given game, i.e. for payoff functions π i A and π i B , a fitness-based process assumes that at each time step an individual is selected for reproduction with a probability proportional to its fitness. This individual produces one identical offspring which replaces a randomly chosen individual in the population. Consequently, the transition probabilities are of the form The subscript 'F' indicates a fitness-based process. We have assumed that the payoffs π A i and π i B translate into reproductive fitness via a mapping βπ . The transition probabilities in equation (1) are then fully specified by the underlying game and by the payoff-to-fitness mapping f.
In a pairwise comparison process, one focal individual and a role model are chosen at random at each time step. The payoff difference between the two individuals determines the probability that the focal individual adopts the strategy of the role model. Specifically, for a focal individual of type A and a role model of type B, this probability is β π where β > 0 is again the intensity of selection. If the focal individual is of type B and the role model of type A this probability is β π must be positive to ensure it is more likely to adopt successful strategies. For a given game and a given adaptation function g this leads to a birth-death process with the transition probabilities The subscript 'I' indicates a pairwise comparison process. For both classes of processes, and for any game, the dynamics will eventually reach one of the two absorbing states: either the mutant goes extinct (absorption at i = 0), or it reaches fixation (i = N). The so-called fixation probability, ϕ 1 , measures how likely it is that a single mutant takes over the entire population, i.e. it is the probability for the system to end up in i = N, if initialised at i = 1. For general birth-death processes this probability is given by [11,17,18]

Non-equivalence between fitness-based processes and pairwise comparison processes in fixation probability
For weak selection, many fitness-based processes and pairwise comparison processes are similar in the fixation probability [23,31,32]. For strong selection, however, they can be qualitatively different (see figure 1). For example, the fixation probability for a fitness-based process with linear payoff-to-fitness mapping can converge to any positive value between zero and one in the limit of strong selection, β → ∞. To see this, we construct the following example: let π = 1 , the denominator will diverge resulting in a fixation probability of zero in , denote j as the number of terms in the sum The denominator is then a sum of + j 1 terms with value one, with the rest of the terms vanishing for β → ∞. This leads to the fixation probability in the strong selection to be + j 1 ( 1). Note that the integer j can range from 0 to − N 1, such that the fixation probability can only take values ⋯ N 1 , , 1 2 and 1 in this case.
Given, that for β → ∞ the fixation probability for the fitness-based process can take any value between 0 and 1, whereas the fixation probability for the pairwise comparison process reaches only discrete values, the behavior of the two processes has to be qualitatively different in terms of the fixation probability.

Equivalence between two evolutionary process classes in fixation probability
Realizing that the fixation probability can be sensitive to the evolutionary process and the selection intensity, we ask the following question: for what choices of the payoff-to-fitness mapping f, and of the imitation function g, do the resulting fitness-based and pairwise comparison processes have the same fixation probability ϕ ϕ = F I , for arbitrary games and intensities of selection? In other words, if we require that the two processes are equivalent in fixation probability for any game and any selection intensity, how do we need to choose the fitness and imitation functions?
3.1. Equivalence between payoff-difference based pairwise comparison processes and fitness-based processes From equations (1) and (2) , we note that A for the fitness-based process, and β π π βπ for the pairwise comparison process. If the functions f and g fulfill for all x y , , we have = i for all i. Using equation (3) this leads to equal fixation probabilities for all games and any selection intensity. Thus, equation (4) is sufficient.
We now show that the condition in equation (4) is also necessary. We are interested in functions f and g such that the equality of fixation probability holds for all games. Therefore, the fixation probabilities need to be equal for games with constant payoffs π π = constants π A and π B can be chosen arbitrarily, as the selection intensity β. The fact that we require γ γ = F I leads to the conclusion that f and g must fulfil equation (4). Hence, the condition in equation (4) is necessary if we require identity of fixation probabilities for all possible games. We stress three points: (i) it may well be possible to construct a game and a pair of functions f and g, which are not of the above form, such that the fixation probabilities of the two resulting processes coincide for this particular game.
However, unless f and g fulfil equation (4), the identity of fixation probabilities will not hold for arbitrary games, as our argument above shows; (ii) the proof does not assume the two-strategy game to be a pairwise matrix game. In fact, it holds for twostrategy multiplayer games and even for games with an arbitrary payoff dependence on i; (iii) equation (4) is also the necessary and sufficient condition such that the fixation probability of m-strategy A mutants ϕ m < ⩽ − m N (1 1) is identical for fitness-based and pairwise comparison processes for any game and any selection intensity (for a proof see appendix A).
Equation (4) implies that the ratio f x f y ( ) ( )has to be a function of the difference − x yalone. Setting Δ = + y x x in = − − f x f y g x y g y x ( ) ( ) ( ) ( )and taking the limit Δ → x 0 leads to the differential equation We note that this differential equation must hold for all x. It is a necessary condition for the equality of fixation probabilities for arbitrary games and arbitrary strength of selection, but it is not a sufficient condition by itself. A necessary and sufficient condition is given by equation (4). We observe that the condition of equation (5) can be relaxed if we limit the equality of fixation probabilities to the weak-selection approximation. It corresponds to expanding the fixation probabilities to linear order in the selection intensity. If we require that f and g lead to identical fixation probabilities only in the linear-order term in β (but not necessarily to higher order) for any payoff functions π i A and π i B we obtain the condition This condition is far less restrictive than equation (5), and it is both necessary and sufficient to have identity of fixation probability for all games up to linear order in β. This can be seen from existing results for weak selection [32]. The only solution of the more restrictive condition, equation (5), is This implies that in order for the fixation probabilities of a fitness-based process to be identical to those of a pairwise comparison process (to any order in the selection intensity), it is necessary that the payoff-to-fitness mapping f(x) is exponential in x, where λ is an arbitrary positive constant. The imitation function g is at this point largely unconstrained, although one finds − = λ g x g x ( ) ( ) e x by setting y = 0 in equation (4). With the additional assumption + − = g x g x ( ) ( ) 1, only a single possible imitation functions remains, the so-called Fermi function We have thus shown that the assumption of equal fixation probabilities for all games together with the mild assumption + − = g x g x ( ) ( ) 1fully restricts the payoff-to-fitness mapping and the imitation function to = The only remaining free parameters are f (0) and the constant λ. However, the choice of f (0) is immaterial as f (0) drops out in equation (1). The constant λ on the other hand can effectively be absorbed in the selection strength, β, so that, to all intents and purposes, our constraints fully specify the payoff-to-fitness mapping and the imitation function. Thus, this pair of processes is unique (see figure 2) and, if chosen otherwise, the precise details of the microscopic model will affect the outcome of the model on the macroscopic level. For example, the popular linear payoff-to-fitness mapping βπ = + f 1 has no corresponding imitation function which depends on payoff differences only and which leads to the same fixation probability for arbitrary games. This is illustrated in figure 3.
The allowed set of imitation functions becomes broader if we relax the constraint and allow functions g with + − ≠ g x g x ( ) ( ) 1. Any imitation function of the form x is permissible as long as the resulting g(x) is increasing, takes values between 0 and 1 (such that it is a probability), and as long as h(x) is even are increasing with selection intensity. Here x is proportional to the selection intensity. Beyond weak selection, however, the fixation probability of the pairwise comparison process decreases to zero with increasing selection intensity; whereas that of the fitness-based process increases to a limit. This example shows that the fitness-based process and pairwise comparison process can differ qualitatively, if selection is non-weak. Here, the payoffs are , where i is the number of A individuals and the population size N is 10. Figure 2. There is a unique pair of a fitness-based process and a pairwise comparison process which are identical in fixation probability for all games and all intensities of selection. The fixation probabilities for many fitness-based process and pairwise comparison processes are similar under weak selection, but in general they are not. If we require that the fixation probability is identical for any selection intensity and any two-strategy game, we arrive at the unique pair: the payoff-to-fitness mapping has to be exponential; while the imitation function must be a Fermi function. Here, the imitation functions are under the constraint of No other pair can lead to identical fixation probabilities for any game and any selection intensity. In other words, for any other pair of a fitness based and a pairwise comparison process, there exists a 2 × 2 game and a selection intensity leading to differences in the fixation probability.
(to ensure To show that such functions h(x) exist, we mention two arbitrary examples,

Equivalence between general pairwise comparison processes and fitness-based processes
We now consider more general pairwise comparison processes in which the imitation probability does not depend on payoff differences alone. Specifically, we allow imitation probabilities with which a focal individual with payoff π foc imitates the strategy of a role model with payoff π rm of the form βπ βπ Q ( , ) foc rm , i.e. Q may depend on the payoffs of both individuals explicitly. The previous case is recovered as = − Q x y g y x ( , ) ( ). To guarantee that the resulting imitation function Q x y ( , )is a probability, it has to take values between 0 and 1. In addition, we require ∂ < Q x y ( , ) 0 x , such that focal individuals with high payoff are less likely to adopt the strategies of others, and ∂ > Q x y ( , ) 0 y , such that role models with higher payoff are more likely to be imitated than those with a low payoff. In this more general case, a fitness-based process with payoff-to-fitness mapping f has the same fixation probability of a single mutant as a pairwise comparison process if in analogy to equation (4). Setting Δ = + y x x and taking the limit Δ → x 0 in equation (8) leads to the necessary condition where x 0 ⎡ ⎣ ⎢ ⎤ ⎦ ⎥ Condition (10) admits payoff-to-fitness mappings f (x) that are not exponential. The constraint that f(x) must be exponential in x derived under the more restrictive assumptions above is a specific consequence of the requirement that the imitation probability depends on payoff differences only. For any given payoff-to-fitness mapping f(x) which is increasing and positive, the function = + Q x y f y f x f y ( , ) ( ) ( ( ) ( ))proposed in [11] is decreasing in x and increasing in y and takes values between 0 and 1. In other words, it fulfills the constraints of an imitation function. Thus, for any payoff-to-fitness mapping, f(x), there is an imitation kernel Q x y ( , )leading to equal fixation probabilities for all games. Restricting the set of permissible kernels to those of the form where ψ x ( )is an increasing function fully specifies the imitation kernel. A short calculation shows that the imitation function is then the only possible imitation choice leading to identical fixation probabilities for all games for a given payoff-to-fitness mapping (see appendix B

Summary
In summary we have challenged some of the key assumptions frequently made in modeling evolutionary dynamics. Fitness-based and pairwise comparison processes are often used as if these approaches were entirely exchangeable. This is appropriate-to a certain extent-when fitness is a positive constant as it is the case in many models of classical population genetics. But in the case of evolutionary games, the choice of the microscopic details of the process does make a difference for the macroscopic outcome of frequency-dependent selection outside the regime of weak selection. As we have shown there are then strong restrictions on the choice of the imitation function and the payoff-to-fitness mapping if one requires that the fixation probabilities in the two classes of processes are identical for any intensity of selection. Furthermore these strong restrictions cannot be relaxed even when the pairwise comparison process class is broadened. These challenges are largely absent in population genetics, where selection is typically constant, and only arise in evolutionary game theory, where selection is frequency dependent.
On the one hand, it is of interest to study when these strict restrictions can be relaxed. One of the possibilities is to further broaden the pairwise comparison process class. There are only two individuals in pairwise comparison processes, a focal individual and a role model. Consequently, the imitation function has at most two arguments. If we allow multiple comparison rules, i.e., k role models, the imitation function could have up to + k 1 arguments. On the other hand, it is also of interest to study situations where these restrictions could be even stronger. In evolutionary games on graphs, a dependence on the microscopic details has been pointed out repeatedly [2][3][4]33]. It is noteworthy that these difficulties are already present in non-spatial well-mixed systems of the type that we have discussed. The complexity of a network structure is therefore not a necessary component. Indeed, we expect it to be much more challenging to construct two processes with identical outcomes on such more complicated geometries in our context. An alternative approach to analyze this problem in island structured populations can be found in [34], but these population structures have so far received little attention among physicists working in this field.

Acknowledgment
We thank Benjamin Werner for helpful discussions.

Appendix A. Fixation probability of m mutants
The fixation probability that m strategy A mutants take over the whole population, ϕ m , is given by [17,18]