Selection pressure transforms the nature of social dilemmas in adaptive networks

We have studied the evolution of cooperation in structured populations whose topology coevolves with the game strategies of the individuals. Strategy evolution proceeds according to an update rule with a free parameter, which measures the selection pressure. We explore how this parameter affects the interplay between network dynamics and strategy dynamics. A dynamical network topology can influence the strategy dynamics in two ways: (i) by modifying the expected payoff associated with each strategy and (ii) by reshaping the imitation network that underlies the evolutionary process. We show here that the selection pressure tunes the relative contribution of each of these two forces to the final outcome of strategy evolution. The dynamics of the imitation network plays only a minor role under strong selection, but becomes the dominant force under weak selection. We demonstrate how these findings constitute a mechanism supporting cooperative behavior.

3 according to his behavior. Such a coevolutionary interplay between strategy evolution and graph evolution may give rise to correlations between the strategies of connected individuals. The strategy of an individual will therefore influence not only the payoff he acquires, but also the probability that he will be considered as a potential role model by someone else. When some strategies are more popular than others, the rate of strategy update may effectively change, depending on the intensity of selection.
We study this issue in the framework of active linking (AL) dynamics [55,56,62,65,72], a model that allows us to study cooperation in adaptive networks analytically in certain limits. Section 2 introduces both our model and the framework in which we will make our analysis.
In section 3, we analyze the gradient of selection as a function of the intensity of selection, assuming that the network topology evolves much faster than the individual behavior. Section 4 presents the conclusions.

A minimal model
Consider a finite population of N individuals interacting in symmetric, one-shot, two-player games of cooperation defined by the payoff matrix We distinguish two possible game strategies: cooperate unconditionally (C) and defect unconditionally (D). Payoff matrix 1 shows that individuals receive a reward R upon mutual cooperation and a punishment P upon mutual defection. When a C meets a D, the C receives the sucker's payoff S, whereas the D acquires the temptation to defect T .
Individuals do not interact with everyone in the population. Instead, a network indicates who meets whom. The structure of this network is dynamic, in the sense that edges appear and disappear over time. Simultaneously, individuals may reconsider their game play. The network dynamics proceed on a characteristic time scale τ a , the strategy dynamics on another time scale τ s . Below, we define each of these two dynamical processes separately.
We use the AL model, developed earlier by Pacheco et al [55,56], to define the network's evolution. Each individual has a propensity α to engage in new interactions, such that new edges are formed at a rate α 2 . The lifetime of existing edges depends on the behavior of the individuals connected by this link. Specifically, the rate at which CC-links, CD-links and DD-links disappear is given by γ CC , γ CD and γ DD , respectively.
Consider a network with k Cs and N − k Ds. The number of CC-links, CD-links and DD-links in such a network can never exceed , respectively. Under the assumption that the individuals stick to their game behavior, for a large number of links, we can describe the time evolution of the number of links of each type using the following ordinary differential equations, where L i j (t) denotes the actual number of links at time t between individuals adopting strategy i and individuals adopting strategy j (i, j ∈ {C, D}). In the steady state, the number of links of each type is given by where φ i j = α 2 (α 2 + γ i j ) −1 denotes the fraction of active ij-links (i, j ∈ {C, D}). Note that the stationary configuration of the network depends on the actual strategy configuration of the population, illustrating the interplay between network evolution and strategy evolution. The second dynamical process in our model, the strategy dynamics in finite populations, is defined by the pairwise-comparison rule [18,69]. At every strategy update event, a randomly selected individual X imitates a random neighbor Y with probability The individual Y can be regarded as the role model of X . X ( Y ) denotes the total payoff X (Y ) receives after interacting once with every neighbor. The parameter β ( 0) controls the intensity of selection. When β is large, the imitation process is driven mainly by the payoff values that the individuals acquire. The game becomes progressively less important for decreasing β.
In the following, we assume that the network dynamics proceed much faster than the strategy dynamics (τ a τ s ). In this limit, the network always reaches a stationary configuration before a strategy update event occurs. The expected payoff of Cs and Ds during strategy update events is therefore given by * These payoff values correspond to those obtained in a well-mixed population (complete network) with the same strategy configuration, but using the following rescaled payoff matrix [55,56], The network dynamics not only affect the payoffs individuals acquire but also influence the imitation process. Some individuals will act more often as a role model than others, depending on their strategy. In section 3, we show that such differences can have a profound effect on the strategy dynamics. If τ a τ s , a D will select a C as his role model with probability The term in the numerator corresponds to the average number of CD-links of each D, whereas the term in the denominator reflects the average total number of links of each D. The overall probability that the number of Cs will increase during a strategy update event equals 5 Similarly, the number of Cs will decrease with probability In well-mixed populations (φ CC = φ CD = φ DD = 1), the difference [69] can be regarded as a finite population analogue of the gradient of selection associated with the replicator equation in infinite, well-mixed populations [2], which is defined , where x ∈ [0, 1] stands for the fraction of cooperators. In both cases, Cs are favored over Ds when g(k) > 0 (ẋ > 0), whereas the opposite is true whenever It is noteworthy that we consider the evolutionary dynamics as a discrete stochastic system while assuming that the number of links is sufficiently large, so that the linking dynamics can be described by a set of ordinary differential equations (see equation (2)). Stochastic effects could be included at the level of the linking dynamics as well, for instance by adopting a discrete version of the AL model like the one proposed recently by Wu et al [65]. Nevertheless, the continuous approximation we use here does not weaken the robustness of our conclusions, as shown in the following section by means of computer simulations (see figure 2(d)).

Results and discussion
We investigate how the gradient of selection depends on the selection pressure. To do so, one can study the shape of g(k) or, alternatively, that of the ratio h(k) = T + k /T − k , which is given by As the solutions of g(x) = 0 are sometimes more conveniently obtained solving h(x) = 1, we use both interchangeably. For large N , h(x) can be approximated by Let us start by studying the two limiting cases: the weak selection limit (β → 0) and the strong selection limit (β → ∞).
In the limit of strong selection, the direction of the gradient of selection depends solelyapart from finite size effects-on the sign of u x + v (see appendix A). g(x) will therefore exhibit one of four possible shapes (see figure 1), depending on the ordering of the payoff values of the transformed game M . When R > T and S > P , g(x) is positive for all x ∈ ]0, 1[. This scenario is known as C-dominance. Selection will always favor Cs over Ds, irrespective of the strategy configuration of the population. When R < T and S < P , we obtain the opposite scenario: D-dominance. The gradient of selection now satisfies g(x) < 0 for all x ∈ ]0, 1[, implying that Ds are always favored over Cs. When R > T and S < P , g(x) has a root in Furthermore, g(x) < 0 for x < x * and g(x) > 0 for x > x * . When the initial fraction of Cs is smaller than x * , evolution favors Cs. Otherwise, Ds will be favored. This is an example of coordination or bistability. Finally, when R < T and S > P , evolution favors a mixture of Cs and Ds. The gradient g(x) has a root in x * again, but unlike before, selection favors Cs for x < x * and Ds for x > x * . Note that one has to correct for self-interactions when computing the finite population analogue of the equilibrium point x * . The resulting equilibrium k * is given by In the other limit, that of weak selection, we have Because this function is monotonic in x, h(x) = 1 can have at most one solution and consequently the gradient g(x) can have at most one root in ]0, 1[. Hence, g(x) exhibits one of the four shapes shown in figure 1. The particular type of shape that occurs depends on the parameters a and b.
In the following, we assume that links between Cs satisfy both individuals and therefore last longer than other links. Two interacting Ds, on the other hand, are both dissatisfied and prefer to break their connection quickly. Following the same reasoning, Cs would like to break CD-links quickly, whereas Ds prefer to keep such links as long as possible. Hence, on average, CD-links will have a longer time span than DD-links, but a shorter one than CC-links. Altogether, we obtain the following ordering for the fractions of active links of each type, which is equivalent to This condition ensures that h(x) > 1 for all x ∈ ]0, 1[. Hence, Cs are dominant in the limit of weak selection, irrespective of the game being played.
Having considered the two limiting cases, we now study the gradient of selection for general values of β. In appendix B, we show that g(x) can have either zero, one or two internal equilibria, depending on the parameter settings. First we discuss examples where the specific intensity of selection leads to a scenario with at most one internal equilibrium. Next we will see that some parameter combinations lead to a scenario with two internal equilibria, which can never occur in the limits of either weak or strong selection.
We reduce the complexity of the game space by normalizing the difference between R and P to 1, taking R = 2 and P = 1. The contours in figure 2 depict the sign of g(k) for three different D-dominance games, using α = 0.8, γ CC = 0.4, γ CD = 0.5 and γ DD = 0.6. In the weak selection limit, these birth/death rates of links lead to dominance of Cs, irrespective of the original payoff matrix. The strong selection limit leads to different scenarios, depending on the original game. For instance, S = 0.5 and T = 2.1 results in bistability (see figure 2(a), S = 0.9 and T = 2.5 in coexistence of Cs and Ds (see figure 2(b)), and S = 0.5 and T = 2.5 in D-dominance (see figure 2(c)). Hence, there exist critical values of β at which the number of internal equilibria changes. We can compute these critical values using the derivative of g(x) in 0 and in 1 (see also figure 1). These derivatives are given bẏ Assuming at most one internal equilibrium, we obtain C-dominance (D-dominance) when botḣ g(0) andġ(1) are larger (smaller) than zero. Bistability occurs whenġ(0) < 0 andġ(1) > 0, and coexistence whenġ(0) > 0 andġ(1) < 0. From equation (18), it follows thatġ(0) > 0 if and only if and thatġ(1) > 0 if and only if This means thatġ(0) will be positive for β smaller than and negative otherwise. Similarly,ġ(1) is positive for β smaller than and negative otherwise. Both β 0 and β 1 are indicated by vertical dotted lines in figure 2. They correspond clearly to the values of β at which each of the game transitions takes place. We further confirm the validity of the analytical results by means of computer simulations. Figure 2(d) shows the probability, as a function of β, that a population with initially 50% cooperators evolves to full cooperation. Circles, representing simulation results, fit nicely with the analytical predictions, which are indicated by a solid line. Each simulation starts with a complete network of size N = 100, 50% of them being Cs, and runs until the population fixates into either full cooperation or full defection. Strategy and network structure evolve simultaneously under asynchronous updating. Strategy update events take place with probability (1 + τ ) −1 , where τ = τ a τ s = 10 −3 ; network update events occur otherwise. We run 10 4 simulations for each value of β and plot the fraction of runs that end in full cooperation. The analytical predictions are calculated using the following formula [69], with i being equal to 50. The expressions for T + j and T − j are given by equations (8) and (9). Earlier in this section, we showed that there can be at most one internal equilibrium for the limiting cases β → 0 and β → ∞. At intermediate intensities of selection, two internal equilibria can, however, occur simultaneously (see also appendix B). Figure 3 shows two examples where this is the case. The upper two panels correspond to a scenario where Cs are expected to disappear from the population in case they are rare. If a sufficiently large fraction of Cs is present, however, a mixture of Cs and Ds is favored. The lower two panels illustrate an example of the opposite scenario, in which Ds are expected to go extinct when they are rare.

Conclusion
In this paper, we have explored, both analytically and numerically, the evolution of cooperation in dynamical networks that evolve side by side with individuals' behavior, the evolution of the network structure being affected by the dynamics of the individuals and vice versa. We indicate that the adaptive nature of the network of contacts affects both the average payoff associated with each game strategy and the likelihood for each of the different strategies to serve as a role model. The intensity of selection, which controls the contribution of game payoff to fitness, regulates the importance of each of these two effects in the final outcome of strategy evolution. When the intensity of selection is strong, the payoff transformation resulting from the network dynamics provides the dominating contribution to evolution. Weakening the intensity of selection enhances the effect of the adaptive imitation network. By doing so, one is able to effectively transform a D-dominance dilemma into any of the conventional 2 × 2 symmetric games. We derive analytical conditions, valid under the assumption that there is only one internal equilibrium, that predict which range of intensities of selection leads to which game scenario. We show also that certain parameter combinations may lead to the occurrence of two internal equilibria at intermediate intensities of selection. Our results clearly demonstrate the relevance of the intensity of selection when evolution proceeds on an adaptive structured population. Furthermore, we indicate the conditions under which this coupled dynamics work as an efficient mechanism for the promotion of cooperative behavior. The equilibrium closest to full defection is unstable and the other one is stable. This is emphasized in (b), where we zoom in on the actual gradient g(k) for β = 1.0. Panels (c) and (d) show the same plots, but using R = 2, S = −8, T = 3, P = 1, α = 0.02, γ CC = 10 −5 , γ CD = 6 * 10 −4 , γ DD = 0.9 and N = 100. The equilibrium closest to full defection is now stable, while the other one is unstable.

Appendix A. The strong selection limit
The following inequality holds, where K > 0 is a constant, whose specific value depends on the parameters a and b. Therefore, Hence, the direction of the gradient of selection depends solely on the sign of u x + v . Cs are favored over Ds if and only if u x + v > 0.

Appendix B. General intensities of selection
Let us designate the rational factor in equation (12) by h 1 (x) and the exponential factor by h 2 (x). Both h 1 (x) and h 2 (x) are monotonic in x: • The derivative d dx h 1 (x)is given by b−a 2 ((a−b)x+b) 2 . Hence, h 1 (x) increases monotonically if b > a 2 and decreases monotonically if b < a 2 .
• The derivative d dx h 2 (x)is given by u e u x+v . Hence, h 2 (x) increases monotonically if u > 0 and decreases monotonically if u < 0.