Motion depending on the strategies of players enhances cooperation in a co-evolutionary prisoner's dilemma game

In the evolution of cooperation, the motion of players plays an important role. In this paper, we incorporate, into an evolutionary prisoner dilemma's game on networks, a new factor that cooperators and defectors move with different probabilities. By investigating the dependence of the cooperator frequency on the moving probabilities of cooperators and defectors, μc and μd, we find that cooperation is greatly enhanced in the parameter regime of μc<μd. The snapshots of strategy pattern and the evolutions of cooperator clusters and defector clusters reveal that either the fast motion of defectors or the slow motion of cooperators always favors the formation of large cooperator clusters. The model is investigated on different types of networks such as square lattices, Erdös–Rényi networks and scale-free networks and with different types of strategy-updating rules such as the richest-following rule and the Fermi rule. The numerical results show that the observed phenomena are robust to different networks and to different strategy-updating rules.


Introduction
How does cooperation emerge from a population of selfish individuals? This is a long-standing problem existing in many real systems, such as social, biological, economic and ecological systems. Understanding the mechanisms of the emergence of cooperation among a population of selfish individuals has drawn growing interest in recent years. Since the pioneering work on iterated games by Axelrod [1], the evolutionary prisoner's dilemma game (PDG) has been a general metaphor for studying cooperative behaviors. In PDG, each player of a pair independently chooses either cooperation or defection as her strategy. Both players receive R under mutual cooperation and P under mutual defection, while a cooperator receives S when playing with a defector, which in turn receives T . The inequalities T > R > P > S and 2R > T + S define the classical PDG. Under these conditions, a rational, self-interested agent has no motivation to cooperate because the payoffs for defecting strictly dominate those for cooperating regardless of her co player's choice. Following common practice [2], payoffs are normalized by taking R = 1 and P = S = 0, and the remaining game parameter is the temptation to the defector T = b > 1. (Actually, the case is termed as weak PDG and we designate it by PDG throughout the paper.) Since the seminal work of Nowak and May [2], evolutionary PDG in spatial networks such as square lattices [3]- [5], regular random networks [6], small-world networks [7]- [9] and scale-free networks [10]- [14] has been an important mechanism for explaining the emergence of cooperation in a population of selfish individuals. In particular, scale-free networks in which individuals' degrees exhibit high heterogeneity have a strong effect on facilitating the evolution of cooperation. However, in the real world, the environment of players, such as network structures, the properties of players and the players' strategy patterns, affect each other. Since the work by Zimmermann et al [15], more and more researchers have been paying attention to co-evolutionary PDG. Co-evolution involves the evolution of the links between players [16]- [23], the teaching ability (reproduction capability) of players [24]- [27] and other factors [28]- [30]. Another important co-evolutionary factor is the motion of players. In [31], Vainstein et al first introduced empty sites as a disordering factor into a square lattice and found that the cooperation in the population may be enhanced. They extended the model by allowing the movement of players to empty neighboring sites with a certain probability [32]. The results of that work showed that the motion of players could further promote cooperation. Furthermore, Helbing et al [33,34] introduced success-driven migration as a specific mechanism leading toward the outbreak of cooperation in a population of selfish and unrelated individuals even under noisy conditions. In the above-mentioned literature, the authors assigned the same mobility to all players. Droz et al [35] introduced influential players to a population and found 3 that a higher level of cooperation can be achieved if the influential players are allowed to move with an optimal rate. Yang et al [36] proposed an aspiration-induced migration in which individuals would migrate to new sites provided that their payoffs are below some aspiration level. They found that the optimal cooperative behavior appears at a moderate aspiration level. Motivated by this work, we introduce a new mechanism where cooperators and defectors are distinguished in motion. As is well known, cooperators and defectors behave differently in the evolution of cooperation in evolutionary PDG. Cooperators usually form clusters to resist the invasion of defectors, whereas defectors tend to damage those cooperators' clusters to lower cooperation. Hence, the motion of cooperators and defectors would have varying influences on the evolution of cooperation. In this paper, we investigate the effects of motion depending on the strategies of players (MDSP) on cooperation in evolutionary PDG on square lattices with the richest-following strategy updating rule (learn from the best) [37]- [39]. The results of this work show that cooperation is strongly promoted when cooperators and defectors move with appropriate but different probabilities. In addition, we extend the model from square lattices to Erdös-Rényi (ER) networks and scale-free networks suggested by Barabási and Albert [40] and to the evolutionary PDG with the Fermi-type strategy updating rule.

The model
We consider an evolutionary PDG on an N = 100 × 100 square lattice with periodic boundary conditions. Each site may be occupied by a player or not. We define the density of players, ρ = (n c + n d )/N , where n c is the number of cooperators and n d is the number of defectors. Initially, empty sites and players are distributed randomly and a player is designated as a cooperator or defector with equal percentage. Each Monte Carlo (MC) simulation procedure has three elementary steps. Firstly, a randomly selected player interacts with her nearest neighbors to accumulate her payoff; meanwhile, her nearest neighbors acquire their payoffs by interacting with their nearest neighbors. Secondly, the selected player updates her strategy according to the richest-following rule. Thirdly, the selected player moves to a site chosen randomly within her four nearest neighbors with a certain probability if the site is empty. If the selected player is a cooperator, the probability of moving is µ c ; otherwise the probability is µ d . We characterize the cooperation with the frequency of cooperators, F c , which is defined as F c = n c /(n c + n d ) and is calculated by averaging over 1 × 10 8 MC steps after sufficiently long transients. The system is updated asynchronously.

Numerical results
In figure 1(a), F c is plotted as a function of ρ for three combinations of µ c and µ d . In the case µ c = µ d = 0, F c approximately equals initial values for extremely low ρ due to rare interactions between players distributed sparsely on square lattices. F c decreases as ρ increases from 0.1 to 0.4, because the chances of players getting encountered increase, yet cooperators are difficult to form compact clusters. For higher ρ (e.g. ρ > 0.4), the tendency for forming cooperator clusters increases and F c starts to increase. In the case µ c = µ d = 0.01, which was studied in [32], cooperation is promoted over the range of 0.6 < ρ < 1. Surprisingly, when cooperators and defectors move with different probabilities, cooperation is enhanced dramatically. For example, in the case µ c = 0.001 and µ d = 0.01, the optimal F c almost reaches 1. Moreover, the optimal regime of ρ, where the cooperation level is higher than that in the case µ c = µ d = 0, is extended  to 0.25 < ρ < 1 from 0.6 < ρ < 1 (µ c = µ d = 0.01). Figure 1(b) shows the dependence of F c on b. With an increase of b, F c displays several discontinuous transitions that are rooted in the nature of the richest-following rule [41,42]. F c drops sharply to zero at b = 1.5 for nonzero µ c and µ d , which was observed in [33,36], and a detailed analysis was made in [33].
To further investigate the effects of µ c and µ d on the evolution of cooperation, the contour graph of F c on µ c and µ d in evolutionary PDG is plotted in figure 2(a). Compared with the case of µ c = µ d = 0, cooperation is promoted in a large area of the µ c -µ d space, provided  Figure 3(a), where µ c = µ d = 0, shows that cooperators form many small clusters distributed uniformly on the lattice against the invasion of defectors. And those cooperator clusters (C-clusters) are surrounded by empty sites that could protect cooperators from the invasion of defectors and help cooperators to survive. The snapshots of strategy pattern at different MC steps in figures 3(b)-(d) show that, as µ d increases from 0.01, some defectors exchange positions with the empty sites close to the edges of C-clusters. Each of these defectors has more empty neighbors and yet only a few cooperative neighbors. As a result, her payoff is lower than that of her cooperative neighbors, who receive support from C-clusters, which leads to a strategy conversion from defection to cooperation. With an increase of µ d , more defectors are turned into cooperators and C-clusters become larger. Hence, in most of the given parameter space, the faster the defectors move, the better the cooperation. Figure 4(b), where µ c is small (µ c = 0.001), shows that the motion of cooperators tends to join small C-clusters together and form large C-clusters, which causes F c to increase dramatically. However, further increasing µ c , the motion of cooperators produces adverse effects on cooperation: the large C-clusters are damaged and break up into many small ones. In addition, cooperators would move into the sea of defectors and be swallowed easily. As shown in figure 4(c), where µ c = 0.1, the strategy pattern includes many small C-clusters and F c obviously decreases compared with that in the case µ c = 0.001. When µ c = 0.2 (figure 4(d)), cooperators move so fast that no C-clusters can be sustained. As a result, cooperators are almost extinct, and defectors prevail over the whole system. Figure 5 shows snapshots of strategy pattern for two systems with equivalent values of F c , b and ρ: a motionless one (µ c = µ d = 0) and one with motion (µ c = 0.1, µ d = 0.005). Although F c is the same in the two snapshots, the strategy patterns are completely different: in the motionless system, cooperators exist in the form of many small clusters, whereas when appropriate motion is introduced, cooperators form large clusters. Therefore, the two graphs give us an intuitive understanding of how the MDSP facilitates cooperation in a population of selfish individuals.
The above work is performed on regular square lattices, but networks in the real world are heterogeneous rather than regular [43,44]. Here, we investigate whether the above results are applicable to heterogeneous networks by exploring this model on ER networks and scalefree networks, both of which have 3000 sites and average degree k = 4. In simulations of scale-free networks, we set b = 2.5 because F c would almost completely reach 1 on the whole µ c -µ d space if b is small, whereas the effects of motion on cooperation would not be obvious.  In addition, although the case b = 2.5 does not belong to social dilemma, it is more harmful to the survival of cooperators. As shown in figure 6, the contour graphs of F c on the µ c -µ d space for the above two types of networks, especially for ER networks, are similar to those for square lattices. Some difference appears in scale-free networks: when 0.1 < µ c < 0.2, F c decreases with an increase of µ d . The phenomenon is contrary to that for square lattices and ER networks. It is mainly caused by the existence of hubs, which have high degrees and usually tend to be occupied by cooperators, in scale-free networks. When µ c is large, cooperators located on the hubs may move away and the sites of hubs become empty. Then as µ d increases, defectors have more chances of occupying the empty hubs to obtain high payoffs because most of their neighbors are cooperators, which favors the spread of defection strategy; thus F c decreases. Instead, if µ d is small, it is difficult for defectors to occupy the empty hubs, and the situation above may not happen.
To give a microscopic view of the effects of µ c and µ d on cooperation in ER networks and scale-free networks, we study the evolution of C-clusters and D-clusters (defector clusters) with time on both types of networks. For each kind of cluster, two quantities are monitored: the mean size of clusters and the number of clusters. It should be noted that isolated defectors (or cooperators) who have no defector (or cooperator) neighbors are not counted in the statistics of clusters. We consider two parameter sets of (µ c = 0.001, µ d = 0) and (µ c = 0, µ d = 0.2) and the results are presented in figure 7. Clearly, the mean size of C-clusters always increases with time and the number of C-clusters always decreases. On the other hand, the mean size of D-clusters always decreases as time elapses. These observations are consistent with figures 3 and 4: the slow motion of cooperators can promote cooperation by forming large clusters to resist the invasion of defectors and the fast motion of defectors tends to cause themselves to become cooperators. The non-monotonic evolution of the number of D-clusters is closely related to the existence of isolated defectors, which increases the number of D-clusters in the initial stage of evolution.
In the above investigations, we adopt the richest-following rule in the strategy updating process. However, the Fermi rule is another strategy updating rule widely used in evolutionary PDG on networks, which is defined as where K denotes the uncertainty. Now it is interesting to investigate how the Fermi rule impacts the effects of MDSP on cooperation. We set K = 0.4 [45] and present in figures 8(a) and (b) the contour graphs of F c on the µ c -µ d space for square lattices and ER networks. Clearly, the enhancement of cooperation caused by the motion of defectors is prominent. However, the optimal behavior of F c against µ c is absent in contrast to the results for the richest-following rule. One possible reason could be given from figure 8(c). Figure 8(c) shows that F c may reach a very high value even when the motion of defectors is extremely slow. The quick rise of F c with µ d outperforms the possible optimization of cooperation induced by the motion of cooperators. As a result, the optimization of cooperation against µ c is lost. The evolutionary PDG on square lattices displays another interesting feature: the cooperation may be greatly improved due to the motion of cooperators (see figure 8(d)). However, such an improvement is localized at µ d = 0. When µ d increases from zero, the strong improvement of cooperation induced by µ c fades away quickly (see figure 8(a)).

Conclusion
In conclusion, we have studied the effects of MDSP on cooperation in evolutionary PDG. Compared with the work of Vainstein et al [32], in which motion is independent of players' strategies, our model shows that further promotion of cooperation may be induced by MDSP, especially in the parameter space of µ c < µ d . There is an optimal µ c,o that leads to the highest level of cooperation, µ c,o = 0.001 for the strategy updating rule with the richest-following and µ c,o = 0 for the rule with the Fermi type. On the other hand, the motion of defectors enhances cooperation in most of the parameter space we investigated. The snapshots of strategy pattern give an intuitive explanation of the roles of cooperators and defectors in motion. The slow motion of cooperators plays a positive role in forming large C-clusters to resist the invasion of defectors, whereas the fast motion of cooperators destroys C-clusters and deteriorates cooperation.
The findings from this work can be used to explain some recent observations regarding the motion of players. In a recent work [36] where the effects of players' motion induced by aspiration on cooperation were investigated, the authors found that optimal cooperative behavior appears at moderate aspiration level. Based on the present work, optimal behavior of cooperation against aspiration could be explained. A low aspiration level indicates slow motion of cooperators and defectors, whereas a high aspiration level indicates fast motion of cooperators and defectors. In both situations, the positive role of players' motion on cooperation is weak. However, at a moderate aspiration level, defectors tend to move fast, whereas cooperators tend to move slowly, which coincides with the condition of µ c < µ d in our model. Consequently, a strong enhancement of cooperation can be observed at a moderate aspiration level in [36].