Payoff-related migration enhances cooperation in the prisoner's dilemma game

In reality, migration of an individual usually correlates with the individual's financial or social status. Here, we consider the situation where migration of a player depends on the player's payoff to an evolutionary prisoner's dilemma game. When the mobility of a player is positively correlated with the player's normalized payoff, Pi, where the mobility of player i is defined as μi=Piα, we found that cooperation could be promoted strongly in the case of a high density of players because of the introduction of this kind of migration. Moreover, the system could reach a state of complete cooperation in a large region on the given parameter space. Interestingly, enhancement of cooperation shows a non-monotonic behavior with an increase in α. We also found that the positive effects of this kind of migration on cooperation are robust in the face of changes to the network structure and the strategy-updating rule. In addition, we consider another situation where the mobility of a player is anti-correlated with the player's normalized payoff, and we observe that cooperation enhancement still exists.


Introduction
Cooperation is a ubiquitous feature of communities of humans and animals, and the evolutionary prisoner's dilemma game (PDG) [1] is employed frequently to investigate the emergence of cooperation within groups of selfish individuals where cooperation goes against the principle of Darwinian natural selection. Mechanisms ranging from kin-selection [2] and direct reciprocity [3] to indirect reciprocity [4] and group selection [5] have been proposed to explain the emergence of cooperation. Since the pioneering work of Nowak and May [6], the evolutionary PDG in different structured populations, such as square lattices [7]- [9], regular random networks [10], small-world networks [11]- [13] and scale-free (SF) networks [14]- [18], has been intensively studied. In recent years, co-evolution has been regarded as a crucial factor for improving cooperation in the evolutionary PDG [19]. The first work on co-evolution was performed by Zimmermann et al [20,21], who considered the co-evolution between strategy patterns and network structures. Since the initial work, more researchers have considered the co-evolutionary PDG and different co-evolutionary scenarios, such as co-evolution between strategy patterns and network structures [22]- [29], co-evolution between strategy patterns and individual properties, such as the teaching ability (reproduction capability) of players [30]- [33], and the aging of players [34], and co-evolution between strategy patterns and learning rules [35]- [39], among others [40,41], have been studied.
In addition to the factors mentioned in connection with the cited research on co-evolution, the migration of players is another important factor in the co-evolutionary PDG. Vainstein and Arenzon [42] first introduced empty sites as a disordering factor in a square lattice and found that cooperation in the population may be enhanced. Then, Vainstein and Silva [43] extended the model by allowing players to jump with a certain probability to empty neighboring sites. The results of their work showed that the migration of players could further promote cooperation. In addition, Helbing and Yu [44,45] introduced success-driven migration as a specific mechanism leading towards the outbreak of cooperation in a population of selfish and unrelated individuals, even under noisy conditions. In these works, the authors assigned the same mobility to all players. Droz et al [46] introduced influential players to the population and found that a higher level of cooperation could be achieved if the influential players were allowed to move at an optimal rate. Cheng et al [47] introduced migration depending on players' strategies and found that both the fast migration of defectors and the slow migration of cooperators may enhance cooperation. Yang et al [48] proposed an aspiration-induced migration in which individuals would migrate to new sites provided that their payoffs were below some aspirational level.
In reality, the migration of an individual may strongly correlate with the individual's financial or social status and the correlation may be different in different situations. In financial 3 activities in human society, migration of individuals may positively correlate with their financial status. As mentioned in [49], the empirical distribution of wealth and income reveals a twoclass distribution. The majority of the population belongs to the lower class characterized by the exponential distribution, whereas a small fraction of the population in the higher class is characterized by the power-law distribution. The formation of the two-class distribution of wealth and income is attributed to individuals in the higher class; for example, successful businessmen in financial markets are more aggressive in exploring new fields by investing, while individuals in the lower class remain passive because of their shortage of money. On the other hand, the migration of individuals may anti-correlate with their financial status in some other situations, such as colonies of microorganisms where starved individuals tend to migrate for nutrition. Considering these situations, in the evolutionary PDG, we introduce a mechanism where the migration of a player depends on the player's payoff and investigate its effects on cooperation.
This paper is organized as follows. The model, including descriptions of evolutionary processes, is presented in section 2. In section 3, we present the results and investigate in detail the causes for our findings, and in the last section, we summarize our findings and discuss the social implications of this work.

Model
In a one-shot, two-player PDG, two players simultaneously decide whether they wish to cooperate or defect under the condition of having no knowledge of the choice of their counterpart. They will receive a reward, R, if both cooperate, and a punishment, P, if both defect. However, if one player defects while the other cooperates, the defector will get the temptation payoff, T , while the cooperator will get the sucker's payoff, S. The rankings of T > R > P > S and 2R > T + S are required by a typical PDG. In this work, we consider an evolutionary PDG on SF networks as suggested by Barabási and Albert [50] with average degree k = 4. Following common practice [6], payoffs are normalized by taking R = 1 and P = S = 0. Since the level of cooperation on SF networks is much higher, the requirement of 2R > T + S is relaxed and we set T = b in the range of [1, 2.5].
In SF networks with N = 3000, each site may or may not be occupied by a player. We define the density of players as ρ = (n c + n d )/N , where n c is the number of cooperators and n d is the number of defectors. Initially, players are distributed randomly on the network and each player is designated as a cooperator or a defector with an equal percentage. Each generation in Monte Carlo (MC) simulations consists of three elementary steps. Firstly, a randomly selected player, i, with degree, k i , interacts with their nearest neighbors to accumulate a payoff, p i . Meanwhile, the player's nearest neighbors acquire payoffs accordingly. Secondly, the selected player updates their strategy according to the richest-following rule [51]- [53]. That is, the selected player adopts the strategy of their neighborhood having the highest payoff (including the selected player). Thirdly, with a certain probability, µ i , the selected player moves to a site chosen randomly within their nearest neighbors if the site is empty. In simulations, a random number in the range from 0 to 1 is drawn, and if the number is less than µ i , a node is randomly chosen from the neighborhood of the selected player. If the chosen node is not occupied by another player, the selected player moves to the node. µ i is defined as µ i = P α i , where P i = p i bk i is the normalized payoff of the player, i. α is a control parameter responsible for the dependence of the moving probability, µ, on the normalized payoff, P. When α = 0, all players move with an equivalent probability (for players with zero normalized payoff, we let their mobility be given by µ i = 1), which reduces to the case of random migration. α > 0 means that the mobility of a player is positively correlated with their normalized payoff (MCP). In the case of MCP, the defectors surrounded by other defectors will not move because of their zero payoff, and for the same reason, isolated players will not move. There are two special situations: the cases when the density of players, ρ, approaches zero and one. When ρ → 0, the players are sparsely distributed on the network, all players become isolated and their migrations stop after a short time, which leads to a final cooperator level close to the initial value. On the other hand, when ρ → 1, although the mobility of players may be high, real migration of players becomes rare because of the shortage of empty nodes. In this case, the evolution of cooperation follows the evolution in a motionless system. We characterize the cooperation with the frequency of cooperators, F c , which is defined as F c = n c /(n c + n d ) and is calculated by averaging over 2000 steps after 18 000 transient steps. Unless otherwise specified, each data point results from an average of 200 realizations.

Numerical results
Initially, we study an evolutionary PDG on SF networks without migration and in the presence of empty sites. The contour plot of F c on the b-ρ space is presented in figure 1(a). Figure 1(a) shows that F c first decreases and then increases with ρ, and the introduction of empty sites does not improve cooperation, which is similar to the case of the square lattice [47]. However, things are quite different when MCP, where cooperation enhancement is prominent, is introduced. Figure 1(b) shows the results for the case, α = 0, where all players move with the same probability. The migration of players has different effects in the case of different ρ. For large ρ, cooperation is enhanced, and there exists an optimal ρ at which the highest level of cooperation is reached. For small ρ, cooperation is suppressed. Additionally, at b = 2, there exists a steep drop in cooperation because of the richest-following rule [44]. Above b = 2, the region of cooperation enhancement shrinks and is limited to large ρ near 1. In SF networks, the high level of cooperation is maintained by hubs whose degrees are much higher. To understand the phenomena above, we focus on the migration of hub players and non-hub players. For large ρ, hubs are always occupied, and the migration of hub players has little influence on their positive role in cooperation. However, the migration of non-hub players brings defectors to cooperator clusters centered at hubs and enhances cooperation, as depicted in previous studies [47]. For small ρ, hubs may be empty for a long time, which weakens the positive role played by hubs in cooperation. Moreover, low player density blocks the formation of large cooperator clusters, and the migration of non-hub players may further break up cooperator clusters. All of these factors suppress cooperation when ρ is small.
In considering the influence of α on cooperation for α > 0, we first discuss the mobility of defectors and cooperators. Since, as discussed above, the migration of players adversely affects the maintenance of cooperation at a low density of players, we focus here on a high density of players where the migration of players may play a positive role in cooperation. When α deviates from zero, the diversity of mobility of players shows up, and the moving probability of either cooperators or defectors (µ c or µ d ) ranges from 0 to 1. For an arbitrary node, the player on the node gets a higher payoff when they adopt defection, which means that µ c < µ d for a player on the specific node in question. The inequality also holds on a large scale, which is verified by panels (a) and (c) of figure 2, where the distributions of the mean normalized payoff of cooperators, P c (k) , and the mean normalized payoff of defectors, P d (k) , on degree, k, are presented at different times. Figure 2 shows that throughout the evolution, P c (k) is always lower than P d (k) for low degrees. That P d (k) is lower than P c (k) at high degrees does not invalidate the inequality of µ c < µ d , which is supported by the results in panels (b) and (d) of figure 2. Figures 2(b) and (d) show the distributions of the cooperator frequency, F c (k), the defector frequency, F d (k), and the fraction of players, F(k), at different times. Clearly, as time advances, the nodes with high degree are quickly occupied by cooperators, which agrees with the statement by Gómez-Gardenes et al [16] that nodes with high degree are always occupied by pure cooperators (the players always cooperate after transient time). A direct consequence of nodes with high degree being occupied by pure cooperators is that the events of strategy updating only occur on nodes with low degree during the evolution of cooperation [54], which ensures the validity of µ c < µ d . Figures 2(b) and (d) also reveal that, after transient time, the empty nodes tend to distribute on nodes with low degree. Recalling that for a high density of players the optimal cooperation enhancement occurs when defectors move faster than cooperators [47], we know that the inequality of µ c < µ d at α > 0 indicates cooperation enhancement in comparison with the case α = 0, where µ c = µ d . The definition of a player's mobility reveals that, for any given P c and P d , with increasing α, the mobility of all players becomes less and the system reduces to the motionless situation at α → ∞, which is also disadvantageous to cooperation enhancement, as stated in [47]. Based on these analyses, we know that F c should display non-monotonic behavior with the variation in α at large ρ. This can be seen from figure 3, where the contour plots of F c on the b-ρ space for different α are presented. Three main features can be drawn from figure 3. Firstly, the cooperation enhancement induced by MCP is prominent. In each plot, there exists a large region on b-ρ space for cooperation enhancement. In particular, an all-C state (all players are cooperators in the network) may exist in most of the region 1 < b < 2 and ρ > 0.4. Secondly, the non-monotonic behavior of cooperation with α is also clear, which can be observed from the territory of the all-C state that first expands and then shrinks. Figure 3 also shows that the optimal α for the highest level of cooperation depends on the combination of ρ and b. Thirdly, for a low density of players, figure 3 shows a monotonic increase in F c with α. Specifically, F c first rises quickly and then stays at an asymptotic limit at large α.
To get microscopic views of the evolution of cooperation on SF networks in the presence of MCP, we first study the evolution of the effective frequency of movement of players, N M , which is defined as the fraction of moving players. The results for different ρ are presented in figure 4 for b = 1.75 and α = 1. We find that N M approaches a constant value after transient time for large ρ; for example, for ρ 0.4. The asymptotic value of N M decreases with ρ because of fewer empty nodes in the networks. On the other hand, interestingly, after transient time, N M = 0 for small ρ. That players become motionless for small ρ indicates the formation of interesting strategy patterns on the network where cooperators form compact clusters and all empty nodes are separated from those cooperator clusters by defectors. The formation of the strategy pattern also gives a possible explanation for the result that cooperation is enhanced in the presence of MCP in comparison with the situation with α = 0. Then, we study the evolutions of C-clusters (cooperator clusters) and D-clusters (defector clusters) at α = 1 and b = 1.75. For each kind of cluster, we monitored two quantities: the mean size of clusters and the number of clusters. The results are presented in figure 5 for ρ = 0.37, ρ = 0.45 and ρ = 0.7. Consistent with the above discussion, the asymptotic mean size of C-clusters increases with ρ, and the asymptotic number of C-clusters decreases with ρ. Also of interest is that at a low density of players, the mean size shows a maximum in the initial stage of evolution. Actually, the fall of mean size after the initial rise is caused by the migration of hub-cooperators, which fragments the C-clusters centered on them and reduces the size of C-clusters. With an increasing density of players, the effects of the   shows a cooperation enhancement, which confirms the positive role of players' migration in cooperation again. However, as shown in figure 6, the levels of cooperation in the above two cases are always lower than those in figure 3(b) for b < 2 and are always higher than those in figure 3(b) for b > 2. The two observations indicate that the strong effects of MCP on cooperation shown in figure 3(b) come from the collective effects of the migrations of both cooperators and defectors.
The possible cooperation enhancement induced by MCP is independent of population structure and the rule for updating strategy. To verify the claims, we first consider the evolutionary PDG with the richest-following rule on square lattices. Figure 7(a) shows the contour graph of F c on the b-ρ space on square lattices with MCP, and figure 7(b) shows the contour graph of the difference between F c on square lattices with MCP and without migration. Similar to the results for SF networks, strong cooperation enhancement is observed at large ρ. Due to the absence of hub players on square lattices, the region for strong cooperation enhancement shrinks. For example, the region for strong cooperation enhancement is limited to the region with ρ > 0.8 and b < 1.5. Then, we considered the evolutionary PDG on SF networks but with the Fermi strategy updating rule instead of the richest-following rule. The Fermi rule is defined as The parameter K denotes the uncertainty in the strategy updating, and we let K = 0.1. The contour plots of F c on SF networks adopting the Fermi rule with MCP and the differences between F c on SF networks adopting the Fermi rule with MCP and without migration are presented in figure 7(c) and (d), respectively. Similarly, the enhancement of cooperation is observed at large ρ. In comparison with the cooperation enhancement on SF networks, the cooperation enhancement on square lattices is weak but may be improved by the choice of a proper α. In short, the cooperation enhancement induced by MCP is independent of both population structure and the strategy-updating rule.
In the above investigations, we adopted the MCP rule in deciding how players move. One question arises: What happens if the mobility of a player is anti-correlated with their normalized payoff (we denote it as MACP)? In the case of MACP, we define the moving probability of a player with normalized payoff, P, as µ = 1 − P α . As an example, we set α = 1. The result for MACP on SF networks with the richest-following rule is presented in figure 8, which is qualitatively similar to the result in figure 1(b) (e.g. cooperation enhancement for large ρ and cooperation suppression for small ρ). Nevertheless, further cooperation enhancement for MACP is observed in figure 8 in comparison with α = 0, which also stems from heterogeneity in the mobility of individuals. As shown in figure 9, the normalized payoffs of defectors are always lower than the normalized payoffs of cooperators, which causes a higher mobility for defectors than for cooperators and leads to cooperation enhancement in comparison with α = 0.

Conclusion
In conclusion, we have investigated an evolutionary PDG on structured populations where players may move to neighboring empty nodes with a probability depending on their normalized payoffs, µ = P α (in the case of MCP). We found that in comparison with a motionless system, cooperation could be promoted strongly for a high density of players at α > 0, and in a large region on the b-ρ space, the all-C state may be reached. Cooperation enhancement shows a non-monotonic behavior with variation of α. Through extensive numerical simulations, we also found that the cooperation enhancement for α > 0 is robust in the face of changes to the population structure and the strategy-updating rule. We further investigated the evolution of cooperation when the mobility of players is defined as µ = 1 − P α (in the case of MACP) and found that the result of MACP is similar to that of MCP with α = 0.
In contrast to many previous co-evolutionary games, we introduce a simple rule for individual mobility, which is controlled by a single parameter, α, and great cooperation enhancement is achieved. However, as we know that heterogeneity in the properties of individuals plays an important role in cooperation, it will be interesting to investigate in future work whether the presence of heterogeneity in α may have further positive effects on cooperation.