Effects of inter-connections between two communities on cooperation in the spatial prisoner's dilemma game

In this paper, we investigate the evolutionary prisoner's dilemma game in a structured population with two communities. The number of inter-connections represents the interaction strength between the two communities. We address the question of how the interaction between two communities influences cooperation by studying five situations with different game dynamics and different community structures. We find that the interaction between communities suppresses cooperation when the levels of cooperation in isolated communities are high, whereas the interaction enhances cooperation when the levels of cooperation in isolated communities are low. For intermediate levels of cooperation in isolated communities, the interaction between communities induces a resonance-type behavior of cooperation.


Introduction
The evolutionary prisoner's dilemma game (PDG) has attracted much attention over the last few decades [1] for gaining understanding of the emergence of cooperation in a population of selfish individuals. In PDG, each individual chooses cooperation (C) or defection (D) as its competing strategy. When the population is well mixed, PDG fails to sustain cooperation, which is often at odds with reality where mutual cooperation may also be the final outcome of the game [2]- [4]. In Nowak and May's seminal work [5,6], a mechanism for cooperation in the evolutionary PDG was introduced where the two-dimensional (2D) square lattice and the interaction between nearest neighbors enable cooperators to protect themselves against defectors' exploitation by forming compact clusters on the lattice. Inspired by the maintenance of cooperation on a square lattice, investigating the evolutionary PDG on structured populations has become an active field. The influences of different types of parameters on cooperation have been studied intensively. For example, a large set of strategies was used in some models [7]- [9], different evolutionary rules were introduced [10]- [12], randomness of a different nature was considered [13]- [16] and the effects of increasing number of neighbors on cooperation were studied [17]. Generally, the opinion that diversity in the personalities of individuals or in population structures can enhance cooperation in the evolutionary PDG [10], [18]- [21] has been widely accepted.
Most of the works mentioned above focus on the spontaneous emergence of cooperation in populations composed of only a single community. However, in reality, populations may be divided into several communities. The relationships or interactions between individuals inside one community are closer than those in different communities. For example, the world is divided into different countries and the industry is divided into different companies. In [22], the authors studied three games, Prisoner's Dilemma, the Hawk-Dove and the Stag Hunt, in three complex networks: BA scale-free network [23], TSN [24] and genetic programming community. The results showed that the structure of social networks plays an extremely important role in the game dynamics and the emergence of cooperation. The inter-community structure and the intra-community structure were designed during the study of PDG in two practical networks in [25,26]. The authors found that the behavior observed in a network with communities depends strongly on the intra-community heterogeneity and also on the inter-community connectivity. By the design of the network with communities in their work, the effects of intra-community heterogeneity and inter-community connectivity on cooperation were mixed up and could not be separated clearly. One cannot quantify how strongly the inter-community connectivity affects the level of cooperation in the population quantitatively. In [27], a structural network with both a tunable clustering coefficient and a tunable degree distribution was used to study the roles of the clustering coefficient and the community structure in the evolution of cooperation. Again, due to the formation of a clustered network, the strength of interaction between communities, which is entangled with the heterogeneity of the network, cannot be seen clearly. So, how to define the strength of interaction between communities? How does it influence the level of cooperation in social dilemma situations? What is the relationship between the overall cooperation and the interaction between communities, and how does the influence of interaction on the overall cooperation depend on the cooperation level in each community? Answers to all these problems are still unknown to us. In this paper, we will try to address these questions.

Model
For simplicity, we consider a population composed of only two communities. Each community is represented by a 2D lattice on which players are located. In the population, the players follow cooperation s x = 1 0 or defection s x = 0 1 . The payoff of a player x, accumulating through the one-shot PDGs with its opponents, can be expressed as where s + x denotes the transpose of the state vector s x . x includes all the neighbors of the player x and itself. Self-interaction is not indispensable. We also consider the cases without selfinteraction and the results remain the same. The payoff matrix, Q, is defined as the following, which is suggested by Nowak and May [5]: In this notation, 1 < 1 + r < 2 measures a defector's temptation to exploit the neighboring cooperator. The game is a weak PDG when c = 0 and a PDG when c = −r . Then, the player x will adopt the strategy s y of a randomly chosen neighbor y with a probability that is determined by the payoff difference between them: where the parameter K , which is analogous to the temperature in the Fermi-Dirac distribution in statistical physics, characterizes the stochastic uncertainties in making decisions for the player x [14,15]. Throughout this work, we set K = 0.1. The parameter ω y is the teaching ability that characterizes the ability of the player y to transfer its strategy to other players. In the absence of diversity in teaching ability, ω y is independent of the player y and we set ω y = 1. First, we will consider weak PDGs, setting c = 0, in three different cases. In the first case, the two communities are identical 2D lattices. Without inter-connections, each player has four neighbors (z = 4). In the second case, the communities are different. In one community, each player has four neighbors, whereas in the other community, each player has eight neighbors (z = 8). In the third case, the population structure is the same as that in the first case. However, in one community, all players have the same teaching ability ω y = 1, whereas diversity in teaching ability is present in the other community. Szabó and Töke [17] report that 20-40% is the optimal percentage of players with high teaching ability for promoting the cooperation level in a square lattice. To be specific, we let the players in the other community to be divided into two groups: 30% of the players have teaching ability ω y = 1, while the remaining players have teaching ability ω y = 0.1. The qualitative results will not change if we choose other ratios.
Secondly, we consider PDGs with c = −r in two more different cases. In the fourth case, the communities are two identical lattices with z = 4, but without self-interaction. That is, x includes all the neighbors of the player except itself. In the fifth case, the communities are different: one is a square lattice with z = 4 and the other is an ER [28] random network with z ave = 8. In this case, the self-interaction is also excluded.
The interactions between two communities are represented by random inter-connections between players in different communities and the number of inter-connections between two communities measures the strength of interaction between the two communities. To exclude the influence of structural heterogeneity, we require that each player in one community may have 4 one inter-connection at most and, once established, the inter-connections between communities remain unchanged during the evolution. The interaction between communities manifests itself not only on performing PDGs but also on strategy updating.
In the Monte Carlo simulations, the number of players in each community is set to be N = 100 × 100 and the 2D lattices satisfy periodic boundary conditions. The controlling parameters in these five cases are r and the number of random inter-connections m, which ranges from 0 to N . Initially, the two strategies of C and D are randomly distributed among the players with equal probability. To measure the cooperation level, the cooperator frequency ρ c will be monitored when the evolution of the strategy pattern reaches its steady state. All the following data are obtained with synchronous updating and each point is gained by averaging 1000 generations after a transient time of 4000 generations and by averaging over 20 realizations corresponding to 20 different initial conditions.

Simulation results and analysis
First, we investigate the evolutionary PDG in different isolated communities. The cooperator frequencies ρ c in these communities against r are presented in figure 1. In figure 1(a), the situation of the weak PDG is considered in the presence of self-interaction. The squares and circles are for the communities with z = 4 and with z = 8, respectively, in the absence of diversity in teaching ability. The open triangles are for the community with z = 4 in the presence of diversity in teaching ability. From figure 1(a), it is clear that the presence of diversity in teaching ability enhances cooperation [17] and the large number of neighbors endangers the cooperation. In figure 1(b), PDGs in square lattices and the ER network are studied. The squares, circles and triangles are for the communities with z = 4, z = 8 and ER with z ave = 8, respectively. Closed symbols are for the situation with self-interaction and open symbols are for the ones without self-interaction. Due to the introduction of self-interaction, the cooperation can survive for rather large r .
Since the two communities in the first case are the same, it is enough to consider the averaged cooperator frequency in the whole population ρ c,t (the overall cooperation). In figure 2, we present the results for this situation. From figure 2, we find that the relationship between the cooperator frequency ρ c,t and the interaction between two communities strongly depends on r . For small r where the cooperator frequencies in isolated communities are high, ρ c,t always decreases with the number of inter-connections m. When r increases where the cooperator frequencies in isolated communities become low, the dependence of ρ c,t on m becomes nonmonotonic: a small amount of random inter-connections can enhance cooperation and a large number of inter-connections may downgrade cooperation. Further increasing r so that the cooperator frequencies in isolated communities become sufficiently low (e.g. r > 0.8), ρ c,t always increases with m. In particular, when r is large enough that the cooperators in isolated communities become extinct, the cooperation may be restored and sustained at a relatively high level in a large range of m. In short, the effects of the inter-connections on cooperation can be summarized as the dependence of the optimal number of inter-connections m opt on r at which the cooperation is strongest: m opt always increases with r . For small r , m opt appears at m opt = 0, whereas at large r , m opt appears at m opt > N . The insets of figures 2(a) and (f) show the uncertainties of ρ c,t in the vicinity of the transition points where extinction of cooperators or defectors occurs. Clearly, the uncertainties increase with r approaching the transition points, which indicates a critical slowing phenomenon [29]. Then, we consider the second case where one community has z = 4 and the other z = 8. Since the two communities are different, the dynamics of the system is richer than the first situation. The cooperator frequencies evaluated in both communities (ρ c1 and ρ c2 ) and in the whole population (ρ c,t ) are monitored. The left column in figure 3 shows ρ c1 in the community with z = 4 against m for different r , the middle column shows the results for the community with z = 8 and the right column shows ρ c,t . From the top to the bottom, the parameter r is increased from 0.16 to 0.84. Interestingly, the influences of the inter-connections on cooperation are similar to those in the first situation. That is, there exists an optimal number of inter-connections for the cooperation, m opt , and m opt increases with r . The difference from the first situation is that, in the second situation, the optimal numbers of inter-connections m opt are different for Furthermore, we consider the third case and the results are presented in figure 4. Once more, the scenarios mentioned above are found again. There is one interesting observation for the communities we used in the third situation. From figure 1, we know that, for r < 0.3, the cooperators take over the whole community regardless of the presence or absence of teaching ability diversity. However, for r > 0.3, the community with teaching ability diversity is better than that without it. When the inter-connections are switched on, as shown in figure 4, the presence of diversity in teaching ability is more harmful to the cooperation than the absence of it. For example, the cooperator frequency ρ c1 in the community without teaching ability diversity is always larger than ρ c2 for r < 0.3. Even for 0.9 > r > 0.3 where the community without diversity in teaching ability has lower cooperation than the community with diversity in teaching ability, the difference between ρ c2 and ρ c1 decreases with the number of inter-connections. The advantage of the teaching ability diversity in maintaining cooperation is manifested only when r > 0.9, where cooperators in both isolated communities are extinct. It should be mentioned that, in the above three cases, the inter-connections are introduced in a random way. However, even if we introduce the inter-connections in a regular way, the scenarios observed above will remain unchanged qualitatively. For example, a node in one community only has interconnection with that at the same location in the other community.   The above three cases show the results for weak PDGs and the regular lattices with selfinteraction. However, the scenarios are the same even if we consider more general situations, for example, the game dynamics taking the form of PDGs in the absence of self-interaction and the communities taking the form of complex networks. In figure 5, we show the results for the fourth case where PDGs are considered on a structure with two identical square lattice in the absence of self-interaction, and in figure 6, we show the results for the fifth case where PDGs are considered on a structure with one square lattice with z = 4 and one ER network with z ave = 8 in the absence of self-interaction. Both figures show that there exists an optimal inter-connection m opt that is dependent on r .
As mentioned above, the influences of the inter-community connections on cooperation are twofold: inter-community connections suppress high level of cooperation and enhance low level of cooperation. For intermediate levels of cooperation, cooperation is enhanced for weak interaction between communities and suppressed for strong interaction between communities. To understand the impacts of inter-community connections on cooperation, we consider the evolution of cooperation in a microscopic view. Generally, the variation of cooperation with inter-community connections is determined by two elementary events: defectors are converted into cooperators and cooperators are converted into defectors. For simplicity, we consider identical communities. The scenarios for systems with different communities are the same qualitatively; however, for a high level of cooperation, the cooperator frequency in one community may decrease when that in the other one may increase. Under the condition of high level of cooperation, most of the defectors in the isolated community survive only when they are located at the boundaries of cooperator clusters and their neighbors are dominated by cooperators. When a defector in this community is linked to a defector in the other community by an inter-connection, there is no contribution to the cooperator frequency in its community. However, if the inter-connection links the defector to a cooperator in the other community, the defector will gain more payoff so that the probability that it changes its cooperator neighbors to defectors increases. Intuitively, with the increase of m, the probability that a defector encounters a cooperator in the other community increases, which leads to an increase of the probability of cooperators turning into defectors. To be addressed, as the defector gets surrounded by more and more defects with the help of its cooperator neighbors in the other community, its payoff decreases and the probability that it takes the strategy of its neighbor in the other community increases. However, the fact that most of the neighbors of the defector in its community have been occupied by defectors indicates another type of event: a defector is converted into a cooperator with the help of inter-connections, which only happens when the defector is inside the defector clusters. Such an event cannot be permanent since, once the defector is converted into a cooperator, its defector neighbors in its community will convert it back to a defector soon. Combining these two events together, we may find that the cooperator frequency in a community with high level of cooperation tends to decrease with m. Then, under the low level of cooperation, cooperators survive only by forming small cooperator clusters. Most of the neighbors of a defector at the boundaries of cooperator clusters are occupied by 10 defectors. The players in cooperator clusters have a high probability to encounter defectors in the sea of defectors in the other community and have high probability to convert these defectors into cooperators temporarily. It is these temporary cooperators in the other community that provide higher payoffs for players in the cooperator clusters and eventually lead to the expansion of cooperator clusters. On the other hand, the events that cooperators are turned into defectors happen only when those cooperators encounter those defectors at the boundaries of cooperator clusters. Because of the low level of cooperation, such events occur with a low probability. Therefore, the events that defectors are turned into cooperators outperform events that cooperators are turned into defectors and this trend will be enhanced by increasing the number of inter-connections, which expose more players in the sea of defectors to those in cooperator clusters. In short, it is the competition between these two fundamental events that leads to the appearance of the optimal number of inter-connections. The fact that these two elementary events depend on the level of cooperation in isolated communities leads to the dependence of the optimal number of inter-connections on the parameter r .

Discussion
It has been concluded that cooperation in real social networks is a complex issue depending on a combination of the effects of several structural features including the inter-and intrastructure of the communities [25]. In [27], the authors found that the clustering coefficient in degree-homogeneous networks inhibited the emergence of cooperation and a larger size of the communities within a whole population would decrease the level of cooperation. But the works mentioned above cannot declare clearly how the interaction between communities affects cooperation since both the clustering coefficient and the size of communities do not have clear a monotonic relationship with the interaction between communities. In the simple construction of a system with two communities in our work, it is easy to establish the relationship between intercommunity interaction and inter-connections. From the investigation of evolutionary PDGs on this simple construction, we find that the inter-connections between two communities can depress the cooperation for small r or enhance the cooperation for large r . In particular, we find that, for intermediate r , the cooperation displays a resonance-type behavior with variation of the number of inter-connections.
The observations in this work are of practical significance. For example, if the levels of cooperation in two societies are high, then to maintain a high level of cooperation, it would be better for these two societies to avoid communication between them. However, if two societies are very low in cooperation, strong interaction between them is required to acquire cooperation. In between, two societies have to be careful to find an optimal interaction strength to acquire the highest level of cooperation. Of course, the models discussed in this work are very simple since we ignore many positive aspects in the communication between two societies, e.g. the effects of the spread of knowledge and technology. Along this line, our future work will consider how to incorporate factors ignored in this work and will investigate their effects on cooperation emerging spontaneously.