Understanding spatial public goods games on three-layer networks

Cooperation abounds in all biological systems. Spatial public goods game (PGG) serves as a baseline model when investigating the evolution of cooperation in collective interactions. Since analyzing collective interactions and dynamics in spatial structures is very complicated, a full theoretical understanding of spatial PGGs is still deficient. Here we generalize spatial PGGs in a three-layer weighted network—investment, benefit allocation, and strategy dispersal layers—which are not necessarily identical and thus cover a wide class of population structures and interaction scenarios. We provide an analytic formula that accurately predicts when cooperation is favored over defection, and that is applicable in populations of any size under weak selection. We prove that in regular networks investment and benefit allocation are essentially symmetric—that exchanging structures of investment and benefit allocation layers does not affect the evolutionary dynamics at all. The success of cooperators relies heavily on the correlation between an individual’s investment in a game and its benefit allocated from the same game. In most cases, the positive correlation, i.e., a greater investment in games with a larger share of benefits, facilitates a cooperative society. Importantly, we also show that diversifying the amounts of investment in different games or benefit allocation to different participants, if implementing improperly, might impede the global cooperation.


Introduction
Cooperation among unrelated individuals [1] has puzzled both evolutionary biologists and sociologists since Darwin [2]. Evolutionary game theory is an efficient mathematical tool for modeling and studying this cooperation conundrum [3]. The public goods game (PGG) is a typical game metaphor frequently used to describe social dilemmas involving multiple agents [4]. In the classical PGG, each of L participants independently chooses cooperation or defection. Each cooperator invests an amount c ('cost') to a public goods pool, and defectors do not. Finally the total investment is multiplied by an enhancement factor r and is evenly allocated to all participants. Because defectors do not invest, they obtain a larger net benefit than cooperators. Thus defection is the stable evolutionary stable strategy. The behavior of free riders, i.e., defection, then drives the population into the tragedy of the commons despite the greater benefit derived from collective cooperation [4].
Investment and payoff allocation are two crucial ingredients in PGGs. Given both theoretical and experimental importance of spatial PGGs, the interplay between investment, payoff allocation and evolution of cooperation must be clear. On the one hand, it is conducive to make clear which factor plays the dominant role in the evolutionary outcome when various factors are coupled. On the other, it may inform new solutions to get rid of the tragedy of the commons without invoking other mechanisms such as reward, punishment [13][14][15][16][17], and reputation [13,20]. Most prior studies assume that in each PGG each cooperator invests the same amount c and that ultimately the produced benefits are equally allocated to all participants [35]. This is in sharp contrast to the behavior observed in real-world animal communities and microbial colonies [37,38]. For example, when guillemots preen each other (a reciprocal behavior), the duration of allopreening bouts ranges from under a second to over a minute, where the longer duration is the larger investment [37]. In addition, a cooperative budding yeast secretes invertase to hydrolyse disaccharide but 99% of the created production diffuses to neighboring yeasts [38]. All these imply nonuniform investment and payoff allocation. Despite the results obtained by a few numerical simulation studies [29,[39][40][41][42][43][44][45], many fundamental questions still remain unanswered. Prior studies always couple various factors (besides nonuniform investment and payoff allocation) into a model, which usually obscures how the heterogeneity in investment or payoff allocation affects the evolutionary outcomes [39,41]. For example, when payoffs are distributed amongst players by assigning each player an individual scaling factor, the nonuniform scaling factors actually lead to twofold heterogeneities, i.e, allocating benefits nonuniformly and possessing different individual attributes [41]. Thus it is hard to tell which factor should be responsible for the evolutionary outcome [41]. More importantly, few work so far has studied investment and payoff allocation simultaneously. Their synergistic effects on the evolution of cooperation thus are still undiscovered. In addition, because both multiplayer game and spatial structures have added complexity to the theoretical exploration [46][47][48][49], analytical results have been deficient. Nevertheless, a recent investigation by Allen et al [50] provides a few analytical insights into evolutionary dynamics when there is an unequal sharing of diffusible common goods. They found that a lower level of diffusion to neighbors is conducive to cooperation.
Here we generalize the spatial PGGs using a three-layer weighted network with investment, payoff allocation, and strategy dispersal layers in which structures in different layers are not necessarily identical. We investigate a minimal model in which each individual can have nonuniform investments and shares of benefits in different games, but all individuals cannot be distinguished except their strategies. This excludes any difference in individuals' attributes and makes the acts of investing and allocating benefits the only variables affecting cooperation. Our analytical investigation arrives at a few unexpected results. We find that in regular networks exchanging structures of investment and payoff allocation layers does not change the evolutionary outcomes, implying that investment and payoff allocation are essentially symmetric. In addition, the evolutionary outcome relies largely on the correlation between the investment in a game and the benefit allocated from the same game. In most cases, the positive correlation, i.e., a large investment in games with a large share of returns, is conducive to societal cooperation. Importantly, our finding indicates that diversifying the amounts of investment or benefit allocation in different games, if implementing improperly, might impede the global cooperation. This finding contrasts sharply with most prior studies addressing that diversity (or heterogeneity) promotes cooperation [36]. Also, it enriches our knowledge of interplay between diversity and the collective behavior.
The rest of the paper is structured as follows. Section 2 briefly describes our model. Section 3 presents the method of identity by descent (IBD) for deriving an exact analytical condition in three-layer weighted networks. Section 4 examines analytical calculation results and shows all findings. Section 5 provides our concluding remarks. Figure 1(a) shows the three-layer networked structure of the spatial PGG. Each node represents an individual and edges capture events associated with investment (investment layer), payoff allocation (payoff allocation layer), and strategy dispersal (dispersal layer). Investment and payoff allocation are modeled as directed networks. In each generation, each individual initiates a PGG centered on itself. Figure 1(b) shows that in the investment layer the direction of the edge indicates who makes an investment to whom-centered game, and the weight indicates the investment fraction. Here the total investment of a cooperator in all PGGs in each generation is normalized to 1. Figure 1(c) shows that in the payoff allocation layer the benefit goes to players being directed and that the edge weight denotes the allocation fraction.

Model
In the PGG initiated by j, i invests a fraction d ij of its total investment, i.e., d ij s i , where s i is i's strategy (s i =1 indicates cooperation and s i =0 defection). Note that d ij is not necessarily identical to d ji . The accumulated investment in PGG centered on j is given by , where V is the set of all individuals. After the investment produces a benefit Π j , a fraction e ji , i.e., e ji Π j , is allocated to i. Similarly, e ij may be different from e ji . Overall, the net benefit for i from PGG centered on j is Here i's payoff is transformed to fitness by F f 1 i i d d = -+ , where δ denotes the selection intensity. Since the fitness of an individual is often the consequence of many factors rather than only games considered here, payoffs derived from games have only marginal effect on individuals' fitness [51]. We thus take weak selection (δ=1).
Following all interactions, in the strategy dispersal layer, the population evolves according to the classic death-birth update rule [52]. That is, a random individual i is chosen with a uniform probability to die and another individual j competes to replace i with a probability proportional to its effective fitness w ji F j , i.e., . We constrain the three-layer structure into a joint transitive network [53][54][55]. A mathematical definition for such a network is that for any two nodes i, jäV, there exists a permutation T such that T(i)=j and d T(x)T(y) =d xy , e T(x)T(y) =e xy , w T(x)T(y) =w xy for any pair of x, y. The node-transitivity gives all nodes the same spatial configuration. Joint transitive networks describe a large class of population structures, such as rings and square lattice. Note that we can recover the traditional spatial setting by making the structures in different layers overlapped fully and the edge weights identical.

Method
When there are no mutations, the system evolves to a homogeneous state in which all individuals are either cooperators or defectors. The fixation probability ρ C is the probability that a cooperator in a random position takes over the whole population of defectors [51]. Analogously, ρ D is the probability a defector in a random position takes over the whole population of cooperators. Here ρ C >ρ D indicates that natural selection favors cooperation over defection. Using the method of IBD [54,56] we find the critical enhancement factor r * for Figure 1. Spatial public goods game with a general setting of investment and payoff allocation. (a) Spatial structure described by a three-layer weighted network (investment, payoff allocation, and strategy dispersal). Nodes represent individuals (marked by 1, 2, 3, 4). Directions of edges in the investment (payoff allocation) layer represent investment (payoff allocation) directions, and weights show investment fractions in different games (benefit fractions allocated to different participants). (b) Investment and payoff allocation associated with individual 1. Individual 1 participates in a self-centered and two neighbor-centered PPGs (highlighted). The investments are d 11 s 1 , d 12 s 1 , and d 14 s 1 , respectively. In each game (taking the game centered on 1 for example), the accumulated investment is multiplied by an enhancement factor and then benefits (Π 1 ) are allocated to all participants, i.e. e 11 1 P , e 12 Π 1 and e 14 Π 1 to individual 1, 2 and 4, respectively. (c) Strategy dispersal according to the death-birth update rule. A random individual 1 is chosen to die. A random neighbor replaces 1 proportional to their efficient fitness (see the main text for details). As illustrated, 2 competes to replace 1 with probability ρ C >ρ D . Because when r * is larger it is harder for cooperation to evolve, the success of cooperators is negatively correlated with r * . Biologically IBD describes that two individuals inherit alleles from a common ancestor. Two individuals are called IBD when no mutation separates them from their common ancestor during evolution. This is frequently used to study the type assortment [56], and the related methods have been used to solve varieties of problems in evolution [57,58]. Allen et al formalize IBD method and apply it to pairwise games on graphs [54]. In this paper, we further extend it to multiplayer games such as PGGs in multilayer networks. The analytical calculation is based on an evolving system with strategy mutation rate μ. The derived results can predict a system with no mutations in the limit of rare mutations ( 0 m  ) [54]. We present the key points for this extension.
We define a (n, m, l)-random walk in the three-layer network to be a random walk with n steps using the weights d ij i j , { } in the investment layer, m steps using the weights e ij i j , { } in the payoff allocation layer, and l steps using the weights w ij i j , { } in the strategy dispersal layer. In the joint transitive network, the commutativity of the adjacency matrices ensures that the probability of terminating at an assigned node is independent of the step order. Here p ( n, m, l) is the probability that a (n, m, l)-random walk terminates at its starting node, p ij n m l , , ( ) the probability that a (n, m, l)-random walk starting at i terminates at j, q ( n, m, l) the probability that a (n, m, l)random walk terminates at a node IBD to the starting node, q ij the stationary probability that individuals occupying i, j are IBD under the neutral drift (δ=0), s ( n, m, l) the probability that an individual accessible to node 1 via a (n, m, l)-random walk is a cooperator, f (n,m,l) the average payoff of individuals who can reach node 1 via a (n, m, l)-random walk, and V n m l . From equation (2), we have = - where Pr s s 1 1 . Note that the analytical result is valid for a wide class of population structures, from networks with no loops (e.g., square lattice) to those with loops (e.g., rings), from overlapped structures in different layers to non-overlapped structures. In particular, we recover a previous conclusion as a special case that gives r k k ) in infinite and unweighted random regular networks with a degree k [30]. Analogously, when the system is updated using the pairwise comparison or birth-death rule [52], we have r N Np Although equation (6) is applicable to many cases, given that the focus in this paper is investment and payoff allocation, we avoid making the spatial structure complicated. Thus we take two constraints: (1) Edges in different layers overlap such that d 0 ij ¹ and e 0 ij ¹ if and only if w 0 ij ¹ , and edges do not form triangle loops.
(2) Edge weights in the dispersal layer are identical, i.e., w ij k 1 = if i and j are connected.

Symmetry between investment and payoff allocation
In each generation, each individual participates in the self-centered and all neighbor-centered games. Thus, it must decide how much to invest in each game. Besides, as the initiator of the self-centered game, it probably plays a dominant role in allocating the produced benefits. A typical example is that a rectangular bacteria Escherichia coli diffuses more public goods to neighboring bacterium contacted along long edges [50]. The question is how these two decisions affect the evolution of cooperation and whether there is any correlation between them. Equation (6) shows that the evolutionary outcomes are determined by p (1,1,0) +p (1,1,1) for a given population of size N. Here we exchange the structures in investment and payoff allocation layers by exchanging their edge weights. That is, individuals make investments based on the structure in the payoff allocation layer and allocate payoffs based on the structure in the investment layer.
) , indicating that switching the structures of the investment and payoff allocation layers does not affect the evolutionary outcomes. The implication is that because investment and payoff allocation are symmetric, targeted interventions to social investment or to benefit distribution have the same outcome. In addition, the symmetry is valid when aforementioned constraints are released and when the system is updated using pairwise comparison or birth-death rule. Figure 2 shows the analytic predictions agree well with Monte Carlo simulations.
Note that this finding is not apparent at the individual level. Equation (2) indicates that after switching structures in interaction and payoff allocation layers, the benefit for individual i is f r e d s s Under the same strategy configuration, f f i i =˜is invalid. Thus switching structures affects i's viability. Although we constrain the research scope of this paper in regular networks, we stress that the symmetry between investment and payoff allocation is also visible in a few heterogeneous networks. Typically, in the starstructured population, a strongly heterogeneous network consisting of a center node and a few leaf nodes, we assume that the total investment amount of each cooperator is identical. Each player invests a fraction d in the self-centered game and invests the rest evenly in all neighbor-centered games. When allocating benefits, each player gains a fraction e of the total earnings from the self-centered game and the rest is distributed to all neighbors equally. We find that exchanging the values of d and e does not affect the evolutionary outcomes at all, applicable to various update rules such as birth-death, death-birth and pairwise comparison rules.

Nonuniform investment and uniform payoff allocation
We proceed with the study about how arrangement of investment and payoff allocation affects the evolutionary dynamics. Due to the node-transitivity of joint networks, p (1,1,0) +p (1,1,1) does not depend on node i. We use edge weights associated with node 1 to rewrite p (1,1,0) +p ( The value of p (1,1,0) +p (1,1,1) relies on the product of d 1j and e j1 . The former is the fraction of individual 1's investment in the PGG centered on j. The latter is the share of benefits allocated to individual 1 from the PGG centered on j.
We apply this finding to clarify a classic issue: how does heterogeneity in investment or payoff allocation affect cooperation? Because of the symmetry between investment and payoff allocation, we take uniform payoff allocation and focus on heterogeneous investment. Figure 3 shows that heterogeneity in investment can promote (o a  ), does not affect (o b  ), or inhibit (o c  ) the evolution of cooperation, implying that its effects on cooperation are multiple. According to figure 3, we can accurately predict when heterogeneity boosts cooperation. Actually, the impact of heterogeneity is determined by d 11 (the investment in the self-centered game) and is unaffected by any other d 1j (the investment in j-centered game). This shows that in the most frequently studied spatial PGGs, individuals' behavior in self-centered game and in neighbor-centered games do not have the same effect on social cooperation. Although benefits are allocated to all participants equally, increasing the investment in the self-centered game can confer cooperation more advantages, an effect never observed in prior studies [29,[39][40][41][42][43][44][45]. Figure 4 shows an example that makes this explicit. The cooperator cluster expands only when the cooperator in the boundary successfully disperses its strategy to the node occupied by the adjacent defector. Assuming a weak selection strength ( ) in figure 4(a). This indicates that investments in neighbor-centered games facilitate the invasion of defectors. Analogously, figures 4(a) and (c) show that investments in neighbor-centered games could weaken the expansion of cooperators. Thus under uniform payoff allocation, investment in self-centered games benefits cooperation more than investment in neighbor-centered games.

Nonuniform investment and nonuniform payoff allocation
Equation (9) shows that the success of cooperators relies on the correlation between individuals' investment in a game and shares of benefits from the same game. Although an individual cannot control the level of benefit received from each game, they can decide their investment level in each game. Here we fix the allocation fraction e ij i j , { } and determine how adjusting the investment can positively affect cooperation. We treat equation (9) as a linear polynomial of d 1j ( jäV ) with a coefficient that is termed return coefficient e j 1 . Here  = for j 1 ¹ . The optimal theory tells that increasing d 1j with a larger return coefficient e j1 causes a larger p (1,1,0) +p (1,1,1) and lowers the barrier for cooperators' success (see equation (6)). Thus, in most cases an increase in the investment to games with a larger share of benefits is more beneficial to cooperation (see figures 5(a)-(c)). For example, in figure 5(c), the benefit shares from games centered on individual 1, 2, 4 are   )does not change the fact that increasing d 11 is optimal to collective cooperation. Overall, the added term to e 11 increases the likelihood that augmenting investment in self-centered games is an optimal choice for establishing a cooperative society. Figure 5(d) shows that in the full configuration space of payoff allocation (e e e , , 11 21 41 ), the optimal investment zone for self-centered games is the largest.

Discussion and conclusions
Here we theoretically demonstrate the symmetry between investment and payoff allocation. This finding expands our understanding of the baseline model of spatial PGGs. It also indicates that in many cases adjusting one's investment strategies or its benefit allocation strategies lead to the same evolutionary outcomes. Thus when the intervention to benefit allocation is not accessible, targeting and structuring the investment could achieve the same goal, with potential use by policy makers, managers, and administrators. In addition, we find that in most cases investing more in games with larger potential returns often facilitates cooperation. The underlying mechanism is that cooperators can benefit more from their investment while defectors lose the chance of free riding on cooperators. This finding therefore highlights the importance to coordinate the investments and payoff benefits in different games. Intriguingly, heterogeneity in investment or payoff allocation is not always beneficial to cooperation. Heterogeneity here can be not only a promoter of cooperation but also its inhibitor, which is in sharp contrast with previous studies [36,59,60]. Traditional investigations into heterogeneity focus on the difference in individuals' attributes, such as possessing different numbers of social ties [59] and inhomogeneous teaching abilities [60] (see reference [36] and references therein). Heterogeneous attributes shape a few distinguished 'hub' individuals who have a greater capacity to spread strategies [59][60][61]. Hub individuals establish homogeneous strategy 'clouds' that surround them, i.e., cooperation 'clouds' around hub cooperators and defection 'clouds' around hub defectors. These clouds strengthen the viability of the hub cooperators and increase survival of cooperation, even under testing conditions. In particular, most prior studies exploring nonuniform investment or payoff allocation [29,[39][40][41][42][43][44][45] incorporate heterogeneity in individuals' attributes. For example, when each individual is assigned a random scaling factor to indicate benefit allocation, the scaling factor difference causes individuals' heterogeneity, which is partly responsible for the persistence of cooperation [41]. Here we investigate 'action' heterogeneity, such as the heterogeneous investment amount and the benefit distribution in different games. We find the effect of action heterogeneity on the evolution of cooperation relies heavily on the correlation among participants' actions. Positive correlations in heterogeneity facilitate the evolution of cooperation and negative correlations impede cooperation, indicating the double edged influence of heterogeneity. In recent studies [32,62], Su et al develop the interactive diversity in which each individual adopts and adaptively adjust different strategies against different opponents. They show that this diversity strengthens the strategy reciprocity between pairs of individuals and maintains large scale cooperation even in completely connected networks [62]. Research on 'action' heterogeneity will significantly enriches our understanding of collective behavior in complex systems.
A tacit assumption in spatial multiplayer games is that each individual engages in a self-centered and all neighbor-centered games in each generation [29,[39][40][41][42][43][44][45]. Few studies have examined how the same behavior in the self-centered game and neighbor-centered games affect the evolution of cooperation differently. Here we show that the former more strongly encourages cooperation. For example, in spatial PGGs with a uniform payoff allocation, an investment in self-centered games provides more advantages to the evolution of cooperation than the same level of investment in neighbor-centered games. This finding provides us new insights into escaping the tragedy of the commons without invoking other mechanisms such as reputation [13,20] and punishment [13][14][15][16][17]. Nevertheless, as seen in most prior studies, update rules often strongly affect the evolutionary dynamics [54]. When the system evolves using the birth-death or pairwise comparison rule (the probability that individual i is replaced by its neighbor j is proportional to their payoff difference f j −f i ), the same behavior in different games become equivalent in terms of stabilizing cooperation (refer to figure 4). Broom et al recently develop a new framework for investigating the interactions between territorial animals in which games are neither initiated by nor centered on individuals [63]. This eliminates the spatial correlations between games and individuals [35]. Ultimately, the same behavior in different games has an identical effect to the evolutionary outcomes. Equally important as evolutionary graph theory, it is a promising framework for exploring how local interactions affect evolutionary dynamics [64].
To focus on investment and payoff allocation, in this paper, we try to make the inter-layer correlation and intra-layer connection simple, i.e., making edges in different layers overlapping and avoiding structural triangles in a single layer. Actually, when these constraints are removed, the obtained formula is still applicable. A further analysis into this formula will help to understand how a few realistic structural features like community structure affect the evolution of social behavior. Nevertheless, we point out that the formula hinges on the widely used assumption of weak selection and regular networks. Extending our results to strong selection and any population structure will be a challenging and important topic in future work.