Hierarchical prisoner’s dilemma in hierarchical game for resource competition

Dilemmas in cooperation are one of the major concerns in game theory. In a public goods game, each individual cooperates by paying a cost or defecting without paying it, and receives a reward from the group out of the collected cost. Thus, defecting is beneficial for each individual, while cooperation is beneficial for the group. Now, groups (say, countries) consisting of individuals also play games. To study such a multi-level game, we introduce a hierarchical game in which multiple groups compete for limited resources by utilizing the collected cost in each group, where the power to appropriate resources increases with the population of the group. Analyzing this hierarchical game, we found a hierarchical prisoner’s dilemma, in which groups choose the defecting policy (say, armament) as a Nash strategy to optimize each group’s benefit, while cooperation optimizes the total benefit. On the other hand, for each individual, refusing to pay the cost (say, tax) is a Nash strategy, which turns out to be a cooperation policy for the group, thus leading to a hierarchical dilemma. Here the group reward increases with the group size. However, we find that there exists an optimal group size that maximizes the individual payoff. Furthermore, when the population asymmetry between two groups is large, the smaller group will choose a cooperation policy (say, disarmament) to avoid excessive response from the larger group, and the prisoner’s dilemma between the groups is resolved. Accordingly, the relevance of this hierarchical game on policy selection in society and the optimal size of human or animal groups are discussed.


Introduction
Hierarchical structures are ubiquitous in society. For example, a human society or a country consists of people, while the world or a higher group of societies consists of countries or lower-level groups. Such a hierarchy exists in some animal societies, too, where herds in a region interact with each other. Within a group or country, individuals may cooperate or not, whereas a group (country) chooses apolicy ofinteractingwith other groups. Thus, interplay between the intra-group strategies of individuals and inter-group policy is important in understanding the social structure of cooperation.
In considering cooperation among individuals within a group, the public goods (PG) game is commonly adopted [1][2][3]. In the PG game, each individual has to pay a certain cost for the goods in the society, while not paying the cost wouldbe advantageous for the individual. How cooperation in a society is achieved has been extensively studied in the PG game [4][5][6]. For example, each person in a country is asked to paytax, with which theycan get equal welfare as a payoff from the country, depending on the total taxes collected. Here, the total payoff forthe country is maximized when all individuals cooperate in paying the required taxes. This achieves the Pareto optimum in an intra-group game. On the other hand, each individual can choose to free-ride, i.e., can tryto receive the payoff without paying the tax, for his or her own benefit. Indeed, the Nash equilibrium [7]or the optimal strategyfor an individual is achieved when everyonefree-rides, which, however, results in smaller payoffs. This situation is common to the standard prisoner's dilemma [8,9].
There have been previous studies on games for hierarchical situations [10][11][12]. In particular, games in which a deterministic contest between groups resultsfrom each individual's action have beeninvestigatedassocial models (i.e., voting) [13][14][15]. In such a simple game, however, only the group in which individuals contribute most in total receives all the maximum reward. To consider a more realistic competition, Rapoport and Amaldoss's model [16] is often used, where the reward is distributed among groups depending on the contribution of individuals. Since then, there have been successive studies onhierarchical games [17,18]. In such a hierarchical game, theindividuals within each group play a kind of publicgoods game, and individuals face the prisoner's dilemma within each group. In a relationship, Traulsen and Nowak discussed an evolutionary game ina hierarchical situation, and proved that cooperation within each group can develop as a result of selection at a group level [19]. Groups are selected depending on the fitness computed from the action of individuals, where a defector (parasite) gets an advantage at the individual level, but a group dominated by such individuals has lower fitness and is eliminated. Although the cooperation in a standard prisoner's dilemma leads to anincrease inbenefit forevery player, thisis not necessarily so for hierarchical games. Bornsteinpointed out that intra-group cooperationin all groups, in contrast, results in the decrease of every individual's benefit under a deterministic contest between groups with identical populations [20]. In other words, the resolution of the intra-group prisoner's dilemma leads to the appearance of another prisoner's dilemma. Such a new type of prisoner's dilemma is specifictohierarchical games, where not only individuals but groups need to play it. Surely in a hierarchical society, each group (e.g., country) also seems to playinter-group games sometimes, choosing an action according to its own policy.
To discuss this situation, westudy here a novel type of hierarchical game, in which each group also plays an inter-group game. In the inter-group game, groups compete with each other for a restricted amount of resources. For example, let us consider a simple allegorical situation in which each country struggles for finite resources using arms, for which taxes are collected from the people. Each country faces a strategic choice:either urge individuals within the country to cooperate in order to collect a large amount of taxes for armament or let them defectso that the smallamount oftaxes collected does not allow for armament. Accordingly, each individual has to choose whether to cooperate by paying tax or evade it. In this sense, the game in question is hierarchical in nature. Defectingat the individual level leads to cooperation at the group level, whereas cooperation at the individual level in paying tax leads to an arms race at the group level. In terms of game theory, the former implies a Pareto optimum between groups, whereas the latter leads to a Nash equilibrium for each one. Disagreement between the two strategies implies a prisoner's dilemma, which exists acrosslevels as well.
Here, we study this type of hierarchical game with a prisoner's dilemma across levels.
Another point we address in the present study is the dependence on group size, (i.e., thepopulation size of each group), which hasnot been discussed previously. Indeed, there often exists a collective effect forobtaining resources as a group. Hence, this group-size effect is introduced as the dependence of the individual payoff on group size for a discussion on how the policy of the group depends on its size. We analyze the optimal strategy choice both at the individual and group levels, and find a dilemma intrinsic to the hierarchical game, which we call the hierarchical prisoner's dilemma. We show that a hierarchical prisoner's dilemma appears when the two groups are not markedly different in size. However, with a large group size difference, defect(i.e., refusal to pay the cost) is favored at the individual level in a relatively smaller group, and the hierarchical dilemma is avoided.

Model
The Rapoport and Amaldoss [16] model is often used as a hierarchical game, in which a contest between groups is introduced as a result of each individual's action. In their model, an individual belongs to one of the groups and pays a cost to his or her own group, and each group utilizes all the costs collected from the individuals in the group to win the competition with the other groups. Based on their model, we make two essential changes. We take into account the games at both group and individual leveland introduce the populationsize dependence of the competition onresource capacity to consider the collective effect.
Thus, the capacity of each group to compete for resources depends not only on the collected cost but also on the number of individuals by considering the collective effect of the population. To be specific, the capacity is assumed to be proportional to both the αth power of the population and the summed cost. Here, α is a positive number that characterizes the efficiency of utilizing the cost for the competition: the larger α is, the greaterthe advantage of the larger group, while previous studies without populationsize dependence (e.g., Rapoport and Amaldoss's model) correspond to the case with a = 0.
This αth power of the population follows, for example, Lanchester's law [21], in which the strength of military forces increases with the αth power of the population. For example, a = 1 holds in a battle on a narrow bridge, and a = 2 in a wide field. Furthermore, this type of law is also applicable to competitions in biological groups [22][23][24].
We now formulate the hierarchical game for resource competition explicitly. Consider a situation in which N individuals are divided into L groups with populations and . is the total amount ofresource, which is divided intoL groups according to the capacity of each group, which is proportional to a X N i i . The first term in u ij gives the reward, that is, the resource distributed among the individuals according to the group's total cost, while the second term gives the cost paid by each individual. The payoff function is defined as the reward reduced by the cost paid. The payoff function equation (1) in the hierarchical game for resource competition is schematically shown in figure 1. Now the total payoff, written as U tot , is given by From equation (2), the moreeach individual or group pays, the less U tot is. U tot is obviously maximal when = x 0 ij holds for all i j , . This maximal U tot can be achieved when there is only one group, where each individual payoff is given by Hence, for all j u , j 1 is maximized when = x 0 j 1 holds: since the competition between groups does not exist, none of theindividuals are motivated to pay any costs. Note that even though the denominator in equation (1) goes to zero in this case, the payoff of equation (1) is not indefinite by itself, as the denominator and numerator are always equal.
On the other hand, wheninter-group competition exists ( > N 0 2 ), the payoff of equation ( . This is because a group can get all its resources by any small cost when none ofthe other groups pay anythingat all. Thus, the solution with = x 0 ij for all i j , does not appear to bean optimal solution. Hence, one does not need to include this  terminthe following study,astheaddition of this term would not change the following result at all. Section 3 provides the basic concept and behavior of our model, by focusing on a case with two groups (i.e., L = 2). We discuss the case of  L 3 in section 4.

Game between individuals
We now consider an intra-group game of individualswithin only a single group (i.e., group 1, the larger group), while the total cost in the other group X 2 is fixed.
First, we define the defect(d) and cooperative (c) strategy. In the former, the individual defectsand pursues his or her own benefit without sufficient cost (see below), while for the latter, individuals cooperate to pay the sufficient cost. Then, the action of the group consisting of d (c) individuals is defined as the C (D) policy respectively. Here, we will use the capital letter for group policy and the lower-case letter for individual strategy.
As will be shown later by explicit calculation, the group consisting of the d strategy does not pay a sufficient cost for the struggle with the other group and is cooperative atgroup level, because the group consisting of d individuals cannot collect sufficient cost, which results inpursuing the total benefit + U U 1 2 , rather than its own benefit U 1 . Therefore, C policy is interpreted as cooperative at group level. On the other hand, the group consisting of c strategy results in the defectpolicy atgroup level, as it collects sufficient costs for the struggle with the other group, for its own benefit,so that we use the term D policy. Now, when each individual chooses d strategy in group 1, the individual j determines his or her own cost as a function of X 2 , denoted as to maximize one's own payoff u 1j , whose condition is given by represents the total cost in the Nash equilibrium of group 1ʼs intra-group game as a function of the fixed cost of the other group. (Note that defectingindividuals does not mean that they do not pay any costs at all.) Then, the total payoff in group 1, denoted as ( ) Second, we consider the cooperative (c) strategy for the intra-group game in group 1, where the individual cooperates and pursues the whole group's benefit. Then, we define the group consisting of c individuals as the D policy. As will also be shown later, the D policy group can be interpreted as defectpolicy, because a sufficient cost by c individuals, in contrast, pursues its own benefit U 1 , not the total benefit + U U 1 2 . Now, to maximize the group's payoff U 1 , the cost of the individual j in group 1, represents the total cost for the individual Pareto optimum. Then, the total payoff in group 1, denoted as ( ) In the above, we defined C/D policy just as the group consisting of all individuals choosing d/c strategy. We now confirm that C/D policy means acooperative/defectgroup in the inter-group competition (see supplementary material for adetailed calculation is available at stacks.iop.org/NJP/19/073008/mmedia). > ( ) ( )

X X
1 D 1 C shows that the total payoff is larger for C policy than D policy. On the other hand, > shows that group 1ʼs own payoff is larger for D policy than C policy. We now show that a prisoner's dilemma exists in group 1ʼs intra-group game. Here, for simplicity, we assume that each individual pays the average cost for its c (d) strategy, in other words, From this, we get Equation (3) indicates that each individual should defect to pursue his or her benefit, while equation (4) indicates that all individuals as a whole should cooperate rather than defect fromeach other. Hence, the individuals play a prisoner's dilemma game for N i persons in group 1 (these two equations are shown by the convexity of the individual payoff function). The same argument can be applied to group 2 because the above equations hold regardless of N 1 . Therefore, the prisoner's dilemma game is also played in group 2 as an intra-group game.

Game between groups
Our game involves an inter-group game where each group chooses between the Cor Dpolicydepending on the individuals' actions. Now, the two groups have in totalfour sets of policies, indicates the policy in group 1 (2). Then, the equilibrium total cost in each group, denoted as ( ) ). With the assumption, as already mentioned, that each individual in the same group pays equally, = , the individual payoff in group 1 is obtained as , respectively. The payoff of each group is determined by each equilibrium point. Although there is a cross point at (0, 0), it is not an optimal strategy, so there is no problem in the indefinite payoff function in equation (1), as mentioned in section 2. This can be stated more accurately as follows: atthe optimal point, all players are required not to get more payoffs by changing theirstrategy. Since the payoff of each group is indefinite at (0, 0), we need to discuss a limit  X X , 0 1 2 in thevicinity. If the origin were optimal for group 1 (2), the payoffs for both groups should decrease for small positive   0 with ), for example, in the case of CC. However, the conditions cannot be satisfied for both groups simultaneously.  For CD, however, the merit of group size power and the demerit in the policy effect counterbalance each other, so that whether  u u j j 1 2 holds depends on N 1 . With large asymmetry between group sizes (  N N 1 2 ), the group size merit outweighs the policy demerit so that group 1 members receive higher payoffs. However, group 2 members receive higher payoffs when group size asymmetry is small (  N N 1 2 ). Second, we discuss the dependence of individual payoff u 1j in the larger group on N 1 . When  u u j j 1 2 holds, there exists an optimal N 1 for group 1 that maximizes u 1j , because the population increase in the dominant group (now group 1) has both merit and demerit. The merit is due to the population advantage for group 1 because the total group reward,  (1), increases with N 1 , which, however, gets saturated in the limit of  N N 1 . The demerit is the decrease in the distributed resources per individual within the group, that is, N 1 1 in equation (1). Thus, the larger group 1 has an optimal size, denoted by N 1 op . Such an optimal size exists only in a hierarchical game where both intra-and inter-group effects exist. On the other hand, when  u u j j 1 2 holds in the case of CD, u 1j monotonically increases with N 1 , because the saturation of u 1j with the increase of N 1 does not occur as it is smaller.
The dependence of the payoff u 2j on N 2 is explained in the same way. Except for the case of CD,  u u j j 1 2 holds, so that the payoff is not saturated with the increase in N 2 , and just increases monotonically, while for CD the saturation leads to an optimal value for N 2 .

Hierarchical prisoner's dilemma
Now we consider whether each group should choose the C or D policy. The payoff matrix of such a game is given by table 1.  1 DD also holds for group 1: consider the case where both groups change their policies from C to D. As group size asymmetry is small, the reward in equation (1) does not increase much with the cost. Since the numerator and denominator increase almost at an equal rate, the payoff (reward-cost) decreases for both groups. The same relationship among payoffs for the four sets of policies also holdsfor group 2. Now, to confirm the existence of a prisoner's dilemma in the present case, we compare the payoffs for CC, CD, DCand DD with those of the standard prisoner's dilemma. Here, we should note that the payoff matrix is different between the two groups as long as ¹ N N 1 2 . Let us then denote the payoff for the first group as T R P , , 1 1 1 and S 1 for the sets of policies DC, CC, DDand CD, following the standard notation in the prisoner's dilemma game (we use the suffix 2 for group 2). Then, the above calculation shows > > > T R P S i i i i for i=1, 2, implying the existence of a prisoner's dilemma.
Recall that the C (D) policy implies that all individuals in the group choose d (c) strategies. Therefore, the inter-group Pareto optimum (all groups choosing C policy) leads to the intra-group Nash equilibrium (all individuals choosing d strategy), while the inter-group Nash equilibrium (all groups choosing D policy) leads to the intra-group Pareto optimum (all individuals choosing c strategy). In other words, the resolution of an intragroup (inter-group) prisoner's dilemma leads to the appearance of an inter-group (intra-group) prisoner's dilemma. This result indicates that a prisoner's dilemma appears throughout the hierarchy, which we call the hierarchical prisoner's dilemma.
As for the analogy to the military game between two countries, each country tends to collect more taxes for the arms race. However, this increases the loss from war. In contrast, the C policy at the inter-group level or, in other words, d strategies at the intra-group level (i.e., refusal to pay taxes) will decrease the military cost, benefiting both countries.
This dilemma, however, does not exist when the asymmetry between group sizes is too large, figure 3 shows < ( ) ( ) u u j j 1 CC 1 DD conversely holds. This is explained as follows: as seen in figure 2 figure 2(A) with 2(B)). The hierarchical prisoner's dilemma when populations are identical between groups (or when the population effect is not included) has already been pointed out [20]. What is important in our study is that whether the hierarchical prisoner's dilemma exists or not depends on population asymmetry.

Excessive response
There is another reason for the disappearance of the hierarchical prisoner's dilemma for  N N 1 2 , where group 2 receives a higher payoff by following the C policy. Indeed, whether ( ) u j 2 YD or ( ) u j 2 YC is larger depends on group size differences, as shown in figure 3 and  2 DC holds. This ER is an abbreviation of excessive response to be explained below. In other words, group 2 should choose the C (cooperative) policy when its size is small.
We now explain the reason for this unexpected outcome. In figure 2(B), hold. When group 2 changes from theC to the D policy, its cost could increase if X 1 is fixed (see the broken blue line in figure 2(B). However, in response to the policy of group 2, group 1 will increase its cost, so that group 1 takes back the resource that could be lost with the cost increase in group 2. This response from group 1 is excessive for group 2, and the cost increase in group 2 is suppressed so that  ( ) ( ) X X 2 YD 2 YC holds (see the crosspoint DC in figure 2(B). Recall that the reward of group 2 is given by + a a a ( ) MX N X N X N 2 2 1 1 2 2 according to equation (1). Then, the reward of group 2 decreases with the increase in X 1 . Therefore, < ( ) ( ) u u j j 2 YD 2 YC holds, so that the hierarchical prisoner's dilemmaexists no more (i.e., > > > T R S P 2 2 2 2 or > > > R T S P 2 2 2 2 holds). As explained above, whether or not this 'excessive response' occurs is determined by the positional relationship between ( ) X i YD and ( ) X i YC , which depends on N N 1 . In summary, an excessive response by group 1 against the D (defect) policy of group 2 leads to this unexpected outcome. When individuals in group 2 play only the intra-group game, where the other group's summed cost is considered to be constant, they attempt to choose c (cooperative) strategies that lead to the D policy. By taking the inter-group game into account, they choose the C policy in order to avoid an excessive response from the other group.
Let us recall the analogy of the present game to the military game between two countries, where the individual cost corresponds to the military tax of the people, and cooperation (c strategies) within each group meansarmament. If a small country collects more tax for an arms race, the larger country increases its armaments in response, so that the payoff of the smaller country decreases. Hence, defection (d strategies), i.e., decreasing the military tax, is a better policy for the smaller country.

Dependence on N
The excessive response and hierarchical prisoner's dilemma discussed so far occurs for any total population N, while the region in which the former exists (defined as > N N 1 1 ER ) decreases with an increase in N. As shown in the supplementary material, the value

Dependence on α
So far, we have adopted a = 2.5. Indeed, the behaviorwe have reported here isuniversally observed as long as a > 0. The phase diagram for the regions with the hierarchical prisoner's dilemma and excessive response are shown in figure 4 against the change in α and Since α represents the advantage of the larger group relative to the smaller group, the region with excessive response (hierarchical prisoner's dilemma) decreases (increases) with a decrease in α.
Furthermore, the decrease in α below 2 alters the inequality between the payoffs in section 3.3. For a < > 1 CD holds. In other words, individuals in the smaller group receive higher payoffs than those in the larger group when the latter's policy is to defect. For details of the behavior for a < 2, see thesupplementary material.

Case with more than two groups
So far, we have studied the case with only two groups. The salient behaviorfound for the two-person case, particularly the hierarchical prisoner's dilemma and excessive response, occur for more than two groups as well, while the emergence of a group with a null payoff is a novel finding here.
We now discuss the case of three groups, again with   > N N N 0 . We define the two ratios ≔ R N N The payoff vanishes because the group is left behind in the competition. The weakest group cannot participate in the competition because competition from theother two groups is too severe, and no room is left for the weakest one. This left-behind phenomenon is also observed in the general L-groupgame. In general, group iʼs power is roughly proportional to a-N i 1 when it chooses the C policy and to a N i when it chooses the D policy. Then, the weakest group often cannot participate in the competition for resources. Now we discuss which policy each of the three groups chooses-D or C?Figure 5 suggests that only the DDD, DDCand DCC set of policies can be achieved to maximize the payoff. Indeed, we have plotted the set of policies that achieves the maximal payoff as a function of R 12 and R 23 in figure 6, which demonstrates the rule: when a smaller group chooses the D policy, larger groups also choose the D policy. When R 12 is small enough, the set of policies achieved is DCC, where an excessive response from group 1 to groups 2 and 3 changes the others' policies to C. Similarly, when only R 23 is small enough, the set of policies is DDC, where excessive response occurs from groups 1 and 2 to group 3. In general, excessive response occurs when the hierarchical game is played among groups with large population imbalances. Indeed, this excessive response and the rule of the allowable set of policies with DDLDCLC generally applies to the L-group case with (as will be discussed later).  Next, we discuss whether the hierarchical prisoner's dilemma exists between thethree groups. To examine this, we compute whetherseveral groups can obtain higher payoffs if togethertheyswitch from the D policy (which maximizes each group's payoff) to C simultaneously. Figure 7 shows the regions with such gains from DDD  CCC, DDC  CCC, and DDD  DCC as a function of R 12 and R 23 . First, we examine the case where both R 12 and R 23 are large enough (i.e.,   N N N 1 2 3 ), (see figure 6). In this case > DDD holds for all three groups, even though the set of policies results in DDD. This result indicates that a prisoner's dilemma happens amongall three groups. As α increases, this region is narrowed because a large α causes the strongly biased distribution of resources among groups. Second, if only R 23 is sufficiently small, i.e., for  3. This result indicates that a prisoner's dilemma happens between groups 2 and 3 while the regime is quite narrow since, when R 12 is smaller, an excessive response from group 1 to groups 2 and 3 forces the latter groups to choose C simultaneously in order to avoid competition with group 1. This result is also consistent with the two-group game.
The arguments so far are also true for more than three groups. Groups with a smaller size choose the C policy where the dilemma is resolved, whilefor groups smaller than some threshold size, the individual payoff turns out to be zero, as in the case with L=3. For example, we consider a situation in which the population decreases with a power law as b = -N i L i for L=7. Figure 8 shows the number of groups that choose the C policy, that is, groups with null payoffs, as a function of a b , . This indicates that they expand with the increase in α and β.

Summary and discussion
In aprevious study, Rapoport and Amaldoss proposed a hierarchical game, where groups probably compete for a limited amount of benefits, depending on each group's contribution collected from the individuals [16]. In the present study, we newly added the populationsize effect tothe typical hierarchical game. In addition, we also consideredan inter-group game where each group chooses either the C or D policy. Figure 6. The achieved set of policies when each group chooses its policy in order to maximize its payoff, for a = 2.5 (left), a = 1.5 (center) and a = 0.5 (right) with N=100. Each of the red, yellow and cyan points indicate the set of policies DDD, DDC and DCC.
Here we define that a group takes the C policy when it gets no payoff, where the payoffs by C and D policies agree. In this game, we investigated a class of dilemma-the hierarchical prisoner's dilemma-which isinherent in the hierarchical game: the cooperative/defect (C/D) policy of each group leads to defect/cooperative (d/c) behaviorfor all individuals in the group, and a prisoner's dilemma appears in both intra-and inter-group games. In the intra-group game, each individual prefers not to pay the cost-i.e., to choose d-but all individuals get lower payoffs when they choose d rather than c. In the inter-group game, each group prefers to pay the cost-i.e., to choose D-but all groups get lower payoffs from choosing D rather than C. Such a hierarchical prisoner's dilemma has already been noted in a certain case [20].
Here, werevealed that whether a hierarchical prisoner's dilemma exists depends on the group-size difference. When the difference is sufficiently large, the hierarchical prisoner's dilemma disappears for two reasons. First, the larger group gets an advantage with DD rather than CC. Second, both groups receive higher payoffs when the smaller group follows the C policy. Indeed, if the smaller group pursues D, i.e., it pays more to compete with the larger group, the larger pays more, too, so that competition for resources increases, and the smaller group suffers a larger loss from what isdescribed here as an excessive response. Hence, the smaller group abandons cost competition, so that the dilemma is avoided.
These findings may have some implications in anarms race or the struggle between groups. If the two groups or countries are not very different in size, they cannot avoid an arms race, which is costly for both members. With cooperative members in each group, the race would be stronger, and the costs larger. In contrast, when the two countries or groups are quite different in size, the smaller group would abandon the race, averting the loss. Of course, in reality, the interaction between groups and resource allocation to each is more complicated, and the choice of policy in each group is not simply determined by the actions of individuals. Nevertheless, the group size dependence of the dilemma and the achieved set of policies may be relevant to understandingreal society.
In the present model, we assume Lanchester's law as the population size effect, i.e., the power-law dependence. Of course, we can adopt other forms of population dependence, where most of our conclusions are valid. For example, consider the populationsize effect with saturation, as defined by In this case, if N i is much smaller than a K 1 , the above form is approximated by a X N i i , i.e., the present game of the power α. However, if a  N K i 1 , it is approximated by a constant (i.e., the 0th power). Hence, we can adopt the result of the present study depending on the group size, with some crossover behavior for the intermediate size N i .
In the present study, we assumedfor simplicitythat all individuals in a group follow the same action:the c (d) strategies in a D (C) policy. In reality, the cost thateach individual within the same group pays can differ according to the stage of the evolutionary process. Since there exists a prisoner's dilemma in both the intra-and the inter-group games in our model, the cost for the struggle paid by the group for the inter-group game decreases if the fraction of free-riders (d strategy) increases as a result of the intra-group game. Thus, the payoff could increase, depending also on the size of each group. Hence, the distribution of costs and payoffs within a group, together with its dependence on group size, wouldbe an interesting topic for discussion. Furthermore, it i . Each color, black, magenta, yellow, cyan, red, green andblue indicates the number of C groups with 0, 1, 2, 3, 4, 5 and6, respectively. For most regions, the payoff of all the C groups isnull. Only in the righthand region of the black solid line (blue and part of the green region) doesthe largest group tochooseC policy get apositive payoff, while the others still get a null payoff.