Successful conditional altruistic strategies and successful conditional spiteful strategies are different

Altruism and spite are costly, so from an evolutionary perspective their existence is mysterious. Both altruism and spite have been studied, but the latter has been investigated less than the former. One set of mechanisms that facilitates the evolution of altruism is repeated interactions. Upon repeated interactions, conditional altruistic strategies can be favoured by natural selection, and successful conditional strategies have already been proposed. Mechanisms that encourage the evolution of altruism can also encourage the evolution of spite. A previous study showed that spite could evolve under repeated interactions, by investigating (just) one conditional spiteful strategy. However, the kinds of conditional spiteful strategies that are successful/unsuccessful have not been comprehensively studied. Here, by using evolutionary game theory, I investigated what kinds of conditional spiteful strategies are favoured by natural selection under repeated interactions. The results showed that a successful spiteful strategy involves choosing the opposite of the opponent's last move and does not involve holding a grudge. In these two aspects, a successful spiteful strategy and a successful altruistic strategy are different. Spite is altruism's evil twin in the sense that repeated interactions facilitate both; however, the conditional strategies that are likely to evolve are different.

Social behaviour refers to the interactions and behaviours exhibited between two (or more) individuals within the same social pair (or group).In social behaviour, an individual can affect its own fitness and that of others.Social behaviour can be divided into four types, namely, altruism, spite, mutual benefit and selfishness, according to whether they have positive or negative effects on their own fitness and that of others (Hamilton, 1964(Hamilton, , 1970)).There is a need for investigation within the framework of natural selection to shed light on the evolutionary origins of the former two behaviours, despite their costliness to the actor.Many researchers have attempted to explain the existence of altruism (Hamilton, 1964;Nowak, 2006), but less attention has been paid to the evolution of spite (but see also Bruner & Smead, 2022;Fulker et al., 2021;Gardner & West, 2004;Konrad & Morath, 2012;Lehmann et al., 2009;Madgwick, 2020;West & Gardner, 2010).
On the one hand, animals engage in behaviours such as sharing food and assisting in each other's parenting.When they repeatedly interact and exchange roles, these actions can be seen as examples of repeated interactions.On the other hand, herring gulls occasionally kill chicks in neighbouring nests, and their behaviour of reusing the same nesting sites also represents a form of repeated interactions (Pierotti, 1980).Thus, repeated interactions are observed in both altruistic and spiteful behaviours among animals.
Spite is altruism's evil twin (Vickery et al., 2003).Mechanisms that facilitate the evolution of altruism can also facilitate the evolution of spite.Vickery et al. (2003), through evolutionarily stable strategy (ESS) analysis, studied repeated interactions and, by investigating one conditional spiteful strategy characterized by being spiteful in the initial encounter with strangers and subsequently adopting the opposite tactic used by the opponent, showed that this conditional spiteful strategy is successful.
This means that the evolution of spite is possible under repeated interactions.However, because Vickery et al. (2003) only investigated a single strategy, it remains unclear what kinds of conditional spiteful strategies are successful and unsuccessful.Thus, compared with research on altruism, less progress has been made on the evolution of spite under repeated interactions.The present study investigates what kinds of conditional spiteful strategies are favoured by natural selection under repeated interactions.
The remainder of this paper is arranged as follows.I first propose a model in which spiteful behaviour can be investigated.Using ESS analysis, I then study a competition between reactive strategies and an unconditional nonspiteful strategy and then competition between the strategy that responds to the opponent's single spiteful action with a nonspiteful behaviour and an unconditional nonspiteful strategy.Finally, I investigate replicator dynamics in a competition involving four strategies.

MODEL
In this model, I assume that individuals interact with other individuals at random.Consider an iterated game where individuals choose to be either spiteful or nonspiteful in each round.We assume that if an individual is spiteful, then it harms the opponent and, as a result, the individual pays a personal cost c while the opponent is damaged by h, where c > 0; (1) I assume that, in each interaction, any given pair continues to play the next game with probability d, where 0 < d < 1 (3) while their relationship terminates with probability 1 À d.As d increases, the expected number of rounds between a pair increases.

COMPETITION BETWEEN THE REACTIVE STRATEGY (R) AND ALLN
Consider the following strategy space.The space of memory-one strategies for a game between two players for the current case would be a vector of three probabilities: f, Q S and Q N , where f is the probability of being spiteful in the first move and Q i is the probability that an individual is spiteful in this round when the opponent adopted i in the last round (where S stands for 'spiteful' and N stands for 'nonspiteful').Here, 0 f ; Q i 1 is satisfied since f and Q i are probabilities.Let us consider the reactive strategy (hereafter referred to as R).When f ¼ Q S ¼ Q N ¼ 0, the strategy is always nonspiteful.This strategy is named AllN.
I explored the competition between these two strategies: R with f > 0 and AllN.An AllN strategy is always an ESS against the invasion of a strategy R with f > 0. The payoff for R when playing against R, denoted as E(R,R), and the payoff for AllN when playing against R, denoted as E(AllN,R), can be derived after performing algebraic calculations (refer to Appendices 1 and 2 for the derivation, respectively).Here, the condition under which strategy R is an ESS against the invasion of the AllN strategy is given by EðR; RÞ > EðAllN; RÞ.When f > 0, this condition becomes c h From ( 1) and (2), it is shown that the left-hand side of (4) is positive.Furthermore, (3) requires d to be a positive value.Therefore, Q N À Q S > 0 is a necessary condition for (4) to hold true.Additionally, when Q N À Q S > 0 is satisfied, there is a higher likelihood of (4) being satisfied for larger values of d.Namely, as the number of interactions increases, R is more likely to be stable against the invasion by AllN.Without repeated interactions, R is unstable because the left-hand side of (4) is positive and the righthand side of (4) is zero.This inequality does not contain the parameter (f ), which implies that the probability of being spiteful in the first move does not affect the evolution of spite.In addition, Q S À Q N can be regarded as the degree of retaliation and, as Q S À Q N increases, the right-hand side of (4) decreases.Retaliation has a negative impact on the evolution of spite.Besides, inequality (4) is the loosest (i.e. the right-hand side of (4) is largest) when Q S ¼ 0 and Q N ¼ 1 at which R is anti-tit-for-tat (ATFT; Baek et al., 2016).Spite is most likely to evolve upon being spiteful towards nonspiteful individuals and being nonspiteful towards spiteful individuals.'An eye for an eye and a tooth for a tooth' is not beneficial for the evolution of spite.When R is ATFT, (4) reduces to: As the number of interactions increases, ATFT is more likely to be stable against the invasion by AllN.

COMPETITION BETWEEN TRIGGER AND ALLN
In the previous section, I obtained the most successful strategy in reactive strategy sets.The strategy involves being spiteful towards nonspiteful individuals and nonspiteful towards spiteful individuals.Here, the interaction between two such strategies is SNSNS….This spiteful strategy harms each of the individuals, which does not efficiently lead to the evolution of spite.To avoid this, I examine a strategy where one will never be spiteful in future if the opponent is spiteful at least once (hereafter referred to as 'Trigger').Trigger involves keeping spiteful as far as the opponent is not spiteful.In the first move, Trigger involves being spiteful.
I explored the competition between the two strategies Trigger and AllN and derived conditions under which the Trigger strategy is an ESS against the invasion of the AllN strategy: As d increases, the right-hand side of (6) increases.Thus, as the number of interactions increases, Trigger is more likely to be stable against the invasion by AllN.Besides, the right-hand side of ( 6) is larger than the right-hand side of (5); therefore, ( 6) is considered a less stringent condition compared to (5).This means that Trigger is stable against AllN over a wider parameter space than ATFT is stable versus AllN.

THE ATFTdTRIGGERdALLSdALLN GAME
In the section Competition between the Reactive strategy (R) and AllN, I found that Trigger is more successful than ATFT when competing with AllN.However, in the model, Trigger and ATFT were not competing directly.Moreover, dealing with AllS (i.e.f ¼ Q S ¼ Q N ¼ 1) well is also important for evolution.In this section, I examine what happens in the replicator dynamics (Taylor & Jonker, 1978), considering a population with a mixture of individuals employing ATFT, Trigger, AllS and AllN.Consider a model with ATFT, Trigger, AllS and AllN and with the following payoff matrix: In a repeated game between two ATFT strategies, ATFTs exhibit spiteful behaviour exclusively during odd-numbered rounds, as depicted in Fig. 1.Similarly, in a repeated game between two Trigger strategies, Triggers engage in spiteful behaviour only in the first round (Fig. 1).This implies that when both players adopt the Trigger strategy, they can effectively avoid engaging in spiteful behaviour towards each other, leading to a more favourable outcome compared to the ATFT strategy.On the other hand, in a repeated game between two AllS strategies, AllSs display spiteful behaviour in every round (Fig. 1).In contrast, in a repeated game between two AllN strategies, AllNs exhibit nonspiteful behaviour in every round (Fig. 1).
Consequently, the payoff rankings are as follows: AllN versus AllN receives the highest payoff (0), followed by Trigger versus Trigger ( À c À h), and then ATFT versus ATFT ( ).Consider a repeated game between an ATFT (anti-tit-for-tat) strategy and a Trigger strategy.ATFT exhibits spiteful behaviour in all rounds except the second (Fig. 1).Conversely, Trigger displays spiteful behaviour only in the first round (Fig. 1).Now, consider a repeated game between an ATFT strategy and an AllS strategy.ATFT exhibits spiteful behaviour only in the first round (Fig. 1).Conversely, AllS exhibits spiteful behaviour in every round (Fig. 1).Next, I examine a repeated game between an ATFT strategy and an AllN strategy.ATFT exhibits spiteful behaviour in every round (Fig. 1).Conversely, AllN demonstrates nonspiteful behaviour in each round (Fig. 1).In a repeated game between a Trigger strategy and an AllS strategy, Trigger engages in spiteful behaviour only in the first round (Fig. 1).On the other hand, AllS exhibits spiteful behaviour in every round (Fig. 1).In a repeated game between a Trigger strategy and an AllN strategy Trigger engages in spiteful behaviour throughout each round (Fig. 1).On the other hand, AllN demonstrates nonspiteful behaviour in every round (Fig. 1).Lastly, in a repeated game between an AllS strategy and an AllN strategy AllS exhibits spiteful behaviour in every round whereas AllN displays nonspiteful behaviour in each round (Fig. 1).
Partly based on the preceding discussion, I derive a payoff matrix M (see Appendix 3).I denote the frequency of strategy i by P i .We have Let us denote F i as the expected payoff of strategy i (i ¼ ATFT; Trigger; AllS; AllN).We have: (9) where i ¼ ATFT; Trigger; AllS; AllN.Then, let F denote the average payoff in the population.We then have: I assume that the evolution (i.e.change of frequencies) of the strategies in the population can be described by the following replicator equation: I present formulas for the equilibria and their existence/local stability conditions in Table 1 (see Supplementary material for the derivations).T i is an equilibrium in which only i is present.T i;j is an equilibrium in which i and j are present.T i;j;k is an equilibrium in which i, j and k are present.T i;j;k;l is an equilibrium in which i, j, k and l are present.I also calculated the conditions under which the equilibria undergo transcritical bifurcations (see Table 2 for details).
Based on Table 1, it turns out that the equilibria that can be evolutionarily stable are (1) the equilibrium T AllN (i.e. a population consisting of individuals employing AllN), (2) the equilibrium T Trigger (i.e. a population consisting of individuals employing Trigger) and (3) the equilibrium T ATFT;AllS (i.e. the coexistence of ATFT and AllS).
Based on Table 1, it also turns out that the parameter space can be divided into three regions based on the presence/absence and stability of the equilibrium points.
Table 3 summarizes the presence or absence of each equilibrium.
For region (i), only (1) (i.e. the equilibrium T AllN ) is stable.The equilibrium T ATFT is stable against invasion by AllS; however, it is not stable against invasion by either Trigger or AllN.The equilibrium T Trigger is unstable against invasion by AllN.I selected ðc =h; dÞ ¼ ð0:3; 0:2Þ from region (i).In Fig. 3a, this specific parameter combination is used.Fig. 3a Unstable on the edge.Always stable against invasion by AllN.

Always unstable against invasion by AllS
T ATFT;AllS ðP ATFT ; P Trigger ; Is always a saddle point.

Always stable against invasion by AllN
T ATFT;Trigger;AllN ðP ATFT ; P Trigger ; Is always an unstable node.

Always unstable against invasion by AllS
which means that the frequency of AllS decreases over time.As a result, AllS vanishes from the population.
For region (iii), (1) (i.e. the equilibrium T AllN ), (2) (i.e. the equilibrium T Trigger ) and (3) (i.e. the equilibrium T ATFT;AllS ) are stable.The equilibrium T ATFT is stable against invasion by both Trigger and AllN; however, it is unstable against invasion by AllS.I selected ðc =h; dÞ ¼ ð0:12; 0:2Þ from region (iii).In Fig. 3c, this specific parameter combination is used.Fig. 3c suggests that the equilibrium to which the state converges depends on the initial state.

DISCUSSION
In this paper I investigated the evolution of spiteful behaviour under repeated interactions.The successful spiteful strategy and the successful altruistic strategy differ in the following two aspects.One is whether the successful strategy is retaliatory or not.From analysis of the repeated prisoner's dilemma game (e.g.Axelrod & Hamilton, 1981), it has been found that the successful altruistic strategy is retaliatory.However, the present study revealed that the successful spiteful strategy is not retaliatory (tit-for-tat) but one involving choosing the opposite of the opponent's last move (like ATFT).Here, note that retaliation in the context of altruism is free from cost and rather confers benefits on the actor, while retaliation in the context of spite is costly.Being retaliatory is beneficial evolutionarily when there is no cost in retaliation, while being retaliatory is not a smart choice when there is a cost to retaliation.
The other aspect by which the successful spiteful strategy and the successful altruistic strategy differ is whether the successful strategy involves holding a grudge or not.Which is more successful: the strategy involving responding to the latest move by the opponent or the strategy in which one will never be spiteful later if the opponent is spiteful at least once?In the context of altruism, the former (i.e.not holding a grudge) tends to be more successful than the latter (i.e.holding a grudge), although there are exceptions to this (see Axelrod & Hamilton, 1981;Zagorsky et al., 2013, for related studies).However, in the context of spite, a population consisting of individuals employing the latter strategy is stable, while a population consisting of individuals employing the former strategy is unstable, at least in my model involving four strategies.Not holding a grudge is a smart choice in the context of altruism, whereas holding a grudge is a smart choice in the context of spite.
In the preceding paragraph, I stated that a population consisting of Triggers is stable.However, even if Trigger can spread throughout the entire population, the actual incidence of spiteful behaviour between Triggers is rare, which does not explain the observed spiteful behaviour.As demonstrated in the section The ATFTdTriggerdAllSdAllN game, the only other stable states are a population consisting of AllN and a coexistence state of AllS and ATFT.The former population does not engage in spite.Therefore, if the population settles at the former state, spite will not be observed.The latter coexistence state (i.e. the coexistence of AllS and ATFT) results in spiteful behaviour, as AllS always employ spite against any other individual.Therefore, if spite is observed, it is restricted to the coexistence state of AllS and ATFT.Considering that the ancestral state was a population consisting of AllN, we can contemplate the transition from a population of AllN to a state where spite is observed.Since AllN is stable, it would require genetic drift or some external force to facilitate the initial evolution of ATFT, followed by the invasion of AllS, leading to the eventual settling into a coexistence state of ATFT and AllS (see Fig. 3c).This represents the scenario I envision for the evolution of spiteful behaviour.
This study dealt with a broader set of strategies than those dealt with by a previous study (Vickery et al., 2003).However, the set of strategies examined in the present study are still limited.For example, I did not investigate strategies that refer to the individual's own action in the previous round(s).In addition, the strategy that I investigated only involves memory of the last action and does not refer to actions further in the past.We are interested in what strategy is the most successful among a broader set of strategies.There is a possibility that we can find successful strategies when extending the set of strategies to directions that are different from the case of altruism; therefore, this should be considered.
I also did not investigate the case where execution errors occur.However, such errors do sometimes occur in actual situations in animal relations, and previous studies (e.g.Nowak & Sigmund, 1992, 1993) pointed out that successful strategies in the presence and absence of execution errors differ, at least in the context of the evolution of altruism.It would be intriguing to determine which conditional spiteful strategy is successful in repeated games in the presence of execution errors.
In this paper, it is assumed that, upon repeated interactions, players always know what the opponent did in the last move.However, in reality, players are sometimes unable to access such information.How should players behave in such a situation?Kurokawa (2017) found that a strategy referring to an individual's Depending on the presence/absence and stability of the equilibrium points, the parameter space can be classified into three distinct regions (i), (ii) and (iii).(i) Only (1) (i.e. the equilibrium T AllN ) is stable; (ii) (1) (i.e. the equilibrium T AllN ) and (2) (i.e. the equilibrium T Trigger ) are stable; and (iii) (1) (i.e. the equilibrium T AllN ), (2) (i.e. the equilibrium T Trigger ) and (3) (i.e. the equilibrium T ATFT;AllS ) are stable.own last move and imitating it is successful in the context of altruism.Persistence facilitates the evolution of altruism.Would the same result also be obtained in the context of spite?That is, does persistence facilitate the evolution of spite?Future studies on this issue are anticipated.
Retaliation has been simply considered to be one's behaviour that decreases the opponent's fitness when the opponent decreases one's fitness.On the one hand, a previous related study attempted to determine a successful strategy in a repeated game in which players choose altruistic behaviour or selfish behaviour in each round.The results showed that the successful strategy was selfish when the opponent was selfish in the previous round.A retaliatory strategy is thus likely to evolve.On the other hand, in this study, the goal was to determine a successful strategy in the repeated game in which players choose spiteful behaviour or mutualistic behaviour in each round.The results showed that the successful strategy was mutualistic when the opponent was spiteful in the previous round.Here, a nonretaliatory strategy is likely to evolve.From this, note that we cannot simply say that retaliatory strategies are likely to evolve.
How should we interpret these two results, which seem incompatible to understand in a uniform manner?Retaliation in both these situations is one's behaviour that decreases the opponent's fitness when the opponent has decreased one's fitness.However, in detail, these two types of retaliation differ in two aspects.The first aspect is whether the effect of the opponent's behaviour's on the opponent themselves is positive or negative (the effect is positive in the former, but negative in the latter).The second aspect is whether the effect of the retaliatory behaviour on the actor is positive or negative (the effect is positive in the former, but negative in the latter).The above two results, which seem incompatible, can be understood uniformly in (at least) the following two ways.One uniform interpretation is that the retaliatory (nonretaliatory) behaviour is likely to evolve when the effect of the opponent's behaviour on the opponent themselves is positive (negative).The other uniform interpretation is that the retaliatory (nonretaliatory) behaviour is likely to evolve when the effect of the retaliatory behaviour on the actor is positive (negative).To reveal which interpretation is appropriate, further study is required.However, irrespective of which interpretation is appropriate, the result that retaliatory strategies are likely to evolve, which has been believed to be true, does not hold true generally.Thus, the result of the present study urges us to divide the concept of retaliation into smaller pieces.
In previous studies, both selfish behaviour towards the opponent's selfish behaviour in the repeated prisoner's dilemma game and rejection when the offer was low in the ultimatum game (Harsanyi, 1961) have been treated as retaliation.Thus, retaliation has not been subdivided into smaller categories.Taking this into consideration, I consider that the present study is significant in that it urges us to subdivide retaliation into smaller categories.This shows a future direction in this research field, so the value of this work goes beyond the finding that the successful strategy is nonretaliatory in the repeated game in which players choose mutual benefit or spite.
My model shows that a population of individuals employing Trigger and a population with the coexistence of ATFT and AllS can be stable under some conditions.On the other hand, a population consisting of nonspiteful individuals (i.e.AllN) is always evolutionarily stable, although the threshold frequency of Trigger and (or) ATFT and (or) AllS necessary to converge on a population of individuals employing Trigger or the coexistence of ATFT and AllS is low when c h is small and (or) d is large.How could spite initially evolve?Although spatial clusters encourage the initial evolution of altruism (Nowak & May, 1992), the initial evolution of spite is considered to be rather disturbed by the presence of spatial clusters.Random drift could be one possible mechanism that helps the initial evolution of spite.An actor is not included in its own set of potential social partners, and, as a consequence, the actor is on average less similar to its social partner than to the population mean.When the population is small, this effect cannot be negligible, and the evolution of spite is possible.Considering this, evolution of spite at an initial stage is also possible (Grafen, 1985;  Vickery et al., 2003).Further study on whether any other mechanism enables the initial evolution of spite is required.
In this paper, social behaviour is categorized into four categories based on whether the impact of behaviour on the actor/recipient, in terms of immediate fitness consequences, is positive or negative, rather than considering its impact on lifetime fitness.However, some papers (e.g.Patel et al., 2020;West & Gardner, 2010) classify behaviours into four categories based on their effects on lifetime fitness.West and Gardner (2010) adopt the perspective of definitions based on lifetime fitness and argue that reciprocity is not altruistic.Similarly, if we adopt this perspective, the conditional spiteful behaviour favoured by natural selection in this study should not be considered spite in terms of lifetime fitness.Spite in terms of lifetime fitness requires kin discrimination (Patel et al., 2020).

Figure 1 .
Figure 1.Behaviours of individuals in the first five rounds.(a)e(d) Scenarios demonstrating interactions between the same two strategies.(e)e(j) Scenarios demonstrating interactions between two different strategies.
dP i dt ¼ P i ðF i À FÞ:
suggests that Trigger dominates the population if AllN is absent.However, after performing algebraic calculations (see Appendix 4 for proof), I can show if AllN is present that Trigger ) are stable.The equilibrium T ATFT is stable against invasion by AllS; however, it is not stable against invasion by either Trigger or AllN.I selected ðc =h; dÞ ¼ ð0:21; 0:2Þ from region (ii).In Fig.3b, this specific parameter combination is used.Fig.3bsuggeststhat Trigger dominates the population if AllN is absent, while AllN dominates the population if Trigger is absent.After performing algebraic calculations (see Appendix 5 for proof), I can show that AllN ðP ATFT ; P Trigger ; P AllS Þ ¼ ð0; 0; 0Þ Always exists Always stable against invasion by ATFT.Always stable against invasion by Trigger.Always stable against invasion by AllS T ATFT;Trigger ðP ATFT ; P Trigger Stable on the edge.Always stable against invasion by AllN.Always stable against invasion by Trigger T ATFT;AllN ðP ATFT ; P Trigger ; P AllS Þ ¼ Unstable on the edge.Always stable against invasion by ATFT.Always stable against invasion by AllS T ATFT;Trigger;AllS ðP ATFT ; P Trigger

Table 3
Summary indicating the presence or absence of each equilibrium ✓ indicates presence, while Â indicates absence.S.Kurokawa / Animal Behaviour 209 (2024) 143e153