On the number of equilibria of the replicator-mutator dynamics for noisy social dilemmas

In this paper, we consider the replicator-mutator dynamics for pairwise social dilemmas where the payoff entries are random variables. The randomness is incorporated to take into account the uncertainty, which is inevitable in practical applications and may arise from different sources such as lack of data for measuring the outcomes, noisy and rapidly changing environments, as well as unavoidable human estimate errors. We analytically and numerically compute the probability that the replicator-mutator dynamics has a given number of equilibria for four classes of pairwise social dilemmas (Prisoner's Dilemma, Snow-Drift Game, Stag-Hunt Game and Harmony Game). As a result, we characterise the qualitative behaviour of such probabilities as a function of the mutation rate. Our results clearly show the influence of the mutation rate and the uncertainty in the payoff matrix definition on the number of equilibria in these games. Overall, our analysis has provided novel theoretical contributions to the understanding of the impact of uncertainty on the behavioural diversity in a complex dynamical system.


Introduction
Evolutionary game theory (EGT), which combines the analysis of game theory with that of dynamic evolutionary processes, provides a powerful mathematical and simulation framework for the study of dynamics of frequencies of competing strategies in large populations [19,15,4].This framework has been successfully used for the investigation of the evolution of collective behaviours such as cooperation, coordination, trust and fairness, and recently, for understanding several pressing societal challenges such as climate change and pandemics mitigation and advanced technology governance [27,21,13,25,22,31,5].
However, existing works on evolutionary game theory mainly focus on deterministic games which could not capture the different random factors that define the interactions, in particular the game payoff matrix [20,27,21].That is, these works assume that the outcomes of interactions for any group of strategists (aka the game payoff matrix) can be defined in advance with certainty.However, the uncertainty in defining such outcomes is in general unavoidable which can arise from a diversity of possible sources, including lack of data for measuring the outcomes, noisy and rapidly changing environments, as well as unavoidable human estimate errors [16,18,11].
On the other hand, existing works that capture such uncertainty factors in EGT usually assume the payoff entries of the game are general random variables whose distributions are not known a prior [7,10,14,38,6].In this line, random social dilemma games where [8,9] were also analysed, where the payoff entries satisfy certain ordering required for particular social dilemmas.These approaches are useful to provide generic properties of the underlying dynamical systems.
However, it might be the case that in many domains/scenarios, some knowledge about the payoff entries is available.In particular, some of the payoff entries might fluctuate around certain known values, which are for example estimated through data analysis or given by domain experts.Capturing the available information in the analysis is essential to more accurately describe system dynamics and evolutionary outcomes.
In this paper, we bridge this gap by analytically investigating the statistic of the number of equilibrium points in pairwise social dilemma games where some payoff entries are drawn from a random distribution with a known mean value and variance.We will consider the full space of (symmetric) two-player games in which players can choose either to cooperate or defect (see detailed definitions in Section 1.2) [23,27].These games have been shown to provide important abstract frameworks to capture collective behaviours in a wide range of biological and social interactions such as cooperation and (anti-)coordination.We adopt in our analysis the replicatormutator dynamics (see Sections 2.1 and 2.2) to model the evolutionary process, capturing both selection and mutation, allowing us to generate insights on how uncertainty in the interaction outcomes and these stochastic factors together influence equilibrium outcomes.Note that most previous works on random games consider replicator equations [14,7,10], which is a simple version of the replicator-mutator ones where mutation is assumed to be negligible, already general enough to encompass a variety of biological contexts from ecology to population genetics and from prebiotic to social evolution [26].A similar setting to our work was considered in [2,3] for studying the impact of uncertainty on the evolution of cooperation, but their analysis was purely based on simulations and did not use replicator or replicator-mutator equations.Moreover, there has been a particular interest in studying average and maximal numbers of equilibrium points in a dynamical system [17,14,1].Our analysis provides close forms for the probability of a concrete number of equilibria to occur, thus generalising these results.
The rest of the paper is organised as follows.In Section 2 we provide the background regarding replicator-mutator dynamics and summarise the main results of the paper.Detailed proofs will follow in Sections 3-5.

The replicator-mutator dynamics
The replicator-mutator equation describes the evolution dynamics in a population of different strategies being in co-presence, where selection and mutation are both captured.It is an well established mathematical framework that integrates the unavoidable mutation observed in various biological and social settings [29,57,62,52,32,61].This framework has been utilised in many application domains, including evolution of collective behaviours [46,20], social networks dynamics [54], language evolution [53], population genetics [42], and autocatalytic reaction networks [59].
We consider an infinitely large population consisting of n different strategies S 1 , • • • , S n .Their frequencies are denoted, respectively, by x 1 , • • • , x n , where n i=1 x i = 1.These strategies undergo selection where their frequency, S i , is determined by its fitness (i.e.average payoff), f i , obtained through interactions with others in the population.Such interactions happen within randomly selected pairs of individuals playing a social dilemma game (see details below).By means of mutation, individuals in the population might change their strategy to another randomly selected strategy, given by the so-called mutation matrix: Q = (q ji ), j, i ∈ {1, • • • , n}.Here, q ji stands for the probability of an S j individual changing its strategy to S i , satisfying that n j=1 q ji = 1, ∀1 ≤ i ≤ n.
Denoting vector x = (x 1 , x 2 , . . ., x n ) and f (x) = n i=1 x i f i (x) the population's average fitness, we can describe the replicator-mutator equation as follows [49,47,48,56] It is important to note that the replicator dynamics can be reproduced from ( 1) with q = 0 (i.e.no mutation).This paper investigates the equilibrium points of the replicator-mutator dynamics, which are solutions in [0, 1] n of the following system of equations where g i (x), i = 1, . . ., n, denotes the right-hand side of (1) In general, knowing equilibrium points in a dynamical system allows us to study states where different strategies might co-exist in the population, indicating the possibility of polymorphism.

Pairwise social dilemmas
Now let us consider a pairwise game with two strategies S 1 and S 2 , with a genaral payoff matrix given below where a 11 is the payoff that a player using strategy S 1 obtains when interacting with another player, who is also using strategy S 1 .Other notations are interpreted similarly.Denoting x as S 1 's frequency (and thus 1 − x as S 2 's frequency), we can simplify the replicatormutator equation as follows Using the identities q 11 = q 22 = 1 − q, q 12 = q 21 = q, Equation (2) reduces to Next, we consider a well-established parameterisation of pairwise social dilemmas, for random game analysis [24,30,28].In these games, players can choose to cooperate or defect in each interaction.Mutual cooperation (punishment) would lead to a payoff R (P ).Unilateral cooperation leads to payoff S while unilateral defection leads to payoff T .Without loss of generality, we normalize R = 1 and S = 0 in all games, and that 0 ≤ a 21 = T ≤ 2 and −1 ≤ a 12 = S ≤ 1.We focus on four important social dilemma games, characterised by different orderings of the payoff entries (i) the Prisoner's Dilemma (PD): 2 ≥ T > 1 > 0 > S ≥ −1 (both players defect), (ii) the Snow-Drift (SD) game: 2 ≥ T > 1 > S > 0 (players prefer unilateral defection to mutual cooperation), (iii) the Stag Hunt (SH) game: 1 > T > 0 > S ≥ −1 (players prefer mutual defection to unilateral cooperation), (iv) the Harmony (H) game: 1 > T ≥ 0, 1 ≥ S > 0 (both players cooperate).
As motivated in the introduction, to be more realistic and to capture the various possible uncertainty, we will consider random games, where T or S or both T and S, are random variables.

Main results
We obtain explicit analytical formulas for the probability that the replicator-mutator dynamics has a certain number of equilibria for the four social dilemmas above in two different cases, where T or S is random and the other is fixed.The distinction is necessary since T and S might play different roles in the equilibrium outcomes, which can be also seen from our results.
Theorem 1 (T is random).Suppose that T is normally distributed with mean T 0 and variance σ 2 , i.e.T ∼ N (T 0 , σ 2 ), and S is a given number where T 0 and S satisfy the corresponding ordering in each of the social dilemmas above.Let T 1 , T 2 , T 3 and s 1 , s 2 be defined in (10), (11) and (12).
Then, for all games, the probability that the replicator-mutator has 2 equilibria is given by The probability that the replicator-mutator has three equilibria is given as follows.First, for SD and H games

Now, for PD and SH games
Thus the probability that the replicator-mutator has one equilibrium is, for SD and H games p 1 = 0, and for PD and SH games, Furthermore, as a function of q, the probability p 2 is decreasing in SD and H games, but is increasing in PD and SH games and satisfies the following small mutation limit 1, in SD and H games, 0, in PD and SH games.
In addition, in SD and H games, it holds that Theorem 2 (S is random).Suppose that S is normally distributed with mean S 0 and variance σ 2 , i.e. S ∼ N (S 0 , σ 2 ), and T 0 is a given number so that T 0 and S satisfy the corresponding ordering in each of the social dilemmas above.Let S 1 , S 2 be defined in (19).The probabilities that the replicator-mutator dynamics has 1,2 and 3 equilibria are given by, respectively, . As a consequence, p 2 is always increasing as a function of q and satisfies the following lower and upper bounds In the following we will provide detailed proofs for these main results.In Section 3 we study the case where T is random and S is deterministic.We compute the probability that the replicatormutator dynamics has p k (k ∈ {1, 2, 3}) equilibria, both analytically and numerically by sampling the payoff matrix space.Similar results where S is random and T is deterministic are obtained in Section 4. In Section 5 we numerically investigate the case where both T and S are random.Summary and outlook is given in the final section, Section 6.

T is random
We first consider the case where only T is random.Specifically, we assume that where ε T is a centered random variable.Suppose that ε T is a centered normal distribution, ε T ∼ N (0, σ 2 ), and T 0 is a fixed number.It follows that This means that we have partial information about the value of T , which is randomly fluctuating (perturbed) around a deterministic value T 0 .In practical applications, this may come from estimations based on expert's advice or data simulations.By taking the value of the variance smaller and smaller, the value of T is more and more concentrated around T 0 , and in particular, by sending the variance to zero, we expect to recover deterministic games.

Equilibrium points
By simplifying the right hand side of (3), equilibria of a social dilemma game are roots in the interval [0, 1] of the following cubic equation It follows that x = 0 is always an equilibrium.If q = 0, (5) reduces to Note that for SH and SD games x * ∈ (0, 1), thus it is always an (internal) equilibrium.On the other hand, for PD-games and H-games, x * ∈ (0, 1), thus it is not an equilibrium.If q = 1 2 then the above equation has two solutions x 1 = 1 2 and x 2 = T +S T +S−1 .In PD, SD and H games, x 2 ∈ (0, 1), thus they have two equilibria x 0 = 0 and x 1 = 1 2 .In the SH game: if T + S < 0 then the game has three equilibria x 0 = 0, x 1 = 1 2 and 0 < x 2 < 1; if T + S ≥ 0 then the game has only two equilibria x 0 = 0, x 1 = 1 2 .Now we consider the case 0 < q < 1 2 .For non-zero equilibrium points we solve the following quadratic equation where we define Set t := x 1−x , then we obtain Thus, an equilibrium point of a social dilemma can be found from a positive solution of the following quadratic equation Using this relation, subsequently we provide numerical simulations and analytical results for the probability p k that each of the game mentioned above has a certain number, k ∈ {1, 2, 3}, of equilibria.For the quadratic function g(t), the discriminant is given by where a and c are defined in terms of T and S in (7).The number of positive roots of g is characterized in the following three cases: (a) g has no positive roots, which happens when g has no real roots (∆ < 0) or g has only negative roots ( ∆ ≥ 0, t 1 ≤ 0, t 2 ≤ 0).In this case, the replicator-mutator equation has only one equilibrium x = 0.
(b) g has one positive root, which happens when g has a positive double root (∆ = 0, or when g has one positive and one negative root (∆ > 0, t 1 t 2 < 0).In this case, the replicatormutator equation has two equilibria.
Since in both cases (Gaussian and uniform distributions) T follows a continuous distribution, we have P (T = T 1 ) = P (T = T 2 ) = 0.It follows that p 21 = 0. Thus which is equivalent to the probability that T > T 3 .Since T ∼ N (T 0 , σ 2 ), we obtain where erf(•) is the error function The following lemma presents some interesting qualitative properties of p 2 .
Lemma 1.As a function of q, the probability p 2 is decreasing in SD and H games, but is increasing in PD and SH games.As a consequence, for SD and H games, it holds that In addition, 1, in SD and H games, 0, in PD and SH games.
It is worth mentioning that the monotonicity and the small mutation limits above are independent of the specific values of S, T 0 and σ.The lower bound (14) indicates that in SD and H games, p 2 is always dominant over p 1 and p 3 since p 1 + p 2 + p 3 = 1.For instance, in the SD game, with the specific values S = 0.5, T 0 = 1.5, σ = 1 we obtain and in the H game with the specific values S = 0.5, T = 0.5, σ = 1, we obtain Therefore, using the chain rule and the fact that d dz erf(z) = 2 √ π e −z 2 , we obtain Since S is positive in SD and H games but is negative in PD and SH games, dp2 dq is negative in SD and H games but is positive in PD and SH games.Thus the probability p 2 , as a function of q, is decreasing in SD and H games, but is increasing in PD and SH games.As a consequence, for SD and H games, we have Now we establish the limit of p 2 as q tends to 0. Since 0 < q < 1 2 we have Together with the fact that lim z→±∞ erf(z) = ±1, we obtain Applying this to the underlying games, we achieve lim q→0 p 2 = 1, in SD and H games, 0, in PD and SH games.

Probability that the replicator-mutator has three equilibria
Now we compute the probability p 3 that the replicator-mutator equation has three equilibria.We consider the following cases, depending on the ordering between S and s 1 < s 2 .
Therefore, ∆ > 0 always holds.It also holds that T 3 < 1 − (S + 1)q 1 − q .Therefore, Thus, when (q − 1)S > q, the probability p 3 that the replicator-mutator dynamics has three equilibria is given by It implies that in this case, p 1 = 0. Note that, SD and H games always satisfy this case since s 2 = q q−1 < 0 < S. Thus, we obtain for SD and H games: (C2) S = s 2 Then 2q 3 − q 2 − (2q 3 − 3q 2 + q)S = 0, we obtain that A direct computation shows that − q(1 − q)S + q 2 + 2q − 1 (1 − q) 2 < T 3 .Thus, the probability that the replicator-mutator equation has 3 equilibria is again p 3 = P(T < T 3 ).Thus we obtain the same results as in the previous case.
It implies that the probability of the replicator-mutator equation having three equilibria is (C4) S ≤ s 1 Similarly to case (C3), ∆ = 0 has two distinct solutions T 1 and T 2 and Thus the probability that replicator-mutator equation has three equilibria is given by Bringing all cases together, we obtain, in PD and SH games, the probability that the replicatormutator equation has three equilibria is The three cases in the formula above can be written in terms of q as follows where q 1 and q 2 are respectively unique solutions of q(q − 1) 1 − 2q = S and q q − 1 = S.
We note that when q 2 ≤ q ≤ 1 2 then p 3 = 1 − p 2 , thus p 1 = 0. Using the above formula, in principle, we can derive qualitative properties for p 3 as a function of q as in Lemma 1.However, to compute the derivatives of T 1 and T 2 with respect to q and determine their signs for general S are very complicated for general S. In the following example, we demonstrate such a result for the specific value S = −0.5.
Example 1.For S = −0.5, we obtain the following formula for p 3 depending on the value of q From the above analytical formula, we can study the beheviour of p 3 as a function of q.For 0 < q < 2− √ 2 2 , we have dT 2 dq > 0.
Thus as a function of q, p 3 is decreasing in 2 ), is increasing in 3 ) and then is decreasing again in ( 13 , 1 2 ).

Numerical simulations
In Figure 1 we plot the probabilities that PD, SD, SH and H games have 1, 2 or 3 equilibria for different values of q using the analytical formula obtained in the previous section (see Theorem 1).For validation, these probabilities are also computed by sampling over 10 6 realizations of T from the normal distribution N (T 0 , 1) and then calculating the solutions of g(t) = 0.In these simulations, the value S 0 (respectively, T 0 ) is taken to be the middle point in the interval that S (respectively, T ) belongs to in each corresponding game, that is S 0 = −0.5 in PD and SH games and S 0 = 0.5 in SD and H games (respectively, T 0 = 1.5 in PD and SD games, T 0 = 0.5 in SH and H games).It can be clearly seen that the simulation results are in accordance with the analytical ones.In particular, we observe that, as a function of q, the probability p 2 is decreasing in SD and H games, and is increasing in PD and SH games.
In Figure 2 we plot the probabilities that PD, SD, SH and H games have 1, 2 or 3 equilibria for different values of σ (fixing q = 0.25), using the analytical formula obtained in the previous section.The value S 0 (respectively, T 0 ) is taken to be the middle point in the interval that S (respectively, T ) belongs to in each corresponding game, that is S 0 = −0.5 in PD and SH games and S 0 = 0.5 in SD and H games (respectively, T 0 = 1.5 in PD and SD games, T 0 = 0.5 in SH and H games).
p 3 that the replicator-mutator dynamics has respectively 1, 2, 3 equilibria as functions of the mutation rate q in PD, SD, SH and H games when T is random and S is fixed.The values from analytical analysis and numerical samplings are in accordance.The probabilities p 1 , p 2 , p 3 that the replicator-mutator dynamics has respectively 1, 2, 3 equilibria as functions of σ, the variance of the random variable T, in PD, SD, SH and H games when T is random and S is fixed.

S is random
In this section, we consider the case where only S is random, assuming that where ε S ∼ N (0, η 2 ) and T = T 0 < 1.Then we have As in Section 3, we will compute the probability that the replicator-mutator has k ∈ {1, 2, 3} equilibria, which is the same as the probability that the polynomial g defined in (8) has k − 1 ∈ {0, 1, 2} positive roots.This has been characterized in three corresponding cases (a), (b) and (c) in Section 3.1.

The probability that the replicator-mutator has two equilibria
In this section, we compute the probability that the replicator-mutator has 2 equilibria, which amounts to computing the probability that the quadratic polynomial g defined in ( 8) has 1 positive root.This can happen in two different cases below.
It follows from ( 17) that c ≥ 0 is equivalent to Y ≥ −q, which gives S ≥ −qT 0 1 − q .
From ( 16) and ( 17), we have Thus ∆ can be seen as a quadratic polynomial of S with a leading coefficient q 2 > 0 and its discriminant is given by ∆ = 16q 2 T 0 (1 − 2q) > 0 since for 0 < q < 1 2 .Thus The equation ∆ = 0 have two real solutions, S 1 < S 2 , given by To proceed, we need to compare −qT0 1−q with S 2 .We have Since both −qT0 1−q and S 2 are negative, it follows that −qT0 1−q > S 2 .Therefore Since S ∼ N (S 0 , η 2 ), the probability that g has a unique positive root, which is the probability that the replicator-mutator dynamics have two equilibria, is given by Lemma 2. As a function of q, the probability p 2 is increasing in all games.As a consequence, Proof.Let z := . Then p 2 = 1 2 − 1 2 erf(z).Thus, since T 0 > 0, we have Thus p 2 is increasing as a function of q, see Figure 3.As a consequence, we obtain the following lower and upper estimates for p 2 : The following example illustrates the above lemma.
Example 2. In the SD game, with the specific values S = 0.5, T 0 = 1.5, η = 1 we get In the PD game, with T 0 = 1.5, S 0 = −0.5, η = 1, we get In the SH game, with T 0 = 0.5, S = −0.5, η = 1, we get In the H game, with T 0 = S 0 = 0.5, η = 1, we get We notice that in SD and H games, p 2 is always dominant.

The probability that the replicator-mutator has three equilibria
In this section, we compute the probability p 3 that the replicator-mutator has three equilibria, that is the probability that g has two positive roots.We have To proceed, by comparing (q−1)(T0−1) q with −qT0 1−q and with S 1 , S 2 , we obtain 2 cases.
Thus, the probability p 3 in this case is given by p 3 = P(S < S 1 ).
(ii) When T 0 < (1−q) 2 1−2q , then we have Thus, the probability p 3 in this case is given by Therefore, since S ∼ N (S 0 , η 2 ), the probability of having three equilibria is given by In conclusion:

Numerical simulations
In Figure 3 we show the probabilities that PD, SD, SH and H games have 1, 2 or 3 equilibria for different values of q using the analytical formula obtained in the previous section (see Theorem 2).Moreover, for validation, these probabilities were calculated by sampling over 10 6 realizations of S from the normal distribution N (S 0 , 1) and then calculating the solutions of g(t) = 0.The values of T 0 and S 0 are the middle points of the corresponding intervals as in Section 3.4.It can be seen that the numerical results are clearly in accordance with theoretical ones.We can also observe that, p 2 is always increasing as a function of q, as stated in Theorem 2.
p 3 that the replicator-mutator dynamics has respectively 1, 2, 3 equilibria, as functions of the mutation strength q in PD, SD, SH and H games when S is random and T is fixed.The values from analytical analysis and numerical samplings are in accordance.

Both T and S are random: numerical investigations
In this section, we numerically compute the probabilities p 1 , p 2 and p 3 when both T and S are random.In Figure 4, we show their values obtained from averaging over 10 6 random samples of T and S with the corresponding distribution in each game.We observe that p 2 tends to increase in all games, while p 1 and p 3 exhibit more complex behaviours.We aim to study this more complex case analytically in future work.

Summary and Outlook
In this paper, we have studied pair-wise social dilemmas where the payoff entries are random variables.The randomness is necessary to capture the uncertainty that is unavoidable in practical applications, which may come from different sources, both subjective and objective, such as lack of data, fluctuating environment as well as human estimate errors.We have focused on four important social dilemma games, namely Prisoner's Dilemma, the Snow-Drift game, the Stag-Hunt game and the Harmony game.For each game, we have analytically computed, and numerically validated, the probability that the replicator-mutator dynamics has a certain number of equilibria, studying their qualitative behaviour as a function of the mutation rate.Our results have clearly shown that the mutation rate and randomness from the payoff matrix have a strong impact on the equilibrium outcomes.Thus, our analysis has provided novel theoretical contributions to the understanding of the impact of uncertainty on the behavioural diversity in a complex dynamical system.
Here we have assumed that the payoff entries are standard normal distributions; however, from formulas such as Equation ( 13), our results can be easily extended to other distributions.One natural and challenging problem for future work is to generalize the equilibrium analysis of the present work to multi-player and multi-strategy games where the payoff entries satisfy more complex conditions.Another direction is to trajectorially characterize statistical properties of the full dynamical systems.

Figure 1 :
Figure1: Probabilities p 1 , p 2 , p 3 that the replicator-mutator dynamics has respectively 1, 2, 3 equilibria as functions of the mutation rate q in PD, SD, SH and H games when T is random and S is fixed.The values from analytical analysis and numerical samplings are in accordance.

Figure 2 :
Figure 2:The probabilities p 1 , p 2 , p 3 that the replicator-mutator dynamics has respectively 1, 2, 3 equilibria as functions of σ, the variance of the random variable T, in PD, SD, SH and H games when T is random and S is fixed.

Figure 3 :
Figure3: Probabilities p 1 , p 2 , p 3 that the replicator-mutator dynamics has respectively 1, 2, 3 equilibria, as functions of the mutation strength q in PD, SD, SH and H games when S is random and T is fixed.The values from analytical analysis and numerical samplings are in accordance.

Figure 4 :
Figure4: Probabilities p 1 , p 2 , p 3 that the replicator-mutator dynamics has respectively 1, 2, 3 equilibria, as functions of the mutation strength q in PD, SD, SH and H games when both T and S are random.The values are numerically obtained from 10 6 samplings.