DeFinetti : A Mathematica program to analyze the replicator dynamics of 3-strategy collective interactions

DeFinetti is a Mathematica program that enables the user to visualize the replicator dynamics of three-strategy games with collective interactions, to study cooperation at different levels of biological complexity. The program computes the replicator dynamics of a game for multiple initial frequencies and shows trajectories, fitness and end points on a ternary plot (a DeFinetti diagram). The user only needs to specify the payoffs, and can change the parameters, visualize their effect, customize the style of the graphs and export the results. ©

. In a ternary plot, three quantities a, b, c corresponding to three types A, B and C are described by the position of a point within an equilateral triangle. The distance of this point from each side of the triangle is proportional to the quantity corresponding to the opposite vertex. Vertices denote pure types.

DeFinetti diagrams
A DeFinetti diagram (also known as ternary plot, ternary graph, triangle plot, simplex plot or simplex) is a plot of three variables that sum to a constant. The position of each point within the plot, which is bounded by an equilateral triangle, shows the values of the three variables. A ternary plot therefore enables the user to represent three variables in a two-dimensional graph. While not strictly necessary, since there are effectively only two variables (as the three variables sum to a constant), an equilateral triangle shows graphically the ratios of the three variables in an intuitive way: the concentration of each variable is 100% (pure type) at each corner of the triangle and decreases linearly with increasing distance from this corner, reaching 0% at the line opposite it (by Viviani's theorem: the sum of the distances from any interior point to the sides of an equilateral triangle equals the length of the triangle's altitude) [ Fig. 1].
Ternary plots are widely used, from chemistry and physics to geology and biology, to show the status of systems composed of three types. In biology, they are often used to describe not only static systems but also the dynamics of populations over time. In population genetics ternary plots have traditionally been used to show the changes in genotypic frequencies of systems with one locus and two alleles (resulting in three genotypes) [1]; ternary plots have also been of interest to evolutionary biologists and behavioral ecologists since the inception of evolutionary game theory [2] to study interactions between individuals with three strategies.

Public goods games
Public goods games (PGGs) are multiplayer games with collective interactions (payoffs are derived from fitness functions that depend on the behavior of multiple other players, not simply on the outcome of pairwise interactions). Examples of PGGs range from microbes secreting diffusible molecules [3] and cancer cells secreting growth factors [4] to cooperative hunting in mammals [5,6], birds [7] and arachnids [8,9], sentinels in cooperative mammals [10] and birds [11]. The exploitation of shared natural resources [12] is a classic case of PGG, and examples of collective interactions in environmental sciences and conservation biology range from land management [13] to problems related to global climate change [14,15]. Analogous collective action problems arise in most human social interactions and interventions for public goods, such as contribution to herd immunity in vaccination campaigns [16]. Hence, collective action problems for the production of public goods occur in many cases from microbiology to ecology and environmental sciences and are thought to be crucial in most of the major transitions in evolution [17]. While many types of PGGs assume that individuals can be either cooperators or defectors (hence use two-strategy games), many cases involve three strategies. Three-way symbioses [18][19][20] are obvious examples in which DeFinetti diagrams are useful. Another general case in which two strategies are not enough to describe cooperation for the provision of public goods occurs when the public good derives from a third party. For example, cooperators among a population of symbionts can provide diffusible nutrients to a host (whereas non-cooperative symbionts free-ride on the contributions of the cooperators) which in turn provides a benefit to the symbionts. The host here is a third type whose fitness and frequency changes over time, hence a three-player game must be used. Examples of indirect provision of public goods exist in cell biology too: in tumors, for example, cancer cells that produce or do not produce diffusible growth factors can engage in a public goods game with two strategies (cooperators and defectors); more commonly, however, cancer cells secrete molecules that induce other ''stromal'' cells to provide beneficial growth factors to the tumor -hence again, a three-way system that is described by DeFinetti diagrams [21,22].

Rationale
While extensions exist for certain programming languages to plot ternary diagrams, they do not allow to plot changes over time, which is often essential in biology. Dynamo [23,24], as well as its more recent companion EvoDyn-3s [25], are notable exceptions that enable the user to visualize different types of dynamics (including the replicator dynamics) of 3-strategy games on ternary plots. The recent Python package egtplot [26] has similar capabilities. They can be used to model three or four-strategy games, two-population, 3 × 2 strategy games and three-population, 2 × 2 × 2 strategy games but not multiplayer games with collective interactions (public goods games: PGGs).
I have developed a Mathematica program to plot the replicator dynamics of three-strategy multiplayer games with collective interactions. Here I describe the software, which is also available on https://github.com/archetti/DeFinetti. Examples of figures produced by the software can be found in references [22] and [27]; further examples are reported here.

Software description
DeFinetti is an executable program written in Mathematica. It contains the source code and intuitive instructions on how to use it. There is no need to compile it -it can be used directly by simply opening it with Mathematica. Parameters and style can be modified using a series of intuitive input boxes, sliders and menus, and the results are updated and exported either automatically and in real time or by pressing a button, without rerunning the program. The code, hidden in the interface, can be revealed (and modified, if necessary) by clicking on each section.

Software architecture
DeFinetti conducts the following operations: • Produces a three-strategy multiplayer public goods game with collective interactions, as described in the Appendix, using the following input provided by the user: names of the three types; group size; the cost c i for each type i in Eqs. • Computes the replicator dynamics numerically for the given starting points.  [20,20,20], [20,20,20], [20,20,20]  • Shows a series of trajectories and arrows showing the dynamics in a ternary plot.
• Shows the end points of the system in the ternary plot.
• Colors the trajectories in the ternary plot according to the average fitness of the population, using a color gradient selected by the user.
• Enables the user to add further starting points directly on the ternary plot.

Software functionalities
The user can specify the parameters of the model and the style of the plots using interactive input fields, sliders and popup menus. The results are updated either automatically in real time as the controls are changed, or they are generated upon pressing specific buttons, enabling a fast update of the results that are affected by the changes, with no need to run the program multiple times.
Files generated by the program are named path/time_filename, where path is the path of the directory where the DeFinetti.nb file is located; time is the time and date the file is generated (in this format: year_month_day_[hours_minutes_seconds]), which identifies each result; filename is the type of the file generated.
A text file (filename = Parameters.txt) listing the values of the parameters can be generated when the parameters are modified.
number, on a logarithmic scale to highlight the initial, usually wider, changes) for given initial frequencies [Fig. 2]. The initial frequencies of the three types can be changed using sliders and the results are produced in real time. The number of generations is initially set to 200, but can be changed. The user can also specify the colors of the plot. This plot is not exported by default, but can be saved by the user.
The core of the program produces ternary plots using the following coordinate transformation: if the pure A type has Cartesian coordinates (0,0) and pure B has coordinates (1,0), then the Cartesian coordinates of the pure C type in a ternary plot are (1/2, √ 3/2); hence a point {a, b, c} has Cartesian coordinates: In the ternary plot, each point represents the three frequencies at each generation; points are not actually shown; instead, they are joined by segments that form a trajectory describing the dynamics of the population over time. The color of each segment shows the average fitness of the population at that generation. The frequency of arrows, as well as the number of initial starting points and the colors representing average fitness can be customized using sliders.
In addition to choosing among a variety of color schemes, it is possible to normalize the fitness values (that is, rescale them between the observed fitness minimum and maximum). This is useful to highlight variations in fitness within the plot, which might be too small for some parameters; on the other hand, not rescaling the values (that is, using absolute fitness values) enables an easier comparison between plots with different parameters.
Finally, arrows are added to the trajectory to show the direction of the dynamics, and it is possible to choose whether to show the end points or not (which is useful, for example, in the case of cyclical dynamics) [Fig. 3].
The plot is exported to a file with filename = Simplex.pdf.
A separate file with a legend showing the values corresponding to the colors for average fitness is also produced and exported automatically (filename = Legend.pdf).
Additional starting points can be added in the final section of the software, by clicking on any point in the simplex [Fig. 3]. This part of the code uses the Locator command in Mathematica, and requires an inverse transformation, from Cartesian to Simplex coordinates: given Cartesian coordinates {x,y}, the corresponding coordinates on the simplex {a, b, c} are: Showing individual trajectories is sometimes useful in order to highlight peculiar dynamics from specific starting points that are not included in the default set. The user can decide whether to overlap the new trajectory with the ones already obtained or not. This plot is not exported by default, but can be saved by the user, and added (using any image processing software) to the ones already shown in the original standard plot.

Illustrative examples
Examples of ternary plots produced using different parameters are shown in Fig. 4. Further examples using different styles for the same parameters are shown in Fig. 5. Fig. 7 shows how DeFinetti enables the user to visualize the results of changing one set of parameters in the study of tumor-stroma interactions in multiple myeloma [22]: the three strategies are the three interacting cell phenotypes: osteoblasts (OB), osteoclasts (OC) and malignant plasma cells (MM). A healthy equilibrium of OB and OC cells can be destabilized by the introduction of MM cells. Fig. 7 shows how changing the position of the inflection points of the cytokines produced by the three types of cells (which simulates targeted therapy against those cytokines) changes and the dynamics and end points of the model.

Differences with previous software
The main difference between DeFinetti and alternate programs such as Dynamo [23,24], EvoDyn-3s [25] and egtplot [26] is that DeFinetti plots three-strategy games with collective interactions (the payoff of each player depends on the strategy of many other players) rather than pairwise interactions (the payoff of each player depends only on the strategy of its single opponent). Additional, minor differences include the ability to visualize the evolution of average population fitness (although at the expense of the visualization of the speed of the dynamics -a notable feature of Dynamo and EvoDyn-3s which is absent here); and the ability to add further starting points directly on the ternary plot, enabling the study of very specific initial starting frequencies.
A notable shortcoming of DeFinetti is that it is limited to the replicator dynamics (Dynamo and EvoDyn-3s can plot different types of dynamics).

Limitations and potential extensions
Determining the equilibria of three-strategy multiplayer nonlinear games with collective interactions is not feasible [27]. The end points in DeFinetti are inferred by looking at the values of the frequencies at the last generation. Because of this, two problems may arise. First, if the number of generations used is not high enough, multiple end points might seem to arise (because each trajectory will be truncated too early); this problem can be easily solved by choosing a high number of generations, and does not usually lead to any issue. Second, if not enough starting points are used, some end points may be missing if their basin of attraction is small (because no starting points are used within those basins of attraction); this is a more serious issue, which can be solved by choosing a large number of starting points -which however leads to the following possible other issue.
The only known serious issue with the current version is that under certain conditions (a high number of starting points and using normalized values for fitness colors), on some machines, the ''update simplex'' button does not actually produce a new simplex after updating the parameters. This seems to happen because the simultaneous use of these two options requires a high computational effort. Three solutions are possible: choosing a lower number of starting points (and if necessary eventually adding individual trajectories as explained above); using absolute values for fitness colors (instead of normalizing them); and plotting the dynamics without colors. These options are all easily selected using interactive commands. In addition, if normalized fitness colors for many trajectories are required, the problem can sometimes be solved by evaluating manually (selecting and pressing shiftenter) the line of code corresponding to the ''update simplex'' button (rather than simply pressing the button). This issue, when it happens, does not have an impact on the other parts of the program.  [20,20,20], [20,20,20], [20,20,20]].
The program allows to change the parameters that specify the benefit functions used for the three types, allowing to study the dynamics of multiplayer public goods games with monotonic nonlinear benefits ranging from linear, concave and convex to sigmoid and step functions. It does not include more complex, non-monotonic functions. Users with a basic knowledge of Mathematica could easily modify the source code in order to extend it to more complex benefit functions or other types of threestrategy games. Alternatives to the replicator dynamics could also be introduced, like Dynamo [23,24] and EvoDyn-3s [25] do for games with pairwise interactions.

Conclusion
DeFinetti enables to visualize the replicator dynamics of threestrategy games with collective interactions, an advance over existing programs that only allow pairwise interactions. The most obvious use of DeFinetti is in the study of public goods games, which is useful in many fields of biology, from microbiology and cancer research to behavioral ecology and human social behavior. I would like to encourage colleagues to suggest additional improvements and extensions.

Declaration of competing interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Appendix. Replicator dynamics
In evolutionary game theory, the change in frequency over time is often assumed to be based on the replicator dynamics [28] x where x i and W (i) are, respectively, the frequency and the fitness of type i. Hence the rate of change x ′ i /x i is equal to the difference between the fitness of type i and the average fitness of the population. As is standard in models of large, well-mixed populations, fitness is calculated by considering the payoffs obtained in randomly formed groups of size N weighted by the probability that such groups occur.
In a system with three types (i = A, B, C ) we can define x A = a; x B = b; x C = c (with a +b+c = 1); in a well-mixed population, the probability that a group contains n A , n B , and N − n A − n B individuals n i of type A, B and C, respectively, is given by and the payoffs for A, B and C are, respectively: where c i is the cost for type i and b i,j describes the effect of type i on type j. Note that each player's strategy has also an effect on itself. The fitness of type i is given by The frequencies change based on the replicator dynamics: x where The benefit function, which describes the effect of each type on each other, is defined by the normalized version of the logistic function The parameters define the effect of type i on type j: B i,j defines the maximum benefit; h i,j defines the position of the inflection point: h i,j →1 gives increasing returns and h i,j →0 diminishing returns; s i,j defines the steepness of the function at the inflection point (s i,j →∞ models a threshold public goods game; s i,j →0 models a linear benefit [Fig. 6]; and the normalization in (9) in prevents the logistic function (10) from becoming constant for s i,j →0).
This framework enables to analyze a wide range of non-linear multiplayer games with collective interactions [27,29].