Defensive Strategies May Stabilize Unstable Competitive Systems: A Model of “Rock-Paper-Scissors-Lizard-Spock” in the Chemostat

Traditionally, “rock-paper-scissors” is a children’s game in which rock attens paper (rock wins), paper covers rock (paper wins), and scissors cut paper (scissors win). When two individuals play the game with their hands, rock is represented by a fist, scissors by two fingers extended to represent a pair of scissors, and paper by a attened hand. At the count of three, the two players extend their hands into one of the forms described. Rock breaks scissors so defeats scissors, scissors cut paper so scissors defeat paper, and paper covers rock so paper defeats rock. Viewing


Introduction
Traditionally, "rock-paper-scissors" is a children's game in which rock attens paper (rock wins), paper covers rock (paper wins), and scissors cut paper (scissors win). When two individuals play the game with their hands, rock is represented by a fist, scissors by two fingers extended to represent a pair of scissors, and paper by a attened hand. At the count of three, the two players extend their hands into one of the forms described. Rock breaks scissors so defeats scissors, scissors cut paper so scissors defeat paper, and paper covers rock so paper defeats rock. Viewing the relationship between the rock, paper, and scissors, each has an advantage over the other. Although "rock-paper-scissors" may be an amusing way to settle simple disagreements between two friends, "rock-paper-scissors" occurs in meaningful and interesting biological situations as well.
Kerr et al. [1], describe an interesting relationship between three populations of Escherichia coli that produce toxins against each other resulting in a biological "rock-paper-scissors" relationship. Their experiments in a chemostat environment showed that coexistence of all three populations could occur or that one or two of the populations might "win" and eliminate one or both other competitors. Generally, they saw that one population evolved (mutated) and dominated the mixture after one week. Roelke and Eldrige [2], expanded on these results in their article.
Another interesting study is by Karolyi et al. [3], who examine a similar problem and demonstrate that the competition is affected by the distribution of the competitors. They further show that by chaotic mixing of the nutrient (with density S(t)) in what follows) that there are oscillations in the population densities, which in this paper we interpret as limit cycles.
Competing species problems have been extensively studied theoretically and experimentally. We will not list an exhaustive reference of such studies but rather mention a few that seem particularly interesting in the context of the model we develop here. In a general sense, competing species in the chemostat are studied in Smith and Waltman [4]. Many variations of the basic model of competing species in the chemostat have been studied. For example, Li [5] studies a competition model with three competing species competing for three nutrients. His study is based on the experimental results seen by Husiman and Weissing [6], which shows that cyclic competition, such as in a "rock-paper-scissors" relationship, can result in periodic oscillations (or limit cycles). More recently, Cameron et al. [7], examine a "rock-paper-scissors" relationship involving a parasitic plant and experimentally show that coexistence of the three competitors occurs and confirm the conclusion with biological observations. Reichenback et al. [8], obtain similar results. More recently, in Hsu and Roeger [9], use the May-Leonard competition models to study more complex competition relationships in the chemostat.
Unlike these previously studied models, we give each competitor in a "rock-paper-scissors-lizard-Spock" relationship a defense by giving it the ability to produce a toxin against its enemies (or competitors). The production of anti-competitor toxins is of interest when the weaker competitor can devote some of its resources to the production of a toxin (or inhibitor) against its competitors at some expense to its own growth. In Hsu and Waltman [6], this was modelled as a constant proportion of the resources. Biologically, this makes sense. For example, in Majeed et al. [10], it is documented that mixing strains of Escherichia coli, some of which produce bacteriocins, which are poisons that have a negative effect on competing organisms, the end result can lead to coexistence of all competitors. Inhibitors (including those added to the environment as well as those produced by the organisms) in the chemostat have been studied in Chao and Levin [11], Levin [12], Hsu and Waltman [13],

Background of competition models in the chemostat
The chemostat is a basic model for competition in an open system and a model for the laboratory bioreactor. Such models have applications in ecology like modeling a simple lake or in biotechnology to model the commercial bio-reactor. See Frederickson and Stephanopoulos [19], or Smith [20], for a general discussion of competition and Hsu et al. [14], or Smith and Waltman [4], for a mathematical description of competition in the chemostat and detailed analysis of Equations 2 and 3 that follow. Refer to Hansen and Hubbell [21], for experimental verification of the match between theory and experiment in the chemostat.
In this paper, we consider a basic, resource-based model of competition of which the chemostat is the standard example. Such models have applications in ecology to model a simple lake, a simple digestive system, or in biotechnology to model the commercial bioreactor. Experimental verification of the match between theory and experiment can be found in Hanson and Hubbel's study [21]. For a general discussion of competition models in the chemostat see Frederickson and Stephanopoulos [19], or Smith [20].
First, we revisit a model of three competitors competing for three nutrients and give each competitor a defense by giving it the ability to produce a toxin against its enemy (or enemies) in a "rock-paper-scissors" relationship. Next, we form a model of five competitors competing for five nutrients and give each competitor a defense by giving each competitor the ability to produce a toxin against its enemy (or enemies) resulting in a "rock-paper-scissors-lizard-Spock" relationship. The simulations we obtained for five competitors mirror those obtained for three competitors and indicate that giving competitors a defense may stabilize some unstable biological relationships.

Background of a model of "rock-paper-scissors" in the chemostat
To formulate a continuous model of a biological "rock-paperscissors-lizard-Spock" relationship in the chemostat, we start with the basic models introduced by Smith and Waltman [4], that are summarized next and then the competition models and, specifically, the model when one competitor produces a toxin studied by Hsu and Waltman in [13]. The model we state here is developed and analyzed analytically in [22]. We restate the model and our approach here is strictly numerical to help motivate the development of the model of "rock-paper-scissors-lizard-Spock" in the subsequent sections [22].
We restate these models of competition and then of "rock-paperscissors" in the chemostat that are developed in [22] to motivate the model of a "rock-paper-scissors-lizard-Spock" in the chemostat where the competitors are given a mechanism (such as a toxin) to defend themselves against their competitors.
We begin with the standard model for two competing organisms in the chemostat. Let S(t) denote the concentration of a nutrient at time t, and x 1 (t) and x 2 (t) denote the concentration of the two competing microorganisms. The basic model for competition in the chemostat between two competitors takes the form (note that ' d dt = = where S (0) is the input concentration of the nutrient j, D is the washout rate, m i , the maximal growth rates, a i , the Michaelis-Menten constants and n i , i=1, 2, the yield constants. This is usually called the Monod Model or the model with Michaelis-Menten dynamics. Refer to Smith and Waltman [4], for extensive details.
When scaled, Equation 1 for two competitors in the chemostat as stated in [4] becomes The scaled competition models in the chemostat for when one competitor produces a toxin at a constant rate are studied in by Hsu and Waltman in [13] In Equations 2 and 3, k represents the fraction of nutrient compensation of the x 2 organism allocated to the production of the toxin. P(t) represents the concentration of the toxin present. We assume that that interaction between the toxin and the affected microorganism is of mass-action form, -γPx 1 . In both equations, S(t) denotes the concentration of the nutrient in the chemostat. In the first equation, x 1 (t) and x 2 (t) denote the concentration (or density) of the competitors. In the second equation, x 1 (t) denotes the density of the toxin sensitive organism and x 2 (t) denotes the density of the toxin producing organism. P(t) denotes the density of the concentration of the toxin in the chemostat.
Using the same notation as used with Equation 3, the model studied by Li [5], for three species competing for three essential (or complementary) nutrients in the chemostat is In Equation 4, D represents the ow rate of the chemostat, (0) j S is the initial concentration of nutrient j, c ji is the content of nutrient j in species i. Because we are assuming that all three nutrients are essential, the growth rate of species i is determined by the nutrient that is most limiting: where f ji (S j ) is the growth rate of the species i when nutrient with density S j is limiting. As described previously, we take where the m ji are the maximal growth rates and a ji are the Michaelis-Menten constants, as in Hsu and Waltman [13], in Equations 2 and 3 that were described earlier.

Stabilizing an Unstable System with Three Competitors Competing for Three Essential Nutrients
Li's [5] primary result essentially says that Equation 4 has limit cycles that result when competitor x 1 is the better competitor for nutrient S 2 and the weaker competitor for nutrient S 1 but if x 1 becomes limited by S 3 , x 2 is the better competitor for nutrient S 3 and the weaker competitor for nutrient S 2 but becomes limited by S 1 , and x 3 is the better competitor for nutrient S 1 and the weaker competitor for nutrient S 3 but becomes limited by S 2 . Hulsman and Weissing [6], numerically illustrate the theorem in [6].
To be consistent with notation with [22] and [23], let S i (t), 1 ≤ i ≤3, denote the concentration of the three nutrients at time t, x 1 (t) the concentration of the organism susceptible to the toxin secreted by the organism with concentration x 3 (t), x 2 (t) the concentration of the organism susceptible to the toxin secreted by the organism with concentration x 1 (t), x 3 (t) the concentration of the organism susceptible to the toxin secreted by the organism with concentration x 2 (t). The concentrations of the toxin producing organisms are given by P i (t) (for x i ). An underlying assumption is that the chemostat is well-stirred so the nutrient is equally available to all competitors.
After incorporating toxin production in the same way as Hsu and Waltman in [13], Equation 4 becomes is the input concentration of the nutrient, D is the washout rate and, as before c ji is the content of nutrient j in species i. (0) j S and D are usually controlled by the experimenter although in real life situations such as studying how Escherichia coli interact in a (human) digestive system might not be controllable by the individual involved. The Equations 7 are usually called the Monod Model or the model with Michaelis-Menten dynamics as mentioned previously [4]. For the purposes, here k i represents the fraction of potential growth devoted to producing the toxin. If one had chosen to not start with a standard chemostat model, another sensible choice would have been starting with the May-Leonard competition equations as in Hsu and Roeger [9].
The interaction between the allelopathic agent (toxin producing organism) and the sensitive organism have been taken to be of mass action form, γ i P i x j . This is common in modeling when an interaction depends on the concentrations.

The case for non-constant k
Both Karolyi et al. [3], and Reichenbach et al. [8] mention that interaction between the organisms was considered or observed. In particular, this would imply that the organisms have some ability to sense those around them. Through mechanisms known as "quorum sensing" bacteria are able to control the expression of their genes in response to density of other bacteria in their environment. Quorum sensing mechanisms have been demonstrated to play a role in the control of gene expression associated with diverse activities like bioluminescence, in massing to form biofilms, and the expression of the gene code for characters responsible for the pathogenicity of these organisms. For example, Sandoz et al. [24], describe a situation in which mutations of the bacteria Pseudomonas aeruginosa use quorum sensing to "cheat" and coexist with non-mutated strains of the bacteria. In a review on quorum sensing, Bassler [25] provides a large bibliography and reading list on this fascinating subject. In her review, she states that "Recent studies show that quorum sensing modulates both intra-and inter-species cell-cell communication. " In discussing a particular case she notes that "the capacity to respond to both intra-and inter-species signals could allow Harveyi to know not only its own cell density, but also the relative frequency of bacteria of its type in mixed populations. " Thus, if a bacterium has the ability to sense the current state of its habitat and the presence of other bacteria, we believe that it is reasonable to conclude that k i is not constant but rather k i =k i (x 1 , x 2 , x 3 ) is a function of x i . The problem is magnified so that little rigorous analytical results can be obtained for a general k i (x 1 , x 2 , x 3 ) and so it is necessary to consider special cases. We note first that there are two undesirable cases. If k i =0, no agent (toxin producing competitor) is produced and the weaker competitor becomes extinct. Similarly, if k i =1, all uptake is devoted to toxin production and none to growth, so again the competing organism cannot survive. k i (x 1 , x 2 , x 3 ) must fall between these extremes of 0 (no toxin produced) and 1 (the growth rate of the organism is then 0 and the organism faces extinction).

My enemy's enemy is my friend
In a rock-paper-scissors relationship, a given organism is affected by both the organism affected by its toxin as well as by the organism that produces a toxin against the organism. For this choice of k i , we consider "my enemy's enemy is my friend" to be a natural strategy to consider. However, a given organism might want to detect the densities of both competitors. To take this into consideration in the competition model, the toxin producing organism, say x 1 , might produce its toxin against the x 2 organism at a rate inversely proportional to the density of the x 3 organism because the (enemy of x 1 ) x 3 organism produces a toxin against the x 1 organism. Thus, when the x 1 organism decreases its toxin production against the x 2 organism, the x 2 organism (friend of x 1 ) is allowed to devote more of its resources to producing its toxin against the x 3 organism (enemy of x 1 ), which then benefits the x 1 organism by lowering the x 3 population and less toxin is produced against the x 1 organism. On the other hand, when the x 2 population (density) is high but the x 3 population (density) is low, x 1 devotes more of its resources to producing its toxin against (its enemy) the x 2 organism. We illustrate the enemy/friend relationship in Table 1. We then continue a Friends Enemies x 3 x 1 x 2 Table 1: Relationships between x 1 , x 2 , and x 3 in the context of a "rock-paperscissors" relationship. Observe that in a "rock-paper-scissors" relationship, if x i is x j 's "friend" then x j is x i 's "enemy". Friendship is not transitive in the "rock-paper scissors" relationship. similar process for the other organisms. Paradoxically, x 2 is a friend of x 1 because x 2 produces a toxin against x 1 enemy x 3 who produces a toxin against x 1 , even though x 1 produces a toxin against x 2 . The same analogy carries over for x 2 and x 3 .
We try to choose simple choices of k i (x 1 , x 2 , x 3 ) that take this into consideration. In our discussion, we choose looks the same as Equation 7 but we keep in mind that the k i (x 1 , x 2 , x 3 ) now are given by Equation 8.

Rock-Paper-Scissors-Lizard-Spock
Some believe that the game of "rock-paper-scissors" has biases that might determine the winner of the game. A generalization of the game is "rock-paper-scissors-lizard-Spock", with rules that follow next. In the following, let x 1 denote the density of rock, x 2 the density of paper, x 3 the density of scissors, x 4 the density of lizard, and x 5 the density of Spock. Also, let P i denote the toxin (or inhibitor) by organism xi. Refer to Figure 1 in the explanations that follow.
The rules of rock-paper-scissors-lizard-Spock are listed as follows. The corresponding toxin production effects are listed in the context of our notation as well (Table 2).
(1) Scissors (x 3 ) cut paper (x 2 ). (x 3 's toxin, P 3 , has a negative effect on the growth rate of x 2 .) (2) Paper (x 2 ) covers rock (x 1 ). (x 2 's toxin, P 2 , has a negative effect on the growth rate of   When you examine the rules of "rock-paper-scissors-lizard-Spock" carefully, some will say that it has some of the same flaws as "rockpaper-scissors" because the hand sign used in the game for "Spock" is difficult for many to implement quickly. The rules of rock-paperscissors-lizard-Spock are summarized by Sheldon Cooper (played by Jim Parsons) on the popular American television show The Big Bang Theory [26]. Refer to Figure 2 and direct your browser to "https://www. youtube.com/watch?v=_PUEoDYpUyQ" to see the complete YouTube video.
Using Figure 1, we adjust Table 1 to mirror the "rock-paper-scissorslizard-Spock" relationship. Observe that "rock-paper-scissors-lizard-Spock" has the same friend paradox as "rock-paper-scissors" in that a given organisms "friends" are those who it produces a toxin against. That is, if x j is organism x i 's friend, then x i is organism x j 's enemy.
For five species competing for five essential nutrients, system (4) is adjusted to x i Friends Enemies  4 , and x 5 in the context of a "rock-paperscissors-lizard-Spock" relationship. Observe that in a "rock-paper-scissors-lizard-Spock" relationship, if x i is x j 's "friend" then x j is x i 's "enemy". Friendship is not transitive in the "rock-paper scissors-lizard-Spock" relationship.

(10)
(0) 0, (0) 0, , 1, 2,3, 4,5 The constants have the same interpretation as in system (4); µ i (S 1 , In the absence of toxin production, Figure 3 show that Equation 10 can exhibit chaotic behaviour. In Figure 3, the values of the parameters used are D= .25, We incorporate a constant toxin production into Equation 10 in the same way as Hsu and Waltman in [13]. Assuming constant toxin production, Equation 10 is adjusted to where the k i are constants.
Our computations indicate that Equation 14 can exhibit a wide range of behavior even assuming constant toxin production. For example, using the same parameter values as used in Figure 3, in Figure  4 we use constant toxin production as given in Equation 15.
In Figure 4, observe that constant toxin production can eliminate the chaotic behavior observed in Figure 3.

Non-constant k
For non-constant k, in the same manner as before, we try to choose simple choices of k i =k i (x 1 , x 2 , x 3 , x 4 , x 5 ). Following the same approach as with Equation 8, we choose 13      For example, if the density of one of the organisms xi is large, a competitor with density x j may increase its toxin production because it is already losing the competition and facing extinction.

Conclusion
In this paper, we have numerically analyzed a fiffteen-dimensional nonlinear system of differential equations that models a "rock-paperscissors-lizard-Spock" relationship. In previous models with fewer competitors or different assumptions, a reduction process was able to be carried out to reduce the dimension of the system. In the cases studied here, reducing the dimension of the studied systems is not possible because of the interpretation of the toxin producing agent, k=k i , that is allowed to produce a toxin (inhibitor) against its competitors.
We have considered several extreme cases of the "cost" of the metabolic load of producing a toxin against a competitor. We examined the case when k=k i is constant and then two cases when k=k i was not constant. To see the coexistence that is biologically documented, we used a non-constant k, which indicates that when coexistence occurs in such situations, quorum sensing is involved in the biological relationship.
Our main results indicate that giving a competitor a defense might help stabilize an unstable system. It would be interesting if one could logically extend this result to other unstable competitive systems, such as social or societal ones. Of course, our assumptions are not exhaustive, but might indicate that in a real biological "rocks-papers-scissors" or "rocks-papers-scissors-lizard-Spock" situation the actual metabolic cost of producing a toxin is quite minimal. An interesting biological analysis might study the value of k=ki, which we theorize is quite small, and examine the impact of the k=ki value, if there is any tangible impact on such biological relationships.
Next interesting steps might be to investigate the results seen by Kerr et al. [1], where they observed multiple mutations after several generations. Although incorporating genetics and non-constant k into the equations developed here would probably only result in numerical results, such results could have meaningful biological significance. Another interesting study would be to adjust the May-Leonard competition equations as in Hsu and Roeger [9], repeat the analysis, and analyze any unexpected results. The authors of this paper think that studying the situation when there are multiple nutrients that the competitors are competing for would be particularly interesting, especially if the availability of the nutrients was dependent on seasons

Computational Notes
In our calculations, we used Mathematica Version 10 (www.wolfram. com) [27]. Primarily, we used Mathematica's graphing capabilities and its numerical methods for solving systems of ordinary differential equations. Mathematica's primary function for numerically solving (systems of) differential equations is NDSolve. Detailed information regarding NDSolve can be obtained from the Wolfram website. Alternatively, if you have access to Mathematica, enter the command ?NDSolve. In a broad sense, NDSolve uses a variety of numerical methods to attempt to solve (systems of) differential equations. Because our simulations are over "large" intervals, We increased the accuracy goal and adjusted other NDSolve options to help assure that the results we illustrate here are (numerically) meaningful.
Our approach is quite general so other choices of the coefficient functions could be investigated with relative small changes in the Mathematica notebooks used in the calculations and plots here. Finally, we note that the positive constant in the denominator keeps the function differentiable at the origin. A smaller constant mimics stricter ratio dependence. In our cases k ij (0, 0, 0)=0: We also chose α j so that 0<k ij <1.