When optimal foragers meet in a game theoretical conﬂict: A model of kleptoparasitism Journal of Theoretical Biology

in the case where foragers interact with each other? In our foraging game we consider predators who engage in contests (contestants) and those who never do (avoiders), and in general those who play a mixture of the two strategies. Here the classical zero-one rule does not hold. Firstly, the pure avoider phenotype is never an ESS. Secondly, the pure contestant can be a strict ESS, but we show this is not necessarily so. Thirdly, we give an example when there is mixed ESS. (cid:1) 2020 The Authors. Published by Elsevier Ltd. ThisisanopenaccessarticleundertheCCBYlicense(http:// creativecommons.org/licenses/by/4.0/).


Introduction
Kleptoparasitism is the stealing of already procured food by one individual from another (Brockmann and Barnard, 1979), and it is observed across several taxonomic groups, including spiders (Coyle et al., 1991), insects (Erlandsson, 1988), mammals (Janson, 1985;Carbone et al., 2005), and birds (Barnard, 1990). The advantage of kleptoparasitic behavior is that it allows individuals to avoid some of the costs of the foraging cycle (searching for, acquiring and handling food items) by exploiting food discovered by another individual's effort (Giraldeau and Caraco, 2000). Clearly, kleptoparasitism can be considered as a game theoretical problem and a foraging tactic at the same time. Starting from this point, the aim of this paper is to combine the basic ideas of two research lines.
The first research line is optimal foraging theory (Stephens and Krebs, 1986). The main assumptions of optimal foraging theory are the following: a) the focal forager has all necessarily information about its prey (cf. omniscient forager e.g. Schmidt andBrown, 1996, andGaray andMóri, 2010); b) the focal forager has absolute control of its own food preferences, i.e. the forager freely accepts or ignores any of its prey types (food items); c) energy collection by a forager does not depend on the food preferences of other foragers, and finally; d) an individual's fitness is its net energy intake rate, which is given by the functional response (Holling, 1959, Jeschke et al., 2002. The overwhelming majority of the derivation of functional responses (see e.g. Garay, 2019) are based on the assumption that either the prey density is renewed after each killing (Cressman et al., 2014, McNamara et al., 2006 or the predators have no (or only a negligible) effect on prey density during the duration of the foraging time (Garay andMóri, 2010, Holling, 1959), thus classical optimal foraging theory assumes that the prey density is fixed.
In the prey choice model (where each forager has different prey types providing different energy intakes and with different handling times), the basic result of optimal foraging theory is the zero-one rule, which claims that a predator accepts a given prey type if its energy/handling time ratio is bigger than the average intake rate on the whole foraging process (Charnov, 1976). In other words the predator either ignores or accepts a given prey type, so it never uses a mixed prey preference.
The second research line is evolutionary game theory (Maynard Smith and Price, 1973) focusing on the fitness consequences of interaction between conspecifics, when individuals' behavior have effects on the fitness of others, often through direct contests. In such contests, they assumed that when two individuals encounter each other then they always play a game. Observe that the latter assumption is not in harmony with the basic view of the optimal foraging, see assumption c) above, where the individual can ignore any interaction with its prey types. In this paper we concentrate on the case where each individual has freedom to interact or not to interact with others it encounters, and each activity needs a period of time.
There are three points, which offer us a way to make a connection between the above two research lines. Firstly, the functional response can take account of the interference between predators, which has an effect on the functional response, since this interference takes time. DeAngelis et al.'s (1975) and Beddington's (1975) functional response takes account of the time duration of the interactions between predators, but these interactions have no effect on the energy intake of predators. In this paper we consider the case when this interference has an effect on the net energy intake of predators as well, i.e. there are game theoretical conflicts between predators for prey. Secondly, in the classical matrix model of evolutionary game theory, Maynard Smith (1982) included a positive basic fitness, which is independent from the phenotypes (i.e. the strategy of players), in order to avoid a negative total fitness. But ''There is no such thing as a free lunch". In biology, the collection of basic fitness at least needs time, as in optimal foraging theory. Thus, the concept of time constraints gives us a way to introduce the ''time cost" of collecting the basic fitness of Maynard Smith (1982). Thirdly, the Nash principle can make a bridge between game theory and optimal foraging theory, namely the zero-one rule and the Nash-equilibrium condition are connected by the rule of time averages (Garay et al., 2015), claiming that ''the optimal predator behavior involves those activities that ensure larger time average intake than the time average of all activities".
Furthermore, there are game theoretical models, which are related to the present paper. Firstly, kleptoparasitism is modeled by ecological games with time constraints (e.g., Broom and Ruxton, 1998;Broom et al., 2004Broom et al., , 2008Broom et al., , 2009Broom et al., , 2010Broom and Rychtář, 2013;Sirot, 2000). The models of Broom and colleagues are compartmental, where individuals follow a Markov transition process between searching, handling and contesting states, with each behavior taking (an exponential amount of) time. Unlike in the present paper, strategic decisions are made at the transition stage, so a searching individual can decide whether to challenge a handler for a food item, after which the handler decides whether to defend it, the winner being decided at random, with no further decisions. The game is thus a type of sequential game. The model of Sirot (2000) had a similar basis, but here individuals made simul-taneous decisions when contesting a food item. Secondly, the present paper builds on a general game-theoretical modeling methodology, namely a matrix game under time constraints Cressman, 2017, Garay et al., 2018a), when each interaction between players has a time duration. Matrix games under time constraint are then characterized by two matrices, the intake matrix A ¼ ða i;j Þ nÂn and the time constraint matrix T ¼ ðt i;j Þ nÂn , i.e. when the focal individual uses the i-th pure strategy and its opponent the j-th one, the focal individual's payoff is a i;j , and the focal individual cannot play the next game during an average time duration t i;j ! 0. If this time duration depends on the strategies that the players use in the interaction, then the matrix game's evolutionary outcome is no longer given solely through its payoff matrix. Instead, an individual's payoff is given at the stationary distribution of a Markov chain that depends on the time constraint matrix. A similar process is followed for the more complex kleptoparasitism model developed that follows.
The aim of this paper is to combine the basic ideas of optimal foraging theory and evolutionary game theory with time constraints. A good combination of two theories should get back these theories as special cases. Clearly, for this aim, we have to keep as many basic assumptions of these theories as possible. From optimal foraging theory we keep the following three assumptions: 1. The predators have no (or a negligible) effect on prey density during the foraging time duration, so the prey density is fixed.
In other words, we use one of the basic assumptions of optimal foraging theory: prey renewal, see assumption d) above. 2. The predator is searching for food, and there are two types of food: (i) free food means that there is no other predator nearby; (ii) not free food means that the predator finds the food of a conspecific, but the acquired food has still not been consumed by the killer. Here we assume the interaction is symmetric, i.e. there is no ownership. In other words, when a predator kills a prey, then the ''ownership" has no effect on the behavior of the killer. The difference between a symmetric game, e.g. hawk-dove game, and an asymmetric version of this game, the hawk-dove-bourgeois game (Maynard Smith, 1982), is well known. 3. As in optimal foraging theory, each forager can neglect all types of food. In other words, when two predators have only one food item, the interaction between them is not a must, as in the basic evolutionary matrix game model. If an individual can evade the interactions, then this kind of individual has two extreme behaviors: either it evades the interactions (thus collects ''basic fitness" alone, i.e., only looking for free food), or it interacts with others, i.e. plays a game.
Thus we will introduce a situation dependent sequential game with time constraints. The first level gives the ratio of the materialization of the interaction. When two foragers encounter each other and one of them has killed but not eaten a prey individual, then they either interact for this killed prey (we call an individual playing this strategy a contestant) or they do not interact (we call an individual playing this strategy an avoider). The avoider (noncontesting) strategy means that before any interaction the avoider predator leaves the place, thus it has neither payoff nor extra time cost. The second level of our sequential game describes the situation when both foragers use the contestant strategy, and we consider the hawk-dove game as a mathematical description of the interaction between predators, when they find the same food item. So the hawk-dove game is a subgame in the sequential game introduced here. Now let us make clear the difference between a noncontest and a non-fight. The contest but not-fight behavior is the dove strategy, needing some extra time when interacting with a hawk and it has extra time and some payoff when interacting with another dove strategy user. We will assume a symmetric situation where all individuals can evade the interactions, so interaction takes place if and only if both individuals want to play the game. We emphasize that this situation is a combination of the basic problem of the optimal foraging theory (where the forager has a free decision on the acceptance of any type of prey) and the matrix game under time constraints, since both methods take account of the time constraints of different activities. In the present paper we will investigate this combined model. The main question of the present paper is whether the zero-one rule remains valid when the foragers interact with each other and assumption c) of optimal foraging theory does not hold.
2. Optimal foragers face game theoretical conflicts with others: a general monomorphic model

Model description
We start from an optimal foraging model (e.g. Stephens andKrebs, 1986, Garay andMóri, 2010), but now we consider two types of food: free food that has not been found by a forager and food in the possession of another forager (called discovered food). A focal individual forager begins in the searching stage, the average time duration of which will be denoted by s s . During this time, the focal forager can either find free food or discovered food. The model described here concentrates on the following question: which foraging behavior is optimal, engaging in a contest with the other forager over discovered food or avoiding contests by focusing only on free food.
Firstly, consider the case where a searching focal forager has found free food without another forager. Then it starts to handle the food item without consuming it (e.g. killing, transporting the food, etc). We call this period the vulnerable stage, the average time duration of which will be denoted by s v . This is the only stage where there is the possibility for the interaction with another forager, one result of which may be the theft of the food item. During the vulnerable stage, either the focal individual does not encounter a searching forager, or such an intruder arrives from the whole population and these two individuals will or will not interact. If there is no encounter, the focal forager passes to the digestive stage, the average time duration of which will be denoted by s d . If there is an encounter, there are the following four conditional events. (i) The focal individual does not retire and the intruder leaves. (ii) The focal individual retires and the intruder does not. In both these cases, there is no interaction between them and the forager who does not retire starts to digest the food in the digestive stage and the other returns to the searching stage. (iii) Both the focal individual and the intruder retire, in which case there is no interaction and each gets the food item with probability ½. Finally, when (iv) neither the focal nor the intruder retire, they interact in a contest, called the subgame, which is modelled as a symmetric matrix game with time constraints. In the interaction in this subgame between the two foragers, one of them possesses the food item and digests it before returning to the searching stage while the other returns to the searching stage. Note that we split the standard notion of ''handling time" into two stages, the vulnerable stage and the digestive stage (cf. Jeschke 2002). Moreover, all time durations are assumed to be independent and exponentially distributed.
Here we assume that the subgame is based on the classical hawk-dove game where pairs of foragers are engaged in a contest over the food item (i.e. the resource) of value B. Prior to the contest, neither forager has any information concerning the behavior (i.e. strategy) of the other forager. Moreover, we assume that this con-test is symmetric (i.e., there is no ownership, so the winning probabilities of the contestants can only depend on the strategies they use, and not on which one discovered the food item and which is the intruder). The subgame is then specified as a matrix game under time constraints characterized by the following intake and time constraint matrices: where the entries of A (respectively, T) are the intake (respectively, time duration) of the row player when interacting with the column player. When two hawks interact, they engage in an escalated fight with one of them winning without getting injured and the other losing with injuries. This is reflected in matrix A where C HW is the winner's cost and C HL is the losing hawk's cost (including the cost of fighting and the cost of recovery). Moreover, s XW (respectively,s XL , X ¼ D; H) is the time duration for the winner (respectively, loser) that is associated with this interaction, including fighting and recovery time. When a hawk and dove interact, the hawk gets the food item immediately (i.e. the time duration is 0), which accounts for the off diagonal terms in matrices A and T. Finally, when two doves interact, there is no fight (one wins the food item and the other loses) and the time duration is s DW for the winner and s DL for the loser (they can differ, e.g., in the time of digestion). We emphasize that, from the game theoretical perspective, the subgame is symmetric. Indeed, in hawk-hawk and dove-dove interactions, both contestants win with the same probability (i.e. who wins the contest does not depend on who discovered the food). Since all time durations are exponentially distributed, the matrix T contains the means of these independent exponential random variables. We note that here we follow the basic modelling methodology of our earlier paper (Garay et al., 2017) on matrix games with time constraints. Namely, the intake matrix A and the time constraint matrix T are independent parameters and the time constraints decrease the number of interactions between individuals. In essence, we build our model in two distinct steps. After setting up a continuous time Markov chain, first we look for the stationary distribution of the chain. This depends on the time constraint matrix. Then we calculate the average payoff determined by the intake matrix at this equilibrium. Thus our model is a static one, similar to the basic model of Maynard Smith and Price (1973), since we are interested in the set of conditions under which a sufficiently rare mutant cannot invade the resident population, but we are not interested in the dynamical frequency change of different phenotypes. In particular, we do not use replicator dynamics (cf. Garay et al., 2018b, Varga et al., 2019. Secondly, consider the case where the focal forager finds discovered food (i.e. food with another forager who is in the vulnerable stage). In this case, the focal forager is the intruder, and these two individuals will or will not interact, leading to a similar ''story" to the one above. If the focal individual leaves, then it starts a new search. If the focal individual does not leave and the other forager retires, then the focal individual gets the food and enters the digestive stage. (For the sake of simplicity, we assume throughout that at most one intruder can find a given food item that is with a forager in the vulnerable stage; i.e., no sequence of encounters can occur among foragers over the same food item.) When the focal individual does not leave and the other forager does not retire, then the above subgame (a matrix game under time constraints) takes place.
In this model, each forager has two types of decision. When a forager in the vulnerable stage and an intruder encounter each other, they can choose to interact or not to interact. Their strategies can be characterized by a real numberr 2 0; 1 ½ ; namely, a rstrategist is willing to interact with probability r. Observe that the subgame will be realized if and only if both foragers are willing to interact. Furthermore, in the subgame under time constraints, an individual forager can use a mixed strategy that can be described by a discrete probability distribution p ¼ ðp 1 ; p 2 Þ where p 1 (respectively p 2 Þ is the probability that the forager plays hawk (respectively, dove) in the subgame. A forager's phenotype is then characterized by its choice of r and p.

Mathematical model
Suppose there are m phenotypes in the forager population with y i the number of foragers with phenotype i (i ¼ 1; Á Á Á ; m). Then y ¼ y 1 þ y 2 þ Á Á Á þ y m is the total number of foragers. An individual forager, labelled as the ordered pair ði; jÞ, corresponds to the j-th forager (where 1 j y i Þof the phenotype i. An individual can be in one of the following stages at any moment: searching stage, denoted by s, vulnerable stage, denoted by v, subgame stage, denoted by g u; w ð Þ or simply by g. This means that the forager, using pure strategy u in the subgame, is interacting with another forager who is using pure strategy w. The duration of this stage depends on the strategies used and may differ for the two contestants. digestive stage, denoted by d.
We emphasize that in our model the subgame stage includes digestion and hence it is not followed by staying in the digesting stage. This is because the duration of digestion may depend on the amount of food, and in a subgame we allow the contestants to share the food in an undetermined proportion. Therefore, separating digestion from the subgame would make the mathematical model significantly more complicated. In all other cases, i.e. when food is acquired outside of a subgame, digestion always presumes a digestive stage. Note that the subgame stage may include recovery from injuries, which can also be different for the contestants.
Here we assume that there are n possible pure strategies a forager can use in the subgame (in the model description of Section 2.1, n ¼ 2). If a forager uses the pure strategy u against an opponent using pure strategy w, its intake is a u;w , and the average time it spends in the subgame stage is t u;w . Thus, following Garay et al. (2017), the subgame is characterized by the intake matrix A ¼ ða i;j Þ nÂn and the time constraint matrix T ¼ ðt i;j Þ nÂn . Phenotype i is then determined by the probability r i that such a forager is willing to interact in the subgame combined with the strategy distribution vector p i ¼ ðp i1 ; Á Á Á ; p in Þ, where p iu is the probability that this phenotype uses the pure strategy u in the subgame; thus P n u¼1 p iu ¼ 1.
Further notations: Let x denote the number of food items in the habitat. Food is assumed to regenerate at the same rate as it is consumed, thus x is assumed constant in time, in other words, we assume food renewal. We introduce here h i is the number of foragers of phenotype i per one food item, and h is the same quantity with respect to all foragers, regardless of the phenotype. Let q s;i ; q v;i ; q g;i ; q d;i denote the proportions of phenotype i in the searching, vulnerable, subgame, and digestive stages, respectively. Moreover, let q s ; q v ; q g ; q d be the equivalent proportions for the whole population. Clearly, q s ¼ -the first term corresponds to the case where individual ði; jÞ finds food discovered by forager ðk; lÞ, the former is willing to interact but the latter is not. If both retire, then each has probability 1 2 to win the food, thus the second term in the rate stands for the case where chance favors individual ði; jÞ. Only phenotype i receives an intake (which we will denote by G i Þ: -a game played with strategies u and w is over. Intake: It is easy to see that this Markov chain is irreducible as every state communicates with the state ðs; Á Á Á ; sÞ, hence it has a unique stationary distribution. Similarly to as in Garay et al. (2017), one can show that the random proportions q s;i , q v;i , q g;i , q d;i converge to constants as the size y of the population and the amount x of food increase to infinity in such a way that the ratios h i converge.
This result allows us to define a game among the m phenotypes where the payoff is taken as the intake rate at the stationary equilibrium and to do this we need to consider cycles.
In what follows, we will focus on a sufficiently large equilibrium population with a single (resident) phenotype, where a mutant phenotype appears. In this general monomorphic model, we then take m ¼ 2 in the above mathematical model. Consider an arbitrary focal forager, resident or mutant. We will distinguish its strategy parameters, r Ã and p Ã , by asterisks. The population parameters then have no asterisks. Let us call a sequence of consecutive stages a cycle if it lasts from the beginning of a searching stage to the next searching. What's going on during a cycle?
1) The focal individual is searching until it finds food, and the average searching time is s s . At the end of searching 2a) The focal individual finds free food with probability 1 À q v h.
Then it moves to the vulnerable stage. Its average time length is because the length of the vulnerable stage is the minimum of two independent exponential time spans, one of them is the length of the uninterrupted vulnerable period, and the other one is the time needed by the fastest searcher to find the focal individual. As is well-known, the minimum of two independent, exponentially distributed random variables is also exponential, with expectation being half of the harmonic mean of the two expectations (equivalently, with hazard rate being the sum of the two hazard rates). According to this, at the end of vulnerable stage there are two possibilities. o Either the focal individual starts digesting, with probability Note that the occurrence of these possibilities is (stochastically) independent of the length of the vulnerable period, i.e. knowing the length of any occurrence of the period (as opposed to its expectation s v ) provides no information on which event will occur.
Then, from the point of view of the focal individual, the following outcomes are possible.

& The focal individual is not willing to interact but the intruder is.
This has probability r 1 À r Ã ð Þand leads to no additional time, and zero intake. & The focal individual is willing to interact but the intruder is not, which happens with probability r Ã ð1 À rÞ. Then the focal individual receives intake B and moves to the digestive stage with average time s d . & Neither the focal nor the intruder are willing to interact. Such a case occurs with probability ð1 À r Ã Þð1 À rÞ. Here the whole food item is taken by one of them, with equal probability for each. The luckier one moves to the digestive stage, the other to the searching stage. Thus the average time left for the focal individual in the cycle is s d =2, and its average intake is B=2.
& Both the focal individual and the intruder are willing to interact, occurring with probability r Ã r. The average time for the game is p Ã Tp, and the average intake is p Ã Ap. 2b) Alternatively, the focal individual finds previously discovered food with probability q v h. Then the following scenarios are possible. o The focal individual is not willing to interact but the intruder is. This has probability ð1 À r Ã Þr. There is no additional time and zero intake. o The focal individual is willing to interact but the intruder is not, with probability r Ã ð1 À rÞ. The focal individual starts digesting with average time s d and intake B.
o Neither the focal individual nor the intruder are willing to interact, with probability ð1 À r Ã Þð1 À rÞ. The focal individual spends average time s d =2 digesting, with average intake B=2. o Both the focal individual and the intruder are willing to interact. The probability of this possibility is r Ã r, and the average time and average intake are p Ã Tp, and p Ã Ap, respectively.
After all of the above, the cycle starts over again. Let s Ã denote the average time of the focal individual's cycle. It has the following components.
searching stage with average length s s , vulnerable stage with average length s v p d , where p d is the probability that free food is found and no intruders arrive, namely, can be interpreted as the probability of getting into a contest situation (i.e., where two individuals, one with food and another without it, meet): the first term stands for the case where an intruder appears, and the second one for the case where the searching focal individual finds previously discovered food. A contest situation leads to a subgame if and only if rr Ã -0. digesting stage of average length s d p d þ p c ð1ÀrÞð1þr Ã Þ 2 . The first term corresponds to the case where free food is found and no intruders come, and the second term stands for the case where in an encounter food is taken without a contest. The multiplier of p c in the above is ð1ÀrÞð1þr Ã Þ , where the first term comes from the case where the focal would fight but the intruder would not, and the second term comes from the case where both retire and the food is awarded randomly. Neither of these cases correspond to a subgame.
The average amount of food taken by the focal individual during one cycle is In order to characterize the equilibrium, let the focal individual belong to the resident population, i.e., there is no need for asterisks, as all quantities tagged with asterisks are equal to their unmarked counterparts. Then the proportions of individuals in searching, vulnerable, subgame, or digestive stages, respectively, are equal to the proportions of time spent in those stages during one cycle. Thus, in equilibrium we have The fourth equation is omitted, because it follows from the preceding three. In detail, we obtain the following system of quadratic equations in the two variables q s and q v ; After this system is solved, the third equation provides us an explicit formula for q g in terms of q s and q v . Since an irreducible, continuous time, finite state space Markov chain always has a stationary distribution, this system does have a feasible solution.
Though the stationary distribution is unique, it does not necessarily imply the uniqueness of the solution of our system of equations. To illustrate our model, we use the game tree method introduced earlier by Cressman et al. (2014). Game trees provide a way to describe the forager's behavior in detail, based on the sequence of its choices at different decision points. The game tree describes all possible foraging situations, which start from the beginning of the search for food (the root of the tree), and end at different random events (the leaves of the tree). In the illustration, we will consider two types of focal foragers, i.e. we consider polymorph model for visualization. 1 The first type, called an avoider (see Fig. 1), is a forager who is never willing to interact (i.e. r ¼ 0). The second type, called a contestant (see Fig. 2), is a forager who is always willing to interact (i.e. r ¼ 1). Accordingly, we will use the notations q s;a , q v;a , q g;a , q d;a , h a (respectively q s;b , q v;b ,q g;b , q d;b , h b ) instead of q s;1 , q s;2 , q v;1 , q v;2 , h 1 , h 2 etc. for the avoider (respectively, contestant). We call the reader's attention to the fact that it is not assumed here that at least one of these phenotypes is arbitrary rare.
In Fig. 1 we consider a focal individual that adopts the avoider strategy in all foraging turns. This individual encounters a food item discovered by another avoider with rate q v;a h a , see (1). Similarly, the focal avoider encounters a food item discovered by a contestant and free food with rates q v;b h b and 1 À q v;a h a À q v;b h b , respectively. The time spent in the vulnerable stage is the minimum of two independent, exponentially distributed random variables, as in (2), so it has mean There is no interaction when no intruder arrives during the vulnerable stage of the focal avoider, that is, when the focal individual can pass to the digestive stage before meeting a searcher. This happens with probability s m =s v . In this case the focal avoider starts digesting its free prey, so in this particular foraging turn, the focal avoider spends time s s þ s m þ s d and gets benefit B.
Next, consider the possibilities of interactions. Firstly, let us start with the case where the focal avoider is in the vulnerable stage and another individual arrives in the meantime. This happens with probability 1 À sm sv . The intruders must be in searching stage.
The probabilities that the intruder plays the subgame or uses the avoider strategy are proportional to the frequencies of the corresponding phenotypes, that is, a contestant individual arrives with probability q s;b h b q s;b h b þq s;a ha . If a contestant arrives, it takes the focal avoider's prey, thus the focal individual is left without prey and in this particular foraging turn the focal individual spends s s þ s m time on average. On the other hand, an avoider individual arrives with probability q s;a ha q s;b h b þq s;a ha , and after the encounter, without a subgame occurring, one of them gets the prey and starts digestion, each with probability 1=2, so the average time duration and benefit are s s þ s m þ s d 2 and B=2. When a focal avoider finds prey with a contestant, the focal individual retires and immediately starts a new search, thus the time duration of this kind of foraging turn is just s s . Finally, when a focal avoider finds prey with another avoider, no contest follows, and both parties have the same chance to take the whole prey. Thus the focal individual spends time s s þ s d 2 and gets B=2 on average.
In Fig. 2 we consider a focal individual that follows the contestant strategy (r ¼ 1) in all foraging turns. Differences only appear on the leaves of the tree. The leftmost leaf (no intruder arrives) is the same as in the case of a focal avoider. When the focal contestant is in the vulnerable stage and another contestant arrives, they start to play the matrix game with time constraints, so in this particular foraging turn the focal contestant spends time s s þ s m þ pTp on average and its average intake is pAp. If the intruder is an avoider, then no contest (subgame) begins: the focal contestant gets the prey and starts digesting, so the average time duration and benefit are s s þ s m þ s d and B, respectively. Similarly, when a focal contestant finds a prey with another contestant, they start to play the game immediately, so this particular foraging turn takes an average s s þ pTp of the focal contestant's time, and the focal individual gets pAp. Finally, when the discovered prey is with an avoider, the focal contestant takes the prey and starts to digest it, so it only spends time s s þ s d and gets benefit B.

Strict ESS
We say that the resident phenotype is strictly evolutionarily stable if for an arbitrary focal different from the resident we have that is, the resident phenotype maximizes the average intake per time unit among all possible phenotypes, and this maximum is unique. This is equivalent to maximizing the long-term payoff of the individual, the standard measure of evolutionary success. We note that an alternative way of approaching this problem was developed in Křivan and Cressman (2017). The fact that these two methods are actually equivalent was shown in Broom et al. (2019).
Claim. If the resident phenotype is strictly evolutionarily stable, then r ¼ 1.
Proof. Let p Ã ¼ p. Then the focal individual's average intake per time unit can be written in the following form: where the coefficients are positive, namely This is a linear rational function of r Ã , hence monotone. Thus, if 0 < r < 1, there exists a mutant with r Ã 2 f0; 1g which is at least as good as the resident. This is excluded by supposition. If r ¼ 0, Þ> 0, so the function f r Ã ð Þ is strictly increasing, therefore the resident can be outperformed by choosing r Ã ¼ 1.j Next we show an example of a strictly evolutionarily stable phenotype.
If r Ã ¼ 1, then This is obviously positive if s v þ s d ð Þ pAp > BpTp, which can be achieved by suitably choosing a and t. Then the resident phenotype is evolutionarily stable.

Mixed ESS
Of course, a strictly evolutionarily stable phenotype does not necessarily exist. For example, if the matrices T and A are given in such a way that a ij ¼ a and t ij ¼ t for every i; j 2 1; 2; Á Á Á ; n f g , then game strategy p is indifferent. Therefore, two phenotypes are equivalent if they have the same contesting probability r. Thus no phenotype can be strictly evolutionarily stable. Apart from this trivial case, if the duration of the game is very long, and the reward is small, it is not worth contesting. Our second example presents a case where a strictly evolutionarily stable phenotype cannot exist.
Example 2. Consider a model where h < 1, that is, there is more food than individuals. It is easy to see that Recalling the lower estimate for p d we can see that Suppose Þ , that is, (5) holds, therefore a mutant with p Ã ¼ p and r Ã ¼ 0 is strictly better than the resident.
Thus, in this model there does not exist a strictly evolutionarily stable phenotype.

Weak ESS
We can also define the weak evolutionary stability property of phenotype ðp; rÞ. It means that for an arbitrary focal with ðp Ã ; r Ã Þ we have G Ã s Ã G s : In Example 2, though there exist no strictly evolutionarily stable phenotypes, still there may be one in the weaker sense. Again, p Ã ¼ p ¼ 1; 0; Á Á Á 0 ð Þ can be assumed, thus p Ã Ap ¼ pAp ¼ a and p Ã Tp ¼ pTp ¼ t. For a weakly evolutionarily stable r one has to solve the equation Q 1 Q 4 À Q 2 Q 3 ¼ 0. It looks quadratic, but in fact it is not, because p d and p c also depend on r through q s and q v , which are only implicitly given. Fixing a; t; B; s s ; s v ; s d so that the conditions of Example 2 are satisfied one solves the equation numerically, by computing Q 1 Q 4 À Q 2 Q 3 for r running from 0 to 1. We shall see that for suitably choosing a; t; B; s s ; s v ; s d the existence of a weakly evolutionarily stable r can be realized.
Example 3. For the sake of simplicity, we suppose s s ¼ s v ¼ s d ¼ 1, that is, all time durations are identically distributed, namely, exponential with mean 1, and let a ¼ 1. Set the positive parameters t; B; h, such that they satisfy (6) (i.e. Bt > 3 and h < BtÀ3 BtÀ1 ). With C ¼ r 2 t þ 1Àr 2 2 , equations (3a) and (3b) take the form: where x and y stand for q s and q v respectively. For r fixed between 0 and 1, we find numerically the unique positive solution of this system of two quadratic equations in x and y that satisfy x þ y < 1. We then plot the function u r which is equal to a positive multiple ð1þxhÞ 2 hðxþyÞ of Q 1 Q 4 À Q 2 Q 3 . When t ¼ 1; B ¼ 4 and h ¼ 0:1; Fig. 3 shows that there is a mixed solution r ffi 0:81 that satisfies the weak evolutionary stability property.
By Example 3, the zero-one rule is not valid in general since the ESS phenotype is ready to contest with probability 0:81 (i.e. the expected outcome is a mixed ESS).

Conclusion
Through considering the functional response, we can see that kleptoparasitism is a special interference between foragers, which does not only take time but also has an effect on the net energy intake of both forager individuals. Thus, kleptoparasitism is an excellent example for a foraging game (e.g. Filippi and Nomakuchi, 2016, Sirot, 2000, Spencer and Broom, 2018. Furthermore, it is also a good example for the game with time constraints, for instance the victim not only lost its acquired food item but also the time it has spent to get this food item before it was stolen. That is, although we only formally introduce time constraints in the subgame, it is clear that time constraints also play an important part in other stages of the sequential game. The novelty of the present work is that we make a bridge between two theoretical research lines: optimal foraging theory and a sequential evolutionary game theory with time constraints. One of the basic ideas of optimal foraging theory is that the densities of different prey types determine the optimal foraging tactics. In our game theoretical model a similar effect takes place. For instance, in Example 2 we found that if there is more food than individuals (i.e. the free food is abundant enough) then there is no strict ESS. Furthermore, in our model, although there is only one prey species, there are different prey types (as a free food item, but also as a food item at the vulnerable stage of foragers, moreover according to which phenotype acquired the prey). During our investigation the relative frequency of these different food items implicitly determines the evolutionary stability (see the role of h i in the main text and Examples). In this sense, the above basic ideas of optimal foraging theory are transferred to the game theory. Moreover, the Nash solution concept in our game (where the payoff is the ratio of average intake to the average time duration of one of the foraging cycle) is equivalent with the rule of time averages (Garay et al., 2015), claiming that ''the optimal predator behavior involves those activities that ensure larger time average intake than the time average of all activities". Thus the time durations of different activities also have effect on the optimal behavior in the game with time constraints.
In our game, where the interactions between predators have an effect on their net energy intake and need extra time, we found that the classical zero-one rule is not valid. Firstly, the avoider phenotype (r ¼ 0) is never an ESS, since if there are only avoiders in the resident population (that is, e.g. the resident never contests), the mutant always get an advantage by stealing the resident's food. Secondly, we point out that the contestant (r ¼ 1) can be a strict ESS, but is not necessarily one. Thirdly, the contestant (r ¼ 1) is sometimes not an ESS, since if the average time duration of the game is very long, and the reward is small, it is not worth contesting. Fourthly, we give an example where a mixed ESS does exist. The kleptoparasitism models of Broom and colleagues did not generally produce mixed solutions, as have been produced here. A key reason for this was the sequential nature of decisions in that model. The challenger decides their choice first and then the defender responds, and if the challenger decided not to challenge then the defender automatically keeps their food. We note that simple sequential games generally have only pure solutions (see Broom and Rychtář, 2013). These food stealing games (see e.g., Broom and Ruxton, 1998;Broom et al., 2004) are not simple but affected by population density. However, the effect of density is destabilizing for mixtures. If all individuals fight then the effective foraging rate is low, meaning the value of any given food item is effectively higher, making it more attractive to fight for. Thus more than one ESS was common. The exceptions that produced mixed strategies were Broom et al. (2008Broom et al. ( , 2009, where individuals which did not attempt to search for conspecifics had a higher rate of finding free food than others, whereas in the other models the efficiency of food finding was assumed the same for all individuals. In the present paper individuals make simultaneous decisions, and they do it without making a distinction in whether they are the challenger or the defender (since our model is symmetric without ownership), in a similar way to Sirot (2000), and so can similarly obtain a mixed solution. We note that there are a number of differences in the current model and the Sirot (2000) model. In the latter Dove versus Dove contests took no time (as in Broom et al., 2004, although there it arose naturally as there was no contest), whereas in the current paper it does, in the spirit of the ''war of attrition" game (see Maynard Smith, 1982). Sirot (2000) also effectively had a simplifying assumption for the payoffs, where the value of a reward compared to the cost of a fight was independent of the population strategy, which is not made here (or in the Broom et al. (2004) models).
Although we concentrate on a theoretical symmetrical selection situation, we think the game-tree method (Cressman et al., 2014) can handle other biological situations, as well. For instance, two different types of asymmetry occur in kleptoparasitism. The first one takes place within the same species, namely ownership, which may have effect on the behavior of owner, like the bourgeois strategy (Maynard Smith, 1982). The effects of ownership on the evolutionary outcome when, unlike kleptoparasitism, it is only interaction times that are strategy dependent, were investigated by Cressman and Křivan (2019). The second one is when kleptoparasitism occurs between different species (e.g. Balme et al., 2017, Garthe andHüppop, 1998). These types of asymmetry (ownership and/or multispecies interactions) can be modelled by the gametree method, but the analysis of these asymmetric games will need more investigation and is left to future research.