Diverse Stochasticity Leads a Colony of Ants to Optimal Foraging

A mathematical model of garden ants (Laius japonicus) is introduced herein to investigate the relationship between the distribution of the degree of stochasticity in following pheromone trails and the group foraging efficiency. Numerical simulations of the model indicate that depending on the systematic change of the feeding environment, the optimal distribution of stochasticity shifts from a mixture of almost deterministic and mildly stochastic ants to a contrasted mixture of almost deterministic ants and highly stochastic ants. In addition, the interaction between the stochasticity and the pheromone path regulates the dynamics of the foraging efficiency optimization. Stochasticity could strengthen the collective efficiency when stochasticity to the sensitivity of pheromone for ants is introduced in the model.


I. INTRODUCTION
Ants efficiently shuttle between their nest and food sources over long distances by depositing and following pheromone trails [1]. As a result of the collective usage of pheromones, ants are able to construct a trail that might be accomplished by accurate pheromone usage. Following the trail ensures that ants can accurately reach food sources. However, a certain fraction of ants in each colony does not accurately follow pheromone trails [2,3]. The concept of stochastic foraging or strategy of error is introduced and validated through experiments and mathematical models to interpret the ecological significance of this stochasticity [2,4,6,[9][10][11][12][13][14]16]. Deneubourg et al. performed numerical experiments to understand the meaning of the very low percentage of Tetramorium impurum reaching a food source following a pheromone trail [2]. Using a simple phenomenological model for the recruiting dynamics of ants, they estimated the optimized degree of stochasticity for each ant to be recruited to an already-found food source in a multi-food-source environment. They found that a less accurate recruitment led the colony to a more efficient foraging as the number of location of food sources increased. These studies recognized the stochasticity of ants in following pheromone trails as a factor that increased their chance to find new food sources and avoid excessive persistence to previously established trails.
However, many basic problems on stochasticityinduced efficient foraging remain unsolved. One among them is the main issue being focused on herein: what * nishimor@hiroshima-u.ac.jp type of population distribution of stochasticity enables optimized foraging in a colony of ants? Previous models of stochastic foraging have implicitly or explicitly introduced the Gaussian or uniform distribution of stochasticity because of their simple forms. However, experiments with existing ants have shown widely dispersed population distributions of stochasticity. As an extreme example, only 18% of Tetramortium inpurum in a colony can follow pheromone trails, and the same quantity decreases down to 9% in the case without a leader [2]. As another example, typically more than 40% individuals in a colony of garden ants species Lasius niger, which usually make use of both visual and chemical cues in their foraging trip, fail to follow pheromone trails if visual cues are made imperfectly available [3]. We believe that the population distribution of stochasticity in following pheromone trails is not always restricted to a Gaussian-like unimodal form. We evaluate this presumption by performing a modelbased study on what type of distribution of stochasticity leads a colony of ants to efficient foraging. We focus on the effects of the intensity of stochasticity of the stochastic ants and their population rate in the colony under different environment conditions. Our study aims to mathematically understand how the stochasticity in a complex collective system based on real biological systems works to optimize the collective behaviors and analyze how the specific stochasticity in the ant colony model works compared to the stochasticity in previous studies [2,4,6,[9][10][11][12][13][14]16]. This paper is organized as follows: Section 2 introduces our agent-based model; Section 3 explains the evaluation procedure of the foraging behaviors with the model; Section 4 shows the numerical results and discusses some combinations of stochasticity intensity and their population rate in the colony; and Section 5 concludes and summarizes the results.

II. MODEL
We employ herein a computational model [15] categorized in multi-agent models [14]. The unique task for each agent (i.e., ant) in the present model is to find food sources and bring food to the nest. The peculiar feature of this model is that in addition to using trail pheromone, ants make use of visual cues to roughly grasp the homing direction, where this non-local walking rule of ants is introduced to make a qualitative correspondence to the behavior of the existing species of ants, namely the Lasius japonicus. The foraging field is a hexagonal twodimensional lattice measuring 150 × 150 with a periodic boundary condition, on which one nest site and two sites of food sources are designated. The dynamical states for each ant, which are labeled as k ∈ {1, 2, . . . N total }, are represented by a positional vector, x k , consisting of a lattice index for the hexagonal two-dimensional lattice and a facing direction vector, η k , consisting of one of the unit vectors of the hexagonal two-dimensional lattice. Each ant at each time step is located at one site, but does not exclusively occupy it, and faces toward one of its nearest six sites. A randomly selected single ant among a total of N total ants moves to the facing site, its right site, or its left site (the detailed process to select a site is explained below). The same processes are repeated for N total times and constituted a unit Monte-Carlo(MC) step. This unit MC step is accompanied by a single step of Eq. (3) used to update the amount of food and Eq. (2a) used for the evolution of the pheromone field. The distance R between the nest and the food source sites and the relative angle θ between two food source sites as seen from the nest are given as the feeding environmental parameters ( Fig. 1). Increasing the food supplies per unit time is an important feeding environment parameter as well, which characterizes the dynamical aspects of this model. The dynamics of food supply and the evaluation methods of the system are explained in the sub-sections that follow.

A. Chemotactic Dynamics
In this model, each ant moves to the facing site or its adjacent sites, which are stochastically decided upon depending on the trail pheromone on the lattice field. The position of the kth ant is given as x k , and its facing direction is η k ; therefore, the site to which the ant faces is represented by x f k = x k + η k , and x k and x r k are the left and the right sites of the facing site, respectively. The probability, p j , denoting the move to one of the three The distance R between the nest and the food source points and the relative angle θ between two food sources as seen from the nest are considered as feeding environmental parameters. Food resource sites are the closest hexagonal sites (blue meshed hexagonal sites) to the food source points (blue dots).
, is presented as: where φ x j k is the trail pheromone density at x j k ; a parameter h = 10 corresponds to the Hill coefficient, z = 0.05; and g(y) is a type of response function defining the individual's susceptibility to the pheromone. Unfortunately, this response function is not directly determined experimentally; thus, we adopted a function with a non-linear effect, like a sigmoid function. Note that α k is the key parameter controlling the stochasticity of the kth ant; hence, the sensitivity of the kth ant to the trail pheromone increases as α k increases. In this sense, we call α k as the "sensitivity parameter." Aside from introducing the sensitivity parameter in this manner, the forms of Eq. (1a) and Eq. (1b) follow those of the previous studies [4,5]. The facing direction of the ant is reset to the moving direction once the moving direction is selected. It then moves to the facing cell. After this movement, the facing directions are kept unchanged until the next move.

B. Three Behavioral Modes
We introduce the three following behavioral modes to incorporate the situation-dependent behavior of ants: exploring mode (EM), homing mode (HM), and emergency homing mode (EHM). The details of each mode are explained below.
(i) Exploring mode: Every ant initially leaves the nest in this mode and explores for food by tracing the trail pheromone deposited by its nest-mates (or the ant itself) in HM. In other words, the ant's movement is based on simply stochastic dynamics. This mode lasts until the ant finds food or becomes exhausted without finding food after a finite time, t th = 150, has passed since it left the nest. A randomly selected ant in each walking step moves into one of its facing sites or its neighboring two sites based on the stochastic dynamics explained in the previous section. Upon reaching one of the six neighboring sites of food sources, if the temporal amount of food exceeds the threshold value f th = 0.1, the ant takes a certain amount of food f e (≤ f th ) from the site, then its mode shifts from EM to HM.
(ii) Homing mode: In HM, the ant tries to return to the nest with the deposited trail pheromone to recruit nest-mates. As a peculiar feature of the walking manner in HM, a combined use of visual and chemical cues is considered assuming the homing manner of Lasius japonicus [17]. Accordingly, visual cues are preferentially relied on. We assume that homing ants roughly "recognize the nest direction by visual cue, for which they make use of landmarks. More specifically, viewing from a homing ant, if the relative angle between the nest direction and its facing direction exceeds 60 • irrespective of the local pheromone field, the ant turns to the right-or left-facing direction to decrease the relative angle to the nest, then walks to one of the newly faced sites, in which it will keep its facing direction until the next move. Alternatively, when the relative direction is less than or equal to 60 • , the ant determines its walking direction depending only on the pheromone densities among its three facing sites according to Eqs. (1a) and (1b). In each walking step in HM, the ant deposits a fixed amount of trail pheromone, φ u = 0.08, at site x i , from which it leaves.
(iii) Emergency homing mode: If an ant in EM cannot find food within a certain time interval t th after leaving the nest or in HM and fails to come back to the nest within the same interval t th after getting food, its mode changes to the EHM. Aside from not depositing the pheromone, an ant in this mode moves in the same way as in the usual HM. Note that the sensitivity parameter is set high as α k = α n = 1 for every ant in this mode to enable the ant a prompt return to the nest after losing its way. Upon arriving at one of the neighboring sites of the nest, the ant returns to EM.

C. Evolution of the Pheromone Field
The pheromone density φ(x, t) in the hexagonal sites x is updated using the following equation: where τ is the unit time of the present model; the second term in the r.h.s. represents discretized form diffusion, in which δφ t ( x) is the difference from average around the nearest sites of site x as shown in the second equation, and D p is a diffusion constant. The third term and the last term represent evaporation and the total amount of deposited pheromone at x during one MC step, respectively, where n h ( x) is the number of times the homing ants leave the site x depositing pheromone during one MC step, and constants e p and φ u are the evaporation rate of the pheromone and the amount of pheromone secreted by an ant, respectively.

D. System Parameters
As mentioned earlier, the present model assumes Lasius japonicas as the corresponding species of ants; therefore, the parameter values are consistent with the assessable data of this ant species. Time and space units are also set accordingly. First, the unit length, the distance between neighboring cells, is set as approximately 1.5 cm, whereas the unit time, one MC step, τ is set as 3 s. Considering each ant in the unit MC simulation walks once in average, the above-mentioned estimations are consistent with the typical walking velocity, ∼ 0.5 cm/s, of Lasius japonicas. In addition, the evaporation rate of pheromone e p = 0.01 in this model is consistent with the experimentally obtained duration time (i.e., 5 min) of the pheromone. Furthermore, the width of foraging trail is within a few centimeters. Therefore, the characteristic width, D p /e p ∼ 1.2 assuming D p = 0.015 in our model, is consistent with the reality. We know that the preceding experiments [7] of Lasius niger estimated 47 min as the life time, and those of Monomorius pharaohs [8] estimated the decay time (to 50%) on the plastic and paper substrates to yield 25 min and 8 min, respectively. We would like to stress again that the evaporation rate (or life time) of the pheromone significantly depends on the species and the experimental situations. Meanwhile, the Hill coefficient (n=2 in ref [4,5]) is "manually" set as h = 10, such that the combination of the walking probability in the three facing directions sharply varies between (1/3, 1/3, 1/3) and one of {(0, 0, 1), (0, 1, 0), (1, 0, 0)} according to the change of α k in the present range of its variation, thereby accentuating the effect of the non-uniformity of the stochasticity in a colony.

E. Normal and Stochastic Ant Distributions
The basic motivation of this study is to investigate what type of population distribution of stochasticity will lead a colony of ants to optimal foraging. As the index controlling the distribution of stochastic ants in each colony, a set of sensitivity parameters {α k }, (k ∈ 1, . . . , N total ) introduced in Eq. (1b) is considered. We restrict the distributions to binary ones to keep the calculations simple while retaining the essential feature of the non-uniform distribution of sensitivity, that is, each colony is assumed to consist of two degrees of stochastic ants: (i) the less stochastic ants, which we call "normal ants," whose sensitivity parameter is fixed as α k = α n = 1 and is the value giving the highest (or close to the highest) foraging efficiency as a uniform colony in the reference foraging environment, R = 50 and θ = 90 • , and (ii) more or equivalently stochastic ants, which we simply call "stochastic ants," whose sensitivity parameter is α k = α s ∈ {1, 10 −1 , . . . 10 −6 }. Note that α s = 1 provides an exceptional case, where the stochastic ants are equivalent to the normal ants, and the distribution becomes uniform. Moreover, fixing the total number of ants in each colony as N total = N n + N s = 500, we varies N n , which is the number of normal ants in the same colony.

F. Feeding Environments
In nature, aside from ants, other animals also have to spend time to find food and avoid the risk of starvation. The longer animals explore the outside of their nests to find food, the higher their risks of being attacked by other animals or dying by accident. Animals manage the balance between the potential risks surrounding them. The difficulty to minimize the total risks comes from uncertain food sources, which are randomly distributed and supply a fluctuating amount of food. The efficiency of an ant colony under reasonable conditions, which represent many different natural environments, must be studied; however, performing a simulation for all the possible situations is difficult. We controll herein the spatial configurations of food sources and the dynamics of the amount of food of individual sources. As we explained in Section 2, the number of food sources is limited to two as the most simplified situation in multiple food sources. The dynamics of the amount of food is unpredictable and complex in nature; thus, its mathematical modeling requires some kind of simplification. Therefore, we set up typical situations with controlling the feeding amount of food to understand the effects of stochasticity under the steady state and the response to dynamical perturbation. In the simulation, the amount of food changes, such that the food at the two feeding sites is constantly supplied with a certain amount of δ and carried away by foragers reaching to the sites. We define this method as the "continuous feeding method." Let f i (t) be the amount of food in the i-th feeding site at time t. The time evolution of f i (t) is given by the following discrete equation: where n f i (t) is the number of ants that succeeded in carrying away a certain amount of food f e from the food source i in unit MC step; δ is the feeding increment of food; and f c is the upper threshold for the amount of food introduced to represent that the food sources are finite in nature. We controll the richness of the food environment by changing δ. Note that the time evolution of the amount of food at individual food sites in Eq. (3) independently proceeds. This feeding method corresponds to the insufficient feeding environment when δ is small and sufficient feeding environment when δ is large.
In addition to the above-mentioned feeding method, we testify on a specialized feeding situation to study how ants manage to find an alternative feeding site when the already discovered food source at one feeding site runs out. To do so, at the beginning, a fixed amount of food is set in a feeding site, and the other feeding site is set empty. The initial food amount, f 1 (0), was set enough to contrast the stable pheromone trail between the first food source site and the nest site. We set f 1 (0) = 20 herein. The time evolution of the amount of food obeys: when ants take all the food on the first feeding site to the same amount of food set on the other site. This feeding method is called the "specialized feeding method."

III. EVALUATION OF THE COLONY ABILITY
In this section, we propose three types of index used to evaluate the foraging-related abilities of individual colonies in our simulations: foraging efficiency F , transportation efficiency E T , and discovery efficiency E D .

A. Foraging Efficiency
First, we define the foraging efficiency F of a colony: the average number of ants in HM per unit MC step. The continuous feeding method introduced in the previous section is employed to calculate this quantity. We calculate the foraging efficiency F as a function of N n and α s defined as follows: where n h (t) is the number of ants reaching the nest carrying food from one of the food sources. We observed the "foraging efficiency landscape" F (N n , α s ; θ, δ) on the two parameters, N n and alpha s , spaced at each pair of the relative angle θ between the two food resource sites (Fig.  1) and the food increment δ in Eq. (3). The distance from the nest to food sources R is fixed as R = 50. All calculations start from t = 0, at which all ants are located in the nest and last until T = 10000, and the statistical results are averaged over 100 ensembles. This duration of calculation is sufficiently longer than the characteristic time for the formation of pheromone trails.

B. Transportation and Discovery Efficiencies
The foraging efficiency F is measured as an averaged quantity over a sufficiently longer time than the characteristic time for trial formation connecting a nest and food source sites with the simulations assuming a stationary environment. We investigate the short-term dynamical aspects of the foraging activity through two kinds of quantities to elucidate the basic mechanism and generate the characteristic features of the landscape of F : E T , the transportation efficiency and E D , the discovery efficiency. We employ the specialized feeding method to calculate T T , which is the duration when the second food in each simulation ran out after it is found, and T D , which is the duration between the timing of the first food running out and the second food found by an ant. Note that we define the timing of the second food found as the time the amount of the second food is reduced to 90% of the initial amount so as to distinguish the random access access and collective foraging. Using T T and T D , we quantify the transportation efficiency of food E T = 1/T T and the discovery efficiency of food E D = 1/T D . With respect to T D , a shorter duration means faster finding of a new food at the alternative feeding site and a high discovery efficiency E D . The introduced two kinds of efficiencies E T and E D are investigated after the simulations to measure the landscape of F for each set of (θ, δ) are performed. Therefore, these efficiencies are represented as a function of N n , α, and θ: E T = E T (N n , α s ; θ) and E D = E D (N n , α s ; θ).

IV. NUMERICAL RESULTS
We build the landscapes for the the quantities F (N n , α; θ, δ), E T (N n , α; θ), and E D (N n , α; θ), which are introduced in the previous section by varying the system parameters N n and α s and the feeding environment parameters θ and δ only for F (N n , α; θ, δ). First, we calculate the landscape of the foraging efficiency F (N n , α; θ, δ) to analyze what type of combination of α s and N n is optimal to achieve the most efficient foraging. Thereafter, the transportation efficiency and the discovery efficiency are calculated to analyze the short-term dynamics by varying α s and N n . With respect to the angle between the two food sources θ, it can be 0 < θ ≤ 180 • ; however, we discuss the results for θ ≤ 90 • in this report because the simulation results of the cases, where θ > 90 • , show that the all efficiencies qualitatively depicted similar landscapes to the case θ = 90 • . Thus, we focus on the cases θ ≤ 90 • .
As for the feeding environment, we conduct simulations using the sequential feeding method for three values of δ; δ = 0.1 as a starving feeding environment, δ = 0.8 as an intermediately feeding environment, and δ = 2.0 as a saturated feeding environment, in which the amount of food is always at or around the upper limit f c . Fig. 2 shows the landscapes of the averaged foraging efficiency F (N n , α s ; θ, δ) for the different feeding environment parameters, θ and δ. The figure shows that the optimal combination of (N n , α s ) found neither along the line N n = 500 nor along α s = α n varies depending on θ and δ. In other words, some sort of stochasticity among ants in the colony is indispensable to achieve the optimal efficiency, and the form of the optimal distribution of the stochasticity is affected by the environment conditions. This result as a whole supports the previously introduced idea of 'strategy of error.' However, the following outcome supplies us with the extended concept of the previous one. More specifically, in the case δ = 0.1, the optimal distribution of the stochasticity largely depends on θ. As shown in Fig. 2(1), where θ = 10 • , the efficiency is optimized when α s = 10 −2 , where a finite number, N n = 200, of normal ants is mixed in the colony. When θ = 45 • , the region of the efficient combi- nations shifts to ones, where the degree of stochasticity of the stochastic ants become higher, α s = 10 −4 , and almost half of the colony is stochastic ants (Fig. 2(2)). In the case of θ = 90 • (Fig. 2(3)), the optimal foraging is achieved when the stochasticity of the stochastic ants is the strongest α s = 10 −6 among the given range of α s , and both stochastic and normal ants coexist in the colony. In these cases of δ = 0.1, the food at the sources easily runs out because of the slow food refill. Thus, these results suggest that, for an efficient foraging, ants have to find another feeding site when the food sources at the discovered site run out, and a finite number of strong stochastic ants are required to keep their mobility between alternative feeding sites. We discuss these hypotheses on the next sub-section.

A. Foraging Efficiency for Two Food Sources
In the case of δ = 0.8 ( Fig. 2(4)-(6)), the trends of the deformation of the landscapes of F (N n , α s ; θ, δ) under the change of θ are kept similar with the above-mentioned case of δ = 0.1; however, the sharpness of the landscapes around the optimized combinations of (N n , α s ) is not so clear as the above-mentioned case. A possible reason for this is that the food sources do not easily run out compared to the above. Thus, the ants that find one of the food sources are easily able to obtain a finite amount of food even if they fail to find the alternative food source. This hypothesis is also discussed in the next sub-section.
In the case of δ = 2.0 ( Fig. 2(7)-(9)), the very strong stochasticity of the stochastic ants seem to rather disturb the optimal foraging in any value of θ.

B. Transportation and Discovery Efficiencies
In the previous section, we observed the landscape of F (N n , α s ; θ, δ) and confirmed that the stochasticity in following pheromone trails increased the foraging efficiency of ants in a colony, assuming that the two food sources were relatively close. This result supports the basic idea of the 'strategy of errors' by Deneubourg [2]. Especially, in the case, where two food sources are separated over a certain distance, θ = 90 • and the amount of food is not sufficient δ = 0.1 or 0.8, the contrasted coexistence of highly and weakly stochastic ants is the task allocation on errors for the optimal foraging, which seems to supply us with the extended concept of the strategy of error. However, the mechanism that improves the foraging efficiency in the latter case is still unclear. Thus, we discuss herein the short-term dynamics in the foraging behaviors to understand how N n and α s affect the collective dynamics by analyzing E T and E D introduced in the previous section. Fig. 3 presents the numerical results. E T represents the ability of ants to transport the food source to the nest, while E D characterizes the ability of ants to promptly explore the alternative food source resisting the attraction of the remaining pheromone around the initially found food source. The required degree of abilities largely depends on the distance between the food sources on θ.
In the case where θ = 10 • , the landscapes of E T and E D have similar optimal combinations (N n , α s ) to that of F . In other words, the existence of mildly stochastic ants in the colony improves the efficiencies in shortterm dynamics. At the same time, the foraging efficiency in the long term is improved with the optimal parameters. In contrast, in the case where θ = 45 • and 90 • (Fig. 3(2-3) and (5-6)), the landscapes of E T and E D do not have optimal parameters similar to those of F . Note that in the cases where parameters N n = 0 and − log α s > 2 ( Fig. 3(4-6)), E D shows extremely higher values than the other combinations of parameters because the strong stochasticity prevents colonies from constructing pheromone trails, and the spatial symmetry is not broken, indicating that no pheromone trail is constructed in the simulation. Accordingly, the extreme values N n = 0 or − log α s > 2 are ignored in the following discussion on the effect of the optimal combination of the stochasticity. The landscapes ofÊ TÊD and the product of normalized E T and E D , have landscapes similar to the optimal parameters of F , including the case where θ = 10 • (Table III), except for the above-mentioned extreme values of (N n , α s ).
These results indicate that the mechanism to achieve optimal foraging in the colony with the insufficient feeding environment can be discussed based on the shortterm dynamics such that the colony of ant must be equipped with the balance of abilities of finding a new food source and efficiently carrying food to the nest using the pheromone trail.

V. SUMMARY
In summary, through the simulations of a mathematical model reflecting the behavioral parameters of the existing species of ants, we confirmed herein that the coexistence of highly stochastic ants and normal or weakly stochastic ants in the same colony is the key to achieving the optimal foraging behavior in an environment with insufficient food supply at finitely separated food sources. In an environment with a sufficient feeding environment or in the case when two feeding sites are located nearby, the feeding efficiency was maximized when the colony was composed of only the weakly stochastic ants or their mixture with normal ants. In the latter case, stable food sources can be achieved by continuing to access the same or nearby source through the shortest pathway; however, in this case, the highly stochastic ants cannot follow the pheromone pathway, decreasing the ability to carry food. In contrast, mildly stochastic ants prevent the colony from being trapped with a too attractive, but non-optimized way of trail. In the former case, the food at each food source was easily and quickly exhausted; thus, the strategy relying on a single food source increased the risk of losing the source because of sudden accidents. Therefore, the probability of discovering other food sources must be increased such that the colony does not lose accessible food sources. In other words, the reason for the coexistence of highly stochastic and weakly stochastic ants in the same colony is such that the colony can adapt to unreliable external environments. We call this effect the "diverse-stochasticityinduced optimization," which is a type of role division of ants based on the degree of stochasticity to complete tasks with the highest efficiency. The concept of diversestochasticity-induced optimization appears to be applicable to a class of systems wider than the present case. However, the details depend on the situation of individual systems. One of which is the recently reported drastic reinforcement of the stochastic resonance of a multiexcitable-element system by exposing them to the noise of an element-dependent unequal amplitude [20,21]. Focusing once more on ants, an ant colony has many types of workers; some of which protect the queen, while others care for the larvae. Our research suggested another class of role division, that is, a role division based on the degree of stochasticity regarding their behavior.