Non-equilibrium steady states of stochastic processes with intermittent resetting

Stochastic processes that are randomly reset to an initial condition serve as a showcase to investigate non-equilibrium steady states. However, all existing results have been restricted to the special case of memoryless resetting protocols. Here, we obtain the general solution for the distribution of processes in which waiting times between reset events are drawn from an arbitrary distribution. This allows for the investigation of a broader class of much more realistic processes. As an example, our results are applied to the analysis of the efficiency of constrained random search processes.


Introduction
Suppose that you are working on a difficult problem. While trying to find a solution, you may get the impression that you are stuck or got on the wrong track. A natural strategy in such a situation is to reset from time to time and start over. This behavior can be modeled by a stochastic exploration process, which is interrupted by a random resetting to the initial condition. Over the last years a special case of such processes, a diffusion process interrupted at constant rate by reset events, has attracted considerable attention, because it represents a particularly simple and analytically approachable example of a non-equilibrium steady state (NESS) [1][2][3][4][5].
The investigation of processes with random resetting is also one of natural interest to the study of first passage times, e.g.in the catalysis time of chemical reactions [6], in kinetic proofreading [7], and in areas where search optimality is crucial [8]. Furthermore, processes with reset are studied in population dynamics, where resets are interpreted as catastrophic events corresponding to the extinction of the population, followed by a resurgence [9].
All of the previous works, including the extensions to Lévy Flights [10] and fluctuating interfaces with stochastic resetting [11], assume the special case that resets occur at a constant rate γ, i.e.the case in which the waiting times between the resets are exponentially distributed. In this case the resetting procedure is memoryless, which has a straight-forward but interesting implication: the NESS of the process is equivalent to the distribution of end-points of an ensemble of trajectories of the same process without resetting that are trapped at a constant rate γ. Consequently, if the propagator of the process without resetting is known, the NESS can be directly obtained by averaging the propagator over the exponential distribution of inter-reset times [1]. However, the limitation to constant rate resetting severely restricts the applicability to memoryless processes. Even the simple example of a process in which multiple identical steps have to be completed for a reset, which gives rise to Gamma-distributed waiting times, cannot be described within the current framework. Such processes occur e.g.in chemical resetting due to multistep dissociation reactions [12]. Neither can the important case of a rate that depends on the time τ elapsed since the last reset event be captured anymore. Processes of this form can be described using time-dependent rates Any further distribution of this work must maintain attribution to the author(s) and the title of the work, journal citation and DOI. more intermittent (regular) reset sequences, as compared to the constant rate case ( 1 a = ). The Weibull distribution is widely applied when the conditions for strict history-independence are violated [13].
In this article, we present the theory of stochastic resetting with arbitrary waiting time distributions ( ) y t , (see figure 1), which includes the aforementioned examples as special cases. While for stochastic processes that reset at a constant rate, the master equation can be immediately formulated, here we first have to derive the governing equation for arbitrary resetting time distributions. The corresponding master equation equation (6) is non-local in time and is one of our main results. Despite the fact that our theory has to include integrals over the complete history of the process, we can explicitly calculate stationary distributions for different waiting time distributions, ( ) y t , and quantify the temporal relaxation of the moments towards this NESS. We find that the history-dependent processes with a general resetting time distribution have a rich structure and exhibit many new properties that are not present in the case of resetting with a constant rate. Furthermore, we demonstrate that the success of a search depends on the full waiting time distribution and not only on the characteristic resetting time scale as one could expect from the memoryless process. A possible application of our theory is the efficiency of random searchers that are confined by their need to regularly return to a home location. We find that, under this constraint, the search success can be optimized by adapting the distribution of waiting times between the returns.

Derivation and solution of the generalized master equation
Consider the motion of a particle that, between reset events, is described by the stochastic differential equation Furthermore, let the waiting time distribution between the resets be ( ) y t , after which the particle is reset to the location x r . The probability t n ( ) n that the nth reset event happens at time t satisfies the renewal equation t t t t t d , stating that the probability to have the nth event at time t is given by the probability that the n 1 th event occurred at time t ¢ and that the next event occurs after time t t -¢ [14]. We assume that the process starts with a reset event. Then, the probability that an event occurs at time t is t t t n n 1 where the δ-distribution accounts for the initial condition.
The probability p x t , ( )to find the particle at location x at time t is where p x x t t , ; r ( ) -¢ is the transition amplitude of the process defined by equation (1). The distribution w t t ( ) -¢ gives the probability that no resetting event occurs between t ¢ and t and is related to the waiting time and applying the convolution and shifting theorem, the Laplace Figure 1. Examples of different processes x(t) with random resetting (marked with red crosses). All cases share the same mean duration between resets, 1 t á ñ = , but their distributions differ. (a) Diffusion with intermittent (bursty) resetting. A resetting event is more likely to occur when another one has happened recently. Distribution of resetting times is heavy tailed. (b) Diffusion with resetting times that are comparatively regular (distribution of resetting times peaked around mean). (c) Stochastic resetting with deterministic, linear motion between resetting events. transform of equation (2) reads we can use equations (3) and (4) to obtain is the initial condition. Finally, Laplace inversion of equation (5) leads to the generalized master equation The first term on the rhs accounts for the dynamics between the resetting events, the second term is the source term and describes the resets to x r . The third term is a sink term; it describes the density of particles that have propagated to x at time t during the time t t -¢ under the influence of the dynamics equation (1) and are then subject to resetting. The integral of the time-evolution kernel corresponds to the time-dependent density of reset The special case of resetting at a constant rate γ is recovered by choosing an Inserting this kernel into equation (6) leads to the Markovian master equation studied in [1]. The solution of equation (6) in Laplace space is Here the first term on the rhs describes the decay of the initial condition and the second term accounts for the resets. While the Laplace inversion of equation (7) can in general only be carried out by numerical means, we will show that the time-dependent moments of the distribution can be calculated analytically. Remarkably, even the full stationary solution p x p x t lim , t st ( ) ( ) = ¥ can be given by employing the Tauberian theorems [17], which allow to consider instead the limit p x s lim , in Laplace space.

Constant advection with intermittent resetting
Before we come to the diffusion processes with resetting, let us first exemplify equation (6) using the example of constant advection with velocity v between the reset events ( figure 1(c)). The discussion of this simplest of all cases helps to illustrate the intricacies that come with the inclusion of non-memoryless resetting. In particular, we set F x v ( ) = and D=0 in equation (1) and consider the Liouvillian x v x ( )  = - ¶ . Without loss of generality let us from now on assume the reset position to be x r =0. Using equation (6), we obtain for the time evolution of the first moment in Laplace space Clearly, the mean of the process depends on the precise form of the waiting time distribution through ŝ ( ) f even in the stationary limit. For concreteness, let us consider the waiting times to be Gamma-distributed with rate γ and shape parameter α, i.e. ; , e . The mean of this distribution is t a g á ñ = and for 1 a = it includes the exponential distribution. The shape parameter α regulates how intermittently the resetting events occur. For 1 a < these events occur more intermittently the smaller α gets, whereas for a  ¥ the resetting events occur regularly with period t á ñ. In figure 2(a) we compare x t ( ) á ñ for Gamma-distributed waiting times with the same mean waiting time but with different rate and shape parameters, calculated using equation (8). Interestingly, the asymptotic value of x t ( ) á ñ depends on the shape of the distribution, even when the mean time between resets, t á ñ, is kept constant. We can also calculate the asymptotic, steady-state value of the mean by expanding equation (8) in s, keeping only the lowest order, and then performing the Laplace inversion. This leads to x v 2 st 1 t á ñ = á ñ a a + , which only for exponentially distributed waiting times is equal to the typical distance x v s t = á ñthat a particle moves between resets ( figure 2(b)). The stationary distribution can be calculated using the same approach. Inserting the Liouvillian , in equation (7), and substituting the appropriate ŝ ( ) f , the resulting expression can be considered in the limit s 0  . For example, in the case of Gamma-distributed resetting times, the resulting expression in Fourier space , which can easily be Fourier inverted. Several examples are given in figure 2(c). The stationary distribution exhibits a transition from a broad-tailed distribution for 1 a < , where rare large excursions dominate the process, to a uniform distribution in the limit a  ¥, where the process is reset deterministically after time t á ñ.

Diffusion with intermittent resetting
Let us now focus on the case of free diffusion between the resets, i.e.F x 0 ( ) = , with x D x 2 ( )  = ¶ . In this case, the process is symmetric around the reset point and the mean is simply x r (set to zero for simplicity). Interestingly, calculating the second moment of the process, x t 2 ( ) á ñ from equation (6), leads to exactly the same equation as the mean of the advection process, but with v replaced by D 2 . Thus, considering the time evolution of the variance and its final state leads to exactly the same curves as shown for the mean of the advection in figures 2(a) and (b), as we have also confirmed numerically. The stationary distribution p x st ( ) can again be calculated by considering equation (7) In figures 3(a) and (b) we compare the analytical solution equation (9) to numerical simulations of the process and find a perfect agreement. More intermittent processes, i.e.smaller α lead to broader (but still exponentially decaying) stationary distributions, since longer diffusive excursions become more likely. The stationary standard deviation, We can again calculate the stationary distribution analytically and observe the same qualitative behavior as for the Gamma distribution. The interesting case of regular resetting at t á ñ can be obtained in the limit of a  ¥ of either the Gamma or the Weibull waiting time distribution (while keeping t á ñ fixed), or indeed as a limit of any waiting time distribution that approaches ( ) ( ) y t d t t = -á ñ as an appropriate limit. In this important case, where resets are precisely timed, we can express the stationary distribution concisely as

Search efficiency
Evidently, the resetting protocol also has a strong influence on the search efficiency to find a target located at distance x 0 from the origin. For the case of constant rate resetting, an x 0 -dependent optimal rate has been obtained in [1]. However, in many situations the search strategy is constrained by the need to return on average after a characteristic time t á ñ. For example, many foraging animals have to return home regularly to rest because of exhaustion of energy or to feed their offspring. Our previous results indicate that the efficiency can also be optimized by adapting the functional form of the distribution. In figure 4(a) we show for Gamma-distributed resetting times with a fixed mean resetting time t á ñ that the mean first passage time indeed exhibits a where the C has to be determined numerically once and for Gamma-distributed resetting times is C 0.83 » . In figure 4 we compare our analytical prediction equation (11) with the numerically determined values of x opt 0 ( ) a and find a very good agreement. For small x x 0 t y p < equation (11) is no longer valid and it is very hard to identify opt a because the stationary distribution becomes increasingly insensitive to variation of α for 1 a > , see figure 4(a). These results indicate that an intermittent return strategy should be adopted when targets remote from the home may exist. While, we have focused here on optimal search in one dimension, extending our work to two and higher dimensions poses no new conceptual difficulties, and it would be interesting to compare such an extension to recent results on intermittent search strategies [18].

Conclusions
In conclusion, our study reveals that the NESS as well as the search efficiency depend on the precise form of the resetting distribution. Interestingly, for continuous time random walks-a related class of models, where the times between successive jumps are drawn from a waiting time distribution-such a dependence of the long time behavior on the precise form of the waiting time distribution is not observed. Here the knowledge of the characteristic waiting time is sufficient to characterize the steady state distribution of the process in a confining potential [19]. Finally, we note that, for general resetting times, the steady state of the process cannot be simply obtained by averaging the propagator over the waiting time distribution. Instead, one has to average over the distribution of times that have elapsed since the last reset event, the so-called backward occurrence time distribution. This distribution is only identical to the waiting time distribution for memoryless processes, i.e.exponentially distributed waiting times. For other waiting time distributions the corresponding backward occurrence time distribution can in general not be obtained in a closed form. Our work opens several further interesting directions for future research. Our theory can be used to consider deterministic dynamics with resetting at random locations, and thus has direct relevance for spiking dynamics in neurons that emit spikes upon reaching a random threshold. Furthermore, it would be interesting to investigate the search efficiency of diffusion processes with generalized resetting in the presence of a binding potential.