Estimating Entropy Production Rates with First-Passage Processes

We consider the problem of estimating the mean entropy production rate in a nonequilibrium process from the measurements of first-passage quantities associated with a single current. For first-passage processes with large thresholds, Refs. [1, 2] identified a ratio of first-passage observables - involving the mean first-passage time, the splitting probability, and the first-passage thresholds - that lower bounds the entropy production rate and is an unbiased estimator of the entropy production rate when applied to a current that is proportional to the stochastic entropy production. Here, we show that also at finite thresholds, a finite number of realisations of the nonequilibrium process, and for currents that are not proportional to the stochastic entropy production, first-passage ratios can accurately estimate the rate of dissipation. In particular, first-passage ratios capture a finite fraction of the total entropy production rate in regimes far from thermal equilibrium where thermodynamic uncertainty ratios capture a negligible fraction of the total entropy production rate. Moreover, we show that first-passage ratios incorporate nonMarkovian statistics in the estimated value of the dissipation rate, which are difficult to include in estimates based on Kullback-Leibler divergences. Taken together, we show that entropy production estimation with first-passage ratios is complementary to estimation methods based on thermodynamic uncertainty ratios and Kullback-Leibler divergences.


Introduction
Empirical measurements of dissipated heat in nonequilibrium, mesoscopic systems are challenging. The obstacle is that heat, defined as the remainder energy required to satisfy the first law of thermodynamics [3], is microscopically the energy exchanged between a system and its environment through interactions with a large number of microscopic degrees of freedom [4]. Therefore, contrarily to the work performed on a system, heat cannot be measured directly.
To illustrate the empirical difficulty of measuring heat in small systems, let us consider the example of a molecular motor in a living cell. A molecular motor dissipates at a power of 10 −18 W [5,6,7], which is several orders of magnitude smaller than the 10 −6 W sensitivity of calorimeters [8,9]. Nevertheless, even if it were possible to measure heat at an accuracy of 10 −18 W , then still it would not be possible to measure the rate of dissipation of a molecular motor in a living cell, as in living cells a large number of dissipative processes take place at once. Therefore, it is important to develop alternative methods to measure dissipation in nonequilibrium, mesoscopic systems.
Instead of measuring the rateṡ of dissipation, also known as the entropy production rate, with calorimeters, the entropy production rate can be estimated from the trajectories of slow, coarse-grained degrees of freedom, which is an example of a thermodynamic inference problem [10]. In this paper, we consider entropy production estimation based on the trajectories of a single current J in a stationary process. An observer measures, e.g., the current in an electrical wire or the position of a molecular motor bound to a biofilament, and aims to estimate the rate of dissipation from the obtained trajectories. If the average current J(t) = 0, then the system is driven out of equilibrium. However, in general, it is not possible to estimate the rate of dissipation from the average current, except when the thermodynamic affinity is known. On the other hand, when the whole trajectory of the current J is available, then it is possible to estimate the entropy production rate from fluctuations in the current [11].
An estimator of dissipation is a real-valued functional evaluated on the trajectories of a nonequilibrium process. In the present paper we limit ourselves to estimators that are evaluated on the trajectories of a single current J, and we also consider estimators that do not require knowledge about the model that determines the statistics of the nonequilibrium process X. These two limitations model physical scenarios for which only partial information about the nonequilibrium process is available, such as, the case when the position of a molecular motor is measured, but the chemical states of its motor heads are not known.
The quality of an estimator is determined by its bias, i.e., the distance between the average value of the estimator and the rate of dissipation. Since we consider problems with partial information, it is in general not possible to obtain an unbiased estimator. In fact, when observing a single current, a good estimator of dissipation will underestimate the amount of dissipation in the process, as in general there exist other dissipative processes that do not directly influence the measured current. A notable exception is when the current J is proportional to the stochastic entropy production. In this case, we expect a good estimator to be unbiased. Lastly, let us mention that the bias of an estimator can depend on the number of trajectories available. In other words, an estimator may have a small bias in the limit of an infinite number of realisations of a nonequilibrium process, while in situations of finite data the same estimator may have a large bias.
As the entropy production estimation problem has been considered before, we first review two estimators that have been well studied in the literature. Subsequently, we present an estimator based on first-passage processes that has been less studied, and which constitutes the main topic of this paper.
The first estimator that we review is the Kullback-Leibler divergence between the probability to observe a trajectory of a current and the probability to observe its timereversed trajectory, see e.g., Refs. [12,13,14,15,16,17]. In general, it is however impractical to evaluate the Kullback-Leibler divergence. This because the statistics of a single current J are nonMarkovian, which complicates significantly the evaluation of the probability to observe a certain trajectory. Because of this complication, nonMarkovian effects in the current are often neglected by evaluating [12,13,14] whereṅ j is the rate at which the current J makes a jump of size j, and J is the set of possible jump sizes. The estimator (1) neglects nonMarkovian statistics in J's trajectory, and consequently the estimator is biased, i.e.,ŝ KL ≤ṡ.
A second estimator of dissipation that we review is the Thermodynamic Uncertainty Ratio (TUR) [18,11,10,19,20] where J(t) is a stochastic current at time t, j = J(t) /t is the average current rate, and σ 2 J(t) = J 2 (t) − J(t) 2 is the variance of the current. The TUR has been proposed as an estimator of dissipation as it lower bounds the entropy production rate in stationary Markov jump processes and overdamped Langevin processes, i.e.,ŝ TUR ≤ṡ. However, the bias of the TUR is large for systems far from thermal equilibrium. For example, it was shown thatŝ TUR captures 20 % of the total dissipation of molecular motors, such as kinesin and myosin [6,7], and the bias of the TUR remains large when J equals the stochastic entropy production [21].
In an attempt of resolving the issues in the bias of the Kullback-Leibler divergence and the TUR, we study in this paper the First-Passage Ratio (FPR) [1,2] s FPR ( + , − ) := where T J is the first time when the stochastic current J leaves the open interval (− − , + ) with − , + > 0, p − is the probability that J first goes below the negative threshold − − before exceeding the positive value + , and p + = 1 − p − . For stationary Markov jump processes and in the limit of large thresholds − and + , the FPR bounds the dissipation rate from below, i.e.,ŝ FPR ≤ṡ [1,2]. In addition, the FPR is an unbiased estimator when J is proportional to the stochastic entropy production [1,2], i.e.,ŝ FPR =ṡ, a result that also holds in the limit of large thresholds.
Although it was established thatŝ FPR equalsṡ for large thresholds and for currents that are proportional to the stochastic entropy production, this is not sufficient to conclude thatŝ FPR is a useful estimator of dissipation. Therefore, in the present paper, we determine the bias in the estimatorŝ FPR for: (i) generic currents that are not proportional to the entropy production; (ii) finite thresholds + and − ; and (iii) a finite number of realisations of the nonequilibrium process. To this aim, we estimate dissipation based on the FPR in two paradigmatic examples of nonequilibrium Markov jump processes, namely, a random walker on a two-dimensional lattice [see Fig. 1

(a)]
(a) Random walk on a two-dimensional lattice < l a t e x i t s h a 1 _ b a s e 6 4 = " j t 5 4 / + E F 1 u p x y N a h t e V y w L x u U a s = " < l a t e x i t s h a 1 _ b a s e 6 4 = " r u P R 7 q P f O L C V q k 5 K A A v H P y + G y 6 0 = " > A A A B 9 H i c b V B N T 8 J A E J 3 i F + I X 6 t H L R m I C F 9 I a g s Y T 0 Y t H T A B J a E O 2 y x Y 2 b L d 1 d 0 t C G n 6 H F w 8 a 4 9 U f 4 8 1 / 4 w I 9 K P i S S V 7 e m 8 n M P D / m T G n b / r Z y G 5 t b 2 z v 5 3 c L e / s H h U f H 4 p K O i R B L a J h G P Z N f H i n I m a F s z z W k 3 l h S H P q e P / v h u 7 j 9 O q F Q s E i 0 9 j a k X 4 q F g A S N Y G 8 k r 1 y s u K r d u 3 H j E K v 1 i y a 7 a C 6 B 1 4 m S k B B m a / e K X O 4 h I E l K h C c d K 9 R w 7 1 l 6 K p W a E 0 1 n B T R S N M R n j I e 0 Z K n B I l Z c u j < l a t e x i t s h a 1 _ b a s e 6 4 = " n y F l v 7 w 8 x N U 6 m 4 n o P S x L 2 V 1 s l l a P S L X 7 2 B p E n I I q C C a O 0 6 d g x e S h R w K t i 0 0 E s 0 i w k d k y F z D Y 1 I y L S X z k + e 4 j O j D H A g l a k I 8 F z 9 P Z G S U O t J 6 J v O k M B I L 3 s z 8 T / P T S C 4 9 l I e x Q m w i C 4 W B Y n A I P H s f z z g i l E Q E 0 M I V d z c i u m I K E L B p F Q w I T j L L 6 + S 9 n n V u a z W H m u l + m 0 W R  paradigmatic examples of nonequilibrium processes that we consider in this paper. Subfigure (a): Random walker jumping on a twodimensional lattice at rates k + 1 , k − 1 , k + 2 , and k − 2 . Subfigure (b): Markov jump process with six states, representing the different chemical configurations of the two motor heads of a molecular motor bound to a biofilament [22,6]. The different motor states are ATP bound (T), ADP bound (D), and nucleotide free (φ). Since there are six states, we also label them with {1, 2, 3, 4, 5, 6} as indicated in the figure. The motor moves forwards along the cycle (2 → 5 → 6 → 1 → 2) and moves backwards along the cycle (5 → 2 → 3 → 4 → 5) at rates j f and j b , respectively. and a model for a molecular motor with two motor heads [see Fig. 1 To evaluate the quality ofŝ FPR , we compare the bias in the estimates of dissipation based on the FPR with those based on the TUR and the Kullback-Leibler divergence.
The paper is organised as follows: In Sec. 2, we introduce the setup of stationary Markov jump processes that we use in this paper, and we define entropy production in this setup. In Sec. 3, we define estimators of entropy production based on the FPR, the TUR, and the Kullback-Leibler divergence. In Sec. 4, we make a study ofŝ FPR as an estimator of the entropy production rate in a two-dimensional random walk process, and in Sec. 5, we make an analogous study for a model of a molecular motor with two motor heads, such as, conventional kinesin. We end the paper with a discussion in Sec. 6, where we compare the advantages and disadvantages of the different estimators for dissipation. The paper also contains three appendices with detailed information on some of the calculations and the parameters used in the models.

Entropy production in stationary Markov jump processes
We first present in Sec. 2.1 the mathematical setup on which we work in this paper, and in Sec. 2.2 we discuss the physical context of the mathematical setup.

General system setup
We focus in this paper on stationary Markov jump processes [23], which play an important role in stochastic thermodynamics [24,25]; doing so, we neglect the possibility of a Hamiltonian evolution [26].
Let X(t) be a Markov jump process taking values in a discrete set X [23], with t ≥ 0 a time index. We denote by P the probability measure that generates the statistics of X, and averages with respect to P are denoted by · P , or more briefly · . Trajectories of X(t) are denoted by X t 0 . We assume that the process X is regular, i.e., it has a finite number of jumps within each time interval of finite length. The probability distribution p X(t) (x) = p(x; t) of X(t) solves Kolmogorov's backward differential system [23] where w x→y ∈ R + are the transition rates to jump from x to y. We consider timehomogeneous Markov jump processes for which the rates w y→x are independent of time t. We assume that X admits a unique stationary probability distribution p ss (x) with p ss (x) > 0 for all x ∈ X , and we set p X(0) (x) = p ss (x), so that the process is stationary. As shown in Ref. [23], in this setup it holds that for generic initial conditions the stationary state is reached in a finite time, i.e., lim t→∞ p(x; t) = p ss (x), and moreover the ergodic theorem implies that lim t→∞ t −1 t 0 f (X(s))ds = x∈X f (x)p ss (x) for all initial distributions and real-valued functions f with x∈X |f (x)|p ss (x) < ∞ [23]. Note that for finite sets X , a sufficient requirement for the uniqueness of the stationary probability distribution is that the graph of nonzero transitions w x→y is strongly connected -a property that is often called irreducibility-and for countable sets X we also require that the process is t-positive recurrent, i.e., the mean time for X to return to its initial state is finite [23].
The last assumption we require is that the Markov jump process is reversible in the sense that Given the above assumptions, we can define the stochastic entropy production by [27,25] where the edge currents denote the difference between the number of times N x→y (t) the process X has jumped from x to y in the time interval [0, t] and the number of times N y→t (t) the process X has jumped from y to x in the same time interval. We denote the mean entropy production rate byṡ which will be the main quantity of interest in this paper.

Physical interpretation ofṡ based on local detailed balance
To give physical content to the entropy production rateṡ-in the sense of dissipation or the entropy production of the second law of thermodynamics -it is necessary to make physical assumptions about the nature of the environment and the interactions between the system and its environment. It should be emphasised that these physical assumptions are mathematically superfluous, as they do not alter the model setup given by Eq. (4). Indeed, Eq. (4) provides a coarse-grained description of a system for which environmental degrees of freedom have already been integrated out. Nevertheless, to understand the physical relevance ofṡ, it is necessary to consider how Eq. (8) appears in the broader picture of a system coupled to an environment. We assume that X ∈ X represents the slow, coarse-grained degrees of freedom of a mesoscopic system that is in contact with one or more equilibrium reservoirs. A given mesoscopic configuration x ∈ X has an energy u(x) and an entropy s int (x) due to the existence of fast, internal degrees of freedom.
Following Refs. [28,24,29,25], we use the principle of local detailed balance, which states that w x→y w y→x = e sxy , with x, y ∈ X , where s xy (= −s yx ) is the change of entropy in the environment and the fast, internal degrees of freedom of the system when X jumps from x to y. Note that Eq. (9) does not require that the system is near equilibrium -asṡ can be large -but it does require that the environment is equilibrated. To have a system far from thermal equilibrium that interacts with an environment in equilibrium, we require, in general, weak coupling between system and environment and fast relaxation of the degrees of freedom in the environment.
The precise form of s xy depends on the physical problem of interest. For a system in thermal equilibrium, where is the free energy of the x-th state, and T env is the temperature of the environment. Note that we use natural units for which Boltzmann's constant is set to one. For a nonequilibrium system in contact with a thermal reservoir, Eq. (10) extends to where Q is the net heat absorbed by the system from the environment. Equation (11) provides a direct link between the entropy production rateṡ, as defined in Eq. (8), and the rate of heat dissipated to the environment, Therefore, in this paper, we callṡ the entropy production rate, as well as, the rate of dissipation. As a last example, let us consider the case of a molecular motor that interacts with m chemostats at chemical potentials µ α , with α = 1, 2, . . . , m, and an external agent that exerts a mechanical force f mech on the motor. In this example [18] where n α (x, y) is the number of particles exchanged with the α-th reservoir when X jumps for x to y, and δ(x, y) is the distance moved by the motor when X jumps for x to y. We follow the usual convention that n α > 0 when the system binds particles and n α < 0 when the system releases particles [3]. The minus sign in front of f mech implies that the mechanical force opposes the motion of the motor when it moves forward.

Estimators ofṡ that rely on the measurement of a single current
We aim to inferṡ based on the measurements of a single stochastic current of the form where c x,y ∈ R; observe that the stochastic entropy production S, as defined in (6), is a specific example of a current. If J = cS, with c a constant, then this is equivalent to an estimation problem with complete information, while otherwise, this is an estimation problem with partial information. Without loss of generality, we assume in what follows that J(t) /t = j > 0.

Estimators of dissipation: general definitions
Given n s independent realisations J (i) (t) of a current J(t), with i = 1, 2, . . . , n s , an estimator F( J t 0 ) ofṡ is a functional defined on the set of trajectories . The bias of the estimator F is its average value is the bias of the estimator in the asymptotic limit of an infinite number of realisations, and g(n s ) is a function that determines how the bias depends on n s ; note that g(n s ) → 0 for n s 1. An estimator is unbiased in the limit of an infinite number of realisations whenŝ =ṡ, and it is an unbiased estimator, tout court, when bothŝ =ṡ and g = 0.
In what follows, we define three estimators ofṡ, namely, F FPR based on the FPR, F TUR based on the TUR, and F KL based on the Kullback-Leibler divergence. As we discuss, all the estimators capture a finite fraction of the dissipation, i.e. s ≤ṡ, but the equality is attained in different circumstances. Consequently, all the functionals we introduce are useful as estimators of dissipation, but they capture dissipation in different scenarios.

3.
2. An estimator based on the first-passage ratioŝ FPR (FPR) 3.2.1. Definition. We define an estimator F FPR ofṡ for which it holds that whereŝ FPR is the first-passage ratio Eq. (3), with and Since an estimator F is defined on n s trajectories of the current J, we consider the first-passage times associated with the different trajectories [J (i) ] t 0 of the current, and the decision variables that determine for each trajectory whether the current crosses first the positive threshold (D (i) = 1) or crosses first the negative threshold (D (i) = −1).
Using T (i) and D (i) , we define, assuming J(t) > 0, the estimator whereT is the empirical mean of the first-passage time, and is the empirical estimate of the splitting probability p − . Analogously, we can define the estimator F FPR for J(t) < 0.
where we added o min (1) to indicate that the inequality holds asymptotically for large thresholds min = min { + , − }; here o min (1) represents the small-o notation, i.e., a function that converges to zero for large values of min . The equality in Eq. (26) is attained when J = cS, with c a constant [1,2], while for currents that are not proportional to S the estimate is in general biased. Now, let us discuss how the bias depends on the number of realisations in the process. For small values of n s it holds that as the number of events with D (i) = −1 is negligible. Consequently, for n s 1/p − , F FPR grows logarithmically in n s , i.e., Notice that in Eq. (28), for our convenience, we have assumed 1/T J ≈ 1/ T J , as this will not effect the scaling arguments in n s . At larger values n s 1/p − , the mean value F FPR saturates to Combining Eq. (28) with Eq. (3), and considering that we assumed j > 0, we obtain Equation (30) implies that around n s ≈ 1/p − the bias of the estimator saturates to its asymptotic value, and the bias grows logarithmically at smaller values smaller of n s .
For the case when J = S, martingale theory for e −S provides exact expressions for p − in terms of the thresholds − and + [30,31,32]. As shown in Appendix A, for Markov jump processes near equilibrium (ṡ ≈ 0), so that we require a number of samples, which grows exponentially in the depth of the negative threshold − . For process that are governed far from thermal equilibrium, we obtain instead (see Appendix A) where is the infimum of the jumps sizes in S(t). Equation (33) implies that the number of samples required grows as which is exponential in either the depth of the negative threshold − or the minimal jump size ∆s, depending on which one is larger. Hence, for processes that are governed far from thermal equilibrium, the number of samples required is independent of the thresholds, but instead depends on the jumps size ∆s.

3.
3. An estimator based on the thermodynamic uncertainty ratioŝ TUR (TUR) 3.3.1. Definition. We define an estimator F TUR ofṡ for which it holds that whereŝ TUR is the thermodynamic uncertainty ratio (2). The corresponding estimator F TUR , defined on a finite number n s of realisations of X, is and hence also the TUR captures a finite fraction of the total dissipation. The equality in Eq. (38) is attained by processes near equilibrium. On the other hand, for processes driven away far from thermal equilibrium, the bias in F TUR is large, i.e.,ŝ TUR ṡ [21,37]. Improvements on the TUR exist, but these rely on measurements of both currents and time-integrated empirical measures [38,39], and thus require more information about the process X.
To determine the dependence of F TUR on n s , we rely on the central limit theorem, from which it follows that where O denotes the big-O notation. The correction term in Eq. (39) is due to sampleto-sample fluctuations in the current.   Since we assumed that the process is reversible, see Eq. (5), it holds that −j ∈ J whenever j ∈ J . Therefore, we define the estimator that estimates the entropy production rate assuming that the process is Markovian; F KL (t) is closely related to the apparent entropy production considered in Refs. [40,16]. In Eq. (43), it should be understood that 0 log(0) = 0.

Bias. It holds that [40, 16]
which follows from applying Jensen's inequality to the definition ofṡ, which states thatṡ is a Kullback-Leibler divegence. The equality is attained when the current J is Markovian and a linear combination of all independent currents in X.
We discuss how the bias F KL depends on n s . Let us assume that J > 0, n s t is small, andṡ is large enough, so that N (i) −j = 0 for j > 0. In this case, where is the total number of jumps observed, and is the fraction of jumps of size j. Setting N (t)/t ≈ṅ n s , we obtain that Hence, analogous to the FPR, the bias of F KL (t) grows logarithmically in the number of samples n s until it reaches its asymptotic valueŝ KL , and therefore F KL (t) takes a form similar to Eq. (30).
In what follows, we compare estimation of dissipation with the three estimators F FPR , F TUR and F KL , in two paradigmatic examples of nonequilibrium processes. As discussed, this boils down to determiningŝ/ṡ and g(n s ), as defined in Eq. (15), for these three estimators.

Random walker on a two-dimensional lattice
We use first-passage processes for currents to infer the rate at which a random walker on a two-dimensional lattice produces entropy. In particular, we compare estimates based on F FPR with those based on F TUR and F KL .
In Sec. 4.1 we define the model. Subsequently, we analyse in Sec. 4.2 the bias in the estimator F FPR for currents that are not proportional to S, and in Secs. 4.3 and 4.4 we analyse the effect of finite thresholds and a finite number of realisations, respectively, on the bias of F FPR .

Model definition
We consider a two-dimensional hopping process X = (X 1 , X 2 ), with X 1 , X 2 ∈ {1, 2, . . . , n} the coordinates of a random walker on a two-dimensional lattice of length n with periodic boundary conditions [see Fig. 1(a) for an illustration]. The coordinates evolve according to where N + i and N − i are two counting process with rates k + i and k − i , respectively, and the boundary conditions are implemented by setting X i ± n = X i .
We parameterise the rates as and so that the total rate Notice that the total rate fixes the characteristic time scale of the process. We assume, without loss of generality, that ν ≥ 0 and ρ ≥ 1.
The stochastic entropy production, as defined by (6), is in the present example given by as p ss (x) = 1/n 2 independent of x. Consequently, the entropy production ratė From Eqs. (51) and (52) it follows that ν is the affinity associated with X 1 and ρ is the ratio between the affinities of X 2 and X 1 . From a geometric point of view, the stochastic process S in Eq. (51) can be represented as a vector in the (X 1 , X 2 )-plane. The parameter ρ fixes the direction of this vector and ν its magnitude. Without loss of generality, we can parametrise stochastic currents in this model by Since the FPR and TUR are independent of c, we set c = 1. If then J(t) is proportional to S(t), which is independent of ν. In the present model, the currents J are Markovian. This is a peculiar feature of the random walk model as, in general, currents in Markov jump processes are nonMarkovian.

Entropy production estimation with currents that are not proportional to S
When J is proportional to S, thenŝ FPR is unbiased in the limit of large thresholds, and hence in this case the FPR is an optimal estimator of dissipation. However, for currents that are not proportional to S,ŝ FPR is biased. We analyse here the effect of the parameter ∆ that determines the current J on the biases inŝ FPR (in the limit of large thresholds),ŝ TUR andŝ KL .
In the limit of large thresholds, the mean first-passage time (see Eq. (17) of Ref. [2]) and the splitting probability (see Proposition 7 of Appendix E of [2]) where z * < 0 is the nonzero solution to  (49), with ρ = 2 and ν = 5; the corresponding value ofṡ ≈ 7.5k total . The ratioŝ TUR is evaluated at t = 20/k total . The dashed, blue, horizontal line is a guide to the eye that indicates the value 1 corresponding to optimal thermodynamic inference.

Hence, for min
which is independent of the ratio + / − . Figure 2 plotsŝ FPR as a function of ∆ and shows graphically thatŝ FPR ≤ṡ, in correspondence with the general bound (26). Moreover, the equalityŝ FPR =ṡ is attained when ∆ is given by Eq. (54), as indicated by the vertical, black, dotted line in Fig. 2, and henceŝ FPR is an unbiased estimator when J(t) = S(t). Indeed, this can be verified analytically, as solves the Eq. (57) when ∆ is given by Eq. (54). Substituting the values of ∆ and z * given by (54) and (59) in Eq. (58), we recover (52), and hence we obtain in this case thatŝ FPR =ṡ. Let us now compare the biases ofŝ FPR andŝ TUR . A direct computation of j and σ 2 J(t) for a random walker on a two dimensional lattice giveŝ  The ratiosŝ FPR /ṡ (in the limit of min → ∞) andŝ TUR /ṡ are plotted as a function of the rate of dissipationṡ/k total for the current with ∆ = 0.6 in the random walk model defined on a two-dimensional lattice. The rates k + 1 , k − 1 , k + 2 , and k − 2 are parametrised according to Eqs. (48) and (49), with ρ = 2, while the parameter ν depends on the ratė s of dissipation. The horizontal dashed line denotes the asymptotic limit lim ν→∞ŝFPR given by Eq. (62). Figure 2 plotsŝ TUR /ṡ as a function of ∆, demonstrating graphically thatŝ TUR <ṡ in correspondence with the bound Eq. (38). However, contrarily toŝ FPR , the TUR captures at most 25% of the total dissipation. This is due to the fact that the rate of dissipationṡ ≈ 7.5k total is large enough, and the TUR does not estimate well dissipation far from thermal equilibrium.
To demonstrate that at high values ofṡ the FPR estimates better the entropy production rate than the TUR, we consider the asymptotic limit of ν 1, corresponding with large dissipation ratesṡ. In this limit, the nonzero solution to Eq. (57) is so that the FPR captures a nonzero fraction of the dissipation in the limit of large entropy production rates, viz., On the other hand, in the limit of large ν, Eq. (60) implies that and hence in the limit ofṡ 1 the TUR captures a zero fraction of the total dissipation.
The fact thatŝ FPR captures a nonzero fraction of the entropy production rate far from thermal equilibrium, whileŝ TUR captures in this limit a zero fraction of the entropy production rate, is demonstrated graphically in Fig. 3.
Another aspect ofŝ FPR that is well illustrated with Fig. 2 is that the maximum value ofŝ FPR is attained when J = cS. Instead, forŝ TUR the most precise current is determined by Hence, the most precise current that maximisesŝ TUR is in general not proportional to the stochastic entropy production [37,41]. For example, in Fig. 2, the most precise current has a value ∆ ≈ 0.0067, which is different from ∆ = 1/3 corresponding with J = S (both are indicated by the vertical dotted lines). Lastly, let us compareŝ FPR withŝ KL . Since J is Markovian, it holds thatŝ KL =ṡ, and hence in the present example estimates based on the Kullback-Leibler divergence are unbiased. This is a specific feature of the random walk model that does not extend to other models, as currents in Markov jump processes are in general nonMarkovian, such that,ŝ KL <ṡ (see Sec. 5 for a general example). For this reason, the random walk model studied in this section is mainly useful to compare entropy production estimates based on FPRs with those based on TURs.

Effect of finite thresholds
So far, we have consideredŝ FPR ( − , + ) in the limit of large thresholds + , − 1. We determine now the effect of finite thresholds onŝ FPR ( − , + )/ṡ. For finite thresholds, we solve numerically a recursion formula to determineŝ FPR ( − , + ), as explained in Appendix B. Although there is no conceptually difficulty in considering asymmetric thresholds, to reduce the number of parameters, we consider symmetric thresholds − = + = . Figure 4 plots the ratioŝ FPR ( − , + )/ṡ for parameters that are the same as in Fig. 2, but now for finite values of . It is evident from Fig. 4 that even at relatively small values of the thresholds,ŝ FPR ( − , + ) estimates dissipation more accurately than s TUR . For example, for = 0.1 the estimatorŝ FPR ( − , + ) captures for ∆ ≈ 0 about 75% ofṡ, which should be compared with the 25% captured by the TUR.
In Fig. 5, we analyseŝ FPR ( − , + ) as a function of for a fixed value of ∆ = 0.6; note that this is a generic example for which J is not proportional to S, and henceŝ FPR does not capture all the dissipation in the process, not even in the limit of large thresholds. The top panel of Fig. 5 plotsŝ FPR ( − , + )/ṡ as a function of for different values of ν, which is the parameter that determines the hopping rates according to Eqs. (48) and (49). At large values ν 1 the rate of dissipation is large, and at small values ν ≈ 0 the system is almost at equilibrium. In Fig. 5, we observe that asymptotically for large ,ŝ FPR ( − , + )/ṡ < 1, in accordance with the universal first-passage bound (26). However, how fastŝ FPR ( − , + )/ṡ converges to its asymptotic limit depends on Parameters are the same as in Fig. 2, except that the thresholds are finite, as given in the legend. The dashed, blue, horizontal line is a guide to the eye that indicates the value 1 corresponding to optimal inference. ν: near equilibrium, e.g., for ν = 0.2, large values of ≈ 15 are required to estimateṡ accurately, while far from thermal equilibrium, e.g., for ν > 2.5, the ratioŝ FPR captures about 90% of the dissipation for any value of . Hence, for systems far from thermal equilibrium,ŝ FPR estimates dissipation well, even at very small values of the thresholds. In order to better understand what value of is required to estimate dissipation, we plot in the bottom panel of Fig. 5 the ratioŝ FPR ( − , + )/ṡ as a function of | ln p − |. This figure shows thatŝ FPR ( − , + ) estimates well the entropy production rate whenever | ln p − | > 3, and hence the relevant asymptotic parameter is p − , rather than the threshold value . Accordingly, for systems near equilibrium, a large threshold is needed to bring p − < 0.05 = e −3 , while for systems far from thermal equilibrium p − < 0.05 = e −3 is reached for any value of the thresholds, as described by Eq. (33).

Effect of a finite number of realisations
So far, we have analysedŝ FPR ( − , + ), which is the value of the estimator F FPR when an infinite number of realisations are available. However, in experimental situations, we have to deal with a finite amount of data, and hence it is crucial to understand how fast F FPR converges toŝ FPR ( − , + ) as a function of n s , the number of realisations of the process X. We address this question here by simulating the process X with a continuous-time Monte Carlo algorithm.. Figure 6 shows F FPR /ṡ as a function of n s for the same parameters as in | ln p − | s FPṘ s Figure 5. Effect of finite thresholds on the estimatorŝ FP (part II). The ratioŝ FPR /ṡ is plotted as a function of − (top) and | ln p − | (bottom) for the current J with ∆ = 0.6 (defined in Eq. (53)) in the random walk process on a two-dimensional lattice. The rates k + 1 , k − 1 , k + 2 , and k − 2 are parametrised according to Eqs. (48) and (49), with ρ = 2 and ν = 0.2, 1, 2.5, 5, as given in the legend; the corresponding values ofṡ/k total are 0.05, 1, 3.5, and 7.5, respectively. The thresholds are symmetric, − = + . The blue, horizontal, dashed line is a guide to the eye that indicates the value 1.
We observe two regimes in Fig. 6: (a) when n s < 1/p − , then F FPR grows logarithmically in n s ; (b) when n s > 1/p − , then F FPR saturates to its asymptotic valueŝ FPR . To make it easier to differentiate these two regimes, Fig. 6 shows the value n s = 1/p − with the vertical dotted lines. Hence, we conclude that, in correspondence with Eq. (30), a number n s ∼ 1/p − of samples are required to optimally estimate the rate of dissipation. The saturation of F FPR at n s ∼ 1/p − has the following intuitive explanation: to determine the magnitude of p − , it is necessary to observe the frequency of events that hit the negative threshold first. In the regime n s /p − the experimenter does not observe events for which the current hits the negative threshold first, while for n s /p − the experimenter observes several such events.
Comparing estimates of dissipation based on the FPR with those based on the TUR, we observe that for small values of ν (orṡ) both estimators are equivalent: their asymptotic values for n s 1 are almost identical, in correspondence with Fig. 3, and the estimators converge equally fast to the asymptotic limit, viz., about n s ∼ 10 2 realisations are required. On other hand, for large values of ν (orṡ), the FPR captures asymptotically a finite fraction of the entropy production rate, while the TUR captures a negligible fraction of the entropy production rate. However, the small bias of the FPR comes at the expense of a large sample size n s that is required to reach the asymptotic limit (about n s ∼ 10 3 realisations are required for the FPR compared to n s ∼ 30 for the TUR).
Comparing estimations of dissipation based on F FPR with those based on F KL , we observe that the bias in F KL is smaller than the bias in F FPR . This is because in the present model the current is Markovian, and consequently the estimator F KL is unbiased. However, the dependence on n s is similar for both estimators, viz., for large values of ν the bias grows logarithmically until it reaches its asymptotic value.
Taken together, Fig. 6 confirms the Eq. (30), and hence although far from thermal equilibrium the FPR estimates a finite fraction of the total dissipation, this comes at the expense of a possibly large number of samples n s 1/p − , depending on how small the splitting probability p − is.

Inference of the efficiency of molecular motors
As an application, we use first-passage processes to estimate the efficiency at which a molecular motor converts chemical fuel into mechanical work. We assume that the position of the molecular motor, which we denote by Y , is observed. Instead, the internal degrees of freedom, such as the chemical states of the motor heads, are not known, and hence this is a problem with partial information. Moreover, in this example the observed current Y is nonMarkovian, and hence with this model we can quantify the influence of nonMarkovian statistics on entropy production estimates. In particular, it will be interesting to compare the biases of F FPR and F KL .

General setup: dissipation and efficiency of molecular motors
Consider a molecular motor that moves on a one-dimensional biofilament at mean velocity v in steps of length δ. We denote by f mech the mechanical load felt by the molecular motor (f mech > 0 denotes a hindering load, while f mech < 0 is an assisting load). When f mech v > 0, then the molecular motor consumes chemical fuel in the hydrolysis of adenosine triphosphate (ATP) into adenosine diphopshate (ADP) and an inorganic phosphate (P) to perform work on an external cargo. Following [18], we define the efficiency of the molecular motor as the ratio between the output power f mech v and the rateẇ chem of consumption of free energy input due to the hydrolysis of ATP into ADP and P. The rate of dissipation is the sum of the rate of entropy change in the chemostats, given byẇ chem /T env , and in the thermal reservoir, given by −f mech v/T env , viz., Combining the Eqs. (65) and (66), we obtain the relatioṅ between the rate of dissipationṡ and the motor efficiency.

A back-of-the-envelope estimate of the efficiency of molecular motors with first-passage processes
Combining the FPR-inequality Eq. (29) with the expression (67), we obtain the upper bound on the efficiency of a molecular motor. The bound Eq. (68) is the FPR equivalent of the bound [18], that follows from the TUR inequality, given by Eq. (38). Analogously, there is the bound based on the Kullback-Leibler divergence. We estimate η FPR based on the experimental data for Kinesin-1 published in Ref. [42]. For forces f mech = 3 pN and [ATP] = 10 µM, a number of 2-8% of backward steps were observed [42]. Hence, if we set the thresholds − = + = δ = 8 nm, then p − ∼ 0.02, and we are in the asymptotic regime of p − 0.05 that was identified in Fig. 5(b), albeit for a different model. Setting T Y ≈ δ/v, with δ = 8nm the step size of the molecular motor, and considering room temperatures T env = 298 × 1.38 × 10 −23 J, we obtain η ≤ η FPR = 1 1 + 298×1.38×0.01 24 × |ln 0.02| ≈ 0.6.
As this simple back-of-the-envelope calculation illustrates, we can estimate dissipation in nonequilibrium processes withŝ FPR . In the following sections, we use a more detailed analysis that relies on a model for Kinesin-1 dynamics to determine the accuracy of this estimate.

Model for molecular motors with two motor heads
In the previous section, we have used the FPR to estimate the rate of dissipation of a Kinesin-1 motor. In this section, we consider a molecular motor model for Kinesin-1 to determine the accuracy of this estimate. We describe the dynamics of Kinesin-1 with a model for two-headed molecular motors. This model is a Markov jump process that determines the transitions between the chemical states of the motor heads, and the model includes also one mechanical step. Each motor head can take three different chemical states, namely, the ADP-bound (D), ATP-bound (T), and nucleotide-free APO states (φ), leading to nine possible states. However, since the kinetics of the two heads are coordinated, see Refs. [22,6], a six state model with the states suffices to describe the dynamics of two-headed molecular motors. For convenience, we also number the states from 1 to 6, as shown in Fig. 1(b). In Fig. 1(b) we show all the possible chemical/mechanical transitions in the sixstate model for two-headed molecular motors. As argued in Ref. [22], the mechanical step, corresponding with the interchange of the front and rear head, takes place during the transition from (D : T) to (T : D), and therefore the position of the motor equals the current where for convenience we have set Y (0) = 0. The model contains three cycles: a forward cycle at rate j f corresponding to an ATP-hydrolysis-induced forward step, a backward cycle at rate j b corresponding to an ATP-hydrolysis-induced backward step, and a third cycle with rate j 0 that hydrolysis two ATP molecules but does not change the position of the motor. The total rate of dissipation isṡ are the affinities of the forward and backward cycles, and is the affinity of the third cycle. Here is the supplied chemical energy due to the hydrolysis of ATP into ADP and P, and f mech δ is the mechanical work done by the system on an external cargo during one molecular motor step of length δ. In Eq. (78), [ATP], [ADP], and [P], are the concentrations in the surrounding chemostat of ATP, ADP, and P, respectively, and K eq is the equilibrium constant of the hydrolysis reaction. Following [22,6], we use for the rates of the chemomechanical transitions between states 2 and 5 the formulae and For the other rates we set and and the explicit dependence on the concentrations [ADP] and [P] is not considered. Due to the equivalence of the two cycles, we also set k 3→2 (0) = k 6→5 (0), k 2→3 (0) = k 5→6 (0), k 3→4 (0) = k 6→1 (0), k 4→3 (0) = k 1→6 (0), k 4→5 (0) = k 1→2 (0), χ 23 = χ 56 , χ 34 = χ 61 , and χ 45 = χ 12 . Moerover, we set such that the six-state model satisfies detailed balance when For these values of the control parameters, there is neither a chemical driving force (∆µ = 0) nor a mechanical force (f mech = 0). The remaining parameters k i→j (0), θ, χ ij , k bi 1→2 , and k bi 4→5 are motor specific, and for Kinesin-1 they have been obtained in Refs. [6,7] from fitting single molecule motility data to the six-state model. Here, we will use these fitted parameters for Kinesin-1, and for convenience we summarise the exact numerical values in Appendix C.
The concentration [ATP] and the mechanical load f mech δ/T env are external, control parameters that can be modified by the experimenter.

Estimating the efficiency of a molecular motor based on the measurement of its position
We estimate the efficiency η from the measurement of the position Y of the molecular motor, which according to Eq. (67) is equivalent to an entropy production estimation problem. We consider the case of [ATP] = 10µM and f mech = 3pN, corresponding with the calculation in Sec. 5.2 based on experimental data.
We define two estimatorsη FPR , andη TUR for the efficiency of η that are equivalent to the estimators F FPR and F TUR for the rate of dissipation, i.e., they are estimates for the efficiency when a finite number of trajectories are available. For estimation based on first-passage processes, we consider wherep − andT Y are the estimates of p − and T Y based on a finite number of realisations of the process, as in Eqs. (24) and (25); note that we assume that v = δ Y (t) /t is known, although we could also take an estimated value for this quantity leading to almost identical results. To estimate the motor efficiency based on the TUR, we usê Figure 7 plots η FPR as a function of n s for three values of = − = + , namely, = 1, = 2, and = 3, and compares the estimate η FPR with the efficiency η = 0.478 and the estimate η TUR based on the TUR.
Just as for the F FPR in Fig. 6, we can differentiate two regimes: (i) for n s < 1/p − , the estimator η FPR decreases logarithmically; and (ii) for n s > 1/p − , the estimator η FPR has reached its asymptotic value. The values n s = 1/p − that separate the two regimes are indicated for different values of by the vertical dotted lines. For = 1, η FPR converges to its asymptotic value 0.6, which is similar to the value we estimated in Sec. 5.2 based on experimental data. The simulation results in Fig. 7 show that this estimate can be improved by considering larger values of , but this comes at a significant expense in the number of realisations n s required. Nevertheless, the estimate η FPR at small thresholds = 1 is better than the estimate η TUR , indicating again that small thresholds are sufficient to obtain accurate estimates of dissipation with FPRs.
Interestingly, Fig. 7 shows that the estimates η FPR are less biased than the estimate η KL , given by Eq. (70), based on the Kullback-Leibler divergence. Note that in the present example Y is a nonMarkovian current. Since η KL does not account for nonMarkovian effects in the current, we can attribute the smaller bias in the FPR due to the nonMarkovian statistics of Y that are accounted for by the FPR.

Discussion
We have studied the problem of inferring the rate of entropy productionṡ from the measurement of repeated realisations of a first-passage process of a single current. To this aim, we have used the estimator F FPR [Eq. (23)] that converges in the limit of an infinite number of realisations of the process to the first-passage ratioŝ FPR [Eq. (3)]. We have compared this estimator with F TUR , an estimator based on the TUR, and with F KL , an estimator based on the Kullback-Leibler divergence. We discuss now the strengths and weakness of these different estimators, and we also provide an outlook on future challenges.
Comparing F FPR with F TUR , we conclude that far from thermal equilibrium the FPR is a better estimator of dissipation. Indeed, as shown in Fig. 3, far from thermal equilibrium the FPR captures a finite fraction ofṡ, while the TUR captures a negligible fractionṡ; note that in the literature modifications of the TUR have been derived that have a smaller bias thanŝ TUR [38,39], but these modified TURs rely on measurements of both currents and time-integrated empirical measures, and hence require more information about the process X than the trajectories of a single current. The estimation of dissipation withŝ FPR does require a number n s 1/p − of realisations of the nonequilibrium process, which can be large for processes far from thermal equilibrium; in particular, we have shown that the number of realisations n s increases exponentially as a function of ∆s, the minimal jump size of S. Nevertheless, since far from thermal equilibrium F TUR /ṡ ≈ 0, it holds that the bias of the FPR is smaller than the bias of the TUR, even for values of n s that are smaller than 1/p − .
Comparing F FPR with F KL , we conclude that for generic Markov jump processes the bias in F FPR is smaller than the bias in F KL , as the FPR captures nonMarkovian effects in the current (see Fig. 7). An exception is when the current J is Markovian, as was the case for the random walk process discussed in Sec. 4. Note that it is possible to include memory effects in the Kullback-Leibler divergence, e.g., by assuming the current is a semi-Markov process [14], but in general this still leads to a biased estimator of dissipation Taken together, the three estimators F FPR , F TUR , and F KL are valid estimators of dissipation, as they can be used to estimate dissipation in different circumstances. Near equilibrium and with limited data available, the TUR is a good estimator of dissipation; for Markovian currents the Kullback-Leibler divergence is the best estimator; and for processes far from equilibrium with nonMarkovian currents the FPR is the estimator with the least bias.
In the present paper, we have considered entropy production estimation based on the measurement of a single current. Since all estimators are bounded from above by the rateṡ of entropy production, we can reduce the bias in the estimators by measuring multiple currents, and maximising the estimator towards linear combinations of these currents. In this regard, the FPR has the appealing property that the maximum of this variational problem is attained by a current that is proportional to the entropy production, and hence the affinities of the process can be inferred from a variational principle.
The purpose of the present paper was to highlight the potential of a first-passage approach to the entropy-production estimation problem. To this aim, we have performed a comparative study using two simple examples of nonequilibrium processes. However, to better understand estimation theory for entropy production rates, we think it will be important to take the present study one step further and make an extensive comparison of the different methods for the estimation of entropy production used in the literature, e.g., by using a larger pool of estimators for entropy production and a larger pool of models representing nonequilibrium systems. In this regard, one should be careful to make a fair comparison of the different methods based on the knowledge they require about the nonequilibrium process and the amount of data used. Also, we hope the present manuscript highlights the importance of developing further the mathematics underlying the estimatorŝ FPR through extensions of the results in Ref. [2].
We end the paper with the discussion of an interesting, potential application in cell biology. Proteins in living cells produce entropy, as they are involved in chemical reactions that operate far from chemical equilibrium. However, due to the diffraction limit, it is challenging to measure the trajectories of individual proteins in living cells at high enough spatial and temporal resolution. Nevertheless, in recent years significant advances have been made in super-resolution live cell imaging, see e.g. Refs. [43,45,46]. If the resolution of these imaging methods can be further improved, then it will certainly be possible to infer entropy production rates of individual proteins in living cells from the measurements of their trajectories. used in Ref. [6]. The reason we prefer to use Eq. (84) is because then the model satisfies detailed balance when the control parameters are given by Eq. (85). Moreover, since this will make no discernible difference in the model fits.
At equilibrium, i.e. for the control parameters given by Eq. (85), the stationary state is where the normalisation constant is