Informations in Models of Evolutionary Dynamics

Abstract

Biological organisms adapt to changes by processing informations from different sources, most notably from their ancestors and from their environment. We review an approach to quantify these informations by analyzing mathematical models of evolutionary dynamics and show how explicit results are obtained for a solvable subclass of these models. In several limits, the results coincide with those obtained in studies of information processing for communication, gambling or thermodynamics. In the most general case, however, information processing by biological populations shows unique features that motivate the analysis of specific models.

Appendices

Appendix 1: Mapping from Eqs. (6) to (2)

The model described by Eq. (6) is mapped to the model described by Eq. (2) by defining

$$\begin{aligned} \tilde{\phi }_t=(\gamma _t,\phi _t),\quad \tilde{x}_t=(0,x_t),\quad \tilde{y}_t=(y_t,z_t) \end{aligned}$$

(59)

and

$$\begin{aligned} \pi (\tilde{\phi }_t|\tilde{\phi }_{t-1},\tilde{y}_t)=H\big (\tilde{\phi }^1_t|\tilde{\phi }^1_{t-1},\tilde{\phi }^2_t,\tilde{y}^2_t\big )D\big (\tilde{\phi }^2_t|\tilde{\phi }^1_{t-1},\tilde{y}^1_t\big ),\qquad \tilde{S}(\tilde{\phi }_t,\tilde{x}_t)=S(\tilde{\phi }^2_t,\tilde{x}^2_t), \end{aligned}$$

(60)

where $\tilde{\phi }^k_t$ corresponds to the k-th component of $\tilde{\phi }_t$, i.e., $\tilde{\phi }_t=(\tilde{\phi }_t^1,\tilde{\phi }_t^2)$. Note that $\tilde{S}$ is of the form $\tilde{S}=\tilde{K}\tilde{\Delta }$ as in Eq. (1) if S is itself of the form $S=K\Delta$.

Appendix 2: Decomposition of the Growth Rate

We detail here the decomposition of the growth rate given in Eq. (14) for the discrete model in absence of inheritance. The idea is to write

$$\begin{aligned} \Lambda= & {} \mathbb {E}_{X,Y}[\ln (K(X)\tilde{\pi }(X|Y))] \nonumber \\= & {} \mathbb {E}_{X,Y}[\ln (K(X)]+\mathbb {E}_{X,Y}[\ln P_X(x)]+\mathbb {E}_{X,Y}\left[ \ln \frac{P_{X|Y}(X|Y)}{P_X(X)}\right] +\mathbb {E}_{X,Y}\left[ \ln \frac{\tilde{\pi }(X|Y)}{P_{X|Y}(X|Y)}\right] ,\nonumber \\ \end{aligned}$$

(61)

and to recognize that $\mathbb {E}_{X,Y}[\ln P_X(x)]=-H(X)$, $\mathbb {E}_{X,Y}[\ln P_{X|Y}(X|Y)/P_X(X)]=I(X;Y)$ and

$$\begin{aligned} \mathbb {E}_{X,Y}\left[ \ln \frac{\tilde{\pi }(X|Y)}{P_{X|Y}(X|Y)}\right]= & {} \sum _{x,y}P_{X,Y}(x,y)\ln \frac{\tilde{\pi }(x|y)}{P_{X|Y}(x|y)}=\sum _y P_Y(y)\sum _{x}P_{X|Y}(x|y)\ln \frac{\tilde{\pi }(x|y)}{P_{X|Y}(x|y)} \nonumber \\= & {} -\sum _y P_Y(y)D(P_{X|Y}(.|y)\Vert \tilde{\pi }(.|y))=-\mathbb {E}_Y[D(P_{X|Y}(.|Y)\Vert \tilde{\pi }(.|Y))].\nonumber \\ \end{aligned}$$

(62)

Appendix 3: Gaussian Random Variables

A Gaussian random variable X is characterized by its mean $x_0=\mathbb {E}[X]$ and its variance $\sigma ^2_x=\mathbb {E}[X^2]-\mathbb {E}[X]^2$, and its probability density is $P_X(x)=G_{\sigma ^2_x}(x-x_0)$, where $G_{\sigma ^2_x}(x)=(2\pi \sigma ^2_x)^{-1/2}\exp (-x^2/2\sigma ^2_x)$.

Its differential entropy $h(X)=-\int \mathrm {d}xP_X(x)\ln P_X(x)$ is

$$\begin{aligned} h(X)=\frac{1}{2}\ln (2\pi e\sigma ^2_x). \end{aligned}$$

(63)

The mutual information $I(X;Y)=h(X)-h(X|Y)$ between X and another Gaussian random variable Y whose conditional probability given x is $P_{Y|X}(y|x)=G_{\sigma ^2_{y|x}}(y-x)$ is

$$\begin{aligned} I(X;Y)=\frac{1}{2}\ln \left( 1+\frac{\sigma ^2_x}{\sigma ^2_{y|x}}\right) . \end{aligned}$$

(64)

The relative entropy between two Gaussian probability densities is

$$\begin{aligned} D\big (G_{\sigma ^2_0}(.-x_0)\Vert G_{\sigma ^2_1}(.-x_1)\big )=\frac{1}{2}\left( \frac{\sigma ^2_0+(x_1-x_0)^2}{\sigma ^2_1}-\ln \frac{\sigma ^2_0}{\sigma ^2_1}-1\right) . \end{aligned}$$

(65)

Finally, given $P_{X_1|X_0}(x_1|x_0)=G_{\sigma ^2_{x_1|x_0}}(x_1-ax_0)$ and $P_{Y_1|X_1}(y_1|x_1)=G_{\sigma ^2_{y_1|x_1}}(y_1-x_1)$, the conditional probability $P_{X_1|Y_1,X_0}$, which by Bayes’ rule is proportional to $P_{Y_1|X_1}P_{X_1|X_0}$, is also Gaussian and given by

$$\begin{aligned} P_{X_1|Y_1,X_0}(x_1|y_1,x_0)=G_{\sigma ^2_{x_1|y_1,x_0}}(x_1-\lambda x_0-\kappa y_1), \end{aligned}$$

(66)

with

$$\begin{aligned} \kappa =\frac{1}{1+\sigma ^2_{y_1|x_1}/\sigma ^2_{x_1|x_0}},\qquad \lambda =a(1-\kappa ),\qquad \sigma ^2_{x_1|y_1,x_0}=\kappa \sigma ^2_{x_1|x_0}, \end{aligned}$$

(67)

or, equivalently, $\sigma ^{-2}_{x_1|y_1,x_0}=\sigma _{x_1|x_0}^{-2}+\sigma _{y_1|x_1}^{-2}$.

Appendix 4: Growth Rate of the Gaussian Model

Equation (30) for the growth rate $\Lambda$ of the Gaussian model is obtained by considering

$$\begin{aligned} n_t(\phi _t)=\frac{1}{W_t}K(x_t)\int \mathrm {d}\phi _{t-1}\ G_{\sigma ^2_s}(\phi _t-x_t)G_{\sigma ^2_\pi }(\phi _t-\lambda \phi _{t-1}-\kappa y_t)n_t(\phi _{t-1}), \end{aligned}$$

(68)

with $n_t(\phi _t)$ of the form $n_t(\phi _t)=G_{\sigma ^2_t}(\phi _t-m_t)$, which leads to

$$\begin{aligned}&\displaystyle m_t=\frac{\sigma ^2_s}{\sigma ^2_s+\sigma ^2_\pi +\lambda ^2\sigma ^2_{t-1}}(\lambda m_{t-1}+\kappa y_t)+\frac{\sigma ^2_\pi +\lambda ^2\sigma ^2_{t-1}}{\sigma ^2_s+\sigma ^2_\pi +\lambda ^2\sigma ^2_{t-1}}x_t\end{aligned}$$

(69)

$$\begin{aligned}&\displaystyle \sigma ^2_t=(\sigma _s^{-2}+(\sigma ^2_\pi +\lambda ^2\sigma ^2_{t-1})^{-1})^{-1}\end{aligned}$$

(70)

$$\begin{aligned}&\displaystyle W_t=K(x_t)G_{\sigma ^2_s+\sigma ^2_\pi +\lambda ^2\sigma ^2_{t-1}}(\lambda m_{t-1}-x_t+\kappa y_t). \end{aligned}$$

(71)

The variance $\sigma ^2_t$ has a fixed point $\sigma ^2_\infty$ in terms of which the growth rate can be rewritten as

$$\begin{aligned} \Lambda =\lim _{t\rightarrow \infty }\mathbb {E}[\ln W_t]=\Lambda ^*-\frac{1}{2}\ln (2\pi \sigma ^2_s)+\frac{1}{2}\ln \frac{\alpha }{\lambda }-\frac{\alpha }{2\lambda \sigma ^2_s}\lim _{t\rightarrow \infty }\mathbb {E}[z^2_t], \end{aligned}$$

(72)

where $\Lambda ^*=\mathbb {E}_X[\ln K(X)]$,

$$\begin{aligned} z_t=\lambda m_{t-1}-x_t+\kappa y_t, \end{aligned}$$

(73)

and

$$\begin{aligned} \alpha =\frac{\lambda \sigma ^2_s}{\sigma ^2_s+\sigma ^2_\pi +\lambda ^2\sigma ^2_\infty }=\frac{2\lambda }{1+\lambda ^2+\beta +((1-\lambda ^2-\beta )^2+4\beta )^{1/2}},\quad \mathrm{with}\quad \beta =\frac{\sigma ^2_\pi }{\sigma ^2_s}.\nonumber \\ \end{aligned}$$

(74)

Given that $x_{t+1}=ax_t+b_t$ and $y_{t+1}=x_{t+1}+b'_{t+1}$ with $b_t\sim \mathcal {N}(0,\sigma ^2_{x_1|x_0})$ and $b'_{t+1}\sim \mathcal {N}(0,\sigma ^2_{y_1|x_1})$, we have

$$\begin{aligned} z_{t+1}=\alpha z_t+\epsilon x_t+(\kappa -1)b_t+\kappa b'_{t+1},\quad \mathrm{with}\quad \epsilon = \lambda -a(1-\kappa ). \end{aligned}$$

(75)

Using $\sum _{k=0}^t\alpha ^{t-k}x_k=\sum _{k=0}^t(\alpha ^{t-k}-a^{t-k})/(\alpha -a)b_k$, we obtain

$$\begin{aligned} z_{t+1}=\frac{1}{\alpha -a}\sum _{k=0}^t(\delta \alpha ^{t-k}-\epsilon a^{t-k})b_k+\kappa \sum _{k=0}^t\alpha ^{t-k}b'_{k+1},\quad \mathrm{with}\quad \delta = \lambda -\alpha (1-\kappa ),\nonumber \\ \end{aligned}$$

(76)

and, since the $b_k$ and $b'_k$ are all independent, with variances $\mathbb {E}[b_k^2]=\sigma ^2_{x_1|x_0}$ and $\mathbb {E}[b_k'^2]=\sigma ^2_{y_1|x_1}$,

$$\begin{aligned} \lim _{t\rightarrow \infty }\mathbb {E}[z^2_{t+1}]= & {} \frac{1}{(\alpha -a)^2}\left( \frac{\delta ^2}{1-\alpha ^2}-\frac{2\delta \epsilon }{1-a\alpha }+\frac{\epsilon ^2}{1-a^2}\right) \sigma ^2_{x_1|x_0}+\kappa ^2\frac{\sigma ^2_{y_1|x_1}}{1-\alpha ^2}\end{aligned}$$

(77)

$$\begin{aligned}= & {} \frac{(\lambda ^2+(1-\kappa )^2)(1+a\alpha )-2\lambda (1-\kappa )(a+\alpha )}{(1-\alpha ^2)(1-a\alpha )(1-a^2)}\sigma ^2_{x_1|x_0}+\kappa ^2\frac{\sigma ^2_{y_1|x_1}}{1-\alpha ^2}.\qquad \end{aligned}$$

(78)

Plugged into Eq. (72), it leads to Eq. (30).

Appendix 5: Decomposition of the Growth Rate of the Gaussian Model

Since the Gaussian model can be obtained as a continuous limit of the discrete model, Eqs. (32)–(37) directly result from Eqs. (14)–(22) by taking the same limit. The decomposition can also be derived directly from the general formula of Eq. (30) as we illustrate it here in the simplest case where the two limits are taken.

The first limit, of perfect selectivity, corresponds to $\sigma ^2_s\rightarrow 0$, such that Eq. (30) becomes

$$\begin{aligned} \Lambda =\Lambda ^*-\frac{1}{2}\ln (2\pi \sigma ^2_\pi )-\frac{1}{2\sigma ^2_\pi }\left[ \frac{\lambda ^2+(1-\kappa )^2-2\lambda (1-\kappa )a}{1-a^2}\sigma ^2_{x_1|x_0}+\kappa ^2\sigma ^2_{y_1|x_1}\right] .\qquad \end{aligned}$$

(79)

The second limit, of no inheritance, simply corresponds to setting $\lambda = 0$ in this equation, so that

$$\begin{aligned} \Lambda =\Lambda ^*-\frac{1}{2}\ln (2\pi \sigma ^2_\pi )-\frac{1}{2\sigma ^2_\pi }\left[ (1-\kappa )^2\sigma ^2_{x_1}+\kappa ^2\sigma ^2_{y_1|x_1}\right] , \end{aligned}$$

(80)

where $\sigma ^2_{x_1}=\sigma ^2_{x_1|x_0}/(1-a^2)$ represents the stationary variance of the environmental process, $\sigma ^2_{x_1}=\mathbb {E}[X_1^2]$. The optimal strategy $\hat{\pi }$ is obtained by optimizing $\Lambda$ over $\kappa$ and $\sigma ^2_\pi$, which leads to

$$\begin{aligned} \hat{\kappa }=\frac{1}{1+\sigma ^2_{y_1|x_1}/\sigma ^2_{x_1}},\qquad \hat{\sigma }_\pi ^2=\hat{\kappa }\sigma ^2_{y_1|x_1}. \end{aligned}$$

(81)

$\hat{\kappa }$ can also be written $\hat{\kappa }=\sigma ^2_{x_1}/\sigma ^2_{y_1}$ where $\sigma ^2_{y_1}=\sigma ^2_{x_1}+\sigma ^2_{y_1|x_1}$ represents the stationary variance of $y_t$. As expected from the analysis of the discrete model, we verify that the optimal strategy implements a Bayesian estimation, i.e., $\hat{\pi }=P_{X|Y}$ [see Appendix 3]. We also verify that the optimal optimal growth rate,

$$\begin{aligned} \hat{\Lambda }=\Lambda ^*-\frac{1}{2}\ln \left( 2\pi \frac{\sigma ^2_{y_1|x_1}\sigma ^2_{x_1}}{\sigma ^2_{x_1}+\sigma ^2_{y_1|x_1}}\right) -\frac{1}{2}, \end{aligned}$$

(82)

is equivalently written

$$\begin{aligned} \hat{\Lambda }=\Lambda ^*-h(X)+I(X;Y), \end{aligned}$$

(83)

where

$$\begin{aligned} h(X)=\frac{1}{2}\ln (2\pi e\sigma ^2_{x_1}),\qquad \mathrm{and}\qquad I(X;Y)=\frac{1}{2}\ln \left( 1+\frac{\sigma ^2_{x_1}}{\sigma ^2_{y_1|x_1}}\right) . \end{aligned}$$

(84)

More generally, by introducing $\sigma _{x_1|y_1}^{-2}=\sigma _{y_1|x_1}^{-2}+\sigma _{x_1}^{-2}$, so that $(1-\kappa )^2\sigma ^2_{x_1}+\kappa ^2\sigma ^2_{y_1|x_1}=\sigma ^2_{x_1|y_1}+(\hat{\kappa }-\kappa )^2\sigma ^2_{y_1}$, we verify that Eq. (80) is equivalent to

$$\begin{aligned} \Lambda =\hat{\Lambda }-\frac{1}{2}\left[ \frac{\sigma ^2_{x_1|y_1}+(\hat{\kappa }-\kappa )^2\sigma ^2_{y_1}}{\sigma ^2_\pi }-\ln \frac{\sigma ^2_{x_1|y_1}}{\sigma ^2_\pi }-1\right] =\hat{\Lambda }-\mathbb {E}_Y[D(P_{X|Y}(.|Y)\Vert \pi (.|Y))],\nonumber \\ \end{aligned}$$

(85)

as expected from Eq. (13).

Appendix 6: Gaussian Model with Individual Sensors

Using the formulae of Appendix 3, the term to maximize in Eq. (42) can be written

$$\begin{aligned}&I(X;Y)-\mathbb {E}_Y\big [D(P_{X|Y}\Vert G_{\sigma ^2_\pi +\kappa ^2 \sigma ^2_{\psi |y}}(.-\kappa Y))\big ]=\frac{1}{2}\ln \left( 1+\frac{\sigma ^2_x}{\sigma ^2_{y|x}}\right) \nonumber \\&\quad -\frac{1}{2}\left( \frac{\sigma ^2_{x|y}+(\kappa -\kappa _0)^2\sigma ^2_y}{\sigma ^2_\pi +\kappa ^2\sigma ^2_{\psi |y}}-\ln \frac{\sigma ^2_{x|y}}{\sigma ^2_\pi +\kappa ^2\sigma ^2_{\psi |y}}-1\right) . \end{aligned}$$

(86)

When $Y\rightarrow X$, $\kappa _0\rightarrow 1$, $\sigma ^2_y\rightarrow 0$, $\sigma ^2_{y|x}\rightarrow 0$ and $\sigma ^2_{x|y}\rightarrow 0$ but $\sigma ^2_{x|y}/\sigma ^2_{y|x}\rightarrow 1$ and it simplifies to

$$\begin{aligned} \frac{1}{2}\left[ \ln \left( \frac{\sigma ^2_x}{\sigma ^2_\pi +\kappa ^2\sigma ^2_{\psi |x}}\right) -\frac{(\kappa -1)^2\sigma ^2_x}{\sigma ^2_\pi +\kappa ^2\sigma ^2_{\psi |x}}+1\right] . \end{aligned}$$

(87)

The maximum over $\sigma ^2_\pi$ is reached for $\sigma ^2_\pi =0$ and taking the derivative with respect to $\kappa$ leads to

$$\begin{aligned} \kappa ^2\sigma ^2_{\psi |x}+(\kappa -1)\sigma ^2_x=0 \end{aligned}$$

(88)

whose solution is given in Eq. (43).

Appendix 7: The Gaussian Model as a Limit of the General Model of Ref. [17]

A general model is defined by Eqs. [S1]–[S2] in the Supporting Information of [17], which we repeat here with only slightly modified notations:

$$\begin{aligned} \gamma _0'= & {} \lambda _0\gamma _0+\kappa _0z_t+\omega _0\phi _0+\nu _H,\quad \nu _H\sim \mathcal {N}(0,\sigma ^2_H),\end{aligned}$$

(89)

$$\begin{aligned} \phi _0= & {} \theta _0\gamma _0+\rho _0y_t+\nu _D,\quad \nu _D\sim \mathcal {N}(0,\sigma ^2_D),\end{aligned}$$

(90)

$$\begin{aligned} S(\phi _0,x_t)= & {} \exp [r_\mathrm{max}-(\phi _0-x_t)^2/(2\sigma ^2_s)],\end{aligned}$$

(91)

$$\begin{aligned} x_t= & {} ax_{t-1}+b_t,\quad b_t\sim \mathcal {N}(0,\sigma ^2_{x_1|x_0}),\end{aligned}$$

(92)

$$\begin{aligned} y_t= & {} x_t+b'_t,\quad b'_t\sim \mathcal {N}(0,\sigma ^2_{y|x}),\end{aligned}$$

(93)

$$\begin{aligned} z_t= & {} x_t+b''_t,\quad b''_t\sim \mathcal {N}(0,\sigma ^2_{z|x}). \end{aligned}$$

(94)

Without loss of generality it can be assumed that $\sigma ^2_s=1$. The formula for the growth rate of this general model is given with an error in Eq. [S3] of [17]. The correct formula is

$$\begin{aligned}&\Lambda =r_\mathrm{max}+\frac{1}{2}\ln \frac{\alpha }{\eta }-\frac{\alpha }{2\eta (1-\alpha ^2)}\left[ \frac{(\upsilon ^2+(1-\rho _0)^2)(1+a\alpha )-2\upsilon (1-\rho _0)(a+\alpha )}{1-a\alpha }\right. \nonumber \\&\left. \qquad \quad \times \, \sigma ^2_x+\rho _0^2 (1-2\alpha \lambda _0+\lambda _0^2)\sigma ^2_{y|x}+\kappa _0^2\theta _0^2\sigma ^2_{z|x}\right] , \end{aligned}$$

(95)

where $\sigma ^2_x\equiv \sigma ^2_{x_1|x_0}/(1-a^2)$, where $\eta$ and $\upsilon$ given by

$$\begin{aligned} \eta =\lambda _0(1+\sigma ^2_D)+\omega _0\theta _0,\qquad \upsilon = (\omega _0+\kappa _0)\theta _0+(1-\rho _0)\lambda _0, \end{aligned}$$

(96)

and

$$\begin{aligned} \alpha =\frac{2\tilde{\lambda }}{1+\tilde{\lambda }^2+\tilde{\sigma }_H^2+\left( (1-\tilde{\lambda }^2-\tilde{\sigma }_H^2)^2+4\tilde{\sigma }_H^2\right) ^{1/2}}, \end{aligned}$$

(97)

with

$$\begin{aligned} \tilde{\sigma }_H^2=\left( \sigma ^2_H+\frac{\omega _0^2\sigma ^2_D}{\sigma ^2_D+1}\right) \frac{\theta _0^{2}}{\sigma ^2_D+1},\quad \quad \tilde{\lambda }=\lambda _0+\frac{\theta _0\omega _0}{\sigma ^2_D+1} \end{aligned}$$

(98)

These formulae reduce to Eq. (30) when taking $\theta _0=\lambda$, $\rho _0=\kappa$, $\omega _0=1$, $\sigma ^2_D=\sigma ^2_\pi$, $\lambda _0=\kappa _0=\sigma ^2_H=\sigma ^2_{z|x}=0$ and $r_\mathrm{max}=\ln K-(1/2)\ln (2\pi \sigma ^2_s)$.

Appendix 8: Feedback Control Out of Equilibrium

The state $x_t$ of a system in contact with a heat bath is measured as $y_t$ at regular intervals of time $\tau$, upon which the potential in which the system evolves is changed from $V_{t-1}(x)$ to $V_t(x)$. This change is done without knowing the current state $x_t$, but may depend on the history of past measurements $y^t=(y_1,\dots ,y_t)$ as well as on the history of past states at the time of these measurements, $x^{t-1}=(x_1,\dots ,x_{t-1})$. If we assume that the potential is controllable by one or several parameters $\ell$, we therefore consider, in the more general case, that $\ell _t=\ell (y^t,x^{t-1})$ [in more constrained cases, $\ell _t$ may depend only on some of variables, e.g., $\ell _t=\ell (y^t)$ when only the present and past measurements are available]. In-between two measurements, the system relaxes in a constant potential $V_t(x)$ but may not reach equilibrium; its dynamics is generally stochastic, due to the interaction with the heat bath, and may for instance be described by a Master equation with rates satisfying detailed balance. When changing the potential from $V_{t-1}(x)$ to $V_t(x)$, a demon extracts a work $\mathcal {W}_t=V_{t-1}(x_t)-V_{t}(x_t)$. The goal of the demon is either to optimize the total extracted work $\mathcal {W}_\mathrm{tot}=\mathbb {E}[\sum _t\mathcal {W}_t]$ or, if $\tau$ itself is controllable, to optimize the power $\mathcal {W}_\mathrm{tot}/\tau$.

To formalize this problem, we denote by $p^{\tau }_{t-1}(x_t)$ the probability of the system to be in state $x_t$ at the time of the t-th measurement: this probability depends explicitly only on $\ell _{t-1}$ and $x_{t-1}$, which characterize, respectively, the potential $V_{t-1}(x)$ and the state of the system when this potential is switched on. Introducing $F_t=-\beta ^{-1}\ln \sum _x e^{-\beta V_t(x)}$ and $p^{(\infty )}_t(x)=e^{\beta [F_t-V_t(x)]}$ (also denoted $\pi$ in the main text), the extracted work may be decomposed as

$$\begin{aligned} \mathcal {W}_t(x^t,y^t)=V_{t-1}(x_t)-V_t(x_t)=\beta ^{-1}\ln \frac{p^{\infty }_t(x_t)}{p^{\tau }_{t-1}(x_t)}+\beta ^{-1}\ln \frac{p^{\tau }_{t-1}(x_t)}{p^{\infty }_{t-1}(x_t)}-(F_t-F_{t-1}).\nonumber \\ \end{aligned}$$

(99)

We now consider the past history $(x^{t-1},y^{t-1})$ as given and average over $(X_t,Y_t)$ to define

$$\begin{aligned} \mathbb {E}_t[\mathcal {W}_t]= \mathbb {E}_{X_t,Y_t|X^{t-1}=x^{t-1},Y^{t-1}=y^{t-1}}[\mathcal {W}_t(X^t,Y^t)]. \end{aligned}$$

(100)

Since $P_{X_t|X^{t-1},Y^{t-1}}(x_t|x^{t-1},y^{t-1})=p^{\tau }_{t-1}(x_t)$, we have

$$\begin{aligned} \beta \mathbb {E}_t[\mathcal {W}_t]=I(X_t;Y_t|x^{t-1},y^{t-1}) - \mathbb {E}[D(q^{\tau }_{t-1}\Vert p^{\infty }_t)]+\mathbb {E}[D(p^{\tau }_{t-1}\Vert p^{\infty }_{t-1})]-\beta \mathbb {E}[F_t-F_{t-1}],\nonumber \\ \end{aligned}$$

(101)

where

$$\begin{aligned} q_{t-1}^\tau (x_t|y_t)=P_{X_t|X^{t-1},Y^t}(x_t|x^{t-1},y^t)=\frac{P_{Y|X}(y_t|x_t)p_{t-1}^\tau (x_t)}{\sum _x P_{Y|X}(y_t|x)p_{t-1}^\tau (x)} \end{aligned}$$

(102)

and

$$\begin{aligned} I(X_t;Y_t|x^{t-1},y^{t-1}) =\sum _{x_t,y_t}P_{Y|X}(y_t|x_t)p^{\tau }_{t-1}(x_t)\ln \frac{q_{t-1}^\tau (x_t|y_t)}{p_{t-1}^\tau (x_t)} \end{aligned}$$

(103)

The total work $\mathcal {W}_\mathrm{tot}$ is obtained as $\mathcal {W}_\mathrm{tot}=\sum _t\mathbb {E}_{X^{t-1},Y^{t-1}}[\mathbb {E}_t[\mathcal {W}_t]]$. When $\tau \rightarrow \infty$, the third term on the right-hand side of Eq. (101) vanishes and we recover the equilibrium result, Eq. (51).

This formalism can be applied to a Brownian particle in a controllable harmonic potential. For simplicity, we assume that only the location of the potential can be controlled, and its stiffness k is fixed to $k=1$. We also set $\beta =1$. The potential $V_t(x)=(x-\ell _t)^2/2$ is characterized by the location $\ell _t$ of its minimum, and $F_t=F_{t-1}$ for all t. We take the relaxation dynamics between measurements to be described by a Fokker-Planck equation,

$$\begin{aligned} \partial _\tau p_t^\tau (x)=\partial _x(\partial _xV_t(x)p_t^\tau (x))+\partial ^2_xp_t^\tau (x), \end{aligned}$$

(104)

with the initial condition $p_t^0(x)=\delta (x-x_t)$. This equation is easily solved as its solution is Gaussian at all time: $p^\tau _t(x)=G_{\varsigma ^2_\tau }(x-\mu _t^\tau )$ with

$$\begin{aligned}&\frac{1}{2}\partial _\tau \mu _t^\tau +\mu _t^\tau =\ell _t,\qquad&\mu ^0_t=x_t,\end{aligned}$$

(105)

$$\begin{aligned}&\frac{1}{2}\partial _\tau \varsigma ^2_\tau +\varsigma ^2_\tau =1,\qquad&\varsigma ^2_0=0, \end{aligned}$$

(106)

so that

$$\begin{aligned} \mu _t^\tau= & {} (1-e^{-\tau })\ell _t+e^{-\tau }x_t,\end{aligned}$$

(107)

$$\begin{aligned} \varsigma ^2_\tau= & {} 1-e^{-2\tau }. \end{aligned}$$

(108)

When $\tau \rightarrow \infty$, $p^\tau _t(x)$ converges to the equilibrium distribution $p_t^\infty (x)=G_{1}(x-\ell _t)$. Using $P_{Y|X}(y_t|x_t)=G_{\sigma ^2_{y|x}}(y_t-x_t)$ and applying Eq. (66), $q_t^\tau$ is found to be

$$\begin{aligned} q_{t-1}^\tau (x_t|y_t)=G_{\sigma ^2_{x|y}}(x_t-(1-\kappa )\mu ^\tau _{t-1}-\kappa y_t),\qquad \text {with}\quad \kappa =\frac{1}{1+\sigma ^2_{y|x}/\varsigma ^2_\tau },\qquad \sigma ^2_{x|y}=\kappa \sigma ^2_{y|x}.\nonumber \\ \end{aligned}$$

(109)

The first term in Eq. (101) is therefore

$$\begin{aligned} I(X_t;Y_t|x^{t-1},y^{t-1})=\frac{1}{2}\ln \left( 1+\frac{\varsigma ^2_\tau }{\sigma ^2_{y|x}}\right) . \end{aligned}$$

(110)

The second term is

$$\begin{aligned} \mathbb {E}[D(q^{\tau }_{t-1}\Vert p^{\infty }_t)]=\frac{1}{2}\left( \sigma ^2_{x|y}+\mathbb {E}[z_t^2]-\ln \sigma ^2_{x|y}-1\right) , \end{aligned}$$

(111)

with

$$\begin{aligned} z_t =(1-\kappa )\mu ^\tau _{t-1}+\kappa y_t-\ell _t. \end{aligned}$$

(112)

The third term is

$$\begin{aligned} \mathbb {E}[D(p^{\tau }_{t-1}\Vert p^{\infty }_{t-1})]=\frac{1}{2}\left( \varsigma ^2_\tau +\mathbb {E}[z_t'^2]-\ln \varsigma ^2_\tau -1\right) , \end{aligned}$$

(113)

with

$$\begin{aligned} z_t'=\mu ^\tau _{t-1}-\ell _{t-1}=e^{-\tau }(x_{t-1}-\ell _{t-1}). \end{aligned}$$

(114)

Given $(x^{t-1},y^{t-1})$, the only term depending on $y_t$ is $\mathbb {E}[z_t^2]$ in Eq. (111). It is optimized by choosing $\ell _t$ so as to have $z_t=0$:

$$\begin{aligned} \hat{\ell }_t=\kappa y_t+(1-\kappa )\mu ^\tau _{t-1}=\kappa y_t+(1-\kappa )[(1-e^{-\tau })\ell _{t-1}+e^{-\tau }x_{t-1}]. \end{aligned}$$

(115)

By taking $\ell _{t-1}=\hat{\ell }_{t-1}$, this defines recursively a series of optimal translations $\hat{\ell }^t$.

To express the optimal work, it remains to evaluate $\mathbb {E}[z_t'^2]$ for $\ell ^t=\hat{\ell }^t$. Since $x_t-\hat{\ell }_t=(1-\kappa )(x_t-\mu _{t-1}^\tau )-\kappa (y_t-x_t)$ where $x_t-\mu _{t-1}^\tau$ and $y_t-x_t$ are statistically independent, we have

$$\begin{aligned} \mathbb {E}[(x_t-\hat{\ell }_t)^2]=(1-\kappa )^2\varsigma ^2_\tau +\kappa ^2\sigma ^2_{y|x}=\sigma ^2_{x|y}, \end{aligned}$$

(116)

and therefore $\mathbb {E}[z_t'^2]=e^{-2\tau }\sigma ^2_{x|y}$. All together, we obtain

$$\begin{aligned} \max _{\ell ^t}\mathbb {E}[\mathcal {W}_t]= & {} \frac{1}{2}\ln \left( 1+\frac{\varsigma ^2_\tau }{\sigma ^2_{y|x}}\right) -\frac{1}{2}\left( \sigma ^2_{x|y}-\ln \sigma ^2_{x|y}-1\right) \nonumber \\&\quad +\,\frac{1}{2}\left( \varsigma ^2_\tau +e^{-2\tau }\sigma ^2_{x|y}-\ln \varsigma ^2_\tau -1\right) , \end{aligned}$$

(117)

which, given that $\varsigma ^2_\tau =1-e^{-2\tau }$ and $\sigma ^2_{x|y}=(\varsigma ^{-2}_\tau +\sigma ^{-2}_{y|x})^{-1}$, simplifies to $\max _{\ell ^t}\mathbb {E}[\mathcal {W}_t]=\varsigma ^2_\tau (1-\sigma ^2_{x|y})/2$, or, in terms of $\tau$ and $\sigma ^2_{y|x}$ only,

$$\begin{aligned} \max _{\ell ^t}\mathbb {E}[\mathcal {W}_t]= \frac{1}{2}(1-e^{-2\tau })(1-\big ((1-e^{-2\tau })^{-1}+\sigma ^{-2}_{y|x}\big )^{-1}). \end{aligned}$$

(118)

When $\tau \rightarrow \infty$, we recover the equilibrium result, $\mathbb {E}[\mathcal {W}_t]\le I(X;Y)-\min _\phi \mathbb {E}_Y[D(P_{X|Y}(.-Y)\Vert G_1(.-\phi (Y)))]$, with $I(X;Y)=[\ln (1+1/\sigma ^2_{y|x})]/2$ and $\min _\phi \mathbb {E}_Y[D(P_{X|Y}(.-Y)\Vert G_1(.-\phi (Y)))]=D(G_{\sigma ^2_{x|y}}\Vert G_1)=(\sigma ^2_{x|y}-\ln \sigma ^2_{x|y}-1)/2$.