Mutual information in changing environments: Nonlinear interactions, out-of-equilibrium systems, and continuously varying diffusivities

Giorgio Nicoletti and Daniel Maria Busiello

Phys. Rev. E 106, 014153 – Published 29 July 2022

Abstract

Biochemistry, ecology, and neuroscience are examples of prominent fields aiming at describing interacting systems that exhibit nontrivial couplings to complex, ever-changing environments. We have recently shown that linear interactions and a switching environment are encoded separately in the mutual information of the overall system. Here we first generalize these findings to a broad class of nonlinear interacting models. We find that a new term in the mutual information appears, quantifying the interplay between nonlinear interactions and environmental changes, and leading to either constructive or destructive information interference. Furthermore, we show that a higher mutual information emerges in out-of-equilibrium environments with respect to an equilibrium scenario. Finally, we generalize our framework to the case of continuously varying environments. We find that environmental changes can be mapped exactly into an effective spatially varying diffusion coefficient, shedding light on modeling of biophysical systems in inhomogeneous media.

Received 4 April 2022
Revised 1 June 2022
Accepted 14 July 2022

DOI:https://doi.org/10.1103/PhysRevE.106.014153

Physics Subject Headings (PhySH)

Complex systems Stochastic processes

Fokker–Planck equation Information theory Langevin equation

Statistical Physics & ThermodynamicsNonlinear DynamicsInterdisciplinary PhysicsGeneral Physics

Authors & Affiliations

Giorgio Nicoletti ¹ and Daniel Maria Busiello^2,*

¹Laboratory of Interdisciplinary Physics, Department of Physics and Astronomy “G. Galilei,” University of Padova, Padova 35121, Italy
²Institute of Physics, École Polytechnique Fédérale de Lausanne–EPFL, 1015 Lausanne, Switzerland

^*Present address: Max Planck Institute for the Physics of Complex Systems, 01187 Dresden, Germany (busiello@pks.mpg.de).

Article Text (Subscription Required)

Click to Expand

References (Subscription Required)

Click to Expand

Issue

Vol. 106, Iss. 1 — July 2022

Reuse & Permissions

Access Options

Author publication services for translation and copyediting assistance advertisement

Images

Figure 1
The environmental contribution to the mutual information as a function of $D_{-} / D_{+}$ and $w_{-} / w_{+}$ in a quartic potential and in the slow-jumps limit. For all plots, $τ = 1$ . (a) The quartic potential considered here. [(b) and (c)] Contour plots of the joint probability distribution and its factorization, respectively, for $D_{+} = 10, D_{-} = 10^{- 2}, w_{-} = w_{+}$ . Notice that the marginalized probability has much longer tails along the axis $x = 0$ and $y = 0$ . (d) The colored surface is the result of a Monte Carlo integration with importance sampling of the mutual information. In the $D_{-} / D_{+} \to 0$ limit, $I_{env}$ approaches $H_{jumps}$ , the black dashed line, which is also its maximum value. (e) Compared with the linear case, the nonlinear relaxation reflects into a considerably slower convergence toward $H_{jumps}$ of the mutual information.
Reuse & Permissions
Figure 2
The total mutual information $I_{sw}$ in the single-well case, as a function of $D_{-} / D_{+}$ and $w_{-} / w_{+}$ , in the slow-jumps limit. For all plots, $τ g = 10$ . (a) The single-well potential considered here. [(b) and (c)] Contour plots of the joint probability distribution and its factorization, respectively, for $D_{+} = 10, D_{-} = 10^{- 2}, w_{-} = w_{+}$ . The marginalized probability has much longer tails along the axis $x = 0$ and $y = 0$ , which tend to be suppressed by the environment. However, these tails are still present in the joint probability as a consequence of the interactions. (d) The colored surface is the result of a Monte Carlo integration with importance sampling of $I_{sw}$ , whereas the gray surface represent the environmental contribution alone $I_{env}$ . (e) $I_{sw}$ (black dots) can be smaller than $I_{env}$ (gray dashed line) and, in general, it is lower than the sum of $I_{sw}^{int}$ and $I_{env}$ (red dashed line). (f) In fact, the term $Ξ_{sw}$ is always negative, showing that the effects of the environment and of the interactions are reciprocally masked at low-enough values of $D_{-} / D_{+}$ . For $D_{-} / D_{+} \to 0, 1$ we find $Ξ_{sw} = 0$ , hence the environmental and the interactions contributions are disentangled.
Reuse & Permissions
Figure 3
The total mutual information $I_{swr}$ in the rotated single-well case, as a function of $D_{-} / D_{+}$ and $w_{-} / w_{+}$ , in the slow-jumps limit. For all plots, $τ g = 10$ . (a) The rotated single-well potential considered here. [(b) and (c)] Contour plots of the joint probability distribution and its factorization, respectively, for $D_{+} = 10, D_{-} = 10^{- 2}, w_{-} = w_{+}$ . The marginalized probability has much longer tails along the axis $x = 0$ and $y = 0$ , which are suppressed in the joint probability as a consequence of the environment. Contrary to the single-well case, the interactions here trigger the presence of tails along the bisectors of the $(x, y)$ plane. (d) The colored surface is the result of a Monte Carlo integration with importance sampling of $I_{swr}$ , whereas the gray surface represent the environmental contribution alone $I_{env}$ . (e) $I_{swr}$ (black dots) is always greater than $I_{env}$ (gray dashed line) and, in general, it is greater than the sum of $I_{swr}^{int}$ and $I_{env}$ (red dashed line). (f) The term $Ξ_{swr}$ is always positive, and in particular it is different from zero at high enough values of $D_{-} / D_{+}$ . For $D_{-} / D_{+} \to 0, 1$ we end up with $Ξ_{swr} = 0$ as expected.
Reuse & Permissions
Figure 4
The total mutual information $I_{dw}$ in the double-well case, as a function of the adimensional parameters $D_{\pm} / (τ g^{2})$ , in the slow-jumps limit. For all plots, $w_{-} / w_{+} = 1, τ = g = 1$ . (a) The double-well potential considered here. [(b) and (c)] Contour plots of the joint probability distribution and its factorization, respectively, for $D_{+} = 10, D_{-} = 10^{- 2}$ . The joint probability has two peaks, corresponding to the two minima of the potential. On the contrary, the marginalized probability is markedly different, with four peaks. (d) The colored surface is the result of a Monte Carlo integration with importance sampling of $I_{dw}$ , whereas the gray surface represent the environmental contribution alone $I_{env}$ , in the plane $(D_{-} / τ g^{2}, D_{+} / τ g^{2})$ . (e) The mutual information $I_{dw}^{int}$ of the interactions only. At large $D$ , we expect the two minima to be less relevant, and indeed the mutual information vanishes. At small $D$ , instead, $I_{dw}^{int}$ is markedly different from zero since the particles are typically trapped in one of the two minima. At intermediate values the mutual information peaks due to an interplay between trapping and diffusion. (e) The term $Ξ_{dw}$ can be either positive and negative, meaning that, at different values of $D_{\pm} / (τ g^{2})$ , we find both constructive and destructive interference. Clearly, when $D_{-} \approx D_{+}$ , we have $Ξ_{dw} \approx 0$ .
Reuse & Permissions
Figure 5
Mutual information in the presence of multiplicative noise. [(a) and (b)] The joint and the marginal probability distributions in the slow-jumps limit. (c) As we increase the temperature gradient $Δ T$ , the mutual information increases. Here the different curves are for $Δ T = {0.15, 0.29, 0.43, 0.57}, T_{0} = τ = 1, γ_{+} = 1.5$ , and $γ_{-} = 0.5$ .
Reuse & Permissions
Figure 6
The mutual information associated to the Langevin equations in Eq. (31). [(a) and (b)] Plots of the joint and the factorized distribution in the limit of a much slower environment, $τ_{X} / τ_{D} ≪ 1$ . (c) Mutual information at different values of $τ_{X} / τ_{D}$ estimated through a $k$ -nearest-neighbors estimator. As expected, in the limit $τ_{X} / τ_{D} ≪ 1$ the mutual information converges to Eq. (36), whereas it vanishes in the opposite limit.
Reuse & Permissions

Physical Review E

covering statistical, nonlinear, biological, and soft matter physics