Role of measurement-feedback separation in autonomous Maxwell's demons

We introduce an information heat engine that is autonomous (i.e., without any time-dependent parameter) but has separated measurement and feedback processes. This model serves as a bridge between the other types of information heat engines inspired by Maxwell's demon; from the original Szilard-engine type systems to the autonomous demonic setups. By analyzing our model on the basis of a general framework introduced in our previous paper [N. Shiraishi and T. Sagawa, Phys. Rev. E {\bf 91}, 012130 (2015).], we clarify the role of the separation of measurement and feedback in the integral fluctuation theorems.


Introduction
"Maxwell's demon" is a thought experiment proposed by J. C. Maxwell [1]: if a thermodynamic system is subjected to feedback control at the level of thermal fluctuations, the second law can be apparently violated. A prominent example is the Szilard engine [2], which is a composite system of an engine and a memory. The memory measures the state of the engine and performs feedback to the engine. The feedback procedure allows positive work to be extracted from the engine through an isothermal cyclic process, which, under the usual circumstance, is prohibited by the second law of thermodynamics. Szilard characterized the two separated steps of measurement and feedback as the processes which change the extent of the correlation between the engine and the memory, and suggested the possibility of an extended framework of thermodynamics for systems with the change in correlation. The consistency of the second law of thermodynamics with the existence of Maxwell's demon has been discussed vigorously [3]. It has been suggested that the key to understand the consistency is the change in the volume of the phase space [4], which characterizes the thermodynamic irreversibility [5]. Modern theories have revealed that this property is captured by the mutual information, which is a quantity that measures the correlation between the engine and the memory. The generalized thermodynamic relations with the mutual information has been discussed for a single feedback process [6,7], continuous feedback processes [8][9][10], feedback cooling [11][12][13], and more general information processing [14][15][16].
Although these information-theoretic frameworks are expected to be useful in various problems in small fluctuating systems such as biochemical sensing [17,18] and quantum mesoscopic systems [19,20], there is a critical remark to be made; the original framework, which is applicable to the Szilard-type demons with stepby-step separated measurement and feedback processes, cannot be directly applied to autonomous stochastic information processes. This is because the processes of measurement and feedback can themselves occur stochastically at any time, and thus they are inseparable in autonomous setups. In purpose to extend the applicability of information thermodynamics to autonomous Maxwell's demons, intensive researches on the model of autonomous demons [20][21][22][23] and information flow [24][25][26][27][28][29][30][31][32][33][34] have been recently made. Autonomous demons can be modeled as bipartite Markovian systems where the measurement and the feedback processes are not, in general, separated. The integral fluctuation theorem (IFT) obtained for the autonomous demons has distinct features from the relation obtained for the Szilard-type systems [15,16]. The crucial difference comes from the fact that in the Szilard-type systems, the state of the engine (memory) is fixed externally during the measurement (feedback) phase, whereas in autonomous demons, it is not.
In this paper, we clarify the difference between Szilard-type demons and autonomous demons on the basis of a general framework introduced in our previous paper [34]. In purpose to build a bridge between these two different setups, we propose a new model of Maxwell's demon, which is autonomous but possesses separated measurement and feedback processes. The key ingredient in the model is an additional stochastic variable, which plays a role of separating the measurement phase and the feedback phase. An IFT with the mutual information can be derived for this model, which is more similar to another IFT satisfied in Szilard-engine type demons [15,16] than to that for autonomous bipartite demons [34]. This paper is organized as follows. In Sec. 2, we review IFTs for Szilard-type demons and autonomous demons, where the mutual information plays a crucial role. We also discuss a general framework that leads to IFT of autonomous and non-autonomous demons in a unified way, on the basis of the concept of the partial entropy production. In Sec. 3, we introduce a model of the autonomous demon with separated measurement and feedback phases, and derive an IFT for this model. On the basis of these discussions, in Sec. 4, we elucidate the role played by the separation of the measurement phase and the feedback phase.

Thermodynamics of small systems
Throughout this paper, we consider Markov jump processes during the time interval 0 ≤ t ≤ T . The system is in the isothermal condition with inverse temperature β, and the possible states of the system are denoted as w 1 , w 2 · · · w M . The transition from w ′ to w is written as w ′ → w, where the transition rate at time t is denoted by P (w ′ → w; t). We assume that for any states w and any time t there exists another state w ′ such that P (w ′ → w; t) = 0. The time evolution of the system is written as the master equation: where P (w, t) is the probability distribution of w at time t, and J(w ′ → w; t) := P (w ′ , t)P (w ′ → w; t) − P (w, t)P (w → w ′ ; t) represents the probability flux from w ′ to w.
Jumps between states occur stochastically in each trajectory, where we denote the number of jumps by N. The i-th jump occurs at time t i (1 ≤ i ≤ N), and correspondingly the state changes from w i−1 to w i . We write the initial and final time of the entire dynamics as t 0 := 0 and t N +1 := T . The state at time t is written as w(t). We define the total entropy production σ tot as [35] where is the heat absorption by the system from the heat bath with the transition w ′ → w at time t, and s(w, t) := − ln P (w, t) represents the stochastic Shannon entropy at the state w and time t. Here, we assumed the local detailed balance condition [36], and normalized the Boltzmann constant k B to 1. The total entropy production characterizes the irreversibility, and satisfies the IFT e −σtot = 1 and the second law of thermodynamics σ tot ≥ 0 [37][38][39][40], where · represents the ensemble average over all trajectories.

General framework
We now describe a general theoretical framework introduced in Ref. [34], which is applicable to both Szilard-type and autonomous demons in a unified way, as discussed in the subsequent subsections. Under this framework, the entropy production for the total system is divided into contributions from subsets of individual transitions, where each partial entropy production satisfies an IFT. We divide a set of all possible transitions w i → w j into two subsets: Ω and its complement Ω c . In Fig. 1(a), for example, the six red arrows represent the transitions included in Ω and the other sixteen black arrows represent the transitions included in Ω c . We divide the total entropy production σ tot into the contributions from Ω and Ω c , which are denoted by σ Ω and σ Ω c , respectively. Note that the subset Ω can be time-dependent in general. First, we divide the probability flux at the state w as J( and J Ω c (w, t) is defined in the same manner. Using Eq. (4), we define the partial entropy production σ Ω as [34] σ Ω := −βQ Ω + ∆s Ω .
The first term of the right-hand side (rhs) of Eq. (5) represents the heat absorption accompanied by the transitions in Ω: where δ Ω (w ′ → w) takes 1 if (w ′ → w) ∈ Ω and takes 0 otherwise. The second term of the rhs of Eq. (5) represents the change in the stochastic entropy induced by the transitions in Ω: The first term of the rhs of Eq. (7) is defined as which quantifies the contributions from jumps (dashed lines in Fig. 1(b)) within Ω. The second term of the rhs of Eq. (7) evaluates the change induced by the probability flux (the solid lines in Fig. 1(b)) within Ω, which is confirmed by It is crucial that the partial entropy production σ Ω satisfies and the IFT [34]: Applying Jensen's inequality to Eq. (11), we obtain In the following two subsections, we show two IFTs for Szilard-type demons (18) and that for autonomous demons (20) which are particular cases of Eq. (11).

IFT for the Szilard-type demon
In this and the next subsections, we consider a bipartite system that consists of subsystems X and Y , where the state is written as w = (x, y), with the transition rates satisfying P ((x, y) → (x ′ , y ′ ); t) = 0 for x = x ′ and y = y ′ . We define the entropy production of subsystem X as where Q X is the heat absorbed by subsystem X, and ∆s X := − ln P (x(T ), T ) + ln P (x(0), 0) is the change in the stochastic Shannon entropy of only x. In contrast to the averaged entropy production of the total system σ tot , that of the subsystem σ X can be negative. In the case of the Szilard engine [2], which is a composite system of an engine and a memory, the entropy production of the engine is negative on average corresponding to the extraction of work from the isothermal cycle.
In previous works [15,16], it has been discussed that modified IFT and second law still hold under these setups, if the change in the correlation between the engine and the memory is taken into account. In the case of the Szilard engine, the measurement process increases the correlation, and the feedback process decreases the correlation. In Transitions in Ω c is allowed.
Transitions in Ω is allowed. general, the correlation is quantified by the mutual information. The stochastic mutual information between the state of the engine X and the state of the memory Y is defined as whose ensemble average is the mutual information [41]. The mutual information is zero if there is no correlation, and becomes larger if the correlation is stronger. In order to generalize the setup of the Szilard engine, we here suppose that there are m-times of measurement and feedback processes in a composite system of X and Y during the time interval from T 0 = 0 to T 2m . During the time interval T 2n ≤ t < T 2n+1 (T 2n+1 ≤ t < T 2n+2 ) with 0 ≤ n ≤ m − 1, only Y (X) can evolve and X (Y ) is fixed (see Fig. 2), corresponding to the measurement (feedback) phase. These conditions are written as where we abbreviated the transition rates as . This setup represents the repeated measurement and feedback processes between the engine X and the memory Y . Here, by applying Eq. (11) with setting all transitions in X as Ω (see Fig. 3), the entropy production of the engine σ X satisfies the following IFT and the generalized second law [16]: where ∆I X,n := I T 2n+2 (x; y)−I T 2n+1 (x; y) represents the change in the mutual information between before and after the n-th feedback process. This IFT implies that if we consume  the correlation between the engine and the memory (i.e., n ∆I X,n < 0), the entropy production in the engine can be negative (i.e., σ X < 0) up to the amount of the consumption. In contrast, if we establish their correlation, we need the corresponding extra cost. Equality (18) is a special case of a more general IFT that is discussed below.

IFT for general measurement and feedback processes
In this subsection, we mention on the case of the general measurement and feedback processes with bipartite systems, which is applicable for example to autonomous information processing in biological systems [17,18]. The transition rate of X depends on the state of memory Y , and vice versa, although the time-separation of the measurement and feedback [Eqs. (16) and (17)] is not assumed. We define the entropy production associated only with X by the same form as Eq. (13). By applying Eq. (11) with setting all transitions in X as Ω (see Fig. 4), it has been shown that the following IFT holds [34]: where ∆I X represents the change in the mutual information contributed from X [26-28, 32] defined as The first term I X,jump represents the change in the mutual information with realized jumps of X defined as where δ is the Kronecker delta. Here, we used notation w i =: (x i , y i ). The second term involves a time integral over the entire time interval; its integrand F X (x, y, t) represents the change in the mutual information induced by the probability flux of X, which is defined as where J X (x, t) := y J X (x, y, t).

Autonomous Maxwell's demon with separated measurement and feedback process
In contrast to the IFT for the Szilard-type demons (18), the IFT for general measurement and feedback (20) includes a time-integral term over the entire time interval. The most important difference between these setups is whether the time interval of the measurement phase and the feedback phase are separated externally. Although autonomous systems are not controlled externally, two phases can be separated even in autonomous systems by introducing an additional stochastic variable which determines whether the present state is in the measurement phase or in the feedback phase. To clarify this point, we construct an autonomous model with separated measurement and feedback phases, and derive an IFT satisfied in this system.

Model and setups
Let us start from simple models with and without measurement-feedback separation. We first consider the autonomous 4-state model introduced in Ref. [34], where the measurement phase and the feedback phase are not separated. The 4-state model is a bipartite system which transports particles from a dense particle bath L to a dilute particle bath R (see Fig. 5(a)). There is a site between two baths with at most one particle, where x ∈ {p, a} represents the presence p or the absence a of a particle at the site. We then introduce a wall which prohibits jumps of particles. If the wall is between the site and bath L (R), we denote the state of the wall as y = l (r). The position of the wall tends to be r (l) if a particle exists (does not exist) at the site. With these processes, particles are transported against the chemical potential gradient. Now we construct an autonomous model named the 8-state model with the measurement-feedback separation, by adding a new stochastic variable e ∈ {0, 1} to the 4-state model (see Fig. 5(b)). The role of e is to determine whether the present state is in the measurement phase or the feedback phase. When e = 0 (1), only particles (the wall) can move and the wall (particles) is fixed, which corresponds to the feedback (measurement) phase. Since the probability distributions and transition rates are independent of time, we omit the argument of t. These conditions are written as P (l → r; x, 0) = P (r → l; x, 0) = 0, P (p → a; y, 1) = P (a → p; y, 1) = 0. We assume that the transition rates of e are independent of x and y, and write P 01 := P (0 → 1; x, y) and P 10 = P (1 → 0; x, y).
We can easily generalize this setup to the case with more than eight states. Suppose a composite system with three variables; x, y, and an additional variable e ∈ {0, 1}, which is in the stationary distribution with time-independent transition rates. We assume that only one of {x, y, e} changes within a single transition. The transition rates also satisfy the following conditions (see also Fig. 6(a)):

IFT for separated autonomous demons
We now discuss an IFT for autonomous demons with separated measurement and feedback (e.g., the 8-state model). The entropy production of engine X is defined as By using the stationary distribution P ss (x) := dydeP ss (x, y, e), we defined the stochastic entropy of x as s(x) := − ln P ss (x). Under the setup described in Sec. 3.1, σ X satisfies the following IFT: Here, I ch 0,i is defined as which subtracts (adds) the mutual information at e = 1 (0) when the state changes from the measurement (feedback) phase e = 1 (0) to the feedback (measurement) phase e = 0 (1). The conditional mutual information is defined as I(x; y|e) := ln P ss (x, y|e) P ss (x|e)P ss (y|e) , where P ss (x|e) := dyP ss (x, y, e)/P ss (e). The definition of ∆I X,n in Eq. (18) is regarded as the change in the mutual information between the endpoint of the measurement processes and the endpoint of the feedback processes. I 0 (x; y) is defined as I 0 (x; y) := I(x; y|e)δ e,0 , which counts the mutual information at the initial and the final states if these states are in e = 0. We now derive Eq. (28) from Eq. (11) by setting Ω to {(x ′ , y ′ , e ′ ) → (x, y, e)|e = 0} (see also Fig. 7). In this case, since the system is in the stationary state, ∆s Ω is calculated as where we used J Ω (x, y, e) = 0 for all (x, y, e). The difference between Q Ω and Q X is equal to the heat absorption accompanying the transitions from e = 1 to e = 0: ln P ss (e i−1 )(δ e i ,0 − δ e i−1 ,0 ) + ln P ss (0)(δ e 0 ,0 − δ e N ,0 ), (32) where we used the stationary condition for e such that P ss (0)P 01 = P ss (1)P 10 . Since y is fixed while e = 0 and x is fixed while e = 1, it is easy to show that We also note that P ss (x|1) = P ss (x|0) = P ss (x) P ss (y|1) = P ss (y|0) = P ss (y) for all x, y (see Fig. 6(b)), which follows from the fact that P 01 and P 10 are constant and independent of (x, y). Finally, we arrive at which implies Eq. (28).

State-space reduction of autonomous demon
In this subsection, we show that the IFT for non-separated autonomous demons (20) in stationary states can be naturally derived from the IFT for separated autonomous demons (28), by considering the reduction of the variable e. Suppose that there are two models, A and B. Model A has three variables (x, y, e), and satisfies the setup described in Sec. 3.1. Model B has two variables (x, y), and satisfies the setup described in Sec. 2.4. Hence, Eq. (28) holds in model A and Eq. (20) holds in model B. In the following, superscript A (B) represents quantities in model A (B). By using the transition rates of model A, we set the transition rates of model B to Here, we introduce a quantity k: which characterizes the typical rate of the transition in e. By taking the limit k → ∞ in model A with fixed P A (x → x ′ ; y, e), P A (y → y ′ ; x, e), P A ss (0), and P A ss (1), we obtain a reduced Markovian dynamics with the variables (x, y), which is equivalent to the dynamics of model B.  Then, the total count of I ch 0,i per unit time during the stay at (x a , y b ) is calculated as In the second line, we used Eq. (34). In the third and fifth lines, we used for e = 0, 1. In the fourth line, we used the stationary condition at (x a , y b , 0): In the last line, we used the fact that the second term of the rhs of Eq. (23) is zero in the stationary state due to From (40) Figure 9. An example of the division of the state space. The subset of states D is {w 1 , w 6 , w 7 }. We set the ten red arrows to Ω in this case.

Role of separated measurement and feedback in IFTs
We discussed in the previous section that the IFT (28) satisfied in our separated autonomous demon model is similar to that of the Szilard-type systems (18). In contrast, the IFT (20) for the general measurement and feedback systems contained a timeintegral term F X (x, y, t), which we found an interpretation as the mutual information flow in the fast switching limit as shown in Eq. (40). We here aim to clarify the relation between the existence of the time-integral term in the IFT and the separation of measurement and feedback dynamics. Let us first recall how we set Ω when we derived the IFTs, from the viewpoint of the general framework in Sec. 2.2. First, we define a set of transitions whose final state is w as S w := {(w ′ → w)|w ′ ∈ W, (w ′ → w) ∈ G}, where W represents the set of all possible states and G represents the set of all possible transitions. Then, the common feature of the choices of Ω for Eq. (18) and Eq. (28) is that S w satisfies S w ⊆ Ω or S w ⊆ Ω c for any w. In the case of Eq. (18), S w is time-dependent, and all S w satisfy S w ⊆ Ω (Ω c ) in the feedback (measurement) phase (see Fig. 3). In the case of Eq. (28), for w with e = 0 (e = 1), S w satisfies S w ⊆ Ω (Ω c ) (see Fig. 7). This property implies the separation of the measurement phase and the feedback phase. We call this condition the separation condition. Conversely, the measurement phase and the feedback phase are unseparated, when we can find some (w, w ′ , w ′′ ) such that (w ′ → w) ∈ Ω and (w ′′ → w) ∈ Ω c (see Fig. 4), as in the case of the general measurement and feedback setup including the reduced dynamics discussed in Sec. 3.3.
To confirm that the separation condition plays a crucial role for the time-integral term in IFT, we show another IFT for the division in a state space. Corresponding to a subset Ω with separation condition, we divide the possible states into two groups (see Fig. 9): Owing to the separation condition, D c is indeed the complement of D. Although D is time-dependent in general, in the following we assume that D is time-independent for simplicity. We now show that σ Ω can be transformed into a form without any timeintegral term. By focusing on the change in the stochastic Shannon entropy, Eq. (7), the integrand of the time-integral term can be written as which is the crucial property of the separation condition. Adding the contribution from the jumps, we have where δ D (w) takes 1 if w ∈ D, and 0 otherwise. Note that δ Ω (w ′ → w) = δ D (w) holds for Ω with the separation condition. We also introduced the entropy exchange between D and D c through the i-th transition The entropy exchange s ch D,i represents the initial and the final entropy of the dynamics in D, which corresponds to I ch 0,i in Eq. (28). Therefore, the IFT for the case of divided states does not include the time-integral term. We note that the contribution from the entropy exchange in Eq. (47) corresponds to the change in the mutual information, which appeared in the IFTs for Szilard-type systems and the model with separated measurement and feedback phases.

Conclusion
In this paper, we clarified the difference between the Szilard-type demons and autonomous bipartite demons. By introducing another type of autonomous demon, in which the measurement phase and the feedback phase are separated, we showed that the presence of a time-integral term in IFTs is related to the unseparated measurement and feedback phases. Since the Szilard-type demons and the 8-state model have the separated measurement and feedback phases, the IFTs for the Szilard  On the basis of the general framework in Sec. 2.2, we clarified the concept of separation as the condition on the choice of the subset of transitions Ω. The separation condition leads to the absence of time-integral terms, which is clearly shown in Eq. (47). Understanding the separation of the measurement phase and the feedback phase is important to analyze the difference between ideal information processing systems (e.g., the Szilard engine) and demons in the real world such as biochemical networks.