Fluctuations When Driving Between Nonequilibrium Steady States

Abstract

Maintained by environmental fluxes, biological systems are thermodynamic processes that operate far from equilibrium without detailed-balanced dynamics. Yet, they often exhibit well defined nonequilibrium steady states (NESSs). More importantly, critical thermodynamic functionality arises directly from transitions among their NESSs, driven by environmental switching. Here, we identify the constraints on excess heat and dissipated work necessary to control a system that is kept far from equilibrium by background, uncontrolled “housekeeping” forces. We do this by extending the Crooks fluctuation theorem to transitions among NESSs, without invoking an unphysical dual dynamics. This and corresponding integral fluctuation theorems determine how much work must be expended when controlling systems maintained far from equilibrium. This generalizes thermodynamic feedback control theory, showing that Maxwellian Demons can leverage mesoscopic-state information to take advantage of the excess energetics in NESS transitions. We also generalize an approach recently used to determine the work dissipated when driving between functionally relevant configurations of an active energy-consuming complex system. Altogether, these results highlight universal thermodynamic laws that apply to the accessible degrees of freedom within the effective dynamic at any emergent level of hierarchical organization. By way of illustration, we analyze a voltage-gated sodium ion channel whose molecular conformational dynamics play a critical functional role in propagating action potentials in mammalian neuronal membranes.

Appendices

Appendix A: Extension to Non-Markovian Instantaneous Dynamics

Commonly, theoretical developments assume state-to-state transitions are instantaneously Markovian given the input. This assumption works well for many cases, but fails in others with strong coupling between system and environment. Fortunately, we can straightforwardly generalize the results of stochastic thermodynamics by considering a system’s observable states to be functions of latent variables ${\varvec{{\mathcal {R}}}}$. The goal in the following is to highlight the necessary changes, so that it should be relatively direct to adapt our derivations to the non-Markovian dynamics. See Ref. [75] for an alternative approach to addressing non-Markovian dynamics.

1.1 Latent States, System States, and Their Many Distributions

Even with constant environmental input, the dynamic over a system’s states need not obey detailed balance nor exhibit any finite Markov order. We assume that the classical observed states $\varvec{\mathcal {S} }$ are functions $f: {\varvec{{\mathcal {R}}}}\rightarrow \varvec{\mathcal {S} }$ of a latent Markov chain. We also assume that the stochastic transitions among latent states are determined by the current environmental input $x \in \mathcal {X}$, which can depend arbitrarily on all previous input and system-state history. The Perron–Frobenius theorem guarantees that there is a stationary distribution over latent states associated with each fixed input x; the function of the Markov chain maps this stationary distribution over latent states into the stationary distribution over system states. These are the stationary distributions associated with system NESSs.

We assume too that the ${\varvec{{\mathcal {R}}}}$-to-${\varvec{{\mathcal {R}}}}$ transitions are Markovian given the input. However, different inputs induce different Markov chains over the latent states. This can be described by a (possibly infinite) set of input-conditioned transition matrices over the latent state set ${\varvec{{\mathcal {R}}}}$: $\{ \mathsf{{T}}^{({\varvec{{\mathcal {R}}}}\rightarrow {\varvec{{\mathcal {R}}}}| x)} \}_{x \in \mathcal {X}}$, where $\mathsf{{T}}^{({\varvec{{\mathcal {R}}}}\rightarrow {\varvec{{\mathcal {R}}}}| x)}_{i,j} = \Pr ({\mathcal {R}}_{t} = r^j | {\mathcal {R}}_{t-1} = r^i , X_t = x)$. Probabilities regarding actual state paths can be obtained from the latent-state-to-state transition dynamic together with the observable-state projectors, which we now define.

We denote distributions over the latent states as bold Greek symbols, such as $\varvec{\mu }$. As in the main text, it is convenient to cast $\varvec{\mu }$ as a row-vector, in which case it appears as the bra $\langle {\varvec{\mu }}|$. The distribution over latent states ${\varvec{{\mathcal {R}}}}$ implies a distinct distribution over observable states $\varvec{\mathcal {S} }$. A sequence of driving inputs updates the distribution: $\varvec{\mu }_{t+n}(\varvec{\mu }_{t}, x_{t:t+n})$. In particular:

$$\begin{aligned} \langle {\varvec{\mu }_{t+n}}|&= \langle {\varvec{\mu }_{t}}| \mathsf{{T}}^{({\varvec{{\mathcal {R}}}}\rightarrow {\varvec{{\mathcal {R}}}}| x_{t:t+n} )} \\&= \langle {\varvec{\mu }_{t}}| \mathsf{{T}}^{({\varvec{{\mathcal {R}}}}\rightarrow {\varvec{{\mathcal {R}}}}| x_{t} )} \mathsf{{T}}^{({\varvec{{\mathcal {R}}}}\rightarrow {\varvec{{\mathcal {R}}}}| x_{t+1} )} \cdots \mathsf{{T}}^{({\varvec{{\mathcal {R}}}}\rightarrow {\varvec{{\mathcal {R}}}}| x_{t+n-1} )} . \end{aligned}$$

(Recall that time indexing is denoted by subscript ranges n : m that are left-inclusive and right-exclusive.) An infinite driving history induces a distribution over the state space, and $\varvec{\pi _x}$ is the specific steady-state distribution over ${\varvec{{\mathcal {R}}}}$ induced by tireless repetition of the single environmental drive x. Explicitly:

$$\begin{aligned} \langle {\varvec{\pi _x}}| = \lim _{n \rightarrow \infty } \langle {\varvec{\mu }_0}| \left( \mathsf{{T}}^{({\varvec{{\mathcal {R}}}}\rightarrow {\varvec{{\mathcal {R}}}}| x)} \right) ^n . \end{aligned}$$

Usefully, $\varvec{\pi _x}$ can also be found as the left eigenvector of $\mathsf{{T}}^{({\varvec{{\mathcal {R}}}}\rightarrow {\varvec{{\mathcal {R}}}}| x)}$ associated with the eigenvalue of unity:

$$\begin{aligned} \langle {\varvec{\pi _x}}| = \langle {\varvec{\pi _x}}| \mathsf{{T}}^{({\varvec{{\mathcal {R}}}}\rightarrow {\varvec{{\mathcal {R}}}}| x)} . \end{aligned}$$

(A1)

The physically relevant steady-state probabilities are this vector’s projection onto observable states: $\pi _x(s) = \langle {\varvec{\pi _{x}} | s}\rangle$, where $|{s}\rangle = |{\delta _{s,f(r)}}\rangle$ has a vector-representation in the latent-state basis with elements of all 0s except 1s where the latent state maps to the observable state s.

Assuming latent-state-to-state transitions are Markovian allows the distribution $\varvec{\mu }$ over these latent states to summarize the causal relevance of the entire driving history.

1.2 Implications

A semi-infinite history induces a particular distribution over system latent states and implies another particular distribution over its observable states. This can be usefully recast in terms of the “start” (or initial) distribution $\varvec{\mu }_0$ induced by the path $x_{-\infty :1}$ and the driving history $x_{1:t+1}$ since then, giving the entropy of the induced state distribution:

$$\begin{aligned} h^{(s | \varvec{\mu }_0 , x_{1:t+1} ) }&= - \ln \Pr (\mathcal {S} _t = s | \varvec{\mu }_0 , x_{1:t+1}) \\&= - \ln \langle {\varvec{\mu }_0}| \mathsf{{T}}^{({\varvec{{\mathcal {R}}}}\rightarrow {\varvec{{\mathcal {R}}}}| x_{1:t+1})} |{s}\rangle . \end{aligned}$$

Or, employing the new distribution and the driving history since then, the path entropy (functional of state and driving history) can be expressed simply in terms of the current distribution over latent states and the candidate observable state s:

$$\begin{aligned} h^{(s | \varvec{\mu } ) }&= - \ln \Pr (\mathcal {S} _t = s | {\mathcal {R}}_{t} \sim \varvec{\mu } ) \\&= - \ln \langle {\varvec{\mu } | s}\rangle . \end{aligned}$$

Averaging the path-conditional state entropy over observable states again gives a genuine input-conditioned Shannon state entropy:

$$\begin{aligned} \langle {h^{(s_t | {\overleftarrow{x}}_{t})} }\rangle _{\Pr (s_t | {\overleftarrow{x}}_{t})}&= {\text {H}}[\mathcal {S} _t | {\overleftarrow{X}}_{t}= {\overleftarrow{x}}_{t}] . \end{aligned}$$

It is again easy to show that the state-averaged path entropy $k_\text {B}{\text {H}}[\mathcal {S} _t | {\overleftarrow{x}}_{t}]$ is an extension of the system’s steady-state nonequilibrium entropy. In steady-state, the state-averaged path entropy reduces to:

$$\begin{aligned} k_\text {B}{\text {H}}[\mathcal {S} _t | {\overleftarrow{X}}_{t}= \dots xxx ]&= - k_\text {B}{\text {H}}[\mathcal {S} _t | {\mathcal {R}}_t \sim \varvec{\pi _x} ] \\&= - k_\text {B}\sum _{s \in \varvec{\mathcal {S} } } \pi _x(s) \ln \pi _x(s) \\&= S_\text {ss}(x) . \end{aligned}$$

The nonsteady-state addition to free energy is:

$$\begin{aligned} \beta ^{-1} \gamma (s | \varvec{\mu } , x) \equiv \beta ^{-1} \ln \frac{\Pr (\mathcal {S} _t = s | {\mathcal {R}}_{t-1} \sim \varvec{\mu }, X_{t} = x )}{ \pi _{x}(s ) } . \end{aligned}$$

Averaging over observable states this becomes the relative entropy:

$$\begin{aligned} \langle {\gamma (s | \varvec{\mu } , x) }\rangle = D_\text {KL} \left[ \Pr (\mathcal {S} _t | {\mathcal {R}}_{t-1} \sim \varvec{\mu } , X_{t} = x ) || \varvec{\pi _x} \right] , \end{aligned}$$

which is always nonnegative.

Using this setup and decomposing:

$$\begin{aligned} \frac{\Pr (\mathcal {S} _{0:N} = s^0 \mathbf {s}| {\mathcal {R}}_{-1} \sim {\varvec{\mu }}_\text {F}, X_{0:N} = x^0 \mathbf {x})}{\Pr (\mathcal {S} _{0:N} = s^{N-1} \mathbf {s}_{\leftarrow }^\text {R}| {\mathcal {R}}_{-1} \sim {\varvec{\mu }}_\text {R}, X_{0:N} = x^N \mathbf {x}^\text {R})} \end{aligned}$$

in analogy with Eq. (25), it is straightforward to extend the remaining results of the main body to the setting in which observed states are functions of a Markov chain. Notably, the path dependencies pick up new contributions from non-Markovity. Also, knowledge of distributions over latent states provides a thermodynamic advantage to Maxwellian Demons.

Appendix B: Integral Fluctuation Theorems with Auxiliary Variables

Recall that we quantify how much the auxiliary variable independently informs the state sequence via the nonaveraged conditional mutual information:

$$\begin{aligned} i[{\overrightarrow{s}}; {\overrightarrow{y}}| {\overrightarrow{x}}, {\varvec{\mu }}_\text {F}]&\equiv \ln \frac{\Pr ({\overrightarrow{s}}, {\overrightarrow{y}}| {\overrightarrow{x}}, {\varvec{\mu }}_\text {F})}{ \Pr ( {\overrightarrow{y}}| {\overrightarrow{x}}, {\varvec{\mu }}_\text {F}) \Pr ( {\overrightarrow{s}}| {\overrightarrow{x}}, {\varvec{\mu }}_\text {F}) } \\&= \ln \frac{\Pr ({\overrightarrow{s}}, {\overrightarrow{y}}, {\overrightarrow{x}}| {\varvec{\mu }}_\text {F})}{ \Pr ( {\overrightarrow{y}}, {\overrightarrow{x}}, {\varvec{\mu }}_\text {F}) \Pr ( {\overrightarrow{s}}| {\overrightarrow{x}}, {\varvec{\mu }}_\text {F}) } . \end{aligned}$$

Note that averaging over the input, state, and auxiliary sequences gives the familiar conditional mutual information:

$$\begin{aligned} \text {I}[\mathcal {S} _{0:N}; Y_{0:N}&| X_{0:N} , {\varvec{\mu }}_\text {F}] = \left\langle i[{\overrightarrow{s}}; {\overrightarrow{y}}| {\overrightarrow{x}}, {\varvec{\mu }}_\text {F}] \right\rangle _{\Pr (x_{0:N} , s_{0:N} , y_{0:N} | {\varvec{\mu }}_\text {F})} . \end{aligned}$$

(Averaging over distributions is the same as being given the distribution, since the distribution over distributions is assumed to be peaked at ${\varvec{\mu }}_\text {F}$.)

Noting that:

$$\begin{aligned} e^{\beta W_\text {diss} + i({\overrightarrow{s}}; {\overrightarrow{y}}| {\overrightarrow{x}}, {\varvec{\mu }}_\text {F}) + \Psi }&= e^{\Omega + i({\overrightarrow{s}}; {\overrightarrow{y}}| {\overrightarrow{x}}, {\varvec{\mu }}_\text {F}) + \Psi + (\gamma _\text {F} - \gamma _\text {R} )} \\&= \frac{\Pr ({\overrightarrow{s}}, {\overrightarrow{y}}, {\overrightarrow{x}}| {\varvec{\mu }}_\text {F})}{ \Pr ( {\overrightarrow{y}}, {\overrightarrow{x}}| {\varvec{\mu }}_\text {F}) \Pr ( s^{N-1} \mathbf {s}_{\leftarrow }^\text {R}| \mathbf {x}^\text {R}x^0 , {\varvec{\mu }}_\text {R}) } \\&= \frac{\Pr ({\overrightarrow{s}}, {\overrightarrow{y}}, {\overrightarrow{x}}| {\varvec{\mu }}_\text {F})}{ \Pr ( {\overrightarrow{y}}, {\overrightarrow{x}}| {\varvec{\mu }}_\text {F}) \Pr ( {\overleftarrow{s}}| {\overleftarrow{x}}, {\varvec{\mu }}_\text {R}) } , \end{aligned}$$

where , we have the integral fluctuation theorem (IFT):

$$\begin{aligned} \left\langle e^{-\beta W_\text {diss} - i({\overrightarrow{s}}; {\overrightarrow{y}}| {\overrightarrow{x}}, {\varvec{\mu }}_\text {F}) - \Psi } \right\rangle _{\Pr ({\overrightarrow{s}}, {\overrightarrow{y}}, {\overrightarrow{x}}| {\varvec{\mu }}_\text {F})}&= \sum _{{\overrightarrow{x}}, {\overrightarrow{s}}, {\overrightarrow{y}}} \Pr ({\overrightarrow{s}}, {\overrightarrow{y}}, {\overrightarrow{x}}| {\varvec{\mu }}_\text {F}) \frac{\Pr ({\overrightarrow{y}}, {\overrightarrow{x}}| {\varvec{\mu }}_\text {F}) \Pr ( {\overleftarrow{s}}| {\overleftarrow{x}}, {\varvec{\mu }}_\text {R})}{\Pr ({\overrightarrow{s}}, {\overrightarrow{y}}, {\overrightarrow{x}}| {\varvec{\mu }}_\text {F})} \\&= \sum _{{\overrightarrow{x}}, {\overrightarrow{s}}, {\overrightarrow{y}}} \Pr ( {\overrightarrow{y}}, {\overrightarrow{x}}| {\varvec{\mu }}_\text {F}) \Pr ( {\overleftarrow{s}}| {\overleftarrow{x}}, {\varvec{\mu }}_\text {F}) \\&= \sum _{{\overrightarrow{x}}, {\overrightarrow{y}}} \Pr ( {\overrightarrow{y}}, {\overrightarrow{x}}| {\varvec{\mu }}_\text {F}) \sum _{{\overleftarrow{s}}} \Pr ( {\overleftarrow{s}}| {\overleftarrow{x}}, {\varvec{\mu }}_\text {R}) \\&= \sum _{{\overrightarrow{x}}, {\overrightarrow{y}}} \Pr ( {\overrightarrow{y}}, {\overrightarrow{x}}| {\varvec{\mu }}_\text {F}) \\&= 1 . \end{aligned}$$

Notably, this relation holds arbitrarily far from equilibrium and allows for the starting and ending distributions to both be nonsteady-state.

It is tempting to conclude that the revised Second Law of Thermodynamics should read:

(B1)

which includes the effects of both irreversibility and conditional mutual information between state-sequence and auxiliary sequence, given input-sequence. However, we expect that $\left\langle Q_\text {hk}\right\rangle > 0$, so Eq. (B1) is not the strongest bound derivable. Dropping $\Psi$ from the IFT still yields a valid equality. However, the derivation runs differently since it depends on the normalization of the dual dynamic—that is, using quantities of the form:

$$\begin{aligned} \frac{\pi _{x^n}(s^{n-1})}{\pi _{x^n}(s^n)} \Pr (s^n | s^{n-1}, x^n) . \end{aligned}$$

These are mathematically-sound transition probabilities $\widetilde{\mathsf{{T}}}_{s^n, s^{n-1}}^{( \varvec{\mathcal {S} } \rightarrow \varvec{\mathcal {S} } | x^n)}$, but only of a nonphysical artificial dynamic. Although IFTs with $\Psi$ may be useful for other reasons, it is the non-$\Psi$ IFTs that seem to yield the tighter bound for the revised Second Laws of information thermodynamics without detailed balance.