⟨⟩

# Nonequilibrium thermodynamics of uncertain stochastic processes

Jan Korbel  Section for Science of Complex Systems, CeMSIIS, Medical
University of Vienna, Spitalgasse 23, 1090 Vienna, Austria
<EMAIL_ADDRESS>Complexity Science Hub Vienna, Josefstädter
Strasse 39, 1080 Vienna, Austria  David H. Wolpert  Santa Fe Institute, Santa
Fe, NM, USA<EMAIL_ADDRESS>http://davidwolpert.weebly.com
Complexity Science Hub Vienna, Josefstädter Strasse 39, 1080 Vienna, Austria
Arizona State University, Tempe, AZ, USA  International Center for Theoretical
Physics, Trieste, Italy  Albert Einstein Institute for Advanced Study, New
York, USA

###### Abstract

Stochastic thermodynamics is formulated under the assumption of perfect
knowledge of all thermodynamic parameters. However, in any real-world
experiment, there is non-zero uncertainty about the precise value of
temperatures, chemical potentials, energy spectrum, etc. Here we investigate
how this uncertainty modifies the theorems of stochastic thermodynamics. We
consider two scenarios: in the (called _effective_) scenario we fix the
(unknown, randomly generated) experimental apparatus and then repeatedly
observe (stochastic) trajectories of the system for that fixed apparatus. In
contrast, in a (called _phenomenological_) scenario the (unknown) apparatus is
re-generated for each trajectory. We derive expressions for thermodynamic
quantities in both scenarios. We also discuss the physical interpretation of
effective (scenario) entropy production (EP), derive the effective mismatch
cost, and provide a numerical analysis of the effective thermodynamics of a
quantum dot implementing bit erasure with uncertain temperature. We then
analyze the protocol for moving between two state distributions that maximize
effective work extraction. Next, we investigate the effective thermodynamic
value of information, focusing on the case where there is a delay between the
initialization of the system and the start of the protocol. Finally, we derive
the detailed and integrated fluctuation theorems (FTs) for the
phenomenological EP. In particular, we show how the phenomenological FTs
account for the fact that the longer a trajectory runs, the more information
it provides concerning the precise experimental apparatus, and therefore the
less EP it generates.

##  I Introduction

The microscopic laws of classical and quantum physics are parameterized sets
of equations that specify the evolution of a closed system starting from a
specific state. To use those equations, we need to know that specific state,
we need to be sure the system is closed, and we need to know the values of the
parameters in the equations [1, 2].

Unfortunately, in many real-world scenarios, we are uncertain about the
precise state of the system, and very often, the system is open rather than
closed, subject to uncertain interactions with the external environment.
Statistical physics accounts for these two types of uncertainty by building on
the microscopic laws of physics in two ways. First, to capture uncertainty
about the state of the system, we replace the exact specification of the
system’s state with a probability distribution over states. Second, to capture
uncertain interactions between the system and the external environment, we add
randomness to the dynamics in a precisely parameterized form 111We note though
there is a substantial literature which adopts the “inclusive” framework, in
which the external environment is finite, and the joint dynamics of the
system-environment is explicitly modeled. This framework is the classical form
of “open quantum thermodynamics”. The inclusive framework has been explored
both for explicit Hamiltonian dynamics over the joint system, where the only
randomness in the initial state [49, 50, 51], and for approximate Hamiltonian
dynamics [52, 53, 54, 55, 56]..

In particular, in the sub-field of classical stochastic thermodynamics [1, 2]
we model the system as a probability distribution evolving under a continuous-
time Markov chain (CTMC) with a precisely specified rate matrix. Often in this
work, we require that the CTMC obeys local detailed balance (LDB). This means
that the rate matrix of the CTMC has to obey certain restrictions, which are
parameterized by the energy spectrum of the system, the number of
thermodynamic reservoirs in the external environment perturbing the system’s
dynamics, and the temperatures and chemical potentials of those reservoirs.
Often we also allow both the Hamiltonian of the system and the rate matrix of
the associated CTMC to change in time in a deterministic manner, perhaps
coupled by LDB. That joint trajectory is referred to as a “protocol”.

However, in addition to uncertainty about the state of the system and
uncertainty about interactions with the external environment, there is an
additional unavoidable type of uncertainty in all real-world systems:
uncertainty about the parameters in the equations governing the dynamics. In
the context of stochastic thermodynamics, this means that even if we impose
LDB, we will never know the reservoir temperatures and chemical potentials to
infinite precision (often even being unsure about the number of such
reservoirs), we will never know the energy spectrum to infinite precision, and
more generally, we will never know the rate matrix and its time dependence to
infinite precision.

At present, almost nothing is known about the thermodynamic consequences of
this third type of uncertainty despite its unavoidability 222We note though
that it is already known that if we do not account for all thermodynamic
reservoirs, we invariably under-estimate the total entropy production in a
process [57].. In this paper, we start to fill in this gap by considering how
stochastic thermodynamics (and non-equilibrium statistical physics more
generally) needs to be modified to account for this third type of uncertainty,
in addition to the two types of uncertainty it already captures.

We define an _apparatus_ $\alpha\in A$ to be any specific set of values of the
thermodynamic parameters of an experiment, including the number of reservoirs,
their temperatures and chemical potentials, the precise initial distribution
over states (i.e., how the system was prepared) the (deterministic
trajectories of the) rate matrices, the (deterministic trajectories of the)
energy functions, etc. Here and throughout, we assume that these thermodynamic
parameters are appropriately related by LDB for any specific $\alpha$. For
simplicity, we also assume that for all apparatuses, the system has the same
state space, $X$. Also for simplicity, we assume that all non-protocol
components of an apparatus (in particular the temperatures and chemical
potentials) do not change in time. In addition, we assume that for all
$\alpha$, the process takes place in the same time interval, $[t_{i},t_{f}]$.
We write an element of $X$ as $x$, and a trajectory of $X$ values across
$[t_{i},t_{f}]$ as ${\pmb{x}}$.

We suppose that $\alpha$ is not precisely known and write its probability
measure as $\mathrm{d}P^\alpha$. Physically, it may be that we have an
infinite set of apparatuses generated by IID sampling $\mathrm{d}P^{\alpha}$.
Alternatively, $\mathrm{d}P^{\alpha}$ could represent Bayesian uncertainty or
a detailed model of the noise in the measuring instruments used to set the
parameters in $\alpha$. (Below, we will often abuse notation/terminology and
refer to a “distribution” over apparatuses when properly speaking, we should
be couching the discussion in terms of a probability measure.) Abusing
notation, we will use $A$ to denote both the random variable with values
$\alpha$, and the event space of that random variable 333Note that $\alpha$ is
a generic characterization of apparatuses, which can be formalized as a vector
with some components finite-valued, some countable, and some uncountable, with
the measure $\mathrm{d}P^\alpha$ defined appropriately..

Concretely, we consider two kinds of experimental scenarios. Both start by
sampling $\mathrm{d}P^{\alpha}$, but they differ after that:

  1. I)

In the effective scenario, we generate an apparatus by sampling
$\mathrm{d}P^\alpha$. For that fixed apparatus we then generate many
stochastic trajectories ${\pmb{x}}$. After running all those trajectories for
that fixed apparatus, we can, if we wish, rerun the scenario, generating
another sample of the distribution over apparatuses, which we then use to
generate a new set of stochastic trajectories. We call this the _effective_
scenario.

Experimentally, in the effective scenario, one can generate and then observe
frequency counts of the distribution $p({\pmb{x}}|\alpha)$ for multiple random
values of $\alpha$, but without ever directly observing $\alpha$. For example,
the experimenter might construct a bit-eraser experimental apparatus involving
a single thermal reservoir whose temperature is fixed throughout the
experiment but only known to the finite precision of $.1^{\circ}K$. The
experimenter then runs their experiment many times using this fixed apparatus
and collected statistics concerning the trajectories across those experiments.
They can then use those estimates to make (perhaps Bayesian) estimates of
thermodynamic functions of a trajectory, like the associated entropy
production.

  2. II)

In the phenomenological scenario, we again generate an apparatus by sampling
$\mathrm{d}P^\alpha$, but the apparatus cannot be fixed while we generate
multiple trajectories. Instead, in order to generate a new trajectory we must
first generate a new apparatus by resampling $\mathrm{d}P^\alpha$. We call
this the _phenomenological_ scenario.

To illustrate the phenomenological scenario we can return to the example where
the experimenter constructs a bit-eraser experimental apparatus involving a
single thermal reservoir whose temperature at the beginning of the experiment
is only known to some finite precision of $.001^{\circ}K$. Suppose though that
the temperature is very slowly drifting randomly in time. Any given run of the
experiment is very fast on the timescale of that drift, so we can treat the
temperature as fixed throughout the run. However, after generating a
trajectory by running the experiment, it takes a long time for the system to
be reinitialized to rerun the experiment, and during that time the temperature
has drifted to a new value that is statistically independent of the value
during the preceding run. As in the effective scenario, the experimenter runs
their experiment many times and collected statistics concerning (functions of)
the trajectories across those experiments. However, in the phenomenological
scenario, one can only observe frequency counts of the $\alpha$-averaged
distribution over trajectories, $\bar{p}({\pmb{x}}):=\int\mathrm{d}P^\alpha
p({\pmb{x}}|\alpha)$.

Figure 1: Comparison of the two uncertain apparatus scenarios considered in
this paper. a) A simple, a three-state system that can be coupled to one of
the three apparatuses, with respective probabilities. b) We will plot
trajectories of the values of a quantity $E$ that has three possible values
across the time interval $[t_{i},t_{f}]$, along with the associated empirical
estimate of the relative probabilities that the system had each of those three
values during $[t_{i},t_{f}]$ c),d),e) Plots for the effective scenario, where
we can estimate the marginal distribution at time $t_{f}$
$p_{t_{f}}(E|\alpha)$ for a fixed apparatus $\alpha$, shown for three separate
instances of the scenario corresponding to the three possible apparatuses. f)
The phenomenological scenario, in which each trajectory is sampled with a
different apparatus, and therefore only $\bar{p}_{t_{f}}(E)$ can be estimated.

Illustrations of both scenarios are depicted in Fig. 1, for a simple three-
state time-homogeneous system coupled to one of the three possible
apparatuses. In each case, we measure the marginal probability distribution at
the final time $t_{f}$. Note that in neither scenario do we allow any direct
measurement of $\alpha$. However, there may be indirect information about
$\alpha$ that arises from the precise trajectory of states that is generated
once $\alpha$ is chosen.

Crucially, the ensemble-level thermodynamic quantities generated in these two
scenarios can differ, since the two types of average involved (once over
$\alpha$, once over $x$) do not necessarily commute. As an example, in the
effective scenario, since we can form an estimate of $p_{t}(x|\alpha)$ by
running many iterations of a fixed experimental apparatus, we can
experimentally estimate the entropy defined as

$\displaystyle\bar{S}(P_{t})=-\int\mathrm{d}P^\alpha\sum_{x}p_{t}(x|\alpha)\ln
p_{t}(x|\alpha)$ (1)

using empirical frequency counts.

This is not possible in the phenomenological scenario, in which we can only
experimentally estimate a more “coarse-grained” version of entropy,

$\displaystyle S(\bar{P}_{t})=-\sum_{x}\left(\int\mathrm{d}P^\alpha
p_{t}(x|\alpha)\right)\ln\left(\int\mathrm{d}P^\alpha p_{t}(x|\alpha)\right)$
(2)

using empirical frequency counts.

Note also that if we are in the effective scenario and have the ability to
force a new apparatus to be (randomly) generated whenever we want, then we can
implement the phenomenological scenario just by forcing a new apparatus to be
generated after every run. In this augmented version of the effective
scenario, we could experimentally estimate the quantity in Eq. 2 using
empirical frequency counts. However, if we are in the effective scenario and
do not have this extra ability, then we cannot estimate the quantity in Eq. 2,
only the quantity in Eq. 1. (In this paper, whenever we discuss the effective
scenario, we will assume we do not have this extra ability.)

The difference between the thermodynamics of the two scenarios will be a
central focus of our analysis below.

### Related research

It is important to distinguish between the focus of this paper and some of the
issues that have been investigated in the recent literature. Some recent
research has considered how to modify stochastic thermodynamics if the
experimentalist is not able to view all state transitions in the system as it
evolves [6, 7]. The uncertainty in these papers concerns what is observed as
the system evolves, whereas we focus on uncertainty in the parameters
governing that evolution. Similarly, some models consider either spatial [8]
or temporal [9] variation of temperature and other parameters, but they assume
that this evolution is known. In contrast, we assume that $\alpha$ is fixed
throughout the interval, but to an unknown value.

Probably the closest research to what we consider in this paper is sometimes
called _superstatistics_. It has long been known that an average over Gibbs
distributions cannot be written as some single Gibbs distribution (Thm. 1 in
[10]). This means that even equilibrium statistical physics must be modified
when there is uncertainty in the temperature of a system. The analysis of
these modifications was begun by Beck and Cohen [11], who developed an
_effective theory_ for thermodynamics with temperature fluctuating in time.
They considered a system coupled to a bath, which is in a _local equilibrium_
under the slow evolution of the temperature of the bath. The main assumption
they exploit is _scale-separation_ : while for short time scales, the
distribution over states of the system is an equilibrium, canonical
distribution with inverse temperature $\beta$, the long-scale behavior is
determined by a _superposition_ of canonical distributions with some
distribution of temperatures $f(\beta)$. The resulting _superstatistical_
distribution $p(E)=\int\mathrm{d}\beta f(\beta)\exp(-\beta E)/Z(\beta)$ was
later identified with the distribution corresponding to generalized entropic
functionals [12, 13] due to the fact that particular generalized entropic
functionals are maximized by the same distribution that can be obtained the
superposition of the canonical distribution with given $f(\beta)$ [14, 15].

Later interpretations of superstatistics are not based on the notion of local
equilibria but rather on the Bayesian approach to systems with uncertain
temperature [16, 17]. These are conceptually closer to the focus of this
paper, which focuses on off-equilibrium systems that are evolving quickly on
the scale of the coupling with the thermal reservoirs, and so cannot be
modeled in terms of time-scale separation.

Similar to the quasi-equilibrium scenarios considered in superstatistics,
other research has focused on deriving an _effective_ description of the
system in local equilibrium averaged over uncertain thermodynamic parameters.
In particular, this is the basis of a very rich and well-studied approach to
analyzing spin glasses [18, 19], in which the coupling constants $J_{ij}$ in
the spin-glass Hamiltonian $H=-\sum_{(ij)}J_{ij}s_{i}s_{j}$, are random
variables drawn from a given distribution $p(J_{ij})$. Given such a
distribution, the famous replica trick $\overline{\ln Z}=\lim_{n\to
0}\frac{Z^n-1}{n}$ [20] can be used to calculate the Helmholtz free energy,
averaged over all $J_{ij}$. Let us note that in the terminology used in
disordered systems, the annealed disorder corresponds to the effective
scenario while quenched disorder corresponds to the phenomenological scenario.

Finally, several authors [21, 22, 23, 24] investigated the case where the
initial distribution differs from the one that would minimize EP. It describes
the situation when the system is designed by a scientist who was mistaken in
their assumption concerning the initial distribution. In this case, the choice
of the non-optimal solution generates the extra entropy production that can be
described by the so-called mismatch cost. These papers do not involve a
distribution different initial distribution that is re-sampled each time the
experiment is re-run.

### Roadmap

One of the major themes of our investigation is that some of the details of
how an experiment is conducted that experimenters currently do not consider in
fact have major effects on the precise forms of various thermodynamic
quantities. This is reflected in the difference between (the thermodynamics
of) the effective and phenomenological scenarios, discussed above. Even within
the effective scenario though, there are some important distinctions between
different ways of running the experiment (and so different ways of defining
thermodynamic quantities). In particular, there is a major effect on the
thermodynamics itself that arises from whether the experimenter the protocol
(time-dependent trajectory of Hamiltonians of the system) changes from one run
of an experiment to the next, or instead is fixed in all runs. We call these
the “unadapted” and “adapted” situations, respectively. We start in Section II
with a simple illustrative example of these two situations, involving a moving
optical tweezer with uncertain stiffness parameter.

We then begin our more general analysis. First, in Section III, we introduce
the necessary notation and briefly recall the main results of traditional,
full-certainty stochastic thermodynamics. In Section IV, we present the
general form that stochastic thermodynamics takes in the effective scenario
(recall the discussion of the effective and phenomenological scenarios in the
introduction). We begin by noting that the evolution of the effective
probability distribution is not Markovian. Then we derive the forms of the
first and second laws of thermodynamics for effective thermodynamic
quantities. Next we discuss the relation between effective EP and effective
dissipated work. We illustrate this discussion with the numerical example of a
fermionic bit erasure with uncertain temperature. We end this section by
investigating the special case where the only uncertainty concerns the initial
distribution, calculating the associated effective mismatch cost.

In Section VI, we focus on (feedback) control protocols for uncertain
apparatuses. In contrast to the conventional case where the apparatus is
precisely known, we assume we cannot tailor the protocol for each (uncertain)
apparatus separately, but instead must use the same protocol for all
apparatuses. We use this setting to investigate how apparatus uncertainty
affects a foundational concern of stochastic thermodynamics: How much work can
be extracted from a system during a process that takes it from a given initial
distribution to a given target distribution.

First, we consider this issue when we are uncertain both about the initial
distribution (though not the final one) and about the temperature of the
system as it evolves. We focus on how that uncertainty changes the results of
the standard analysis of this issue, in which we suppose a {quench;
equilibrate; semi-statically-evolve} process is applied to the system
immediately after the initial distribution is generated.

Next, we use this analysis to consider how uncertainty affects the
“thermodynamic value of information” to a feedback controller [25, 26]. We
restrict attention to the special case of the analysis where the temperature
is known exactly, so the only uncertainty is in the initial distribution. We
also suppose that there is a (perfectly known) delay between when the initial
distribution is generated, $t_{i}$, and the time $\tau$ when the {quench;
equilibrate; semi-statically-evolve} process can begin, during which time the
system evolves according to a (perfectly known) rate matrix. In particular, we
derive expressions for how the thermodynamic value of information varies with
the length of the delay.

In Section VII, we investigate the ensemble entropy production calculated from
effective trajectory probabilities, i.e., from trajectory probabilities given
by averaging over apparatuses. We call this the _phenomenological (ensemble)
EP_. We begin by proving that phenomenological ensemble EP is a lower bound on
the average over apparatuses of the effective ensemble EP. So fixing the
apparatus and averaging over the trajectories — though without knowing what
value the apparatus is fixed to — and then averaging over apparatuses
increases EP, compared to the case where we average apparatuses before
averaging trajectories.

The difference between effective EP and phenomenological EP is called
_likelihood EP_. It measures the difference between log-likelihood functions
estimated from the forward and time-reversed trajectories.

Considering trajectory versions of all three EPs, we establish three detailed
fluctuation theorems (DFT). In addition to the well-studied DFT in the
literature which concerns a single, known apparatus, we establish the DFT for
the phenomenological EP and for likelihood EP. The former represents the
effective irreversibility of the system by coarse-graining all the
apparatuses. The latter represents how irreversibility affects the estimation
of the apparatus’ parameters when estimated by observing the forward and time-
reversed trajectories. These results are illustrated by a simple example of a
two-state system coupled to one heat reservoir with uncertain temperature.

The paper ends with a discussion section in which we describe just a few of
the myriad directions for future work.

##  II Illustrative example

In this section we illustrate the importance of accounting for the uncertainty
of the system parameters in an experiment, with a simple example of a
colloidal particle in a moving laser trap. The dynamics of the particle is
given by the overdamped Langevin equation

$\dot{x}=-\mu\frac{\partial V}{\partial x}+\xi$

where $\xi$ is the white noise, and $V$ is the potential. Let us consider that
the particle is dragged by an optical tweezer with the harmonic potential

$V_{k}(x,t)=\frac{k}{2}(x-\lambda(t))^2$

where $k$ is the stiffness parameter and $\lambda(t)$ is the control protocol.
The average work is given by the Sekimoto formula

$W[\lambda(t)]=\int_{0}^{t_{f}}\mathrm{d}t\dot{\lambda}\left\langle\frac{V_{k}(\lambda(t),x(t))}{\partial\lambda}\right\rangle$

where $\langle..\rangle$ is the ensemble average. Let us consider $\mu=1$. Our
aim is to move the trap from $\lambda_{i}=0$ at time $t_{i}=0$ to
$\lambda_{f}$ at time $t_{f}$ such that the average work is minimal. Following
[27], it is possible to express the optimal protocol starting that minimizes
the average work as

$\lambda_{k}^\star=\frac{\lambda_{f}(1+kt)}{2+kt_{f}}$

and the corresponding optimal work as

$\displaystyle W_{k}^\star=\frac{k\lambda_{f}^2}{2+kt_{f}}$ (3)

The complete derivation is done in Appendix A.

We focus on the realistic situation where the experimenter has to measure the
stiffness parameter to be able to determine the optimal protocol. The
estimation is typically done by repeated measurement of $k$, which leads to a
histogram of $k$. In practice, often an experimenter will implicitly assume
that the uncertainty in $k$ is due to the measurement and takes the average
value of stiffness $\bar{k}$ as the single possible value. However, often the
uncertainty in the parameters can have a physical reason, e.g., imprecise
calibration of the laser. In those kinds of scenarios the stiffness can change
for each run of the experiment, and so the experimenter’s implicit assumption
is invalid.

Write the stiffness parameter that the experimenter uses to set up the control
protocol as $k$, with the real stiffness parameter written as $\kappa$ (which
in general differs from $k$). In Appendix A we show that the work can then be
expressed as

$\displaystyle W_{\kappa}[\lambda_{k}(t)]$ $\displaystyle=$ $\displaystyle
W^\star_{k}$
$\displaystyle+\frac{\lambda_{f}^2}{(2+kt_{f})^2}\left(\frac{\kappa^2-k^2}{\kappa}+\frac{(k-\kappa)^2}{\kappa}e^{-\kappa
t_{f}}\right)$

using the definition in Eq. 3.

Let us now assume that the experimenter repeats the experiment many times. In
each run, the stiffness parameter $\kappa$ is drawn from a distribution
$p(\kappa)$. We want to compare two broad types of scenario. In the first type
of scenario, the experimenter re-measures the stiffness parameter before each
run of the experiment, so that they can adapt their protocol in that run to
the new value of the stiffness parameter in order to minimizes the average
work for that run. This is an example of what we call an _adapted_ scenario
below, in Section V. The average work in this scenario can be expressed as

$W_{ad.}=\int\mathrm{d}\kappa W_{\kappa}[\lambda_{\kappa}(t)]$

In the second scenario, the experimenter ignores the uncertainty in stiffness,
and for each run, they assume that the system is described by the same
stiffness parameter $\bar{\kappa}=\int\mathrm{d}\kappa p(\kappa)\kappa$. This
is an example of what we call an _unadapted_ scenario below, in Section V. The
minimal expected work in this unadapted scenario can be expressed as

$W_{unad.}=\int\mathrm{d}\kappa W_{\kappa}[\lambda_{\overline{\kappa}}(t)]$

Recall that the “dissipated work” of a particular protocol is the difference
between the work it expended and the minimal work that would have been
expended by an optimal protocol. Dissipated work is one of the central
concerns in non-equilibrium statistical physics. Because the “minimal work”
differs between the adapted and unadapted scenarios, we would expect that the
dissipated work does as well. We illustrate this for the current case of a
particle in a trap in Fig. 2. To generate that figure we chose $p(\kappa)\sim
LogNormal(\mu,\sigma)$, where $\mu$ and $\sigma$ are the mean and the variance
of the log-normal distribution 444N.b., this is in contrast with what is
arguably the most common parametrization, where the parameters are the mean
and the variance of the corresponding normal distribution. The transformation
between these two parametrizations is given in Appendix A.. The parameters of
the lognormal are $\lambda_{f}=1$, $t_{f}=1$,
$\mu\equiv\bar{\kappa}=\frac{3}{2}$. Fig. 2 demonstrates the general fact that
the adapted and unadapted scenarios can differ in their thermodynamics; with
the increasing variance of the distribution, the difference is more pronounced
(Note that both adapted and unadapted work decrease with increasing $\sigma$
which is caused by the fact that the expected work is in the case a concave
function of stiffness).

Note that choosing the protocol $\lambda_{\overline{\kappa}}(t)$ in this
scenario is not the optimal choice that minimizes the average work. (The issue
of protocols that optimize the average work are discussed in Section VI.)

Figure 2: The comparison of the average work for an unadapted and adapted
scenario for the moving optical tweezer with uncertain stiffness.

##  III Preliminaries

For simplicity, throughout this paper, we assume the system of interest has a
countable state space $X$ with elements generically written as $x\in X$. A
trajectory of values of $X$ across some non-infinitesimal time interval
$[0,t]$ will be written as ${\pmb{x}}_{t}$, or just ${\pmb{x}}$ for short. The
trajectory’s state at a specific time $t$ is denoted as ${\pmb{x}}(t)$.
Throughout this paper, we will often leave the time $t$ implicit. The
Kronecker delta function with arguments $x,x^{\prime}$ is written as
$\delta_{x,x^{\prime}}$ and $D_{x}(p||q)=\sum_{x}p(x)\ln\frac{p(x)}{q(x)}$ is
the Kullback-Leibler (KL) divergence between distributions $p$ and $q$, also
known as the relative entropy [29]. When the KL divergence is also averaged
over apparatuses, we denote it as
$D_{x,\alpha}(p||q)=\int\mathrm{d}P^\alpha\sum_{x}p(x|\alpha)\ln\frac{p(x|\alpha)}{q(x|\alpha)}$.
Throughout this paper, capital letters indicate random variables that are
averaged over, while lowercase letters indicate specific values of such random
variables.

###  III.1 Instance-level, trajectory-level and ensemble-level quantities

A generic function $y_{t}(x)$ depending on state $x$ (and possibly on time
$t$) is called an _instance-level_ value. Instance-level values are denoted by
lowercase letters. Given a distribution over the states $p_{t}(x)$, we write

$\displaystyle Y_{t}:=\langle y_{t}\rangle=\int\mathrm{d}x\,p_{t}(x)y_{t}(x)$
(4)

for the expectation of the instance value $y_{t}(x)$ under a distribution
$p_{t}(x)$. We refer to such expected values specified by a particular $t$ as
_ensemble_ values, and denote them in uppercase letters. Also, we will often
leave both the distribution and the time $t$ implicit when they are obvious
from context.

We refer to quantities specified in terms of a sequence ${\pmb{x}}$ of
instance values over a time interval $[t_{0},t]$ as _trajectory-level_ ,
typically denoted as $\pmb{y}_{[t_{0},t]}(\pmb{x})$. When the initial time
$t_{0}$ is fixed, we use only $\pmb{y}_{t}(\pmb{x})$. Trajectories of values
are denoted by bold lowercase letters, with an index $t$ indicating the ending
time of that trajectory. Given a distribution over trajectories
$\mathbb{P}({\pmb{x}})$, we denote the expected value of an associated
sequence as the ensemble(-trajectory) value

$\displaystyle\pmb{Y}_{t}:=\boldlangle\pmb{y}_{t}\boldrangle=\int_{t_{0}}^t\mathcal{D}{\pmb{x}}\,\mathbb{P}({\pmb{x}})\pmb{y}_{t}(\pmb{x}).$
(5)

where $\mathcal{D}{\pmb{x}}$ denotes the path integration over all
trajectories.

###  III.2 Brief review of stochastic thermodynamics

We now quickly review conventional stochastic thermodynamics, in which there
is no uncertainty about the thermodynamic parameters [2, 1, 30]. A more
extensive review can be found in Appendix B. Readers already comfortable with
stochastic thermodynamics can skip to the next section.

Conventional stochastic thermodynamics considers a system with a specified
Hamiltonian that is coupled to $N$ independent infinite thermal reservoirs,
which have associated inverse temperatures $\beta^\nu$, This coupling between
the systems and the reservoirs results in the system evolving according to a
CTMC, where as described below, the physical requirement of microreversibility
puts constraints on how the rate matrix of the CTMC is related to the physical
parameters of the process. Below we go through the main definitions of
relevant quantities used in stochastic thermodynamics. Their explicit formulas
as well as the extension to the case of particle reservoirs are shown in
Appendix B.

#### Ensemble thermodynamics

The ensemble internal energy is written as $U_{t}:=\langle
u_{t}\rangle=\sum_{x}p_{t}(x)u_{t}(x)$. The system exchanges energy with each
of the reservoirs (e.g., via kinetic molecular collisions). Additionally, one
can consider that the system exchanges particles with some of the reservoirs.
This extension is discussed in Appendix B and used later in one of the
examples. _The first law of thermodynamics_ can be formulated as

$\Delta U_{t}=\pmb{Q}_{t}+\pmb{W}_{t}$ (6)

where $\Delta U_{t}:=U_{t}-U_{t_{i}}$
$\pmb{Q}_{t}:=\int_{t_{i}}^t\mathrm{d}t^{\prime}\dot{\pmb{Q}}_{t^{\prime}}=\int_{t_{i}}^t\mathrm{d}t^{\prime}\sum_{x}\dot{p}_{t^{\prime}}(x)u_{t^{\prime}}(x)$
is total heat flow into the system during the interval $[t_{i},t]$, and
$\pmb{W}_{t}:=\int_{t_{i}}^t\mathrm{d}t^{\prime}\dot{\pmb{W}}_{t^{\prime}}=\int_{t_{i}}^t\mathrm{d}t^{\prime}\sum_{x}p_{t^{\prime}}(x)\dot{u}_{t^{\prime}}(x)$
is the total work on the system during that interval. The heat flow rate
$\dot{\pmb{Q}}_{t}$ can be decomposed into heat flows in from the separate
reservoirs, i.e., direct energy flows in from the separate reservoirs:
$\dot{\pmb{Q}}_{t}=\sum_{\nu}\dot{\pmb{Q}}^\nu_{t}$.

The entropy rate can be decomposed as

$\displaystyle\Delta S_{t}=\pmb{\Sigma}_{t}+\pmb{\mathcal{E}}_{t}$ (7)

where $\Delta S_{t}=S_{t}-S_{t_{i}}$, $\pmb{\Sigma}_{t}$ is the entropy
production (EP) and $\pmb{\mathcal{E}}_{t}$ is the entropy flow (EF). _The
second law of thermodynamics_ is enforced by the fact that for any rate
matrix, EP rate is non-negative, i.e., $\dot{\pmb{\Sigma}}_{t}\geq 0$ and so
${\pmb{\Sigma}}_{t}\geq 0$.

When LDB holds, the EF rate can be expressed in terms of _thermodynamic
entropy_ , i.e.,

$\displaystyle\dot{\pmb{\mathcal{E}}}_{t}=\sum_{\nu}\beta^\nu\dot{\pmb{Q}}^{\nu}_{t}\,.$
(8)

In this case, Eq. 7 means that

$\displaystyle{\pmb{\Sigma}}_{t}=\Delta S_{t}-\beta{\pmb{Q}}_{t}$ (9)

if there is only a single heat bath, with inverse temperature $\beta$ By Eq. 6
this can be written as

$\displaystyle\dfrac{{\pmb{\Sigma}}_{t}}{\beta}=\pmb{W}_{t}-\left(\Delta
U_{t}-\dfrac{\Delta S_{t}}{\beta}\right)=\pmb{W}_{t}-\Delta F_{t}$ (10)

where $\Delta F_{t}$ is the difference in the Helmholtz free energy between
time $0$ and time $t$.

Suppose we wish to minimize the work expended over all possible protocols
$(u_{t},K_{t})$ acting on such a system, so long as they take some specified
initial pair of an energy function and distribution
$(u^*_{t_{0}},p^*_{t_{0}})$ to some specified ending pair,
$(u^*_{t_{f}},p^*_{t_{f}})$. For that specified pair, $\Delta U_{t_{f}}$ is
fixed, independent of the protocol. So by Eq. 6 the minimal such work is
$\Delta U_{t_{f}}-\min_{(u_{t},K_{t})}\pmb{Q}^{(u_{t},K_{t})}$ where
$\pmb{Q}^{(u_{t},K_{t})}$ is the total heat over the interval $[t_{i},t_{f}]$
flowing into the system if it follows protocol $(u_{t},K_{t})$. By the second
law, that minimum is just $\Delta S_{t_{f}}/\beta$.

Plugging in, the minimal work over all protocols is $\Delta U_{t_{f}}-\Delta
S_{t_{f}}/\beta$. Therefore the quantity on the RHS of Eq. 10 is the
difference between the actual work expended to go from
$(u^*_{t_{0}},p^*_{t_{0}})$ to $(u^*_{t_{f}},p^*_{t_{f}})$ and the minimal
possible. Accordingly, that quantity is called the dissipated work, and by the
first law it can be written as a function of the initial distribution as
follows:

$\displaystyle\pmb{W}_{\mathrm{diss}}(p_{t_{i}}):=\dfrac{\Delta
S_{t_{f}}}{\beta}(p_{t_{i}})-\pmb{Q}_{t_{f}}(p_{t_{i}})$ (11)

In this way Eq. 10 connects a purely thermodynamic quantity to a purely
dynamic quantity, defined in terms of rate matrices. Note that the energy
function and distribution are treated specially in the definition of
dissipated work — no other parameters of the thermodynamic process are allowed
to vary when determining the “minimal” work.

Let us now define the prior initial distribution as the one that minimizes
that function when Eq. 11. We can consider the directional derivative of that
function, evaluated for any initial distribution, in the direction of any
other initial distribution. In particular, that directional derivative of the
dissipated work must equal $0$ when evaluated at the prior initial
distribution $q_{t_{i}}(x)$ in the direction of any other initial
distribution, $p_{t_{i}}(x)$. This can be used to show that

$\displaystyle\\!\\!\\!\\!\\!\\!\\!\\!\pmb{\Sigma}(p_{t_{i}})$
$\displaystyle=\pmb{W}_{\mathrm{diss}}(p_{t_{i}})$
$\displaystyle=\left[D(p_{t_{i}}||q_{t_{i}})-D(p_{t_{f}}||q_{t_{f}})\right]$
$\displaystyle\;\;\;+\pmb{\Sigma}^\alpha(q_{t_{i}})$ (12)

where the distributions $p_{t_{i}}(x)$ and $q_{t_{i}}(x)$ evolve into the
distributions $p_{t_{f}}(x)$ and $q_{t_{f}}(x)$, respectively.
$\pmb{\Sigma}(q_{t_{i}})$ is called the residual EP 555Formally, Eq. 12
assumes that the conditional distribution $P(x(t_{f})|x(t_{i}))$ has a single
“island”; the extension of the analysis here to the case of multiple islands
is straight-forward. See [21, 22, 23, 24].. The thermodynamic process
implemented by a given apparatus is thermodynamically reversible iff the
residual EP of the process is zero, and the initial distribution happens to
equal the prior of the process.

The drop in KL divergence in Eq. 12 is called the _mismatch cost_ of running
the process with initial distribution $p_{t_{i}}$. By the data-processing
inequality for KL divergence, the mismatch cost is nonzero, and by inspection,
it equals zero if the initial distribution equals the prior. So mismatch cost
is the extra EP generated by running the process with initial distribution
$p_{t_{i}}$ rather than the prior $q_{t_{i}}$, in addition to the residual EP
which would be generated if the process were run with the prior as the initial
distribution.

#### Trajectory thermodynamics

The _trajectory internal energy_ is written as $\pmb{u}_{t}({\pmb{x}})$. The
first law of thermodynamics on the trajectory level for any time $t$ is

$\frac{\mathrm{d}}{\mathrm{d}t}\pmb{u}_{t}({\pmb{x}})=\dot{\pmb{q}}_{t}({\pmb{x}})+\dot{\pmb{w}}_{t}({\pmb{x}})$
(13)

where
$\dot{\pmb{q}}_{t}({\pmb{x}})=\sum_{\nu}\dot{\pmb{q}}_{t}^\nu({\pmb{x}})$ is
the trajectory heat and
$\dot{\pmb{w}}_{t}({\pmb{x}})=\sum_{x}\delta_{x,{\pmb{x}}(t)}\dot{u}_{t}{(x)}$
trajectory work. Trajectory entropy is defined as $s_{t}({\pmb{x}}):=-\ln
p_{t}({\pmb{x}}(t))$. The time derivative of entropy can be decomposed as

$\displaystyle\frac{\mathrm{d}}{\mathrm{d}t}\pmb{s}_{t}({\pmb{x}})$
$\displaystyle=\dot{\pmb{\sigma}}_{t}({\pmb{x}})+\dot{\pmb{\epsilon}}_{t}({\pmb{x}})$
(14)

where $\dot{\pmb{\sigma}}_{t}({\pmb{x}})$ is the trajectory EP rate and
$\dot{\pmb{\epsilon}}_{t}({\pmb{x}})$ is the trajectory EF rate. It is
straightforward to verify that by averaging these trajectory-level quantities
over all trajectories, we recover the ensemble-level versions, i.e.,
$\boldlangle{\dot{\pmb{q}}}^\nu_{t}\boldrangle=\dot{\pmb{Q}}^\nu_{t}$,
$\boldlangle\dot{\pmb{\epsilon}_{t}}\boldrangle=\dot{\pmb{\mathcal{E}}}_{t}$,
and $\boldlangle\dot{\pmb{\sigma}}_{t}\boldrangle=\dot{\pmb{\Sigma}}_{t}$. Due
to LDB, trajectory EP can be expressed as

$\pmb{\sigma}({\pmb{x}})=\ln\frac{\pmb{P}({\pmb{x}})}{\pmb{P}^{\dagger}({\pmb{x}^{\dagger}})}$
(15)

where $\pmb{P}({\pmb{x}})$ is the probability of observing trajectory
${\pmb{x}}$ and $\pmb{P}^{\dagger}({{\pmb{x}^{\dagger}}})$ is the probability
of observing time-reversed trajectory
${\pmb{x}^{\dagger}}(t):={\pmb{x}}(t_{f}-t)$ under time-reversed protocol.
Defining
$P(\sigma):=\int\mathcal{D}{\pmb{x}}{\mathbb{P}}({\pmb{x}})\delta(\sigma-\pmb{\sigma}({\pmb{x}}))$,
it is straightforward to show that the trajectory EP $\sigma$ fulfills the
_detailed fluctuation theorem_ ,

$\frac{P(\sigma)}{{P}^{\dagger}(-\sigma)}=e^{\sigma}$ (16)

where $P^{\dagger}$ denotes the probability under a time-reversed protocol and
$\sigma$ is random variable [2]. Finally, ensemble EP can be expressed as
Kullback-Leibler divergence between probabilities of forward and reversed
trajectories [32]

$\pmb{\Sigma}=D_{{\pmb{x}}}(\pmb{P}({\pmb{x}})||\pmb{P}^{\dagger}({\pmb{x}^{\dagger}}))\,.$
(17)

##  IV Effective thermodynamics

As described in the introduction, we are interested in how the conventional
laws of stochastic thermodynamics concerning the evolution of a system change
when there is uncertainty about the parameters of that evolution, but we
assume LDB holds, whatever those parameters are.

In Section IV.1, we describe a general framework of effective quantities that
are averaged over different apparatuses. In Section V.2 we continue the
description of effective ensemble thermodynamics in the general case where the
transition rates of the systems are not known with infinite precision. Such
uncertainty must arise whenever we do not know the exact number of heat
reservoirs, their temperatures/and or chemical potentials. However,
uncertainty can even arise when we do know those quantities exactly, and even
when we impose LDB. This is because even if we knew those quantities to
infinite precision, LDB does not uniquely fix the rate matrix $K$, and so
there can still be uncertainty concerning $K$.

We exemplify the effective thermodynamics approach in Section IV.3 on the
example of a bit erasure for the case of fermionic bits. We start with the
brief review of system when coupled to a heat bath with certain temperature.
Then we continue with the generalized case when the system is coupled to a
heat bath with uncertain temperature.

###  IV.1 Effective quantities

Suppose we are given some generic quantity $Y^\alpha$, which depends on the
apparatus $\alpha$, and which may also depend on time and/or the random
trajectory through the system’s state space (where any of the dependencies may
be implicit). The probability distribution at time $t$ is denoted as
$p_{t}(x|\alpha)$. Trajectory probability is denoted as
${\mathbb{P}}({\pmb{x}}\,|\,\alpha)$.

We define the _effective_ value of $Y$ as its expectation over $\alpha$, and
write it as

$\displaystyle\overline{Y}$ $\displaystyle:=\int\mathrm{d}P^\alpha\,Y^\alpha$
(18)

where we require that the $\alpha$-dependence of all arguments of $Y^\alpha$
is made explicit. So for example, if $Y^\alpha$ depends on the initial
probability distribution over states, that distribution occurs in its argument
list as $p_{t_{0}}(x|\alpha)$. In particular, the effective state probability
at time $t$ is

$\displaystyle\overline{p}_{t}(x)$
$\displaystyle=\int\mathrm{d}P^{\alpha}\,p_{t}(x|\alpha)$ (19)
$\displaystyle=\int\mathrm{d}P^{\alpha}\,p_{t}^\alpha(x)$ (20)

where we use the shorthand $p_{t}^\alpha(x):=p_{t}(x|\alpha)$. We also write
the effective trajectory probability up to time $t$

$\overline{{\mathbb{P}}}_{t}({\pmb{x}})=\int\mathrm{d}P^{\alpha}\,{\mathbb{P}}_{t}({\pmb{x}}\,|\,\alpha)$
(21)

Note that in both scenarios I and II, since the apparatus is fixed throughout
$[t_{i},t_{f}]$ once it is sampled, the joint dynamics over $X\times A$ is
given by the master equation,

$\displaystyle\dot{\overline{p}}_{t}(x)$
$\displaystyle=\sum_{x^{\prime}}\int\mathrm{d}P^{\alpha^{\prime}}\,K^{\alpha\alpha^{\prime}}_{xx^{\prime}}p_{t}^{\alpha^{\prime}}(x^{\prime})$
(22)

where
$K^{\alpha\alpha^{\prime}}_{xx^{\prime}}=K^\alpha_{xx^{\prime}}\,\delta(\alpha^{\prime},\alpha)$.
Averaging both sides of Eq. 22 over $\alpha$ and interchanging the derivative
and average on the LHS, we get

$\displaystyle\dot{\overline{p}}_{t}(x)=\sum_{x^{\prime}}K_{xx^{\prime}}\overline{p}_{t}(x^{\prime})\,$
(23)

where

$\displaystyle
K_{xx^{\prime}}:=\int\mathrm{d}P^{\alpha}\,K^{\alpha\alpha^{\prime}}_{xx^{\prime}}p_{t}(x^{\prime}|\alpha^{\prime})$
(24)

So the dynamics over the system considered by itself is simply the dynamics of
a coarse-graining of the joint system-apparatus. Since the transition rate
matrix of that coarse-grained dynamics depends on the probability
distribution, the effective dynamics is not described by a linear Markov
master equation.

###  IV.2 Effective ensemble stochastic thermodynamics

We write $\pmb{W}^\alpha$ for the ensemble energetic work expended for a
specific apparatus $\alpha$ as it sends $p^\alpha_{t_{i}}$ to
$p^\alpha_{t_{f}}$. By conservation of energy

$\displaystyle\pmb{W}^\alpha=\Delta{U}^\alpha-\pmb{Q}^\alpha$ (25)

where the change in the internal energy for each apparatus $\alpha$ during the
process is

$\displaystyle\Delta U^\alpha$
$\displaystyle=\sum_{x}[p^\alpha_{t_{f}}(x)u^\alpha_{t_{f}}(x)-p^\alpha_{t_{i}}(x)u^\alpha_{t_{i}}(x)]$
(26)

In addition, by the second law, for any specific apparatus $\alpha$,

$\displaystyle\sum_{\nu}\beta^{\nu,\alpha}\pmb{Q}^{\nu,\alpha}\leq
S(p^\alpha_{t_{f}})-S(p^\alpha_{t_{i}})$ (27)

These quantities are given by integrating over time and fixing a specific
apparatus. If we instead average over apparatuses and fix a specific time, we
get the formulas for the effective ensemble energy and effective ensemble
entropy at time $t$:

$\displaystyle\overline{U}_{t}$
$\displaystyle=\int\mathrm{d}P^\alpha\sum_{x}p^\alpha_{t}(x)u_{t}^\alpha(x)$
(28) $\displaystyle\overline{S}_{t}$
$\displaystyle=-\int\mathrm{d}P^\alpha\sum_{x}p^\alpha_{t}(x)\ln
p_{t}^\alpha(x)$ (29)

If we run over both time and apparatuses, we get the formulas for the
effective work on the system and heat transferred to the reservoir(s) up to
time $t$:

$\displaystyle\overline{\pmb{W}}_{t}$
$\displaystyle=\int\mathrm{d}P^\alpha\int_{t_{i}}^t\mathrm{d}t^{\prime}\sum_{x}p_{t^{\prime}}^\alpha(x)\dot{u}_{t^{\prime}}^\alpha(x)$
(30) $\displaystyle\overline{\pmb{Q}}_{t}$
$\displaystyle=\int\mathrm{d}P^\alpha\int_{t_{i}}^t\mathrm{d}t^{\prime}\sum_{x}\dot{p}_{t^{\prime}}^\alpha(x)u_{t^{\prime}}^\alpha(x)$
(31)

In addition, the effective ensemble EF rate is

$\displaystyle\dot{\overline{\pmb{\mathcal{E}}}}_{t}$
$\displaystyle=\int\mathrm{d}P^\alpha\sum_{\nu}\beta^{\alpha,\nu}\dot{\pmb{Q}}^{\alpha,\nu}_{t}$
(32)
$\displaystyle=\int\mathrm{d}P^\alpha\sum_{xx^{\prime}\nu}K^{\alpha,\nu}_{x,x^{\prime}}p^\alpha_{t}(x^{\prime})\ln\dfrac{K^{\alpha,\nu}_{x,x^{\prime}}}{K^{\alpha,\nu}_{x^{\prime},x}}\,$
(33)

and
$\overline{\pmb{\mathcal{E}}}_{t}=\int_{t_{i}}^{t}\mathrm{d}t\,\dot{\overline{\pmb{\mathcal{E}}}}_{t}$.
Similarly, the effective ensemble EP is
$\overline{\pmb{\Sigma}}_{t}=\int_{t_{i}}^{t}dt\dot{\overline{\pmb{\Sigma}}}_{t}$,
where the effective ensemble EP rate is

$\displaystyle\dot{\overline{\pmb{\Sigma}}}_{t}$
$\displaystyle=\int\mathrm{d}P^\alpha\dot{\pmb{\Sigma}}^\alpha_{t}\geq 0$ (34)

Plugging in the expression for $\dot{\pmb{\Sigma}}^\alpha_{t}$ in terms of
rate matrices, we confirm that

$\displaystyle\dfrac{\mathrm{d}\overline{S}}{\mathrm{d}t}$
$\displaystyle=\dot{\overline{\pmb{\mathcal{E}}}_{t}}+\dot{\overline{\pmb{\Sigma}}}_{t}$
(35)

where

$\displaystyle\dot{\overline{\pmb{\Sigma}}}_{t}$
$\displaystyle=\int\mathrm{d}P^\alpha\sum_{xx^{\prime}\nu}K^{\alpha,\nu}_{x,x^{\prime}}p^\alpha_{t}(x^{\prime})\ln\dfrac{K^{\alpha,\nu}_{x,x^{\prime}}p_{t}(x^{\prime})}{K^{\alpha,\nu}_{x^{\prime},x}p_{t}(x)}$
(36)

Combining these definitions, we can write the first and second law of
thermodynamics for effective ensemble quantities as

$\displaystyle\Delta\overline{U}_{t}$
$\displaystyle=\overline{\pmb{Q}}_{t}+\overline{\pmb{W}}_{t}$ (37)
$\displaystyle\Delta\overline{S}_{t}$
$\displaystyle=\overline{\pmb{\Sigma}}_{t}+\overline{\pmb{\mathcal{E}}}_{t}$
(38)

respectively, simply by averaging the first and second laws over all $\alpha$.
Note in particular that since $\dot{\overline{\pmb{\Sigma}}}_{t}\geq 0$, the
total effective ensemble EP generated in the interval is non-negative. This is
why Eq. 38 can be identified as the effective scenario’s second law.

On the other hand though, Eq. 32 shows that in general the effective ensemble
EF rate cannot be expressed in terms of the expected effective heat flows for
the separate reservoirs,
$\dot{\overline{\pmb{Q}^{\nu}_{t}}}=\int\mathrm{d}P^\alpha\dot{\pmb{Q}}^{\nu,\alpha}_{t}$,
despite our assumption that LDB holds for each apparatus separately. This
discrepancy between the two rates would exist even if we knew with certainty
that there was only one reservoir, i.e., $N=1$; the discrepancy reflects the
fact that there can be statistical coupling between $\beta^\alpha$ (and
$K^{\alpha,\nu}$ and therefore) $\dot{\pmb{Q}}^\alpha_{t}$, due to the
uncertainty over $\alpha$. This discrepancy means that the time-derivative of
the effective ensemble entropy is not necessarily lower-bounded by the
ensemble effective heat flow rate. So the version of the effective scenario’s
second law of thermodynamics, Eq. 38 is not as consequential as the standard
version in which there is no uncertainty about $\alpha$.

###  IV.3 Example: Bit erasure of information stored in fermionic bits with
uncertain temperature

We now illustrate how environment uncertainty affects the design of an
experimenter’s protocol as well as the associated EP for the effective
scenario, using the example of bit erasure of a fermionic bit in finite time
[33]. First, we review the analysis when there is no uncertainty about
temperature. Then we extend that analysis by introducing uncertainty about the
temperature of the single heat bath, and therefore (in order to enforce LDB)
in the trajectories of the rate matrices and so of the energy function.

#### Bit erasure of information stored in fermionic bits

Our system has two states, labeled $0$ and $1$. The probabilities of those two
states are denoted as $p_{t}(1)\equiv p(t)$ and $p_{t}(0)\equiv 1-p(t)$
respectively. In this subsection, we will denote the dependence of a generic
quantity $x$ on time $t$ as $x(t)$ (rather than $x_{t}$) to clarify the
reasoning. In particular, the chemical potential of an electron in the quantum
dot is written as $\mu(t)$. (Recall that in Appendix B we discuss the
extension of stochastic thermodynamics to include chemical potentials.)

For simplicity, we will only consider variations in the trajectory of energy
functions across $t\in[t_{i},t_{f}]$, with a fixed map specifying the rate
matrix that goes with each possible energy function. Specifically, we set the
rate matrix at all times $t$ to

$K^{\beta}(t)=\left(\begin{array}[]{cc}K^\beta_{00}(t)&K^\beta_{01}(t)\\\
K^\beta_{10}(t)&K^\beta_{11}(t)\\\
\end{array}\right)=\left(\begin{array}[]{cc}-k^\beta(t)&1-k^\beta(t)\\\
k^\beta(t)&-(1-k^\beta(t))\\\ \end{array}\right)$ (39)

where

$k^\beta(t)=\frac{1}{1+\exp(\beta h(t))}$ (40)

and we use the shorthand $h(t)=u(t)-\mu(t)n_{1}$ for the difference between
the energy and the chemical potential times $n_{1}=1$. Since there is no
remaining freedom to vary the trajectory of rate matrices across
$t\in[t_{i},t_{f}]$ once we specify a trajectory of energy functions across
$t\in[t_{i},t_{f}]$, we will sometimes refer to that trajectory of energy
functions as a “control protocol”.

Given Eq. 39, we can write

$\displaystyle
K^\beta_{10}(t)(1-p^\beta(t))-K^\beta_{01}(t)p^\beta(t)=k^\beta(t)-p^\beta(t)$
(41)

and so

$\dot{p}^\beta(t)=k^\beta(t)-p^\beta(t)$ (42)

The solution is

$p^\beta(t)=e^{-t}\left(p(t_{i})+\int_{t_{i}}^t\mathrm{d}\tau e^\tau
k^\beta(\tau)\right)\,.$ (43)

From now on, for simplicity, we take $t_{i}=0$. Consider a special type of
control protocol chosen so that

$k^\beta(t)\equiv k(t)=(1-t/t_{f})p(0)+t/t_{f}\delta$ (44)

for all $t$, arbitrary $\delta>0$. Note that the RHS is independent of
$\beta$. Plugging in to Eq. 40, the associated control protocol is

$h^\beta(t)=\frac{1}{\beta}\ln\left(\frac{1+p(0)t+t\delta}{p(0)(t-1)-t\delta}\right).$
(45)

Similarly, plugging in to Eq. 43 shows that

$p(t)=e^{-t}\left(p(0)(2e^t-te^t-1)+\delta(1+te^t-e^t)\right)$ (46)

Note that despite the notation, this quantity does not depend on the
temperature. Finally, the EP is

$\displaystyle\pmb{\Sigma}^\beta\equiv\pmb{\Sigma}=\int_{0}^t\mathrm{d}\tau\dot{p}(\tau)\left(\ln\frac{1-p(\tau)}{p(\tau)}-\beta
h^\beta(\tau)\right)$
$\displaystyle\;=\int_{0}^t\mathrm{d}\tau\left[p(t)(1-k(t))\ln\left(\frac{p(t)(1-k(t))}{(1-p(t))k(t)}\right)\right.$
$\displaystyle\qquad\qquad\left.+(1-p(t))k(t)\ln\left(\frac{(1-p(t))k(t)}{p(t)(1-k(t))}\right)\right]$
(47)

Note that EP also does not depend on temperature, due to the choice of the
control protocol $h^\beta(t)$.

#### Bit erasure with temperature uncertainty

We now consider the variant of this bit erasure scenario where the
experimentalist does not know the temperature exactly but instead has some
distribution $\mathrm{d}P^\beta=p(\beta)\mathrm{d}\beta$. We suppose that the
experimenter is in full control of the control protocol, but that the same,
single protocol will be used for all apparatuses.

Averaging both sides of Eq. 43 gives the formula for the evolution of the
effective distribution:

$\displaystyle\overline{p}(t)=e^{-t}\left(p(t_{i})+\int_{t_{i}}^t\mathrm{d}\tau
e^\tau\overline{k}(\tau)\right)\ $ (48)

with

$\overline{k}(t)=\int\mathrm{d}\beta p(\beta)\,\frac{1}{1+\exp(\beta h^*(t))}$
(49)

where $h^*(t)$ is the protocol chosen by the experimentalist to be used for
all apparatuses. Note that this is the same evolution one would get by first
averaging both sides of Eq. 42,

$\dot{\overline{p}}(t)=\overline{k}(t)-\overline{p}(t)$ (50)

and then solving for $\overline{p}(t)$. This reflects the fact that in this
particular situation, where Eq. 41 holds, we can write
$\overline{K(t)p(t)}=\overline{K}(t)\cdot\overline{p}(t)$, and so the dynamics
of the effective distribution is Markovian (in contrast with the general
case).

Similarly to the no-uncertainty analysis, we assume the transition rate matrix
has the form

$\overline{k}(t)=(1-t/t_{f})p(0)+t/t_{f}\delta$ (51)

To this end, we should find a control protocol (independent of $\beta$) such
that Eq. 51 equals Eq. 46.

Next, define the function

$\Psi(h^*):=\int\mathrm{d}\beta p(\beta)\,\frac{1}{1+\exp(\beta h^*)}$ (52)

Thus, the control protocol $h^\star(t)$ can be obtained by solving the
equation for the transition rate $\bar{k}(t)=\Psi(h^*(t))$. By solving the
equation, we obtain

$h^*(t)=\Psi^{-1}\left((1-t)p(0)+t\delta\right)$ (53)

Note that $h^*(t)$ is not the $\beta$-average of $h^\beta(t)$ which can be
expressed as $\bar{h}(t)=h^{\tilde{\beta}}(t)$, where
$\tilde{\beta}=\left(\int\mathrm{d}\beta P^\beta 1/\beta\right)^{-1}$ is the
harmonic mean of beta (corresponding to the arithmetic mean of temperature).

In the actual experiment, however, the system is coupled to a bath with the
certain temperature $\beta$ and therefore the transition rate is

$k^{\beta}_{h^*}(t)=\frac{1}{1+\exp(\beta h^*(t))}$ (54)

Thus, the probability distribution of a system coupled to a heat reservoir
with temperature $\beta$ using the protocol $h^*$, can be obtained from the
equation

$\dot{p}_{h^*}^{\beta}(t)=k_{h^*}^\beta(t)-p_{h^*}^{\beta}(t)\,.$ (55)

As an illustration, consider the special case where there are two possible
temperatures. The first temperature $T_{1}=1/\beta_{1}$ occurs with
probability $P(T_{1})$ and the second temperature $T_{2}=1/\beta_{2}$ occurs
with probability $P(T_{2})=1-P(T_{1})$. Thus, the distribution $P(T)$ can be
expressed as

$P(T)=P(T_{1})\delta_{TT_{1}}+P(T_{2})\delta_{TT_{2}}\,.$ (56)

and therefore

$\overline{k}(t)=\frac{P(T_{1})}{1+\exp\left(\frac{h^*(t)}{T_{1}}\right)}+\frac{P(T_{2})}{1+\exp\left(\frac{h^*(t)}{T_{2}}\right)}$
(57)

As usual, the control protocol can be obtained by solving Eq. 53. Moreover,
since that protocol is independent of $T$, by definition the effective
ensemble entropy production is

$\bar{\pmb{\Sigma}}_{h^*}=P(T_{1})\,\pmb{\Sigma}^{T_{1}}_{h^*}+P(T_{2})\,\pmb{\Sigma}^{T_{2}}_{h^*}$
(58)

Figure 3: Bit erasure of a fermionic bit with uncertain temperature. The
design of the experiment is depicted in the top-left panel. For each site, the
bit is coupled to one of the metallic leads with different temperatures
$T_{1}=1$ and $T_{2}=10$ with equal probability. The control protocol for each
temperature and for the case of uncertain temperature is depicted in the top-
right panel. The actual probability distribution for a given temperature and
control protocol is depicted on the bottom-right panel. The entropy production
for the case of a certain temperature as well as an uncertain temperature is
depicted in the bottom-right panel.

In Fig. 3 we plot the control protocol $h(t)$, probability distribution
$p_{1}(t)$, and entropy production $\pmb{\Sigma}(t)$ for for this case of bit
erasure with uncertainty about which of two possible temperatures the bath
has. We assume parameters $T_{1}=1$, $T_{2}=10$, $t_{f}=10$ and $\delta=0.1$,
and take $P(T_{1})=P(T_{2})=1/2$.

In the top-right panel of Fig. 3 we plot $h^*(t)$ along with the control
protocols $h^T(t)$ for the two cases where there is no temperature
uncertainty, for the two possible values of $T=\\{T_{1},T_{2}\\}$. The
expected distribution $\bar{p}(t)$ is displayed in the bottom-left panel,
along with the two distributions $p^T_{h^*}$ which could actually occur in the
experiment, when the experimenter fixes the protocol to $h^*(t)$ and the
temperature of the bath is one of $T=\\{T_{1},T_{2}\\}$. Finally, we compare
the total EP for the case of certain temperature $\pmb{\Sigma}^T(t)$ (which
does not vary with $T$), with the case of total EP for the case of uncertain
temperature given by $\overline{\pmb{\Sigma}}_{h^*}$ in the bottom-right panel
of Fig. 3. Despite the fact that $p(t)$ coincides with $\bar{p}(t)$, the
effective ensemble EP increases to the case of EP for the case of a certain
temperature. For comparison, we depict $\pmb{\Sigma}^T_{h^*}$ for
$T=\\{T_{1},T_{2}\\}$.

The main reason why $\overline{\pmb{\Sigma}}_{h^*}$ is higher than
$\pmb{\Sigma}$ is the fact that for the situation when $T_{1}$ is chosen, the
protocol $h^*$ forces the distribution $p_{h^\star}^{T_{1}}(t)$ to decrease
much below $p(t)$, which on one hand makes the bit erasure more efficient (the
final probability distribution is closer to $(1,0)$), but one has to pay much
more dissipated work and consequently entropy production
$\pmb{\Sigma}_{h^*}^{T_{1}}$ to get the distribution closer to the ideal bit
erasure. This work is not compensated in the other case when the temperature
is equal to $T_{2}$. We see that while the entropy production
$\pmb{\Sigma}_{h^*}^{T_{2}}$ is lower than $\pmb{\Sigma}$, this does not
compensate enough the entropy production $\pmb{\Sigma}_{h^*}^{T_{1}}$ and the
average entropy production $\overline{\pmb{\Sigma}}_{h^*}$ is larger than
$\Sigma$ in the case when the temperature is certain. Let us finally mention
that this is a similar situation hidden Markov pump [34] when the actual EP
(here corresponding to $\pmb{\Sigma}$ calculated from the complete microscopic
structure of the hidden pump in lower than when the entropy is calculated for
a coarse-grained pump. I.e., the experimenter’s knowledge of the setup affects
the obtained EP. While in the case of [34] the experimenter lacks the precise
information about the system, there the experimenter lacks the precise
knowledge of the heat bath and its temperature.

##  V The two types of effective dissipated work

In much of the rest of this paper we focus on the case of a single bath.
Specifically, we investigate how the properties of dissipated work change from
those in Eq. 11 when we that number of baths but introduce uncertainty in the
other parameters .

It turns out that there (at least) two natural ways to extend the reasoning
that results in Eq. 11 to that case where the number of baths is fixed to one
but there is uncertainty about the other parameters defining the apparatus.
These two extensions reflect two different ways of defining “minimal possible
effective work”.

###  V.1 The adapted and unadapted scenarios

Write ${\hat{\alpha}_{t}}:=(u^\alpha_{t},K^\alpha_{t})$ to mean the components
of $\alpha$ that specify the protocol, evaluated at time $t$. Similarly write
$\underline{\alpha}_{t}$ to mean all components of $\alpha_{t}$ _other_ than
those that specify the protocol, e.g., the initial probability distribution,
the temperature, chemical potentials, and other thermodynamic forces, etc.
Thus, $\alpha_{t}$ can be decomposed as
$\alpha_{t}=({\hat{\alpha}_{t}},\underline{\alpha}_{t})$. Recall though that
for simplicity we are assuming that all non-protocol components of $\alpha$
are time-independent, so we can simplify this to
$\alpha_{t}=({\hat{\alpha}_{t}},\underline{\alpha})$.

In the sequel, when we are considering the entire trajectory over times $t$ of
the values $(u^\alpha_{t},K^\alpha_{t})$ specified by a particular
$\hat{\alpha}$, we write $(\textbf{u}^\alpha,\textbf{K}^\alpha)$, in keeping
with our convention that bold characters indicate trajectories. Similarly, we
write $\hat{\alpha}$ without a subscript to mean the entire trajectory of the
protocol, $(\textbf{u}^\alpha,\textbf{K}^\alpha)$.

In the first approach to extending the definition of the minimal possible
effective work, we define it as the least work that could be achieved if the
energy component of the protocol, u, were fully fixed in an explicitly known
manner by the experimentalist, before the other uncertain parameters,
$\underline{\alpha}$, were (randomly) determined. To make sure that
uncertainty in parameters like the temperature do not cause LDB to be violated
even though the parameters other than $\bm{u}^\alpha$ are set independently of
$\bm{u}^\alpha$, we must allow the rate matrix trajectories
$\textbf{K}^\alpha$ to be statistically dependent on both $\bm{u}^\alpha$ and
$\underline{\alpha}$. So in this approach, we restrict attention to measures
over $\alpha$ of the form

$\displaystyle{\mathrm{d}}P^{\textbf{u}^\alpha}{\mathrm{d}}P^{\underline{\alpha}}{\mathrm{d}}P^{(\textbf{K}^\alpha|\textbf{u}^\alpha,\underline{\alpha})}$
(59)

with no statistical coupling between the energy component of the the protocol
and the non-protocol parameters defining the apparatus. (It is the term
${\mathrm{d}}P^{(\textbf{K}^\alpha|\bm{u}^\alpha,\underline{\alpha})}$ in Eq.
59 that allows us to ensure that LDB holds).

We use the term unadapted to refer to measures of this form. For unadapted
measures, the minimal possible work is

$\displaystyle
W^{unad}_{\min}=\min_{{\hat{\alpha}}}\int\mathrm{d}P^{\underline{\alpha}}\big{(}\Delta
U^{{\hat{\alpha}},{\underline{\alpha}}}-\pmb{Q}^{{\hat{\alpha}},{\underline{\alpha}}}\big{)}$
(60)

We also use the term “unadapted” to refer specifically to the kind of optimal
protocol given in Eq. 60, and to the associated definitions of least possible
(effective expected) work and of dissipated (effective expected) work. In
particular, the unadapted dissipated work is the amount of work that could
have been saved if the experimentalist has intervened to change the marginal
distribution $\mathrm{d}P^{{\hat{\alpha}}}$, leaving all other aspects of the
experimental setting the same. (An example of unadapted dissipated work is
presented in Section IV.3.)

In the second approach, we do not compare the actual work to the best that
could have been achieved if the experimentalist had changed
$\mathrm{d}P^{{\hat{\alpha}}}$ to be some delta function, leaving
$\mathrm{d}P^{\underline{\alpha}}$ unchanged. Rather we consider the best
possible work that could have occurred “by luck “, if there were the best
possible statistical coupling between the protocol and the other thermodynamic
parameters, again leaving $\mathrm{d}P^{\underline{\alpha}}$ unchanged.
Formally, in this second approach we define the “least possible” work for a
given $\mathrm{d}P^\alpha$ to be the minimum of the expected work as one
varies over the conditional $P^{\hat{\alpha}|\underline{\alpha}}$.

We refer to this measure over apparatuses as the adapted statistical coupling
between the protocol and the other parameters specifying the apparatus that
could have occurred, counterfactually, for the specified (fixed) marginal over
those other parameters. We also refer to the associated minimal effective
ensemble work as the adapted minimal (effective ensemble) work,

$\displaystyle
W_{\min}^{ad}=\int\mathrm{d}P^{\underline{\alpha}}\min_{{\hat{\alpha}}}\big{(}\Delta
U^{{\hat{\alpha}},{\underline{\alpha}}}-\pmb{Q}^{{\hat{\alpha}},{\underline{\alpha}}}\big{)}$
(61)

and refer to the associated dissipated effective ensemble work as the adapted
(effective ensemble) dissipated work.

Note that we can self-consistently define the adapted dissipated work for a
single apparatus $\alpha$,

$\displaystyle W^{{\hat{\alpha}},\underline{\alpha}}_{\mathrm{diss}}$
$\displaystyle:=W^{{\hat{\alpha}},\underline{\alpha}}-\min_{{\hat{\alpha}^{\prime}}}\big{(}\Delta
U^{{\hat{\alpha}}^{\prime},{\underline{\alpha}}}-\pmb{Q}^{{\hat{\alpha}}^{\prime},{\underline{\alpha}}}\big{)}$
(62)

in the sense that the adapted effective dissipated work is the
${\alpha}$-average of the adapted dissipated work for a single $\alpha$. The
analogous property does not hold for the unadapted dissipated work in general,
since the min and the integral in Eq. 60 will typically not commute. (Indeed,
if they did commute, then the unadapted dissipated work would equal the
adapted dissipated work.)

###  V.2 Physical interpretation of effective EP for a specific adapted
scenario

We now describe a relationship between effective dissipated work and effective
ensemble EP, to provide a thermodynamic interpretation of effective ensemble
EP. In this subsection we focus on calculating the minimal adapted dissipated
work.

For simplicity, we suppose that all the apparatuses $\alpha$ with nonzero
probability take the same initial distribution $p^*_{t_{i}}(x)$ to the same
ending distribution $p^*_{t_{f}}(x)$, i.e., for all $\alpha$ with nonzero
probability, $p^\alpha_{t_{i}}(x)=p^*_{t_{i}}(x)$ and
$p^\alpha_{t_{f}}(x)=p^*_{t_{f}}(x)$. This means that $\Delta S^{\alpha}$ is
independent of $\alpha$.

We similarly assume that the initial and final energy functions are
independent of $\alpha$. By Eq. 26, these two assumptions mean that $\Delta
U^\alpha$ is independent of $\alpha$, and so in particular it is independent
of the protocol. So by Eq. 61, the adapted minimal effective ensemble work is

$\displaystyle\Delta
U-\int\mathrm{d}P^{{\underline{\alpha}}}\\!\min_{{\hat{\alpha}}}\pmb{Q}^{{\hat{\alpha}},{\underline{\alpha}}}$
(63)

Also for simplicity, from now on we assume that each reservoir for any
specific apparatus $\alpha$ has the same temperature, so that we can write
$\beta^{\nu,\alpha}=\beta^\alpha$. Under this assumption the second law
reduces to the inequality

$\displaystyle\dfrac{(S(p^\alpha_{t_{f}})-S(p^\alpha_{t_{i}}))}{\beta^\alpha}$
$\displaystyle=\dfrac{(S(p_{t_{f}})-S(p_{t_{i}}))}{\beta^\alpha}$ (64)
$\displaystyle\geq\pmb{Q}^\alpha$ (65)

In general, the minimum of Eq. 63 will occur when the protocol
${\hat{\alpha}}$ accompanying the rest of the apparatus $\underline{\alpha}$
saturates the second law. As a result, the adapted minimal work is

$\displaystyle\Delta U-\int\mathrm{d}P^{{\underline{\alpha}}}\dfrac{\Delta
S^{\underline{\alpha}}}{\beta^{\underline{\alpha}}}$ $\displaystyle=\Delta
U-\Delta
S\int\mathrm{d}P^{{\underline{\alpha}}}(\beta^{\underline{\alpha}})^{-1}$ (66)
$\displaystyle=\Delta U-\Delta S\overline{\beta^{-1}}$ (67)

Plugging into Eq. 37 and combining, the adapted dissipated work is

$\displaystyle\overline{\pmb{W}}_{\mathrm{diss}}$ $\displaystyle={\Delta
S}\overline{\left(\beta^{-1}\right)}-\overline{\pmb{Q}}$ (68)

On the other hand, multiplying both sides of Eq. 62 by $\beta^\alpha$ before
averaging over $\alpha$, then again using the fact that $\Delta U^\alpha$ is
independent of $\alpha$, we get

$\displaystyle\overline{\beta\pmb{W}_{\mathrm{diss}}}$
$\displaystyle=\int\mathrm{d}P^\alpha\,\beta^\alpha[\pmb{W}^\alpha-\Delta
U]+\int\mathrm{d}P^\alpha\beta^\alpha\dfrac{\Delta S}{\beta^\alpha}$ (69)
$\displaystyle=\Delta S-\overline{\beta\pmb{Q}}=\overline{\pmb{\Sigma}}$ (70)

Plugging this into Eq. 68 and rearranging gives

$\displaystyle\overline{\pmb{\Sigma}}=\left(\dfrac{\overline{\pmb{W}}_{\mathrm{diss}}+\overline{\pmb{Q}}}{\overline{\left({\beta^{-1}}\right)}}\right)-\overline{\beta\pmb{Q}}$
(71)

So in general, in this setting where all apparatuses map $p^*_{t_{i}}(x)$ to
$p^*_{t_{f}}(x)$ and $\Delta U^\alpha$ is independent of $\alpha$, the
effective ensemble EP is not proportional to the effective (adapted)
dissipated work, in contrast to the no-uncertainty case. Instead it equals the
effective inverse-temperature-weighted dissipated work (Eq. 70), or
alternatively it is an affine function of the effective dissipated work (Eq.
71). Note though that if $\beta$ is fixed, independent of $\alpha$, then the
relationship between adapted effective dissipated work and effective EP
mirrors their relationship in the no-uncertainty case:

$\displaystyle\beta\overline{\pmb{W}}_{\mathrm{diss}}=\overline{\pmb{\Sigma}}$
(72)

(Compare to Eq. 70.)

As a variant of this scenario, suppose instead that we still have a single
reservoir, one which does not exchange particles with the system, and also
still suppose that the other thermodynamic parameters can vary with $\alpha$.
However, now suppose that both $p^{\underline{\alpha}}_{t_{i}}$ and
$p^{\underline{\alpha}}_{t_{f}}$ can also vary with $\underline{\alpha}$,
unlike before. To have the definition of minimal adapted work still be
meaningful in this situation, assume that the protocol cannot change either of
those two distributions, although it can change intermediate distributions.
Formally, this means that the support of $P(\hat{\alpha}|\underline{\alpha})$
is restricted so that for all $\underline{\alpha}$, no $\hat{\alpha}$ is
possible which affects the initial and final distributions. (If changing the
protocol were allowed to change the initial and / or final distributions, then
in general the minimal adapted work would be arbitrarily negative.) Under this
assumption changes to $\hat{\alpha}$ without any changes to
$\underline{\alpha}$ do not change the drop in entropy, $\Delta
S^{\hat{\alpha},\underline{\alpha}}$. Nor do they change $\Delta U^\alpha$,
which is still independent of $\alpha$. However, now changes to
$\underline{\alpha}$ will change the drop in entropy. In this case Eq. 68 gets
replaced by

$\displaystyle\overline{\pmb{W}}_{\mathrm{diss}}$
$\displaystyle=\overline{\left(\dfrac{\Delta
S}{\beta}\right)}-\overline{\pmb{Q}}$ (73)

###  V.3 Mismatch cost and effective ensemble EP for a specific unadapted
scenario

A common setting in which we have uncertainty about thermodynamic parameters
is where we know everything about the apparatus with complete certainty — the
number of reservoirs and their parameters, the paths followed by the
Hamiltonian and the rate matrix, etc. — except that we do not know the initial
distribution that is run with those parameters. Concretely, this setting
arises whenever we have a fixed physical apparatus that is run with a randomly
generated initial distribution. (So we are concerned with unadapted dissipated
work.) One common example is a computer that is used by different users; each
user implicitly fixes a distribution over the inputs to the computer, i.e.,
fixes its the initial distribution over the states of the computer. Other
common examples are a single cell floating in different environments, or a
fixed digital gate that can be positioned at different locations in a digital
circuit (and so have different distributions over the inputs it receives from
the rest of the circuit). For simplicity, for the rest of this subsection take
$\beta=1$.

In the situation under consideration, where we are certain about the
temperature of the process, effective dissipated work and effective EP are
identical, given by Eq. 72. By applying the formula for the mismatch cost Eq.
12, we get the formula for the effective mismatch cost

$\displaystyle\overline{\pmb{W}}_{\mathrm{diss}}$
$\displaystyle=\overline{\pmb{\Sigma}}$
$\displaystyle\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!\\!=\int\mathrm{d}P^\alpha\bigg{[}D(p^\alpha_{t_{i}}||q^\alpha_{t_{i}})-D(p^\alpha_{t_{f}}||q^\alpha_{t_{f}})+{\Sigma}^\alpha(q^\alpha_{t_{i}})\bigg{]}$
(74)

In particular, the expected effective mismatch cost is

$\displaystyle\overline{\pmb{W}}_{\mathrm{diss}}$
$\displaystyle=\int\mathrm{d}P^\alpha\left[D(p^\alpha_{t_{i}}||q^\alpha_{t_{i}})-D(p^\alpha_{t_{f}}||q^\alpha_{t_{f}})\right]$
$\displaystyle=\int\mathrm{d}P^\alpha\left[D(p^\alpha_{t_{i}}||q^\alpha_{t_{i}})-D(R^\alpha
p^\alpha_{t_{i}}||R^\alpha q^\alpha_{t_{i}})\right]$ (75)

where $R^\alpha$ is the transition matrix of the apparatus $\alpha$, with
entries $P^\alpha(x(t_{f})|x(t_{i}))$.

This effective mismatch cost is strictly positive in general since the
integrand is always non-negative and only equals zero in degenerate cases. In
particular, in the situation considered in this subsection, the rate matrix
trajectory is independent of $\alpha$, and therefore so is the prior. So the
effective mismatch cost reduces to

$\displaystyle\overline{\pmb{W}}_{\mathrm{diss}}$
$\displaystyle=\int\mathrm{d}P^\alpha\left[D(p^\alpha_{t_{i}}||q_{t_{i}})-D(Rp^\alpha_{t_{i}}||Rq_{t_{i}})\right]$
(76)

As an illustration, suppose that $R$ takes all initial distributions to the
same ending distribution (i.e., $Rp^\alpha_{t_{i}}=Rq_{t_{i}}$). For example,
this happens in bit erasure, or in complete relaxation of a system to its
stationary state. In this case the ending KL divergence is zero, no matter
what $p^\alpha_{t_{i}}$ is. Since the KL divergence is a convex function of
its arguments, this means that the prior that would minimize effective
mismatch cost is just the effective initial distribution,
$q_{t_{i}}(x)=\overline{p}_{t_{i}}(x)$.

If we plug this into Eq. 76 we can evaluate the minimal value of the
contribution to EP arising from uncertainty about the initial distribution.
Since effective residual EP is non-negative, we see that

$\displaystyle\overline{\pmb{\Sigma}}\geq\int\mathrm{d}P^\alpha
D(p^\alpha_{t_{i}}||\overline{p}_{t_{i}})=D_{JS}(\\{p^\alpha_{t_{i}}\\},P^\alpha)$
(77)

where $D_{JS}(\\{p^\alpha_{t_{i}}\\},P^\alpha)$ denotes the Jensen-Shannon
divergence among the set of distributions $\\{p^\alpha_{t_{i}}\\}$ distributed
according to $P^\alpha$. So minimal EP is strictly positive, as long as
$P^\alpha$ puts nonzero probability mass on at least one distribution
$p^\alpha\neq\overline{p}$. This provides a strengthened version of the second
law, applicable whenever there is uncertainty about the initial distribution,
and all initial distributions with nonzero probability get mapped to the same
final distribution.

##  VI Control protocols in a specific unadapted scenario

As discussed at the end of Section III.2, along with Sections V and V.2, in
no-uncertainty stochastic thermodynamics, the “dissipated work” is defined as
the difference between two amounts of work. The first is the actual work under
a given protocol that maps an initial distribution $p_{t_{i}}(x)$ to a final
distribution $p_{t_{f}}(x)$. The second is the least possible amount of work
that would be required to implement that map under any (counterfactual)
protocol, with all thermodynamic parameters other than the protocol, left the
same. One way to modify this definition of dissipated work for the case of
uncertain thermodynamic parameters was investigated at the beginning of
Section V.2. In that investigation, the initial and final distributions were
both fixed, independent of the (non-protocol components of the) apparatus
$\underline{\alpha}$. We also supposed that
$P(\hat{\alpha}|\underline{\alpha})=P({\bm{u},\bm{K}}|\alpha^{-(\bm{u},\bm{K})})$
was optimized for extracting work, whether due to conscious intervention by
the experimentalist or just by chance. That optimizing conditional
distribution was called the adapted protocol.

In the next subsection we introduce a modification of that scenario considered
in Section V.2. In the following subsection, we analyze the dissipated work
for that modified scenario.

###  VI.1 Modifying the adapted scenario

First we list the modifications, and then discuss their formal subtleties:

  1. 1.

One of the reasons for interest in dissipated work in no-uncertainty
stochastic thermodynamics is because often the experimentalist can intervene
in their experiment, in an (essentially) arbitrary way, even if they do not
know $\underline{\alpha}$. In the language of Section V, this is an unadapted
scenario. Formally, in such a scenario the distribution over the set of
counterfactual protocols, $P(\hat{\alpha}|\underline{\alpha})$, is independent
of $\underline{\alpha}$, since the experimentalist cannot choose the protocol
they implement to match the other uncertain parameters specifying the
apparatus (by definition of their being uncertain about those other
parameters). Physically, this means that the experimentalist can set the
protocol — but can only do so before the other thermodynamic parameters are
generated by sampling the distribution over apparatuses, in a way that is
independent of the protocol they have set.

  2. 2.

Often the experimentalist does not in fact have complete freedom to vary the
entire protocol arbitrarily. Often they will be able to set the trajectory of
the energy function directly, but cannot directly set the trajectory of rate
matrices. Instead, for each different apparatus, LDB will constrain the
relationship between the rate matrix and the quantity that the experimentalist
can control, the energy function.

  3. 3.

Often, the experimentalist will want to only consider the counterfactual
situations that result in some pre-specified final distribution, $p^*(x)$.
However, they will be uncertain about the initial distribution, as well as
other thermodynamic parameters that govern the dynamics, like the temperature.

Item 1 means that the protocol of the apparatus is statistically independent
of the other thermodynamic parameters:

$\displaystyle P(\alpha)=P({\hat{\alpha}})P(\underline{\alpha})$ (78)

To ensure Item 2 while requiring that LDB holds for each apparatus we make
several simplifying assumptions. First, we assume that there is only a single
reservoir in all of the apparatuses that have a nonzero probability of
occurring and that the reservoir only exchanges energy with the system, not
particles. So the only uncertainty parameter concerning the reservoirs that is
relevant to ensuring LDB is the single number $\beta^\alpha$ (which we will
sometimes write as $\underline{\alpha}^\beta$). In addition, we suppose that
there is a single-valued function $M:(u_{t},\beta^\alpha)\rightarrow
K^\alpha(t)$, a function which the experimentalist can set arbitrarily
(subject to the constraint of LDB), and so will know with zero uncertainty. So
by “varying the set of counterfactual protocols” we vary the marginal
distribution $P(\hat{\alpha}^{\bm{u}})$ and the function $M$, with

$\displaystyle P\left(K^{\alpha}(t),\underline{\alpha}|{u^\alpha_{t}}\right)$
$\displaystyle\qquad=\delta\left(K^\alpha(t)-M({u^\alpha_{t}},\underline{\alpha}^\beta))\right)P(\underline{\alpha})$
(79)

In our analysis below, it will also be convenient to assume that the support
of $\mathrm{d}P^{\underline{\alpha}^\beta}$ has a finite minimum (so that
there is a maximal possible temperature), and to require that $M$ be
differentiable with bounded derivative.

There are various formal challenges that arise in the analysis, depending on
the precise definition of Item 3. Here we clarify Item 3 to mean that averaged
over apparatuses, the final distribution is $p^\ast$, rather than requiring it
have that form for all $\underline{\alpha}$ with nonzero probability. That
then leaves the formal challenge of ensuring that we can meet this Item 3. If
the protocol is chosen so that at least one of the apparatuses is non-
infinitesimally off-equilibrium at $t_{f}$, then if that apparatus happens to
occur, in general the state distribution $p_{t}(x)$ would pass through
$p^*(x)$ transitionally at the time $t_{f}$, after which $p_{t}(x)$ would keep
evolving to other distributions, no matter what the protocol happens to be
then. Indeed, in general, it may be that two apparatuses
$\underline{\alpha},\underline{\alpha^{\prime}}$ both cause the state
distribution to pass through $p^*(x)$ at $t_{f}$, but result in different
state distributions at all other times, both before and after $t_{f}$. This is
a complication that doesn’t occur in the conventional, no-uncertainty scenario
for the case of a single reservoir, where the counterfactual protocol is
completely arbitrary, and so the optimal counterfactual protocol results in
the system being at thermal equilibrium at all times.

To see how to address this problem with ensuring that the condition in Item 3
is met, recall the conventional, two-step {quench; semi-static protocol}
process often considered in analyses of the minimal thermodynamic cost, in
which there is no uncertainty, and one needs to set a protocol to change
$(u_{t_{i}},p_{t_{i}})\mapsto(u_{t_{f}},p_{t_{f}})$ [35, 36]. Here we restrict
attention to protocols that expand that conventional two-step process, into a
three-step {quench; equilibrate; semi-static protocol} process. In other
words, we insert an intermediate step into the conventional process, a step in
which we wait long enough for the system to relax to equilibrium, no matter
what its temperature, before starting the step with the semi-static protocol.
(We are assured of being able to do this due to our restrictions on the
support of $\mathrm{d}P^{\beta^\alpha}$.) Due to our assumption that the
function $M(.)$ has a bounded derivative, we can choose the semi-static
protocol in which $u(x)$ varies in that subsequent third step slowly enough so
that (the rate matrix evolves slowly enough so that) the system is always at
equilibrium throughout that step, again no matter what $\alpha$ is.

As a formal point, we also assume that the very first energy function,
$u_{t_{i}}$, cannot be chosen by the experimentalist, i.e., it is randomly
distributed as specified by $P(\alpha)$. However, the experimentalist “takes
over” the specification of the trajectory of the energy function starting with
the energy function that the system is quenched to, the one immediately
following $u_{t_{i}}$.

As in the conventional no-uncertainty version of the two-step protocol, we
require that the distribution over states does not change during the quench
step of this expanded, three-step protocol. This is true no matter what the
temperature of the bath is. This means both that there is no heat exchange
with the bath in that step, and that there is no change in Shannon entropy in
that step, no matter what $\alpha$ is. So this step is thermodynamically
reversible; if we inverse-quenched right away, we would return to the initial
distribution and energy $u$, with zero net work expenditure. Moreover, since
the system is at equilibrium throughout the semi-static evolution step, that
step is also thermodynamically reversible; we could run it backward, returning
to the energy and (equilibrium) distribution at the beginning of that step,
and there would be zero net work. This is also true no matter what $\alpha$
is.

On the other hand, in contrast to the quench and semi-static steps, in
general, the intermediate, equilibration step will result in dissipated work
for each $\alpha$ considered by itself. More precisely, while no work is
extracted during that step, in a counterfactual process work could have been
extracted, if the protocol could be tailored for that $\alpha$.

The first of the following subsections contain a preliminary analysis of this
dissipated work. In the subsection after that, we investigate some related
issues that arise when a feedback control protocol is used to define a
“thermodynamic value of information about the actual apparatus”.

###  VI.2 Optimal work extraction when the temperature is uncertain

Write the Boltzmann distribution for arbitrary energy function $u$ and inverse
temperature $\beta^\alpha$ as

$\displaystyle\pi^\alpha_{u}(x)=\dfrac{e^{-\beta^\alpha
u(x)}}{Z(u,\beta^\alpha)}$ (80)

Also write $u^\alpha_{t_{i}}$ for the initial, pre-quench energy function for
the case where the apparatus is $\alpha$, and write $u_{q}$ for the energy
function that the experimentalist chooses for the system to quench to.
Similarly, write $u_{t}$ (with no $\alpha$ superscript) for the energy
function at times $t$ during the semi-static evolution (times at which the
experimentalist chooses the energy function).

With this notation the protocol of the process for each $\alpha$ (which is
determined before the process starts) can be written as the following:

$(p^\alpha_{t_{i}},u^\alpha_{t_{i}})$$(p^\alpha_{t_{i}},u_{q})$$(\pi^\alpha_{u_{q}},u_{q})$$(p^\alpha_{t_{f}},u_{t_{f}})$quenchequilibratesemi-
static ev.

where due to the semi-static nature of the evolution in the third leg,
$p^\alpha_{t_{f}}=\pi^\alpha_{u_{t_{f}}}$. As a shorthand, it will also be
convenient to write the non-equilibrium internal energy and free energy for
arbitrary energy $u$, state distribution $p$, and apparatus $\alpha$ as

$\displaystyle U(p,u)$ $\displaystyle:=\sum_{x}p(x)u(x)$ (81) $\displaystyle
F^\alpha(p,u)$ $\displaystyle:=U(p,u)-\dfrac{S(p)}{\beta^\alpha}$ (82)

where in general, $u$ and / or $p$ may depend on $\alpha$.

Note that our requirement that $\int\mathrm{d}P^\alpha
p^\alpha_{t_{f}}(x)=p^*(x)$ means that

$\displaystyle p^*(x)$
$\displaystyle=\int\mathrm{d}P^\alpha\pi^\alpha_{u_{t_{f}}}(x)$ (83)
$\displaystyle=\int\mathrm{d}P^\alpha\dfrac{e^{-\left[\beta^{\alpha}u_{t_{f}}(x)\right]}}{Z(u_{t_{f}},\beta^\alpha)}$
(84)

This imposes a strong condition on the ending energy function that the
experimentalist chooses. Indeed, in the no-uncertainty case, this requirement
would uniquely fix that energy function, up to an overall additive constant.

Since no work is expended or extracted during the equilibration process
$(p_{t_{i}},u_{q})\mapsto(\pi^\alpha,u_{q})$, the total work is given by
summing the work during the quench and semi-static evolution steps. Whatever
the protocol is, for each $\alpha$ this total work is

$\displaystyle\pmb{W}^\alpha$
$\displaystyle=\underbrace{U(p^\alpha_{t_{i}},u_{q})-U(p^\alpha_{t_{i}},u^\alpha_{t_{i}})}_{\mathrm{quench}}$
$\displaystyle\qquad\qquad+\underbrace{F^\alpha(p^\alpha_{t_{f}},u_{t_{f}})-F^\alpha(\pi^\alpha_{u_{q}},u_{q})}_{\mathrm{semi-
static\ ev.}}$ (85)

Adding and subtracting $S(p^\alpha_{t_{i}})/\beta^\alpha$ to Eq. 85 and
rearranging, we can rewrite this total work as

$\displaystyle\pmb{W}^\alpha=$ $\displaystyle
F^\alpha(p^\alpha_{t_{f}},u_{t_{f}})-F^\alpha(p^\alpha_{t_{i}},u_{t_{i}})$
$\displaystyle+F^\alpha(p^\alpha_{t_{i}},u_{q})-F^\alpha(\pi^\alpha_{u_{q}},u_{q})$
(86)

The expected work is the $\alpha$-average of this expression.

The first term on the RHS of Eq. 86, the difference between the nonequilibrium
free energies immediately preceding the quench and at the end of the process,
is independent of $u_{q}$. So the experimentalist cannot affect this term by
appropriate choice of protocol. The second term is instead the change in
nonequilibrium free energy that arises in going from just before to
immediately following the equilibration step.

To evaluate that second term, first, note that since no work is done on the
system in the equilibration step, the change of the expected energy is due to
the heat flow, i.e., $\Delta U=\pmb{Q}$. So the expected value of that second
term is

$\displaystyle\int\mathrm{d}P^\alpha\left[F^\alpha(p^\alpha_{t_{i}},u_{q})-F^\alpha(\pi^\alpha_{u_{q}},u_{q})\right]$
(87)
$\displaystyle\qquad=\int\mathrm{d}P^\alpha\left[\frac{1}{\beta^\alpha}\left(S(\pi^\alpha_{u_{q}}-S(p^\alpha_{t_{i}})\right)-{Q}^\alpha\right]$
(88) $\displaystyle\qquad=\overline{\pmb{W}}_{diss}$ (89)

where abusing notation, ${Q}^\alpha$ is defined to be the heat flow occurring
in just the second, equilibration step, for apparatus $\alpha$. Eq. 89
establishes that the ($\alpha$-average of the) second term in Eq. 86 is just
the effective dissipated work of the equilibration step, and therefore the
effective dissipated work of the entire process. (See Section V.2.)

Because $\pi^\alpha_{u_{q}}$ is the Boltzmann distribution for energy function
$u_{q}$ and inverse temperature $\beta^\alpha$, as usual in equilibrium
thermodynamics,

$\displaystyle F^\alpha(\pi^\alpha_{u_{q}},u_{q})=-\dfrac{\ln
Z(u_{q},\beta^\alpha)}{\beta^\alpha}$ (90)

In addition, $S(p^\alpha_{t_{i}})$ is independent of $u_{q}$. Combining, the
$u_{q}$ that minimizes the expression in Eq. 87 is the one that minimizes

$\int\mathrm{d}P_{\alpha}\,\left[\sum_{x}p^\alpha_{t_{i}}(x)u_{q}(x)+\frac{1}{\beta^\alpha}\ln
Z(u_{q},\beta^\alpha)\right]$

To solve for the $u_{q}$ minimizing this expression, take
$\frac{\partial}{\partial u_{q}(x)}$ for each $x$ and set it to zero. This
gives a set of coupled equations that $u_{q}(x)$ must satisfy:

$\displaystyle\int\mathrm{d}P_{{\alpha}}\,\left[p^\alpha_{t_{i}}(x)-\dfrac{e^{-{{\beta^\alpha}}u_{q}(x)}}{\sum_{x^{\prime}}e^{-{{\beta^\alpha}}u_{q}(x^{\prime})}}\right]=0$
(91)

i.e.,

$\displaystyle\overline{p}_{t_{i}}(x)$
$\displaystyle=\int\mathrm{d}P_{{\alpha}}\,\dfrac{e^{-{{\beta^\alpha}}u_{q}(x)}}{\sum_{x^{\prime}}e^{-{{\beta^\alpha}}u_{q}(x^{\prime})}}$
(92) $\displaystyle=\int\mathrm{d}P^\alpha\pi^\alpha_{u_{q}}(x)$ (93)

where

$\displaystyle\overline{p}_{t_{i}}(x):=\int\mathrm{d}P_{{\alpha}}\,p^\alpha_{t_{i}}(x)$
(94)

We denote this optimizing solution to Eq. 92 as $u^*_{q}$. Note that it is
independent of $u_{t_{i}},u_{t_{f}}$ and $p^*$.

As an example, suppose there is no uncertainty in the temperature, only in the
initial distribution. Then we could immediately invert Eq. 92 to get

$\displaystyle u^*_{q}(x)$ $\displaystyle=-\ln\overline{p}_{t_{i}}(x)/\beta$
(95)

up to an irrelevant additive constant. Note that this is also the energy
function that would be optimal if with zero uncertainty we knew that the
initial distribution were $\overline{p}_{t_{i}}(x)$.

More generally, suppose that there is both a nonzero minimal value and a
finite maximal value of $\operatorname*{supp}(\mathrm{d}P_{\beta^\alpha})$,
i.e., a nonzero minimal temperature and a finite maximal temperature that have
nonzero probabilities under $\mathrm{d}P_{\alpha}$. Then up to an overall
additive constant, there is one and only one solution $u_{q}$ to Eq. 91 for
any given $\overline{p}_{t_{i}}$ and $\mathrm{d}P_{\alpha}$. (This is proven
in Appendix C.)

By plugging $u^*_{q}(x)$ into Eq. 86 and averaging over $\alpha$, we get a
formula for the expected work during the full protocol:

$\displaystyle\overline{\pmb{W}}=$
$\displaystyle\int\mathrm{d}P^\alpha\left[F^\alpha(p^\alpha_{t_{f}},u_{t_{f}})-F^\alpha(p^\alpha_{t_{i}},u_{t_{i}})\right]$
$\displaystyle+\int\mathrm{d}P^\alpha\left[F^\alpha(p^\alpha_{t_{i}},u_{q}^*)-F^\alpha(\pi^\alpha_{u_{q}^*},u_{q}^*)\right]$
(96)

The first line in Eq. 96 is the expected work that would be required if we
were able to use a different (optimal) protocol that was set by the realized
value of $\alpha$, i.e., it is the minimal work, discussed in Section V.2. So
the second line in Eq. 96 is the extra work required due to our not being able
to use such a protocol that depends on the realized value of $\alpha$. In
other words, it is the extra work due to our having limited information about
the random apparatus. This kind of increase in the minimal work if the
protocol is prevented from depending on the realized value of a random
variable is sometimes called “thermodynamic value of information” in the
literature [35, 37, 38, 39, 40]. What differs between our analysis here and
the previous analyses is that we are concerned with the thermodynamic value of
information of the parameter $\alpha$, whereas those previous analyses instead
considered the thermodynamic value of information on the initial state of the
system.

###  VI.3 Dynamics of the thermodynamic value of information

In this subsection, we modify the scenario considered in Section VI.2 in two
ways. First, we assume that the experimenter can only set the protocol
starting at a time $\tau>t_{i}$, whereas $\mathrm{d}P_{\alpha}$ is sampled at
$t_{i}$. (We refer to the protocol during the interval starting at $\tau$ as
the “experimenter’s protocol”.) Second, we assume that we know with certainty
that $\beta^\alpha=1$ for all $\alpha$ during that time of the experimenter’s
protocol, following $\tau$. In particular, $\beta^\alpha=1$ during the
equilibration process. So the only uncertainty the experimenter faces in
choosing their protocol is in what the distribution is when that protocol
takes over the dynamics.

In general, the system will evolve between $t_{i}$ and $\tau$, potentially
according to an uncertain rate matrix. Our concern is with how the size of the
gap $\tau-t_{i}$ affects the minimal work that must be expended during the
experimenter’s protocol, in light of that evolution between $t_{i}$ and
$\tau$. In particular, we investigate how the thermodynamic value of knowing
the precise $\alpha$ in that interval depends on the length of that interval,
i.e., the derivative with respect to $\tau$ of that value of information.

Since there is no uncertainty in the temperature during the experimenter’s
protocol, we can plug into Eq. 95 with $t_{i}$ replaced by $\tau$, to see that
the optimal energy function for the experimenter to quench to at time $\tau$
is

$\displaystyle u_{q}^*(x)$ $\displaystyle=-\ln\overline{p}_{\tau}(x)$ (97)
$\displaystyle=-\ln\int\mathrm{d}P^\alpha p_{\tau}^\alpha(x)$ (98)

up to an overall additive constant 666Note that even though the temperature
during the experimenter’s protocol has no uncertainty, that is not the case
during $[t_{i},\tau)$. As a result, ${p}^\alpha_{\tau}$ can have values
different from $\pi_{u^*_{q}}$, depending on $P^\alpha$. In fact, even if
temperature were fixed during $[t_{i},\tau)$, uncertainty in the initial
distribution ${p}^\alpha_{t_{i}}$ would again mean that ${p}^\alpha_{\tau}$
can have values different from $\pi_{u_{q}}$..

If the experimenter had had no uncertainty when they perform the quench, the
protocol starting at $\tau$ would have resulted in zero dissipated work.
Accordingly, we define the _thermodynamic value of information_ for this
scenario as the effective dissipated work during the experimenter’s protocol
that does arise, since there in fact is uncertainty.

Recall that by the analysis in Section VI.2, this dissipated work arises
during the equilibration that starts immediately following the time of the
quench, which in our current scenario is the time $\tau$. This dissipated work
is given by modifying Eq. 87 to reflect the fact that there is no temperature
uncertainty during the experimenter’s protocol:

$\displaystyle\int\mathrm{d}P^\alpha\left[F^\alpha(p^\alpha_{\tau},u^*_{q})-F(\pi_{u_{q}^*},u^*_{q})\right]$
(99)

where $u^*_{q}$ is given by the solution to Eq. 98. Expanding the two free
energies and plugging in, we can write this effective dissipated work as

$\displaystyle I_{{\tau}}$
$\displaystyle=S\left(\overline{p}_{\tau}\right)-\overline{S}_{\tau}$ (100)
$\displaystyle=S\left(\int\mathrm{d}P^\alpha
p^\alpha_{\tau}\right)-\int\mathrm{d}P^\alpha S(p^\alpha_{\tau})$ (101)

(See the discussion at the end of Section VI.2.)

This is the Jensen-Shannon divergence of the set of distributions
$\\{p^\alpha(x)\\}$, weighted according to $\mathrm{d}P_{\alpha}$.
Furthermore, since $\alpha$ is a random variable, we can re-express each
distribution $p^\alpha(x)$ as a conditional distribution $p(x|\alpha)$. This
allows us to also identify $I_{{\tau}}$ as the mutual information at time
$\tau$ between $x$ and $\alpha$, mutual information that vanishes by the end
of the equilibration step. So $I_{{\tau}}$ is the usual expression for the
thermodynamic value of information found in previous analyses in the
literature [35, 38, 42]. Here though it reflects the thermodynamic value in
knowing what the initial distribution is, while in those previous analyses, it
reflects the value in knowing what the precise initial state is 777As an
aside, in this paper we are considering the thermodynamic value of information
in scenarios in which the (expected) target distribution, $p_{t_{f}}(x)$ is
fixed, independent of the initial distribution. An alternative is to consider
scenarios in which the conditional distribution, $p(x(t_{f})|x(t_{i}))$ is
fixed, independent of the initial distribution. Such scenarios include, for
example, all computational systems more complicated than simple bit erasure.
(In the case of computational systems, uncertainty about the initial
distribution amounts to uncertainty about which of a set of possible users of
the computer will set its initial state in any given run of that computer.) In
general, to implement a fixed conditional distribution $p(x(t_{f})|x(t_{i}))$
requires there to be “hidden states”, in addition to $X$ Nonetheless, here too
the Jensen-Shannon divergence characterizes the value of information in
knowing the actual initial distribution [42], just like in Eq. 101..

We wish to evaluate the derivative of $I_{\tau}$ with respect to $\tau$. To
begin, we consider the case where the rate matrix $K_{xx^{\prime}}(t)$ during
the interval $t\in[t_{i},\tau)$ has no uncertainty. We can see how the
thermodynamic value of information of $\alpha$ depends on $\tau$ in this case,
by taking the derivative of the RHS of Eq. 101 with respect to $\tau$. In
Appendix D we show that the value of information in this case, $I_{\tau}$, is
monotonically decreasing in time, i.e.,

$\frac{\mathrm{d}I_{\tau}}{\mathrm{d}\tau}\leq 0$ (102)

stopping to fall only when the system reaches equilibrium. (We can derive the
same result from the data-processing inequality.) Physically, this means that
the thermodynamic value of information of $\alpha$ decreases the longer we
wait to use it.

In addition to the $\tau$-dependence of the thermodynamic value of information
of $\alpha$, one might also be interested in how the total effective EP (i.e.,
effective dissipated work), generated during the entire interval
$[t_{i},t_{f}]$, changes with changes to $\tau\in[t_{i},t_{f}]$. There are two
contributions to that EP: the thermodynamic value of information $I_{\tau}$,
generated during the equilibration step, and the EP generated earlier, during
the interval $[t_{i},\tau)$. So to get the derivative of the total effective
EP generated during the entire interval $[t_{i},t_{f}]$, we must add two
terms. The first is the derivative of the effective EP under $K(t)$, i.e., the
effective EP rate, evaluated at $t=\tau$. The second is the time derivative of
the value of information evaluated at that time.

From Eq. 34, that effective EP rate at $\tau$ is

$\displaystyle\dot{\overline{\pmb{\Sigma}}}_{\tau}=\int\mathrm{d}P^\alpha\,\sum_{x,x^{\prime}}K_{xx^{\prime}}(\tau)p_{\tau}^\alpha(x^{\prime})\ln\dfrac{K_{x^{\prime}x}(\tau)p_{\tau}^\alpha(x^{\prime})}{K_{x^{\prime}x}(\tau)p^\alpha_{\tau}(x)}$
(103)

Adding this to Eq. 163 of Appendix D gives the derivative of the total
effective EP generated during $[t_{i},t_{f}]$ as

$\displaystyle\dfrac{d\overline{\Sigma}}{dt}=\sum_{x,x^{\prime}}K_{xx^{\prime}}(\tau)\overline{p}_{\tau}(x^{\prime})\ln\dfrac{K_{xx^{\prime}}(\tau)\overline{p}_{\tau}(x^{\prime})}{K_{x^{\prime}x}(\tau)\overline{p}_{\tau}(x)}$
(104)

This is the EP rate generated by the effective dynamics of $\overline{p}$,
evaluated at $\tau$. So the derivative of the effective EP generated during
$[t_{i},t_{f}]$ with respect to the time $\tau\in[t_{i},t_{f}]$ at which the
experimentalist takes over the protocol is the phenomenological EP generated
at $\tau$.

As a final comment, suppose that $\alpha$ indexes not just the distribution,
but also the rate matrix, i.e., suppose that we are unsure about the dynamics
as well as the distribution. This means that

$\displaystyle\dfrac{d\overline{p}_{\tau}(x)}{d\tau}$
$\displaystyle=\int\mathrm{d}P^\alpha\dfrac{d{p}^\alpha_{\tau}(x)}{d\tau}$
$\displaystyle=\int\mathrm{d}P^\alpha\sum_{x^{\prime}}K^\alpha_{xx^{\prime}}(\tau){p}^\alpha_{\tau}(x^{\prime})$
(105)

In this case, the value of information of $\alpha$ can increase with $\tau$.
As a simple example, suppose that the initial distribution is independent of
$\alpha$, i.e., $p^\alpha_{t_{i}}(x)=p_{t_{i}}(x)$. Then $I_{p_{t_{i}}}=0$.
Suppose as well that $\alpha$ has two possible values, as does $x$, and that
$dP_{\alpha}$ is uniform over $\alpha$’s two values. Finally, also suppose
that for $\alpha=0$, the associated trajectory of rate matrices
$K^0_{xx^{\prime}}(t)$ sends $p_{t_{i}}(t)$ to the ending distribution
$p^0_{\tau}(x)=\delta(x,0)$ with arbitrarily high accuracy, while for
$\alpha=1$, the associated trajectory of rate matrices $K_{xx^{\prime}}^1(t)$
sends $p_{t_{i}}(x)$ to the ending distribution $p^1_{\tau}(x)=\delta(x,1)$
with arbitrarily high accuracy. Then $I_{p_{\tau}}$ is arbitrarily close to
$\ln 2$, which establishes the claim.

To illustrate the richness of value of information due to uncertainty, we also
consider the value of information for the case where there is uncertain
temperature, and therefore (imposing LDB) uncertain rate matrices leading up
to the time that the experimentalist’s protocol takes over. It is unknown what
the general necessary and sufficient conditions would be for the value of
information due to uncertainty to be nondecreasing as $\tau$ increases. This
is a potentially fruitful area of future research. To exemplify this fact, we
calculate the value of information for the example from Section IV.3, i.e.,
the bit erasure for the case of the uncertain temperature. We observe that the
value of information increases with time, until it reaches its maximal value
around $t\approx 8$ and then starts to decrease.

Figure 4: Value of information for the case of bit erasure with uncertain
temperature from Section IV.3.

##  VII Phenomenological EP and fluctuation theorems

We now focus on the second, phenomenological scenario, where the apparatus
changes after each stochastic trajectory is generated. In this scenario we are
not able to measure ${\mathbb{P}}({\pmb{x}}|\alpha)$ for any (unknown)
apparatus $\alpha$, but only the average probabilities
$\overline{{\mathbb{P}}}({\pmb{x}})$.

###  VII.1 EP in the phenomenological scenario

To begin, recall that the $\alpha$-average of the effective EP can be
expressed as

$\displaystyle\overline{\pmb{\Sigma}}=\int\mathrm{d}P^\alpha
D({\mathbb{P}}({\pmb{x}}\,|\,\alpha)||\
({\mathbb{P}}^\alpha)^{\dagger}({\pmb{x}^{\dagger}}|\,\alpha))\,.$ (106)

Writing $\mathrm{d}P^\alpha=p(\alpha)\mathrm{d}\alpha$ as shorthand, the joint
trajectory probability is
${\mathbb{P}}({\pmb{x}},\alpha)={\mathbb{P}}({\pmb{x}}|\alpha)p(\alpha)$. So
the $\alpha$-averaged effective ensemble EP is

$\displaystyle{\overline{\pmb{\Sigma}}}$
$\displaystyle=D_{{\pmb{x}},\alpha}({\mathbb{P}}({\pmb{x}},\alpha)||{\mathbb{P}^{\dagger}}({\pmb{x}^{\dagger}},\underline{\alpha}))$
$\displaystyle=\int\mathrm{d}\alpha\sum_{{\pmb{x}}}{\mathbb{P}}({\pmb{x}},\alpha)\ln\frac{{\mathbb{P}}({\pmb{x}},\alpha)}{{\mathbb{P}^{\dagger}}({\pmb{x}^{\dagger}},\underline{\alpha})}$
(107)

Now by Bayes’ theorem,

$\displaystyle
P(\alpha|{\pmb{x}})=\frac{{\mathbb{P}}({\pmb{x}}|\alpha)}{\overline{\mathbb{P}}({\pmb{x}})}p(\alpha)=\frac{{\mathbb{P}}({\pmb{x}},\alpha)}{\overline{\mathbb{P}}({\pmb{x}})}$
(108)

where we used the fact that
$\overline{\mathbb{P}}({\pmb{x}})=\int\mathrm{d}P^\alpha{\mathbb{P}}({\pmb{x}}|\alpha)$.
Combining this with the chain rule for KL divergence [44], we derive

$\bar{\pmb{\Sigma}}=D_{{\pmb{x}}}(\overline{{\mathbb{P}}}({\pmb{x}})||\overline{{\mathbb{P}}}^{\dagger}({\pmb{x}^{\dagger}}))+D_{{\pmb{x}},\alpha}({\mathbb{P}}(\alpha|{\pmb{x}})||{\mathbb{P}^{\dagger}}(\alpha|{\pmb{x}^{\dagger}}))$
(109)

For later use, introduce shorthand for the first term on the RHS of Eq. 109,

$\displaystyle\pmb{\Phi}:=D_{{\pmb{x}}}(\overline{{\mathbb{P}}}({\pmb{x}})||\overline{{\mathbb{P}}}^{\dagger}({\pmb{x}^{\dagger}}))$
(110)

which we call _phenomenological EP_. Phenomenological EP measures the
irreversibility of the dynamics quantified by the effective (i.e., apparatus-
averaged) probabilities.

Similarly, define the second term on the RHS of Eq. 109 as

$\displaystyle\pmb{\Lambda}:=D_{{\pmb{x}},\alpha}({\mathbb{P}}(\alpha|{\pmb{x}})||{\mathbb{P}^{\dagger}}(\alpha|{\pmb{x}^{\dagger}}))$
(111)

which measures the difference between distributions of $\alpha$ estimated from
forward and reverse trajectories. Like many forms of EP, $\pmb{\Lambda}$ is a
Kullback-Leibler divergence and is zero only if the forward probability is the
same as the backward probability. For these reasons, we refer to
$\pmb{\Lambda}$ as _likelihood EP_ , even though it need not have a
straightforward relation to dissipated work.

All three EPs in Eq. 109, Eq. 110, and Eq. 111 have associated trajectory-
level versions:

$\displaystyle\pmb{\sigma}({\pmb{x}}|\alpha)$
$\displaystyle:=\ln\frac{{\mathbb{P}}({\pmb{x}}|\alpha)}{{\mathbb{P}^{\dagger}}({\pmb{x}^{\dagger}}|\alpha)}$
(112) $\displaystyle\pmb{\phi}({\pmb{x}})$
$\displaystyle:=\ln\frac{\overline{{\mathbb{P}}}({\pmb{x}})}{\overline{{\mathbb{P}}}^{\dagger}({\pmb{x}^{\dagger}})}$
(113) $\displaystyle\pmb{\lambda}(\alpha|{\pmb{x}})$
$\displaystyle:=\ln\frac{{{\mathbb{P}}}(\alpha|{\pmb{x}})}{{{\mathbb{P}^{\dagger}}}(\alpha|{\pmb{x}^{\dagger}})}$
(114)

The first EP corresponds to the effective scenario, where we randomly fix the
apparatus and generate an infinite set of trajectories for that apparatus. The
second EP corresponds to the phenomenological scenario, where the apparatus
randomly changes after generating each trajectory.

In general, there will both be $\alpha$ for which
$\pmb{\sigma}({\pmb{x}}|\alpha)>\pmb{\phi}({\pmb{x}})$ and $\alpha$ for which
$\pmb{\sigma}({\pmb{x}}|\alpha)<\pmb{\phi}({\pmb{x}})$. However, Eq. 109 means
that

$\overline{\pmb{\Sigma}}\geq\pmb{\Phi}\,.$ (115)

So fixing $\alpha$ to the same value all stochastically generated
trajectories, calculating the associated trajectory-averaged EP, and then
averaging over the unknown values of $\alpha$ increases the ensemble EP,
compared to the case where we average over apparatuses to calculate EP, and
only then average over apparatuses.

The last of these trajectory-level EPs in Eq. 114, the likelihood EP, is the
difference between the first two. While the likelihood EP is a log-likelihood
ratio, in contrast to the common case in which log-likelihood ratios are based
on the same data but different parametric models, this one is based on the
same set of apparatuses, but on forward, resp. reversed trajectories.

The trajectory version of Eq. 109 can be written as

$\pmb{\sigma}(\alpha,{\pmb{x}})=\pmb{\phi}({\pmb{x}})+\pmb{\lambda}(\alpha|{\pmb{x}})\,.$
(116)

For the first EP $\sigma$, we get to derive the ordinary fluctuation theorem
for a given $\alpha$. For the case of phenomenological EP and likelihood EP,
we define the following probabilities:

$\displaystyle P(\phi)$
$\displaystyle:=\int\mathcal{D}{\pmb{x}}\,\overline{{\mathbb{P}}}({\pmb{x}})\delta(\phi-\pmb{\phi}({\pmb{x}}))$
(117) $\displaystyle P(\lambda_{{\pmb{x}}})$
$\displaystyle:=\int\mathrm{d}P^\alpha\,\delta(\lambda_{{\pmb{x}}}-\pmb{\lambda}(\alpha|{\pmb{x}}))$
(118)

we can write down detailed fluctuation theorems:

$\displaystyle\frac{P(\phi)}{{P}^{\dagger}(-\phi)}$ $\displaystyle=e^\phi\,,$
(119)
$\displaystyle\frac{P(\lambda_{{\pmb{x}}})}{{P}^{\dagger}(-\lambda_{{\pmb{x}^{\dagger}}})}$
$\displaystyle=e^{\lambda_{{\pmb{x}}}}\,.$ (120)

The consequence of the detailed fluctuation theorem is the integrated
fluctuation theorem, i.e, $\langle e^{-\phi}\rangle=1$, resp. $\langle
e^{-\lambda_{{\pmb{x}}}}\rangle=1$. The first of those IFTs implies that
$\langle\phi\rangle\equiv\Phi\geq 0$. The second of those IFTs means that

$\displaystyle\pmb{\Lambda}_{\pmb{x}}=\langle\lambda_{{\pmb{x}}}\rangle=\int\mathrm{d}P(\lambda_{{\pmb{x}}})\lambda_{{\pmb{x}}}\geq
0$ (121)

We call this _the second law of inference_. It tells us that for each
trajectory ${\pmb{x}}$, the $\alpha$-averaged log-likelihood
$\int\mathrm{d}P^\alpha\ln{\mathbb{P}}(\alpha|{\pmb{x}})$ is larger than the
$\alpha$-averaged likelihood obtained from the time-reversed trajectory. the
reverse process. Of course, by averaging over all trajectories with
probability $\overline{{\mathbb{P}}}({\pmb{x}})$, we obtain
$\pmb{\Lambda}=\boldlangle\pmb{\Lambda}_{\pmb{x}}\boldrangle\geq 0$.

###  VII.2 Example: two-state system with uncertain temperature

Figure 5: Illustration of the second law of inference on a two-state system
with uncertain temperature. Before the experiment, the temperature is randomly
chosen from the prior distribution $p(T)$ (top-left panel). The experimenter
cannot measure the temperature, but can observe the trajectory ${\pmb{x}}(t)$
(top-right panel). From observing the trajectory, it is possible to update the
temperature distribution $P(T)\mapsto P(T|{\pmb{x}})$ (bottom-left panel).
Similarly, the temperature distribution obtained from observing the time-
reversed trajectory under the time-reversed protocol is
$\tilde{P}(T|{\pmb{x}^{\dagger}})$. From their log ratio, it is possible to
express the likelihood EP $\pmb{\lambda}(T|{\pmb{x}})$ (bottom-right panel).
By calculating the histogram of $\lambda_{\pmb{x}}$, we observe that negative
likelihood EP is observed much less common than positive likelihood EP, which
corresponds to the detailed fluctuation theorem. Finally, the average value of
likelihood EP remains positive, which is in agreement with the second law of
inference.

To illustrate these results, consider a two-state system with states
$\\{0,1\\}$, as depicted in Fig. 5. The energy levels corresponding to the
states are $E_{0}$ and $E_{1}$. We consider the transition rate matrix

$K=\left(\begin{array}[]{cc}-e^{\frac{E_{0}-E_{1}}{T}}&e^{\frac{E_{1}-E_{0}}{T}}\\\
e^{\frac{E_{0}-E_{1}}{T}}&-e^{\frac{E_{1}-E_{0}}{T}}\\\ \end{array}\right)$
(122)

The initial distribution is $p_{0}=\\{1/2,1/2\\}$. Suppose that the
temperature is randomly drawn from the Gamma distribution $P(T)=Te^{-T}$ (top-
left panel of Fig. 5).

The trajectory entropy production is

$\pmb{\sigma}^T({\pmb{x}})=\ln p_{t_{0}}({\pmb{x}}(t_{0}))-\ln
p(t_{f})({\pmb{x}}(t_{f}))-\frac{1}{T}\left(E_{{\pmb{x}}(t_{f})}-E_{{\pmb{x}}(t_{0})}\right)$
(123)

By observing a trajectory ${\pmb{x}}$ (top-right panel of Fig. 5), we
calculate an updated distribution ${\mathbb{P}}(T|{\pmb{x}})$ (bottom-right
panel of Fig. 5). Finally, we can also calculate the likelihood EP (bottom-
right panel of Fig. 5). In the inset histogram of the bottom-right panel, we
see that the likelihood EP $\pmb{\lambda}_{\pmb{x}}$ can attain negative
values, however $\bar{\lambda}_{\pmb{x}}=\Lambda_{\pmb{x}}$ is greater than
zero, which is in agreement with the detailed fluctuation theorem and the
second law of inference.

##  VIII Discussion and Future work

In any real-world experimental test of a system there are three major types of
uncertainty: a) uncertainty about the state of the system b) uncertainty about
the state of the external environment that the system is interacting with, and
c) uncertainty about the parameters of the equations governing the dynamics of
the system and its interaction with its external environment. In (classical)
stochastic thermodynamics, the first type of uncertainty is addressed by
replacing the specification of the system’s state (e.g., coarse-graining), and
the second one is typically addressed by assuming the environment is infinite,
at equilibrium, and evolving far faster than does the system (“separation of
time scales”). In essence, the entire field of stochastic thermodynamics
concerns the consequences of those two types of uncertainty for the dynamics
of energy and particle counts in the system. However, very little attention
has been paid so far to the third type of uncertainty. Here we begin an
investigation of the consequences of that third type of uncertainty, showing
that it entails major modifications to the standard results previously derived
in stochastic thermodynamics.

Our investigations have only scratched the surface of issues involved with
this third type of uncertainty. In the light of recent studies [45] where the
system is coupled to a reservoir with fluctuating temperature, it makes sense
to extend the analysis from a static distribution $p(\alpha)$ to a
distribution over dynamic trajectories of the parameters, i.e.,
$\pmb{P}(\pmb{\alpha}_{t})$. Another possible future direction can motivated
by extending the framework to the case of stochastic thermodynamics hidden
Markov models [46] with the application to biophysical systems as flashing
ratchets [47, 48] when the experimenter cannot directly observe whether the
ratchet is on or off, and therefore does not also know the switching rate.

Some of the more immediate questions to be addressed in future work include
the extension to the stochastic control protocols. The first possible question
is whether the {quench; equilibrate; semi-statically-evolve} protocol is the
optimal protocol which maps $p_{x}(t_{i})\rightarrow p_{x}(t_{f})$ or if there
is an alternative protocol that generates less effective dissipated work.

Furthermore, it is reasonable to assume that in systems with limited
possibility of measuring the system’s parameters, it will be even more
difficult to realize the desired control protocol with infinite precision. The
related question is how the partial knowledge of the system obtained by the
(imprecise) measurement from observing a trajectory, i.e., $P(\alpha)\mapsto
P(\alpha|{\pmb{x}}_{t})$ can be utilized to adjust the control protocol
$\epsilon^{\star}(x)\mapsto\epsilon^{\star}(x|{\pmb{x}}_{t})$. In more
realistic situations, the measurement cannot be done continuously but at given
time instants and might bear some costs. The question of optimal measurement
and update of control protocol in uncertain environments is challenging but
definitely a crucial question to be answered.

###### Acknowledgements.

DHW would like to thank the Santa Fe Institute and the US National Science
Foundation under grant 2221345 for support. JK was supported by the Austrian
Science Fund (FWF) project No. P 34994 and project No. P 33751 and Austrian
Science Promotion Agency (FFG) Project Grant 857136.

## References

  * Seifert [2012] U. Seifert, Stochastic thermodynamics, fluctuation theorems and molecular machines, Reports on Progress in Physics 75, 126001 (2012).
  * Van den Broeck and Esposito [2015] C. Van den Broeck and M. Esposito, Ensemble and trajectory thermodynamics: A brief introduction, Physica A: Statistical Mechanics and its Applications 418, 6 (2015).
  * Note [1] We note though there is a substantial literature which adopts the “inclusive” framework, in which the external environment is finite, and the joint dynamics of the system-environment is explicitly modeled. This framework is the classical form of “open quantum thermodynamics”. The inclusive framework has been explored both for explicit Hamiltonian dynamics over the joint system, where the only randomness in the initial state [49, 50, 51], and for approximate Hamiltonian dynamics [52, 53, 54, 55, 56].
  * Note [2] We note though that it is already known that if we do not account for all thermodynamic reservoirs, we invariably under-estimate the total entropy production in a process [57].
  * Note [3] Note that $\alpha$ is a generic characterization of apparatuses, which can be formalized as a vector with some components finite-valued, some countable, and some uncountable, with the measure $\mathrm{d}P^\alpha$ defined appropriately.
  * Bisker  _et al._ [2017] G. Bisker, M. Polettini, T. R. Gingrich, and J. M. Horowitz, Hierarchical bounds on entropy production inferred from partial information, Journal of Statistical Mechanics: Theory and Experiment 2017, 093210 (2017).
  * Shiraishi and Sagawa [2015] I. S. K. K. Shiraishi, N. and T. Sagawa, Role of measurement-feedback separation in autonomous Maxwell’s demons, New Journal of Physics (2015).
  * Matsuo and ichi Sasa [2000] M. Matsuo and S. ichi Sasa, Stochastic energetics of non-uniform temperature systems, Physica A 276, 188 (2000).
  * Fiore and de Oliveira [2019] C. E. Fiore and M. J. de Oliveira, Entropy production and heat capacity of systems under time-dependent oscillating temperature, Phys. Rev. E 99, 052131 (2019).
  * Wolpert  _et al._ [1993] D. H. Wolpert _et al._ , On the use of evidence in neural networks, Advances in neural information processing systems , 539 (1993).
  * Beck and Cohen [2003] C. Beck and E. Cohen, Superstatistics, Physica A 322, 267 (2003).
  * Hanel  _et al._ [2011] R. Hanel, S. Thurner, and M. Gell-Mann, Generalized entropies and the transformation group of superstatistics, Proceedings of the National Academy of Sciences 108, 6390 (2011), https://www.pnas.org/content/108/16/6390.full.pdf .
  * García-Morales and Krischer [2011] V. García-Morales and K. Krischer, Superstatistics in nanoscale electrochemical systems, Proceedings of the National Academy of Sciences 108, 19535 (2011).
  * Beck [2006] C. Beck, Stretched exponentials from superstatistics, Physica A 365, 96 (2006), fundamental Problems of Modern Statistical Mechanics.
  * Mathai and Haubold [2007] A. Mathai and H. J. Haubold, Pathway model, superstatistics, tsallis statistics, and a generalized measure of entropy, Physica A 375, 110 (2007).
  * Sattin [2006] F. Sattin, Bayesian approach to superstatistics, The European Physical Journal B-Condensed Matter and Complex Systems 49, 219 (2006).
  * Davis and Gutiérrez [2018] S. Davis and G. Gutiérrez, Temperature is not an observable in superstatistics, Physica A: Statistical Mechanics and its Applications 505, 864 (2018).
  * Edwards and Anderson [1975] S. F. Edwards and P. W. Anderson, Theory of spin glasses, Journal of Physics F: Metal Physics 5, 965 (1975).
  * Sherrington and Kirkpatrick [1975] D. Sherrington and S. Kirkpatrick, Solvable model of a spin-glass, Phys. Rev. Lett. 35, 1792 (1975).
  * Mézard  _et al._ [1987] M. Mézard, G. Parisi, and M. A. Virasoro,  _Spin glass theory and beyond: An Introduction to the Replica Method and Its Applications_ , Vol. 9 (World Scientific Publishing Company, 1987).
  * Kolchinsky and Wolpert [2017] A. Kolchinsky and D. H. Wolpert, Dependence of dissipation on the initial distribution over states, Journal of Statistical Mechanics: Theory and Experiment , 083202 (2017).
  * Wolpert [2019] D. H. Wolpert, The stochastic thermodynamics of computation, Journal of Physics A: Mathematical and Theoretical (2019).
  * Wolpert and Kolchinsky [2020] D. Wolpert and A. Kolchinsky, The thermodynamics of computing with circuits, New Journal of Physics (2020).
  * Riechers and Gu [2021] P. M. Riechers and M. Gu, Initial-state dependence of thermodynamic dissipation for any quantum process, Phys. Rev. E 103, 042145 (2021).
  * Cao and Feito [2009] F. J. Cao and M. Feito, Thermodynamics of feedback controlled systems, Physical Review E 79, 041118 (2009).
  * Sagawa and Ueda [2012] T. Sagawa and M. Ueda, Nonequilibrium thermodynamics of feedback control, Physical Review E 85, 021104 (2012).
  * Gomez-Marin  _et al._ [2008a] A. Gomez-Marin, T. Schmiedl, and U. Seifert, Optimal protocols for minimal work processes in underdamped stochastic thermodynamics, The Journal of Chemical Physics 129, 10.1063/1.2948948 (2008a), 024114, https://pubs.aip.org/aip/jcp/article-pdf/doi/10.1063/1.2948948/6770253/024114_1_online.pdf .
  * Note [4] N.b., this is in contrast with what is arguably the most common parametrization, where the parameters are the mean and the variance of the corresponding normal distribution. The transformation between these two parametrizations is given in Appendix A.
  * Cover and Thomas [2012] T. M. Cover and J. A. Thomas,  _Elements of information theory_ (John Wiley & Sons, 2012).
  * Rao and Esposito [2019] R. Rao and M. Esposito, Detailed fluctuation theorems: A unifying perspective (Santa Fe Institute Press, 2019).
  * Note [5] Formally, Eq. 12 assumes that the conditional distribution $P(x(t_{f})|x(t_{i}))$ has a single “island”; the extension of the analysis here to the case of multiple islands is straight-forward. See [21, 22, 23, 24].
  * Parrondo  _et al._ [2009] J. M. R. Parrondo, C. V. den Broeck, and R. Kawai, Entropy production and the arrow of time, New Journal of Physics 11, 073008 (2009).
  * Diana  _et al._ [2013] G. Diana, G. B. Bagci, and M. Esposito, Finite-time erasing of information stored in fermionic bits, Physical Review E 87, 012111 (2013).
  * Esposito and Parrondo [2015] M. Esposito and J. M. Parrondo, Stochastic thermodynamics of hidden pumps, Physical Review E 91, 052114 (2015).
  * Parrondo  _et al._ [2015] J. M. Parrondo, J. M. Horowitz, and T. Sagawa, Thermodynamics of information, Nature Physics 11, 131 (2015).
  * Hasegawa  _et al._ [2010] H.-H. Hasegawa, J. Ishikawa, K. Takara, and D. Driebe, Generalization of the second law for a nonequilibrium initial state, Physics Letters A 374, 1001 (2010).
  * Sagawa and Ueda [2013] T. Sagawa and M. Ueda, Role of mutual information in entropy production under information exchanges, New Journal of Physics 15, 125012 (2013).
  * Sagawa and Ueda [2009] T. Sagawa and M. Ueda, Minimal energy cost for thermodynamic information processing: measurement and information erasure, Physical Review Letters 102, 250602 (2009).
  * Kolchinsky and Wolpert [2021] A. Kolchinsky and D. H. Wolpert, Work, entropy production, and thermodynamics of information under protocol constraints, Physical Review X 11, 041024 (2021).
  * Kolchinsky and Wolpert [2018] A. Kolchinsky and D. H. Wolpert, Semantic information, autonomous agency and non-equilibrium statistical physics, Interface focus 8, 20180041 (2018).
  * Note [6] Note that even though the temperature during the experimenter’s protocol has no uncertainty, that is not the case during $[t_{i},\tau)$. As a result, ${p}^\alpha_{\tau}$ can have values different from $\pi_{u^*_{q}}$, depending on $P^\alpha$. In fact, even if temperature were fixed during $[t_{i},\tau)$, uncertainty in the initial distribution ${p}^\alpha_{t_{i}}$ would again mean that ${p}^\alpha_{\tau}$ can have values different from $\pi_{u_{q}}$.
  * Wolpert [2015] D. H. Wolpert, Extending Landauer’s bound from bit erasure to arbitrary computation (2015), arXiv:1508.05319 [cond-mat.stat-mech].
  * Note [7] As an aside, in this paper we are considering the thermodynamic value of information in scenarios in which the (expected) target distribution, $p_{t_{f}}(x)$ is fixed, independent of the initial distribution. An alternative is to consider scenarios in which the conditional distribution, $p(x(t_{f})|x(t_{i}))$ is fixed, independent of the initial distribution. Such scenarios include, for example, all computational systems more complicated than simple bit erasure. (In the case of computational systems, uncertainty about the initial distribution amounts to uncertainty about which of a set of possible users of the computer will set its initial state in any given run of that computer.) In general, to implement a fixed conditional distribution $p(x(t_{f})|x(t_{i}))$ requires there to be “hidden states”, in addition to $X$ Nonetheless, here too the Jensen-Shannon divergence characterizes the value of information in knowing the actual initial distribution [42], just like in Eq. 101.
  * Gomez-Marin  _et al._ [2008b] A. Gomez-Marin, J. M. R. Parrondo, and C. Van den Broeck, Lower bounds on dissipation upon coarse graining, Phys. Rev. E 78, 011107 (2008b).
  * Karimi and Pekola [2020] B. Karimi and J. P. Pekola, Quantum trajectory analysis of single microwave photon detection by nanocalorimetry, Phys. Rev. Lett. 124, 170601 (2020).
  * Bechhoefer [2015] J. Bechhoefer, Hidden markov models for stochastic thermodynamics, New Journal of Physics 17, 075003 (2015).
  * Prost  _et al._ [1994] J. Prost, J.-F. m. c. Chauwin, L. Peliti, and A. Ajdari, Asymmetric pumping of particles, Phys. Rev. Lett. 72, 2652 (1994).
  * Roldán and Parrondo [2010] E. Roldán and J. M. R. Parrondo, Estimating dissipation from single stationary trajectories, Phys. Rev. Lett. 105, 150607 (2010).
  * Kawai  _et al._ [2007] R. Kawai, J. M. R. Parrondo, and C. Van den Broeck, Dissipation: the phase-space perspective, Physical Review Letters 98, 080602 (2007).
  * Esposito  _et al._ [2010] M. Esposito, K. Lindenberg, and C. Van den Broeck, Entropy production as correlation between system and reservoir, New Journal of Physics 12, 013013 (2010).
  * Ptaszyński [2018] K. Ptaszyński, First-passage times in renewal and nonrenewal systems, Physical Review E 97, 012127 (2018).
  * Seifert [2016] U. Seifert, First and second law of thermodynamics at strong coupling, Physical Review Letters 116, 020601 (2016).
  * Strasberg and Esposito [2017] P. Strasberg and M. Esposito, Stochastic thermodynamics in the strong coupling regime: An unambiguous approach based on coarse graining, Physical Review E 95, 062101 (2017).
  * Talkner and Hänggi [2020] P. Talkner and P. Hänggi, Colloquium: Statistical mechanics and thermodynamics at strong coupling: Quantum and classical, Reviews of Modern Physics 92, 041002 (2020).
  * Talkner and Hänggi [2020] P. Talkner and P. Hänggi, Comment on “measurability of nonequilibrium thermodynamics in terms of the hamiltonian of mean force”, Phys. Rev. E 102, 066101 (2020).
  * Strasberg and Esposito [2020] P. Strasberg and M. Esposito, Measurability of nonequilibrium thermodynamics in terms of the hamiltonian of mean force, Physical Review E 101, 050101 (2020).
  * Esposito and Van den Broeck [2010] M. Esposito and C. Van den Broeck, Three faces of the second law. i. master equation formulation, Physical Review E 82, 011143 (2010).

##  Appendix A Detailed calculation of moving optical tweezer with uncertain
stiffness

Let us now show a detailed derivation of the moving optical tweezer with
uncertain stiffness from the main text. Let us consider a particle described
by an overdamped Langevin equation

$\dot{x}=-\mu\frac{\partial V}{\partial x}+\xi$ (124)

where $\xi(t)$ is the white noise, and $V$ is the potential. Let us consider
the potential in the form

$V_{k}(x,t)=\frac{k}{2}(x-\lambda(t))^2$ (125)

where $k$ is the stiffness parameter and $\lambda(t)$ is the control protocol.
The Sekimoto formula for the average work is

$W[\lambda(t)]=\int_{t_{i}}^{t_{f}}\mathrm{d}t\dot{\lambda}\left\langle\frac{V_{k}(\lambda(t),x(t))}{\partial\lambda}\right\rangle$
(126)

where $\langle..\rangle$ is the ensemble average. Let us consider $\mu=1$. By
introducing $u(t)=\langle x(t)\rangle$, we obtain

$\dot{u}_{k}=k(\lambda-u_{k})$ (127)

where we omit the dependence of both $u$ and $\lambda$ on $t$. From this, we
can express $\lambda$ as

$\lambda=\frac{\dot{u}_{k}}{k}+u_{k}$ (128)

Let us now express
$\left\langle\frac{V_{k}(\lambda(t),x(t))}{\partial\lambda}\right\rangle$ as

$\left\langle\frac{V_{k}(\lambda(t),x(t))}{\partial\lambda}\right\rangle=-k\langle
x-\lambda\rangle=k(\lambda-u_{k})=\dot{u}_{k}$ (129)

By plugging into the formula for work, we obtain

$W[\lambda(t)]=\int_{t_{i}}^{t_{f}}\mathrm{d}t\left(\dot{u}_{k}^2+\frac{\ddot{u}_{k}\dot{u}_{k}}{k}\right)=\int_{t_{i}}^{t_{f}}\mathrm{d}t\dot{u}_{k}^2+\frac{[\dot{u}_{k}^2]_{t_{i}}^{t_{f}}}{2k}$
(130)

Extremization of the work functional can obtained by $\frac{\delta
W[\lambda(t)]}{\delta\lambda(t)}=0$ which leads to the Euler-Lagrange equation

$\ddot{u}_{k}=0$ (131)

which leads to $u(t)=mt$, where $m$ is a parameter to be determined. The