for model updating (a.k.a. information fusion). Kapteyn et al. (2021)
suggested DBNs as the foundation of a unifying mathematical formulation for
digital twin at scale, by integrating Bayesian statistics, dynamical systems,
and control theory. They also demonstrated the DBN concept using the UAV
digital twin example as given in Fig. 13.

The directed acyclic graph in a BN or DBN is usually constructed based on
physical knowledge of the causality relationships between different nodes,
such as the logical relationships defined by the ontology and SysML as been
described in Sec. 3.5. It is possible that the logical or causal relationship
is unknown. In that case, the directed acyclic graph/causal relationships
could be learned from data using BN learning methods, such as minimal
description length, Bayesian–Dirichlet equivalence, and mutual information
test (Scutari, 2009, Murphy, 2002, Hu and Mahadevan, 2018b). In BN learning,
the most probable topology of the graph is identified by minimizing or
maximizing a score function, such as the aforementioned minimal description
length, Bayesian–Dirichlet equivalence, or mutual information. A tutorial on
BN learning can be found in (Heckerman, 2008). Also, there are several open-
sources toolkits available for the inference, learning, and construction of
DBNs, such as the Graphical Models Toolkit (Bilmes, ) and the Bnlearn Python
package (Taskesen, ).

####  4.2.4 State and parameter estimation

When a digital twin involves multiple types of physics, the time scales of
different digital state variables could be quite different. For example, the
capacity of a lithium-ion battery cell typically fades slowly and only shows
noticeable changes in months or years. In contrast, the cell’s SOC typically
changes quickly with time and may go from 0$\%$ (fully depleted) to 100$\%$
(fully charged) within minutes in a fast charging cycle. As will be discussed
in the battery case study in Sec. 3 of Part 2 of the review paper, SOC and
capacity are two key performance measures that together assist in managing the
health of lithium-ion batteries. The state estimation literature usually
refers to estimating digital state variables that change very slowly or do not
change with time as _parameter estimation_. Estimating digital state variables
that change rapidly with time is called _state estimation_. Note that both the
hidden state ${\bf{x}}$ and parameters ${\boldsymbol{\uptheta}}$ are digital
state variables and part of the digital state.

When the state ${\bf{x}}$ is known and noise-free, then state estimation will
not be needed, and only parameter estimation is needed. A straightforward way
to estimate ${\boldsymbol{\uptheta}}$ is to solve a least squares problem,
formulated based on the measurement function $g(\cdot)$ for a one-dimensional
known state $x$ and measurable response $y$ as follows:

$\mathop{\min}\limits_{\boldsymbol{\uptheta}}\;\sum\limits_{k=1}^{K}{{{({y_{k}}-g({x_{k}},\;{\boldsymbol{\uptheta}}))}^{2}}},$
(7)

where $K$ is the total number of measurements available for estimating
${\boldsymbol{\uptheta}}$ and $g(\cdot)$ is the measurement function that
predicts $y$ at state $x$, given the vector of model parameters
${\boldsymbol{\uptheta}}$. Here, $g(\cdot)$ can be a physics-based model as
discussed in Sec. 3.2 or a data-driven model as discussed in Sec. 3.3. Three
examples of least squares parameter estimation for a physics-based model are
reported in (1) Wang et al. (2019) for estimating two fault-sensitive
parameters of a rotor dynamics model for unbalance diagnostics of a rotor
system, (2) Ramadesigan et al. (2011) for estimating five degradation-
sensitive parameters of an electrochemical model for every charge/discharge
cycle for battery capacity fade prediction, and (3) Peng et al. (2020) for
estimating five degradation-sensitive parameters of an equivalent circuit
model for condition monitoring of a buck DC-DC converter. Least squares
estimation can also be used to estimate the parameters of data-driven models,
such as estimating the weights and biases of an artificial neural network from
a training dataset $\\{(x_{1},y_{1}),…,(x_{K},y_{K})\\}$.

If $g(\cdot)$ is linear, Eq. (7) can be solved using linear least squares.
When measurement data comes in a stream, the Kalman filter discussed in Sec.
4.2 (b) and recursive least squares can be used to ensure a fixed
computational cost at each time step Simon (2006). They produce optimal
estimates, and if the assumption of Gaussian measurement noise holds, they are
equivalent.

If $g(\cdot)$ is nonlinear, numerical optimization algorithms, such as Gauss-
Newton, gradient descent, and Levenberg–Marquardt, and population-based
metaheuristic optimization algorithms, such as genetic algorithms and particle
swarm optimization, can be used to solve Eq. (7). However, a potential issue
is that the resulting estimate of ${\boldsymbol{\uptheta}}$ is deterministic.
This estimate does not capture prior knowledge on ${\boldsymbol{\uptheta}}$ or
uncertainty in parameter estimation (e.g., due to low identifiability and
measurement noise). This issue can be mitigated by deriving an approximate
posterior using a Markov chain Monte Carlo method combined with a prior
distribution of ${\boldsymbol{\uptheta}}$ under a Bayesian calibration
framework (Ramadesigan et al., 2011). Although sequential Bayesian calibration
(e.g., a sequential version of a well-known Bayesian calibration framework
referred to as the Kennedy and O’Hagan (KOH) framework (Kennedy and O’Hagan,
2001)) has been shown to track time-varying parameters, this approach may be
computationally intensive and require longer run times than the response time
needed to track sudden changes (Ward et al., 2021). A detailed discussion on
the applicability of Bayesian calibration to UQ of dynamic system models is
given in Sec. 2.1.2 in Part 2 of the review paper. Sequential, recursive
filtering alternatives to these iterative optimizers are nonlinear extensions
of the Kalman filter (e.g., the extended Kalman filter and unscented Kalman
filter) and particle filters, which are more attractive in computational
efficiency. It is not uncommon to see these recursive filtering alternatives
succeed in training nonlinear models like neural networks with fixed optimal
parameters (Puskorius and Feldkamp, 1994, Aitchison, 2020), with a possible
extension to time-varying parameters (Wagner et al., 2021). A new state-space
representation can be defined for parameter estimation in the presence of a
known and clean state. This new state-space model takes the following form:

$\begin{split}&{\rm{State}}\;{\rm{transition}}:\;{{\boldsymbol{\uptheta}}_{k}}={{\boldsymbol{\uptheta}}_{k-1}}+{{\bf{r}}_{k}},\cr&{\rm{Measurement}}:\;{{\bf{y}}_{k}}={\bf{g}}({{\bf{x}}_{k}},\;{{\boldsymbol{\uptheta}}_{k}})+{{\bf{v}}_{k}},\cr\end{split}$
(8)

Now suppose that $g$ is highly nonlinear, ${\boldsymbol{\uptheta}}$ is high-
dimensional (e.g., consisting of $>$ 5 parameters), and the computational time
and cost are not a concern. In that case, population-based optimization
algorithms, such as genetic algorithms and particle swarm optimization, could
be better alternatives due to the simple implementation and applicability to a
wide variety of $g$ function forms and nonlinearities (Wang et al., 2019, Peng
et al., 2020).

When the state is unknown or given with noise, state estimation and parameter
estimation are needed. In these cases, state-space models have unknown
parameters ${\boldsymbol{\uptheta}}$ and hidden states ${\bf{x}}$ that need to
be estimated. As mentioned in Sec. 4.2 (a), ${\boldsymbol{\uptheta}}$ is part
of the digital state, and these parameters may first be calibrated offline but
must be updated online. With both a hidden state and unknown parameters, the
state-space model in Eq. (3) becomes

$\begin{split}&{\rm{State}}\;{\rm{transition}}:\;{{\bf{x}}_{k}}={\bf{f}}({{\bf{x}}_{k-1}},\;{{\boldsymbol{\uptheta}}_{k-1}},\;{{\bf{u}}_{k-1}})+{{\boldsymbol{\upomega}}_{k}},\cr&\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;{{\boldsymbol{\uptheta}}_{k}}={{\boldsymbol{\uptheta}}_{k-1}}+{{\bf{r}}_{k}},\cr&{\rm{Measurement}}:\;{{\bf{y}}_{k}}={\bf{g}}({{\bf{x}}_{k}},\;{{\boldsymbol{\uptheta}}_{k}})+{{\bf{v}}_{k}},\cr\end{split}$
(9)

where ${\boldsymbol{\uptheta}}$ is the vector of unknown model parameters
which changes slowly or do not change over time, ${{\bf{r}}_{k}}$ is the
vector of process noise variables for the parameters at the $k^{\mathrm{th}}$
time step.

Similar to when the state is known and clean, iterative optimization
algorithms can be used to jointly estimate ${\bf{x}}$ and
${\boldsymbol{\uptheta}}$. Sequential approaches such as recursive Bayesian
filtering become favorable alternatives for online, close to real-time state
estimation with limited computational power (e.g., in an embedded
application). A straightforward sequential approach is to combine the state
and parameters into an augmented state, ${\bf{z}}=[{\bf{x}}\mathchar
24635\relax\;{\boldsymbol{\uptheta}}]$, and estimate ${\bf{z}}$ using the
extended Kalman filter or unscented Kalman filter. This approach is referred
to as joint (extended or unscented) Kalman filtering Cox (1964), which was
attempted more than half a century ago. As a new measurement from a physical
system becomes available, this joint version of the extended Kalman filter
simultaneously updates the state and parameter estimates. A drawback reported
for this approach is divergence and bias in the parameter estimation Ljung
(1979). An alternative is the dual extended or unscented Kalman filter that
separately considers state and parameter estimation (Wan and Nelson, 2001).
Instead of using fixed state transition and measurement functions such as in
Eq. (#), these approaches alternate between estimating the state given the
most recent parameter estimate and estimating the model parameters given the
most recent state estimate. Two notable applications of the dual extended
Kalman filter are battery state (SOC) and parameter (capacity) estimation
Plett (2004) and vehicle state (e.g., longitudinal and lateral velocities) and
parameter (e.g., vehicle mass and moments of inertia) estimation Wenzel et al.
(2006). Compared to the hidden state ${\bf{x}}$, the parameters
${\boldsymbol{\uptheta}}$ tend to vary more slowly with time. For example, the
SOC and capacity of a lithium-ion battery cell change on two largely different
time scales, as mentioned earlier. The different time scales of these digital
state variables present unique challenges in estimating these variables. More
recently, a multiscale filtering framework (see Fig. 17) was proposed to
estimate the SOC and capacity of a lithium-ion battery cell (Hu et al., 2012).
Multiscale filtering utilizes the temporal nature of SOC (fast varying) and
capacity (slowly varying) to improve the efficiency and accuracy in estimating
both the state and parameter over the joint and dual versions of the extended
Kalman filtering.

Figure 17: Illustration of multiscale extended Kalman filtering for battery
SOC and capacity estimation (adapted from Hu et al. (2012)).

###  4.3 ML model updating

ML models are an increasingly critical component of digital twins, and they
are often deployed in an environment with periodic operational changes in the
physical world, such as sensors addition/removal, manufacturing process
updates (e.g., add/remove a step in a manufacturing process), changes in
measurement device settings/performance, etc. These significant changes might
lead to test data that is sufficiently far from the distribution of the data
on which the ML model is trained. Such phenomenon is typically referred to as
“out-of-distribution” in the ML field (Lee et al., 2018, Zhang et al., 2022b).
An ML model is often trained with a closed-world assumption (i.e., the
training data and the test data are assumed to follow the same distribution).
As a result, if out-of-distribution test data is not incorporated in model
training timely and properly, ML model performance will drift significantly.

It is crucial to detect the out-of-distribution situation and update the ML
model to account for the operational changes associated with the physical
asset. ML model updating will prevent severe performance degradation of a
trained and deployed ML model. Updating digital models based on the continuous
monitoring of the physical counterpart is an important characteristic of
digital twins (Chakraborty et al., 2021). Besides, the timely update of the ML
model is also the key to ensuring a consistent and acceptable predictive
performance in characterizing the trend of quantities of interest associated
with physical assets. One of the most commonly adopted paradigms is to utilize
a performance-based indicator to determine when model drift onsets and
subsequently trigger an update of the deployed ML model. ML model updating can
be implemented in the cloud or on the edge. In the ML community, model drift
(also referred to as model decay) is developed as an umbrella term to
characterize the degradation of a model’s predictive power due to
unforeseeable changes in the environment. In practice, model drift often
manifests itself in two principal forms:

Figure 18: Demonstration of data drift. Adopted from Zhang et al. (2022c)

  1. 1.

Data drift (also referred to as covariate drift): As mentioned before, the
training of ML models is established upon the assumption that the training
data distribution is representative of the test data distribution. However,
this assumption can easily get violated in many real-world applications
(Sugiyama et al., 2008). The term “ data drift” is used to describe the
situation that statistical properties of the production data differ
substantially from those of the training data. and it can be mathematically
represented as $p^{\textrm{train}}\left(\mathcal{X}\right)\neq
p^{\textrm{test}}\left(\mathcal{X}\right)$.

Fig. 18 showcases an instance of data drift, where the relationship between
$\mathcal{X}$ and $\mathcal{Y}$ remains the same (see the target function in
Fig. 18). Unfortunately, $\mathcal{X}$ follows two different distributions in
the training data and test data. As a result of data drift, the function
learned from the training data leads to erroneous predictions on the test
data.

  2. 2.

Concept drift: This form of model drift characterizes the phenomenon in which
the underlying relationship between $\mathcal{X}$ and $\mathcal{Y}$ changes in
non-stationary environments (Lu et al., 2018). In essence, the quantity we are
trying to predict evolves over time. Mathematically, concept drift is
represented as $p^{\textrm{train}}\left(\mathcal{Y}|\mathcal{X}\right)\neq
p^{\textrm{test}}\left(\mathcal{Y}|\mathcal{X}\right)$.

Fig. 19 demonstrates an example of concept shift using the same regression
problem. Obviously, the underlying functional relationship between
$\mathcal{X}$ and $\mathcal{Y}$ has substantially changed from the training
data to test data.

Figure 19: Demonstration of concept drift

In either case, model drift poses significant robustness and reliability
challenges in practical ML applications. Such drift has led to substantial ML
model performance degradation (Tsymbal, 2004). Safeguarding an ML model in
digital twins from this issue entails a close monitoring of ML performance
variation using a drift indicator and proactive update of the ML model on an
as-needed basis. Here, a drift indicator is typically a quantitative
application-specific metric. For example, one approach is to monitor an ML
model’s test data for drift from the training data with a statistic metric
(e.g., Kullback–Leibler divergence or Jensen-Shannon divergence). A second
approach is to monitor the variation of model predictive quality over a time
window, allowing trending model quality. A third approach is to calculate a
mean error between the ML predictions and actual measurements in the recent
past and tune the last fully-connected layer or the last few layers, in the
case of a neural network. In this direction, state-of-the-art studies have
developed several drift-based approaches to determining when to update a
deployed ML model. For example, Chakraborty and Adhikari (2021) investigated
ML model updating in the digital twin of a multi-time scale dynamical system,
where physical data was periodically used to update the ML model in learning
the time-evolution of system parameters. Davis et al. (2017) studied the
deterioration of ML model performance in predictive analytics for the
development of acute kidney injury with ten years of patient data and
highlighted the importance of routine calibration of ML models when they were
incorporated into clinical decision support systems.

Q-learning is a reinforcement learning algorithm that identifies the best
action given the current state of a physical system. Sun et al. (2020a) took
advantage of Q-learning to sensitively quantify the deviation between the
measurable response (e.g., CPU frequency of industrial devices and computer
clusters or energy consumption) of a physical device (e.g., a sensor or a
monitor) and its mapped value in the digital twin of an IIoT system and
updated the ML model in the IIoT system to achieve a better trade-off between
local update and global parameter aggregation. Wu et al. (2019a) developed
error- and event-triggered mechanisms to update a recurrent neural network
model using the most recent process data for model predictive control (MPC) in
a chemical process subject to time-varying disturbances.

It is worth noting that ML model updating can be formulated as a parameter
estimation problem, expressed as a discrete-time state-space model. In this
state-space model, the ML model parameters to be updated (often a small subset
of all ML model parameters) become ${\boldsymbol{\uptheta}}$ that changes by a
vector of process noises (state transition equation), and the ML model becomes
the measurement function that maps the ML model input and parameters to the
measurement $\bf{y}$ (measurement equation). As new measurements from the
physical system become available, Bayesian filters (Cox, 1964, Gustafsson,
2010), such as the extended Kalman filter and particle filters, can be used to
solve this state-state model for online updating the ML model parameters. See
a more detailed discussion in Sec. 4.2.4.

Accurate ML model prediction plays a cornerstone role in digital twins. ML
model updating is just one of the mainstream ways to ensure satisfactory
generalization performance of ML models in the open world. In addition to
regular ML model updating, if the physics characterizing the underlying
physical processes are known, incorporating these physics in the ML model can
also be pursued as another viable path to improve ML models’ generalizability.
See a more detailed discussion of physics-informed ML in Sec. 3.4.

###  4.4 Fault diagnostics and failure prognostics

####  4.4.1 Overview of PHM

Over the past two decades, sensor data collection, fault diagnostics, failure
prognostics, and maintenance decision making have drawn significant attention,
collectively forming a research field called prognostics and health management
(PHM). This research field has been largely promoted by (1) various workshop
committees on structural health monitoring (SHM), a research field largely
parallel to and synergistic with PHM, (2) the PHM Society, and (3) the rise of
journals like Structural Health Monitoring: An International Journal. While
fault diagnostics and failure prognostics form an essential, integral part of
PHM, the system health management goes beyond the predictions of the RUL and
provides decision support for optimal maintenance and logistics decisions by
integrating also the considerations on the available maintenance resources,
the operating requirements, and the economic, environmental and operational
impact of different faults. In general, health management can be considered as
the process of taking optimal maintenance decisions based on the outputs from
diagnostics and prognostics models, while taking the relevant operational,
resource and monetary constraints into consideration. In short, the goal of
PHM can be considered as optimally managing the health of components and
systems by minimizing the operational and economic impact of failures, and
proactively controlling the direct and indirect maintenance costs Lee et al.
(2014).

The concept of PHM has been exploited in different application domains,
leading to a diverse set of terms, algorithms, and tools. For example,
machinery health monitoring is an engineering discipline dedicated to
monitoring the health of machine components, including bearings, gears,
shafts, motors, and pumps, and tracking and predicting their health evolution
over time. Vibration-based fault diagnostics is a mature subdiscipline of
machinery health monitoring that has a much longer history than PHM, with many
algorithms and tools available for processing vibration signals (Antoni and
Randall, 2006) and diagnosing bearing faults (Randall and Antoni, 2011) and
many applications in industrial sectors (Randall, 2021). Another example is
that research on damage diagnostics and prognostics of civil infrastructures
has, over time, formed an engineering discipline called SHM. As described
earlier, SHM and PHM are two largely parallel yet synergistic research fields,
with SHM having a much longer history than PHM. SHM primarily focuses on
monitoring and detecting structural damage that alters the material and
geometric properties of a structure (Farrar and Worden, 2007). In SHM, “damage
assessment” is a more appropriate and widely used term than fault diagnostics.
It deals with detecting, localizing, and identifying the types of, and
quantifying the severity of, damage as just defined. Given that structural
failure is rare and difficult to introduce, prognostics plays less of an
important role than damage assessment. A third example is when PHM is applied
to lithium-ion batteries, creating an active discipline of battery health
management. In this discipline, a better alternative to the term “fault
diagnostics” is “degradation diagnostics”, which identifies and quantifies
degradation modes and mechanisms driving cell-level degradation (Birkl et al.,
2017). Metrics used to measure cell-level degradation are called the state of
health parameters, as discussed in 4.2 (a). Two representative metrics in this
regard are capacity and internal resistance, which together determine the
energy and power capability of a battery cell.

A few representative review papers on failure prognostics and SHM are (1) Heng
et al. (2009), Lee et al. (2014), Lei et al. (2018) for machinery health
monitoring, with the first focusing on comparing traditional reliability
engineering and prognostics, the second providing a balanced review on
methodologies and industrial applications, and the third focusing on datasets
and methodologies for health assessment and RUL prediction, and (2)
Rezvanizaniani et al. (2014), Waag et al. (2014), Hu et al. (2020) for battery
health management, with the first focusing on SOC and state of health
monitoring as well as lifetime prognostics, the second on SOC and state of
health monitoring, and the third on battery lifetime prognostics.

####  4.4.2 Diagnostics vs. prognostics

Before discussing diagnostics and prognostics in detail, let us first look at
the key differences between fault diagnostics and failure prognostics. Fault
diagnostics is focused on identifying the presence (fault detection), location
(fault localization), type (fault identification) and possibly severity (fault
severity assessment) of damage/defects representative of the system’s current
health state. In contrast, failure prognostics is tasked with tracking the
time evolution of fault properties and projecting their progression into the
future to yield an updated estimate of the physical system’s health state or
RUL, defined as how long the physical system can operate before it reaches a
failure threshold or completely breaks down. Time evolution of damage or
degradation is an essential factor that plays an almost dominant role in RUL
prediction. In short, diagnostics gives us the current damage state, and
prognostics projects that into the future to explore questions about
criticality that could suggest actions such as maintenance, repair, etc.

####  4.4.3 Fault diagnostics approaches

Fault diagnostics is a broad topic that has been extensively studied in many
application domains over the past decades. We intend to provide a brief survey
of this topic in this subsection, focusing on ML-based fault diagnostics.
Interested readers can gain more knowledge and insights by looking at review
papers on fault diagnostics in respective domains, such as Samuel and Pines
(2005), Jardine et al. (2006), Li et al. (2016), Zhao et al. (2019) for
machinery fault diagnostics, Lynch and Loh (2006), Fan and Qiao (2011) for
sensors in SHM applications, Farrar and Worden (2007), Fan and Qiao (2011) for
structural damage detection and identification, and Berecibar et al. (2016),
Li et al. (2019b) for battery state of health estimation. Furthermore, a
recent special issue of the Structural Health Monitoring journal featured ML-
based approaches (Structural Health Monitoring 20(4), 1353-2239, 2021.)

In ML-based fault diagnostics, ML models are either classification tools to
classify the health state of a physical system (e.g., healthy, slightly
damaged, or severely damaged of a roller bearing in a hydraulic motor (Shen et
al., 2021)) or identify its fault type (e.g., unbalance, misalignment,
mechanical looseness, rubbing, or oil whirl in a journal bearing rotor system
(Oh et al., 2017)) or predictive tools to quantify a fault/damage present in a
physical system. Approaches have been developed using conventional ML and
emerging DL methods. As indicated in Fig. 20, running a forward pass on
conventional ML models typically involves two steps: feature extraction and
classification/regression. Before training these ML models, their input
features need to be properly defined. Low-dimensional features (typically less
than 20) are first extracted from preprocessed sensor data and then used with
other low-dimensional monitoring data (e.g., measurements of operating
conditions) for fault classification or damage assessment. The commonly used
feature extraction techniques include statistical feature extraction
(Jegadeeshwaran and Sugumaran, 2015), principal component analysis (Misra et
al., 2002, Pei et al., 2021), wavelet transform (Peng and Chu, 2004), and fast
Fourier transform (Zhang et al., 2013). For example, Pei et al. (2021) used
principal component analysis to extract important features from manufacturing
process monitoring signals. Afterward, a support vector machine was used to
map the extracted features to processing quality which is labeled as
acceptable or abnormal according to the level of tolerance deviation.

Figure 20: A graphical comparison of traditional ML and emerging DL approaches
to fault diagnostics

While successes using conventional ML methods for fault diagnostics can be
found in some application domains, these success stories have been limited to
cases where the domain knowledge required to extract hand-crafted features
from the raw data and select appropriate features is available. Furthermore,
extracting and selecting input features can be tedious and time-consuming. It
is an iterative, manual process known as feature engineering, where features
are added and removed, and ML models are redefined, trained, and evaluated.
Feature engineering can account for more than 90% of the total effort in
building ML models.

Emerging DL methods provide end-to-end fault diagnostics capability,
eliminating the need for manual feature extraction, as indicated in Fig. 20.
Unlike traditional ML algorithms, DL algorithms, such as those described in
Sec. 3.3.2, can automatically extract high-level, complex representations (or
features) from large volumes of data. The DL models take preprocessed sensor
data directly as input and produce the health class or fault/damage size as
output. For instance, melt pool images and thermal images collected by cameras
have been directly used as the inputs of DL models for defect detection in
both additive and more conventional manufacturing processes (Liu et al.,
2020a, Wang et al., 2020b, Franciosa et al., 2020).

In recent years, hundreds or even thousands of DL-based fault diagnostics
approaches have been reported in the literature. Given large volumes of
labeled data, DL approaches show better accuracy in fault diagnostics than
their conventional ML counterparts in most applications. A review of DL
methods for fault diagnostics can be found in Zhang et al. (2020b). As
expected, the effectiveness of ML models for fault diagnostics is
significantly affected by the quantity and quality of the data used to train
the model. In particular, data from faulty states are much more difficult to
collect than data from the healthy state. This data collection issue could
result in data imbalance and poor accuracy in fault diagnostics. Three
strategies have been investigated to address this data challenge, and they all
build on known physics.

  * •

_Synthetic data generation_ : As reviewed in Sec. 3.4, a variety of physics-
informed ML approaches can be used in physical system modeling to improve
generalizability, to increase interpretability, and to alleviate the data-size
requirement over purely data-driven ML approaches. Similarly, physics-informed
ML can also improve fault diagnostic performance. If we consider that data
under faulty states are very difficult to collect, “ synthetic” faulty data
can be generated from high-fidelity physics-based simulations and used to
augment the original training dataset. The synthetic data can be generated by
changing health-relevant parameters of the physics-based model (e.g., reducing
material stiffness for selected regions where structural damage is intended to
be injected (Kapteyn et al., 2021)). Combining the synthetic data with
available real data to train and validate ML models is essentially a strategy
for physics-informed ML, illustrated as Approaches 2, 3, and 5 in Fig. 10.
When high-fidelity physics-based models are not available, an alternative
solution to synthetic data generation is to use a DL-based generative model
called a generative adversarial network (GAN). After being trained on a real
dataset, a GAN can generate synthetic healthy/faulty data, which share a
similar distribution as the real training dataset (Shao et al., 2019).

  * •

_Physics-informed architecture or loss function design_ : In many cases,
physical domain knowledge can be leveraged to customize mathematical
operations or node connections in ways that mimic essential physics and make
black-box ML models more interpretable. The design of the model architecture
can be designed to constrain the output of the model to better reflect the
governing physics. For instance, Sadoughi and Hu (2019) proposed a physics-
based (or better termed as physics-informed) CNN for fault classification of
rolling element bearings. The uniqueness of their physics-informed CNN is that
the kernel functions in its first convolution layer are designed based on
known physical knowledge about bearings and their fault characteristics. Kim
et al. (2022) developed a CNN-based classifier for gearbox fault diagnostics
based on a health-adaptive time-scale representation that was built using
multiscale convolutional filters. These convolutional filters were specially
designed to incorporate known physics of faults, more specifically, the fault-
related time and frequency characteristics in the vibration signals from a
faulty gearbox. Other physics-informed ML approaches to fault diagnostics
utilize frequency components of interest to customize the loss function used
in ML model training, as reported in Shen et al. (2021), Russell and Wang
(2022). These physics-informed ML approaches can help address the data
challenge in ML-based fault diagnostics, as they can improve the
generalizability of ML models to test samples unseen during model training,
including those that may outside of the training distribution, as more
extensively discussed in 3.4. A a result, compared to a purely data-driven ML
model, a physics-informed ML model may be trained on a smaller set of faulty
data to achieve a similar generalization performance.

  * •

_Fault diagnostics based on physics-based modeling_ : For physical systems in
the SHM domain, such as civil infrastructure systems and large aircraft
structures, or in safe-critical industrial applications, it is unlikely one
would be able to collect faulty data since these physical systems are
generally well-maintained, and failures are extraordinarily rare. Thus, the
amount of available faulty data is very limited. Even if it is possible to
collect a reasonable quantity of faulty data and build a training dataset,
this dataset may not be representative of other structures operating under
different loading and environmental conditions. As a result, it is challenging
to build ML models readily deployable for structures or machines that these
models have not seen. One way to tackle this challenge is model-based fault
diagnostics taking advantage of physics-based models. This approach classifies
the health state of a physical system or quantifies its fault/damage by
studying model predictions under healthy and faulty conditions (Park et al.,
2016), comparing model prediction with sensor data, or using sensor data to
update a physics-based model (see Bayesian model updating in Sec. 4.2),
without resorting to building data-driven ML models. For example, Park et al.
(2016) proposed a model-based fault diagnostics method to detect faults in
planetary gears. A lumped parametric model was developed to simulate
transmission error signals that contained gear fault-related information. The
development of this physics-based model allowed for a simulation-based
investigation into how to extract fault-sensitive features from these
transmission error signals, informing the implementation based on actual
sensor data in practice.

####  4.4.4 Failure prognostics approaches

Failure prognostics approaches can be broadly divided into _physics-based_ ,
_data-driven_ , and _hybrid approaches_. An ideal example of a physics-based
approach is using a physics-based model, as described in Sec. 3.2, depicting
the physical processes, in combination with a degradation model, depicting the
time evolution of damage or degradation parameters used in the physics-based
model. Such a physics-based approach has been attempted in Ramadesigan et al.
(2011), as described in Sec. 4.2.4. These ideal physics-based approaches can
capture the physical processes and failure mechanisms, therefore offering good
interpretability and generalisability. However, their applicability to
prognostics is limited by difficulties in calibrating large numbers of
parameters in the physics-based models due to, for example, poor model
identifiability, as mentioned in Sec. 3.2 and 4.2 (a), an insufficient
understanding of the degradation mechanisms and their long-term evolution, and
high computational costs (not suitable for real-time decision making). These
limitations motivated researchers to look for models that are not strictly
physics-based but rather semi-empirical or empirical. Examples of semi-
empirical models include the Paris’ law equation describing fatigue crack
growth widely used for fatigue damage prognostics (Sankararaman et al., 2011)
and the square-root-of-time dependence capturing battery capacity fade due to
the solid electrolyte interphase growth for battery lifetime prediction (Bloom
et al., 2001, Smith et al., 2011). Empirical models do not have any physical
meaning but show good agreement with data. Three representative examples of
empirical models are (1) the statistical degradation models described in Sec.
3.3 (b) for general-purpose prognostics (i.e., health forecasting of a general
physical system), (2) exponential degradation models, originally developed for
general-purpose prognostics (Gebraeel et al., 2005) and further developed and
applied to bearing prognostics (Li et al., 2015) and battery prognostics (He
et al., 2011), and (3) power-law models for battery prognostics (Lui et al.,
2021, Gasper et al., 2021), one of which reduces to the traditional model when
the temporal power exponent equals 0.5.

Physics-based approaches, plus prognostics approaches using semi-empirical and
empirical mathematical models, form a broader category called _model-based
prognostics_. Figure 21 shows a diagram illustrating a model-based approach
using an empirical exponential model for battery capacity forecasting and RUL
prediction. This battery prognostics problem can be formulated as a state-
space model for parameter estimation, as discussed in Sec. 4.2.4, and solved
using particle filtering, as detailed in Sec. 4.2 (b). The model-based
approach outlined in Fig. 21 is used for online estimate of the RUL of a
battery cell. In a digital twin, the RUL prediction could then be used in
other process and control optimization algorithms to, for example, determine
the optimal time to remove the cell from its first-life application. This
application of a model-based prognostics method within a digital twin
framework is extensively investigated as a case study in Part 2 of this review
on digital twin.

Figure 21: A schematic illustrating model-based prognostics for battery
capacity forecasting and RUL prediction. $Q$ denotes the capacity of a battery
cell normalized by its initial capacity, and $k$ denotes the cycle number. EOL
in the figure stands for end of life.

Suppose a good degradation model is unavailable, or degradation parameters are
hard to estimate from observational data. In that case, data-driven approaches
may be a better alternative to mode-based approaches, assuming medium to large
volumes of degradation data are available. This is usually only possible in
applications where life safety issues are not a concern. Data-driven
approaches can be categorized further into direct mapping and time series
forecasting approaches, as illustrated in Fig. 22. Direct mapping approaches
bypass health forecasting and directly map features (ML model input) to RUL
(ML model output) using ML models such as standard feedforward neural networks
(Huang et al., 2007, Tian, 2012), deep CNN (Li et al., 2018, 2019a), and LSTM
recurrent neural networks (Yuan et al., 2016, Huang et al., 2019). Some
variants of direct mapping approaches do not output RUL and instead estimate a
health indicator (e.g., an overall measure of engine/milling machine health
(Malhotra et al., 2016) or bearing health (Guo et al., 2017), battery capacity
(Shen et al., 2020)), or a battery degradation parameter such as an electrode
aging parameter (Tian et al., 2021, Thelen et al., 2021)). ML-based
forecasting approaches predict how a health indicator or reliability metric
(such as time to failure or survival probability) evolves beyond the current
time as an intermediate step for RUL prediction. ML models used for
forecasting include support vector machine (Pham et al., 2012), relevance
vector machine (Wang et al., 2013), Gaussian process regression (Richardson et
al., 2017), feedforward neural networks (Fink et al., 2014), and LSTM
recurrent neural networks (Nemani et al., 2021), just to name a few.
Applications of DL to PHM are well-reviewed in Khan and Yairi (2018), Zhao et
al. (2019), Fink et al. (2020), with the first and second papers focusing on
machinery health monitoring, the second paper providing a comparative study on
the tool wear prediction performance of ten ML and DL algorithms, and the
third paper focusing on successes of DL in various application domains
including PHM.

Figure 22: A graphical comparison of two distinct strategies for data-driven
prognostics. HI and EOL stand for health indicator and end of life,
respectively.

Data-driven ML approaches can discover complex degradation patterns relevant
to RUL prediction from large volumes of data, are simpler to implement than
physics-based approaches, and are agnostic to the physical and degradation
processes. However, large ML models can have many more parameters than
physics-based models and require large training datasets to ensure low risks
of overfitting. In reality, collecting large and representative run-to-failure
datasets can be costly and time-consuming. Another well-known issue with ML
models is that they have difficulties in generalizing to out-of-distribution
data unseen during model training, as also discussed in Sec. 3.4 and 4.3, and
may produce predictions that do not make physical sense. In conclusion, the
performance of purely ML models is limited by the quality and quantity of the
available training data. Without consideration of physics, the accuracy
numbers on a test dataset that falls within the training data distribution may
not hold for other datasets outside the training distribution.

As discussed above, purely physics-based and data-driven approaches have
complementary advantages that can be leveraged to tackle the limitations. For
example, physics-based models offer opportunities to generate synthetic data
that can help alleviate the data-size requirement of data-driven approaches,
extending the applicability of these approaches to cases where run-to-failure
trajectories are scarce. Using physics-based models to generate synthetic
training data is illustrated as Approaches 2, 3, and 5 in Fig. 10. Another
example is that data-driven ML models can learn complex bias functions from
data to compensate for discrepancies in physics-based models. This strategy in
combining physics-based and model-based approaches is illustrated in Fig. 10
as Approaches 4 and 5, where the bias functions respectively capture the
prediction inaccuracy of a physics-based model (Approach 4) and the prediction
inaccuracy of an ML model trained with physics-based synthetic data (Approach
5). All the approaches described in Sec. 3.4 can be adapted for failure
prognostics, with some successes recently reported in Chao et al. (2022),
Kohtz et al. (2022), Yucesan and Viana (2022).

###  4.5 Ontology-based reasoning

Ontology maps and knowledge graphs provide a straightforward way to represent
empirical knowledge in a structured manner. The unified knowledge engineering
and representation through descriptive logic lays a firm foundation for
knowledge reasoning over knowledge graph in the semantic space. In practice,
they are primarily implemented through ontology markup languages (Chen, 2010),
such as Web Ontology Language, XML Schema, Resource Description Framework
Schema (RDFS).

The proper construction of an ontology map and knowledge graph facilitates
knowledge inference along the semantic dimension (Chen et al., 2020a). In
general, knowledge reasoning methods can be categorized into three classes.

  1. 1.

Logic rules-based reasoning. Rules-based reasoning manifests in several forms:
(a) first-order predicate logic rules, such as First-Order Inductive Learner
(FOIL) (Schoenmackers et al., 2010), that is to use the predicate as the basic
unit for knowledge reasoning; (b) manually inject user-defined or simply
learned rules in knowledge graph to generate new knowledge, such as the
inference component in the Never-Ending Language Learning system (NELLs)
(Mitchell et al., 2018). (c) to incorporate frequently present patterns,
constraints or paths to extract new knowledge. That is to reason with the Web
Ontology Language. (d) identify path rules and inject them into knowledge
graph to facilitate reasoning. For example, Lao and Cohen (2010) utilized a
path ranking algorithm to identify the associations between edge types and the
instances of edge types, and exploited such associations to predict missing
edges in the graph.

  2. 2.

Distributed representation-based reasoning. This line of studies aim to
project the basic elements in knowledge graph (i.e., entities, relations,
attributes) into a continuous vector space through embedding-based approaches
(Nickel et al., 2011), including tensor decomposition, distance-based mapping,
and semantic matching models. The distributed representation allows to
discover latent rules and connections in the transformed space.

  3. 3.

Neural network-based knowledge reasoning. In recent years, more and more
studies have explored the representation learning ability in deep neural
networks for knowledge reasoning. Specifically, the CNN-based learner enables
to account for the fine-grained multi-source heterogeneous information encoded
in entity description (Xie et al., 2016); the recurrent neural network is
capable of reasoning about multi-hop relations in knowledge graph (Neelakantan
et al., 2015). These unique advantages have positioned neural network as an
appealing paradigm for knowledge inference in the semantic space.

Over the past few years, there has been an increasing investigation of
ontology techniques for data management as well as knowledge representation
and inference in digital twins. For example, Ladj et al. (2021) combined
business rules with the knowledge deriving from industrial data analytics to
build ontology-based knowledge model for incident detection in the machining
process, and the knowledge base served as an inference platform to detect
detrimental incidents. Lim et al. (2020b) employed knowledge graph-based
method to accommodate heterogeneous data sources, such as asset ontology,
environment ontology, and system ontology. The comprehensive data integration
in knowledge graph drives the inference of smart solutions for asset
configuration, resource planing and component maintenance. Liu et al. (2021d)
developed multi-scale product quality knowledge model to characterize the
relationship between product quality and the product quality factors
explicitly.

##  5 Virtual-to-physical (V2P) twinning enabling technologies

Figure 23 summarizes the modeling and twinning methods by application in
different phases of a physical system’s life cycle. As illustrated in this
figure, there are numerous ways of establishing V2P connection, such as system
reconfiguration, process control, production planning, maintenance scheduling,
and path planning. In this section, we mainly concentrate on two classical V2P
twinning enabling technologies, namely model predictive control and predictive
maintenance.

Figure 23: Classification of modeling and twinning techniques according to
phase of a physical system’s life cycle

###  5.1 Model predictive control

Model predictive control (MPC) includes a wide range of advanced control
methods, which in common use a model to predict the future behavior of the
process and determine the optimal control within a set of constraints. MPC is
an essential component for smart manufacturing, which enables not only the
optimal use of resources for high-quality products, and but also quick
production responses to changes in market demands and supply chains (Lu et
al., 2016). It is one of the most widely used methods to establish the V2P
connection in a digital twin.

Figure 24: MPC structure Nikolaou (2001)

Figure 24 shows a general structure of MPC, including a process model, an
objective function, process measurements, constraints and sampling points
(Nikolaou, 2001).

Assuming an arbitrary process with a state space representation:

$\begin{split}&{{\bf{x}}({k+1})}={\bf{f}}({{\bf{x}}(k}),\;{{\bf{u}}(k})),\cr&{{\bf{y}}({k+1})}={\bf{h}}({{\bf{x}}({k+1})}).\cr\end{split}$
(10)

MPC minimizes a user-defined cost function $J$,

$\begin{split}\mathop{\min\;}\limits_{\bf{u}}\sum\limits_{t=0}^{k}{J({\bf{x}}(t),\;{\bf{u}}(t))},\cr\end{split}$
(11)

in which
$J({\bf{x}}(t),\;{\bf{u}}(t)):{\mathbb{R}^{s}}\times{\mathbb{R}^{c}}\to\mathbb{R}$
is the cost function at $t$.

For a trajectory following process control problem, the objective functions
can be any norm of the tracking error between the reference vector $\bf{r}$
and the model output $\bf{y}$, as shown in Eq. (12). The optimization problem
is defined under hard state and input constraints:

$\begin{split}&\min_{u}\;\;\;\;{\sum^{N_{2}}_{i=N_{1}}}\;\lVert{{\bf{r}}({{k}+i}\mid{k})-{\bf{y}}({{k}+i}\mid{k}})\rVert\cr&s.t.\;\;\;\;\;\;\;\;\;{\bf{u}}_{lb}\leq{\bf{u}}({k}+j\mid{k})\leq{\bf{u}}_{ub}\cr&\;\;\;\;\;\;\;\;\;\;\;\;\;\;{\bf{y}}_{lb}\leq{\bf{y}}({k}+j\mid{k})\leq{\bf{y}}_{ub}\cr&\;\;\;\forall\;\;i\in\\{N_{1},...,N_{2}\\}\;\;and\;\;j\in\\{0,...,N_{u}\\},\cr\end{split}$
(12)

where $N_{1}\leq N_{u}\leq N_{2}$.

The iterative, finite-horizon optimization of a process can be further
illustrated in Fig. 25. At time $t_{k}$ the current process measurements are
acquired and a cost minimizing control strategy is computed to minimize the
error between the system output $y$ and the given reference $r$ for a horizon
$N_{2}$. $N_{2}$ must be long enough to represent the effect of a change in
the control commands $u$ on the control variable $y$. If there a pure delay
exists, the sum of the objective function starts from $N_{1}$. Only the first
value of the optimal control commands is implemented, then the process
measurements are sampled again and the optimization is repeated, yielding new
sets of control commands and new predicted state path. The prediction horizon
keeps being shifted forward so MPC is also called receding horizon control.

Figure 25: Iterative Horizon Receding Control of MPC Schwenzer et al. (2021)

MPC has become the most widely implemented advanced process control technology
and is offered by major automation suppliers, with applications in process
industries such as refining, petrochemicals, paper and food processing
(Industry, , Automation, ) . In the last decade, MPC is extended and also
adopted in power plant control, build thermal management and discrete
manufacturing (Deng et al., 2014, Farzan et al., 2014, Swief et al., 2019). In
the era of smart manufacturing, the process and manufacturing industries are
embracing tremendous opportunities brought by the Internet-of-Things (IoT) and
advanced data analytical technologies, which also enhance the application of
MPC by expanding the objective function to more some general performance
measure (Ellis, 2015), and by combining machine learning techniques in process
modeling (Essien and Giannetti, 2021). Economic MPC employs cost function
connected to the economics of the considered process which makes economic MPC
well suited as a tool to achieve the goals of smart manufacturing. Machine
learning-based MPC is reported to use recurrent neural network modeling
approaches for a general class of nonlinear dynamic process systems to improve
prediction accuracy and control performance (Wong et al., 2018). There are
also efforts reported to integrate a DL architecture with MPC for robot
manipulation control (Lenz et al., 2015). Rather than developing a dynamics
model from first principles, the future MPC will more likely to learn digital
models using a novel deep architecture and learning algorithms directly from
data. For example, some of the data-driven models summarized in Sec. 3.3 can
be used as predictive models for MPC in digital twins.

###  5.2 Predictive maintenance

A recent industry-wide shift called _Industry 4.0_ capitalizes on increased
interconnectivity and automation to make machines on a production floor
smarter. Benefits of this digital transformation to Industry 4.0 include
increased productivity, improved workflow efficiency, and improved product
quality. A key enabling step of Industry 4.0 is predictive maintenance, mostly
discussed in the manufacturing domain. Predictive maintenance is a proactive
approach to equipment maintenance that centers on (1) identifying signatures
in sensor data (continuous monitoring or periodic inspections) indicative of
changes in equipment health (see fault diagnostics discussed in Sec. 4.4.3),
(2) predicting when a machine, component or part might fail (see failure
prognostics discussed in Sec. 4.4.4), and (3) scheduling maintenance work
during planned downtime just before equipment failure, sometimes referred to
as “just-in-time maintenance” (Lee et al., 2013a, b).

Now let us answer a simple question: why is predictive maintenance needed, for
example, for a rotating machine (e.g., motor, pump, and fan)? The answer is
quite simple. The unexpected failure of this rotating machine often incurs
high maintenance and downtime costs, reduces customer satisfaction with a
produced good, and may even cause human injuries and fatalities. These
consequences also impact the machinery manufacturer by tarnishing their
reputation and potentially putting them at a competitive disadvantage.
Therefore, it will be value-added to develop, implement, and deploy predictive
maintenance solutions, especially justifiable for high-value equipment whose
failure cost is high. Note that predictive maintenance differs from the more
traditional _reactive maintenance_ , which performs maintenance work after an
equipment failure and may lead to costly unexpected downtime, and _preventive
maintenance_ , which performs maintenance work at a fixed time interval,
typically more often than necessary, and may cause a waster of time and
resources due to unnecessary maintenance.

A typical ML pipeline for building and deploying a predictive maintenance
solution for a rotating machine is shown in Fig. 26 and consists of four main
steps.

  1. 1.

Acquire sensor data (P2V): This first step identifies critical machine assets
for condition monitoring and installs sensors on these assets to measure
signals that are sensitive to machine health. For example, sensor signals
sensitive to bearing health include acoustics, vibration, oil wear particle
count, and temperature, ranked from the earliest to latest in detecting a
bearing fault. Note that in SHM, the placement of a sensor network needs to be
optimized to maximize the amount of useful information for structural damage
assessment, as discussed in detail in Sec. 2.2.1 of Part 2 of the review
paper.

Figure 26: An overview of an ML pipeline for predictive maintenance

  2. 2.

Preprocess data (P2V): Field data is rarely clean; it contains outliers and
noise that need to be removed to avoid corrupting ML model performance. An
example in bearing health monitoring is that vibration signal processing is
almost always needed to denoise vibration signals by, for example, designing
frequency-domain filters used to remove background noise irrelevant to machine
faults (Antoni and Randall, 2006). Another example is that outliers in battery
voltage and current measurements must be removed before they are fed into ML
models for capacity estimation or RUL prediction enabling battery health
management.

  3. 3.

Train ML models (P2V): This step trains ML models to classify the machine
condition as healthy/faulty (green/red) or healthy/lightly damaged/heavily
damaged (green/yellow/red) and estimate the RUL of the machine. For both fault
classification and RUL prediction, the predictive uncertainty of the ML models
needs to be estimated, accurately reflecting the models’ confidence in making
a health class/RUL prediction (see Sec. 2.1.1 in Part 2 of the review paper
for a detailed discussion on ML model uncertainty). A probabilistic estimate
of RUL can be in the form of a mean RUL and an associated confidence interval
representing the predictive uncertainty.

  4. 4.

Optimize maintenance decision making (V2P): This last step optimizes
maintenance schedules based on the predictive information from Step 3. In
practice, what is preferred is to repair or replace equipment during a
scheduled downtime just before failure occurs, without affecting the chance of
meeting production goals. Maintenance decisions need to be made from a risk
perspective, given that ML model predictions are associated with varying
degrees of uncertainty. A dedicated review of maintenance schedule
optimization will be given in Sec. 2.2.3 (c) in Part 2 of the review paper.

Digital twin can be an enabler of predictive maintenance. As discussed earlier
in Sec. 4.4.3 and 4.4.4, one of the main issues with diagnostics/prognostics
is the lack of faulty/run-to-failure data (i.e., data acquired from a physical
system whose health state or RUL is known). This data challenge can be
addressed by using a physics-based model in a digital twin, calibrated offline
and updated online (Sec. 4.2), to generate synthetic faulty/run-to-failure
data. This strategy for tackling the data challenge has already been discussed
in Sec. 4.4.3.

An example where a digital twin can help with the online phase of predictive
maintenance is that a physics-based model in a digital twin, considering both
physical and degradation processes, takes sensor data as input, estimates
digital state variables that significantly affect lifetime, and predicts RUL.
Sometimes, ML models can assist physics-based models in estimating the digital
state variables. This strategy is similar to the ML-assisted approach
(Approach 6) shown in Fig. 10, except that ML models may not always be used in
the digital twin. A lot of success stories have been reported in the
manufacturing industry on this strategy, for example, Ansys Twin Builder and
PTC thingworx collaboratively working on estimating the maximum temperature of
the rotor, stator, and case of an electric motor based on sensor measurements
to predict the motor’s RUL (ANSYS, ).

Figure 27: An IIoT platform for predictive maintenance of industrial assets

Another industrial success story is the predictive maintenance alarm system in
MATLAB developed by Baker Hughes for the positive displacement pumps on their
trucks. The core of this system is the pump health monitoring software that
analyzes real-time pressure, vibration, and timing signals and uses a trained
neural network to predict pump failures. It is shown that the software can
classify pump health into one of three health classes, namely “normal
operation”, “monitor closely”, and “maintenance needed”, enabling better
maintenance planning. However, it is unclear whether the software has
predictive capabilities (i.e., whether it can predict the RUL of a pump). As
the emergence of IIoT platforms facilitates the industry-scale adoption of
digital twins, we can expect the number of industrial success stories of
predictive maintenance to continue increasing rapidly. Figure 27 illustrates
an industry-grade IIoT platform for proactively predicting and preventing the
failure of rotating machinery on a manufacturing floor. In this IIoT platform,
a network of smart devices perform vibration analysis, fault detection, and
failure prediction on the edge, and then send detection and prediction results
to a web-based user interface that provides real-time analytics, dashboards,
and alert capabilities to the maintenance and reliability engineers
responsible for keeping rotating equipment in service. Maybe talk about the
possibility of building centralized digital twins and distributed digital
twins on the edge and sharing ML model information among clients without
sharing data using federated learning in Sec. 6.1.

###  5.3 Real-time perception and decision making

After sensor measurements characterizing the state of a physical system are
transmitted to the digital counterpart, simulation and optimization models
enable the P2V and V2P connections within the digital twin. Specifically,
simulation enables precisely modeling of the behavior of the physical system
under uncertainty (P2V), and optimization drives control, maintenance, and
decision making on the physical asset (V2P) given its current state. Ideally,
the simulation and optimization algorithms in the digital model will be
performed in real-time and will not have significant latency. However, this is
not the case, as simulation and optimization are known to be time-consuming
and computationally expensive, which substantially hampers their ability to
provide timely results in digital twins which operate on physical systems with
short time scales. To this end, the power of ML can be fully exploited to
alleviate many of the inherent issues incurred by using classical simulation
and optimization models. Various ML techniques can be used to facilitate real-
time or near real-time (sometimes termed just-in-time) perception and decision
making. Specifically, ML can be leveraged to infer the parameters of or
replace the simulation model for accurately modeling the physical system
(Goodwin et al., 2022). For example, Zohdi (2020) used ML to ascertain the
parameters of a multi-submodel system for fast and accurate simulation of fire
propagation in order to support continuous update of the digital twin of the
physical counterpart in near real-time. In each case, the simulations are
expedited by using pre-trained ML models which are very computationally
inexpensive to evaluate. These examples are a few of many where ML can be used
to speed up simulation operations in digital twins.

Regarding optimization, the expensive iterations in classical optimization
algorithms can be replaced by an end-to-end ML-based optimization model proxy.
In this method, the ML-based optimizer typically adopts a predict-then-repair
paradigm. The basic idea is to first reformulate the optimization problem as a
learning problem and then incorporate fast feasibility-restoration procedures
to guarantee the feasibility of the produced solution (Bengio et al., 2021,
Barry et al., 2022). ML-based optimization proxies have been shown to surpass
conventional optimization algorithms by a large margin in computational speed.
For example, Van Hentenryck (2021) showcased the role of ML in approximating
the optimal solutions of power flow problems and the great potential of ML-
based optimization proxy in speeding up existing algorithms. Chen et al.
(2022) proposed an end-to-end DL model that can predict an optimal solution
for Security-Constrained Economic Dispatch (SCED) in milliseconds to meet the
need of power gird in real-time operations. Creating this end-to-end DL model
overcomes the fundamental limitation of classical optimization techniques in
computational efficiency.

Ultimately, machine learning will play a large role in enabling real-time
simulation, optimization, and control of physical systems. Researchers are
actively investigating methods to increase the speed of physics-based
simulations and traditional optimization algorithms. These techniques will
pave the way for digital twins which operate on the millisecond and sub-
millisecond timescales.

##  6 Perspectives on modeling and twinning in digital twins

###  6.1 Federated learning in digital twins

For data-driven digital twins, a single system or component may not have
sufficient condition monitoring data to train a representative model. This is
particularly due to the fact that industrial and infrastructure systems are
operated under varying conditions. Thus, monitoring a system for a limited
period of time may not cover all possible expected operating conditions.
Moreover, since faults or critical system states are rare, one single system
may not observe a sufficient number and type of faults to train the model.
Therefore, data from similar systems can be leveraged to improve the
performance and generalization of the developed model.

Accumulating data from multiple similar systems can eventually result in
large-scale datasets that enable representative data-driven digital twins to
perform well under all relevant operating conditions. Generally, compiling
such representative datasets could be achieved through sharing the data
between different stakeholders, for example between different operators of
similar systems. However, even without direct competition, companies are often
reluctant to share data mainly due to the concern of disclosing proprietary
company information. Federated learning could potentially overcome that
concern enabling to take advantage of the benefits of a shared model without
the need of data disclosure. Federated learning is a privacy preserving
learning concept. It enables to benefit from the local data subsets (e.g., of
several power plant operators) without the need of sharing the data (McMahan
et al., 2017, Li et al., 2020a). It is a decentralized machine learning
technique that enables to train a neural network across multiple local
datasets without exchanging or sharing data samples across the decentralized
nodes. A global model is constructed by aggregating (typically averaging) the
parameters (either gradients or weights) of the locally trained models. Hence,
it can make predictions based on the experience of the entire fleet but
without the stakeholders having access to each other’s data. Each individual
system retains its own collected condition monitoring data on the edge,
without sharing it with other systems. It gets a current version of the model,
improves it based on its collected data and sends the updates to the model.
Two different concepts of federated learning are typically distinguished:
cross-device federated learning (also referred to as sample-based or
horizontal federated learning) (Yang et al., 2019) and cross-silo federated
learning (also referred to as feature-based or vertical federated learning)
(Yang et al., 2019). In the context of digital twins, in fact, particularly
the horizontal federated learning concept is the most relevant. However, there
are also contexts where vertical federated learning could be applied.

Privacy preservation is an essential characteristic of federated learning.
However, even though data is not directly shared, only sharing model updates
during the training process can still potentially reveal sensitive information
(Zhu et al., 2019a). Different privacy preserving approaches have been
proposed that provide privacy guarantees and prevent potential indirect data
leakage (Truex et al., 2019, Chen et al., 2020b, Yin et al., 2021). Federated
learning has not yet been broadly applied for digital twins. Federated
learning will then face similar challenges as other data-driven approaches,
such as the lack of representative datasets, differences in operating
conditions and system configurations between different entities of the digital
twins. One of the potential directions to tackle these challenges is combining
federated learning and transfer learning.

###  6.2 Domain adaptation in digital twins

Data-driven digital twins can show a significant performance drop if they are
applied under operating or environmental conditions that they have not been
trained on. However, evolving operating conditions are very common in
practical applications. Moreover, if a data-driven digital twin is applied
across different units of a fleet, the performance may also drop significantly
due to discrepancies either in system configurations or operating regimes of
different units of a fleet. It is, however, unrealistic to assume that
sufficient data can be collected for all units and for all operating
conditions. Different operating and environmental conditions and different
system configurations result in the so-called domain shift or distributional
shift, a change in the data distribution between the source (training) and the
target (testing) dataset (Fink et al., 2020, Li et al., 2022), which is also
described in Sec. 4.3 in the context of machine learning. Typically, no labels
are available in the target dataset, so that fine-tuning of the trained model
cannot be performed. Further to the two scenarios of domain shifts caused by
differences in operating conditions or by differences between different units
of a fleet, a third scenario, particularly relevant to digital twins is the
synthetic to real gap existing between synthetically generated data stemming
potentially from model-based simulators and the actual observation (Bousmalis
et al., 2018, Wang et al., 2021a, Li and Lee, 2021).

Domain adaptation, a subfield of transfer learning, aims at overcoming the
differences between the domains so that the algorithms are able to generalize
across domains and learn domain-invariant features. Domain adaptation solves
the domain shift problem by finding a mapping from the source data
distribution to the target distribution. Different approaches for domain
adaptation have been proposed. One of the most obvious approaches to overcome
the domain gap is distribution alignment approaches by statistics matching,
including Adaptive Batch Normalization (AdaBN) and Automatic DomaIn Alignment
Layers (AutoDIAL), using the statistics of source and target mini-batches at
different layers to align the two distributions (Long et al., 2015, Yang et
al., 2016, Carlucci et al., 2017). Some of the other commonly applied
approaches have been relying on the principle of minimizing a divergence-based
criterion between source and target distributions, including such criteria as
Maximum Mean Discrepancy (MMD) or Wasserstein Discrepancy (Yan et al., 2017,
Lee et al., 2019). Due to the superior performance on many different tasks,
adversarial domain adaptation has recently emerged as one of the most
frequently applied approaches to confront domain shift (Ganin and Lempitsky,
2015, Tzeng et al., 2017, Ganin et al., 2016). A domain discriminator is then
applied to ensure that the extracted features become indistinguishable between
the two domains, while the performance on the main classification task is
maximized.

Domain adaptation has been recently broadly applied to many different tasks in
prognostics and health management, partly also for digital twins (Li et al.,
2022, Liu et al., 2020b). One of the additional challenges that is typically
encountered in real applications is that the label space of the source
(training) is often not identical with that of the target (testing) dataset.
This is particularly relevant in the context of fault diagnostics, where
different systems may have experienced different fault types within the
(short) observation time period. Possible setups in real applications are that
some classes are missing in the target dataset (Partial Domain Adaptation),
some classes may also be missing the source datasets (OpenSet Domain
Adaptation). Or we may be even facing only a partial overlap of the label
space between source and target datasets, resulting in a combination of the
two types (Partial & OpenSet Domain Adaption) (Boris et al., 2021). Some
approaches have been recently proposed to tackle such challenges (Li et al.,
2020c, Wang et al., 2020c, Li and Zhang, 2020, Rombach et al., 2022), on the
one hand by overcoming misalignment in case of different label spaces and on
the other hand by developing generative models for a controlled generation of
the missing fault types. While domain adaptation has been flourishing for
classification tasks, its application to regression has been rather limited
(Boris et al., 2021). This has been also reflected in the tasks tackled in the
field of prognostics and health management, where the main application field
has been fault diagnostics, with only few research works addressing
prognostics tasks as well (Wu et al., 2019b, da Costa et al., 2020). Similar
to classification tasks, domain adaptation for regression is also facing the
challenge of label discrepancy between the source and the target label spaces
(Boris et al., 2021).

###  6.3 Deep reinforcement learning in digital twins

Reinforcement learning is concerned with learning an optimal policy for
sequential decision making problems through an agent’s interactions with the
environment over time by trial and error (Mnih et al., 2015). Towards policy
learning, reinforcement learning not only considers the immediate reward of an
action, but also takes into account the accumulative reward resulting from the
action in a discounted manner. In general, reinforcement learning is composed
of several key components: state space, action space, policy to map from state
space to action space, state transition probability, and reward function. The
advantage of reinforcement learning in long-term cumulative reward modeling
has put it in a good position to tackle complex sequential decision making
problems.

Deep neural networks have recently been used to approximate such functions as
value functions (value function is used to predict the expected cumulative
reward indicting how good a state-action pair is), policies (policy describes
which action to take given a specific state), reward functions, and state
transition function. For example, the well-known deep Q-network (DQN)
developed by Mnih et al. (2015) approximates the state-value function in a
Q-Learning framework with a neural network. Similarly, deep deterministic
policy gradient (Silver et al., 2014) concurrently learned the action-value
function and the policy.

For digital twins, deep reinforcement learning has only recently started to be
applied. There are two main directions that have been pursued: 1) Establishing
the P2V connection for model updating; 2) Building upon the operational
digital twin, deep reinforcement learning has been used for control tasks and
decision support, in particular due to its learning capabilities. As one
example for the P2V connection, Tian et al. (2022a) trained an agent with
Lyapunov-based maximum entropy deep reinforcement learning to perform model
calibration and infer the uncertain model parameters of the physics-based
model, such that the response of physical model matched the observed data. The
proposed framework provides a similar performance for the model calibration as
model-based approaches, such as UKF, but provides real-time update capability.
As an example for the the P2V connection, Xia et al. (2021a) investigated the
application of deep reinforcement learning in industrial process control for
smart manufacturing, where deep reinforcement learning was trained to learn an
intelligent operations scheduler. Some other examples include a deep
reinforcement learning algorithm trained to find the optimal off-loading
policy in an IoT digital twin framework with a goal of reducing energy
consumption while enhancing data processing efficiency Dai et al. (2020).
Moreover, Khan et al. (2020) provided an interesting perspective postulating
that the digital twin as a technology for decision automation could be treated
as a deep reinforcement learning model, where the physical system provides
states and rewards information to the digital system via the P2V connection
and digital system recommends actions to physical system to maximize the long-
term reward.

Future research directions could make progress both directions: on the one
hand integrating deep reinforcement learning more in the learning process of
data-driven (and hybrid) digital twin, such for example for meta-learning
tasks; on the other hand deep reinforcement learning is essential for
interweaving more the digital twin and decision support, respectively control
tasks enabling to harness more benefits from digital twin technology. One of
such directions could be fore example, prescriptive operations, where deep
reinforcement learning is used to prescribe an optimal course of actions with
a potential objective to prolong the remaining useful lifetime Tian et al.
(2022b).

##  7 Conclusion

In this paper, we propose a five-dimensional definition for digital twin based
on how data flows between the physical and virtual systems. The five
dimensions are the: physical system, digital system, an updating engine (P2V),
a prediction engine (V2P), and an optimization dimension (OPT). The five-
dimensional digital twin provides a clear and complete picture on the
different level of interactions between the physical system and its digital
counterpart (i.e., data exchange, modeling, actions). Following the five-
dimensional digital twin definition, we comprehensively review the underlying
techniques that are commonly used in the state-of-the-art literature to enable
each dimension of the digital twin and present our findings in a series of two
papers. This paper covers the enabling techniques for three specific
dimensions of digital twin: physical system modeling, P2V, and V2P. In the
following paper, we will concentrate on articulating how to fully incorporate
UQ and optimization in the three dimensions of digital twin described in this
paper.

Specific to this paper, we provide a comprehensive and rigorous examination on
various means of modeling different aspects of a physical system (e.g.,
geometry, structure, system dynamics, relationship), the key P2V enabling
techniques, and the actions and benefits (e.g., model predictive control,
predictive maintenance) enabled by V2P twinning techniques across different
phases of a physical system’s life cycle. The accurate representation of a
physical system together with the constant interactions enabled by P2V and V2P
techniques form a closed loop to drive the implementation of digital twin in
practice. In Part 2 of the review paper, we investigate how to ensure robust
performance of a digital twin model by incorporating UQ and optimization
techniques into the underlying enabling technologies.

## Appendix A: A generic particle filter algorithm

Particle filtering is a widely used Bayesian filtering technique for state
estimation in generic state-space models and DBNs. It is a key enabler in many
digital twin applications where probabilistic model updating with state
estimation plays an essential role. Let us briefly look at how particle
filtering works. Particle filters represent a family of algorithms that
recursively execute Bayesian filtering through sequential Monte Carlo
simulation (Arulampalam et al., 2002). In a particle filter, the marginal
posterior of a state at the current time step $k$ is approximated using a
large set of weighted particles. This approximation takes the following form

$\begin{split}p({{\bf{x}}_{k}}|{{\bf{y}}_{1:k}})\approx\sum\limits_{j=1}^{{N_{p}}}{w_{k}^{j}\delta({{\bf{x}}_{k}}-{\bf{x}}_{k}^{j})},\end{split}$
(13)

where ${\bf{x}}_{k}^{j}$ is the jth particle of the state x at the kth time
step, $w_{k}^{j}$ is its associated weight, NP is the total number of
particles, and $\delta$ is the Dirac delta function. Algorithm 1 depicted in
Fig. 28 gives the pseudo-code of a generic particle filter (Hu et al., 2018,
Cappé et al., 2007). Three key steps of the particle filtering procedure are
described as follows:

  * •

State Transition: Each particle of the state transitions forward in time by
one step (from the previous, $(k–1)^{th}$ step to the current, $k^{th}$ step
according to the state transition equation in Eq. (13) (Line 5). These
particles are equally weighed ($1/N_{P}$), and they form the prior of
${{\bf{x}}_{k}}$ before the current measurement is used.

  * •

Weight Evaluation and Normalization: The weight of each particle is updated
using the likelihood of the current measurement given the state values of the
particle, i.e.,
${\bf{w}}_{k}^{i}\propto{\bf{w}}_{k-1}^{i}p({{\bf{y}}_{k}}|{\bf{x}}_{k}^{i})$
(Line 6). The likelihood is calculated by comparing the measurement
distribution predicted by the measurement equation ${\bf{g}}$ in Eq. (13) to
the actual measurement ${{\bf{y}}_{k}}$. After that, the updated weights are
normalized, and the normalized weights sum up to 1 (Lines 8-12).

  * •

Resampling: Particles with negligible weights are replaced by new particles
that are copies of particles with higher weights (Line 13). This step
mitigates the issue of particle degeneracy, where the weights become overly
concentrated on a very small subset of particles (in an extreme case, only all
by one particle have close-to-zero weights).

Figure 28: Pseudocode of a generic particle filter algorithm

## Appendix B: Decomposition of likelihood and Bayesian inference in a DBN

Using the DBN given in Fig. 16 as an example, the posterior distribution
$p({x_{1,k}},{x_{2,k}},{x_{3,k}},{x_{4,k}}|{y_{1,k}})$ of state variables
${x_{1,k}}$, ${x_{2,k}}$, ${x_{3,k}}$, and ${x_{4,k}}$ for given observation
${y_{1,k}}$ at time $t_{k}$ is given by

$\begin{split}&p({x_{1,k}},{x_{2,k}},{x_{3,k}},{x_{4,k}}|{y_{1,k}})\cr&\propto
p({y_{1,k}}|{x_{3,k}},{x_{4,k}})p({x_{3,k}}|{x_{2,k}},{x_{1,k}})\cr&\times
p({x_{4,k}}|{x_{2,k}},{x_{1,k}})p^{\prime}({x_{1,k}},{x_{2,k}},{x_{3,k}},{x_{4,k}}),\cr\end{split}$
(14)

where $p({y_{1,k}}|{x_{3,k}},{x_{4,k}})$, $p({x_{3,k}}|{x_{2,k}},{x_{1,k}})$,
and $p({x_{4,k}}|{x_{2,k}},{x_{1,k}})$ are the conditional probability tables
or conditional probability distributions describing the probabilistic
causality relationship between the parent nodes (e.g., $x_{2,k}$ and
$x_{1,k}$) and a child node (e.g., $x_{3,k}$),
$p^{\prime}({x_{1,k}},{x_{2,k}},{x_{3,k}})$ is the prior distribution of state
variables ${x_{1,k}},{x_{2,k}}$, ${x_{3,k}}$, and ${x_{4,k}}$ at $t_{k}$,
which is obtained at each time step by recursively performing Bayesian
inference and uncertainty propagation based on observations and state
transition probabilities given in Eq. (6).

Theoretically, all the Bayesian inference methods discussed in Sec. 4.2.2 can
be employed to update the posterior of state variables based on the
formulation given in Eq. (14). In the context of digital twins, particle
filters such as the one presented in Appendix A are usually used in
conjunction with DBNs to update digital states for two main reasons: (1)
particle filters have fewer assumptions on the nonlinearity of the state
transition and measurement functions and noise distributions than Kalman
filters (see Table 4); and (2) using surrogate models to approximate
conditional probability distributions (see Eq. (5)) allows for an efficient
evaluation of likelihood functions, such that a large number of particles can
be used for the inference. Using Eq. (14) as an example, in a particle filter
implementation, we first generate equally weighted particles of state
variables ${x_{1,k}},{x_{2,k}}$, ${x_{3,k}}$, and ${x_{4,k}}$ as their prior
at $t_{k}$ according to the transition probabilities of state variables
defined by the transition BN (i.e., Step 5 in Fig. 28). After that, the
likelihood of each particle is computed using the decomposed likelihood
function
$p({y_{1,k}}|{x_{3,k}},{x_{4,k}})p({x_{3,k}}|{x_{2,k}},{x_{1,k}})\times
p({x_{4,k}}|{x_{2,k}},{x_{1,k}})$ given in Eq. (14). This corresponds to the
computation of particle weights in Step 6 of Fig. 28. The likelihood values of
the particles are then normalized and the particles are subsequently re-
sampled based on the normalized weights to obtain the posterior samples of the
state variables at $t_{k}$. This process is implemented repeatedly over time
to dynamically update the state variables.

## Acknowledgements

Adam Thelen and Chao Hu would like to thank the financial support from the
U.S. National Science Foundation under Grant No. ECCS-2015710. Xiaoge Zhang is
supported by a grant from the Research Committee of The Hong Kong Polytechnic
University under project code 1-BE6V and G-UAMR. Sankaran Mahadevan
acknowledges the support of the National Institute of Science and Technology.
Michael D. Todd and Zhen Hu received financial support from the U.S. Army
Corps of Engineers through the U.S. Army Engineer Research and Development
Center Research Cooperative Agreement W912HZ-17-2-0024.

## Competing interests

The authors have no relevant financial or non-financial conflicts of interest
to disclose.

## Replication of results

The Python code and preprocessed dataset used for the battery case study are
available for download on Github .

## Authors’ contributions

All authors read and approved the final manuscript. Hu, C. and Hu, Z. devised
the original concept of the review paper. Hu, Z, Thelen, A., and Zhang, X.
were responsible for the literature review. Thelen, A. was responsible for
geometric modeling. Hu, C., Thelen, A., and Hu, Z. were responsible for
physics-based modeling. Hu, Z. was responsible for data-driven modeling. Hu,
C. and Hu, Z. were responsible for physics-informed ML. Zhang, X. was
responsible for system modeling. Hu, C., and Hu, Z., were responsible for
probabilistic model updating. Zhang, X. was responsible for ML model updating.
Hu, C. and Hu., Z were responsible for fault diagnostics, failure prognostics,
and predictive maintenance. Lu, Y. was responsible for MPC. Fink, O. was
responsible for federated learning and domain adaptation. Zhang, X., Hu, Z.,
and Fink, O. were responsible for deep reinforcement learning. Hu, C. was
responsible for UQ of ML models. Hu, Z. was responsible for UQ of dynamic
system models, optimization for sensor placement, and optimization for
physical system modeling. Lu, Y. was responsible for the optimization of
additive manufacturing processes. Zhang, X. and Hu, Z. were responsible for
real-time mission planning. Thelen, A. and Hu, C. were responsible for the
case study and predictive maintenance scheduling. Hu, C. was responsible for
open-source software and data. Ghosh, S. was responsible for the industry
demonstration. Hu, C., Todd, M., and Mahadevan, S. were responsible for
perspectives. All authors participated in manuscript writing, review, editing,
and comment.

## References

  * Aitchison (2020) L. Aitchison.  Bayesian filtering unifies adaptive and non-adaptive neural network optimization methods.  _Advances in Neural Information Processing Systems_ , 33:18173–18182, 2020.
  * Akroyd et al. (2021) J. Akroyd, S. Mosbach, A. Bhave, and M. Kraft.  Universal digital twin-a dynamic knowledge graph.  _Data-Centric Engineering_ , 2, 2021.
  * Allu et al. (2014) S. Allu, S. Kalnaus, W. Elwasif, S. Simunovic, J. A. Turner, and S. Pannala.  A new open computational framework for highly-resolved coupled three-dimensional multiphysics simulations of li-ion cells.  _Journal of Power Sources_ , 246:876–886, 2014.
  * Almalki et al. (2022) A. Almalki, D. Downing, B. Lozanovski, R. Tino, A. Du Plessis, M. Qian, M. Brandt, and M. Leary.  A digital-twin methodology for the non-destructive certification of lattice structures.  _JOM_ , 74(4):1784–1797, 2022.
  * Angjeliu et al. (2020) G. Angjeliu, D. Coronelli, and G. Cardani.  Development of the simulation model for digital twin applications in historical masonry buildings: The integration between numerical and experimental reality.  _Computers & Structures_, 238:106282, 2020.
  * (6) I. ANSYS.  Ansys and ptc partner to create a digital twin of an electric motor.  https://www.ansys.com/resource-center/demo/ansys-ptc-demo.  Last Accessed: 2022-05-20.
  * Antoni and Randall (2006) J. Antoni and R. B. Randall.  The spectral kurtosis: application to the vibratory surveillance and diagnostics of rotating machines.  _Mechanical Systems and Signal Processing_ , 20(2):308–331, 2006.
  * Arendt et al. (2012) P. D. Arendt, D. W. Apley, and W. Chen.  Quantification of model uncertainty: Calibration, model discrepancy, and identifiability.  2012\.
  * Arulampalam et al. (2002) M. S. Arulampalam, S. Maskell, N. Gordon, and T. Clapp.  A tutorial on particle filters for online nonlinear/non-gaussian bayesian tracking.  _IEEE Transactions on Signal Processing_ , 50(2):174–188, 2002.
  * Åström and Eykhoff (1971) K. J. Åström and P. Eykhoff.  System identification—a survey.  _Automatica_ , 7(2):123–162, 1971.
  * (11) R. Automation.  Inca mpc: Model-predictive multi-variable controller.  https://literature.rockwellautomation.com/idc/groups/literature/documents/br/rsbrp8-br001_-en-p.pdf.  Last Accessed: 2022-05-29.
  * Ayani et al. (2018) M. Ayani, M. Ganebäck, and A. H. Ng.  Digital twin: Applying emulation for machine reconditioning.  _Procedia CIRP_ , 72:243–248, 2018.
  * Ayerbe et al. (2021) E. Ayerbe, M. Berecibar, S. Clark, A. A. Franco, and J. Ruhland.  Digitalization of battery manufacturing: Current status, challenges, and opportunities.  _Advanced Energy Materials_ , page 2102696, 2021.
  * Aykol et al. (2021) M. Aykol, C. B. Gopal, A. Anapolsky, P. K. Herring, B. van Vlijmen, M. D. Berliner, M. Z. Bazant, R. D. Braatz, W. C. Chueh, and B. D. Storey.  Perspective—combining physics and machine learning to predict battery lifetime.  _Journal of The Electrochemical Society_ , 168(3):030525, 2021.
  * Bachelor et al. (2019) G. Bachelor, E. Brusa, D. Ferretto, and A. Mitschke.  Model-based design of complex aeronautical systems through digital twin and thread concepts.  _IEEE Systems Journal_ , 14(2):1568–1579, 2019\.
  * Banerjee et al. (2017) A. Banerjee, R. Dalal, S. Mittal, and K. P. Joshi.  Generating digital twin models using knowledge graphs for industrial production lines.  _UMBC Information Systems Department_ , 2017.
  * Bangalore and Patriksson (2018) P. Bangalore and M. Patriksson.  Analysis of scada data for early fault detection, with application to the maintenance management of wind turbines.  _Renewable Energy_ , 115:521–532, 2018.
  * Barricelli et al. (2019) B. R. Barricelli, E. Casiraghi, and D. Fogli.  A survey on digital twin: Definitions, characteristics, applications, and design implications.  _IEEE Access_ , 7:167653–167671, 2019.
  * Barry et al. (2022) N. Barry, M. Chatzos, W. Chen, D. Han, C. Huang, R. Joseph, M. Klamkin, S. Park, M. Tanneau, P. Van Hentenryck, et al.  Risk-aware control and optimization for high-renewable power grids.  _arXiv preprint arXiv:2204.00950_ , 2022.
  * Bellalouna (2021) F. Bellalouna.  Case study for design optimization using the digital twin approach.  _Procedia CIRP_ , 100:595–600, 2021.
  * Bengio et al. (2021) Y. Bengio, A. Lodi, and A. Prouvost.  Machine learning for combinatorial optimization: a methodological tour d’horizon.  _European Journal of Operational Research_ , 290(2):405–421, 2021.
  * Berecibar et al. (2016) M. Berecibar, I. Gandiaga, I. Villarreal, N. Omar, J. Van Mierlo, and P. Van den Bossche.  Critical review of state of health estimation methods of li-ion batteries for real applications.  _Renewable and Sustainable Energy Reviews_ , 56:572–587, 2016.
  * Bichon et al. (2008) B. J. Bichon, M. S. Eldred, L. P. Swiler, S. Mahadevan, and J. M. McFarland.  Efficient global reliability analysis for nonlinear implicit performance functions.  _AIAA journal_ , 46(10):2459–2468, 2008.
  * Billings (2013) S. A. Billings.  _Nonlinear system identification: NARMAX methods in the time, frequency, and spatio-temporal domains_.  John Wiley & Sons, 2013.
  * (25) J. Bilmes.  Graphical models toolkit.  https://web.archive.org/web/20150228005644/http://melodi.ee.washington.edu/gmtk/.  Last Accessed: 2022-04-24.
  * Birkl et al. (2017) C. R. Birkl, M. R. Roberts, E. McTurk, P. G. Bruce, and D. A. Howey.  Degradation diagnostics for lithium ion cells.  _Journal of Power Sources_ , 341:373–386, 2017.
  * Bloom et al. (2001) I. Bloom, B. Cole, J. Sohn, S. A. Jones, E. G. Polzin, V. S. Battaglia, G. L. Henriksen, C. Motloch, R. Richardson, T. Unkelhaeuser, et al.  An accelerated calendar and cycle life study of li-ion cells.  _Journal of power sources_ , 101(2):238–247, 2001\.
  * Boje et al. (2020) C. Boje, A. Guerriero, S. Kubicki, and Y. Rezgui.  Towards a semantic construction digital twin: Directions for future research.  _Automation in Construction_ , 114:103179, 2020.
  * Boris et al. (2021) C. Boris, A. Sadek, and C. Wolf.  Universal domain adaptation in ordinal regression.  _arXiv preprint arXiv:2106.11576_ , 2021.
  * Bousmalis et al. (2018) K. Bousmalis, A. Irpan, P. Wohlhart, Y. Bai, M. Kelcey, M. Kalakrishnan, L. Downs, J. Ibarz, P. Pastor, K. Konolige, et al.  Using simulation and domain adaptation to improve efficiency of deep robotic grasping.  In _2018 IEEE international conference on robotics and automation (ICRA)_ , pages 4243–4250. IEEE, 2018.
  * Box (1976) G. E. Box.  Science and statistics.  _Journal of the American Statistical Association_ , 71(356):791–799, 1976.
  * Brincker and Andersen (2006) R. Brincker and P. Andersen.  Understanding stochastic subspace identification.  In _Conference Proceedings: IMAC-XXIV: A Conference & Exposition on Structural Dynamics_. Society for Experimental Mechanics, 2006.
  * Brincker et al. (2001) R. Brincker, L. Zhang, and P. Andersen.  Modal identification of output-only systems using frequency domain decomposition.  _Smart Materials and Structures_ , 10(3):441, 2001\.
  * Broo et al. (2022) D. G. Broo, M. Bravo-Haro, and J. Schooling.  Design and implementation of a smart infrastructure digital twin.  _Automation in Construction_ , 136:104171, 2022.
  * Brooks (1998) S. Brooks.  Markov chain monte carlo method and its application.  _Journal of the Royal Statistical Society: Series D (the Statistician)_ , 47(1):69–100, 1998.
  * Bzdok et al. (2017) D. Bzdok, M. Krzywinski, and N. Altman.  Machine learning: a primer.  _Nature methods_ , 14(12):1119, 2017.
  * Cai et al. (2020) Y. Cai, Y. Wang, and M. Burnett.  Using augmented reality to build digital twin for reconfigurable additive manufacturing system.  _Journal of Manufacturing Systems_ , 56:598–604, 2020.
  * Cappé et al. (2007) O. Cappé, S. J. Godsill, and E. Moulines.  An overview of existing methods and recent advances in sequential monte carlo.  _Proceedings of the IEEE_ , 95(5):899–924, 2007\.
  * Carlucci et al. (2017) F. M. Carlucci, L. Porzi, B. Caputo, E. Ricci, and S. R. Bulò.  Autodial: Automatic domain alignment layers.  In _ICCV_ , pages 5077–5085, 2017.
  * Castells et al. (2006) P. Castells, M. Fernandez, and D. Vallet.  An adaptation of the vector-space model for ontology-based information retrieval.  _IEEE Transactions on Knowledge and Data Engineering_ , 19(2):261–272, 2006.
  * Chai and Zhao (2019) Z. Chai and C. Zhao.  Enhanced random forest with concurrent analysis of static and dynamic nodes for industrial fault classification.  _IEEE Transactions on Industrial Informatics_ , 16(1):54–66, 2019.
  * Chakraborty and Adhikari (2021) S. Chakraborty and S. Adhikari.  Machine learning based digital twin for dynamical systems with multiple time-scales.  _Computers & Structures_, 243:106410, 2021.
  * Chakraborty et al. (2021) S. Chakraborty, S. Adhikari, and R. Ganguli.  The role of surrogate models in the development of digital twins of dynamic systems.  _Applied Mathematical Modelling_ , 90:662–681, 2021.
  * Chao et al. (2019) M. A. Chao, C. Kulkarni, K. Goebel, and O. Fink.  Hybrid deep fault detection and isolation: Combining deep neural networks and system performance models.  _International Journal of Prognostics and Health Management_ , 10(4), 2019.
  * Chao et al. (2022) M. A. Chao, C. Kulkarni, K. Goebel, and O. Fink.  Fusing physics-based and deep learning models for prognostics.  _Reliability Engineering & System Safety_, 217:107961, 2022.
  * Chen and Billings (1992) S. Chen and S. A. Billings.  Neural networks for nonlinear dynamic system modelling and identification.  _International journal of control_ , 56(2):319–346, 1992.
  * Chen et al. (2022) W. Chen, S. Park, M. Tanneau, and P. Van Hentenryck.  Learning optimization proxies for large-scale security-constrained economic dispatch.  _Electric Power Systems Research_ , 213:108566, 2022.
  * Chen et al. (2020a) X. Chen, S. Jia, and Y. Xiang.  A review: Knowledge reasoning over knowledge graph.  _Expert Systems with Applications_ , 141:112948, 2020a.
  * Chen et al. (2020b) Y. Chen, F. Luo, T. Li, T. Xiang, Z. Liu, and J. Li.  A training-integrity privacy-preserving federated learning scheme with trusted execution environment.  _Information Sciences_ , 522:69–79, 2020b.
  * Chen (2010) Y.-J. Chen.  Development of a method for ontology-based empirical knowledge representation and reasoning.  _Decision Support Systems_ , 50(1):1–20, 2010\.
  * Cheng et al. (2018) J. Cheng, W. Chen, F. Tao, and C.-L. Lin.  Industrial iot in 5g environment towards smart manufacturing.  _Journal of Industrial Information Integration_ , 10:10–19, 2018.
  * Chiachío et al. (2020) J. Chiachío, M. L. Jalon, M. Chiachío, and A. Kolios.  A markov chains prognostics framework for complex degradation processes.  _Reliability Engineering & System Safety_, 195:106621, 2020.
  * Choi et al. (2022) S. H. Choi, K.-B. Park, D. H. Roh, J. Y. Lee, M. Mohammed, Y. Ghasemi, and H. Jeong.  An integrated mixed reality system for safety-aware human-robot collaboration using deep learning and digital twin generation.  _Robotics and Computer-Integrated Manufacturing_ , 73:102258, 2022.
  * Chorin et al. (1990) A. J. Chorin, J. E. Marsden, and J. E. Marsden.  _A mathematical introduction to fluid mechanics_ , volume 3.  Springer, 1990.
  * Cimino et al. (2019) C. Cimino, E. Negri, and L. Fumagalli.  Review of digital twin applications in manufacturing.  _Computers in Industry_ , 113:103130, 2019.
  * Cohn et al. (1996) D. A. Cohn, Z. Ghahramani, and M. I. Jordan.  Active learning with statistical models.  _Journal of artificial intelligence research_ , 4:129–145, 1996.
  * Compare et al. (2016) M. Compare, F. Martini, S. Mattafirri, F. Carlevaro, and E. Zio.  Semi-markov model for the oxidation degradation mechanism in gas turbine nozzles.  _IEEE Transactions on Reliability_ , 65(2):574–581, 2016.
  * Compton et al. (2012) M. Compton, P. Barnaghi, L. Bermudez, R. Garcia-Castro, O. Corcho, S. Cox, J. Graybeal, M. Hauswirth, C. Henson, A. Herzog, et al.  The ssn ontology of the w3c semantic sensor network incubator group.  _Journal of Web Semantics_ , 17:25–32, 2012.
  * Corral-Acero et al. (2020) J. Corral-Acero, F. Margara, M. Marciniak, C. Rodero, F. Loncaric, Y. Feng, A. Gilbert, J. F. Fernandes, H. A. Bukhari, A. Wajdan, et al.  The ‘digital twin’to enable the vision of precision cardiology.  _European Heart Journal_ , 41(48):4556–4564, 2020\.
  * Cox (1964) H. Cox.  On the estimation of state variables and parameters for noisy dynamic systems.  _IEEE Transactions on Automatic Control_ , 9(1):5–12, 1964.
  * Culler and McNamara (2010) A. J. Culler and J. J. McNamara.  Studies on fluid-thermal-structural coupling for aerothermoelasticity in hypersonic flow.  _AIAA journal_ , 48(8):1721–1738, 2010.
  * da Costa et al. (2020) P. R. d. O. da Costa, A. Akçay, Y. Zhang, and U. Kaymak.  Remaining useful lifetime prediction via deep domain adaptation.  _Reliability Engineering & System Safety_, 195:106682, 2020.
  * Dai et al. (2020) Y. Dai, K. Zhang, S. Maharjan, and Y. Zhang.  Deep reinforcement learning for stochastic computation offloading in digital twin networks.  _IEEE Transactions on Industrial Informatics_ , 17(7):4968–4977, 2020.
  * Danilo et al. (2018) B. Danilo, A. Naomi, and K. Martin.  Statistics versus machine learning.  _Nature Methods_ , 15(4):233, 2018.
  * Davis et al. (2017) S. E. Davis, T. A. Lasko, G. Chen, E. D. Siew, and M. E. Matheny.  Calibration drift in regression and machine learning models for acute kidney injury.  _Journal of the American Medical Informatics Association_ , 24(6):1052–1061, 2017.
  * DeLaurentis and Mavris (2000) D. A. DeLaurentis and D. Mavris.  Uncertainty modeling and management in multidisciplinary analysis and synthesis.  In _38th Aerospace Sciences Meeting and Exhibit_ , page 422, 2000\.
  * Deng et al. (2014) K. Deng, Y. Sun, S. Li, Y. Lu, J. Brouwer, P. G. Mehta, M. Zhou, and A. Chakraborty.  Model predictive control of central chiller plant with thermal energy storage via dynamic programming and mixed-integer linear programming.  _IEEE Transactions on Automation Science and Engineering_ , 12:565–579, 2014.
  * Deng et al. (2021) T. Deng, K. Zhang, and Z.-J. M. Shen.  A systematic review of a digital twin city: A new pattern of urban governance toward smart cities.  _Journal of Management Science and Engineering_ , 6(2):125–134, 2021.
  * Ding et al. (2019) K. Ding, F. T. Chan, X. Zhang, G. Zhou, and F. Zhang.  Defining a digital twin-based cyber-physical production system for autonomous manufacturing in smart shop floors.  _International Journal of Production Research_ , 57(20):6315–6334, 2019.
  * Doerr et al. (2018) A. Doerr, C. Daniel, M. Schiegg, N.-T. Duy, S. Schaal, M. Toussaint, and T. Sebastian.  Probabilistic recurrent state-space models.  In _International Conference on Machine Learning_ , pages 1280–1289. PMLR, 2018.
  * Döhler et al. (2011) M. Döhler, E. Reynders, F. Magalhaes, L. Mevel, G. D. Roeck, and A. Cunha.  Pre-and post-identification merging for multi-setup oma with covariance-driven ssi.  In _Dynamics of Bridges, Volume 5_ , pages 57–70. Springer, 2011\.
  * Downey et al. (2019) A. Downey, Y.-H. Lui, C. Hu, S. Laflamme, and S. Hu.  Physics-based prognostics of lithium-ion battery using non-linear least squares with dynamic bounds.  _Reliability Engineering & System Safety_, 182:1–12, 2019\.
  * Dröder et al. (2018) K. Dröder, P. Bobka, T. Germann, F. Gabriel, and F. Dietrich.  A machine learning-enhanced digital twin approach for human-robot-collaboration.  _Procedia Cirp_ , 76:187–192, 2018.
  * Elayan et al. (2021) H. Elayan, M. Aloqaily, and M. Guizani.  Digital twin for intelligent context-aware iot healthcare systems.  _IEEE Internet of Things Journal_ , 8(23):16749–16757, 2021.
  * Ellis (2015) M. J. Ellis.  Economic model predictive control theory: Computational efficiency and application to smart manufacturing.  2015\.
  * Erkoyuncu et al. (2020) J. A. Erkoyuncu, I. F. del Amo, D. Ariansyah, D. Bulka, R. Roy, et al.  A design framework for adaptive digital twins.  _CIRP annals_ , 69(1):145–148, 2020.
  * Errandonea et al. (2020) I. Errandonea, S. Beltrán, and S. Arrizabalaga.  Digital twin for maintenance: A literature review.  _Computers in Industry_ , 123:103316, 2020.
  * Essien and Giannetti (2021) A. Essien and C. Giannetti.  A deep learning model for smart manufacturing using convolutional lstm neural network autoencoders.  _IEEE Transactions on Industrial Informatics_ , 16:6069, 2021.
  * Fan et al. (2013) L. Fan, J. Khodadadi, and A. Pesaran.  A parametric study on thermal management of an air-cooled lithium-ion battery module for plug-in hybrid electric vehicles.  _Journal of Power Sources_ , 238:301–312, 2013.
  * Fan and Qiao (2011) W. Fan and P. Qiao.  Vibration-based damage identification methods: a review and comparative study.  _Structural health monitoring_ , 10(1):83–111, 2011.
  * Fang et al. (2019) Y. Fang, C. Peng, P. Lou, Z. Zhou, J. Hu, and J. Yan.  Digital-twin-based job shop scheduling toward smart manufacturing.  _IEEE Transactions on Industrial Informatics_ , 15(12):6425–6435, 2019.
  * Farrar and Worden (2007) C. R. Farrar and K. Worden.  An introduction to structural health monitoring.  _Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences_ , 365(1851):303–315, 2007.
  * Farzan et al. (2014) F. Farzan, M. A. Jafari, R. Masiello, and Y. Lu.  Toward optimal day-ahead scheduling and operation control of microgrids under uncertainty.  _IEEE Transactions on Smart Grid_ , 6:499–507, 2014.
  * Fernández-Godino et al. (2016) M. G. Fernández-Godino, C. Park, N.-H. Kim, and R. T. Haftka.  Review of multi-fidelity models.  _arXiv preprint arXiv:1609.07196_ , 2016.
  * Fink et al. (2014) O. Fink, E. Zio, and U. Weidmann.  Predicting component reliability and level of degradation with complex-valued neural networks.  _Reliability Engineering & System Safety_, 121:198–206, 2014.
  * Fink et al. (2020) O. Fink, Q. Wang, M. Svensen, P. Dersin, W.-J. Lee, and M. Ducoffe.  Potential, challenges and future directions for deep learning in prognostics and health management applications.  _Engineering Applications of Artificial Intelligence_ , 92:103678, 2020.
  * Franciosa et al. (2020) P. Franciosa, M. Sokolov, S. Sinha, T. Sun, and D. Ceglarek.  Deep learning enhanced digital twin for closed-loop in-process quality improvement.  _CIRP Annals_ , 69(1):369–372, 2020.
  * Friedman et al. (1997) N. Friedman, D. Geiger, and M. Goldszmidt.  Bayesian network classifiers.  _Machine learning_ , 29(2):131–163, 1997.
  * Fuller et al. (2020) A. Fuller, Z. Fan, C. Day, and C. Barlow.  Digital twin: Enabling technologies, challenges and open research.  _IEEE Access_ , 8:108952–108971, 2020.
  * Ganin and Lempitsky (2015) Y. Ganin and V. Lempitsky.  Unsupervised domain adaptation by backpropagation.  In _Proceedings of the 32nd International Conference on International Conference on Machine Learning-Volume 37_ , pages 1180–1189. JMLR. org, 2015.
  * Ganin et al. (2016) Y. Ganin, E. Ustinova, H. Ajakan, P. Germain, H. Larochelle, F. Laviolette, M. Marchand, and V. Lempitsky.  Domain-adversarial training of neural networks.  _The Journal of Machine Learning Research_ , 17(1):2096–2030, 2016.
  * Gao et al. (2021) H. Gao, L. Sun, and J.-X. Wang.  Phygeonet: Physics-informed geometry-adaptive convolutional neural networks for solving parameterized steady-state pdes on irregular domain.  _Journal of Computational Physics_ , 428:110079, 2021.
  * Gasper et al. (2021) P. Gasper, K. Gering, E. Dufek, and K. Smith.  Challenging practices of algebraic battery life models through statistical validation and model identification via machine-learning.  _Journal of the Electrochemical Society_ , 168(2):020502, 2021.
  * Gebraeel et al. (2005) N. Z. Gebraeel, M. A. Lawley, R. Li, and J. K. Ryan.  Residual-life distributions from component degradation signals: A bayesian approach.  _IIE Transactions_ , 37(6):543–557, 2005.
  * Giorgio et al. (2011) M. Giorgio, M. Guida, and G. Pulcini.  An age-and state-dependent markov model for degradation processes.  _IIE Transactions_ , 43(9):621–632, 2011.
  * Giselle Fernández-Godino et al. (2019) M. Giselle Fernández-Godino, C. Park, N. H. Kim, and R. T. Haftka.  Issues in deciding whether to use multifidelity surrogates.  _AIAA Journal_ , 57(5):2039–2054, 2019.
  * (97) Github.  Link to github repository where the preprocessed data and python scripts used to generate all the results and figures in the case study section reside.  https://github.com/acthelen/battery_digital_twin.
  * Glaessgen and Stargel (2012) E. Glaessgen and D. Stargel.  The digital twin paradigm for future nasa and us air force vehicles.  In _53rd AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics and Materials Conference 20th AIAA/ASME/AHS Adaptive Structures Conference 14th AIAA_ , page 1818, 2012.
  * Glatt et al. (2021) M. Glatt, C. Sinnwell, L. Yi, S. Donohoe, B. Ravani, and J. C. Aurich.  Modeling and implementation of a digital twin of material flows based on physics simulation.  _Journal of Manufacturing Systems_ , 58:231–245, 2021.
  * Goodwin et al. (2022) T. Goodwin, J. Xu, N. Celik, and C.-H. Chen.  Real-time digital twin-based optimization with predictive simulation learning.  _Journal of Simulation_ , pages 1–18, 2022.
  * Gorjian et al. (2010) N. Gorjian, L. Ma, M. Mittinty, P. Yarlagadda, and Y. Sun.  A review on degradation models in reliability analysis.  In _Engineering asset lifecycle management_ , pages 369–384. Springer, 2010.
  * Grieves (2014) M. Grieves.  Digital twin: manufacturing excellence through virtual factory replication.  _White paper_ , 1:1–7, 2014.
  * Guillaume et al. (2003) P. Guillaume, P. Verboven, S. Vanlanduit, H. Van Der Auweraer, and B. Peeters.  A poly-reference implementation of the least-squares complex frequency-domain estimator.  In _Proceedings of IMAC_ , volume 21, pages 183–192. A Conference & Exposition on Structural Dynamics, Society for Experimental …, 2003.
  * Guivarch et al. (2019) D. Guivarch, E. Mermoz, Y. Marino, and M. Sartor.  Creation of helicopter dynamic systems digital twin using multibody simulations.  _CIRP Annals_ , 68(1):133–136, 2019.
  * Guo et al. (2021) H. Guo, M. Chen, K. Mohamed, T. Qu, S. Wang, and J. Li.  A digital twin-based flexible cellular manufacturing for optimization of air conditioner line.  _Journal of Manufacturing Systems_ , 58:65–78, 2021.
  * Guo et al. (2017) L. Guo, N. Li, F. Jia, Y. Lei, and J. Lin.  A recurrent neural network based health indicator for remaining useful life prediction of bearings.  _Neurocomputing_ , 240:98–109, 2017.
  * Guo et al. (2022) S. Guo, W. Guo, L. Bian, and Y. Guo.  A deep-learning-based surrogate model for thermal signature prediction in laser metal deposition.  _IEEE Transactions on Automation Science and Engineering_ , 2022.
  * Gustafsson (2010) F. Gustafsson.  Particle filter theory and practice with positioning applications.  _IEEE Aerospace and Electronic Systems Magazine_ , 25(7):53–82, 2010.
  * Ha et al. (2018) S. Ha, S. Coros, A. Alspach, J. Kim, and K. Yamane.  Computational co-optimization of design parameters and motion trajectories for robotic systems.  _The International Journal of Robotics Research_ , 37(13-14):1521–1536, 2018.
  * Haag and Anderl (2018) S. Haag and R. Anderl.  Digital twin–proof of concept.  _Manufacturing letters_ , 15:64–66, 2018.
  * Haghighat et al. (2021) E. Haghighat, M. Raissi, A. Moure, H. Gomez, and R. Juanes.  A physics-informed deep learning framework for inversion and surrogate modeling in solid mechanics.  _Computer Methods in Applied Mechanics and Engineering_ , 379:113741, 2021.
  * He and Bai (2021) B. He and K.-J. Bai.  Digital twin-based sustainable intelligent manufacturing: A review.  _Advances in Manufacturing_ , 9(1):1–21, 2021\.
  * He et al. (2019) R. He, G. Chen, C. Dong, S. Sun, and X. Shen.  Data-driven digital twin technology for optimized control in process systems.  _ISA Transactions_ , 95:221–234, 2019.
  * He et al. (2011) W. He, N. Williard, M. Osterman, and M. Pecht.  Prognostics of lithium-ion batteries based on dempster–shafer theory and the bayesian monte carlo method.  _Journal of Power Sources_ , 196(23):10314–10321, 2011.
  * Heckerman (2008) D. Heckerman.  A tutorial on learning with bayesian networks.  _Innovations in Bayesian networks_ , pages 33–82, 2008.
  * Heng et al. (2009) A. Heng, S. Zhang, A. C. Tan, and J. Mathew.  Rotating machinery prognostics: State of the art, challenges and opportunities.  _Mechanical Systems and Signal Processing_ , 23(3):724–739, 2009.
  * Hochreiter and Schmidhuber (1997) S. Hochreiter and J. Schmidhuber.  Long short-term memory.  _Neural Computation_ , 9(8):1735–1780, 1997.
  * Hsu and Bazilevs (2012) M.-C. Hsu and Y. Bazilevs.  Fluid–structure interaction modeling of wind turbines: simulating the full machine.  _Computational Mechanics_ , 50(6):821–833, 2012\.
  * Hu and Youn (2011) C. Hu and B. D. Youn.  Adaptive-sparse polynomial chaos expansion for reliability analysis and design of complex engineering systems.  _Structural and Multidisciplinary Optimization_ , 43(3):419–442, 2011.
  * Hu et al. (2012) C. Hu, B. D. Youn, and J. Chung.  A multiscale framework with extended Kalman filter for lithium-ion battery SOC and capacity estimation.  _Applied Energy_ , 92:694–704, 2012.
  * Hu et al. (2018) C. Hu, H. Ye, G. Jain, and C. Schmidt.  Remaining useful life assessment of lithium-ion batteries in implantable medical devices.  _Journal of Power Sources_ , 375:118–130, 2018.
  * Hu et al. (2021a) W. Hu, Y. He, Z. Liu, J. Tan, M. Yang, and J. Chen.  Toward a digital twin: Time series prediction based on a hybrid ensemble empirical mode decomposition and bo-lstm neural networks.  _Journal of Mechanical Design_ , 143(5), 2021a.
  * Hu et al. (2021b) W. Hu, T. Zhang, X. Deng, Z. Liu, and J. Tan.  Digital twin: A state-of-the-art review of its enabling technologies, applications and challenges.  _Journal of Intelligent Manufacturing and Special Equipment_ , 2021b.
  * Hu et al. (2022a) W. Hu, J. Shao, Q. Jiao, C. Wang, J. Cheng, Z. Liu, and J. Tan.  A new differentiable architecture search method for optimizing convolutional neural networks in the digital twin of intelligent robotic grasping.  _Journal of Intelligent Manufacturing_ , pages 1–19, 2022a.
  * Hu et al. (2022b) W. Hu, C. Wang, F. Liu, X. Peng, P. Sun, and J. Tan.  A grasps-generation-and-selection convolutional neural network for a digital twin of intelligent robotic grasping.  _Robotics and Computer-Integrated Manufacturing_ , 77:102371, 2022b.
  * Hu et al. (2020) X. Hu, L. Xu, X. Lin, and M. Pecht.  Battery lifetime prognostics.  _Joule_ , 4(2):310–346, 2020.
  * Hu and Mahadevan (2018a) Z. Hu and S. Mahadevan.  Adaptive surrogate modeling for time-dependent multidisciplinary reliability analysis.  _Journal of Mechanical Design_ , 140(2), 2018a.
  * Hu and Mahadevan (2018b) Z. Hu and S. Mahadevan.  Bayesian network learning for data-driven design.  _ASCE-ASME J Risk and Uncert in Engrg Sys Part B Mech Engrg_ , 4(4), 2018b.
  * Hu and Mourelatos (2018) Z. Hu and Z. P. Mourelatos.  A sequential accelerated life testing framework for system reliability assessment with untestable components.  _Journal of Mechanical Design_ , 140(10), 2018.
  * Huang et al. (2019) C.-G. Huang, H.-Z. Huang, and Y.-F. Li.  A bidirectional lstm prognostics method under multiple operational conditions.  _IEEE Transactions on Industrial Electronics_ , 66(11):8792–8802, 2019.
  * Huang et al. (2007) R. Huang, L. Xi, X. Li, C. R. Liu, H. Qiu, and J. Lee.  Residual life predictions for ball bearings based on self-organizing map and back propagation neural network methods.  _Mechanical Systems and Signal Processing_ , 21(1):193–207, 2007.
  * Huang et al. (2022a) S. Huang, G. Wang, and Y. Yan.  Building blocks for digital twin of reconfigurable machine tools from design perspective.  _International Journal of Production Research_ , 60(3):942–956, 2022a.
  * Huang et al. (2022b) X. Huang, T. Xie, Z. Wang, L. Chen, Q. Zhou, and Z. Hu.  A transfer learning-based multi-fidelity point-cloud neural network approach for melt pool modeling in additive manufacturing.  _ASCE-ASME J Risk and Uncert in Engrg Sys Part B Mech Engrg_ , 8(1), 2022b.
  * Ibrahim et al. (2020) M. S. Ibrahim, J. Fan, W. K. Yung, A. Prisacaru, W. van Driel, X. Fan, and G. Zhang.  Machine learning and digital twin driven diagnostics and prognostics of light-emitting diodes.  _Laser & Photonics Reviews_, 14(12):2000254, 2020.
  * Im et al. (2021) S. Im, J. Lee, and M. Cho.  Surrogate modeling of elasto-plastic problems via long short-term memory neural networks and proper orthogonal decomposition.  _Computer Methods in Applied Mechanics and Engineering_ , 385:114030, 2021.
  * (136) S. Industry.  Model predictive control - rockwell model predictive control delivers results.  https://mall.industry.siemens.com/mall/en/WW/Catalog/Products/10017369.  Last Accessed: 2022-05-29.
  * Jagtap et al. (2020a) A. D. Jagtap, K. Kawaguchi, and G. E. Karniadakis.  Adaptive activation functions accelerate convergence in deep and physics-informed neural networks.  _Journal of Computational Physics_ , 404:109136, 2020a.
  * Jagtap et al. (2020b) A. D. Jagtap, E. Kharazmi, and G. E. Karniadakis.  Conservative physics-informed neural networks on discrete domains for conservation laws: Applications to forward and inverse problems.  _Computer Methods in Applied Mechanics and Engineering_ , 365:113028, 2020b.
  * Jain et al. (2019) P. Jain, J. Poon, J. P. Singh, C. Spanos, S. R. Sanders, and S. K. Panda.  A digital twin approach for fault diagnosis in distributed photovoltaic systems.  _IEEE Transactions on Power Electronics_ , 35(1):940–956, 2019.
  * Jardine et al. (2006) A. K. Jardine, D. Lin, and D. Banjevic.  A review on machinery diagnostics and prognostics implementing condition-based maintenance.  _Mechanical Systems and Signal Processing_ , 20(7):1483–1510, 2006.
  * Jegadeeshwaran and Sugumaran (2015) R. Jegadeeshwaran and V. Sugumaran.  Fault diagnosis of automobile hydraulic brake system using statistical features and support vector machines.  _Mechanical Systems and Signal Processing_ , 52:436–446, 2015.
  * Ji et al. (2021) S. Ji, S. Pan, E. Cambria, P. Marttinen, and S. Y. Philip.  A survey on knowledge graphs: Representation, acquisition, and applications.  _IEEE Transactions on Neural Networks and Learning Systems_ , 2021\.
  * Jiang et al. (2022) C. Jiang, M. A. Vega, M. D. Todd, and Z. Hu.  Model correction and updating of a stochastic degradation model for failure prognostics of miter gates.  _Reliability Engineering & System Safety_, 218:108203, 2022.
  * Jiang et al. (2021) F. Jiang, L. Ma, T. Broyd, and K. Chen.  Digital twin and its implementations in the civil engineering sector.  _Automation in Construction_ , 130:103838, 2021.
  * Jones et al. (2020) D. Jones, C. Snider, A. Nassehi, J. Yon, and B. Hicks.  Characterising the digital twin: A systematic literature review.  _CIRP Journal of Manufacturing Science and Technology_ , 29:36–52, 2020.
  * Juang and Pappa (1985) J.-N. Juang and R. S. Pappa.  An eigensystem realization algorithm for modal parameter identification and model reduction.  _Journal of Guidance, Control, and Dynamics_ , 8(5):620–627, 1985.
  * Kapteyn et al. (2020) M. G. Kapteyn, D. J. Knezevic, D. Huynh, M. Tran, and K. E. Willcox.  Data-driven physics-based digital twins via a library of component-based reduced-order models.  _International Journal for Numerical Methods in Engineering_ , 2020\.
  * Kapteyn et al. (2021) M. G. Kapteyn, J. V. Pretorius, and K. E. Willcox.  A probabilistic graphical model foundation for enabling predictive digital twins at scale.  _Nature Computational Science_ , 1(5):337–347, 2021.
  * Kapusuzoglu and Mahadevan (2020) B. Kapusuzoglu and S. Mahadevan.  Physics-informed and hybrid machine learning in additive manufacturing: application to fused filament fabrication.  _Jom_ , 72(12):4695–4705, 2020.
  * Karniadakis et al. (2021) G. E. Karniadakis, I. G. Kevrekidis, L. Lu, P. Perdikaris, S. Wang, and L. Yang.  Physics-informed machine learning.  _Nature Reviews Physics_ , 3(6):422–440, 2021\.
  * Karve et al. (2020) P. M. Karve, Y. Guo, B. Kapusuzoglu, S. Mahadevan, and M. A. Haile.  Digital twin approach for damage-tolerant mission planning under uncertainty.  _Engineering Fracture Mechanics_ , 225:106766, 2020.
  * Kennedy and O’Hagan (2001) M. C. Kennedy and A. O’Hagan.  Bayesian calibration of computer models.  _Journal of the Royal Statistical Society: Series B (Statistical Methodology)_ , 63(3):425–464, 2001.
  * Kerschen et al. (2006) G. Kerschen, K. Worden, A. F. Vakakis, and J.-C. Golinval.  Past, present and future of nonlinear system identification in structural dynamics.  _Mechanical Systems and Signal Processing_ , 20(3):505–592, 2006.
  * Khan and Yairi (2018) S. Khan and T. Yairi.  A review on the application of deep learning in system health management.  _Mechanical Systems and Signal Processing_ , 107:241–265, 2018.
  * Khan et al. (2020) S. Khan, M. Farnsworth, R. McWilliam, and J. Erkoyuncu.  On the requirements of digital twin-driven autonomous maintenance.  _Annual Reviews in Control_ , 50:13–28, 2020.
  * Kharoufeh et al. (2010) J. P. Kharoufeh, C. J. Solo, and M. Y. Ulukus.  Semi-markov models for degradation-based reliability.  _IIE Transactions_ , 42(8):599–612, 2010.
  * Kim et al. (2022) Y. Kim, K. Na, and B. D. Youn.  A health-adaptive time-scale representation (htsr) embedded convolutional neural network for gearbox fault diagnostics.  _Mechanical Systems and Signal Processing_ , 167:108575, 2022.
  * Kohtz et al. (2022) S. Kohtz, Y. Xu, Z. Zheng, and P. Wang.  Physics-informed machine learning model for battery state of health prognostics using partial charging segments.  _Mechanical Systems and Signal Processing_ , 172:109002, 2022.
  * Kritzinger et al. (2018) W. Kritzinger, M. Karner, G. Traar, J. Henjes, and W. Sihn.  Digital twin in manufacturing: A categorical literature review and classification.  _IFAC-PapersOnLine_ , 51(11):1016–1022, 2018\.
  * Ladj et al. (2021) A. Ladj, Z. Wang, O. Meski, F. Belkadi, M. Ritou, and C. Da Cunha.  A knowledge-based digital shadow for machining industry in a digital twin perspective.  _Journal of Manufacturing Systems_ , 58:168–179, 2021.
  * Lao and Cohen (2010) N. Lao and W. W. Cohen.  Relational retrieval using a combination of path-constrained random walks.  _Machine learning_ , 81(1):53–67, 2010.
  * LeCun et al. (2015) Y. LeCun, Y. Bengio, and G. Hinton.  Deep learning.  _Nature_ , 521(7553):436–444, 2015.
  * Lee et al. (2019) C.-Y. Lee, T. Batra, M. H. Baig, and D. Ulbricht.  Sliced wasserstein discrepancy for unsupervised domain adaptation.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 10285–10295, 2019.
  * Lee et al. (2013a) J. Lee, E. Lapira, B. Bagheri, and H.-a. Kao.  Recent advances and trends in predictive manufacturing systems in big data environment.  _Manufacturing letters_ , 1(1):38–41, 2013a.
  * Lee et al. (2013b) J. Lee, E. Lapira, S. Yang, and A. Kao.  Predictive manufacturing system-trends of next-generation production systems.  _IFAC proceedings volumes_ , 46(7):150–156, 2013b.
  * Lee et al. (2014) J. Lee, F. Wu, W. Zhao, M. Ghaffari, L. Liao, and D. Siegel.  Prognostics and health management design for rotary machinery systems—reviews, methodology and applications.  _Mechanical Systems and Signal Processing_ , 42(1-2):314–334, 2014.
  * Lee et al. (2018) K. Lee, K. Lee, H. Lee, and J. Shin.  A simple unified framework for detecting out-of-distribution samples and adversarial attacks.  _Advances in Neural Information Processing Systems_ , 31, 2018.
  * Lee et al. (2010) S. Lee, L. Li, and J. Ni.  Online degradation assessment and adaptive fault detection using modified hidden markov model.  _Journal of Manufacturing Science and Engineering_ , 132(2), 2010.
  * Lei et al. (2018) Y. Lei, N. Li, L. Guo, N. Li, T. Yan, and J. Lin.  Machinery health prognostics: A systematic review from data acquisition to rul prediction.  _Mechanical Systems and Signal Processing_ , 104:799–834, 2018.
  * Leng et al. (2019) J. Leng, H. Zhang, D. Yan, Q. Liu, X. Chen, and D. Zhang.  Digital twin-driven manufacturing cyber-physical system for parallel controlling of smart workshop.  _Journal of Ambient Intelligence and Humanized Computing_ , 10(3):1155–1166, 2019.
  * Leng et al. (2020) J. Leng, Q. Liu, S. Ye, J. Jing, Y. Wang, C. Zhang, D. Zhang, and X. Chen.  Digital twin-driven rapid reconfiguration of the automated manufacturing system via an open architecture model.  _Robotics and Computer-Integrated Manufacturing_ , 63:101895, 2020.
  * Lenz et al. (2015) I. Lenz, R. A. Knepper, and A. Saxena.  Deepmpc: Learning deep latent features for model predictive control.  In _Robotics: Science and Systems_ , 2015.
  * Leser et al. (2020) P. E. Leser, J. E. Warner, W. P. Leser, G. F. Bomarito, J. A. Newman, and J. D. Hochhalter.  A digital twin feasibility study (part ii): Non-deterministic predictions of fatigue life using in-situ diagnostics and prognostics.  _Engineering Fracture Mechanics_ , 229:106903, 2020.
  * Li and Lee (2021) C. Li and G. H. Lee.  From synthetic to real: Unsupervised domain adaptation for animal pose estimation.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 1482–1491, 2021.
  * Li et al. (2017) C. Li, S. Mahadevan, Y. Ling, S. Choze, and L. Wang.  Dynamic bayesian network for aircraft wing health monitoring digital twin.  _Aiaa Journal_ , 55(3):930–941, 2017.
  * Li et al. (2015) N. Li, Y. Lei, J. Lin, and S. X. Ding.  An improved exponential model for predicting remaining useful life of rolling element bearings.  _IEEE Transactions on Industrial Electronics_ , 62(12):7762–7773, 2015.
  * Li and Yang (2021) S. Li and Y. Yang.  Hierarchical deep learning for data-driven identification of reduced-order models of nonlinear dynamical systems.  _Nonlinear Dynamics_ , 105(4):3409–3422, 2021\.
  * Li et al. (2020a) T. Li, A. K. Sahu, A. Talwalkar, and V. Smith.  Federated learning: Challenges, methods, and future directions.  _IEEE Signal Processing Magazine_ , 37(3):50–60, 2020a.
  * Li et al. (2020b) W. Li, M. Rentemeister, J. Badeda, D. Jöst, D. Schulte, and D. U. Sauer.  Digital twin for battery systems: Cloud battery management system with online state-of-charge and state-of-health estimation.  _Journal of Energy Storage_ , 30:101557, 2020b.
  * Li et al. (2022) W. Li, R. Huang, J. Li, Y. Liao, Z. Chen, G. He, R. Yan, and K. Gryllias.  A perspective survey on deep transfer learning for fault diagnosis in industrial scenarios: Theories, applications and challenges.  _Mechanical Systems and Signal Processing_ , 167:108487, 2022.
  * Li and Zhang (2020) X. Li and W. Zhang.  Deep learning-based partial domain adaptation method on intelligent machinery fault diagnostics.  _IEEE Transactions on Industrial Electronics_ , 68(5):4351–4361, 2020.
  * Li et al. (2018) X. Li, Q. Ding, and J.-Q. Sun.  Remaining useful life estimation in prognostics using deep convolution neural networks.  _Reliability Engineering & System Safety_, 172:1–11, 2018\.
  * Li et al. (2019a) X. Li, W. Zhang, and Q. Ding.  Deep learning-based remaining useful life estimation of bearings using multi-scale feature extraction.  _Reliability Engineering & System Safety_, 182:208–218, 2019a.
  * Li et al. (2020c) X. Li, W. Zhang, H. Ma, Z. Luo, and X. Li.  Partial transfer learning in machinery cross-domain fault diagnostics using class-weighted adversarial networks.  _Neural Networks_ , 129:313–322, 2020c.
  * Li et al. (2019b) Y. Li, K. Liu, A. M. Foley, A. Zülke, M. Berecibar, E. Nanini-Maury, J. Van Mierlo, and H. E. Hoster.  Data-driven health estimation and lifetime prediction of lithium-ion batteries: A review.  _Renewable and Sustainable Energy Reviews_ , 113:109254, 2019b.
  * Li et al. (2021) Y. Li, D. M. Vilathgamuwa, E. Wikner, Z. Wei, X. Zhang, T. Thiringer, T. Wik, and C. Zou.  Electrochemical model-based fast charging: Physical constraint-triggered pi control.  _IEEE Transactions on Energy Conversion_ , 36(4):3208–3220, 2021.
  * Li et al. (2016) Z. Li, Y. Jiang, C. Hu, and Z. Peng.  Recent progress on decoupling diagnosis of hybrid failures in gear transmission systems using vibration sensor signal: A review.  _Measurement_ , 90:4–19, 2016.
  * Liao et al. (2021) G. Liao, H. Yin, M. Chen, and Z. Lin.  Remaining useful life prediction for multi-phase deteriorating process based on wiener process.  _Reliability Engineering & System Safety_, 207:107361, 2021.
  * Lim et al. (2020a) K. Y. H. Lim, P. Zheng, and C.-H. Chen.  A state-of-the-art survey of digital twin: techniques, engineering product lifecycle management and business innovation perspectives.  _Journal of Intelligent Manufacturing_ , 31(6):1313–1337, 2020a.
  * Lim et al. (2020b) K. Y. H. Lim, P. Zheng, C.-H. Chen, and L. Huang.  A digital twin-enhanced system for engineering product family design and optimization.  _Journal of Manufacturing Systems_ , 57:82–93, 2020b.
  * Lin et al. (1996) T. Lin, B. G. Horne, P. Tino, and C. L. Giles.  Learning long-term dependencies in narx recurrent neural networks.  _IEEE Transactions on Neural Networks_ , 7(6):1329–1338, 1996.
  * Lin et al. (2015) Y. Lin, Z. Liu, M. Sun, Y. Liu, and X. Zhu.  Learning entity and relation embeddings for knowledge graph completion.  In _Twenty-ninth AAAI Conference on Artificial Intelligence_ , 2015\.
  * Liu et al. (2020a) C. Liu, L. Le Roux, C. Körner, O. Tabaste, F. Lacan, and S. Bigot.  Digital twin-enabled collaborative data management for metal additive manufacturing systems.  _Journal of Manufacturing Systems_ , 2020a.
  * Liu et al. (2020b) C. Liu, A. Ricardo Mauricio, J. Qi, D. Peng, and K. Gryllias.  Domain adaptation digital twin for rolling element bearing prognostics.  _Online Proceedings of PHM2020_ , pages 1–10, 2020b.