# Investigating a domain adaptation approach for integrating different
measurement instruments in a longitudinal clinical registry

Maren Hackenberg∗,1,2, Michelle Pfaffenlehner1,2, Max Behrens1,2,
Astrid Pechmann3, Janbernd Kirschner3, and Harald Binder1,2,4

###### Abstract

In a longitudinal clinical registry, different measurement instruments might
have been used for assessing individuals at different time points. To combine
them, we investigate deep learning techniques for obtaining a joint latent
representation, to which the items of different measurement instruments are
mapped. This corresponds to domain adaptation, an established concept in
computer science for image data. Using the proposed approach as an example, we
evaluate the potential of domain adaptation in a longitudinal cohort setting
with a rather small number of time points, motivated by an application with
different motor function measurement instruments in a registry of spinal
muscular atrophy (SMA) patients. There, we model trajectories in the latent
representation by ordinary differential equations (ODEs), where person-
specific ODE parameters are inferred from baseline characteristics. The
goodness of fit and complexity of the ODE solutions then allows to judge the
measurement instrument mappings. We subsequently explore how alignment can be
improved by incorporating corresponding penalty terms into model fitting. To
systematically investigate the effect of differences between measurement
instruments, we consider several scenarios based on modified SMA data,
including scenarios where a mapping should be feasible in principle and
scenarios where no perfect mapping is available. While misalignment increases
in more complex scenarios, some structure is still recovered, even if the
availability of measurement instruments depends on patient state. A reasonable
mapping is feasible also in the more complex real SMA dataset. These results
indicate that domain adaptation might be more generally useful in statistical
modeling for longitudinal registry data.

1 Institute of Medical Biometry and Statistics, Faculty of Medicine and
Medical Center, University of Freiburg, Freiburg, Germany
2 Freiburg Center for Data Analysis and Modeling, University of Freiburg,
Freiburg, Germany
3 Department of Neuropediatrics and Muscle Disorders, Faculty of Medicine and
Medical Center, University of Freiburg, Freiburg, Germany
4 Centre for Integrative Biological Signalling Studies (CIBSS), University of
Freiburg, Freiburg, Germany
∗ Corresponding author<EMAIL_ADDRESS>

Keywords: domain adaptation, deep learning, latent representation, dynamic
modeling

##  1 Introduction

In longitudinal data collection in a clinical registry, often different
measurement instruments are used over time to assess individuals, e.g., as
some instruments might be more appropriate for individuals of a certain age,
or some might be used only sporadically. To obtain more information on an
individual’s development for modeling, it would thus be beneficial to
integrate these instruments. For high-volume data, such as images, domain
adaptation approaches have been quite successful in mapping data from
different domains, such as different modes of microscopy, into a joint latent
representation (Long et al., 2015; Tzeng et al., 2017). Therefore, we want
investigate the potential of such a domain adaptation approach in the setting
of clinical registries, with a particular focus on setting with a small number
of time points. This is motivated by our work with the SMArtCARE registry
(Pechmann et al., 2019) comprising patients with spinal muscular atrophy
(SMA). There, different physiotherapeutic assessments, each comprising a
larger number of motor function characteristics as items, are available for
each of the affected patients over time, reflecting individual disease
development. Due to limited time for assessment, a patient’s age, and disease
severity, different physiotheurapeutic tests have been used at different time
points for capturing different aspects of motor function. For example, the
Hammersmith Functional Motor Scale Expanded (HFMSE) test (Glanzman et al.,
2011; O’Hagen et al., 2007) assesses gross motor function skills while the
Revised Upper Limb Module (RULM) test (Mazzone et al., 2017) is used to assess
fine motor function skills of the upper limbs. While both tests are
informative about the underlying SMA disease development, they may capture
some overlapping, but also some distinct aspects. Domain adaptation then means
to map the items of these different measurement instruments to a low-
dimensional joint latent representation.

In computer science, domain adaptation is a well-established concept, and a
large number of approaches have been proposed (for a recent overview, see,
e.g., Farahani et al., 2021). These address the challenge of linking knowledge
between data from a source and a target domain, based on the assumption that
an underlying latent concept is shared between the domains, and that it is
feasible to map both domains into a joint representation. Typically, deep
learning approaches are used to learn such domain-invariant representations
(e.g., Chen et al., 2012; Long et al., 2015; Tzeng et al., 2017). So far, many
approaches focus on image data (e.g., Csurka, 2017; Ganin & Lempitsky, 2015)
and have been successfully applied also in the biomedical domain (for an
overview, see Guan & Liu, 2022), e.g., for MR imaging (e.g., Goetz et al.,
2016; Ghafoorian et al., 2017; Karani et al., 2018) or microscopy (e.g.,
Becker et al., 2015). Recently, similar approaches have also been adapted to
high-frequency time series data (e.g., Jin et al., 2022; He et al., 2023; Lai
et al., 2023), including biomedical applications (e.g., Purushotham et al.,
2016; Jude et al., 2022; Raghu et al., 2023). As these approaches focus on
rather large datasets with many individuals and time points per individual,
they cannot be directly adapted for typical clinical registry settings with a
potentially small number of patients, such as for a rare disease, and only few
time points per patient.

For our investigation, we therefore propose an approach tailored to this
setting. It is based on variational autoencoders (VAEs) (Kingma & Welling,
2014, 2019), which use artificial neural networks for learning an encoder,
i.e., a mapping to a latent representation, such that the most important
features of the data can be reconstructed based on this representation. We use
separate encoders for mapping the items of each measurement instrument, at
each time point where it is available, to a joint latent representation,
corresponding to applying different transformations to the measurement
instruments. The joint representation then allows to model an individual’s
dynamics. We focus on modeling trajectories by ordinary differential equations
(ODEs), to reflect growth dynamics and inhibition by disease. The ODE
solutions, fitted in the joint representation, then allow to assess how
closely the representation of each measurement instrument aligns to the common
trajectory. Specifically, we allow for patient-specific trajectories by
inferring individual ODE parameters from a patient’s baseline characteristics
with an additional neural network, as these characteristics are assumed to be
informative about individuals’ disease dynamics (Hackenberg et al., 2022). To
encourage alignment in the latent space, we employ an additional penalty term
as a regularization based on the differences of the representations of
individual measurement instruments to the common ODE solution. This
corresponds to the concept of adversarial domain adaptation (Tzeng et al.,
2017), where a model is trained such that an adversarial classifier is unable
to distinguish between data from different domains, i.e., different
measurement instruments, in the latent representation, inspired by generative
adversarial networks (Goodfellow et al., 2014).

Still, it may not be clear whether the measurement instruments can actually be
perfectly aligned in a latent representation or whether there are systematic
discrepancies in the measured information. For assessing this challenge, we
draw inspiration from the rich statistical literature on measurement error
(see Crainiceanu et al. (2006) for an overview). In particular, the problem of
obtaining equivalent information from different instruments measuring the same
constructs has been addressed by latent variable models (e.g., Bauer &
Hussong, 2009; Van Buuren et al., 2005; van den Heuvel et al., 2020), which
often require anchor items present across different instruments. Also, latent
class models have been employed for describing disease progression based on
multivariate longitudinal data (Proust-Lima et al., 2014, 2023).

For evaluating agreement between single score variables, simple measures can
be used (Bland & Altman, 1990). To address potentially more complex settings
in our evaluation, we specifically want to consider settings with different
degrees of overlap in the information of the two measurement instruments, such
as shifts in one instrument for a subgroup of patients, or systematic
differences in the time intervals where each measurement instrument has been
used. For this, we have designed several scenarios based on artificially
modified data from one measurement instrument in the SMArtCARE registry. This
includes scenarios where a perfect mapping is available in principle and
scenarios where this is not feasible.

We proceed by giving a brief overview of VAEs for learning a latent
representation for each measurement instrument and describe our approach for
combining VAEs with ODEs, for obtaining a joint latent trajectory and aligning
the representations of different measurement instruments, in Section 2. In
Section 3, we empirically evaluate the potential of such a domain adaptation
approach, based on the different scenarios described above. We further
illustrate the approach on data from two SMArtCARE measurement instruments and
discuss our findings in Section 4.

##  2 Methods

###  2.1 VAEs for a per-measurement-instrument mapping to a latent space

To infer an underlying latent representation from the observed data of the
items of each measurement instrument, we employ a VAE, a generative deep
learning approach that uses artificial neural networks to learn a low-
dimensional representation (Kingma & Welling, 2014, 2019). Specifically, we
use a separate VAE for each measurement instrument to map it to a latent
representation that is shared between instruments. The VAE latent space is
defined as a low-dimensional random variable ${\bf Z}$ with prior distribution
$P^{{\bf Z}}$. Two distinctly parameterized neural networks, called the
encoder and the decoder, map an observation ${\bf x}\in\mathbb{R}^{p}$, i.e.,
the items of a measurement instrument for an individual at a specific time
point, to the latent space and back to data space. The parameters of the
encoder and decoder are jointly optimized to infer a low-dimensional
representation that allows to reconstruct the original data well. Formally,
the encoder and decoder parameterize the conditional distributions $q_{{\bf
Z}\mid{\bf x}}(\cdot,{\boldsymbol{\varphi}})$ and $p_{{\bf X}\mid{\bf
z}}(\cdot,{\boldsymbol{\theta}})$, where we abbreviate conditional
distributions and densities by writing, e.g., $p_{{\bf X}\mid{\bf z}}({\bf
x},{\boldsymbol{\theta}})$ for $p_{{\bf X}\mid{\bf Z}={\bf z}}({\bf
x},{\boldsymbol{\theta}})$. The model is trained, i.e., the parameters
${\boldsymbol{\theta}}$ and ${\boldsymbol{\varphi}}$ of the encoder and
decoder networks are optimized, by maximizing the evidence lower bound (ELBO).
The ELBO is a lower bound on the data likelihood $p_{{\bf X}}$, derived based
on variational inference (Blei et al., 2017):

$\mathrm{ELBO}({\bf
x},{\boldsymbol{\varphi}},{\boldsymbol{\theta}})=E_{q_{{\bf Z}\mid{\bf
x}}(\cdot,{\boldsymbol{\varphi}})}[\log(p_{{\bf X}\mid{\bf z}}({\bf
x},{\boldsymbol{\theta}}))]-D_{\mathrm{KL}}(q_{{\bf Z}\mid{\bf
x}}(\cdot,{\boldsymbol{\varphi}})\|p_{{\bf Z}})\leq\log(p_{{\bf X}}).$

It can be shown that maximizing the ELBO is equivalent to minimizing the
Kullback-Leibler divergence between the true but intractable posterior
$p_{{\bf Z}\mid{\bf x}}(\cdot,{\boldsymbol{\theta}})$ and its approximation by
a member of a parametric variational family $\\{q_{{\bf Z}\mid{\bf
x}}(\cdot,{\boldsymbol{\varphi}})\\}$, typically assumed as a Gaussian with
diagonal covariance matrix (Blei et al., 2017; Kingma & Welling, 2014, 2019).
The first term of the ELBO can be interpreted as a reconstruction error, while
the second term acts as a regularizer that encourages densities close to the
standard normal prior. Training a VAE then corresponds to maximizing the ELBO
as a function of the parameters ${\boldsymbol{\theta}}$ and
${\boldsymbol{\varphi}}$ of the encoder and decoder, specifying the
conditional distributions, which yields both an approximate maximum likelihood
estimate for ${\boldsymbol{\theta}}$ and an optimal variational density
$q_{{\bf Z}\mid{\bf x}}(\cdot,{\boldsymbol{\varphi}})$. Such optimization is
typically performed by stochastic gradient descent (Kingma & Ba, 2015), where
the so-called reparameterization trick is used to obtain gradients of the ELBO
w.r.t. the variational parameters ${\boldsymbol{\varphi}}$ (Kingma & Welling,
2014, 2019).

To map data from two different measurement instruments into a joint space, we
use separate VAEs for each of them and align their latent spaces.
Specifically, for two measurement instruments $R$ and $S$, we represent the
observed longitudinal data of the $i$th individual’s measurement items as
matrices ${\bf x}^{R}_{i}\in\mathbb{R}^{p\times(T^{R}_{i}+1)}$, ${\bf
x}^{S}_{i}\in\mathbb{R}^{q\times(T^{S}_{i}+1)}$, of measurements of $p$ and
$q$ items, respectively, at $T^{R}_{i}+1$ and $T^{S}_{i}+1$ time points,
respectively. We define the individual-specific and measurement instrument-
specific time points as $({\bf
t}^{R}_{i}):=(t_{0},t^{R,i}_{1}...,t^{R,i}_{T^{R}_{i}})$ and $({\bf
t}^{S}_{i}):=(t_{0},t^{S,i}_{1}...,t^{S,i}_{T^{S}_{i}})$, where $t_{0}$
denotes the common baseline time point. We define corresponding separate VAEs,
$\mathrm{VAE}_{R}$ with parameters
${\boldsymbol{\theta}}_{R},{\boldsymbol{\varphi}}_{R}$ and $\mathrm{VAE}_{S}$
with parameters ${\boldsymbol{\theta}}_{S},{\boldsymbol{\varphi}}_{S}$, and
for each latent space specify the variational posterior as multivariate
Gaussian with diagonal covariance matrix, parameterized by the respective
VAE’s encoder.

The encoder of $\mathrm{VAE}_{R}$ then maps the observed time series to the
posterior mean
${\boldsymbol{\mu}}^{R}_{i}:=({\boldsymbol{\mu}}_{i}^{R,t})_{t\in({\bf
t}_{i}^{R})}\in\mathbb{R}^{d\times(T^{R}_{i}+1)}$ and standard deviation
${\boldsymbol{\sigma}}^{R}_{i}:=({\boldsymbol{\sigma}}_{i}^{R,t})_{t\in({\bf
t}_{i}^{R})}\in\mathbb{R}^{d\times(T^{R}_{i}+1)}$, where $d$ is the dimension
of the latent space. For the measurement instrument $S$, the posterior mean
${\boldsymbol{\mu}}^{S}_{i}$ and standard deviation
${\boldsymbol{\sigma}}^{S}_{i}$, obtained from the encoder of
$\mathrm{VAE}_{S}$, are defined analogously.

###  2.2 Aligning the latent spaces of each measurement instrument by fitting
a joint trajectory via ODEs

To align the two latent spaces of $\mathrm{VAE}_{R}$ and $\mathrm{VAE}_{S}$,
we solve a differential equation, using the latent representations from both
measurement instruments to fit a joint trajectory. We then evaluate the joint
solution at the time points of each measurement instruments and use the values
of the joint trajectory at the time points of $R$ as input for the decoder of
$\mathrm{VAE}_{R}$, and at the time points of $S$ for the decoder of
$\mathrm{VAE}_{S}$.

For obtaining the joint ODE solution, we build on an approach that we have
proposed previously (Hackenberg et al., 2023). Specifically, we solve a linear
ODE system multiple times, using each latent value obtained from the items of
both measurement instruments observed in the course of time as the initial
condition. The resulting individual solutions are combined into an inverse-
variance weighted average using time-dependent weights, to obtain an unbiased
minimum variance estimator of the true underlying dynamics (Shahar, 2017).
This allows to reduce dependence of the ODE solution on the initial condition,
i.e., the first value observed in the course of time, and incorporate all
observed values into the fitting process. Specifically, we assume that for
each individual $i$ the posterior means
${\boldsymbol{\mu}}_{i}^{R},{\boldsymbol{\mu}}_{i}^{S}$ of the latent
representations of both measurement instruments are governed by a common
underlying $d$-dimensional linear ODE system with constant coefficients
$A_{i},{\bf c}_{i}$. We drop the index $i$ for ease of notation and consider
for a specific individual and for each
$k=t_{0},t^{R}_{1},\dots,t^{R}_{T^{R}},t^{S}_{1},\dots,t^{S}_{T_{S}}$ the
initial value problem

$\displaystyle\begin{split}\frac{d}{dt}{\boldsymbol{\mu}}(t)&=A\cdot{\boldsymbol{\mu}}(t)+{\bf
c};\\\
{\boldsymbol{\mu}}(k)&={\boldsymbol{\mu}}^{\\{R,S\\},k}:=\begin{cases}{\boldsymbol{\mu}}^{R,k};&k\in({\bf
t}^{R}),\\\ {\boldsymbol{\mu}}^{S,k};&k\in({\bf
t}^{S}),\end{cases}\end{split}$ (1)

where the latent mean at the $k$-th time point is used as the initial
condition. For $A=0$, the solution to (1) is given by
${\boldsymbol{\mu}}(t)={\bf c}\cdot t+{\boldsymbol{\mu}}^{\\{R,S\\},k}$. For
$A\neq 0$, the solution can be computed analytically (see (Hackenberg et al.,
2023) for details):

$\displaystyle{\boldsymbol{\mu}}(t)=\exp(A(t-k))\cdot(A^{-1}{\bf
c}+{\boldsymbol{\mu}}^{\\{R,S\\},k})-A^{-1}{\bf c}.$ (2)

We define the solutions of Equation (1) for each
$k=t_{0},t^{R}_{1},\dots,t^{R}_{T^{R}},t^{S}_{1},\dots,t^{S}_{T_{S}}$ as

$\displaystyle\widetilde{{\boldsymbol{\mu}}}_{k}(t,{\boldsymbol{\eta}}):=\begin{cases}(A^{-1}{\bf
c}+{\boldsymbol{\mu}}^{\\{R,S\\},k})\exp(A(t-k))-A^{-1}{\bf c};&A\neq 0\\\
{\bf c}\cdot t+{\boldsymbol{\mu}}^{\\{R,S\\},k};&A=0,\end{cases}$

where ${\boldsymbol{\eta}}=\\{A,{\bf c}\\}$ and
${\boldsymbol{\mu}}^{\\{R,S\\},k}$ is used as initial condition. The solutions
are then combined to a time-dependent inverse-variance weighted average as an
estimator for the true underlying dynamics (see (Hackenberg et al., 2023) for
details):

$\displaystyle\widetilde{{\boldsymbol{\mu}}}(t,{\boldsymbol{\eta}}):=\frac{\sum_{k\in({\bf
t}^{R})\cup({\bf
t}^{S})}\mathrm{Var}\left[\widetilde{{\boldsymbol{\mu}}}_{k}(t,{\boldsymbol{\eta}})\right]^{-1}\widetilde{{\boldsymbol{\mu}}}_{k}(t,{\boldsymbol{\eta}})}{\sum_{k\in({\bf
t}^{R})\cup({\bf
t}^{S})}\mathrm{Var}\left[\widetilde{{\boldsymbol{\mu}}}_{k}(t,{\boldsymbol{\eta}})\right]^{-1}}.$
(3)

For calculating the weights in Equation (3), we need to estimate the unknown
variance
$\mathrm{Var}[\widetilde{{\boldsymbol{\mu}}}_{k}(t,{\boldsymbol{\eta}})]$ at a
time point $t$ of the ODE solution with initial value
${\boldsymbol{\mu}}^{\\{R,S\\},k}$. For this, we use all encoded values
${\boldsymbol{\mu}}^{\\{R,S\\},j}$ for $k<j<t$, i.e., from time points between
the current initial time point $k$ and the time point $t$ of interest, and
calculate the sample variance $s^{2}$ of the corresponding ODE solutions
$\widetilde{{\boldsymbol{\mu}}}_{j}(t,{\boldsymbol{\eta}})$ for all $j$ with
$k<j<t$.

In our application setting of a clinical registry, typically a more extensive
patient characterization at baseline is available, including, e.g., the age at
symptom onset or treatment information, which we assume to be informative of
intra-individual differences in the underlying disease dynamics. To obtain
personalized dynamics conditional on this baseline information, we use an
additional neural network to map the baseline variables to individual ODE
parameters ${\boldsymbol{\eta}}_{i}$, as in our previous work (Hackenberg et
al., 2022). From Equation (3) we thus obtain estimators of person-specific
trajectories $\widetilde{{\boldsymbol{\mu}}}_{i}(t,{\boldsymbol{\eta}}_{i})$
for each individual $i$.

We subsequently evaluate
$\widetilde{{\boldsymbol{\mu}}}_{i}(\cdot,{\boldsymbol{\eta}}_{i})$ at $({\bf
t}_{i}^{R})$, the individual’s measurement time points of measurement
instrument $R$, and analogously at $({\bf t}_{i}^{R})$, the time points of
$S$, to obtain the posterior means according to the joint trajectory for both
measurement instruments. We then sample ${\bf
z}_{i}^{R,k_{i}}\sim\mathcal{N}(\widetilde{{\boldsymbol{\mu}}}_{i}(k_{i},{\boldsymbol{\eta}}_{i}),{\boldsymbol{\sigma}}_{i}^{k_{i}})$
for $k_{i}\in({\bf t}^{R}_{i})$ for measurement instrument $R$ and pass the
sampled latent longitudinal data to the decoder of $\mathrm{VAE}_{R}$ to
obtain reconstructed longitudinal data $\widehat{{\bf x}}_{i}^{R}$.
Analogously, for $S$, we sample ${\bf
z}_{i}^{S,k_{i}}\sim\mathcal{N}(\widetilde{{\boldsymbol{\mu}}}_{i}(k_{i},{\boldsymbol{\eta}}_{i}),{\boldsymbol{\sigma}}_{i}^{k_{i}})$
for $k_{i}\in({\bf t}^{S}_{i})$ and use the decoder of $\mathrm{VAE}_{S}$ to
obtain $\widehat{{\bf x}}_{i}^{S}$. The models are trained to provide
reconstructed longitudinal data that is as similar as possible to the original
observations, to ensure a good fit in the latent space, which allows for a
good reconstruction of the original data.

###  2.3 Optimization based on a joint loss function and adversarial penalty
term

We jointly optimize both VAEs and the neural network for obtaining the ODE
parameters by maximizing the respective ELBOs of $\mathrm{VAE}_{R}$ and
$\mathrm{VAE}_{S}$, where we use $\widetilde{{\boldsymbol{\mu}}}_{i}({\bf
t}_{i}^{R},{\boldsymbol{\eta}}_{i})$ and
$\widetilde{{\boldsymbol{\mu}}}_{i}({\bf t}_{i}^{S},{\boldsymbol{\eta}}_{i})$
as the latent posterior means obtained from the joint ODE solution,
respectively. Maximizing the ELBOs encourages the model to learn a latent
space based on which the original data can be reconstructed well. More
specifically, an optimal model fit should provide a close alignment of both
measurement instruments to the latent ODE while still explaining underlying
variance, i.e., it should avoid a trivial solution such as a constant
trajectory.

To enforce a good mapping where the latent representations of each measurement
instruments are aligned to the common trajectory, we propose an additional
regularizing penalty term based on the ODE solution. Specifically, we optimize
the encoders of each measurement instrument such that it is not possible to
distinguish between their latent representations based on their difference to
the ODE solution. This corresponds to an adversarial domain adaptation
approach (Tzeng et al., 2017), where the model is encouraged to find an
invariant representation, in which an adversarial classifier is unable to
distinguish between domains, i.e., representations of different measurement
instruments. Conceptually, this approach is inspired by generative adversarial
networks (GANs) (Goodfellow et al., 2014). In a GAN, a generator and a
discriminator neural network are trained in a competitive way to learn the
distribution of the input data. The generator creates synthetic observations
which the discriminator tries to distinguish from real data observations. The
generator is optimized to fool the discriminator, and thus learns to
approximate the data-generating distribution. In our scenario, we optimize the
VAE encoders of both measurement instruments to find a representation in which
the latent values from each instrument are not distinguishable based on the
ODE solution, i.e., the encoders are trained as generator to fool a
discriminator that uses the ODE solution for judging differences between
measurement instruments. To achieve this, we compare the respective average
differences between each measurement instrument’s encoded latent values and
the ODE solution across instruments. In particular, we do not use absolute or
quadratic deviations, such that encoded values with the same difference above
and below the ODE solution cancel out. The ODE solution can thus be viewed as
an adversarial classifier that distinguishes between the measurement
instruments based on average distance. We then add the difference between the
per-instrument differences as a penalty term to the loss function, such that
the function would be minimized with respect to this term if the difference
were zero, i.e., both measurement instruments’ encoded values are equally well
aligned to the ODE solution. To further encourage consistency of each latent
representation before and after solving the ODE, we add the squared Euclidean
distance between the encoded values ${\boldsymbol{\mu}}_{i}^{R}$,
${\boldsymbol{\mu}}_{i}^{S}$ and the posterior means according to the smooth
dynamics, $\widetilde{{\boldsymbol{\mu}}}_{i}({\bf
t}_{i}^{R},{\boldsymbol{\eta}}_{i})$, $\widetilde{{\boldsymbol{\mu}}}_{i}({\bf
t}_{i}^{S},{\boldsymbol{\eta}}_{i})$.

To ensure that there is no trade-off between a close alignment and explaining
variability, and in particular to avoid a trivial constant fit, we encourage
similar variances of the ODE solutions and encoded values, separately for $R$
and $S$. Specifically, we add the ratio between the sample variance of all
solutions $\widetilde{{\boldsymbol{\mu}}}_{i,k}({\bf
t}_{i}^{R},{\boldsymbol{\eta}}_{i})$ and the sample variance of all encoded
values ${\boldsymbol{\mu}}_{i}^{R}$, summed across all observed time points,
where an offset of $1$ is added to avoid values near zero, and analogously for
$S$.

The final loss function is thus given by

$\begin{split}\mathcal{L}({\bf x}_{i}^{R},{\bf
x}_{i}^{S},&{\boldsymbol{\eta}}_{i},{\boldsymbol{\theta}}_{R},{\boldsymbol{\varphi}}_{R},{\boldsymbol{\theta}}_{S},{\boldsymbol{\varphi}}_{S})\\\
=&-\mathrm{ELBO}_{R}({\bf
x}_{i}^{R},{\boldsymbol{\eta}}_{i},{\boldsymbol{\theta}}_{R},{\boldsymbol{\varphi}}_{R})\\\
&-\mathrm{ELBO}_{S}({\bf
x}_{i}^{S},{\boldsymbol{\eta}}_{i},{\boldsymbol{\theta}}_{S},{\boldsymbol{\varphi}}_{S})\\\
&+\alpha\left(\frac{1}{T_{i}^{R}+1}\sum_{k\in({\bf
t}_{i}^{R})}\left(\widetilde{{\boldsymbol{\mu}}}_{i}(k,{\boldsymbol{\eta}}_{i})-{\boldsymbol{\mu}}_{i}^{R,k}\right)-\frac{1}{T_{i}^{S}+1}\sum_{k\in({\bf
t}_{i}^{S})}\left(\widetilde{{\boldsymbol{\mu}}}_{i}(k,{\boldsymbol{\eta}}_{i})-{\boldsymbol{\mu}}_{i}^{S,k}\right)\right)\\\
&+\beta\left(\|{\boldsymbol{\mu}}_{i}^{R}-\widetilde{{\boldsymbol{\mu}}}_{i}({\bf
t}_{i}^{R},{\boldsymbol{\eta}}_{i})\|_{2}^{2}+\|{\boldsymbol{\mu}}_{i}^{S}-\widetilde{{\boldsymbol{\mu}}}_{i}({\bf
t}_{i}^{S},{\boldsymbol{\eta}}_{i})\|_{2}^{2}\right)\\\
&+\gamma\left(\sum_{k\in({\bf
t}_{i}^{R})}\frac{s^{2}(\widetilde{{\boldsymbol{\mu}}}_{i}({\bf
t}_{i}^{R},{\boldsymbol{\eta}}_{i}))+1}{s^{2}({\boldsymbol{\mu}}_{i}^{R})+1}+\sum_{k\in({\bf
t}_{i}^{S})}\frac{s^{2}(\widetilde{{\boldsymbol{\mu}}}_{i}({\bf
t}_{i}^{S},{\boldsymbol{\eta}}_{i}))+1}{s^{2}({\boldsymbol{\mu}}_{i}^{S})+1}\right)\\\
\end{split}$ (4)

where $\alpha,\beta,\gamma\in\mathbb{R}_{+}$ are hyperparameters balancing the
loss components.

##  3 Empirical evaluation

###  3.1 Rare disease registry application

To illustrate our approach, we consider longitudinal data on SMA patients’
motor function development collected during routine visits into the SMArtCARE
registry, a prospective multicenter cohort study (Pechmann et al., 2019).
Ethical approval of the study has been granted by the Ethics Committee of the
University of Freiburg, No.56/18. So far, the registry provides only few time
points per patient with irregular timing and frequency of follow-up visits. In
addition to an extensive baseline characterization, motoric ability is
assessed with physiotherapeutic tests, each comprising several items, at
follow-up visits. SMA is caused by a homozygous deletion in the survival motor
neuron (SMN) 1 gene on chromosome 5, which is essential for normal motor
neuron function (Lefebvre et al., 1995). As a result of the SMN deficiency,
muscles do not receive signals from motor neurons, which leads to atrophy,
i.e., muscle degeneration. Three different disease-modifying drugs
(Nusinersen, Risdiplam, and Onasemnogene Abeparvovec) are available for the
treatment of SMA patients that aim to increase production of the SMN protein
to maintain functional motor neurons (Schorling et al., 2020; Messina et al.,
2022). These underlying disease processes cannot be observed directly, but are
implicitly reflected in the motor function assessments.

Specifically, we consider patients treated with Nusinersen who completed the
RULM assessment (Mazzone et al., 2017) and/or the HFMSE assessment (Glanzman
et al., 2011). The HFMSE assessment comprises $33$ items with a maximum score
of $66$ to evaluate gross motor function of the whole body and can be
conducted for patients with the ability to sit and/or walk older than two
years, while the RULM assessment comprises $20$ items with a maximum sum score
of $37$ to evaluate motor function of upper limbs and can be conducted for all
patients older than two years and able to sit in a wheelchair (Pechmann et
al., 2019).

In our investigation, we use all test items as time-dependent variables and
all available baseline information, including, e.g., SMA subtype, age at
symptom onset and first treatment, and genetic test results. For each patient
and each of the two motor function assessments, we removed outlier time points
where the difference in HFMSE/RULM sum score to the previous time point
exceeded two times the interquartile range of all sum score differences
between adjacent time points. For each assessment separately, we filtered out
patients with less than two observation time points, and patients with
HFMSE/RULM sum score variance smaller than $0.5$, i.e., with nearly constant
trajectories across all items. This resulted in a dataset of $558$ patients
with HFMSE and/or RULM measurements. Of these, $431$ have HFMSE measurements
at between $2$ and $15$ time points (median $7$ time points), corresponding to
in total $3103$ observations of $33$ time-dependent variables. $427$ patients
have RULM measurements at between $2$ and $13$ time points (median $7$ time
points), corresponding to in total $2984$ observations of $20$ time-dependent
variables. In addition, we used $24$ baseline variables. Integer-valued test
items of both RULM and HFMSE were rescaled and a logit transformation was
applied to account for the Gaussian generative distribution parameterized by
the VAE decoder.

###  3.2 Synthetic dataset scenarios with different error sources

To investigate different potential scenarios of systematic differences in
measurement instruments and their effect on how well they can be aligned to a
common trajectory in a latent space, we create modifications of the dataset
and artificially introduce different sources of discrepancies. Specifically,
we use data from one measurement instrument only and use a subscale, i.e., a
subset of items, as the synthetic second measurement instrument, as
schematically illustrated in Figure 1a. We use the RULM assessment in the
SMArtCARE registry as the original measurement instrument $R$. Thus, from the
dataset described in Section 3.1, we consider only the patients with RULM
measurements. We use a subscale of the RULM items, comprising $5$ items that
concern the handling of small weights as second measurement instrument $S$.
When no modifications are introduced, the information in our second
measurement instrument is fully contained in the first measurement instrument,
i.e., a close alignment should be feasible in principle. We consider this
scenario as a baseline and then create different modifications of the second
measurement instrument, corresponding to the subscale, to reflect different
potential sources of systematic discrepancies between measurement instruments.

Figure 1: a: Creating an artificial second measurement instrument by using a
subscale of a single existing measurement instrument. b: Different dataset
modifications: We artificially modify our synthetic second measurement
instrument to create different hypothetical scenarios of how discrepancies
between instruments could arise. Top left: Baseline scenario with no
discrepancies. Top right: Time points are randomly deleted for all patients.
Middle left: A shift is added to the items of the second measurement
instrument for all patients. Middle right: Time points are deleted
preferentially at later time points for all patients. Bottom left: A shift is
added for a random subgroup of patients. Bottom right: The second measurement
instrument is only available if the sum score of the first instrument exceeds
a given threshold.

The baseline scenario and different modifications are schematically
illustrated in Figure 1b. Specifically, we consider two scenarios where a
shift is added to all items of the second measurement, reflecting the
assumption of a systematic bias in this instrument. In the first scenario, the
shift is added for all patients and at all time points, reflecting a patient-
independent and time-independent source of error (Figure 1b, middle left). In
the second scenario, the shift is added only for a random subgroup of patients
(Figure 1b, bottom left), reflecting a patient-dependent and time-independent
error that could be explained by, e.g., differences in (potentially
unobserved) patient characteristics that cause a bias in the measurements.

Next, we consider scenarios where discrepancies are introduced by deleting the
information of the second measurement instrument at certain time points.
First, we randomly remove observation time points of the second measurement
instrument with an equal dropout probability at all time points and for all
patients (Figure 1b, top right), to reflect a scenario where the instrument
was generally applied less frequently for all patients. We then randomly
remove observation time points of the second measurement instrument
preferentially at later time points, i.e., with a higher dropout probability
for later time points, reflecting the assumption of a time-dependent but
patient-independent error (Figure 1b, middle right). Finally, we consider a
scenario where the second measurement instrument is used conditional on a
patient’s score in the first measurement, i.e., where discrepancies depend on
patients’ disease development. In particular, we consider observations of the
second measurement instrument only if the sum score of all items of the first
measurement instrument exceeds a given threshold, and delete all other
observations (Figure 1b, bottom right). This reflects the assumption that the
second measurement instrument is applied additionally after a ceiling effect
is observed in the first measurement instrument, i.e., to still detect changes
in a patient’s development when the first instrument may not be informative
anymore. For example, in the SMArtCARE registry, the RULM test is sometimes
used as an additional assessment when ceiling effects in the HFMSE score are
observed (see also Mazzone et al., 2017).

Implementation details for each modification are provided in the Appendix,
Section A.2.

###  3.3 Results on synthetically modified datasets

On each of the synthetically created and modified datasets described in the
previous Section 3.2, we train the model presented in Section 2, using
separate VAEs for the original RULM measurements as measurement instrument $R$
and the (modified) subscale as second measurement instrument $S$. We use a
two-dimensional latent space and a homogeneous linear ODE system, i.e., where
$\frac{d}{dt}{\boldsymbol{\mu}}(t)=A\cdot{\boldsymbol{\mu}}(t)\in\mathbb{R}^{2}$
and $c=0$. We use the same model architecture and hyperparameter configuration
for all datasets to ensure a fair comparison (for details, see Section A.1).
To investigate the effect of the synthetic modifications on the alignment in
the latent representation and evaluate the performance of our approach, we
consider the misalignment between the latent representations of both
measurement instruments after training the model. Specifically, for each
patient we calculate the mean absolute difference between the latent
representations of each measurement instrument at all time points where both
measurement instruments were observed:

$\Delta_{i}^{R,S}:=\frac{1}{|{\bf t}_{i}^{R}\cap{\bf
t}_{i}^{S}|}\sum_{t\in{\bf t}_{i}^{R}\cap{\bf
t}_{i}^{S}}\mid{\boldsymbol{\mu}}_{i}^{R,t}-{\boldsymbol{\mu}}_{i}^{S,t}\mid\in\mathbb{R}^{d}.$

Yet, the absolute values of $\Delta_{i}^{R,S}$ depend on the scale of the
values in the latent representation, which can be flexibly set by the encoder
neural network. In addition, what may be considered a large difference depends
on the dynamics of the common latent trajectory. For a patient with not much
dynamics and a nearly constant ODE trajectory, we would expect smaller
differences between the representations of each measurement instrument than
for a patient with large changes in dynamics, where the same absolute
differences between representations are smaller in relation to the dynamics
and might thus be still acceptable. To compare the values of
$\Delta_{i}^{R,S}$ across patients, we thus consider them in relation to the
variability of each patient’s ODE trajectory. Specifically, for each patient
we calculate the absolute difference between the values of the ODE solution at
the first and last time point of the time interval,

$\Delta_{i}^{\mathrm{ODE}}:=\mid\widetilde{{\boldsymbol{\mu}}}_{i}(t_{\max\\{T_{i}^{R},T_{i}^{S}\\}}),{\boldsymbol{\eta}}_{i})-\widetilde{{\boldsymbol{\mu}}}_{i}(t_{0},{\boldsymbol{\eta}}_{i})\mid\in\mathbb{R}^{d}.$

For each scenario, we calculate $\Delta_{i}^{R,S}$ and
$\Delta_{i}^{\mathrm{ODE}}$ for all patients $i=1,\dots,N$ and visualize them
in scatterplots. Ideally, the variability in ODE dynamics should be greater
than the discrepancies between measurement instruments, i.e., the better the
alignment, the more points lie above the diagonal.

First, we focus on the baseline scenario where no modification was introduced
in the subscale. This scenario can be viewed as a performance reference for
the following scenarios with modifications, as all the information of the
second measurement instrument is already contained in the first one, such that
a close alignment should be feasible in principle. Nonetheless, due to the
random initialization of each model and the iterative optimization of a non-
convex loss function, empirically it might not be possible to achieve a
perfect mapping. This scenario thus allows to quantify potential discrepancies
due to other reasons.

Figure 2: Evolution of the alignment of measurement instruments in the
baseline scenario where no modification was applied over the course of $25$
training epochs. Each subplot corresponds to the state of the model after
training for the specified number of epochs. The two panels of each subplot
correspond to the two latent dimensions. In each panel, we show the values of
$(\Delta_{i}^{R,S})_{i}$, i.e., the mean absolute difference between the
latent representations of the two measurement instruments in the first resp.
second latent dimension, on the $x$ axis and the corresponding values of
$(\Delta_{i}^{\mathrm{ODE}})_{i}$, i.e., the differences in ODE dynamics in
the first resp. second latent dimension, on the $y$ axis. The solid red line
corresponds to the diagonal, and the points above the diagonal, i.e., where
$\Delta_{i}^{\mathrm{ODE}}>\Delta_{i}^{R,S}$ are colored in dark grey, while
the points below the diagonal where
$\Delta_{i}^{\mathrm{ODE}}<\Delta_{i}^{R,S}$ are colored in light grey.

In Figure 2, we visualize the changes in the
$(\Delta_{i}^{R,S},\Delta_{i}^{\mathrm{ODE}})_{i=1,\dots,N}$ values over the
course of the first $25$ training epochs, separately for each of the two
latent dimensions. This might also be used as a reference for assessing
performance later in more complex scenarios. At the beginning of training, for
many patients the difference between the two measurement instruments is
greater than the variability explained by the ODE dynamics (indicated by many
values below the diagonal). As training progresses, this ratio changes
(indicated by more and more values below the diagonal), until after $25$
epochs, for nearly all patients the difference between the measurement
instruments is (often considerably) smaller than the changes in ODE dynamics.
Note that also the scale of the differences between measurement instruments
changes from $\left[0,4\right]$ to $\left[0,0.5\right]$, i.e., not only the
ratio of the two values changes, but also the absolute values of
$(\Delta_{i}^{R,S})_{i}$ become much smaller. Thus, in a scenario where a
close alignment is feasible, the model is effective in minimizing the
differences between measurement instruments and achieves a very close, yet not
perfect mapping.

Figure 3 shows scatterplots of each patient’s $\Delta_{i}^{R,S}$ and
$\Delta_{i}^{\mathrm{ODE}}$ values after model training in all scenarios with
a synthetic modification of the second measurement instrument described in
Section 3.2.

In the first row, we show results for two modifications where for all
patients, time points of the second measurement instrument were deleted at
random time points (left) or preferentially at later time points (right). In
both scenarios, for most patients $\Delta_{i}^{R,S}$ is smaller than
$\Delta_{i}^{\mathrm{ODE}}$, and the absolute values of
$(\Delta_{i}^{R,S})_{i}$ are mostly in a similar range as in the baseline
scenario (almost all values below $0.5$, two/four values between $0.5$ and
$0.7$ and one outlier $>1$). Deleting items with a higher probability at later
time points corresponds to a more challenging task, as the model has to deal
with potentially longer periods of missing information, and has to extrapolate
rather than interpolate. Yet, the number of values above the diagonal is only
slightly higher in this scenario, indicating that the approach is generally
capable of handling also these more demanding challenges.

In the second row, we show results for applying a shift to all items of the
second measurement instrument at all time points either for all patients
(left) or for a randomly sampled subgroup of patients (right). In both
scenarios, the number of points below the diagonal is smaller than for the
scenarios where time points were deleted, and in a similar range as in the
baseline scenario (all values $<0.5$, only one outlier in the second scenario
slightly larger than $0.5$). Particularly in the first scenario of a patient-
independent shift, the alignment is only slightly worse than in the baseline
scenario with no modification. This shows that a simple additive
transformation can be easily handled by the model, even if it is only present
in a subgroup.

Figure 3: Results of model training on different dataset modifications with
artificially introduced discrepancies. Each of the five subplots corresponds
to a specific dataset modification, symbolized by the icon in the top left
corner (see Figure 1 and the corresponding explanation in Section 3.2). The
two panels of each subplot correspond to the two dimensions of the latent
representation. In each panel, we show the values of $(\Delta_{i}^{R,S})_{i}$,
i.e., the mean absolute difference between the latent representations of the
two measurement instruments in the first resp. second latent dimension, on the
$x$ axis and the corresponding values of $(\Delta_{i}^{\mathrm{ODE}})_{i}$,
i.e., the differences in ODE dynamics in the first resp. second latent
dimension, on the $y$ axis. The solid red line corresponds to the diagonal,
and the points above the diagonal, i.e., where
$\Delta_{i}^{\mathrm{ODE}}>\Delta_{i}^{R,S}$ are colored in dark grey, while
the points below the diagonal where
$\Delta_{i}^{\mathrm{ODE}}<\Delta_{i}^{R,S}$ are colored in light grey.

In the third row, we show results for the scenario where the observation of
the second measurement depends on the sum score of all items of the first
measurement instrument. Specifically, the second measurement instrument is
only observed if the sum score of the first measurement instrument exceeds the
threshold, i.e., is applied when ceiling effects are detected in the first
measurement instrument. This scenario is considerably more challenging, as the
second measurement instrument is missing completely for a subgroup of patients
(as can be seen by the overall smaller number of points in the scatterplot).
The scale of the $(\Delta_{i}^{R,S})_{i}$ values is still comparable to the
other scenarios, with only $2$ values $>0.5$. Yet, here the scale of the
$(\Delta_{i}^{\mathrm{ODE}})_{i}$ values is smaller, in particular in the
first dimension, i.e., the ODE solution is more strongly regularized, as
overall less information is available for fitting. Still, also in this
challenging scenario, for most patients
$\Delta_{i}^{R,S}<\Delta_{i}^{\mathrm{ODE}}$, i.e., a decent alignment is
still feasible when the second measurement instrument is observed dependent on
a patient’s status, and generally much less frequently.

###  3.4 Results on two different measurement instruments in the SMArtCARE
registry

We apply our model in a real data example with two real measurement
instruments, i.e., without artificial modifications, where discrepancies
between measurement instruments could arise due to more complex reasons.
Specifically, we consider data from the HFMSE and RULM measurement instruments
in the SMArtCARE data as described in Section 3.1. Measurement instrument $R$
corresponds to patients’ HFMSE measurements, and measurement instrument $S$
corresponds to patients’ RULM measurements.

On this dataset, we compare different versions of penalties in our approach
with respect to their effectiveness in achieving a close alignment.
Correspondingly, we train the model with four different loss function
versions. We use (1) neither the adversarial penalty term nor the penalty for
aligning each measurement’s representation separately to the ODE trajectory
(referred to as the ’ODE penalty’ in the following), i.e., $\alpha=\beta=0$.
Then, we train models with (2) only the ODE penalty, i.e., $\alpha=0,\beta\neq
0$, and (3) only the adversarial penalty, i.e., $\alpha\neq 0,\beta=0$, and
finally (4) with both penalty terms, i.e., $\alpha,\beta\neq 0$. All other
hyperparameters are kept constant across the four types of model estimation to
ensure comparability.

In Figure 4, we visualize exemplary model fits for $12$ randomly selected
individuals from the SMArtCARE registry obtained for the four considered
penalty approaches. Applying the ODE penalty only (Figure 4, top right) helps
to reduce the distances between the latent representations
${\boldsymbol{\mu}}_{i}^{R}$, ${\boldsymbol{\mu}}_{i}^{S}$ to the common ODE
trajectory $\widetilde{{\boldsymbol{\mu}}}(t,{\boldsymbol{\eta}})$. The
adversarial penalty is even more effective in achieving a close alignment
between the representations of the two different measurement instruments
(Figure 4, bottom left). When both terms are applied together, the closest
alignment is achieved. Yet, the ODE penalty also has a regularizing effect on
the fitted dynamics, i.e., the ODE trajectories tend to become closer to
constant if that term is added, whereas the adversarial penalty term has no
effect on the variance explained by the ODE solution. In this specific
setting, we would thus recommend to only use the adversarial penalty term for
obtaining a representation that provides a good alignment yet still explains
underlying variance.

Figure 4: Exemplary model fits of $12$ randomly selected individuals from the
SMArtCARE registry for four model fitting versions, using different
combinations of penalty terms for alignment, shown in subplots. In each
subplot, each panel shows the latent representation and fitted ODE trajectory
of the data from one patient, the $x$ axis corresponds to the time in months
and the $y$ axis to the value of the latent representation. The
$2$-dimensional latent representations ${\boldsymbol{\mu}}_{i}^{R}$ obtained
directly from the encoder of the HFMSE measurements are depicted as squares in
light colors, blue for the first dimension and orange for the second
dimension, whereas the latent representations ${\boldsymbol{\mu}}_{i}^{S}$
obtained from the RULM encoder are depicted as circles in darker colors, again
using blue for the first dimension and orange for the second dimension The
common ODE trajectory $\widetilde{{\boldsymbol{\mu}}}_{i}$ is shown as blue
resp. orange line for the first resp. second latent dimension.

##  4 Discussion

Domain adaptation is a prominent concept in computer science, but has rarely
been considered in statistical modeling of longitudinal data so far. In
particular, it is implicitly assumed that an underlying latent concept is
shared between data from two different domains, and a fully domain-invariant
mapping is in principle feasible. Yet, in more complex clinical cohort
applications such as a rare disease registry with small numbers of patients
and few and irregular time points per patient, it is not clear to what extent
different measurement instruments can be aligned or whether no useful mapping
is possible.

To investigate the potential of domain adaptation for such settings, we have
therefore developed and evaluated an approach that is tailored for mapping
longitudinal measurements from different measurement instruments in a clinical
registry with few time points. As a motivating application, we have considered
different measurement instruments used to assess motor function of individuals
in an SMA registry at different time points. For aligning the different motor
function measurement instruments, each comprising a larger number of
individual items, we have separately mapped the items of each instrument to a
latent space using a VAE, for learning a flexible low-dimensional
representation based on neural networks. We have allowed for individual
trajectories by inferring person-specific ODE parameters from baseline
variables with an additional neural network, jointly optimizing the VAEs for
the different measurement instruments, the neural network for inferring ODE
parameters, and the dynamic model in the latent space. This is to find a
representation that is optimized to provide a good mapping between measurement
instruments. To enforce a close alignment during model optimization, we have
developed an additional penalty term, inspired by adversarial classifiers in
domain adaptation and GANs, which encourages the model to find a latent space
where the representations of different measurement instruments cannot be
distinguished based on the ODE solution. To specifically investigate what
happens in scenarios when no perfect mapping can be achieved, we have designed
synthetic modifications of an SMA dataset by artificially introducing
discrepancies with varying degrees of complexity, to characterize the effect
of such systematic discrepancies on the alignment.

Our findings show that our proposal, as a prototypical domain adaptation
approach, can achieve a close alignment in simple settings, and still provides
a reasonable mapping in complex settings, e.g., when one measurement
instrument is observed only if the sum score of another measurement instrument
exceeds a given threshold, i.e., when it is observed dependent on a patient’s
status and not at all for many patients. To exemplify how domain adaptation
approaches could be adapted in a real world setting, where discrepancies might
arise due to even more intricate reasons, we have considered data from two
different measurement instruments in the SMArtCARE registry. We investigated
different types of model estimation in our approach, using different penalty
terms. There, we could show that the adversarial penalty was particularly
important for achieving a close alignment, and that a reasonable, yet not
perfect, mapping was achievable despite the increased complexity. This
application also illustrates potential trade-offs in complex settings where a
perfect mapping might not be available, as enforcing a closer alignment to the
ODE solution with an ODE penalty term also implied more strongly regularized
ODE solutions, and thus less explained variance in the dynamics.

In our application, we have still observed differences in the alignment across
individuals, and also between different observed time intervals for the same
individual, suggesting that there are time-dependent and time-independent
sources of intra-individual differences which prevent a closer alignment.
While in the present manuscript, we have focused on investigating the general
potential of a domain adaptation approach, these differences might need to be
characterized in more detail in future work. For example, the observed
patient-and time-dependent differences between measurement instruments could
be linked to patient characteristics observed in the baseline variables.
Further, the choice of the ODE system used for obtaining the common trajectory
might influence the quality of the alignment, e.g., depending on whether an
interaction between different latent dimensions is possible or not. Due to the
small number of patients, we have focused on a simple linear system and have
not investigated the effect of different choices of ODE systems in depth so
far.

In summary, we have illustrated how a domain adaptation approach can be
adapted to address the challenge of different measurement instruments in
clinical cohorts with few time points, and how the performance of such an
approach can be characterized. Specifically, combining deep learning
components and dynamic modeling by differential equations allowed for aligning
the representations of measurement instruments in a latent space, indicating
that such a combination of approaches could be more generally useful for
addressing challenging modeling scenarios in longitudinal data, such as from
rare disease registries. Overall, our results suggest that domain adaptation
techniques might have considerable potential for more generally integrating
information of different measurement instruments.

Acknowledgement This work was funded by the Deutsche Forschungsgemeinschaft
(DFG, German Research Foundation) – Project-ID 322977937 – GRK 2344 (MH) and
Project-ID 499552394 – SFB 1597 (HB, MB, and MH). Biogen and Novartis provide
financial support for the SMArtCARE registry. AP was supported by the Berta-
Ottenstein clinician scientist program of the University of Freiburg.

Conflict of Interest

The authors have declared no conflict of interest.

## Appendix

###  A.1 Implementation details

The following section provides details on the implementation of the proposed
domain adaptation approach.

The code to run all experiments and models is written in the Julia programming
language (Bezanson et al., 2017) of version 1.8.3 with the additional packages
CSV.jl (v0.10.11), DataFrames.jl (v1.6.1), Distances.jl (v0.10.10),
Distributions.jl (v0.25.102), Flux.jl (v0.13.17), LaTeXStrings.jl (v1.3.0),
Measures (v.0.3.2), MultivariateStats (v.0.10.2), Optim.jl (v1.7.8),
OrderedCollections.jl (v1.6.2), Parameters.jl (v0.12.3), Plots.jl (v1.23.5),
ProgressMeter.jl (v1.39.0), StatsBase.jl (v0.34.2), VegaLite.jl (vv3.2.3), and
Zygote.jl (v.0.6.59).

In the VAEs for each measurement instrument, the encoder and decoder each have
one hidden layer, where the number of hidden units is equal to the number of
input dimensions, i.e., the number of variables in each dataset, corresponding
to the number of items in each measurement instrument. In all hidden layers,
we use a $\tanh$-activation function. Each latent space is two-dimensional,
and the respective mean and variance are obtained as affine linear
transformations of the hidden layer values without a non-linear activation.
Each decoder parameterizes mean and variance of a Gaussian distribution,
corresponding to the conditional distribution of the input data given the
latent representation, calculated from the decoder hidden layer using an
affine linear transformation.

The additional network for mapping the baseline variables to individual-
specific ODE parameters has two hidden layers in addition to input and output
layer. In the first hidden layer, the number of hidden units is equal to the
number of baseline variables and a $\tanh$-activation is used. In the second
hidden layer, the number of hidden units is equal to the number of ODE
parameters and the activation function is a sigmoid function, shifted by
$-0.5$ and scaled by $0.5$. This corresponds to a prior for the range of the
estimated ODE parameters. Deviations from this range are still possible, as an
affine linear transformation with a diagonal matrix is added as the final
layer.

In the ELBO of each VAE, the KL-divergence between the prior and posterior is
scaled by a factor of $0.5$ to slightly reduce the regularizing effect of the
prior. The sum of the squared decoder parameter is added with a weighting
factor of $0.01$ to the loss function, corresponding to a standard penalty
term to prevent exploding decoder parameters.

In the experiments on synthetic modifications of data from one measurement
instrument presented in Section 3.3, we use a two-dimensional latent space and
a homogeneous linear ODE system, i.e., where
$\frac{d}{dt}{\boldsymbol{\mu}}(t)=A\cdot{\boldsymbol{\mu}}(t)\in\mathbb{R}^{2}$
and $c=0$. All additional loss function penalty terms are added with a weight
of $5$, i.e., in the notation of Equation (4), we have
$\alpha=\beta=\gamma=5$. In the application on data from two measurement
instruments in the SMArtCARE registry presented in Section 3.4, we also use a
two-dimensional latent space and an inhomogeneous linear ODE system, i.e.,
where
$\frac{d}{dt}{\boldsymbol{\mu}}(t)=A\cdot{\boldsymbol{\mu}}(t)+c\in\mathbb{R}^{2}$,
to account for the higher complexity of this setting. In the loss function, we
set $\gamma=5$ in all four types of model estimation. In the model with
neither ODE penalty nor adversarial penalty, we set $\alpha=\beta=0$. In the
model with only the ODE penalty, we set $\alpha=0,\beta=5$, while in the model
with only the adversarial penalty, we set $\alpha=5,\beta=0$. In the model
with both ODE and adversarial penalty, we set $\alpha=\beta=5$

All models are trained using on stochastic gradient descent with the ADAM
optimizer (Kingma & Ba, 2015). For the experiments on the dataset
modifications from Section 3.3, we use a learning rate of $0.001$ and $30$
training epochs for all modifications, chosen based on monitoring convergence
of the loss function and visualizing exemplary fits. For the results on two
measurement instruments in the SMArtCARE registry from Section 3.4, we use a
learning rate of $0.00003$ and $10$ training epochs for all four versions of
model estimation.

###  A.2 Dataset modifications

In the following, we provide details of the implementation of each
modification presented in Section 3.2. For the scenarios with a shift, we add
an offset of $2$ to all items. In the scenario where the shift is only applied
to a random subgroup of patients, we sample the subgroup with $p=0.5$. In the
scenario where observation time points of the second measurement instrument
are deleted randomly for all patients, we use a dropout probability of $0.5$.
For removing observation time points preferentially at later time points, for
a patient $i$, at each time point $k=1,\dots,T_{i}^{R}+1$ we sample from a
Bernoulli distribution with $p=\frac{k}{T_{i}^{R}+4}$ and remove the time
points where a $1$ was sampled, such that removal of later time points is more
likely. For removing observations in the second measurement instrument where
the sum score of all items of the first measurement exceeds a given threshold,
we use as cutoff the $0.6$ quantile of the sum scores of all patients and time
points, corresponding to a value of $25$, where $44$ is the maximum sum score.

## References

  * (1)
  * Bauer & Hussong (2009) Bauer, D. J. & Hussong, A. M. (2009), ‘Psychometric approaches for developing commensurate measures across independent studies: traditional and new models’, Psychological Methods 14(2), 101–125.
  * Becker et al. (2015) Becker, C., Christoudias, C. M. & Fua, P. (2015), ‘Domain Adaptation for Microscopy Imaging’, IEEE Transactions on Medical Imaging 34(5), 1125–1139.
  * Bezanson et al. (2017) Bezanson, J., Edelman, A., Karpinski, S. & Shah, V. B. (2017), ‘Julia: A fresh approach to numerical computing’, SIAM Review 59(1), 65–98.
  * Bland & Altman (1990) Bland, J. M. & Altman, D. G. (1990), ‘A note on the use of the intraclass correlation coefficient in the evaluation of agreement between two methods of measurement’, Computers in Biology and Medicine 20(5), 337–340.
  * Blei et al. (2017) Blei, D. M., Kucukelbir, A. & McAuliffe, J. D. (2017), ‘Variational inference: A review for statisticians’, Journal of the American Statistical Association 112(518), 859–877.
  * Chen et al. (2012) Chen, M., Xu, Z., Weinberger, K. Q. & Sha, F. (2012), ‘Marginalized Denoising Autoencoders for Domain Adaptation’, ArXiv .  arXiv preprint: https://arxiv.org/abs/1206.4683.
  * Crainiceanu et al. (2006) Crainiceanu, D. R., Stefanski, L. A. & Ciprian M., R. J. C. (2006), Measurement Error in Nonlinear Models: A Modern Perspective, Second Edition, 2 edn, Chapman and Hall/CRC, New York.
  * Csurka (2017) Csurka, G. (2017), ‘Domain Adaptation for Visual Applications: A Comprehensive Survey’.  arXiv preprint: https://arxiv.org/abs/1702.05374.
  * Farahani et al. (2021) Farahani, A., Voghoei, S., Rasheed, K. & Arabnia, H. R. (2021), A Brief Review of Domain Adaptation, in R. Stahlbock, G. M. Weiss, M. Abou-Nasr, C.-Y. Yang, H. R. Arabnia & L. Deligiannidis, eds, ‘Advances in Data Science and Information Engineering’, Transactions on Computational Science and Computational Intelligence, Springer International Publishing, Cham, pp. 877–894.
  * Ganin & Lempitsky (2015) Ganin, Y. & Lempitsky, V. (2015), Unsupervised Domain Adaptation by Backpropagation, in ‘Proceedings of the 32nd International Conference on Machine Learning’, PMLR, pp. 1180–1189.
  * Ghafoorian et al. (2017) Ghafoorian, M., Mehrtash, A., Kapur, T., Karssemeijer, N., Marchiori, E., Pesteie, M., Guttmann, C. R. G., de Leeuw, F.-E., Tempany, C. M., van Ginneken, B., Fedorov, A., Abolmaesumi, P., Platel, B. & Wells, W. M. (2017), Transfer Learning for Domain Adaptation in MRI: Application in Brain Lesion Segmentation, in M. Descoteaux, L. Maier-Hein, A. Franz, P. Jannin, D. L. Collins & S. Duchesne, eds, ‘Medical Image Computing and Computer Assisted Intervention - MICCAI 2017’, Lecture Notes in Computer Science, Springer International Publishing, Cham, pp. 516–524.
  * Glanzman et al. (2011) Glanzman, A. M., O’Hagen, J. M., McDermott, M. P., Martens, W. B., Flickinger, J., Riley, S., Quigley, J., Montes, J., Dunaway, S., Deng, L., Chung, W. K., Tawil, R., Darras, B. T., De Vivo, D. C., Kaufmann, P., Finkel, R. S., Pediatric Neuromuscular Clinical Research Network for Spinal Muscular Atrophy (PNCR) & Muscle Study Group (MSG) (2011), ‘Validation of the Expanded Hammersmith Functional Motor Scale in spinal muscular atrophy type II and III’, Journal of Child Neurology 26(12), 1499–1507.
  * Goetz et al. (2016) Goetz, M., Weber, C., Binczyk, F., Polanska, J., Tarnawski, R., Bobek-Billewicz, B., Koethe, U., Kleesiek, J., Stieltjes, B. & Maier-Hein, K. H. (2016), ‘DALSA: Domain Adaptation for Supervised Learning From Sparsely Annotated MR Images’, IEEE Transactions on Medical Imaging 35(1), 184–196.
  * Goodfellow et al. (2014) Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A. & Bengio, Y. (2014), Generative adversarial nets, in ‘Advances in neural information processing systems’, Vol. 27.
  * Guan & Liu (2022) Guan, H. & Liu, M. (2022), ‘Domain adaptation for medical image analysis: A survey’, IEEE Transactions on Biomedical Engineering 69(3), 1173–1185.
  * Hackenberg et al. (2022) Hackenberg, M., Harms, P., Pfaffenlehner, M., Pechmann, A., Kirschner, J., Schmidt, T. & Binder, H. (2022), ‘Deep dynamic modeling with just two time points: Can we still allow for individual trajectories?’, Biometrical Journal 64, 1426–1445.
  * Hackenberg et al. (2023) Hackenberg, M., Pechmann, A., Kreutz, C., Kirschner, J. & Binder, H. (2023), ‘A statistical approach to latent dynamic modeling with differential equations’.  arXiv preprint: https://arxiv.org/abs/2311.16286.
  * He et al. (2023) He, H., Queen, O., Koker, T., Cuevas, C., Tsiligkaridis, T. & Zitnik, M. (2023), ‘Domain Adaptation for Time Series Under Feature and Label Shifts’.  arXiv preprint: http://arxiv.org/abs/2302.03133.
  * Jin et al. (2022) Jin, X., Park, Y., Maddix, D. C., Wang, H. & Wang, Y. (2022), ‘Domain Adaptation for Time Series Forecasting via Attention Sharing’.  arXiv preprint: http://arxiv.org/abs/2102.06828.
  * Jude et al. (2022) Jude, J., Perich, M. G., Miller, L. E. & Hennig, M. H. (2022), ‘Robust alignment of cross-session recordings of neural population activity by behaviour via unsupervised domain adaptation’.  arXiv preprint: http://arxiv.org/abs/2202.06159.
  * Karani et al. (2018) Karani, N., Chaitanya, K., Baumgartner, C. & Konukoglu, E. (2018), A Lifelong Learning Approach to Brain MR Segmentation Across Scanners and Protocols, in A. F. Frangi, J. A. Schnabel, C. Davatzikos, C. Alberola-López & G. Fichtinger, eds, ‘Medical Image Computing and Computer Assisted Intervention – MICCAI 2018’, Lecture Notes in Computer Science, Springer International Publishing, Cham, pp. 476–484.
  * Kingma & Ba (2015) Kingma, D. P. & Ba, J. (2015), Adam: A method for stochastic optimization, in Y. Bengio & Y. LeCun, eds, ‘3rd International Conference on Learning Representations (ICLR), Conference Track Proceedings’.
  * Kingma & Welling (2014) Kingma, D. P. & Welling, M. (2014), Auto-encoding variational Bayes, in Y. Bengio & Y. LeCun, eds, ‘2nd International Conference on Learning Representations (ICLR), Conference Track Proceedings’.
  * Kingma & Welling (2019) Kingma, D. P. & Welling, M. (2019), ‘An introduction to variational autoencoders’, Foundations and Trends® in Machine Learning 12(4), 307–392.
  * Lai et al. (2023) Lai, K.-H., Wang, L., Chen, H., Zhou, K., Wang, F., Yang, H. & Hu, X. (2023), ‘Context-aware Domain Adaptation for Time Series Anomaly Detection’.  arXiv preprint: http://arxiv.org/abs/2304.07453.
  * Lefebvre et al. (1995) Lefebvre, S., Bürglen, L., Reboullet, S., Clermont, O., Burlet, P., Viollet, L., Benichou, B., Cruaud, C., Millasseau, P. & Zeviani, M. (1995), ‘Identification and characterization of a spinal muscular atrophy-determining gene’, Cell 80(1), 155–165.
  * Long et al. (2015) Long, M., Cao, Y., Wang, J. & Jordan, M. I. (2015), ‘Learning Transferable Features with Deep Adaptation Networks’, ArXiv .  arXiv preprint: https://arxiv.org/abs/1502.02791.
  * Mazzone et al. (2017) Mazzone, E. S., Mayhew, A., Montes, J., Ramsey, D., Fanelli, L., Young, S. D., Salazar, R., Sanctis, R. D., Pasternak, A., Glanzman, A., Coratti, G., Civitello, M., Forcina, N., Gee, R., Duong, T., Pane, M., Scoto, M., Pera, M. C., Messina, S., Tennekoon, G., Day, J. W., Darras, B. T., Vivo, D. C. D., Finkel, R., Muntoni, F. & Mercuri, E. (2017), ‘Revised upper limb module for spinal muscular atrophy: Development of a new module’, Muscle Nerve 55(6), 869–874.
  * Messina et al. (2022) Messina, S., Sframeli, M., Maggi, L., D’Amico, A., Bruno, C., Comi, G. & Mercuri, E. (2022), ‘Spinal muscular atrophy: state of the art and new therapeutic strategies’, Neurological Sciences: Official Journal of the Italian Neurological Society and of the Italian Society of Clinical Neurophysiology 43(Suppl 2), 615–624.
  * O’Hagen et al. (2007) O’Hagen, J. M., Glanzman, A. M., McDermott, M. P., Ryan, P. A., Flickinger, J., Quigley, J., Riley, S., Sanborn, E., Irvine, C., Martens, W. B., Annis, C., Tawil, R., Oskoui, M., Darras, B. T., Finkel, R. S. & De Vivo, D. C. (2007), ‘An expanded version of the Hammersmith Functional Motor Scale for SMA II and III patients’, Neuromuscular disorders: NMD 17(9-10), 693–697.
  * Pechmann et al. (2019) Pechmann, A., König, K., Bernert, G., Schachtrup, K., Schara, U., Schorling, D., Schwersenz, I., Stein, S., Tassoni, A., Vogt, S., Walter, M. C., Lochmüller, H. & Kirschner, J. (2019), ‘Smartcare - a platform to collect real-life outcome data of patients with spinal muscular atrophy’, Orphanet Journal of Rare Diseases 14(18).
  * Proust-Lima et al. (2023) Proust-Lima, C., Saulnier, T., Philipps, V., Traon, A. P.-L., Péran, P., Rascol, O., Meissner, W. G. & Foubert-Samier, A. (2023), ‘Describing complex disease progression using joint latent class models for multivariate longitudinal markers and clinical endpoints’, Statistics in Medicine 42(22), 3996–4014.
  * Proust-Lima et al. (2014) Proust-Lima, C., Séne, M., Taylor, J. M. & Jacqmin-Gadda, H. (2014), ‘Joint latent class models for longitudinal and time-to-event data: A review’, Statistical methods in medical research 23(1), 74–90.
  * Purushotham et al. (2016) Purushotham, S., Carvalho, W., Nilanon, T. & Liu, Y. (2016), Variational recurrent adversarial deep domain adaptation, in ‘International Conference on Learning Representations’.
  * Raghu et al. (2023) Raghu, A., Chandak, P., Alam, R., Guttag, J. & Stultz, C. M. (2023), ‘Sequential Multi-Dimensional Self-Supervised Learning for Clinical Time Series’.  arXiv preprint: http://arxiv.org/abs/2307.10923.
  * Schorling et al. (2020) Schorling, D. C., Pechmann, A. & Kirschner, J. (2020), ‘Advances in treatment of spinal muscular atrophy - new phenotypes, new challenges, new implications for care’, Journal of neuromuscular diseases 7(1), 1–13.
  * Shahar (2017) Shahar, D. J. (2017), ‘Minimizing the variance of a weighted average’, Open Journal of Statistics 7(2), 216–224.
  * Tzeng et al. (2017) Tzeng, E., Hoffman, J., Saenko, K. & Darrell, T. (2017), Adversarial Discriminative Domain Adaptation, in ‘2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)’, IEEE, Honolulu, HI, pp. 2962–2971.
  * Van Buuren et al. (2005) Van Buuren, S., Eyres, S., Tennant, A. & Hopman-Rock, M. (2005), ‘Improving comparability of existing data by Response Conversion.’, Journal of Official Statistics 21, 53–72.
  * van den Heuvel et al. (2020) van den Heuvel, E. R., Griffith, L. E., Sohel, N., Fortier, I., Muniz-Terrera, G. & Raina, P. (2020), ‘Latent variable models for harmonization of test scores: A case study on memory’, Biometrical Journal 62(1), 34–52.