# Online Domain Adaptation for Continuous Cross-Subject Liver Viability
Evaluation Based on Irregular Thermal Data

Sahand Hajifara and Hongyue Sunb CONTACT Hongyue Sun. Email:
<EMAIL_ADDRESS>a,bDepartment of Industrial and Systems Engineering,
University at Buffalo, Buffalo, NY 14260, USA

###### Abstract

Accurate evaluation of liver viability during its procurement is a challenging
issue and has traditionally been addressed by taking invasive biopsy on liver.
Recently, people have started to investigate on the non-invasive evaluation of
liver viability during its procurement using the liver surface thermal images.
However, existing works include the background noise in the thermal images and
do not consider the cross-subject heterogeneity of livers, thus the viability
evaluation accuracy can be affected. In this paper, we propose to use the
irregular thermal data of the pure liver region, and the cross-subject liver
evaluation information (i.e., the available viability label information in
cross-subject livers), for the real-time evaluation of a new liver’s
viability. To achieve this objective, we extract features of irregular thermal
data based on tools from graph signal processing (GSP), and propose an online
domain adaptation (DA) and classification framework using the GSP features of
cross-subject livers. A multiconvex block coordinate descent based algorithm
is designed to jointly learn the domain-invariant features during online DA
and learn the classifier. Our proposed framework is applied to the liver
procurement data, and classifies the liver viability accurately.

###### keywords:

Irregular Data; Graph Signal Processing; Liver Viability; Online Domain
Adaptation; Real-time Classification

##  1 Introduction

According to the Organ Procurement and Transplantation Network, there are over
120,000 patients needing an organ transplant as of Sep. 3rd, 2020 (HRSA,
2020). Every 9 minutes, someone is added to the national transplant waiting
list, and 95 transplants take place each day in the US on average (HRSA,
2020). One challenge for the organ transplantation is that the maximum
viability of organs under ideal procurement conditions is limited, only 4-6
hours for lungs and hearts and 8-12 hours for livers and pancreas. The actual
viability of an organ is dependent on many factors such as donor health and
procurement conditions. It is very important to accurately evaluate an organ’s
viability before its transplantation.

In this paper, we focus on the liver disease. According to the National Center
for Health Statistics, there are 4.5 million adults being diagnosed with liver
disease in 2018 (CDC, 2020). An early diagnosis and treatment of liver disease
can be challenging (Mueller et al., 2014; Badebarin et al., 2017), and many
patients eventually suffer from end-stage liver disease. For end-stage liver
disease, which accounts for 41,743 deaths in the US in 2017, liver
transplantation has proven to be the most effective method to extend patients’
life (CDC, 2020; Lan et al., 2018). However, after a liver being harvested
from a donor, it needs to wait for a matching recipient and be transported to
the recipient. During this period, the liver is usually kept in a cold box
with circulating perfusion fluid for hours to reduce the decrease of
functionality (Cameron and Cornejo, 2015; Lan et al., 2018). Before
transplantation, pathologists need to evaluate the liver viability and make
sure it is safe to transplant the liver.

Currently, there are mainly two approaches being used to evaluate the liver
viability: visual inspection and biopsy (Keeffe, 2001; Lan et al., 2015). In
visual inspection, the pathologists look at the surface of a liver and judge
its viability. The visual inspection may suffer from the pathologists’
inadequate experiences and is subject to the lack of consistent standards
(Petrick et al., 2015). In addition, the appearances of a viable and an
unviable organ may not be distinguishable (Gao et al., 2020). On the other
hand, the biopsy examination and histologic analysis technique was initially
used in organ transplantation in the 1980s, and has proven to be a more
accurate evaluation technique (Vazquez-Martul and Papadimitriou, 2004).
However, the biopsy examination technique suffers from two disadvantages: (1)
the judgment on biopsy results depends on the pathologists’ expertise and is
prone to the subjective errors; and (2) taking biopsy is an invasive procedure
and causes very small but irremediable damage to the organ (Rothuizen and
Twedt, 2009; Lan et al., 2015). It was reported that the lack of accurate non-
invasive assessment methods wastes 20% of donor organs every year (Lan, 2020).

In recent years, infrared thermography (IRT) has gained considerable attention
in medical applications as a fast, passive, non-contact and non-invasive
measuring technique (Lahiri et al., 2012). For instance, a research team from
Virginia Tech designed a machine perfusion and corresponding non-invasive
thermal imaging system to measure the surface temperature of livers, as shown
in Figure 1 (a). Several analyses were performed for the liver viability
assessment, and revealed a close relationship between the organ surface
temperatures and viability based on this system (Lan et al., 2015, 2018; Gao
et al., 2018, 2020). As will be discussed in detail in the literature review,
existing works on thermal imaging based organ viability evaluation have the
limitations of: (1) using the full thermal images and thus including the
background data in addition to the pure liver region (shown in Figure 1(b)) in
the analysis. The background data may vary over time, and are irrelevant to
the liver viability evaluation; (2) only considering the data in one liver or
directly combining data from multiple livers in the analysis without
considering the cross-subject liver heterogeneity. The variations in donors
and perfusion conditions bring in the heterogeneous thermal images among
livers, which need to be explicitly considered in the analysis; (3) cannot be
used for the online cross-subject liver viability evaluation. The thermal
images are collected continuously over time, and a model needs to be updated
to accommodate for the latest information for the liver being evaluated, which
will allow for the accurate online viability evaluation.

Figure 1: (a) Perfusion and non-invasive thermal imaging system (Lan, 2020).
Note that biopsy extraction was not a part of this work and was used for the
purpose of viability evaluation; (b) Thermal images (on the left) of a viable
liver versus an unviable liver, and the corresponding irregular pure liver
regions (on the right)

The objective of this paper is to design an online non-invasive liver
viability evaluation framework based on the pure liver region thermal data and
cross-subject evaluation information. This framework leverages the thermal
data and biopsy evaluation information from heterogeneous livers, to predict
the viability of a new liver based on its thermal images in real time without
the need of taking biopsies. To achieve this objective, we face several
challenges: (1) The pure liver region is irregular (irregular data format),
where traditional feature and image processing methods are not directly
applicable; (2) The heterogeneity of livers, because of different donors and
machine perfusion conditions, prevent us from directly training a model for
cross-subject liver viability evaluation; and (3) The continuously collected
thermal images contain rich information on the liver condition, and one needs
to update the model with the streaming images while still achieve the online
viability evaluation. To address these challenges, we propose a novel online
domain adaptation (DA) and classification framework based on the graph signal
processing (GSP) features of irregular pure liver thermal data. The
contributions of the proposed framework are:

  * •

We propose to model the irregular thermal data of the pure liver region by
graph signal processing for the first time, and use graph Fourier transform
(GFT) to extract the graph features from the irregular data.

  * •

We propose a novel online DA and classification framework based on the graph
features for the online cross-subject liver viability evaluation.

  * •

We develop a novel multiconvex block coordinate descent based algorithm to
jointly learn the domain-invariant features and the classifier.

The reminder of the paper is organized as follows. In Section 2, we review the
relevant literature on organ viability evaluation based on thermal images, on
GSP and on DA. Our proposed method is introduced in Section 3, and being
demonstrated in Section 4. Finally, we conclude the paper and discuss about
the future work in Section 5.

##  2 Literature Review

###  2.1 Organ Viability Evaluation based on Thermal Images

IRT has been applied to a variety of applications including, but not limited
to, kidney transplantation, brain imaging, breast cancer and liver disease
(Lahiri et al., 2012). Researchers have revealed a close association between
the histomorphologic quality and surface temperatures of the organ (Gorbach et
al., 2009; Skowno and Karpelowsky, 2014; Vidal et al., 2014; Lan et al., 2015;
Kochan et al., 2015; Lan et al., 2018; Gao et al., 2018, 2020).

Based on the collected data, different statistical methods can be utilized for
organ viability evaluation. Cox regression models were applied to determine
the main factors that would result into liver failure in the post-transplant
period (Feng et al., 2006). A hierarchical regression method was utilized to
estimate donor risk index based on patient and transplant center
characteristics (Volk et al., 2011). Lan et al. (2015) applied quantitative
and qualitative models to predict the number of dead cells in a liver and
viability of the liver. Lan et al. (2018) extracted the principal components
from the thermal images, and performed intra liver classification with
logistic regression and inter liver classification with multitask learning.
The classification can distinguish the viable versus unviable images within a
liver accurately but did not generalize well to multiple livers since it did
not consider the cross-subject heterogeneity among livers. The above works did
not consider the updated liver information over time thus cannot achieve
online viability evaluation. Gao et al. (2018, 2020) performed the change
detection of surface temperature variation for a single liver in an online
manner, and found that the temperature measurements show a high variation when
the liver is viable and a low variation if the liver is unviable. However,
these works only detected the change for a single liver and did not borrow the
valuable pathologists’ evaluation information from other livers, which can be
beneficial to the viability evaluation.

There is a lack of online organ viability evaluation method to explicitly
consider the heterogeneous livers’ thermal images and evaluation information,
and to address the irregular thermal data of the pure liver region.

###  2.2 Graph Signal Processing

GSP is a quickly developing subfield of signal processing that generalizes
classical techniques of signal processing to irregular graph domain by
considering the graph structure of multivariate signals (Ménoret et al.,
2017). According to Ortega et al. (2018), GSP has seen major applications in
sensor networks, image and 3-D point cloud processing, and biological
networks.

We review the related research on biological networks, where many works
modeled the data as a network (i.e., graph), such as the human brain (Ortega
et al., 2018; Tran et al., 2019). Unlike in classical signal processing
(Sartipi et al., 2020; Kang et al., 2020), GSP aims at learning and leveraging
the underlying graph structure of the brain (Ortega et al., 2018). In
particular, the human brain activity signals can be modeled as a graph
structure, where nodes and edge weights represent brain regions and structural
connectivity between brain regions, respectively (Sporns, 2010; Bullmore and
Sporns, 2012). Hu et al. (2016a) modeled the progression of brain atrophy
through a diffusion model over the brain connectivity graph. Hu et al. (2016b)
proposed a matched signal detection theory to be applied to the neuroimaging
data classification problem of Alzheimer’s disease. Pirayre et al. (2015,
2017) used GSP to infer gene interactions, which play a vital role in the
recognition of novel regulatory processes in cells. The application of GSP in
other biological networks is not explored yet and still remains a research
gap.

In this paper, we will use the tools in GSP for the feature extraction of the
irregular thermal data of pure liver region.

###  2.3 Domain Adaptation

DA, a subfield of transfer learning, can transfer the information learnt from
source (training) domain to a new target (test) domain such that the costly
endeavour in data recollection and model rebuilding can be removed (Huang et
al., 2012; Hoffman et al., 2014; Khosla et al., 2012; Torralba and Efros,
2011; Hoffman et al., 2014; Cheng et al., 2020; Kontar et al., 2020). In
general, DA can be classified into semi-supervised and unsupervised DA (Ding
et al., 2018). We focused on the unsupervised DA where the target data do not
have labels, which is the case in our problem setting (i.e., no biopsy labels
are available for the target data of a new liver).

There are generally three categories of methods in unsupervised DA (Sun et
al., 2019): (1) Minimizing some distributional discrepancy measure to yield
the matching between the source and target features in a common feature space.
For instance, Long et al. (2015) minimized the maximum mean discrepancy (MMD)
and Long et al. (2017) minimized the joint MMD between source and target
domains. (2) Learning generative models to transform from the source data to
the target data (Taigman et al., 2016; Hoffman et al., 2018). For instance, a
direct transformation was conducted on the image pixels rather than generating
a common feature space (Sun et al., 2019). (3) Training a model on the labeled
source data to estimate the pseudo-labels on the target data. Afterwards,
another training was conducted on the most confident pseudo-labels of target
data (Sun et al., 2019). This method was given different names such as self-
ensembling (French et al., 2017) and co-training (Saito et al., 2017).

The above methods cannot achieve online DA. Online DA has gained increasing
attention in recent years, where the target data is provided in a streaming
manner (Hoffman et al., 2014; Bitarafan et al., 2016). Hoffman et al. (2014)
posed the question of “what happens when test data not only differs from
training data, but differs from it in a continually evolving way?”. The
authors proposed continuous domain adaptation to address the scenario of
evolving domains in a classification setting. However, their method first
learned domain-invariant features and then trained classifiers in two steps,
which may yield sub-optimal classification accuracy.

In this paper, we propose an online DA and classification framework that can
jointly learn the domain-invariant features and the classifier.

##  3 Proposed Framework

An illustration of the proposed framework is shown in Figure 2, which includes
(1) graph generation and feature extraction from irregular liver surface
thermal data, and (2) joint online domain adaptation and classification.
Firstly, we remove the background and extract the pure liver region from the
raw liver thermal images, and then downsample the images to reduce the
dimension. Thus, the later analysis is free from the background noise but is
challenged by the irregular pure liver region. To address this challenge, we
construct the graph for GSP of the irregular thermal data, and then extract
the features from the irregular data using GFT. Secondly, we propose a novel
method to jointly perform the online DA and classification for the cross-
subject liver viability classification with continuous model update. In the
following, we will first introduce the GSP for irregular data analysis in
Section 3.1, then describe our graph construction in Section 3.2, and finally
introduce the proposed online DA and classification and its joint optimization
algorithm in Sections 3.3 and 3.4, respectively.

Figure 2: An illustration of the proposed framework

###  3.1 Graph Signal Processing

Throughout this paper, let $\mathcal{G}$ be a weighted graph containing a set
$\mathcal{V}$ of $M$ vertices
($\mathcal{V}=\\{\mathpzc{v}_{1},\ldots,\mathpzc{v}_{M}\\}$). In our
application, the vertices correspond to pixels of interest in the liver
thermal data (i.e., the pixels after background removal and image
compression). Let $\bm{A}$ be the symmetric adjacency matrix of the graph,
where $A_{ij}\in\mathbb{R}^{+}$ represents the weight between vertices
$\mathpzc{v}_{i}$ and $\mathpzc{v}_{j}$. The Laplacian matrix of graph
$\mathcal{G}$ is denoted by $\bm{L}$ and calculated by $\bm{L}=\bm{D}-\bm{A}$,
where $\bm{D}$ denotes the diagonal matrix of degrees obtained by $\forall
i:D_{ii}=\sum_{j}A_{ij}$. $\bm{L}$ is symmetric and real-valued and can be
factorized as $\bm{L}=\bm{F}\bm{\Lambda}\bm{F}^{T}$, such that $\bm{F}$ is an
orthonormal matrix, and $\bm{\Lambda}$ is a diagonal matrix, whose diagonal
consists of the eigenvalues of $\bm{L}$ in ascending order.

This factorization of the Laplacian matrix serves as the building block of
GFT, which extracts a spectral representation of signals situated on a graph
(Ménoret et al., 2017). In particular, let $\bm{s}_{t}\in\mathbb{R}^{M}$ be a
vector, which denotes a realization of signal at time $t$ over the $M$
vertices of graph $\mathcal{G}$. $\bm{s}_{t}$ is a column of $\bm{S}$ (i.e.,
$\bm{S}_{.,t}=\bm{s}_{t}$), which is the matrix that keeps all measurements
collected throughout the experiment time, where $S_{\mathpzc{v}_{m},t}$
denotes the temperature of vertex $\mathpzc{v}_{m}$ at time $t$.

The GFT of the signal is calculated by
$\hat{\bm{s}}_{t}=\bm{F}^{T}\bm{s}_{t}$. The first elements of
$\hat{\bm{s}}_{t}$, related to the lower eigenvalues in $\bm{\Lambda}$, are
named low frequencies (LF) and its last elements are named high frequencies
(HF). As shown in Figure 2, the HF elements are close to zero, and we consider
the first $D$ LF elements to be the initial features input to DA. Actually,
these LF elements preserve the main patterns in the temperature data, as shown
in the case study. Let $\hat{\bm{s}}_{t}^{\prime}$ be a vector with the first
$D$ elements being the same as $\hat{\bm{s}}_{t}$, and zero elsewhere. We use
$\hat{\bm{s}}_{t,1:D}^{\prime}$ as inputs to online DA. For simplicity, we
denote $\bm{f}_{t}=\hat{\bm{s}}_{t,1:D}^{\prime}$ hereafter.

###  3.2 Graph Construction

One needs to construct the graph $\mathcal{G}$ for GFT. To preserve the main
graph structure while allow the online modeling, we construct our graph as
follows. Notice that GFT requires an eigendecomposition to be computed, which
can be time consuming for a large matrix. We therefore construct the graph
from an averaged signal $\tilde{S}_{i}=\bar{\bm{S}}_{i,1:\tilde{t}}$ of the
beginning of the data collection, so that the time-consuming eigendecompostion
needs to be computed only once. Then, the weight $\bm{A}$ is defined as
follows:

$A_{i,j}=\begin{cases}\text{exp}\Big{(}-\frac{|\tilde{S}_{i}-\tilde{S}_{j}|^{2}}{\sigma}\Big{)}&i\neq
j\\\ 0&i=j\end{cases}$ (1)

where $\sigma$ is a scale parameter. This graph can provide a good
reconstruction of the irregular thermal data by using the first elements of
the transformed signals $\hat{\bm{s}}_{t}$, see details in the case study.

###  3.3 Online Domain Adaptation and Classification

Let $\bm{X}\in\mathbb{R}^{N\times D}$ be the feature matrix for the graph
signal $S$ of the irregular pure liver thermal data, where the $t$-th row
$\bm{X}_{t,.}=\bm{f}_{t}^{T}$. $N$ is the number of samples (i.e., thermal
images for a liver), and $D$ is the number of features (i.e., low frequency
variables in GFT). Let $\bm{Y}$ be the class labels (viable or unviable) for
thermal images. Adopting the notations from domain adaptation, we denote
$\bm{X}_{S}$ and $\bm{X}_{T}$ as the source and target features, and
$\bm{Y}_{S}$ and $\bm{Y}_{T}$ as the source and target labels. Our objective
here is to: (1) train a classifier for the continuous target domain liver
viability classification based on the labels in the source domain, and (2)
find a subspace where the transformed features in source and target domains
align well.

To achieve the objective, we propose a novel online DA and classification
framework. At time $t$, we minimize the following objective function

$min\>L(\phi_{t}(\bm{X}_{S}),\bm{Y}_{S})+\lambda_{1}D(t)+\lambda_{2}P(t),$ (2)

where the first term $L(\phi_{t}(\bm{X}_{S}),\bm{Y}_{S})$ is the
classification loss at time $t$, $\phi(\cdot)$ is the kernel function from
Reproducing Kernel Hilbert Space (RKHS) for DA. The second term
$D(t)=MMD(\bm{X}_{S},\bm{X}_{T_{t}})=\left\lVert\frac{1}{N_{S}}\sum_{i=1}^{N_{S}}\phi(\bm{X}_{S,i})-\frac{1}{N_{T_{t}}}\sum_{i=1}^{N_{T_{t}}}\phi(\bm{X}_{T_{t},i})\right\rVert$
measures the alignment of the source domain and target domain up to time
$T_{t}$ in the subspace $\phi(\cdot)$ (Pan et al., 2010; Long et al., 2017).
Here, we will use the widely adopted Gaussian kernel for $\phi(\cdot)$. After
finding the mapping to minimize MMD, the features of source and target domains
are expected to be close to each other. Finally, the third term $P(t)$ is a
penalty to enforce the function $\phi(\cdot)$ at two close-by time to be
similar during the online DA.
$P(t)=\left\lVert\bm{W}_{S_{t}}-\bm{W}_{S_{t-1}}\right\rVert_{F}^{2}=\left\lVert\bm{H}\bm{W}_{t}-\bm{H}\bm{W}_{t-1}\right\rVert_{F}^{2}$,
where $\bm{W}_{t}=\begin{bmatrix}\bm{W}_{S_{t}}\\\
\bm{W}_{T_{t}}\end{bmatrix}$ is the mapping matrix in the kernel to be defined
in Section 3.4, and $\bm{H}_{t}=\begin{bmatrix}\bm{I}_{S\times
S}&\bm{0}_{S\times t}\end{bmatrix}$. In the following, we will elaborate
details of each term in Equation 2.

Firstly, to consider the potential nonlinear relationship between the features
and label, we use support vector machine (SVM) as the classifier. The SVM
classification loss is,

$\begin{gathered}min\>L(\phi(\bm{X}_{S}),\bm{Y}_{S})=\>\frac{1}{N_{S}}\sum_{i=1}^{N_{S}}hinge(\phi(\bm{X}_{S,i}),\bm{Y}_{S,i}|\bm{\beta}_{t},\beta_{0,t})+\frac{\lambda}{2}\left\lVert\bm{\beta}_{t}\right\rVert^{2}\\\
=\>\frac{1}{N_{S}}\sum_{i=1}^{N_{S}}\Big{(}1-Y_{S,i}(\langle\bm{\beta}_{t},\phi(\bm{X}_{S,i})\rangle+\beta_{0,t})\Big{)}_{+}+\frac{\lambda}{2}\left\lVert\bm{\beta}_{t}\right\rVert^{2}\end{gathered}$

where $\beta_{0,t}$ and $\bm{\beta}_{t}$ are model coefficients. According to
the representer theorem (Schölkopf et al., 2001), $\bm{\beta}_{t}$ can be
written as
$\bm{\beta}_{t}=\alpha_{i,t}\sum_{i=1}^{N_{S}}Y_{S,i}\phi(\bm{X}_{S,i})$.
Substituting $\bm{\beta}_{t}$ into the above equation, and letting
$K(a,b)=\phi(a)^{T}\phi(b)$ and $\bm{Z}_{S}=diag(Y_{S,1},\ldots,Y_{S,N_{S}})$,
we have

$\begin{gathered}min\>L(\phi(\bm{X}_{S}),\bm{Y}_{S})=\>\frac{1}{N_{S}}\sum_{i=1}^{N_{S}}\Big{(}1-Y_{S,i}(\bm{\alpha}_{t}^{T}\bm{Z}_{S}K(\bm{X}_{S,i},\cdot)+\beta_{0,t})\Big{)}_{+}+\frac{\lambda}{2}\bm{\alpha}_{t}^{T}\bm{Z}_{S}K(\cdot,\cdot)\bm{Z}_{S}\bm{\alpha}_{t}\\\
\>\approx\frac{1}{N_{S}}\sum_{i=1}^{N_{S}}\psi\Big{(}Y_{S,i}(\bm{\alpha}_{t}^{T}\bm{Z}_{S}K(\bm{X}_{S,i},\cdot)+\beta_{0,t})\Big{)}+\frac{\lambda}{2}\bm{\alpha}_{t}^{T}\bm{Z}_{S}K(\cdot,\cdot)\bm{Z}_{S}\bm{\alpha}_{t}\end{gathered}$

where $\bm{\alpha}_{t}=(\alpha_{1,t},\ldots,\alpha_{N_{s},t})^{T}$, and the
$\cdot$ in $K(\bm{X}_{S,i},\cdot)$ and $K(\cdot,\cdot)$ refers to all the
possible elements. Due to the nested structure of the DA transformation and
SVM parameters, the above problem is bi-convex.

We propose to develop a block-coordinate proximal gradient method based
algorithm to solve the non-convex problem, which has guaranteed global
convergence and high computational speed (Xu and Yin, 2013). However, the
block-coordinate proximal gradient method requires the objective to be
differentiable. We therefore use the huberized SVM, a differential
approximation of the SVM formulation (Wang et al., 2008), as shown in the
second line of the above equation. Here, $\psi(a)=\begin{cases}0&\text{for
$a>1$}\\\ \frac{(1-a)^{2}}{2\delta}&\text{for $1-\delta<a\leq 1$}\\\
1-a-\frac{\delta}{2}&\text{for $a\leq 1-\delta$}\end{cases}$ is derivable, and
$\delta$ is a pre-specified constant.

In summary, we have the objective function at time $t$,

$\begin{gathered}min\>\frac{1}{N_{S}}\sum_{i=1}^{N_{S}}\psi\Big{(}Y_{S,i}(\bm{\alpha}_{t}^{T}\bm{Z}_{S}K(\bm{X}_{S,i},\cdot)+\beta_{0,t})\Big{)}+\frac{\lambda}{2}\bm{\alpha}_{t}^{T}\bm{Z}_{S}K(\cdot,\cdot)\bm{Z}_{S}\bm{\alpha}_{t}\\\
+\lambda_{1}\left\lVert\frac{1}{N_{S}}\sum_{i=1}^{N_{S}}\phi(\bm{X}_{S,i})-\frac{1}{N_{T_{t}}}\sum_{i=1}^{N_{T_{t}}}\phi(\bm{X}_{T_{t},i})\right\rVert+\lambda_{2}\left\lVert\bm{H}\bm{W}_{t}-\bm{H}\bm{W}_{t-1}\right\rVert_{F}^{2}.\end{gathered}$
(3)

Next, we describe the proposed joint optimization to solve the online DA and
classification in Equation 3.

###  3.4 Joint Optimization

Our proposed joint optimization algorithm to solve Equation 3 at time $t$ is
shown in Algorithm 1. The algorithm belongs to block-coordinate proximal
descent algorithm, and will iteratively solve for
$(\bm{\alpha}_{t},\beta_{0,t})$ and $\bm{W}_{t}$ in each iteration $k$, given
the source and target inputs, source labels and initial parameters. In this
section, we will introduce the details of this algorithm.

Input : $\bm{X}_{S},\bm{Y}_{S},\bm{X}_{T_{t}},\lambda,\lambda_{1},\lambda_{2}$

Initialize : $\bm{u}^{-1}_{t},\bm{u}^{0}_{t}$

for  _$k=1,2,\ldots$_ do

Update $\bm{u}_{t}^{k}$ based on Eq. 5

Update $\bm{W}_{t}^{k}$ based on Eq. 7

if  _stopping criterion is satisfied_ then

Return($\hat{\bm{u}_{t}},\hat{\bm{W}}_{t}$) ;

Algorithm 1 Joint Optimization Algorithm for Online DA and Classification

Firstly, we solve for $(\bm{\alpha}_{t}^{k},\beta_{0,t}^{k})$ in Equation 3
while fixing $\bm{W}_{t}^{k-1}$. In Equation 3, only the first two terms are
related to $(\bm{\alpha}_{t},\beta_{0,t})$, thus to solve for
$(\bm{\alpha}_{t}^{k},\beta_{0,t}^{k})$, we have

$\begin{gathered}min_{(\bm{\alpha}_{t},\beta_{0,t})}\>\frac{1}{N_{S}}\sum_{i=1}^{N_{S}}\psi\Big{(}Y_{S,i}(\bm{\alpha}_{t}^{T}\bm{Z}_{S}K(\bm{X}_{S,i},\cdot)+\beta_{0,t})\Big{)}+\frac{\lambda}{2}\bm{\alpha}_{t}^{T}\bm{Z}_{S}K(\cdot,\cdot)\bm{Z}_{S}\bm{\alpha}_{t}\\\
=min_{(\bm{\alpha}_{t},\beta_{0,t})}\>\frac{1}{N_{S}}\bm{1}^{T}\bm{\psi}(\bm{Z}_{S}\bm{VV}^{T}\bm{Z_{S}\alpha}_{t}+\bm{Z}_{S}\bm{1}\beta_{0,t})+\frac{\lambda}{2}\bm{\alpha}_{t}^{T}\bm{Z}_{S}\bm{VV}^{T}\bm{Z_{S}\alpha}_{t}\\\
=min_{\bm{u}_{t}}\>\frac{1}{N_{S}}\bm{1}^{T}\bm{\psi}(\tilde{\bm{X}}_{S}\bm{u}_{t})+\frac{\lambda}{2}\bm{u}_{t}^{T}\bm{e}^{T}\bm{eu}_{t}=min_{\bm{u}_{t}}f(\bm{u}_{t})+\frac{\lambda}{2}\bm{u}_{t}^{T}\bm{e}^{T}\bm{eu}_{t},\end{gathered}$
(4)

The second line is the vector form of the first line in Equation 4, where
$\bm{\psi}(\bm{a})=\Big{(}\psi(a_{1}),\psi(a_{2}),\ldots,\psi(a_{N})\Big{)}^{T}$.
In addition, the kernel matrix for the source domain
$K(\bm{X}_{S},\bm{X}_{S})$ is approximated by
$K(\bm{X}_{S},\bm{X}_{S})=\Phi\approx\bm{V}\bm{V}^{T}$, where $\bm{V\in
R^{N_{S}\times r}}$ is a rank $r$ matrix with the $i$-th row $\bm{v}_{i}$ to
increase the kernel computational efficiency (Zhang et al., 2008). We further
simplify Equation 4 as shown in its last line, where we denote
$\bm{u}_{t}=(\beta_{0,t};\bm{\eta}_{t})$,
$\bm{\eta_{t}}=\bm{V}^{T}\bm{Z}_{S}\bm{\alpha}_{t}$,
$\bm{e}=diag(0,\bm{1}_{r\times r})$,
$\tilde{\bm{X}}_{S}=(\bm{Z}_{S}\bm{1}\;\bm{Z}_{S}\bm{V})$, and
$f=\frac{1}{N_{S}}\bm{1}^{T}\bm{\psi}(\hat{\bm{X}}_{S}\bm{u}_{t})$.

Equation 4 can be solved by the proximal linear update for $\bm{u}_{t}$ (Xu
and Yin, 2013)

$\begin{gathered}\bm{u}_{t}^{k}=\min_{\bm{u}_{t}}\langle\nabla
f^{k}(\hat{\bm{u}}_{t}^{k-1}),\bm{u}_{t}-\hat{\bm{u}}_{t}^{k-1}\rangle+\frac{L_{1}^{k-1}}{2}\left\lVert\bm{u}_{t}-\hat{\bm{u}}_{t}^{k-1}\right\rVert^{2}+\frac{\lambda}{2}\bm{u}_{t}^{T}\bm{e}^{T}\bm{eu}_{t}\\\
=\Big{(}L_{1}^{k-1}\bm{I}_{(r+1)\times(r+1)}+\lambda\bm{e}^{T}\bm{e}\Big{)}^{-1}\Big{(}L_{1}^{k-1}\hat{\bm{u}}_{t}^{k-1}-\nabla
f^{k}(\hat{\bm{u}}_{t}^{k-1})\Big{)},\end{gathered}$ (5)

where $\nabla
f^{k}(\hat{\bm{u}}_{t}^{k-1})=(\tilde{\bm{X}_{S}})^{T}\bm{\psi}^{\prime}(\tilde{\bm{X}_{S}}\hat{\bm{u}}_{t}^{k-1})$,
$\hat{\bm{u}}_{t}^{k-1}=\bm{u}_{t}^{k-1}+w_{1}^{k-1}(\bm{u}_{t}^{k-1}-\bm{u}_{t}^{k-2})$
is an extrapolated point, $w_{1}^{k-1}$ is the extrapolation weight, and
$L_{1}^{k-1}$ is the Lipschitz constant. The extrapolation can significantly
accelerate the convergence speed (Xu and Yin, 2013).

Secondly, we solve for $\bm{W}_{t}^{k}$, the mappings for the kernel matrix,
while fixing $(\bm{\alpha}_{t}^{k},\beta_{0,t}^{k})$ in Equation 3. Then,
Equation 3 can be converted to,

$\begin{gathered}min_{\bm{W}_{t}}\>\frac{1}{N_{S}}\sum_{i=1}^{N_{S}}\psi\Big{(}Y_{S,i}[\tilde{\bm{\alpha}}_{t}^{k}]^{T}\tilde{\bm{Z}}_{S}\bm{KW}_{t}\bm{W}_{t}^{T}\bm{K}_{\bm{X}_{S,i}}+Y_{S,i}\beta_{0,t}^{k}\Big{)}+\frac{\lambda}{2}[\tilde{\bm{\alpha}}_{t}^{k}]^{T}\tilde{\bm{Z}}_{S}\bm{KW}_{t}\\\
\bm{W}_{t}^{T}\bm{K}\tilde{\bm{Z}}_{S}[\tilde{\bm{\alpha}}_{t}^{k}]+\lambda_{1}tr(\bm{KW}_{t}\bm{W}_{t}^{T}\bm{K}\bm{S})+\lambda_{2}\left\lVert\bm{H}_{t}\bm{W}_{t}-\bm{H}_{t}\bm{W}_{t-1}\right\rVert_{F}^{2}\end{gathered}$
(6)

where
$\tilde{\bm{\alpha}}_{t}^{k}=[\bm{\alpha}_{t}^{k};\bm{0}_{N_{T_{t}}\times
1}]$, $\tilde{\bm{Z}}_{S}=diag(\bm{Z}_{S},\bm{0}_{N_{T_{t}}\times
N_{T_{t}}})$, the empirical
$\tilde{\bm{K}}(\bm{X}_{i},\bm{X}_{j})\approx\bm{K}_{\bm{X}_{i}}^{T}\bm{W}_{t}\bm{W_{t}}^{T}\bm{K}_{\bm{X}_{j}}$,
where
$\bm{K}_{X}=[\bm{K}(\bm{X}_{1},\bm{X}),\ldots,\bm{K}(\bm{X}_{S+t},\bm{X})]^{T}$,
the empirical $\tilde{\bm{K}}=\begin{bmatrix}\bm{K}_{S,S}&\bm{K}_{S,t}\\\
\bm{K}_{t,S}&\bm{K}_{t,t}\end{bmatrix}\approx\bm{KW}_{t}\bm{W}_{t}^{T}\bm{K}$,
$\bm{S}=\begin{bmatrix}\frac{1}{N_{S}^{2}}\bm{I}_{N_{S}}&-\frac{1}{N_{S}\times
N_{T_{t}}}\bm{I}_{N_{S}\times N_{T_{t}}}\\\ -\frac{1}{N_{S}\times
N_{T_{t}}}\bm{I}_{N_{T_{t}}\times
N_{S}}&\frac{1}{N_{T_{t}}^{2}}\bm{I}_{N_{T_{t}}}\end{bmatrix}$ (Pan et al.,
2010). Here, $\bm{K}_{S,S}$ is the kernel matrix of the source domain,
$\bm{K}_{t,t}$ is the kernel matrix of the target domain at time $t$, and
$\bm{K}_{S,t}=\bm{K}_{t,S}^{T}$ is the kernel matrix between the source and
target domain at time $t$. The approximation to the empirical kernel can
significantly reduce the number of parameters to be estimated, from the
dimension of the sample to estimating $\bm{W}_{t}$. Then, the proximal linear
update solves for

$\bm{W}_{t}^{k}=\min_{\bm{W}_{t}}\langle\nabla
g^{k}(\hat{\bm{W}}_{t}^{k-1}),\bm{W}_{t}-\hat{\bm{W}}_{t}^{k-1}\rangle+\frac{L_{2}^{k-1}}{2}\left\lVert\bm{W}_{t}-\hat{\bm{W}}_{t}^{k-1}\right\rVert^{2}+\lambda_{2}\left\lVert\bm{H}_{t}\bm{W}_{t}-\bm{H}_{t-1}\bm{W}_{t-1}\right\rVert_{F}^{2}$

where $g$ is the first three items of Equation 6,
$\hat{\bm{W}}_{t}^{k-1}=\bm{W}_{t}^{k-1}+w_{2}^{k-1}(\bm{W}_{t}^{k-1}-\bm{W}_{t}^{k-2})$
is an extrapolated point, $w_{2}^{k-1}$ is the extrapolation weight and
$L_{2}^{k-1}$ is Lipschitz constant.

After taking derivative of the above equation with regard to $\bm{W}_{t}$ and
setting it to zero, we have the update for $\bm{W}_{t}$ as follows, see the
Appendix for details.

$\begin{gathered}\bm{W}_{t}^{k}=(L_{2}^{k-1}\bm{I}+2\lambda_{2}\bm{H}_{t}^{T}\bm{H}_{t})^{-1}\bigg{(}L_{2}^{k-1}\hat{\bm{W}}_{t}^{k-1}+2\lambda_{2}\bm{H}_{t}^{T}\bm{H}_{t-1}\bm{W}_{t-1}-\\\
\frac{1}{N_{S}}\sum_{i=1}^{N_{S}}\psi^{\prime}\Big{(}Y_{S,i}[\tilde{\bm{\alpha}}_{t}^{k}]^{T}\tilde{\bm{Z}}_{S}\bm{K}\hat{\bm{W}}_{t}^{k-1}(\hat{\bm{W}}_{t}^{k-1})^{T}\bm{K}_{\bm{X}_{S,i}}+Y_{S,i}\beta_{0,t}^{k}\Big{)}\\\
\Big{(}\bm{K}_{\bm{X}_{S,i}}Y_{S,i}[\tilde{\bm{\alpha}}_{t}^{k}]^{T}\tilde{\bm{Z}}_{S}\bm{K}+\bm{K}\tilde{\bm{Z}}_{S}[\tilde{\bm{\alpha}}_{t}^{k}]Y_{S,i}\bm{K}_{\bm{X}_{S,i}}^{T}\Big{)}\hat{\bm{W}}_{t}^{k-1}-\lambda(\bm{K}\tilde{\bm{Z}}_{S}[\tilde{\bm{\alpha}}_{t}^{k}])\\\
(\bm{K}\tilde{\bm{Z}}_{S}[\tilde{\bm{\alpha}}_{t}^{k}])^{T}\hat{\bm{W}}_{t}^{k-1}-2\lambda_{1}\bm{K}\bm{S}\bm{K}\hat{\bm{W}}_{t}^{k-1}\bigg{)}\end{gathered}$
(7)

Finally, after the algorithm converges, the classification at time $t$ for the
new liver viability evaluation is made by

$sign\big{(}[\hat{\bm{\alpha}}_{t};\bm{0}_{N_{t}\times
1}]^{T}\tilde{\bm{Z}}_{S}\bm{K}\hat{\bm{W}}_{t}\hat{\bm{W}}_{t}^{T}\bm{K}_{\bm{X}_{S,i}}+\hat{\beta}_{0,t}\big{)},$
(8)

where $\hat{\beta}_{0,t}=\hat{\bm{u}}[1]$,
$\hat{\bm{\eta}}=\hat{\bm{u}}[2:end]$ and
$\hat{\bm{\alpha}}_{t}=(\bm{K}_{S,S}\bm{Z}_{S})^{-1}\bm{V}\hat{\bm{\eta}}_{t}$.

##  4 Case Study for Cross-subject Liver Viability Classification

###  4.1 Porcine Liver Procurement Data

We applied our proposed online DA and classification framework to the cross-
liver viability classification in experimental data for porcine liver
procurement originally reported in Lan et al. (2015). Here, we briefly
introduce the experiment and data. After being harvested from the donor, the
livers were kept in the perfusion system perfused with a physiologic perfusion
fluid (modified Krebs’ solution). In total, there were four livers in the case
study, where the perfusion temperature for Liver 1 and Liver 2 was $4^{\circ}$
C, while for Liver 3 and Liver 4 was $22^{\circ}$ C. The livers were
continuously monitored with an infrared camera (FLIR Systems, Boston, MA) for
24 hours to collect the surface temperature information. In particular, the
resolution of the images were $640\times 480$ and the sampling frequency was 1
frame/minute. Every four hours, a biopsy was taken from the liver and judged
by a pathologist on its viability. As reported in Lan et al. (2015, 2018), the
livers usually became unviable after the 8-th hour while it was impossible to
know the exact point where a liver changes from viable to unviable. In
conformity with Lan et al. (2015, 2018), the images 3 hours before and after
the 8-th hour in each liver were excluded from the classification, and a
sampling frequency of 10-minute per image was used. Interested reader is
referred to Lan et al. (2015, 2018) for more details.

###  4.2 Data Pre-processing

The pre-processing consists of background removal and data compression of the
original liver images. For a thermal image in Figure 3 (a), firstly, we
extracted the pure liver region by defining a mask, and applying the mask to
the subsequent streaming images. An irregular thermal data after the
background removal is shown in Figure 3 (b). Afterwards, we performed the data
compression by taking a moving average of a $5\times 5$ window. In case part
of the $5\times 5$ window falls outside of the liver region at a pixel, the
average was performed based on those pixels of $5\times 5$ window inside the
liver region. As a result, Figure 3 (c) shows the compressed data for GSP,
where each pixel was treated as a vertex in the graph (for instance, there
were 7,756 vertices in Liver 1), and the temperature value over the vertices
was a graph signal.

Figure 3: Data preprocessing steps: (a) Original thermal image; (b) Irregular
pure liver thermal data after background removal; (c) Compressed irregular
thermal data

###  4.3 Irregular Liver Thermal Data Feature Extraction using Graph Signal
Processing

We introduce the graph construction and GFT to extract the features from the
irregular thermal data in this section. Firstly, to construct the graph, the
images measured over the first three hours of the experiments were used to
compute the averaged signal $\tilde{\bm{S}}$ in each liver. Thereafter,
Equation 1 was used to obtain the $M\times M$ weight matrix. We then computed
the Laplacian matrix and its factorization
$\bm{L}=\bm{D}-\bm{A}=\bm{F}\bm{\Lambda}\bm{F}^{T}$. Secondly, we transformed
the graph signal, using $\hat{\bm{s}}_{t}=\bm{F}^{T}\bm{s}_{t}$, to the graph
spectral domain. The transformed signal $\hat{\bm{s}}_{t}$ has the same
dimension as the original signal $\bm{s}_{t}$. However, as shown in Figure 2,
only the first few elements of $\hat{\bm{s}}_{t}$ related to low frequencies
are non-zero. In our case, the first 50 elements of $\hat{\bm{s}}_{t}$ were
selected. This can significantly reduce the dimension of the problem in later
analysis, while keep the patterns in the graph signal.

Figure 4 illustrates an example of the irregular thermal data reconstruction
using only the first 50 elements of $\hat{\bm{s}}_{t}$. The reconstruction
error can be quantified by
$\frac{||(\bm{F}^{T})^{-1}\hat{\bm{s}}_{t}^{\prime}-\bm{s}_{t}||_{2}}{||\bm{s}_{t}||_{2}}$,
where $\hat{\bm{s}}_{t}^{\prime}$ is a vector with the first 50 elements being
the same as $\hat{\bm{s}}_{t}$, and zero elsewhere. On average, the
reconstruction error resulted by storing first 50 elements of
$\hat{\bm{s}}_{t}$ was 0.08 over the livers, which is reasonably small.

Figure 4: Illustration of irregular thermal data reconstruction of Liver 1
using: (a) All elements in $\hat{\bm{s}}_{t}$; (b) The first 50 elements in
$\hat{\bm{s}}_{t}$.

###  4.4 Online Domain Adaptation and Classification via Joint Optimization

Note that in our problem, the source domain data (both irregular thermal data
and viability labels) were fully available, while the irregular thermal data
in the target domain (without viability labels) were observed as a continuous
data stream. We continuously made the viability classification once a
streaming image is observed, and updated the model every one hour. In
particular, Equation 3 was hourly optimized to update the model, and predict
the labels of the forthcoming images in the target domain. The model update
typically took around less than 3 seconds when performed on an HP laptop
equipped with a 2.10 GHz processor and 16.0 GB of RAM running a 64-bit version
of Windows 10. Here, a hourly updated was used to balance the information
accumulation and domain evolution in the target domain, and other numbers can
be used in other applications.

We compare our proposed method and four benchmark methods: offline SVM, online
SVM, offline subspace alignment and online subspace alignment. A leave-one-
out-cross-validation was used to evaluate the performance of the proposed
algorithm in comparison with benchmarks, where livers were classified every 10
minutes. Specifically, in each iteration, one of the livers was considered as
target and the others were considered as sources. A majority voting approach
was applied, in which the final predicted label at a specific time was the one
that gained more than half of the votes.

The first two benchmarks, offline SVM and online SVM, did not benefit from any
DA. In offline SVM, the underlying assumption was that all of the features
associated with sources and target were known. Therefore, the features could
be normalized and SVM could be applied to the normalized features. However, in
online SVM, we consider the streaming nature of the target data, and the
features could not be normalized and SVM was directly applied to the features
obtained from GFT. For the last two benchmarks, subspace alignment (SA) is an
unsupervised DA method, which was originally developed for the purpose of
offline DA (Fernando et al., 2013). In this method, the source and target
features are represented by subspaces based on eigenvectors. The offline SA
first perform the SA in Fernando et al. (2013), and then perform the SVM
classification. The online SA considered the streaming nature of the target
data, and an SA was implemented every time that a new observation was
received, followed by the SVM classification (i.e., the DA and classification
are done in two separate steps).

Next, we will discuss the details of the optimization process and results
comparison.

####  4.4.1 Details of the Optimization Process

We set $r$ to be 50 in the approximation of $K(\bm{X}_{S},\bm{X}_{S})$ with
$\bm{V}\bm{V}^{T}$, and our results demonstrated that the average
reconstruction error associated with this factorization was $<0.03$. In
Algorithm 1, Lipschitz constants and extrapolation weights were specified as
$L_{1}^{k-1}=1000$, $L_{2}^{k-1}=1000$, $w_{1}^{k-1}=0.01$ and
$w_{2}^{k-1}=0.01$ (Xu and Yin, 2013). The $\delta$ was specified as 1 and the
number of iterations was set to 50, which is sufficient for the algorithm to
converge in our case study.

We set $\lambda$ to 1 and tuned the two weights associated with DA and
smoothness loss functions ($\lambda_{1}$ and $\lambda_{2}$) in Equation 3 via
leave-one-out-cross-validation. In particular, we applied the leave-one-out-
cross-validation to the source domain data at combinations of values of
$\lambda_{1}$ and $\lambda_{2}$, and $\lambda_{1}$ and $\lambda_{2}$ were
selected to maximize the average accuracy of the classification over all
observation times and all left-out source domain data. Note that the tuning
process only needs the source domain data, and can be performed offline. Here,
the range of $\lambda_{1}$ and $\lambda_{2}$ are
$\lambda_{1}=[20,40,60,\ldots,200]$ and
$\lambda_{2}=[0.002,0.004,0.006,\ldots,0.020]$, respectively. The tuning
parameter in SA was also selected using leave-one-out-cross-validation.

####  4.4.2 Comparative Analysis

A summary of the accuracy (ACC), precision (PRC) and recall (RCL) results for
the target domain data (i.e., testing data) with different methods is given in
Table 1. From Table 1, when there is not any DA (offline and online SVM), the
accuracy ranges from 0.75 to 0.97 for offline SVM (with an average of 0.86)
and from 0.35 to 0.81 for online SVM (with an average of 0.67) across the
livers. On average, the recalls associated with offline and online SVM are
0.37 and 0.25, respectively. In both methods, there are cases in which NaN
values are reported for precision because no livers are classified as viable
in those cases. We attribute the low classification performance of these two
methods to the heterogeneity of livers. However, it can be noted that
normalization in offline SVM can slightly mitigate the liver-to-liver
variation.

Table 1: Accuracy, precision and recall results for different methods

Liver Number SVM (offline) SVM (online) Subspace Alignment (offline) Subspace
Alignment (online) Proposed (our method) ACC PRC RCL ACC PRC RCL ACC PRC RCL
ACC PRC RCL ACC PRC RCL Liver 1 0.81 NaN 0.00 0.81 NaN 0.00 1.00 1.00 1.00
0.83 1.00 0.11 0.91 0.67 1.00 Liver 2 0.75 0.00 0.00 0.76 NaN 0.00 0.82 1.00
0.25 0.82 1.00 0.25 0.94 0.80 1.00 Liver 3 0.97 0.96 0.92 0.35 0.27 1.00 0.99
1.00 0.96 0.76 NaN 0.00 0.79 0.53 1.00 Liver 4 0.89 1.00 0.54 0.76 NaN 0.00
0.81 0.56 1.00 0.99 0.96 1.00 1.00 1.00 1.00 Avg. 0.86 NaN 0.37 0.67 NaN 0.25
0.91 0.89 0.80 0.85 NaN 0.34 0.91 0.75 1.00

When the heterogeneity is addressed by DA, the accuracy ranges from 0.79 to
1.00 (with an average of 0.91) for our algorithm, from 0.76 to 0.99 (with an
average of 0.85) for online SA and from 0.81 to 1.00 (with an average of 0.91)
for offline SA across the livers. Considering accuracy, precision and recall,
our algorithm outperforms its online counterpart (online SA) in accuracy and
recall and shows a lower precision compared to online SA (in case we ignore
NaN for online SA); however, its recall is much better than that of online SA.

If we focus on the comparison between DA methods and methods without DA, it
can be noted that offline SA outperforms offline SVM, and online SA and our
proposed method perform better that online SVM. This result justifies the
heterogeneity of the livers and highlights the necessity of using DA to
alleviate this issue. A comparison between online and offline methods shows
that offline methods generally outperform online ones, which is an intuitive
observation. In particular, offline SVM outperforms online SVM because it
benefits from scaling through normalization. Moreover, offline SA outperforms
(in two out of three measures) online SA because learning the transformation
is more effective when the whole target data is available and shows a
comparable performance to our proposed method. Our proposed method outperforms
online SA (it shows better accuracy and significantly better recall) because
of the two following reasons:

  * •

Our proposed method uses block coordinate proximal descent optimization to
jointly learn the domain-invariant features and classifiers, whereas SA learns
the subspaces and classifiers in a two-stage procedure.

  * •

To regularize learning and improve performance, we use a loss function to
assert that the mapping matrix changes slowly in time.

Finally, offline SA was the only method that showed a performance comparable
to our proposed method. However, it is evident that an offline scenario is
very idealistic and it is not applicable to real-world applications.

##  5 Conclusion and Future Work

Organ transplantation is the most effective treatment for end-stage liver
disease. However, the accurate evaluation of organ viability before
transplantation has been a challenging issue. Traditionally, the viability is
evaluated by repetitively taking invasive biopsies on the organ. Recently,
viability evaluation by infrared imaging has gained increasing interest due to
its noninvasive nature. However, the existing studies used the full thermal
images with background noise rather than the irregular pure liver region
thermal data, and did not fully address the heterogeneity of cross-subject
livers during the viability evaluation.

In this paper, we propose a novel online DA and classification framework using
the irregular thermal data for the liver viability evaluation. In particular,
we extract features of irregular thermal data by GSP, and develop a joint
optimization algorithm to jointly learn the domain-invariant features of the
cross-subject livers and learn the classifiers. Our proposed method excels
other benchmark methods, and has accurate online viability evaluations.

We will pursue several directions in the future. First, in our work, in spite
of having multiple sources, we considered the problem as multiple single-
source DA problems and the output prediction was done based on majority
voting. In the future, we will extend our framework to the multi-source DA. In
addition, there have been some works on the variance change-point detection of
the original thermal images (Gao et al., 2018, 2020). We will perform the
graph-based change-point detection for the irregular thermal data change-point
detection.

##  6 Appendix

We provide details on the derivation of the update of $\bm{W}_{t}$ in Equation
7. Recall that the proximal update solves for

$\bm{W}_{t}^{k}=\min_{\bm{W}_{t}}\langle\nabla
g^{k}(\hat{\bm{W}}_{t}^{k-1}),\bm{W}_{t}-\hat{\bm{W}}_{t}^{k-1}\rangle+\frac{L_{2}^{k-1}}{2}\left\lVert\bm{W}_{t}-\hat{\bm{W}}_{t}^{k-1}\right\rVert^{2}+\lambda_{2}\left\lVert\bm{H}_{t}\bm{W}_{t}-\bm{H}_{t-1}\bm{W}_{t-1}\right\rVert_{F}^{2}$

Taking derivative of the above equation with regard to $\bm{W}_{t}$ and
setting it to zero, we have

$\nabla
g^{k}(\hat{\bm{W}}_{t}^{k-1})+L_{2}^{k-1}(\bm{W}_{t}-\hat{\bm{W}}_{t}^{k-1})+2\lambda_{2}\bm{H}_{t}^{T}(\bm{H}_{t}\bm{W}_{t}-\bm{H}_{t-1}\bm{W}_{t-1})=0,$
(9)

where $\nabla
g^{k}(\hat{\bm{W}}_{t}^{k-1})=\frac{1}{N_{S}}\sum_{i=1}^{N_{S}}\frac{\partial\psi\Big{(}Y_{S,i}[\tilde{\bm{\alpha}}_{t}^{k}]^{T}\tilde{\bm{Z}}_{S}\bm{KW}_{t}\bm{W}_{t}^{T}\bm{K}_{\bm{X}_{S,i}}+Y_{S,i}\beta_{0,t}^{k}\Big{)}}{\partial\bm{W}_{t}}+\frac{\lambda}{2}\frac{\partial([\tilde{\bm{\alpha}}_{t}^{k}]^{T}\tilde{\bm{Z}}_{S}\bm{KW}_{t}\bm{W}_{t}^{T}\bm{K}\tilde{\bm{Z}}_{S}[\tilde{\bm{\alpha}}_{t}^{k}])}{\partial\bm{W}_{t}}+\lambda_{1}\frac{\partial\big{(}tr(\bm{KW}_{t}\bm{W}_{t}^{T}\bm{K}\bm{S})\big{)}}{\partial\bm{W}_{t}}$.
We will focus on the derivative for each of the term as follows.

According to Petersen and Pedersen (2012) Eqs. 70-72, 77,

$\begin{gathered}\frac{1}{N_{S}}\sum_{i=1}^{N_{S}}\frac{\partial\psi\Big{(}Y_{S,i}[\tilde{\bm{\alpha}}_{t}^{k}]^{T}\tilde{\bm{Z}}_{S}\bm{KW}_{t}\bm{W}_{t}^{T}\bm{K}_{\bm{X}_{S,i}}+Y_{S,i}\beta_{0,t}^{k}\Big{)}}{\partial\bm{W}_{t}}=\frac{1}{N_{S}}\sum_{i=1}^{N_{S}}\psi^{\prime}\Big{(}Y_{S,i}[\tilde{\bm{\alpha}}_{t}^{k}]^{T}\tilde{\bm{Z}}_{S}\\\
\bm{KW}_{t}\bm{W}_{t}^{T}\bm{K}_{\bm{X}_{S,i}}+Y_{S,i}\beta_{0,t}^{k}\Big{)}\Big{(}\bm{K}_{\bm{X}_{S,i}}Y_{S,i}[\tilde{\bm{\alpha}}_{t}^{k}]^{T}\tilde{\bm{Z}}_{S}\bm{K}+\bm{K}\tilde{\bm{Z}}_{S}[\tilde{\bm{\alpha}}_{t}^{k}]Y_{S,i}\bm{K}_{\bm{X}_{S,i}}^{T}\Big{)}\bm{W}_{t},\end{gathered}$

$\begin{gathered}\frac{\lambda}{2}\frac{\partial\big{(}[\tilde{\bm{\alpha}}_{t}^{k}]^{T}\tilde{\bm{Z}}_{S}\bm{KW}_{t}\bm{W}_{t}^{T}\bm{K}\tilde{\bm{Z}}_{S}[\tilde{\bm{\alpha}}_{t}^{k}]\big{)}}{\partial\bm{W}_{t}}=\lambda(\bm{K}\tilde{\bm{Z}}_{S}[\tilde{\bm{\alpha}}_{t}^{k}])(\bm{K}\tilde{\bm{Z}}_{S}[\tilde{\bm{\alpha}}_{t}^{k}])^{T}\bm{W}_{t}.\end{gathered}$

According to Petersen and Pedersen (2012) Eq. 100,

$\lambda_{1}\frac{\partial\big{(}tr(\bm{KW}_{t}\bm{W}_{t}^{T}\bm{K}\bm{S})\big{)}}{\partial\bm{W}_{t}}=\lambda_{1}\frac{\partial\big{(}tr(\bm{W}_{t}^{T}\bm{K}\bm{S}\bm{KW}_{t})\big{)}}{\partial\bm{W}_{t}}=2\lambda_{1}\bm{K}\bm{S}\bm{KW}_{t}$

Substituting the above derivative terms into Equation 9, we have

$\begin{gathered}\frac{1}{N_{S}}\sum_{i=1}^{N_{S}}\psi^{\prime}\Big{(}Y_{S,i}[\tilde{\bm{\alpha}}_{t}^{k}]^{T}\tilde{\bm{Z}}_{S}\bm{K}\hat{\bm{W}}_{t}^{k-1}(\hat{\bm{W}}_{t}^{k-1})^{T}\bm{K}_{\bm{X}_{S,i}}+Y_{S,i}\beta_{0,t}^{k}\Big{)}\Big{(}\bm{K}_{\bm{X}_{S,i}}Y_{S,i}[\tilde{\bm{\alpha}}_{t}^{k}]^{T}\tilde{\bm{Z}}_{S}\bm{K}+\\\
\bm{K}\tilde{\bm{Z}}_{S}[\tilde{\bm{\alpha}}_{t}^{k}]Y_{S,i}\bm{K}_{\bm{X}_{S,i}}^{T}\Big{)}\hat{\bm{W}}_{t}^{k-1}+\lambda(\bm{K}\tilde{\bm{Z}}_{S}[\tilde{\bm{\alpha}}_{t}^{k}])(\bm{K}\tilde{\bm{Z}}_{S}[\tilde{\bm{\alpha}}_{t}^{k}])^{T}\hat{\bm{W}}_{t}^{k-1}+2\lambda_{1}\bm{K}\bm{S}\bm{K}\hat{\bm{W}}_{t}^{k-1}+\\\
L_{2}^{k-1}(\bm{W}_{t}-\hat{\bm{W}}_{t}^{k-1})+2\lambda_{2}\bm{H}_{t}^{T}(\bm{H}_{t}\bm{W}_{t}-\bm{H}_{t-1}\bm{W}_{t-1})=0\end{gathered}$

Simplifying the equation, we have the update of $\bm{W}_{t}$ as shown in
Equation 7

$\begin{gathered}\bm{W}_{t}=(L_{2}^{k-1}\bm{I}+2\lambda_{2}\bm{H}_{t}^{T}\bm{H}_{t})^{-1}\bigg{(}L_{2}^{k-1}\hat{\bm{W}}_{t}^{k-1}+2\lambda_{2}\bm{H}_{t}^{T}\bm{H}_{t-1}\bm{W}_{t-1}-\frac{1}{N_{S}}\sum_{i=1}^{N_{S}}\psi^{\prime}\Big{(}Y_{S,i}\\\
[\tilde{\bm{\alpha}}_{t}^{k}]^{T}\tilde{\bm{Z}}_{S}\bm{K}\hat{\bm{W}}_{t}^{k-1}(\hat{\bm{W}}_{t}^{k-1})^{T}\bm{K}_{\bm{X}_{S,i}}+Y_{S,i}\beta_{0,t}^{k}\Big{)}\Big{(}\bm{K}_{\bm{X}_{S,i}}Y_{S,i}[\tilde{\bm{\alpha}}_{t}^{k}]^{T}\tilde{\bm{Z}}_{S}\bm{K}+\bm{K}\tilde{\bm{Z}}_{S}[\tilde{\bm{\alpha}}_{t}^{k}]Y_{S,i}\\\
\bm{K}_{\bm{X}_{S,i}}^{T}\Big{)}\hat{\bm{W}}_{t}^{k-1}-\lambda(\bm{K}\tilde{\bm{Z}}_{S}[\tilde{\bm{\alpha}}_{t}^{k}])(\bm{K}\tilde{\bm{Z}}_{S}[\tilde{\bm{\alpha}}_{t}^{k}])^{T}\hat{\bm{W}}_{t}^{k-1}-2\lambda_{1}\bm{K}\bm{S}\bm{K}\hat{\bm{W}}_{t}^{k-1}\bigg{)}.\end{gathered}$

## References

  * Badebarin et al. (2017) Davoud Badebarin, Saeid Aslanabadi, Amir Teimouri-Dereshki, Masoud Jamshidi, Tuba Tarverdizadeh, Kaveh Shad, Kamyar Ghabili, and Ghazal Khajir.  Different clinical presentations of choledochal cyst among infants and older children: A 10-year retrospective study.  _Medicine_ , 96(17), 2017.
  * Bitarafan et al. (2016) Adeleh Bitarafan, Mahdieh Soleymani Baghshah, and Marzieh Gheisari.  Incremental evolving domain adaptation.  _IEEE Transactions on Knowledge and Data Engineering_ , 28(8):2128–2141, 2016.
  * Bullmore and Sporns (2012) Ed Bullmore and Olaf Sporns.  The economy of brain network organization.  _Nature Reviews Neuroscience_ , 13(5):336–349, 2012.
  * Cameron and Cornejo (2015) Andrew M Cameron and Jose F Barandiaran Cornejo.  Organ preservation review: history of organ preservation.  _Current opinion in organ transplantation_ , 20(2):146–151, 2015.
  * CDC (2020) CDC.  Chronic liver disease and cirrhosis, 2020.  URL https://www.cdc.gov/nchs/fastats/liver-disease.htm.
  * Cheng et al. (2020) Longwei Cheng, Kai Wang, and Fugee Tsung.  A hybrid transfer learning framework for in-plane freeform shape accuracy control in additive manufacturing.  _IISE Transactions_ , pages 1–15, 2020.
  * Ding et al. (2018) Zhengming Ding, Sheng Li, Ming Shao, and Yun Fu.  Graph adaptive knowledge transfer for unsupervised domain adaptation.  In _Proceedings of the European Conference on Computer Vision (ECCV)_ , pages 37–52, 2018.
  * Feng et al. (2006) S Feng, NP Goodrich, JL Bragg-Gresham, DM Dykstra, JD Punch, MA DebRoy, Stuart M Greenstein, and RM Merion.  Characteristics associated with liver graft failure: the concept of a donor risk index.  _American Journal of Transplantation_ , 6(4):783–790, 2006.
  * Fernando et al. (2013) Basura Fernando, Amaury Habrard, Marc Sebban, and Tinne Tuytelaars.  Unsupervised visual domain adaptation using subspace alignment.  In _Proceedings of the IEEE international conference on computer vision_ , pages 2960–2967, 2013.
  * French et al. (2017) Geoffrey French, Michal Mackiewicz, and Mark Fisher.  Self-ensembling for domain adaptation.  _arXiv preprint arXiv:1706.05208_ , 2017.
  * Gao et al. (2018) Zhenguo Gao, Zuofeng Shang, Pang Du, and John L Robertson.  Variance change point detection under a smoothly-changing mean trend with application to liver procurement.  _Journal of the American Statistical Association_ , 2018.
  * Gao et al. (2020) Zhenguo Gao, Pang Du, Ran Jin, John L Robertson, et al.  Surface temperature monitoring in liver procurement via functional variance change-point analysis.  _Annals of Applied Statistics_ , 14(1):143–159, 2020.
  * Gorbach et al. (2009) Alexander M Gorbach, David B Leeser, Hengliang Wang, Douglas K Tadaki, Carlos Fernandez, David DeStephano, Douglas Hale, Allan D Kirk, Fred A Gage, and Eric A Elster.  Assessment of cadaveric organ viability during pulsatile perfusion using infrared imaging.  _Transplantation_ , 87(8):1163, 2009.
  * Hoffman et al. (2014) Judy Hoffman, Trevor Darrell, and Kate Saenko.  Continuous manifold based adaptation for evolving visual domains.  In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , pages 867–874, 2014.
  * Hoffman et al. (2018) Judy Hoffman, Eric Tzeng, Taesung Park, Jun-Yan Zhu, Phillip Isola, Kate Saenko, Alexei Efros, and Trevor Darrell.  Cycada: Cycle-consistent adversarial domain adaptation.  In _International conference on machine learning_ , pages 1989–1998, 2018.
  * HRSA (2020) HRSA.  Organ procurement and transplantation network: National data, 2020.  URL https://optn.transplant.hrsa.gov/data/view-data-reports/national-data/#.
  * Hu et al. (2016a) Chenhui Hu, Xue Hua, Jun Ying, Paul M Thompson, Georges E Fakhri, and Quanzheng Li.  Localizing sources of brain disease progression with network diffusion model.  _IEEE journal of selected topics in signal processing_ , 10(7):1214–1225, 2016a.
  * Hu et al. (2016b) Chenhui Hu, Jorge Sepulcre, Keith A Johnson, Georges E Fakhri, Yue M Lu, and Quanzheng Li.  Matched signal detection on graphs: Theory and application to brain imaging data classification.  _NeuroImage_ , 125:587–600, 2016b.
  * Huang et al. (2012) Shuai Huang, Jing Li, Kewei Chen, Teresa Wu, Jieping Ye, Xia Wu, and Li Yao.  A transfer learning approach for network modeling.  _IIE transactions_ , 44(11):915–931, 2012.
  * Kang et al. (2020) Xiaoning Kang, Xiaoyu Chen, Ran Jin, Hao Wu, and Xinwei Deng.  Multivariate regression of mixed responses for evaluation of visualization designs.  _IISE Transactions_ , pages 1–13, 2020.
  * Keeffe (2001) Emmet B Keeffe.  Liver transplantation: current status and novel approaches to liver replacement.  _Gastroenterology_ , 120(3):749–762, 2001.
  * Khosla et al. (2012) Aditya Khosla, Tinghui Zhou, Tomasz Malisiewicz, Alexei A Efros, and Antonio Torralba.  Undoing the damage of dataset bias.  In _European Conference on Computer Vision_ , pages 158–171. Springer, 2012.
  * Kochan et al. (2015) Kamila Kochan, E Maslak, S Chlopicki, and M Baranska.  Ft-ir imaging for quantitative determination of liver fat content in non-alcoholic fatty liver.  _Analyst_ , 140(15):4997–5002, 2015.
  * Kontar et al. (2020) Raed Kontar, Garvesh Raskutti, and Shiyu Zhou.  Minimizing negative transfer of knowledge in multivariate gaussian processes: A scalable and regularized approach.  _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , 2020.
  * Lahiri et al. (2012) BB Lahiri, S Bagavathiappan, T Jayakumar, and John Philip.  Medical applications of infrared thermography: a review.  _Infrared Physics & Technology_, 55(4):221–235, 2012.
  * Lan (2020) Qing Lan.  _Organ Viability Assessment in Transplantation based on Data-driven Modeling_.  PhD thesis, Virginia Tech, 2020.
  * Lan et al. (2015) Qing Lan, Ran Jin, and John L Robertson.  Quantitative and qualitative evaluation for organ preservation in transplant.  In _IIE Annual Conference. Proceedings_ , page 2229. Institute of Industrial and Systems Engineers (IISE), 2015.
  * Lan et al. (2018) Qing Lan, Hongyue Sun, John Robertson, Xinwei Deng, and Ran Jin.  Non-invasive assessment of liver quality in transplantation based on thermal imaging analysis.  _Computer methods and programs in biomedicine_ , 164:31–47, 2018.
  * Long et al. (2015) Mingsheng Long, Yue Cao, Jianmin Wang, and Michael Jordan.  Learning transferable features with deep adaptation networks.  In _International conference on machine learning_ , pages 97–105. PMLR, 2015.
  * Long et al. (2017) Mingsheng Long, Han Zhu, Jianmin Wang, and Michael I Jordan.  Deep transfer learning with joint adaptation networks.  In _International conference on machine learning_ , pages 2208–2217, 2017.
  * Ménoret et al. (2017) Mathilde Ménoret, Nicolas Farrugia, Bastien Pasdeloup, and Vincent Gripon.  Evaluating graph signal processing for neuroimaging through classification and dimensionality reduction.  In _2017 IEEE Global Conference on Signal and Information Processing (GlobalSIP)_ , pages 618–622. IEEE, 2017.
  * Mueller et al. (2014) Sebastian Mueller, Helmut Karl Seitz, and Vanessa Rausch.  Non-invasive diagnosis of alcoholic liver disease.  _World journal of gastroenterology: WJG_ , 20(40):14626, 2014.
  * Ortega et al. (2018) Antonio Ortega, Pascal Frossard, Jelena Kovačević, José MF Moura, and Pierre Vandergheynst.  Graph signal processing: Overview, challenges, and applications.  _Proceedings of the IEEE_ , 106(5):808–828, 2018\.
  * Pan et al. (2010) Sinno Jialin Pan, Ivor W Tsang, James T Kwok, and Qiang Yang.  Domain adaptation via transfer component analysis.  _IEEE Transactions on Neural Networks_ , 22(2):199–210, 2010.
  * Petersen and Pedersen (2012) KB Petersen and MS Pedersen.  The matrix cookbook, version 20121115.  _Technical Univ. Denmark, Kongens Lyngby, Denmark, Tech. Rep_ , 3274, 2012.
  * Petrick et al. (2015) Anthony Petrick, Peter Benotti, G Craig Wood, Christopher D Still, William E Strodel, John Gabrielsen, David Rolston, Xin Chu, George Argyropoulos, Anna Ibele, et al.  Utility of ultrasound, transaminases, and visual inspection to assess nonalcoholic fatty liver disease in bariatric surgery patients.  _Obesity surgery_ , 25(12):2368–2375, 2015.
  * Pirayre et al. (2015) Aurélie Pirayre, Camille Couprie, Frédérique Bidard, Laurent Duval, and Jean-Christophe Pesquet.  Brane cut: biologically-related a priori network enhancement with graph cuts for gene regulatory network inference.  _BMC bioinformatics_ , 16(1):1–12, 2015.
  * Pirayre et al. (2017) Aurélie Pirayre, Camille Couprie, Laurent Duval, and Jean-Christophe Pesquet.  Brane clust: Cluster-assisted gene regulatory network inference refinement.  _IEEE/ACM transactions on computational biology and bioinformatics_ , 15(3):850–860, 2017.
  * Rothuizen and Twedt (2009) Jan Rothuizen and David C Twedt.  Liver biopsy techniques.  _Veterinary Clinics: Small Animal Practice_ , 39(3):469–480, 2009.
  * Saito et al. (2017) Kuniaki Saito, Yoshitaka Ushiku, and Tatsuya Harada.  Asymmetric tri-training for unsupervised domain adaptation.  _arXiv preprint arXiv:1702.08400_ , 2017.
  * Sartipi et al. (2020) Shadi Sartipi, Hashem Kalbkhani, Peyman Ghasemzadeh, and Mahrokh G Shayesteh.  Stockwell transform of time-series of fmri data for diagnoses of attention deficit hyperactive disorder.  _Applied Soft Computing_ , 86:105905, 2020.
  * Schölkopf et al. (2001) Bernhard Schölkopf, Ralf Herbrich, and Alex J Smola.  A generalized representer theorem.  In _International conference on computational learning theory_ , pages 416–426. Springer, 2001.
  * Skowno and Karpelowsky (2014) Justin J Skowno and Jonathan Saul Karpelowsky.  Near-infrared spectroscopy for monitoring renal transplant perfusion.  _Pediatric Nephrology_ , 29(11):2241–2242, 2014\.
  * Sporns (2010) Olaf Sporns.  _Networks of the Brain_.  MIT press, 2010.
  * Sun et al. (2019) Yu Sun, Eric Tzeng, Trevor Darrell, and Alexei A Efros.  Unsupervised domain adaptation through self-supervision.  _arXiv preprint arXiv:1909.11825_ , 2019.
  * Taigman et al. (2016) Yaniv Taigman, Adam Polyak, and Lior Wolf.  Unsupervised cross-domain image generation.  _arXiv preprint arXiv:1611.02200_ , 2016.
  * Torralba and Efros (2011) Antonio Torralba and Alexei A Efros.  Unbiased look at dataset bias.  In _CVPR 2011_ , pages 1521–1528. IEEE, 2011.
  * Tran et al. (2019) Hoang M Tran, Satish TS Bukkapatnam, and Mridul Garg.  Detecting changes in transient complex systems via dynamic network inference.  _IISE Transactions_ , 51(3):337–353, 2019.
  * Vazquez-Martul and Papadimitriou (2004) E Vazquez-Martul and JC Papadimitriou.  Importance of biopsy evaluation and the role of the pathologist in solid organ transplant programs.  In _Transplantation proceedings_ , volume 3, pages 725–728, 2004\.
  * Vidal et al. (2014) Enrico Vidal, Angela Amigoni, Valentina Brugnolaro, Giulia Ghirardo, Piergiorgio Gamba, Andrea Pettenazzo, Giovanni Franco Zanon, Chiara Cosma, Mario Plebani, and Luisa Murer.  Near-infrared spectroscopy as continuous real-time monitoring for kidney graft perfusion.  _Pediatric Nephrology_ , 29(5):909–914, 2014\.
  * Volk et al. (2011) Michael L Volk, Heidi A Reichert, Anna SF Lok, and Rodney A Hayward.  Variation in organ quality between liver transplant centers.  _American Journal of Transplantation_ , 11(5):958–964, 2011.
  * Wang et al. (2008) Li Wang, Ji Zhu, and Hui Zou.  Hybrid huberized support vector machines for microarray classification and gene selection.  _Bioinformatics_ , 24(3):412–419, 2008.
  * Xu and Yin (2013) Yangyang Xu and Wotao Yin.  A block coordinate descent method for regularized multiconvex optimization with applications to nonnegative tensor factorization and completion.  _SIAM Journal on imaging sciences_ , 6(3):1758–1789, 2013.
  * Zhang et al. (2008) Kai Zhang, Ivor W Tsang, and James T Kwok.  Improved nyström low-rank approximation and error analysis.  In _Proceedings of the 25th international conference on Machine learning_ , pages 1232–1239, 2008.