# Towards Cross-Domain Continual Learning ††thanks: This research is supported
by the Agency for Science, Technology and Research (A*STAR) under its RIE 2025
– Industry Alignment Fund – Pre Positioning (IAF-PP) funding scheme (Project
No: M23L4a0001).

1st Marcus de Carvalho  Nanyang Technological University
50 Nanyang Ave, 639798, Singapore
<EMAIL_ADDRESS>2nd Mahardhika Pratama  University of South Australia
Mawson Lakes, Adelaide, 5095, Australia
<EMAIL_ADDRESS>3rd Jie Zhang  Nanyang Technological University
50 Nanyang Ave, 639798, Singapore
<EMAIL_ADDRESS>4th Chua Haoyan  Nanyang Technological University
50 Nanyang Ave, 639798, Singapore
<EMAIL_ADDRESS>5th Edward Yapp  Singapore Institute of Manufacturing
Technology (SIMTech)
Agency for Science, Technology and Research (A*STAR)
5 CleanTechLoop #01-01, CleanTechTwo Blk B, 636732, Singapore
<EMAIL_ADDRESS>
###### Abstract

Continual learning is a process that involves training learning agents to
sequentially master a stream of tasks or classes without revisiting past data.
The challenge lies in leveraging previously acquired knowledge to learn new
tasks efficiently, while avoiding catastrophic forgetting. Existing methods
primarily focus on single domains, restricting their applicability to specific
problems.

In this work, we introduce a novel approach called Cross-Domain Continual
Learning (CDCL) that addresses the limitations of being limited to single
supervised domains. Our method combines inter- and intra-task cross-attention
mechanisms within a compact convolutional network. This integration enables
the model to maintain alignment with features from previous tasks, thereby
delaying the data drift that may occur between tasks, while performing
unsupervised cross-domain (UDA) between related domains. By leveraging an
intra-task-specific pseudo-labeling method, we ensure accurate input pairs for
both labeled and unlabeled samples, enhancing the learning process. To
validate our approach, we conduct extensive experiments on public UDA
datasets, showcasing its positive performance on cross-domain continual
learning challenges. Additionally, our work introduces incremental ideas that
contribute to the advancement of this field.

We make our code and models available to encourage further exploration and
reproduction of our results: https://github.com/Ivsucram/CDCL

###### Index Terms:

Transformers, Continual Learning, Unsupervised Domain Adaptation, Data
Streaming, Transfer Learning

##  I Introduction

Continual learning (CL) has gained significant attention in the context of
data streaming systems, particularly for self-learning agents and robots.
While offline neural networks approaches are typically designed around data
collected in controlled environments, aggregated into static datasets, such
strategies deviate significantly from the more neural learning conditions
experienced by humans and other mammals throughout their lifespans [5].

In stark contrast to current dominant artificial learning status quo, human
learning involves accumulating and retaining past experiences, seamlessly
integrating them when acquiring new skills and tackling novel challenges. The
human brain excels at integrating multisensory information, facilitating
effective adaptation to uncertain sensory scenarios and leveraging knowledge
from complementary domains to accomplish common tasks [40]. However, deep
learning models, while showing promise on static data, often suffer from
catastrophic forgetting, a consequence of a weak stability-plasticity trade-
off [6]. This phenomenon results in new information overwriting past
experiences, leading to a significant decline in performance on previous
tasks. Tackling this challenge in CL has been a long-standing pursuit in the
fields of machine learning, neural networks, and artificial intelligence [18,
41].

Despite extensive research on unsupervised domain adaptation [49, 32, 14, 10,
25], its practical application in CL remains largely unexplored. Current
approaches in this field predominantly focus on either single-domain continual
learning [9, 28, 4, 8, 47] or supervised multi-domain continual learning [39],
both of which have limitations in fully harnessing the wealth of available
world knowledge. The challenge lies in addressing two fundamental issues faced
by unsupervised domain adaptation methods: task drift, which involves adapting
to changes in the conditional distributions of given labels, and domain drift,
concerning changes in the marginal distributions of given domains.
Consequently, single-domain and supervised multi-domain continual learning
methods are susceptible to feature-alignment catastrophic forgetting when
applied to unsupervised multi-domain problems. This vulnerability results in
not only overwriting past experiences but also compromising the understanding
of domain invariance.

To overcome these challenges, more comprehensive approaches are required to
effectively handle both task and domain drifts, ensuring robust performance
across diverse domains while preserving acquired knowledge for continual
learning scenarios. A compelling real-world example would be a hypothetical
self-driving car that initially learns to navigate the streets of London using
supervised datasets and then expands that knowledge to unsupervised roads in
the countryside. Subsequently, the same car needs to adapt and learn how to
drive in France, facing new laws, road signs, landmarks, and driving on the
opposite side of the road. The car must maintain its previous knowledge from
London and extend it to unsupervised related domains, such as mountain roads
and countryside paths, to achieve successful and safe navigation in this new
setting.

In this research, we present a pioneering framework called Cross-Domain
Continual Learning (CDCL), aimed at addressing the challenges of unsupervised
domain adaptation in the continual learning setting. CDCL introduces a unique
mechanism known as inter intra-task cross-attention, facilitating category-
level feature alignment between labeled source domains and unlabeled target
domains within the continual learning context. Additionally, CDCL incorporates
an innovative intra-task center-aware pseudo-label procedure, strategically
organizing similar source and target domain samples while reducing noise
interference. This procedure contributes to more effective learning in a
continual setup. Moreover, CDCL leverages a rehearsal memory that not only
stores previous source and target data samples but also retains previously
generated logits. By doing so, the model is compelled to mimic acquired
knowledge when learning new tasks, leading to improved performance and
knowledge retention during continual learning processes.

The main contributions of this paper are four-fold:

  * •

A novel framework for Cross-Domain Continual Learning (CDCL) to solve the
unsupervised cross-domain (UDA) task-incremental learning problem, while
maintaining a good stability-plasticity trade-off.

  * •

An inter- intra-task cross-attention mechanism that aligns domain features in
current tasks and consolidates previous alignment knowledge, alleviating the
feature-alignment catastrophic forgetting of related knowledge domains.

  * •

An intra-task-based center-aware pseudo-labeling method that identifies
similar task-specific samples between domains.

  * •

An extensive experimental analysis showcasing the success of applying the
cross-attention mechanism in the cross-domain task-incremental learning
problem, while providing hypotheses towards incremental contributions towards
cross-domain class-incremental learning solutions.

##  II Related work

###  II-A Unsupervised Domain-Adaptation

Domain adaptation seeks to create a knowledge representation in which an
algorithm cannot discern the origin of input data from either the source or
target domain. Specifically, unsupervised domain adaptation (UDA) imposes an
additional constraint, wherein the target domain possesses limited or scarce
labeled data.

UDA methods currently exist in two main categories. The first is domain-level
UDA [32, 14, 10], which aims to create a latent space that is invariant to
domain-specific characteristics. On the other hand, category-level UDA [49,
25] focuses on generating a feature-aligned network between the two domains,
ensuring that their category distributions are closely situated in the latent
space. It is worth noting that category-level UDA methods often outperform
their domain-level counterparts, given that the resulting domain-invariant
latent space allows the categories from both domains to be represented in
close proximity. This proximity facilitates effective knowledge transfer and
adaptation between the domains, leading to enhanced performance in the
category-level UDA methods.

While many computational vision UDA methods still rely on convolutional neural
networks (CNN) architecture, vision transformers (ViT) [16] have recently
emerged as a cutting-edge driver for UDA [49, 12]. Despite their state-of-the-
art performance, one drawback of the ViT architecture is its substantial data
requirements. Consequently, researchers have made several efforts to develop
data-efficient visual transformers. Among these endeavors, the most popular
ones explicitly incorporate convolutional approaches into the transformer
architecture [19, 13, 17, 43]. Of particular interest is the CCT
(Convolutional-Contrastive Training) method [19], which addresses the need for
a class token and positional embedding by employing a sequence pooling
strategy and convolutional operations. This innovation leads to more efficient
data utilization, reducing the data burden for ViT-based UDA approaches. By
integrating convolutional techniques into vision transformers, researchers aim
to strike a balance between data efficiency and performance, paving the way
for more accessible and effective UDA solutions in the realm of computational
vision.

Pseudo-labeling [48, 24] has emerged as a recent catalyst for enhancing the
accuracy of UDA procedures. This approach involves generating synthetic labels
for samples, effectively incorporating confident unlabeled data into the
training set. In the context of domain adaptation, pseudo-labeling techniques
typically leverage the source domain data or a classifier pre-trained on a
related domain, such as ImageNet [15], to serve as the pseudo-label generator
for the target domain data. Several methodologies have been proposed to
leverage pseudo-labeling for domain adaptation, such as: Separating sparse
feature representations [51] \- this technique aims to disentangle the feature
representations between the source and target domains, enabling better
alignment and adaptation of the model to the target domain; Incorporating
regularization terms [11] \- by adding regularization terms to the loss
function, pseudo-labeling encourages the model to generalize better across
domains and reduce overfitting; Fine-tuning [53] \- After pseudo-labels are
assigned to the target domain data, fine-tuning the model using this augmented
dataset allows for domain-specific adaptation, leading to improved performance
on the target domain. These advancements in pseudo-labeling have shown promise
in bridging the domain gap and achieving more accurate and robust UDA
solutions. By harnessing synthetic labels and exploiting the relationship
between different domains, pseudo-labeling contributes to the effective
transfer of knowledge from the source to the target domain, ultimately leading
to improved performance and generalization in UDA tasks.

In order to mitigate the adverse impact of noisy labeling, often referred to
as miss-synthetic labeling, various approaches have been proposed in the
literature. Some works explicitly focus on re-evaluating and correcting
erroneous labels [27], while others concentrate on improving the selection of
unlabeled samples using the confidence of the model [31, 26]. Additionally,
some methods introduce objective functions that exhibit tolerance to labeling
noise [46]. However, these approaches were primarily designed for static
environments and do not address the challenges of continual learning, where a
model faces sequential tasks from source and target domains. Consequently,
they are susceptible to the issue of catastrophic forgetting in such dynamic
scenarios.

To address the limitations of existing methods and tackle cross-domain
continual learning problems, CDCL (Cross-Domain Continual Learning) introduces
an enhancement to the self-supervised technique proposed in [26]. CDCL
augments this approach by incorporating intra-task information to generate
task-relevant pseudo-labels. By leveraging such intra-task information, the
model can better handle the complexities of continual learning in cross-domain
settings, ensuring a more seamless and efficient adaptation to sequential
tasks. CDCL’s ability to handle both cross-domain and continual learning
challenges makes it a promising solution for real-world applications requiring
adaptive and lifelong learning capabilities.

###  II-B Continual learning

Continual learning is the field where a single model, or ensemble of models,
needs to sequentially learn a stream of tasks, whereby previous tasks become
unavailable due to computational constraints, which impossibilities the re-fit
of the whole dataset. Three scenarios [42] were proposed to standardize
obtained results and enable fairer comparisons:

  * •

Task-incremental learning (TIL): The models receive a task identifier during
inference. This is the most straightforward continual learning scenario, and
typical solutions present a ”multi-head” output layer to handle prediction.

  * •

Domain-incremental learning (DIL): The task identifier is unavailable during
inference. However, models do not need to infer the receiving task, as a
single task is requested here, referred to as a domain. Typical solutions
handle structures where the task is always the same, but the input
distribution changes. This is the least explored scenario, as most researchers
classify it as an intermediate stage between TIL and CIL.

  * •

Class-incremental learning (CIL): The task identifier is available only during
training time, and models must be able to solve and predict the receiving task
during inference time. Typical solutions present a ”single-head” output layer,
as the model needs to learn to recognize new classes.

By defining clear evaluation protocols for TIL, DIL, and CIL, the current
field can make significant progress towards developing more robust and
adaptive continual learning algorithms for a wide range of real-world
applications.

Furthermore, existing works are mainly divided into three categories:

  * •

Structure-based approach: This approach tends to evolve the network by
creating new nodes to handle incoming tasks while freezing the rest of the
network parameters [35, 30].

  * •

Regularization-based approach: These models penalize changes to parameters
that are important to previously learned tasks by estimating their importance
[21, 37, 52].

  * •

Rehearsal-based approach: This strategy replays old, or augmented, samples
stored in memory when learning a new task [28, 9]. Strategies to optimize
which samples to be stored in the memory are also typical [1]. DER/DER++ [4]
incorporates previous tasks dark knowledge to align past and current outputs.
HAL [8] adds additional replay objectives, reducing forgetting of key learned
data points. Pseudo-rehearsal is an alternative, where an external generative
model sequentially trains on all tasks and generates pseudo-samples from those
data distributions [38].

Current CL approaches are domain-specific and thus not adaptable to different
but related unlabeled domains. The latter is a challenging problem, where a
stream of tasks is generated from the labeled source domain and the label-free
target domain. The proposed CDCL simultaneously addresses the task-incremental
learning problem over multiple domains.

##  III Problem formulation

Let $(\mathcal{D}_{\mathrm{S}_{i}},\mathcal{D}_{\mathrm{T}_{i}})$ be an data
stream system, with the data tuples
$(\mathbf{x}_{\mathrm{S}_{i}},\mathbf{y}_{\mathrm{S}_{i}})$ satisfying
$(\mathbf{x}_{\mathrm{S}_{i}},\mathbf{y}_{\mathrm{S}_{i}})\stackrel{{\scriptstyle
iid}}{{\sim}}\mathcal{D}_{\mathrm{S}_{i}}(X_{\mathrm{S}},Y_{\mathrm{S}})$, and
$(\mathbf{x}_{\mathrm{T}_{i}})$ satisfying
$(\mathbf{x}_{\mathrm{T}_{i}})\stackrel{{\scriptstyle
iid}}{{\sim}}\mathcal{D}_{\mathrm{T}_{i}}(X_{\mathrm{T}})$. The data tuples
contain a labeled set $X_{\mathrm{S}_{i}}$ paired with its target set
$Y_{\mathrm{S}_{i}}$, along with an unlabeled input set $X_{\mathrm{T}_{i}}$,
organized into sequential tasks $t_{i}\in\mathcal{T}=\\{1,\cdots,T\\}$, where
the total number of tasks $T$ is unknown a priori. The goal is to learn a
predictor
$f:(\mathcal{X}_{\mathrm{S}}\cup\mathcal{X}_{\mathrm{T}})\times\mathcal{T}\rightarrow\mathcal{Y}_{\mathrm{T}}$,
which can be queried at any time to predict the target vector $y_{\mathrm{T}}$
associated to a sample in the target domain $x_{\mathrm{T}}$, where
$(x_{\mathrm{T}},y_{\mathrm{T}})\sim P_{t}$.

The solution to this problem is not trivial, as the model needs to handle
label scarcity in the target domain, and also possibles task drifts
$P_{i}(X_{\mathrm{S}},Y_{\mathrm{S}})\neq
P_{i+1}(X_{\mathrm{S}},Y_{\mathrm{S}})\wedge
P_{i}(X_{\mathrm{T}},Y_{\mathrm{T}})\neq
P_{i+1}(X_{\mathrm{T}},Y_{\mathrm{T}})$ and domain drifts
$P_{t}(X_{\mathrm{S}})\neq P_{t}(X_{\mathrm{T}})\wedge
P_{t}(Y_{\mathrm{S}}|X_{\mathrm{S}})=P_{t}(Y_{\mathrm{T}}|X_{\mathrm{T}})$
along the training.

##  IV Proposed method

Figure 1: The proposed framework. The main contribution - the inter- intra-
task cross attention - is highlighted with purple, which is responsible for
aligning the source and target feature domains, and mitigating its feature-
alignment catastrophic forgetting when new tasks arrive. When the network is
faced with only a single input at a time, source (represented by the dashed
blue line) or target (represented by the blocked red line), CDCL will process
it via the self-attention mechanism (left side block). When the network is
faced with both source and target inputs at the same time, CDCL will process
them via the cross-attention mechanism (right side block), which outputs a
mixed signal (represented by the blocked green open arrow). $\mathbf{b}_{i}$
is omitted for simplicity. The $f^{\text{CIL}}(\cdot)$ is a single-head output
used for CIL scenarios along with the latest $\mathbf{K}_{T}$ and
$\mathbf{b}_{T}$ instantiated. Meanwhile, the $f^{\text{TIL}}(\cdot)$ is a
multi-head output used for TIL scenarios with the respective $\mathbf{K}_{i}$
and $\mathbf{b}_{i}$, as the task-identifier $t_{i}$ is provided.

We propose a cross-domain task-incremental learning framework (CDCL) that
alleviates the catastrophic forgetting of domain invariant features
effectively. Figure 1 presents an overview of the framework. The major
contributions components follow:

1) Inter- intra-task cross-attention in the incremental transformer block
implicitly retain previous tasks information into the attention maps.

2) Intra-task center-aware pseudo-labeling generates synthetic labels for the
incoming unlabeled target inputs based on the intra-task information,
integrating the same into the inter- intra-task cross-attention maps.

The rest of this section gives more details about the inter- intra-task cross-
attention procedure; the intra-task center-aware pseudo-label strategy; the
sample rehearsal configuration; the computational time complexity; and the
upper-bound error of the proposed method.

###  IV-A Inter- intra-task cross-attention mechanism

The proposed cross-attention mechanism is based on the CCT [19] sequential
pooling, an attention-based method that pools over the convolution tokenizer.
Given an image $\mathbf{x}_{i}\in\mathbb{R}^{H\times W\times C}$ presented in
task $t_{i}$:

$\mathbf{x}_{ct}=\textit{MaxPool}(\textit{ReLU}(\textit{Conv2d}(\mathbf{x}_{i})))$
(1)

where Conv2d has $d$ filters, the same number as the embedding dimension of
the transformer backbone - maintaining local spatial information.
$\mathbf{x}_{ct}$ is the outputted convolution, replacing ViT’s patch tokens.
$\mathbf{x}_{ct}$ is then projected into global queries
$\mathbf{Q}\in\mathbb{R}^{n\times d}$ and global values
$\mathbf{V}\in\mathbb{R}^{n\times d}$. Furthermore, $\mathbf{x}_{ct}$ is also
projected into a task-related keys $\mathbf{K}_{i}\in\mathbb{R}^{n\times d}$
and task-related bias $\mathbf{b}_{i}\in\mathbb{R}^{1\times n}$.

The self-attention operation is now applied:

$\mathbf{x}_{Li}=\frac{\mathbf{Q}\mathbf{K}_{i}^{T}+\mathbf{b}_{i}}{\sqrt{d}}\mathbf{V}\in\mathbb{R}^{b\times
n\times d}$ (2)

where $\mathbf{x}_{Li}$ is the output of an $L$ layer transformer encoder, $b$
is the batch size, $n$ is the senquence length, and $d$ is the total embedding
dimension. The cross-attention module is derived from the self-attention
module. The difference is that the input of the cross-attention is a pair of
images, i.e. $\mathbf{x}_{\mathrm{S}_{i}}$ and $\mathbf{x}_{\mathrm{T}_{i}}$.
The cross-attention module can be calculated as follows:

$\mathbf{x}_{Li}=\frac{\mathbf{Q}_{\mathrm{S}}\mathbf{K}_{i}^{T}+\mathbf{b}_{i}}{\sqrt{d}}\mathbf{V}_{\mathrm{T}}\in\mathbb{R}^{b\times
n\times d}$ (3)

where $\mathbf{Q}_{\mathrm{S}}$ are the global queries from image
$\mathbf{x}_{\mathrm{S}_{i}}$ and $\mathbf{V}_{\mathrm{T}}$ are the global
values from image $\mathbf{x}_{\mathrm{T}_{i}}$. Note that
$\mathbf{x}_{\mathrm{T}_{i}}$ convolution token is also projected into the
task-related $\mathbf{K}_{i}$ and $\mathbf{b}_{i}$, producing an inter-task
projection. A new pair $\mathbf{K}_{i}$ and $\mathbf{b}_{i}$ is created when a
new task arrives, while previously learned $\mathbf{K}_{1,\cdots,i}$ and
$\mathbf{b}_{1,\cdots,i}$ are frozen, preserving the knowledge from the
previous feature aligned tasks and building the current feature aligned
knowledge based on the global $\mathbf{Q}$ and $\mathbf{V}$.

The attention maps are used to generate an importance weighting for each input
convolution token:

$\mathbf{x}_{Li}^{\prime}=\text{softmax}(g(\mathbf{x}_{Li})^{T})\in\mathbb{R}^{b\times
1\times n}$ (4)

$\mathbf{z}_{i}=\mathbf{x}_{Li}^{\prime}\mathbf{x}_{Li}=\text{softmax}(g(\mathbf{x}_{Li})^{T})\times\mathbf{x}_{Li}$
(5)

Throughout the paper, this whole operation is simplified by the given
notation:

$a(\mathbf{x}_{i})=\mathbf{z}_{i}\in\mathbb{R}^{b\times 1\times d}$ (6)

The final feature output is generated by flattening
$\mathbf{z}_{i}\in\mathbb{R}^{b\times d}$. These features can be sent through
a classifier for both training and evaluation. CDCL evaluates $\mathbf{z}_{i}$
in both inter-task (CIL) and intra-task (TIL) manner:

$\mathbf{y}^{\text{TIL}}=f^{\text{TIL}}(z_{i})\in\mathbb{R}^{b\times u}$ (7)

$\mathbf{y}^{\text{CIL}}=f^{\text{CIL}}(z_{i})\in\mathbb{R}^{b\times u}$ (8)

where $u$ is the number of classes in the current task. From here, the
following objective functions are applied:

$\mathcal{L}_{\mathrm{S}}^{\text{CIL}}=-\sum\mathbf{y_{\mathrm{S}}}\log(\mathbf{y}_{\mathrm{S}}^{\text{CIL}})\in\mathbb{R}^{b}$
(9)

$\mathcal{L}_{\mathrm{T}}^{\text{CIL}}=-\sum\mathbf{y_{\mathrm{S}}}\log(\mathbf{y}_{\mathrm{T}}^{\text{CIL}})\in\mathbb{R}^{b}$
(10)

Input: $\mathcal{D}_{\mathrm{S}},\mathcal{D}_{\mathrm{T}}$

Output:
$f:(\mathbf{x}_{\mathrm{S}}\cup\mathbf{x}_{\mathrm{T}})\times\mathcal{T}\rightarrow\mathbf{y}_{\mathrm{T}}$

1

2for  _$t_{i}$ in $\mathcal{T}$_ do

3
$\mathbf{x}_{\mathrm{S}},\mathbf{y}_{\mathrm{S}}\leftarrow\mathcal{D}_{\mathrm{S}_{i}}$;

4 $\mathbf{x}_{\mathrm{T}}\leftarrow\mathcal{D}_{\mathrm{T}_{i}}$;

5 Random initialize $K_{i}$ and $b_{i}$;

6 for  _epoch in epochs_ do

7
$\mathcal{L}^{\text{CIL}}\leftarrow\mathcal{L}^{\text{TIL}}\leftarrow\mathcal{L}_{\mathrm{R}}\leftarrow
0$;

8 if  _epoch $<$ warm-up-epochs_ then

9
$\mathcal{L}_{\mathcal{S}}^{\text{CIL}}\leftarrow-\sum\mathbf{y_{\mathrm{S}}}\log(\mathbf{y}_{\mathrm{S}}^{\text{CIL}})$;
# Eq.9

10
$\mathcal{L}_{\mathcal{S}}^{\text{TIL}}\leftarrow-\sum\mathbf{y_{\mathrm{S}}}\log(\mathbf{y}_{\mathrm{S}}^{\text{TIL}})$;
# Eq.12

11

12 else

13 Compute $\mathbf{c}_{k}$ and $\hat{\mathbf{y}}_{\mathrm{T}}$; # Eqs. 17 and
18

14 Compute $\mathbb{P}$; # Eq. 19

15
$\mathcal{L}^{\text{CIL}}\leftarrow\mathcal{L}_{\mathrm{S}}^{\text{CIL}}+\mathcal{L}_{\mathrm{T}}^{\text{CIL}}+\mathcal{L}_{\mathrm{D}}^{\text{CIL}}$;
# Eq.15

16
$\mathcal{L}^{\text{TIL}}\leftarrow\mathcal{L}_{\mathrm{S}}^{\text{TIL}}+\mathcal{L}_{\mathrm{T}}^{\text{TIL}}+\mathcal{L}_{\mathrm{D}}^{\text{TIL}}$;
# Eq.16

17 if  _$t_{i} >1$_ then

18
$\mathcal{L}_{R}\leftarrow\mathcal{L}_{R}^{\mathrm{ST}}+\mathcal{L}_{R}^{\mathrm{D}}+\mathcal{L}_{R}^{\mathrm{Z}}$;
# Eq.23

19

20

21
$\mathcal{L}\leftarrow\mathcal{L}^{\text{CIL}}+\mathcal{L}^{\text{TIL}}+\mathcal{L}_{\mathrm{R}}$;

22 $\theta\leftarrow\theta-\lambda\nabla_{\theta}\mathcal{L}$; # Update model
parameters

23 Stores $\lfloor|\mathcal{M}|/t_{i}\rfloor$ records per task in memory
$\mathcal{M}$;

24

Algorithm 1 CDCL learning pseudo-algorithm

$\mathcal{L}_{\mathrm{D}}^{\text{CIL}}=\sum\mathbf{y}_{\mathrm{S+T}}^{\text{CIL}}\log(\mathbf{y}_{\mathrm{T}}^{\text{CIL}})\in\mathbb{R}^{b}$
(11)

where $\mathbf{y_{\mathrm{S}}}$ is the label vector from the source domain,
$\mathbf{y}_{\mathrm{S}}^{\text{CIL}},\mathbf{y}_{\mathrm{T}}^{\text{CIL}}$
are respectively the output vector of the source input and target input
through the intra-task classifier, and
$\mathbf{y}_{\mathrm{S+T}}^{\text{CIL}}$ is the output vector of the inter-
intra-task cross-attention applied to both the source and target image vector
through the intra-task classifier, i.e., they map the source and target
classes information between the tasks. Similar objectives are optimized
through the inter-task classifier, which maps the source and target classes
information within a task:

$\mathcal{L}_{\mathrm{S}}^{\text{TIL}}=-\sum\mathbf{y_{\mathrm{S}}}\log(\mathbf{y}_{\mathrm{S}}^{\text{TIL}})\in\mathbb{R}^{b}$
(12)

$\mathcal{L}_{\mathrm{T}}^{\text{TIL}}=-\sum\mathbf{y_{\mathrm{S}}}\log(\mathbf{y}_{\mathrm{T}}^{\text{TIL}})\in\mathbb{R}^{b}$
(13)

$\mathcal{L}_{\mathrm{D}}^{\text{TIL}}=\sum\mathbf{y}_{\mathrm{S+T}}^{\text{TIL}}\log(\mathbf{y}_{\mathrm{T}}^{\text{TIL}})\in\mathbb{R}^{b}$
(14)

We will refer to the inter-task and intra-task losses as single objectives
throughout the paper:

$\mathcal{L}^{\text{CIL}}=\mathcal{L}_{\mathrm{S}}^{\text{CIL}}+\mathcal{L}_{\mathrm{T}}^{\text{CIL}}+\mathcal{L}_{\mathrm{D}}^{\text{CIL}}\in\mathbb{R}^{b}$
(15)

$\mathcal{L}^{\text{TIL}}=\mathcal{L}_{\mathrm{S}}^{\text{TIL}}+\mathcal{L}_{\mathrm{T}}^{\text{TIL}}+\mathcal{L}_{\mathrm{D}}^{\text{TIL}}\in\mathbb{R}^{b}$
(16)

###  IV-B Intra-task center-aware pseudo-label

To ensure a feature alignment on correct samples, we build a set $\mathbb{P}$
containing pairs of similar inputs from the source and target domain.
$\mathbb{P}$ is built with both feature similarity and label similarity in
mind. To improve inter-task alignment, we increment [26] by adding intra-task
knowledge into their center-aware pseudo-label mechanism, i.e., we use only
information extracted from the current task to build the current task
centroids. When a new task arrives, a warm-up stage starts where the
classifiers will train solely in the source domain inputs. After the warm-up
stage finishes, we build category centroids in the target domain by weighted
k-means clustering based on the intra-task classification information,
updating it at every training epoch111We recreate the centroids at every
training epoch.:

$\mathbf{c}_{k}=\frac{\sum_{\mathcal{D}_{\mathrm{T}_{i}}}\mathbf{y}_{\mathrm{T}}^{\text{TIL}}a(\mathbf{x}_{\mathrm{T}})}{\sum_{\mathcal{D}_{\mathrm{T}_{i}}}\mathbf{y}_{\mathrm{T}}^{\text{TIL}}}$
(17)

where $\mathbf{y}_{\mathrm{T}}^{\text{TIL}}$ corresponds to the intra-task
prediction on the target input samples and $k$ refers to the number of classes
in the current task. Pseudo-labels of the target data are produced via the
nearest neighbor classifier:

$\hat{\mathbf{y}}_{\mathrm{T}}=\arg\min_{k}d(\mathbf{c}_{k},a(\mathbf{x}_{\mathrm{T}}))$
(18)

where $d(\cdot,\cdot)$ is a distance metric, as cosine similarity or Euclidean
distance. Finally, the set $\mathbb{P}$ is updated accordingly to the pseudo-
labels, discarding noise and only allowing paired records that match the
source input label with the target-generated pseudo-label:

$\displaystyle\mathbb{P}=\\{(x_{S},y_{S},x_{T})|$ $\displaystyle\forall
x_{S}\in a(X_{S}),$ (19) $\displaystyle\forall x_{T}\in a(X_{T}),$
$\displaystyle\arg\min_{(x_{S},y_{S},x_{T})}d(x_{S},x_{T})\wedge
y_{S}=\hat{y}_{T}\\}$

###  IV-C Sample rehearsal

Although neural networks are susceptible to catastrophic forgetting, it is
believed that some residual from past experiences can still be found within
its weights. Therefore, rehearsing samples is an effective strategy to drive
these residuals toward previous knowledge presentation. However, memory size
is constrained by having a fixed size or growing very slowly with the number
of incoming tasks. Therefore, it is important to select samples that better
describe previous feature alignment knowledge, hence having a better
regeneration over residuals. At the end of the training of the current task
$t_{i}$, $\lfloor|\mathcal{M}|/t_{i}\rfloor$ records222Each record consist of
the tuple
($\mathbf{x}_{\mathrm{S}},\mathbf{x}_{\mathrm{T}},\mathbf{y}_{\mathrm{S}},\mathbf{y}_{\mathrm{S}}^{\text{CIL}},\mathbf{y}_{\mathrm{T}}^{\text{CIL}})$
in memory with respective notation
$(\mathbf{x}_{\mathrm{S}}^{\mathrm{R}},\mathbf{x}_{\mathrm{T}}^{\mathrm{R}},\mathbf{y}^{\mathrm{R}},\mathbf{y}_{\mathrm{S}}^{\mathrm{R}},\mathbf{y}_{\mathrm{T}}^{\mathrm{R}})$
where $\mathrm{R}$ denotes ”Rehearsal”. with highest intra-task confidence,
$max(\mathbf{y}_{S}^{TIL})\vee\max(\mathbf{y}_{T}^{TIL})$, are stored in the
memory $\mathcal{M}$.

To preserve the knowledge about previous tasks, we seek to minimize the
following losses:

$\mathcal{L}_{R}^{\mathrm{ST}}=-\sum\mathbf{y^{\mathrm{R}}}\log(f^{\text{CIL}}(a(\mathbf{x}_{\mathrm{S}}^{\mathrm{R}}))f^{\text{CIL}}(a(\mathbf{x}_{\mathrm{T}}^{\mathrm{R}})))\in\mathbb{R}^{b}$
(20)

$\mathcal{L}_{R}^{D}=\sum
f^{\text{CIL}}(a(\mathbf{x}_{\mathrm{S}}^{\mathrm{R}}),a(\mathbf{x}_{\mathrm{T}}^{\mathrm{R}}))f^{\text{CIL}}(a(\mathbf{x}_{\mathrm{T}}^{\mathrm{R}}))\in\mathbb{R}^{b}$
(21)

where
$\mathbf{x}_{\mathrm{S}}^{\mathrm{R}},\mathbf{x}_{\mathrm{T}}^{\mathrm{R}},\mathbf{y}^{\mathrm{R}}$
are respectively the source image, target image and original source labels
recorded in the memory $\mathcal{M}$. Additionally, CDCL also attempts to
maintain previously learned task boundaries by logit replay:

$\mathcal{L}_{R}^{\mathrm{Z}}=\sum\mathbf{y}_{\mathrm{S}}^{\mathrm{R}}\log(\frac{\mathbf{y}_{\mathrm{T}}^{\mathrm{R}}}{f^{\text{CIL}}(a(\mathbf{x}_{\mathrm{T}}^{\mathrm{R}}))}\frac{\mathbf{y}_{\mathrm{S}}^{\mathrm{R}}}{f^{\text{CIL}}(a(\mathbf{x}_{\mathrm{S}}^{\mathrm{R}}))})\in\mathbb{R}^{b}$
(22)

where
$\mathbf{y}_{\mathrm{S}}^{\mathrm{R}},\mathbf{y}_{\mathrm{T}}^{\mathrm{R}}$
are respectively the source logits and target logits recorded in the memory
$\mathcal{M}$.

Finally, we consolidated the rehearsal objective333Note that the pseudo-label
centroid were built using intra-task knowledge (information within a task),
but the rehearsal loss apply it via inter-task outputs (information between
all tasks) as:

$\mathcal{L}_{R}=\mathcal{L}_{R}^{\mathrm{ST}}+\mathcal{L}_{R}^{\mathrm{D}}+\mathcal{L}_{R}^{\mathrm{Z}}\in\mathbb{R}^{b}$
(23)

Algorithm 1 depicts a simplified representation of how and when CDCL optimizes
each of its objectives.

###  IV-D Time Complexity Analysis

The computational time complexity of the proposed method is derived into two
main procedures: the convolution tokenizer operation; and the inter- intra-
task cross-attention operation.

The convolution tokenizer in equation 1 performs $O(n)$ operations per layer
$L_{c}$, where $n$ is the input feature space size of $x_{i}$. A cross-
attention operation has a linear transformation of its input into the three
projections $\mathbf{Q},\mathbf{V},$ and $\mathbf{K}$, followed by a post-
multiplication of shape $(d\times d)$, resulting in $O(nd^{2})$. Computing the
output of the attention mechanism following equation 3 has complexity
$O(dn^{2})$ to compute
$\frac{\mathbf{Q}\mathbf{K}_{i}^{T}+\mathbf{b}_{i}}{\sqrt{d}}$, and
$O(nd^{2})$ for the post-multiplication with $\mathbf{V}$, repeated by $L_{a}$
layers. Therefore, a single forward-pass time complexity of CDCL is defined
by:

$O(\underbrace{nL_{c}}_{\text{Conv
Token}}+\underbrace{(dn^{2}+nd^{2})L_{a}}_{\text{Cross-attentions}})$ (24)

This time complexity equation doesn’t take into consideration the impact of
general-purpose hardware architectures in the algorithm, which explains how
transformers (and consequently CDCL) run faster than previousnetworks, even
though it presents a higher computational complexity [44].

###  IV-E Theoretical analysis

In this section, we describe the error bound for the general cross-domain
continual learning problem and how it influenced the proposed method
architecture:

######  Theorem 1.

Theorem 1: Given two domain distributions
$\mathcal{D}_{\mathrm{S}}(X_{\mathrm{S}})$ and
$\mathcal{D}_{\mathrm{T}}(X_{\mathrm{T}})$, the target domain error
$\varepsilon_{\mathrm{T}}$ is bound by [2]:

$\small
d_{\mathcal{H}\Delta\mathcal{H}}(X_{\mathrm{S}},X_{\mathrm{T}})=\underset{\eta\in\mathcal{H}}{2sup}\left|{P[\eta(X_{\mathrm{S}})\\!=\\!1]}-{P[\eta(X_{\mathrm{T}})\\!=\\!1]}\right|$
(25)

$\varepsilon_{\mathrm{T}}\leq\varepsilon_{\mathrm{S}}+d_{\mathcal{H}\Delta\mathcal{H}}(X_{\mathrm{S}},X_{\mathrm{T}})+C^{*}$
(26)

where $d_{\mathcal{H}\Delta\mathcal{H}}(X_{\mathrm{S}},X_{\mathrm{T}})$ is the
$\mathcal{H}\Delta\mathcal{H}$ divergence, which relies on the capacity of the
hypothesis class $\mathcal{H}$ to distinguish between examples generated by
$\mathcal{D}_{\mathrm{S}}(X_{\mathrm{S}})$ from examples generated by
$\mathcal{D}_{\mathrm{T}}(X_{\mathrm{T}})$. $C^{*}$ is the error of an optimal
classifier for both source domain $\mathcal{D}_{\mathrm{S}}$ and target domain
$\mathcal{D}_{\mathrm{T}}$.

######  Theorem 2.

Theorem 2: For every new sequential task
$t_{i}\in\mathcal{T}=\\{1,\cdots,T\\}$ and incoming domain distributions
$\mathcal{D}_{\mathrm{S}_{i}}(X_{\mathrm{S}})$ and
$\mathcal{D}_{\mathrm{T}_{i}}(X_{\mathrm{T}})$, let
$\lambda_{i}=d_{\mathcal{H}\Delta\mathcal{H}}(\mathbf{z}_{\mathrm{S}_{i}},\mathbf{z}_{\mathrm{T}_{i}})$
be the domain discrepancy of the feature distributions
$\mathbf{z}_{\mathrm{S}_{i}}$ and $\mathbf{z}_{\mathrm{T}_{i}}$. The target
domain error $\varepsilon_{\mathrm{T}_{i}}$ at task $t_{i}$ is bound by:

$\varepsilon_{\mathrm{T}_{i}}\leq\varepsilon_{\mathrm{S}_{i}}+\lambda_{i}+C_{i}^{*}$
(27)

where
$C_{i}^{*}=\min_{\theta}(\varepsilon_{\mathrm{S}_{i}}+\varepsilon_{\mathrm{T}_{i}})$
is the error of an optimal classifier for both source domain
$\mathcal{D}_{\mathrm{S}_{i}}$ and target domain
$\mathcal{T}_{\mathrm{S}_{i}}$ at task $t_{i}$.

######  Theorem 3.

Theorem 3: For every new sequential task $t\in\mathcal{T}=\\{1,\cdots,T\\}$
and incoming domain distributions
$\mathcal{D}_{\mathrm{S}_{i}}(X_{\mathrm{S}})$ and
$\mathcal{D}_{\mathrm{T}_{i}}(X_{\mathrm{T}})$, let
$\lambda_{i}=d_{\mathcal{H}\Delta\mathcal{H}}(\mathbf{z}_{\mathrm{S}_{i}},\mathbf{z}_{\mathrm{T}_{i}})$
be the domain discrepancy of the feature distributions
$\mathbf{z}_{\mathrm{S}_{i}}$ and $\mathbf{z}_{\mathrm{T}_{i}}$, and
$P_{\mathrm{R}_{i}},P_{\mathcal{M}_{i}}$ denote respectively the conditional
distributions of labels $\mathbf{y}_{\mathrm{S}_{i}}$ given the raw features
$\mathbf{z}_{\mathrm{S}_{i}}$ and memory stored features
$\mathbf{z}_{\mathcal{M}_{i}}$. The total error of the target domain error
$\varepsilon_{\mathrm{T}}$ at task $t$ is bound by:

$\varepsilon_{\mathrm{T}}\leq\sum_{i=1}^{t}(\varepsilon_{\mathrm{S}_{i}}+\lambda_{i})+\sum_{i=1}^{t-1}\text{KL}(P_{\mathcal{M}_{i}}||P_{\mathrm{R}_{i}})+C^{*}$
(28)

where
$C^{*}=\min_{\theta}\sum_{i=1}^{t}(\varepsilon_{\mathrm{S}_{i}}+\varepsilon_{\mathrm{T}_{i}})$
is the error of an optimal classifier for both source domain
$\mathcal{D}_{\mathrm{S}_{\mathcal{T}}}$ and target domain
$\mathcal{D}_{\mathrm{T}_{\mathcal{T}}}$ across all tasks. Here, we choose the
KL-divergence due to its simplification and similarity to other common
information theory equations, but any divergence equation following Eq. 26 can
be used.

Theorem 2 assists us to prove theorem 3, which is proved by [22] for a similar
problem and adapted here.

###### Proof.

Proof 1: At task $t$, the error of previous learned target experiences
$\mathcal{D}_{\mathrm{T}_{i}}(X_{\mathrm{T}})$ for $i\in\\{1,\cdots,t-1\\}$ is
estimated by $\hat{\varepsilon_{\mathrm{T}_{i}}}$ since we rehearse
$\mathbf{z}_{\mathcal{M}_{i}}$ to mimic $\mathbf{z}_{\mathcal{S}_{i}}$ during
training. The total error of the target domain error
$\varepsilon_{\mathrm{T}}$ at task $t_{i}$ is:

$\displaystyle\varepsilon_{\mathrm{T}}$
$\displaystyle=\varepsilon_{\mathrm{T}_{t}}+\sum_{i=1}^{t-1}\hat{\varepsilon_{\mathrm{T}_{i}}}$
(29)
$\displaystyle\leq\varepsilon_{\mathrm{S}_{t}}+\lambda_{t}+\sum_{i=1}^{t-1}(\hat{\varepsilon_{\mathrm{S}_{i}}}+\lambda_{i})+C^{*}\cdot$

Additionally, the divergence between $P_{\mathrm{R}_{i}}$ and
$P_{\mathcal{M}_{i}}$ can be understood as:

KL
$\displaystyle\big{(}P_{\mathcal{M}_{i}}(\mathbf{y}_{\mathrm{S}_{i}}|\mathbf{z}_{\mathcal{M}_{i}})||P_{\mathrm{R}_{i}}(\mathbf{y}_{\mathrm{S}_{i}}|\mathbf{z}_{\mathrm{S}_{i}})\big{)}$
(30)
$\displaystyle=\sum_{y\in\mathbf{y}}P_{\mathcal{M}_{i}}(\mathbf{y}_{\mathrm{S}_{i}}|\mathbf{z}_{\mathcal{M}_{i}})\log\frac{P_{\mathcal{M}_{i}}(\mathbf{y}_{\mathrm{S}_{i}}|\mathbf{z}_{\mathcal{M}_{i}})}{P_{\mathrm{R}_{i}}(\mathbf{y}_{\mathrm{S}_{i}}|\mathbf{z}_{\mathrm{S}_{i}})}$
$\displaystyle=\mathbb{E}\Big{[}\log\frac{P_{\mathcal{M}_{i}}(\mathbf{y}_{\mathrm{S}_{i}}|\mathbf{z}_{\mathcal{M}_{i}})}{P_{\mathrm{R}_{i}}(\mathbf{y}_{\mathrm{S}_{i}}|\mathbf{z}_{\mathrm{S}_{i}})}\Big{]}$
$\displaystyle=\mathbb{E}[\log{P_{\mathcal{M}_{i}}(\mathbf{y}_{\mathrm{S}_{i}}|\mathbf{z}_{\mathcal{M}_{i}})}]-\mathbb{E}[\log{P_{\mathrm{R}_{i}}(\mathbf{y}_{\mathrm{S}_{i}}|\mathbf{z}_{\mathrm{S}_{i}})}]$
$\displaystyle=\hat{\varepsilon_{\mathrm{S}_{i}}}-\varepsilon_{\mathrm{S}_{i}}\cdot$

Hence, we can estimate the error for each source domain
$\varepsilon_{\mathrm{S}_{i}}$ in a specific task $t$ as:

$\varepsilon_{\mathrm{S}_{t}}=\hat{\varepsilon_{\mathrm{S}_{t}}}-\text{KL}\big{(}P_{\mathcal{M}_{t}}||P_{\mathrm{R}_{t}}\big{)}$
(31)

Finally, we can estimate the total error bound for the target domain
$\varepsilon_{\mathrm{T}}$ at a task $t$ by:

$\displaystyle\varepsilon_{\mathrm{T}}$
$\displaystyle\leq\varepsilon_{\mathrm{S}_{t}}+\lambda_{t}+\sum_{i=1}^{t-1}(\hat{\varepsilon_{\mathrm{S}_{i}}}+\lambda_{i})+C^{*}$
(32)
$\displaystyle=\varepsilon_{\mathrm{S}_{t}}+\lambda_{t}+\sum_{i=1}^{t-1}(\varepsilon_{\mathrm{S}_{i}}+\text{KL}\big{(}P_{\mathcal{M}_{i}}||P_{\mathrm{R}_{i}}\big{)}+\lambda_{i})+C^{*}$
$\displaystyle=\sum_{i=1}^{t}(\varepsilon_{\mathrm{S}_{i}}+\lambda_{i})+\sum_{i=1}^{t-1}\text{KL}\big{(}P_{\mathcal{M}_{i}}||P_{\mathrm{R}_{i}}\big{)}+C^{*}\cdot$

∎

Finally, we can use theorem 3 to explain the objective choices in CDCL based
on proof 1:
$\mathcal{L}_{\mathrm{T}}^{\text{CIL}},\mathcal{L}_{\mathrm{D}}^{\text{CIL}},\mathcal{L}_{\mathrm{T}}^{\text{TIL}},\mathcal{L}_{\mathrm{D}}^{\text{TIL}}$
contribute to the creation of a feature-aligned domain-invariant latent space,
being related to $\lambda_{i}$; $\mathcal{L}_{\mathrm{R}}$ is related to the
KL term, which is minimized by the samples chosen by the intra-task center-
aware pseudo-label approach; and finally,
$\mathcal{L}_{\mathrm{S}}^{\text{CIL}},\mathcal{L}_{\mathrm{S}}^{\text{TIL}}$
fits CDCL in the source domain, decreasing the incremental source domain error
rate $\varepsilon_{\mathrm{S}_{t}}$. $C^{*}$ represents how suitable the model
architecture is to find an optimal solution for both data domains.

##  V Experiments

###  V-A Benchmarks

The proposed method is verified in five popular UDA benchmarks, including
VisDA-2017 [34], Office-Home [45], Office-31 [36], DomainNet [33], and
MNIST$\leftrightarrow$USPS [23, 20].

MNIST$\leftrightarrow$USPS: Gray-scale images of hand-written digits. The 10
classes are split into 5 tasks of 2 classes each.

VisDA-2017: Consists of a source dataset of synthetic 2D renderings of 3D
models generated from different angles and with different lighting conditions
and a target dataset of photo-realistic images. The 12 classes are split into
4 tasks of 3 classes each.

DomainNet: Contains 345 classes of ordinary objects, represented in six
domains: Clipart (clp), Infographics (inf), Painting (pnt), Quickdraw (qdr),
Real (rel), and Sketch (skt). The 345 classes are divided into 15 tasks of 23
classes each.

Office31: Contains 31 object categories in three domains: Amazon (A), DSLR
(D), and Webcam (W). The 31 categories in the dataset consist of objects
commonly encountered in office settings. To ease the baseline setup, the
category ”trash can” was dropped, making this a problem of 30 classes divided
into 5 tasks of 6 classes each.

Office-Home: Consists of 65 categories on four domains: Art (Ar), Clipart
(Cl), Product (Pr), and Real-World (Re). The 65 classes are divided into 13
tasks of 5 classes each.

###  V-B Baselines

We compare our method with the state-of-the-art method on UDA tasks, CDTrans-S
and CDTrans-B [49], as well as with state-of-the-art methods in continual
learning, such as DER [4], DER++ [4], and HAL [8]. We also compare our method
against MLS [39], which perform supervised cross-domain continual learning.
Finally, we showcase the performance of the Transferable Vision Transformer
(TVT) [50], another state-of-the-art UDA method, as a full trainig upper bound
baseline, clearly depicting the catastrophic forgetting gap between static
learning and continual learning.

We created two instances of CDCL for the experiments: A small-sized instance
for the MNIST$\leftrightarrow$USPS experiments and a larger one for all the
other benchmarks. The small version of CDCL had 7 transformer encoder layers,
a 2-layer convolution tokenizer with a 7x7 kernel size, and a convolution
input feature space of size 28x28x1. The larger version had 14 transformer
encoder layers, a 2-layer convolution tokenizer with a 7x7 kernel size, and a
convolution input feature space of size 224x224x3.

All baselines were executed over 125 epochs (25 warm-up epochs and 25 cooldown
epochs) with a maximum and fixed memory size of 1000 records. CDTrans-S and
CDTrans-B were performed with the same hyper-parameters described in the
original paper [49]. DER, DER++, and HAL were performed with the best hyper-
parameters described in the Mammoth library [3]. CDCL uses AdamW optimizer
[29] with a warm-up learning-rate $\lambda=10^{-5}$, a cosine annealing
learning-rate starting at $\lambda=5*10^{-5}$ and a minimum learning-rate of
$\lambda=10^{-6}$. All experiments were executed in an NVIDIA GeForce RTX
2080Ti with 11GB of VRAM.

###  V-C Metrics

The continual learning protocol is followed, where we observe two metrics:

$\small\textbf{\text{Average Accuracy:
}}\text{ACC}_{\uparrow}=\frac{1}{T}\sum_{i=1}^{T}R_{T,i}$ (33)

$\small\textbf{\text{Forgetting:
}}\text{FGT}_{\downarrow}=\frac{1}{T-1}\sum_{i=1}^{T-1}\max\limits_{i\in{1,\cdots,T-1}}(R_{i,i}-R_{T,i})$
(34)

where $R\in\mathbb{R}^{T\times T}$ is a test classification matrix on the
target domain, with $R_{i,j}$ representing the accuracy in task $t_{j}$ after
thoroughly learning $t_{i}$. The details are given by [28, 7].

###  V-D Numerical results

Method |  A$\to$D |  A$\to$W |  D$\to$A |  D$\to$W |  W$\to$A |  W$\to$D |  MN$\to$US |  US$\to$MN |  VisDA-2017
---|---|---|---|---|---|---|---|---|---
DER |  TIL |  4.45 |  4.20 |  3.99 |  11.50 |  4.29 |  9.14 |  62.79 |  73.16 |  11.43
DER++ |  |  4.21 |  4.18 |  4.21 |  8.51 |  3.74 |  8.42 |  79.81 |  73.24 |  10.24
HAL |  |  4.62 |  5.02 |  3.93 |  5.77 |  3.70 |  5.28 |  80.97 |  73.38 |  9.84
MSL |  |  4.75 |  4.10 |  3.89 |  11.65 |  4.67 |  9.23 |  63.01 |  71.43 |  12.21
CDTrans-S |  |  2.50 |  5.20 |  4.50 |  5.20 |  4.50 |  2.50 |  10.78 |  10.68 |  8.90
CDTrans-B |  |  5.96 |  3.90 |  4.70 |  3.60 |  4.70 |  5.80 |  8.77 |  9.42 |  7.85
Ours (ACC) |  |  26.22 |  22.43 |  28.74 |  55.44 |  26.54 |  53.21 |  91.91 |  81.48 |  40.80
Ours (FGT) |  |  5.95 |  6.86 |  0.33 |  6.38 |  3.41 |  9.38 |  7.38 |  10.22 |  7.94
DER |  CIL |  2.07 |  3.62 |  5.23 |  24.81 |  7.34 |  26.50 |  29.71 |  29.34 |  10.09
DER++ |  |  2.07 |  3.75 |  6.36 |  11.11 |  4.43 |  19.25 |  40.82 |  33.06 |  9.32
HAL |  |  4.35 |  5.81 |  3.23 |  5.43 |  4.43 |  11.18 |  29.08 |  25.94 |  8.50
MSL |  |  3.23 |  4.42 |  4.43 |  25.23 |  8.01 |  24.20 |  30.22 |  30.56 |  12.23
Ours (ACC) |  |  9.68 |  10.98 |  7.02 |  27.97 |  6.73 |  28.98 |  66.73 |  52.50 |  10.16
Ours (FGT) |  |  4.14 |  4.37 |  1.19 |  8.30 |  1.54 |  7.20 |  37.03 |  27.81 |  26.20
TVT (Static UDA) |  96.39 |  96.35 |  84.91 |  99.37 |  86.05 |  100.0 |  98.26 |  99.70 |  83.92
TABLE I: Comparison with SoTA methods’ ACC on Office-31, MNIST$\leftrightarrow$USPS, and VisDA-2017. The best result in each experiment is highlighted with bold font, while the second-best is highlighted with underline font. Method |  Ar$\to$Cl |  Ar$\to$Pr |  Ar$\to$Re |  Cl$\to$Ar |  Cl$\to$Pr |  Cl$\to$Re |  Pr$\to$Ar |  Pr$\to$Cl |  Pr$\to$Re |  Re$\to$Ar |  Re$\to$Cl |  Re$\to$Pr
---|---|---|---|---|---|---|---|---|---|---|---|---
DER |  TIL |  2.26 |  2.38 |  2.93 |  2.29 |  3.37 |  3.09 |  2.31 |  2.62 |  3.18 |  3.75 |  1.77 |  1.72
DER++ |  |  2.25 |  2.40 |  2.77 |  2.60 |  3.34 |  3.26 |  2.32 |  2.51 |  3.12 |  4.02 |  3.41 |  3.56
HAL |  |  2.08 |  2.10 |  2.53 |  2.36 |  2.84 |  2.83 |  2.07 |  2.42 |  2.55 |  3.43 |  3.07 |  3.40
MSL |  |  2.26 |  2.35 |  4.01 |  4.02 |  3.94 |  3.65 |  3.34 |  3.32 |  3.72 |  4.02 |  2.24 |  2.14
CDTrans-S |  |  1.50 |  1.80 |  1.50 |  1.50 |  1.80 |  1.50 |  1.50 |  1.50 |  1.50 |  1.50 |  1.50 |  1.80
CDTrans-B |  |  1.00 |  1.20 |  1.40 |  1.40 |  1.20 |  1.40 |  1.40 |  1.00 |  1.65 |  1.70 |  1.12 |  1.20
Ours (ACC) |  |  24.44 |  25.18 |  26.20 |  21.25 |  26.64 |  23.54 |  22.89 |  24.21 |  29.44 |  26.25 |  26.27 |  31.25
Ours (FGT) |  |  7.33 |  8.17 |  10.97 |  10.90 |  9.93 |  11.06 |  9.05 |  10.32 |  13.89 |  12.75 |  11.05 |  19.17
DER |  CIL |  4.40 |  4.75 |  8.45 |  4.86 |  10.63 |  9.30 |  4.45 |  6.58 |  9.55 |  5.00 |  2.27 |  2.21
DER++ |  |  3.89 |  4.93 |  6.47 |  5.52 |  11.11 |  11.06 |  4.49 |  5.50 |  8.45 |  6.84 |  4.00 |  5.23
HAL |  |  4.01 |  3.27 |  6.24 |  4.70 |  7.16 |  7.28 |  3.38 |  5.18 |  4.66 |  3.54 |  3.03 |  4.74
MSL |  |  4.02 |  4.35 |  6.02 |  4.70 |  6.44 |  6.82 |  4.40 |  5.10 |  7.12 |  4.42 |  3.64 |  3.66
Ours (ACC) |  |  4.15 |  4.77 |  6.01 |  5.56 |  8.32 |  6.35 |  3.92 |  5.02 |  6.29 |  6.45 |  6.26 |  8.35
Ours (FGT) |  |  10.36 |  8.15 |  13.52 |  10.15 |  12.76 |  11.78 |  11.96 |  11.48 |  21.80 |  19.45 |  13.06 |  27.14
TVT (Static UDA) |  74.89 |  86.82 |  89.47 |  82.78 |  87.95 |  88.27 |  79.81 |  71.94 |  90.13 |  85.46 |  74.62 |  90.56
TABLE II: Comparison with SoTA methods’ ACC on Office-Home. The best result in each experiment is highlighted with bold font, while the second-best is highlighted with underline font. DER |  clp |  inf |  pnt |  qdr |  rel |  skt | DER |  clp |  inf |  pnt |  qdr |  rel |  skt
---|---|---|---|---|---|---|---|---|---|---|---|---|---
TIL |  clp |  - |  0.53 |  0.55 |  0.49 |  0.51 |  0.53 |  CIL |  clp |  - |  1.29 |  1.32 |  1.21 |  1.32 |  1.31
|  inf |  0.55 |  - |  0.56 |  0.52 |  0.52 |  0.51 |  |  inf |  1.26 |  - |  1.32 |  1.32 |  1.24 |  1.32
|  pnt |  0.54 |  0.52 |  - |  0.54 |  0.49 |  0.50 |  |  pnt |  1.32 |  1.33 |  - |  1.27 |  1.31 |  1.26
|  qdr |  0.53 |  0.52 |  0.51 |  - |  0.49 |  0.52 |  |  qdr |  1.26 |  1.30 |  1.29 |  - |  1.19 |  1.26
|  rel |  0.53 |  0.53 |  0.50 |  0.49 |  - |  0.51 |  |  rel |  1.29 |  1.27 |  1.22 |  1.23 |  - |  1.23
|  skt |  0.51 |  0.52 |  0.51 |  0.52 |  0.53 |  - |  |  skt |  1.32 |  1.29 |  1.20 |  1.30 |  1.32 |  -
DER++ |  clp |  inf |  pnt |  qdr |  rel |  skt | DER++ |  clp |  inf |  pnt |  qdr |  rel |  skt
TIL |  clp |  - |  0.55 |  0.55 |  0.48 |  0.56 |  0.57 |  CIL |  clp |  - |  1.30 |  1.34 |  1.18 |  1.24 |  1.27
|  inf |  0.55 |  - |  0.53 |  0.55 |  0.52 |  0.52 |  |  inf |  1.23 |  - |  1.31 |  1.34 |  1.28 |  1.28
|  pnt |  0.52 |  0.52 |  - |  0.56 |  0.57 |  0.54 |  |  pnt |  1.35 |  1.30 |  - |  1.24 |  1.26 |  1.23
|  qdr |  0.57 |  0.58 |  0.57 |  - |  0.54 |  0.52 |  |  qdr |  1.32 |  1.27 |  1.35 |  - |  1.25 |  1.35
|  rel |  0.55 |  0.54 |  0.54 |  0.55 |  - |  0.56 |  |  rel |  1.36 |  1.28 |  1.25 |  1.31 |  - |  1.24
|  skt |  0.55 |  0.52 |  0.54 |  0.57 |  0.53 |  - |  |  skt |  1.29 |  1.29 |  1.20 |  1.22 |  1.24 |  -
HAL |  clp |  inf |  pnt |  qdr |  rel |  skt | HAL |  clp |  inf |  pnt |  qdr |  rel |  skt
TIL |  clp |  - |  0.48 |  0.48 |  0.43 |  0.48 |  0.47 |  CIL |  clp |  - |  0.88 |  0.87 |  0.81 |  0.90 |  0.86
|  inf |  0.48 |  - |  0.50 |  0.45 |  0.49 |  0.45 |  |  inf |  0.91 |  - |  0.87 |  0.85 |  0.85 |  0.85
|  pnt |  0.45 |  0.47 |  - |  0.51 |  0.49 |  0.48 |  |  pnt |  0.92 |  0.86 |  - |  0.84 |  0.86 |  0.86
|  qdr |  0.48 |  0.46 |  0.46 |  - |  0.48 |  0.46 |  |  qdr |  0.88 |  0.91 |  0.88 |  - |  0.84 |  0.85
|  rel |  0.47 |  0.45 |  0.45 |  0.48 |  - |  0.47 |  |  rel |  0.87 |  0.85 |  0.88 |  0.87 |  - |  0.92
|  skt |  0.46 |  0.46 |  0.46 |  0.49 |  0.45 |  - |  |  skt |  0.94 |  0.91 |  0.89 |  0.90 |  0.88 |  -
MSL |  clp |  inf |  pnt |  qdr |  rel |  skt | MSL |  clp |  inf |  pnt |  qdr |  rel |  skt
TIL |  clp |  - |  0.53 |  0.53 |  0.48 |  0.53 |  0.53 |  CIL |  clp |  - |  1.30 |  1.32 |  1.20 |  1.30 |  1.30
|  inf |  0.54 |  - |  0.55 |  0.54 |  0.54 |  0.50 |  |  inf |  1.24 |  - |  1.30 |  1.30 |  1.26 |  1.30
|  pnt |  0.55 |  0.52 |  - |  0.54 |  0.52 |  0.52 |  |  pnt |  1.35 |  1.30 |  - |  1.26 |  1.30 |  1.26
|  qdr |  0.55 |  0.55 |  0.55 |  - |  0.50 |  0.52 |  |  qdr |  1.30 |  1.30 |  1.30 |  - |  1.20 |  1.24
|  rel |  0.55 |  0.53 |  0.52 |  0.53 |  - |  0.55 |  |  rel |  1.32 |  1.27 |  1.23 |  1.27 |  - |  1.23
|  skt |  0.53 |  0.52 |  0.53 |  0.55 |  0.53 |  - |  |  skt |  1.30 |  1.29 |  1.20 |  1.25 |  1.30 |  -
CDTrans-S |  clp |  inf |  pnt |  qdr |  rel |  skt | CDTrans-B |  clp |  inf |  pnt |  qdr |  rel |  skt
TIL |  clp |  - |  0.00 |  0.00 |  0.00 |  0.00 |  0.00 |  TIL |  clp |  - |  0.00 |  0.00 |  0.00 |  0.00 |  0.00
|  inf |  0.30 |  - |  0.40 |  0.30 |  0.30 |  0.30 |  |  inf |  0.20 |  - |  0.20 |  0.30 |  0.20 |  0.20
|  pnt |  0.20 |  0.30 |  - |  0.30 |  0.0 |  0.40 |  |  pnt |  0.20 |  0.40 |  - |  0.30 |  0.20 |  0.20
|  qdr |  0.20 |  0.30 |  0.40 |  - |  0.30 |  0.30 |  |  qdr |  0.20 |  0.40 |  0.20 |  - |  0.20 |  0.20
|  rel |  0.00 |  0.00 |  0.00 |  0.00 |  - |  0.00 |  |  rel |  0.00 |  0.00 |  0.00 |  0.00 |  - |  0.00
|  skt |  0.30 |  0.30 |  0.40 |  0.30 |  0.30 |  - |  |  skt |  0.20 |  0.40 |  0.20 |  0.30 |  0.20 |  -
Ours (ACC) |  clp |  inf |  pnt |  qdr |  rel |  skt | Ours (ACC) |  clp |  inf |  pnt |  qdr |  rel |  skt
TIL |  clp |  - |  10.86 |  5.70 |  4.68 |  7.48 |  3.40 |  CIL |  clp |  - |  1.33 |  0.63 |  0.45 |  2.06 |  2.17
|  inf |  21.10 |  - |  10.12 |  2.03 |  13.50 |  8.77 |  |  inf |  2.59 |  - |  1.26 |  0.25 |  1.64 |  1.02
|  pnt |  22.80 |  10.84 |  - |  2.52 |  14.61 |  9.96 |  |  pnt |  2.98 |  1.31 |  - |  0.28 |  1.77 |  1.18
|  qdr |  16.43 |  2.32 |  3.85 |  - |  6.96 |  4.64 |  |  qdr |  1.94 |  0.29 |  0.48 |  - |  0.89 |  0.58
|  rel |  26.22 |  12.26 |  11.85 |  2.57 |  - |  10.23 |  |  rel |  3.24 |  1.50 |  1.49 |  0.31 |  - |  1.20
|  skt |  27.62 |  11.58 |  11.07 |  5.15 |  13.93 |  - |  |  skt |  3.49 |  1.35 |  1.31 |  0.58 |  1.64 |  -
Ours (FGT) |  clp |  inf |  pnt |  qdr |  rel |  skt | Ours (FGT) |  clp |  inf |  pnt |  qdr |  rel |  skt
TIL |  clp |  - |  4.22 |  6.42 |  1.44 |  10.51 |  2.44 |  CIL |  clp |  - |  5.19 |  8.12 |  1.25 |  11.39 |  2.54
|  inf |  7.73 |  - |  3.93 |  0.81 |  4.96 |  3.45 |  |  inf |  9.46 |  - |  4.59 |  0.96 |  6.27 |  4.07
|  pnt |  9.04 |  4.21 |  - |  0.97 |  5.42 |  3.88 |  |  pnt |  11.18 |  5.22 |  - |  1.17 |  6.73 |  4.61
|  qdr |  6.19 |  0.89 |  1.50 |  - |  2.83 |  1.91 |  |  qdr |  7.37 |  1.08 |  1.83 |  - |  3.41 |  2.32
|  rel |  10.25 |  4.70 |  4.59 |  1.06 |  - |  3.60 |  |  rel |  12.36 |  5.89 |  5.79 |  1.24 |  - |  4.49
|  skt |  11.14 |  4.32 |  4.38 |  1.85 |  5.52 |  - |  |  skt |  13.20 |  5.17 |  5.51 |  2.47 |  6.70 |  -
TVT (Static UDA) |  clp |  inf |  pnt |  qdr |  rel |  skt |  |  |  |  |  |  |  |
Static |  clp |  - |  24.2 |  48.9 |  15.5 |  63.9 |  50.7 |  |  |  |  |  |  |  |
|  inf |  43.5 |  - |  44.9 |  6.5 |  58.8 |  37.6 |  |  |  |  |  |  |  |
|  pnt |  52.8 |  23.3 |  - |  6.6 |  64.6 |  44.5 |  |  |  |  |  |  |  |
|  qdr |  31.8 |  6.1 |  15.6 |  - |  23.4 |  18.9 |  |  |  |  |  |  |  |
|  rel |  58.9 |  26.3 |  56.7 |  9.1 |  - |  45.0 |  |  |  |  |  |  |  |
|  skt |  60.0 |  21.1 |  48.4 |  16.6 |  61.7 |  - |  |  |  |  |  |  |  |
TABLE III: Comparison with SoTA methods’ ACC on DomainNet. The rows represent
the source dataset, while the columns represent the target dataset. The best
result in each experiment is highlighted with bold font.

We conducted a comprehensive analysis of all baselines’ capabilities in
mitigating feature alignment catastrophic forgetting. The results for both the
Task-Incremental Learning (TIL) and Class-Incremental Learning (CIL) scenarios
are presented in Tables I, II, and III. These tables compare the average
accuracy of all baselines alongside CDCL’s additional coverage of average
forgetting.

Our findings reveal that CDCL exhibits superior performance over all continual
baselines in the TIL setup. This success can be attributed to CDCL’s inter-
intra-task cross-attention mechanism, which effectively retains previous task
information in the attention maps (queries and values projections).
Additionally, CDCL maximizes intra-task knowledge in the key vectors from both
source and target domains. Furthermore, CDCL excels in scenarios where the
target domain is closely related to the source domain, such as in the
MNIST$\leftrightarrow$USPS experiments. This success is primarily due to
CDCL’s intra-task center-aware pseudo-labeling mechanism, which enables
successful semi-supervised prediction of the target domain. Remarkably, CDCL’s
performance in the D$\rightarrow$W and W$\rightarrow$D experiments further
reinforces this claim. Significantly, in the DomainNet experiments, CDCL
stands out as the only continual method capable of demonstrating any signal of
learning and catastrophic forgetting mitigation.

However, it is important to acknowledge certain limitations. First, all
continual learning that presents task-related parameters, such as CDCL,
suffers from a theoretical infinitely growing numbers of parameter. Usually,
this weakness is overlooked because of the small footprint added to the
computation complexity of the model as more tasks are introduced.
Additionally, such limitation is easy handled by spliting the parameters
records into the primary memory (RAM) and the secondary memories (HDDs or
SSDs), loading only the necessary parameters during inference time. Finally,
all methods, including CDCL, still need improvements in addressing the
challenges of the CIL scenario. In tackling such demanding experiments, CDCL
adopts the experimental replay strategy introduced by DER and DER++ to recover
forgotten information swiftly. Consequently, all three baselines perform
similarly in these scenarios. The hypothesis behind this method lies in the
fact that forgotten knowledge may still reside in residual forms within the
network parameters, which can be partially recovered through experience
replay. For future work, it is essential to devise an improved mechanism that
preserves inter-task common knowledge while effectively adapting to new tasks,
problem also depict on Fig. 2 with the VisDA-2017 benchmark. Addressing this
aspect will contribute to enhancing CDCL’s performance in handling the CIL
scenario and advancing the field of cross-domain continual learning.444The
work presented on [22] seems to tackle the CIL scenario. However, the authors
didn’t release their method source code; and their experimental setup is very
different from the standard continual learning procedure, presented here.

Figure 2: The evolution of CDCL’s ACC in the VisDA-2017 for both TIL and CIL
scenarios. The shared area represents the standard deviation of
$R_{i,j},i\in[1,j]$, the accuracy on such a task by a model that learned only
previous tasks.

###  V-E Ablation study

In our investigation, we delved into the influence of the three losses-block
on the final ACC performance of CDCL. Additionally, we explored the impact of
the inter- intra-task cross-attention, which involves evaluating how a
standard simple attention mechanism on the source domain enables the network
to generalize to the target domain. The results of our analysis are presented
in Table IV.

The examination of the losses-block revealed their relative importance in
determining CDCL’s performance. Notably, the intra-task loss emerged as the
most critical factor, followed closely by the rehearsal loss. This finding
suggests that leveraging intra-task information plays a significant role in
enhancing the model’s performance in CDCL.

Interestingly, the omission of the inter-task loss had a relatively low-
performance impact, indicating that much of the knowledge is effectively
preserved in the $\mathbf{K}_{i}$ and $\mathbf{b}_{i}$ projections. This
observation highlights the effectiveness of the intra-task and rehearsal
losses in capturing and retaining crucial information during the learning
process.

Furthermore, in the context of the class-incremental learning (CIL) scenario,
the rehearsal loss demonstrated a higher level of importance in preserving and
adapting inter-task information. This aspect is especially valuable when
addressing the continual learning of new classes, as the rehearsal loss aids
in retaining knowledge from previous tasks, facilitating effective adaptation
to new ones.

Finally, it was noted that when utilizing the standard simple attention
mechanism, CDCL’s main contribution is compromised, leading to a performance
that closely resembles that of DER (Domain-Adversarial Network for Domain-
Adaptive Visual Recognition) and DER++. This outcome underscores the unique
and significant impact of CDCL’s approach to cross-domain continual learning
and the importance of its inter-intra-task cross-attention mechanism in
achieving superior performance over alternative methods.

Experiment | Loss / Module | MN$\to$US | US$\to$MN
---|---|---|---
$\mathcal{L}^{\text{CIL}}$ | $\mathcal{L}^{\text{TIL}}$ | $\mathcal{L}_{\mathrm{R}}$ | TIL | CIL | TIL | CIL
- | ✓ | ✓ | ✓ | 91.91 | 66.73 | 81.48 | 52.50
A | - | ✓ | ✓ | 81.88 | 63.71 | 67.31 | 47.86
B | ✓ | - | ✓ | 59.17 | 46.33 | 60.20 | 34.65
C | ✓ | ✓ | - | 68.71 | 19.59 | 71.72 | 15.83
Simple attention | ✓* | ✓* | ✓* | 62.72 | 29.82 | 73.20 | 29.50

TABLE IV: Ablation study analyzing the effect of each loss module into CDCL’s
ACC.

##  VI Conclusion

In this paper, we introduced a novel framework called Cross-Domain Continual
Learning (CDCL), which effectively tackles the challenge of unsupervised
cross-domain problems in the task-incremental learning setup. CDCL capitalizes
on two key mechanisms: the inter-intra-task cross-attention block and the
intra-task center-aware pseudo-labeling procedure. Leveraging the transformer
architecture, CDCL enables simultaneous learning across multiple domains in a
continual learning context.

Through extensive numerical experiments, we demonstrated the superiority of
CDCL over its baselines in the task-incremental learning scenario. The results
underscore the effectiveness of the inter-intra-task cross-attention and the
center-aware pseudo-labeling strategies in enhancing CDCL’s performance.

Looking ahead, our future work will focus on further refining CDCL in the
direction of fully class-incremental learning. We aim to extend the
capabilities of CDCL to effectively handle the continual introduction of new
classes, thus advancing the field of cross-domain continual learning and its
practical applications.

## References

  * [1] Rahaf Aljundi, Min Lin, Baptiste Goujaud, and Yoshua Bengio.  Gradient Based Sample Selection for Online Continual Learning.  Curran Associates Inc., Red Hook, NY, USA, 2019.
  * [2] Shai Ben-David, John Blitzer, Koby Crammer, Alex Kulesza, Fernando Pereira, and Jennifer Wortman Vaughan.  A theory of learning from different domains.  Mach. Learn., 79(1–2):151–175, May 2010.
  * [3] Matteo Boschini, Lorenzo Bonicelli, Pietro Buzzega, Angelo Porrello, and Simone Calderara.  Class-incremental continual learning into the extended der-verse.  IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022.
  * [4] Pietro Buzzega, Matteo Boschini, Angelo Porrello, Davide Abati, and Simone Calderara.  Dark experience for general continual learning: a strong, simple baseline.  In H. Larochelle, M. Ranzato, R. Hadsell, M. F. Balcan, and H. Lin, editors, Advances in Neural Information Processing Systems, volume 33, pages 15920–15930. Curran Associates, Inc., 2020.
  * [5] Angelo Cangelosi and Matthew Schlesinger.  From babies to robots: The contribution of developmental robotics to developmental psychology.  Child Development Perspectives, 12(3):183–188, Sept. 2018.
  * [6] G.A. Carpenter and S. Grossberg.  The art of adaptive pattern recognition by a self-organizing neural network.  Computer, 21(3):77–88, 1988.
  * [7] Arslan Chaudhry, Puneet K. Dokania, Thalaiyasingam Ajanthan, and Philip H. S. Torr.  Riemannian walk for incremental learning: Understanding forgetting and intransigence.  In Computer Vision – ECCV 2018: 15th European Conference, Munich, Germany, September 8-14, 2018, Proceedings, Part XI, page 556–572, Berlin, Heidelberg, 2018. Springer-Verlag.
  * [8] Arslan Chaudhry, Albert Gordo, Puneet K. Dokania, Philip Torr, and David Lopez-Paz.  Using hindsight to anchor past knowledge in continual learning, 2020.
  * [9] Arslan Chaudhry, Marc’Aurelio Ranzato, Marcus Rohrbach, and Mohamed Elhoseiny.  Efficient lifelong learning with a-gem.  ArXiv, abs/1812.00420, 2019.
  * [10] Yun-Chun Chen, Yen-Yu Lin, Ming-Hsuan Yang, and Jia-Bin Huang.  Crdoco: Pixel-level domain transfer with cross-domain consistency.  In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2019.
  * [11] Hyeonwoo Cho, Kazuya Nishimura, Kazuhide Watanabe, and Ryoma Bise.  Effective pseudo-labeling based on heatmap for unsupervised domain adaptation in cell detection.  Medical Image Analysis, 79:102436, 2022.
  * [12] Ren Chuan-Xian, Zhai Yi-Ming, Luo You-Wei, and Li Meng-Xue.  Towards unsupervised domain adaptation via domain-transformer, 2022.
  * [13] Zihang Dai, Hanxiao Liu, Quoc V Le, and Mingxing Tan.  Coatnet: Marrying convolution and attention for all data sizes.  arXiv preprint arXiv:2106.04803, 2021.
  * [14] Marcus de Carvalho, Mahardhika Pratama, Jie Zhang, and Edward Yapp.  Acdc: Online unsupervised cross-domain adaptation, 2021.
  * [15] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei.  Imagenet: A large-scale hierarchical image database.  In 2009 IEEE Conference on Computer Vision and Pattern Recognition, pages 248–255, 2009.
  * [16] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby.  An image is worth 16x16 words: Transformers for image recognition at scale.  In International Conference on Learning Representations, 2021.
  * [17] Benjamin Graham, Alaaeldin El-Nouby, Hugo Touvron, Pierre Stock, Armand Joulin, Herve Jegou, and Matthijs Douze.  Levit: A vision transformer in convnet’s clothing for faster inference.  In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 12259–12269, October 2021.
  * [18] Demis Hassabis, Dharshan Kumaran, Christopher Summerfield, and Matthew Botvinick.  Neuroscience-inspired artificial intelligence.  Neuron, 95(2):245–258, 2017.
  * [19] Ali Hassani, Steven Walton, Nikhil Shah, Abulikemu Abuduweili, Jiachen Li, and Humphrey Shi.  Escaping the big data paradigm with compact transformers.  2021\.
  * [20] J. J. Hull.  A database for handwritten text recognition research.  IEEE Transactions on Pattern Analysis and Machine Intelligence, 16(5):550–554, 1994.
  * [21] James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A. Rusu, Kieran Milan, John Quan, and Tiago Ramalho.  Overcoming catastrophic forgetting in neural networks.  Proceedings of the National Academy of Sciences, 114(13):3521–3526, 2017.
  * [22] Qicheng Lao, Xiang Jiang, Mohammad Havaei, and Yoshua Bengio.  A two-stream continual learning system with variational domain-agnostic feature replay.  IEEE Transactions on Neural Networks and Learning Systems, 33(9):4466–4478, 2022.
  * [23] Yann LeCun and Corinna Cortes.  MNIST handwritten digit database.  ATT Labs, 2010.
  * [24] Dong-Hyun Lee.  Pseudo-label : The simple and efficient semi-supervised learning method for deep neural networks.  ICML 2013 Workshop : Challenges in Representation Learning (WREPL), 07 2013.
  * [25] Jichang Li, Guanbin Li, Yemin Shi, and Yizhou Yu.  Cross-domain adaptive clustering for semi-supervised domain adaptation.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 2505–2514, June 2021.
  * [26] Jian Liang, Dapeng Hu, and Jiashi Feng.  Do we really need to access the source data? source hypothesis transfer for unsupervised domain adaptation.  In Proceedings of the 37th International Conference on Machine Learning, ICML’20. JMLR.org, 2020.
  * [27] Sheng Liu, Zhihui Zhu, Qing Qu, and Chong You.  Robust training under label noise by over-parameterization.  In Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepesvari, Gang Niu, and Sivan Sabato, editors, Proceedings of the 39th International Conference on Machine Learning, volume 162 of Proceedings of Machine Learning Research, pages 14153–14172. PMLR, 17–23 Jul 2022.
  * [28] David Lopez-Paz and Marc' Aurelio Ranzato.  Gradient episodic memory for continual learning.  In I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc., 2017.
  * [29] Ilya Loshchilov and Frank Hutter.  Decoupled weight decay regularization.  In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019. OpenReview.net, 2019.
  * [30] Nicolas Y. Masse, Gregory D. Grant, and David J. Freedman.  Alleviating catastrophic forgetting using context-dependent gating and synaptic stabilization.  Proceedings of the National Academy of Sciences, 115(44):E10467–E10475, 2018.
  * [31] Kento Nishi, Yi Ding, Alex Rich, and Höllerer, Tobias.  Augmentation strategies for learning with noisy labels.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 8022–8031, June 2021.
  * [32] Sinno Jialin Pan, James T. Kwok, and Qiang Yang.  Transfer learning via dimensionality reduction.  In Proceedings of the 23rd National Conference on Artificial Intelligence - Volume 2, AAAI’08, page 677–682. AAAI Press, 2008.
  * [33] Xingchao Peng, Qinxun Bai, Xide Xia, Zijun Huang, Kate Saenko, and Bo Wang.  Moment matching for multi-source domain adaptation.  In Proceedings of the IEEE International Conference on Computer Vision, pages 1406–1415, 2019.
  * [34] Xingchao Peng, Ben Usman, Neela Kaushik, Judy Hoffman, Dequan Wang, and Kate Saenko.  Visda: The visual domain adaptation challenge, 2017.
  * [35] Andrei A. Rusu, Neil C. Rabinowitz, Guillaume Desjardins, Hubert Soyer, James Kirkpatrick, Koray Kavukcuoglu, Razvan Pascanu, and Raia Hadsell.  Progressive neural networks.  ArXiv, abs/1606.04671, 2016.
  * [36] Kate Saenko, Brian Kulis, Mario Fritz, and Trevor Darrell.  Adapting visual category models to new domains.  volume 6314, pages 213–226, 09 2010.
  * [37] Jonathan Schwarz, Wojciech Czarnecki, Jelena Luketina, Agnieszka Grabska-Barwinska, Yee Whye Teh, Razvan Pascanu, and Raia Hadsell.  Progress andamp; compress: A scalable framework for continual learning.  In Jennifer Dy and Andreas Krause, editors, Proceedings of the 35th International Conference on Machine Learning, volume 80 of Proceedings of Machine Learning Research, pages 4528–4537. PMLR, 10–15 Jul 2018.
  * [38] Hanul Shin, Jung Kwon Lee, Jaehong Kim, and Jiwon Kim.  Continual learning with deep generative replay.  In I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc., 2017.
  * [39] Christian Simon, Masoud Faraki, Yi-Hsuan Tsai, Xiang Yu, Samuel Schulter, Yumin Suh, Mehrtash Harandi, and Manmohan Chandraker.  On generalizing beyond domains in cross-domain continual learning.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 9265–9274, June 2022.
  * [40] Barry Stein, Terrence Stanford, and Benjamin Rowland.  Development of multisensory integration from the perspective of the individual neuron.  Nature reviews. Neuroscience, 15:520–35, 07 2014.
  * [41] Sebastian Thrun and Tom M. Mitchell.  Lifelong robot learning.  Robotics and Autonomous Systems, 15(1):25–46, 1995.  The Biology and Technology of Intelligent Autonomous Agents.
  * [42] Gido M van de Ven and Andreas S Tolias.  Generative replay with feedback connections as a general strategy for continual learning.  arXiv preprint arXiv:1809.10635, 2018.
  * [43] Ashish Vaswani, Prajit Ramachandran, Aravind Srinivas, Niki Parmar, Blake Hechtman, and Jonathon Shlens.  Scaling local self-attention for parameter efficient visual backbones, 2021.
  * [44] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, and Illia Polosukhin.  Attention is all you need.  In Proceedings of the 31st International Conference on Neural Information Processing Systems, NIPS’17, page 6000–6010, Red Hook, NY, USA, 2017. Curran Associates Inc.
  * [45] Hemanth Venkateswara, Jose Eusebio, Shayok Chakraborty, and Sethuraman Panchanathan.  Deep hashing network for unsupervised domain adaptation.  In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 5018–5027, 2017.
  * [46] Y. Wang, X. Ma, Z. Chen, Y. Luo, J. Yi, and J. Bailey.  Symmetric cross entropy for robust learning with noisy labels.  In 2019 IEEE/CVF International Conference on Computer Vision (ICCV), pages 322–330, Los Alamitos, CA, USA, nov 2019. IEEE Computer Society.
  * [47] Zhen Wang, Liu Liu, Yiqun Duan, Yajing Kong, and Dacheng Tao.  Continual learning with lifelong vision transformer.  In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 171–181, 2022.
  * [48] Weiwei Weng, Mahardhika Pratama, Choiru Za’in, Marcus de Carvalho, Rakaraddi Appan, Andri Ashfahani, and Edward Yapp Kien Yee.  Autonomous cross domain adaptation under extreme label scarcity.  IEEE Transactions on Neural Networks and Learning Systems, pages 1–12, 2022.
  * [49] Tongkun Xu, Weihua Chen, Pichao Wang, Fan Wang, Hao Li, and Rong Jin.  Cdtrans: Cross-domain transformer for unsupervised domain adaptation, 2021.
  * [50] Jinyu Yang, Jingjing Liu, Ning Xu, and Junzhou Huang.  TVT: transferable vision transformer for unsupervised domain adaptation.  CoRR, abs/2108.05988, 2021.
  * [51] Chang’an Yi, Haotian Chen, Yonghui Xu, Yong Liu, Lei Jiang, and Haishu Tan.  Atpl: Mutually enhanced adversarial training and pseudo labeling for unsupervised domain adaptation.  Knowledge-Based Systems, 250:108831, 2022.
  * [52] Friedemann Zenke, Ben Poole, and Surya Ganguli.  Continual learning through synaptic intelligence.  In Doina Precup and Yee Whye Teh, editors, Proceedings of the 34th International Conference on Machine Learning, volume 70 of Proceedings of Machine Learning Research, pages 3987–3995. PMLR, 06–11 Aug 2017.
  * [53] Can Zhang and Gim Hee Lee.  Ca-uda: Class-aware unsupervised domain adaptation with optimal assignment and pseudo-label refinement, 2022.


# Study and perspective on neutron beam divergence improvement achievable by
the combination of two or more neutron collimating systems

Oriol Sans-Planell1,2 Francesco Cantini3,4 Marco Costa1,2 Francesco Grazzi3,4
Manuel Morgano5 Masako Yamada6 1Università degli Studi di Torino, Via Pietro
Giuria 1, 10125 Torino, Italy
2INFN Sezione di Torino, Via Pietro Giuria 1, 10125 Torino, Italy
3Consiglio Nazionale delle Ricerche, Istituto dei Sistemi Complessi, Via
Madonna del Piano 10, 50019 Sesto Fiorentino, Italy
4INFN Sezione di Firenze, Via Madonna del Piano 10, 50019 Sesto Fiorentino,
Italy
5European Spallation Source, Partikelgatan 2, 22484 Lund, Sweden
6Paul Scherrer Institute, Forschungsstrasse 111, 5232 Villigen, Switzerland
<EMAIL_ADDRESS>
###### Abstract

This communication presents the results obtained at an experimental campaign
at PSI BOA beamline using the combination of the ANET Compact Neutron
Collimator (CNC) with the actual BOA pin-hole system. Through extensive
resolution campaigns, it has been possible to quantify and understand the
effects of improvement on the beam divergence when combining the two
collimating systems. A new theoretical approach to this problem is described
and discussed. The effect is expected not to be limited to the specific case
that has been studied at PSI BOA but to have a more general validity for
neutron collimation systems.

##  1 Introduction

The study, here reported, derives its motivation from measurements performed
at the TU-Delft FISH beam-line [1], where an unexpected improvement on the
quality of the beam collimation [2], beyond what a simple geometrical model
could predict, have been shown. This indicates that more complicated effects
arise when combining two collimation systems.
Commonly a geometrical model states that, when two collimators are
concatenated in a set-up, the one with the higher L/D will determine the final
collimation power of the system. This assumption has certainly some
fundamental of truth, but our measurements showed also its limitations, since
the model only takes into consideration the geometrical properties of the
system but it does not include that the divergence distribution of the
neutrons after the first collimator is different from that of the original
source. In order to replicate the results obtained at TU-Delft and a better
understanding of the observed behavior, a more extensive measurement campaign
has been done at the BOA beam-line [3], at the Paul Scherrer Institut.
A hypothesis of the formal description of the convolutional effect between
both collimators is described in the following sections.

##  2 The experiment

The experiment has been performed at the Paul Scherrer Institut, at the BOA
facility [3]. It is a multi-purpose beamline composed by a 18$m$ long tube
with a cold neutron flux of $10^{8}cm^{-2}s^{-1}mA^{-1}$ at the exit window.
The beamline is composed by a series of neutrons guides and a bender, which
creates some alterations on the beam divergence. The beam exit window can also
be confined, through a pinhole, to different sizes, up to 40 x 150 $mm^{2}$.
For the measurements, the beam was restricted to a rectangular shape of 40 x
80 $mm^{2}$.

Figure 1: Design of the experimental set-up present at the campaign.

The set-up used was analogous to the one already defined on previous
experimental campaigns [2, 4]. It made use of three towers - labeled as T4, T5
and T6 -. Each one has a Z-axis moving stage parallel to the beam: the first
one (T4) held the Stewart Platform and the collimator, the second one (T5)
held a secondary X-axis linear stage with the samples, and the third one (T6)
held the detector. The three T stages were configured so that the distance
between the exit window of the collimator and the scintillator of the detector
was 190$mm$. The two standard reference samples were chosen to be a gadolinium
knife edge and a Siemens star, both manufactured at the PSI [5], and mounted
in the middle stage (T5). The stage moved the samples at 8 different
positions, from 10$mm$ up to 80$mm$ from the scintillator at 10$mm$ steps -
labeled as micro-positions -. The T5 X-axis motor served to change from one
sample to the other.

Position | T4 | T5 | T6 |
---|---|---|---|---
1 | 4480 | 4920 | 5360 |
2 | 5480 | 5920 | 6360 |
3 | 6480 | 6920 | 7360 |
4 | 7480 | 7920 | 8360 |
Table 1: Distance from the primary pin-hole to each motor (in $mm$) at each
macro-position (1-4) for the measurements at BOA.

Table 1 describes the position of each stage, from T4 to T6 at each macro-
position with respect to the primary pin-hole.
The camera used to acquire the images was an ANDOR 16-bit cooled CCD with a
field of view of 4.00 x 4.00 $cm^{2}$ over a resolution of 1024 x 1024,
leading to a projected pixel size at the scintillator of 33.65$\mu m$. The BOA
system was controlled through the software package NIKOS, which is a custom
distribution of the open source EPICS [6], while the Stewart platform was
controlled via LAN using Labview.

##  3 Experimental results

The measurements taken with both Siemens Star and gadolinium knife-edge
reference samples have been analysed following the procedure in [2], and the
results are shown in figure 2

Figure 2: Graph showing the divergence angle calculated on each macro-position
with both the Siemens star and the gadolinium knife-edge.

The expected beam divergence should not be affected by a collimator whose
collimation power is smaller than the beam divergence itself. The measurements
shown in figure 2 point at different conclusions. Not only the divergence is
improved when the ANET CNC is included within the beamline, but it greatly
reduces the beam angular divergence beyond its expected capabilities. A second
appreciation is that the absolute improvement is not the same for the vertical
and horizontal axis, despite the collimator applying the same potential
correction factor by having a square section geometry. A hypothesis why this
phenomenon happens will be studied in the following section.

##  4 Mathematical description

The approximate geometrical behaviour that a neutron would have when passing
through a collimator made of a perfect absorbing material can be described
through the following formula, for the 2D case:

$P_{2D}(\theta)=\int_{x}\rho(x,\theta)\delta(\theta|x)dx$ (1)

This equation describes the probability distribution of a neutron to survive
the collimator, $P(\theta)$, calculated through the integral of the initial
space and angular distribution, $\rho(x,\theta)$, and a step function which
can either take values 0 or 1, depending on whether the neutrons are absorbed
or not, which is determined by the angle given a certain position. To
calculate the shape of this function, a very simple empirical simulation has
been designed: since the probability of passing through the collimator is
either 0 or 1, a simple channel was designed, with given characteristics (L
and D), and every possible neutron within an angular range of
[-$\theta$,+$\theta$] and an spatial range of [$-\frac{D}{2}$,$+\frac{D}{2}$)
was simulated. The output of this model is the distribution $P(\theta)$ which
describes the probability of survival through the given collimator.

Figure 3: 2D acceptance curves calculated at different values of L/D. The
graphic shows the probability of ”survival” of a neutron depending on the
divergence angle.

Figure 3 shows some acceptance curves generated with different L/D values.
These curves can be parametrised with a straight line function, which is:

$P_{2D}(\theta)=\theta\Big{(}arctan(\frac{1}{L/D})\Big{)}^{-1}+1$ (2)

The value $\Big{(}arctan(\frac{1}{L/D})\Big{)}^{-1}$ essentially describes the
limit angle over which the probability of survival is 0111As it is a
probability, negative values are equal to 0..
The model can be expanded to 3D, in order to evaluate the total divergence,
instead of evaluating only one axis. When treating the collimator in 3D, two
more variables are added: there are two spatial coordinates $(x,y)$ and two
angular coordinates, the divergence from the propagation axis, $\theta$ and
the rotation along the propagation axis, $\phi$, which is sampled in
$[0,2\pi]$. The formula that describes the mathematical model is:

$P_{3D}(\theta)=\int_{x}\int_{y}\int_{\phi}\rho(x,y,\theta,\phi)\delta(\theta,\phi|x,y)dxdyd\phi$
(3)

The solution of $P_{3D}(\theta)$, contrary to the 2D version, depends heavily
on the geometry of the collimator. In the ANET CNC, the 3D distribution
follows:

$P_{3D}\Big{(}\sqrt{\theta_{1}^{2}+\theta_{2}^{2}}\Big{)}=P_{2D}(\theta_{1})\cdot
P_{2D}(\theta_{2})$ (4)

This relation is due to the square geometry of the collimator individual
channels. Figure 4 has the result of some calculations of the 3D acceptance
curves.

Figure 4: Different acceptance curves calculated in 3D while varying the L/D
factor. The probability of survival of the neutrons decrease with the
increasing angle with respect to the propagation axis.

Contrary to the 2D case, in the 3D version the $\theta$ value has a minimum
value of 0. This is due to the $\phi$ being sampled from 0 to $2\pi$.
This function acts as a mask over the incident beam, altering the shape
regardless of the initial beam divergence. To test this hypothesis, the
angular distributions from the BOA facility simulation, done in McStas, have
been used. The beam divergence distributions before and after the collimator
have been simulated. If the mathematical formula is correct, multiplying the
angular distribution present at the entrance window of the collimator by the
mask should result in the beam’s angular distribution at the exit window. This
is discussed in the following section.

##  5 Testing the Acceptance Curve parametrisation

The set-up has been simulated using McStas [7] in 2 positions: with the CNC at
5.23$m$ from the BOA exit window, and at 20$cm$. The calculation test has been
done considering both the 2D and 3D cases.

###  5.1 The 2D cases

Both close (figure 5) and far (figure 6) configurations have been simulated.
The plots in the figures have been normalised to the maximum value of each
plot, in order to compare the shapes of the distributions. All cases show a
very good agreement between the calculated distributions and the simulated
ones. Those plots render evident the difference in the BOA facility between
the vertical and horizontal distributions. The bender present along the guide
creates distortions on the horizontal axis, as expected. The plots of the
vertical divergence shows a wide and uniform distribution before the CNC,
which is strongly reduced by the collimator down to the shape of the
acceptance curve. On the horizontal axis, instead, the irregular shape of the
angular distribution is still very well predicted by the model, as it adapts
onto the shape of the acceptance curve.

Figure 5: Vertical and horizontal angular divergence distribution plots before
and after the ANET CNC along with the predicted distribution. Simulation done
with the CNC at 20 $cm$ from the exit window of BOA.

When the ANET CNC is installed far from the exit window of BOA (figure 6), the
pin-hole collimation of the beam reduces the maximum angular divergence to
less than 0.5 degrees. On the vertical axis, the distribution has, at the
tails, two secondary peaks, artifacts generated by the beam construction.
Those artifacts get eliminated by the collimator, which also shrinks the
distribution.

Figure 6: Vertical and horizontal angular divergence distribution plots before
and after the ANET CNC along with the predicted distribution. Simulation done
with the CNC at 5.23 $m$ from the exit window of BOA.

On the horizontal axis, the effect of collimation is less prominent than on
the vertical axis, but there is still a reduction of the more divergent
neutrons.

###  5.2 The 3D cases

Regarding the 3D calculation, the formula used to extract the analytical shape
is reported in equation 4, leading to the following results in figure 7. The
effect of the ANET CNC is very apparent in both cases, but specially when the
collimator is close to the exit window of BOA. All the neutrons beyond the
0.50 limit angle are absorbed, while the whole distribution shifts towards
smaller divergence angles. On the far set-up configuration, the angular
distribution without the collimator is more irregular, and it becomes smoother
when the ANET CNC is present in the set-up. In both close and far
configurations, the model accurately predicts the impact of the ANET CNC on
the angular divergence of the beam.

Figure 7: Results of the simulations of the 3D neutron angular distribution
before and after the ANET CNC, along with the analytical calculation of the
resulting angular distribution.

The same test can be applied when calculating the neutron angular distribution
while combining two curves of acceptance: the pin-hole effect from the BOA
exit window to the ANET CNC, and the ANET CNC itself. The result is shown in
figure 8.

Figure 8: Results of the simulations of the 3D neutron angular distribution at
the BOA exit window and after the ANET CNC. The graph also shows the
calculated neutron distribution after the ANET CNC.

The normalised calculated curve adapts fairly well to the simulated angular
distribution obtained from the McStas simulations.

##  6 Conclusions

A first important consequence of the curve of the mathematical acceptance
model is the fact that the collimator has an active effect in altering the
quality of the image, regardless of the angular divergence of the initial
beam. A second consequence is that the collimator doesn’t alter the absolute
divergence if the beam is more collimated, as the endpoint of the distribution
after the collimator doesn’t change. The improvement in the image quality is
due to the statistical reduction of the number of neutrons present at high
divergence angle, not because the collimator fully eliminates them.
This hypothesis is not limited to the ANET CNC, as at no point within the
calculations the full ANET geometry has been imposed. The model has been done
for a single collimation channel, and the calculations are perfectly
applicable to a circular collimator or a pinhole. Using this hypothesis, the
combination of more than one pinhole, or with the ANET collimator itself, can
be used as a beam divergence corrector. Equations 3 and 4 imply that the
effect of the collimator is null on the perfectly straight neutrons, while the
greater the angle, the better the absorption, up to the value of 1 (perfect
absorption) when the angle is beyond the geometrical acceptance of the
collimator. The effect that the collimation has on the neutron angular
distribution implies that, at some point, the most-divergent neutrons will be
”smoothed out” and they will be not statistically significant to contribute
above the image noise, becoming than not distinguishable. The macroscopic
consequence of this effect is that the measured neutron divergence is better
than what can be calculated. The authors would like to acknowledge the INFN
for the financial support of the ANET project. A special thanks also to the
support given by members of the Paul Scherrer Institute, for their insights
and help on the experimental campaigns, in particular Pierre Boillat, Pavel
Trtik, Anders Kaestner and Uwe Filges.

## References

## References

  * [1] Zhou, Zhou, et al. ”FISH: A thermal neutron imaging station at HOR Delft.” Journal of Archaeological Science: Reports 20 (2018): 369-373.
  * [2] Sans-Planell, Oriol, et al. ”Improved methodologies to study the performance of the ANET Compact Neutron Collimator.” arXiv preprint arXiv:2301.07749 (2023).
  * [3] Morgano, M., et al. ”Neutron imaging options at the BOA beamline at Paul Scherrer Institut.” Nuclear Instruments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment 754 (2014): 46-56.
  * [4] Sans-Planell, Oriol, et al. ”First results with the ANET compact thermal neutron collimator.” Journal of Instrumentation 16.11 (2021): P11025.
  * [5] Kaestner, A. P., et al. ”Samples to determine the resolution of neutron radiography and tomography.” Physics Procedia 88 (2017): 258-265.
  * [6] Dalesio, Leo R., A. J. Kozubal, and M. R. Kraimer. EPICS architecture. No. LA-UR-91-3543; CONF-911116-9. Los Alamos National Lab., NM (United States), 1991.
  * [7] Lefmann, Kim, and Kristian Nielsen. ”McStas, a general software package for neutron ray-tracing simulations.” Neutron news 10.3 (1999): 20-23.


# Lesion segmentation using U-Net network

Adrien Motsch  Sébastien Motsch  Arizona State University, Tempe (USA)
Thibaut Saguet

###### Abstract

This paper explains the method used in the segmentation challenge (Task 1) in
the International Skin Imaging Collaboration’s (ISIC) Skin Lesion Analysis
Towards Melanoma Detection challenge held in 2018. We have trained a U-Net
network to perform the segmentation. The key elements for the training were
first to adjust the loss function to incorporate unbalanced proportion of
background and second to perform post-processing operation to adjust the
contour of the prediction.

##  1 Introduction

Segmentation is an important task to highlight zone of interests in medical
images. In the ISIC 2018 challenge, the goal is to perform lesion boundary
segmentation. To perform the segmentation, we have chosen to use a U-Net
architecture [2] as it is known to perform well in medical image and without a
large data-set. The details of the training is explained below and an example
of mask generation is given in figure 1. Our data was extracted from the ISIC
2018: Skin Lesion Analysis Towards Melanoma Detection grand challenge data-
sets [4, 3].

Figure 1: Example of segmentation (image ISIC_0000009.jpg from the training
set 2018): the original image is on the left, in the middle the mask manually
generated (’ground truth’), and on the right the prediction of our network
based on the U-Net architecture.

##  2 Methods

###  2.1 Pre-processing

Before the training of the U-Net network [2], we first pre-processed the data-
set. The very first statistic consisted in estimating the average pixel by
estimating the proportion of each color ( $\langle
Red,Green,Blue\rangle=(0.708,0.582,0.536))$ (red is slightly more prominent)
and the standard deviation of the data-set $(0.0978,0.113,0.127)$. This
information was used in the training to re-center the training set. Another
important information was the average proportion of the mole in the image
which was estimated as:

$\text{prop. mole}=\frac{\\#{\text{pixels ground truth}}}{\\#{\text{pixels in
image}}}=0.214.$ (2.1)

Masks represent in average only $21\%$ of the image. The loss function was
modified to even this ratio. Lastly, we also estimated the position of the
mole inside the image. As expected, moles are more likely present in the
center (see fig. 2-left). This information can be used as a prior for a
Bayesian approach (not used here).

Figure 2: Left: Probability that a pixel is part of a mole for the all
training set. The probability at the center is around $p=.9$ and close to zero
at the boundary ($p\approx.005$). Right: Evolution of the loss function (blue)
and Jaccard index (orange) per epochs.

###  2.2 Training

For the training of the U-net network, we used the cross entropy as loss
function and the Adam optimizer to update the parameters. Learning rate was
set-up as $10^{-4}$ and reduced when the loss function was not decaying. The
evolution per epoch of a typical training is given in figure 2-right. The
orange curve gives the evolution of the so-called Jaccard index used to rank
the method. Usually data-augmentation methods were used (random flipping and
image rotation). The code has been implemented using PyTorch.

###  2.3 Post-processing

The U-net gives a two scores for each pixel (scores $s_{0}$ for the background
and $s_{1}$ foreground). These scores $(s_{0},\,s_{1})$ are then transformed
into probability $(p_{0},\,p_{1})$ via softmax function:

$p_{0}=\frac{\mathrm{e}^{s_{0}}}{\mathrm{e}^{s_{0}}+\mathrm{e}^{s_{1}}},\quad
p_{1}=\frac{\mathrm{e}^{s_{1}}}{\mathrm{e}^{s_{0}}+\mathrm{e}^{s_{1}}}.$ (2.2)

The mask is then taken as the pixel with probability $p_{1}$ higher than $.5$.
However, we find out that the masks can be further improved using several
post-processing methods on the scores $(s_{0},s_{1})$. First, we apply a
Gaussian filter on both scores ($\sigma=5$). Plotting the difference $\Delta
s=s_{1}-s_{0}$ (see fig. 3-left), we observe three levels: at the boundary the
difference is below ($\Delta s<-15$), at the center where the mole is, the
values are around $\Delta s>5$, and finally the normal skin the values are in
between $-10$ and $0$. The histogram of all these values $\Delta s$
(regardless of their position on the image) is given in fig. 3-right. The
threshold separating ’skin’ to ’mole’ seems to be around $\Delta s_{*}=2.5$.
We use the Otsu algorithm [1] to estimate automatically this threshold.

Figure 3: Distribution of the score (foreground - background, i.e. $\Delta
s=s_{1}-s_{0}$) returns by the U-net network. Rather than taking a zero
threshold to determine the mask, we first estimate the threshold separating
distribution of scores (Otsu method).

###  2.4 Results

Our method score a Jaccard index of $.75$ over various test set. For the
validation test, the network has a score of $.71$ due to the metric used
(scores below $.65$ are considered zero).

Acknowledgments Authors would like to thank Thomas Laurent and Xavier Bressan
for fruitful suggestions. S. Motsch thanks Renate Mittelman, Salil Malik and
people at research computing (ASU) for their support during the computation.

## References

  * [1] Nobuyuki Otsu.  A threshold selection method from gray-level histograms.  IEEE transactions on systems, man, and cybernetics, 9(1):62–66, 1979.
  * [2] Olaf Ronneberger, Philipp Fischer, and Thomas Brox.  U-net: Convolutional networks for biomedical image segmentation.  In International Conference on Medical image computing and computer-assisted intervention, pages 234–241. Springer, 2015.
  * [3] Philipp Tschandl, Cliff Rosendahl, and Harald Kittler.  The HAM10000 Dataset: A Large Collection of Multi-Source Dermatoscopic Images of Common Pigmented Skin Lesions.  arXiv preprint arXiv:1803.10417, 2018.
  * [4] Eduardo Valle, Michel Fornaciali, Afonso Menegola, Julia Tavares, Flávia Vasques Bittencourt, Lin Tzy Li, and Sandra Avila.  Data, depth, and design: learning reliable models for melanoma screening.  arXiv preprint arXiv:1711.00441, 2017.