# Continual learning in Deep Neural Networks:
an Analysis of the Last Layer

Timothée Lesort  Université de Montréal, MILA - Quebec AI Institute  Thomas
George  Université de Montréal, MILA - Quebec AI Institute  Irina Rish
Université de Montréal, MILA - Quebec AI Institute

###### Abstract

We study how different output layer parameterizations of a deep neural network
affects learning and forgetting in continual learning settings. The following
three effects can cause catastrophic forgetting in the output layer: (1)
weights modifications, (2) interference, and (3) projection drift. In this
paper, our goal is to provide more insights into how changing the output layer
parameterization may address (1) and (2). Some potential solutions to those
issues are proposed and evaluated here in several continual learning
scenarios. We show that the best-performing type of output layer depends on
the data distribution drifts and/or the amount of data available. In
particular, in some cases where a standard linear layer would fail, changing
parameterization is sufficient to achieve a significantly better performance,
without introducing any continual-learning algorithm but instead by using
standard SGD to train a model. Our analysis and results shed light on the
dynamics of the output layer in continual learning scenarios and suggest a way
of selecting the best type of output layer for a given scenario.

##  1 Introduction

Continual deep learning algorithms usually rely on end-to-end training of deep
neural networks, making them difficult to analyze given the complexity (non-
linearity, non-convexity) of the training dynamics. In this work, we instead
propose to evaluate the ability of a network to learn or forget in a
simplified learning setting: We decompose our model as a _feature extractor_
part consisting of all but the final layer, and a _classifier_ part which is
given by the output layer. Using this decomposition, we can isolate the role
of the output layer parameterization in continual learning scenarios.

Recent works on continual learning (CL) point out some drawbacks of
parameterizing the output layer as a linear layer, especially in incremental
settings [35, 36, 14]. Among these problems is the unbalance of bias and/or
norm of the output layer’s vectors. These works propose some solutions but do
not study them independently from the feature extractor. Indeed, in CL
scenarios, the function learned by the feature extractor changes, and
therefore the embedding space changes accordingly; we call this change the
projection drift. This drift characterizes the change through time of the
embedding on a given observation.

By isolating the output layer from the feature extractor, we can study it in a
controlled environment, i.e., without projection drifts. Decoupling the
feature extractor from a sub-network has shown to be helpful, e.g., in
reinforcement learning [20, 26, 31].

We now list our contributions:
$\bullet$ We propose an evaluation of a large panel of output layer types in
incremental, lifelong, and mixed continual scenarios. We show that depending
on the type of scenario, the best output layer may vary.

$\bullet$ We describe the different sources of performance decrease in
continual learning for the output layer: forgetting, interference, and
projection drifts.

$\bullet$ We propose different solutions to address catastrophic forgetting
(CF) in the output layer: a simplified weight normalization layer, and two
masking strategies Sec. 3.

$\bullet$ We provide a proof of concept for end-to-end training by applying a
reparameterized layer to a vanilla replay method Sec. 6.

##  2 Related Works

In most deep continual learning papers, the last layer of the deep neural
network is parameterized as a linear layer as in typical classifier
architectures in i.i.d. settings such as VGG [30], ResNet [13], and GoogleNet
[32]. Recently, several CL approaches questioned this approach, showing that
the norm or the bias can be unbalanced for the last classes observed in class-
incremental settings. For example, BIC [35] proposed to train two additional
parameters using a subset of the training dataset after training the rest of
the model to correct for the unbalance of the bias. In the same spirit, [36]
proposed to compute a Weight Aligning value to apply to vector to balance the
norm of past classes vectors and the new classes vectors of a linear layer.
[36] also experiments with a layer, the weight normalization layer [28], that
decouples the norm of the weight matrix from its direction to normalize the
vector of linear output layers. [14, 3] proposed to apply a cosine
normalization of the layer to avoid the norm and bias unbalance.

Linear Discriminant Analysis (LDA) Classifiers in incremental settings have
been used for dozens of years [1, 5, 6]. The streaming version SLDA from [29]
has recently been revisited in deep continual learning in [12]. SLDA combines
the mean and the covariance of the training data features to classify new
observations. A more straightforward approach, as implemented in iCaRL [27] is
to average the features for each class and apply a nearest neighbor classifier
also called Nearest Mean Classifier (NMC).

In this work, we study the capability of those various types of top layers
with a fixed pre-trained model in incremental and lifelong settings as defined
in [19]. Many different strategies have been proposed in the continual
learning community: rehearsal, generative replay, dynamic architectures or
regularization. In this study, we do not aim at modifying the training process
or proposing a new one. We instead study how different layer types learn and
forget in various continual scenarios without any additional CL approach.

##  3 Output Layer Types

We here review the output layer parameterizations studied in this work. We
introduce the notations and present the main characteristics of linear layers.
Then, we explain how the standard layer parameterization struggles to learn
continually, and we present different parameterizations to overcome the
problems. Finally, we present several other types of output layers not trained
by gradient descent.

###  3.1 Notations

We will study functions $f_{\theta}(\cdot)$ parameterized by a vector of
parameters $\theta$ representing the set of weight matrices and bias vectors
of a deep network. In continual learning, the goal is to train
$f_{\theta}(\cdot)$ on a sequence of task
$[\mathcal{T}_{0},\mathcal{T}_{1},...,\mathcal{T}_{T-1}]$, such that
$\forall(x,y)\sim\mathcal{T}_{t}$ ($t\in[0,T-1]$), $f_{\theta}(x)=y$.

We can decompose $f_{\theta}(\cdot)$ into two parts, (1) a feature extractor
$\phi_{\theta^{-}}(\cdot)$ which encodes the observation $x$ into a feature
vector $z$, (2) an output layer $g_{\theta^{+}}(\cdot)$ which transforms the
feature vector into a non-normalized vector $o$ (the logits). $\theta^{-}$ and
$\theta^{+}$ are two complementary subsets of $\theta$.

###  3.2 Linear Layer

A linear layer is parameterized by a weight matrix $A$ and bias vector $b$,
respectively of size $N\times h$ and $N$, where $h$ is the size of the latent
vector (the activations of the penultimate layer) and $N$ is the number of
classes. For $z$ a latent vector, the output layer computes the operation
$o=Az+b$. We can formulate this operation for a single class $i$ with $\langle
z,A_{i}\rangle+b_{i}=o_{i}$, where $\langle\cdot\rangle$ is the euclidean
scalar product, $A_{i}$ is the $i$th row of the weight matrix viewed as a
vector and $b_{i}$ is the corresponding scalar bias.

It can be rewritten:

$\lVert z\rVert\lVert A_{i}\rVert\cdot cos(\angle(z,A_{i}))+b_{i}=o_{i}$ (1)

Where $\angle(\cdot,\cdot)$ is the angle between two vectors and
$\lVert\cdot\rVert$ denotes here the euclidean norm of a vector.

Eq. 1 highlights the 3 components that need to be learned to make a correct
prediction $\hat{y}=\operatorname*{argmax}_{i}(o_{i})$: The norm of $A_{i}$,
its angle with the latent representation, and the bias. In particular, if the
training dynamics make a particular $\lVert A_{i}\rVert$ or $b_{i}$ larger
than others, then the network will be considerably biased into predicting the
class $i$. In incremental learning, the vectors $A_{i}$ and values $b_{i}$ are
learned sequentially, hence the learning algorithm will grow the norm $\lVert
A_{i}\rVert$ and bias $b_{i}$ for the current classes and reduce them for past
classes to ease the current task (Figure 1). This may lead to forgetting.

###  3.3 Linear classifiers in continual learning and reparameterizations

In continual learning, assuming that the optimal function $f(x)=y$ is fixed,
there are two main cases of data distribution drift: either we get new
examples from new classes (virtual concept drift $\xrightarrow[]{}$
incremental scenario) or from known classes but in slightly different setups
(domain drift $\xrightarrow[]{}$ lifelong scenario). For a linear output
layer, forgetting is caused by different mechanisms in both scenarios.

In incremental learning, the $A_{i}$ and $b_{i}$ are learned sequentially (one
by one or set by set). When learning a new $A_{i}$ (and $b_{i}$), the training
algorithm does not have access to past classes, and hence cannot ensure that
learning weight vectors for new classes does not interfere with past classes
(eq. 1). As an illustration, suppose two classes are alike in the latent
space. In that case, their respective latent vectors might be similar, and
interference might happen between the two classes (see fig. 3), a fortiori if
the norm and bias are not balanced.

Figure 1: Illustration of norm and bias unbalance at the end of a CIFAR10
continual experiment. 5 tasks with 2 classes each. (left) the norm of each
output vector at the end of the last task, (middle) bias at the end of the
last task, and (right) the difference between the last task’s bias and current
bias. We can see a clear unbalance in norm and bias for the last active
vectors (classes 8 and 9). The middle right figure shows us that the imbalance
of bias is primarily due to the modification of bias from the previous task
(classes 6 and 7).

(a) No masking

(b) Single masking

Figure 2:  Illustration of Forgetting: we plot the difference between weights
before and after task 45 in the scenario Core10Mix (first 50 dimensions only).
Task 45 is composed of only one class that is different from 44’s one. Those
two figures show the impact of masking. Indeed, it avoids the modification of
weights of other classes.

One solution found in the bibliography to avoid interference is to counteract
unbalance between the norms of vectors or bias. We can modify eq. 1 in several
ways to mitigate such unbalance:
Removing the bias (Linear_no_bias layer):

$\lVert z\rVert\lVert A_{i}\rVert\cdot cos(\angle(z,A_{i}))=o_{i}$ (2)

Normalizing output vectors (WeightNorm layer):

$\lVert z\rVert\cdot cos(\angle(z,A_{i}))=o_{i}$ (3)

Measuring only the angle (CosLayer):

$cos(\angle(z,A_{i}))=o_{i}$ (4)

WeightNorm (eq. 3) is similar to the original WeightNorm layer ([28], here
denoted by Original WeightNorm) experimented in [36] in a continual learning
context:

$\gamma_{i}\lVert z\rVert\cdot cos(\angle(z,A_{i}))+b_{i}=o_{i}$ (5)

However, in the original WeightNorm, the additional scaling parameter $\gamma$
and the bias $b$ are learned during training. These parameters are akin to the
parameters in BatchNorm layers [15], which have been shown to be more prone to
catastrophic forgetting in the intermediate layers in continual learning [23].
Hence, our proposed WeightNorm layer (eq. 3) avoids such interference by
ensuring a unit norm for all vectors and removing bias and gamma parameters.

Forgetting might also be caused by weight modification: this happens when the
optimizer modifies weight vectors from past classes that are not found in the
current task. In the literature, regularization strategies (e.g. EWC [16])
aims at avoiding such forgetting by penalizing modification of important
parameters.

We introduce a more radical strategy to avoid that updating a specific class
affects another past one. The strategy consists in masking some classes during
the update step. We propose two types of masking: in single masking, we only
update weights for the output vector of the true target and in group masking,
we mask all classes that are not in the mini-batch. With the following
strategies, the update step cannot change $A_{i}$ and $b_{i}$ of past classes
and avoid sub-sequential interference. In Figure 2, we illustrate the impact
of single masking in a task with only one class from one experiment of the
paper. In incremental scenarios, all masking strategies can also be seen as a
regularization strategy that strictly forbids the modification of past weights
in the last layer.

The masking strategy resembles the update step in a multi-head architecture
[33] for gradient descent since in this case, the gradient is computed only
for a subset of selected classes. However, in our setup, there is only one
head, and the inference is achieved without the help of any external
supervision, such as a task label.

(a) Angles between vectors

(b) Mean angles vectors-data.

(c) Interference risk

Figure 3: Illustration of _interference_ for the linear layer: We plot (left)
the angles (in degree) between the output vectors $A_{i}$, (middle) the mean
angle between the latent vectors $z$ of each class and the $A_{i}$ vectors and
(right) ratio between the mean angle with wrong classes, and the mean angle of
the target class: a high value indicates a higher risk of interference. In
this experiment, there is a high risk of interference between examples for
class 4 and examples of class 1 or 3, as pictured by a dark red cell.

In lifelong learning, forgetting happens when new instances of a previously
learned class make the output layer forget features that were important for
past instances or modify the direction of the output vectors. However, those
problems are usually less prone to catastrophic forgetting (CF) but may still
have a significant impact in complex scenarios. One of the reasons that CF is
less important is that all classes are available simultaneously, which makes
it possible for the output vectors to be coordinated and avoid interference.

###  3.4 Classifiers Trained without Gradient

To avoid interference and weight modification, we experiment with another
family of layer types that rely on the similarity between training latent
vectors and testing latent vectors but are not trained by gradient descent.

One of the most popular classifiers that can easily be adapted into a
continual or online learning setting is the k-nearest neighbors (KNN)
classifier. This classifier searches for the k nearest exemplars of a given
observation in the training set and predicts the class based on the class of
the k neighbors. This classifier’s hyper-parameters are the number of
neighbors $k$ and the distance used (here, the usual euclidean norm).

A lighter strategy is the nearest mean classifier (denoted MeanLayer) as
proposed in iCarL [27], this strategy consists of saving the mean of the
features of each class $k$ while learning (called a _prototype_ $\mu_{k}$) and
predict the class given by the nearest mean for test latent vectors.

An alternative is the linear discriminant analysis classifier, which uses both
the mean of the features and a covariance. The online version of this
algorithm is streaming linear discriminant analysis (SLDA) [29].

While similarity-based classifiers have proven very effective in practice,
they cannot be directly used to train the feature extractor since they are not
differentiable. Hence, they need to store samples from past tasks as rehearsal
strategies or be applied in a setting without projection drift.

In this section, we introduced the different output layer types that we will
use in experiments. We introduce two modifications: (1) a simplified
WeightNorm (referred to as WeightNorm), and, (2) two masking strategies,
single masking and group masking, compatible with any reparameterization of
the output layer.

##  4 Experiments

In this section, we will review how the different output layer types,
presented in section 3, can solve various continual scenarios. We isolate the
output layer from the remaining of the neural network by using a frozen pre-
trained model. This methodology consists of creating a scenario where no
projection drift occurs. Indeed, projection drifts may only lead to a
deterioration of performance. Hence our study can eliminate approaches that
would, in all cases, fail under such conditions. Our findings can also be
directly transferred to settings where the projection drift is negligible.

In a preliminary experiment, we benchmark all output layer types in an i.i.d.
setting to fix some of their hyper-parameters for further continual training.
The conclusion of these experiments are in Sec. 4.1. In a second setup, we
train the various output layers in continual learning scenarios. Finally, we
experiment with a vanilla continual approach close to a rehearsal approach: we
train using only on a subset of the training data, and compare layer
parameterizations in this low data regime.

###  4.1 Datasets and pre-trained Models

We conduct our experiments on CIFAR10/CIFAR100 [17] and Core50 [22]. Core50 is
a dataset composed of 50 objects from 10 categories (e.g. glasses, phone,
cup…), filmed in 11 environments (8 for training and 3 for evaluation). We
denote Core10, the 10 classes version where we only predict the category, and
Core50, the full 50 classes version. We use both to create various types of
scenarios.

The preliminary experiments consist of testing the basic output layer (without
masking) to select a learning rate (among $0.1$, $0.01$ and $0.001$ ) and
architecture for pre-trained models (among ResNet, VGG16 and GoogleNet
available on the torchvision library [24]). While choosing pre-trained models,
we made sure that pre-training had not been done using the same dataset that
we used in our continual learning experiments (e.g. we did not choose a model
pre-trained on CIFAR10 for our CIFAR10 experiments), which would otherwise be
"cheating" as the pre-training would have been made using all available data,
and not been restricted to data sequentially made available as in continual
learning.

To maximize the performance of all layers, we selected hyper-parameters and
architecture on i.i.d. settings with all data. We selected a learning rate of
$0.01$ for the original linear layer and original linear layer without bias
(Linear_no_bias) as well as for their masked counterpart and $0.1$ for the
other layers. Secondly, we found that for Core50 and Core10 experiments, the
ResNet model performs best. We also eliminated the CIFAR100 dataset from our
incremental experiments because the pre-trained model we tested (pre-trained
on CIFAR10) did not permit us to achieve a good enough accuracy. We kept
CIFAR10, Core10, and Core50 for subsequent experiments. We added CUB200 and a
modified version of CIFAR100 (referred to as CIFAR100Lifelong), for continual
experiments. Moreover, we see that 5 epochs per task are sufficient on these
datasets, and we will keep this number of epochs for all experiments.

###  4.2 Continual Experiments Settings

In these experiments, we evaluate the capacity of every output layer to learn
continually with a fixed feature extractor. We evaluate the CL performance on
virtual concept drifts (incremental scenario), on domain drifts (lifelong
scenario) (cf Sec. 3.3). Incremental settings consist of a sequence of tasks
where all new tasks bring new unknown classes. Lifelong settings consist of a
sequence of tasks where each new task brings new examples of known classes. We
also evaluate the best layers in a mixed scenario with both kinds of drifts in
the end. Our scenarios can be reproduced easily with continuum library [9].
Our experiments use the single-head framework [10], as opposed to multi-head
where the task id is used for inference to pre-select a subset of classes. We
do not use task labels during training or testing. The change in labels can
signal a change of tasks. However, it does not trigger any continual
mechanisms in our experiments, and all training are done using stochastic
gradient descent with a momentum of $0.9$.

  * •

Incremental Scenarios: CIFAR10, Core50, CUB200: Incremental scenarios are
characterized by a virtual concept drift between two tasks. These settings
evaluate the capacity of learning incremental new classes and distinguishing
them from the others. The core50 scenario is composed of 10 tasks; each of
them is composed of 5 classes, the CIFAR10 scenario is composed of 5 tasks
with 2 classes each, and the CUB200 scenario is composed of 10 tasks of 20
classes each.

  * •

Lifelong Scenarios: Core10Lifelong, CIFAR100Lifelong:  Lifelong settings are
characterized by a domain drift between two tasks. The classes stay the same,
but the instances change. These settings evaluate the capacity of improving at
classifying with new data. The Core10Lifelong scenario is composed of 8 tasks
with 10 classes in a given environment, each new task, we visit a new
environment. CIFAR100Lifelong is composed of 5 tasks, with 20 labels each that
also stay the same. Data are labeled with the coarse labels of CIFAR100.
However, data are shared between tasks using the original label to ensure a
domain drift between tasks.

  * •

Mixed Scenario: Core10Mix:  In this scenario, a new task is triggered by
either a virtual concept drift or a domain drift. In this case, the domain
drift is characterized by the transition from one object to another in the
same category. This scenario is composed of 50 tasks, where each task
corresponds to a new object (with category annotation) in all its training
domains. So a new task is always a new object, but it is either data from a
new class or from a known class visited through another object.

The results for those experiments are in Sec. 5.1. We report at each epoch the
average accuracy over the whole test set of the scenario (with data of all
tasks). This measure makes it possible to report the accuracy on a fixed test
set and makes the visualization of progress in solving the complete scenario
easier.

###  4.3 Subset Experiments Settings

In many continual learning approaches, a subset of the training is saved,
either for rehearsal purposes or for validation purposes [2, 8, 25]. In this
experiment, we evaluate the capacity of each layer to be trained from scratch
with such subsets. It could be more efficient to continually train the feature
extractor in a continual end-to-end setting and learn a suitable output layer
afterward from a subset of data. This procedure is very fast, and it avoids
all problems that projection drifts or any other drift can create. The only
(not so simple) problem to solve is which examples to store and how many.
Nevertheless, the successes in few-shot learning [18, 11, 34] or fast
adaptation [4] show that when the feature space is good enough, only a small
set of data is enough to identify a class.

##  5 Results

###  5.1 Continual Experiments

Figure 4: Linear layers comparison: Each layer has been experimented on on 8
different task orders for each scenario.

(a) Incremental vs Lifelong

(b) Incremental Scenarios

(c) gradient vs no gradient

Figure 5: Experiments in continual scenarios. Highlights of masking effects
and similarity based layers.

We split the results of continual experiments into four steps; first (Fig. 4),
we compare the various parameterizations of a linear layer presented in Sec.
3, secondly (Fig. 5(a)), we compare the effect of masking in incremental and
lifelong scenarios, thirdly (Fig. 5(b)), we analysis which layer (among the
selected in the first step) benefits the most from masking, finally (Fig.
5(c)), we compare the best combinations of layer-masking with layers that are
not trained by gradient descent, namely: KNN, MeanLayer, and SLDA. (NB: we
keep also the original linear layer as baseline).

The results presented in the various figures lead us to the following
conclusions:

  * •

Among the reparameterizations of linear layers, Coslayer and WeightNorm are
the overall best performing. Indeed, in incremental settings, they are usually
outperforming other layers (cf Fig. 4). Both of them only rely on the angle
between data and output vectors, which reduces the risk of interference as
described in Sec. 3. The only difference between these layers is that in
WeightNorm, the predictions are weighted by the norm of the latent vectors (cf
Coslayer eq. 4, WeightNorm eq. 3), which scales the gradients thus may change
the learning dynamics.

  * •

Linear layers do not perform similarly while exposed to different types of
drift. A tendency that emerges from these experiments (Fig. 4) is that all
layers are compatible with lifelong learning even if masking may create some
instability. While, in incremental and mixed scenarios, the layers relying
only on angles between data and output vectors and masking helps to avoid
weight modification.

  * •

Removing the bias on the linear layer has no effect. In figure 4, we do not
observe any improvement by removing the bias from the linear layers in our
experiments. This questions [35]’s observations that correcting the bias can
mitigate catastrophic forgetting (CF). Nevertheless, we hypothesize that the
effect of the bias in continual learning depends on the task at hand.

  * •

Masking is efficient in incremental settings to avoid weight modifications.
The second step (Fig. 5(a)) compares the effect of masking in incremental and
lifelong scenarios, by averaging results of all layers in scenarios of both
types and comparing them with their masked counterpart. The figure 5(a) shows
clearly that incremental scenarios benefit the most from masking in particular
single masking. The figure 5(a), compares layers in the incremental scenarios.
It shows us that the layer that benefits the most from masking is Coslayer, it
even outperforms all the other combinations. Hence, for later experiment we
keep the single masked coslayer ( Coslayer_Masked ) and the WeightNorm as the
best not masked layer.

  * •

In lifelong, we cannot avoid weight modification with masking. In lifelong
scenarios, tasks use the same output vectors, i.e., the same weights; Hence,
masking is less interesting than in incremental scenarios, indeed, the figure
5(a) shows some degradation of the original performance in lifelong scenarios.
Note that in lifelong group masking is the same as no masking since all
classes are in all tasks.

  * •

Similarity-based layers perform well regardless of the data drift type (fig.
5(c)). These layers are easy to train since they need to see the data only
once to achieve their best performance. In fig. 4 these layers over all
scenarios perform well in comparison with the best linear layers, especially
SLDA. Nevertheless, those layers can not be used to backpropagate gradient and
therefore can not be used to finetune the encoder and necessarily rely on a
pre-trained model. They could, however, be associated with another layer to
train the feature extractor as in iCARL [27].

Those experiments show us that the reparametrization of the output layer such
as weightnorm and coslayer improves the ability to learn continually in
comparison with the original linear layer. It also indicates the best
parametrization depends on the data distribution drifts. In the next section,
to better understand how the various layers could perform with a continual
algorithm. We study the various layers trained iid with only a subset of the
data, it simulates an approach of continual learning with replay.

###  5.2 Training with a Subset of Data

Table 1: Subset experiments: Mean Accuracy and standard deviation on 8 runs
with different seeds on Core50. In bold, best performing layers, in the subset
setting.

OutLayer | Dataset $\backslash$ Subset | 100 | 200 | 500 | 1000 | All
---|---|---|---|---|---|---
CosLayer | Core50 |  $10.30$$\pm 1.98$ |  $12.43$$\pm 3.56$ |  $18.11$$\pm 4.17$ |  $26.48$$\pm 4.16$ |  $53.93$$\pm 0.90$
WeightNorm | Core50 |  $32.06$$\pm 2.93$ |  $44.32$$\pm 1.92$ |  $61.91$$\pm 1.10$ |  $69.44$$\pm 0.67$ |  $77.04$$\pm 0.33$
OriginalWeightNorm | Core50 |  $33.19$$\pm 2.62$ |  $45.68$$\pm 1.96$ |  $62.39$$\pm 1.01$ |  $69.17$$\pm 0.65$ |  $76.85$$\pm 0.61$
CosLayer-Masked | Core50 |  $30.62$$\pm 2.57$ |  $41.22$$\pm 1.68$ |  $56.47$$\pm 1.13$ |  $63.10$$\pm 0.62$ |  $63.43$$\pm 5.78$
WeightNorm-Masked | Core50 |  $12.95$$\pm 2.07$ |  $14.98$$\pm 2.89$ |  $21.13$$\pm 1.22$ |  $23.42$$\pm 2.49$ |  $31.18$$\pm 17.50$
OriginalWeightNorm-Masked | Core50 |  $12.27$$\pm 2.59$ |  $17.42$$\pm 2.52$ |  $20.82$$\pm 4.94$ |  $22.21$$\pm 5.50$ |  $22.43$$\pm 4.36$
Linear | Core50 |  $32.80$$\pm 2.67$ |  $44.88$$\pm 1.94$ |  $61.80$$\pm 1.06$ |  $69.03$$\pm 0.65$ |  $76.29$$\pm 0.34$
Linear-Masked | Core50 |  $7.37$$\pm 1.98$ |  $8.20$$\pm 2.68$ |  $14.02$$\pm 5.88$ |  $18.46$$\pm 2.60$ |  $27.25$$\pm 28.39$
Linear-no-bias-Masked | Core50 |  $7.36$$\pm 1.97$ |  $8.22$$\pm 2.67$ |  $14.04$$\pm 5.88$ |  $18.48$$\pm 2.55$ |  $25.81$$\pm 29.13$
KNN | Core50 |  $25.95$$\pm 3.09$ |  $34.17$$\pm 2.14$ |  $45.50$$\pm 2.04$ |  $52.13$$\pm 1.19$ |  $65.50$$\pm 0.14$
SLDA | Core50 |  $17.30$$\pm 1.23$ |  $30.23$$\pm 5.58$ |  $17.71$$\pm 1.47$ |  $60.14$$\pm 0.93$ |  $78.55$$\pm 0.03$
MeanLayer | Core50 |  $28.81$$\pm 2.68$ |  $40.71$$\pm 1.65$ |  $56.52$$\pm 1.29$ |  $63.20$$\pm 0.79$ |  $71.51$$\pm 0.00$
MedianLayer | Core50 |  $26.83$$\pm 2.26$ |  $36.03$$\pm 1.65$ |  $53.07$$\pm 1.15$ |  $60.73$$\pm 0.77$ |  $70.22$$\pm 0.00$

(d) Comparison of performance of a significant set of output layers.

(e) Impact of masking.

Figure 6: Subset Experiments realized on 8 seeds. The test accuracy is on the
full test set. Results are averaged over all datasets experimented (CIFAR10,
Core50, CUB200, Core10, CIFAR100Lifelong). We compare the different layer
types in various subset setting. _Masked denotes layer trained with single
masking.

In these experiments, we randomly select a sub-sample of the training dataset,
and we train the output layers on it. The experiment’s results are gathered in
Fig. 6. We show also a representative subset of results with layer
performances on Core50 in Table 5.2.

The goal is to measure how the various layers can learn with a low amount of
data. It gives us also a good continual baseline close to a replay approach
such as GDump [25]. We save samples in the memory, and we train at the end of
the scenario only on the memory’s samples. This approach performs particularly
well in comparison to many continual approaches. It gives us some insight into
how layers can perform in few-shot scenarios and replay scenarios with limited
data. Here are some conclusions of these experiments:

$\bullet$ Masking does not work in an iid setting. In table 5(e), most of the
masked layers underperform, except the CosLayer, which performs better with a
mask (cf Fig. 5(d)).

$\bullet$ WeightNorm and CosLayer_Masked are still very competitive among
linear layers. In figure 5(d), they are among the best linear layers. The
original linear layer, which was bad performing in continual settings is well-
performing when the distribution is i.i.d. Hence, the original linear layer is
still a good option for few-shot learning or with replay.

$\bullet$ Similarity-based layers tend to slightly under-perform linear
layers. In Fig. 5(d), they do not perform as well as the best linear layers,
losing the advantages they had in continual settings. They can still have a
decent accuracy.

These experiments with iid training on a low data regime show that in such
settings, the reparameterizations of the linear layer are well-performing. In
conclusion of all the experiments, we recommend using WeigthNorm (especially
in lifelong) or CosLayer_Masked in continual scenarios whatever the setting
and the training procedure.

##  6 Application for End-to-End with Replay

Figure 7: Final test accuracy vs. replay balance: Group masking reduces the
need for replay. The replay balance is ratio between instances of a replayed
class and a new class.

The adaptation of the output layer can also be helpful for end-to-end
training. In this example experiment, we show that group masking improves
performance with replay, particularly when the amount of replay is low.

We train a resnet model on a 5-tasks class-incremental setting with CIFAR10.
We use the WeightNorm layer with group masking and without masking. We select
this baseline since it is the most consistently well performing in our
previous experiments. The CL approach is a vanilla replay method with a random
selection of samples to populate the buffer (2000 examples per class). We use
a large buffer since memory is generally not the main bottleneck. In the
experiment, we variate the amount of replay balance to control the replay
balance. The replay balance is the ratio between new and old classes’
instances. $0.5$ replay balance means two times more instances for new
classes. The lower the replay balance, the more imbalanced the class
distribution is, and the lower the total compute is. Indeed, by replaying
less, they are fewer examples to process. We use a batch size of 8 with 3
seeds. The results are provided in Fig. 7. Group masking improves final
performance, especially with low replay balance. This shows that group masking
makes training more compute efficient when using replay and lowers the need
for replay.

##  7 Discussion

In this paper, we studied and proposed various approaches that mitigate CF in
the output layer by addressing weights modification and interference, by
restricting ourselves to a setup where there is no projection drift.

On the opposite, multi-head architectures use the test task label and use a
head specific to the current task. It avoids interference and weight
modification by training separate heads. Interestingly, in multi-heads
settings, using regularization or dynamic architecture , have been shown to
perform well on end-to-end tasks. Then, they are able to deal with projection
drifts: the head of task 0 still works when applied to the feature extractor
trained up until the last task.

Unfortunately, end-to-end approaches that work in multi-heads can not be
directly connected with our findings to create a good end-to-end single head
approach. Indeed, single-head models need discriminative inter-task
representations that are not necessary for multi-head settings. Therefore, the
constraints on the feature extractor are not the same, and direct transfer is
not possible.

Dealing with projection drift in single-head settings stays one of the biggest
challenges in continual learning. Replay might be a simple solution to
simulate an iid training and avoid problems with projection drift. However,
even if replay might be necessary for continual learning scenarios [7, 21] ,
replaying data may have consequences in the memory needs and compute needs.
Therefore, it is worth finding training procedures, architectures, or
regularizations that do not necessarily replace replay but at least reduce its
need as group masking does in experiment of Sec. 6.

##  8 Conclusion

In this paper, we conduct an empirical evaluation of the various output layers
of deep neural networks in CL scenarios. This evaluation gives us clear
insights into how output layers learn continually or on a low data regime. We
introduced WeigthNorm and CosLayer_Masked layers, that showed clear advantages
in all setting types. We also showed how distribution drifts might affect the
output layers differently, namely in lifelong and incremental settings. This
perspective should incite researchers to study more the connection between
distribution drifts and catastrophic forgetting.

On one hand, classifiers such as KNN, MeanLayer, and SLDA provide very strong
baselines that are very effective with a frozen feature extractor. On the
other hand, our results show that by a reparameterization of the classical
linear layer, we can greatly improve performance, especially in incremental
settings. This reparameterization helps to reduce the risks of interference
and weight modifications. Some output layer types can learn continually when
there is no projection drift with stochastic gradient training. These findings
could be integrated into all end-to-end training to make continual training
more efficient.

## References

  * [1] Y. Aliyari Ghassabeh, F. Rudzicz, and H. A. Moghaddam.  Fast incremental lda feature extraction.  Pattern Recognition, 48(6):1999–2012, 2015.
  * [2] P. Buzzega, M. Boschini, A. Porrello, D. Abati, and S. Calderara.  Dark experience for general continual learning: a strong, simple baseline.  In H. Larochelle, M. Ranzato, R. Hadsell, M. F. Balcan, and H. Lin, editors, Advances in Neural Information Processing Systems 33, pages 15920–15930. Curran Associates, Inc., 2020.
  * [3] L. Caccia, R. Aljundi, T. Tuytelaars, J. Pineau, and E. Belilovsky.  Reducing representation drift in online continual learning.  arXiv preprint arXiv:2104.05025, 2021.
  * [4] M. Caccia, P. Rodriguez, O. Ostapenko, F. Normandin, M. Lin, L. Caccia, I. Laradji, I. Rish, A. Lacoste, D. Vazquez, and L. Charlin.  Online fast adaptation and knowledge accumulation: a new approach to continual learning.  NeurIPS, 2020.
  * [5] C. Chatterjee and V. Roychowdhury.  On self-organizing algorithms and networks for class-separability features.  IEEE Transactions on Neural Networks, 8(3):663–678, 1997.
  * [6] G. Demir and K. Ozmehmet.  Online local learning algorithms for linear discriminant analysis.  Pattern Recognition Letters, 26(4):421–431, 2005.  ICAPR 2003.
  * [7] T. Diethe, T. Borchert, E. Thereska, B. d. B. Pigem, and N. Lawrence.  Continual learning in practice.  In NeurIPS Continual Learning Workshop, 2018.
  * [8] A. Douillard, M. Cord, C. Ollion, T. Robert, and E. Valle.  Podnet: Pooled outputs distillation for small-tasks incremental learning.  In Proceedings of the IEEE European Conference on Computer Vision (ECCV), 2020.
  * [9] A. Douillard and T. Lesort.  Continuum: Simple management of complex continual learning scenarios, 2021\.
  * [10] S. Farquhar and Y. Gal.  Towards robust evaluations of continual learning.  arXiv preprint arXiv:1805.09733, 2018.
  * [11] L. Fei-Fei, R. Fergus, and P. Perona.  One-shot learning of object categories.  IEEE transactions on pattern analysis and machine intelligence, 28(4):594–611, 2006.
  * [12] T. L. Hayes and C. Kanan.  Lifelong machine learning with deep streaming linear discriminant analysis.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pages 220–221, 2020.
  * [13] K. He, X. Zhang, S. Ren, and J. Sun.  Deep residual learning for image recognition.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016.
  * [14] S. Hou, X. Pan, C. C. Loy, Z. Wang, and D. Lin.  Learning a unified classifier incrementally via rebalancing.  In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2019.
  * [15] S. Ioffe and C. Szegedy.  Batch normalization: Accelerating deep network training by reducing internal covariate shift.  In International conference on machine learning, pages 448–456. PMLR, 2015.
  * [16] J. Kirkpatrick, R. Pascanu, N. Rabinowitz, J. Veness, G. Desjardins, A. A. Rusu, K. Milan, J. Quan, T. Ramalho, A. Grabska-Barwinska, et al.  Overcoming catastrophic forgetting in neural networks.  Proc. of the national academy of sciences, 2017.
  * [17] A. Krizhevsky and G. Hinton.  Learning multiple layers of features from tiny images.  Technical Report, 2009.
  * [18] B. Lake, R. Salakhutdinov, J. Gross, and J. Tenenbaum.  One shot learning of simple visual concepts.  In Proceedings of the Annual Meeting of the Cognitive Science Society, volume 33, 2011.
  * [19] T. Lesort, M. Caccia, and I. Rish.  Understanding continual learning settings with data distribution drift analysis.  arXiv preprint arXiv:2104.01678, 2021.
  * [20] T. Lesort, N. Díaz-Rodríguez, J.-F. Goudou, and D. Filliat.  State representation learning for control: An overview.  Neural Networks, 2018.
  * [21] T. Lesort, A. Stoian, and D. Filliat.  Regularization shortcomings for continual learning.  arXiv preprint arXiv:1912.03049, 2019.
  * [22] V. Lomonaco and D. Maltoni.  Core50: a new dataset and benchmark for continuous object recognition.  In S. Levine, V. Vanhoucke, and K. Goldberg, editors, Proceedings of the 1st Annual Conference on Robot Learning, volume 78 of Proceedings of Machine Learning Research, pages 17–26. PMLR, 13–15 Nov 2017.
  * [23] V. Lomonaco, D. Maltoni, and L. Pellegrini.  Rehearsal-free continual learning over small non-iid batches.  In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pages 989–998. IEEE Computer Society, 2020.
  * [24] A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga, A. Desmaison, A. Kopf, E. Yang, Z. DeVito, M. Raison, A. Tejani, S. Chilamkurthy, B. Steiner, L. Fang, J. Bai, and S. Chintala.  Pytorch: An imperative style, high-performance deep learning library.  In H. Wallach, H. Larochelle, A. Beygelzimer, F. dAlché Buc, E. Fox, and R. Garnett, editors, Advances in Neural Information Processing Systems 32, pages 8024–8035. Curran Associates, Inc., 2019.
  * [25] A. Prabhu, P. H. Torr, and P. K. Dokania.  Gdumb: A simple approach that questions our progress in continual learning.  In European Conference on Computer Vision, pages 524–540. Springer, 2020.
  * [26] A. Raffin, A. Hill, K. R. Traoré, T. Lesort, N. Díaz-Rodríguez, and D. Filliat.  Decoupling feature extraction from policy learning: assessing benefits of state representation learning in goal based robotics.  Workshop on “Structure and Priors in Reinforcement Learning” (SPiRL) at ICLR, 2019.
  * [27] S.-A. Rebuffi, A. Kolesnikov, G. Sperl, and C. H. Lampert.  icarl: Incremental classifier and representation learning.  In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 2001–2010, 2017.
  * [28] T. Salimans and D. P. Kingma.  Weight normalization: A simple reparameterization to accelerate training of deep neural networks.  arXiv preprint arXiv:1602.07868, 2016.
  * [29] Shaoning Pang, S. Ozawa, and N. Kasabov.  Incremental linear discriminant analysis for classification of data streams.  IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 35(5):905–914, 2005.
  * [30] K. Simonyan and A. Zisserman.  Very deep convolutional networks for large-scale image recognition.  arXiv preprint arXiv:1409.1556, 2014.
  * [31] A. Stooke, K. Lee, P. Abbeel, and M. Laskin.  Decoupling representation learning from reinforcement learning.  arXiv preprint arXiv:2009.08319, 2020.
  * [32] C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, and A. Rabinovich.  Going deeper with convolutions, 2014.
  * [33] G. M. van de Ven and A. S. Tolias.  Three scenarios for continual learning.  arXiv preprint arXiv:1904.07734, 2019.
  * [34] W. Wang, V. W. Zheng, H. Yu, and C. Miao.  A survey of zero-shot learning: Settings, methods, and applications.  ACM Trans. Intell. Syst. Technol., 10(2):13:1–13:37, Jan. 2019\.
  * [35] Y. Wu, Y. Chen, L. Wang, Y. Ye, Z. Liu, Y. Guo, and Y. Fu.  Large scale incremental learning.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 374–382, 2019.
  * [36] B. Zhao, X. Xiao, G. Gan, B. Zhang, and S.-T. Xia.  Maintaining discrimination and fairness in class incremental learning.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13208–13217, 2020.

## Appendix

##  Appendix A Scenarios description

###  A.1 CIFAR10 5 tasks

A classical incremental setting, with 2 classes per task with original CIFAR10
dataset.

####  A.1.1 CIFAR100Lifelong 5 tasks

CIFAR100 dataset has two labelization systems: the classes labels and the
coarse labels. There are 20 coarse labels gathering 5 classes each. To create
CIFAR100Lifelong 5 tasks, we share data into tasks such as having one data for
each coarse label in each task. On the other hand, we use class labels to
ensure that the data is different for each task.

We have then 5 tasks with data for all coarse labels and from 20 different
classes. A class does not appear in 2 tasks, as in an incremental. However,
the model should predict the coarse label. We obtain a scenario with 5 tasks
with always the same 20 labels. However, the data change, and the model should
learn continually to predict the coarse labels for data for all classes
correctly. This scenario makes it possible to measure the resistance to mode
change (change of class) while continually learning the coarse label
classification.

###  A.2 Core50 10 tasks

A classical incremental setting, with 5 classes per task with the 50 objects
core50.

###  A.3 Core10Lifelong 8 tasks

Core50 is composed of 50 objects, equitably shared into 10 classes filmed in
11 environments (8 environments for train and 3 for tests.) In Core10Lifelong,
we have 8 tasks where we explore each time one environment with all objects.
Objects are annotated with their class labels. Hence, we have 8 tasks of
10-way classification. The object stays the same from one task to another;
only the environment changes. This scenario makes it possible to measure the
resistance to background change while learning continually.

###  A.4 Core10Mix 50 tasks

In this scenario, each task contains the data corresponding to one object in
all of its environments. We have hence a sequence of 50 tasks. However,
objects are annotated with their classes labels. Hence all classes are
revisited 5 times in the scenario (in a random way). This scenario is a
mixture of incremental and lifelong to measure the ability to solve a scenario
with several types of data distribution drifts.

##  Appendix B SLDA Layer

The approach we use (proposed by [12]) stores one mean vector per class
$\mu_{k}$ with a counter $c_{k}\in\mathbb{R}$ as well as a covariance matrix
of the same size as $A$ (common to all classes). The covariance matrix’s
update is done with:

$\Sigma_{t+1}=\frac{t\Sigma_{t}+\Delta_{t}}{t+1}$ (6)

Where $\Delta_{t}$ is:

$\Delta_{t}=\frac{t(z_{t}-\mu_{k})^{2}}{t+1}$ (7)

For prediction, SLDA uses a precision matrix
$\Lambda=[(1-\epsilon)\Sigma+\epsilon I]^{-1}$, where $\epsilon=10^{-4}$ is
called the shrinkage parameter and $I\in\mathbb{R}^{d\times d}$ is the
identity matrix.

Then two vectors $w$ and $b$ are computed $w_{k}=\Lambda\mu_{k}$,
$b_{k}=-\frac{1}{2}(\mu_{k}^{\top}\Lambda\mu_{k})$

And finally: $o_{k}=\langle z_{t},w_{k}\rangle+b_{k}$

##  Appendix C Preliminary Experiments

###  C.1 Preliminary Experiments Setting

The preliminary experiments are designed to see how the different output
layers work without continual learning constraints. This experiment is also
conducted to select the learning rate and architecture for further
experiments.

We experimented with CIFAR10 and CIFAR100 datasets, and we use resnet20 pre-
trained model from here 111https://github.com/chenyaofo/pytorch-cifar-models
(BSD 3-Clause License). The main idea is to compare those results with our
results. We take the frozen pre-trained model and replace the output layer
with a new one we train on all data. (1) We experiment with training on
CIFAR10 with a neural network pre-trained on CIFAR100 and the opposite. (2) We
also trained on CIFAR10 with the model pre-trained on CIFAR10 to see if we
would recover the original accuracy, and we did the same for CIFAR100. Doing
continual learning on a dataset with a model pre-trained on this dataset is
not recommended since it means having access to the whole data from the
beginning of training. We did not use (2) setting in the further experiments,
and it was just used as a reference.

(3) We also experiment also with Core50 dataset [22]. We used models pre-
trained on ImageNet available on the torchvision library [24]. We used VGG16,
GoogleNet and Resnet, which have a latent space of dimension 2048, 1024, and
512. We experimented with the Core50 dataset, Core50 setting with 50 classes
corresponding to the object id, and Core10 setting with 10 classes
corresponding to the object category.

In these preliminary experiments, we evaluate all layers without their masking
variant to verify that the pre-trained models were suited to the continual
learning scenarios.

###  C.2 Preliminary Experiments

The summary of the preliminary results is reported in Table C.2. The full
results can be found in Table C.2. As described in section C.1, we experiment
with the simplest output layer for this preliminary experiment, i.e., linear
layer, linear layer without bias (Linear_NB), CosLayer, WeightNorm layer,
Original WeightNorm (OWeightNorm), KNN, SLDA, and MeanLayer.

Table 2: Preliminary experiments to select learning rate and architectures
with existing type of layers. Experiment realized on 8 seeds
$[0,1,2,3,4,5,6,7]$.

Dataset | LR | Architecture | Pretraining | Linear | Linear_NB | CosLayer | WeightNorm | OWeightNorm | KNN | SLDA | MeanLayer
---|---|---|---|---|---|---|---|---|---|---|---
CIFAR10 | 0.1 | resnet | CIFAR10 |  $92.55$$\pm 0.02$ |  $92.55$$\pm 0.02$ |  $92.49$$\pm 0.05$ |  $92.56$$\pm 0.03$ |  $92.55$$\pm 0.04$ |  $91.79$$\pm 0.26$ |  $92.37$$\pm 0.00$ |  $92.44$$\pm 0.00$
CIFAR10 | 0.1 | resnet | CIFAR100 |  $65.37$$\pm 2.74$ |  $68.58$$\pm 9.55$ |  $66.55$$\pm 9.83$ |  $68.55$$\pm 0.21$ |  $69.21$$\pm 0.36$ |  $60.26$$\pm 0.28$ |  $66.90$$\pm 0.02$ |  $63.27$$\pm 0.00$
CIFAR100 | 0.1 | resnet | CIFAR10 |  $17.32$$\pm 0.23$ |  $17.14$$\pm 0.22$ |  $13.09$$\pm 0.22$ |  $15.11$$\pm 0.22$ |  $16.88$$\pm 0.30$ |  $14.20$$\pm 0.17$ |  $26.44$$\pm 0.03$ |  $14.48$$\pm 0.00$
CIFAR100 | 0.1 | resnet | CIFAR100 |  $63.97$$\pm 0.10$ |  $63.94$$\pm 0.11$ |  $60.91$$\pm 0.24$ |  $63.62$$\pm 0.09$ |  $63.71$$\pm 0.15$ |  $60.69$$\pm 0.12$ |  $61.56$$\pm 0.01$ |  $61.84$$\pm 0.00$
Core50 | 0.1 | resnet | ImageNet |  $76.58$$\pm 0.41$ |  $76.57$$\pm 0.41$ |  $53.93$$\pm 0.90$ |  $77.04$$\pm 0.33$ |  $76.85$$\pm 0.61$ |  $65.50$$\pm 0.14$ |  $78.55$$\pm 0.03$ |  $71.51$$\pm 0.00$
Core50 | 0.1 | vgg | ImageNet |  $71.69$$\pm 0.50$ |  $71.44$$\pm 0.30$ |  $61.13$$\pm 0.39$ |  $72.13$$\pm 0.45$ |  $71.53$$\pm 0.23$ |  $56.86$$\pm 0.08$ |  $69.98$$\pm 0.03$ |  $66.67$$\pm 0.00$
Core50 | 0.1 | googlenet | ImageNet |  $70.13$$\pm 0.27$ |  $70.13$$\pm 0.27$ |  $47.81$$\pm 3.26$ |  $66.37$$\pm 0.74$ |  $69.27$$\pm 0.35$ |  $59.45$$\pm 0.11$ |  $70.60$$\pm 0.11$ |  $60.85$$\pm 0.15$
Core10Lifelong | 0.1 | resnet | ImageNet |  $85.45$$\pm 0.51$ |  $85.62$$\pm 0.58$ |  $78.76$$\pm 0.36$ |  $86.29$$\pm 0.29$ |  $86.07$$\pm 0.51$ |  $75.71$$\pm 1.50$ |  $88.06$$\pm 0.01$ |  $79.61$$\pm 0.00$
Core10Lifelong | 0.1 | vgg | ImageNet |  $85.52$$\pm 0.57$ |  $85.34$$\pm 0.57$ |  $79.39$$\pm 0.18$ |  $85.95$$\pm 0.61$ |  $85.56$$\pm 0.83$ |  $75.14$$\pm 0.88$ |  $83.69$$\pm 0.03$ |  $78.55$$\pm 0.00$
Core10Lifelong | 0.1 | googlenet | ImageNet |  $85.08$$\pm 0.47$ |  $85.08$$\pm 0.47$ |  $78.23$$\pm 0.33$ |  $83.05$$\pm 0.38$ |  $84.97$$\pm 0.29$ |  $77.57$$\pm 0.96$ |  $86.33$$\pm 0.18$ |  $76.92$$\pm 0.10$

The results of those preliminary experiments are that globally the learning
rate of $0.1$ is adapted to all settings. Secondly, we found that for Core50
and Core10 experiments, the ResNet model was the best suited. Moreover, we see
that 5 epochs per task are sufficient to learn a correct solution. Concerning
CIFAR experiments, the reported top-1 accuracy on CIFAR10 and CIFAR100 of the
original pre-trained models are respectively $92.60\%$, $69.83\%$. The
settings CIFAR10 pre-trained on CIFAR10 (same for CIFAR100) show us that the
training procedure can recover almost the best performance. In CIFAR100, there
is a drop of $6\%$ of accuracy, but it stays reasonable in a 100-way
classification task.

Even if not very surprising, one interesting result is that training on
CIFAR100 with a model pre-trained on CIFAR10 does not work at all. It
illustrates that using a pre-trained model is not always a solution in machine
learning or continual learning. They should be selected carefully, even if the
data might look similar to CIFAR10 and CIFAR100. On the other hand, the
experiment on CIFAR10 with CIFAR100 models shows a significant decrease in the
performance for all layers.

These preliminary experiments make us able to fix some hyper-parameters to
study all layers in a common setting. It also offers us interesting insight
into the drop of performance in simple experiments when using a pre-trained
model instead of training end-to-end. The results of this experiment can be
used as a baseline for continual experiments and subsets experiments.

Table 3: Preliminary experiments to select learning rate and architectures
with existing type of layers

Dataset | LR | Architecture | Pretraining | Linear | Linear_NB | CosLayer | WeightNorm | OWeightNorm | KNN | SLDA | MeanLayer
---|---|---|---|---|---|---|---|---|---|---|---
CIFAR10 | 0.1 | resnet | CIFAR10 |  $92.55$$\pm 0.02$ |  $92.55$$\pm 0.02$ |  $92.49$$\pm 0.05$ |  $92.56$$\pm 0.03$ |  $92.55$$\pm 0.04$ | - | - | -
CIFAR10 | 0.01 | resnet | CIFAR10 |  $92.59$$\pm 0.04$ |  $92.59$$\pm 0.03$ |  $92.48$$\pm 0.03$ |  $92.54$$\pm 0.02$ |  $92.60$$\pm 0.03$ | - | - | -
CIFAR10 | 0.001 | resnet | CIFAR10 |  $92.49$$\pm 0.04$ |  $92.49$$\pm 0.04$ |  $67.03$$\pm 10.29$ |  $92.39$$\pm 0.10$ |  $92.57$$\pm 0.04$ | - | - | -
CIFAR10 | n/a | resnet | CIFAR10 | - | - | - | - | - |  $91.79$$\pm 0.26$ |  $92.37$$\pm 0.00$ |  $92.44$$\pm 0.00$
CIFAR10 | 0.1 | resnet | CIFAR100 |  $65.37$$\pm 2.74$ |  $68.58$$\pm 9.55$ |  $66.55$$\pm 9.83$ |  $68.55$$\pm 0.21$ |  $69.21$$\pm 0.36$ | - | - | -
CIFAR10 | 0.01 | resnet | CIFAR100 |  $75.13$$\pm 10.09$ |  $75.13$$\pm 10.08$ |  $66.46$$\pm 15.03$ |  $68.43$$\pm 0.08$ |  $69.27$$\pm 0.25$ | - | - | -
CIFAR10 | 0.001 | resnet | CIFAR100 |  $73.28$$\pm 11.11$ |  $73.30$$\pm 11.10$ |  $18.85$$\pm 12.54$ |  $60.71$$\pm 0.41$ |  $67.79$$\pm 0.14$ | - | - | -
CIFAR10 | n/a | resnet | CIFAR100 | - | - | - | - | - |  $60.26$$\pm 0.28$ |  $66.90$$\pm 0.02$ |  $63.27$$\pm 0.00$
CIFAR100 | 0.1 | resnet | CIFAR10 |  $17.32$$\pm 0.23$ |  $17.14$$\pm 0.22$ |  $13.09$$\pm 0.22$ |  $15.11$$\pm 0.22$ |  $16.88$$\pm 0.30$ | - | - | -
CIFAR100 | 0.01 | resnet | CIFAR10 |  $14.86$$\pm 0.20$ |  $14.86$$\pm 0.18$ |  $3.02$$\pm 0.43$ |  $12.18$$\pm 0.28$ |  $15.20$$\pm 0.23$ | - | - | -
CIFAR100 | 0.001 | resnet | CIFAR10 |  $6.71$$\pm 0.48$ |  $6.71$$\pm 0.48$ |  $1.22$$\pm 0.33$ |  $2.40$$\pm 0.37$ |  $6.77$$\pm 0.44$ | - | - | -
CIFAR100 | n/a | resnet | CIFAR10 | - | - | - | - | - |  $14.20$$\pm 0.17$ |  $26.44$$\pm 0.03$ |  $14.48$$\pm 0.00$
CIFAR100 | 0.1 | resnet | CIFAR100 |  $63.97$$\pm 0.10$ |  $63.94$$\pm 0.11$ |  $60.91$$\pm 0.24$ |  $63.62$$\pm 0.09$ |  $63.71$$\pm 0.15$ | - | - | -
CIFAR100 | 0.01 | resnet | CIFAR100 |  $63.70$$\pm 0.18$ |  $63.71$$\pm 0.19$ |  $6.95$$\pm 1.19$ |  $62.45$$\pm 0.26$ |  $63.96$$\pm 0.11$ | - | - | -
CIFAR100 | 0.001 | resnet | CIFAR100 |  $54.58$$\pm 0.32$ |  $54.54$$\pm 0.32$ |  $1.29$$\pm 0.25$ |  $21.35$$\pm 1.20$ |  $55.54$$\pm 0.48$ | - | - | -
CIFAR100 | n/a | resnet | CIFAR100 | - | - | - | - | - |  $60.69$$\pm 0.12$ |  $61.56$$\pm 0.01$ |  $61.84$$\pm 0.00$

Core50 & 0.1 resnet ImageNet $76.58$$\pm 0.41$ $76.57$$\pm 0.41$ $53.93$$\pm
0.90$ $77.04$$\pm 0.33$ $76.85$$\pm 0.61$ \- - -

Core50 0.01 resnet ImageNet $76.29$$\pm 0.34$ $76.29$$\pm 0.34$ $5.80$$\pm
0.89$ $73.01$$\pm 0.51$ $76.98$$\pm 0.29$ \- - -

Core50 0.001 resnet ImageNet $62.64$$\pm 0.72$ $62.65$$\pm 0.72$ $2.21$$\pm
0.56$ $30.55$$\pm 1.05$ $63.42$$\pm 0.66$ \- - -

Core50 n/a resnet ImageNet \- - - - - $65.50$$\pm 0.14$ $78.55$$\pm 0.03$
$71.51$$\pm 0.00$

Core50 0.1 vgg ImageNet $71.69$$\pm 0.50$ $71.44$$\pm 0.30$ $61.13$$\pm 0.39$
$72.13$$\pm 0.45$ $71.53$$\pm 0.23$ \- - -

Core50 0.01 vgg ImageNet $72.29$$\pm 0.28$ $72.29$$\pm 0.28$ $27.15$$\pm 1.66$
$71.43$$\pm 0.35$ $71.91$$\pm 0.37$ \- - -

Core50 0.001 vgg ImageNet $68.79$$\pm 0.32$ $68.78$$\pm 0.31$ $3.47$$\pm 0.79$
$59.54$$\pm 0.43$ $69.06$$\pm 0.32$ \- - -

Core50 n/a vgg ImageNet \- - - - - $56.86$$\pm 0.08$ $69.98$$\pm 0.03$
$66.67$$\pm 0.00$

Core50 0.1 googlenet ImageNet $70.13$$\pm 0.27$ $70.13$$\pm 0.27$ $47.81$$\pm
3.26$ $66.37$$\pm 0.74$ $69.27$$\pm 0.35$ \- - -

Core50 0.01 googlenet ImageNet $65.99$$\pm 0.50$ $65.99$$\pm 0.50$ $8.71$$\pm
1.31$ $61.77$$\pm 0.21$ $66.65$$\pm 0.52$ \- - -

Core50 0.001 googlenet ImageNet $49.61$$\pm 0.85$ $49.62$$\pm 0.85$ $2.55$$\pm
0.64$ $21.91$$\pm 1.17$ $49.76$$\pm 0.66$ \- - -

Core50 n/a googlenet ImageNet \- - - - - $59.45$$\pm 0.11$ $70.60$$\pm 0.11$
$60.85$$\pm 0.15$

Core10Lifelong 0.1 resnet ImageNet $85.45$$\pm 0.51$ $85.62$$\pm 0.58$
$78.76$$\pm 0.36$ $86.29$$\pm 0.29$ $86.07$$\pm 0.51$ \- - -

Core10Lifelong 0.01 resnet ImageNet $86.19$$\pm 0.33$ $86.19$$\pm 0.33$
$55.26$$\pm 1.84$ $85.46$$\pm 0.40$ $86.44$$\pm 0.36$ \- - -

Core10Lifelong 0.001 resnet ImageNet $82.53$$\pm 0.52$ $82.53$$\pm 0.52$
$13.38$$\pm 3.59$ $76.42$$\pm 1.03$ $83.41$$\pm 0.45$ \- - -

Core10Lifelong n/a resnet ImageNet \- - - - - $75.71$$\pm 1.50$ $88.06$$\pm
0.01$ $79.61$$\pm 0.00$

Core10Lifelong 0.1 vgg ImageNet $85.52$$\pm 0.57$ $85.34$$\pm 0.57$
$79.39$$\pm 0.18$ $85.95$$\pm 0.61$ $85.56$$\pm 0.83$ \- - -

Core10Lifelong 0.01 vgg ImageNet $85.85$$\pm 0.30$ $85.89$$\pm 0.27$
$72.44$$\pm 1.92$ $85.37$$\pm 0.24$ $85.74$$\pm 0.65$ \- - -

Core10Lifelong 0.001 vgg ImageNet $84.20$$\pm 0.14$ $84.20$$\pm 0.14$
$24.59$$\pm 3.17$ $82.14$$\pm 0.47$ $84.51$$\pm 0.11$ \- - -

Core10Lifelong n/a vgg ImageNet \- - - - - $75.14$$\pm 0.88$ $83.69$$\pm 0.03$
$78.55$$\pm 0.00$

Core10Lifelong 0.1 googlenet ImageNet $85.08$$\pm 0.47$ $85.08$$\pm 0.47$
$78.23$$\pm 0.33$ $83.05$$\pm 0.38$ $84.97$$\pm 0.29$ \- - -

Core10Lifelong 0.01 googlenet ImageNet $83.67$$\pm 0.26$ $83.67$$\pm 0.25$
$64.27$$\pm 2.53$ $82.31$$\pm 0.28$ $83.97$$\pm 0.33$ \- - -

Core10Lifelong 0.001 googlenet ImageNet $79.51$$\pm 0.36$ $79.52$$\pm 0.35$
$14.85$$\pm 2.09$ $73.90$$\pm 0.63$ $80.13$$\pm 0.30$ \- - -

Core10Lifelong n/a googlenet ImageNet \- - - - - $77.57$$\pm 0.96$ $86.33$$\pm
0.18$ $76.92$$\pm 0.10$

##  Appendix D Forgetting in Lifelong Scenarios

###  D.1 CIFAR100Lifelong

(a) CIFAR10, 5 tasks

Figure 8: Test accuracy on task 0 in CIFAR100Lifelong Scenario.

In Fig. 8, we can see that the model is forgetting the first tasks and the
masking have no effect on it. Moreover, since the single masking makes
learning less efficient with many classes, it can make the performance bad
from the beginning.

###  D.2 Core10Lifelong

Unfortunately, the Core50 test set does not make it possible to evaluate the
forgetting in each task of lifelong scenarios. Indeed, the environments of the
test set are different from the train environments. Then it is not possible to
evaluate with the test set the evolution of accuracy on past tasks. In the
lifelong scenario, we can only evaluate the performance on 3 other unknown
environments, similarly to an out of distribution evaluation.

##  Appendix E Full subsets results

Table 4: Subset experiments: Mean Accuracy and standard deviation on 8 runs
with different seeds.

OutLayer | Dataset $\backslash$ Subset | 100 | 200 | 500 | 1000 | All
---|---|---|---|---|---|---
CosLayer | CIFAR10 |  $31.81$$\pm 5.84$ |  $39.00$$\pm 4.90$ |  $48.23$$\pm 3.51$ |  $54.09$$\pm 2.66$ |  $66.55$$\pm 9.83$
WeightNorm | CIFAR10 |  $50.82$$\pm 1.91$ |  $56.90$$\pm 0.82$ |  $62.64$$\pm 0.78$ |  $65.41$$\pm 0.49$ |  $68.55$$\pm 0.21$
OriginalWeightNorm | CIFAR10 |  $48.22$$\pm 2.19$ |  $52.33$$\pm 1.53$ |  $55.91$$\pm 0.77$ |  $61.66$$\pm 0.74$ |  $69.21$$\pm 0.36$
CosLayer-Masked | CIFAR10 |  $53.45$$\pm 1.63$ |  $58.54$$\pm 0.54$ |  $61.25$$\pm 0.67$ |  $62.51$$\pm 0.46$ |  $67.36$$\pm 9.52$
WeightNorm-Masked | CIFAR10 |  $36.55$$\pm 3.26$ |  $39.04$$\pm 2.23$ |  $41.23$$\pm 2.15$ |  $42.44$$\pm 1.62$ |  $33.89$$\pm 5.33$
OriginalWeightNorm-Masked | CIFAR10 |  $22.79$$\pm 10.50$ |  $22.68$$\pm 9.30$ |  $21.07$$\pm 9.97$ |  $22.45$$\pm 6.73$ |  $33.18$$\pm 3.31$
Linear | CIFAR10 |  $49.13$$\pm 2.31$ |  $54.31$$\pm 1.43$ |  $60.16$$\pm 0.72$ |  $64.02$$\pm 0.77$ |  $75.13$$\pm 10.09$
Linear-no-bias | CIFAR10 |  $49.11$$\pm 2.27$ |  $54.31$$\pm 1.42$ |  $60.15$$\pm 0.73$ |  $64.04$$\pm 0.77$ |  $75.13$$\pm 10.08$
Linear-Masked | CIFAR10 |  $23.02$$\pm 8.05$ |  $25.78$$\pm 5.89$ |  $27.45$$\pm 3.77$ |  $30.02$$\pm 4.71$ |  $42.57$$\pm 29.59$
Linear-no-bias-Masked | CIFAR10 |  $23.01$$\pm 8.12$ |  $25.96$$\pm 6.00$ |  $27.74$$\pm 3.64$ |  $30.01$$\pm 4.84$ |  $39.20$$\pm 30.87$
KNN | CIFAR10 |  $45.21$$\pm 2.03$ |  $49.47$$\pm 1.47$ |  $54.19$$\pm 1.13$ |  $57.00$$\pm 0.58$ |  $60.26$$\pm 0.28$
SLDA | CIFAR10 |  $49.23$$\pm 22.67$ |  $61.51$$\pm 16.98$ |  $68.47$$\pm 13.43$ |  $70.47$$\pm 12.44$ |  $66.90$$\pm 0.02$
MeanLayer | CIFAR10 |  $62.34$$\pm 17.28$ |  $66.58$$\pm 14.84$ |  $68.65$$\pm 13.71$ |  $69.58$$\pm 13.23$ |  $63.27$$\pm 0.00$
MedianLayer | CIFAR10 |  $60.65$$\pm 18.27$ |  $65.50$$\pm 15.53$ |  $63.59$$\pm 10.88$ |  $68.84$$\pm 13.61$ |  $70.44$$\pm 12.72$
CosLayer | CIFAR100Lifelong |  $28.42$$\pm 4.38$ |  $36.61$$\pm 5.10$ |  $47.39$$\pm 3.01$ |  $53.30$$\pm 1.87$ |  $65.42$$\pm 0.28$
WeightNorm | CIFAR100Lifelong |  $49.19$$\pm 2.30$ |  $57.33$$\pm 1.14$ |  $63.67$$\pm 0.65$ |  $66.39$$\pm 0.44$ |  $70.52$$\pm 0.22$
OriginalWeightNorm | CIFAR100Lifelong |  $47.68$$\pm 2.52$ |  $55.16$$\pm 1.10$ |  $60.70$$\pm 0.61$ |  $62.83$$\pm 0.69$ |  $70.53$$\pm 0.19$
CosLayer-Masked | CIFAR100Lifelong |  $53.27$$\pm 1.74$ |  $58.88$$\pm 0.76$ |  $62.88$$\pm 0.54$ |  $64.56$$\pm 0.33$ |  $65.74$$\pm 0.06$
WeightNorm-Masked | CIFAR100Lifelong |  $23.17$$\pm 3.88$ |  $26.00$$\pm 5.85$ |  $27.69$$\pm 5.08$ |  $28.60$$\pm 6.01$ |  $30.78$$\pm 4.98$
OriginalWeightNorm-Masked | CIFAR100Lifelong |  $16.13$$\pm 8.01$ |  $18.49$$\pm 8.51$ |  $16.16$$\pm 7.56$ |  $18.12$$\pm 6.86$ |  $24.80$$\pm 5.16$
Linear | CIFAR100Lifelong |  $47.84$$\pm 2.81$ |  $55.88$$\pm 1.04$ |  $62.46$$\pm 0.79$ |  $65.56$$\pm 0.50$ |  $70.36$$\pm 0.10$
Linear-no-bias | CIFAR100Lifelong |  $47.24$$\pm 2.64$ |  $55.70$$\pm 0.89$ |  $62.35$$\pm 0.70$ |  $65.38$$\pm 0.48$ |  $70.28$$\pm 0.29$
Linear-Masked | CIFAR100Lifelong |  $11.41$$\pm 4.22$ |  $14.29$$\pm 3.81$ |  $14.90$$\pm 2.85$ |  $14.76$$\pm 2.52$ |  $16.39$$\pm 2.11$
Linear-no-bias-Masked | CIFAR100Lifelong |  $11.81$$\pm 3.97$ |  $12.41$$\pm 3.47$ |  $12.77$$\pm 2.43$ |  $13.16$$\pm 2.30$ |  $14.53$$\pm 2.80$
KNN | CIFAR100Lifelong |  $44.78$$\pm 2.39$ |  $52.98$$\pm 1.33$ |  $61.87$$\pm 0.85$ |  $65.93$$\pm 0.77$ |  $66.71$$\pm 1.01$
SLDA | CIFAR100Lifelong |  $37.03$$\pm 2.04$ |  $55.02$$\pm 0.85$ |  $63.07$$\pm 0.44$ |  $65.75$$\pm 0.28$ |  $68.17$$\pm 0.01$
MeanLayer | CIFAR100Lifelong |  $51.63$$\pm 2.17$ |  $58.01$$\pm 1.05$ |  $62.32$$\pm 0.45$ |  $64.13$$\pm 0.32$ |  $65.55$$\pm 0.00$
MedianLayer | CIFAR100Lifelong |  $48.23$$\pm 1.74$ |  $55.30$$\pm 1.46$ |  $60.44$$\pm 0.35$ |  $62.76$$\pm 0.37$ |  $64.17$$\pm 0.00$
CosLayer | Core50 |  $10.30$$\pm 1.98$ |  $12.43$$\pm 3.56$ |  $18.11$$\pm 4.17$ |  $26.48$$\pm 4.16$ |  $53.93$$\pm 0.90$
WeightNorm | Core50 |  $32.06$$\pm 2.93$ |  $44.32$$\pm 1.92$ |  $61.91$$\pm 1.10$ |  $69.44$$\pm 0.67$ |  $77.04$$\pm 0.33$
OriginalWeightNorm | Core50 |  $33.19$$\pm 2.62$ |  $45.68$$\pm 1.96$ |  $62.39$$\pm 1.01$ |  $69.17$$\pm 0.65$ |  $76.85$$\pm 0.61$
CosLayer-Masked | Core50 |  $30.62$$\pm 2.57$ |  $41.22$$\pm 1.68$ |  $56.47$$\pm 1.13$ |  $63.10$$\pm 0.62$ |  $63.43$$\pm 5.78$
WeightNorm-Masked | Core50 |  $12.95$$\pm 2.07$ |  $14.98$$\pm 2.89$ |  $21.13$$\pm 1.22$ |  $23.42$$\pm 2.49$ |  $31.18$$\pm 17.50$
OriginalWeightNorm-Masked | Core50 |  $12.27$$\pm 2.59$ |  $17.42$$\pm 2.52$ |  $20.82$$\pm 4.94$ |  $22.21$$\pm 5.50$ |  $22.43$$\pm 4.36$
Linear | Core50 |  $32.80$$\pm 2.67$ |  $44.88$$\pm 1.94$ |  $61.80$$\pm 1.06$ |  $69.03$$\pm 0.65$ |  $76.29$$\pm 0.34$
Linear-no-bias | Core50 |  $32.80$$\pm 2.68$ |  $44.89$$\pm 1.94$ |  $61.80$$\pm 1.06$ |  $69.02$$\pm 0.65$ |  $76.29$$\pm 0.34$
Linear-Masked | Core50 |  $7.37$$\pm 1.98$ |  $8.20$$\pm 2.68$ |  $14.02$$\pm 5.88$ |  $18.46$$\pm 2.60$ |  $27.25$$\pm 28.39$
Linear-no-bias-Masked | Core50 |  $7.36$$\pm 1.97$ |  $8.22$$\pm 2.67$ |  $14.04$$\pm 5.88$ |  $18.48$$\pm 2.55$ |  $25.81$$\pm 29.13$
KNN | Core50 |  $25.95$$\pm 3.09$ |  $34.17$$\pm 2.14$ |  $45.50$$\pm 2.04$ |  $52.13$$\pm 1.19$ |  $65.50$$\pm 0.14$
SLDA | Core50 |  $17.30$$\pm 1.23$ |  $30.23$$\pm 5.58$ |  $17.71$$\pm 1.47$ |  $60.14$$\pm 0.93$ |  $78.55$$\pm 0.03$
MeanLayer | Core50 |  $28.81$$\pm 2.68$ |  $40.71$$\pm 1.65$ |  $56.52$$\pm 1.29$ |  $63.20$$\pm 0.79$ |  $71.51$$\pm 0.00$
MedianLayer | Core50 |  $26.83$$\pm 2.26$ |  $36.03$$\pm 1.65$ |  $53.07$$\pm 1.15$ |  $60.73$$\pm 0.77$ |  $70.22$$\pm 0.00$
CosLayer | Core10Lifelong |  $31.45$$\pm 10.30$ |  $43.25$$\pm 9.15$ |  $52.17$$\pm 10.29$ |  $65.40$$\pm 5.68$ |  $78.76$$\pm 0.36$
WeightNorm | Core10Lifelong |  $68.52$$\pm 4.46$ |  $76.78$$\pm 1.44$ |  $81.38$$\pm 0.89$ |  $83.89$$\pm 0.62$ |  $86.29$$\pm 0.29$
OriginalWeightNorm | Core10Lifelong |  $67.50$$\pm 5.30$ |  $75.86$$\pm 1.15$ |  $80.33$$\pm 0.89$ |  $82.42$$\pm 0.75$ |  $86.07$$\pm 0.51$
CosLayer-Masked | Core10Lifelong |  $65.58$$\pm 4.18$ |  $72.28$$\pm 2.19$ |  $76.89$$\pm 1.28$ |  $78.69$$\pm 0.66$ |  $79.56$$\pm 0.65$
WeightNorm-Masked | Core10Lifelong |  $42.05$$\pm 5.58$ |  $48.26$$\pm 4.63$ |  $54.12$$\pm 6.72$ |  $57.78$$\pm 4.81$ |  $58.74$$\pm 16.05$
OriginalWeightNorm-Masked | Core10Lifelong |  $26.76$$\pm 5.72$ |  $34.75$$\pm 5.45$ |  $30.12$$\pm 11.94$ |  $31.73$$\pm 7.00$ |  $46.05$$\pm 4.96$
Linear | Core10Lifelong |  $68.47$$\pm 4.57$ |  $76.33$$\pm 1.32$ |  $80.79$$\pm 0.91$ |  $83.08$$\pm 0.57$ |  $86.19$$\pm 0.33$
Linear-no-bias | Core10Lifelong |  $68.47$$\pm 4.57$ |  $76.33$$\pm 1.32$ |  $80.79$$\pm 0.91$ |  $83.08$$\pm 0.57$ |  $86.19$$\pm 0.33$
Linear-Masked | Core10Lifelong |  $23.17$$\pm 6.06$ |  $24.29$$\pm 6.54$ |  $29.25$$\pm 7.45$ |  $30.79$$\pm 10.56$ |  $38.08$$\pm 28.04$
Linear-no-bias-Masked | Core10Lifelong |  $23.11$$\pm 5.98$ |  $24.06$$\pm 6.49$ |  $28.82$$\pm 7.34$ |  $30.76$$\pm 10.48$ |  $38.27$$\pm 28.03$
KNN | Core10Lifelong |  $55.05$$\pm 4.00$ |  $64.80$$\pm 2.23$ |  $72.20$$\pm 2.05$ |  $76.51$$\pm 1.55$ |  $75.71$$\pm 1.50$
SLDA | Core10Lifelong |  $63.00$$\pm 3.03$ |  $63.96$$\pm 2.35$ |  $35.55$$\pm 1.38$ |  $73.46$$\pm 0.82$ |  $88.06$$\pm 0.01$
MeanLayer | Core10Lifelong |  $65.06$$\pm 4.08$ |  $71.30$$\pm 2.70$ |  $76.25$$\pm 1.37$ |  $78.33$$\pm 0.49$ |  $79.61$$\pm 0.00$
MedianLayer | Core10Lifelong |  $61.46$$\pm 4.26$ |  $69.28$$\pm 2.51$ |  $74.81$$\pm 1.24$ |  $76.69$$\pm 0.85$ |  $78.16$$\pm 0.03$
CosLayer | CUB200 |  $5.19$$\pm 0.77$ |  $5.41$$\pm 1.01$ |  $6.77$$\pm 1.19$ |  $8.74$$\pm 1.94$ |  $30.00$$\pm 0.73$
WeightNorm | CUB200 |  $11.13$$\pm 1.19$ |  $15.78$$\pm 0.83$ |  $25.39$$\pm 0.98$ |  $34.96$$\pm 0.93$ |  $54.41$$\pm 0.50$
OriginalWeightNorm | CUB200 |  $11.17$$\pm 0.87$ |  $16.16$$\pm 0.96$ |  $26.10$$\pm 0.93$ |  $35.50$$\pm 0.57$ |  $52.90$$\pm 0.67$
CosLayer-Masked | CUB200 |  $11.43$$\pm 1.20$ |  $16.85$$\pm 0.41$ |  $26.79$$\pm 0.77$ |  $36.03$$\pm 0.74$ |  $33.07$$\pm 0.63$
WeightNorm-Masked | CUB200 |  $7.51$$\pm 1.02$ |  $9.10$$\pm 1.04$ |  $12.37$$\pm 1.37$ |  $14.40$$\pm 0.92$ |  $16.24$$\pm 0.85$
OriginalWeightNorm-Masked | CUB200 |  $3.96$$\pm 1.02$ |  $3.92$$\pm 0.82$ |  $4.39$$\pm 0.74$ |  $4.96$$\pm 1.26$ |  $5.75$$\pm 1.02$
Linear | CUB200 |  $10.88$$\pm 0.82$ |  $15.38$$\pm 0.86$ |  $24.27$$\pm 0.86$ |  $33.29$$\pm 0.93$ |  $28.84$$\pm 0.75$
Linear-no-bias | CUB200 |  $10.73$$\pm 0.82$ |  $15.07$$\pm 0.73$ |  $24.23$$\pm 0.86$ |  $33.49$$\pm 0.91$ |  $28.92$$\pm 0.78$
Linear-Masked | CUB200 |  $2.07$$\pm 0.53$ |  $2.53$$\pm 0.57$ |  $2.55$$\pm 0.55$ |  $2.77$$\pm 0.42$ |  $2.24$$\pm 0.24$
Linear-no-bias-Masked | CUB200 |  $2.42$$\pm 0.33$ |  $2.15$$\pm 0.43$ |  $2.36$$\pm 0.59$ |  $2.72$$\pm 0.58$ |  $2.08$$\pm 0.29$
KNN | CUB200 |  $10.33$$\pm 1.22$ |  $14.31$$\pm 0.56$ |  $20.41$$\pm 1.07$ |  $25.54$$\pm 0.75$ |  $43.79$$\pm 0.00$
SLDA | CUB200 |  $1.94$$\pm 0.65$ |  $2.37$$\pm 0.70$ |  $3.73$$\pm 0.44$ |  $15.93$$\pm 2.68$ |  $58.61$$\pm 0.12$
MeanLayer | CUB200 |  $10.37$$\pm 1.25$ |  $15.07$$\pm 0.46$ |  $25.24$$\pm 1.05$ |  $34.55$$\pm 0.74$ |  $54.14$$\pm 0.00$
MedianLayer | CUB200 |  $10.77$$\pm 1.02$ |  $14.92$$\pm 0.54$ |  $23.07$$\pm 0.90$ |  $31.91$$\pm 1.00$ |  $53.28$$\pm 0.00$

##  Appendix F Visualizations

(a) Linear: Angles between the different vectors of the output layer

(b) Linear: Mean Angles between the vectors and data class by class.

(c) Linear: Interference Risks

(d) WeightNorm: Angles between the different vectors of the output layer

(e) WeightNorm: Mean Angles between the vectors and data class by class.

(f) WeightNorm: Interference Risks

##  Appendix G Task orders

from continuum.scenarios import create_subscenario import numpy as np

# we suppose scenario, num_tasks and seed already set np.random.seed(seed)
task_order = np.arange(num_tasks) scenario = create_subscenario(scenario,
task_order)

(g) Code to change task order of the original scenario. We used seeds
$[0,1,2,3,4,5,6,7]$. Note: for seed 0, we use original order without
modification.

##  Appendix H Scenario reproducibilty

from continuum.datasets import Core50 from continuum import ClassIncremental

dataset = dataset = Core50("./datasets", download=True, train=train) scenario
= ClassIncremental(dataset, nb_tasks=10, transformations=transform)

(h) Code for Core50 scenario

from continuum.datasets import Core50 from continuum import ContinualScenario

dataset = Core50("./datasets", scenario="domains", classification="category",
train=train) scenario = ContinualScenario(dataset, transformations=transform)

(i) Code for Core10Lifelong scenario

from continuum.datasets import Core50 from continuum import ContinualScenario

dataset = Core50("./datasets", scenario="objects", classification="category",
train=train) scenario = ContinualScenario(dataset, transformations=transform)

(j) Code for Core10Mix scenario

from continuum.datasets import CUB200 from continuum import ClassIncremental

dataset = CUB200("./datasets", train=train) scenario =
ClassIncremental(dataset, nb_tasks=10, transformations=transform)

(k) Code for CUB200 scenario

from continuum.datasets import CIFAR100 from continuum import
ContinualScenario

dataset = CIFAR100("./datasets", labels_type="category",
task_labels="lifelong", train=train) scenario = ContinualScenario(dataset,
transformations=transform)

(l) Code for CIFAR100Lifelong scenario

Figure 9: Code to reproduce the scenarios used in the paper with continuum
library.