# Unlocking Continual Learning Abilities in Language Models

Wenyu Du1 Shuang Cheng2∗ Tongxu Luo3 Zihan Qiu4 Zeyu Huang5
Ka Chun Cheung6 Reynold Cheng1 Jie Fu733footnotemark: 3
1The University of Hong Kong 2ICT, Chinese Academy of Sciences 3CUHK-SZ
4Tsinghua University 5University of Edinburgh 6NVIDIA 7HKUST
<EMAIL_ADDRESS><EMAIL_ADDRESS>Equal Contributions. Work done
during interning at NVIDIA. Corresponding Authors.

###### Abstract

Language models (LMs) exhibit impressive performance and generalization
capabilities. However, LMs struggle with the persistent challenge of
catastrophic forgetting, which undermines their long-term sustainability in
continual learning (CL). Existing approaches usually address the issue by
incorporating old task data or task-wise inductive bias into LMs. However, old
data and accurate task information are often unavailable or costly to collect,
hindering the availability of current CL approaches for LMs. To address this
limitation, we introduce “MIGU” (MagnItude-based Gradient Updating for
continual learning), a rehearsal-free and task-label-free method that only
updates the model parameters with large magnitudes of output in LMs’ linear
layers. MIGU is based on our observation that the L1-normalized magnitude
distribution of the output in LMs’ linear layers is different when the LM
models deal with different task data. By imposing this simple constraint on
the gradient update process, we can leverage the inherent behaviors of LMs,
thereby unlocking their innate CL abilities. Our experiments demonstrate that
MIGU is universally applicable to all three LM architectures (T5, RoBERTa, and
Llama2), delivering state-of-the-art or on-par performance across continual
finetuning and continual pre-training settings on four CL benchmarks. For
example, MIGU brings a 15.2% average accuracy improvement over conventional
parameter-efficient finetuning baselines in a 15-task CL benchmark. MIGU can
also seamlessly integrate with all three existing CL types to further enhance
performance.

Unlocking Continual Learning Abilities in Language Models

Wenyu Du1††thanks:  Equal Contributions.††thanks:  Work done during interning
at NVIDIA. Shuang Cheng2∗ Tongxu Luo3 Zihan Qiu4 Zeyu Huang5 Ka Chun Cheung6
Reynold Cheng1††thanks:  Corresponding Authors. Jie Fu733footnotemark: 3 1The
University of Hong Kong 2ICT, Chinese Academy of Sciences 3CUHK-SZ 4Tsinghua
University 5University of Edinburgh 6NVIDIA 7HKUST<EMAIL_ADDRESS><EMAIL_ADDRESS>

##  1 Introduction

Figure 1: The different output values and the varying magnitude distributions
are observed across the BoolQA, COPA, and Yelp datasets. The output values and
L1-normalized magnitude distributions are from the real sample of the first
linear layer in FFN of the last Transformer block of T5. The detailed
magnitude distributions are illustrated in Figure 32 of Appendix D.4.

Neural networks suffer from catastrophic forgetting McCloskey and Cohen
(1989), i.e. learning new knowledge and tasks at the cost of forgetting
previously acquired ones. Recently, language models (LMs) have demonstrated
impressive performance and generalization capabilities across the spectrum of
NLP research Liu et al. (2019); Brown et al. (2020); Touvron et al. (2023) and
beyond Zhang et al. (2023a). Nevertheless, they still suffer from catastrophic
forgetting Shi et al. (2024); Wu et al. (2024), undermining the capacity for
continual learning (CL) Wang et al. (2024a). In light of the large scale and
high cost of training LMs Achiam et al. (2023), models with strong continual
learning capabilities would enable more economical reuse of these resource-
intensive models, a vital trajectory for driving both scientific development
and societal benefits.

To make LMs better continual learners, the research community pursues three
main directions Shi et al. (2024): (1) rehearsal-based approaches that mix new
task data with a small buffer of past task examples Scialom et al. (2022);
Wang et al. (2024d), (2) architecture-based methods that introduce new
components like adapters to incorporate new tasks Gururangan et al. (2021);
Qin et al. (2022); Zhao et al. (2024); Wang et al. (2024b), and (3) parameter-
based approaches that either apply regularization to penalize changes in
important parameters for old tasks Zheng et al. (2023); Zhu et al. (2024) or
update parameter gradients for each task into orthogonal subspaces Wang et al.
(2023b).

However, rehearsal-based methods require data from previously learned tasks,
which is not always available Touvron et al. (2023). The architecture and
parameter-based approaches typically rely on task labels to design techniques
to mitigate gradient conflicts between tasks by updating the parameters task-
wise. However, obtaining accurate task labels can be challenging or infeasible
in LMs’ scenarios. This paper explores an alternative approach, examining
whether the model’s inherent features or behaviors can be utilized instead of
task labels to mitigate gradient conflicts between tasks. Concretely, we
examine the distribution of the L1-normalized output magnitude of the linear
layers in LMs. The output is computed as the dot product between the input
$\mathbf{x}\in\mathbb{R}^{d_{\text{in}}}$ and the weight
$\mathbf{W}\in\mathbb{R}^{d_{\text{in}}\times d_{\text{out}}}$ of the layer,
and then the output is normalized using the L1-norm, resulting in a vector
$\mathbf{n}\in\mathbb{R}^{d_{\text{out}}}$. Our analysis reveals an intriguing
finding: the L1-normalized output $\mathbf{n}$ exhibits distinct magnitude
distributions for different tasks111The term ‘(L1-)normalized output magnitude
distribution’ will be referred to interchangeably as ‘magnitude distribution’
for brevity throughout the paper.. The process and observation described above
are illustrated in Figure 1, which presents real output data for the first
linear layer of the Feedforward Network (FFN) in the last Transformer block of
T5. We can observe that the magnitude distributions differ significantly for
three example tasks - BoolQA, COPA, and Yelp. Motivated by this observation,
we argue that the differences in magnitude distributions within LMs could
serve as a natural, label-free alternative to replace the need for external
task labels in mitigating gradient conflicts to update the model’s parameters
task-wise. However, this potential is locked during conventional continual
learning settings.

To this end, we introduce “MIGU” (MagnItude-based Gradient Updating for
continual learning), leveraging the inherent differences in magnitude
distributions of the L1-normalized output in LMs’ linear layers to enable
continual learning without relying on task labels. Specifically, during the
forward propagation phase, we cache and normalize the output of the linear
layers using the L1-norm. Then, in the backward propagation phase, we only
update the parameters with the $T$ largest values in L1-normalized magnitude,
where $T$ is a predefined threshold ratio. Since different tasks exhibit
distinct magnitude distribution patterns, MIGU can effectively harness the
LMs’ inherent features to update the parameters with large magnitudes per
task, alleviating gradient conflicts and unlocking their innate continual
learning potential.

We evaluate MIGU across three main LM architectures: the encoder-only RoBERTa
Liu et al. (2019), the encoder-decoder T5 model Raffel et al. (2023), and the
decoder-only Llama2 Touvron et al. (2023). Furthermore, we consider two
continual pre-training settings for LMs: continual pre-training and continual
finetuning, using four CL datasets. Notably, our approach can seamlessly
integrate three mainstream CL approaches - rehearsal-based, architecture-
based, and parameter-based - to further enhance the CL abilities of LMs. When
evaluated on the four datasets, our experimental results achieve comparable or
superior performance to the current state-of-the-art methods. For example, in
a 15-task long sequence CL dataset, the MIGU leads to a 15.2% accuracy
improvement over the conventional parameter-efficient finetuning baseline.
Furthermore, combining MIGU with three types of CL methodologies substantially
improves these individual CL approaches. We also provide detailed ablation
studies and visualizations on MIGU, revealing that CL with MIGU pushes the
magnitude distribution similarity between tasks farther apart and better
avoids conflicts. We believe the work presents a novel perspective on
exploring CL in LMs. Our code is publicly available at
https://github.com/wenyudu/MIGU.

##  2 Related Work

#### Continual Learning for Language Models.

Continual learning is a long-standing challenge through the history of machine
learning and deep learning McCloskey and Cohen (1989); Wu et al. (2024).
Recent studies for CL in LMs can be roughly categorized into three categories.
1\. Rehearsal-based approach that mixes new task data with a small buffer of
past task examples Scialom et al. (2022); Wang et al. (2024d). 2\.
Architecture-based approach that expands new modules like adapters to
incorporate new tasks Gururangan et al. (2021); Qin et al. (2022); Zhao et al.
(2024); Wang et al. (2024b). 3\. Parameter-based method that updates
parameters in a task-aware manner. Some literature Wang et al. (2023a) splits
the parameter-based into either regularization-based approaches that add a
regularization term to penalize changes in important weights of the earlier
learned tasks Zheng et al. (2023); Zhu et al. (2024), or optimization-based
approaches that updates parameters gradients for each task into orthogonal
subspaces to avoid conflicts Wang et al. (2023b). These methods rely on either
old task data or accurate task labels, which are hard or expensive to collect
for LMs’ continual training. In contrast, MIGU only leverages LMs’ innate
features for CL.

#### Partially Updating Parameters in Continual Learning.

Among existing CL methods for LMs, our approach and regularization-based Zheng
et al. (2023); Zhu et al. (2024) approach both partially update parameters,
but ours fundamentally diverges from the regularization-based methods in
motivation and design. While they rely on backward gradients to identify and
protect important weights for old tasks, we leverage the differences in
magnitude distribution across tasks during the feed-forward phase.
Additionally, our method’s ability to freely mask at the sample level sets us
apart from their fixed gradient mask approach. Furthermore, our method does
not require task labels, enabling it to work in broader scenarios where task
labels are unavailable. Lastly, the layer output distributions are naturally
obtained during the feed-forward phase training, whereas they normally require
an additional subset to derive the gradient mask before training on a task.

#### Finding Important Weights.

One may classify our method as a broader research cluster centered on finding
important weights, a topic that has been extensively explored in continual
learning Zhu et al. (2023), model pruning and compression Frankle and Carbin
(2019), efficient training and inference Ansell et al. (2024), as well as
investigations into activation sparsity Zhang et al. (2023b); Song et al.
(2024), and other related areas. However, these works mostly use weight or
gradient magnitude to define a fixed size of the important weights. A few
works on activation sparsity use the sparsity patterns after the activation
function for either efficient inference Zhang et al. (2022) or performance
improvements Qiu et al. (2024). None of the above explore the general dot
product of weights and layer input. The closest work to ours is an
unstructured pruning work Sun et al. (2024) using the dot product of weight
and input, demonstrating a superior method to pure weight-based pruning.
However, this prior work fails to consider the varying patterns of important
weights across different tasks. In contrast, our method utilizes the
L1-normalized dot product of weight and input as an inherent indicator of
importance in CL settings.

##  3 Method

In Table 1, we compare MIGU with common CL methods. Our approach is only one
rehearsal-free, task-id-free method that supports both continual pre-training
and continual finetuning.

###  3.1 Preliminary - Continual Learning Setup

Continual learning Ke and Liu (2022); Wang et al. (2023c); Zhao et al. (2024)
aims to tackle the challenges that arise within the ongoing sequence.
Formally, tasks $\left\\{\mathcal{T}_{1},\ldots,\mathcal{T}_{T}\right\\}$
arrive in sequentially. Each task
$\mathcal{T}_{t}=\left\\{\left(x_{t}^{i},y_{t}^{i}\right)\right\\}_{i=1}^{n_{t}}$
contains a separate target dataset with the size of $n_{t}$. For any time step
$t$, the model is expected to not only adapt itself to the $t$-th task, but
also retain its capabilities across all the previous tasks it has been trained
on. This study explores two distinct CL settings. In the first setting, where
only the MIGU method is employed, the task label is unavailable during the
training and testing phases. Secondly, when combined with the three existing
types of CL techniques, the model can be exposed to old task data or task
information during the training phase.

|  RF |  TIFT |  CIT |  CPT
---|---|---|---|---
LFPT5 Qin and Joty (2021) |  |  | $\checkmark$ |
EPI Wang et al. (2023d) | $\checkmark$ |  | $\checkmark$ |
O-LoRAWang et al. (2023b) | $\checkmark$ |  | $\checkmark$ |
MoCL Wang et al. (2024c) | $\checkmark$ |  | $\checkmark$ |
SAPT Zhao et al. (2024) | $\checkmark$ |  | $\checkmark$ |
DAS Ke et al. (2023) | $\checkmark$ |  |  | $\checkmark$
MIGU | $\checkmark$ | $\checkmark$ | $\checkmark$ | $\checkmark$

Table 1: The comparison between MIGU and other CL methods. Specifically, RF
indicates whether the method is rehearsal-free. TIFT indicates whether the
method is task-id-free during training. CIT indicates whether the method
supports instruction finetuning.CPT indicates whether the method supports
continual pre-training.

###  3.2 MIGU - MagnItude-based Gradient Updating for Continual Learning.

Figure 2: Proposed method: MIGU. During 1) the forward phase, our method 2)
caches the output magnitude of the linear layers, and 3) after
backpropagation, 4) MIGU masks the gradients by cached magnitudes to update
parameters accordingly.

Our approach employs a two-step process to leverage the inherent differences
in magnitude distributions across various tasks for continual learning.: 1)
Caching output magnitudes and 2) Updating gradient via a magnitude-based mask.
We show the process in Figure 2. To illustrate our method, we first consider
the fundamental component in LMs, a single linear layer222For simplicity, we
omit the bias term $\mathbf{b}$ here. with weight $\mathbf{W}$ and only feed
an input token $\mathbf{x}$ into LMs.

#### Feedforward: Caching Output Magnitudes.

Given the weight matrix $\mathbf{W}\in\mathbb{R}^{d_{\text{in}}\times
d_{\text{out}}}$, we interpret the columns of $\mathbf{W}$ as a set of
$d_{\text{out}}$ vectors, each with dimension $d_{\text{in}}$:

$\mathbf{W}=[\mathbf{w_{1}},\dots,\mathbf{w_{i}},\dots\mathbf{w_{d_{out}}}],\text{where
}\mathbf{w_{i}}\in\mathbb{R}^{d_{\text{in}}}$ (1)

Given the input vector of the layer $\mathbf{x}\in\mathbb{R}^{d_{\text{in}}}$,
the operation of the layer can be viewed as the dot product between
$\mathbf{x}$ and each weight vector $\mathbf{w_{i}}$:

${h_{i}}=\mathbf{x}\cdot\mathbf{w_{i}}$ (2)

We then compute the normalized product magnitude ${n_{i}}$ using the L1-norm
by : ${n_{i}}=\|{h_{i}}\|_{1}$, where $\|\cdot\|_{1}$ denotes the L1-norm.
Thus, we have the L1-normalized magnitude product distribution vector
$\mathbf{n}$ for $\mathbf{W}$.

#### Backward Propagation: Updating gradient via a magnitude-based mask.

After calculating the gradient in the backward phase, we obtain the gradient
matrix $\nabla\mathbf{W}$ for the weight $\mathbf{W}$, which presents the
optimization direction given the input $\mathbf{x}$. We then define a mask
matrix $\mathbf{M}$ to partially mask $\nabla\mathbf{W}$ using the
L1-normalized product magnitudes cached during the forward phase. Formally, we
sort the product magnitudes in the descending order and mask the corresponding
gradients as follows:

$\displaystyle{t}$ $\displaystyle=\lfloor T\times d_{\text{out}}\rfloor$ (3)
$\displaystyle\mathbf{M}$ $\displaystyle=\text{BinaryTopT}(\textbf{n},t)$ (4)
$\begin{array}[]{l}\text{BinaryTopT}(\mathbf{n_{i}},t)\\\
=\begin{cases}1&\text{if }\mathbf{n_{i}}\text{ is in the top }1-t\text{
elements of }\mathbf{n}\text{.}\\\ 0&\text{otherwise,}\end{cases}\end{array}$
(5)

where $T$ is the threshold ratio to mask gradient, ${t}$ is the actual number
t to mask, $\lfloor.\rfloor$ is the floor rounding. The model update rule is
then given by:

$\mathbf{W}_{\text{new}}\leftarrow\mathbf{W}-\eta\cdot\mathbf{M}\odot\nabla\mathbf{W}$
(6)

where $\eta$ is the learning rate. This formulation ensures that only those
weights with L1-normalized magnitudes exceeding the threshold ${T}$ are
updated.

###  3.3 MIGU in Practice

In practice, to apply MIGU, we average the product magnitudes of all tokens on
a batch to generate the mask for simple implementation.

#### MIGU in Transformer Block.

For a Transformer block, we apply our method from Section 3.2 to the Query,
Key, Value, and Output linear layer of the multi-head attention (MHA)
component, and two (for T5 and RoBERTa) or three (for Llama) linear layers in
the FFN component.

#### MIGU in LoRA Implementation.

We also implement MIGU for parameter-efficient finetuning (PEFT) of LMs,
particularly we employ Low-Rank Adaptation (LoRA) Hu et al. (2022). The
standard LoRA is mathematically represented as follows:

$\displaystyle\mathbf{x_{A}}$ $\displaystyle=\mathbf{x}\cdot\mathbf{A}$ (7)
$\displaystyle\mathbf{x_{B}}$ $\displaystyle=\mathbf{x_{A}}\cdot\mathbf{B}$
(8) $\displaystyle\mathbf{x_{O}}$
$\displaystyle=\mathbf{x}\cdot\mathbf{W}+\frac{\alpha}{r}\cdot\mathbf{x_{B}},$
(9)

where $\mathbf{x}$ denotes the input representation of the layer,
$\mathbf{A}\in\mathbb{R}^{d_{in}\times r}$ and
$\mathbf{B}\in\mathbb{R}^{r\times d_{out}}$ are the low-rank matrices,
$\alpha$ is a scaling constant, $\mathbf{W}$ is the original weight matrix of
the standard linear, and $\mathbf{x_{O}}$ is the output after applying the
LoRA transformation.

To implement MIGU, we apply the same method in Section 3.2 for the matrix
$\mathbf{A}$. But for the matrix $\mathbf{B}$, we use the output of
$\mathbf{x_{O}}$ in Equation 8 rather than the output of $\mathbf{x_{B}}$ in
Equation 9 to compute the magnitude distribution vector.

##  4 Experiments

CL Benchmark | Standard | Long
---|---|---
LFPT5 | 72.7 | 69.2
EPI | 65.3 | -
MoCL | 75.9 | -
SAPT-LoRA | - | 82.0
IncLoRA | 68.8 | 64,7
\+ MIGU | 76.4(7.6$\uparrow$) | 68.7(4.0$\uparrow$)
OIncLoRA | 75.8 | 69.6
\+ MIGU |  76.6(0.8$\uparrow$) | 70.0(0.4$\uparrow$)
LoRAReplay | 74.5 | 75.2
\+ MIGU | 76.2(1.7$\uparrow$) | 76.5(1.3$\uparrow$)
MoELora | 54.1 | 27.6
FT | 75.7 | 68.3
\+ MIGU |  78.8(3.1$\uparrow$) |  73.8(5.5$\uparrow$)
LoRA | 67.9 | 46.0
\+ MIGU | 73.3(5.4$\uparrow$) | 61.2(15.2$\uparrow$)

Table 2: Average accuracy on standard CL benchmark (Order 1,2,3) and long CL
benchmark (Order 4,5,6) with T5-large model. The top block contains CL methods
with extra old task data or task labels, while the bottom does not.

We use three language models adopted by the previous lines of works in CL for
NLP: encoder-only RoBERTa Liu et al. (2019), encoder-decoder T5 model Raffel
et al. (2023) and decoder-only Llama2 Touvron et al. (2023). We start with
continual finetuning T5-large Raffel et al. (2020) on two CL datasets
following the settings from Qin and Joty (2021); Wang et al. (2023b).We
implement MIGU upon vanilla finetuning and PEFT with LoRA Hu et al. (2022). We
also combine our method with three main types of CL approaches to examine the
seamless integration of our method with the existing CL methodologies. Next,
we use encoder-only RoBERTa to continual pre-traning domain adaptive data,
following the setting Ke et al. (2023). We further scale our experiment to
decoder-only Llama2-7B Touvron et al. (2023) and test the trade-off between
base model ability and new task ability. All experimental results are reported
as the average of 3 runs. Please refer to the Appendix A.1 for more detailed
settings.

###  4.1 Continual Finetuning on T5-large

#### Two Benchmarks.

We evaluate our approach to continual finetuning on T5-large using the
standard CL benchmark and long sequence benchmark. We follow the setup from
Qin and Joty (2021); Wang et al. (2023b) to shuffle the four text
classification tasks from the LM dataset Zhang et al. (2015) into three
different orders to form Order 1, 2, 3 for standard CL benchmark. Similarly,
we shuffle a mix of 15 tasks (five classification tasks, nine GLUE and
SuperGLUE tasks, and the IMDB dataset) to form Orders 4, 5, and 6 for the long
sequence benchmark. For the details on benchmark and sequence, please refer to
the appendix C.1.

#### Baselines.

We separate the baselines into two categories: without old data or task
information and with old data or task information during training. For the
first category, we include vanilla FT, which trains all model parameters on a
sequence of tasks, and vanilla LoRA, in which fixed-size LoRA parameters are
trained on a sequence of tasks. For the second category, we have rehearsal-
based approaches: LoRAReplay that trains new tasks on LoRA with mixing a 2%
past task, LFPT5 Qin and Joty (2021) continuously trains a soft prompt that
simultaneously learns to solve the tasks and generate training samples for
experience replay; architecture-based approaches: IncLoRA that incremental
learning of new LoRA parameters on a sequential series of tasks, MoELora Luo
et al. (2024), a vanilla MoE with LoRA number equals to the task number, SAPT-
LoRA Zhao et al. (2024) extends IncLoRA by aligning learning process and
selection process of LoRA, and MoCL Wang et al. (2024c) continually adds new
modules and composes them with existing modules; parameter-based approaches
OIncLoRA Wang et al. (2023b)333O-LoRA is original name, we rename it to
OIncLoRA to emphasize it is build upon IncLoRA and align with our notation.
extends IncLoRA to learn different LoRAs into orthogonal subspaces.

#### Metrics.

ACC (Accuracy Chaudhry et al. (2018)). The average performance of all tasks
after training on the last task, i.e.,
$A_{\mathcal{T}}=\frac{1}{\mathcal{T}}\sum_{t=1}^{\mathcal{T}}a_{\mathcal{T},t}$.

#### Results on T5.

Table 2 shows that our proposed approach (+MIGU) improves the performance of
all five CL approaches. Notably, when our method is applied, the vanilla FT
and LoRA baselines see substantial improvements. Some results obtained using
our approach are comparable to the SOTA CL methods that leverage task labels
or old task data. Notably, the LoRA+MIGU approach surpasses the vanilla LoRA
method by a substantial 15.2% on the long sequence benchmark, significantly
mitigating the drawbacks of LoRA in the CL setting with long sequences. We
choose to combine our method with three LoRA-based techniques to integrate
with three CL approaches that leverage old data or additional labels. The
parameter-based IncLoRA+MIGU exhibits the most significant improvement over
the original IncLoRA, implying that our magnitude-based approach can
effectively mitigate the conflicts among the sequentially learned LoRA
parameters in IncLoRA. The relatively marginal improvement of parameter-based
OIncLoRA+MIGU indicates a similar function between our approach and projecting
LoRAs into orthogonal subspaces, but our method does not require task labels
during the continual training process. SAPT-LoRA achieves the SoTA performance
in long sequence benchmark, but it requires both task labels and past data,
which are often infeasible or costly in LMs settings. We also report an
efficiency study in Appendix D.2 Table 9 to show our approach only leads to a
minor overhead over the vanilla methods, which is assumed to be more efficient
than other CL methods. We provide a full experiment in Appendix D.1 Table 8.
We also draw the Violin Plot to show the statistical significance of our
approach over baselines in Appendix D.1.

###  4.2 Continual Pre-training on RoBERTa

#### Benchmark.

In contrast to the previous continual finetuning setting, Ke et al. (2023)
introduces DAS, a new benchmark for continual pre-training of LMs. DAS is
composed of six unlabeled domain corpora, which contain three review domains
and three academic paper domains. It is then evaluated using six corresponding
classification datasets. Please refer to the Appendix B.1 for the details.

#### Metrics.

For continual pre-training, we utilize MF1 (Macro-F1) and ACC (Accuracy)
following Ke et al. (2023) to evaluate the performance after pre-training on
the last domain.

#### Baselines.

We choose top baselines ranging from vanilla methods that pre-train RoBERTa on
domains sequentially with full parameters FT and with PEFT Adapter to
rehearsal-based (DER++ (Buzzega et al., 2020)), architecture-based (DEMIX
(Gururangan et al., 2021)), and parameter-based HAT-Adapter (Serrà et al.,
2018) and DAS Ke et al. (2023).

#### Results on RoBERTa.

Metrics | MF1 | ACC
---|---|---
DEMIX | 74.70 | 79.66
DER++ | 75.78 | 80.46
HAT-Adapter | 74.63 | 79.78
DAS | 77.90 | 81.90
DAS∗ | 76.59 | 81.07
Adapter | 74.05 | 79.48
FT | 76.36 | 80.77
\+ MIGU |  76.73(0.37$\uparrow$) |  81.19(0.42$\uparrow$)

Table 3: Average MF1, ACC on the DAS benchmark after continual pre-training on all domains and finetuning on their corresponding end-task datasets. The top block contains CL methods with extra old task data or task labels, while the bottom does not. DAS∗ is the average result we reproduced in 3 runs. | Avg. Domain 1-2 | Avg. Domain 5-6
---|---|---
FT | 79.69 | 82.07
DAS |  80.30(0.61$\uparrow$) | 81.16(0.91$\mathbf{\downarrow}$)
MIGU | 80.14(0.45$\uparrow$) |  82.41(0.34$\uparrow$)
Table 4: The average ACC of the first and last two learned domains in the DAS
benchmark.

We evaluate MIGU in another setting in which, we continually pre-train a
RoBERTa model to six domains sequentially (domain-adaptive pre-training). Our
experimental results in Table 3 also show promising results of our approach
over or on par with the sophisticated CL methods with task labels or old data.
For instance, FT+MIGU achieves 0.37% improvement in MF1 and 0.42% in ACC. We
also explore the performance of the domains in different orders. We report the
average ACC of the first and last two learned domains in Table 4. The results
indicate that while the DAS model exhibits less forgetting in the earlier
learned domains, but it also learns less in the last domains, possibly due to
the strong regularization used to constrain its parameter updates during the
CL process over a long sequence. In contrast, MIGU demonstrates a more
sustainable method, exhibiting robust performance on the earlier and recently
learned domains.

###  4.3 Forgetting Less and Learning the Same: Scaling to Llama2

#### Results on Llama2.

Figure 4: Performance comparison of LoRA with MIGU and the baseline vanilla
LoRA on Llama2-7B instruction tuning, evaluated using the Humaneval Chen et
al. (2021), as well as on LM benchmarks: HellaSwag Zellers et al. (2019),
Winogrande Sakaguchi et al. (2019), and ARC-Challenge Clark et al. (2018).

We further assess our approach on a more demanding LLM continual instruction
tuning setting. We finetune a base Llama2-7B on Magicoder-Evol-Instruct-110K
for 32 epochs. This dataset Wei et al. (2024) contains 72.97M tokens of
programming questions and answers. However, due to computation constraints, we
sample 20% of data and conduct experiments on LoRA. We follow Biderman et al.
(2024) to assess LoRA+MIGU’s capabilities on both the base ability (forgetting
domain) and the code ability (learning domain). To evaluate code learning
performance, we utilize the Humaneval benchmark Chen et al. (2021), which
contains 164 problems that generate a Python program with a docstring and a
function signature. A generation is considered correct if it passes all
supplied unit tests. To quantify how much they have forgotten previous
knowledge, we follow Biderman et al. (2024) that utilizes average scores of
three benchmarks, HellaSwag Zellers et al. (2019), WinoGrade Sakaguchi et al.
(2019) and ARC-challenge Clark et al. (2018). The experiments are shown in
Figure 4. Compared to baseline FT, our method learns a similar level of new
code knowledge but exhibits significantly less forgetting of previous
knowledge. This suggests our approach achieves a better trade-off point on the
Pareto frontier between learning plasticity and memory stability Huang (2003);
Wang et al. (2024a). For example, after 32 training epochs, the average
accuracy across the three benchmarks for our method is 59.4, while the
baseline model only achieves 58.4.

##  5 Discussions

We then provide ablations on gradient mask threshold and components as well as
a visualization.

###  5.1 Ablation on Gradient Mask Threshold

We plot all five curves of our approach (+MIGU) for gradient mask threshold
from 0.0 to 0.9 in Section 4.1. The optimal threshold value for FT+MIGU,
LoRA+MIGU, and IncLoRA+MIGU settings is 0.7 while LoRAReplay+MIGU is 0.4 as
shown in Figure 5444The ablation on threshold search only reports one run, so
it does not align with the results in Section 4.1.. OIncLoRA+MIGU is only 0.1,
which may due to the parameter updating regularized by the OIncLoRA method
itself. The optimal value for IncLoRA+MIGU is 0.6, close to FT+MIGU,
LoRA+MIGU, and IncLoRA+MIGU settings.

Figure 5: Ablation study on the gradient mask threshold. The curves illustrate
that the optimal value is concentrated around 0.7 for FT+MIGU, LoRA+MIGU, and
IncLoRA+MIGU, 0.4 and 0.1 for the LoRAReplay+MIGU and OIncLoRA+MIGU settings
respectively.

Surprisingly, with only 5% ($T=0.95$) or 1% ($T=0.99$) parameters updating,
LoRA+MIGU still beats LoRA by a wide margin. This interesting finding may
indicate that only a small proportion of proportional weights with large
magnitudes is crucial for successful CL settings, which may be worth future
investigation.

###  5.2 Ablation on Gradient Mask Components

| Order1 | Order2 | Order3 | Avg
---|---|---|---|---
FT + MIGU | 79.6 | 80.3 | 79.2 | 79.7
FT | 75.3 | 76.1 | 78.0 | 76.5
\+ FFN $1_{st}$-L | 76.9 | 75.9 | 75.8 | 76.2
\+ FFN all | 77.2 | 77.2 | 76.7 | 77.0
\+ Attention Q | 77.2 | 76.4 | 78.3 | 77.3
\+ Attention K | 76.9 | 73.4 | 75.6 | 75.3
\+ Attention V | 75.4 | 76.3 | 78.0 | 76.6
\+ Attention O | 76.2 | 75.9 | 76.0 | 76.0
\+ Attention all | 80.2 | 78.8 | 79.0 | 79.3

Table 5: The ablation results from applying MIGU to different LM components.“+
FFN all” means only applying MIGU to all the linear operators in FFN layers.
The results demonstrate implementing MIGU across all linear layers leads to
the most benefits.

We further investigate which components within a transformer block should
utilize MIGU. Typically, a transformer block consists of six linear layers:
the query, key, and value (QKV) linear layers and the output linear layer (O)
in the MHA module, as well as the two linear layers in the FFN. Our analysis
in Table 5 shows that employing MIGU across all these linear layers achieves
the best overall performance, suggesting that the magnitude-based approach is
effective for linear layers in different parts of the transformer
architecture.

###  5.3 Visualization

We evaluate task similarity by counting the overlapping ratio of updated
parameters (large magnitudes) positions by using 100 samples per task. In
Figure 9, we visualize the task similarity for the first layer of FFN in the
last Transformer block of T5-large, comparing FT and FT+MIGU in the Order 6
setting. The results clearly show that MIGU increases the degree of parameter
isolation across tasks, achieving a similar effect by using task information
but without relying on such explicit task labels. We further highlight the
similarity between the BoolQA, COPA, and Yelp tasks and the notable decrease
in similarity among these three tasks. Analyzing the performance results shown
in Table 6, we find that the significant reduction in overlapping ratio across
tasks considerably alleviates the task conflicts, resulting in much more
significant performance gains. For example, the accuracy improvement for the
COPA dataset is exactly 10%. We put the full visualization of all linear
layers in Appendix D.4.

Figure 9: The visualization of magnitude distribution similarity across
different tasks. FT+MIGU is lower, indicating that MIGU reduces the
possibility of weight conflicts between tasks. The two sub-figures at the
bottom are three highlighted task samples: BoolQA, COPA and Yelp.

| BoolQA | COPA | Yelp
---|---|---|---
FT | 67.3 | 45.0 | 39.1
FT + MIGU | 78.3(11.0$\uparrow$) | 55.0(10.0$\uparrow$) | 47.6(8.5$\uparrow$)

Table 6: The improvement on BoolQA, COPA and Yelp in Order 6.

##  6 Conclusion

We propose MIGU, a rehearsal-free and task-label-free method that only updates
the model parameters with large output magnitudes in LM’s linear layers. By
imposing this simple constraint on the gradient update process, we can
leverage the inherent behaviors of LMs, thereby unlocking their innate CL
abilities. Our experiments, applied to all three LM architectures (T5, RoBERTa
and Llama2), on two CL scenarios (continual finetuning and continual pre-
training) and four CL benchmarks, consistently deliver better performance. Our
method can also be seamlessly integrated with existing CL solutions to further
improve their performance.

##  7 Limitations

We acknowledge two limitations for this work. Due to computation limitations,
although we finetune Llama2-7B with LoRA, we are unable to scale our
experiments to LM continual pre-training or full tuning. However, our
experimental performance on continual pre-training RoBERTa indicates the great
potential for the scalability of this general approach. Another limitation is
we only explore an approach for unlocking the inherent CL potential of LMs
through updating the gradient by the magnitude of output. There exists more
discussions on exploiting innate features such as activation sparsity as
discussed in the Related Work section. These limitations can be further
addressed in future work.

## References

  * Achiam et al. (2023) Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023.  Gpt-4 technical report.  _arXiv preprint arXiv:2303.08774_.
  * Ansell et al. (2024) Alan Ansell, Ivan Vulić, Hannah Sterz, Anna Korhonen, and Edoardo M. Ponti. 2024.  Scaling sparse fine-tuning to large language models.  _Preprint_ , arXiv:2401.16405.
  * Biderman et al. (2024) Dan Biderman, Jose Gonzalez Ortiz, Jacob Portes, Mansheej Paul, Philip Greengard, Connor Jennings, Daniel King, Sam Havens, Vitaliy Chiley, Jonathan Frankle, Cody Blakeney, and John P. Cunningham. 2024.  Lora learns less and forgets less.  _Preprint_ , arXiv:2405.09673.
  * Brown et al. (2020) Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020.  Language models are few-shot learners.  _Advances in neural information processing systems_ , 33:1877–1901.
  * Buzzega et al. (2020) Pietro Buzzega, Matteo Boschini, Angelo Porrello, Davide Abati, and Simone Calderara. 2020.  Dark experience for general continual learning: a strong, simple baseline.  _arXiv preprint arXiv:2004.07211_.
  * Chaudhary (2023) Sahil Chaudhary. 2023.  Code alpaca: An instruction-following llama model for code generation.  https://github.com/sahil280114/codealpaca.
  * Chaudhry et al. (2018) Arslan Chaudhry, Puneet K. Dokania, Thalaiyasingam Ajanthan, and Philip H. S. Torr. 2018.  _Riemannian Walk for Incremental Learning: Understanding Forgetting and Intransigence_, page 556–572.  Springer International Publishing.
  * Chen et al. (2021) Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, Nick Ryder, Mikhail Pavlov, Alethea Power, Lukasz Kaiser, Mohammad Bavarian, Clemens Winter, Philippe Tillet, Felipe Petroski Such, Dave Cummings, Matthias Plappert, Fotios Chantzis, Elizabeth Barnes, Ariel Herbert-Voss, William Hebgen Guss, Alex Nichol, Alex Paino, Nikolas Tezak, Jie Tang, Igor Babuschkin, Suchir Balaji, Shantanu Jain, William Saunders, Christopher Hesse, Andrew N. Carr, Jan Leike, Josh Achiam, Vedant Misra, Evan Morikawa, Alec Radford, Matthew Knight, Miles Brundage, Mira Murati, Katie Mayer, Peter Welinder, Bob McGrew, Dario Amodei, Sam McCandlish, Ilya Sutskever, and Wojciech Zaremba. 2021.  Evaluating large language models trained on code.  _Preprint_ , arXiv:2107.03374.
  * Clark et al. (2018) Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. 2018.  Think you have solved question answering? try arc, the ai2 reasoning challenge.  _arXiv preprint arXiv:1803.05457_.
  * de Masson D’Autume et al. (2019) Cyprien de Masson D’Autume, Sebastian Ruder, Lingpeng Kong, and Dani Yogatama. 2019.  Episodic memory in lifelong language learning.  _Advances in Neural Information Processing Systems_ , 32.
  * Ding et al. (2008) Xiaowen Ding, Bing Liu, and Philip S Yu. 2008.  A holistic lexicon-based approach to opinion mining.  In _Proceedings of the 2008 international conference on web search and data mining_.
  * Frankle and Carbin (2019) Jonathan Frankle and Michael Carbin. 2019.  The lottery ticket hypothesis: Finding sparse, trainable neural networks.  _Preprint_ , arXiv:1803.03635.
  * Gururangan et al. (2021) Suchin Gururangan, Mike Lewis, Ari Holtzman, Noah A Smith, and Luke Zettlemoyer. 2021.  Demix layers: Disentangling domains for modular language modeling.  _arXiv preprint arXiv:2108.05036_.
  * Houlsby et al. (2019) Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly. 2019.  Parameter-efficient transfer learning for nlp.  In _International Conference on Machine Learning_ , pages 2790–2799. PMLR.
  * Hu et al. (2022) Edward J Hu, yelong shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022.  LoRA: Low-rank adaptation of large language models.  In _International Conference on Learning Representations_.
  * Hu and Liu (2004) Minqing Hu and Bing Liu. 2004.  Mining and summarizing customer reviews.  In _Proceedings of ACM SIGKDD_.
  * Huang (2003) Guang-Bin Huang. 2003.  Learning capability and storage capacity of two-hidden-layer feedforward networks.  _IEEE Transactions on Neural Networks_ , 14(2):274–281.
  * Jiang et al. (2024) Albert Q Jiang, Alexandre Sablayrolles, Antoine Roux, Arthur Mensch, Blanche Savary, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Emma Bou Hanna, Florian Bressand, et al. 2024.  Mixtral of experts.  _arXiv preprint arXiv:2401.04088_.
  * Jurgens et al. (2018) David Jurgens, Srijan Kumar, Raine Hoover, Daniel A. McFarland, and Dan Jurafsky. 2018.  Measuring the evolution of a scientific field through citation frames.  _TACL_.
  * Ke and Liu (2022) Zixuan Ke and Bin Liu. 2022.  Continual learning of natural language processing tasks: A survey.  _ArXiv_ , abs/2211.12701.
  * Ke et al. (2023) Zixuan Ke, Yijia Shao, Haowei Lin, Tatsuya Konishi, Gyuhak Kim, and Bing Liu. 2023.  Continual pre-training of language models.  _arXiv preprint arXiv:2302.03241_.
  * Ke et al. (2021) Zixuan Ke, Hu Xu, and Bing Liu. 2021.  Adapting bert for continual learning of a sequence of aspect sentiment classification tasks.  In _NAACL_ , pages 4746–4755.
  * Kringelum et al. (2016) Jens Kringelum, Sonny Kim Kjaerulff, Søren Brunak, Ole Lund, Tudor I Oprea, and Olivier Taboureau. 2016.  Chemprot-3.0: a global chemical biology diseases mapping.  _Database_ , 2016.
  * Liu et al. (2019) Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019.  Roberta: A robustly optimized bert pretraining approach.  _arXiv preprint arXiv:1907.11692_.
  * Lo et al. (2020) Kyle Lo, Lucy Lu Wang, Mark Neumann, Rodney Kinney, and Daniel S. Weld. 2020.  S2ORC: the semantic scholar open research corpus.  In _ACL_.
  * Luan et al. (2018) Yi Luan, Luheng He, Mari Ostendorf, and Hannaneh Hajishirzi. 2018.  Multi-task identification of entities, relations, and coreference for scientific knowledge graph construction.  In _ACL_.
  * Luo et al. (2024) Tongxu Luo, Jiahe Lei, Fangyu Lei, Weihao Liu, Shizhu He, Jun Zhao, and Kang Liu. 2024.  Moelora: Contrastive learning guided mixture of experts on parameter-efficient fine-tuning for large language models.  _arXiv preprint arXiv:2402.12851_.
  * Luo et al. (2023) Ziyang Luo, Can Xu, Pu Zhao, Qingfeng Sun, Xiubo Geng, Wenxiang Hu, Chongyang Tao, Jing Ma, Qingwei Lin, and Daxin Jiang. 2023.  Wizardcoder: Empowering code large language models with evol-instruct.  _Preprint_ , arXiv:2306.08568.
  * Maas et al. (2011) Andrew L. Maas, Raymond E. Daly, Peter T. Pham, Dan Huang, Andrew Y. Ng, and Christopher Potts. 2011.  Learning word vectors for sentiment analysis.  In _Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies_ , pages 142–150, Portland, Oregon, USA. Association for Computational Linguistics.
  * McCloskey and Cohen (1989) Michael McCloskey and Neal J Cohen. 1989.  Catastrophic interference in connectionist networks: The sequential learning problem.  In _Psychology of learning and motivation_ , volume 24, pages 109–165. Elsevier.
  * Ni et al. (2019) Jianmo Ni, Jiacheng Li, and Julian J. McAuley. 2019.  Justifying recommendations using distantly-labeled reviews and fine-grained aspects.  In _EMNLP_ , pages 188–197. Association for Computational Linguistics.
  * Qin and Joty (2021) Chengwei Qin and Shafiq Joty. 2021.  Lfpt5: A unified framework for lifelong few-shot language learning based on prompt tuning of t5.  _arXiv preprint arXiv:2110.07298_.
  * Qin et al. (2022) Yujia Qin, Jiajie Zhang, Yankai Lin, Zhiyuan Liu, Peng Li, Maosong Sun, and Jie Zhou. 2022.  Elle: Efficient lifelong pre-training for emerging data.  _arXiv preprint arXiv:2203.06311_.
  * Qiu et al. (2023) Zihan Qiu, Zeyu Huang, and Jie Fu. 2023.  Emergent mixture-of-experts: Can dense pre-trained transformers benefit from emergent modular structures?  _arXiv preprint arXiv:2310.10908_.
  * Qiu et al. (2024) Zihan Qiu, Zeyu Huang, and Jie Fu. 2024.  Unlocking emergent modularity in large language models.  _Preprint_ , arXiv:2310.10908.
  * Raffel et al. (2020) Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. 2020.  Exploring the limits of transfer learning with a unified text-to-text transformer.  _Journal of Machine Learning Research_ , 21:1–67.
  * Raffel et al. (2023) Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2023.  Exploring the limits of transfer learning with a unified text-to-text transformer.  _Preprint_ , arXiv:1910.10683.
  * Razdaibiedina et al. (2023) Anastasia Razdaibiedina, Yuning Mao, Rui Hou, Madian Khabsa, Mike Lewis, and Amjad Almahairi. 2023.  Progressive prompts: Continual learning for language models.  In _The Eleventh International Conference on Learning Representations_.
  * Sakaguchi et al. (2019) Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. 2019.  Winogrande: An adversarial winograd schema challenge at scale.  _arXiv preprint arXiv:1907.10641_.
  * Scialom et al. (2022) Thomas Scialom, Tuhin Chakrabarty, and Smaranda Muresan. 2022.  Fine-tuned language models are continual learners.  In _Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing_ , pages 6107–6122.
  * Serrà et al. (2018) Joan Serrà, Didac Suris, Marius Miron, and Alexandros Karatzoglou. 2018.  Overcoming catastrophic forgetting with hard attention to the task.  In _ICML_.
  * Shi et al. (2024) Haizhou Shi, Zihao Xu, Hengyi Wang, Weiyi Qin, Wenyuan Wang, Yibin Wang, and Hao Wang. 2024.  Continual learning of large language models: A comprehensive survey.  _arXiv preprint arXiv:2404.16789_.
  * Song et al. (2024) Yixin Song, Haotong Xie, Zhengyan Zhang, Bo Wen, Li Ma, Zeyu Mi, and Haibo Chen. 2024.  Turbo sparse: Achieving llm sota performance with minimal activated parameters.  _Preprint_ , arXiv:2406.05955.
  * Sun et al. (2024) Mingjie Sun, Zhuang Liu, Anna Bair, and J. Zico Kolter. 2024.  A simple and effective pruning approach for large language models.  _Preprint_ , arXiv:2306.11695.
  * Touvron et al. (2023) Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. 2023.  Llama: Open and efficient foundation language models.  _arXiv preprint arXiv:2302.13971_.
  * Wang et al. (2018) Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R Bowman. 2018.  Glue: A multi-task benchmark and analysis platform for natural language understanding.  _arXiv preprint arXiv:1804.07461_.
  * Wang et al. (2023a) Liyuan Wang, Xingxing Zhang, Hang Su, and Jun Zhu. 2023a.  A comprehensive survey of continual learning: Theory, method and application.  _ArXiv_ , abs/2302.00487.
  * Wang et al. (2024a) Liyuan Wang, Xingxing Zhang, Hang Su, and Jun Zhu. 2024a.  A comprehensive survey of continual learning: Theory, method and application.  _IEEE Transactions on Pattern Analysis and Machine Intelligence_.
  * Wang et al. (2024b) Mingyang Wang, Heike Adel, Lukas Lange, Jannik Strötgen, and Hinrich Schütze. 2024b.  Rehearsal-free modular and compositional continual learning for language models.  _arXiv preprint arXiv:2404.00790_.
  * Wang et al. (2024c) Mingyang Wang, Heike Adel, Lukas Lange, Jannik Strötgen, and Hinrich Schütze. 2024c.  Rehearsal-free modular and compositional continual learning for language models.  _Preprint_ , arXiv:2404.00790.
  * Wang et al. (2023b) Xiao Wang, Tianze Chen, Qiming Ge, Han Xia, Rong Bao, Rui Zheng, Qi Zhang, Tao Gui, and Xuan-Jing Huang. 2023b.  Orthogonal subspace learning for language model continual learning.  In _Findings of the Association for Computational Linguistics: EMNLP 2023_ , pages 10658–10671.
  * Wang et al. (2023c) Xiao Wang, Yuansen Zhang, Tianze Chen, Songyang Gao, Senjie Jin, Xianjun Yang, Zhiheng Xi, Rui Zheng, Yicheng Zou, Tao Gui, et al. 2023c.  Trace: A comprehensive benchmark for continual learning in large language models.  _arXiv preprint arXiv:2310.06762_.
  * Wang et al. (2024d) Yifan Wang, Yafei Liu, Chufan Shi, Haoling Li, Chen Chen, Haonan Lu, and Yujiu Yang. 2024d.  Inscl: A data-efficient continual learning paradigm for fine-tuning large language models with instructions.  _arXiv preprint arXiv:2403.11435_.
  * Wang et al. (2023d) Zhicheng Wang, Yufang Liu, Tao Ji, Xiaoling Wang, Yuanbin Wu, Congcong Jiang, Ye Chao, Zhencong Han, Ling Wang, Xu Shao, and Wenqiu Zeng. 2023d.  Rehearsal-free continual language learning via efficient parameter isolation.  _ArXiv_.
  * Wei et al. (2024) Yuxiang Wei, Zhe Wang, Jiawei Liu, Yifeng Ding, and Lingming Zhang. 2024.  Magicoder: Empowering code generation with oss-instruct.  _Preprint_ , arXiv:2312.02120.
  * Wu et al. (2024) Tongtong Wu, Linhao Luo, Yuan-Fang Li, Shirui Pan, Thuy-Trang Vu, and Gholamreza Haffari. 2024.  Continual learning for large language models: A survey.  _arXiv preprint arXiv:2402.01364_.
  * Xu et al. (2019) Hu Xu, Bing Liu, Lei Shu, and Philip S. Yu. 2019.  BERT post-training for review reading comprehension and aspect-based sentiment analysis.  In _NAACL-HLT_.
  * Zellers et al. (2019) Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. 2019.  Hellaswag: Can a machine really finish your sentence?  In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_.
  * Zhang et al. (2023a) Hang Zhang, Xin Li, and Lidong Bing. 2023a.  Video-llama: An instruction-tuned audio-visual language model for video understanding.  _arXiv preprint arXiv:2306.02858_.
  * Zhang et al. (2015) Xiang Zhang, Junbo Zhao, and Yann LeCun. 2015.  Character-level convolutional networks for text classification.  _Advances in neural information processing systems_ , 28.
  * Zhang et al. (2022) Zhengyan Zhang, Yankai Lin, Zhiyuan Liu, Peng Li, Maosong Sun, and Jie Zhou. 2022.  MoEfication: Transformer feed-forward layers are mixtures of experts.  In _Findings of ACL 2022_.
  * Zhang et al. (2023b) Zhengyan Zhang, Zhiyuan Zeng, Yankai Lin, Chaojun Xiao, Xiaozhi Wang, Xu Han, Zhiyuan Liu, Ruobing Xie, Maosong Sun, and Jie Zhou. 2023b.  Emergent modularity in pre-trained transformers.  _Preprint_ , arXiv:2305.18390.
  * Zhao et al. (2024) Weixiang Zhao, Shilong Wang, Yulin Hu, Yanyan Zhao, Bing Qin, Xuanyu Zhang, Qing Yang, Dongliang Xu, and Wanxiang Che. 2024.  Sapt: A shared attention framework for parameter-efficient continual learning of large language models.  _arXiv preprint arXiv:2401.08295_.
  * Zheng et al. (2023) Junhao Zheng, Shengjie Qiu, and Qianli Ma. 2023.  Learn or recall? revisiting incremental learning with pre-trained language models.  _arXiv preprint arXiv:2312.07887_.
  * Zhu et al. (2024) Didi Zhu, Zhongyi Sun, Zexi Li, Tao Shen, Ke Yan, Shouhong Ding, Kun Kuang, and Chao Wu. 2024.  Model tailor: Mitigating catastrophic forgetting in multi-modal large language models.  _arXiv preprint arXiv:2402.12048_.
  * Zhu et al. (2023) Xunyu Zhu, Jian Li, Yong Liu, Can Ma, and Weiping Wang. 2023.  A survey on model compression for large language models.  _Preprint_ , arXiv:2308.07633.

##  Appendix A Experimental Details

All experiments are run on an A100 $\times$ 8 DGX-machine.

###  A.1 Continual finetuning on T5

We adapted the code-base from O-LORA.555https://github.com/cmnfriend/O-LoRA

#### Finetuning (FT) and FT with MIGU.

  * •

The batch size is set to 64.

  * •

The optimization is performed using the AdamW algorithm with hyperparameters
$\beta_{1}=0.9$, $\beta_{2}=0.999$, and a weight decay coefficient of $0.01$.

  * •

The initial learning rate is set to $1\times 10^{-4}$, alongside a static
learning rate scheduler.

  * •

The threshold for mask selection is set at $0.7$ across orders 1 to 6 in the
FT+MIGU configuration.

#### Low-Rank Adaptation (LoRA) and LoRA with MIGU.

  * •

LoRA configuration: $r=8,\alpha=32$, dropout $=0.05$.

  * •

The learning rate is set to $1\times 10^{-3}$, with all other hyperparameters
being consistent with the FT+MIGU configuration.

#### Incremental LoRA (IncLoRA) and IncLoRA with MIGU.

  * •

For each LoRA module: $r=8,\alpha=32$, dropout $=0.05$.

  * •

Hyperparameters are identical to those specified in the LoRA and LoRA with
MIGU settings.

#### Order-Incremental LoRA (OIncLoRA) and OIncLoRA with MIGU.

  * •

The threshold for mask selection is set at $0.05$ across orders 1 to 6 in the
FT+MIGU configuration.

  * •

All remaining hyperparameters are consistent with the LoRA and LoRA with MIGU
settings.

#### LoRA Replay and LoRA Replay with MIGU.

  * •

The threshold for mask selection is set at $0.4$ across orders 1 to 6 in the
FT+MIGU configuration.

  * •

All remaining hyperparameters are consistent with the LoRA and LoRA with MIGU
settings.

###  A.2 Continual pre-training finetune on RoBERTa

We adapted the code-base from DAS.666https://github.com/UIC-Liu-
Lab/ContinualLM

#### Pre-training.

  * •

The batch size is set to 248.

  * •

The optimization is performed using the AdamW algorithm with hyperparameters
$\beta_{1}=0.9$, $\beta_{2}=0.999$, and a weight decay coefficient of $0$.

  * •

The initial learning rate is set to $1\times 10^{-4}$, alongside a linear
learning rate scheduler.

  * •

The threshold for mask selection is set at $0.7$ on the sequence of tasks.

#### Tuning.

  * •

The batch size is set to 16.

  * •

The optimization is performed using the AdamW algorithm with hyperparameters
$\beta_{1}=0.9$, $\beta_{2}=0.999$, and a weight decay coefficient of $0.01$.

  * •

The initial learning rate is set to $3\times 10^{-5}$, alongside a linear
learning rate scheduler.

###  A.3 Instruct finetuning on Llama2.

  * •

The optimization is performed using the AdamW algorithm with hyperparameters
$\beta_{1}=0.9$, $\beta_{2}=0.95$, and a weight decay coefficient of $0$.

  * •

The initial learning rate is set to $5\times 10^{-4}$, alongside a cosine
learning rate scheduler with warmup $=0.1$ of the total duration.

  * •

LoRA configuration: $\alpha=32$, dropout $=0.05$.

##  Appendix B Benchmark Instruction

Benchmark | Order | Task Sequence
---|---|---
Standard CL | 1 | dbpedia → amazon → yahoo → ag
2 | dbpedia → amazon → ag → yahoo
3 | yahoo → amazon → ag → dbpedia
Long sequence | 4 |  | mnli → cb → wic → copa → qqp → boolqa → rte → imdb →
---
Yelp → amazon → sst-2 → dbpedia → ag → multirc → yahoo
5 |  | multirc → boolqa → wic → mnli → cb → copa → qqp → rte
---
→ imdb → sst-2 → dbpedia → ag → Yelp → amazon → yahoo
6 |  | Yelp → amazon → mnli → cb → copa → qqp → rte → imdb →
---
sst-2 → dbpedia → ag → yahoo → multirc → boolqa → wic
DAS | 7 | Restaurant $\to$ ACL $\to$ AI $\to$ Phone $\to$ PubMed $\to$ Camera
Table 7: Task Sequence Orders for Continual Learning Experiments. Orders 1-3
represent the conventional task sequences employed in standard continual
learning benchmarks Zhang et al. (2015). Orders 4-6 extend to longer
sequences, encompassing 15 tasks each Razdaibiedina et al. (2023). Order 7
comprises a sequence of 6 tasks derived from unsupervised pre-training
domains, in accordance with Ke et al. (2023).

###  B.1 Dataset Information

Our experimental section encompasses datasets including the Standard CL
benchmark and Long sequence benchmark, both of which are utilized for
instruction finetuning on the T5-large model; the DAS benchmark, which is used
for continual pre-training on RoBERTa; the Magicoder-Evol-Instruct-110K, which
pertains to instruction tuning on Llama-2-7B; and the datasets Hellaswag,
WinoGrande, and ARC-Challenge for evaluating the finetuned Llama-2-7B.

#### Standard CL benchmark.

For continual finetuning, we use MTL5 dataset introduced by Zhang et al.
(2015), and follow the setup from LFPT5 and O-LoRA Qin and Joty (2021); Wang
et al. (2023b) to pick four text classification datasets (AG News, Amazon
reviews, DBpedia and Yahoo Answers) and shuffle the tasks into three different
orders.

#### Long sequence benchmark.

Razdaibiedina et al. (2023) extends the Standard CL benchmark by introducing a
long sequence benchmark for continual learning benchmark with 15 datasets.
This includes five tasks from CL benchmark, four from GLUE benchmark (MNLI,
QQP, RTE, SST2) Wang et al. (2018), five from SuperGLUE benchmark (WiC, CB,
COPA, MultiRC, BoolQ) Wang et al. (2018), and the IMDB movie reviews dataset
Maas et al. (2011). Following Razdaibiedina et al. (2023), we select 1000
random samples for training each task and hold out 500 samples per class for
validation.

#### DAS Benchmark.

(Ke et al., 2023) introduce a new benchmark for continual pre-training of LMs,
which is more challenging as the data required to pre-train is much larger and
LMs are easier to forget previous knowledge. DAS is composed of 6 unlabeled
domain corpora, which contain 3 reviews: Yelp Restaurant Xu et al. (2019),
Amazon Phone Ni et al. (2019), Amazon Camera Ni et al. (2019); 3 of them are
academic papers: ACL Papers Lo et al. (2020), AI Papers Lo et al. (2020), and
PubMed Papers 777https://pubmed.ncbi.nlm.nih.gov/. and evaluated by 6
corresponding classification datasets are: Restaurant
888https://alt.qcri.org/semeval2014/task4/, PhoneDing et al. (2008); Hu and
Liu (2004), Camera Ding et al. (2008); Hu and Liu (2004), ACL (ACL-ARC in
(Jurgens et al., 2018)), AI (SCIERC in (Luan et al., 2018)), and PubMed
(CHEMPORT in (Kringelum et al., 2016)).

#### Magicoder-Evol-Instruct-110K.

This dataset Wei et al. (2024) contains 72.97M tokens of programming questions
and answers. It reproduces the “Evol-Instruct” dataset of WizardCoder Luo et
al. (2023): an LLM (GPT-4) is iteratively prompted to increase the difficulty
of a set of question-answer pairs (from Code Alpaca Chaudhary (2023)). Due to
computation constraints, we pick contain 20% the samples to instruct tuning
the Llama-2-7B model.

#### HellaSwag, WinoGrade and ARC-challenge.

For how much they forget the old knowledge, we follow the Biderman et al.
(2024) that averages three benchmarks, HellaSwag Zellers et al. (2019),
WinoGrade Sakaguchi et al. (2019) and ARC-challenge Clark et al. (2018).
HellaSwag benchmark includes 70K problems, each describing an event with
multiple possible continuations. The task is to pick the most plausible
continuation, requiring inferences about nuanced everyday situations.
WinoGrande benchmark also assesses commonsense reasoning. It includes 44K
problems with sentences that require ambiguous pronoun resolution. ARC-
Challenge benchmark consists of 7,787 grade-school level, multiple-choice
science questions, testing capabilities in complex reasoning and understanding
scientific concepts.

| Standard CL Benchmark (4 tasks) | Longer CL Benchmark (15 tasks)
---|---|---
| Order-1 | Order-2 | Order-3 | avg | Order-4 | Order-5 | Order-6 | avg
SeqFT | 18.9 | 24.9 | 41.7 | 28.5 | 7.4 | 7.4 | 7.5 | 7.4
SeqLoRA | 44.6 | 32.7 | 53.7 | 43.7 | 2.3 | 0.6 | 1.9 | 1.6
IncLoRA | 66 | 64.9 | 68.3 | 66.4 | 63.3 | 58.5 | 61.7 | 61.2
Replay | 55.2 | 56.9 | 61.3 | 57.8 | 55 | 54.6 | 53.1 | 54.2
EWC | 48.7 | 47.7 | 54.5 | 50.3 | 45.3 | 44.5 | 45.6 | 45.1
LwF | 54.4 | 53.1 | 49.6 | 52.3 | 50.1 | 43.1 | 47.4 | 46.9
L2P | 60.3 | 61.7 | 61.1 | 60.7 | 57.5 | 53.8 | 56.9 | 56.1
LFPT5 | 67.6 | 72.6 | 77.9 | 72.7 | 70.4 | 68.2 | 69.1 | 69.2
MoELoRA | 52.8 | 49.6 | 59.8 | 54.1 | 36.3 | 31.4 | 15.1 | 27.6
O-LoRA | 77.1 | 76.2 | 76.6 | 76.6 | 68.4 | 68.8 | 71.4 | 69.5
MoCL | 75.6 | 75.4 | 76.7 | 75.9 | - | - | - | -
ProgPrompt | 75.2 | 75 | 75.1 | 75.1 | 78.0 | 77.7 | 77.9 | 77.9
PerTaskFT | 70.0 | 70.0 | 70.0 | 70.0 | 78.1 | 78.1 | 78.1 | 78.1
MTL | 80.0 | 80.0 | 80.0 | 80.0 | 76.5 | 76.5 | 76.5 | 76.5
FT | 74.4 | 75.0 | 77.5 | 75.7 | 70.6 | 69.7 | 65.6 | 68.3
MIGU + FT | 78.3 | 79.8 | 78.3 | 78.8 | 77.1 | 73.6 | 70.7 | 73.8
LORA | 60.7 | 70.0 | 73.1 | 67.9 | 53.7 | 44.4 | 39.8 | 46.0
MIGU + LORA | 74.8 | 71.6 | 73.5 | 73.3 | 66.9 | 64.8 | 51.8 | 61.2
OIncLoRA | 77.1 | 76.2 | 76.6 | 76.6 | 68.4 | 68.8 | 71.4 | 69.5
MIGU + OIncLoRA | 77.1 | 77.0 | 75.6 | 76.6 | 67.3 | 68.5 | 74.2 | 70.0
LORAReplay | 77.1 | 73.4 | 73.2 | 74.5 | 74.5 | 75.4 | 75.7 | 75.2
MIGU + LORAReplay | 77.8 | 75.1 | 75.9 | 76.2 | 75.4 | 76.8 | 77.2 | 76.5
Table 8: Summary of the results on two standard CL benchmarks with T5-large
model. Averaged accuracy after training on the last task is reported. All
results in the last block are averaged over 3 runs. (We reuse the table
template and experiment results from O-LoRA Wang et al. (2023b) to construct
the results of the top two blocks). It is noticeable some baselines in some
previous literature show significant lower performance than ours, we assume
this may due to our strict parameter grid search for baseline methods.

###  B.2 Training orders

The training orders in 3 benchmarks on T5-large and RoBERTa models are shown
in table 7.

##  Appendix C Baselines for all settings

###  C.1 Baselines on Standard CL benchmark and Long sequence benchmark

We reuse some baseline descriptions from O-LoRA Wang et al. (2023b).

  * •

FT de Masson D’Autume et al. (2019): train all model parameters on a sequence
of tasks (without adding any regularization or replaying samples from the
previous tasks).

  * •

LoRA: fixed-size LoRA parameters are trained on a sequence of tasks (without
adding any regularization or replaying samples from the previous tasks).

  * •

IncLoRA: incremental learning of new LoRA parameters on a sequential series of
tasks (without adding any regularization or replaying samples from the
previous tasks).

  * •

Replay: finetune the whole model with a memory buffer, and replay samples from
old tasks when learning new tasks to avoid forgetting.

  * •

LFPT5 Qin and Joty (2021): continuously train a soft prompt that
simultaneously learns to solve the tasks and generate training samples, which
are subsequently used in experience replay.

  * •

OIncLoRA Wang et al. (2023b): learns tasks in different LoRA subspaces that
are kept orthogonal to each other and sums all LoRA weights up at testing
time.

  * •

MoCL Wang et al. (2024c): MoCL continually adds new modules to language models
and composes them with existing modules.

  * •

SAPT Zhao et al. (2024): In the SAPT method, a Shared Attentive Learning and
Selection Module (SALS) is employed to guide training samples through optimal
PET blocks for task-specific learning, using a unique instance-level attention
mechanism. This process ensures efficient continual learning for large
language models.

  * •

MoELORA Luo et al. (2024): MoELoRA considers LoRA as a Mixture of Experts,
leveraging the modeling capabilities of multiple experts for complex data
domains, as well as utilizing LoRA’s parameter-efficient characteristics.

  * •

NCL (Naive CL) continually DAP-trains the RoBERTa;

  * •

NCL-Adapter continually DAP-trains a set of adapters (Houlsby et al., 2019)

  * •

DER++ (Buzzega et al., 2020) is a replay method based on knowledge
distillation. 16.4K tokens are saved for each domain in the replay memory.

  * •

DEMIX (Gururangan et al., 2021) adds a new adapter for each new domain and
initializes it with a previous adapter nearest to the new domain;

  * •

HAT-Adapter (Serrà et al., 2018): HAT is an effective parameter-isolation
method. HAT is applied to Transformer layers (i.e., self-attention,
intermediate and output layers).

  * •

HAT-Adapter (Ke et al., 2021): HAT-Adapter uses HAT within adapters.

  * •

DAS Ke et al. (2023) DAS proposes a soft-masking method to overcome CF and to
encourage KT, and a constrative learning-based method for knowledge
integration.

##  Appendix D Experimental Results

###  D.1 Experiment on T5

We report more detailed results on the Standard CL benchmark and Long sequence
benchmark in table 8, including each order results and their corresponding
average results. To more intuitively display our results compared to the
baseline, we plotted violin graphs showing the performance with and without
our method under the condition of full finetuning as Figure 10 11 12 13.

Figure 10: Performance comparison on the standard cl benchmark under full
finetuning setting, with and without the implementation of our method.. Figure
11: Average performance comparison on the standard cl benchmark under full
finetuning setting, with and without the implementation of our method.. Figure
12: Performance comparison on the standard cl benchmark under full finetuning
setting, with and without the implementation of our method.. Figure 13:
Average performance comparison on the standard cl benchmark under full
finetuning setting, with and without the implementation of our method..

###  D.2 Experiment on RoBERTa

#### Detailed experiment results

The violin graphs results is shown as Figure 14.

Figure 14: Performance comparison on the DAS benchmark under continual domain
pre-training setting, with and without the implementation of our method.
Figure 15: Ablation on LoRA MIGU+cluster design Figure 18: Accuracy on ARC-
Challenge Clark et al. (2018), HellaSwag Zellers et al. (2019) and Winogrande
Sakaguchi et al. (2019), evaluating on Llama-2-7B by MIGU with LoRA and
valinna LoRA instruct tuning pre-trained on Magicoder-Evol-Instruct-110k Wei
et al. (2024).

#### Efficiency

We also conduct an efficiency ablation by comparing FT, FT+MIGU and DAS
because continual-pre-training is a relatively computational-intensive
setting. DAS is a typical parameter-based regularization methods. We record
the wall time required for the first three dataset given the same GPU
configuration: A100 $\times$ 2\. As shown in the Table 9, FT+MIGU only occur
an approximately 10% overhead in wall time, due to the extra masking step in
the backward propagation phase while DAS achieves a magnitude larger overhead.

| Restaurant | ACL | AI
---|---|---|---
FT | 25.3(+0.0%) | 26.7(+0.0%) | 15.5(+0.0%)
FT + MIGU | 27.3(+7.9%) | 29.2(+9.4%) | 17.0(+12.9%)
DAS | 78.0(+208%) | 66.5(+154%) | 45.0(+190%)

Table 9: The wall time($\min$) on three domain pre-training dataset. Figure
19: Differences between (a)Dense, (b)MoE and (c)MIGU

###  D.3 Experiment on Llama2

The detailed violin graphs results about ARC-Challenge Clark et al. (2018),
HellaSwag Zellers et al. (2019) and Winogrande Sakaguchi et al. (2019) are
seperately shown in Figure LABEL:fig:Llama_code_arcc,
LABEL:fig:Llama_code_hellaswag, LABEL:fig:Llama_code_winogrande.

###  D.4 Visualization

To investigate how our method enhances model performance, we visualized the
variation in product magnitudes between an FT model and an FT model augmented
with our MIGU technique in Figures 23,31. We employed heatmaps to depict the
similarity in product magnitude distributions across different tasks. Our
findings reveal that task similarity in the FT model with MIGU implementation
is markedly reduced. This suggests that the models trained with our method
exhibit more distinctive weight activations for different tasks, thereby
mitigating their conflict. This distinction in activation patterns indicates
our method’s ability to foster more task-specific representations within the
model, contributing to its improved performance across varied learning
scenarios.

#### Magnitude Distribution.

We plot the L1-normalized magnitude distribution of COPA sample, BoolQA
sample, and Yelp sample on the first linear layer of 23-th FFN layer of
T5-large model in Figure 32.

##  Appendix E Ablation on MIGU+Cluster

Informed by the works on Mixture of Experts (MoE) Jiang et al. (2024),
Emergent MoE (EMoE) Qiu et al. (2023), and MoEfication Zhang et al. (2022), we
investigate explicit clustering of weight vectors in LMs to construct expert
groups. Technically, we treat the linear layer’s weight matrix $\mathbf{W}$ as
a set of $d_{out}$ vectors, each of dimension $d_{in}$. These vectors are then
partitioned into $N$ clusters, analogous to MoE experts.

###  E.1 Implementation

As detailed in § 3.2, our method encompasses four core processes in cluster-
based implementation. During the data forward phase, the product magnitudes of
the weight vectors are computed and tracked. Subsequently, in the second
phase, MIGU caches these magnitudes and employs an L1-norm normalization to
derive a gradient mask. This mask is pivotal for modulating the gradients in
the subsequent phases. The third phase involves the standard backpropagation
to calculate the gradients of the parameters. Finally, in the fourth phase,
the earlier computed gradient mask is applied to the obtained gradients,
ensuring a modulated update of the parameters. This modulation is consistent
within each cluster, thereby maintaining the integrity of the expert groupings
and enhancing the model’s learning efficacy. We also plot a Figure 19 to
illustrate the differences between Dense, MoE and ours in forward and backward
phase.

We explored two distinct clustering strategies:

  * •

Weight Cluster Combination: The weight vectors are clustered into $N$ groups
based on their proximity in the weight space.

  * •

Co-magnitude Guided Combination: Using a subset of the dataset, we group
weight vectors into clusters based on the similarity of their product
magnitudes.

###  E.2 Result & Analysis

The outcomes of two distinct clustering approaches, alongside our
implementation within LoRA, are illustrated in Figure 15. It is evident that,
except for the second order, the “Weight Cluster” method surpasses the ’No
Cluster’ approach, which does not employ explicit clustering. However, the ’No
Cluster’ method demonstrates superior performance across the remaining orders,
highlighting its robustness and effectiveness. Nonetheless, the other two
explicit clustering techniques still significantly outperform the baseline
vanilla continual learning LoRA, indicating their potential for further
exploration.

Figure 23: The product magnitude distribution similarity of different tasks in
the FFN of the last transformer block: (a,b) 1-st linear layer; (c,d) 2-nd
linear layer. Figure 31: The product magnitude distribution similarity of
different tasks in the MHA of the last transformer block: (a,b) query linear
layer; (c,d) key linear layer; (e,f) value linear layer; (g,h) output linear
layer. Figure 32: The Magnitudes distribution of COPA sample, BoolQA sample,
and Yelp sample on the first linear layer of 23-th FFN layer of T5-large
model.


# ON LOCALLY FINITE DIMENSIONAL TRACES

Massoud Amini, Mehdi Moradi

###### Abstract.

We partially resolve two open questions on approximation properties of traces
on simple $C^{*}$-algebras. We answer a question raised by Nate Brown by
showing that locally finite dimensional (LFD) traces form a convex set for
simple $C^{*}$-algebras. We prove that all the traces on the reduced
$C^{*}$-algebra $C^{*}_{r}(\Gamma)$ of a discrete amenable ICC group $\Gamma$
are LFD, and conclude that $C^{*}_{r}(\Gamma)$ is strong-NF in the sense of
Blackadar-Kirchberg in this case. This partially answers another open question
raised by Brown.

###### 1991 Mathematics Subject Classification:

Primary 46L35 ; Secondary 19K14

##  1\. Introduction

In his monumental paper [10] Alain Connes introduced the notion of amenable
trace (cf., Definition 2.1 below) under the title of “hyper-trace” and proved
that a type II1-factor is hyperfinite if and only if its unique tracial state
is amenable. Exploiting this fact, Connes proves in [11] the non-existence of
finite summable Fredholm modules on reduced group $C^{*}$-algebra of a free
group. In the same vein, Eberhard Kirchberg defined liftable traces on
$C^{*}$-algebras and proved that a discrete group with Kazhdan property $(T)$
is residually finite if and only if the canonical trace of full group
$C^{*}$-algebra is liftable [16]. Also, he proves that a general discrete
group $\Gamma$ has factorization property if and only if its canonical trace
of full group algebra, $C^{*}(\Gamma)$, is liftable.

Eventually, it was Nate Brown who recognized the central role of amenable
tracial states and elaborated on how they could be employed in studying finite
dimensional approximation properties of $C^{*}$-algebras, and used them to
give characterizations of central notions such as liftability [5]. Based on
the pioneering work of Sorin Popa [22], Brown also introduced and studied the
notions of quasidiagonal and locally finite dimensional traces and their
uniform counterparts, but unlike his thorough analysis of amenable traces, he
gave no characterization of quasidiagonality or local finite dimensionality.

In their celebrated paper [3], Blackadar and Kirchberg examined the class of
$C^{*}$-algebras that can be written as a generalized inductive limit of
finite dimensional $C^{*}$-algebras. In particular, they introduced the class
of strong-NF algebras, where the connecting maps are complete order
embeddings, and characterized them as exactly the class of nuclear and inner-
QD $C^{*}$-algebras, for example every RFD $C^{*}$-algebra is inner-QD hence
any nuclear and RFD algebra is strong-NF. They further showed that for
separable $C^{*}$-algebras, inner quasidiagonality is equivalent to the
existence of a separating sequence of irreducible quasidiagonal
representations [3]. For this reason, inner quasidiagonality is not homotopy
invariant and does not pass to subalgebras or quotients in general 111For the
lack of homotopy invariance and passing to subalgebras, consider the cone
$C_{0}((0,1],\mathcal{O}_{n})$ of the Cuntz algebra, and note that its
irreducible representations factor through point evaluations on $(0,1]$ and so
cannot be QD, since $\mathcal{O}_{n}$ is not QD, also the cone embeds into an
AF algebra which is inner-QD (actually strong QD) [20]. For the lack of
passing to quotients, consider $C^{*}(\mathbb{F}_{2})$, the maximal group
$C^{*}$-algebra of free group on two generators, which is RFD [9] and
$\mathcal{O}_{2}$ arises as its quotient..

One of the central problems in approximation theory of traces on
$C^{*}$-algebras is the question that in what circumstances all amenable
traces are quasidiagonal [30, 7.3.2], [6]. The first major breakthrough was a
result of Tikuisis-White-Winter, who showed that faithful traces on separable
nuclear $C^{*}$-algebras in the UCT class are quasidiagonal [26]. In
particular, as examples in [8] and [25] suggest, it is plausible that for a
discrete group $G$, the full $C^{*}$-algebra $C^{*}(G)$ is inner quasidiagonal
provided that $G$ is amenable. A partial answer to this question is provided
by Proposition D below.

The main objective of the current paper is to give affirmative answers to some
natural questions along the above lines. First, we obtain a new
characterization of locally finite dimensional traces, as follows:

Theorem A. A tracial state $\tau$ on a simple $C^{*}$-algebra $A$ is locally
finite dimensional iff for every $\epsilon>0$, every positive integer $n\geq
1$, and every $a_{1},...,a_{n}\in A$, there are disjoint irreducible
representations $\pi_{j}:A\to B(H_{j})$ and finite rank projections $p_{j}\in
B(H_{j})$, for $1\leq j\leq n$, such that

$\Big{|}\tau(a_{i})-\frac{\sum_{j=1}^{n}{Tr(p_{j}\pi_{j}(a_{i}))}}{\sum_{j=1}^{n}{Tr(p_{j})}}\Big{|}<\epsilon,$

for each $1\leq i\leq n$, and

$\max_{1\leq j\leq n}{\|[p_{j},\pi_{j}(a_{i})]\|}<\epsilon,$

where $Tr$ is the canonical trace on the matrix algebra $p_{j}B(H_{j})p_{j}$
i.e. the sum of eigenvalues counted with multiplicities.

This is then used to prove that for simple $C^{*}$-algebras, these tracial
states form a convex set, partially answering an open question raised by Brown
[5, page 29].

Corollary B. The set of all locally finite dimensional traces on a simple
separable $C^{*}$-algebra form a convex set.

To highlight the importance of LFD traces, we observe that if for each
separable $C^{*}$-algebra $A$, uniformly amenable QD traces are LFD, it
follows that hyperfinite type II1-factor $\mathcal{R}$ is QD, something which
seems to be wide-open at present.

Proposition C. The following statements are equivalent:

$(i)$ For every separable $C^{*}$-algebra $C$, $UAT(C)\cap AT_{QD}(C)\subset
AT_{LFD}(C)$,

$(ii)$ The hyperfinite type $II_{1}$-factor $\mathcal{R}$ is quasidiagonal,
and for every unital inclusion of separable $C^{*}$-algebras $B\subset A$ if
$\tau\in UAT(A)\cap AT_{LFD}(A)$ then $\tau|_{B}\in AT_{LFD}(B)$.

Our last result gives a characterization of inner quasidiagonality of the
reduced group $C^{*}$-algebra of a discrete group, giving an inner-QD analog
of the celebrated Rosenberg conjecture, already settled by Tikuisis-White-
Winter in [26]. Indeed, we prove a little more.

Proposition D. Let $\Gamma$ be a discrete amenable ICC group, then all traces
on the reduced $C^{*}$-algebra $C^{*}_{r}(\Gamma)$ are automatically locally
finite dimensional, and in particular, $C^{*}_{r}(\Gamma)$ is strong-NF.

##  2\. Preliminaries and conventions

In this section we recall a number of preliminary facts on approximation
properties of traces and inner quasidiagonality, needed in the rest of this
paper. In this paper, traces are always meant to be tracial states and $T(A)$
denotes the collection of all traces. We write $\mathbb{M}_{n}$ for the
$C^{*}$-algebra of all $n\times n$ complex matrices with canonical non
normalized tracial functional $Tr$. We denote the normalized trace by $tr$ or
more specifically by $tr_{n}$. For $x\in\mathbb{M}_{n}$, we write
$\|x\|_{2}:=tr_{n}(x^{*}x)^{\frac{1}{2}}$. Also, for a $C^{*}$-algebra $A$, we
denote the positive cone of $A$ by $A_{+}$.

When two representations $\pi:A\to B(H)$ and $\sigma:A\to B(K)$ are
(unitarily) equivalent via a unitary $v:H\to K$ we write $\pi\sim_{v}\sigma$.
For a representation $\pi:A\to B(H)$ and projection $p\in B(H)$, $A_{p}$ is
the set of all $a\in A$ with $p\pi(a)=\pi(a)p$.

For a non-degenerate representation $\rho$, we denote the central cover of
$\rho$ by $c(\rho)$. This is defined to be the central projection
$c(\rho):=1_{A^{**}}-e_{\rho}$, where $e_{\rho}$ is the unit of the
$W^{*}$-algebra $ker(\rho)^{**}$. It is known that $c(\rho)A^{**}$ is
isomorphic to $\rho(A)^{\prime\prime}$ and that $c(\rho)$ determines $\rho$ up
to quasi-equivalence [7, Page 6]. We say that an irreducible representation
$\rho$ is GCR if its range contains $\mathbb{K}(H_{\rho}).$ Finally, we say
that a projection $p\in A^{**}$ is in the socle of $A^{**}$ if the cut-down
$pA^{**}p$ is finite dimensional. Note that every finite rank projection in
$B(H_{\rho})$ actually lives in the socle of $A^{**}$.

In the rest of this section we recall the definition of certain classes of
traces and use the same notations as of the ones used by Brown in [5].

######  Definition 2.1.

A trace $\tau$ on a $C^{*}$-algebra $A$ is amenable if there exists a net of
c.c.p. (contractive completely positive) maps
$\varphi_{n}:A\to\mathbb{M}_{k(n)}$ such that

$\lim_{n\to\infty}{\|\varphi_{n}(ab)-\varphi_{n}(a)\varphi_{n}(b)\|_{2}}=0,\ \
\tau(a)=\lim_{n\to\infty}{tr_{k(n)}\circ\varphi_{n}(a)},$

for $a,b\in A$. An amenable trace $\tau$ is called QD (quasidiagonal) if
moreover

$\lim_{n\to\infty}{\|\varphi_{n}(ab)-\varphi_{n}(a)\varphi_{n}(b)\|}=0,$

for $a,b\in A,$ in the operator norm. Finally, an amenable trace $\tau$ on a
$C^{*}$-algebra $A$ is called LFD (locally finite dimensional) if there are
c.c.p. maps $\varphi_{n}:A\to\mathbb{M}_{k(n)}$ with

$d(a,A_{\varphi_{n}})\to 0,\ \ \
\tau(a)=\lim_{n\to\infty}{tr_{k(n)}\circ\varphi_{n}(a)},$

for $a\in A$, where $A_{\varphi_{n}}$ is the multiplicative domain of
$\varphi_{n}$.

When $A$ is unital, in the above definition, one could always arrange the
required maps to be unital, and when $A$ is separable, there is a sequence
(instead of a net) witnessing the above approximations. The sets of amenable,
QD, and LFD traces are denoted by $AT(A)$, $AT(A)_{QD}$, and $AT(A)_{LFD}$,
respectively.

Next, we recall the notion of QD $C^{*}$-algebras, introduced and studied
first by Javier Thayer [27] and later by Norberto Salinas [24], Dan Voiculescu
[28], [29]. It includes the natural subclass of inner-QD $C^{*}$-algebras of
Blackadar-Kirchberg [3].

######  Definition 2.2.

A separable $C^{*}$-algebra $A$ is QD (quasidiagonal) if there exists a
sequence of c.c.p. maps $\varphi_{n}:A\to\mathbb{M}_{k(n)}$ such that

$\lim_{n\to\infty}{\|\varphi_{n}(ab)-\varphi_{n}(a)\varphi_{n}(b)\|}=0,\ \
\lim_{n\to\infty}{\|\varphi_{n}(a)\|}=\|a\|,$

for $a,b\in A,$ and it is called inner-QD (inner quasidiagonal) if for each
$\epsilon>0$, each $n\geq 1$, and every $x_{1},...,x_{n}\in A$, there is a
projection $p$ in the socle of $A^{**}$ with $\|px_{j}p\|>\|x_{j}\|-\epsilon$
and $\|[p,x_{j}]\|<\epsilon$, for each $1\leq j\leq n$.

First let us recall a characterization of inner quasidiagonality, due to
Blackadar and Kirchberg (cf., [7, Corollary 11.3.7]).

######  Proposition 2.3.

A separable $C^{*}$-algebra $A$ is inner-QD if and only if there is a sequence
of c.c.p. maps $\varphi_{n}:A\to\mathbb{M}_{k(n)}$ such that
$\|a\|=\lim\|\varphi_{n}(a)\|$ and $d(a,A_{\varphi_{n}})\to 0$, for $a\in A$.

For a free ultrafilter $\omega$ on the set of natural numbers, let
$\mathcal{Q}_{\omega}$ be the ultrapower of the universal UHF algebra
$\mathcal{Q}$. Let
$q_{\omega}:\ell^{\infty}(\mathcal{Q})\to\mathcal{Q}_{\omega}$ be the
canonical quotient map and define the canonical trace
$\tau_{\mathcal{Q}_{\omega}}$ on $\mathcal{Q}_{\omega}$ by

$\tau_{\mathcal{Q}_{\omega}}(q_{\omega}(a_{1},a_{2},...))=\lim_{n\to\omega}{\tau_{\mathcal{Q}}(a_{n})}.$

This is then the unique tracial state on $\mathcal{Q}_{\omega}$ ([21, Theorem
8]). A c.c.p. map $\tilde{\Phi}:A\to\mathcal{Q}_{\omega}$ is said to be
liftable if there is a c.c.p. map $\Phi:A\to\ell^{\infty}(\mathcal{Q})$ such
that $q_{\omega}\circ\Phi=\tilde{\Phi}$.

Next we recall further characterizations of QD $C^{*}$-algebras and traces in
terms of liftability (see; [26, Proposition 1.4], [17, Theorem 4], and [14,
Proposition 3.4]).

######  Proposition 2.4.

For a separable $C^{*}$-algebra $A$,

($i$) $A$ is quasidiagonal iff there is a liftable *-monomorphism
$\Phi:A\to\mathcal{Q}_{\omega}$.

($ii$) If $A$ is unital then $A$ is inner QD iff there is a sequence of u.c.p.
maps $\varphi_{n}:A\to\mathbb{M}_{k(n)}$ with irreducible minimal
Stinespring’s dilation such that the induced map
$\Phi:A\to\frac{\prod\mathbb{M}_{k(n)}}{\sum\mathbb{M}_{k(n)}}$ is a faithful
$*$-homomorphism.

$(iii)$ A trace $\tau$ on $A$ is QD iff there is a liftable *-homomorphism
$\Phi:A\to\mathcal{Q}_{\omega}$ with
$\tau=\tau_{\mathcal{Q}_{\omega}}\circ\Phi$.

We have the following de facto result which we omit its straightforward proof
(using Propositions 2.3 and 2.4).

######  Proposition 2.5.

Let $A$ be a separable $C^{*}$-algebra admitting a faithful LFD trace then $A$
is inner-QD. Moreover, any unital, inner-QD $C^{*}$-algebra has at least one
LFD trace.

By a well-known result of Voiculescu, the cone of every $C^{*}$-algebra is
quasidiagonal (more is true, as quasidiagonality is homotopy invariant). An
analogous result in the context of QD traces is known as ”Gabe’s order-zero
quasidiagonality” (cf., [6, Proposition 3.2]).

######  Proposition 2.6.

Let $A$ be a separable $C^{*}$-algebra and $\tau\in AT(A)$ then there is a
c.c.p. order-zero map $\Phi:A\to\mathcal{Q}_{\omega}$ which is liftable and
$\tau(a)=\tau_{\mathcal{Q}_{\omega}}(\Phi(a)\Phi(1)^{n-1})$ for $n=1,2,\cdots$
and $a\in A$. In particular, every amenable trace on the cone of $A$ is
quasidiagoanl.

##  3\. Proof of the main results

In this section we show that the set of LFD traces is convex. For this, we
need a characterization of LFD traces in terms of irreducible representations
(to control the multiplicative domain of the underlying c.c.p. maps). First we
recall a result of Blackadar-Kirchberg [3], which gives a way to calculate the
distance to the multiplicative domain (cf., [7, Proposition 11.3.6]).

######  Lemma 3.1.

For a separable $C^{*}$-algebra $A$, let $p\in A^{**}$ be in the socle and
$A_{p}$ be the multiplicative domain of the map $a\mapsto pap$. Then
$d(a,A_{p})=\|[a,p]\|$, for each $a\in A$.

Next, let us prove the first main result of this paper.

Proof of Theorem A. Let $\tau$ be a LFD trace, and for $\epsilon>0$, and
contractions $a_{1},...,a_{n}\in A$, choose positive integer $N\geq 1$ and a
c.c.p. map $\varphi:A\to\mathbb{M}_{N}$ with

$d(a_{i},A_{\varphi})<\epsilon/4,\ \
\big{|}\tau(a_{i})-\frac{Tr(\varphi(a_{i}))}{N}\big{|}<\epsilon,$

for $1\leq i\leq n$. Choose contractions $b_{1},...,b_{n}\in A_{\varphi}$ such
that $\|a_{i}-b_{i}\|<\epsilon/2$. Since the restriction
$\varphi|_{A_{\varphi}}$ of $\varphi$ is a finite dimensional representation,
it decomposes into a direct sum of finitely many irreducible representations
of finite dimension say,
$\varphi|_{A_{\varphi}}=\sigma_{1}\oplus...\oplus\sigma_{r}$, by [18, Theorem
5.5.1.] there are irreducible representations $\tilde{\sigma}_{i}:A\to
B(\tilde{H_{i}})$ and closed vector subspaces
$H^{\prime}_{i}\subset\tilde{H_{i}}$ which are invariant for
$\sigma_{i}(A_{\varphi})$ such that the restriction of $\tilde{\sigma}_{i}$ to
$A_{\varphi}$ and Hilbert space $H^{\prime}_{i}$ is unitarily equivalent to
$\sigma_{i}$. After obvious identifications, let $p_{i}$ be the corresponding
orthogonal projection of $\tilde{H}_{i}$ onto $H_{i}$ which has a finite rank
as $dimH_{i}$ is finite. Since $A$ is simple, each $\tilde{\sigma}_{i}$ is
non-GCR, hence by [3, Theorem A.2] there are uncountably many disjoint
irreducible representations say $\pi_{i}$ with the same kernel as
$\tilde{\sigma}_{i}$ (hence approximate unitarily equivalent by Voiculescu’s
Theorem [12, Theorem II.5.8.]). Let $u_{i}$ be a unitary operator implementing
$\|\tilde{\sigma}_{i}(b_{j})-u_{i}^{*}\pi_{i}(b_{j})u_{i}\|<\epsilon/3$ for
$j=1,2,...,n$. Put $q_{i}:=u_{i}p_{i}u_{i}^{*}$, then,

$\Big{|}\tau(b_{j})-\frac{\sum_{i=1}^{r}{Tr(q_{i}\pi_{i}(b_{j}))}}{\sum_{i=1}^{r}{Tr(q_{i})}}\Big{|}<3\epsilon,$
$\max_{i}{\|[q_{i},\pi_{i}(b_{j})]\|}<\epsilon,$

for $j=1,2,...,n.$

Conversely, let such representations $\pi_{j}$ and projections $p_{j}$ exist
and put

$N:=\sum_{j=1}^{n}{Tr(p_{j})},\ \
\varphi(x):=\bigoplus_{j=1}^{n}{p_{j}\pi_{j}(x)p_{j}},\ \ p:=p_{1}+...+p_{n}.$

Let us identify each $B(H_{j})=\pi_{j}(A)^{\prime\prime}$ with a unique
summand of $A^{**}$, then each $p_{j}$ lives in the socle of $A^{**}$. Since
the representations $\pi_{j}$ are disjoint, these summands are different,
hence the projections $p_{j}$ are orthogonal and sum to a projection $p$ in
the socle of $A^{**}$. Now $A_{\varphi}$ consists of those elements commuting
with projections $p_{j}$ and hence with $p$, therefore, $A_{\varphi}\subset
A_{p}$. By Lemma 3.1,

$d(a,A_{p})=\|[a,p]\|=\max_{1\leq j\leq n}\|[p_{j},\pi_{j}(a)]\|,$

and we are done.

Blackadar and Kirchberg introduced the notion of pure matricial states in
[BKirredinnQD], here we need an analogous notion.

######  Definition 3.2.

Let $A$ be a $C^{*}$-algebra, $\mathfrak{F}\subset A$ be a finite set and
$\epsilon>0$. We call $f\in A^{*}$ an $(\mathfrak{F},\epsilon)$-ATS
(approximate tracial state) if there are finitely many disjoint irreducible
representations $\sigma_{i}:A\to B(H_{i})$, $i=1,2,...,n$ and finite rank
projections $p_{i}\in B(H_{i})$, such that
$f(a)=\frac{\sum_{i}{Tr(p_{i}\sigma_{i}(a))}}{\sum_{i}{Tr(p_{i})}}$, and
$\max_{i}{\|[p_{i},\sigma_{i}(x)]\|}<\epsilon$, for every $a\in A$ and
$x\in\mathfrak{F}$.

The following lemma is the key technical tool to prove convexity of
$AT_{LFD}$.

######  Lemma 3.3.

$(i)$ For disjoint irreducible representations $\sigma_{i}:A\to B(H_{i})$,
$i=1,2,...,n$, and finite rank projections $p_{i}\in B(H_{i})$ as above, the
map

$\varphi:A\to B(\bigoplus_{i}p_{i}H_{i});\ \
\varphi(a):=\oplus_{i=1}^{n}{{p_{i}\sigma_{i}(a)}p_{i}}\ \ (a\in A),$

is c.c.p. with $d(x,A_{\varphi})<\epsilon$, for all $x\in\mathfrak{F}.$

$(ii)$ If $A$ is simple and $f$ and $g$ are $(\mathfrak{F},\epsilon)$-ATS, for
$\delta>0$ there is a $(\mathfrak{F},\epsilon+\delta)$-ATS, $h$, with
$|h(x)-\frac{1}{2}(f+g)(x)|<\delta$, for all $x\in\mathfrak{F}$.

###### Proof.

The first assertion of part $(i)$ directly follows from the definition, and
the second follows from Lemma 3.1. For part $(ii)$, let
$f(x)=\frac{\sum_{i}{Tr(p_{i}\sigma_{i}(x))}}{\sum_{i}{Tr(p_{i})}}$ and
$g(x)=\frac{\sum_{r}{Tr(p^{\prime}_{r}\sigma^{\prime}_{r}(x))}}{\sum_{r}{Tr(p^{\prime}_{r})}}$,
with $\max_{i}{\|[p_{i},\sigma_{i}(x)]\|}<\epsilon,\ \
\max_{r}{\|[p^{\prime}_{r},\sigma^{\prime}_{r}(x)]\|}<\epsilon,$ for
$x\in\mathfrak{F}$ and $i=1,2,...,n$, $r=1,2,...,n^{\prime}$. Set
$d:=\sum_{i}{Tr(p_{i})}$ and $d^{\prime}:=\sum_{r}{Tr(p^{\prime}_{r})}$. Then,

$\frac{1}{2}(f(x)+g(x))=\frac{d^{\prime}\sum_{i}{Tr(p_{i}\sigma_{i}(x))}+d\sum_{r}{Tr(p^{\prime}_{r}\sigma^{\prime}_{r}(x))}}{2dd^{\prime}},$

for $x\in\mathfrak{F}.$ Since $A$ is simple, there are mutually disjoint
irreducible representations $\\{\pi_{i,j}\\}$, $i=1,...,n$,
$j=1,...,d^{\prime}$, and $\\{\pi^{\prime}_{r,s}\\}$, $r=1,...,n^{\prime}$,
$s=1,...,d$ such that $\pi_{i,j}$ are approximate unitarily equivalent to
$\sigma_{i}$ and the same happens for $\pi^{\prime}_{r,s}$ and
$\sigma^{\prime}_{r}$, that is, there are families $\\{u_{i,j}\\}$ and
$\\{v_{r,s}\\}$ of unitary operators such that

$\|\sigma_{i}(x)-u_{i,j}^{*}\pi_{i,j}(x)u_{i,j}\|<\delta/2,\ \
\|\sigma^{\prime}_{r,s}(x)-v_{r,s}^{*}\pi^{\prime}_{r,s}(x)v_{r,s}\|<\delta/2,\
\ (x\in\mathfrak{F}).$

Then, for $q_{i,j}:=u_{i,j}p_{i}u_{i,j}^{*}$ and
$q^{\prime}_{r,s}:=v_{r,s}p^{\prime}_{r}v_{r,s}^{*}$,

$|Tr(p_{i}\sigma_{i}(x))-Tr(q_{i,j}\pi_{i,j}(x))|<\delta Tr(q_{i,j})=\delta
Tr(p_{i}),$
$|Tr(p^{\prime}_{r}\sigma^{\prime}_{r}(x))-Tr(q^{\prime}_{r,s}\pi^{\prime}_{r,s}(x))|<\delta
Tr(p^{\prime}_{r}),$

and

$\|[q_{i,j},\pi_{i,j}(x)]\|\leq\|[p_{i},\sigma_{i}(x)]\|+2\|\sigma_{i}(x)-u_{i,j}^{*}\pi_{i,j}(x)u_{i,j}\|<\epsilon+\delta,$

for $x\in\mathfrak{F}.$ Similarly,

$\|[q^{\prime}_{r,s},\pi^{\prime}_{r,s}(x)]\|<\epsilon+\delta,\ \
(x\in\mathfrak{F}).$

To finish the proof let us put,

$\displaystyle h(a)$
$\displaystyle:=\frac{\sum_{i,j}{Tr(q_{i,j}\pi_{i,j}(a))}+\sum_{r,s}{Tr(q^{\prime}_{r,s}\pi^{\prime}_{r,s}(a))}}{\sum_{i,j}{Tr(q_{i,j})}+\sum_{r,s}{Tr(q^{\prime}_{r,s})}}$
$\displaystyle=\frac{\sum_{i,j}{Tr(q_{i,j}\pi_{i,j}(a))}+\sum_{r,s}{Tr(q^{\prime}_{r,s}\pi^{\prime}_{r,s}(a))}}{2dd^{\prime}},$

for $a\in A$. Then,

$\displaystyle|h(x)-\frac{1}{2}(f(x)+g(x))|$
$\displaystyle\leq\frac{1}{2dd^{\prime}}\Big{(}\sum_{i,j}{\Big{|}Tr(q_{i,j}\pi_{i,j}(x))-Tr(p_{i}\sigma_{i}(x))\Big{|}}$
$\displaystyle+\sum_{r,s}{\Big{|}Tr(q^{\prime}_{r,s}\pi^{\prime}_{r,s}(x))-Tr(p^{\prime}_{r}\sigma^{\prime}_{r}(x))\Big{|}}\Big{)}$
$\displaystyle\leq\frac{1}{2dd^{\prime}}(\sum_{i,j}{\delta
Tr(p_{i})}+\sum_{r,s}{\delta Tr(p^{\prime}_{r})})<\delta,$

for all $x\in\mathfrak{F},$ as required. ∎

The convexity now follows.

Proof of Corollary B. Given LFD traces $\tau$, $\tau^{\prime}$, $\epsilon>0,$
and a finite set $\mathfrak{F}$ of contractions, $\tau$ and $\tau^{\prime}$
are $(\mathfrak{F},\epsilon/3)$-ATS by Theorem A. Hence,
$\frac{1}{2}(\tau+\tau^{\prime})$ is $(\mathfrak{F},\epsilon)$-ATS, by Lemma
3.3$(ii)$ for $\delta=\epsilon/3$, and so is LFD by Theorem A. Now the set of
LFD traces is closed under taking dyadic convex combinations, and so is convex
by weak∗-closedness.

Next we prove Proposition C. This follows from Gabe’s order-zero
quasidiagonality for separable subalgebras $A$ of $\mathcal{R}$ and
equivalence of inner quasidiagonality for $A$ and its cone.

Proof of Proposition C. $(i)\Rightarrow(ii)$. Given an inclusion $B\subset A$
and $\tau\in UAT(A)\cap AT_{LFD}(A)$, since $\tau|_{B}$ is quasidiagonal and
uniformly amenable, $\tau|_{B}\in AT_{LFD}(B)$. Let $C\subset\mathcal{R}$ be a
separable simple subalgebra, then by Gabe’s order-zero quasidiagonality,
$C_{0}(0,1]\otimes C$ has a faithful, uniformly amenable, quasidiagonal trace.
Indeed, $C$ has a faithful, uniformly amenable tracial state coming from the
normalized tracial state of $\mathcal{R}$, and integrating this against the
Lebesque measure on $(0,1]$ provides a faithful, uniformly amenable tracial
state on cone of $C$. This trace is then LFD by $(i)$, which implies that
$C_{0}(0,1]\otimes C$ is inner quasidiagonal by Corollary B. Since inner
quasidiagonality of $C$ and $C_{0}(0,1]\otimes C$ are equivalent [3, Corollary
3.11], $C$ is quasidiagonal. Now since all separable subalgebras of
$\mathcal{R}$ are quasidiagonal and every separable subalgebra embeds into a
simple subalgebra (by simplicity of $\mathcal{R}$), it follows that
$\mathcal{R}$ is quasidiagonal.

$(ii)\Rightarrow(i)$. Given a separable $C^{*}$-algebra $C$ and $\tau\in
UAT(C)\cap AT_{QD}(C)$, by a result of Brown [5, Proposition 3.2.2.],
$\pi_{\tau}(C)^{\prime\prime}$ is hyperfinite. Since every separable, finite
and hyperfinite von Neumann algebra embeds into $\mathcal{R}$ (see the proof
of [MuRo2020, Theorem A.1]), we may assume without loss of generality that
$\pi_{\tau}(C)^{\prime\prime}\subset\mathcal{R}$. Now let
$\pi_{\tau}(C)\subset D\subset\mathcal{R}$ be a separable, simple, monotracial
$C^{*}$-algebra, then by quasidiagoanlity of $\mathcal{R}$, $D$ is strongly
quasidiagonal with a unique LFD trace [5, 6.1.14]. In particular, $\tau\in
AT_{LFD}(C)$.

The next corollary illustrates that simplicity is rather technical
obstruction. Indeed, the lamplighter group
$\Gamma:=\mathbb{Z}_{2}\wr\mathbb{Z}$ is an amenable, ICC group such that
$C^{*}(\Gamma)$ is RFD hence far from being simple but every trace on
$C^{*}(\Gamma)$ is LFD.

Proof of Proposition D. First note that the reduced $C^{*}$-algebra
$C^{*}(\Gamma)$ has a faithful irreducible representation $\pi$ such that
$\pi(C^{*}(\Gamma))\cap\mathbb{K}(H_{\pi})=0$. Indeed,
$C^{*}_{r}(\Gamma)^{\prime\prime}=\mathcal{R}$ since $\Gamma$ is ICC, and by
amenability of $\Gamma$, $C^{*}(\Gamma)=C^{*}_{r}(\Gamma)$, hence
$C^{*}(\Gamma)$ admits a faithful, $II_{1}$-factor representation. Therefore,
by a result of Glimm [3, Theorem A.2], there are uncountably many non-GCR
irreducible representations which are faithful (note that by separability,
prime ideals and primitive ideals are the same [1, II.6.5.15]). Invoking the
main result of Tikusis-White-Winter [26, Corollary 6.1], we conclude that
every trace on $C^{*}(\Gamma)$ is QD. Combining this with a result of Nate
Brown [5, Proposition 3.3.2], we obtain an increasing sequence $(p_{n})$ of
finite rank projections in $\mathbb{B}(H_{\pi})$ such that for each $a\in
C^{*}(\Gamma)$, $\|[p_{n},\pi(a)]\|\to 0$, and for every trace $\tau\in
T(C^{*}(\Gamma))$, there is a subsequence $p_{n(k)}$ with
$\tau(a)=\lim_{k\to\infty}{\frac{Tr(p_{n(k)}\pi(a))}{Tr(p_{n(k)})}}$.
Therefore, $\tau$ satisfies the conditions of (the proof of) Theorem A, and so
it is LFD. In particular, the (faithful) canonical trace of $C^{*}(\Gamma)$ is
LFD. Hence $C^{*}(\Gamma)$ is inner-QD (by Proposition 2.5) and nuclear, which
is equivalent to being strong-NF.

Remark.$(i)$ The main result of [13] and similar argument as above reveals
that for separable, unital, residually finite dimensional $C^{*}$-algebras
$A_{1}$ and $A_{2}$ with $(dim(A_{1})-1)(dim(A_{2})-1)\geq 2$, the full free
product, $A=A_{1}*A_{2}$ satisfies $AT_{QD}(A)=AT_{LFD}(A)$ and every LFD
trace has the form described in Theorem A.
$(ii)$ The simplicity assumption on $A$ is used in the proof of Theorem A to
make sure that $A$ has no GCR irreducible representation. In particular, this
result is also valid in the non simple case, as long as this extra condition
holds. The same holds for Corollary B and Proposition C. A practical instance
is a non simple $\mathcal{Z}$-stable $C^{*}$-algebra, where $\mathcal{Z}$ is
the Jiang-Su algebra (as easily seen by the Kirchberg slice lemma [23, Lemma
4.1.9]).

##  4\. Acknowledgment

We would like to thank Professor Wilhelm Winter for pointing out a gap in an
earlier version of this paper. We also thank the anonymous referee for careful
reading and suggestions to improve the paper.

## References

  * [1] B. Blackadar. Operator algebras. Theory of $C^{*}$-algebras and von Neumann algebras. Berlin: Springer, 2006.
  * [2] Bruce Blackadar and Eberhard Kirchberg, Generalized inductive limits of finite-dimensional $C^{*}$-algebras., Math. Ann. 307 (1997), Number 3, 343–380.
  * [3] Bruce Blackadar and Eberhard Kirchberg, Inner quasidiagonality and strong NF algebras., Pac. J. Math. 198 (2001), Number 2, 307–329.
  * [4] Bruce Blackadar and Eberhard Kirchberg, Irreducible representations of inner quasidiagonal $C^{*}$-algebras., Can. Math. Bull. 54 (2011), Number 3, 385–395.
  * [5] Nathanial P. Brown, Invariant means and finite representation theory of $C^{*}$-algebras., Memoirs of the American Mathematical Society 865, 2006, Providence, RI: American Mathematical Society
  * [6] Nathanial P. Brown and José R. Carrión and Stuart White, Decomposable approximations revisited., Operator algebras and applications. The Abel symposium, August 7–11, 2015, 2016, Cham: Springer.
  * [7] Nathanial P. Brown and Narutaka Ozawa, $C^{*}$-algebras and finite-dimensional approximations., Graduate Studies in Mathematics 88, 2008, Providence, RI: American Mathematical Society.
  * [8] José R. Carrión and Marius Dadarlat and Caleb Eckhardt, On groups with quasidiagonal $C^{\ast}$-algebras., J. Funct. Anal. 265 (2013), Number 1, 135–152.
  * [9] Man-Duen Choi, The full C*-algebra of the free group on two generators, Pac. J. Math. 87 (1980), 41–48.
  * [10] Alain Connes, Classification of injective factors. Cases $\mathrm{II}_{1}$, $\mathrm{II}_{\infty}$, $\mathrm{III}_{\lambda}$, $\lambda\neq 1$., Ann. Math. (2) 104 (1976), 73–115.
  * [11] Alain Connes, Compact metric spaces, Fredholm modules, and hyperfiniteness, Ergodic Theory Dyn. Syst. 9 (1989), Number 2, 207–220.
  * [12] Kenneth R. Davidson, $C^{*}$-algebras by example, Fields Institute Monographs 6, 1996, Providence, RI: American Mathematical Society.
  * [13] Ken Dykema, and Francisco Torres-Ayala, Primitivity of unital full free products of residually finite dimensional $C^{\ast}$-algebras, J. Funct. Anal. 267 (2014), Number 11, 4519–4558.
  * [14] James Gabe, Quasidiagonal traces on exact $C^{\ast}$-algebras., J. Funct. Anal. 272 (2017), Number 3, 1104–1120.
  * [15] Richard V. Kadison and John R. Ringrose, Fundamentals of the theory of operator algebras. Vol. I: Elementary theory. 2nd printing with correct, Graduate Studies in Mathematics 15, 1997, Providence, RI: American Mathematical Society.
  * [16] Eberhard Kirchberg, Discrete groups with Kazhdan’s property $T$ and factorization property are residually finite, Math. Ann. 299 (1994), Number = 3, 551–563.
  * [17] Qihui Li and Ze Li, A note on inner quasidiagonal C*-algebras, Proc. Am. Math. Soc. 144 (2016), 4861-4872.
  * [18] G. J. Murphy. $C^{*}$-algebras and operator theory. Boston: Academic Press, 1990.
  * [19] Magdalena Musat and Mikael Rørdam, Non-closure of quantum correlation matrices and factorizable channels that require infinite dimensional ancilla (With an appendix by Narutaka Ozawa), Commun. Math. Phys. 375 (2020), Number 3, 1761–1776.
  * [20] Narutaka Ozawa, Homotopy invariance of AF-embeddability, Geom. Funct. Anal. 13 (2003), Number 1, 216–222.
  * [21] Narutaka Ozawa, Dixmier approximation and symmetric amenability for C*-algebras, J. Math. Sci., Tokyo 20 (2013), Number 3, 349–374.
  * [22] Sorin Popa, On local finite dimensional approximation of $C^{*}$-algebras, Pac. J. Math. 181 (1997), Number 1, 141–158.
  * [23] Mikael Rørdam and Erling Størmer, Classification of nuclear $C^{*}$-algebras. Entropy in operator algebras., 2002, Berlin: Springer.
  * [24] N. Salinas. Homotopy invariance of $Ext(A)$, Duke Math. J., 44:777-794, 1977.
  * [25] Christopher Schafhauser, Subalgebras of simple AF-algebras, Ann. Math. (2) 192 (2020), 309-352.
  * [26] Aaron Tikuisis and Stuart White and Wilhelm Winter, Quasidiagonality of nuclear $\mathrm{C}^{\ast}$-algebras., Ann. Math. (2) 185, Number 1, 229–284.
  * [27] F. J. Thayer. Quasi-diagonal $C^{*}$-algebras, J. Funct. Anal., 25:50-57, 1977.
  * [28] Dan Voiculescu, A note on quasi-diagonal $C^{*}$-algebras and homotopy, Duke Math. J. 62 (1991), Number 2, 267–271.
  * [29] Dan Voiculescu, Around quasidiagonal operators, Integral Equations Oper. Theory 17 (1993), Number 1, 137–149.
  * [30] Wilhelm Winter, QDQ vs. UCT., Operator algebras and applications. The Abel symposium, August 7–11, 2015, 2016, Cham: Springer.