ENEM-50k and HSLS, but its performance falters in COMPAS and Adult. EqOdds
produces a point with very low MEO but with a substantial loss in accuracy.
LevEqOpp also yields a point with low MEO but with a much smaller accuracy
drop. Even though LevEqOpp only optimizes for FNR difference between two
groups, it performs surprisingly well in terms of MEO in all four datasets.
However, we note that LevEqOpp can only produce a point, not a curve, and it
does not enjoy the generality of Fair Projection as it is specifically
designed for binary-class, binary-group predictions and minimizing Equalized
Opportunity difference.

####  B.3.2 Accuracy-fairness trade-off in multi-class/multi-group
classification

In the main text, we showed the performance of FairProjection-CE on multi-
class prediction with 5 classes and 2 groups (see Figure 2). We include
results under a few different multi-class settings here. First, we show
results on ENEM-50k-5-5 which has 5 classes and 5 groups in Figure 11 and 12 .
We obtain 5 groups by not binarizing the race feature. Then, we show results
on binary classification with 5 groups in Figure 13 and 14. Finally, we
include the extended version of Figure 2 that include both FairProjection-CE
and FairProjection-KL in Figure 15.

To measure multi-class performance, we extend the definition of mean equalized
odds (MEO) and statistical parity as follows:

MEO
$\displaystyle=\max_{i\in\mathcal{Y}}\max_{s_{1},s_{2}\in\mathcal{S}}(|\textsf{TPR}_{i}(s_{1})-\textsf{TPR}_{i}(s_{2})|+|\textsf{FPR}_{i}(s_{1})-\textsf{FPR}_{i}(s_{2})|)/2$
(117) Statistical Parity
$\displaystyle=\max_{i\in\mathcal{Y}}\max_{s_{1},s_{2}\in\mathcal{S}}|\textsf{Rate}_{i}(s_{1})-\textsf{Rate}_{i}(s_{2})|$
(118)

where we denote $\textsf{TPR}_{i}(s)=P(\widehat{Y}=i\mid Y=i,S=s)$,
$\textsf{FPR}_{i}(s)=P(\widehat{Y}=i\mid Y\neq i,S=s)$, and
$\textsf{Rate}_{i}(s)=P(\widehat{Y}=i\mid S=s)$.

In all experiments, FairProjection reduces MEO and statistical parity
significantly (e.g., 0.22 to 0.14) with a negligible sacrifice in accuracy.

Figure 3: Accuracy-fairness curves of FairProjection and benchmark methods on
the HSLS dataset with 3 different models (Logistic regression, Random forest,
GBM). The fairness constraint is MEO.

Figure 4: Accuracy-fairness curves of FairProjection and benchmark methods on
the HSLS dataset with 3 different models (Logistic regression, Random forest,
GBM). The fairness constraint is statistical parity.

Figure 5:  Accuracy-fairness curves of FairProjection and benchmark methods on
the ENEM-50k-2C dataset with 3 different models (Logistic regression, Random
forest, GBM). The fairness constraint is MEO.

Figure 6: Accuracy-fairness curves of FairProjection and benchmark methods on
the ENEM-50k-2C dataset with 3 different models (Logistic regression, Random
forest, GBM). The fairness constraint is statistical parity.

Figure 7: Accuracy-fairness curves of FairProjection and benchmark methods on
COMPAS with 3 different models (Logistic regression, Random forest, GBM). The
fairness constraint is MEO.

Figure 8: Accuracy-fairness curves of FairProjection and benchmark methods on
COMPAS with 3 different models (Logistic regression, Random forest, GBM). The
fairness constraint is statistical parity.

Figure 9:  Accuracy-fairness curves of FairProjection and benchmark methods on
the Adult dataset with 3 different models (Logistic regression, Random forest,
GBM). The fairness constraint is MEO.

Figure 10: Accuracy-fairness curves of FairProjection and benchmark methods on
the Adult dataset with 3 different models (Logistic regression, Random forest,
GBM). The fairness constraint is statistical parity.

Figure 11: Accuracy-fairness curves of FairProjection-CE and FairProjection-KL
on ENEM-50k with with 5 labels, 5 groups and different base classifiers base
classifiers. The fairness constraint is MEO.

Figure 12: Accuracy-fairness curves of FairProjection-CE and FairProjection-KL
on ENEM-50k with with 5 labels, 5 groups and different base classifiers base
classifiers. The fairness constraint is SP.

Figure 13: Accuracy-fairness curves of FairProjection-CE and FairProjection-KL
on ENEM-50k with with 2 labels, 5 groups and different base classifiers base
classifiers. The fairness constraint is MEO.

Figure 14: Accuracy-fairness curves of FairProjection-CE and FairProjection-KL
on ENEM-50k with with 2 labels, 5 groups and different base classifiers base
classifiers. The fairness constraint is SP.

Figure 15: Comparison of FairProjection-CE and FairProjection-KL with
Adversarial on ENEM-50k-5-2, meaning 5 labels, 2 groups. The reason for the
difference comparing to Fig. 2 is that we resampled 50k data points from ENEM.

###  B.4 More on related work

Method | Feature |
---|---|---
Multiclass | Multigroup | Scores | Curve | Parallel | Rate | Metric
Reductions [5] | ✕ | ✓ | ✓ | ✓ | ✕ | ✓ | SP, (M)EO
Reject-option [44] | ✕ | ✓ | ✕ | ✓ | ✕ | ✕ | SP, (M)EO
EqOdds [35] | ✕ | ✓ | ✕ | ✕ | ✕ | ✓ | EO
LevEqOpp [13] | ✕ | ✕ | ✕ | ✕ | ✕ | ✕ | FNR
CalEqOdds [50] | ✕ | ✕ | ✓ | ✕ | ✕ | ✓ | MEO
FACT [42] | ✕ | ✕ | ✕ | ✓ | ✕ | ✕ | SP, (M)EO
Identifying [39] | $\text{{\color[rgb]{0,0.88,0}{\char 33\relax}}}^{\text{{\color[rgb]{0.72,0,0}{\char 35\relax}}}}$ | ✓ | ✓ | ✓ | ✕ | ✕ | SP, (M)EO
FST [53, 54] | ✕ | ✓ | ✓ | ✓ | ✕ | ✓ | SP, (M)EO
Overlapping [55] | ✓ | ✓ | ✓ | ✓ | ✕ | ✕ | SP, (M)EO
Adversarial [57] | ✓ | ✓ | N/A | ✓ | ✓ | ✕ | SP, (M)EO
FairProjection (ours) | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | SP, (M)EO

Copy of Table 1. Comparison between benchmark methods. Multiclass/multigroup:
implementation takes datasets with multiclass/multigroup labels; Scores:
processes raw outputs of probabilistic classifiers; Curve: outputs fairness-
accuracy tradeoff curves (instead of a single point); Parallel: parallel
implementation (e.g., on GPU) is available; Rate: convergence rate or sample
complexity guarantee is proved. Metric: applicable fairness metric, with
SP$\leftrightarrow$Statistical Parity, EO$\leftrightarrow$Equalized Odds,
MEO$\leftrightarrow$Mean EO.

Our method is a model-agnostic post-processing method, so we focus our
comparison on such post-processing fairness intervention methods. In the above
table, the only exception is Adversarial [57], which we use to benchmark
multi-class prediction. Adversarial [57] is an in-processing method based on
generative-adversarial network (GAN) where the adversary tries to guess the
sensitive group attribute $S$ from $Y$ and $\widehat{Y}$. Even though this
GAN-based approach is applicable to multi-class, multi-group prediction, it
cannot be universally applied to any pre-trained classifier like our method.

EqOdds [35], CalEqOdds [50] and LevEqOpp [13] are post-processing methods
designed for binary prediction with binary groups. They find different
decision thresholds for each group that equalize FNR and FPR of two groups.
CalEqOdds [50] has an additional constraint that the post-processed classifier
must be well-calibrated, and we observe in our experiments that this stringent
constraint leads to a low-accuracy classifier especially when there is a big
gap in the base rate between the two groups. FACT [42] follows a similar
approach but generalizes this to an optimization framework that can have both
equalized odds and statistical parity constraints and flexible accuracy-
fairness trade-off. The optimization formulation finds a desired confusion
matrix, and their proposed post-processing method flips the predictions to
match the desired confusion matrix. Reject-option [44] is similar in that it
flips predictions near the decision threshold. In [44], instead of finding the
optimal confusion matrix, it performs grid search to find the optimal margin
around the decision threshold that can minimize either equalized odds or
statistical parity. For these methods that center around modifying decision
thresholds, it is not straightforward to extend to multi-class and multi-group
as one will have to consider
$\binom{|\mathcal{Y}|}{2}\cdot\binom{|\mathcal{S}|}{2}$ boundaries.

FST [53, 54] tackles fairness intervention via minimizing cross-entropy for
binary classes. Their method is inherently tailored to binary classification
_and_ only a cross-entropy objective function, and our FairProjection-CE
reduces to FST for the case of CE and binary classification tasks. Identifying
[39] is a method for minimizing KL-divergence for group-fairness intervention,
which changes the label weights (via a convex combination) between unweighted
and weighted samples, but it is not clear that this would navigate a good
fairness-accuracy trade-off curve. Their method can be extended to non-binary
prediction with non-binary groups by an appropriate choice of base classifier
and fairness constraints, which is a non-trivial extension of the accompanying
code, and we chose not to pursue this. Note that [39] and FairProjection solve
the KL-divergence minimization in very different ways. In particular, the
runtime of [53, 54] on 350k train is longer than 30 minutes using logistic
regression as a base classifier (in comparison, the runtime of FairProjection
for a 500k dataset is less than 1 minute). This is because they require
reweighing the data and retraining a large number of times. Hence, it is
inherently non-parallelizable.

####  B.4.1 Fairness in Multi-Class Prediction

Methods that are based on optimization with a fairness regularizer often can
be easily extended to multi-class prediction as it only requires a small
change in the regularizer. For example, instead of using
$|\text{FNR}_{0}(x)-\text{FNR}_{1}(x)|$, one can replace this with

$\sum_{i\in\mathcal{Y}}\sum_{j\neq i\in\mathcal{Y}}|P(\widehat{Y}=j\mid
Y=i,S=0)-P(\widehat{Y}=j\mid Y=i,S=1)|.$ (119)

FERM [27] mentions how their method can be extended to multi-class sensitive
attribute. Similarly, we believe that their method can be used for multi-class
labels as well. The reductions approach [5] assumes binary labels but is has
natural extension to multi-class, which is explored in [55]. In-processing
methods proposed in [17] and [57] allow for both multi-class labels and multi-
class group attributes. [57] aims to achieve the independence between the
sensitive attribute $S$ and $\widehat{Y}$ or $\widehat{Y}$ given $Y$ by
training an adversary who tries to figure out $\widehat{S}$. [17] directly
estimates the fairness loss (e.g., 119) using kernel density estimation. They
also demonstrate the empirical performance in a three-class classification
using synthetic data. Another in-processing method is [2] where the authors
propose a way to incorporate multi-class fairness constraints into decision
tree training. The preprocessing method suggested in [19] is conceptually
similar to our methods in that it aims to minimize the KL-divergence between
the original distribution and preprcoessed distribution while satisfying
fairness constraints. Their method, however, requires all feature vectors to
be binary, and applies only to demographic parity or representation rate.
There exist other notions of fairness, which is different from commonly-used
group fairness metrics such as envy-freeness [9] or best-effort [43], which
can be applied to multi-class prediction tasks.

Finally, there are unpublished works [24, 56] that could handle multi-class
classification. Specifically, [24] presents a post-processing method that
selects different thresholds for each group to achieve demographic parity.
[56] formulates SVM training as a mixed-integer program and integrates
fairness regularizer in the objective, which can also deal with multi-class.

##  Appendix C Datasheet for ENEM 2020 dataset

#### Questions

The questions below are derived from [33] and aim to provide context about the
ENEM-2020 dataset. We highlight that we did not create the dataset nor collect
the data included in it. Instead, we simply provide a link to the ENEM-2020
data at [37]. At the time of writing, the ENEM-2020 dataset is open and made
freely available by the Brazilian Government at [37] under a Creative Commons
Attribution-NoDerivs 3.0 Unported License [20]. We provide the datasheet below
to clarify certain aspects of the dataset (e.g., motivation, composition,
etc.) since the original information is available in Portuguese at [37], thus
limiting its access to a broader audience. The website [37] contains a link to
download a .zip file which contains the ENEM-2020 data in .csv format and
extensive accompanying documentation.

The datasheet below is not a substitute for the explanatory files that are
downloaded together with the dataset at [37], and we emphatically recommend
the user to familiarize themselves with associated documentation prior to
usage. We also strongly recommend the user to carefully read the “Leia-Me”
(readme) file Leia_Me_Enem_2020.pdf available in the same .zip folder that
contains the dataset. The answers in the datasheet below are based on an
English translation of information available at [37] and may be incomplete or
inaccurate. The datasheet below is based on our own independent analysis and
in no way represents or attempts to represent the opinion or official position
of the Brazilian Government and its agencies.

We also note that we do not distribute the ENEM-2020 dataset directly nor host
the dataset ourselves. Instead, we provide a link to download the data from a
public website hosted by the Brazilian Government. The dataset may become
unavailable in case the link in [37] becomes inaccessible.

#### Motivation

  * •

For what purpose was the dataset created? According to the “Leia-me” (Read Me)
file that accompanies the data, the dataset was made available to fufill the
mission of the Instituto Nacional de Estudos e Pesquisas Educacionais Anísio
Teixeira (INEP) of developing and disseminating data about exams and
evaluations of basic education in Brazil.

  * •

Who created the dataset (e.g., which team, research group) and on behalf of
which entity (e.g., company, institution, organization)? The dataset was
developed by INEP, which is a government agency connected to the Brazilian
Ministry of Education.

  * •

Who funded the creation of the dataset? The data is made freely available by
the Brazilian Government.

#### Composition

  * •

What do the instances that comprise the dataset represent (e.g., documents,
photos, people, countries)? The instances of the dataset are information about
individual students who took the Exame Nacional do Ensino Médio (ENEM). The
ENEM is the capstone exam for Brazilian students who are graduating or have
graduated high school.

  * •

How many instances are there in total (of each type, if appropriate)? The raw
data provided in at [37] has approximately 5.78 million entries. The processed
version we use in our experiments has approximately 1.4 million entries.

  * •

Does the dataset contain all possible instances or is it a sample (not
necessarily random) of instances from a larger set? The data provided is the
lowest level of aggregation of data collected from ENEM exam-takers made
available by INEP.

  * •

What data does each instance consist of? We provide a brief description of the
features available in the raw public data provided at [37]. Upon downloading
the data, a detailed description of features and their values are available
(in Portuguese) in the file titled Dicionário_Mircrodados_ENEM_2020.xsls. The
features include:

    * –

Information about exam taker: exam registration number (masked), year the exam
was taken (2020), age range, sex, marriage status, race, nationality, status
of high school graduation, year of high school graduation, type of high school
(public, private, n/a), if they are a “treineiro” (i.e., taking the exam as
practice).

    * –

School data: city and state of participant’s school, school administration
type (private, city, state, or federal), location (urban or rural), and school
operation status.

    * –

Location where exam was taken: city and state.

    * –

Data on multiple-choice questions: The exam is divided in 4 parts (translated
from Portuguese): natural sciences, human sciences, languages and codes, and
mathematics. For each part there is data if the participant attended the
corresponding portion of the exam, the type of exam book they received, their
overall grade, answers to exam questions, and the answer sheet for the exam.

    * –

Data on essay question: if participant took the exam, grade on different
evaluation criteria, and overall grade.

    * –

Data on socio-economic questionnaire answers: the data include answers to 25
socio-economic questions (e.g., number of people who live in your house,
family average income, if the your house has a bathroom, etc.).

  * •

Is there a label or target associated with each instance? No, there is no
explicit label. In our fairness benchmarks, we use grades in various
components of the exam as a predicted label.

  * •

Is any information missing from individual instances? Yes, certain instances
have missing values.

  * •

Are relationships between individual instances made explicit (e.g., users’
movie ratings, social network links)? No explicit relationships identified.

  * •

Are there recommended data splits (e.g., training, development/validation,
testing)? No.

  * •

Are there any errors, sources of noise, or redundancies in the dataset? The
data contains missing values and, according to INEP, was collected from
individual exam takers. The information is self-reported and collected at the
time of the exam.

  * •

Is the dataset self-contained, or does it link to or otherwise rely on
external resources (e.g., websites, tweets, other datasets)? Self-contained.

  * •

Does the dataset contain data that might be considered confidential (e.g.,
data that is protected by legal privilege or by doctor–patient
confidentiality, data that includes the content of individuals’ non-public
communications)? According to the _Leia-me_ (readme) file (in Portuguese) that
accompanies the dataset and our own inspection, the dataset does not contain
any feature that allows direct identification of exam takers such as name,
email, ID number, birth date, address, etc. The exam registration number has
been substituted by a sequentially generated mask. INEP states that the
released data is aligned with the Brazilian _Lei Geral de Proteção dos Dados_
(LGPD, General Law for Data Protection). We emphatically recommend the user to
view the Readme file prior to usage.

  * •

Does the dataset contain data that, if viewed directly, might be offensive,
insulting, threatening, or might otherwise cause anxiety? The official
terminology used by the Brazilian Government to denote race can be viewed as
offensive. Specifically, the term used to describe the race of exam takers of
Asian heritage is “Amarela,” which is the Portuguese word for the color
yellow. Moreover, the term “Pardo,” which roughly translates to brown, is used
to denote individuals of multiple or mixed ethnicity. This outdated and
inappropriate terminology is still in official use by the Brazilian
Government, including in its population census. The dataset itself includes
integers to denote race, which are mapped to specific categories through the
variable dictionary.

  * •

Does the dataset relate to people? Yes.

  * •

Does the dataset identify any subpopulations (e.g., by age, gender)? Yes.
Information about age, sex, and race are included in the dataset.

  * •

Is it possible to identify individuals (i.e., one or more natural persons),
either directly or indirectly (i.e., in combination with other data) from the
dataset? The _Leia-me_ (readme) file notes that the individual exam-takers
cannot be directly identified from the data. However, in the same file, INEP
recognizes that the Brazilian data protection law (LGPD) does not clearly
define what constitutes a reasonable effort of de-identification. Thus, INEP
adopted a cautious approach: this dataset is a simplified/abbreviated version
of the ENEM micro-data compared to prior releases and aims to remove any
features that may allow identification of the exam-taker.

  * •

Does the dataset contain data that might be considered sensitive in any way
(e.g., data that reveals racial or ethnic origins, sexual orientations,
religious beliefs, political opinions or union memberships, or locations;
financial or health data; biometric or genetic data; forms of government
identification, such as social security numbers; criminal history)? The data
includes race information and socio-economic questionnaire answers.

#### Collection Process

Since we did not produce the data, we cannot speak directly about the
collection process. Our understanding is that the data contains self-reported
answers from exam-takers of the ENEM collected at the time of the exam. The
exam was applied on 17 and 24 of January 2021 (delayed due to COVID). The data
was aggregated and made publicly available by INEP at [37]. After consulting
the IRB office at our institution, no specific IRB was required to use this
data since it is anonymized and publicly available.

#### Preprocessing/cleaning/labeling

  * •

Was any preprocessing/cleaning/labeling of the data done (e.g., discretization
or bucketing, tokenization, part-of-speech tagging, SIFT feature extraction,
removal of instances, processing of missing values)? Some mild pre-processing
was done on the data to ensure anonymity, as indicated in the “Leia-me” file.
This includes aggregating participant ages, masking exam registration numbers,
and removing additional information that could allow de-anonymization.

  * •

Was the “raw” data saved in addition to the preprocessed/cleaned/labeled data
(e.g., to support unanticipated future uses)? The raw data is not publicly
available.

### Uses

  * •

Has the dataset been used for any tasks already? We have used this dataset to
benchmark fairness interventions in ML in the present paper. ENEM microdata
has also been widely used in studies ranging from public policy in Brazil to
item response theory in high school exams.

  * •

Are there tasks for which the dataset should not be used? INEP does not
clearly define tasks that should not be used on this dataset. However, no
attempt should be made to de-anonymize the data.

#### Distribution and Maintenance

The ENEM-2020 dataset is open and made freely available by the Brazilian
Government at [37] under a Creative Commons Attribution-NoDerivs 3.0 Unported
License [20] at the time of writing. The dataset may become unavailable in
case the link in [37] becomes inaccessible.


# Learning Purified Feature Representations
from Task-irrelevant Labels

Yinghui Li1,∗, Chen Wang2,∗, Yangning Li1,∗, Hai-Tao Zheng1,4,†, Ying Shen3,†
∗ indicates equal contribution.† Corresponding author: Hai-Tao Zheng, Ying
Shen. (E-mail<EMAIL_ADDRESS><EMAIL_ADDRESS>1
Tsinghua Shenzhen International Graduate School, Tsinghua University
2Department of Computer Science and Technology, Tsinghua University
3School of Intelligent Systems Engineering, Sun-Yat Sen University
4Peng Cheng Laboratory
{liyinghu20, wchen20<EMAIL_ADDRESS>
###### Abstract

Learning an empirically effective model with generalization using limited data
is a challenging task for deep neural networks. In this paper, we propose a
novel learning framework called _Purified Learning_ to exploit task-irrelevant
features extracted from task-irrelevant labels when training models on small-
scale datasets. Particularly, we purify feature representations by using the
expression of task-irrelevant information, thus facilitating the learning
process of classification. Our work is built on solid theoretical analysis and
extensive experiments, which demonstrate the effectiveness of Purified
Learning. According to the theory we proved, Purified Learning is model-
agnostic and doesn’t have any restrictions on the model needed, so it can be
combined with any existing deep neural networks with ease to achieve better
performance.

###### Index Terms:

Deep Learning Theory, Model Generalization, Computer Vision

##  I Introduction

With sufficient manually annotated training samples, deep neural networks
could automatically perform feature extraction and achieve unprecedented
performances on various classification tasks [9]. However, collecting and
annotating adequate training data is an extremely time-consuming and expensive
process, leading that in many instances the training samples are insufficient
and even noisy [39]. Under this circumstance, the performance of deep models
always drops gravely on most classification tasks [2, 22]. The essential
reason for the phenomenon is that the goal of “minimizing the empirical risk”
is not reliable when the training data is inadequate [34], as a result, deep
neural models will easily overfit the training data. Therefore, training
effective deep neural models with remarkable generalization performance on
small training samples is of great practical importance in terms of vastly
expanding the scalability of deep learning methods.

Many techniques have been developed to tackle the issue. Some approaches
directly expanding training samples [7, 4, 6, 40, 26] to alleviate the
shortage of annotated data, but may be limited because expansion based on
small-scale training samples is only of theoretical significance and not
operability. On the other hand, previous studies also attempt to break through
the limitation of a specific dataset and find training samples in a broad
sense. Based on this motivation, transfer learning [35, 17] exploits knowledge
from additional datasets with relevant content and labels, achieving
remarkable improvements on the target task. However, they only achieve good
results when labels are relevant enough since irrelevant labels could bring
out massive negative transfer [36, 23]. And to date, there have been few
studies that use task-irrelevant features to improve the performance of deep
learning models. In this paper, we argue that while transferring task-relevant
labels from other datasets, task-irrelevant labels could also be utilized to
improve the generalization of classification on small datasets without
conflicts.

Consider a running example in computer vision: facial expression recognition
a.k.a FER, which aims at constructing a model to accurately predict the
expression of unseen facial pictures. In this task, it is obvious that
_smiling_ is a task-relevant feature, while _hair color_ is a proper task-
irrelevant one. Assume we have a small-scale FER dataset, which is often the
case in real-world deep learning tasks. Since the training dataset is
extremely small, it is unavoidable that the distribution of task-irrelevant
features i.e., _hair color_ have non-negligible bias. For example, most people
with black hair might be labeled as _happy_ and people with white hair are
exactly labeled as _sad_. This kind of bias is a ubiquitous problem when
training samples are insufficient, which has already been observed in many
previous works [31, 32].

As we mentioned before, the task-irrelevant features create a substantial
barrier for the learning process due to the problem of “minimizing the
empirical risk”, which leads to the models inevitably overfit hair features.
As a result, the performance of the trained models on unseen facial expression
samples are severely affected. An intuitive method to address the issue is to
make use of massive related facial information from other facial datasets by
transfer learning, but as we mentioned, there would be severe negative
transfer when using irrelevant facial information (_hair color_). Our
methodology is motivated by the predicament, which is to utilize vast samples
that has task-irrelevant features to tackle this problem.

In this paper, we propose Purified Learning to explore task-irrelevant
features from large-scale and easily available datasets with the same content
as the training set. By minimizing the Wasserstein distance between the
distribution of representation extracted from samples with a fixed task-
irrelevant label and that from the entire dataset, we reduce the influence of
task-irrelevant features. This suppression of task-irrelevant features plays a
“two negatives make a positive” role that further highlights the
representation ability of task-relevant features and finally improve model
generalization. In summary, in this paper we make three-fold contributions:

  * •

We theoretically prove that task-irrelevant labels can help to extract helpful
features while only small-scale training dataset is available.

  * •

We propose Purified Learning, which use task-irrelevant labels to facilitate
the learning process, finally derive a purified feature representation that
has minimum task-irrelevant components.

  * •

We conduct extensive experiments and analyses on FER and digit recognition.
Results show that we achieve state-of-the-art performance on digit recognition
task when the training set is small.

##  II Theory

###  II-A Unreliability of Minimizing Empirical Risk

For a learning task $\mathcal{T}$, a training sample set including $I$ labeled
instances $D_{train}=\\{(x_{1},y_{1}),(x_{2},y_{2}),...(x_{I},y_{I})\\}$ is
given. Let $p(x,y)$ be the ground-truth joint probability distribution of
sample $x$ and label $y$. We denote the space of input instance by
$\mathcal{X}$, and the space of task labels by $\mathcal{Y}$,
$x\in\mathcal{X}$ and $y\in\mathcal{Y}$. The goal of deep learning is to learn
a model $f(x;\theta):\\{\mathcal{X};\Theta\\}\rightarrow\mathcal{Y}$
parameterized by $\theta\in\Theta$ from training data to minimize the expected
risk, i.e.,

$\min_{f}R(f)=\int\mathcal{L}(f(x;\theta),y)dp(x,y),$ (1)

where $\mathcal{L}$ refers to a certain loss function, e.g., mean squared
error or cross entropy loss.

As $p(x,y)$ is unknown, the most classical approach is to approximate the
expected risk by minimizing the empirical risk [33, 19], which is the average
of sample losses over the training set $D_{train}$ of $I$ samples:

$\displaystyle\min_{f}R_{emp}(f)=\frac{1}{I}\sum_{i=1}^{I}\mathcal{L}\left(y_{i},f\left(x_{i}\right)\right),\text{
s.t. }(x_{i},y_{i})\in D_{train}.$ (2)

However, obviously that when the distribution of the training sample differs
greatly from the true distribution $p(x,y)$, especially when $I$ is small, the
empirical risk $R_{emp}(f)$ may then be far from being a good approximation of
the expected risk $R(f)$, which makes it no longer reliable [34]. The bias of
task-irrelevant features in the training dataset is one of the main causes of
the deviation between training data distribution and $p(x,y)$. We believe that
weakening the impact of task-irrelevant features will help train models with
better generalization, which requires introducing knowledge of task-irrelevant
features from samples with task-irrelevant labels.

###  II-B Task-Irrelevant Labels

Task-irrelevant features are defined as the features irrelevant to the target
task, and task-relevant features are features relevant to the target task. For
example, hair color is a task-irrelevant feature for the FER task.
Correspondingly, if samples from some irrelevant tasks contain the same
content but different labels with the target task, the labels are termed as
task-irrelevant labels, and the samples are termed as task-irrelevant samples.
Note that in practice, the task irrelevance of a label is related to the
performance loss of transfer learning, not by human judgments. The more
negative the transferring effect is, the more irrelevant the label is.

For a specific target task, and a dataset with task-irrelevant labels,
$S=\\{(x,y_{tir})\\}$, we denote the feature space by $X$, $x\in X$, and the
label space of task-irrelevant labels by $Y$,
$Y=\\{y_{1},y_{2},y_{3}...y_{n}\\}$, $y_{tir}\in Y$. Input $x$ contains some
features that are task-relevant, denoted by $x_{tr}$ and the distribution of
$x_{tr}$ in $S$ is denoted by $\mathbb{S}$.

We select all samples $S_{y_{1}}$ whose $y_{tir}=y_{1}$ from $S$:

$S_{y_{1}}=\\{(x,y_{tir})|y_{tir}=y_{1},(x,y_{tir})\in S\\},$ (3)

and the distribution of $x_{tr}$ in $S_{y_{1}}$ is denoted by
$\mathbb{S}_{y_{1}}$. Since $y_{tir}$ is irrelevant with $x_{tr}$, we have:

$\mathbb{S}=\mathbb{S}_{y_{1}}.$ (4)

In deep models, the feature extractor $f_{e}$ is used to extract
representation $r$ from input data. Here, $R$ and $R_{y_{1}}$ are extracted
from $S$ and $S_{y_{1}}$,

$\begin{split}R&=\\{(r,y_{tir})|r=f_{e}(x),(x,y_{tir})\in S\\}\\\
R_{y_{1}}&=\\{(r,y_{tir})|r=f_{e}(x),(x,y_{tir})\in S_{y_{1}}\\}.\end{split}$
(5)

Similarly, the distributions of $r$ in $R$ and $R_{y_{1}}$ are denoted by
$\mathbb{R}$ and $\mathbb{R}_{y_{1}}$.

Under ideal conditions, a good extractor only extracts task-relevant features,
so $\mathbb{R}=\mathbb{R}_{y_{1}}$. However, in reality, the extractor
inevitably extracts task-irrelevant features and leads to
$\mathbb{R}\neq\mathbb{R}_{y_{1}}$. The greater the difference between
$\mathbb{R}$ and $\mathbb{R}_{y_{1}}$, the more the extractor is influenced by
task-irrelevant features. To reduce this difference, Purified Learning aims to
minimize the divergence between these two distributions,

$\min(\operatorname{WD}(\mathbb{R},\mathbb{R}_{y_{1}})),$ (6)

$\operatorname{WD}$ is Wasserstein distance that is used to measure the
divergence between two distributions.

###  II-C Theoretical Analysis of Purified Learning

Motivated by [25], we provide a theoretical analysis of Wasserstein distance’s
efficacy and its generalization bound here. The classification model is
divided into two parts, a classifier $f_{c}$, and a feature extractor $f_{e}$.
After sending an input instance to the extractor $f_{e}$, we get a
representation $r$, so $f_{e}:\mathcal{X}\rightarrow\mathcal{R}$ where
$\mathcal{R}$ denotes and $f_{c}:\mathcal{R}\rightarrow\mathcal{Y}$ where
$\mathcal{Y}$ is the space of labels as defined before.

$H$ is a hypothesis class that for every $h\in H$,
$h:\mathcal{R}\rightarrow\mathcal{Y}$ and $h$ is K-Lipschitz continuous. In
neural networks, we limit the scale of the weights so that $f_{c}$ is
$K$-Lipschitz continuous and $f_{c}\in H$. For every distribution $\mathbb{D}$
on $\mathcal{X}$, the corresponding distribution of representation is denoted
by $\mathbb{R}$, $\mathbb{R}=f_{e}(\mathbb{D})$. The difference of $h_{1}$ and
$h_{2}$ ($h_{1},h_{2}\in H$) on $\mathbb{R}$ are defined by:

$\epsilon_{\mathbb{R}}(h_{1},h_{2})=\underset{r\sim\mathbb{R}}{\mathbb{E}}[||h_{1}(r)-h_{2}(r)||].$
(7)

Similarly,

$\epsilon_{\mathbb{D}}(h_{1}\circ f_{e},h_{2}\circ
f_{e})=\underset{x\sim\mathbb{D}}{\mathbb{E}}||h_{1}(f_{e}(x))-h_{2}(f_{e}(x))||.$
(8)

Figure 1: An overview of Purified Learning framework, which contains a feature
extractor, a discriminator and a classifier. The top of the diagram shows how
it can be easily combined with existing models that use task-relevant
information.

Theorem 1: For representation distributions $\mathbb{R}_{1},\mathbb{R}_{2}$ on
$\mathcal{R}$, and $h_{1},h_{2}\in H$. Then the following holds:

$\epsilon_{\mathbb{R}_{1}}\left(h_{1},h_{2}\right)\leq\epsilon_{\mathbb{R}_{2}}\left(h_{1},h_{2}\right)+2K\cdot\operatorname{WD}\left(\mathbb{R}_{1},\mathbb{R}_{2}\right).$
(9)

_Proof._ We first prove that $\left|h_{1}-h_{2}\right|$ is $2K$-Lipschitz
continuous. Using the triangle inequality, we have:

$\displaystyle||h_{1}(r)-h_{2}(r)||\leq$
$\displaystyle||h_{1}(r)-h_{1}(r^{\prime})||+||h_{1}(r^{\prime})-h_{2}(r)||$
(10) $\displaystyle\leq$
$\displaystyle||h_{1}(r)-h_{1}(r^{\prime})||+||h_{1}(r^{\prime})-h_{2}(r^{\prime})||$
$\displaystyle+||h_{2}(r)-h_{2}(r^{\prime})||,$

thus,

$\displaystyle||h_{1}(r)-h_{2}(r)||-||h_{2}(r^{\prime})-h_{2}(r^{\prime})||$
(11) $\displaystyle\leq$
$\displaystyle||h_{1}(r)-h_{1}(r^{\prime})||+||h_{1}(r)-h_{2}(r^{\prime})||,$

then, because both $h_{1}$ and $h_{2}$ are $K$-Lipschitz continuous,

$\displaystyle\frac{||h_{1}(r)-h_{2}(r)||-||h_{1}(r^{\prime})-h_{2}(r^{\prime})||}{\rho(r,r^{\prime})}$
(12) $\displaystyle\leq$
$\displaystyle\frac{|h_{1}(r)-h_{1}(r^{\prime})|+||h_{2}(r)-h_{2}(r^{\prime})||}{\rho(r,r^{\prime})}\leq
2K.$

Thus, $\left|h_{1}-h_{2}\right|$ is $2K$-Lipschitz continuous. Then we have:

$\displaystyle\epsilon_{\mathbb{R}_{1}}(h_{1},h_{2})-\epsilon_{\mathbb{R}_{2}}(h_{1},h_{2})$
(13)
$\displaystyle=\underset{r\in\mathbb{R}_{1}}{\mathbb{E}}[||h_{1}(r)-h_{2}(r)||]-\underset{r\in\mathbb{R}_{2}}{\mathbb{E}}[||h_{1}(r)-h_{2}(r)||]$
$\displaystyle\leq\sup_{\|f\|_{L}\leq
2K}\underset{r\sim\mathbb{R}_{1}}{\mathbb{E}}[f(r)]-\underset{r\sim\mathbb{R}_{2}}{\mathbb{E}}[f(r)]$
$\displaystyle=2K\cdot\operatorname{WD}\left(\mathbb{R}_{1},\mathbb{R}_{2}\right).$

So far, Theorem 1 is proven.

Next, we give the upper bound of generalization error by Wasserstein distance.
Denoting the ideal classifier for a specific classification task by
$f^{*}_{c}$, both $f_{c}$ and $f^{*}_{c}$ are $K$-Lipschitz continuous. To
achieve the optimal performance, $f_{c}$ needs to approach the ideal
classifier $f^{*}_{c}$. So the error of $f_{c}$ on distribution $\mathbb{R}$
is defined as $\gamma_{\mathbb{R}}(f_{c})$:

$\gamma_{\mathbb{R}}(f_{c})=\epsilon_{\mathbb{R}}(f_{c},f^{*}_{c}).$ (14)

For two distribution of input instances , denoted by $\mathbb{D}_{1}$ and
$\mathbb{D}_{2}$, their corresponding representation distributions are
$\mathbb{R}_{1}$ and $\mathbb{R}_{2}$ According to Theorem 1, we have:

$\epsilon_{\mathbb{R}_{1}}(f_{c},f^{*}_{c})\leq\epsilon_{\mathbb{R}_{2}}(f_{c},f^{*}_{c})+2K\cdot\operatorname{WD}\left(\mathbb{R}_{1},\mathbb{R}_{2}\right),$
(15)

thus,

$\gamma_{\mathbb{R}_{1}}(f_{c})\leq\gamma_{\mathbb{R}_{2}}(f_{c})+2K\cdot\operatorname{WD}\left(\mathbb{R}_{1},\mathbb{R}_{2}\right).$
(16)

Correspondingly,

$\gamma_{\mathbb{D}_{1}}(f_{c}\circ
f_{e})\leq\gamma_{\mathbb{D}_{2}}(f_{c}\circ
f_{e})+2K\cdot\operatorname{WD}(\mathbb{R}_{1},\mathbb{R}_{2}).$ (17)

$\operatorname{WD}$ is Wasserstein distance that is used to measure the
divergence between two distributions. Thus, for an unknown test distribution
$\mathbb{D}_{1}$, minimizing error on $\mathbb{D}_{1}$ can be divided into two
goals. The first one is to minimize error on the given training sample
distribution $\mathbb{D}_{2}$. The second is to minimize the Wasserstein
distance between $\mathbb{R}_{1}$ and $\mathbb{R}_{2}$. Since $\mathbb{R}_{1}$
is unknown, we need to approximate it using another dataset. According to
unreliability of empirical risk, we need another large-scale dataset to
approximate the distribution of $\mathbb{R}_{1}$. Large-scale data ensures
that the distribution is more close to the testing set. In practice, we use
the feature distribution $R$ of $S$, mentioned in Sec. II-B to do the
approximation.

##  III Framework of Purified Learning

Purified Learning focuses on exploiting the knowledge contained in task-
irrelevant labels from auxiliary samples and transferring it to the target
task. With this knowledge, we measure the influence of task-irrelevant
features, and then use the adversarial learning method based on Wasserstein
distances to limit such influence during feature extraction. In this way,
original extract features is purified, increasing the percentage of task-
relevant features. Generally, the framework is divided into three parts: a
feature extractor $f_{e}$, a linear classifier $f_{c}$ and an additional
discriminator $f_{d}$, as shown in Figure 1. We denote the dataset of the
target task by $\mathcal{S}_{tgt}$, and the task-irrelevant dataset by
$\mathcal{S}_{src}$. Note that Purified Learning is a theoretically validation
framework, the extractor, discriminator and classifier are all flexible and
have multiple choices. A practice choice is to set them according to
performance. To utilize both $\mathcal{S}_{tgt}$ and $\mathcal{S}_{src}$, two
optimization goals are set according to Sec. II.

###  III-A Goal 1: Empirical Risk Minimization

For training sample $x$ and label $y$, $(x,y)\in\mathcal{S}_{tgt}$, the
classification probability $p=\operatorname{softmax}(f_{c}(f_{e}(x)))$, and
the classification loss is calculated by cross-entropy loss, which is
consistent with the loss in empirical risk.

$\begin{split}{\mathcal{L}_{\text{classification}}}=&\\\
-\sum_{x,y\in\mathcal{S}_{tgt}}&y*\log\left(\operatorname{softmax}\left(f_{c}(f_{e}(x))\right)\right).\end{split}$
(18)

Goal 1 is the empirical risk minimization that we minimize the classification
error so that the model learns the knowledge contained in the samples of
$\mathcal{S}_{tgt}$. However, the generalization ability of the model trained
on Goal 1 depends heavily on the consistency of the training data distribution
and the real data distribution as we discussed before, which is usually hard
to achieve on a small dataset. Thus, a new optimization goal should be
imported to utilize large-scale task-relevant or task-irrelevant data to
reduce this bias.

###  III-B Goal 2: Wasserstein Distance Minimization

This goal aims to minimize the Wasserstein distance between
$\mathbb{R}_{y_{1}}$ and $\mathbb{R}$ as Equation 6, and it is necessary to
estimate the two distributions by sampling. For $\mathcal{S}_{src}$, we
randomly select two groups of samples, one from samples with a specific task-
irrelevant label $y_{1}$ in $\mathcal{S}_{src}$, denoted by $A$, and the other
from the entire $\mathcal{S}_{src}$, denoted by $B$. For $A$ and $B$, the
representations extracted by $f_{e}$ are denoted by $R_{A}$, $R_{B}$:

$\begin{split}R_{A}=&\\{f_{e}(x)|x\in A\\}\\\ R_{B}=&\\{f_{e}(x)|x\in
B\\}.\end{split}$ (19)

Algorithm 1 Algorithm of Purified Learning.

Require: target training sample set $\mathcal{S}_{tgt}$; task-irrelevant
sample set $\mathcal{S}_{src}$; learning rate for discriminator $\alpha_{1}$;
learning rate for feature extractor and classifier $\alpha_{2}$; batch size
$m$; Iteration numbers $n_{1}$, $n_{2}$.
Notation: function fitted by the feature extractor, $f_{e}$; function fitted
by the classifier, $f_{c}$; function fitted by the discriminator, $f_{d}$.

1:Initialize feature extractor, linear classifier, discriminator with random
weights $\theta_{e}$ , $\theta_{c}$ , $\theta_{d}$.

2:for $n_{1}$ steps do

3: Sample minibatch $A=\\{a^{(i)}\\}_{i=1}^{m}$ from $\mathcal{S}_{src}$ with
a fixed task-irrelevant label.

4: Sample minibatch $B=\\{b^{(i)}\\}_{i=1}^{m}$ from $\mathcal{S}_{src}$ with
random task-irrelevant labels.

5:
$g_{d}\leftarrow\bigtriangledown_{d}(\frac{1}{m}\sum^{m}_{i=1}(f_{d}(f_{e}(b^{(i)}))-f_{d}(f_{e}(a^{(i)})))$

6: $\theta_{d}\leftarrow\theta_{d}+\alpha_{1}Adam(\theta_{d},g_{d})$

7: for $n_{2}$ steps do

8: Sample minibatch $C=\\{x^{(i)},y^{(i)}\\}_{i=1}^{m}$ from
$\mathcal{S}_{tgt}$.

9: Goal 1:

10: $g_{c}\leftarrow\bigtriangledown_{c}(\sum^{m}_{i=1}(-y^{(i)}\cdot
log(f_{c}(f_{e}(x^{(i)})))))$

11: $\theta_{c}\leftarrow\theta_{c}-\alpha_{2}SGD(\theta_{c},g_{c})$

12: $g_{e}\leftarrow\bigtriangledown_{e}(\sum^{m}_{i=1}(-y^{(i)}\cdot
log(f_{c}(f_{e}(x^{(i)})))))$

13: $\theta_{e}\leftarrow\theta_{e}-\alpha_{2}SGD(\theta_{e},g_{e})$

14: Goal 2:

15:
$g_{e}\leftarrow\bigtriangledown_{e}(\frac{1}{m}\sum^{m}_{i=1}(-f_{d}(f_{e}(x^{(i)}))))$

16: $\theta_{e}\leftarrow\theta_{e}-\alpha_{2}SGD(\theta_{e},g_{e})$

When the sizes of $A$ and $B$ are large enough, the representation
distributions of $R_{A}$ and $R_{B}$, denoted by $\mathbb{R}_{A}$ and
$\mathbb{R}_{B}$, are used as reasonable estimations of $\mathbb{R}_{y_{1}}$
and $\mathbb{R}$. Thus, Equation 6 is rewritten as:

$\min(\operatorname{WD}(\mathbb{R}_{A},\mathbb{R}_{B})).$ (20)

We substitute the $\operatorname{WD}$ with Wasserstein’s Kantorovich-
Rubinstein duality form,

$\min_{f_{e}}\sup_{\left\|f_{d}\right\|_{L\leq 1}}\left(\underset{x\in
A}{\mathbb{E}}\left[f_{d}(f_{e}(x))\right]-\underset{x\in
B}{\mathbb{E}}\left[f_{d}(f_{e}(x))\right]\right).$ (21)

Inspired by WGAN [3], Equation 21 could be divided into two steps. Firstly,
the discriminator $f_{d}$ is trained by:

$\max_{f_{d}}\left(\underset{x\in
A}{\mathbb{E}}\left[f_{d}(f_{e}(x))\right]-\underset{x\in
B}{\mathbb{E}}\left[f_{d}(f_{e}(x))\right]\right).$ (22)

Secondly, we select a group of training samples, $C$, from training dataset of
target task $\mathcal{S}_{tgt}$. The feature representations extracted by
$f_{e}$ are denoted by $R_{C}$,

$R_{C}=\\{f_{e}(x)|x\in C\\},$ (23)

then, the extractor $f_{e}$ is trained by:

$\min_{f_{e}}\left(-\underset{x\in
C}{\mathbb{E}}\left[f_{d}(f_{e}(x))\right]\right),$ (24)

and the corresponding Wasserstein loss is:

${\mathcal{L}}_{\text{wasserstein}}=-\frac{1}{\|C\|}\underset{x\in
C}{\sum}\left[f_{d}(f_{e}(x))\right],$ (25)

where $\|C\|$ is the size of the sample group C.

Combining $\mathcal{L}_{\text{classification}}$ and
$\mathcal{L}_{\text{wasserstein}}$, the complete loss function is written as
follows, where $\lambda_{i}$ are weighting factors:

$\mathcal{L}_{\text{Purify
Learning}}=\lambda_{1}{\mathcal{L}}_{\text{classification}}+\lambda_{2}{\mathcal{L}}_{\text{wasserstein}}.$
(26)

In practice, the training algorithm is shown in Alg. 1.

##  IV Experiments

Feature Extractor | Train $\rightarrow$ Test Dataset | Goal 1 Only |  | TL
---
(Hair Color)
| MTL
---
(Hair Color)
| AMTL
---
(Hair Color)
| PL
---
(Hair Color)
AlexNet | ck+ $\rightarrow$ mmi | 35.91 | 38.28 | 33.73 | 34.57 | 37.61
ck+ $\rightarrow$ oulu | 34.75 | 25.54 | 33.08 | 33.15 | 38.31
mmi $\rightarrow$ ck+ | 56.36 | 44.97 | 57.09 | 57.94 | 61.45
mmi $\rightarrow$ oulu | 22.12 | 19.89 | 23.66 | 28.73 | 22.61
oulu $\rightarrow$ ck+ | 55.03 | 54.42 | 54.67 | 43.52 | 59.64
oulu $\rightarrow$ mmi | 39.46 | 40.00 | 40.30 | 28.33 | 45.36
Average | 40.61 | 37.18 | 40.42 | 37.37 | 44.16
ResNet34 | ck+ $\rightarrow$ mmi | 50.42 | 46.54 | 46.54 | 44.35 | 50.42
ck+ $\rightarrow$ oulu | 50.94 | 54.29 | 47.24 | 43.75 | 51.78
mmi $\rightarrow$ ck+ | 65.33 | 68.73 | 66.18 | 64.61 | 69.21
mmi $\rightarrow$ oulu | 44.87 | 46.13 | 40.96 | 42.15 | 42.43
oulu $\rightarrow$ ck+ | 73.45 | 73.33 | 72.93 | 66.18 | 80.12
oulu $\rightarrow$ mmi | 54.13 | 49.01 | 51.43 | 41.99 | 54.30
Average | 56.52 | 56.34 | 54.21 | 50.51 | 58.04
VggNet19 | ck+ $\rightarrow$ mmi | 45.53 | 41.10 | 37.10 | 33.05 | 45.53
ck+ $\rightarrow$ oulu | 56.66 | 44.52 | 32.24 | 47.66 | 57.22
mmi $\rightarrow$ ck+ | 65.33 | 62.30 | 64.61 | 57.82 | 66.79
mmi $\rightarrow$ oulu | 45.08 | 32.17 | 46.69 | 40.20 | 46.76
oulu $\rightarrow$ ck+ | 76.73 | 71.27 | 72.12 | 31.88 | 78.91
oulu $\rightarrow$ mmi | 45.03 | 43.68 | 40.00 | 20.24 | 51.43
Average | 55.73 | 49.17 | 48.79 | 38.48 | 57.77

TABLE I: Results (Accuracy $\%$) on FER task, where TL refers to transfer
learning, MTL refers to multi-task learning, AMTL refers to adversarial multi-
task learning, and PL refers to Purified Learning. ck+$\rightarrow$mmi means
model is trained on ck+ and tested on mmi dataset. TL (Hair Color) means using
_hair color_ label in transfer learning.

###  IV-A Experimental Setup

####  IV-A1 Model

As mentioned before, the framework of Purified Learning consists of three
parts: a feature extractor, a linear classifier, and a discriminator. The
discriminator consists of four fully connected layers. The linear classifier
is a fully connected layer with the number of neurons depending on the
specific task. The flexibity of Purified Learning enables that various
mainstream models can be embed as feature extractor.

####  IV-A2 Evaluation Metrics

To evaluate the results of model generalization, the cross-dataset test
results are taken as evaluation metrics. This is based on a reasonable
assumption: the training set and the real data distribution are different, and
the unseen test set is sampled from the real data distribution, so there
exists a gap between the distribution of test data and that of training data.
Therefore, the higher the accuracy in our test means the better the
generalization performance of the model.

####  IV-A3 Hyperparameters

We implement all experiments without data augmentation. The model is trained
by an SGD optimizer with an initial learning rate of 0.001, the momentum of
0.9, StepLR(step size is 7), and $\gamma$ of 0.1. We apply widely used
classification models (e.g ResNet) as a feature extractor and set the output
dimension of the penultimate layer to 128. The classifier is a fully connected
layer and the output dimension is equal to the number of classes (7 for FER
and 10 for digit recognition). The discriminator consists of four fully
connected layers with 512, 256, 10, 1 nodes, the discriminator is trained
using an Adam with the same learning rate, and a weight limit 0.1. In
addition, the step ratio of $n_{2}$ in Alg.1 is 3. The factors $\lambda_{1}$
and $\lambda_{2}$ are both 1. We use batch size 32 in FER, and 128 in digit
recognition. All results are obtained after the model has been trained for 50
epochs.

###  IV-B Experimental implementation details

We provide implementation details of the comparison method used in our
experiments.

####  IV-B1 Facial Expression Recognition

  * •

Transfer learning: We adopt the pre-training/fine-tuning approaches to
transfer learning based on shared parameters [37]. We first pre-train the
feature extractor as well as the classifier on CelebA for hair color
classification and then train the whole model on the expression dataset.

  * •

Multi-task learning: We adopt the multi-task learning approach with hard
parameter sharing of hidden layers [5]. Specifically, a shared feature
extractor accepts both images with hair color tags from CelebA and images with
expression tags from a training set. And two independent classifier networks
make separate predictions for hair color and expressions. The entire network
gets trained by backpropagating the final loss which is calculated by adding
up the classification loss of hair color and the classification loss of
expression.

  * •

Adversarial Multi-task Learning: Based on the multi-tasking learning model, a
new gradient reversal layer (GRL) is added between the feature extractor and
the hair color classifier, where the idea is consistent with Shinohara [27].

####  IV-B2 Digit Recognition

  * •

DANN: DANN is a framework for unsupervised domain adaptation based on
adversarial multitasking learning proposed by Ganin [8]. As a comparison
method for the simultaneous use of MNIST and SVHN data, it is implemented by
adding a new gradient reversal layer (GRL) and a domain classifier with
2-dimensional outputs for domain classification to the existing feature
extractor and classifier. The feature extractor accepts both images from MNIST
and SVHN. One classifier then classifies the feature representation of the
MNIST data from 0 to 9, and another classifier distinguishes whether the
feature representation originates from MNIST or SVHN.

###  IV-C Experiments on Facial Expression Recognition

In this part, we compare Purified Learning with some methods related to our
work on the FER task. These methods are briefly described below, and their
implementation details are shown in Section IV-B:

  * •

Goal 1 Only: As an empirical baseline, we use labeled training data to train
the model with the optimization goal 1 only.

  * •

Transfer learning : We use the hair color recognition task to pre-train the
network, and then fine-tune on FER task.

  * •

Multi-task learning : We train FER and hair color recognition tasks using a
shared feature extractor.

  * •

Adversarial multi-task learning: We add a gradient inversion layer for the
hair color recognition task on the multi-task learning method above.

####  IV-C1 Datasets and Feature Extractors

In FER experiments, we use small-scale datasets, including ck+ [16], oulu [41]
and mmi [21] for cross-dataset evaluation. We select AlexNet [13], VggNet19
[28] and ResNet34 [10] as feature extractors.

We use $224\times 224$ resolution for all RGB pictures and preprocess them
through MTCNN [38] for alignment. More importantly, the _hair color_ label in
CelebA [15] is regarded as task-irrelevant label. Therefore, we select samples
with fixed _hair color_ labels and samples randomly selected from CelebA, and
then using Purified Learning on them.

####  IV-C2 Results

Table I is the comparison between Purified Learning and baseline methods,
which shows that our method achieves better results in 14 of 18 cross-dataset
tests.

Since hair color is not relevant to FER, it leads to negative transfer for
transfer learning methods including transfer learning, multi-task learning and
adversarial multi-task learning (lower accuracy even than Goal 1 Only in most
cases). The performance of the adversarial multi-task learning is the worst
because it requires training samples with both task label and task-irrelevant
label. However, in our experiments, the training samples are only annotated
with task labels, and the task-irrelevant labels are from additional samples.
By contrast, Purified Learning is able to make use of the knowledge from task-
irrelevant labels, so it improves the accuracy of Goal 1 only, thus
successfully resolving negative transfer and improving the generalization
performance of the model.

###  IV-D Experiments on Digit Recognition

In this part, we evaluate Purified Learning on Digit Recognition task. In this
experiment, we add another baseline, DANN [1], a domain transfer framework
based on adversarial multi-task learning. DANN is currently the state-of-the-
art method of transferring from MNIST to MNIST-M111Link to Paperwithcode.
Since we use small-scale training set, our accuracy is lower than original
paper. The model details of DANN have been shown in Section IV-B.

####  IV-D1 Datasets and Feature Extractors

In the digit recognition experiments, regarding _background color_ as the
task-irrelevant features, we also use a small-scale training set, which are
20000 pictures selected from MNIST [14]. The test sets are SVHN [20] and
MNIST-M [8]. The two groups of samples are randomly sampled from MNIST and the
combination of MNIST and SVHN respectively. Besides, all pictures are
converted to RGB pictures and we use $32\times 32$ resolution.

Additionally, we select ResNet18 [10], VggNet11 [28], DenseNet121 [12], SeNet
[11], EﬃcientNet [29], MobileNetV2 [24] and ShuﬄeNetV2 [18] as feature
extractors.

####  IV-D2 Results

| Train $\rightarrow$ Test
---
Dataset
| Feature
---
Extractor
| Goal 1
---
Only
DANN |  | PL
---
(Background Color)
MNIST $\rightarrow$ SVHN | DenseNet121 | 13.85 | 10.64 | 16.44
EfficientNet | 22.99 | 13.99 | 33.74
MobileNetV2 | 20.06 | 11.35 | 22.11
ResNet18 | 15.01 | 12.00 | 18.83
SeNet | 15.44 | 10.69 | 18.22
ShuffleNetV2 | 14.29 | 11.98 | 20.72
VggNet11 | 19.50 | 13.79 | 25.71
| Average | 17.31 | 12.06 | 22.25
MNIST $\rightarrow$ MNIST-M | DenseNet121 | 25.26 | 14.28 | 35.21
EfficientNet | 40.78 | 38.71 | 57.64
MobileNetV2 | 29.77 | 21.75 | 48.76
ResNet18 | 25.10 | 22.54 | 37.64
SeNet | 17.24 | 15.83 | 39.71
ShuffleNetV2 | 27.96 | 20.34 | 47.65
VggNet11 | 39.18 | 42.09 | 48.98
| Average | 29.33 | 25.08 | 45.08

TABLE II: Results (Accuracy $\%$) on digit recognition task.

Table II is the comparison between our method and two baselines. Compared with
baselines, Purified Learning achieves higher accuracy than DANN and Goal 1
Only method. The Although transferring from MNIST to SVHN is difficult since
SVHN is much more complex than MNIST, Purified Learning improves DANN by more
than 10% accuracy, which proves the effectiveness of using task-irrelevant
features. We also achieve better results than DANN in the “MNIST to MNIST-M”
task.

###  IV-E Discussion

####  IV-E1 The impact of different levels of task-irrelevance.

| Feature
---
Extractor
| Train $\rightarrow$ Test
---
Dataset
| Goal 1
---
Only
| PL
---
(Smiling)
| PL
---
(Hair Color)
AlexNet | ck+ $\rightarrow$ mmi | 35.91 | 32.38 | 37.61
ck+ $\rightarrow$ oulu | 34.75 | 33.91 | 38.31
mmi $\rightarrow$ ck+ | 56.36 | 35.39 | 61.45
mmi $\rightarrow$ oulu | 22.12 | 18.77 | 22.61
oulu $\rightarrow$ ck+ | 55.03 | 49.33 | 59.64
oulu $\rightarrow$ mmi | 39.46 | 38.45 | 45.36
Average | 40.61 | 34.71 | 44.16
ResNet34 | ck+ $\rightarrow$ mmi | 50.42 | 47.89 | 50.42
ck+ $\rightarrow$ oulu | 50.94 | 50.31 | 51.78
mmi $\rightarrow$ ck+ | 65.33 | 65.70 | 69.21
mmi $\rightarrow$ oulu | 44.87 | 39.22 | 42.43
oulu $\rightarrow$ ck+ | 73.45 | 73.70 | 80.12
oulu $\rightarrow$ mmi | 54.13 | 51.43 | 54.30
Average | 56.52 | 54.71 | 58.04
VggNet19 | ck+ $\rightarrow$ mmi | 45.53 | 40.47 | 45.53
ck+ $\rightarrow$ oulu | 56.66 | 42.99 | 57.22
mmi $\rightarrow$ ck+ | 65.33 | 63.64 | 66.79
mmi $\rightarrow$ oulu | 45.08 | 44.59 | 46.76
oulu $\rightarrow$ ck+ | 76.73 | 75.03 | 78.91
oulu $\rightarrow$ mmi | 45.03 | 50.59 | 51.43
Average | 55.73 | 52.89 | 57.77

TABLE III: Results (Accuracy $\%$) on FER task, using different labels
(_smiling_ and _hair color_) in Purified Learning.

As different task-relevance can affect the performance of transfer learning,
it is essential to study the impact of different levels of task-irrelevance on
Purified Learning. Except for the _hair color_ label, we use the _smiling_
label from CelebA. We apply these labels to the Purified Learning framework
and compare their performance.

Table III indicates that the experiment results that compared with Goal 1 Only
method, the performance of Purified Learning remains the same or gets worse
when using the _smiling_ label, but have significant improvement when using
the _hair color_ label. The experiment results suggest that using a more
irrelevant label is better for the Purified Learning framework to improve the
performance since it tries to use the knowledge of task-irrelevant features.
On the other hand, if we misuse task-relevant samples in Purified Learning, it
will have a negative impact. This is similar to negative transfer while using
task-irrelevant labels in transfer learning. In general, this phenomenon means
that Purified Learning differs from transfer learning, since it benefits from
the irrelevance of data instead of relevance.

####  IV-E2 PCA analysis on feature representations

Based on the maximum variance theory [30], if the percentage of variance
explained by principal components increase, the representation vector will
contain more information about task-relevant features, and task-relevant
information will be better retained during the process of dimension reduction.

Figure 2: PCA results of ResNet18 on digit recognition task. The larger the
proportion of the dark green part (principal component 1 $\&$ 2), the larger
the proportion of the components that are decisive to the task in the
representation.

Figure 2 shows the proportion of principal components of different methods in
digit recognition tasks. Compared with other methods (Goal 1 Only and DANN),
the proportion of task-relevant information is significantly higher in
Purified Learning, which means that Purified Learning learns knowledge from
task-irrelevant labels and avoid extracting task-irrelevant features during
feature extraction.

##  V Conclusion

In this paper, we propose a novel framework, Purified Learning, to exploit the
knowledge from additional task-irrelevant labels in order to solve the problem
caused by the unreliable empirical risk minimization when the training dataset
is small. Based on detailed theoretical analysis, we illustrate that samples
with task-irrelevant labels can be used in improving the generalization
performance of the model, based on which we propose Purified Learning, which
directly obtains and utilizes knowledge from a wide range of task-irrelevant
labels. Furthermore, Purified Learning can be well combined with the existing
task-relevant learning methods. We believe that the exploration of task-
irrelevant labels in our work provides a valuable insight for future research.

##  VI Acknowledgement

This research is supported by National Natural Science Foundation of China
(Grant No. 6201101015 and 61602013), Beijing Academy of Artificial
Intelligence (BAAI), the Natural Science Foundation of Guangdong Province
(Grant No. 2021A1515012640), the Basic Research Fund of Shenzhen City (Grant
No. JCYJ20210324120012033 and JCYJ20190813165003837), the Shenzhen General
Research Project (Grant No. JCYJ20190808182805919), the National Key R&D
Program of China (No. 2021ZD0112905), the Overseas Cooperation Research Fund
of Tsinghua Shenzhen International Graduate School (Grant No. HW2021008) and
the 173 program (Grant No. 2021-JCJQ-JJ-0029).

## References

  * [1] H. Ajakan, P. Germain, H. Larochelle, F. Laviolette, and M. Marchand, “Domain-adversarial neural networks,” _arXiv preprint arXiv:1412.4446_ , 2014.
  * [2] A. Ajiboye, R. Abdullah-Arshah, and Q. Hongwu, “Evaluating the effect of dataset size on predictive model using supervised learning technique,” 2015.
  * [3] M. Arjovsky, S. Chintala, and L. Bottou, “Wasserstein gan,” _arXiv preprint arXiv:1701.07875_ , 2017.
  * [4] C. Bowles, L. Chen, R. Guerrero, P. Bentley, R. Gunn, A. Hammers, D. A. Dickie, M. V. Hernández, J. Wardlaw, and D. Rueckert, “Gan augmentation: Augmenting training data using generative adversarial networks,” _arXiv preprint arXiv:1810.10863_ , 2018.
  * [5] R. Caruana, “Multitask learning: A knowledge-based source of inductive bias,” in _Proceedings of the Tenth International Conference on Machine Learning_. Morgan Kaufmann, 1993, pp. 41–48.
  * [6] T. Chen, S. Kornblith, M. Norouzi, and G. Hinton, “A simple framework for contrastive learning of visual representations,” _arXiv preprint arXiv:2002.05709_ , 2020.
  * [7] E. D. Cubuk, B. Zoph, D. Mane, V. Vasudevan, and Q. V. Le, “Autoaugment: Learning augmentation policies from data,” _arXiv preprint arXiv:1805.09501_ , 2018.
  * [8] Y. Ganin, E. Ustinova, H. Ajakan, P. Germain, H. Larochelle, F. Laviolette, M. Marchand, and V. Lempitsky, “Domain-adversarial training of neural networks,” _The Journal of Machine Learning Research_ , vol. 17, no. 1, pp. 2096–2030, 2016.
  * [9] J. Gu, Z. Wang, J. Kuen, L. Ma, A. Shahroudy, B. Shuai, T. Liu, X. Wang, G. Wang, J. Cai _et al._ , “Recent advances in convolutional neural networks,” _Pattern Recognition_ , vol. 77, pp. 354–377, 2018.
  * [10] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2016, pp. 770–778.
  * [11] J. Hu, L. Shen, and G. Sun, “Squeeze-and-excitation networks,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2018, pp. 7132–7141.
  * [12] G. Huang, Z. Liu, L. Van Der Maaten, and K. Q. Weinberger, “Densely connected convolutional networks,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2017, pp. 4700–4708.
  * [13] A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classification with deep convolutional neural networks,” in _Advances in neural information processing systems_ , 2012, pp. 1097–1105.
  * [14] Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, “Gradient-based learning applied to document recognition,” _Proceedings of the IEEE_ , vol. 86, no. 11, pp. 2278–2324, 1998.
  * [15] Z. Liu, P. Luo, X. Wang, and X. Tang, “Deep learning face attributes in the wild,” in _Proceedings of International Conference on Computer Vision (ICCV)_ , December 2015.
  * [16] P. Lucey, J. F. Cohn, T. Kanade, J. Saragih, Z. Ambadar, and I. Matthews, “The extended cohn-kanade dataset (ck+): A complete dataset for action unit and emotion-specified expression,” in _2010 ieee computer society conference on computer vision and pattern recognition-workshops_. IEEE, 2010, pp. 94–101.
  * [17] Z. Luo, Y. Zou, J. Hoffman, and L. F. Fei-Fei, “Label efficient learning of transferable representations acrosss domains and tasks,” in _Advances in Neural Information Processing Systems_ , 2017, pp. 165–177.
  * [18] N. Ma, X. Zhang, H.-T. Zheng, and J. Sun, “Shufflenet v2: Practical guidelines for efficient cnn architecture design,” in _Proceedings of the European conference on computer vision (ECCV)_ , 2018, pp. 116–131.
  * [19] M. Mohri, A. Rostamizadeh, and A. Talwalkar, _Foundations of Machine Learning_ , ser. Adaptive computation and machine learning. MIT Press, 2012.
  * [20] Y. Netzer, T. Wang, A. Coates, A. Bissacco, B. Wu, and A. Y. Ng, “Reading digits in natural images with unsupervised feature learning,” 2011.
  * [21] M. Pantic, M. Valstar, R. Rademaker, and L. Maat, “Web-based database for facial expression analysis,” in _2005 IEEE international conference on multimedia and Expo_. IEEE, 2005, pp. 5–pp.
  * [22] J. Prusa, T. M. Khoshgoftaar, and N. Seliya, “The effect of dataset size on training tweet sentiment classifiers,” in _2015 IEEE 14th International Conference on Machine Learning and Applications (ICMLA)_. IEEE, 2015, pp. 96–102.
  * [23] M. T. Rosenstein, Z. Marx, L. P. Kaelbling, and T. G. Dietterich, “To transfer or not to transfer,” in _NIPS 2005 workshop on transfer learning_ , vol. 898, 2005, pp. 1–4.
  * [24] M. Sandler, A. Howard, M. Zhu, A. Zhmoginov, and L.-C. Chen, “Mobilenetv2: Inverted residuals and linear bottlenecks,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2018, pp. 4510–4520.
  * [25] J. Shen, Y. Qu, W. Zhang, and Y. Yu, “Wasserstein distance guided representation learning for domain adaptation,” _arXiv preprint arXiv:1707.01217_ , 2017.
  * [26] Y. Shinohara, “Adversarial multi-task learning of deep neural networks for robust speech recognition.” in _Interspeech_. San Francisco, CA, USA, 2016, pp. 2369–2372.
  * [27] ——, “Adversarial multi-task learning of deep neural networks for robust speech recognition.” in _Interspeech_. San Francisco, CA, USA, 2016, pp. 2369–2372.
  * [28] K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” in _International Conference on Learning Representations_ , 2015.
  * [29] M. Tan and Q. V. Le, “Efficientnet: Rethinking model scaling for convolutional neural networks,” _arXiv preprint arXiv:1905.11946_ , 2019.
  * [30] M. E. Tipping and C. M. Bishop, “Mixtures of probabilistic principal component analyzers,” _Neural computation_ , vol. 11, no. 2, pp. 443–482, 1999.
  * [31] T. Tommasi, N. Patricia, B. Caputo, and T. Tuytelaars, “A deeper look at dataset bias,” in _Domain adaptation in computer vision applications_. Springer, 2017, pp. 37–55.
  * [32] A. Torralba and A. A. Efros, “Unbiased look at dataset bias,” in _CVPR 2011_. IEEE, 2011, pp. 1521–1528.
  * [33] V. Vapnik, “Principles of risk minimization for learning theory,” in _Advances in Neural Information Processing Systems 4, [NIPS Conference, Denver, Colorado, USA, December 2-5, 1991]_ , J. E. Moody, S. J. Hanson, and R. Lippmann, Eds. Morgan Kaufmann, 1991, pp. 831–838.
  * [34] Y. Wang, Q. Yao, J. T. Kwok, and L. M. Ni, “Generalizing from a few examples: A survey on few-shot learning,” _ACM Computing Surveys (CSUR)_ , vol. 53, no. 3, pp. 1–34, 2020.
  * [35] Y.-X. Wang and M. Hebert, “Learning to learn: Model regression networks for easy small sample learning,” in _European Conference on Computer Vision_. Springer, 2016, pp. 616–634.
  * [36] K. Weiss, T. M. Khoshgoftaar, and D. Wang, “A survey of transfer learning,” _Journal of Big data_ , vol. 3, no. 1, p. 9, 2016.
  * [37] J. Yosinski, J. Clune, Y. Bengio, and H. Lipson, “How transferable are features in deep neural networks?” in _Advances in neural information processing systems_ , 2014, pp. 3320–3328.
  * [38] K. Zhang, Z. Zhang, Z. Li, and Y. Qiao, “Joint face detection and alignment using multitask cascaded convolutional networks,” _IEEE Signal Processing Letters_ , vol. 23, no. 10, pp. 1499–1503, 2016.
  * [39] X. Zhang, Z. Wang, D. Liu, Q. Lin, and Q. Ling, “Deep adversarial data augmentation for extremely low data regimes,” _IEEE Transactions on Circuits and Systems for Video Technology_ , 2020.
  * [40] Z. Zhang, P. Luo, C. C. Loy, and X. Tang, “Facial landmark detection by deep multi-task learning,” in _European conference on computer vision_. Springer, 2014, pp. 94–108.
  * [41] G. Zhao, X. Huang, M. Taini, S. Z. Li, and M. PietikäInen, “Facial expression recognition from near-infrared videos,” _Image and Vision Computing_ , vol. 29, no. 9, pp. 607–619, 2011.