# Provable Unrestricted Adversarial Training without Compromise with
Generalizability

Lilin Zhang, Ning Yang, Yanchao Sun, Philip S. Yu  Lilin Zhang is with the
School of Computer Science, Sichuan University, China. E-mail:
<EMAIL_ADDRESS>Ning Yang is the corresponding author and with the
School of Computer Science, Sichuan University, China. E-mail:
<EMAIL_ADDRESS>Yanchao Sun is with the Department of Computer Science,
University of Maryland, College Park, USA. E-mail<EMAIL_ADDRESS>Philip S. Yu is
with the Department of Computer Science, University of Illinois at Chicago,
USA. E-mail<EMAIL_ADDRESS>
###### Abstract

Adversarial training (AT) is widely considered as the most promising strategy
to defend against adversarial attacks and has drawn increasing interest from
researchers. However, the existing AT methods still suffer from two
challenges. First, they are unable to handle unrestricted adversarial examples
(UAEs), which are built from scratch, as opposed to restricted adversarial
examples (RAEs), which are created by adding perturbations bound by an $l_{p}$
norm to observed examples. Second, the existing AT methods often achieve
adversarial robustness at the expense of standard generalizability (i.e., the
accuracy on natural examples) because they make a tradeoff between them. To
overcome these challenges, we propose a unique viewpoint that understands UAEs
as imperceptibly perturbed unobserved examples. Also, we find that the
tradeoff results from the separation of the distributions of adversarial
examples and natural examples. Based on these ideas, we propose a novel AT
approach called Provable Unrestricted Adversarial Training (PUAT), which can
provide a target classifier with comprehensive adversarial robustness against
both UAE and RAE, and simultaneously improve its standard generalizability.
Particularly, PUAT utilizes partially labeled data to achieve effective UAE
generation by accurately capturing the natural data distribution through a
novel augmented triple-GAN. At the same time, PUAT extends the traditional AT
by introducing the supervised loss of the target classifier into the
adversarial loss and achieves the alignment between the UAE distribution, the
natural data distribution, and the distribution learned by the classifier,
with the collaboration of the augmented triple-GAN. Finally, the solid
theoretical analysis and extensive experiments conducted on widely-used
benchmarks demonstrate the superiority of PUAT.

###### Index Terms:

Adversarial Robustness, Adversarial Training, Unrestricted Adversarial
Examples, Standard Generalizability.

##  1 Introduction

Deep neural network (DNN) has achieved ground-breaking success in a number of
artificial intelligence application areas, including computer vision, object
identification, and natural language processing, among others. Despite the
remarkable success of DNN, seminal researches reveal its vulnerability to
adversarial examples (AEs), which are inputs with non-random perturbations
unnoticeable to humans but intentionally meant to cause victim models to
deliver false outputs [1, 2, 3, 4]. The harms brought on by AEs have prompted
ongoing efforts to increase DNN’s adversarial robustness (i.e., the accuracy
on AEs), among which adversarial training (AT) has been largely acknowledged
as the most promising defensive strategy [5, 6, 7].

The fundamental idea of AT was first proposed by Szegedy et al. [2], which
adds AEs to the training set and uses a min-max game to create a robust
classifier that can withstand worst-case attacks. In particular, AT
alternately maximizes an adversarial loss incurred by AEs that are
purposefully crafted using adversarial attacking methods such as FGSM [8] and
PGD [9], and minimizes it by adjusting the target classifier’s parameters via
a regular supervised training on the augmented training set. Numerous works
have suggested strengthening the basic AT in order to increase adversarial
robustness [10, 11, 12, 13, 14, 15]. Although the existing AT approaches have
advanced significantly, we argue that the problem of adversarial robustness is
still far from being well solved partly due to the following two challenges.

(a) Adversarial robustness against RAE

(b) Adversarial robustness against UAE

Figure 1: Illustration of adversarial robustness against RAE and UAE. The blue
dots and green triangles represent two classes of observed examples,
respectively, and the gray ones represent the unobserved examples of the two
classes. Each example is surrounded by a box representing its neighborhood
with small distance. The purple stars represent RAEs while the red stars UAEs.
The yellow curves are decision boundaries obtained by adversarial training.

The issue of Unrestricted Adversarial Example (UAE) is the first obstacle. As
previously mentioned, AT is conducted on a training set that includes AEs.
However, the conventional AT approaches are frequently limited to
perturbation-based AEs, which are created by adding adversarial perturbations
with magnitudes restricted by an $l_{p}$ norm to observed natural examples. We
refer to such kind of AEs as restricted AE (RAE). In fact, RAEs only account
for a small portion of possible AEs in real world. Recent works reveal the
existence of UAEs, which are dissimilar to any observable example since they
are built from scratch rather than by perturbing existing examples [16, 17,
18, 19, 20]. Figure 1 provides an illustration showing how UAEs (represented
by the red stars) are not the neighbors of any observed example (blue circle
or green triangle), in contrast to RAEs (shown by the purple stars). Despite
the adversarial robustness against RAE provided by conventional AT approaches,
as seen in Figure 1(a), the classifier (with the yellow curve as the decision
boundary) can still be deceived by UAEs because they also appear natural [16].
UAEs are more numerous, more covert, and more dangerous than RAEs since UAEs
are not dependent on observed examples which are typically sparse. However,
the existing works on UAE mainly focus on the generation of UAEs, which suffer
from two defects. First, no explanation for why UAEs exist and why they are
imperceptible to humans is provided in the existing works. Second, adversarial
robustness against UAE still remains largely unexplored. Therefore, we need a
new AT method that is able to offer the robustness against both UAE and RAE,
which we refer to as comprehensive adversarial robustness. As shown in Figure
1(b), the classifier with comprehensive adversarial robustness is able to
correctly predict the labels of both the UAEs and the RAEs.

The second challenge faced by the current AT approaches is the problem of the
deterioration of standard generalizability (i.e., the accuracy on
natural/natural testing examples). Early researches show that the standard
generalizability of a classifier is sacrificed in order for conventional AT
approaches to obtain the adversarial robustness for the classifier [9]. Zhang
et al. [21] further demonstrate the existence of an upper-bound of the
standard generalizability given the adversarial robustness, and Attias et al.
[22] suggest lowering the upper-bound by increasing sample complexity.
However, recent studies argue that the tradeoff may not be inevitable, and
adversarial robustness and standard generalizability can coexist without
negatively affecting each other [15, 23, 24, 25]. For example, Stutz et al.
[15] demonstrate that the adversarial robustness against the AEs existing on
the manifold of natural examples will equivalently lead to better
generalization. Song et al. [24] and Xing et al. [25] propose to improve the
standard generalizability by introducing robust local features and $l_{1}$
penalty to AT, respectively. The existing efforts, however, only concentrate
on RAEs. It is yet unknown that if and how the comprehensive adversarial
robustness and the standard generalizability can both be improved
simultaneously.

To address the first challenge, in this paper, we propose a unique viewpoint
that understands UAEs as imperceptibly perturbed unobserved examples (gray dot
or triangle in Figure 1), which logically explains why UAEs are dissimilar to
observed examples and why UAEs can fool a classifier without confusing humans.
Essentially, RAE can be thought of as a special UAE resulting from perturbing
observed examples. If we generalize the concept of UAE to imperceptibly
perturbed natural examples, regardless of whether they are observable, then
UAE can be regarded as a general form of AE, which makes it feasible to
provide comprehensive adversarial robustness.

For the second challenge, we find that the tradeoff exists because the target
classifier can not generalize over two separated distributions, the AE
distribution and the natural example distribution, and the distribution
learned by the classifier is a mix of them. Therefore, if we can align the
distributions of UAEs and natural examples with the distribution learned by
the target classifier, then the classifier can generalize over UAEs and
natural examples without conflict, and we can expect to eliminate the tradeoff
between comprehensive adversarial robustness and standard generalization and
achieve a simultaneous improvement of both.

Based on these ideas, we propose a novel AT method called Provable
Unrestricted Adversarial Training (PUAT), which integrates the generation of
UAEs and an extended AT on UAEs with a GAN-based framework, so that the
distributions of UAEs and natural examples can be aligned with the
distribution learned by the target classifier for simultaneously improving its
comprehensive adversarial robustness and standard generalizability. At first,
we design a UAE generation module consisting of an attacker $A$ and a
conditional generator $G$, treating a UAE as an adversarially perturbed
natural example. Different from the existing generative models for AEs, which
often result in suboptimal AEs due to the gradient conflicts during the single
optimization for both imperceptibility and harmfulness of AEs, our UAE
generation module decouples the optimizations of imperceptibility and
harmfulness of UAEs. Particularly, $A$ is in charge of producing an
adversarial perturbation that can fool the classifier, based on which $G$ will
generate a UAE for a certain class, which is equivalent to perturbing a
natural example. PUAT ensures the imperceptibility of UAEs by aligning the UAE
distribution learned by $G$ with the natural example distribution via a G-C-D
GAN, in contrast to conventional AT approaches that accomplish
imperceptibility by an explicit norm constraint on the perturbations. The
G-C-D GAN is a novel augmented triple-GAN that combines two conditional GANs:
the G-D GAN, which consists of $G$ and a discriminator $D$, and the C-D GAN,
which consists of the target classifier $C$ and $D$. Different from the
original triple-GAN proposed by Li et al. [26], in G-C-D GAN the generator $G$
is shared with the UAE generation module, which augments the input of $G$ with
the output of the attacker $A$ and consequently allows $G$ to draw UAEs from
the UAE distribution. To facilitate the distributional alignment, we also make
use of unlabeled data to create pseudo-labeled data for the training of G-C-D
GAN since labeled data are usually insufficient for capturing the real data
distribution. By leveraging partially labeled data, G-D GAN and C-D GAN can
cooperate to help $G$ capture the natural data distribution and the UAE
generation module understand how to build a legitimate AE for a certain class.

In order to simultaneously improve the target classifier’s comprehensive
adversarial robustness and standard generalizability, PUAT conducts an
extended AT between the target classifier $C$ and the attacker $A$, which
incorporates a supervised loss of the target classifier with the adversarial
loss. The extended AT together with the G-C-D GAN leads to the alignment
between the UAE distribution, the natural example distribution, and the
distribution learned by $C$, which enables $C$ to generalize without conflict
on UAEs and natural examples. Finally, we theoretically and empirically
demonstrate that PUAT realizes a consistent optimization for both the
comprehensive adversarial robustness and the standard generalizability,
eliminating the tradeoff between them and resulting in their mutual advantage.
Our contributions can be summarized as follows:

  * •

We propose a unique viewpoint that understands UAEs as imperceptibly perturbed
unobserved examples, which offers a logical and demonstrable explanation for
UAEs’ discrepancies from observed examples, imperceptibility to humans, and
the feasibility of comprehensive adversarial robustness against both RAE and
UAE.

  * •

We propose a novel AT approach called Provable Unrestricted Adversarial
Training (PUAT), which, with theoretical guarantee, can simultaneously
increase a classifier’s comprehensive adversarial robustness and standard
generalizability through a distributional alignment.

  * •

We provide a solid theoretical analysis to demonstrate that PUAT can eliminate
the tradeoff between the comprehensive adversarial robustness and the standard
generalizability through the distributional alignment.

  * •

The extensive experiments conducted on widely adopted benchmarks verify the
superiority of PUAT.

Figure 2: Architecture of PUAT. The attacker $A$ seeks the perturbation
$z_{a}$, and the generator $G$ synthesizes the UAEs
$\\{(\tilde{x}_{g},y_{g})\\}$ based on the perturbation and the specified
class $y_{g}$. In G-D GAN, the discriminator $D$ aims to adversarially
distinguish the UAEs and the unobserved natural examples $\\{(x_{g},y_{g})\\}$
generated by $G$ from the true natural examples $\\{(x_{l},y_{l})\\}$ with
loss $\mathcal{L}^{G,D}_{gan}$, while in C-D GAN, $D$ aims to adversarially
distinguish the pseudo labeled examples $\\{(x_{c},y_{c})\\}$ generated by the
target classifier $C$ from $\\{(x_{l},y_{l})\\}$ with loss
$\mathcal{L}^{C,D}_{gan}$. The extended AT is conducted between $A$ and $C$
with the adversarial loss $\mathcal{L}^{C,A}_{adv}$ plus the supervised loss
$\mathcal{L}^{C}_{sup}$.

##  2 Preliminaries and Problem Formulation

In this section, we first give the formal definition of UAE, then present our
understanding of UAEs, and finally, formulate the AT on UAEs.

###  2.1 Definition of UAE

Let $x\in\mathcal{X}$ and $y\in\mathcal{Y}$ be an example and a label,
respectively, where $\mathcal{X}$ represents the data space with distribution
$P(x)$ while $\mathcal{Y}$ the label space with distribution $P(y)$. Let
$o(\cdot):\mathcal{O}\subseteq\mathcal{X}\rightarrow\mathcal{Y}$ be an oracle
which is defined on its domain $\mathcal{O}$ and tells the true label of an
example $x\in\mathcal{O}$. Usually, the oracle $o$ corresponds to humans and
$\mathcal{O}$ consists of all the examples that look natural to and can not
confuse humans. Let $C(\cdot):\mathcal{X}\rightarrow\mathcal{Y}$ be the target
classifier. Following the idea of [16], we define UAE as follow:

######  Definition 1 (UAE).

For a well-trained classifier $C$, an example $\tilde{x}$ with label
$y=o(\tilde{x})$ is an unrestricted adversarial example if $\tilde{x}\sim
P(x|y)$ and $C(\tilde{x})\neq y$, where $P(x|y)$ is the class likelihood.

###  2.2 Understanding of UAE

From Definition 1 we can see that similar to RAE, the harmfulness of a UAE is
still defined by its ability to fool the classifier, i.e., $C(\tilde{x})\neq
y$. But in contrast to RAE, the imperceptibility of a UAE is due to its
capability of being drawn from the realistic distribution ($\tilde{x}\sim
P(x|y)$) rather than a norm-constraint. However, it is challenging how to
enforce both imperceptibility and harmfulness for UAE. For this purpose, we
understand UAE from the viewpoint of imperceptibly perturbed unobserved
examples. In particular, for a natural example $x\sim P(x|y)$, there exists a
perturbation $z_{a}$ satisfying the imperceptibility $\tilde{x}=G(x,z_{a})\sim
P(x|y)$, and the harmfulness $C(\tilde{x})\neq y$, where $G$ is a learnable
generative function. Obviously, $G$ can be non-linear and the perturbation
$z_{a}$ is not necessarily additive. Following this idea, we can generate a
UAE through two steps: drawing natural example and generating appropriate
perturbation for it. It is noteworthy that such understanding brings the
advantage that RAE can be viewed as a special UAE and UAEs are a superset of
RAEs, thus enabling the adversarial robustness against UAEs to cover RAEs.

###  2.3 Adversarial Training on UAEs

To offer the comprehensive adversarial robustness to the target classifier, we
want to conduct adversarial training on UAEs, which is defined as the
following min-max game:

$\min_{C}\max_{\mathcal{A}}\mathcal{L}_{adv}=\frac{1}{|\mathcal{A}|}\sum_{(\tilde{x}_{g},y_{g})\in\mathcal{A}}\mathcal{L}_{c}\big{(}\tilde{x}_{g},y_{g}\big{)},$
(1)

where $\mathcal{L}_{adv}$ is the adversarial loss, $\mathcal{L}_{c}(x,y)$ is a
loss function of the target classifier, e.g., cross-entropy, $\mathcal{A}$ is
a labeled UAE set, and $\tilde{x}_{g}$ is a generated UAE with label $y_{g}$.
In Equation (1), the inner maximization seeks the most harmful UAE set
$\mathcal{A}$ resulting in the poorest performance of the target classifier,
while the outer minimization aims at adjusting the target classifier to reduce
its sensitivity to UAEs. It is noteworthy that unlike traditional AT where AEs
are searched explicitly with respect to a norm-constraint, the AT on UAEs
defined by Equation (1) does not tell us how to find out the qualified UAEs
that can reconcile the adversarial robustness and standard generalizability of
the target classifier, which is another key challenge we want to address in
this paper.

##  3 Proposed Method

In this section, we first give an overview of our Provable Unrestricted
Adversarial Training (PUAT) model, and then describe its components and
learning procedure in detail.

###  3.1 Overview of PUAT

As shown in Figure 2, PUAT consists of four components, an attacker $A$, a
conditional generator $G$, a discriminator $D$ and the target classifier $C$.
$A$ and $G$ constitute the UAE generation module. According to our idea that a
UAE can be regarded as the perturbed version of an unobserved natural example,
we first use $G$ to produce a labeled example $(x_{g},y_{g})\sim P_{G}(x,y)$
representing an unobserved natural example, where $P_{G}(x,y)$ is the
distribution of the data-label pairs generated by $G$ and the randomness stems
from a white noise $z$. Then we invoke $A$ and $G$ to generate its perturbed
version $(\tilde{x}_{g},y_{g})$ as a UAE, where the perturbation $z_{a}$ is
produced by $A$ based on the same $z$. To ensure the harmfulness of the UAEs,
PUAT will try to adjust $A$ via the AT between $A$ and $C$ so that the
generated UAEs can fool the target classifier $C$, i.e. $C(\tilde{x}_{g})\neq
y_{g}$. Thus the UAEs follow the conditional distribution $P_{G}(x,y|C(x)\neq
y)$. At the same time, to ensure the imperceptibility of the generated UAEs,
we want to align the distribution $P_{G}(x,y)$ with the true data distribution
$P(x,y)$. For this purpose, we introduce a discriminator $D$ to distinguish
the examples $\\{(x_{g},y_{g})\\}$, which includes the UAEs
$\\{(\tilde{x}_{g},y_{g})\\}$, from the true labeled examples
$\\{(x_{l},y_{l})\sim P(x,y)\\}$, which together with $G$ constitutes a class-
conditional GAN, called G-D GAN. The optimization of the G-D GAN will adjust
$G$ so that the distribution $P_{G}(x,y)$ converges to $P(x,y)$. The alignment
of $P_{G}(x,y)$ and the true data distribution $P(x,y)$ will make a UAE drawn
from $P_{G}(x,y)$ look like a natural example, which results in the
imperceptibility of UAEs.

However, in traditional conditional GAN, the discriminator $D$ also plays the
role to predict the class label of $x_{g}$, for which the optimization
objective conflicts with that for distinguishing $\\{(x_{g},y_{g})\\}$ from
$\\{(x_{l},y_{l})\\}$. Such conflict will lead to a suboptimal $D$ [26, 27],
and consequently weaken the ability of the G-D GAN to accurately align
$P_{G}(x,y)$ with $P(x,y)$. To alleviate this issue, we introduce another
conditional GAN, called C-D GAN, which together with the G-D GAN constitutes
the G-C-D GAN. The C-D GAN consists of the target classifier $C$ and the
discriminator $D$, where $C$ aims at predicting the label $y_{c}$ for
unlabeled examples $\\{x_{c}\sim P(x)\\}$ to build pseudo-labeled examples
$\\{(x_{c},y_{c})\\}$ that can fool $D$, while $D$ tries to distinguish
$\\{(x_{c},y_{c})\\}$ from true examples $\\{(x_{l},y_{l})\\}$. The
adversarial game between $C$ and $D$ improves the ability of $D$ to identify
whether a label matches an example $x$. As $D$’s role in C-D GAN is consistent
with its role in G-D GAN, $D$ strengthened by C-D GAN will ultimately help $G$
capture the true data distribution $P(x,y)$ more accurately.

At last, to realize the comprehensive adversarial robustness of the target
classifier $C$, we want to align the $P_{G}(x,y)$ with the distribution
$P_{C}(x,y)$ learned by $C$, which equivalently improves the robust
generalizability of $C$ on UAEs. For this purpose, we conduct AT between the
attacker $A$ and $C$, with an adversarial loss $\mathcal{L}^{A,C}_{adv}$ over
the generated UAEs. At the same time, to simultaneously improve the standard
generalizability of $C$ on natural examples, we further enforce the
distribution $P_{C}(x,y)$ to be aligned with the true data distribution
$P(x,y)$, by extending the AT with minimization of the supervised loss
$\mathcal{L}^{C}_{sup}$ of $C$ over the labeled data $\\{(x_{l},y_{l})\\}$.
The overall logic of PUAT can be summarized as follows:

  1. (1)

For the imperceptibility of UAEs, $P_{G}(x,y)$ is aligned with the true data
distribution $P(x,y)$, denoted by $P_{G}(x,y)=P(x,y)$, via the G-C-D GAN.

  2. (2)

The harmfulness of UAEs is offered by the maximization of
$\mathcal{L}^{A,C}_{adv}$ during the AT.

  3. (3)

The adversarial robustness is offered by $P_{C}(x,y)=P_{G}(x,y)$, via the
inner minimization of $\mathcal{L}^{A,C}_{adv}$ of the AT between $A$ and $C$.

  4. (4)

The standard generalizability is offered by $P_{C}(x,y)=P(x,y)$, via the
minimization of $\mathcal{L}^{C}_{sup}$.

It is noteworthy that traditional AT methods only realize (3) and (4), which
causes $P_{C}(x,y)$ to be a mix of $P_{G}(x,y)$ and $P(x,y)$ and consequently
incurs a tradeoff between adversarial robustness and standard
generalizability. In sharp contrast with the traditional methods, PUAT
eliminates such tradeoff because (1) ($P_{G}(x,y)=P(x,y)$), together with (3)
and (4), results in the equivalence between adversarial robustness
($P_{C}(x,y)=P_{G}(x,y)$) and standard generalizability ($P_{C}(x,y)=P(x,y)$),
i.e., $P_{C}(x,y)=P_{G}(x,y)=P(x,y)$.

###  3.2 UAE Generation

Our idea to generate a UAE $(\tilde{x}_{g},y_{g})$ is to perturb a generated
natural example. Let $P(y)$ be the label distribution and $P_{\epsilon}(z)$ be
a standard normal distribution. At first, we sample a label $y_{g}\sim P(y)$,
and a seed noise $z\sim P_{\epsilon}(z)$. Then we invoke $G$ to generate an
example $(x_{g},y_{g})\sim P_{G}(x,y)$, where

$x_{g}=G(z,y_{g}).$ (2)

Note that since the label distribution $P(y)$ is unknown, we use the labels
appearing in the labeled dataset $\mathcal{D}_{l}=\\{(x_{l},y_{l})\sim
P(x,y)\\}$ to approximate it. In particular, the label $y_{l}$ of each example
in $\mathcal{D}_{l}$ will serve as a $y_{g}$ once.

To generate the UAE corresponding to $(x_{g},y_{g})$, we further feed
$(z,y_{g})$ into the attacker $A$ to produce the perturbation,

$z_{a}=A(z,y_{g}).$ (3)

Next, $z_{a}$ together with $y_{g}$ is fed into the generator $G$ to produce
perturbed data

$\tilde{x}_{g}=G(z_{a},y_{g}).$ (4)

Finally, $\tilde{x}_{g}$ and $y_{g}$ together form a labeled UAE
$(\tilde{x}_{g},y_{g})$.

It is noteworthy that that $(x_{g},y_{g})$ can be regarded as an unobserved
natural example if $P_{G}(x,y)$ is aligned with the true data distribution
$P(x,y)$. For each $(x_{g},y_{g})$, we will generate a corresponding UAE
$(\tilde{x}_{g},y_{g})$, which is actually the perturbed $(x_{g},y_{g})$ as
they are generated based on the same $(z,y_{g})$ with the only difference that
whether $z$ or its perturbed version $z_{a}=A(z,y_{g})$ is fed into $G$. At
the same time, as we will see later, during the AT between A and C, $A$ will
be adjusted to ensure $(\tilde{x}_{g},y_{g})$ be an adversarial example that
is able to attack the classifier $C$, i.e., $C(\tilde{x}_{g})\neq y_{g}$.
Therefore, the UAEs $\\{(\tilde{x}_{g},y_{g})\\}$ are a subset of
$\\{(x_{g},y_{g})\\}$ such that $(\tilde{x}_{g},y_{g})\sim P_{G}(x,y|C(x)\neq
y)$, and essentially, $A$ plays the role of finding out
$\\{(\tilde{x}_{g},y_{g})\\}$ from $\\{(x_{g},y_{g})\\}$.

###  3.3 Distribution Alignment for Imperceptibility

As mentioned before, the imperceptibility of UAEs results from the alignment
of the distribution $P_{G}(x,y)$ with the true data distribution $P(x,y)$,
which is realized through the cooperation of G-D GAN and C-D GAN. Let
$\mathcal{D}_{l}=\\{(x_{l},y_{l})\sim P(x,y)\\}$ be a labeled natural dataset,
where $y_{l}=o(x_{l})$. Then the optimization objective of G-D GAN is defined
as

$\min_{G}\max_{D}\mathcal{L}^{G,D}_{gan}=\mathcal{L}_{D}+\mathcal{L}_{G},$ (5)

where

$\displaystyle\mathcal{L}_{D}$ $\displaystyle=\mathbb{E}_{(x_{l},y_{l})\sim
P(x,y)}\log D(x_{l},y_{l})$ (6)
$\displaystyle=\frac{1}{|\mathcal{D}_{l}|}\sum_{(x_{l},y_{l})\in\mathcal{D}_{l}}\log
D(x_{l},y_{l})$

and

$\displaystyle\mathcal{L}_{G}$ $\displaystyle=\mathbb{E}_{(x_{g},y_{g})\sim
P_{G}(x,y)}\log\big{(}1-D(x_{g},y_{g})\big{)}$ (7)
$\displaystyle=\frac{1}{|\mathcal{D}_{l}|}\sum_{(x_{l},y_{l})\in\mathcal{D}_{l}}\log\bigg{(}1-D\Big{(}G\big{(}z,y_{l}\big{)},y_{l}\Big{)}\bigg{)}.$

In Equation (7), the label $y_{l}$ of each example
$(x_{l},y_{l})\in\mathcal{D}_{l}$ serves as $y_{g}\sim P(y)$. For each
$y_{l}$, we first sample a noise $z\sim P_{\epsilon}(z)$ and then invoke
$G\big{(}z,y_{l}\big{)}$ to generate $x_{g}$, which together with $y_{l}$
forms a data-label pair $(x_{g},y_{g})$.

In G-D GAN, the generation of a data-label pair $(x_{g},y_{g})$ can be
understood as the procedure of first drawing $y_{g}\sim P(y)$ and then drawing
$x_{g}|y_{g}\sim P_{G}(x|y)$, where $P_{G}(x|y)$ is the conditional
distribution characterized by $G$. Therefore, $(x_{g},y_{g})$ $\sim$
$P_{G}(x,y)$ $=P(y)P_{G}(x|y)$. As $y_{g}$ is drawn from the true label
distribution $P(y)$, to adversarially distinguish $(x_{g},y_{g})$ from true
examples, what $D$ should do is to identify whether $x_{g}|y_{g}$ is true.
There is one of two reasons causing a false $x_{g}|y_{g}$. One is $x_{g}$ is
false, which asks $D$ to identify whether $x_{g}\sim P(x)$. The other is
although $x_{g}$ is true, it does not belong to class $y_{g}$, which asks $D$
to identify whether $y_{g}|x_{g}\sim P(y|x)$. However, as we have mentioned
before, these two tasks conflict with each other for $D$, which leads to a
suboptimal $D$ and consequently weakens $G$. To alleviate this issue, we
introduce C-D GAN and use an unlabeled natural dataset
$\mathcal{D}_{c}=\\{x_{c}\sim P(x)\\}$ to train it with the following
optimization objective:

$\min_{C}\max_{D}\mathcal{L}^{C,D}_{gan}=\mathcal{L}_{D}+\mathcal{L}_{C},$ (8)

where

$\displaystyle\mathcal{L}_{C}$ $\displaystyle=\mathbb{E}_{(x_{c},y_{c})\sim
P_{C}(x,y)}\log\big{(}1-D(x_{c},y_{c})\big{)}$ (9)
$\displaystyle=\frac{1}{|\mathcal{D}_{c}|}\sum_{x_{c}\in\mathcal{D}_{c}}\log\Big{(}1-D\big{(}x_{c},y_{c}=C(x_{c})\big{)}\Big{)}.$

To calculate $\mathcal{L}_{C}$, for each $x_{c}\in\mathcal{D}_{c}$, we invoke
the target classifier $C$ to label it with the prediction $y_{c}=C(x_{c})$,
which generates a data-label pair $(x_{c},y_{c})$. This is equivalent to the
procedure of first drawing $x_{c}\sim P(x)$ and then drawing $y_{c}|x_{c}\sim
P_{C}(y|x)$. Therefore $(x_{c},y_{c})\sim P_{C}(x,y)=P(x)P_{C}(y|x)$. C-D GAN
offers two benefits to $D$. The first is in C-D GAN, $C$ provides the
adversarial signal $y_{c}|x_{c}$ to help $D$ learn $P(y|x)$, which ultimately
promotes $D$’s performance in G-D GAN for a better $G$. The second is in both
C-D GAN and G-D GAN, $D$ plays the same role of distinguishing the generated
examples ($\\{(x_{g},y_{g})\\}$ or $\\{(x_{c},y_{c})\\}$) from true ones
($\\{(x_{l},y_{l})\\}$), which eliminates the conflict optimization of $D$ in
traditional conditional GAN.

We refer to the combination of G-D GAN and C-D GAN as G-C-D GAN. By combining
Equations (5) and (8), we obtain the following optimization objective for
G-C-D GAN:

$\min_{C,G}\max_{D}\mathcal{L}^{G,C,D}_{gan}=\mathcal{L}_{D}+\frac{1}{2}\mathcal{L}_{G}+\frac{1}{2}\mathcal{L}_{C}.$
(10)

###  3.4 Extended Adversarial Training

Since the harmfulness of the generated UAEs is caused by the attacker $A$, we
conduct the AT between $A$ and the target classifier $C$. Meanwhile, to
improve both the adversarial robustness and standard generalizability, we
extend the AT defined in Equation (1) by incorporating the supervised loss of
$C$, which leads to the following min-max game:

$\min_{C}\max_{A}\big{[}\mathcal{L}^{C}_{sup}+\lambda\mathcal{L}^{C,A}_{adv}\big{]},$
(11)

where $\lambda$ is a non-negative constant controlling the weight of
adversarial robustness and the standard generalizability. The supervised loss
of the target classifier $C$ is defined as

$\displaystyle\mathcal{L}^{C}_{sup}$ $\displaystyle=\mathbb{E}_{(x,y)\sim
P(x,y)}\big{[}-\log P_{C}(y|x)\big{]}$ (12)
$\displaystyle=\frac{1}{|\mathcal{D}_{l}|}\sum_{(x_{l},y_{l})\in\mathcal{D}_{l}}\big{[}-\log
P_{C}(y_{l}|x_{l})\big{]}.$

where the conditional probability $P_{C}(y|x)$ is the output of $C$. The
adversarial loss is defined as

$\displaystyle\mathcal{L}^{C,A}_{adv}$ $\displaystyle=\mathbb{E}_{(x,y)\sim
P_{G}(x,y)}\big{[}-\log P_{C}(y|\tilde{x})\big{]}$ (13)
$\displaystyle=\frac{1}{|\mathcal{A}|}\sum_{(\tilde{x},y)\in\mathcal{A}}\big{[}-\log
P_{C}(y|\tilde{x})\big{]},$

where $\mathcal{A}$ is the generated UAE set. Note that for each $(x,y)\sim
P_{G}(x,y)$, we generate its corresponding UAE $(\tilde{x},y)$ by invoking
Equations (3) and (4) (see Section 3.2).

In Equation (11), the inner maximization of $\mathcal{L}^{C,A}_{adv}$ enforces
$A$ to be adjusted so that a generated UAE $(\tilde{x}_{g},y_{g})$ is harmful
enough, i.e., $C(\tilde{x}_{g})\neq y_{g}$, which causes the UAE generation
module (consisting of $A$ and $G$) to converge to the underlying UAE
distribution $P_{G}(x,y|C(x)\neq y)$ which maximizes the adversarial loss. The
outer minimization realizes the alignment $P_{C}(x,y)=P_{G}(x,y|C(x)\neq y)$
by adjusting $C$, which results in $C$’s robust generalizability over UAEs
(i.e. the adversarial robustness). As proved later, the KL-divergence of
$P_{C}(x,y)$ and $P_{G}(x,y|C(x)\neq y)$ is the upper bound of the KL-
divergence of $P_{C}(x,y)$ and $P_{G}(x,y)$. Therefore, the minimization of
$\mathcal{L}^{C,A}_{adv}$ will approximately lead to $P_{C}(x,y)=P_{G}(x,y)$.
At the same time, one can note that the minimization of
$\mathcal{L}^{C}_{sup}$ will be achieved when $P_{C}(x,y)=P(x,y)$, which
offers $C$ the standard generalizability on natural examples. Later we will
prove that with the help of the G-C-D GAN (Equation(10)), the extended AT
defined by Equation(11) will achieve the optimal solution at
$P_{C}(x,y)=P_{G}(x,y)=P(x,y)$, which means the adversarial robustness and the
standard generalizability are satisfied simultaneously.

###  3.5 Joint Training

By combining Equations (10) and (11), we can obtain the following overall
optimization objective for joint learning of $A$, $G$, $C$ and $D$:

$\min_{C,G}\max_{A,D}\big{[}\mathcal{L}^{G,C,D}_{gan}+\mathcal{L}^{C}_{sup}+\lambda\mathcal{L}^{C,A}_{adv}\big{]}.$
(14)

Basically, Equation (14) integrates the adversarial generation of UAEs and the
AT over UAEs within a unified min-max game. Algorithm 1 gives the sketch of
the joint training, where the Lines 6 and 7 solve the inner maximization with
stochastic gradient ascent, while the Lines 8 and 9 solve the outer
minimization with stochastic gradient descent.

Algorithm 1 Training Algorithm of PUAT

0: Datasets $\mathcal{D}_{l}$ and $\mathcal{D}_{u}$;

0: Well-trained target classifier $C$;

1: Random initialize $A$, $C$, $G$, and $D$;

2: for $T$ epochs do

3: Generate UAE set $\mathcal{A}$ using Equations (3) and (4).

4: Generate examples $\\{(x_{g},y_{g})\sim P_{G}(x,y)\\}$ based on
$\mathcal{D}_{l}$ w.r.t. Equation (7).

5: Generate examples $\\{(x_{c},y_{c})\sim P_{C}(x,y)\\}$ based on
$\mathcal{D}_{u}$ w.r.t. Equation (9).

6: Update $D$ by ascending stochastic gradient
$\nabla_{D}(\mathcal{L}^{C,D}_{gan}$ $+\mathcal{L}^{G,D}_{gan})$;

7: Update $A$ by ascending stochastic gradient
$\nabla_{A}(\mathcal{L}^{C,A}_{adv})$;

8: Update $C$ by descending stochastic gradient
$\nabla_{C}(\mathcal{L}^{C,D}_{gan}+\mathcal{L}^{C}_{supC}+\lambda\mathcal{L}^{C,A}_{adv})$;

9: Update $G$ by descending stochastic gradient
$\nabla_{G}(\mathcal{L}^{G,D}_{gan})$;

10: end for

##  4 Theoretical Justification

In this section, we theoretically justify PUAT’s ability to simultaneously
improve adversarial robustness and standard generalizability of the target
classifier without compromising either of them. Recall that PUAT offers the
adversarial robustness by $P_{C}(x,y)=P_{G}(x,y)$, while the standard
generalizability by $P_{C}(x,y)=P(x,y)$. As Equation (14) defines a unified
framework integrating the UAE generation with the AT against UAEs, proving
PUAT’s ability to eliminate the tradeoff is equivalent to proving that the
optimal solution of Equation (14) will be achieved at
$P_{C}(x,y)=P_{G}(x,y)=P(x,y)$, which means the adversarial robustness and the
standard generalizability agree with each other under the framework of
Equation (14). For the brevity of later derivations, in the following text
$P(x,y)$, $P_{G}(x,y)$ and $P_{C}(x,y)$ will be abbreviated to $P$, $P_{G}$
and $P_{C}$, respectively, when the context is unambiguous. We first give the
equilibrium of the G-C-D GAN by the following lemma:

######  Lemma 1.

The optimal solution ($G^{*}$, $C^{*}$, $D^{*}$) of $\min_{C,G}$ $\max_{D}$
$\mathcal{L}^{G,C,D}_{gan}$ (Equation (10)) is achieved at $p$
$=\frac{1}{2}(P_{G}$ $+P_{C})$.

###### Proof.

Let $P_{GC}=\frac{1}{2}(P_{G}$ $+P_{C})$. According to Equations (6), (7), (9)
and (10), it is easy to see

$\displaystyle\mathcal{L}^{G,C,D}_{gan}=$
$\displaystyle\iint{P\log{D}dxdy}+\iint{P_{GC}\log(1-D)dxdy}.$ (15)

Fixing $G$ and $C$, the optimal discriminator [28]

$D^{*}=\operatorname*{argmax}_{D}\mathcal{L}^{G,C,D}_{gan}=\frac{P}{P+P_{GC}}.$
(16)

Pluging $D^{*}$ in Equation (15) and by some tedious derivations, we can get

$\displaystyle\mathcal{L}^{G,C,D^{*}}_{gan}$
$\displaystyle=\iint\Bigg{[}P\log\Big{(}\frac{P}{\frac{1}{2}(P+P_{GC})}\Big{)}$
(17)
$\displaystyle+P_{GC}\log\Big{(}\frac{P_{GC}}{\frac{1}{2}(p+P_{GC})}\Big{)}\Bigg{]}dxdy-2\log
2$ $\displaystyle=\text{JS}(P\parallel P_{GC})-2\log 2,$

where JS is Jensen-Shannon divergence. Since $\text{JS}(p\parallel P_{GC})\geq
0$, $G^{*}$ and $C^{*}$ are achieved at JS $(P$ $\parallel$ $P_{GC})$ $=0$,
which leads to $P=P_{GC}=\frac{1}{2}(P_{G}$ $+P_{C})$. ∎

Now we show that $\min_{C}\mathcal{L}^{C}_{sup}$ and $\min_{C}$$\max_{A}$
$\mathcal{L}^{C,A}_{adv}$ are two consistent optimization problems with the
same optimal solution $C$ at which $P_{C}(x,y)=P_{G}(x,y)=P(x,y)$.

######  Theorem 1.

In Equation (11), $\min_{C}\mathcal{L}^{C}_{sup}$ and $\min_{C}$$\max_{A}$
$\mathcal{L}^{C,A}_{adv}$ can be achieved at the same optimal $C$ subjected to
$P_{C}(x,y)$ $=P_{G}(x,y)=P(x,y)$.

###### Proof.

(1) By the definition of $\mathcal{L}^{C}_{sup}$ (Equation (12)), the optimal
$C$ for $\min_{C}\mathcal{L}^{C}_{sup}$ is

$\displaystyle{}\operatorname*{argmin}_{C}$
$\displaystyle\mathcal{L}^{C}_{sup}$ $\displaystyle=$
$\displaystyle{}\operatorname*{argmin}_{C}$
$\displaystyle\iint\Big{[}-P(x,y)\log\big{(}P_{C}(y|x)P(x)\big{)}+$
$\displaystyle P(x,y)\log P(x,y)\Big{]}dxdy$ $\displaystyle=$
$\displaystyle{}\operatorname*{argmin}_{C}$ $\displaystyle\iint
P(x,y)\log\frac{P(x,y)}{P_{C}(x,y)}dxdy$ $\displaystyle=$
$\displaystyle{}\operatorname*{argmin}_{C}$
$\displaystyle\text{KL}\big{(}P(x,y)\parallel P_{C}(x,y)\big{)},$

where KL is Kullback–Leibler divergence, the first equality holds because
$P(x)$ and $P(x,y)$ are independent of $C$ without changing the monotonicity
of $\mathcal{L}^{C}_{sup}$, and the second equality holds because
$P_{C}(x,y)=P(x)P_{C}(y|x)$. Therefore, $\min_{C}\mathcal{L}^{C}_{sup}$ is
achieved at $P_{C}(x,y)=P(x,y)$. Note that when $C$ is adjusted with respect
to $P_{C}(x,y)=P(x,y)$, G-C-D GAN (Equation (10)) will synchronically adjust
$G$ with respect to $P_{G}(x,y)=P(x,y)$, to keep the equilibrium of $G$, $C$
and $D$ (Lemma 1). We hence can safely substitute $P(x,y)$ for $P_{C}(x,y)$ in
$P(x,y)=\frac{1}{2}\big{(}P_{G}(x,y)$ $+P_{C}(x,y)\big{)}$ (Lemma 1), which
follows that

$P_{C}(x,y)=P_{G}(x,y)=P(x,y).$

(2) According to Equation (13), we have

$\displaystyle{}\operatorname*{argmin}_{C}$
$\displaystyle\max_{A}\mathcal{L}^{C,A}_{adv}$ (18) $\displaystyle=$
$\displaystyle{}\operatorname*{argmin}_{C}$ $\displaystyle\max_{A}\iint-
P_{G}(x,y)\log P_{C}(y|\tilde{x})dxdy$ (19) $\displaystyle=$
$\displaystyle{}\operatorname*{argmin}_{C}$
$\displaystyle\max_{A}\iint\Big{[}-P_{G}(x,y)\log P_{C}(\tilde{x},y)$ (21)
$\displaystyle+P_{G}(x,y)\log P_{G}(x,y)\Big{]}dxdy$ $\displaystyle=$
$\displaystyle{}\operatorname*{argmin}_{C}$ $\displaystyle\max_{A}\iint
P_{G}(x,y)\log\frac{P_{G}(x,y)}{P_{C}(\tilde{x},y)}dxdy$ (22) $\displaystyle=$
$\displaystyle{}\operatorname*{argmin}_{C}$
$\displaystyle\max_{A}\text{KL}\big{(}P_{G}(x,y)\parallel
P_{C}(\tilde{x},y)\big{)}.$ (23)

Obviously,

$\displaystyle{}\max_{A}$ $\displaystyle\iint
P_{G}(x,y)\log\frac{P_{G}(x,y)}{P_{C}(\tilde{x},y)}dxdy$ $\displaystyle{}\geq$
$\displaystyle\iint P_{G}(x,y)\log\frac{P_{G}(x,y)}{P_{C}(x,y)}dxdy$
$\displaystyle{}=$ $\displaystyle\text{KL}\big{(}P_{G}(x,y)\parallel
P_{C}(x,y)\big{)},$

which indicates that $\min_{C}$$\max_{A}$ $\mathcal{L}^{C,A}_{adv}$
essentially minimizes the upper-bound of the KL-divergence of $P_{G}$$(x,y)$
and $P_{C}$$(x,y)$.

Let $P_{G|A}(x,y)=P_{G}(x,y|C(x)\neq y)$ be the UAE distribution. Noting that
the UAEs $\\{(x,y)\sim P_{G|A}(x,y)\\}$ maximize the adversarial loss
$\mathcal{L}^{C,A}_{adv}$ (Equation (13)), which are found out by $A$, we
hence also have

$\displaystyle{}\operatorname*{argmin}_{C}$
$\displaystyle\max_{A}\mathcal{L}^{C,A}_{adv}$ (24) $\displaystyle=$
$\displaystyle{}\operatorname*{argmin}_{C}$ $\displaystyle\iint-
P_{G|A}(x,y)\log P_{C}(y|x)dxdy$ (25) $\displaystyle=$
$\displaystyle{}\operatorname*{argmin}_{C}$ $\displaystyle\iint-
P_{G|A}(x,y)\log P_{C}(x,y)dxdy$ (26) $\displaystyle=$
$\displaystyle{}\operatorname*{argmin}_{C}$
$\displaystyle\iint\Big{[}-P_{G|A}(x,y)\log P_{C}(x,y)$ (28)
$\displaystyle+P_{G|A}(x,y)\log P_{G|A}(x,y)\Big{]}dxdy$ $\displaystyle=$
$\displaystyle{}\operatorname*{argmin}_{C}$ $\displaystyle\iint
P_{G|A}(x,y)\log\frac{P_{G|A}(x,y)}{P_{C}(x,y)}dxdy$ (29) $\displaystyle=$
$\displaystyle{}\operatorname*{argmin}_{C}$
$\displaystyle\text{KL}\big{(}P_{G|A}(x,y)\parallel P_{C}(x,y)\big{)}.$ (30)

Considering Equations (23) and (30), we can see that aligning $P_{G|A}(x,y)$
with $P_{C}(x,y)$ (i.e., minimizing KL $\big{(}P_{G|A}(x,y)$ $\parallel
P_{C}(x,y)$$\big{)}$) is equivalent to minimizing the upper-bound of
$\text{KL}\big{(}P_{G}(x,y)\parallel P_{C}(x,y)\big{)}$, which approximately
leads to $P_{C}(x,y)=P_{G}(x,y)$ and also results in

$P_{C}(x,y)=P_{G}(x,y)=P(x,y).$

(3) The part (1) and part (2) together tell us both
$\min_{C}\mathcal{L}^{C}_{sup}$ and
$\min_{C}$$\max_{A}$$\mathcal{L}^{C,A}_{adv}$ will lead to $P_{C}(x,y)$ $=$
$P_{G}(x,y)$ $=$ $P(x,y)$. Therefore, they are consistent optimization
problems with the same optimal solution $C$. ∎

Theorem 1 together with Lemma 1 tells us that $G$, $C$ and $D$ will reach
their equilibrium when $P_{C}(x,y)=P_{G}(x,y)=P(x,y)$, which implies that the
adversarial robustness ($P_{C}(x,y)=P_{G}(x,y)$) and the standard
generalizability ($P_{C}(x,y)=P(x,y)$) can be simultaneously achieved by PUAT.

##  5 Experiments

The goal of experiments is to answer the following research questions:

  * •

RQ1 How does PUAT perform as compared to the state-of-the-art baselines in
terms of the standard generalizability and comprehensive adversarial
robustness?

  * •

RQ2 Can PUAT provide comprehensive adversarial robustness against black-box
attacks?

  * •

RQ3 How does the sensitivity of PUAT to the hyper-parameter $\lambda$ in
Equation (14) show that the tradeoff between standard generalizability and
adversarial robustness is eliminated by PUAT?

  * •

RQ4 How to visually show the superiority of PUAT?

###  5.1 Experimental Setting

####  5.1.1 Datasets

We conduct the experiments on CIFAR10 [29] and SVHN [30], which are widely
used benchmarks for evaluating adversarial training. CIFAR10 contains 60,000
images distributed across 10 classes, while SVHN is a set of 99,559 digit
images. Each dataset is randomly divided into two parts, labeled data and
unlabeled data, where unlabeled data are built by removing the labels of the
images. The labeled data are further randomly divided into training set,
validation set, and testing set. Table I shows the dataset configurations. We
will repeat the random division of the dataset three times, and report the
average results with standard deviation.

TABLE I: Configuration of the datasets. Datasets | unlabeled data | labeled data
---|---|---
training | training | validation | testing
CIFAR10 | 46,000 | 4,000 | 1,000 | 9,000
SVHN | 72,257 | 1,000 | 250 | 25,782
TABLE II: Characteristics of the baseline methods and PUAT. Methods | RAE | UAE |  distribution alignment |  using labeled data |  using unlabeled data
---|---|---|---|---|---
Regular-training |  |  |  | $\checkmark$ |
TRADES [31] | $\checkmark$ |  |  | $\checkmark$ |
GAT [14] | $\checkmark$ |  |  | $\checkmark$ |
GPGD-AT [32] |  | $\checkmark$ |  | $\checkmark$ |
RST [33] | $\checkmark$ |  |  | $\checkmark$ | $\checkmark$
PUAT |  | $\checkmark$ | $\checkmark$ | $\checkmark$ | $\checkmark$

####  5.1.2 Baseline Methods

To verify the superiority of PUAT, we compare it with the following baseline
methods, whose characteristics are summarized in Table II.

  * •

Regular-training Regular-training method trains the target classifier using
natural data with cross-entropy loss, which plays the role of performance
benchmark.

  * •

TRADES [31] TRADES is an AT method based on RAEs that are generated by PGD
based on labeled data, which can trade adversarial robustness off against
standard generalizability by optimizing a regularized surrogate loss.

  * •

GAT [14] GAT is an AT method based on RAEs that are generated by a GAN, where
the generator produces explicitly norm-constrained adversarial perturbations
for natural labeled examples with respect to the gradients of the target
classifier loss.

  * •

GPGD-AT [32] GPGD-AT is an AT method based on UAEs that are generated by GPGD
[32]. To generate a UAE, GPGD first employs a conditional GAN to produce an
unobserved natural labeled example, and then adds adversarial perturbation to
it based on the gradient of the target classifier loss incurred by it.

  * •

RST [33] RST is also an AT method based on RAEs that are generated by PGD.
Different from TRADES, RST first trains a classifier to predict labels for
unlabeled data and then feeds these pseudo-labeled data together with labeled
data into the sequel adversarial training algorithm.

Additionally, to verify the contribution of unlabeled data, we also compare
PUAT with its variant PUAT-L, which does not use unlabeled data
$\mathcal{D}_{c}$ in Equation (9) but instead uses $x_{l}$ in labeled data
$\mathcal{D}_{l}$ as $x_{c}$ to compute $\mathcal{L}_{C}$.

TABLE III: Hyper-parameter setting of the attacking methods. Attacks |  Hyper-parameter setting
---|---
FGSM [8] |  $\epsilon=0.0031$; step size = 0.007.
PGD [9] |  $\epsilon=0.0031$; step size = 0.007; number of iterations = 10.
GPGD [32] |  $\epsilon=0.1$ on CIFAR10, 0.3 on SVHN; step size = 0.007; number of iterations = 10.
USong [16] |  $\epsilon=0.1$ on CIFAR10, 0.3 on SVHN; step size = 0.1, number of iterations = 200; $\lambda_{1}=100,\lambda_{2}=100$.
TABLE IV: Model architecture of $C$, $D$, $G$ and $A$ of PUAT. Classifier C | Discriminator D | Generator G | Attacker A
---|---|---|---
Input: $3\times 32\times 32$ image $x$ |  Input: $3\times 32\times 32$ image $x$, label $y$ |  Input: noise $z$, label $y$ |  Input: noise $z$, label $y$
$3\times 3$ conv, 128, lReLU, weight norm $3\times 3$ conv, 128, lReLU, weight norm $3\times 3$ conv, 128, lReLU, weight norm $2\times 2$ max-pooling, 0.5 dropout |  2-layer $3\times 3$ residual block, 512, ReLU, spectral norm downsample $512\times 16\times 16$ 2-layer $3\times 3$ residual block, 512, ReLU, spectral norm |  MLP $8\times 64\times 16$ units reshape $512\times 4\times 4$ upsample $512\times 8\times 8$ 2-layer residual block, $3\times 3$ conv, 512, ReLU, batch norm |  MLP $256+256$ units ReLU
$3\times 3$ conv, 128, lReLU, weight norm $3\times 3$ conv, 128, lReLU, weight norm $3\times 3$ conv, 128, lReLU, weight norm $2\times 2$ max-pooling, 0.5 dropout |  downsample $512\times 8\times 8$ 2-layer $3\times 3$ residual block, 512, ReLU, spectral norm |  upsample $512\times 16\times 16$ 2-layer residual block, $3\times 3$ conv, 512, ReLU, batch norm
$3\times 3$ conv, 512, lReLU, weight norm NIN, 256, lReLU, weight norm NIN, 128, lReLU, weight norm global pool |  downsample $512\times 4\times 4$ 2-layer $3\times 3$ residual block, 512, ReLU, spectral norm global pool |  upsample $512\times 32\times 32$ 2-layer residual block, $3\times 3$ conv, 512, ReLU, batch norm
10-class softmax, weight norm |  MLP 1 units, spectral norm |  $1\times 1$ conv, 3, tanh, batch norm |  MLP $256$ units, ReLU

####  5.1.3 Evaluation Protocol

We will use PUAT and the baseline methods to train the target classifier and
compare their performances from the perspectives of the target classifier’s
standard generalizability and adversarial robustness. The generalizability is
evaluated in terms of the target classifier’s natural accuracy on testing
natural examples, while the adversarial robustness is evaluated in terms of
the target classifier’s robust accuracy on the testing adversarial examples
which are generated by various attack methods. In particular, for the
robustness against RAE, we choose FGSM [8] and PGD [9] to generate testing
RAEs, which are the attacking methods widely adopted by the existing works.
For the robustness against UAE, we choose GPGD [32] and USong [16] as the
attacking methods, both of which are GAN-based methods and dedicated to the
generation of UAEs. The hyper-parameters of the attacking methods are set to
their optimal values, which are shown in Table III. To evaluate the
performance of PUAT and the baseline methods, we first invoke the attacking
methods to generate testing AEs that can cause worst-case loss of the target
classifier well trained by PUAT or a baseline method, and then check the
classifier’s accuracy on those testing AEs.

To comprehensively evaluate the performance of PUAT and the baseline methods,
we will investigate the robustness of the well-trained target classifier under
two threat models, i.e., white-box attack and black-box attack. To examine a
classifier’s robustness under white-box attacks, we first invoke the attacking
methods to generate adversarial examples (RAEs or UAEs) that can cause worst-
case loss of the target classifier trained by PUAT or a baseline method, and
then check the classifier’s accuracy on those adversarial examples. To carry
out the black-box attacks, each classifier trained by PUAT or a baseline
method will also play as a surrogate model of the classifiers trained by the
other methods. To examine a classifier’s robustness under black-box attacks,
we first invoke the attacking methods to conduct while-box attacks on its
surrogate models, and then check its accuracy on the adversarial examples
generated for the white-box attacks.

####  5.1.4 Hyper-parameter Setting and Architecture

The hyper-parameters are tuned on validation dataset. In particular, the
$\lambda$ in Equation 14 is set to 1. The perturbation constraint $\epsilon$
in TRADES, GAT and RST is set to 0.031. The perturbation step size and the
number fof iterations in TRADES and RST are set to 0.007 and 10, respectively.
The architecture of PUAT is shown in Table IV, where the generator $G$,
discriminator $D$, and target classifier $C$ are implemented with respect to
those adopted in [26]. In the baseline methods, the architecture of the target
classifiers is as the same as that of $C$.

TABLE V: Performance under white-box attacks on CIFAR10. Methods |  Natural Accuracy | Robust Accuracy
---|---|---
RAE Attack | UAE Attack
FGSM | PGD | GPGD | USong
Regular-training | $70.52\pm 1.62$ | $29.43\pm 1.10$ | $24.97\pm 1.08$ | $28.21\pm 3.83$ | $44.83\pm 5.08$
TRADES | $65.59\pm 2.25$ | $57.09\pm 0.93$ | $55.65\pm 0.96$ | $19.71\pm 1.24$ | $37.14\pm 2.12$
GAT | $69.68\pm 2.00$ | $50.57\pm 3.14$ | $44.94\pm 3.09$ | $27.01\pm 4.54$ | $45.59\pm 4.84$
GPGD-AT | $72.62\pm 0.77$ | $29.06\pm 1.03$ | $24.52\pm 1.22$ | $37.55\pm 2.66$ | $49.32\pm 0.75$
RST | $73.89\pm 0.22$ | $65.34\pm 0.82$ | $63.73\pm 0.81$ | $27.47\pm 0.99$ | $56.42\pm 2.23$
PUAT-L | $76.96\pm 0.89$ | $49.64\pm 2.11$ | $43.27\pm 1.97$ | $55.18\pm 1.19$ | $63.83\pm 2.43$
PUAT | $83.47\pm 0.60$ | $67.51\pm 0.10$ | $58.76\pm 0.29$ | $59.49\pm 2.14$ | $81.65\pm 0.51$
TABLE VI: Performance under white-box attacks on SVHN Methods |  Natural Accuracy | Robust Accuracy
---|---|---
RAE Attack | UAE Attack
FGSM | PGD | GPGD | USong
Regular-training | $86.12\pm 0.43$ | $52.16\pm 2.25$ | $50.51\pm 2.33$ | $58.77\pm 2.48$ | $80.42\pm 2.14$
TRADES | $80.86\pm 1.96$ | $57.15\pm 2.90$ | $56.51\pm 2.90$ | $27.56\pm 7.75$ | $62.79\pm 3.14$
GAT | $81.01\pm 2.35$ | $55.52\pm 0.81$ | $53.50\pm 0.77$ | $45.72\pm 4.85$ | $64.56\pm 3.64$
GPGD-AT | $77.99\pm 1.88$ | $44.16\pm 0.11$ | $42.23\pm 0.29$ | $49.42\pm 0.75$ | $58.34\pm 1.69$
RST | $87.59\pm 0.62$ | $62.69\pm 2.84$ | $62.04\pm 2.85$ | $44.78\pm 4.70$ | $79.19\pm 0.89$
PUAT-L | $87.64\pm 0.68$ | $59.17\pm 1.32$ | $56.75\pm 1.24$ | $70.41\pm 0.90$ | $88.14\pm 2.11$
PUAT | $90.35\pm 0.33$ | $67.08\pm 0.58$ | $64.33\pm 0.46$ | $77.39\pm 1.59$ | $95.52\pm 0.80$

###  5.2 Performance under White-box Attack (RQ1)

Tables V and VI show the results under white-box attacks on CIFAR10 and SVHN,
respectively, from which we can make the following observations:

  * •

On both datasets, PUAT surpasses all baselines in robust accuracy under all
assaults, with the exception of RST under PGD attack on CIFAR10, and achieves
the highest natural accuracy, surpassing even regular-training. PUAT is the
only AT approach that simultaneously increases standard generalizability and
adversarial robustness against all attacks on both datasets, in contrast to
baseline AT methods, which all produce at least one output that is subpar to
regular-training. By aligning the AE distribution and natural data
distribution, which results in a consistent generalization on AEs and natural
data, we contend that this finding demonstrates PUAT’s capacity to eliminate
the tradeoff between the standard generalizability and the adversarial
robustness.

  * •

We can see that, under the PGD attack, PUAT is superior to RST on SVHN but
inferior to RST on CIFAR10. As a result, RST’s robustness against PGD attack
is superior on datasets with dense labeled data, like CIFAR10, but inferior on
datasets with sparse labeled data, like SVHN. This is because RST uses PGD to
generate RAEs based on observed labeled examples, which makes it more robust
against PGD attack on dense datasets. In other words, because of its
distinctive generative model G-C-D GAN, PUAT can provide a stronger defense
than baseline methods especially on sparse datasets.

  * •

On both datasets, it is important to notice that PUAT beats nearly all
defensive methods against RAE attacks (provided by FGSM and PGD) and UAE
attacks (provided by GPGD and USong). The horizontal view, meanwhile,
demonstrates that PUAT performs comparably to both sorts of attacks. These
results show how effective PUAT is at providing comprehensive adversarial
robustness against RAE and UAE attacks. We contend that the comprehensive
adversarial robustness is made possible by our innovative notion to generalize
the UAE concept to imperceptibly perturbed natural examples, whether
observable or unobservable.

  * •

The RAE-based AT methods TRADES and GAT enhance robust accuracy under RAE
attacks at the expense of natural accuracy when compared to regular-training.
This demonstrates distribution-shift between natural data and AEs will result
from traditional AT methods that put norm-constraint on perturbations, which
leads to a tradeoff between the standard generalizability and the adversarial
robustness. They are also less accurate under UAE attacks than regular-
training, which shows that they are unable to provide adversarial robustness
against UAEs.

  * •

PUAT absolutely defeats GPGD-AT under UAE attacks on both datasets, despite
the fact that it is also an AT method based on UAE. This is mostly due to the
introduction of the C-D GAN and the unlabeled data for improved distribution
alignment in PUAT. Target classifier $C$ particularly aids the discriminator
$D$ in comprehending the distribution $P(y|x)$ via C-D GAN, and finally via
G-D GAN aids the generator $G$ in accurately capturing the natural data
distribution. In contrast, GPGD-AT only uses one conditional GAN, which
prevents it from properly aligning the UAE distribution with the natural data
distribution. Meanwhile, it is noteworthy that GPGD-AT, which relies on
labeled data, performs even worse under UAE attacks on the sparse dataset SVHN
than regular-training.

  * •

When it comes to natural accuracy and robust accuracy against RAE attacks, RST
is the second-best method after PUAT. When compared to other RAE-based
methods, RST’s generalization over natural examples and RAEs is superior
because it introduces unlabeled examples to help it achieve a tighter upper-
bound of the tradeoff. This method also mitigates the tradeoff between
standard generalizability and adversarial robustness. The tradeoff hasn’t gone
away, though, and we can see that because of RAE-based AT’s fundamental
limitations, RST could hardly defend against UAE attacks on either dataset.

  * •

Finally, we observe that PUAT-L performs significantly worse than PUAT, which
validates the contribution of unlabeled examples to PUAT. As previously
mentioned, PUAT trains the target classifier to provide pseudo labels for
unlabeled examples. This aids the G-C-D GAN in accurately understanding
$P(y|x)$ and, in turn, enables the target classifier to generalize
consistently over aligned distributions of natural data and AEs. However, due
to G-C-D GAN’s capability to align the UAE distribution with natural data
distribution, PUAT-L is still superior to the other baseline methods in
natural accuracy and robust-accuracy under UAE attacks.

###  5.3 Performance under Black-box Attack (RQ2)

TABLE VII: Adversarial robustness against black-box attacks of PGD on CIFAR10 Target Models | Surrogate Models trained by
---|---
Regular-training | TRADES | GAT | GPGD-AT | RST | PUAT-L | PUAT
Regular-training |  | $33.03\pm 1.20$ | $31.50\pm 1.18$ | $31.50\pm 1.31$ | $32.58\pm 1.28$ | $31.89\pm 1.32$ | $\mathbf{33.23\pm 1.29}$
TRADES | $58.78\pm 0.99$ |  | $58.57\pm 0.96$ | $58.77\pm 0.93$ | $57.82\pm 0.91$ | $58.58\pm 0.89$ | $\mathbf{58.84\pm 0.89}$
GAT | $55.06\pm 3.07$ | $55.95\pm 2.83$ |  | $55.50\pm 2.89$ | $55.63\pm 3.01$ | $55.06\pm 2.93$ | $\mathbf{56.40\pm 3.03}$
GPGD-AT | $36.43\pm 2.94$ | $38.24\pm 2.63$ | $37.04\pm 2.56$ |  | $37.94\pm 2.66$ | $36.69\pm 2.58$ | $\mathbf{38.39\pm 2.63}$
RST | $66.93\pm 0.76$ | $65.97\pm 0.80$ | $66.66\pm 0.85$ | $66.94\pm 0.78$ |  | $66.79\pm 0.72$ | $66.89\pm 0.77$
PUAT-L | $52.23\pm 2.27$ | $52.89\pm 2.04$ | $51.97\pm 2.27$ | $51.88\pm 2.04$ | $52.60\pm 2.03$ |  | $\mathbf{53.09\pm 2.10}$
PUAT | $74.55\pm 0.38$ | $74.55\pm 0.26$ | $73.59\pm 0.40$ | $74.17\pm 0.50$ | $74.03\pm 0.25$ | $73.37\pm 0.49$ |
TABLE VIII: Adversarial robustness against black-box attacks of GPGD on CIFAR10 Target Models | Black-box from Surrogate Models trained by
---|---
Regular-training | TRADES | GAT | GPGD-AT | RST | PUAT-L | PUAT
Regular-training |  | $49.77\pm 1.12$ | $46.16\pm 1.34$ | $50.49\pm 3.15$ | $40.09\pm 1.78$ | $59.07\pm 0.89$ | $\mathbf{63.36\pm 1.50}$
TRADES | $57.61\pm 1.30$ |  | $55.04\pm 0.76$ | $58.21\pm 0.36$ | $37.41\pm 1.05$ | $62.11\pm 1.46$ | $\mathbf{64.39\pm 1.91}$
GAT | $49.40\pm 2.49$ | $46.21\pm 1.15$ |  | $54.12\pm 3.53$ | $42.83\pm 1.04$ | $61.29\pm 1.26$ | $\mathbf{64.07\pm 1.31}$
GPGD-AT | $53.81\pm 1.00$ | $54.56\pm 0.36$ | $56.11\pm 1.65$ |  | $49.77\pm 0.59$ | $63.46\pm 0.25$ | $\mathbf{67.38\pm 0.75}$
RST | $61.88\pm 1.87$ | $47.96\pm 1.41$ | $62.91\pm 0.49$ | $65.60\pm 1.67$ |  | $68.70\pm 0.60$ | $\mathbf{71.31\pm 1.11}$
PUAT-L | $54.91\pm 1.09$ | $55.20\pm 1.09$ | $57.72\pm 0.63$ | $57.23\pm 2.26$ | $47.54\pm 0.98$ |  | $\mathbf{67.93\pm 1.17}$
PUAT | $78.09\pm 0.22$ | $77.84\pm 0.54$ | $78.40\pm 0.61$ | $77.80\pm 1.59$ | $71.60\pm 0.46$ | $79.09\pm 0.50$ |
TABLE IX: Adversarial robustness against black-box attacks of PGD on SVHN Target Models | Black-box from Surrogate Models trained by
---|---
Regular-training | TRADES | GAT | GPGD-AT | RST | PUAT-L | PUAT
Regular-training |  | $53.37\pm 2.19$ | $53.08\pm 2.28$ | $53.64\pm 2.25$ | $53.35\pm 2.22$ | $53.22\pm 2.18$ | $\mathbf{54.05\pm 2.07}$
TRADES | $58.04\pm 2.71$ |  | $57.96\pm 2.71$ | $58.33\pm 2.69$ | $57.69\pm 2.71$ | $58.03\pm 2.69$ | $\mathbf{58.39\pm 2.63}$
GAT | $57.24\pm 1.01$ | $57.4\pm 1.12$ |  | $58.21\pm 1.04$ | $57.70\pm 1.08$ | $57.41\pm 0.98$ | $\mathbf{58.34\pm 1.11}$
GPGD-AT | $45.87\pm 0.43$ | $46.27\pm 0.47$ | $46.16\pm 0.41$ |  | $46.31\pm 0.48$ | $46.10\pm 0.38$ | $\mathbf{46.71\pm 0.34}$
RST | $63.09\pm 2.79$ | $62.87\pm 2.82$ | $63.11\pm 2.80$ | $63.39\pm 2.83$ |  | $63.14\pm 2.77$ | $63.30\pm 2.74$
PUAT-L | $60.03\pm 1.45$ | $60.35\pm 1.37$ | $60.06\pm 1.29$ | $60.93\pm 1.22$ | $60.39\pm 1.30$ |  | $60.89\pm 1.26$
PUAT | $69.13\pm 0.98$ | $69.03\pm 0.90$ | $68.89\pm 0.84$ | $69.45\pm 0.99$ | $68.88\pm 1.04$ | $68.91\pm 0.93$ |
TABLE X: Adversarial robustness against black-box attacks of GPGD on SVHN Target Models | Black-box from Surrogate Models trained by
---|---
Regular-training | TRADES | GAT | GPGD-AT | RST | PUAT-L | PUAT
Regular-training |  | $55.73\pm 6.88$ | $57.04\pm 2.68$ | $65.77\pm 3.15$ | $50.93\pm 5.20$ | $72.18\pm 0.53$ | $\mathbf{76.73\pm 1.44}$
TRADES | $60.88\pm 0.84$ |  | $55.45\pm 3.93$ | $61.71\pm 2.34$ | $45.99\pm 4.41$ | $68.86\pm 0.14$ | $\mathbf{73.37\pm 1.95}$
GAT | $59.47\pm 1.32$ | $43.66\pm 6.54$ |  | $62.20\pm 2.84$ | $47.76\pm 4.04$ | $69.27\pm 1.64$ | $\mathbf{73.79\pm 1.09}$
GPGD-AT | $58.23\pm 1.33$ | $40.92\pm 5.58$ | $54.24\pm 2.68$ |  | $48.06\pm 3.33$ | $67.59\pm 1.09$ | $\mathbf{73.58\pm 3.18}$
RST | $64.51\pm 0.26$ | $47.51\pm 4.66$ | $61.98\pm 3.51$ | $68.32\pm 1.27$ |  | $73.81\pm 0.93$ | $\mathbf{78.06\pm 1.27}$
PUAT-L | $65.65\pm 1.95$ | $48.16\pm 8.76$ | $61.32\pm 3.40$ | $69.91\pm 2.68$ | $54.77\pm 4.80$ |  | $\mathbf{75.44\pm 2.74}$
PUAT | $73.73\pm 1.34$ | $65.18\pm 3.91$ | $72.23\pm 4.08$ | $77.79\pm 2.29$ | $61.37\pm 4.98$ | $79.62\pm 0.76$ |

As previously mentioned, to conduct the black-box attacks, we first invoke an
attacking method to attack the surrogate model pre-trained by a defensive
method to generate AEs, and then use these AEs to attack the target
classifiers trained by other defensive methods. Since PGD and GPGD are the
most successful white-box attacking methods in our experiments, we select them
as the black-box attackers. Tables VII and VIII display the results for
CIFAR10, whereas Tables IX and X display the results for SVHN.

Let’s start by focusing the last row of each table. As we can see, PUAT
achieves the best robust accuracy in each scenario, demonstrating that it is
more powerful than all baseline methods at resisting black-box attacks.
Considering PUAT’s excellent performance in defending against white-box
attacks, we are not surprised by this outcome. The most intriguing part of
each table is the last column, which shows how well baseline approaches fight
against AEs produced by attacking the surrogate model trained by PUAT. We can
see that in the majority of cases, the black-box attacks based on the
surrogate model trained by PUAT result in better robust accuracy for baseline
methods than those based on surrogate models trained by other baseline
methods, demonstrating the lower threat of the surrogate model trained by
PUAT. As a result, this finding shows that PUAT is not only the most resilient
against black-box attacks but also does not support them.

###  5.4 Elimination of The Tradeoff (RQ3)

(a) PUAT

(b) RST

Figure 3: Elimination of the tradeoff between standard generalizability and
adversarial robustness.

Now we examine PUAT’s performance throughout a range of values of the factor
$\lambda$, which regularizes the weights of standard generalization and
adversarial robustness in Equation (14), to see if it can eliminate the
tradeoff between them. Figure 3(a) shows the curves of PUAT’s natural accuracy
and robust accuracies under different attacks on CIFAR10. As we can see, all
of the curves first rise as we increase $\lambda$ from 0 to 1, and then
roughly stabilize, which demonstrates that when $\lambda$ is bigger than 0, it
has no regulatory influence on the training of PUAT. In other words, PUAT can
consistently optimize both adversarial robustness and standard generalization
without having to make a tradeoff between them. This is due to PUAT’s ability
to reconcile the robust generalization on AEs and the standard generalization
on natural data by aligning the AE generator distribution $P_{G}(x,y)$, the
target classifier distribution $P_{C}(x,y)$ and the natural data distribution
$P(x,y)$, as previously proved in Section 4.

Similarly, RST also has a balance factor $\lambda$ to weight the importance of
adversarial loss during its training. In contrast to PUAT, the robust accuracy
of RST on RAEs (by FGSM and PGD) is gradually increasing as $\lambda$
increases, but its natural accuracy decreases in parallel, as shown in Figure
3(b). This indicates that there is a tradeoff between the adversarial
robustness and standard generalizability due to their conflicting optimization
objectives in RST. At the same time, one can note that since GPGD and USong
generate UAEs from the natural data distribution, the decrease in the standard
generalization of RST on natural data naturally leads to a concomitant
decrease in its adversarial robustness against UAEs. In sharp contrast, in
Figure 3(a), PUAT’s robust accuracies on GPGD and USong exhibit the same
pattern as its robust accuracies on PGD and FGSM, demonstrating once more that
PUAT is capable of providing consistent adversarial robustness against both
UAE and RAE since PUAT regards RAE as a special UAE.

###  5.5 Visual Study (RQ4)

(a) Alignment on CIFAR10

(b) Misalignment on CIFAR10

(c) Alignment on SVHN

(d) Misalignment on SVHN

Figure 4: Visualized distribution alignment or misalignment, where different colors denote classes and different patterns denote distributions. TABLE XI: The sample number of different classes from different distributions. Distribution | CIFAR10 | SVHN
---|---|---
”airplane” | ”automobile” | ”bird” | ”0” | ”1” | ”2”
$P(y)$ | 200 | 200 | 200 | 133 | 391 | 318
$P_{G}(y)$ | 200 | 200 | 200 | 133 | 391 | 318
$P_{C}(y)$ | 213 | 203 | 170 | 148 | 338 | 279
misaligned $P_{C}(y)$ | 8 | 1 | 42 | 46 | 2 | 0

Now, we visually demonstrate PUAT’s capability to align the natural data
distribution $P(x,y)$, the distribution $P_{G}(x,y)$ learned by the generator,
and the distribution $P_{C}(x,y)$ learned by target classifier. For this
purpose, on each testing set, we first sample three different class labels
$y_{1}$, $y_{2}$ and $y_{3}$ from $P(y)$, with $y_{1}="airplane"$,
$y_{2}="automobile"$, $y_{3}="bird"$ on CIFAR10 and $y_{1}="0"$, $y_{2}="1"$,
$y_{3}="2"$ on SVHN. Then for each $y_{i}$, $i=1,2,3$, we invoke $G$ to
generate a set of $\\{(x_{g},y_{i})\\}$ such that $\big{|}\\{(x_{g},y_{i})\sim
P_{G}(x,y)\\}\big{|}=\big{|}\\{(x_{l},y_{i})\sim P(x,y)\\}\big{|}$. At the
same time, all the testing examples are also fed into $C$, and those
classified as $y_{1}$, $y_{2}$ or $y_{3}$ make up $\\{(x_{c},y_{i})\sim
P_{C}(x,y)\\}$. At last, we plot the sample points $\\{(x_{l},y_{i})\\}$,
$\\{(x_{g},y_{i})\\}$, and $\\{(x_{c},y_{i})\\}$ ($i=1,2,3$) in Figure 4 by
using t-SNE algorithm, where different colors denote classes and different
patterns denote distributions.

Figures 4(a) and 4(c) show the results of the complete PUAT on CIFAR10 and
SVHN, respectively. From Figures 4(a) and 4(c) we can see that the data points
are distributed in three different colored clusters with clear boundaries,
where the examples from different distributions (with different patterns) but
of the same class (with the same color) are grouped in the same cluster, and
there are essentially no data points in any cluster that have a different
color from the majority. However, once we remove the extended AT from PUAT so
that Theorem 1 does not hold, the cluster boundaries become blurred and the
colors in the same cluster become impure, as shown in Figures 4(b) and 4(d).
Therefore, we can contend that PUAT aligns the three conditional distributions
$P(x|y)$, $P_{C}(x|y)$ and $P_{G}(x|y)$. Furthermore, as shown in Table XI,
when the distributions are aligned, the number of examples belonging to
$y_{i}$ is approximately the same in different distributions, which indicates
$P(y)=P_{G}(y)=P_{C}(y)$, so we can confirm that $P(x,y)$, $P_{C}(x,y)$ and
$P_{G}(x,y)$ are aligned by PUAT.

##  6 Related Work

In this section, we briefly introduce the related works from three domains,
adversarial attacks, adversarial training, and standard generalizability, and
comment their differences from our work.

###  6.1 Adversarial Attacks

Generally, adversarial attacks aim to fool a target model well-trained on
natural data with AEs that are imperceptible to humans [4, 34]. The existence
of AEs is first discovered by the seminal works of Szegedy et al. [2] and
Goodfellow et al. [8]. In [2], Szegedy et al. propose an attack method called
L-BFGS which can imperceptibly perturb a natural example with respect to its
second-order gradient to cause misclassifications. In [8], Goodfellow et al.
further propose the fast gradient sign method (FGSM), which searches
adversarial perturbations just based on a single first-order gradient step.
Following the idea of [8], various adversarial attack methods have been
proposed by enhancing the first-order method. For example, Madry et al. [9]
propose a projected gradient descent (PGD) method which can be regarded as an
iterative FGSM projecting AEs into a ball constrained by $L_{\infty}$ norm.
Instead of restricting the perturbation norms to a fixed value, Moosavi-
Dezfooli et al. [35] propose an optimization based attack method Deepfool to
seek adversarial perturbations for differentiable classifiers by minimizing
$L_{2}$ norm. Similarly, Carlini and Wagner [36] propose another optimization
based method called C&W attack, which can generate stronger AEs with
perturbations minimizing $L_{0}$, $L_{2}$ and $L_{\infty}$ norms at the
expense of higher computation complexity. In contrast to gradient based
methods, another line of works employs GAN to generate AEs for better
imperceptibility. For example, Xiao et al. [37] propose AdvGAN to generate
perturbations for observed examples, which extends standard GAN with explicit
perturbation norm loss to ensure the imperceptibility. Additionally, instead
of generating an AE by perturbing a single example, Moosavi-Dezfooli et al.
[38] also further propose universal adversarial perturbations which are
computed in an example-agnostic fashion and can fool the target classifier on
any examples.

The above-mentioned adversarial attacks focus on the AEs which are generated
by adding imperceptible perturbations restricted by $L_{p}$ norm on observed
examples, which we call restricted AE (RAE). Recently, some researchers have
discovered that there exist unrestricted AEs (UAEs) that are norm-constrained.
To ensure the imperceptibility of UAEs, one line of UAE generation methods
manipulates example features by perturbations within perceptual distances that
are aligned with human perception [39, 40, 18, 19]. At the same time, some
researchers find that in a broader sense, UAEs can be completely new examples
built from scratch and dissimilar to any observed example but can still fool
classifiers without confusing humans [16]. Following this idea, a few of UAE
generation methods based on generative models are proposed [16, 17, 32, 20,
41, 42]. For example, Song et al.[16] and Xiao et al. [32] propose to first
generate perturbations based on initial random noise and then feed them into a
pre-trained conditional GAN (e.g. AC-GAN [43]) to produce UAEs, while
Poursaeed et al. [41] propose to use a Style-GAN [44] to synthesize UAEs with
stylistic perturbations.

The existing works for UAE generation, however, do not explain why UAEs pose
threats, or rather, where the imperceptibility of UAEs stems. By contrast, our
work proposes a novel viewpoint to view UAEs as perturbed unobserved examples,
which provides a logical and provable explanation of UAE’s dissimilarity to
observed examples, imperceptibility to humans, and the feasibility of
comprehensive adversarial robustness. Based on this understanding, we further
propose a novel G-C-D GAN for UAE generation, which together with the attacker
$A$ can synthesize UAEs that are more adversarial by aligning UAE distribution
with natural data distribution.

###  6.2 Adversarial Training

AT has proved to be the strongest principled defensive technique against
adversarial attacks [5, 6, 7]. The basic idea of AT is initially proposed by
Szegedy et al. [2], which uses a min-max game to incorporate deliberately
crafted AEs to the training process of a DNN model to obtain immunity against
AEs. Madry et al. [9] are the first time to theoretically justify the
adversarial robustness offered by the min-max optimization used in AT.

Over the past few years, many enhancements of AT have been proposed to employ
various perturbation generation methods to improve adversarial robustness [10,
11, 12, 13, 14, 15]. For example, Wang et al. [10] propose a bilateral
adversarial training method that perturbs both normal images and real labels
with respect to a smaller gradient magnitude. Wang et al. [11] propose a
misclassification-aware adversarial training (MART) method, which defines a
novel adversarial loss with a regularization on misclassified examples. Cheng
et al. [12] argue that large perturbations may be more adversarial for the
robustness of a target model and propose a customized adversarial training
(CAT) method which can offer adaptive perturbations for natural examples.
Shafahi et al. [13] propose a universal adversarial training method for the
robustness against universal attacks [38]. Among others, generative model-
based AT methods are verified as a promising approach due to the GAN’s
capability of distributional alignment. For example, to strengthen the
adversaries, Lee et al. [14] propose a generative adversarial trainer (GAT)
which employs a GAN to produce adversarial perturbations with respect to the
gradients of the natural examples, while Stutz et al. [15] propose to use VAE-
GAN to produce AEs whose distribution is aligned with that of the natural
examples. As discussed in 6.1, GAN-based methods strengthen AEs with stronger
imperceptibility because of the alignment between the distributions of AEs and
the natural examples, which is equivalent to drawing AEs on the manifold
(distribution) of the natural examples.

The above-mentioned AT methods are basically based on supervised learning,
which often suffer from the sparsity of labeled data. Recently, some
researchers have discovered that introducing richer unlabeled data can
increase the performance of AT, and hence proposed semi-supervised AT methods
leveraging partially labeled data [45, 33, 46]. For example, Zhang et al. [45]
argue that the performance of AT will be impaired by the perturbations
generated only based on labeled data since they are unable to reflect the
underlying distribution of all potential examples, and propose a GAN-based
semi-supervised AT method where AEs can preserve the inner structure of
unlabeled data. Carmon et al. [33] propose a robust self-training (RST) model
which first uses an intermediate classifier trained on labeled examples to
infer the labels for unlabeled examples, and then generate AEs for AT from the
labeled and pseudo-labeled examples. Stanforth et al. [46] propose an
unsupervised adversarial training (UAT) method which estimates the worst-case
AEs with KL-divergence between the distributions of the perturbed unlabeled
examples and the natural unlabeled examples.

However, the AEs used in traditional AT methods are generated by adding norm-
bounded perturbations, even though which are generated by GAN, to observed
examples, which limits traditional AT methods to providing only the
adversarial robustness against RAEs. To overcome this limitation, recently,
Xiao et al. [32] propose a new AT method called GPGD-AT, which conducts AT
over UAEs generated by their GPGD. In particular, GPGD first employs a
conditional GAN to generate an unobserved labeled example, then adds
adversarial perturbation to it based on the gradient of the target classifier
loss incurred by the generated example. However, as the problem of UAE
generation, AT on UAEs still remains under-explored.

Different from the existing AT methods providing adversarial robustness
against either RAE or UAE, our PUAT focuses on the concept of generalized UAE
which makes it feasible to offer comprehensive adversarial robustness against
both UAE and RAE. At the same time, PUAT is both GAN-based and semi-
supervised, which improves the AT performance with stronger UAEs compared with
GPGD-AT.

###  6.3 Standard Generalizability

A model’s standard generalizability and adversarial robustness are measured by
its accuracy on testing natural examples and testing AEs, respectively.
Traditional AT methods are based on AEs with norm-constraints, which will lead
to distribution-shift between AEs and natural examples [47, 32]. Therefore,
there is a belief that AT forces the target model to generalize on two
separated distributions and hence the tradeoff between standard
generalizability and adversarial robustness inevitably comes into being, or in
other words, adversarial robustness comes at the expense of standard
generalizability [9, 21, 31, 47].

However, some researchers have opposing opinions that the tradeoff may be not
inevitable, with support of recent empirical evidences and theoretical
analyses [15, 23, 48, 24, 25]. One line of works tries to extend the existing
AT methods with regularizations to mitigate the tradeoff. For example, Song et
al. [24] and Xing et al. [25] propose to improve the generalizability by
introducing robust local features and $l_{1}$ penalty to AT, respectively. The
other line aims at breaking the tradeoff by exploiting the distributional
property of AEs and natural examples. For example, Yang et al. [23] discover
the $r$-separation phenomenon in widely-used image datasets, namely the
distributions of different classes are at least distance 2$r$ apart from each
other in pixel space. Based on this discovery, they theoretically prove that a
classifier with sufficient local Lipschitz smoothness can achieve both high
accuracy on natural examples and acceptable robustness on AEs with
perturbations of size up to $r$. Differently, Stutz et al. [15] find that
there exist AEs on the manifold of natural examples, and hence the adversarial
robustness against the on-manifold AEs will equivalently improve the standard
generalizability. Similarly, Staib et al. [48] propose Distributionally Robust
Optimization (DRO), which offers a more general perspective that the tradeoff
can be eliminated by the alignment between the distributions of AEs and
natural examples. Instead of generating point-wise perturbations with norm-
constrained magnitude, DRO aims to seek worst-case AE distribution by
restricting the distance between the AE distribution $\tilde{p}$ and the
natural data distribution $p$ through the following min-max game:

$\min_{C}\max_{\tilde{P}(\tilde{x},y):W_{p}(P,\tilde{P})\leq\epsilon}\mathbb{E}_{P(x,y)}\left[\mathcal{L}\left(C(\tilde{x}),y\right)\right],$

where $W_{p}$ is a distributional distance measure, e.g., Wasserstein
distance. As $\epsilon$ approaching 0, $\tilde{P}$ is gradually aligning with
$P$, and consequently, the tradeoff tends to disappear. Indeed, on-manifold
AEs can be seen as a special case of DRO when $\epsilon=0$.

Our work provides new arguments for the opinion that the tradeoff is
eliminable. In particular, we make two contributions which distinguish our
work from the existing works addressing the tradeoff: (1) We extend the
research scope to UAEs by proposing a novel AT method PUAT; (2) By solid
theoretical analysis and extensive empirical investigation, we show that the
standard generalizability and comprehensive adversarial robustness are both
achievable for PUAT.

##  7 Conclusion

In this paper, we propose a unique viewpoint that understands UAEs as
imperceptibly perturbed unobserved examples, which rationalizes why UAEs exist
and why they are able to deceive a well-trained classifier. This understanding
allows us to view RAE as a special UAE, thus providing the feasibility of
achieving comprehensive adversarial robustness against both UAE and RAE. Also,
we find that if the UAE distribution and the natural data distribution can be
aligned, the conflict between robust generalization and standard
generalization in traditional AT methods can be eliminated, thus improving
both the adversarial robustness and standard generalizability of a target
classifier. Based on these ideas, we propose a novel AT method called Provable
Unrestricted Adversarial Training (PUAT). PUAT utilizes partially labeled data
to achieve efficient UAE generation by accurately capturing the real data
distribution through a well-designed G-C-D GAN. At the same time, PUAT extends
the traditional AT by introducing the supervised loss of the target classifier
into the adversarial training and achieves alignment between the UAE
distribution, the natural data distribution, and the distribution learned by
the classifier, with the collaboration of the G-C-D GAN. The solid theoretical
analysis and the extensive experiments demonstrate that PUAT can eliminate
tradeoffs between adversarial robustness and standard generalizability through
distributional alignment, and compared to the baseline methods, PUAT enables a
target classifier to better defend against both UAE attacks and RAE attacks,
as well as significantly increases its standard generalizability.

## Acknowledgments

This work is supported by National Natural Science Foundation of China under
grant 61972270, and in part by NSF under grants III-1763325, III-1909323,
III-2106758, and SaTC-1930941.

## References

  * [1] B. Biggio, I. Corona, D. Maiorca, B. Nelson, N. Šrndić, P. Laskov, G. Giacinto, and F. Roli, “Evasion attacks against machine learning at test time,” in _Joint European Conference on Machine Learning and Knowledge Discovery in Databases_ , 2013, pp. 387–402.
  * [2] C. Szegedy, W. Zaremba, I. Sutskever, J. Bruna, D. Erhan, I. Goodfellow, and R. Fergus, “Intriguing properties of neural networks,” in _International Conference on Learning Representations_ , 2014.
  * [3] A. Ilyas, S. Santurkar, D. Tsipras, L. Engstrom, B. Tran, and A. Madry, “Adversarial examples are not bugs, they are features,” _Advances in Neural Information Processing Systems_ , vol. 32, 2019.
  * [4] D. J. Miller, Z. Xiang, and G. Kesidis, “Adversarial learning targeting deep neural network classification: A comprehensive review of defenses against attacks,” _Proceedings of the IEEE_ , vol. 108, no. 3, pp. 402–433, 2020\.
  * [5] T. Bai, J. Luo, J. Zhao, B. Wen, and Q. Wang, “Recent advances in adversarial training for adversarial robustness,” in _Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence (IJCAI-21)_ , 2021.
  * [6] N. Akhtar, A. Mian, N. Kardan, and M. Shah, “Advances in adversarial attacks and defenses in computer vision: A survey,” _IEEE Access_ , vol. 9, pp. 155 161–155 196, 2021.
  * [7] W. Zhao, S. Alwidian, and Q. H. Mahmoud, “Adversarial training methods for deep learning: A systematic review,” _Algorithms_ , vol. 15, no. 8, p. 283, 2022.
  * [8] I. J. Goodfellow, J. Shlens, and C. Szegedy, “Explaining and harnessing adversarial examples,” in _International Conference on Learning Representations_ , 2014.
  * [9] A. Madry, A. Makelov, L. Schmidt, D. Tsipras, and A. Vladu, “Towards deep learning models resistant to adversarial attacks,” in _International Conference on Learning Representations_ , 2018.
  * [10] J. Wang and H. Zhang, “Bilateral adversarial training: Towards fast training of more robust models against adversarial attacks,” in _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , 2019.
  * [11] Y. Wang, D. Zou, J. Yi, J. Bailey, X. Ma, and Q. Gu, “Improving adversarial robustness requires revisiting misclassified examples,” in _International Conference on Learning Representations_ , 2020.
  * [12] M. Cheng, Q. Lei, P.-Y. Chen, I. Dhillon, and C.-J. Hsieh, “Cat: Customized adversarial training for improved robustness,” _CoRR_ , vol. abs/2002.06789, 2020.
  * [13] A. Shafahi, M. Najibi, Z. Xu, J. Dickerson, L. S. Davis, and T. Goldstein, “Universal adversarial training,” in _Proceedings of the AAAI Conference on Artificial Intelligence_ , vol. 34, no. 04, 2020, pp. 5636–5643.
  * [14] H. Lee, S. Han, and J. Lee, “Generative adversarial trainer: Defense to adversarial perturbations with gan,” _CoRR_ , vol. abs/1705.03387, 2017.
  * [15] D. Stutz, M. Hein, and B. Schiele, “Disentangling adversarial robustness and generalization,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2019, pp. 6976–6987.
  * [16] Y. Song, R. Shu, N. Kushman, and S. Ermon, “Constructing unrestricted adversarial examples with generative models,” _Advances in Neural Information Processing Systems_ , vol. 31, 2018.
  * [17] I. Dunn, H. Pouget, T. Melham, and D. Kroening, “Adaptive generation of unrestricted adversarial inputs,” _CoRR_ , vol. abs/1905.02463, 2019.
  * [18] A. Bhattad, M. J. Chong, K. Liang, B. Li, and D. A. Forsyth, “Unrestricted adversarial examples via semantic manipulation,” in _International Conference on Learning Representations_ , 2020.
  * [19] H. Naderi, L. Goli, and S. Kasaei, “Generating unrestricted adversarial examples via three parameteres,” _Multimedia Tools and Applications_ , pp. 1–20, 2022.
  * [20] T. Xiang, H. Liu, S. Guo, Y. Gan, and X. Liao, “Egm: An efficient generative model for unrestricted adversarial examples,” _ACM Transactions on Sensor Networks_ , 2022.
  * [21] D. Tsipras, S. Santurkar, L. Engstrom, A. Turner, and A. Madry, “Robustness may be at odds with accuracy,” in _International Conference on Learning Representations_ , 2019.
  * [22] I. Attias, A. Kontorovich, and Y. Mansour, “Improved generalization bounds for adversarially robust learning,” _Journal of Machine Learning Research_ , vol. 23, no. 175, pp. 1–31, 2022.
  * [23] Y.-Y. Yang, C. Rashtchian, H. Zhang, R. R. Salakhutdinov, and K. Chaudhuri, “A closer look at accuracy vs. robustness,” _Advances in Neural Information Processing Systems_ , vol. 33, 2020.
  * [24] C. Song, K. He, J. Lin, L. Wang, and J. E. Hopcroft, “Robust local features for improving the generalization of adversarial training,” in _International Conference on Learning Representations_ , 2020.
  * [25] Y. Xing, Q. Song, and G. Cheng, “On the generalization properties of adversarial training,” in _International Conference on Artificial Intelligence and Statistics_. PMLR, 2021, pp. 505–513.
  * [26] C. Li, T. Xu, J. Zhu, and B. Zhang, “Triple generative adversarial nets,” _Advances in Neural Information Processing Systems_ , vol. 30, 2017.
  * [27] C. Li, K. Xu, J. Zhu, J. Liu, and B. Zhang, “Triple generative adversarial networks,” _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , vol. 44, no. 12, pp. 9629–9640, 2022.
  * [28] I. J. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. C. Courville, and Y. Bengio, “Generative adversarial nets,” in _Advances in Neural Information Processing Systems_ , 2014.
  * [29] A. Krizhevsky, “Learning multiple layers of features from tiny images,” _Citeseer_ , 2009.
  * [30] Y. Netzer, T. Wang, A. Coates, A. Bissacco, B. Wu, and A. Y. Ng, “Reading digits in natural images with unsupervised feature learning,” in _Advances in Neural Information Processing Systems Workshop on Deep Learning and Unsupervised Feature Learning_ , 2011.
  * [31] H. Zhang, Y. Yu, J. Jiao, E. Xing, L. El Ghaoui, and M. Jordan, “Theoretically principled trade-off between robustness and accuracy,” in _International Conference on Machine Learning_. PMLR, 2019, pp. 7472–7482.
  * [32] J. Xiao, L. Yang, Y. Fan, J. Wang, and Z.-Q. Luo, “Understanding adversarial robustness against on-manifold adversarial examples,” _CoRR_ , vol. abs/2210.00430, 2022.
  * [33] Y. Carmon, A. Raghunathan, L. Schmidt, J. C. Duchi, and P. S. Liang, “Unlabeled data improves adversarial robustness,” in _Advances in Neural Information Processing Systems_ , 2019.
  * [34] X. Zhang, X. Zheng, and W. Mao, “Adversarial perturbation defense on deep neural networks,” _ACM Computing Surveys (CSUR)_ , vol. 54, no. 8, pp. 1–36, 2021.
  * [35] S.-M. Moosavi-Dezfooli, A. Fawzi, and P. Frossard, “Deepfool: a simple and accurate method to fool deep neural networks,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2016.
  * [36] N. Carlini and D. Wagner, “Towards evaluating the robustness of neural networks,” in _2017 ieee symposium on security and privacy (sp)_ , 2017, pp. 39–57.
  * [37] C. Xiao, B. Li, J.-Y. Zhu, W. He, M. Liu, and D. Song, “Generating adversarial examples with adversarial networks,” _CoRR_ , vol. abs/1801.02610, 2018.
  * [38] S.-M. Moosavi-Dezfooli, A. Fawzi, O. Fawzi, and P. Frossard, “Universal adversarial perturbations,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2017.
  * [39] A. S. Shamsabadi, R. Sanchez-Matilla, and A. Cavallaro, “Colorfool: Semantic adversarial colorization,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2020, pp. 1151–1160.
  * [40] Z. Zhao, Z. Liu, and M. Larson, “Towards large yet imperceptible adversarial image perturbations with perceptual color distance,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2020.
  * [41] O. Poursaeed, T. Jiang, Y. Goshu, H. Yang, S. Belongie, and S.-N. Lim, “Fine-grained synthesis of unrestricted adversarial examples,” _CoRR_ , vol. abs/1911.09058, 2019.
  * [42] X. Wang, K. He, and J. E. Hopcroft, “At-gan: A generative attack model for adversarial transferring on generative adversarial nets,” _CoRR_ , vol. abs/1904.07793, 2019.
  * [43] A. Odena, C. Olah, and J. Shlens, “Conditional image synthesis with auxiliary classifier gans,” in _International Conference on Machine Learning_. PMLR, 2017, pp. 2642–2651.
  * [44] T. Karras, S. Laine, and T. Aila, “A style-based generator architecture for generative adversarial networks,” in _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , 2019.
  * [45] H. Zhang and J. Wang, “Defense against adversarial attacks using feature scattering-based adversarial training,” in _Advances in Neural Information Processing Systems_ , 2019.
  * [46] J.-B. Alayrac, J. Uesato, P.-S. Huang, A. Fawzi, R. Stanforth, and P. Kohli, “Are labels required for improving adversarial robustness?” in _Advances in Neural Information Processing Systems_ , 2019.
  * [47] S. Yang, T. Guo, Y. Wang, and C. Xu, “Adversarial robustness through disentangled representations,” in _Proceedings of the AAAI Conference on Artificial Intelligence_ , 2021.
  * [48] M. Staib and S. Jegelka, “Distributionally robust deep learning as a generalization of adversarial training,” in _NIPS workshop on Machine Learning and Computer Security_ , 2017.

|  Lilin Zhang obtained her bachelor’s degree from the School of Information
and Security Engineering, Zhongnan University of Economics and Law, in 2021.
She is now pursuing the master’s degree in the School of Computer Science,
Sichuan University, China. Her research interest focuses on adversarial
machine learning and its applications.
---|---
|  Ning Yang is an associate professor at Sichuan University, China. He
obtained his PhD degree in Computer Science from Sichuan University in 2010.
His research interests include adversarial machine learning, graph learning,
and recommender systems.
---|---
|  Yanchao Sun is a Ph.D. candidate in University of Maryland, College Park,
USA, where she is advised by Prof. Furong Huang. Her research mainly focuses
on reinforcement learning (RL), including adversarial RL, multi-task RL,
sample-efficient RL, etc.
---|---
|  Philip S. Yu received the PhD degree in electrical engineering from
Stanford University. He is a distinguished professor in computer science at
the University of Illinois at Chicago and is also the Wexler chair in
information technology. His research interests include big data, data mining,
and social computing. He is a fellow of the ACM and the IEEE.
---|---