# Adversarial Learning with Cost-Sensitive Classes

Haojing Shen, Sihong Chen, Ran Wang, , Xizhao Wang Haojing Shen, Sihong Chen,
and Xizhao Wang are with Big Data Institute, College of Computer Science and
Software Engineering, Guangdong Key Lab. of Intelligent Information
Processing, Shenzhen University, Shenzhen 518060, Guangdong, China (Email:
<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>).Ran Wang is
with the College of Mathematics and Statistics, Shenzhen University, Shenzhen
518060, China and also with the Shenzhen Key Laboratory of Advanced Machine
Learning and Applications, Shenzhen University, Shenzhen 518060, China.
(e-mail: wangran@szu.edu.cn).Corresponding author: Xizhao Wang.

###### Abstract

It is necessary to improve the performance of some special classes or to
particularly protect them from attacks in adversarial learning. This paper
proposes a framework combining cost-sensitive classification and adversarial
learning together to train a model that can distinguish between protected and
unprotected classes, such that the protected classes are less vulnerable to
adversarial examples. We find in this framework an interesting phenomenon
during the training of deep neural networks, called Min-Max property, that is,
the absolute values of most parameters in the convolutional layer approach
zero while the absolute values of a few parameters are significantly larger
becoming bigger. Based on this Min-Max property which is formulated and
analyzed in a view of random distribution, we further build a new defense
model against adversarial examples for adversarial robustness improvement. An
advantage of the built model is that it does no longer need adversarial
training, and thus, has a higher computational efficiency than most existing
models of needing adversarial training. It is experimentally confirmed that,
regarding the average accuracy of all classes, our model is almost as same as
the existing models when an attack does not occur and is better than the
existing models when an attack occurs. Specifically, regarding the accuracy of
protected classes, the proposed model is much better than the existing models
when an attack occurs.

###### Index Terms:

Adversarial examples, adversarial training, robustness, cost-sensitive, attack
and defense.

##  I Introduction

(a) The framework of adversarial training (b) The framework of CSA and CSE
algorithms

Figure 1: A framework for adversarial training and an overview of CSA and CSE
algorithms. (a) The clean examples and adversarial examples are fed into the
network alternately during training. For example, firstly, clean examples were
fed into the network in Stage 1. Then adversarial examples are generated from
clean examples in Stage 2. Finally, in Stage 3, the adversarial examples are
applied to train the network. (b) A framework for the CSA and CSE algorithms.
We take a LeNet [1] network as example. Note that, for the CSE algorithm, the
convolutional parameters will be applied to the loss function with some
formulation to attain the Min-Max property.

Recent studies have shown that the deep neural network is relatively fragile
[2], and it is vulnerable to adversarial examples that are composed of some
slight perturbations and clean samples. The methods to generate adversarial
examples are called adversarial attacks [2, 7, 10, 11, 6, 12]. Simultaneously,
to defend adversarial attacks, recently, many works have been proposed
adversarial defenses [3, 4, 5, 2, 6, 7, 8, 9]. There is an arms race between
adversarial attacks and adversarial defenses.

Adversarial training [2, 6, 7] is a simple and effective adversarial defense
method. Many studies [6, 7] have shown that adversarial training can
effectively defend against white-box attacks. However, it can only defend
against a given attack method, which may fail for other stronger or unknown
attacks. Meanwhile, it is found in some works that the adversarial training
would step into the trap, called obfuscated gradients [3, 13]. The obfuscated
gradients give a false sense that the model has a good adversarial robustness
against adversarial attack by limiting the attacker to calculate the gradient
of the model. However, an attacker can successfully attack a model with
adversarial training by using a transfer attack method, building a new
approximation function, or using a non-gradient attack.

The main purpose of existing defense methods [2, 4] is to improve the model’s
robustness for overall classes. These defense methods treat every class in the
dataset equally, i.e., the adversarial robustness of each class needs to be
improved simultaneously. However, constructing such an ideal model, which
improves the adversarial robustness of each class, is challenging. Actually,
in practical applications, not all the classes are equally important in some
tasks, i.e., samples of certain classes are more important than those of
others. For example, there are various denominations of dollar such as 1$, 2$,
10$, 20$, 50$, 100$ and so on. Obviously, in the task of identifying dollar
bills, we hope that the model can be more accurate of large denominations. As
for the adversary, of course, they prefer to attack the large denomination of
the bill to obtain the maximum profit. From this perspective, we hope to
propose a method to particularly improve the adversarial robustness of more
important classes.

Some related works can be retrieved from the literature [14, 15]. Zhang et al.
[14] point out that in practical application, the cost of adversarial
transformation depends on specific classes. They combine cost-sensitive
learning with robust learning [16] and advocate using the cost-sensitive
robustness to measure the classifier’s performance for some tasks where the
adversarial transformation is more important than other transformations. They
code the cost of each type of adversarial transformation into a cost matrix,
combine it with the robust learning method [16], and propose a new objective
function. However, they do not explain why cost-sensitive robustness learning
affects the robustness of each class, and the performance of the model is
heavily dependent on the robustness learning methods in [16]. Based on optimal
transmission theory and Wasserstein Distance, Terzi et al. [15] propose the
optimal cost of transferring from one kind of distribution to another to
improve the model’s cost-sensitive robustness. Their proposed WPGD method can
be used to solve the cost-sensitive problem, data imbalance problem, or the
balance problem of robustness and accuracy.

In this paper, we focus on a new problem, i.e., protecting a particular class
under adversarial attacks. Unlike traditional defense strategies, we consider
the accuracy rate of specific categories to minimize the impact of adversarial
examples. Of course, the accuracy rate of other categories may be sacrificed
to some extent, but it is expected that the overall performance is similar to
that of the standard training model. We find that this problem has one thing
in common with the cost-sensitive problem [17, 18], i.e., the cost of
misclassification depends on different labels. This is different from the
traditional classification problem, which assumes that all misclassification
cases have the same cost. In real life, the misclassification cost of many
problems is related to the individual categories. For example, in medical
diagnosis, the cost of misdiagnosing a cancer patient as a flu patient is much
higher than misdiagnosing a flu patient as a cancer patient. It is noteworthy
that the traditional cost-sensitive learning, which is different with the
proposed problem, does not consider the adversarial robustness of the model or
particularly protect a certain class against adversarial attacks.

This paper proposes a cost-sensitive adversarial learning model (CSA), which
can well resist the adversarial attacks against special classes. We point out
that the good robustness of the model is due to a special property of the
convolutional layer parameters in the model, which is reflected in LeNet
networks as the fact that the absolute values of most parameters in the
convolutional layer approach zero while the absolute values of a few
parameters are significantly larger than others. This property indicates that
the absolute values of a major part of weight parameters of the convolutional
layer in the model attain the minimum (zero), and the absolute values of a
minor part of weight parameters go to maximum. Thus we name it as Min-Max
property of weight parameters. Actually, the Min-Max property refers to a kind
of approximate sparseness of weight parameters in convolutional layers, which
reflects the essential of convolution from low level to high level features.
Furthermore, when the model makes predictions for the samples of a specific
class, only some of the parameters play a key role. Therefore, we explain why
the CSA model could improve the adversarial robustness of special classes: the
adversarial training brings the Min-Max property to the model parameters,
while the cost matrix can locate the parameters that play a decisive role in
the prediction results of the target class. When applying the cost matrix to
adversarial training, it makes the model endowing part of the positioned
parameters with more obvious Min-Max property than other parameters during the
training, thus improving the adversarial robustness of class the target class.
According to this explanation, we propose to build a new learning model,
called cost-sensitive adversarial extension (CSE), that does not depend on
adversarial training. CSE is an end-to-end learning method that does not need
adversarial training but can improve the adversarial robustness of the model.
An overview of the proposed model is shown in Fig. 1.

The contributions of this paper are listed as follows:

  * •

by incorporating the cost-sensitivity into the adversarial training, we
provide an algorithm (CSA), which can specifically protect an important class
and improve its adversarial robustness.

  * •

We give a new explanation to the robustness of the model against adversarial
attacks and propose a novel robust learning algorithm (CSE), which can make
the model resist the adversarial examples without adversarial training. It is
noted that most of the state-of-the-art models of adversarial robustness
learning do need adversarial training.

  * •

We also verify the validity of the CSA model and CSE model experimentally.
Compared with the traditional adversarial training models, our model has
better overall robustness and can effectively improve the adversarial
robustness of the protected class.

##  II Related Work

Since the first discovery of adversarial examples [2] in deep learning, there
have been many works [3, 6, 7, 12, 13, 4] studying adversarial examples
generation. The adversarial example is indistinguishable to humans but can
easily fool machines into misclassification. This imperceptibility, which
depends largely on human perception, is not measurable. But we usually use a
small perturbation, which is limited in the $l_{p}$-norm, to evaluate this
imperceptibility when generating adversarial examples. In this way, we can
represent the adversarial examples as the following form:

$A(\mathbf{x})=\\{\mathbf{x}^{\prime}|f(\mathbf{x}^{\prime})\neq
f(\mathbf{x}),||\mathbf{x}^{\prime}-\mathbf{x}||_{p}\leq\epsilon\\}$

where $A$ is a set, $\mathbf{x}$ is a clean example, $\mathbf{x}^{\prime}$ is
the adversarial example, $\|\mathbf{x}^{\prime}-\mathbf{x}\|$ represents the
size of perturbation, and $\epsilon$ is the maximum perturbation. Some
examples and corresponding adversarial examples are represented in Fig. 2.

###  II-A Adversarial attack

According to whether we can know the model’s structure and weights, the
adversarial attacks can be divided into two types: white-box attack [7, 6, 19,
11] and black-box attack [20, 21]. In a white-box attack, the adversary knows
the model structure and weights when attacking, such as FGSM [20, 21],
DeepFool [11], PGD [6], CW [12]. FGSM is a gradient-based and one-step attack
method that updates along the direction of the pixel gradient’s signal
function to obtain the adversarial examples. DeepFool looks for the minimum
distance from a clean sample to an adversarial example and defines this
distance as the model’s adversarial robustness. DeepFool takes advantage of a
linear approximation method to generate adversarial examples iteratively. CW
[12] is a non-gradient based adversarial attack, one of the most powerful
attacks. Carlini et al. [12] propose several objective functions to generate
adversarial examples, among which the most effective attack method can
effectively attack Distillation Defense [4]. From the perspective of robust
optimization, Madry et al. [6] study the model’s adversarial robustness,
formulate the adversarial training process of the model as a saddle point
problem, and use projected gradient descent to search for more aggressive
adversarial examples. They prove that the PGD method is the strongest attack
method among the first-order attack methods.

In a black-box attack, the attacker knows nothing about the model except for
the model’s output. Generally, one can attack these models by taking advantage
of the transferability of adversarial examples [20, 22] or to generate
adversarial examples with GAN [21]. The so-called transferability of
adversarial examples means that adversarial examples of networks can attack
neural networks with different structures, and even the two networks are
trained on different datasets [22]. Liu et al. [20] study the transferability
of adversarial examples on the large-scale network model and large-scale
dataset. They find that the adversarial examples with the non-targeted attack
are easy to generate, while the adversarial examples with targeted attack
could hardly be transferred to the target label. A GAN based algorithm is
proposed [21] to generate adversarial examples. The combination of image and
text domains allows the generated adversarial examples to be more natural and
interpretable, which helps understand the black-box model’s local behaviour.

(a)  (b)

Figure 2: There are clean examples (left) and adversarial examples (right),
which were generated by FGSM [7]. (a) Some examples from MNIST and their
corresponding adversarial examples generated by FGSM with $\epsilon=0.3$. (b)
Some examples from CIFAR10 and their corresponding adversarial examples
generated by FGSM with $\epsilon=0.03$.

###  II-B Adversarial training

Adversarial training is one of the most effective defense methods. Since the
model is required to classify the adversarial examples correctly, the
adversarial examples can be generated and added to the training set to retrain
the model. The framework of adversarial training is shown in Fig. 1. Many
defense works are based on this framework. For example, Goodfellow et al. [7]
use FGSM to generate adversarial examples to retrain the model. They point out
that the adversarial training procedure can be seen as minimizing the worst-
case error when data is perturbed by an adversary. After adversarial training,
the adversarial robustness of the model is significantly improved. Some
conclusions are drawn in [23]. However, both [7] and [23] conduct their
experiments only on the MNIST dataset. Some more complicated experiments on
ImageNet are conducted by [24]. When they train their networks, they utilize
adversarial examples and clean examples in each training step. They present
that adversarial training increases deep neural networks’ robustness against
one-step attacks but would not help under iterative attacks. To search more
powerful attack method, Madry et al. [6] mention that adversarial training
could be formulated as a minimum-maximization problem and propose the PGD
attack method based on projected gradient descent. They prove that their
method is the most powerful among the first-order attack methods. The model
based on adversarial training with PGD obtains high performance against
adversarial examples [13].

Nowadays, many defense methods are based on the minimum-maximum optimization
problem mentioned in [6]. Liu et al. [25] propose a novel single-step
adversarial training method, which can resist single-step and iterative
adversarial examples. Zhang et al. [26] propose TRADES, the minimization
algorithm of the strictest upper limit in theoretical probability distribution
and measurable predictive variables, and win the first place in NeurIPS2018.
Moreover, recently, some works [27, 28, 29, 30] propose to reduce the
computation of adversarial examples, such as [30] proposes a linear
regularization to solve the problem that the computational cost of adversarial
training grows prohibitively as the size of the model and number of input
dimensions increase. Moreover, some works [3, 13] point out that adversarial
training would cause obfuscated gradients. Athalye et al. [13] identify
obfuscated gradients would cause a false sense of security in defenses against
adversarial examples. According to this weakness, they successfully attack
eight methods which were proposed in ICLR 2018.

###  II-C Cost-sensitive learning

Cost-sensitive learning [17] is a common algorithm to solve unequal
misclassification or unbalanced data learning. The main idea is: although a
model can improve the overall performance, some problems bring more serious
consequences than others. In other words, misclassification of different
problems may bring different levels of consequences. If an important
performance cannot be guaranteed by only considering the overall performance,
it may bring unimaginable bad effects. For example, in medical practice, it is
clear that misdiagnosing a person with real cancer as a healthy person is much
higher than the cost of misdiagnosing a healthy person as a cancer patient.
Since cancer patients are in the minority in real life, the method tends to
diagnose patients as healthy. Cost-sensitive learning is an algorithm to solve
such problems. Its core element is the cost matrix. Taking medical diagnosis
as an example, its cost-sensitive matrix may be written as the following form:

$C=\begin{bmatrix}0&100\\\ 1&0\end{bmatrix}$

where the column represents the actual label, and the row represents the
predictive label such as $C(1,2)$ represents the cost of diagnosing a cancer
patient as a healthy person.

Many existing works on cost-sensitive learning [17, 18, 31, 32, 33, 34, 35,
36] fall roughly into two categories. One is to adjust the samples’
distribution [17, 32, 33, 35, 36]. It transforms the frequencies of categories
to proportions according to the cost of misclassification. The advantage is
that the change of sample distributions may affect the performance of the
algorithm. The other is meta-cost learning [18, 31, 34], a method to transform
the general classification model into a cost-sensitive model. Kukar et al.
[31] firstly apply cost-sensitive learning to neural networks. Although there
are many works on cost-sensitive learning, there are few studies on cost-
sensitive adversarial learning. In [37], cost-sensitive learning is first
applied to adversarial training, and a minimum-maximization method for
generating robust classifiers is proposed. This method can directly minimize
the error cost of convex optimization problems, but it is only applicable to
linear classifiers. Asif et al. [37] encode each adversarial transformation
into a matrix called the cost matrix. They then use the adversarial robust
learning method in [16] to propose a new objective function for training the
cost-sensitive robust classifier. Terzi et al. [15] propose the WPGD method
and use it in Adversarial Training. It provides a simple method to make the
model cost-sensitive to control the balance of accuracy-robustness.

##  III Adversarial learning with cost-sensitive classes

Section III-A gives some basic symbol definitions such as empirical risk
minimization, adversarial training, and cost-sensitive learning. Then, we
formally describe the problem this paper solved. Section III-B shows the
detail about the CSA algorithm, which combines cost-sensitive learning and
adversarial training to improve the model’s adversarial robustness. Actually,
by analyzing the convolutional layer’s parameters, we find some
characteristics of the model trained by adversarial training. For the sake of
description, we abbreviate these characteristics as the Min-Max property.
According to the MinMax property, we propose a new algorithm that could
improve the model’s adversarial robustness without adversarial training,
called CSE. The CSE algorithm is described in Section III-C.

###  III-A Symbol definition and problem description

##### Empirical Risk Minimization

Given a training set with $N$ samples
$D=\\{\mathbf{x}^{(i)},y^{(i)}\\}^{N}_{i=1}\subset R^{n}\times Y$ where
$Y=\\{0,1,\dots,m-1\\}$ and $m$ is the number of classes, we can train a
classifier by machine learning $f:R^{n}\rightarrow[0,1]^{m}$, where
$\hat{y}=\mathop{\arg\max}_{i}f_{i}(\mathbf{x})$ represents the prediction
result of the classifier on sample
$(\mathbf{x},y)\in\\{\mathbf{x}^{(i)},y^{(i)}\\}^{N}_{i=1}$. For the
classification problem of $m$ categories, the neural network is used to
represent the classifier, and its training process can be described as the
following optimization problem:

$\min\mathop{E}_{(\mathbf{x},y)\backsim
D}[L(f(\mathbf{x}),y)]=\min\sum_{i=1}^{N}L(f(\mathbf{x}^{(i)}),y^{(i)})$ (1)

where $L$ represents the loss function.

##### Adversarial Training

The set of adversarial examples concerning a sample $(\mathbf{x},y)$ is
defined as
$B_{l}(\mathbf{x},y,f)=\\{\mathbf{x}_{adv}|\quad\|\mathbf{x}_{adv}-\mathbf{x}\|_{l}\leq\epsilon\quad
and\quad\arg\max_{j}f_{j}(\mathbf{x}_{adv})\neq y\\}$, where $l$ represents
$l$-norm. Then, the process of its adversarial training can be formulated as
follows:

$\min\mathop{E}_{(\mathbf{x},y)\backsim D}[\max_{\mathbf{x}_{adv}\in
B_{l}}L(f(\mathbf{x}_{adv}),y)]$ (2)

##### Cost-Sensitive Learning

The cost of categorization is usually represented by a cost matrix $C$ as
below:

$C(i,j)=\begin{cases}e_{ij}\quad&i\neq j\\\ 0\quad&i=j\end{cases}$

where $e_{ij}$ represents the cost of misclassifing an example from class $j$
to class $i$.

For any sample $\mathbf{x}$ belonging to class $j$, the optimal decision is
equivalent to minimizing the following loss function:

$L(\mathbf{x},j)=\sum_{i}P(i|\mathbf{x})C(i,j)$

That is to say, $L(\mathbf{x},j)$ represents the cost expectation of the class
$j$ predicted by the model under the given sample $\mathbf{x}$.

##### The Proposed Problem Formulation

Given a training set with $N$ samples,
$D=\\{\mathbf{x}^{(i)},y^{(i)}\\}^{N}_{i=1}$, where a special class, say the
$p$-th class we need to protect. Our goal is to train a classifier
$f:X\rightarrow Y$, which can classify any clean sample
$(\mathbf{x},y)\in\\{\mathbf{x}^{(i)},y^{(i)}\\}^{N}_{i=1}$ while has stronger
robustness against adversarial examples $\mathbf{x}_{adv}\in
B_{l}(\mathbf{x},y_{p},f)$ in the condition of adversarial attack. In short,
when improving the overall robustness of the classifier $f$, category $p$ is
given priority to ensure the robustness.

###  III-B Cost-Sensitive Adversarial Model (CSA)

This subsection introduces cost-sensitive learning and adversarial training.
Then, we proposes a cost-sensitive adversarial model (hereinafter referred to
as CSA model for convenience). CSA model can effectively improve the
robustness of a certain class $p$ in the classification problem against
adversarial attacks.

Note that we want to protect class $p$ and improve the robustness of the model
regarding class $p$. To solve this problem, two subquestions need to be
answered:

  * •

How to improve the overall robustness of the model?

  * •

How to give priority to improve the robustness of class $p$ in the model?

Intuitively, we can improve the overall robustness of the model through
adversarial training. Simultaneously, to particularly improve the robustness
regarding class p, we add the cost matrix under the framework of the
adversarial training and use the cost matrix to indicate which classes of the
classifier need to be specially protected. In conclusion, the cost of model
misclassification of class $p$ into non-$p$ or non-$p$ into $p$ is higher than
that of other misclassification. So, we define the following cost matrix:

$C(i,j)=\begin{cases}0\quad&i=j,\\\ c\quad&i=p\ or\ j=p,\\\
1\quad&else.\end{cases}$ (3)

where $p$ is the protected class number, $j$ represents the true label, and
$i$ represents the class number predicted by the classifier. This matrix
indicates that the cost is zero when the classifier correctly identifies the
sample. When a classifier misclassifies a $p$ class as a non-$p$ class or
misclassifies a non-$p$ class as a $p$ class, the cost is $c$. And the cost is
1 in other conditions.

$\min\mathop{E}_{(\mathbf{x},y)\backsim D}[\max_{\mathbf{x}_{adv}\in
B_{l}}L(f(\mathbf{x}_{adv}),y)+\sum_{i}f_{i}(\mathbf{x}_{adv})C(i,y)]$ (4)

The first term in Eq. 4 is a traditional loss function, which can be the
cross-entropy function, square deviation, or any existing loss function. It
mainly plays a role in improving the overall performance of the model. The
second term represents the expected output cost of the model. When $c=1$, the
CSA model degenerates back to a standard adversarial training model.

Eq. 4 is a non-convex problem. Thus finding a solution is a bit of challenge.
There is a natural explanation for adversarial training proposed in [2]; that
is, an attack method is a solution to the internal max part of Eq. 4. In
contrast, the external min part works on the whole data set, making the
model’s loss the least. For example, Eq. 5 is an attack the method called FGSM
[7].

$(\mathbf{x}_{adv})_{FGSM}=\mathbf{x}+\epsilon sign(\bigtriangledown
L(f(\mathbf{x}),y))$ (5)

Then, Eq. 4 can be solved by the follow:

$\min\mathop{E}_{(\mathbf{x},y)\backsim
D}[L(f(\mathbf{x}_{fgsm}),y)+\sum_{i}f_{i}(\mathbf{x}_{fgsm})C(i,y)]$ (6)

The sensitivity of the model to the protected category depends on the cost
matrix $C$. Obviously, the cost of the protected category misclassified by the
model is higher than that of the general category. When $C(i,j)=1,i\neq j$,
the model’s penalty for misclassification of any category is the same. The
implementation of the CSA algorithm is presented in Algorithm 1.

Algorithm 1 CSA Algorithm

1: Initialize parameters of network $\mathbf{\theta}$ with random weights

2: Initialize dataset $D=\\{\mathbf{x}^{(i)},y^{(i)}\\}^{N}_{i=1}$

3: Initialize the batchsize $B$, epochs $T$

4: for i=1 to $T$ do

5: Initialize cost-sensitive matrix $C$

6: Sample $B$ examples $Q_{1}=\\{(\mathbf{x}^{(i)},y^{(i)})\\}_{1}^{B}$

7: Get adversarial example $Q_{2}=\\{(\mathbf{x}_{adv},y)|(\mathbf{x},y)\in
Q_{1}$ }

8: Stage 1: Update $\mathbf{\theta}$ with clean examples $({\mathbf{x}},y)\in
Q_{1}$

9: Updated $\mathbf{\theta}$ by minimizing loss function Eq. (4)

10: Stage 2: Update $\mathbf{\theta}$ with adversarial examples
$({\mathbf{x}_{adv}},y)\in Q_{2}$

11: Updated $\mathbf{\theta}$ by minimizing loss function Eq. (4)

12: end for

13: Output $\mathbf{\theta}^{*}$

###  III-C Cost-Sensitive Adversarial Extension (CSE)

This subsection first gives a explanation of the adversarial robustness of the
CSA model, then gives an extension of CSA model, called CSE model. Compared
with the CSA model, a significant feature is the CSE model does not need
adversarial training to make the model robust.

####  III-C1 Empirical Observations on CSA

(a)

(b)

Figure 3: CSA & ADV: The parameter’s size in the first convolutional layer of
the LeNet network (Sort from left to right, top to bottom).

To further study the principle of the CSA model, we analyze the difference
between the CSA model and the ADV model based on the parameters of the
convolutional layer (ADV represents the model that is training with standard
adversarial learning. Turn to Section IV for details). Fig.3a is the parameter
distribution diagram of the convolution of the CSA model and the ADV model in
the first layer. Fig. 3b arranges the convolution parameters of the CSA model
and ADV model from left to right and top to bottom. Fig. 3a shows that in the
first convolution parameter of the CSA model, most values are relatively
small, and the number of values close to 0 is several times that of the ADV
model. Fig. 3b shows that in both the CSA model and ADV model, part of the
parameters in the first convolutional layer are relatively high, and the value
of the CSA model is higher than that of the ADV model.

In order to further observe the influence of parameter distribution of
convolutional layer, we added the L2 norm term to the original CSA model,
hoping that the absolute values of the parameter of model CSA+L2 would become
a little smaller to observe what happens to the model’s adversarial
robustness. Eq. 7 is the new optimization equation.

$\min\mathop{E}_{(\mathbf{x},y)\backsim
D}[L(f(\mathbf{x}_{fgsm}),y)+\sum_{i}f_{i}(\mathbf{x}_{fgsm})C(i,y)+\|\theta_{f}\|_{2}]$
(7)

(a)

(b)

Figure 4: CSA+L2 & CSA & ADV: The parameter’s size in the first convolutional
layer of the LeNet network (Sort from left to right, top to bottom).

Through these experiments, the protective effect of three models on category
$p$ is: $ADV<CSA+L2<CSA$. Section IV shows the experimental details. Fig. 4a
is the distribution diagram of parameters of the first convolutional layer of
CSA model, CSA+L2 model, and ADV model. From Figs. 4a and 4b, we find that the
parameters of the first layer of the convolutional layer have the following
features:

  * •

The number of parameter values approaching zero: $ADV<CSA+L2<CSA$;

  * •

Among the many parameters, the absolute values of a small number of parameters
are relatively large, and for the parameter values of the three models at the
corresponding positions, we have $ADV<CSA+L2<CSA$

Therefore, we give the following two propositions based on the above
experimental findings:

###### Proposition 1

The model has adversarial robustness possibly due to a Min-Max property which
parameters follow, i.e., the absolute values of most parameters in the
convolutional layer approach zero while the absolute values of a few
parameters are significantly larger than others.

###### Proposition 2

When the model predicts sample, the parameters that have real impaction on
prediction result are only a part.

Two above-mentioned observations and analysis give some explanations why the
CSA model can improve the robustness of category $p$. The adversarial training
gives the Min-Max property to the parameters in the model. The cost matrix can
locate the parameters that play a decisive role in the prediction results of
category $p$. When the adversarial training is combined with the cost matrix,
the positioning effect of the cost matrix makes these parameters having the
Min-Max property during the training to improve the robustness of the category
$p$.

####  III-C2 Extension of CSA model (CSE)

This section further mines the information of Proposition 1 and uses this
information to design the CSE model.

Proposition 1 shows that, if the model parameters have the Min-Max property,
then the model has stringer adversarial robustness. Through the analysis in
the previous section, we give two features of the Min-Max property:

###### Feature 1

Most of the parameters in the convolutional layer of the model tend to zero.

###### Feature 2

In the convolutional layer parameters of the model, some are very large
relatively.

Now, we analyze Features 1 and 2 in convolutional layers from the viewpoint of
uncertainty. Let $\mathbf{W_{conv}}$ represent the parameters of convolutional
layers. According to [38, 39], the fuzziness vector can be defined as

$\displaystyle\begin{split}Fuzziness(\mathbf{W_{conv}})=-\frac{1}{N}\sum_{1}^{N}[\rho_{i}log\rho_{i}+\\\
(1-\rho_{i})log(1-\rho_{i})],\end{split}$ (8)

where $N$ is the size of vector $\mathbf{W_{conv}}$ and $\rho_{i}$ is defined
as follows:

$\displaystyle\rho_{i}=\frac{1}{1+e^{|w_{i}|}}$ (9)

By minimizing Eq. 8, we have

$w_{i}\rightarrow 0\Rightarrow\rho_{i}\rightarrow 1,$
$w_{i}\rightarrow\infty\Rightarrow\rho_{i}\rightarrow 0.$

Therefore, we can train a model with Features 1 and 2 by minimizing Eq. 8.
Then, we have

$\displaystyle\min\mathop{E}_{(\mathbf{x},y)\backsim D}[$ $\displaystyle
L(f(\mathbf{x}),y)+\sum_{i}f_{i}(\mathbf{x})C(i,y)+$ (10) $\displaystyle\gamma
Fuzziness(\mathbf{W_{conv}})]$

where $\gamma$ is a hyperparameter. Compared with Eq. 4, Eq. 10 omits the
calculation of the Max function but adds a regular terms
$Fuzziness(\mathbf{W_{conv}})$. The implementation of the CSE algorithm is
presented in Algorithm 2.

In a word, the CSE is a novel method of defense against adversarial attacks.
The difference between the CSE model and the traditional adversarial defense
methods (such as adversarial training) is that the CSE model does not need to
calculate the Max function inside the optimization formula, which greatly
reduces the training time of the model.

Algorithm 2 CSE Algorithm

1: Initialize parameters of network
$f=(\mathbf{x};\mathbf{\theta}=[\mathbf{\theta}_{conv},\mathbf{\theta}_{other}])$
with random weights

2: Initialize dataset $D=\\{\mathbf{x}^{(i)},y^{(i)}\\}^{N}_{i=1}$

3: Initialize the batchsize $B$, learning rate $\alpha$, epochs $T$, gamma
$\gamma$

4: for i=1 to $T$ do

5: Initialize cost-sensitive matrix $C$

6: Sample $B$ examples $Q=\\{(\mathbf{x}^{(i)},y^{(i)})\\}_{1}^{B}$

7: Calculate entropy loss $l_{1}=\sum_{1}^{B}{L(f(x),y)}$

8: Calculate cost-sensitive loss
$l_{2}=\sum_{1}^{B}{\sum_{i}{f_{i}(x)C(i,y)}}$

9: $l_{3}=\gamma Fuzziness(\mathbf{\theta}_{conv})$

10: Then, $L=l_{1}+l_{2}+l_{3}$

11: Update parameters of network
$\mathbf{\theta}\leftarrow\mathbf{\theta}-\alpha\triangledown L$

12: end for

13: Output $\mathbf{\theta}^{*}$

###  III-D Min-Max property

The Min-Max property is first discovered by Shen et al. in [38]. Shen et al.
propose that the neural network models with Min-Max property have stronger
adversarial robustness. The Min-Max property means that, after using a
combination between L1 and L2 normalizations as the loss function, the
training process will result in such a phenomenon that the weights in
convolutional layers will tend towards zero (the minimum) or a maximum value.
This paper considers more complex architectures of neural networks to enhance
the representation ability and advocate to measure Min-Max property by
minimizing fuzziness of convolutional layers.

It is worth noting that the Min-Max property is similar to an off-center
technique proposed in metric learning [40]. The off-center technique is to
achieve a better representation in feature space based on such an idea that,
given the center being 0.5, the similarity after transformation is require to
tend towards zero (the minimum) or one (the maximum) if the similarity before
the transformation is less than or more than 0.5, respectively. It is observed
that the parameters are close to zero or far further away from zero after
several rounds of adversarial training in LeNet. It is an interesting
observation which confirms from the angle of off-center that convolution can
indeed summarize the features from low to high levels while high-level
features have more representative abilities.

The process of training neural networks such that the weights in convolutional
layer weights going to extremes can be implemented in different ways, e.g., by
minimizing the uncertainty of the convolutional layer [38] or by adding the
regularization term in the loss function. Furthermore, it is found that a DNN
with strong robustness against adversarial examples may not have the Min-Max
property. However, based on the observation from a considerable number of
simulations, a DNN with the Min-Max property usually has strong robustness
against adversarial examples where the adversarial robustness means a
tolerance to adversarial noise.

We recall some results in [38] where two neural network models with simple
architectures are considered.

  * •

$Model\\#1$:
$L_{a}(\mathbf{x};\mathbf{W_{a}})=-\mathbf{y}^{T}log(softmax(\mathbf{W_{a}}\mathbf{x}))$,
where $W_{a}=(a_{ij})$ and $a_{ij}\sim U(0,1)$.

  * •

$Model\\#2$:
$L_{b}(\mathbf{x};\mathbf{W_{b}})=-\mathbf{y}^{T}log(softmax(\mathbf{W_{b}}\mathbf{x}))$,
where $W_{b}=(b_{ij})$ and $b_{ij}\sim B(1,p_{1})$. And $B(1,p_{1})$ is the
binomial distribution. $p_{1}$ represents the probability of $b_{ij}=1$ where
$p_{1}>0,p_{1}<<1$.

Based on both models, a theorem regarding the normal and bi-nominal
distributions of weight parameters was given as follows:

###### Theorem 1

Suppose $\mathbf{x}$ is a vector and $\mathbf{x}_{i}$ is the $i$th subscript
in $\mathbf{x}$, $\forall i$, the following inequality holds

$\lvert\frac{\partial\mathbb{E}_{b_{ij}\sim
B(1,p_{1})}L_{b}(\mathbf{x};\mathbf{W_{b}})}{\partial\mathbf{x}_{i}}\rvert\\\
\leq\lvert\frac{\partial\mathbb{E}_{a_{ij}\sim
U(0,1)}[L_{a}(\mathbf{x};\mathbf{W_{a}})]}{\partial\mathbf{x}_{i}}\rvert$

.

According to this theorem, we have the following inequality:

$\displaystyle\begin{split}\lvert
L_{b}(\mathbf{x};\mathbf{W_{b}})-L_{b}(\mathbf{x+\Delta};\mathbf{W_{b}})\rvert\leq\\\
\lvert
L_{a}(\mathbf{x};\mathbf{W_{a}})-L_{a}(\mathbf{x+\Delta};\mathbf{W_{a}})\rvert\end{split}$
(11)

which indicates that $Model\\#2$ (with Min-Max property) is really having the
adversarial robustness stronger than $Model\\#1$ (without the Min-Max
property).

##  IV Experiments

In this section, we use Python3.6 and Jupyter Notebook to implement our
algorithms. Using Advertorch [41], we adopt some adversarial attacks (FGSM
[7], PGD [6], CW [12], MIA [42], L2BIA [42] and LinfBIA [42]) to evaluate the
adversarial robustness of the model. We compare the performance of the
following four models:

  * •

Standard model (STD): This model is trained with clean examples by adopting
cross-entropy as the loss function.

  * •

The extension of the cost-sensitive adversarial model (CSE): This is a model
trained with clean examples by adopting a loss function (Eq.10).

  * •

Cost-sensitive adversarial model (CSA): This is an adversarial training model
trained with Eq. 4. The adversarial examples are generated by PGD.

  * •

A model combined CSE and adversarial training (CSE+ADV): This is a adversarial
training model trained with Eq. 10. The adversarial examples are generated by
PGD.

We evaluate these models on three standard datasets MINST [43], CIFAR10 [44]
and CIFAR100 [44]:

  * •

MNIST dataset consists of 60,000 training samples and 10,000 samples, each of
which is a $28\times 28$ pixel handwriting digital image.

  * •

CIFAR10 dataset consists of 60,000 32x32 colour images in 10 classes, with
6000 images per class. There are 50,000 training images and 10,000 testing
images.

  * •

CIFAR100 is just like the CIFAR10, except it has 100 classes containing 600
images each. There are 500 training images and 100 testing images per class.
Some examples are shown in Fig. 2.

In the experiments, all the adversarial perturbations are limited to a
$l_{\infty}$-norm ball. Let $M_{p}$ represent the model $M$ which particularly
protects category $p$. Before training network, samples from datasets will be
regularized. The preprocessing is described below:

$\mathbf{x}=\frac{\mathbf{x}-\mathbf{\mu}}{\mathbf{\sigma}}$

where the $\mathbf{\mu}$ and $\mathbf{\sigma}$ in different dataset are shown
in Tabel I.

TABLE I: The mean value and standard deviation in MNIST, CIFAR10 and CIFAR100 | $\mathbf{\mu}$ | $\mathbf{\sigma}$ |  |
---|---|---|---|---
MNIST | [0.1307] | [0.3081] |  |
CIFAR10 | [0.4914, 0.4822, 0.4465] | [0.2023, 0.1994, 0.2010] |  |
CIFAR100 | [0.5070, 0.4865, 0.4409] | [0.2673, 0.2564, 0.2761] |  |

###  IV-A MINST

TABLE II: MNIST: The accuracy of protected categories under various attack methods. P represents the category of protection. And $O_{i}$ presents the accuracy of category $i$ for the data in the table. |  |  | The protected class: $p$. Only showing the accuracy of protected class.
---|---|---|---
Attacks |  ADV Training | Models |  $O_{0}$, p=0 |  $O_{1}$, p=1 |  $O_{2}$, p=2 |  $O_{3}$, p=3 |  $O_{4}$, p=4 |  $O_{5}$, p=5 |  $O_{6}$, p=6 |  $O_{7}$, p=7 |  $O_{8}$, p=8 |  $O_{9}$, p=9
FGSM | No | STD | 0.8771 | 0.9491 | 0.7667 | 0.8401 | 0.7230 | 0.8573 | 0.8381 | 0.6053 | 0.8227 | 0.6830
CSE | 0.9448 | 0.9647 | 0.9109 | 0.7696 | 0.8849 | 0.8574 | 0.8424 | 0.9052 | 0.9048 | 0.7099
Yes | CSA | 0.8894 | 0.9679 | 0.9363 | 0.9099 | 0.9562 | 0.8827 | 0.9437 | 0.9057 | 0.9254 | 0.9287
CSE+ADV | 0.9566 | 0.9792 | 0.9426 | 0.9229 | 0.9013 | 0.9311 | 0.9642 | 0.9443 | 0.9406 | 0.8953
PGD | No | STD | 0.7157 | 0.7749 | 0.4019 | 0.6155 | 0.2730 | 0.5787 | 0.6061 | 0.1425 | 0.3767 | 0.2096
CSE | 0.8740 | 0.7954 | 0.7341 | 0.6293 | 0.7497 | 0.7284 | 0.5535 | 0.7187 | 0.5696 | 0.2096
Yes | CSA | 0.8336 | 0.9066 | 0.7978 | 0.8023 | 0.7398 | 0.7094 | 0.8664 | 0.7841 | 0.8023 | 0.7404
CSE+ADV | 0.8508 | 0.9553 | 0.9099 | 0.8667 | 0.9225 | 0.8961 | 0.9367 | 0.9137 | 0.8805 | 0.9069
CW | No | STD | 0.5837 | 0.9036 | 0.4403 | 0.6052 | 0.5519 | 0.6539 | 0.5500 | 0.5192 | 0.2413 | 0.3317
CSE | 0.7497 | 0.9118 | 0.6314 | 0.4859 | 0.6105 | 0.6614 | 0.4966 | 0.7477 | 0.4196 | 0.3317
Yes | CSA | 0.4524 | 0.6934 | 0.6194 | 0.6362 | 0.7059 | 0.6969 | 0.7442 | 0.7296 | 0.4248 | 0.4982
CSE+ADV | 0.7070 | 0.8541 | 0.7428 | 0.6548 | 0.5446 | 0.6247 | 0.7448 | 0.7926 | 0.5427 | 0.4781
MIA | No | STD | 0.7509 | 0.8342 | 0.4107 | 0.6717 | 0.3211 | 0.6280 | 0.6180 | 0.1795 | 0.4031 | 0.2629
CSE | 0.8455 | 0.5360 | 0.6835 | 0.5440 | 0.7150 | 0.6397 | 0.4971 | 0.6592 | 0.5031 | 0.2629
Yes | CSA | 0.8403 | 0.9171 | 0.8085 | 0.8271 | 0.7706 | 0.7162 | 0.8656 | 0.8089 | 0.8294 | 0.7680
CSE+ADV | 0.8771 | 0.9542 | 0.9049 | 0.8737 | 0.9305 | 0.8964 | 0.9462 | 0.9141 | 0.8811 | 0.9073
L2BIA | No | STD | 0.9926 | 0.9795 | 0.9647 | 0.9532 | 0.9467 | 0.9504 | 0.9672 | 0.9613 | 0.9477 | 0.9729
CSE | 0.9738 | 0.9927 | 0.9762 | 0.9859 | 0.9760 | 0.9789 | 0.9693 | 0.9810 | 0.9827 | 0.9729
Yes | CSA | 0.9834 | 0.9934 | 0.9918 | 0.9879 | 0.9935 | 0.9936 | 0.9800 | 0.9827 | 0.9884 | 0.9916
CSE+ADV | 0.9893 | 0.9935 | 0.9881 | 0.9854 | 0.9819 | 0.9852 | 0.9911 | 0.9872 | 0.9925 | 0.9813
LinfBIA | No | STD | 0.7485 | 0.7075 | 0.3370 | 0.6562 | 0.2400 | 0.5700 | 0.5774 | 0.1222 | 0.3371 | 0.1734
CSE | 0.8845 | 0.8519 | 0.7626 | 0.6330 | 0.7651 | 0.7375 | 0.5640 | 0.7542 | 0.5900 | 0.2010
Yes | CSA | 0.8375 | 0.8940 | 0.7771 | 0.8029 | 0.7197 | 0.7129 | 0.8531 | 0.7704 | 0.7952 | 0.7182
CSE+ADV | 0.8388 | 0.9533 | 0.9031 | 0.8670 | 0.9229 | 0.8868 | 0.9350 | 0.9142 | 0.8650 | 0.9097

For the MNIST dataset, we adopt LeNet [45] as our network structure. LeNet
network is composed of two convolutional layers, two pooling layers and two
full connection layers. Two convolutional layers contain 5 and 16 convolution
kernels respectively. And the convolutional layer is followed by a pooling
layer, the padding is set to 2. The number of two full connection layer hidden
nodes are 120 and 84, respectively. RELU [46] function is adopted as the
activation function in the network. During the training stage, the batch size
is 256, and the learning rate is 0.01, the epoch is 20, momentum is 0.95. For
the CSA model and CSE+ADV model, the first 10 epochs are trained with clean
samples, and the last 10 epochs are trained with adversarial examples
generated by PGD. Maximum perturbation of the attack $\epsilon=0.3$. We set
the constant $c=10$.

We respectively conduct 10 experiments for CSA, CSE and $CSE+ADV$ to protect
each class (0-9). Table II is the result of various models on the testing
dataset, where $\epsilon=0.3$. We only show the accuracy of the particularly
protected class. For example, ”$O_{1},p=1$” represents the accuracy of class
”one” in a model that is trained under protecting class ”one”. However, the
STD model, as a baseline, does not adopt any protection strategy. It can be
seen from the table that the CSA, CSE and CSE + ADV can effectively improve
the adversarial robustness of the model than STD. Compared the CSE with STD
model, we find that CSE can achieve high adversarial robustness although both
models do not use adversarial training. When adding adversarial training, it
is worth-noted that CSE + ADV achieves better performance than CSA in most
scenarios. These results experimentally validated that the Min-Max property of
the model can improve the model’s adversarial robustness. Moreover, it is an
effective way to reduce the fuzziness of parameters in convolutional layers.

Figs. 5a, 5b and 5c show the change trend of loss function, fuzziness of
parameters in convolutional layer and the robustness. The protected label is
set to the first class. We find that CSE is harder to converge than STD. It
would take more time for training CSE than STD. However, shown in Fig.5b, as
the number of iterations increases, the fuzziness of convolutional layer
decreases gradually in CSE but remaining stable in STD. This phenomenon
demonstrates that our method can effectively reduce the uncertainty of
parameters which gives a model Min-Max property which reflects in the result
that the adversarial robustness of CSE is stronger than STD as shown in Fig.
5c. Besides, we note that the fuzziness of convolutional layer in the STD and
CSE is around 0.7 at beginning. This is due to the fact that the parameters
are initialized randomly in a same way.

(a) The loss on MNIST.

(b) The fuzziness of first convolutional layer on MNIST.

(c) The robustness on MNIST. Evaluation under PGD.

(d) The loss on CIFAR10.

(e) The fuzziness of first convolutional layer on CIFAR10.

(f) The robustness on CIFAR10. Evaluation under PGD.

Figure 5: The changing trend of the loss function, fuzziness and robustness on
MNIST and CIFAR10.

###  IV-B CIFAR10

TABLE III: CIFAR10: The accuracy of protected categories under various attack methods. P represents the category of protection. And $O_{i}$ presents the accuracy of category $i$ for the data in the table. |  |  | The protected class: $p$. Only showing the accuracy of protected class.
---|---|---|---
Attacks |  ADV Training | Models |  $O_{0}$, p=0 |  $O_{1}$, p=1 |  $O_{2}$, p=2 |  $O_{3}$, p=3 |  $O_{4}$, p=4 |  $O_{5}$, p=5 |  $O_{6}$, p=6 |  $O_{7}$, p=7 |  $O_{8}$, p=8 |  $O_{9}$, p=9
FGSM | No | STD | 0.3093 | 0.3258 | 0.1539 | 0.0697 | 0.1169 | 0.1561 | 0.2877 | 0.3170 | 0.3320 | 0.2823
CSE | 0.4269 | 0.4399 | 0.2212 | 0.0862 | 0.1325 | 0.2136 | 0.3580 | 0.4029 | 0.4566 | 0.3748
Yes | CSA | 0.5671 | 0.6010 | 0.3371 | 0.2204 | 0.3082 | 0.3644 | 0.5726 | 0.5746 | 0.6173 | 0.5467
CSE+ADV | 0.5582 | 0.5904 | 0.3423 | 0.1982 | 0.2817 | 0.3539 | 0.5937 | 0.5497 | 0.6284 | 0.5260
PGD | No | STD | 0.2190 | 0.2584 | 0.0918 | 0.0270 | 0.0439 | 0.1065 | 0.1804 | 0.2586 | 0.2424 | 0.2144
CSE | 0.3628 | 0.4059 | 0.1975 | 0.0682 | 0.0810 | 0.1770 | 0.3118 | 0.3881 | 0.4039 | 0.3561
Yes | CSA | 0.5485 | 0.6015 | 0.3344 | 0.2154 | 0.2883 | 0.3531 | 0.5699 | 0.5493 | 0.6123 | 0.5614
CSE+ADV | 0.5677 | 0.5602 | 0.3186 | 0.1904 | 0.2770 | 0.3431 | 0.5850 | 0.5509 | 0.6115 | 0.5104
CW | No | STD | 0.0000 | 0.0000 | 0.0000 | 0.0000 | 0.0000 | 0.0000 | 0.0000 | 0.0000 | 0.0000 | 0.0000
CSE | 0.0000 | 0.0000 | 0.0000 | 0.0000 | 0.0000 | 0.0000 | 0.0000 | 0.0000 | 0.0000 | 0.0000
Yes | CSA | 0.0000 | 0.0000 | 0.0000 | 0.0000 | 0.0000 | 0.0000 | 0.0000 | 0.0000 | 0.0000 | 0.0000
CSE+ADV | 0.0000 | 0.0000 | 0.0000 | 0.0000 | 0.0000 | 0.0000 | 0.0000 | 0.0000 | 0.0000 | 0.0000
MIA | No | STD | 0.2041 | 0.2546 | 0.0988 | 0.0322 | 0.0404 | 0.1049 | 0.1752 | 0.2446 | 0.2255 | 0.2051
CSE | 0.3702 | 0.4069 | 0.1961 | 0.0618 | 0.0756 | 0.1815 | 0.3011 | 0.3703 | 0.3837 | 0.3463
Yes | CSA | 0.5449 | 0.5917 | 0.3260 | 0.1997 | 0.3000 | 0.3370 | 0.5492 | 0.5455 | 0.6236 | 0.5554
CSE+ADV | 0.5636 | 0.5791 | 0.3101 | 0.1849 | 0.2851 | 0.3367 | 0.5845 | 0.5501 | 0.6105 | 0.5116
L2BIA | No | STD | 0.6796 | 0.7365 | 0.4834 | 0.4313 | 0.5427 | 0.4984 | 0.6787 | 0.6666 | 0.7128 | 0.6816
CSE | 0.7044 | 0.7411 | 0.4936 | 0.3861 | 0.5114 | 0.4915 | 0.6854 | 0.6891 | 0.7761 | 0.6783
Yes | CSA | 0.7020 | 0.7439 | 0.4935 | 0.3987 | 0.5277 | 0.5175 | 0.7424 | 0.6842 | 0.7612 | 0.7137
CSE+ADV | 0.7016 | 0.7445 | 0.4893 | 0.3505 | 0.5079 | 0.5184 | 0.7641 | 0.6845 | 0.7620 | 0.6875
LinfBIA | No | STD | 0.2732 | 0.3134 | 0.1448 | 0.0470 | 0.0956 | 0.1449 | 0.2326 | 0.2938 | 0.3195 | 0.2508
CSE | 0.4128 | 0.4447 | 0.2070 | 0.0819 | 0.1175 | 0.2010 | 0.3329 | 0.3940 | 0.4555 | 0.3709
Yes | CSA | 0.5407 | 0.5871 | 0.3393 | 0.2026 | 0.3009 | 0.3536 | 0.5586 | 0.5548 | 0.5995 | 0.5586
CSE+ADV | 0.5699 | 0.5830 | 0.3425 | 0.1870 | 0.2902 | 0.3556 | 0.5903 | 0.5504 | 0.6289 | 0.5136

In the experiment of CIFAR10, the neural network still is a LeNet network. The
difference with the network on MNIST is that the first layer convolution has
three channels, and it does not need padding operation. The hyperparameters
are showed as follows: the epochs are 50, the batch size is 256, the optimizer
is Adam, the learning rate is 0.01. Every 10 epochs, the learning rate will
drop by half. When training the CSA and CSE + ADV, both clean examples and
adversarial examples are used as inputs every iteration. The adversarial
examples are generated by PGD.

In the test of the model’s adversarial robustness, assuming the adversarial
perturbation is limited in $l_{\infty}$-norm ball with $\epsilon=8/255$, we
evaluate the model’s performance under FGSM, PGD, CW, MIA, L2BIA and LinfBIA
attack methods. The results are shown in Table III. To convenient for showing
protecting performance, we only show the accuracy of the protected class in
each scenario as same as that on MNIST. Under most adversarial attack methods
except for CW, CSE has stronger adversarial robustness than STD. The CSA and
CSE+ADV models have much stronger adversarial robustness than STD.

However, it is worth-note that all the models are vulnerable to adversarial
examples generated by CW. Actually, CW is a strong attack method which does
not depend on gradients. It has been found that adversarial training would
cause gradient obfuscation which gives one a false sense that the model has
adversarial robustness [13]. Therefore, we conclude that the Min-Max propery
would cause gradient obfuscation in another way.

The CIFAR10 dataset is much more complex than MNIST. Therefore, the number of
iterations may be much more than that of MNIST, as shown in Figs. 5a and 5d.
When the number of iterations is sufficient, both STD and CSE can converge
eventually. This is not obvious when training CSE on MNIST (so it does not
require too many iterations). Similar to the MNIST experiments, with the
increase of iterations, the fuzziness of the convolutional layer decreases
gradually, e.g., the Min-Max property of convolution becomes more obvious.
Thus the model becomes more adversarial robust.

###  IV-C CIFAR100

In this section, we will test our proposed methods on a larger, more realistic
dataset. On the CIFAR100 dataset, the simple LeNet network will no longer be
applicable. Therefore, ResNet18 is adopted as the structure of the model. The
following is the introduction of some hyperparameters: Learning rate is 0.1.
And it will be halved every 60 epochs; epochs are 180; batch size is 256;
optimizer is SGD; the constant in cost matrix is 10. When training CSA and CSE
+ ADV, alternate training modes of clean examples and adversarial examples are
adopted. All adversarial examples in training are generated by PGD.

Since CIFAR100 has 100 classes, it would be a waste of time to train
$100\times 3$ models (ADV, CSA and CSE) if each class is protected once.
Therefore, as shown in Table IV, we only show partial results since CIFAR100
contains 100 classes. Nevertheless, we find our methods improve
insignificantly adversarial robustness of model on deeper neural networks and
more complicated datasets. To find the failure reason, we further explore the
convolutional layer’s fuzziness in ResNet18. Fig. 6 shows the fuzziness of
convolutional layer parameters in ResNet18 on CIFAR100. For the CSE model, the
protected label is 21. We discover that the fuzziness of convolutional layer
parameters is almost hard to become small on CIFARI100. The fuzziness of each
convolutional layer in the STD model is almost identical to that of the CSE
model. We think this is due to the depth of the network. And the proposed
optimization algorithm cannot well control the fuzziness of parameters in the
deep network convolutional layer, so the proposed method cannot improve the
robustness of the model. As a result, in shallow neural networks, we can add a
regular term ($Fuzziness(\mathbf{W_{conv}})$) to control convolutional layer
parameters’ fuzziness, but this method may not be effective for deep networks.
If some skills can make the deep network’s convolutional layer parameters
follow the Min-Max property, we speculate that the network should also have
stronger adversarial robustness.

TABLE IV: CIFAR100: The accuracy of protected categories under various attack methods. P represents the category of protection. And $O_{i}$ presents the accuracy of category $i$ for the data in the table. Attacks |  ADV Training | Models |  $O_{9}$,p=9 |  $O_{21}$,p=21 |  $O_{36}$,p=36
---|---|---|---|---|---
FGSM | No | Std | 0.0262 | 0.0000 | 0.0111
CSE | 0.0200 | 0.0000 | 0.0000
Yes | CSA | 0.0135 | 0.0000 | 0.0000
CSE+Adv | 0.0251 | 0.0068 | 0.0110
PGD | No | Std | 0.0000 | 0.0000 | 0.0042
CSE | 0.0043 | 0.0000 | 0.0000
Yes | CSA | 0.0241 | 0.0000 | 0.0000
CSE+Adv | 0.0211 | 0.0000 | 0.0044
CW | No | Std | 0.0381 | 0.0130 | 0.0242
CSE | 0.0000 | 0.0000 | 0.0000
Yes | CSA | 0.0405 | 0.0000 | 0.0000
CSE+Adv | 0.0135 | 0.0000 | 0.0000
MIA | No | Std | 0.0044 | 0.0000 | 0.0035
CSE | 0.0042 | 0.0048 | 0.0072
Yes | CSA | 0.0203 | 0.0000 | 0.0035
CSE+Adv | 0.0047 | 0.0000 | 0.0108
L2BIA | No | Std | 0.0313 | 0.0141 | 0.0164
CSE | 0.0063 | 0.0000 | 0.0000
Yes | CSA | 0.0197 | 0.0000 | 0.0000
CSE+Adv | 0.0227 | 0.0000 | 0.0188
LinfBIA | No | Std | 0.0000 | 0.0000 | 0.0072
CSE | 0.0000 | 0.0000 | 0.0000
Yes | CSA | 0.0348 | 0.0000 | 0.0000
CSE+Adv | 0.0190 | 0.0000 | 0.0067
Figure 6: The fuzziness of convolutional layers of parameters in ResNet18.

##  V Conclusion

A great deal of work has been done to improve the overall adversarial
robustness of a model. However, in some specific problems, the cost of each
class of adversarial attack often varies greatly. Therefore, we consider to
protect the adversarial robustness of specific categories. While improving the
overall adversarial robustness of a model, we prioritize improving the
adversarial robustness of specific classes. Experimentally we show that the
cost-sensitive training can effectively protect specific categories from
adversarial attacks. Moreover, we find that the robustness of model is closely
related to the convolutional layer parameter distribution in LeNet networks.
Also we experimentally find that, the more obvious Min-Max property of the
convolutional layer parameter in LeNet is, the stronger the adversarial
robustness of the model will be.

There exist cases in which our method may not be successful in improving the
adversarial robustness of the model when we apply the proposed method to a
more complicated dataset. These cases may need deeper network structures to be
designed. How to effectively control the uncertainty of convolutional layer
parameters in a deep network, which significantly has impact on adversarial
robustness, remains to be studied further.

However, this failure on CIFAR100 dose not negate our theoretical results. The
Min-Max property can still be verified on small datasets. Therefore, how to
control the Min-Max property of the convolutional layer in a large-scale
network is a relatively challenging task.

## Acknowledgment

This work was supported in part by Key Project of Natural Science Foundation
of China (Grant 61732011), in part by the National Natural Science Foundation
of China (Grants 61976141, 61772344 and 61732011), in part by the Natural
Science Foundation of SZU (827-000230), and in part by the Interdisciplinary
Innovation Team of Shenzhen University.

## References

  * [1] M. Cheng, Q. Lei, P.-Y. Chen, I. Dhillon, and C.-J. Hsieh, “Cat: Customized adversarial training for improved robustness,” _arXiv preprint arXiv:2002.06789_ , 2020.
  * [2] C. Szegedy, W. Zaremba, I. Sutskever, J. Bruna, D. Erhan, I. Goodfellow, and R. Fergus, “Intriguing properties of neural networks,” _arXiv preprint arXiv:1312.6199_ , 2013.
  * [3] A. Kurakin, D. Boneh, F. Tramèr, I. Goodfellow, N. Papernot, and P. McDaniel, “Ensemble adversarial training: Attacks and defenses,” 2018.
  * [4] N. Papernot, P. McDaniel, X. Wu, S. Jha, and A. Swami, “Distillation as a defense to adversarial perturbations against deep neural networks,” in _2016 IEEE Symposium on Security and Privacy (SP)_. IEEE, 2016, pp. 582–597.
  * [5] A. Raghunathan, J. Steinhardt, and P. Liang, “Certified defenses against adversarial examples,” _arXiv preprint arXiv:1801.09344_ , 2018.
  * [6] A. Madry, A. Makelov, L. Schmidt, D. Tsipras, and A. Vladu, “Towards deep learning models resistant to adversarial attacks,” _arXiv preprint arXiv:1706.06083_ , 2017.
  * [7] I. J. Goodfellow, J. Shlens, and C. Szegedy, “Explaining and harnessing adversarial examples,” _arXiv preprint arXiv:1412.6572_ , 2014.
  * [8] H. Zhang, Y. Yu, J. Jiao, E. P. Xing, L. E. Ghaoui, and M. I. Jordan, “Theoretically principled trade-off between robustness and accuracy,” _arXiv preprint arXiv:1901.08573_ , 2019.
  * [9] A. Athalye, N. Carlini, and D. A. Wagner, “Obfuscated gradients give a false sense of security: Circumventing defenses to adversarial examples,” in _Proceedings of the 35th International Conference on Machine Learning, ICML 2018, Stockholmsmässan, Stockholm, Sweden, July 10-15, 2018_ , ser. Proceedings of Machine Learning Research, J. G. Dy and A. Krause, Eds., vol. 80. PMLR, 2018, pp. 274–283. [Online]. Available: http://proceedings.mlr.press/v80/athalye18a.html
  * [10] N. Papernot, P. McDaniel, S. Jha, M. Fredrikson, Z. B. Celik, and A. Swami, “The limitations of deep learning in adversarial settings,” in _2016 IEEE European symposium on security and privacy (EuroS &P)_. IEEE, 2016, pp. 372–387.
  * [11] S.-M. Moosavi-Dezfooli, A. Fawzi, and P. Frossard, “Deepfool: a simple and accurate method to fool deep neural networks,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2016, pp. 2574–2582.
  * [12] N. Carlini and D. Wagner, “Towards evaluating the robustness of neural networks,” in _2017 ieee symposium on security and privacy (sp)_. IEEE, 2017, pp. 39–57.
  * [13] A. Athalye, N. Carlini, and D. Wagner, “Obfuscated gradients give a false sense of security: Circumventing defenses to adversarial examples,” _arXiv preprint arXiv:1802.00420_ , 2018.
  * [14] X. Zhang and D. Evans, “Cost-sensitive robustness against adversarial examples,” _arXiv preprint arXiv:1810.09225_ , 2018.
  * [15] M. Terzi, G. A. Susto, and P. Chaudhari, “Directional adversarial training for cost sensitive deep learning classification applications,” _Engineering Applications of Artificial Intelligence_ , vol. 91, p. 103550, 2020.
  * [16] E. Wong and J. Z. Kolter, “Provable defenses against adversarial examples via the convex outer adversarial polytope,” _arXiv preprint arXiv:1711.00851_ , 2017.
  * [17] C. Elkan, “The foundations of cost-sensitive learning,” in _International joint conference on artificial intelligence_ , vol. 17, no. 1. Lawrence Erlbaum Associates Ltd, 2001, pp. 973–978.
  * [18] Z.-H. Zhou and X.-Y. Liu, “On multi-class cost-sensitive learning,” _Computational Intelligence_ , vol. 26, no. 3, pp. 232–257, 2010.
  * [19] M. Abbasi and C. Gagné, “Robustness to adversarial examples through an ensemble of specialists,” _arXiv preprint arXiv:1702.06856_ , 2017.
  * [20] Y. Liu, X. Chen, C. Liu, and D. Song, “Delving into transferable adversarial examples and black-box attacks,” _arXiv preprint arXiv:1611.02770_ , 2016\.
  * [21] Z. Zhao, D. Dua, and S. Singh, “Generating natural adversarial examples,” _arXiv preprint arXiv:1710.11342_ , 2017.
  * [22] N. Papernot, P. McDaniel, and I. Goodfellow, “Transferability in machine learning: from phenomena to black-box attacks using adversarial samples,” _arXiv preprint arXiv:1605.07277_ , 2016.
  * [23] R. Huang, B. Xu, D. Schuurmans, and C. Szepesvári, “Learning with a strong adversary,” _arXiv preprint arXiv:1511.03034_ , 2015.
  * [24] A. Kurakin, I. Goodfellow, and S. Bengio, “Adversarial machine learning at scale,” _arXiv preprint arXiv:1611.01236_ , 2016.
  * [25] G. Liu, I. Khalil, and A. Khreishah, “Using single-step adversarial training to defend iterative adversarial examples,” _arXiv preprint arXiv:2002.09632_ , 2020.
  * [26] H. Zhang, Y. Yu, J. Jiao, E. P. Xing, L. E. Ghaoui, and M. I. Jordan, “Theoretically principled trade-off between robustness and accuracy,” _arXiv preprint arXiv:1901.08573_ , 2019.
  * [27] A. Shafahi, M. Najibi, M. A. Ghiasi, Z. Xu, J. Dickerson, C. Studer, L. S. Davis, G. Taylor, and T. Goldstein, “Adversarial training for free!” in _Advances in Neural Information Processing Systems_ , 2019, pp. 3358–3369.
  * [28] D. Zhang, T. Zhang, Y. Lu, Z. Zhu, and B. Dong, “You only propagate once: Accelerating adversarial training via maximal principle,” in _Advances in Neural Information Processing Systems_ , 2019, pp. 227–238.
  * [29] E. Wong, L. Rice, and J. Z. Kolter, “Fast is better than free: Revisiting adversarial training,” _arXiv preprint arXiv:2001.03994_ , 2020.
  * [30] C. Qin, J. Martens, S. Gowal, D. Krishnan, K. Dvijotham, A. Fawzi, S. De, R. Stanforth, and P. Kohli, “Adversarial robustness through local linearization,” in _Advances in Neural Information Processing Systems_ , 2019, pp. 13 847–13 856.
  * [31] M. Kukar, I. Kononenko _et al._ , “Cost-sensitive learning with neural networks.” in _ECAI_ , vol. 98, 1998, pp. 445–449.
  * [32] N. Abe, B. Zadrozny, and J. Langford, “An iterative method for multi-class cost-sensitive learning,” in _Proceedings of the tenth ACM SIGKDD international conference on Knowledge discovery and data mining_ , 2004, pp. 3–11.
  * [33] L. Jiang, C. Li, and S. Wang, “Cost-sensitive bayesian network classifiers,” _Pattern Recognition Letters_ , vol. 45, pp. 211–216, 2014.
  * [34] P. Domingos, “Metacost: A general method for making classifiers cost-sensitive,” in _Proceedings of the fifth ACM SIGKDD international conference on Knowledge discovery and data mining_ , 1999, pp. 155–164.
  * [35] W.-Y. Loh, “Fifty years of classification and regression trees,” _International Statistical Review_ , vol. 82, no. 3, pp. 329–348, 2014.
  * [36] B. Zadrozny, J. Langford, and N. Abe, “Cost-sensitive learning by cost-proportionate example weighting,” in _Third IEEE international conference on data mining_. IEEE, 2003, pp. 435–442.
  * [37] K. Asif, W. Xing, S. Behpour, and B. D. Ziebart, “Adversarial cost-sensitive classification.” in _UAI_ , 2015, pp. 92–101.
  * [38] H. Shen, S. Chen, and R. Wang, “A study on the uncertainty of convolutional layers in deep neural networks,” _CoRR_ , vol. abs/2011.13719, 2020. [Online]. Available: https://arxiv.org/abs/2011.13719
  * [39] J. Basak, R. K. De, and S. K. Pal, “Unsupervised feature selection using a neuro-fuzzy approach,” _Pattern Recognition Letters_ , vol. 19, no. 11, pp. 997–1006, 1998.
  * [40] D. Yan, X. Zhou, X. Wang, and R. Wang, “An off-center technique: Learning a feature transformation to improve the performance of clustering and classification,” _Inf. Sci._ , vol. 503, pp. 635–651, 2019. [Online]. Available: https://doi.org/10.1016/j.ins.2019.06.068
  * [41] G. W. Ding, L. Wang, and X. Jin, “AdverTorch v0.1: An adversarial robustness toolbox based on pytorch,” _arXiv preprint arXiv:1902.07623_ , 2019.
  * [42] Y. Dong, F. Liao, T. Pang, H. Su, J. Zhu, X. Hu, and J. Li, “Boosting adversarial attacks with momentum,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2018, pp. 9185–9193.
  * [43] L. Deng, “The mnist database of handwritten digit images for machine learning research [best of the web],” _IEEE Signal Processing Magazine_ , vol. 29, no. 6, pp. 141–142, 2012.
  * [44] A. Krizhevsky, G. Hinton _et al._ , “Learning multiple layers of features from tiny images,” 2009.
  * [45] S. Haykin and B. Kosko, _GradientBased Learning Applied to Document Recognition_ , 2001, pp. 306–351.
  * [46] X. Glorot, A. Bordes, and Y. Bengio, “Deep sparse rectifier neural networks,” in _Proceedings of the fourteenth international conference on artificial intelligence and statistics_ , 2011, pp. 315–323.


# Data-Free Quantization with Accurate Activation Clipping and Adaptive Batch
Normalization

Yefei He, Luoming Zhang, Weijia Wu, Hong Zhou
College of Biomedical Engineering & Instrument Science
Zhejiang University
Hangzhou, China
<EMAIL_ADDRESS><EMAIL_ADDRESS>

###### Abstract

Data-free quantization compresses the neural network to low bit-width without
access to original training data. Most existing data-free quantization methods
cause severe performance degradation due to inaccurate activation clipping
range and quantization error, especially for low bit-width. In this paper, we
present a simple yet effective data-free quantization method with accurate
activation clipping and adaptive batch normalization. Accurate activation
clipping (AAC) improves the model accuracy by exploiting accurate activation
information from the full-precision model. Adaptive batch normalization (ABN)
firstly proposes to address the quantization error from distribution changes
by updating the batch normalization layer adaptively. Extensive experiments
demonstrate that the proposed data-free quantization method can yield
surprisingly performance, achieving $\mathbf{64.33\%}$ top-1 accuracy of 4-bit
ResNet18 on ImageNet dataset, with $\mathbf{3.7\%}$ absolute improvement
outperforming the existing state-of-the-art methods.

##  1 Introduction

Deep learning has achieved breakthrough successes in many fields, such as
computer vision[11, 6] and natural language processing[4]. Over-
parameterization is an obvious feature of deep learning models compared to
traditional methods. Since its birth, deep learning models have the
disadvantages of massive parameters and high computational complexity. For
example, ResNet18[5] network has 11.7M parameters and 1.8GFLOPS calculations.
These shortcomings limit the application of deep learning models in edge
devices such as mobile phones. To solve this problem, model compression
methods such as model quantization, distillation, and pruning have emerged in
recent years. Among them, model quantization is one of the most commonly used
compression methods. Quantization refers to mapping model parameters or
activations from floating-point numbers to integers according to certain
rules, thereby greatly reduce the size of the model and accelerate the
inference process.

Quantizing a full-precision model to low bit-width directly often leads to a
severe accuracy drop. To address this, two quantization methods, i.e.,
quantization-aware-training (QAT) and post-training quantization (PTQ) are
proposed. The former aims to retrain the quantized model and consumes lots of
computation resources. The latter directly quantize the pre-trained floating-
point model and uses part of the training data to calibrate it. Both methods
are data-driven that require real data in their quantization process. However,
in many practical scenarios, the training dataset is not available for privacy
policy or security issues.

Fortunately, data-free quantization can compress models without accessing any
original training data. Many excellent prior studies[1, 12] try to solve the
task by the generative methods with synthetic data. ZeroQ[1] and DSG[14]
proposed to generate fake data from the distribution (i.e., mean and standard
deviation) of BN layers. After using the fake data to update the range of
quantized activations, the method achieved competitive result compared to the
previous QAT methods at 8-bit, but the performance degrades significantly at
low bit-width (especially 4-bit or lower). Although the synthetic data
conforms to the distribution of BN layers, the activation range is actually
determined by the peak value and there is no necessary connection between the
extreme values of the data and its distribution. Therefore, we argut that the
updated activation clipping range is inaccurate. Besides, the quantization
process inevitably brings deviations to both weights and activations, thus
disturbing the distribution of intermediate feature maps. The distribution
mismatch between the feature maps and the fixed BN statistics will further
aggravate the performance decrease. However, this problem is disregarded by
previous data-free quantization methods and how to alleviate this mismatch
remains an open question. Furthermore, recent studies [12, 2, 15] proposed
neural network-based data generators to fine-tune the quantized model. After
fine-tuning, their work achieved better results than [1] when models were
quantized to 4-bit. Nevertheless, training a generator requires a lot of time
and computing resources, and fine-tuning can be less useful or even harmful
for 8-bit quantization. How to quantize models generally and efficiently is
still a problem.

To address the above issues, we present a simple yet effective data-free
quantization method with two core components, i.e., accurate activation
clipping and adaptive batch normalization. Accurate activation clipping uses
the classification results of the teacher model and an absolute value loss
function to optimize the synthetic data, the generated data contains accurate
activation range information of the original dataset. Adaptive batch
normalization takes the shift of quantized feature map distribution into
consideration. After determining the activation value range, we propose to
adaptively update the BN statistics of the quantized model, alleviating the
distribution mismatch. Finally, we propose to perform fine-tuning only when
necessary to further improve accuracy. Even with limited computing resources
that can’t perform fine-tuning, our method can achieve competitive performance
compared to fine-tuning-based method.

We summarize our main contributions as follows:

  * •

We rethink the determinants of activation range and propose a novel method to
generate data specifically for accurate activation clipping. This provides a
new approach for extracting dataset’s activation information from full-
precision models.

  * •

An adaptive batch normalization is proposed to alleviate the disturbance from
distribution mismatch of BN layers.

  * •

Combining the above methods with optional fine-tuning, we get a brand new
three-step data-free quantization pipeline. Extensive experiments on the large
scale ImageNet dataset prove that quantization with our method can surpass the
state-of-the-art methods with $\mathbf{3.7\%}$ absolute improvement on 4-bit
ResNet18.

Figure 1: Proposed three-step pipeline for data-free quantization. Here, "AC"
means activation clipping and "FP Model" denotes full-precision model.

##  2 Related Work

Data-free quantization was first proposed by DFQ[7] and quickly became the
focus of research. There are two important questions in the field of data-free
quantization: how to generate synthetic data and how to apply these data to
improve the quantized model. Updating gaussian input using gradient
backpropagation is a wildly adopted approach. Methods like ZeroQ[1] and
DSG[14] followed this scheme and extract information from BN layers to
generate synthetic data for activation clipping. In this case, the key problem
is how to recover the activation information of the training dataset from the
full-precision model. Another common approach uses a network-based generator
to synthesize data[12, 15, 2]. After generating data, they fine-tuned the
quantized model to improve the accuracy. However, this can be much more time-
consuming than the first approach.

To further improve the limit of data-free quantization, we argue that the
method of generating data should be considered corporately with the method of
using it. In other words, questions one and two should be considered together.
For activation clipping, we analyzed the cause of the peak value of
activations and proposed a new loss function $\mathcal{L}_{\mathrm{ABS}}$ that
can generate data dedicated to activation clipping. Meanwhile, we propose a
new method to apply the synthetic data, i.e., updating the BN layers’
statistics to alleviate the distribution mismatch. Both methods are fast and
effective, and can be further combined with fine-tuning-based methods.

##  3 Method

Figure 2: Activation range comparison for ResNet18 on ImageNet when quantized
to 4-bit. Among them, the best clipping range is obtained by iterate through
all values between $[0,50]$ (which basically covering all activation values)
while inputting real dataset into the model.

###  3.1 Preliminaries

We use uniform quantization in our study and experiments. For uniform
quantization, given the bit-width $b$ and clip range $[l,u]$ for weight or
activation, quantization-dequantization process as follows:

$\Delta=\frac{u-l}{2^{b}-1}$ (1)
$Q(x)=\operatorname{round}(\frac{x-l}{\Delta})$ (2) $D(x)=Q(x)*\Delta+l$ (3)

where $\Delta$ is the interval length,$Q(x)$ is the quantized representation
of the data and $D(x)$ is the result of dequantization process of a value
$Q(x)$.

There will be two main problems when it comes to data-free quantization.
First, since the weight has been trained, the range of the weight is its
minimum/maximum value. However, the clip range for activations of each layers
depends on the specific input and is still unknown. Second, the statistics
($\mu$ and $\sigma$) of BN layers depend on the input and feature maps of the
network, and have been fixed in the model. However, quantizing the model may
shift the distribution of the intermediate feature maps, which consequently
become inconsistent with the BN statistics.

In this section, we propose two approaches, i.e., accurate activation clipping
and adaptive batch normalization to address the corresponding two problems,
which can achieve remarkable results in a short time. In low-bit quantization,
we further use fine-tuning to improve the accuracy, thus having a three-step
pipeline for data-free quantization, as shown in Figure 1.

###  3.2 Accurate Activation Clipping

To determine the clipping range of activations (i.e. $u$ and $l$ in Equation
(1)), one common way is to generate synthetic data and conduct a forward
propagation with it. The peak value of activations is stored as the clipping
range parameter. While previous studies[1, 12] generate fake data with the
distribution of BN layers, it only provides a coarse prediction for the
activation range. We argue that it is not the optimal choice for updating the
clipping range because the peaks are not directly related to the distribution
of the data. This is illustrated in Figure 2, where we plot the best
activation clipping range and the data-determined range for every layer in the
ResNet18 model. With distribution-consistent data from BN layers, ZeroQ[1]
presents a dissatisfied performance compared to the optimal activation value.
So what exactly determines the activation clipping range?

Figure 3: A common block structure of quantized CNNs

Figure 3 presents a common block structure of quantized CNNs. Before clipping,
ReLU takes feature map of convolution layers as input. Thus, the lower bound
$l$ is zero without doubt, while the upper bound $u$ depends on the maximum
value of feature map. The recent study [16] revealed that feature map is the
response of the network to category features where high response is related to
significant features and low response is related to irrelevant features such
as background. Therefore, to emulate the response of the real dataset,
synthetic data should make the network to be highly responsive.

To this end, we propose a novel method to generate accurate activation
clipping data. We notice that deep learning models perform image
classification tasks on large-scale datasets well with rich category-related
information. Thus we try to let the model to learn the most responsive data by
itself. For example, given a target label "flowers" and a gaussian-random
image, if we input the gaussian image to the network directly, the result is
unlikely to classify it as a flower. However, if we have an appropriate loss
function, calculate the loss according to the target label and backpropagate
to the image, the confidence of classifying as "flowers" will increase as we
iterate. Finally, the network generates the "most flower-liked" image
adaptively. With the high-quality generated image used as input, the model
will emerge higher response and emulate the real dataset’s activation peaks.
In other words, we have to solve the following optimization problems:

$\min_{x}\mathbb{E}_{x,y}\left[\ell({M}(x),y)\right],$ (4)

where $x$ is the synthetic input data, $y$ is the target label, $\ell$ is the
loss function and $M$ is the full-precision model.

To select an appropriate loss function, we take a deeper look at this problem.
The Cross-Entropy (CE) loss is the most commonly used loss function in image
classification tasks and it’s a good indicator of how well the model classify
input images. Therefore, we use the CE loss to judge the quality of the
generated data, where lower loss means better generated data. However, when it
comes to backpropagation, CE loss is not the best choice. A toy example is
used to illustrate the problem, as shown in Figure 4(a). We can see that using
CE loss for backpropagation cannot optimize the loss function to the optimum.
Details of the toy experiment can be found in Appendix 6.1.

Figure 4: Toy experiment of optimization using different loss functions. We
take M(x)=x as the model and optimize the input. $p$ represents prediction
probability of target class.

The derivative computed by CE loss can be formulated as:

$\displaystyle\frac{\partial\mathcal{L}_{\mathrm{CE}}}{\partial
M_{y}(x)}=\frac{\partial\mathcal{L}_{\mathrm{CE}}}{\partial
p_{y}}\cdot\frac{\partial p_{y}}{\partial M_{y}(x)}$ (5)
$\displaystyle=-\frac{1}{p_{y}}\cdot p_{y}\cdot(1-p_{y})$ (6)
$\displaystyle=p_{y}-1$ (7)

where $p_{y}$ is the probability of classify to label y (i.e., the result of
$\operatorname{softmax}$ operation). With the increase of $p_{y}$, the
derivative used for backpropagation will be smaller, which makes the
optimization process saturated before it reaches the optimal state.

It can be seen from Equation (7) that the problem is mainly caused by
$softmax$ operation, where all the scores are brought into calculation.
Actually, in this peak value problem, we only care about the score
corresponding to the target label and hope to reflect it on the input image.
Scores of other labels or their appearance on the input image are irrelevant
to the model’s response. Therefore, we propose a hard-label absolute value
loss (ABS Loss):

$\mathcal{L}_{\mathrm{ABS}}=-{M}_{y}(x)$ (8)

Thus, only the regions that the model considers to have an impact on the
"flower" features will be enhanced, and the intermediate feature map will also
produce a higher response. Using Equation (8) as loss functions in the
algorithm also make the Cross-Entropy loss drop to 0 quickly, making the data
generation process even faster (see Figure 4(b)). It can also be seen from the
Figure 2 that the data we generate is very close to the optimal value of the
activation clipping range.

###  3.3 Adaptive Batch Normalization

Figure 5: Comparison between featuremaps of original model and quantized
model. (a): layer1.1.conv2, (b): layer2.0.conv2.

Previous research[7] has found that when the convolutional layer is quantized,
the output featuremap distribution will have a certain offset compared to the
original model. When we quantize models to low bit-width, this phenomenon will
be more obvious, as shown in Figure 5. In this case, since the distribution of
the input data does not match the statistic stored in the BN layer, the BN
layer not only fails to normalize the data, but may also have a side effect
that harms the model accuracy. This statistic inconsistency is caused by
quantization errors. When we have the training dataset, we can use QAT to
retrain the weights and BN layers of the network. However, in the case of
data-free quantization, we do not have training data to optimize the
quantization error. Hence, in this study we look at this problem from another
perspective. Since the model has a large number of weight parameters, it’s
difficult to update without training data. But the BN layers contains few
parameters and only depends on the distribution of the input data. In order to
alleviate the distribution mismatch and improve the accuracy of the quantized
model, we choose to update the statistics of BN layers and let BN layers adapt
to the errors caused by quantization.

Although the data generated in section 3.2 can restore the activation range
very well, it is not suitable for updating the statistics of the BN layer. As
analyzed above, updating the activation range is an extremum problem while
updating the BN statistics is a data distribution problem. Therefore, we use
the distribution-consistent data proposed in [1] to update the BN statistics.

The BN statistics (BNS) loss function is defined as follows:

$\displaystyle\mathcal{L}_{\mathrm{BNS}}=\sum_{i=0}^{L}{\|\widehat{\mu_{i}}-\mu_{i}\|_{2}}^{2}+{\|\widehat{\sigma_{i}}-\sigma_{i}\|_{2}}^{2}$
(9)

Among them, $\mu_{i}$, $\sigma_{i}$ are the mean and standard deviation
information stored in the i-th BN layer in the full-precision model.
$\widehat{\mu_{i}}$ and $\widehat{\sigma_{i}}$ are the mean and standard
deviation of input data at layer $i$. Similar to AAC, the random input image
is backpropagated through the loss function (9), and the generated data
obtained contains the distribution information of the training dataset. By
feeding this data into the network, we can measure the distribution
disturbance and update the BN statistics well.

###  3.4 Three-Step Pipeline

In the case of low bit-width (especially 4-bit or lower) quantization, it is
difficult to achieve ideal results without fine-tuning the weights. Methods
like GDFQ[12] propose to use a network-based generator to generate fake data
and perform retraining on the quantized model.

However, as analyzed in section3.1, determining activation range and BN
statistics are key problems in data-free quantization. While retraining the
weights improve the accuracy, these two problems are neglected by previous
fine-tuning methods. Meanwhile, our corresponding methods proposed above are
effective and fast that can be completed within one minute. Therefore, we use
our method as the basis for fine-tuning, which makes up for their shortcomings
and greatly improves the accuracy. At this point, we get a three-step data-
free quantization pipeline, as shown in Figure 1.

Fine-tuning can consume lots of time and computing resources while our method
are efficient. Therefore, we try to seek a balance between time and effect. In
the case of 8-bit, our two-step method (without fine-tuning) has been able to
achieve ideal results, and the improvement brought by fine-tuning is subtle.
In the case of low bit-width, although the improvement brought by fine-tuning
is still considerable and a complete three-step method is worth performing,
the two-step method achieves a huge improvement over recent study DSG[14] and
is comparable with fine-tuning based method, which will be further analyzed in
section 4.2.

##  4 Experiments

###  4.1 Implementation Details

This research is based on the Pytorch framework and all experiments are
conducted using RTX3090 GPUs. For the generation of accurate activation
clipping data, we used a learning rate of 0.2 to optimize 200 iterations; for
the generation of distribution-consistent data, we used a learning rate of 0.5
to optimize 500 iterations. For data generator, we follow the settings
introduced by [12]. To make a fair comparison, the method of quantizing models
is the same as [1] and [12], where all the convolution layers and linear
layers are quantized to the same bit-width.

We conduct all experiments on the large-scale benchmark dataset ImageNet
(ILSVRC12)[3]. In order to verify the versatility of our method, we tested
several commonly used network structures, including ResNet[5], VGG[9],
Inception v3[10] and MobileNetV2[8]. All networks use the pre-trained model
provided by pytorchcv[13].

###  4.2 Comparison with State-of-the-arts

Table 1: Comparisons of different network architecture on ImageNet. Here, "No Data" means no training data is used, "No FT" stands for no retraining the weight, and "FP" means full-precision pretrained model. Model | No Data | No FT | Method | Bit-width(W/A) | Acc.(%)
---|---|---|---|---|---
ResNet18 | $\times$ | $\times$ | FP | 32/32 | 71.47
$\surd$ | $\surd$ |  ZeroQ[1] | 4/4 | 26.04
$\surd$ | $\surd$ |  DSG[14] | 4/4 | 34.53
$\surd$ | $\surd$ | Ours | 4/4 | 55.06
$\surd$ | $\times$ |  GDFQ[12] | 4/4 | 60.60
$\surd$ | $\times$ | Ours | 4/4 | 64.33
$\times$ | $\times$ | FP | 32/32 | 71.47
$\surd$ | $\surd$ |  DFQ[7] | 8/8 | 69.70
$\surd$ | $\times$ |  GDFQ[12] | 8/8 | 70.80
$\surd$ | $\surd$ |  ZeroQ[1] | 8/8 | 71.43
$\surd$ | $\surd$ | Ours | 8/8 | 71.43
$\surd$ | $\surd$ |  DSG[14] | 8/8 | 71.49
BN-VGG16 | $\times$ | $\times$ | FP | 32/32 | 74.28
$\surd$ | $\surd$ |  ZeroQ[1] | 4/4 | 1.15
$\surd$ | $\times$ |  GDFQ[12] | 4/4 | 67.10
$\surd$ | $\surd$ | Ours | 4/4 | 68.59
$\surd$ | $\times$ | Ours | 4/4 | 70.10
Inception V3 | $\times$ | $\times$ | FP | 32/32 | 78.80
$\surd$ | $\surd$ |  ZeroQ[1] | 4/4 | 26.84
$\surd$ | $\surd$ |  DSG[14] | 4/4 | 34.89
$\surd$ | $\surd$ | Ours | 4/4 | 69.20
$\surd$ | $\times$ |  GDFQ[12] | 4/4 | 70.39
$\surd$ | $\times$ | Ours | 4/4 | 72.74
MobileNet V2 | $\times$ | $\times$ | FP | 32/32 | 73.03
$\surd$ | $\times$ |  GDFQ[12] | 4/4 | 59.24
$\surd$ | $\times$ | Ours | 4/4 | 59.50
$\surd$ | $\surd$ |  DFQ[7] | 8/8 | 72.33
$\surd$ | $\times$ |  GDFQ[12] | 8/8 | 72.88
$\surd$ | $\surd$ |  ZeroQ[1] | 8/8 | 72.91
$\surd$ | $\surd$ | Ours | 8/8 | 72.93

In this subsection, we evaluate our method by comparing it with existing
state-of-the-arts data-free quantization methods over various network
architecture. Since the accuracy in 8-bit quantization is very close to the
full-precision model, we mainly report the results at 4-bit, as shown in
Table1.

We can observe that for W4A4 (i.e., quantize both weights and activations to
4-bit), our method outperforms all other methods by large margins on different
network architectures. This illustrates the effectiveness of our accurate
activation clipping and BN statistics update. Even if our method does not use
fine-tuning on many models, the effect is very close to or exceeds that of
GDFQ, which requires fine-tuning. As we all know, fine-tuning consumes a lot
of GPU time. Our approach (without FT, two-step) only takes about one minute
to complete, strikes a better balance between time and results.

For W8A8, the effects of various methods are very close to the full-precision
model. In this case, we find that if the generated data is used for fine-
tuning, the effect is subtle or may even be worse than the result of not using
fine-tuning. We recommend not using fine-tuning for 8-bit quantization.

###  4.3 Ablation Study

####  4.3.1 Three-step pipeline

To further verify the effects of each part of three-step pipeline, we used the
ResNet18 model to perform ablation experiments on ImageNet dataset. Table 2
shows the performance under different settings. We can see that all three
steps have played an important role in improving accuracy. When we combine
them, the proposed three-steps pipeline obtained achieves the effect of SOTA,
indicating that the improvements brought by the three steps can be
superimposed. The three steps consider a better activation clipping range,
updating BN statistics to adapt to the quantization error, and fine-tuning the
weight separately, that’s why our method can obtain better results. It is
worth mentioning that when only the activation clipping range is updated, our
performance greatly surpasses ZeroQ[1], indicating that our method can better
restore the activation value range of the real dataset.

Table 2: Ablation study of three-step pipeline on 4-bit quantization. Here,
"AC" donotes activation clipping, "BN Update" means updating the statistics in
BN layers of quantized model and "FT" means using synthetic data to fine-tune
the quantized model.
With AC | With BN update | With FT | Acc.(%)
---|---|---|---
$\surd$ (ZeroQ) | $\times$ | $\times$ | 26.04
$\times$ | $\times$ | $\times$ | 10.22
$\surd$ | $\times$ | $\times$ | 43.03
$\surd$ | $\surd$ | $\times$ | 55.06
$\surd$ | $\surd$ | $\surd$ | 64.33

What’s more, we valid the effects of proposed method over several state-of-
the-art fine-tuning-based methods. As shown in Table 3. In all cases, our
method can further improve the accuracy, proving the robustness of our method
and that an accurate activation range is an important basis for the fine-
tuning-based method.

Table 3: Results of 4-bit ResNet18 on ImageNet. The results are obtained with the official code from authors Method | Top-1 Acc.(%)
---|---
real data | 67.90
GDFQ[12] | 60.60
GDFQ+Ours | 64.33 (+3.73)
Qimera[2] | 62.98
Qimera+Ours | 63.72 (+0.74)
IntraQ[15] | 66.47
IntraQ+ours | 67.06 (+0.59)

####  4.3.2 Selection of Loss Function

We further investigate the performance of generating accurate activation data
using different loss functions. The ResNet18 model is quantized to 4-bit. We
iterate on the gaussian image for 600 epochs to generate synthetic data, and
use this data to update the activation clipping range. Table 4 shows the
performance under different settings, where “MAE” denotes mean average error
and "MSE" stands for mean square error. We feed the generated data into the
model and calculate the Cross-Entropy loss to measure how well the model
responds to the data.

Table 4: Selection of different loss functions
Model | Method | Loss | Acc.(%)
---|---|---|---
ResNet18 | ABS+BNS | 2.86 | 28.46
Cross-entropy | 1.77 | 33.68
MAE | 1.10 | 41.13
MSE | 0.25 | 41.39
ABS | 0 | 43.03

It can be seen that the data generated by the proposed ABS loss can make the
model predict labels exactly as we expected. This also achieved the highest
classification accuracy after updating the quantized model.

An intuitive idea is to allow the synthetic data to make the model highly
responsive while conforming to the distribution determined by the BN
statistics. However, in experiments we found that if we add the two loss
functions together, the optimization process becomes unstable and is unable to
converge.

###  4.4 Quantization Efficiency

In this subsection, we compare our two-step and three-step pipeline with the
existing data-free quantization methods, as shwon in Table 5. Methods that
update gaussian input to determine the activation range is way faster than
methods with fine-tuning. Among them, our method achieves the best accuracy
and brings 21% absolute improvement on 4-bit ResNet18. This is very useful on
devices that have limited computing power and cannot perform fine-tuning. When
fine-tune is required, our method can greatly improve the accuracy while
barely increasing the time consumption.

Table 5: Comparison of time cost of 4-bit ResNet-18 with different data-free quantization methods. Method | With FT | Acc.(%) | Time (min)
---|---|---|---
ZeroQ | $\times$ | 26.08 | 0.83
DSG | $\times$ | 34.53 | -
Ours (two-step) | $\times$ | 55.06 | 1.38
GDFQ | $\surd$ | 60.60 | 139.6
Ours (three-step) | $\surd$ | 64.33 | 141.2

##  5 Conclusion and Discussion

In this paper, we propose two novel techniques: accurate activation clipping
and adaptive batch normalization to improve the accuracy of data-free
quantization. First, we analyze the origin of activation peak values and
construct a new scheme to generate synthetic data that can restore the
activation range of the original dataset, thus helping better quantizing the
activations. Next, we analyze the mismatch between quantized featuremap
distribution and statistics stored in BN layers. We propose to update the BN
statistics adaptively and let the BN layers adapt to the errors caused by
quantization. Finally, we combine the above two methods with an optional fine-
tune module and get a three-step quantization pipeline, which can further
improve the accuracy, allowing users to strike a balance between time
consumption and results. Extensive experiments prove that our methods
outperforms the existed data-free quantization methods by a large margin.

We also find that, since the existing methods mainly focus on updating the
clipping range of activation values, they do not work well with models like
MobileNet that do not use traditional $\operatorname{ReLU}$ as the activation
function. How to quantize networks like MobileNet effectively is a question
worth studying in the future.

## References

  * [1] Y. Cai, Z. Yao, Z. Dong, A. Gholami, and K. Keutzer.  Zeroq: A novel zero shot quantization framework.  In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020.
  * [2] K. Choi, D. Hong, N. Park, Y. Kim, and J. Lee.  Qimera: Data-free quantization with synthetic boundary supporting samples.  Advances in Neural Information Processing Systems, 34, 2021.
  * [3] J. Deng.  Imagenet : A large-scale hierarchical image database.  Proc. CVPR, 2009, 2009.
  * [4] J. Devlin, M. W. Chang, K. Lee, and K. Toutanova.  Bert: Pre-training of deep bidirectional transformers for language understanding.  2018\.
  * [5] K. He, X. Zhang, S. Ren, and J. Sun.  Deep residual learning for image recognition.  2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016.
  * [6] Z. Liu, H. Mao, C.-Y. Wu, C. Feichtenhofer, T. Darrell, and S. Xie.  A convnet for the 2020s.  arXiv preprint arXiv:2201.03545, 2022.
  * [7] M. Nagel, M. V. Baalen, T. Blankevoort, and M. Welling.  Data-free quantization through weight equalization and bias correction.  In 2019 IEEE/CVF International Conference on Computer Vision (ICCV), 2019.
  * [8] M. Sandler, A. Howard, M. Zhu, A. Zhmoginov, and L. C. Chen.  Inverted residuals and linear bottlenecks: Mobile networks for classification, detection and segmentation.  2018\.
  * [9] K. Simonyan and A. Zisserman.  Very deep convolutional networks for large-scale image recognition.  Computer Science, 2014.
  * [10] C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens, and Z. Wojna.  Rethinking the inception architecture for computer vision.  IEEE, pages 2818–2826, 2016.
  * [11] C.-Y. Wang, A. Bochkovskiy, and H.-Y. M. Liao.  Scaled-YOLOv4: Scaling cross stage partial network.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 13029–13038, June 2021.
  * [12] S. Xu, H. Li, B. Zhuang, J. Liu, J. Cao, C. Liang, and M. Tan.  Generative low-bitwidth data free quantization.  2020\.
  * [13] A. You, X. Li, Z. Zhu, and Y. Tong.  Torchcv: A pytorch-based framework for deep learning in computer vision.  https://github.com/donnyyou/torchcv, 2019.
  * [14] X. Zhang, H. Qin, Y. Ding, R. Gong, Q. Yan, R. Tao, Y. Li, F. Yu, and X. Liu.  Diversifying sample generation for accurate data-free quantization.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 15658–15667, 2021.
  * [15] Y. Zhong, M. Lin, G. Nan, J. Liu, B. Zhang, Y. Tian, and R. Ji.  Intraq: Learning synthetic images with intra-class heterogeneity for zero-shot network quantization.  arXiv preprint arXiv:2111.09136, 2021.
  * [16] B. Zhou, A. Khosla, A. Lapedriza, A. Oliva, and A. Torralba.  Learning deep features for discriminative localization.  In CVPR, 2016.

##  6 Appendix

###  6.1 Implementation details of toy experiment

In the toy experiment shown in Figure 4, we let the model to be an identity
transform. Given a target label, we calculate different loss functions and
backpropagate to the input, which will make the value of target class higher
during iteration. We run both experiments for 300 iterations. The algorithm is
summarized below.

Algorithm 1 Toy experiment of optimizing by different loss functions

Require: Identity transform model $M$, a loss function $\mathcal{L}$, random
data $\mathbf{x}\in\mathbb{R}^{n}$ and a target label $t\in[0,n-1]$.

for $i=1$ to $num\\_iterations$ do

Forward propagate $M(\mathbf{x})=\mathbf{x}$

Calculate loss value according to the choice of loss function $\mathcal{L}$

Backward propagate and update $\mathbf{x}$

end for

Return: Target value $\mathbf{x}_{t}$