# Both Spatial and Frequency Cues Contribute to High-Fidelity Image Inpainting

Ze Lu, Yalei Lv∗, Wenqi Wang, Pengfei Xiong
Shopee MMU indicates equal contributions.corresponding author.
<EMAIL_ADDRESS>
###### Abstract

Deep generative approaches have obtained great success in image inpainting
recently. However, most generative inpainting networks suffer from either
over-smooth results or aliasing artifacts. The former lacks high-frequency
details, while the latter lacks semantic structure. To address this issue, we
propose an effective Frequency-Spatial Complementary Network (FSCN) by
exploiting rich semantic information in both spatial and frequency domains.
Specifically, we introduce an extra Frequency Branch and Frequency Loss on the
spatial-based network to impose direct supervision on the frequency
information, and propose a Frequency-Spatial Cross-Attention Block (FSCAB) to
fuse multi-domain features and combine the corresponding characteristics. With
our FSCAB, the inpainting network is capable of capturing frequency
information and preserving visual consistency simultaneously. Extensive
quantitative and qualitative experiments demonstrate that our inpainting
network can effectively achieve superior results, outperforming previous
state-of-the-art approaches with significantly fewer parameters and less
computation cost. The code will be released soon.

##  1 Introduction

Figure 1:  We show an image and the corresponding frequency spectrum in each
column. Zoom in for a better view. The center point of the frequency spectrum
denotes the lowest frequency and the outermost region denotes the highest
frequency. LaMa [26] loses much high-frequency information and produces over-
smooth artifacts in the hair area. Wavefill [38] neglects to explore the
spatial features and produces unpleasant perceptual quality, e.g. aliasing
artifacts. Compared with them, our result is more similar to the ground truth
in both the frequency and spatial domains.

Image inpainting aims to recover visually realistic texture in incomplete
images. It has played an important role in various applications, such as
object removal, old photo restoration, and face completion. Recently,
significant progress has been achieved with the development of various
powerful deep learning based methods [23, 2, 8, 17, 13, 31, 10, 5, 26, 38,
12]. However, restoring realistic and high-fidelity images still remains a
challenging task.

Most of the existing inpainting methods [26, 14, 31, 37, 18] explore only
spatial features, with no exploitation of the frequency features. These
methods can usually recover plausible global textures. However, they typically
generate frequency spectra deviating from the ground-truth spectra, leading to
unsatisfied perceptual quality in the visual space, such as over-smooth or
checkerboard artifacts. As shown in Fig. 1, LaMa [26], one of the state-of-
the-art spatial domain methods, suffers from the loss of high-frequency
information, which corresponds to a lack of important fine-grained details in
the spatial space. The frequency gap may be attributed to the inherent bias of
neural networks. Xu et al. [35] propose F-Principle that neural networks first
quickly capture low-frequency components and then slowly capture the high-
frequency ones. Therefore, with no supervision on the high-frequency
components, neural networks fail to maintain important high-frequency
information.

In order to decrease the deviation in the frequency space, frequency-based
methods, such as Wavefill [38], DFT [22], GLama [17], decompose the input
image into multiple frequency bands using discrete wavelet transform and
recover the corrupted regions in each band individually and explicitly. The
separately completed features in different frequency bands are then
transformed back into the spatial domain to produce the final image.
Nevertheless, the final image generated by directly combining the individually
completed frequency features suffers from neglecting features in the space
domain and incorrect structures or distortions, resulting in poor perceptual
quality as shown in Fig. 1.

To address the above problems, we propose a Frequency-Spatial Complementary
Network (FSCN), which exploits both the frequency and spatial information to
improve image inpainting quality. The key idea of FSCN is to introduce an
extra Frequency Branch and an additional Frequency Loss on the spatial-based
network to explicitly provide a frequency constraint, and a Frequency-Spatial
Cross-Attention Block (FSCAB) to fuse multi-domain features in both the
spatial and frequency branches.

Specifically, in the frequency branch, we apply the Fast Fourier
Transformation to each image and adopt several residual blocks to fit training
data in the frequency domain. The frequency branch provides great impedance to
prevent the deviation of important frequency components due to the inherent
bias of neural networks. Existing inpainting methods usually adopt loss
functions in the spatial domain, while we introduce an additional frequency
loss. The frequency loss directly regulates the consistency and penalizes
deviation of images in the frequency domain during training. After that, a
fusion block is adopted to enhance mutual information in frequency and spatial
domains and explicitly supervise each other via a cross-attention block
(FSCAB). By rescaling original spatial features based on the correlation with
frequency features, FSCAB can fully combine the advantages of both frequency
and spatial features, not only to capture the frequency information but also
to achieve visually plausible results.

We conduct extensive experiments on the CelebA-HQ [9] and Places [42] datasets
for evaluation. Quantitative and qualitative results demonstrate that FSCN can
effectively achieve superior results. Furthermore, our model outperforms MAT
[12] in terms of the SSIM metric ($0.9184$ versus $0.9033$), with parameters
decreasing from $61.6$M to $12.3$M and MACs decreasing from $418.7$G to
$62.72$G.

The main contributions are summarized as follows:

  * •

We propose a Frequency-Spatial Complementary Network, which includes an extra
Frequency Branch and a Frequency-Spatial Cross-Attention Block to fuse multi-
domain features, taking full advantage of characteristics of both the
frequency and spatial features. To the best of our knowledge, this is the
first demonstration of joint frequency and spatial domains in feature
constraint.

  * •

We propose an additional Frequency Loss with regard to the commonly used
spatial-domain network. Such design helps narrow down the frequency gap due to
the inherent bias of neural networks and ensures that the inpainted images
share similar frequency spectra with ground truth images.

  * •

We demonstrate that our proposed FSCN can effectively improve model
performance against existing SOTA methods in terms of both quantitative and
qualitative metrics by extensive experimental studies on public datasets. It
brings visually more realistic results with significantly less computational
cost.

Figure 2: The overall architecture of our proposed FSCN. An input image is fed
into the spatial branch and frequency branch simultaneously, and the generated
features are then fused via a Frequency-Spatial Cross-Attention Block (FSCAB)
to combine the characteristics of both the frequency and spatial features. The
frequency loss and spatial loss are introduced to regulate both frequency and
spatial consistency for high-fidelity reconstruction.

##  2 Related Works

###  2.1 Deep Image Inpainting

Recently, deep learning based image inpainting algorithms show superior
performance over the traditional ones [15, 19, 24, 41, 25, 36]. Among them,
[20] first proposes a deep learning inpainting method with an encoder-decoder
architecture. Partial convolution [14] and gated convolution [37] are proposed
where the convolution is masked and conditioned only on valid pixels. In [33],
a two-stage generative model is proposed to handle the image inpainting task
in an end-to-end manner. Edgeconnect [18] is proposed to predict the complete
edge map in the first stage, which is used as prior knowledge to guide the
second-stage inpainting network to restore more refined results. Transformers
and convolutions are also unified to model long-range interactions to
guarantee high-fidelity image inpainting [12]. Despite their immense success,
most methods only explore spatial-based features and pose no constraint on the
high-frequency components. Due to the inherent bias of neural networks, they
will first quickly capture low-frequency components and then slowly capture
the high-frequency ones, leading to the lack of important high-frequency
details, which can be perceived as over-smooth artifacts. Our method
introduces an extra frequency branch and an additional frequency loss to the
commonly used spatial-domain network to narrow the gap between the inpainted
image and the ground truth image in the frequency domain.

###  2.2 Frequency-based Methods

Frequency analysis has been incorporated into various computer vision tasks,
such as quality enhancement [28, 39], image editing [3] and image super-
resolution [1]. Xu et al. [34] show that frequency-domain learning can better
preserve image information than the common spatial approaches and consequently
achieve better performance. A focal frequency loss [8] is introduced to allow
neural networks to adaptively focus on the frequency components that are more
difficult to generate by mining hard frequencies. Since the reconstruction
loss and adversarial loss focus on generating different frequency components,
WaveFill [38] is proposed to decompose images into multiple frequency bands by
discrete wavelet transform to mitigate inter-frequency conflicts. To tackle
the problem of blind face inpainting, a two-stage method named FT-TDR is
proposed in [27] with a network detecting the corrupted regions with the
exploration of frequency information. These frequency-based methods try to
narrow the information gap in the frequency domain, while they still obtain
unpleasant perceptual quality, as they fail to fully exploit spatial-domain
features. Different from them, we propose a Frequency-Spatial Cross-Attention
Block to fuse the cross-domain features and make full advantage of the feature
correlations both in the frequency and spatial domains, achieving both less
frequency loss and better perceptible results.

##  3 Proposed Method

The overall framework of our Frequency-Spatial Complementar Network (FSCN) is
shown in Fig. 2. Specifically, we adopt the frequency and spatial branches
simultaneously to fill the corrupted regions from respective perspectives and
regulate both the frequency and spatial consistency. Then, we introduce the
Frequency-Spatial Cross-Attention Block (FSCAB) to fuse multi-domain features
and make full use of the advantages of these branches. In the following, we
describe our proposed method along with the training objective functions in
detail.

###  3.1 Frequency Branch

According to the F-Principle [35], with no constraint on the frequency
components, neural networks tend to fit them from low to high frequencies. As
a result, the generated frequency spectra often deviate from the ground truth,
which corresponds to different artifacts in the visual space, e.g., over-
smooth. To this end, we introduce a frequency branch to complete the input
image from the frequency perspective and regulate frequency consistency, which
helps to recover more high-frequency details and improve image quality.

We use the Fast Fourier Transformation (FFT) to convert an image into its
frequency representation. Given a real 2D image $I$ of size $H\times W$, we
apply the Discrete Fourier Transform $\mathcal{F}$:

$\mathcal{F}(I)(u,v)=\sum^{H}_{h=1}\sum^{W}_{w=1}I(h,w)\cdot e^{-2\pi
i\left(\frac{hu}{H}+\frac{wv}{W}\right)},$ (1)

where $I(h,w)$ is the pixel value coordinated at $(h,w)$ in the spatial
domain, $\mathcal{F}(I)(u,v)$ is the complex frequency value coordinated at
$(u,v)$ in the frequency domain, and $i$ is the imaginary unit.

After transforming an image into the frequency space, we apply several
residual blocks to extract features and recover corrupted regions from the
frequency perspective. The generated frequency features are then fused with
spatial features. According to the spectral convolution theorem in Fourier
theory, each point in the spectral domain would globally affect all the input
pixels of the spatial domain, which contributes to non-local receptive fields
and global feature representational ability.

###  3.2 Spatial Branch

Complement to the frequency branch that helps recover high-frequency details,
the spatial branch is introduced to explore positional and structural
information, which is beneficial for reducing artifacts such as misalignment
and aliasing. In the spatial branch, we design a UNet architecture [21] based
on residual blocks. Skip connections are added to an encoder-decoder to
connect feature maps and masks. The input to the final residual block includes
the concatenation of the original input image with holes and original mask,
encouraging the model to duplicate non-hole pixels. The spatial features are
extracted via convolutions, which only operate in a local manner and interact
with adjacent pixels to encourage local consistency. Therefore, the frequency
branch and the spatial branch are complementary in extracting both local and
global features.

###  3.3 Frequency-Spatial Cross-Attention Block

Since the frequency branch and the spatial branch complementarily address
different problems, a Frequency-Spatial Cross-Attention Block (FSCAB) is added
to the bottleneck of the UNet architecture to aggregate the frequency and
spatial features.

As illustrated in Fig. 2, we propose FSCAB inspired by the principle of non-
local block [30]. Given the input frequency feature $x^{F}$ and the input
spatial feature $x^{S}$, we first implement a convolution $\theta$ to
transform the frequency feature $x^{F}$ to an embedding space. Then we reshape
$x^{F}$ into $C\times HW$ and $x^{S}$ into $C\times HW$, where $H$ and $W$
denote the height and width respectively, and $C$ denotes the number of
channels.

Afterwards, we conduct a matrix multiplication between the transpose of
frequency feature and spatial feature to calculate the inter-pixel attention
scores. The attention scores are activated by ReLU to suppress negative
values,

$f(x_{i}^{F},x_{j}^{S})={\rm
ReLU}\left(\frac{\theta(x_{i}^{F})^{T}x_{j}^{S}}{\|\theta(x_{i}^{F})\|\cdot\|x_{j}^{S}\|}\right),$
(2)

where $f(x_{i}^{F},x_{j}^{S})$ denotes the correlation score between the
$i$-th frequency feature vector and the $j$-th spatial feature vector.

Next, we feed forward the spatial feature $x^{S}$ and perform a matrix
multiplication between $x^{S}$ and the attention score matrix, and then resize
it back to the original size $C\times H\times W$. In this way, the original
spatial features are rescaled taking into account the correlation with
frequency features. Thus, the output mixed feature $y$ can be calculated as:

$y_{i}=\frac{1}{\mathcal{C}(x)}\sum_{\forall j}f(x_{i}^{F},x_{j}^{S})\cdot
x_{j}^{S},$ (3)

where $i$ is the position index of the output feature. The input spatial
feature $x^{S}_{j}$ at each position is aggregated into position $i$ with the
attention score $f(x_{i}^{F},x_{j}^{S})$. The output feature is normalized by
$\mathcal{C}(x)=\sum_{\forall j}f(x_{i}^{F},x_{j}^{S})$.

Compared to the original non-local block, our FSCAB contains fewer parameters
by removing unnecessary embedding functions $\phi$ and $g$, being able to fuse
features efficiently and effectively.

Such multi-domain feature fusion brings in two advantages: on one hand, since
the frequency and spatial features are extracted in a global and local manner
respectively, the mixed features generated by our FSCAB achieve cross-scale
aggregation. Such cross-scale aggregation is beneficial to image inpainting
since high-level semantic features can guide the completion of low-level
features. On the other hand, the mixed features can make full use of the
characteristics of both the frequency and spatial domains, being able to not
only capture the frequency information but also achieve visually pleasing
results.

###  3.4 Loss Functions

To effectively guide the training process, it is important to devise the loss
function to measure the distance between the generated images and ground
truth. Therefore, we introduce multiple loss functions from different aspects
as follows. Motivated by PatchGAN [7], we adopt a discriminator that works on
local levels and discriminates whether a patch is real or fake.

Let $G$ be the generator and $D$ be the discriminator. Given a binary mask $M$
($1$ for holes) and a ground truth image $I_{gt}$, the input image with hole
$I_{in}$ is formulated as dot product of $I_{gt}$ and $M$, $I_{in}=I_{gt}\odot
M.$ Taking the input image $I_{in}$, image inpainting aims to restore high-
quality and semantic consistent result $I_{out}=G(I_{in},M)$.

Firstly, we introduce the frequency loss to penalize frequency deviation from
the ground truth images and impede loss of frequency information. In order to
stabilize training, we transform $\mathcal{F}$ from the complex number domain
to the real number domain and use a log form:

$\begin{split}\mathcal{F}_{R}(I)(u,v)=\log(1+\sqrt{\left[{\rm
Real}\mathcal{F}(I)(u,v)\right]^{2}}\\\ +\sqrt{\left[{\rm
Imag}\mathcal{F}(I)(u,v)\right]^{2}}+\epsilon),\end{split}$ (4)

where $\rm Real$ and $\rm Imag$ indicate the real and imaginary parts of
$\mathcal{F}(I)(u,v)$ and $\epsilon=1\times 10^{-8}$ is included for numerical
stability. Then we calculate the L1 reconstruction loss between the output of
the frequency branch $I_{fre}$ and the ground truth image $I_{gt}$ in the
Fourier space:

$\mathcal{L}_{fre}=\|\mathcal{F}_{R}(I_{gt})-\mathcal{F}_{R}(I_{fre})\|_{1}.$
(5)

Secondly, we adopt the L1 loss between the inpainted result $I_{gt}$ and the
ground truth $I_{out}$ in the spatial domain. We calculate the loss regarding
only the unmasked pixels as:

$L_{rec}=\|(1-M)\odot(I_{gt}-I_{out})\|_{1},$ (6)

where $\odot$ denotes element-wise multiplication.

Thirdly, we use an adversarial loss to guarantee that our model can generate
realistic images. The adversarial loss is calculated as:

$\mathcal{L}_{D}=-{E}_{{I}_{gt}}[\log(D({I}_{gt}))]-{E}_{{I}_{out}}[\log(1-D({I}_{out}))]$
(7) $\mathcal{L}_{G}=-{E}_{I_{out}}[\log(D(I_{out}))]$ (8)

What is more, following pix2pixHD [29], we adopt the feature matching loss to
match intermediate representations of the discriminator as

$\mathcal{L}_{fm}={E}\left(\sum_{i=1}^{L}\frac{1}{N_{i}}\|D^{i}(I_{gt})-D^{i}(I_{out})\|_{1}\right),$
(9)

where $N_{i}$ denotes the number of elements in the $i$-th layer, and $L$ is
the number of total layers of the discriminator.

Lastly, we adopt a perceptual loss to evaluate the discrepancy between
features extracted from the inpainted and ground truth images by a pretrained
network $\Phi(\cdot)$.

$\mathcal{L}_{perc}=\sum_{i=1}^{T}\|\Phi^{i}(I_{gt})-\Phi^{i}(I_{out})\|_{2},$
(10)

where $T$ is the total number of layers in the network, and $\Phi^{i}$ denotes
the $i$-th activation map of the pretrained network ResNet50 with dilated
convolutions, following LaMa [26]. The final loss function for our model is:

$\mathcal{L}=\mathcal{L}_{fre}+\lambda_{r}\mathcal{L}_{rec}+\lambda_{g}\mathcal{L}_{G}+\lambda_{fm}\mathcal{L}_{fm}+\lambda_{p}\mathcal{L}_{perc}.$
(11)

We set $\lambda_{r}=10,\lambda_{g}=10,\lambda_{fm}=100,\lambda_{p}=30$
empirically to form a linear combination of the above losses.

##  4 Experiments

###  4.1 Experiment Setup

Datasets. Our model is trained on two public datasets: CelebA-HQ [9] and
Places [42]. The CelebA-HQ dataset is a high-quality version of CelebA [16]
that consists of $30,000$ human face images. We randomly sample $26,000$ of
them as training images, $2000$ as validation images, and $2,000$ as test
images. The Places dataset contains more than $10$ million images, including
more than $434$ unique scene categories. We randomly sample $100,000$ training
images, $2,000$ validation images and $30,000$ test images,

Evaluation Metrics. All results are evaluated by the metrics of Frechet
Inception Distance (FID) [4], Learned Perceptual Image Patch Similarity
(LPIPS) [40] and Structural Similarity (SSIM) [32], all of which are more
consistent with human visual neurobiology and perception. FID compares the
distribution of generated images with that of real images, and LPIPS
calculates the L2 distance of features extracted by a pretrained model. SSIM
is composed of three measurements with regard to luminance, contrast, and
structure.

Implementation Details. For the spatial branch, we adopt a UNet [21]
architecture with $4$ downsampling residual blocks and $4$ upsampling residual
blocks. There are $5$ residual blocks in the frequency branch. Our model is
trained using $256\times 256$ images with random masks generated following the
settings in LaMa [26]. We use Adam optimizer [11] with $\beta_{1}=0.9$ and
$\beta_{2}=0.999$, and set fixed learning rates as $0.001$ and $0.0001$ for
the generator and discriminators respectively. Our model is trained on NVIDIA
V100 GPUs with a batch size of $30$ for $1$ million iterations. We prepare
test sets with irregular random masks of different widths (thin, medium,
thick) following LaMa [26]. This strategy uniformly uses samples from polygon
chains extended by rectangles of high random width and arbitrary aspect ratio.
We set different values of the following parameters to generate different
types of masks: a probability of a polygonal chain mask, min number of
segments, max number of segments, max length of a segment in polygonal chain,
max width of a segment in polygonal chain, min bound for the number of box
primitives, max bound for the number of box primitives, min length of a box
side, max length of a box side, etc. We show samples of masks in Fig. 3.

|  |
---|---|---
(a) Thin | (b) Medium | (c) Thick
Figure 3: Samples of different types of masks.

###  4.2 Ablation Study

To study the effects of individual components in the proposed model FSCN, we
conduct several ablation studies on the dataset of CelebA-HQ. Specifically, we
train five model variants including (a) removing the frequency branch and
retaining only the spatial branch (“w/o fre branch”), (b) retaining both the
frequency and spatial branches but removing the frequency loss (“w/o fre
loss”), (c) replacing our feature fusion module FSCAB by concatenating multi-
domain features (“concat”), (d) concatenating the frequency and spatial
features and then adopt self-attention to fuse multi-domain features (“self-
attention”), (e) our final model with both the frequency and spatial branches,
the frequency loss and feature fusion block FSCAB.

Table 1: Quantitative evaluation of ablation studies on CelebA-HQ dataset. Our
full model equipped with the proposed frequency branch, the frequency loss and
feature fusion block FSCAB achieves the best results.

Model | Thin | Medium | Thick
---|---|---|---
FID $\downarrow$ |  LPIPS $\downarrow$ |  FID $\downarrow$ |  LPIPS $\downarrow$ |  FID $\downarrow$ |  LPIPS $\downarrow$
w/o fre branch | 6.2787 | 0.0909 | 5.5051 | 0.0826 | 5.6125 | 0.0948
w/o fre loss | 5.3982 | 0.0793 | 5.1040 | 0.0770 | 5.3505 | 0.0898
concat | 5.4659 | 0.0852 | 5.0711 | 0.0811 | 5.3650 | 0.0945
self-attention | 5.4459 | 0.0847 | 5.0489 | 0.0782 | 5.2837 | 0.0910
ours | 5.1956 | 0.0768 | 5.0189 | 0.0762 | 5.2123 | 0.0897

The results of the five model variants are reported in Table 1. As it is
shown, removing the frequency branch results in a significant performance
decrease. The frequency branch can help preserve the frequency information and
decrease the discrepancy between the generated images and the ground truth
images in the frequency domain, which can be reflected in the spatial space as
perceptible details. Removing the frequency loss causes a further decrease
since the frequency loss penalizes deviating from the frequency spectra of the
ground truth images and helps ensure frequency consistency. Visual evaluation
in Fig. 4 (b) and (c) is consistent with quantitative experiments, suggesting
that the model fails to recover fine-grained details without the frequency
branch and frequency loss, e.g. unreal texture fog in (b) and unreasonable
color distortion in (c). Simply concatenating the frequency and spatial
features is not able to make full use of multi-domain feature aggregation, and
adding self-attention does not help much. Although image details are restored,
such insufficient fusion leads to mismatches in frequency and spatial domains,
as shown by the unreasonable color around the mouth and nose in Fig. 4 (d) and
the ghost artifacts on the contour of the face in Fig. 4 (e). With our FSCAB,
the effectively mixed features can take full advantage of the characteristics
of both the frequency and spatial features to capture frequency information
and obtain visually pleasing results simultaneously.

|  |  |
---|---|---|---
(a) original | (b) w/o fre branch | (c) w/o fre loss |
|  |  |
(d) concat | (e) self-attention | (f) ours |
Figure 4: Visual ablation study results on CelebA-HQ dataset (zoom in for a
better view). The proposed frequency branch, frequency loss and feature fusion
block contribute to suppress artifacts and recover high-quality results.

###  4.3 Quantitative Evaluation

We compare our model with other state-of-the-art methods: LaMa [26], Wavefill
[38], MAT [12], MADF [43], DMFN [6], Deepfillv2 [37] and EdgeConnect [18]. To
make the comparison fair, only the publicly available pretrained models are
used to compute the evaluation metric values. For each dataset, we validate
the performance across three different types of masks.

All the results of compared methods are reported in Table 2 for the CelebA-HQ
dataset and Table 3 for the Places dataset. Note that, we abbreviate
EdgeConnect as EC and Deepfillv2 as GC. The results demonstrate that our model
consistently achieves the best performance on different benchmarks in most
cases, no matter in the scenery of a large mask or a thin mask. On the CelebA-
HQ dataset, compared with the previous SOTA method LaMa with only the spatial
domain features, FID of our proposed FSCN is improved by $24.02\%$. Compared
to previous SOTA method Wavefill with only features in the frequency domain,
SSIM is improved from $0.9207$ to $0.9247$. Although a thick mask leads to
incoherent structure, there is still a significant improvement in all the
metrics compared with previous works. It implies that our model can make full
use of informative features from both the frequency and spatial domains, which
attributes to our proposed frequency branch, frequency loss, and multi-domain
feature fusion block. Subsequently, our model can capture global semantic
information and preserve the overall structure and outline of images, which is
beneficial to high-fidelity image inpainting.

Table 2:  Quantitative evaluation on CelebA-HQ dataset [9]. The best and
second best results are highlighted.

Method | Thin | Medium | Thick
---|---|---|---
FID $\downarrow$ | LPIPS $\downarrow$ | SSIM $\uparrow$ | FID $\downarrow$ | LPIPS $\downarrow$ | SSIM $\uparrow$ | FID $\downarrow$ | LPIPS $\downarrow$ | SSIM $\uparrow$
EC $(ICCV^{\prime}2019)$ [18] | 7.0074 | 0.0921 | 0.9124 | 6.1678 | 0.0919 | 0.9033 | 6.9193 | 0.1107 | 0.8783
GC $(ICCV^{\prime}2019)$ [37] | 11.1879 | 0.1295 | 0.8964 | 8.1926 | 0.1063 | 0.8954 | 9.9133 | 0.1213 | 0.8690
MADF $(TIP^{\prime}2021)$ [43] | 6.9000 | 0.0858 | 0.9166 | 10.2428 | 0.1027 | 0.8992 | 16.4815 | 0.1331 | 0.8706
DMFN $(arXiv^{\prime}2020)$ [6] | 7.8631 | 0.1055 | 0.9070 | 6.2192 | 0.0888 | 0.9062 | 6.9401 | 0.1041 | 0.8839
Wavefill $(ICCV^{\prime}2021)$ [38] | 5.2502 | 0.0790 | 0.9207 | 5.4269 | 0.0815 | 0.9096 | 5.6187 | 0.0963 | 0.8889
LaMa $(WACV^{\prime}2022)$ [26] | 5.7965 | 0.0825 | 0.9208 | 5.2425 | 0.0792 | 0.9141 | 5.5395 | 0.0924 | 0.8942
Our FSCN | 4.6739 | 0.0762 | 0.9247 | 4.9603 | 0.0766 | 0.9162 | 5.4149 | 0.0897 | 0.8971

Table 3:  Quantitative evaluation on Places dataset [42]. The best and second
best results are highlighted.

Method | Thin | Medium | Thick
---|---|---|---
FID $\downarrow$ | LPIPS $\downarrow$ | SSIM $\uparrow$ | FID $\downarrow$ | LPIPS $\downarrow$ | SSIM $\uparrow$ | FID $\downarrow$ | LPIPS $\downarrow$ | SSIM $\uparrow$
EC (ICCV’2019) [18] | 1.3542 | 0.1110 | 0.9347 | 3.6796 | 0.1347 | 0.8720 | 8.5293 | 0.1588 | 0.8492
GC (ICCV’2019) [37] | 1.0668 | 0.1044 | 0.9061 | 2.7125 | 0.1300 | 0.8688 | 5.2606 | 0.1541 | 0.8391
Wavefill (ICCV’2021) [38] | 0.9794 | 0.0994 | 0.9075 | 1.1959 | 0.2828 | 0.6473 | 1.3204 | 0.3679 | 0.5467
MADF (TIP’2021) [43] | 0.5787 | 0.0857 | 0.9140 | 1.6844 | 0.1145 | 0.8796 | 3.7856 | 0.1377 | 0.8560
MAT (CVPR’2022) [12] | 0.6298 | 0.0951 | 0.9033 | 1.2386 | 0.1216 | 0.8676 | 1.8765 | 0.1430 | 0.8428
LaMa (WACV’2022) [26] | 0.6380 | 0.0905 | 0.9153 | 1.3152 | 0.1121 | 0.8857 | 2.2150 | 0.1334 | 0.8669
Our FSCN | 0.5436 | 0.0807 | 0.9184 | 1.1555 | 0.1077 | 0.8860 | 2.1703 | 0.1316 | 0.8640

On the Places dataset, our results still outperform most of the other methods,
both in the frequency domain and spatial domain. In the case of a thick mask,
it is slightly worse than two competitive methods LaMa and MAT. One possible
reason is that thick masks pose higher requirements on the generation ability
for models to recover high-fidelity results. Both LaMa and MAT are complex
with a larger amount of calculation and huge model capacity. Specifically,
LaMa uses approximately $2\times$ and MAT employs about $5\times$ parameters
than FSCN. However, these two methods are not superior in terms of
visualization results in Fig. 6. In terms of EdgeConnect under a thin mask, it
uses edge map as prior knowledge to guide the reconstruction. Thus it helps
preserve structural information and obtains better SSIM values.

|
---|---

|
---|---
(a) Masked Input |  (b) Output

Figure 5:  Our model produces realistic results for the real-world object
removal task (zoom in for a better view).

###  4.4 Qualitative Evaluation

As shown in Fig. 6 and Fig. 7, we also compare exemplar visual results of
various existing SOTA inpainting methods with ours. The figures show that our
model produces visually pleasing results with finer details and image
structures, outperforming other methods significantly. For example, in the
“pizza” image in bottom row of Fig. 6, MAT recovers images with abnormal color
and LaMa suffers from grid-like artifacts, while our model produces more
faithful results with fewer artifacts and finer fine-grained details. What is
more, the model complexity also brings side effects to MAT. As shown in Fig.
6, MAT generates a semantically inconsistent object “windows” in the bush
(first row), which does not match the image context. Since deep neural
networks pay more attention to low-frequency components, the generated
frequency spectra often deviate from the ground truth spectra. This can be
reflected in the visual space as the lack of important fine-grained details,
leading to unpleasant perceptual quality. However, the frequency branch in
FSCN helps prevent the deviation of important frequency components due to the
inherent bias of neural networks and decreases the distance between the
generated images and the ground truth images in the frequency domain. Also, in
Fig. 7, FSCN recovers reasonable face shape and edge details, while LaMa fails
in recovering fine-grained details of the facial region, and WaveFill leads to
unreal facial organs. Since the frequency and spatial features are extracted
in a global and local manner respectively, the well-fused cross-scale features
are beneficial to the image inpainting task since high-level semantic features
can guide the completion of low-level features, resulting in better perceptual
quality. See supplementary materials for more results.

|  |  |  |  |
---|---|---|---|---|---

|  |  |  |  |
---|---|---|---|---|---
(a) Original |  (b) EdgeConnect |  (c) Deepfillv2 |  (d) LaMa |  (e) MAT |  (f) Ours

Figure 6: Qualitative comparison of the SOTA methods with our model on the
Places [42] dataset. Our results are more visually pleasing compared with SOTA
methods. Zoom in for a better view.

|  |  |  |  |
---|---|---|---|---|---

|  |  |  |  |
---|---|---|---|---|---
(a) Original |  (b) EdgeConnect |  (c) Deepfillv2 |  (d) LaMa |  (e) WaveFill |  (f) Ours

Figure 7: Qualitative comparison of the SOTA methods with our model on the
CelebA-HQ [9] dataset. Our model generates more realistic inpainting results
with fewer artifacts compared with SOTA methods. Zoom in for a better view.

###  4.5 Real-world Applications

We also conduct experiments on real-world applications as shown in Fig. 5.
More results can be seen in supplementary materials. In the first case of
object removal, with dense structures and textures, our model recovers
semantically consistent textures. In the second case, it effectively restores
the building structure and recovers the background with fine-grained details,
producing a visually realistic image, which demonstrates the effectiveness of
our model.

###  4.6 Model Complexity Analyses

Figure 8: Model size comparison. The smaller the circle, the fewer the MACs.

We compare model parameters and MACs with other advanced inpainting methods in
Fig. 8. It is observed that our model is located in the upper left, indicating
that our model can achieve better results with fewer parameters and less
computation cost. Our model is superior to all other methods with fewer
parameters and MACs. More specifically, our model surpasses LaMa with regard
to all evaluation metrics on the CelebA-HQ dataset with only half parameters
($27.0$M versus $12.3$M) and one-third MACs ($42.85$G versus $15.68$G). It
illustrates the superiority of our model over other state-of-the-art
approaches.

##  5 Conclusion

In this paper, we propose a Frequency-Spatial Complementary Network (FSCN) for
high-fidelity image inpainting. The network contains an extra Frequency Branch
and Frequency Loss to exploit features in the frequency domain and ensure
frequency consistency, and a Frequency-Spatial Cross-Attention Block to fuse
multi-domain features for better completion. Quantitative and qualitative
experiments demonstrate that FSCN can effectively produce high-fidelity
results with semantically consistent structures and fine-grained details,
outperforming the SOTA methods with fewer parameters and less computational
cost.

## References

  * [1] Xin Deng, Ren Yang, Mai Xu, and Pier Luigi Dragotti.  Wavelet domain style transfer for an effective perception-distortion tradeoff in single image super-resolution.  In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 3076–3085, 2019.
  * [2] Ricard Durall, Margret Keuper, and Janis Keuper.  Watch your up-convolution: Cnn based generative deep neural networks are failing to reproduce spectral distributions.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7890–7899, 2020.
  * [3] Yue Gao, Fangyun Wei, Jianmin Bao, Shuyang Gu, Dong Chen, Fang Wen, and Zhouhui Lian.  High-fidelity and arbitrary face editing.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16115–16124, 2021.
  * [4] Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter.  Gans trained by a two time-scale update rule converge to a local nash equilibrium.  Advances in Neural Information Processing Systems, 30, 2017.
  * [5] Jain-Kai Huang, Tsung-Jung Liu, and Kuan-Hsien Liu.  Image inpainting with frequency domain wavelet convolution.  In Proceedings of the IEEE Visual Communications and Image Processing, 2022.
  * [6] Zheng Hui, Jie Li, Xiumei Wang, and Xinbo Gao.  Image fine-grained inpainting.  arXiv preprint arXiv:2002.02609, 2020.
  * [7] Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, and Alexei A Efros.  Image-to-image translation with conditional adversarial networks.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1125–1134, 2017.
  * [8] Liming Jiang, Bo Dai, Wayne Wu, and Chen Change Loy.  Focal frequency loss for image reconstruction and synthesis.  In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 13919–13929, 2021.
  * [9] Tero Karras, Timo Aila, Samuli Laine, and Jaakko Lehtinen.  Progressive growing of gans for improved quality, stability, and variation.  arXiv preprint arXiv:1710.10196, 2017.
  * [10] Soo Ye Kim, Kfir Aberman, Nori Kanazawa, Rahul Garg, Neal Wadhwa, Huiwen Chang, Nikhil Karnad, Munchurl Kim, and Orly Liba.  Zoom-to-inpaint: Image inpainting with high-frequency details.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022.
  * [11] Diederik P Kingma and Jimmy Ba.  Adam: A method for stochastic optimization.  arXiv preprint arXiv:1412.6980, 2014.
  * [12] Wenbo Li, Zhe Lin, Kun Zhou, Lu Qi, Yi Wang, and Jiaya Jia.  Mat: Mask-aware transformer for large hole image inpainting.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10758–10768, 2022.
  * [13] Guilin Liu, Fitsum A. Reda, Kevin J. Shih, Ting-Chun Wang, Andrew Tao, and Bryan Catanzaro.  Image inpainting for irregular holes using partial convolutions.  In Proceedings of the European Conference on Computer Vision, 2018\.
  * [14] Guilin Liu, Fitsum A Reda, Kevin J Shih, Ting-Chun Wang, Andrew Tao, and Bryan Catanzaro.  Image inpainting for irregular holes using partial convolutions.  In Proceedings of the European Conference on Computer Vision (ECCV), pages 85–100, 2018.
  * [15] Jiaying Liu, Shuai Yang, Yuming Fang, and Zongming Guo.  Structure-guided image inpainting using homography transformation.  IEEE Transactions on Multimedia, 20(12):3252–3265, 2018.
  * [16] Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang.  Deep learning face attributes in the wild.  In Proceedings of the IEEE International Conference on Computer Vision, pages 3730–3738, 2015.
  * [17] Zeyu Lu, Junjun Jiang, Junqin Huang, Gang Wu, and Xianming Liu.  Glama: Joint spatial and frequency loss for general image inpainting.  In Proceedings of the IEEE/CVF International Conference on Computer Vision, 2022.
  * [18] Kamyar Nazeri, Eric Ng, Tony Joseph, Faisal Qureshi, and Mehran Ebrahimi.  Edgeconnect: Structure guided image inpainting using edge prediction.  In Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops, pages 0–0, 2019.
  * [19] Tien-Dat Nguyen, Beomsu Kim, and Min-Cheol Hong.  New hole-filling method using extrapolated spatio-temporal background information for a synthesized free-view.  IEEE Transactions on Multimedia, 21(6):1345–1358, 2019.
  * [20] Deepak Pathak, Philipp Krahenbuhl, Jeff Donahue, Trevor Darrell, and Alexei A Efros.  Context encoders: Feature learning by inpainting.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2536–2544, 2016.
  * [21] Olaf Ronneberger, Philipp Fischer, and Thomas Brox.  U-net: Convolutional networks for biomedical image segmentation.  In International Conference on Medical Image Computing and Computer-assisted Intervention, pages 234–241. Springer, 2015.
  * [22] Hiya Roy, Subhajit Chaudhury, Toshihiko Yamasaki, and Tatsuaki Hashimoto.  Image inpainting using frequency domain priors.  arXiv preprint arXiv:2012.01832, 2012.
  * [23] Katja Schwarz, Yiyi Liao, and Andreas Geiger.  On the frequency bias of generative models.  Advances in Neural Information Processing Systems, 34:18126–18136, 2021.
  * [24] Hongyi Sun, Wanhua Li, Yueqi Duan, Jie Zhou, and Jiwen Lu.  Learning adaptive patch generators for mask-robust image inpainting.  IEEE Transactions on Multimedia, pages 1–1, 2022.
  * [25] Jiande Sun, Fanfu Xue, Jing Li, Lei Zhu, Huaxiang Zhang, and Jia Zhang.  Tsinit: A two-stage inpainting network for incomplete text.  IEEE Transactions on Multimedia, pages 1–11, 2022.
  * [26] Roman Suvorov, Elizaveta Logacheva, Anton Mashikhin, Anastasia Remizova, Arsenii Ashukha, Aleksei Silvestrov, Naejin Kong, Harshith Goka, Kiwoong Park, and Victor Lempitsky.  Resolution-robust large mask inpainting with fourier convolutions.  In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 2149–2159, 2022.
  * [27] Junke Wang, Shaoxiang Chen, Zuxuan Wu, and Yu-Gang Jiang.  Ft-tdr: Frequency-guided transformer and top-down refinement network for blind face inpainting.  IEEE Transactions on Multimedia, pages 1–1, 2022.
  * [28] Jianyi Wang, Xin Deng, Mai Xu, Congyong Chen, and Yuhang Song.  Multi-level wavelet-based generative adversarial network for perceptual quality enhancement of compressed video.  In European Conference on Computer Vision, pages 405–421. Springer, 2020.
  * [29] Ting-Chun Wang, Ming-Yu Liu, Jun-Yan Zhu, Andrew Tao, Jan Kautz, and Bryan Catanzaro.  High-resolution image synthesis and semantic manipulation with conditional gans.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8798–8807, 2018.
  * [30] Xiaolong Wang, Ross Girshick, Abhinav Gupta, and Kaiming He.  Non-local neural networks.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7794–7803, 2018.
  * [31] Yi Wang, Xin Tao, Xiaojuan Qi, Xiaoyong Shen, and Jiaya Jia.  Image inpainting via generative multi-column convolutional neural networks.  In Proceedings of the Neural Information Processing Systems, 2018\.
  * [32] Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero P Simoncelli.  Image quality assessment: from error visibility to structural similarity.  IEEE Transactions on Image Processing, 13(4):600–612, 2004.
  * [33] Haiwei Wu, Jiantao Zhou, and Yuanman Li.  Deep generative model for image inpainting with local binary pattern learning and spatial attention.  IEEE Transactions on Multimedia, 24:4016–4027, 2022.
  * [34] Kai Xu, Minghai Qin, Fei Sun, Yuhao Wang, Yen-Kuang Chen, and Fengbo Ren.  Learning in the frequency domain.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1740–1749, 2020.
  * [35] Zhi-Qin John Xu, Yaoyu Zhang, and Yanyang Xiao.  Training behavior of deep neural network in frequency domain.  In International Conference on Neural Information Processing, pages 264–274. Springer, 2019.
  * [36] Jiahui Yu, Zhe Lin, Jimei Yang, Xiaohui Shen, Xin Lu, and Thomas S Huang.  Generative image inpainting with contextual attention.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5505–5514, 2018.
  * [37] Jiahui Yu, Zhe Lin, Jimei Yang, Xiaohui Shen, Xin Lu, and Thomas S Huang.  Free-form image inpainting with gated convolution.  In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 4471–4480, 2019.
  * [38] Yingchen Yu, Fangneng Zhan, Shijian Lu, Jianxiong Pan, Feiying Ma, Xuansong Xie, and Chunyan Miao.  Wavefill: A wavelet-based generation network for image inpainting.  In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 14114–14123, 2021.
  * [39] Linfeng Zhang, Xin Chen, Xiaobing Tu, Pengfei Wan, Ning Xu, and Kaisheng Ma.  Wavelet knowledge distillation: Towards efficient image-to-image translation.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12464–12474, 2022.
  * [40] Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang.  The unreasonable effectiveness of deep features as a perceptual metric.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 586–595, 2018.
  * [41] Ruisong Zhang, Weize Quan, Yong Zhang, Jue Wang, and Dong-Ming Yan.  W-net: Structure and texture interaction for image inpainting.  IEEE Transactions on Multimedia, pages 1–12, 2022.
  * [42] Bolei Zhou, Agata Lapedriza, Aditya Khosla, Aude Oliva, and Antonio Torralba.  Places: A 10 million image database for scene recognition.  IEEE Transactions on Pattern Analysis and Machine Intelligence, 40(6):1452–1464, 2017.
  * [43] Manyu Zhu, Dongliang He, Xin Li, Chao Li, Fu Li, Xiao Liu, Errui Ding, and Zhaoxiang Zhang.  Image inpainting by end-to-end cascaded refinement with mask awareness.  IEEE Transactions on Image Processing, 30:4855–4866, 2021.


# Breaking BERT: Understanding its Vulnerabilities for Named Entity
Recognition through Adversarial Attack

Anne Dirkson
&Suzan Verberne
Leiden Institute of Advanced Computer Science, Leiden University
Niels Bohrweg 1, 2333 CA Leiden, the Netherlands
{a.r.dirkson, s.verberne<EMAIL_ADDRESS>
&Wessel Kraaij

###### Abstract

Both generic and domain-specific BERT models are widely used for natural
language processing (NLP) tasks. In this paper we investigate the
vulnerability of BERT models to variation in input data for Named Entity
Recognition (NER) through adversarial attack. Experimental results show that
the original as well as the domain-specific BERT models are highly vulnerable
to entity replacement: They can be fooled in 89.2 to 99.4% of the cases to
mislabel previously correct entities. BERT models are also vulnerable to
variation in the entity _context_ with 20.2 to 45.0% of entities predicted
completely wrong and another 29.3 to 53.3% of entities predicted wrong
partially. Often a single change is sufficient to fool the model. BERT models
seem most vulnerable to changes in the local context of entities. Of the two
domain-specific BERT models, the vulnerability of BioBERT is comparable to the
original BERT model whereas SciBERT is even more vulnerable. Our results chart
the vulnerabilities of BERT models for NER and emphasize the importance of
further research into uncovering and reducing these weaknesses.

##  1 Introduction

Self-attentive neural models, such as BERT (Devlin et al., 2019), attain a
high performance on a wide range of natural language processing (NLP) tasks.
Despite their excellent performance, the robustness of BERT-based models is
contested: Various studies Jin et al. (2020); Hsieh et al. (2019); Li et al.
(2020); Zang et al. (2020); Sun et al. (2020) recently showed that BERT is
vulnerable to adversarial attacks. Adversarial attacks are deliberate attempts
to fool the model into giving the incorrect output by providing it with
carefully crafted input samples, also called adversarial examples. These
findings are in line with those of Niven and Kao (2019) who found that BERT
exploits spurious statistical cues resulting from uneven distributions of
certain words and phrases (e.g. ‘cannot’ or ‘are’) over the target labels and
that its performance is reduced drastically if these cues are removed.

Both in NLP and computer vision, neural models have been shown to be
vulnerable to adversarial attacks (Szegedy et al., 2014; Alshemali and Kalita,
2019). In contrast to the vision domain however, adding imperceptable changes
to text is almost impossible (Alzantot et al., 2018). Instead, adversarial
input in the NLP field generally aims to be (1) semantics-preserving and (2)
grammatically correct. The underlying idea is that grammatically correct
sentences should be indistinguishable from text generated by humans and thus
constitutes valid input that the model should be able to handle.

At present, the work on adversarial attack of Named Entity Recognition (NER)
models is limited to a single study. Araujo et al. (2020) attack biomedical
BERT models by simulating spelling errors and replacing entities with their
synonyms. They find that both attacks drastically reduce performance of these
domain-specific BERT models on medical NER tasks.

Here, we propose two additional adversarial attack methods to investigate
different variations in NER input: using synonyms in the context of entities
and replacing entities themselves with others of the same type. In contrast to
previous work, our methods are adaptive and specifically target BERT’s
weaknesses: We create adversarial examples by making the changes to the input
that either manage to fool the model or bring it closest to making a mistake
(i.e. lower the prediction score for the correct output) instead of randomly
introducing noise or variation. This allows for a systematic test of the
model’s robustness under severe stress conditions and thereby allows us to
interpret which variation in entities and entity contexts BERT models are most
vulnerable to. In turn, this will further our understanding of what BERT
models do and do not learn.

To date, the vulnerability of generic BERT models for NER has not been
studied. It is also unknown to what extent the vulnerability of generic and
domain-specific BERT models differs for NER tasks and to what extent the
vulnerability of BERT is impacted by applying it to domain-specific data.

We address the following research questions:

  1. 1.

How vulnerable are BERT models to adversarial attack on general and domain-
specific NER?

  2. 2.

To which types of variation are BERT models for NER the most vulnerable?

Designing methods for direct adversarial attack of NER models poses additional
challenges compared to the attack of text classification models as labels are
predicted per word, sentences can contain multiple entities and entities can
contain multiple words. To ensure that labels remain accurate in our
adversarial sentences, we substitute entities only by entities of the same
type when attacking the entity itself (i.e. an _entity attack_) and constrain
synonym replacements to non-entity words when altering the context of the
entity (i.e. an _entity context attack_).

We assume a black-box setting, which means that the adversarial method has no
knowledge of the data, parameters or model architecture (Alzantot et al.,
2018). This allows our methods to also be used for other neural architectures.
Although we use English data, our methods are largely language-independent.
Only an appropriate language model for synonym selection would be required.

The contributions of this paper are twofold: We adapt existing adversarial
attack methods to sequence labelling tasks and evaluate the vulnerability of
general and domain-specific BERT models for NER. We make our code available
for follow up research.111Our code is available at:
https://github.com/AnneDirkson/breakingBERT

##  2 Related Work

Within the field of NLP, token-level black-box methods for adversarial attack
have mainly been developed for classification and textual entailment Alshemali
and Kalita (2019). Substituting tokens with their synonyms is the most popular
choice for perturbing at the token level. Synonyms are often found using
nearest neighbours in a word embeddings model. One major challenge when
selecting synonyms based on word embeddings is that antonyms will also be
close in the embedding space. To solve this issue, recent studies (Jin et al.,
2020; Li et al., 2019; Hsieh et al., 2019) require a minimal semantic
similarity between the generated and original sentence. Additionally, some
methods (Alzantot et al., 2018; Jin et al., 2020) use word embeddings with
additional synonymy constraints Mrkšić et al. (2016). We will employ both
techniques.

Approaches also differ in how they select the word that is perturbed: while
some select words randomly, it is more common to use the importance of the
word for the output Alshemali and Kalita (2019). The importance is often
operationalized as the difference in output before and after removing the
word. We follow this approach in our method.

Most adversarial attack methods were developed for attacking recurrent neural
models. However, there has been a growing interest in attacking self-attentive
models in the last year Sun et al. (2020); Hsieh et al. (2019); Jin et al.
(2020); Li et al. (2020); Zang et al. (2020); Araujo et al. (2020);
Balasubramanian et al. (2020); Mondal (2021). Nonetheless, the only study that
has attacked BERT models for NER is the study by Araujo et al. (2020). They
perform two types of character-level (i.e. swapping letters and replacing
letters with adjacent keys on the keyboard) and one type of token-level
perturbation (i.e. replacing entities with their synonyms). The authors find
that biomedical BERT models perform far worse on NER tasks when spelling
mistakes are included or synonyms of entities are used.

Our work differs from Araujo et al. (2020) in three ways. First, our
adversarial examples are generated based on the importance of words for the
correct output instead of through random changes. Thereby, we are able to test
the robustness of BERT on the most severe stress conditions, while Araujo et
al. (2020) evaluate the scenario where the input data is noisy due to spelling
mistakes and use of synonyms. Second, we analyze the impact of replacing
entities with others of the same type (e.g. ‘France’ with ‘Britain’) and
replacing words in the context of entities (see Table 1 for an example)
instead of replacing entities with their synonyms. Third, we will test our
method on the original BERT model as well as biomedical BERT models and on
both generic and biomedical NER.

##  3 Methods

In this section, we describe two methods for generating adversarial examples
designed to fool NER models, namely through (1) entity replacement (_entity
attack_) and (2) synonym replacements in the entity context (_entity context
attack_). These are described in Sections 3.1.2 and 3.1.3, respectively.

###  3.1 Aim of the attacks

We aim to generate adversarial examples in which the entire entity is no
longer recognized correctly. This can be either because it has become a false
negative or it has been assigned a different entity type. The attack is
considered a success when the correct label has been changed, unless it has
changed from the I-tag to the B-tag of the same entity type under the IOB
schema. An example of this can be seen in Table 1: Here, the start of the
entity is mislabelled but the last part of the entity is still recognized. We
consider this a partial success.

The | Republic | of | China | bought | flowers
---|---|---|---|---|---
$<$O$>$ |  $<$B-LOC$>$ |  $<$I-LOC$>$ |  $<$I-LOC$>$ |  $<$O$>$ |  $<$O$>$
The | Republic | of | China | purchased | flowers
$<$O$>$ |  $<$O$>$ |  $<$O$>$ |  $<$B-LOC$>$ |  $<$O$>$ |  $<$O$>$
Table 1: Example of a partial success. The bold word has been changed to
attack the entity ‘Republic of China’.

####  3.1.1 Metrics for evaluation of the attacks

The success of the attack and thus the vulnerability of the model is evaluated
by the percentage of entities that were originally correctly labelled but are
mislabelled after attack. For entity context attacks entities can also be
partially mislabelled i.e. only some words in the entity are mislabelled. This
is captured by the partial success rate: the percentage of entities for which
not the whole entity but at least half of the entity is mislabelled. For
entity context attacks we also include a metric (‘Words perturbed’) to measure
how much the sentence needed to be changed before the attack was successful:
the average percentage of words that were perturbed out of the total amount of
out-of-mention words in the sentences. This metric functions as a proxy for
how difficult it is to fool the model (Jin et al., 2020).

####  3.1.2 Entity Attack

To explore to what extent models rely on the words of the entity, we replace
the entity with one of the same type e.g. we change ‘Japan’ to another
location. If a sentence contains multiple entities, an adversarial sentence is
generated for each entity. The replacement entity is selected from a list of
all entities in the data that are of the same type. We randomly select 50
candidate replacements from the entity list. We filter the generated sentences
for a sufficiently high semantic similarity to the original sentence. Semantic
similarity is calculated with the Universal Sentence Encoder (USE) (Cer et
al., 2018). The USE encodes the sentences into 512 dimensional vectors and
their cosine similarity (normalized between 0 and 1) approximates their
semantic similarity. The resulting sentences need to be above the semantic
similarity threshold $\epsilon$. After filtering, we check if the predicted
label of any of the replacements is incorrect. If so, we select the successful
attack replacement with the highest semantic similarity at the sentence level.
If not, the attack was unsuccessful.

####  3.1.3 Entity Context Attack

To investigate the impact of the context on the correct labelling of the
entity, we adapt the method of Jin et al. (2020), which was designed for text
classification, to sequence labelling tasks. For each entity in the sentence,
a separate adversarial example is created, as models may rely on different
contextual words for different entities.

##### Step 1: Choosing the word to perturb

We use the importance ranking function shown in Equation 1 to rank words based
on their importance for assigning the correct label to the entity. The
importance ($I_{w}$) of a word $w$ for a token in the entity is calculated as
the change in the predictions (logits) of the correct label before and after
deleting the word from the sentence Jin et al. (2020). If the deletion of the
word leads to an incorrect label for the entity token, the importance of the
word is increased by also adding the raw prediction score (logits) attributed
to the incorrect label.

If the entity consists of multiple words, we rank words based on their summed
importance for correctly labelling each of the individual words in the entity.
Besides stop words, we also exclude other entities from being perturbed. We
adapt the function so that for any word with an I-tag, both the I and B label
of the entity type (e.g. B-PER and I-PER) are considered correct.

Given a sentence of $n$ words $X={w_{1},w_{2},...,w_{n}}$, the importance
($I_{w}$) of a word $w$ for a token in the entity is formally defined as:

$\begin{split}I_{w}=&F_{Y}(X)-F_{Y}(X_{-w})\\\
&\textrm{if}F(X_{-w})=Y\lor(F(X)=Y_{I}\land F(X_{-w})=Y_{B})\\\
&F_{Y}(X)-F_{Y}(X_{-w})+F\\_{\bar{Y}}(X_{-w})-F_{\overline{Y}}(X)\\\
&\textrm{if}F(X_{-w})\neq Y\\\ \end{split}$ (1)

where $F_{Y}$ is the prediction score for the correct label, $F_{\bar{Y}}$ is
the prediction score of the predicted label, $F$ is the predicted label, $Y$
is the correct label, $Y_{I}$ is the I-tag version of the correct label,
$Y_{B}$ is the B-tag version of the correct label and $X_{-w}$ is the sentence
X after deleting the word $w$.

##### Step 2: Gathering synonyms

For each word, we select synonyms from the Paragram-SL999 word vectors Mrkšić
et al. (2016) with a similarity to the original word above the threshold
$\delta$. Mrkšić et al. (2016) injected antonymy and synonymy constraints into
the vector space representation to specifically gear the embeddings space
towards synonymy. These embeddings achieved state-of-the-art performance on
SimLex-999 (Hill et al., 2015) and were also used by Jin et al. (2020) and
Alzantot et al. (2018). We chose 0.5 as the minimal similarity threshold
$\delta$ for synonym selection in contrast to the threshold of 0 used by
previous work in order to better guarantee semantic similarity. Regardless of
$\delta$, a maximum of 50 synonyms are selected. Examples of word pairs above
a $\delta$ of 0.5 are ‘bought’ and ‘obtained’; and ‘cat’ and ‘puss’. Below
this threshold but within the first 50 synonyms fall ‘bought’ and ‘forfeited’;
and ‘cat’ and ‘dustpan’.

##### Step 3: Filtering synonyms

To preserve syntax, synonyms must have the same POS tag as the original word.
If the data did not include POS tags, we added POS tags using NLTK. We also
exclude synonyms that result in sentences falling below the similarity
threshold $\epsilon$.

##### Step 4: Selecting the final synonym

After filtering, we check whether any of the synonyms can change the entity
label(s) fully. If there are multiple options, we select the one that leads to
the highest sentence similarity ($\epsilon$) to the original sentence. If
there are none, we select the synonym which can reduce the (summed) prediction
scores of the correct label(s) the most. If no synonyms are left after
filtering or none manage to reduce the prediction scores, we do not replace
the original word.

For multi-word entities, it is possible that a synonym changes some, but not
all, labels. From the synonyms that change the most labels, we select the one
that leads to the largest reduction in the (summed) prediction scores for the
unchanged labels (i.e. the labels that are still predicted correctly by the
model) (see Equation 1). Which labels are still correct can differ per
synonym.

##### Finalizing the adversarial examples

For each word in this ranking, we go through step 2-4 until either the
label(s) of the entity have been changed fully or there are no words left to
perturb. Once the attack is partially successful, only the predictions of the
not yet incorrectly labelled words in the entity are considered for subsequent
iterations.

##  4 Experiments

###  4.1 Data

We use two general-domain NER data sets for evaluating our method: the English
CoNLL-2003 data (Tjong Kim Sang and de Meulder, 2003) and the W-NUT 2017 data
(Derczynski et al., 2017). The goal of the latter was to investigate
recognition of unusual, previously-unseen entities in the context of online
discussions. Additionally, we use two data sets from the biomedical domain:
BC5CDR (Li et al., 2016) and the NCBI disease corpus (Doǧan et al., 2014).
Both data sets have been used to evaluate domain-specific BERT models for NER
in the biomedical domain (Peng et al., 2019; Beltagy et al., 2019; Lee et al.,
2019). See Table 2 for more details on the data sets.

Domain | Dataset |  Entity types | Dev. | Train | Test |  Subset for Eval.* (# of Entities)
---|---|---|---|---|---|---
General | CoNLL-2003 |  Person, Location, Organization, Miscellaneous | 3,466 | 14,987 | 3,684 |  500 (1,343)
General | W-NUT 2017 |  Person, Location, Corporation, Product, Creative-work, Group | 1,008 | 1,000 | 1,287 |  500 (787)
Biomedical | BC5CDR |  Disease, Chemical | 4,580 | 4,559 | 4,796 |  500 (1,221)
Biomedical | NCBI disease |  Disease | 922 | 5,432 | 939 |  487 (897)
Table 2: Size of the data sets (number of sentences). *This subset is used for
automatic evaluation

###  4.2 Target models

We fine-tune three BERT models (base-cased) for each data set with different
initialization seeds (1, 2 & 4) using the Huggingface implementation (Wolf et
al., 2019). We set the learning rate at $5\times 10^{-5}$ and optimized the
number of epochs using the range (3,4) as described in Devlin et al. (2019)
for NER. We select the number of epochs based on the first BERT model
(seed=1). We find that for all data sets except W-NUT 2017, 4 epochs is
optimal.

For the biomedical data sets, we additionally fine-tune two domain-specific
BERT models BioBERT (base-cased) Lee et al. (2019) and SciBERT (scivocab-
cased) Beltagy et al. (2019). Each model is trained in three folds with three
different initialization seeds (1, 2 & 4).

###  4.3 Evaluation of the adversarial attacks

##### Automatic evaluation

We randomly select 500 eligible sentences from each test set. Table 2 shows
the number of entities in each subset. We considered sentences to be eligible
if they contain at least one entity and one verb. For the NCBI-disease data,
only 487 sentences fulfill these criteria.

We use models trained on the original training and development data to perform
NER on the selected subset of the test data. We then generate one adversarial
example for each entity in the sentence that was initially predicted
correctly. We evaluate to what degree models are fooled only for entities that
were predicted correctly in the original sentences. We set the semantic
similarity threshold at $\epsilon=0.8$ following Li et al. (2019).

##### Human evaluation

In order to evaluate the quality of our adversarial examples from the
CoNLL-2003 and BC5CDR data, 100 original sentences and 100 adversarial
sentences from each type of attack are scored for grammaticality by human
judges. Grammaticality is evaluated on a five-point scale following the
reading comprehension benchmark DUC2006 (Dang, 2006). Our annotators are four
linguists222We opted for linguists as they would be more acquainted with
assessing the grammaticality of a sentence than domain experts: two for each
data set with 20% overlap. We choose to present annotators with different
original sentences than the ones on which the adversarial sentences they
evaluate are based to prevent bias.

##  5 Results

###  5.1 Entity Attack

| CoNLL-2003 | W-NUT 2017 | NCBI-disease | BC5CDR
---|---|---|---|---
| BERT | BERT | BERT | BERT
Success rate (%) | 97.5 $\pm$ 0.037 | 89.5 $\pm$ 0.886 | 99.2 $\pm$ 0.114 | 99.4 $\pm$ 0.073
Of which: |  |  |  |
– Missed entity(%) | 21.3 $\pm$ 12.3 | 71.4 $\pm$ 1.9 | 100 | 86.1 $\pm$ 0.5
– Entity type error(%) | 78.8 $\pm$ 12.3 | 28.6 $\pm$ 1.9 | 0 | 13.9 $\pm$ 0.5
Median semantic similarity | 0.959 $\pm$ 0.001 | 0.928 $\pm$ 0.003 | 0.952 $\pm$ 0.001 | 0.962 $\pm$ 0.000
Table 3: Automatic evaluation results for entity attacks on BERT models. Results are the mean of three models. | NCBI-disease | BC5CDR
---|---|---
| BioBERT | SciBERT | BioBERT | SciBERT
Success rate (%) | 99.2 $\pm$ 0.259 | 99.4 $\pm$ 0.054 | 99.4 $\pm$ 0.070 | 99.3 $\pm$ 0.089
Of which: |  |  |  |
– Missed entity(%) | 100 | 100 | 94.0 $\pm$ 0.7 | 91.7 $\pm$ 1.5
– Entity type error(%) | 0 | 0 | 6.0 $\pm$ 0.7 | 8.3 $\pm$ 1.5
Median semantic similarity | 0.955 $\pm$ 0.001 | 0.953 $\pm$ 0.002 | 0.961 $\pm$ 0.001 | 0.962 $\pm$ 0.001
Table 4: Automatic evaluation results for entity attacks on biomedical BERT
models. Results are the mean of three models.

The main results of adversarial entity attack on BERT models are presented in
Table 3. BERT models appear highly vulnerable to adversarial attacks on the
entities themselves despite the high similarity between adversarial and
original sentences. On average, BERT models are fooled for 97.5% of entities
that were initially predicted correctly on the CoNLL data and 89.2% on W-NUT
data. BERT models appear even more vulnerable to entity attacks on domain-
specific data with success rates above 99%.

Table 4 shows that domain-specific BERT models do not resolve this issue. They
are also highly vulnerable with over 99% of all initially correctly predicted
entities now predicted incorrectly. The high success rates of entity attacks
both on general domain and domain-specific data suggests that BERT models,
similar to traditional models, are unable to predict entities correctly based
solely on the context of the entity. Replacing the entity word with another of
the same entity type can easily fool the model. This suggests a strong
dependency on the entities that the model has seen previously, making these
models vulnerable to new or emergent entities.

This is corroborated by an analysis of which entities were chosen in
successful attacks. For all BERT models with the exception of the CoNLL data,
these entities are significantly less frequent in training and development
data than the original entities.

A possible explanation for why BERT models for CoNLL are the exception is that
there is a stronger match between the pretraining data and the data at hand
than for the other data sets. This may make the model less vulnerable to
infrequent entities, despite not being less vulnerable to entity replacement
overall. Manual inspection further revealed that BERT models appear to be
sensitive to the capitalization of entities (e.g. BERT models trained on CoNLL
were fooled by transforming ‘New York’ to ‘NEW YORK’).

###  5.2 Entity Context Attack

| CoNLL-2003 | W-NUT 2017 | NCBI-disease | BC5CDR
---|---|---|---|---
| BERT | BERT | BERT | BERT
Success rate (%) | 36.3 $\pm$ 0.612 | 42.2 $\pm$ 0.677 | 20.2 $\pm$ 0.443 | 38.8 $\pm$ 0.862
Of which: |  |  |  |
– Missed entity (%) | 47.4 $\pm$ 2.9 | 61.3 $\pm$ 5.1 | 100 | 90.4 $\pm$ 4.2
– Entity type error (%) | 52.6 $\pm$ 2.9 | 38.7 $\pm$ 5.1 | 0 | 9.6 $\pm$ 4.2
Partial success rate (%) | 51.0 $\pm$ 0.465 | 51.6 $\pm$ 1.6 | 29.3 $\pm$ 0.841 | 45.9 $\pm$ 1.1
Median semantic similarity | 0.928 $\pm$ 0.009 | 0.926 $\pm$ 0.017 | 0.920 $\pm$ 0.040 | 0.946 $\pm$ 0.002
Words perturbed (%) | 15.6 $\pm$ 0.306 | 13.2 $\pm$ 1.2 | 12.4 $\pm$ 1.0 | 12.3 $\pm$ 0.04
Table 5: Automatic evaluation results for the context attacks on BERT models. Results are the mean of the three models | NCBI-disease | BC5CDR
---|---|---
| BioBERT | SciBERT | BioBERT | SciBERT
Success rate (%) | 20.9 $\pm$ 0.762 | 26.4 $\pm$ 0.875 | 37.9 $\pm$ 0.388 | 45.0 $\pm$ 0.665
Of which: |  |  |  |
– Missed entity (%) | 100 | 100 | 86.5 $\pm$ 2.3 | 87.1 $\pm$ 2.3
– Entity type error (%) | 0 | 0 | 13.5 $\pm$ 2.3 | 12.9 $\pm$ 2.3
Partial success rate (%) | 30.1 $\pm$ 1.032 | 39.0 $\pm$ 0.954 | 44.8 $\pm$ 0.331 | 53.3 $\pm$ 0.821
Median semantic similarity | 0.921 $\pm$ 0.031 | 0.936 $\pm$ 0.030 | 0.921 $\pm$ 0.003 | 0.936 $\pm$ 0.008
Words perturbed (%) | 9.1 $\pm$ 2.5 | 8.7 $\pm$ 2.9 | 9.8 $\pm$ 0.3 | 8.5 $\pm$ 0.8
Table 6: Automatic evaluation results for the context attacks on biomedical
BERT models. Results are the mean of the three models

Attacks on the context of entities are less successful at completely fooling
BERT models than entity attacks (see Table 5). Nonetheless, BERT models can be
fooled into predicting entities partially or fully wrong (Partial + full
success rate) for 87.3% and 93.8% of entities for CoNLL and W-NUT
respectively. Moreover, for over 75% of the cases the BERT models were fooled
by a single change.

SciBERT appears more vulnerable than BERT, both to completely being fooled
(+6.2 and +6.2% point) and being fooled partially (+9.7 and +7.4 % point) by
context attack. Also the domain-specific models were often fooled by only one
word being replaced with its synonym; BioBERT was fooled by a single change 65
and 75% of the time whereas SciBERT was fooled by a single change 68 and 76%
of the time.

We analyzed the sentence statistics for successful and failed attacks.
Specifically for BERT models, we see that the following cases are more
vulnerable to attacks: longer sentences; sentences with more words that could
be replaced by synonyms; and shorter entities. Manual analysis of successful
attacks reveals that BERT models are vulnerable to rare synonyms replacing
more common words (e.g. ‘salubrious’ instead of ‘healthy’).

(a) BERT

(b) BioBERT and SciBERT

Figure 1: Distance of successful synonym replacements relative to the entity
(at 0)

Figure 1(a) shows where in the sentence changes have occurred in order to fool
BERT. BERT models seem most vulnerable to changes in the local context of
entities: only 1-2 words left or right of the entity. Manual analysis revealed
that these words are often verbs. Although less influential, long distance
context does appear to be used for predicting entities in some cases. We
manually inspected sentences with long distance changes (>20 words). Lists
stood out as a prime example of a sentence type for which long distance
context is important. For the BioBERT model, the distribution is strikingly
similar to that of the original BERT model (see Figure 1(b)). This is likely
due to either the vocabulary or the training data333BioBERT includes all the
original BERT training data as well as additional domain-specific data that
these models share. SciBERT models which share neither training data nor
vocabulary with the original BERT model are even more vulnerable to changes in
the local context of the entity (see Figure 1(b)).

###  5.3 Evaluating the necessity of importance ranking

To investigate the effect of adding the word importance ranking to the entity
context attack we perform an ablation study on the CoNLL-2003 test set. As can
be seen in Table 7, removing the word importance ranking leads to a stark drop
in both the average full success of adversarial attack (from 37.3% to 9.5%)
and the average partial success rate (from 52.8% to 20.1%). The number of
words that need to be perturbed also drops, by 6.9% point, meaning that
attacks need less changes on average to be successful. Thus, it appears that
the word importance ranking is crucial to the success of the adversarial
attack algorithm.

| Importance ranking
---|---
|  Yes | No
Success rate (%) |  37.3 $\pm$ 0.515 | 9.5 $\pm$ 4.3
Partial success rate (%) |  52.8 $\pm$ 0.356 | 20.1 $\pm$ 9.2
Semantic similarity |  0.922 $\pm$ 0.006 | 0.983 $\pm$ 0.006
Words perturbed (%) |  13.8 $\pm$ 0.238 | 6.9 $\pm$ 2.2
Table 7: Comparison of the effect of context attacks with and without
importance ranking on CoNLL-2003 data

###  5.4 Results of human evaluation

On CoNLL-2003, the annotators have a fair inter-annotator agreement (weighted
$\kappa=0.353$). On BC5CDR, the inter-annotator agreement is slight (weighted
$\kappa=0.177$). Investigation of the annotations reveals that this is most
likely because biomedical sentences are more difficult to assess for laymen.
Because of the limited agreement we report grammaticality assessments per
annotator.

Table 8 show that although entity attacks do not significantly alter the
grammaticality of the sentences, attacks on the context of the entity do.
Although this reduction is consistent across data sets, the mean
grammaticality of the adversarial sentences remains above 3 (acceptable) and
the mean absolute reduction is less than a full point.

| CoNLL | BC5CDR
---|---|---
Annotator | 1 | 2 | 3 | 4
Original | 3.51 | 4.34 | 4.43 | 4.78
After context attack | 3.05* | 3.68** | 3.86** | 4.35**
After entity attack | 3.30 | 4.37 | 3.85 | 4.67
Table 8: Mean grammaticality of the original and adversarial sentences.
*$p<0.05$ **$p<0.01$ compared to the original sentences according to Mann-
Whitney U tests.

##  6 Discussion

###  6.1 Error analysis

We manually analysed the generated adversarial examples and found that our
adversarial examples are susceptible to word sense ambiguity. For example, the
top 50 synonyms for ‘surfed’ in ‘surfed the Internet’ contains both correct
synonyms like ‘googled’ and incorrect ones like ‘paddled’. There are also some
cases where the adversarial examples suffer from words that occur in multiple
languages e.g. ‘vie’ in ‘to vie for top UN post’ is replaced with ‘existence’
which is a synonym of the French ‘vie’ (i.e. life). Moreover, occasionally
synonyms are not English words (e.g. ‘number’ to ‘nombre’). These issues are
caused by foreign words influencing or being included in the Paragram-SL999
word vectors Mrkšić et al. (2016).

Furthermore, our adversarial examples are susceptible to grammatical errors.
Grammatically poor adversarial sentences often suffer from changes from verbs
to nouns or vice versa that are not caught by the POS-filter (e.g. ‘open’ to
‘openness’ and ‘influence’ to ‘implication’). These cases may be particularly
difficult as ‘open’ and ‘influence’ can be both a verb and an adjective or
noun. Another common error is singular-plural inconsistencies (e.g. ‘one
dossiers’). To mitigate these issues, future work could focus on removing non-
English words from the embedding space, and altering how the POS-tag of the
synonym is determined.

We also looked specifically at the adversarial examples generated using
domain-specific data. Here we see that synonyms are not always appropriate for
the context (e.g. ‘worst affected’ is changed to ‘shittiest affected’). A
domain-specific embeddings model may partly alleviate this problem.

###  6.2 Interpreting vulnerabilities

Analysis of which perturbations are able to fool the model might give valuable
insight into its vulnerabilities and robustness to real-world data. However,
there are some caveats to keep in mind when interpreting weaknesses based on
successful attacks. The architecture of self-attentive models means that the
attention weight of a word is context-dependent. Thus, if changing that word
fools the model, this might only be true in that context. Additionally, if
multiple words were changed for a successful attack, their interactions may
contribute to the success and it cannot simply be interpreted as caused by
this combination of words.

##  7 Conclusions

We studied the vulnerability of BERT models in NER tasks under a black-box
setting. Our experiments show that BERT models are highly vulnerable to
entities being replaced by less frequent entities of the same type. They can
also be fooled by changes in single context words being replaced by their
synonyms.

Our analysis of BERT’s vulnerabilities can inform fruitful directions for
future research. Firstly, our results reveal that rare or emergent entities
remain a problem for both generic and domain-specific NER models.
Consequently, we recommend further research into zero or few-shot learning.
Moreover, the masking of entities during fine-tuning may be an interesting
avenue for research. Secondly, BERT models also appear vulnerable to words it
has not seen or rarely seen during training in the entity context. To combat
this vulnerability, the use of adversarial examples designed specifically to
include more infrequently used words could be explored. Another possible
avenue for research could be alternative pre-training schemes for BERT such as
curriculum learning Elman (1993). Thirdly, we find that SciBERT is more
vulnerable to changes in the entity context than BioBERT or BERT. This may be
due to the domain-specific biomedical vocabulary that SciBERT employs, which
could make it more vulnerable to out-of-entity words being replaced by more
common English terms. This trade-off between robustness and domain-specificity
of BERT models may be another worthwhile research direction.

We consider our work to be a first step towards understanding to what extent
BERT models for NER are vulnerable to token-level changes and to which changes
they are most vulnerable. We hope others will build on our work to further our
insight into self-attentive models and to mitigate these vulnerabilities.

## References

  * Alshemali and Kalita (2019) Basemah Alshemali and Jugal Kalita. 2019.  Improving the reliability of deep neural networks in NLP: A review.  _Knowledge-Based Systems_ , 10520.
  * Alzantot et al. (2018) Moustafa Alzantot, Yash Sharma, Ahmed Elgohary, Bo-Jhang Ho, Mani B Srivastava, and Kai-Wei Chang. 2018.  Generating Natural Language Adversarial Examples.  In _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing_ , pages 2890–2896. Association for Computational Linguistics.
  * Araujo et al. (2020) Vladimir Araujo, Andrés Carvallo, and Denis Parra. 2020.  Adversarial evaluation of bert for biomedical named entity recognition.  In _Proceedings of the The Fourth Widening Natural Language Processing Workshop_ , pages 79–82.
  * Balasubramanian et al. (2020) Sriram Balasubramanian, Naman Jain, Gaurav Jindal, Abhijeet Awasthi, and Sunita Sarawagi. 2020.  What’s in a Name? Are BERT Named Entity Representations just as Good for any other Name? In _Proceedings of the 5th Workshop on Representation Learning for NLP_ , pages 205–214, Stroudsburg, PA, USA. Association for Computational Linguistics.
  * Beltagy et al. (2019) Iz Beltagy, Kyle Lo, and Arman Cohan. 2019.  SciBERT: A pretrained language model for scientific text.  In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_ , pages 3615–3620, Hong Kong, China. Association for Computational Linguistics.
  * Cer et al. (2018) Daniel Cer, Yinfei Yang, Sheng-yi Kong, Nan Hua, Nicole Limtiaco, Rhomni St. John, Noah Constant, Mario Guajardo-Cespedes, Steve Yuan, Chris Tar, Brian Strope, and Ray Kurzweil. 2018.  Universal sentence encoder for English.  In _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing: System Demonstrations_ , pages 169–174, Brussels, Belgium. Association for Computational Linguistics.
  * Dang (2006) Hoa Trang Dang. 2006.  Overview of DUC 2006.  In _Proceedings of HLT-NAACL 2006_ , pages 1–12.
  * Derczynski et al. (2017) Leon Derczynski, Eric Nichols, and Marieke van Erp. 2017.  Results of the WNUT2017 Shared Task on Novel and Emerging Entity Recognition.  In _Proceedings ofthe 3rd Workshop on Noisy User-generated Text_ , pages 140–147. Association for Computational Linguistics.
  * Devlin et al. (2019) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019.  BERT: Pre-training of deep bidirectional transformers for language understanding.  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_ , pages 4171–4186, Minneapolis, Minnesota. Association for Computational Linguistics.
  * Doǧan et al. (2014) Rezarta Islamaj Doǧan, Robert Leaman, and Zhiyong Lu. 2014.  NCBI disease corpus: A resource for disease name recognition and concept normalization.  _Journal of Biomedical Informatics_ , 47:1–10.
  * Elman (1993) Jeffrey L Elman. 1993.  Learning and development in neural networks: The importance of starting small.  _Cognition_ , 48(1):71–99.
  * Hill et al. (2015) Felix Hill, Roi Reichart, and Anna Korhonen. 2015.  SimLex-999: Evaluating Semantic Models With (Genuine) Similarity Estimation.  _Computational Linguistics_ , 41(4):665–695.
  * Hsieh et al. (2019) Yu-Lun Hsieh, Minhao Cheng, Da-Cheng Juan, Wei Wei, Wen-Lian Hsu, and Cho-Jui Hsieh. 2019.  On the Robustness of Self-Attentive Models.  In _Proceedings of the 57th Annual Meeting ofthe Association for Computational Linguistics_ , pages 1520–1529. Association for Computational Linguistics.
  * Jin et al. (2020) Di Jin, Zhijing Jin, Joey A Tianyi Zhou, and Peter Szolovits. 2020.  Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailment.  In _AAAI_ , page Accepted for publication.
  * Lee et al. (2019) Jinhyuk Lee, Wonjin Yoon, Sungdong Kim, Donghyeon Kim, Sunkyu Kim, Chan Ho So, and Jaewoo Kang. 2019.  BioBERT: a pre-trained biomedical language representation model for biomedical text mining.  _Bioinformatics_.  Btz682.
  * Li et al. (2016) Jiao Li, Yueping Sun, Robin J Johnson, Daniela Sciaky, Chih-Hsuan Wei, Robert Leaman, Allan Peter Davis, Carolyn J Mattingly, Thomas C Wiegers, and Zhiyong Lu. 2016.  BioCreative V CDR task corpus: a resource for chemical disease relation extraction.  _Database_ , 2016:68.
  * Li et al. (2019) Jinfeng Li, Shouling Ji, Tianyu Du, Bo Li, and Ting Wang. 2019.  TEXTBUGGER: Generating Adversarial Text Against Real-world Applications.  In _Network and Distributed Systems Security (NDSS) Symposium 2019_.
  * Li et al. (2020) Linyang Li, Ruotian Ma, Qipeng Guo, Xiangyang Xue, and Xipeng Qiu. 2020.  Bert-attack: Adversarial attack against BERT using BERT.  _arXiv_.
  * Mondal (2021) Ishani Mondal. 2021.  BBAEG: Towards BERT-based biomedical adversarial example generation for text classification.  In _Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 5378–5384, Online. Association for Computational Linguistics.
  * Mrkšić et al. (2016) Nikola Mrkšić, Diarmuid Ó Séaghdha, Blaise Thomson, Milica Gašíc, Lina Rojas-Barahona, Pei-Hao Su, David Vandyke, Tsung-Hsien Wen, and Steve Young. 2016.  Counter-fitting Word Vectors to Linguistic Constraints.  In _Proceedings of NAACL-HLT 2016_ , pages 142–148.
  * Niven and Kao (2019) Timothy Niven and Hung-Yu Kao. 2019.  Probing neural network comprehension of natural language arguments.  In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_ , pages 4658–4664, Florence, Italy. Association for Computational Linguistics.
  * Peng et al. (2019) Yifan Peng, Shankai Yan, and Zhiyong Lu. 2019.  Transfer learning in biomedical natural language processing: An evaluation of bert and elmo on ten benchmarking datasets.  In _Proceedings of the 2019 Workshop on Biomedical Natural Language Processing (BioNLP 2019)_.
  * Sun et al. (2020) Lichao Sun, Kazuma Hashimoto, Wenpeng Yin, Akari Asai, Jia Li, Philip Yu, and Caiming Xiong. 2020.  Adv-BERT: BERT is not robust on misspellings! Generating nature adversarial samples on BERT.  _arXiv_.
  * Szegedy et al. (2014) Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. 2014.  Intriguing properties of neural networks.  In _International Conference on Learning Representations_.
  * Tjong Kim Sang and de Meulder (2003) Erik Tjong Kim Sang and Fien de Meulder. 2003.  Introduction to the CoNLL-2003 Shared Task: Language-Independent Named Entity Recognition.  In _Proceedings of the Seventh Conference on Natural Language Learning at HLT-NAACL 2003_ , pages 142–147.
  * Wolf et al. (2019) Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, and Jamie Brew. 2019.  Transformers: State-of-the-art Natural Language Processing.  _ArXiv_.
  * Zang et al. (2020) Yuan Zang, Bairu Hou, Fanchao Qi, Zhiyuan Liu, Xiaojun Meng, and Maosong Sun. 2020\.  Learning to attack: Towards textual adversarial attacking in real-world situations.  _arXiv_.