# Global Regular Network for Writer Identification

Shiyu Wang  Xi’an Jiaotong University
Xi’an, China
Email<EMAIL_ADDRESS>
###### Abstract

Writer identification has practical applications for forgery detection and
forensic science. Most models based on deep neural networks extract features
from character image or sub-regions in character image, which ignoring
features contained in page-region image. Our proposed global regular network
(GRN) pays attention to these features. GRN network consists of two branches:
one branch takes page handwriting as input to extract global features, and the
other takes word handwriting as input to extract local features. Global
features and local features merge in a global residual way to form overall
features of the handwriting. The proposed GRN has two attributions: one is
adding a branch to extract features contained in page; the other is using
residual attention network to extract local feature. Experiments demonstrate
the effectiveness of both strategies. On CVL dataset, our models achieve
impressive 99.98% top-1 accuracy and 100% top-5 accuracy with shorter training
time and fewer network parameters, which exceeded the state-of-the-art
structure. The experiment shows the powerful ability of the network in the
field of writer identification. The source code is available at
https://github.com/wangshiyu001/GRN.

##  I Introduction

Just like language, fingerprints, iris and human faces, handwriting is an
important feature of person. Writer identification based on handwriting plays
a significant role in finance, insurance, public security departments,
criminal investigations, court trials and other fields. However, handwriting
is different from other biological characteristics. It mainly includes: (1)
The handwriting of people at different ages will change; (2) At different time
periods and locations, the writer will write changeable handwriting. How to
recognize people’s identity in changing handwriting is a huge challenge in the
field of computer vision. The early method of writer identification based on
feature point extraction. Many feature extraction algorithms are applied in
this field[1, 2, 3, 4, 5, 6]. In recent years, Convolutional Neural Networks
have achieved great success and were quickly applied in this field[7, 8, 9,
10, 11, 12]. However, most of the handwriting identification systems based on
convolutional neural networks pay attention to local features extraction. But
in fact, the characteristics of handwriting not only include local
characteristics such as the thickness of the strokes and the degree of
curvature, but also many global characteristics. In most cases, human
handwriting is not written in a grid, but on empty white paper. The overall
characteristics such as the arrangement of characters, and the arrangement of
lines also affect the writer’s inherent writing habits. For example, we often
say that a person’s writing is neat or scribbled, in which we focus on the
global features of the handwriting rather than the local features. Therefore,
the recognition of handwriting not only needs to pay attention to the local
features, but also the overall features.

The global regular network (GRN) pays attention to the global characteristics
of the handwriting. The input of the network consist of two parts: the page
handwriting and the word handwriting from the same writer: the page
handwriting at the global level, which contains numerous global features; the
word handwriting at the local level, which contains numerous local features.
Our network extracts these two handwriting features in two branches. One is
the global feature extraction network, which takes the page handwriting as
input to extract the global features; the other is the local feature
extraction network, which takes the word handwriting as the input to extract
the local features. Finally, the global feature works on the local features in
the way of parameter weights to form the overall feature of the handwriting.
Our network specifically uses one network to extract the global features and
combines them with local features to extract more abundant handwriting
features.

Besides a branch to extract global features, residual attention network
proposed by Wang et al.[13] is used to extract the local features of word
handwriting. Features of word handwriting were always contained in special
places like the corner of strokes and the attention mechanism[14, 15, 16, 17]
used by residual attention network will help to extract these features.
Sec.IV-C shows improved performance of GRN with residual attention network.

The network structure has two novel highlights: On the one hand, different
from the traditional focus on local feature extraction of handwriting, this
network has adds a global feature extraction network to extract the global
features of the handwriting and combine it with the local features to
characterize the features of the handwriting. With this method, GRN can
extract handwriting features at multiple levels. On the other hand, our
network uses the residual attention network to extract the local features of
the handwriting, which more improves the ability of network.

##  II Related work

In today’s society, writer identification is playing an increasingly important
role in the fields of finance, insurance, public security departments,
criminal investigations, and court trials[18, 19, 20, 21]. According to the
type of handwriting, writer identification can be divided into text-related
writer identification and text-independent writer identification; according to
approaches used for writer identification, writer identification can be
divided into traditional handcrafted approaches and deep learning approaches.

Text-related writer identification is that the contents of the handwriting to
be recognized are the same. The typical text-related writer identification is
signature verification. Many writer identification work based on this[18, 19,
21, 13]. Text-independent writer identification is that the content of the
handwriting to be recognized is different. Since text-independent writer
identification does not rely on text, it has wider applicability than text-
related writer identification, and it is also more difficult to recognize. The
global regular network(GRN) proposed in this paper is text-independent writer
identification.

Handcrafted approaches for writer identification were based on feature
extraction. Many feature extraction algorithms, such as Gabor filter[1], Local
binary patterns (LBP)[5, 6], and SIFT[2, 3, 4] are applied to feature
extraction. Then various classifiers are used to classify, such as distance-
based classifier, Support Vector Machines (SVM)[22], Hidden Markov Model
(HMM)[23], Fuzzy based classifier[24] and so on. These methods have laid a
solid foundation for writer identification. Handcrafted approaches have
achieved good results on some datasets, but it is difficult to apply to a
wider range of handwriting identification. In order to overcome the
difficulties faced by feature point extraction algorithms, convolutional
neural networks have been applied to writer identification[7, 8, 9, 10, 11,
12]. One of the first attempts at writer identification with CNN was proposed
by Fiel and Sablatnig[7] in 2015. They used CNN to extract features and the
feature vectors output from CNN will be compared with precalculated feature
vectors of the dataset by nearest neighbor classification. Christlein et
al.[8] proposed a similar network. The different place is that they encoded
the feature vectors with Gaussian mixture models (GMM) super vector encoding.
Above these, Xing et al.[10] proposed a method based on pixel scanning
strategy, data augmentation and combined multi-stream parallel CNN. On the
standard dataset of HWDB[25], obtaining 97.0% recognition rate using small
amount of handwriting. However, these networks only use single character or
smaller pixel blocks as input to extract the local features, ignoring the
handwriting features contained in the larger handwriting scale and failing to
extract more handwriting features.

The attention mechanism[14, 15, 16, 17] is an effective way to capture stroke
information. Chen et al.[26] used the attention reversal mechanism to capture
prominent objects. Inspired by the attention mechanism, Wang et al.[13]
proposed the residual attention network. This kind of network has the
following appealing properties: it can incorporate with state-of-art feed
forward network architecture in an end-to-end training fashion; attention
modules can extract features in different levels. In experiment, this method
achieves 0.6% top-1 accuracy improvement with 46% trunk depth and 69% forward
Flops comparing to ResNet-200. In our work, residual attention network is used
for local feature extraction.

##  III Global Regular Network

Different people have different handwriting, the characteristics of these
handwriting are not only contained in the local characteristics of the
handwriting, such as the thickness, length, inclination degree, degree of
curvature of each stroke, and distribution of strokes, but also in the global
characteristics of the handwriting, such as the distribution of words, the
arrangement of the rows and columns in the handwriting. These global features
of handwriting contain a wealth of information. The traditional handwriting
feature extraction method based on the handwriting pixel block can often only
capture the local features of the handwriting, but cannot effectively capture
the global information, which is a great waste of dataset resources.

The global regular network(GRN) proposed in this paper captures both the
global and local characteristics of handwriting, realizing the adequate
exploitation of handwriting.

Figure 1: Illustration of Global Regular Network Figure 2: The overall
architecture of Global Regular Network

We illustrate our thoughts with Fig.1. The network extracts the handwriting
features in two branches, one extracts the global features of the handwriting,
another one extracts the local features of the handwriting, and then combines
the two features as the general features of the handwriting. For input images,
We believe that to extract the global characteristics of the handwriting, the
input data must be the page scale of the handwriting including multiple words,
so that the handwriting will contain rich global features; and to extract the
local features of the handwriting, the single word handwriting meet the
requirement, the reason is: too large region of handwriting will import global
features (such as the distribution of different words), which will become the
noise of local feature extraction; too small region of handwriting such as a
single stroke will lose many local features (such as continuous writing in
word handwriting), so single word handwriting is the ideal region for local
feature extraction. For the merge of local features and global features,
suppose page handwriting and word handwriting are x and y, respectively. We
get global handwriting feature G(x) after page handwriting going through the
global feature extract network and local handwriting feature L(y) after word
handwriting going through the local feature extract network. The general
feature T(x, y) is obtained like below:

$T(x,y)=(1+G(x))\times L(y)$ (1)

In this formula, the global features of the handwriting act as weight
parameters to adjust the local features. We use the combination of local
features and global features in Equ.1 instead of the combination like
$T(x,y)=G(x)\times L(y)$, because we believe that the local features of the
handwriting contain the main handwriting information, and should be taken as
the main body to characterize the handwriting features while global features
just are taken as parameters to adjust the local handwriting features, playing
a role in the global regularization of the local features, which is also the
origin of our network name. In addition, in Equ.1, even if the global feature
is 0, the network will still work correctly, and the network result after
adding global feature extraction network will only be better. In Sec.IV-C, we
show the effects of different combinations of global and local features on the
experimental results.

###  III-A Architecture

In Fig.2, the input handwriting consist of the above page handwriting and
below word handwriting. The global regular network consists of the global
feature extraction network above and the local feature extraction network
below. Christlein et al.[8] verify that resnet[27] networks can extract
handwriting better than other networks. Considering GPU occupancy, parameters
number, and training efficiency, we chose resnet34 as the overall architecture
of the global feature extraction network which include: a convolution layer,
initializing the convolution kernel to 64, a $3\times 3$ maximum pooling
layer, and 4 residual modules. The number of residual units in each residual
module is 3, 4, 6, 3, respectively. The final output features merge with local
feature, forming general feature. Compared with the classic resnet34 model,
the global feature extraction network has been revised in two parts: the
convolutional layer convolutional kernel size of the input image increased
from 3 to 7, because we believe that the larger convolutional kernel will be
more conducive to pay attention to the global characteristics of the
handwriting; the second change is to insert the maximum pooling layer after
the first convolutional layer, in order to match global feature with local
feature.

Residual attention network proposed by Wang et al.[13] is used to extract
local features. As shown in the below branch of Fig.2, residual attention
network includes: a convolutional layer to initialize the convolution kernel
to 64, a $3\times 3$ maximum pooling layer, and a series of residual attention
blocks and residual units. Details of residual attention network have been
listed in Table.I. We also can see the structure of residual attention blocks:
the trunk branch containing two residual units and another soft mask branch
using bottom-up top-down structure[28, 29, 30]. The outputs of trunk branch
and soft mask branch merge with a residual way. Detail of residual attention
blocks are showed in Sec.III-B.

Four global residual connections were used between global feature extraction
network and local feature extraction network. The purpose of these connections
is to make the global features adequately work on the local features. Finally,
the general handwriting features flow into the average pooling layer obtaining
$1\times 1$ handwriting features; due to the small dataset, we add a dropout
layer to the network to reduce the over-fitting. Full connected layer is
connected to the end. Detail of overall architecture is showed in Table.I.

The entire network is trained in end-to-end fashion. Two mechanisms are used
for writer identification. First, two branches are used to extract richer
characterizes in handwriting. The global features of the handwriting act as
weight parameters to adjust the local features, playing a role in the global
regularization of the local features. The second is that residual attention
module is used to extract the local features of the handwriting.

###  III-B Residual Attention Block

Figure 3: The architecture of Residual Attention Block

Residual attention blocks are used to extract local features. The
architectures are showed in 3: the branch above is trunk branch, which is the
main path of network; another below is soft mask branch. Soft mask branch uses
bottom-up top-down structure[28, 29, 30]. Input features go through three
maximum pooling layers and reach minimum resolution; Then bilinear
interpolation layers with the same number of maximum pooling layers operate on
the intermediate features. The size of output features is same with the input
features. In order to extract more information contained in input with
different size, two another branches were added in where the input size reduce
two times and four times. Then the output go through two convolutional layers
and a sigmoid normalization layer which scales the value of output between(0,
1). Finally, similar to the combination of global features and local features,
the features of two branches also carry out the same strategy: Suppose the
input feature is x, the output feature of trunk branch is T(X), the output of
soft mask branch is M(X), and the final output is:$H(X)=(1+M(X))\times T(X)$.

TABLE I: GLOBAL REGULAR NETWORK ARCHITECTURE DETAILS Layer-Res | Layer—Att | Resnet | Attention | Output Size
---|---|---|---|---
Conv1 |  $7\times 7$,64,stride 2 | $128\times 128$
Max pooling |  $3\times 3$,stride 2 | $64\times 64$
Residual Unit | Residual Unit | $\left(\begin{array}[]{c}1\times 1,64\\\ 3\times 3,64\\\ \end{array}\right)\times 3$ | $\left(\begin{array}[]{c}1\times 1,64\\\ 3\times 3,64\\\ \end{array}\right)\times 1$ | $64\times 64$
Attention Module | $Attention\times 1$
Residual Unit | Residual Unit | $\left(\begin{array}[]{c}1\times 1,128\\\ 3\times 3,128\\\ \end{array}\right)\times 4$ | $\left(\begin{array}[]{c}1\times 1,128\\\ 3\times 3,128\\\ \end{array}\right)\times 1$ | $32\times 32$
Attention Module | $Attention\times 1$
Residual Unit | Residual Unit | $\left(\begin{array}[]{c}1\times 1,256\\\ 3\times 3,256\\\ \end{array}\right)\times 6$ | $\left(\begin{array}[]{c}1\times 1,256\\\ 3\times 3,256\\\ \end{array}\right)\times 1$ | $16\times 16$
Attention Module | $Attention\times 1$
Residual Unit | Residual Unit | $\left(\begin{array}[]{c}1\times 1,512\\\ 3\times 3,512\\\ \end{array}\right)\times 3$ | $\left(\begin{array}[]{c}1\times 1,512\\\ 3\times 3,512\\\ \end{array}\right)\times 3$ | $8\times 8$
Average pooling |  $8\times 8$,stride 1 | $1\times 1$
Dropout,FC | 1000

##  IV Result

In this chapter, we introduce our experiment in three parts. The first part
introduces the dataset used in training, data preprocessing and the settings
used in training. In the second part, we show the results of our network,
which exceeds most state-of-the-art writer identification structures. In the
third part, we show the effectiveness of the two highlights in this article:
the effectiveness of using global regularization strategies and the
effectiveness of using residual attention network.

###  IV-A Data Preprocessing and Training Settings

The dataset of the CVL[31] contains 27 writers in training dataset and 283
writers in test dataset. Every writer contributes 7 writing materials and 5
writing materials in the training dataset and test dataset respectively.
Writers contribute a German material, and the rest are English materials. Each
piece of material consists of three parts: page handwriting, line handwriting
segmented from page handwriting, and word handwriting segmented from page
handwriting.

We preprocess the dataset. We merge the original training dataset and test
dataset to form a new dataset. It contains 310 writers, 1550 page handwriting,
and 96180 word handwriting. Since the page handwriting are not enough, we do
data augmentation on this part. We randomly divide one page handwriting into 9
smaller square handwriting as new page handwriting to increase the number of
page handwriting and cut off the blank areas around the handwriting before
segmentation to avoid the import of noise. After that, the new page
handwriting is scaled down to $256\times 256$ size. For word handwriting,
because of its different sizes, we first fill the picture to square shape and
then reshape to $256\times 256$ size. During training, each batch consists of
one word handwriting and a random page handwriting from the same writer. It
should be noted that the word handwriting and the page handwriting are from
the same writer, but not necessarily from the same written material. Since
even the same writer, the handwriting written in different environments will
change, these pairs of input handwriting will force the network to capture the
same characteristics of the handwriting written in different environments. We
train the model based on pytorch1.7.1 platform with PTX-P2 GPU. The optimizer
uses Adam, the initial learning rate is 0.001, and half every 30 epochs.

###  IV-B Comparison With State of Art Structures

We compare the training results with several other structures. Fiel et al.[7]
firstly used CNN for writer identification. CNN was used to extract features,
and feature vectors output from CNN will be compared with precalculated
feature vectors of the dataset by nearest neighbor classification. Christlein
et al.[8] proposed a similar network. The different place is that they encoded
the feature vectors with Gaussian mixture models (GMM) super vector encoding.
WU et al.[32] uses an isotropic log filter to divide the handwriting into word
sizes, and then extracts the SIFT descriptor of the handwriting and then
matches the registered handwriting. DLS-CNN[33] first performs line
segmentation on the page handwriting, then cuts each line into small pixel
blocks of the same size, and uses resnet50 for writer identification at the
pixel block level. Table.II shows that our structure has achieved the best
performance with results of 99.98% top1 accuracy and 100% top5 accuracy. We
attribute it to the combination of global and local features. The previous
network focused on the extraction of local features, and our network added a
branch to extract global features, allowing the network to extract more
handwriting features. Compared with the DLS-CNN network, we used a smaller
model(resnet34) but achieved better results, which reflects the effectiveness
of our strategy. In the experimental results, the Top-1 of DLS-CNN are much
lower than other networks. We attribute it as the input image pixel block is
too small (such as the stroke level, smaller than the word size), such
handwriting will lose many word level information (distribution of strokes and
continuous strokes) which greatly reduce the training accuracy of the network.

TABLE II: COMPARISON OF GRN AND OTHER PROPOSED WRITER IDENTIFICATION STRUCTURES | Input materials | Top-1 | Top-5
---|---|---|---
Fiel[7] | one page | 98.9 | 99.3
Wu[32] | one page | 99.2 | 99.5
Christlein[8] | one page | 99.4 | N/A
Tang[9] | one page | 99.7 | 99.8
DLS-CNN[33] | 256 pixel | 95.8 | 99.9
Ours | page+word | 99.92 | 100

###  IV-C Comparison With Different Structures

Figure 4: The structure of Net1

We have designed two structures of the network to compare with our global
regular network. One structure is shown in Fig.4. We mark it Net-1. The global
feature extraction network and the local feature network in this structure are
not connected. The page handwriting go through the above global feature
extraction network, the output calculate the cross entropy loss with
corresponding ground truth, we mark it Loss-G; similarly, word handwriting go
through below local feature extraction network, the output calculate the cross
entropy loss with corresponding ground truth, we mark it Loss-L. The total
loss is a convex combination of Loss-G and Loss-L: $Loss-T=k\times
Loss-G+(1-k)\times Loss-L$ where k is a hyperparameter. The region of k is
(0-1). 0,0.5,1 are assigned to k for training respectively. When k=0, the
network only updates Loss-L, which mains just the local feature extraction
network takes part in training; when k=1, the network only updates Loss-G,
which mains just the global feature extraction network takes part in training;
when k=0.5, Both the global feature network and the local feature network will
be added to the training, and this is another combination method that is
different from the global residual connection.

The second structure is almost same to global regular network, but just saving
the last global residual connection. We mark it Net2. The comparison of these
structures with GRN will verify the effect of the adequacy of global residual
connection.

TABLE III: COMPARISON OF GRN AND OTHER DESIGNED STRUCTURES Handwriting | Train-Loss | Test-loss
---|---|---
Net1-word | 0.010 | 0.052
Net1-page | 0.031 | 0.057
Net1-both | 0.020 | 0.040
Net2 | 0.0074 | 0.008
Ours | 0.0065 | 0.004

For the consideration of training efficiency, the dataset used for the
comparison experiment just need to be the same, we randomly select 60 classes
from the original dataset of 310 classifications as the new dataset for the
comparison experiment. In addition, due to the small number of datasets, we
found that the accuracy of some networks easily reach 100% on some iterations
in the final test, and the accuracy index will no longer effectively reflect
the ability of the network. And we found that the loss index will remain at a
stable level. So we choose the loss in training process and testing process as
the evaluation index of network writer identification ability. The lower the
train-loss and test-loss, the better the result. The experimental results are
shown in TableIII.

First, we compare the three experiments of Net1. For train-loss, Net1-page is
0.031, Net1-both is 0.020, and Net1-word is 0.010. We have two findings: one
is that the training effect of using page as dataset is not well as using
word; the other is, the smaller rate of loss-G in loss-T, the higher the
accuracy of training. We conclude that the training effect of page handwriting
is not as good as that of word handwriting. We attribute it to the fact that
human handwriting features are mainly contained in local features. For test-
loss, Net1-page is 0.057, Net1-both is 0.040, and Net1-word is 0.052. We found
that the effect of word input is better than that of page input, but worse
than the test result obtained by mixed input of word and page. We attribute it
to the fact that the overfitting introduced by using only one type of
handwriting as input and the mixed input allows the network to classify the
handwriting from multiple sources, resulting in the introduction of
regularization to reduce the effect of overfitting. The second network train-
loss is 0.0074, test-loss is 0.008, and the effect is much better than the
first three networks. This reflects the effectiveness of global residual
connection methods. Compared with Net2, our network processes global residual
connections on the four modules, and achieves train-loss and test-loss of
0.0065 and 0.004. The performance is further improved on the basis of Net2,
which reflects the validity of the four-way global residual connection model.
The greatest improvement is test-loss, which we attribute to the four-way
global residual connection that makes the local features and global features
more fully merged, and the regular effect of global features is further
improved.

We also verify the efficiency of the residual attention network to extract
local features. We design a comparative experiment: only use the traditional
resnet34 network to extract local features. That means the global feature
extraction network and the local feature extraction network have the same
structure. The result compared with attention residual network is shown in
Table.IV. The results show that there is a big gap between the network
performance without the attention module and with the attention module. On the
other hand, in actual experiments, the network using the attention module can
be trained at a faster speed.

TABLE IV: COMPARISON OF GRN WITH RESIDUAL ATTENTION NETWORK AND GRN WITHOUT RESIDUAL ATTENTION NETWORK Handwriting | Train-Loss | Test-loss
---|---|---
Net-without attention | 0.025 | 0.023
Net-with attention | 0.0065 | 0.004

##  V Conclusion

We propose global regular network (GRN) for writer identification. The network
has two highlights: First, the network pays attention to the global
characteristics of handwriting. The network extracts handwriting features in
two ways: one is to extract the global features of the handwriting, and the
other is to extract the local features of the handwriting. Finally, the global
feature and the local feature are combined in a global residual way. With this
operation, more information will be extracted by our network; the second is
that GRN uses the residual attention network to extract the local features of
the handwriting, so that the network can extract more local features. We
performed writer identification tasks on CVL dataset, and achieved amazing
results of 99.98% top1 accuracy and 100% top5 accuracy, surpassing the current
state-of-the-art architecture. The core idea of the network design is to
extract the features in multiple levels. We can not only use this strategy in
the field of writer identification, but also applies to many other areas of
feature extraction.

## References

  * [1] B. Helli and M. E. Moghaddam, “A text-independent persian writer identification based on feature relation graph (frg),” _Pattern Recognition_ , vol. 43, no. 6, pp. 2199–2209, 2010. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0031320309004452
  * [2] V. Christlein, D. Bernecker, F. Hönig, A. Maier, and E. Angelopoulou, “Writer identification using gmm supervectors and exemplar-svms,” _Pattern Recognition_ , vol. 63, pp. 258–267, 2017. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0031320316303211
  * [3] V. Christlein, M. Gropp, S. Fiel, and A. Maier, “Unsupervised feature learning for writer identification and writer retrieval,” in _2017 14th IAPR International Conference on Document Analysis and Recognition (ICDAR)_ , vol. 01, 2017, pp. 991–997.
  * [4] V. Christlein and A. Maier, “Encoding cnn activations for writer recognition,” in _2018 13th IAPR International Workshop on Document Analysis Systems (DAS)_ , 2018, pp. 169–174.
  * [5] D. Bertolini, L. Oliveira, E. Justino, and R. Sabourin, “Texture-based descriptors for writer identification and verification,” _Expert Systems with Applications_ , vol. 40, no. 6, pp. 2069–2080, 2013. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S095741741201130X
  * [6] M. B. Yılmaz and B. Yanıkoğlu, “Score level fusion of classifiers in off-line signature verification,” _Information Fusion_ , vol. 32, pp. 109–119, 2016, sI Information Fusion in Biometrics. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S1566253516300033
  * [7] S. Fiel and R. Sablatnig, “Writer identification and retrieval using a convolutional neural network,” in _Computer Analysis of Images and Patterns_ , G. Azzopardi and N. Petkov, Eds. Cham: Springer International Publishing, 2015, pp. 26–37.
  * [8] V. Christlein, D. Bernecker, A. Maier, and E. Angelopoulou, “Offline writer identification using convolutional neural network activation features,” in _Pattern Recognition_ , J. Gall, P. Gehler, and B. Leibe, Eds. Cham: Springer International Publishing, 2015, pp. 540–552.
  * [9] Y. Tang and X. Wu, “Text-independent writer identification via cnn features and joint bayesian,” in _2016 15th International Conference on Frontiers in Handwriting Recognition (ICFHR)_ , 2016, pp. 566–571.
  * [10] L. Xing and Y. Qiao, “Deepwriter: A multi-stream deep cnn for text-independent writer identification,” in _2016 15th International Conference on Frontiers in Handwriting Recognition (ICFHR)_ , 2016, pp. 584–589.
  * [11] H. T. Nguyen, C. T. Nguyen, T. Ino, B. Indurkhya, and M. Nakagawa, “Text-independent writer identification using convolutional neural network,” _Pattern Recognition Letters_ , vol. 121, pp. 104–112, 2019, graphonomics for e-citizens: e-health, e-society, e-education. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0167865518303180
  * [12] S. He and L. Schomaker, “Fragnet: Writer identification using deep fragment networks,” _IEEE Transactions on Information Forensics and Security_ , vol. 15, pp. 3013–3022, 2020.
  * [13] F. Wang, M. Jiang, C. Qian, S. Yang, C. Li, H. Zhang, X. Wang, and X. Tang, “Residual attention network for image classification,” in _2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_ , 2017, pp. 6450–6458.
  * [14] D. Walther, L. Itti, M. Riesenhuber, T. Poggio, and C. Koch, “Attentional selection for object recognition - A gentle way,” in _BIOLOGICALLY MOTIVATED COMPUTER VISION, PROCEEDINGS_ , ser. LECTURE NOTES IN COMPUTER SCIENCE, Bulthoff, HH and Lee, SW and Poggio, TA and Wallraven, C, Ed., vol. 2525. Max Planck Inst Biol Cybernet; Univ Tubingen; Comp Koch, 2002, pp. 472–479, 2nd International Workshop on Biologically Motivated Computer Vision (BMCV 2002), TUBINGEN, GERMANY, NOV 22-24, 2002.
  * [15] L. Itti and C. Koch, “Computational modelling of visual attention,” _Nature Reviews Neuroscience_ , vol. 2, pp. 194–203, 2001. [Online]. Available: https://doi.org/10.1038/35058500
  * [16] V. Mnih, N. Heess, A. Graves, and K. Kavukcuoglu, “Recurrent Models of Visual Attention,” in _ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS 27 (NIPS 2014)_ , ser. Advances in Neural Information Processing Systems, Ghahramani, Z and Welling, M and Cortes, C and Lawrence, ND and Weinberger, KQ, Ed., vol. 27, 2014, 28th Conference on Neural Information Processing Systems (NIPS), Montreal, CANADA, DEC 08-13, 2014.
  * [17] B. Zhao, X. Wu, J. Feng, Q. Peng, and S. Yan, “Diversified visual attention networks for fine-grained object classification,” _IEEE Transactions on Multimedia_ , vol. 19, no. 6, pp. 1245–1256, 2017.
  * [18] E. N. Zois, I. Theodorakopoulos, and G. Economou, “Offline handwritten signature modeling and verification based on archetypal analysis,” in _2017 IEEE International Conference on Computer Vision (ICCV)_ , 2017, pp. 5515–5524.
  * [19] M. B. Yilmaz, B. Yanikoglu, C. Tirkaz, and A. Kholmatov, “Offline signature verification using classifier combination of hog and lbp features,” in _2011 International Joint Conference on Biometrics (IJCB)_ , 2011, pp. 1–7.
  * [20] Y. Guerbai, Y. Chibani, and B. Hadjadji, “The effective use of the one-class svm classifier for handwritten signature verification based on writer-independent parameters,” _Pattern Recognition_ , vol. 48, no. 1, pp. 103–113, 2015. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0031320314002751
  * [21] L. G. Hafemann, R. Sabourin, and L. S. Oliveira, “Learning features for offline handwritten signature verification using deep convolutional neural networks,” _Pattern Recognition_ , vol. 70, pp. 163–176, 2017. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0031320317302017
  * [22] K. Franke, O. Bunnemeyer, and T. Sy, “Ink texture analysis for writer identification,” in _Proceedings Eighth International Workshop on Frontiers in Handwriting Recognition_ , 2002, pp. 268–273.
  * [23] H. Said, T. Tan, and K. Baker, “Personal identification based on handwriting,” _Pattern Recognition_ , vol. 33, no. 1, pp. 149–160, 2000\. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0031320399000060
  * [24] G. X. Tan, C. Viard-Gaudin, and A. C. Kot, “Automatic writer identification framework for online handwritten documents using character prototypes,” _Pattern Recognition_ , vol. 42, no. 12, pp. 3313–3323, 2009, new Frontiers in Handwriting Recognition. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0031320308005256
  * [25] C.-L. Liu, F. Yin, D.-H. Wang, and Q.-F. Wang, “Casia online and offline chinese handwriting databases,” in _2011 International Conference on Document Analysis and Recognition_ , 2011, pp. 37–41.
  * [26] S. Chen, X. Tan, B. Wang, and X. Hu, “Reverse attention for salient object detection,” in _Computer Vision – ECCV 2018_ , V. Ferrari, M. Hebert, C. Sminchisescu, and Y. Weiss, Eds. Cham: Springer International Publishing, 2018, pp. 236–252.
  * [27] K. He, X. Zhang, S. Ren, and J. Sun, “Identity mappings in deep residual networks,” in _Computer Vision – ECCV 2016_ , B. Leibe, J. Matas, N. Sebe, and M. Welling, Eds. Cham: Springer International Publishing, 2016, pp. 630–645.
  * [28] E. Shelhamer, J. Long, and T. Darrell, “Fully convolutional networks for semantic segmentation,” _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , vol. 39, no. 4, pp. 640–651, 2017.
  * [29] A. Newell, K. Yang, and J. Deng, “Stacked hourglass networks for human pose estimation,” in _Computer Vision – ECCV 2016_ , B. Leibe, J. Matas, N. Sebe, and M. Welling, Eds. Cham: Springer International Publishing, 2016, pp. 483–499.
  * [30] H. Noh, S. Hong, and B. Han, “Learning deconvolution network for semantic segmentation,” in _2015 IEEE International Conference on Computer Vision (ICCV)_ , 2015, pp. 1520–1528.
  * [31] F. Kleber, S. Fiel, M. Diem, and R. Sablatnig, “Cvl-database: An off-line database for writer retrieval, writer identification and word spotting,” in _2013 12th International Conference on Document Analysis and Recognition_ , 2013, pp. 560–564.
  * [32] X. Wu, Y. Tang, and W. Bu, “Offline text-independent writer identification based on scale invariant feature transform,” _IEEE Transactions on Information Forensics and Security_ , vol. 9, no. 3, pp. 526–536, 2014.
  * [33] S. M. Chen and Y. S. Wang, “A robust off-line writer identification method,” _Zidonghua Xuebao/Acta Automatica Sinica_ , vol. 46, pp. 108–116, 1 2020\.