11institutetext:  School of Electrical and Computer Engineering
National Technical University of Athens, Greece
22institutetext: Computational Intelligence Laboratory
Institute of Informatics and Telecommunications, National Center for
Scientific Research “Demokritos”, Greece
33institutetext: Department of Computer Science and Engineering
University of Ioannina, Greece
44institutetext: Department of Surveying and Geoinformatics Engineering
University of West Attica, Greece
44email<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>

# Best Practices for a Handwritten Text Recognition System

George Retsinas 11 Giorgos Sfikas 22 3 3 4 4 Basilis Gatos 22 Christophoros
Nikou 33

###### Abstract

Handwritten text recognition has been developed rapidly in the recent years,
following the rise of deep learning and its applications. Though deep learning
methods provide notable boost in performance concerning text recognition, non-
trivial deviation in performance can be detected even when small pre-
processing or architectural/optimization elements are changed. This work
follows a “best practice” rationale; highlight simple yet effective empirical
practices that can further help training and provide well-performing
handwritten text recognition systems. Specifically, we considered three basic
aspects of a deep HTR system and we proposed simple yet effective solutions:
1) retain the aspect ratio of the images in the preprocessing step, 2) use
max-pooling for converting the 3D feature map of CNN output into a sequence of
features and 3) assist the training procedure via an additional CTC loss which
acts as a shortcut on the max-pooled sequential features. Using these proposed
simple modifications, one can attain close to state-of-the-art results, while
considering a basic convolutional-recurrent (CNN+LSTM) architecture, for both
IAM and RIMES datasets. Code is available at
https://github.com/georgeretsi/HTR-best-practices/.

###### Keywords:

Handwritten Text Recognition, Convolution - Recurrent Neural Network, Best
Practices

##  1 Introduction

Handwritten Text Recognition (HTR) is an active area of research, combining
ideas from both computer vision and natural language processing. Unlike
recognition of machine-printed text, handwriting is related to a number of
unique characteristics that make the task much more challenging than
traditional optical character recognition (OCR). The challenging nature of
handwriting recognition stems mostly from the potentially high writing
variability between individuals. To this end, along with visually decoding an
image into sequence of characters, several HTR works adopt language models to
reduce this innate ambiguity of handwritten characters, making use of
contextual and semantic information.

In general, designing an effective and generalizable learning system is a
ongoing challenge, with transferability between different learned writing
styles more being not a given in most cases [22]. Neural Networks (NNs), among
a variety of other learning systems, have been used for the recognition of
handwriting from early on, with a span ranging between simpler subtasks such
as single digit recognition [1] up to full, unconstrained offline HTR [7, 21].
Following the rise of deep learning and its applications, recent developments
in HTR are monopolized by Deep Neural Networks (DNNs). The seminal work of
Graves et al. [9] played a pivotal role in the rise of deep learning for HTR
applications by enabling the training of recurrent nets without assuming any
prior character segmentation. A plethora of subsequent works for HTR relied on
Graves et al. in order to train modern and notably effective DNNs [21, 14, 24,
16].

This work focuses on finding best practices for building modern HTR systems.
We explore a set of guidelines for training HTR DNNs, re-examining and
extending ideas from several previous works of ours [25, 23, 24]. We start
with a fairly common deep network architecture for HTR, consisting of a CNN
backbone and a BiLSTM head, and we make simple yet effective architectural and
training choices. These best practice suggestions can be categorized and
summarized as follows:

  1. 1.

pre-processing: retain aspect ratio of images and use batches of padded images
in order to effectively use mini-batch Stochastic Gradient Descent (SGD)

  2. 2.

architectural: replace the the column-wise concatenation step between the CNN
backbone and the recurrent head with a max-pooling step. Such a choice not
only reduces the required parameters but has an intuitive motivation: we care
only about the existence of a character and not its vertical position.

  3. 3.

training: add an extra shortcut branch, consisting of a single 1D convolution
layer, at the output of the CNN backbone. This branch results to an extra
character sequence estimation, trained in parallel to the recurrent branch.
Both branches use a CTC loss. The motivation behind such a choice comes from
the increased difficulty of training recurrent layers. However, if such a
straightforward shortcut exists, the output of the CNN backbone should
converge to more discriminative features, ideal for fully harnessing the power
of recurrent layers compared to an end-to-end training scheme.

The contribution of this paper is best highlighted through the experimental
section, where we achieve state-of-the-art results with the aforementioned
choices, despite the simplicity of the employed network. Furthermore, other
state-of-the-art existing methods propose complex architectures and
augmentation schemes which are orthogonal to our approach, highlighting the
importance of the suggested best practices.

##  2 Related Work

As is the case with most, if not all, tasks in computer vision, modern HTR
literature is dominated by neural network-based methods. Recurrent neural
networks have become the baseline [15, 21], as they naturally fit to the
sequential nature of handwriting.

Recurrent-based approaches have thus practically overshadowed the previous
state-of-the-art, which was based mostly on Hidden Markov Model (HMM)-based
approaches. Since the introduction of the standard recurrent network paradigm
[7, 6], many key advances have emerged paving the way for very efficient HTR
systems. A characteristic example is the integration of the Long Short-Term
Memory models (LSTMs) into HTR systems [10], that effectively dealt with the
vanishing gradient problem. More importantly, Graves et al. [9] introduced a
very effective algorithm algorithm for training such HTR systems with
sequence-based loss using dynamic programming. Specifically, this
Connectionist Temporal Classification (CTC) method and corresponding output
layer [8], a differentiable output layer that maps a sequential input into
per-time unit softmax outputs, allows simultaneous sequence alignment and
recognition with a suitable decoding scheme. Multi-dimensional recurrent
networks have been considered for HTR [15], however there has been criticism
that the extra computational overhead may not translate to an analogous
increase in efficiency [21].

Even though in this work we will focus only on greedy decoding of a CTC-
trained network, research on decoding schemes is also active [4], with the
beam search algorithm being a popular approach, capable of exploiting an
external lexicon as an implicit language model.

Sequence-to-Sequence approaches, involving translating an input sequence to an
output sequence of a different length in general, became very popular when
achieved state-of-the-art results in Natural Language Processing and gradually
evolved to Transformer networks with attention mechanisms [29]. Such
approaches were later adopted successfully by the HTR community [27, 3, 19,
30, 25].

Recent research directions include complex augmentation schemes ( [14, 31,
16]), novel network architectures/modules (e.g. Seq2Seq/Transformers, Spatial
Transformer Networks [5], deformable convolutions [24]) and multi-task losses
with auxiliary training feeds (e.g. n-gram training [28]).

##  3 Proposed HTR System

In what follows, we will describe in detail the proposed HTR system with
emphasis given on the suggested best practice modifications. The described
system takes as input either a word or a line image and then returns the
predicted sequence of characters based on an unconstrained greedy CTC decoding
algorithm [8].

###  3.1 Preprocessing

The pre-processing steps, applied to every image, are:

  1. 1.

All images are resized to a resolution of $128\times 1024$ pixels for line
images or $64\times 256$ pixels. Initial images are padded (using the image
median value, usually zero) in order to attain the aforementioned fixed size.
If the initial image is greater than the predefined size, the image is
rescaled. The padding option aims to preserve the existing aspect ratio of the
text images.

  2. 2.

During training, image augmentation is performed. A simple global affine
augmentation is applied at every image, considering only rotation and skew of
small magnitude in order to generate valid images. Additionally, gaussian
noise is added to the images.

  3. 3.

Each word/line transcription has spaces added before and after, i.e. ”He rose
from” is changed to ” He rose from ”. This operation aims to assist the system
to adapt to the marginal spaces that exist in the majority of the images
during the training phase. For the testing phase, these additional spaces are
discarded.

Augmentation operations are part of every modern deep learning system and can
consistently provide increased performance, allowing better generalization
[21]. The used augmentation scheme is very basic, trying to have minimal
overhead from this step.

The addition of extra spaces in the transcription is not explicitly referred
to recent existing works, but it is intuitive given the pad operation of step
1 which creates large empty margins. It has a minor yet positive impact to our
system and thus is added as a step. Due to the reduced significance of this
step in the overall performance, it is not explored in the experimental
section.

On the other hand, we found the padding operation critical in many settings. A
typical trade-off, met in many recent text recognition/spotting works,
concerns the definition of the input size: using a predefined fixed size can
assist the architectural design of CNNs and training time requirements, while
retaining the initial image size by processing individually each image
(e.g.[26]) may lead to better performance at the cost of discarding the mini-
batch option.

Modern DNN training relies on creating batches of several images, since batch
manipulation of images can notably affect the training time by fully utilizing
the GPU resources. Thus image resizing is a widely-used first step for any
vision problem when DNNs are involved. On the contrary, when using different
sized images by processing each image individually and update the network’s
weights after a predefined number of images, as if a batch was processed,
leads to an impractical time-consuming training procedure to otherwise
lightweight DNNs, where the existing hardware is under-utilized.

In this work, contrary to the majority of existing approaches, we propose a
simple, yet elegant, solution: we aim to retain the aspect ratio of the images
and simultaneously organize them into batches. The images are transformed into
the same, predefined, shape without resizing, if possible. Specifically, if
the image size is smaller than the predefined size, we pad the image
accordingly. The padding operation is performed equally at each direction,
positioning the initial image at the center of the new one, with a fixed
value, the median value of the initial image. If the image is larger than the
predefined size, it is resized,affecting the aspect ratio. To assist the
proposed approach, we can compute the average height and width over the whole
set of the initial images and select an appropriate size in order to perform
the aforementioned resize operation scarcely (only for very large
words/sentences) and thus avoid deformations that are generated by frequently
violating the aspect ratio.

Figure 1: Overview of the DNN architecture. Apart from the CNN backbone, the
recurrent head, we also depict the auxiliary CTC shortcut branch which will be
the core component of the proposed training modification. Furthermore, we
visualize the column-wise max-pooling operation that is performed between the
CNN backbone and the recurrent head.

###  3.2 Network Architecture

The model that we will use to test the proposed technique can be characterized
as a convolutional-recurrent architecture (an architecture overview is
depicted in Figure 1). The convolutional-recurrent architecture can be broadly
defined as a convolutional backbone being followed by a recurrent head,
typically connected to a CTC loss. Convolutional-recurrent variants have given
routinely very good results for HTR [5, 21].

####  3.2.1 Convolutional Backbone:

In our model, the convolutional backbone is made up of standard convolutional
layers and ResNet blocks [12], interspersed with max-pooling and dropout
layers. In particular, the first layer is a $7\times 7$ convolution with $32$
output channels, followed by cascades of $3\times 3$ ResNet blocks [12]: a
series of $2$ ResNet blocks with $64$ output channels, $4$ ResNet blocks with
$128$ output channels and $4$ ResNet blocks with $256$ output channels. The
standard convolution and the ResNet blocks are all followed by ReLU
activations, Batch Normalization and dropout. Between cascades of blocks we
downscale the produced feature map with $2\times 2$ max-pooling operations of
stride $2$, as shown in Figure 1. Overall, the convolutional backbone accepts
a line image and outputs a tensor of size $h\times w\times d$ (e.g. assuming
the line image case, the tensor is of size $16\times 128\times 256$).

####  3.2.2 Flattening Operation:

The convolutional backbone output should be transformed into a sequence of
features in order to processed by recurrent networks. Typical HTR approaches,
assume a column-wise approach (towards the writing direction) to ideally
simulate a character by character processing. In our work, the CNN output is
flattened by a max-pooling operation in a column-wise manner. Flattening of
the extracted feature maps by the widely-used concatenation operation would
result into a sequence of length $w$ with feature vectors of size $hd$, while
max-pooling results to reduced feature vectors of size $d$. Apart from the
apparent computational advantage, _column-wise max-pooling_ achieves model
translation invariance in the vertical direction. In fact, the reasoning
behind max-pooling is that we care only about the existence of features
related to a character and not their spatial position. This has been the major
motivation for _column-wise max-pooling_ , as successfully employed in our
previous works [28, 24, 25].

####  3.2.3 Recurrent Head:

The recurrent component consists of $3$ stacked Bidirectional Long Short-Term
Memory (BiLSTM) units of hidden size $256$. These are followed by a linear
projection layer, which converts the sequence to a size equal to the number of
possible character tokens, $n_{classes}$ (including the blank character,
required by CTC). The final output of the recurrent part can be translated
into a sequence of probability distributions by applying a softmax operation.
During evaluation, the aforementioned greedy decoding is performed by
selecting the character with the highest probability at each step and then
removing the blank characters from the resulting sequence [8].

###  3.3 Training Scheme

The training of the HTR system is performed via an Adam [13] optimizer using
an initial learning rate of $0.001$ which gradually decreases using a
multistep scheduler. The overall training epochs are 240 and the scheduler
decreases the learning rate by a factor of $0.1$ at 120 and 180 epochs.

This optimizing scheme, with minor modifications, is commonly used for HTR
systems. Nonetheless, we assume an end-to-end training approach where both the
convolutional and the recurrent parts of the system are optimized through the
final CTC loss. Even though this typical approach produces well-performing
solutions, the LSTM head may encumbers the overall training procedure, since
recurrent modules are known to exhibit convergence difficulties.

To circumvent this training hindrance, we introduce an auxiliary branch as
shown in Figure 1. We dub this extra module as a “CTC shortcut”. In what
follows, we describe this module and its functionality in detail.

####  3.3.1 CTC shortcut:

Architecture-wise, the CTC shortcut module consists only of a single 1D
convolutional layer, with kernel size 3. Its output channels equal to the
number of the possible character tokens ($n_{classes}$). Therefore, the 1D
convolutional layer is responsible for straightforwardly encoding context-wise
information and providing an alternative decoding path. Note that we strive
for simplicity for this auxiliary component, since its aim is to assist the
training of the main branch and thus a shallow convolutional part of only one
layer is ideal for this task. We do not expect from the CTC branch to result
to precise decodings.

The CTC shortcut is trained along with the main architecture using a multi-
task loss by adding the corresponding CTC losses of the two branches with the
appropriate weights. Specifically, if $f_{cnn}$ represents the convolutional
backbone, $f_{rec}$ represents the recurrent part and $f_{shortcut}$
represents the proposed shortcut branch, while $I$ is an input image and $s$
its corresponding transcription, the multi-task loss is written as:

$L_{CTC}(f_{rec}(f_{cnn}(I));s)+0.1\,L_{CTC}(f_{shortcut}(f_{cnn}(I));s)$ (1)

Since CTC shortcut acts only as an auxiliary training path, it is weighted by
$0.1$ to reduce its relative contribution to the overall loss.

The motivation behind this extra branch is rather simple: overall convergence
is assisted by quickly generating discriminative features at the top of the
CNN backbone through the straightforward 1D convolutional path, simplifying
the training task for the recurrent part. Due to its training-oriented
assisting nature, CTC shortcut is used only during training and omitted during
evaluation. Therefore, this proposed shortcut does not introduces any overhead
during inference.

##  4 Experimental Evaluation

Evaluation of the proposed system is performed on two widely used datasets,
IAM[18] and Rimes [11]. The ablation study, considering different settings of
the proposed methodology, is performed on the challenging IAM dataset,
consisting of handwritten text from 657 different writers and partitioned into
writer-independent train/validation/test sets (we use the same set partition
as in [21]). All experiments follow the same setting: line-level or word-level
recognition using a lexicon-free unconstrained greedy CTC decoding scheme.
Character Error Rate (CER) and Word Error Rate (WER) metrics are reported in
all cases (lower values are better).

###  4.1 Ablation Study

First, we explore the impact of the proposed modifications over both the
validation and the test set of IAM dataset. Moreover, both line-level
recognition (Table 1) and word-level recognition (Table 2) are considered.
Specifically, we investigate the difference in performance when we: 1) use
_resized_ or _padded_ (retain aspect-ratio case) input images, 2) use
_concatenation_ of _max-pooling_ flattening operation between the
convolutional backbone and the recurrent head and 3) use or not the _CTC
shortcut during_ the training process.

Table 1: Line-level recognition results for IAM dataset: Exploring the impact
of the proposed modifications.

|  |  | Validation | Test
---|---|---|---|---
Preprocessing |  Flattening |  CTC Shortcut |  CER(%) |  WER(%) |  CER(%) |  WER(%)
resized |  concatenation |  no |  4.28 |  15.29 |  5.93 |  19.57
|  |  yes |  3.72 |  13.18 |  5.11 |  16.96
resized |  max-pooling |  no |  3.73 |  13.54 |  5.28 |  17.77
|  |  yes |  3.47 |  12.77 |  4.85 |  16.19
padded |  concatenation |  no |  4.06 |  14.40 |  5.54 |  18.60
|  |  yes |  3.37 |  12.22 |  4.71 |  15.94
padded |  max-pooling |  no |  3.46 |  12.55 |  4.93 |  16.81
|  |  yes |  3.21 |  11.89 |  4.62 |  15.89

Table 2: Word-level recognition results for IAM dataset: Exploring the impact
of the proposed modifications.

|  |  | Validation | Test
---|---|---|---|---
Preprocessing |  Flattening |  CTC Shortcut |  CER(%) |  WER(%) |  CER(%) |  WER(%)
resized |  concatenation |  no |  4.35 |  12.55 |  5.58 |  15.46
|  |  yes |  4.27 |  12.02 |  5.46 |  15.13
resized |  max-pooling |  no |  4.25 |  12.17 |  5.69 |  15.87
|  |  yes |  4.09 |  11.65 |  5.23 |  14.40
padded |  concatenation |  no |  4.17 |  11.99 |  5.66 |  15.66
|  |  yes |  3.98 |  11.50 |  5.37 |  14.98
padded |  max-pooling |  no |  4.00 |  11.25 |  5.43 |  15.06
|  |  yes |  3.76 |  10.76 |  5.14 |  14.33

The following observations can be made:

  * •

Retaining the aspect-ratio of the images (padded option) achieves improved
results for the majority of cases.

  * •

Performing the flattening operation via max-pooling not only is more cost-
effective, but it has a positive impact on performance. This is more evident
in line-level recognition setting.

  * •

Training with a CTC shortcut module provides notable boost over all cases. For
example, in line-level recognition the significant difference in performance
when considering different flatting operations is considerably decreased when
the CTC shortcut approach is adopted (e.g. for padded line-level recognition
the WER performance difference drops from 1.79% to only 0.05%). This hints
that the initial difference in performance is mainly attributed to
difficulties in training (concatenated version has a much larger feature
vector to manage). Note that evaluating the CTC shortcut branch yields poor
decodings, despite the notable performance increase of the main network. For
example, assuming line-level recognition and the padded/max-pooling setting,
we report 5.26% CER/19.76% WER for the validation set and 7.36% CER/25.66% WER
for the test set.

  * •

Applying all three modifications together achieves the best results across all
setting and metrics.

  * •

Word recognition reports improved results compared to line-level recognition
with respect to the WER metric. This was expected, since word-level setting
assumes perfect word segmentation. Interestingly enough, this is not the case
for the CER metric. This can be explained by the lack of sufficient context
(i.e. find a capital letter or a punctuation from the whole line information).

We also explore in more depth the CTC shortcut option, which seems to provide
the best boost in performance. Specifically, we report the progress of both
the loss and the CER/WER metrics (over the validation set) during the training
procedure for the line-level recognition setting. The loss curves are depicted
in Figure 2, while the validation set evaluation metrics are reported in
Figure 3. As we can see, loss curves are similar, but the case of CTC shortcut
consistently has slightly better behavior. The impact of the CTC shortcut is
more clearly shown in CER/WER curves and thus solutions with greater
generalization properties are expected when a model is trained along with the
CTC shortcut.

Figure 2: Behavior of HTR performance in terms of loss value with and without the extra CTC shortcut branch during the training phase. Reported curves correspond to the proposed line-level HTR system trained on the IAM dataset. |
---|---
Figure 3: Behavior of HTR performance in terms of CER (left) and WER (right)
metrics with and without the extra CTC shortcut branch during the training
phase. Reported curves correspond to the proposed line-level HTR system,
trained on IAM dataset and evaluated on the validation set.

###  4.2 Comparison to State-of-the-Art Systems

Finally, we compare our method to several existing state-of-the-art methods,
as shown in Table 3. The reported methods follow the same setting: line-level
lexicon-free recognition. The proposed HTR system along with the suggested
modifications achieves results comparable to the best performing methods.
Notably, it outperforms the majority of existing works for both datasets and
metrics despite many of the reported methods propose novel elements to further
increase performance that are in general orthogonal to our approach. For
example, the work of Chowdhury et al. [3] presents better WER for the RIMES
dataset while using a sequence-to-sequence approach (such models can produce
increased WER as implicit language models can be learnt [25]), while our
previous work [24] achieves better CER for the IAM dataset while using similar
network (max-pooling flattening and padded input images) along with deformable
convolutions and a post processing uncertainty reduction algorithm.

Moreover, the very recent work of Luo et al. [16] manages to outperform our
method for the word-level recognition setting on the IAM dataset by using a
STN component and a complex augmentation method, where ”optimal” augmentations
are learnt. Specifically, our method achieves 5.14% CER / 14.33%, while Luo et
al. achieve 5.13% CER / 13.35% WER for the exact same setting. Nonetheless,
their initial baseline network, stripped of all the extra modules (which could
be added to the proposed architecture without any problem), performs poorly:
7.39% CER and 19.12% WER.

Overall, we achieve very competitive results (outperforming other existing
lexicon-free methods for line-level recognition on IAM) by only using a
typical convolutional-recurrent architecture along with a set of simple, yet
intuitive and effective modifications, forming an effective set of best
practice suggestions which can be applied to the majority of HTR systems.

Table 3: Performance comparison for IAM/RIMES datasets (line-level recognition)  | IAM | RIMES
---|---|---
Method |  CER(%) |  WER(%) |  CER(%) |  WER(%)
Chen et al. [2] |  11.15 |  34.55 |  8.29 |  30.5
Pham et al. [20] |  10.8 |  35.1 |  6.8 |  28.5
Khrishnan et al. [14] |  9.78 |  32.89 |  - |  -
Chowdhury et al. [3] |  8.10 |  16.70 |  3.59 |  9.60
Puigcerver [21] |  6.2 |  20.2 |  2.60 |  10.7
Khrishnan et al. [14] |  9.78 |  32.89 |  - |  -
Markou et al. [17] |  6.14 |  20.04 |  3.34 |  11.23
Dutta et al. [5] |  5.8 |  17.8 |  5.07 |  14.7
Wick et al. [30] |  5.67 |  - |  - |  -
Michael et al. [19] |  5.24 |  - |  - |  -
Tassopoulou et al. [28] |  5.18 |  17.68 |  - |  -
Yousef et al. [32] |  4.9 |  - |  - |  -
Retsinas et al. [24] |  4.55 |  16.08 |  3.04 |  10.56
Proposed |  4.62 |  15.89 |  2.75 |  9.93

##  5 Conclusions

In this paper, we proposed a series of best practice modifications over
typical convolutional-recurrent networks trained with CTC loss. Apart from
presenting a fairly compact architecture based on residual blocks, we present
three impactful modifications: 1) retain aspect-ratio of input images gathered
in batches through a padding operation, 2) apply a column-wise max-pooling
operation between the convolutional backbone and the recurrent head of a
typical HTR architecture for reduced computational effort and increased
performance and 3) enhance performance through a CTC shortcut during training
in order to circumvent an end-to-end training over recurrent networks, which
have been proven ”difficult” to train in various settings. All proposed
modifications have proven to be very helpful, considerably increasing the
performance of the vanilla network. Overall, the proposed system achieves
results in the ballpark of state-of-the-art, while being orthogonal to the
majority of modern deep learning modules and approaches.

## Acknowledgments

This research has been partially co - financed by the EU and Greek national
funds through the Operational Program Competitiveness, Entrepreneurship and
Innovation, under the calls : “RESEARCH - CREATE - INNOVATE”, project
_Culdile_ (code T1E$\Delta$K - 03785) and “OPEN INNOVATION IN CULTURE”,
project _Bessarion_ (T6YB$\Pi$ \- 00214).

## References

  * [1] Bishop, C.M.: Pattern Recognition and Machine Learning. Springer (2006)
  * [2] Chen, Z., Wu, Y., Yin, F., Liu, C.L.: Simultaneous script identification and handwriting recognition via multi-task learning of recurrent neural networks. In: 14th IAPR International Conference on Document Analysis and Recognition (ICDAR). vol. 1, pp. 525–530. IEEE (2017)
  * [3] Chowdhury, A., Vig, L.: An efficient end-to-end neural model for handwritten text recognition (2018)
  * [4] Collobert, R., Hannun, A., Synnaeve, G.: A fully differentiable beam search decoder. In: International Conference on Machine Learning. pp. 1341–1350. PMLR (2019)
  * [5] Dutta, K., Krishnan, P., Mathew, M., Jawahar, C.: Improving CNN-RNN hybrid networks for handwriting recognition. In: 2018 16th International Conference on Frontiers in Handwriting Recognition (ICFHR). pp. 80–85. IEEE (2018)
  * [6] Fischer, A., Keller, A., Frinken, V., Bunke, H.: Lexicon-free handwritten word spotting using character HMMs. Pattern Recognition Letters 33(7), 934–942 (2012)
  * [7] Fischer, A.: Handwriting recognition in historical documents. Ph.D. thesis, Verlag nicht ermittelbar (2012)
  * [8] Graves, A.: Connectionist temporal classification. In: Supervised Sequence Labelling with Recurrent Neural Networks, pp. 61–93. Springer (2012)
  * [9] Graves, A., Fernández, S., Gomez, F., Schmidhuber, J.: Connectionist temporal classification: labelling unsegmented sequence data with recurrent neural networks. In: Proceedings of the 23rd international conference on Machine learning. pp. 369–376 (2006)
  * [10] Greff, K., Srivastava, R.K., Koutník, J., Steunebrink, B.R., Schmidhuber, J.: LSTM: A search space odyssey. IEEE Transactions on Neural Networks and Learning Systems 28(10), 2222–2232 (2016)
  * [11] Grosicki, E., Carre, M., Brodin, J.M., Geoffrois, E.: Rimes evaluation campaign for handwritten mail processing (2008)
  * [12] He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 770–778 (2016)
  * [13] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. In: Proceedings of the International Conference on Learning Representations (ICLR) (2015)
  * [14] Krishnan, P., Dutta, K., Jawahar, C.: Word spotting and recognition using deep embedding. In: 2018 13th IAPR International Workshop on Document Analysis Systems (DAS). pp. 1–6. IEEE (2018)
  * [15] Leifert, G., Strau, T., Gr, T., Wustlich, W., Labahn, R., et al.: Cells in multidimensional recurrent neural networks. Journal of Machine Learning Research 17(97), 1–37 (2016)
  * [16] Luo, C., Zhu, Y., Jin, L., Wang, Y.: Learn to augment: Joint data augmentation and network optimization for text recognition. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 13746–13755 (2020)
  * [17] Markou, K., Tsochatzidis, L., Zagoris, K., Papazoglou, A., Karagiannis, X., Symeonidis, S., Pratikakis, I.: A convolutional recurrent neural network for the handwritten text recognition of historical greek manuscripts. In: International Workshop on Pattern Recognition for Cultural Heritage (PATRECH) (2020)
  * [18] Marti, U.V., Bunke, H.: The iam-database: an english sentence database for offline handwriting recognition. International Journal on Document Analysis and Recognition 5(1), 39–46 (2002)
  * [19] Michael, J., Labahn, R., Grüning, T., Zöllner, J.: Evaluating sequence-to-sequence models for handwritten text recognition. In: 2019 International Conference on Document Analysis and Recognition (ICDAR). pp. 1286–1293. IEEE (2019)
  * [20] Pham, V., Bluche, T., Kermorvant, C., Louradour, J.: Dropout improves recurrent neural networks for handwriting recognition. In: 2014 14th international conference on frontiers in handwriting recognition. pp. 285–290. IEEE (2014)
  * [21] Puigcerver, J.: Are multidimensional recurrent layers really necessary for handwritten text recognition? In: 2017 14th IAPR International Conference on Document Analysis and Recognition (ICDAR). vol. 1, pp. 67–72. IEEE (2017)
  * [22] Retsinas, G., Sfikas, G., Gatos, B.: Transferable deep features for keyword spotting. In: Multidisciplinary Digital Publishing Institute Proceedings. vol. 2, p. 89 (2018)
  * [23] Retsinas, G., Sfikas, G., Nikou, C.: Iterative weighted transductive learning for handwriting recognition. In: International Conference on Document Analysis and Recognition. pp. 587–601. Springer (2021)
  * [24] Retsinas, G., Sfikas, G., Nikou, C., Maragos, P.: Deformation-invariant networks for handwritten text recognition. In: 2021 IEEE International Conference on Image Processing (ICIP). pp. 949–953. IEEE (2021)
  * [25] Retsinas, G., Sfikas, G., Nikou, C., Maragos, P.: From Seq2Seq recognition to handwritten word embeddings. In: Proceedings of the British Machine Vision Conference (BMVC) (2021)
  * [26] Sudholt, S., Fink, G.A.: PHOCNet: A deep convolutional neural network for word spotting in handwritten documents. In: Proceedings of the $15^{th}$ International Conference on Frontiers in Handwriting Recognition (ICFHR). pp. 277–282 (2016)
  * [27] Sueiras, J., Ruiz, V., Sanchez, A., Velez, J.F.: Offline continuous handwriting recognition using sequence to sequence neural networks. Neurocomputing 289, 119–128 (2018)
  * [28] Tassopoulou, V., Retsinas, G., Maragos, P.: Enhancing handwritten text recognition with n-gram sequence decomposition and multitask learning. In: 2020 25th International Conference on Pattern Recognition (ICPR). pp. 10555–10560. IEEE (2021)
  * [29] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L., Polosukhin, I.: Attention is all you need. In: Advances in neural information processing systems. pp. 5998–6008 (2017)
  * [30] Wick, C., Zöllner, J., Grüning, T.: Transformer for handwritten text recognition using bidirectional post-decoding. In: International Conference on Document Analysis and Recognition. pp. 112–126. Springer (2021)
  * [31] Wigington, C., Stewart, S., Davis, B., Barrett, B., Price, B., Cohen, S.: Data augmentation for recognition of handwritten words and lines using a cnn-lstm network. In: 2017 14th IAPR International Conference on Document Analysis and Recognition (ICDAR). vol. 1, pp. 639–645. IEEE (2017)
  * [32] Yousef, M., Hussain, K.F., Mohammed, U.S.: Accurate, data-efficient, unconstrained text recognition with convolutional neural networks. Pattern Recognition 108, 107482 (2020)


# Inverse-Faraday effect from the orbital angular momentum of light

Vage Karakhanyan  vage.karakhanyan.femto-st.fr Clément Eustache
<EMAIL_ADDRESS>Yannick Lefier<EMAIL_ADDRESS>Thierry
Grosjean<EMAIL_ADDRESS>Optics Department – FEMTO-ST
Institute
UMR 6174 - University of Bourgogne Franche-Comté – CNRS - Besançon, France.

###### Abstract

It is usually admitted that the inverse Faraday effect (IFE) originates from
the spin angular momentum (SAM) of light. In this paper, we evidence that part
of the IFE in a metal is induced by the orbital angular momentum (OAM) of
light. On the basis of a hydrodynamic model of the conduction electron gas, we
describe the dependence of the IFE on the spin and orbital angular momenta as
well as spin-orbit interaction in a non-paraxial light beam. We also
numerically quantify the relative contributions of the SAM and OAM of light to
the IFE in a thin gold film illuminated by different focused beams carrying
SAM and/or OAM. The OAM of light provides a new degree of freedom in the
control of the IFE and resulting optomagnetic field, thus potentially
impacting various research fields including all-optical magnetization
switching and spin-wave excitation.

††preprint: APS/123-QED

##  I Introduction

Light is known to possess polarization and spatial degrees of freedom,
manifested by its linear momentum as well as spin and orbital angular
momenta[1]. Remarkably, the spin angular momentum (SAM) of light can be
transferred to electrons in matter, a phenomenon which refers to as the
inverse Faraday effect (IFE) [2, 3, 4]. The inverse Faraday effect (IFE) has
attracted much attention for its ability to generate light-induced
magnetization, thereby opening the prospect of an ultrafast magnetic data
storage [5, 6, 7] and a non-contact excitation of spin-waves [8, 9, 10, 11,
12]. Plasmonic nanostructures have recently been investigated to locally
enhance and control the IFE in non-magnetic metals [13, 14, 15, 16, 17, 18,
19, 20, 21, 22, 23] and in hybrid structures including magnetic materials [24,
25, 26, 27, 28, 29].

So far, the orbital counterpart of the spin-based IFE has been hardly
addressed. To our knowledge, an optomagnetism induced by the OAM of light has
not been reported yet. As a first evidence of the interaction of magnetism and
OAM of light, Sirenko et al showed a vortex beam dichroism in a magnetized
material at THz frequencies [30]. The OAM of light and its combination to the
optical SAM holds promise of new opportunities and strategies in tailoring
optomagnetic effects [31].

In this paper, we provide a spin and orbital angular momentum representation
of the IFE in a metal. In the case of axisymmetrical optical systems
(including vortex beams), we analytically show the role of the SAM and OAM of
light, as well as spin-orbit interaction (SOI), in the generation of an opto-
induced magnetization. We find that the SAM contributes to the IFE only in
non-paraxial optical beams (carrying noticeable longitudinal light fields),
whereas the OAM has a non negligible contribution even in the paraxial optical
regime. Finally, we numerically quantify the spin and orbital parts of the IFE
in a thin gold film under illumination with different focused beams carrying
SAM and/or OAM. We numerically confirm the importance of the SOI of light in
the IFE and resulting optomagnetic field, which manifests via SAM-to-OAM or
OAM-to-SAM conversions at focus.

##  II Model Description

In a hydrodynamic approach, the conduction electron dynamics in a metal can be
described from the Euler’s equation [32, 33]:

$m_{e}\frac{\partial\mathbf{v}}{\partial
t}+m_{e}(\mathbf{v}\cdot\mathbf{\nabla})\mathbf{v}=-\frac{m_{e}}{\tau}\mathbf{v}+e\mathbf{E}+\mu_{0}e\mathbf{v}\times\mathbf{H}-m_{e}\frac{\beta^{2}}{n}\nabla
n.$ (1)

where $m_{e}$, $n$, $\tau$ and $\mathbf{v}$ are the effective mass, the
conduction electron fluid density, the collision time and velocity of the
conduction electrons, respectively. $\mathbf{E}$ and $\mathbf{H}$ are the
applied electric and magnetic optical fields. The last term in Eq. (1) is due
to the electron gas pressure, with $\beta$ proportional to the Fermi velocity
$v_{F}$. $\mu_{0}$ and $e$ are the permeability of free space and the
elementary charge, respectively. $\mathbf{E}(\mathbf{r},t)$,
$\mathbf{H}(\mathbf{r},t)$, $\mathbf{v}(\mathbf{r},t)$, $n(\mathbf{r},t)$ are
time and space dependent variables. $n$ and $\mathbf{j}$ satisfy the
continuity equation:

$\mathbf{\nabla}\cdot\mathbf{j}=-e\frac{\partial n}{\partial t},$ (2)

where $\mathbf{j}=ne\mathbf{v}=\partial\mathbf{P}/\partial t$ is the current
density and $\mathbf{P}$ is the polarization density vector.

Eq. (2) can be rewritten as:

$n(\mathbf{r},t)=n_{0}-\dfrac{1}{e}\mathbf{\nabla}\cdot\mathbf{P},$ (3)

where $n_{0}$ is the background, equilibrium charge density in the absence of
any applied fields.

Assuming that the time variation of the conduction electron fluid density is
relatively small (i.e., $\dot{n}<<n$), the ratio $\dot{n}/n$ may be expanded
in powers of $1/(n_{0}e)$:

$\begin{split}\dfrac{\dot{n}(r,t)}{n(r,t)}=-\dfrac{1}{n_{0}e}\mathbf{\nabla}\cdot\dot{\mathbf{P}}\left(1-\dfrac{1}{n_{0}e}\mathbf{\nabla}\cdot\mathbf{P}\right)^{-1}\approx\\\
-\dfrac{1}{n_{0}e}\mathbf{\nabla}\cdot\dot{\mathbf{P}}\left(1+\dfrac{1}{n_{0}e}\mathbf{\nabla}\cdot\mathbf{P}\right).\end{split}$
(4)

Thus, we have:

$\dfrac{\dot{n}(r,t)}{n(r,t)}\approx-\dfrac{1}{n_{0}e}\mathbf{\nabla}\cdot\dot{\mathbf{P}}-\dfrac{1}{n^{2}_{0}e^{2}}(\mathbf{\nabla}\cdot\mathbf{P})(\mathbf{\nabla}\cdot\dot{\mathbf{P}}).$
(5)

By applying the above-described assumptions in Eq. (1), we find [33, 34]:

$\begin{split}\dfrac{\partial\mathbf{j}}{\partial
t}+\frac{\mathbf{j}}{\tau}=\frac{e^{2}n_{0}}{m_{e}}\mathbf{E}-e\beta^{2}\mathbf{\nabla}n-\frac{e}{m_{e}}\left(\mathbf{\nabla}\cdot\mathbf{P}\right)\mathbf{E}+\\\
+\frac{\mu_{0}e}{m_{e}}\mathbf{j}\times\mathbf{H}-\dfrac{1}{en_{0}}\left[\left(\mathbf{\nabla}\cdot\mathbf{j}\right)\mathbf{j}+(\mathbf{j}\cdot\mathbf{\nabla})\mathbf{j}\right]\end{split}$
(6)

.

##  III Time harmonic regime

In the time harmonic (i.e., monochromatic) regime, the electric and magnetic
optical fields become $\mathbf{E}=\mathbf{E_{\omega}}(\mathbf{r})e^{-i\omega
t}+c.c$ and $\mathbf{H}=\mathbf{H_{\omega}}(\mathbf{r})e^{-i\omega t}+c.c$,
where $\omega$ is the angular frequency, $t$ is time, $c.c$ is the complex
conjugate. To predict both the linear and nonlinear responses of the metal, we
solve Eq. (6) using a perturbation approach. To this end, the current density
$\mathbf{j}$ is written as:

$\mathbf{j}=(\mathbf{j_{\omega}}e^{-i\omega t}+c.c)+\mathbf{j_{NL}}.$ (7)

$\mathbf{j}_{\bm{\omega}}$ and $\mathbf{j_{NL}}$ are the linear and nonlinear
contributions to the current density, respectively. In the following, we focus
on the nonlinear optical process defined by the drift current density
$\mathbf{j_{d}}=\langle\mathbf{j_{NL}}\rangle$ where the operator
$\langle\rangle$ denotes time averaging. $\mathbf{j_{d}}$ originates from an
optical rectification process [35]. Because $\mathbf{j_{NL}}$ is small as
compared to $\mathbf{j}_{\bm{\omega}}$, the perturbation method turns Eq (6)
into the following couple of equations:.

$\mathbf{j_{\omega}}=\gamma_{\omega}\mathbf{E}_{\mathbf{\omega}}-e\beta^{2}\mathbf{\nabla}n,$
(8)

$\begin{split}\mathbf{j_{d}}=-\frac{\tau}{n_{0}e}\mathbf{Re}\left[\frac{\gamma_{0}i}{\omega\tau}(\nabla\cdot\mathbf{j}_{\omega})\mathbf{E^{*}_{\omega}}+\right.\left(\nabla\cdot\mathbf{j_{\omega}}\right)\mathbf{j^{*}_{\omega}}+\\\
+(\mathbf{j_{\omega}}\cdot\nabla)\mathbf{j^{*}_{\omega}}-\mu_{0}\gamma_{0}\mathbf{j_{\omega}}\times\mathbf{H_{\omega}^{*}}\left.\vphantom{\frac{\gamma_{0}i}{\omega\tau}}\right],\end{split}$
(9)

where

$\gamma_{\omega}=\dfrac{\gamma_{0}}{1-i\omega\tau},$ (10)

and

$\gamma_{0}=\dfrac{n_{0}e^{2}\tau}{m_{e}},$ (11)

are the linear (i.e., dynamic) and DC conductivities, respectively. This
system of equations describes both the linear and nonlinear (rectification)
responses of a material to an optical field. It can be numerically solved
using for instance the two-fluid plasma model solver [36]. However, these
solvers require high computational power and are time consuming.

##  IV Inverse Faraday effect

Assuming that the optical response of a metal is mainly driven by its
conduction electrons, the IFE can be described by the orbital magnetization
defined as:

$\mathbf{M}=\dfrac{1}{2V}\int_{V}\mathbf{L}dV,$ (12)

where:

$\begin{split}\mathbf{L}=\mathbf{r}\times\mathbf{j_{d}},\end{split}$ (13)

is the opto-induced local orbital angular momentum in the conduction electron
gas. In that case, the IFE directly depends on the opto-induced drift current
density in the metal.

###  IV.1 IFE in the metal bulk

The last terms of Eqs. (1) and (8) both describe nonlocal effects in the
metal. In the case of smooth and slowly varying charge densities $n$, it is
possible to neglect nonlocal effects (i.e., considering $\beta\rightarrow 0$)
and take into account only the local response [34]. This local response
approximation simplifies the resolution of Eqs. (8) and (9) [20, 37]. It is
however only valid within the metal bulk. The extension of the local
approximation method to metal surfaces will be addressed in section IV.2.

In the framework of a local response approximation, Eqs. (8) and (9) become:

$\mathbf{j_{\omega}}=\gamma_{\omega}\mathbf{E}_{\mathbf{\omega}},$ (14)

and

$\begin{split}\mathbf{j_{d}}=-\frac{\tau}{n_{0}e}\mathbf{Re}\left[\frac{i}{\omega\tau}(\nabla\cdot\mathbf{j}_{\omega})\mathbf{j^{*}_{\omega}}+(\mathbf{j_{\omega}}\cdot\nabla)\mathbf{j^{*}_{\omega}}\right.\\\
-\dfrac{\mu_{0}\gamma_{0}}{\tau}\mathbf{j_{\omega}}\times\mathbf{H_{\omega}^{*}}\left.\vphantom{\frac{\gamma_{0}i}{\omega\tau}}\right],\end{split}$
(15)

respectively. Using the following vector identity:

$\begin{split}(\mathbf{j_{\omega}}\cdot\nabla)\mathbf{j^{*}_{\omega}}&=\mathbf{j_{\omega}}\cdot\nabla\mathbf{j^{*}_{\omega}}-\mathbf{j_{\omega}}\times(\nabla\times\mathbf{j^{*}_{\omega}}),\\\
&=\mathbf{j_{\omega}}\cdot\nabla\mathbf{j^{*}_{\omega}}+i\omega\mu_{0}\gamma^{*}\mathbf{j_{\omega}}\times\mathbf{H_{\omega}^{*}},\end{split}$
(16)

Eq. (15) becomes:

$\begin{split}\mathbf{j_{d}}=&-\frac{\tau}{n_{0}e}\vphantom{\frac{\gamma_{0}i}{20}}\left(\frac{1}{\omega\tau}\mathbf{Im}\left[\vphantom{\frac{\gamma_{0}i}{20}}(\nabla\cdot\mathbf{j}_{\omega})\mathbf{j^{*}_{\omega}}\right]\right.+\\\
&+\mathbf{Re}\left[\vphantom{\frac{\gamma_{0}i}{20}}\mathbf{j_{\omega}}\cdot\nabla\mathbf{j^{*}_{\omega}}\right]-\dfrac{\mu_{0}|\gamma_{\omega}|^{2}}{\tau}\mathbf{Re}\left[\mathbf{E_{\omega}}\times\mathbf{H_{\omega}^{*}}\left.\vphantom{\frac{\gamma_{0}i}{\omega\tau}}\right]\right).\end{split}$
(17)

When the spatial variation of $\gamma_{\omega}$ are small enough to be
neglected, Eq. (17) reads:

$\begin{split}\mathbf{j_{d}}=&-\frac{|\gamma_{\omega}|^{2}}{n_{0}e}\vphantom{\frac{\gamma_{0}i}{20}}\left(\frac{1}{\omega}\mathbf{Im}\left[\vphantom{\frac{\gamma_{0}i}{20}}(\nabla\cdot\mathbf{E}_{\omega})\mathbf{E^{*}_{\omega}}\right]\right.+\\\
&+\tau\mathbf{Re}\left[\vphantom{\frac{\gamma_{0}i}{20}}\mathbf{E_{\omega}}\cdot\nabla\mathbf{E^{*}_{\omega}}\right]-\mu_{0}\mathbf{Re}\left[\mathbf{E_{\omega}}\times\mathbf{H_{\omega}^{*}}\left.\vphantom{\frac{\gamma_{0}i}{\omega\tau}}\right]\right).\end{split}$
(18)

Using vector identities and Maxwell’s equations, Eq. (18) can be rewritten as:

$\begin{split}\mathbf{j_{d}}=\dfrac{|\gamma_{\omega}|^{2}}{n_{0}e\omega}\left(\vphantom{\frac{1}{2}}-\dfrac{\tau\omega}{2}\nabla\left(|\mathbf{E_{\omega}}|^{2}\right)+\mathbf{Im}\left[\mathbf{E_{\omega}^{*}}\cdot\nabla\mathbf{E_{\omega}}\right]\right.\\\
+\left.\dfrac{1}{2}\nabla\times\mathbf{Im}[\mathbf{E^{*}_{\omega}}\times\mathbf{E_{\omega}}]\right)\end{split}$
(19)

According to Refs. [38, 39], the last two terms of Eq. (19) represent the
orbital and spin parts of the time-averaged Poynting vector
$\mathbf{\Pi}=\mathbf{Re}\left[\mathbf{E_{\omega}}\times\mathbf{H_{\omega}^{*}}\right]$,
respectively. We have:

$\mathbf{\Pi}=\mathbf{\Pi^{orb}}+\mathbf{\Pi^{spn}},$ (20)

where
$\mathbf{\Pi^{orb}}=\mathbf{Im}\left[\mathbf{E_{\omega}^{*}}\cdot\nabla\mathbf{E_{\omega}}\right]$
and $\mathbf{\Pi^{spn}}=\nabla\times\mathbf{S}$. Vector
$\mathbf{S}=\mathbf{Im}[\mathbf{E_{\omega}^{*}}\times\mathbf{E_{\omega}}]$ is
proportional to the SAM density of light [40, 38, 41, 42]. The opto-induced
drift current density in the metal bulk becomes:

$\mathbf{j_{d}}=\mathbf{j^{ig}_{d}}+\mathbf{j^{orb}_{d}}+\mathbf{j^{spn}_{d}},$
(21)

where,

$\mathbf{j^{ig}_{d}}=-\dfrac{\tau|\gamma_{\omega}|^{2}}{2n_{0}e}\nabla|\mathbf{E_{\omega}}|^{2},$
(22)
$\mathbf{j^{orb}_{d}}=\dfrac{|\gamma_{\omega}|^{2}}{n_{0}e\omega}\mathbf{\Pi^{orb}},$
(23)
$\mathbf{j^{spn}_{d}}=\dfrac{|\gamma_{\omega}|^{2}}{2n_{0}e\omega}\mathbf{\Pi^{spn}}=\dfrac{|\gamma_{\omega}|^{2}}{2n_{0}e\omega}\nabla\times\mathbf{S}.$
(24)

The opto-induced drift current density thus combines three source terms (Eq.
(21)). Term $\mathbf{j^{ig}_{d}}$ is linked to the so-called intensity-
gradient force [43, 44] that is central in optical tweezing applications [45].
The other two source terms are related to the orbital and spin parts of the
Poynting vector.

In the monochromatic regime, the linear momentum of light is proportional to
the time averaged Poynting vector [38, 40]. Eqs. (23) and (24) thus describe
momentum transfer from light to the conduction electrons in a metal, leading
to an optical drag effect [46]. $\mathbf{\Pi^{orb}}$ and $\mathbf{\Pi^{spn}}$
are thus proportional to the canonical (orbital) and spin momentum densities
of light, respectively. When vector-multiplied by $\mathbf{r}$, the source
terms of Eqs (23) and (24) give the orbital and spin contributions to the
angular momentum $\mathbf{L}$ of the conduction electron gas, respectively
[38, 39]. The local angular momentum (Eq. (13)) and resulting orbital
magnetization (Eq. (12)) of the conduction electrons then read:

$\begin{split}\mathbf{L}=\mathbf{r}\times\mathbf{j_{d}^{ig}}+\mathbf{r}\times\mathbf{j_{d}^{orb}}+\mathbf{r}\times\mathbf{j_{d}^{spn}},\end{split}$
(25)

and

$\mathbf{M}=\mathbf{M^{ig}}+\mathbf{M^{orb}}+\mathbf{M^{spn}},$ (26)

respectively. $\mathbf{M^{ig}}$, $\mathbf{M^{orb}}$ and $\mathbf{M^{spn}}$ are
the intensity-gradient, spin and orbital contributions to the IFE,
respectively. We have
$\mathbf{M^{id}}=1/(2V)\int_{V}\mathbf{r}\times\mathbf{j_{d}^{id}}dV$, where
”$\mathbf{id}$” stands for ”$\mathbf{ig}$”, ”$\mathbf{orb}$” and
”$\mathbf{spn}$”. Therefore, the IFE does not solely originate from the SAM of
light. Eq. (26) shows that part of the IFE in the metal bulk relies on the
transfer of OAM from a light beam to the conduction electron gas of a metal.
It appears from Eqs. (23) and (26) that the orbital part of the IFE relies on
an axis-symmetrical optical drag effect induced and controlled by the OAM of
light.

###  IV.2 IFE at metal surfaces

As previously stated, the local approximation approach used to analytically
solve Eqs. (8) and (9) is only valid within the metal bulk. To overcome
ambiguities of this simplified model at metal interfaces (where strong
variations of the electron fluid density occur), an analytical method has been
proposed to describe opto-induced surface currents [47, 34, 48, 49, 22, 23].
This method brings a non-local correction to the local response approximation
at metal surfaces.

The idea is to define a thin metal layer beneath interfaces, whose thickness
matches Thomas-Fermi length ($\lambda_{TF}\simeq$ 0.1 nm for noble metals).
This layer is considered to be a ”surface layer”, where the electron gas
pressure and the spatial variations of the conductivity are not negligible.
Out of this layer, in the metal bulk, the local model applies
($\beta\rightarrow 0$). Within the interface layer, the component
$j_{\omega}^{T}$ of the linear current density that is locally parallel to the
surface preserves whereas the normal component $j_{\omega}^{N}$ decays to
zero. This additional boundary condition on $j_{\omega}^{N}$, which is
required to solve the nonlocal problem, is attributed to a neglected electron
”spill-out” at interfaces [34].

We define $\rho$ as the spatial coordinate normal to the surfaces so that the
metal bulk is located at $\rho<0$ and the surface layer corresponds to
$0<\rho<\lambda_{TF}$. In that case, we have in the surface layer
$j_{\omega}^{T}(\rho)\approx j_{\omega}^{T}(0^{-})$ and
$j_{\omega}^{N}=j_{\omega}^{N}(0^{-})\sigma(\rho)$, where $\sigma$ is a
decaying function defined by
$\int_{0}^{\lambda_{TF}}\sigma^{\prime}(\rho)d\rho=-1$ [34]. $\sigma^{\prime}$
is the derivative with respect to $\rho$. The $\sigma(\rho)$ function has the
following useful properties:

$\int_{0}^{\lambda_{TF}}\sigma(\rho)^{m}\sigma^{\prime}(\rho)d\rho=-\dfrac{1}{m+1},$
(27)

$\int_{0}^{\lambda_{TF}}\sigma(\rho)^{m}d\rho=0.$ (28)

This non-local correction is applied by redefining the linear conductivity of
the metal as a tensor. The linear current density (cf. Eq. (14)) then becomes:

$\mathbf{j_{\omega}}=\left[\begin{array}[]{ccc}\gamma_{\omega}&0&0\\\
0&\gamma_{\omega}&0\\\
0&0&\gamma_{\omega}\sigma(\rho)\end{array}\right]\left[\begin{array}[]{c}E_{1}^{T}\\\
E_{2}^{T}\\\ E^{N}\end{array}\right]$ (29)

where $E_{1}^{T}$, $E_{2}^{T}$ and $E^{N}$ are the vector components of the
optical electric field that are transverse and normal to the interfaces.
Finally, the opto-induced drift current density at metal surfaces is deduced
by replacing $\mathbf{j_{\omega}}$ in Eq. (17) by its expression given in Eq.
(29).

##  V IFE in axisymmetrical optical systems

We now focus on axisymmetrical optical systems where both the metal structure
and light intensity are axially symmetric. In that case, the IFE mainly
involves a magnetization oriented along the symmetry axis [20, 37, 22]. The
azimuthal component $\left[j_{d}\right]_{\xi}$ of the drift current density
(in the cylindrical coordinates $(r,\xi,z)$) then becomes the main contributor
to the IFE (see Eqs. (12) and (13)).

Since $\nabla_{\xi}|\mathbf{E_{\omega}}|^{2}$=0 in light beams showing axis-
symmetrical intensity, $\mathbf{j_{d}^{ig}}$ has no contribution to the IFE in
axis-symmetrical optical systems (i.e., $[j_{d}^{ig}]_{\xi}=0$). Only the
orbital and spin parts of the opto-induced drift current density are now
involved (cf. Eqs. (23) and (24), respectively). The angular momentum of the
conduction electron gas now reads:

$\begin{split}\mathbf{L}&=\mathbf{r}\times\mathbf{j_{d}^{orb}}+\mathbf{r}\times\mathbf{j_{d}^{spn}},\\\
&=\dfrac{|\gamma_{\omega}|^{2}}{n_{0}e\omega}\left[\mathbf{r}\times\mathbf{\Pi^{orb}}+\mathbf{r}\times\mathbf{\Pi^{spn}}\right].\end{split}$
(30)

In the following, we consider the incoming light to be a Laguerre-Gauss vortex
beam described by a polarization helicity $s$ ($-1\leq s\leq 1$) and a
topological charge $l$ [50, 51, 52], respectively. Laguerre-Gauss beams
fulfill the cylindrical symmetry condition required to cancel the contribution
of the intensity-gradient source term $\mathbf{j_{d}^{ig}}$.

###  V.1 IFE in the metal bulk

From Eq. (19), we find that the azimuthal component of the drift current
density within the metal bulk reads:

$\displaystyle\left[j_{d}\right]_{\xi}^{bulk}$ $\displaystyle=$
$\displaystyle\kappa\left[\dfrac{l+s}{r}|\mathbf{E_{\omega}}|^{2}-\dfrac{2}{r}S_{z}\right]+\kappa\left[\dfrac{\partial
S_{r}}{\partial z}-\dfrac{\partial S_{\xi}}{\partial r}\right]$ (31)
$\displaystyle=$
$\displaystyle\left[j^{orb}_{d}\right]_{\xi}^{bulk}+\left[j^{spn}_{d}\right]_{\xi}^{bulk},$
(32)

where $\kappa=|\gamma_{\omega}|^{2}/(n_{0}e\omega)$. $S_{r}$, $S_{\xi}$,
$S_{z}$ are the components of the SAM density $\mathbf{S}$ of light in
cylindrical coordinates. We have $S_{r}=\mathbf{Im}[E_{\xi}^{*}E_{z}]$,
$S_{\xi}=\mathbf{Im}[E^{*}_{z}E_{r}]$ and
$S_{z}=\mathbf{Im}[E_{r}^{*}E_{\xi}]$.

The first term of Eq. (31), namely:

$\displaystyle\left[j^{orb}_{d}\right]_{\xi}^{bulk}$ $\displaystyle=$
$\displaystyle\kappa\left[\dfrac{l+s}{r}|\mathbf{E_{\omega}}|^{2}-\dfrac{2}{r}S_{z}\right]$
(33) $\displaystyle=$
$\displaystyle\dfrac{\kappa}{r}\left(l|\mathbf{E_{\omega}}|^{2}+\left[s|\mathbf{E_{\omega}}|^{2}-2S_{z}\right]\right)$
(34)

refers to the azimuthal component of $\mathbf{j^{orb}_{d}}$ (cf. Eq. (23)).
This part of the opto-induced drift current density evidences the contribution
to the IFE of the OAM of the light within the metal.

The second term of Eq. (31):

$\left[j^{spn}_{d}\right]_{\xi}^{bulk}=\kappa\left[\dfrac{\partial
S_{r}}{\partial z}-\dfrac{\partial S_{\xi}}{\partial r}\right].$ (35)

describes the spin part of the IFE.

Therefore, in an axis-symmetrical optical problem, our model suggests that the
IFE in the metal bulk relies on both the SAM and OAM of light. From Eq. (34),
the OAM manifests itself via the topological charge $l$ of the incoming vortex
beam and as an intermediate between the SAM of light and the OAM of the
electron gas, via optical SOI (see section V.3).

###  V.2 IFE at metal surfaces

In the following, we consider the opto-induced drift current density at the
surfaces perpendicular to (0z). We thus have $\rho=\pm z$, where $\rho$ is
defined in section (IV.2). From section (IV.2) and Eqs. (33) and (35), the
orbital and spin parts of the azimuthal drift current density at metal
surfaces read:

$\left[j_{d}^{orb}\right]_{\xi}^{surf}=\kappa\left(\dfrac{l+s}{r}|\mathbf{E_{\parallel}}|^{2}-\dfrac{2}{r}S_{z}(0^{-})\right),$
(36)

and

$\left[j_{d}^{spn}\right]_{\xi}^{surf}=\kappa\left(\dfrac{\partial\sigma(\rho)S_{r}(0^{-})}{\partial\rho}-\dfrac{\partial\sigma(\rho)S_{\xi}(0^{-})}{\partial
r}\right),$ (37)

respectively. We have
$|\mathbf{E_{\parallel}}|^{2}=E_{r}(0^{-})E^{*}_{r}(0^{-})+E_{\xi}(0^{-})E^{*}_{\xi}(0^{-})$.
The definition of the optical electric field within the surface layer follows
that of the linear current density (see section IV.2). We have
$E_{\omega}^{T}(\rho)\approx E_{\omega}^{T}(0^{-})$ and
$E_{\omega}^{N}(\rho)=E_{\omega}^{N}(0^{-})\sigma(\rho)$.

We now define $i^{s}_{\xi}$ as the linear density of the surface current. We
have:

$i^{s}_{\xi}=\int_{0}^{\lambda_{TF}}\left[j_{d}^{spn}\right]_{\xi}^{surf}+\left[j_{d}^{orb}\right]_{\xi}^{surf}d\rho$
(38)

Using Eqs. (27) and (28), we find:

$\begin{split}i^{s}_{\xi}=\kappa\left[\dfrac{l+s}{r}|\mathbf{E_{\parallel}}|^{2}\lambda_{TF}-\dfrac{2\lambda_{TF}}{r}S_{z}(0^{-})-S_{r}(0^{-})\right].\end{split}$
(39)

The terms containing $\lambda_{TF}$ being negligible, we finally obtain:

$i^{s}_{\xi}\simeq-\kappa S_{r}(0^{-}),$ (40)

where $S_{r}(0^{-})=\mathbf{Im}[E_{\xi}^{*}(0^{-})E_{z}(0^{-})]$. At metal
surfaces, the IFE is mainly driven by the radial component of the SAM of
light.

###  V.3 Paraxial approximation, spin-orbit interaction

In the paraxial approximation, the optical field is considered to be purely
transverse ($E_{z}=0$) and the SAM of light reduces to
$S_{z}=s/2|\mathbf{E_{\omega}}|^{2}$ ($S_{r}=S_{\xi}=0$) [40]. In that case,
the second term (in brackets) of Eq. (34) vanishes, leading to:

$\left[j^{orb}_{d}\right]_{\xi}^{bulk}=\dfrac{\kappa}{r}l|\mathbf{E_{\omega}}|^{2}.$
(41)

The second term of Eq. (34) thus analytically describes the spin-dependence of
the OAM of light in non-paraxial beams, by virtue of SOI (SAM-to-OAM
conversion) [53, 54]. We will numerically see in section VI that SOI (OAM-to-
SAM conversion) is also inherent in the spin part of the IFE (cf. Eq. (35)).

The spin part of the IFE being dependent on $S_{\rho}$ and $S_{\xi}$ (see Eqs.
(35) and (40)), the contribution of the SAM of light to the IFE vanishes in
the paraxial optical regime. According to our model, the IFE induced by
paraxial optical beams is solely driven by the OAM of light, which manifests
via the topological charge $l$.

Such a property is consistent with the fact that in the paraxial
approximation, the orbital momentum of light (proportional to $\Pi^{orb}$; at
the origin of the OAM) is the only observable contribution to the optical
momentum [38, 40]. The orbital momentum is here proportional to the wave
vector of the optical field and is solely responsible for energy
transport[38]. The spin momentum (proportional to $\mathbf{\Pi^{spn}}$; at the
origin of the SAM) represents a solenoidal current, which does not contribute
to energy transport [38, 40]. In paraxial approximation, the spin momentum has
no observable contribution to the IFE in a metal.

##  VI Optomagnetism in a thin gold layer

To estimate the relative contributions of the SAM and OAM in the IFE and
resulting optomagnetic field, we simulate the optomagnetic response of a thin
gold film under illumination with single focused light beams carrying SAM
and/or OAM.

###  VI.1 Design and theory

The configuration under study is shown in Fig. 1. A 20-nm thick gold layer
lies on a semi-infinite glass substrate. An incoming vortex beam of
topological charge $l$, polarization helicity $s$, and of Gaussian or
Laguerre-Gaussian profile is focused onto the backside of the gold film, in
the substrate. The $1/e$ width of the beam waist coincides with the pupil
diameter of the microscope objective. Operating in the immersion regime, the
objective shows a numerical aperture $NA$ of 1.3. On the basis of the theory
established by Richards and Wolf [55, 56], the optical electric field at focus
can be written as:

$\textbf{E}(r,\xi,z)=-\frac{ik_{1}f\exp[-ik_{1}f]}{2\pi}\frac{1}{\sqrt{n_{1}}}\times\int^{\theta_{M}}_{0}\cos^{\frac{1}{2}}(\theta)\sin(\theta)F(\theta)\int^{2\pi}_{0}\textbf{e}(\theta,\psi,z)\exp\left[i\upalpha
r\cos(\psi-\xi)\right]d\theta d\psi,$ (42)

where $(r,\xi,z)$ are cylindrical coordinates, $f$ is the focal length of the
microscope objective, $\theta$ and $\psi$ are directional angles and
$\upalpha$ is a function of $\theta$. We have $\theta_{M}=\arcsin(NA/n_{1})$,
where $n_{1}$ is the refractive index of the substrate. The (0z)-axis matches
the symmetry axis of the microscope objective and is perpendicular to the
metal surfaces (see Fig. 1).

Vector $\textbf{e}(\theta,\psi,z)$ takes the form:

$\textbf{e}(\theta,\psi,z)=\exp[il\psi]\begin{pmatrix}E_{s}t_{s}(z)\sin\psi-
E_{p}t^{r}_{p}(z)\cos\theta\cos\psi\\\ -E_{s}t_{s}(z)\cos\psi-
E_{p}t_{p}^{r}(z)\cos\theta\sin\psi\\\
E_{p}t_{p}^{z}(z)\sin\theta\end{pmatrix}$ (43)

For circular polarization ($s=\pm 1$), we have:

$\displaystyle E_{s}$ $\displaystyle=$ $\displaystyle-
is(\sqrt{2}/2)\exp[is\psi],$ (44) $\displaystyle E_{p}$ $\displaystyle=$
$\displaystyle-\;(\sqrt{2}/2)\exp[is\psi],$ (45)

The apodization function at the exit pupil plane of the microscope objective
reads:

$F(\theta)=\frac{2}{w_{0}}\sqrt{\frac{Z_{0}P_{0}}{\pi|l|!}}\left(\frac{\sqrt{2}f\sin\theta}{w_{0}}\right)^{|l|}\exp\left[\frac{-f^{2}\sin^{2}\theta}{w_{0}^{2}}\right],$
(46)

with :

$\displaystyle w_{0}$ $\displaystyle=$ $\displaystyle f\sin\theta_{M},\hskip
35.56593pt\text{if $l$=0,}$ (47) $\displaystyle w_{0}$ $\displaystyle=$
$\displaystyle\frac{f\sin\theta_{M}}{(2|l|)^{\frac{1}{2}}},\hskip
31.2982pt\text{ if $|l|\geq$1.}$ (48)

$P_{0}$ and $w_{0}$ are the power and $1/e$ width of the incoming beam and
$Z_{0}$ is the vacuum impedance.

Figure 1: Schematic diagram of the focusing system which involves a microscope
objective of circular pupil.

In the case of a radially-polarized vector vortex beams of the first order
($l=1$, $s=0$), the optical field at focus is defined by:

$E_{p}=1\text{, }E_{s}=0,$ (49)

and we assume the Gaussian apodization function at the pupil plane given by
Eq. (46) for $l$=0.

For all the above-described vortex beams, the optical magnetic field is
calculated by replacing $\textbf{e}(r,\xi,z)$ by
$\textbf{h}(r,\xi,z)=\textbf{k}\times\textbf{e}(r,\xi,z)/\omega\mu_{0}$ in Eq.
(42).

Predicting the IFE in a metallic structure requires the calculation of the
optical field inside the metal (see Eqs. (19) and (40)). In the basic
configuration treated here, the optical field in the metal layer is described
by coefficients $t_{s}$, $t_{p}^{r}$ and $t_{p}^{z}$ under the form
$C^{+}\exp[iw_{2}z]+C^{-}\exp[-iw_{2}z]$ where $w_{2}$ is the component of the
wave vector normal to the surfaces (i.e., along (0z)). Coefficients $C^{+}$
and $C^{-}$, which are obtained by applying boundary conditions of the optical
fields onto the film interfaces, can be found for instance in Ref. [57]. The
dispersion properties of gold at $\lambda$=800nm are defined by the Drude
model with $\varepsilon_{r}=-24.76+0.88i$ and $\gamma_{0}=1.5718\cdot
10^{7}$S.m-1. From the calculated optical field in the metal, we anticipate
the distribution of optically-induced drift current density using the
formalism detailed in section V.

###  VI.2 OAM and SAM-driven optomagnetism

In the following, we consider four different light beams carrying SAM or OAM,
or a combination of SAM and OAM. We examine the circularly polarized beam
($l=0,s=1$), the radially polarized vortex beam ($l=1,s=0$) leading for
instance to optical skyrmions [58], and two circularly polarized vortex beams
of opposite topological charges ($l=\pm 1,s=1$) [59, 60]. The incoming light
waves are characterized by a maximum intensity of $10^{12}$W.cm-2 at focus and
a wavelength ($\lambda$) of 800 nm. The waist of the incident beam coincides
with the pupil plane of the microscope objective. Its $1/e$-width matches the
pupil diameter.

Fig. 2 shows cross-sections of the bulk and surface currents generated in the
metal film under illumination with the above-described light beams. The drift
currents are calculated by numerically integrating the azimuthal components of
the bulk and surface current densities (cf. Eqs. (31) and (40), respectively )
over areas of 0.1$\times$0.1 nm2. The optomagnetic fields originating from the
SAM and OAM-driven loops of drift current are represented in Fig. 3 together
with the overall optomagnetic field.

Figure 2: Opto-induced current loops (a-d) at the bottom surface and (e-p) in
the bulk of a thin gold film illuminated with (a,e,i,m) a circularly polarized
light beam ($s$=1, $l$=0), (b,f,j,n) a radially polarized vortex beam ($s$=0,
$l$=1), (c,g,k,o) a circularly polarized vortex beam ($s$=1, $l$=1), and
(d,h,l,p) the same circularly polarized vortex with an opposite topological
charge ($s$=1, $l$=-1). The helicity of each beam is shown with the $l$ and
$s$ numbers on top of each column of the figure. The bottom surface
((xOy)-plane, see (d)) of the thin gold film matches the focal plane of the
incoming light. The orientation of the surface currents in (a-d) is
represented with white arrows. Eq. (37) is used to calculate the surface
currents as it is assumed to be solely driven by the SAM of light (cf. Table
1). Currents in the metal bulk are represented along the longitudinal
(x0z)-plane (see (e)) and are decomposed in their (e-h) spin and (i-l) orbital
parts (cf. Eqs. (35) and (33), respectively). The overall bulk current
combining spin and orbital contributions is shown as well in (m-p) (cf. Eq.
(32)). The color code used to represent these out-of-plane currents is defined
in (e). All currents are given in nA.

SOI [61] is known to tailor the helicity of focused light. As an example, in
the focal region of a circularly polarized beam, the SAM is partly converted
into OAM [60]. A reciprocal OAM-to-SAM conversion occurs at the focus of
vector vortex beams [62, 58]. An immediate consequence of the SOI of light on
the IFE is that opto-induced drift currents show both orbital and spin
components even when the incoming beam solely carries SAM or OAM. We see from
Figs. 2(e) and 2(i) for the circularly polarized beam (carrying SAM) and Figs.
2(f) and 2(j) for the radially polarized vortex beam (carrying OAM) that in
the metal bulk both the orbital and spin parts of the drift currents are non
negligible. The SAM dependence of the orbital part is analytically evidenced
in Eqs. (33) and (36), whereas the OAM dependence of the spin part is
contained in the spatial derivatives of Eqs. (35) and (37). As anticipated in
Eq. (40), the orbital part of the drift current at metal surfaces is
negligible. This property is numerically verified in Table 1 using Eq. (39).

The effect of the SOI in the IFE is also visible in the case of circularly
polarized vortex beams (carrying both SAM and OAM; cf. the two last columns of
Fig. 2). By comparing Figs. 2(c) and 2(d), and Figs. 2(k) and 2(l), we see
that the spin part of the drift current in the metal bulk and at metal
surfaces is reversed and shows a different morphology when the sign of the
topological charge $l$ (i.e. the OAM) of the incoming beam is flipped while
the polarization helicity $s$ (i.e., the SAM) remains unchanged.

To quantify the relative contributions of the SAM and OAM of light to the IFE,
we consider coefficients $\beta_{spn}$ and $\beta_{orb}$, respectively,
defined as:

$\beta_{spn}=\dfrac{\mu_{z}^{spn}}{\mu_{z}^{spn}+\mu_{z}^{orb}},$ (50)

$\beta_{orb}=\dfrac{\mu_{z}^{orb}}{\mu_{z}^{spn}+\mu_{z}^{orb}}.$ (51)

$\mu_{z}^{spn}$ and $\mu_{z}^{orb}$ are the $z$-component of vectors
$\bm{\mu}^{\mathbf{spn}}$ and $\bm{\mu}^{\mathbf{orb}}$ defined as being
derived from the spin and orbital parts of the opto-induced magnetization
($\mathbf{M^{spn}}$ and $\mathbf{M^{orb}}$, respectively).

We first focus on the ability of the SAM and OAM of light to create local
orbital angular momentum in the electron gas (e-OAM), i.e., to move conduction
electrons in the form of drift current loops. To this end, we define

$\bm{\mu}^{\mathbf{id}}=\dfrac{1}{2V}\int_{V}\mathbf{r}\times|\mathbf{j_{d}^{id}}|dV,$
(52)

with
$|\mathbf{j_{d}^{id}}|=|[\mathbf{j^{id}_{d}}]^{\mathbf{{surf}}}|+|[\mathbf{j^{id}_{d}}]^{\mathbf{bulk}}|$,
”$\mathbf{id}$” stands for ”$\mathbf{orb}$” or ”$\mathbf{spn}$”. Here,
$\mu_{z}=\mu_{z}^{spn}+\mu_{z}^{orb}$ quantifies the total amount of opto-
induced e-OAM. We thus avoid angular momentum compensation effects that could
be induced by current loops of opposite handedness to focus on how much of the
total opto-induced e-OAM comes from the SAM and OAM of light.

From these considerations, it appears that the OAM of light is the main source
of e-OAM in the metal bulk for three of the four helicities of the incoming
light (see Table 1). In contrast, the SAM of light is the main contributor to
e-OAM at metal surfaces (calculated from Eq. (39)), which confirms the
simplification made in Eq. (40). Table 1 also shows that the overall e-OAM
(combining surface and bulk contributions) is mainly driven by the SAM of
light. The existence of a non-negligible OAM-driven magnetization (SAM-driven
magnetization, respectively) for an incoming beam solely carrying SAM (OAM,
respectively) highlights the crucial role of the SOI of light in the IFE. In
the case of the radially polarized vortex beam solely carrying OAM ($l=1$,
$s=0$), the contribution of the SAM to the total generated e-OAM is even three
times as high as the contribution of the OAM. OAM-to-SAM conversion in such a
beam has already been evidenced in Ref [58], leading to the concept of an
optical skyrmion. When the incoming beam carries both OAM and SAM ($l=\pm 1$
and $s=1$), the OAM and SAM contributions to the overall e-OAM are almost
balanced.

|  | l=0,s=1 | l=1,s=0 | l=1,s=1 | l=-1,s=1
---|---|---|---|---|---
Bulk | $\beta_{spn}$ | 73% | 55% | 45% | 55%
| $\beta_{orb}$ | 27% | 45% | 55% | 45%
Surface | $\beta_{spn}$ | 99.8% | 99.5% | 99.3% | 99.2%
| $\beta_{orb}$ | 0.2% | 0.5% | 0.7% | 0.8%
Overall | $\beta_{spn}$ | 88% | 76% | 69% | 71%
| $\beta_{orb}$ | 12% | 24% | 31% | 29%
Table 1: $\beta_{spn}$ and $\beta_{orb}$ coefficient for four different
combinations of the $l$ and $s$ parameters defining the incoming beam entering
the microscope objective. $\beta_{spn}$ and $\beta_{orb}$ are defined from
Eqs. (50) and (51), where $\bm{\mu}^{\mathbf{orb}}$ and
$\bm{\mu}^{\mathbf{spn}}$ are given in (52) (total amount of e-OAM).

We now study the observable magnetization associated to the IFE. $\bm{\mu}$ is
then defined as the amplitude of the magnetization.:

$\bm{\mu}^{\mathbf{id}}=|\mathbf{M}|=\dfrac{1}{2V}\left|\int_{V}\mathbf{r}\times\mathbf{j_{d}^{id}}dV\right|,$
(53)

”$\mathbf{id}$” stands for ”$\mathbf{orb}$” or ”$\mathbf{spn}$”. The e-OAM
compensation effects induced by current loops of opposite handedness are now
taken into account. We see for instance by comparing Figs. 2 (b-d) and 2
(j-l)) that the SAM-driven drift currents at metal surfaces and bulk can be of
opposite handedness.

The SAM and OAM contributions to the overall IFE (combining surface and bulk
contributions) are given in Table 2. We find that except for an incoming
circularly polarized beam ($l=0$, $s=1$), the contribution of the OAM of light
dominates the IFE. The spin part of the IFE undergoes compensation effects due
to surface and bulk currents of opposite handedness, which reduces the
resulting optomagnetization. The SAM of light thus induces a larger amount of
e-OAM, but due to compensation effects between surfaces and bulk contributions
(electrons move in opposite directions), the SAM of light can be less
efficient than its orbital counterpart to generate an observable
magnetization. This surface-to-bulk compensation effect can be almost perfect,
leading to a near zero contribution of the SAM to the IFE (as observed from
Tables 1 and 2 with the radially polarized vortex beam ($l=1$, $s=0$)).

| l=0,s=1 | l=1,s=0 | l=1,s=1 | l=-1,s=1
---|---|---|---|---
$\beta_{spn}$ | 78% | 0.2% | 36% | 49%
$\beta_{orb}$ | 22% | 99.8% | 64% | 51%
Table 2: $\beta_{spn}$ and $\beta_{orb}$ coefficient for four different
combinations of the $l$ and $s$ parameters defining the incoming beam entering
the microscope objective. $\beta_{spn}$ and $\beta_{orb}$ are defined in Eqs.
(50) and (51), where $\bm{\mu}^{\mathbf{orb}}$ and $\bm{\mu}^{\mathbf{spn}}$
are given in (53) (observable magnetization). Figure 3: Optically induced
static magnetic field in a (x0z)-plane perpendicular to the metal surfaces.
The black arrows show the local orientation of the magnetic field. (a-d)
Optomagnetic field (in linear scale) from the contribution of the OAM of light
to the IFE. (e-h) Optomagnetic field (in logarithmic scale) from the
contribution of the SAM of light to the IFE. (i-l) Overall optomagnetic field
(in logarithmic scale). In all figures, the horizontal yellow lines are used
to localize the 20nm thick gold layer. The gold film is illuminated from the
substrate (see indications in (a)).

The resulting optomagnetic field is calculated from the drift currents shown
in Fig. 2 using Biot and Savart law. We show in Fig. 3 the optomagnetic field
generated from the spin and orbital parts of the drift currents (Figs. 3(a-d)
and 3(e-h), respectively) as well as the overall magnetic field combining
these two parts (Figs.3(i-l)). Figs. 3(i-l) reveal that the distribution of
the overall optomagnetic field strongly depends on the OAM of the incoming
beam. When $l=1$ and $s=1$, the optomagnetic field forms a ring-like pattern
and points upward at the center of the light beam (Fig. 3(k)). The amplitude
of the static magnetic field reaches 1.2 mT. When the sign of $l$ is reversed,
the optomagnetic field is both flipped and confined at the beam center (Fig.
3(i)) and it peaks at 1.8 mT. The topological charge (i.e., OAM) of the
collimated light entering the microscope objective thus provides a new degree
of freedom in the control of the optomagnetism.

##  VII Conclusion

On the basis of a hydrodynamic model of the conduction electron gas, we give a
spin and orbital angular momentum representation of the IFE in a metal. Both
SAM and OAM contributions to the IFE in the metal bulk rely on an optical drag
effect [46, 63]: the underlying opto-induced current densities are
proportional to the Poynting energy flow (optical momentum) inside the metal.
In the case of an axisymmetrical optical system, we show a direct
proportionality between the opto-induced drift current at metal surfaces and
the radial component of the SAM of light. In the paraxial approximation, the
contribution of the SAM vanishes and the IFE is solely driven by the OAM of
light, which is consistent with the interpretation of the spin and orbital
angular momenta of purely transverse light fields [38, 40]. We also evidence
that the SOI of light plays a significant role in the IFE. Finally, we
numerically quantify the relative contributions of the SAM and OAM to the IFE
in a thin gold film illuminated with four different focused beams carrying SAM
and/or OAM. We find that the SAM of light is the main source of drift current
density regardless of the helicity of the incident light. However,
compensation effects between SAM-driven surface and bulk currents of opposite
handedness reduces the contribution of the SAM to the observable opto-induced
magnetization. Except for circular polarization, the OAM of light is found to
be the main contributor to the IFE. We also numerically confirm the importance
of the SOI of light in the IFE, which manifests both via SAM-to-OAM and OAM-
to-SAM conversions at focus. The OAM of light thus opens new degrees of
freedom in the control of the IFE in metals, thus potentially impacting
various research fields including all-optical magnetization switching and
spin-wave excitation.

## References

  * Allen  _et al._ [1992] L. Allen, M. W. Beijersbergen, R. Spreeuw, and J. Woerdman, Phys. Rev. A 45, 8185 (1992).
  * Pershan  _et al._ [1966] P. Pershan, J. Van der Ziel, and L. Malmstrom, Phys. Rev. 143, 574 (1966).
  * Popova  _et al._ [2011] D. Popova, A. Bringer, and S. Blügel, Phys. Rev. B 84, 214421 (2011).
  * Hertel [2006] R. Hertel, J. Magn. Magn. Mater. 303, L1 (2006).
  * Beaurepaire  _et al._ [1996] E. Beaurepaire, J.-C. Merle, A. Daunois, and J.-Y. Bigot, Phys. Rev. Lett. 76, 4250 (1996).
  * Stanciu  _et al._ [2007] C. Stanciu, F. Hansteen, A. Kimel, A. Kirilyuk, A. Tsukamoto, A. Itoh, and T. Rasing, Phys. Rev. Lett. 99, 047601 (2007).
  * Kirilyuk  _et al._ [2010] A. Kirilyuk, A. V. Kimel, and T. Rasing, Rev. Mod. Phys. 82, 2731 (2010).
  * Kimel  _et al._ [2005] A. Kimel, A. Kirilyuk, P. Usachev, R. Pisarev, A. Balbashov, and T. Rasing, Nature 435, 655 (2005).
  * Kalashnikova  _et al._ [2008] A. Kalashnikova, A. Kimel, R. Pisarev, V. Gridnev, P. Usachev, A. Kirilyuk, and T. Rasing, Phys. Rev. B 78, 104301 (2008).
  * Satoh  _et al._ [2012] T. Satoh, Y. Terui, R. Moriya, B. A. Ivanov, K. Ando, E. Saitoh, T. Shimura, and K. Kuroda, Nat. Photonics 6, 662 (2012).
  * Savochkin  _et al._ [2017] I. Savochkin, M. Jäckl, V. Belotelov, I. Akimov, M. Kozhaev, D. Sylgacheva, A. Chernov, A. Shaposhnikov, A. Prokopov, V. Berzhansky, _et al._ , Sci. Rep. 7, 1 (2017).
  * Matsumoto  _et al._ [2020] K. Matsumoto, I. Yoshimine, K. Himeno, T. Shimura, and T. Satoh, Phys. Rev. B 101, 184407 (2020).
  * Smolyaninov  _et al._ [2005] I. I. Smolyaninov, C. C. Davis, V. N. Smolyaninova, D. Schaefer, J. Elliott, and A. V. Zayats, Phys. Rev. B 71, 035425 (2005).
  * Gu and Kornev [2010] Y. Gu and K. G. Kornev, JOSA B 27, 2165 (2010).
  * Cheng  _et al._ [2020a] O. H.-C. Cheng, D. H. Son, and M. Sheldon, Nat. Photonics 14, 365 (2020a).
  * Koshelev  _et al._ [2015] K. Koshelev, V. Y. Kachorovskii, and M. Titov, Phys. Rev. B 92, 235426 (2015).
  * Hamidi  _et al._ [2015] S. Hamidi, M. Razavinia, and M. Tehranchi, Opt. Commun. 338, 240 (2015).
  * Lefier [2016] Y. Lefier,  _Etudes du couplage spin-orbite en nano-photonique. applications à l’excitation unidirectionnelle de modes plasmoniques guidés et à la génération d’opto-aimants nanométriques contrôlables par l’état de polarisation de la lumière_ , Ph.D. thesis, Université de Franche-Comté (2016).
  * Nadarajah and Sheldon [2017] A. Nadarajah and M. T. Sheldon, Opt. Express 25, 12753 (2017).
  * Hurst  _et al._ [2018] J. Hurst, P. M. Oppeneer, G. Manfredi, and P.-A. Hervieux, Phys. Rev. B 98, 134439 (2018).
  * Mondal  _et al._ [2015] R. Mondal, M. Berritta, C. Paillard, S. Singh, B. Dkhil, P. M. Oppeneer, and L. Bellaiche, Phys. Rev. B 92, 100402 (2015).
  * Karakhanyan  _et al._ [2021a] V. Karakhanyan, Y. Lefier, C. Eustache, and T. Grosjean, Opt. Lett. 46, 613 (2021a).
  * Karakhanyan  _et al._ [2021b] V. Karakhanyan, C. Eustache, Y. Lefier, and T. Grosjean, OSA Continuum 4, 1598 (2021b).
  * Liu  _et al._ [2015] T.-M. Liu, T. Wang, A. H. Reid, M. Savoini, X. Wu, B. Koene, P. Granitzka, C. E. Graves, D. J. Higley, Z. Chen, _et al._ , Nano Lett. 15, 6862 (2015).
  * Dutta  _et al._ [2017] A. Dutta, A. V. Kildishev, V. M. Shalaev, A. Boltasseva, and E. E. Marinero, Opt. Mat. Express 7, 4316 (2017).
  * Chu  _et al._ [2020] A. H. Chu, B. Beauchamp, D. Shah, A. Boltasseva, V. M. Shalaev, E. E. Marinero, _et al._ , Opt. Mat. Express 10, 3107 (2020).
  * Ignatyeva  _et al._ [2019] D. Ignatyeva, C. Davies, D. Sylgacheva, A. Tsukamoto, H. Yoshikawa, P. Kapralov, A. Kirilyuk, V. Belotelov, and A. Kimel, Nat. Commun. 10, 1 (2019).
  * Im  _et al._ [2019] S.-J. Im, J.-S. Pae, C.-S. Ri, K.-S. Ho, and J. Herrmann, Phys. Rev. B 99, 041401 (2019).
  * Cheng  _et al._ [2020b] F. Cheng, C. Wang, Z. Su, X. Wang, Z. Cai, N. X. Sun, and Y. Liu, Nano Lett. 20, 6437 (2020b).
  * Sirenko  _et al._ [2019] A. Sirenko, P. Marsik, C. Bernhard, T. Stanislavchuk, V. Kiryukhin, and S.-W. Cheong, Phys. Rev. Lett. 122, 237401 (2019).
  * Maccaferri  _et al._ [2020] N. Maccaferri, I. Zubritskaya, I. Razdolski, I.-A. Chioar, V. Belotelov, V. Kapaklis, P. M. Oppeneer, and A. Dmitriev, J. Appl. Phys. 127, 080903 (2020).
  * Euler [1757] L. Euler, Mémoires de l’Académie des Sciences de Berlin , 274 (1757).
  * Scalora  _et al._ [2010] M. Scalora, M. Vincenti, D. De Ceglia, V. Roppo, M. Centini, N. Akozbek, and M. Bloemer, Phys. Rev. A 82, 043828 (2010).
  * Ciracì  _et al._ [2012] C. Ciracì, E. Poutrina, M. Scalora, and D. R. Smith, Phys. Rev. B 85, 201403 (2012).
  * Shen [1984] Y.-R. Shen, New York (1984).
  * Morel  _et al._ [2021] B. Morel, R. Giust, K. Ardaneh, and F. Courvoisier, Sci. Rep. 11, 1 (2021).
  * Sinha-Roy  _et al._ [2020] R. Sinha-Roy, J. Hurst, G. Manfredi, and P.-A. Hervieux, ACS Photonics 7, 2429 (2020).
  * Berry [2009] M. V. Berry, J.Opt. A 11, 094001 (2009).
  * Bekshaev and Soskin [2007] A. Y. Bekshaev and M. Soskin, Opt. Commun. 271, 332 (2007).
  * Bliokh  _et al._ [2014] K. Y. Bliokh, A. Y. Bekshaev, and F. Nori, Nat.communications 5, 1 (2014).
  * Neugebauer  _et al._ [2015] M. Neugebauer, T. Bauer, A. Aiello, and P. Banzer, Phys. Rev. Lett. 114, 063901 (2015).
  * Aiello  _et al._ [2015] A. Aiello, P. Banzer, M. Neugebauer, and G. Leuchs, Nature Photonics 9, 789 (2015).
  * Gordon [1973] J. P. Gordon, Phys. Rev. A 8, 14 (1973).
  * Ashkin and Gordon [1983] A. Ashkin and J. P. Gordon, Opt. Lett 8, 511 (1983).
  * Xu  _et al._ [2020] X. Xu, M. Nieto-Vesperinas, C.-W. Qiu, X. Liu, D. Gao, Y. Zhang, and B. Li, Laser Photonics Rev 14, 1900265 (2020).
  * Goff and Schaich [1997] J. E. Goff and W. Schaich, Phys. Rev. B 56, 15421 (1997).
  * Sipe  _et al._ [1980] J. Sipe, V. So, M. Fukui, and G. Stegeman, Phys. Rev. B 21, 4389 (1980).
  * Raza  _et al._ [2011] S. Raza, G. Toscano, A.-P. Jauho, M. Wubs, and N. A. Mortensen, Phys. Rev. B 84, 121412 (2011).
  * Toscano  _et al._ [2012] G. Toscano, S. Raza, A.-P. Jauho, N. A. Mortensen, and M. Wubs, Opt. Express 20, 4176 (2012).
  * Allen  _et al._ [2016] L. Allen, S. M. Barnett, and M. J. Padgett,  _Optical angular momentum_ (CRC press, 2016).
  * Andrews and Babiker [2012] D. L. Andrews and M. Babiker,  _The angular momentum of light_ (Cambridge University Press, 2012).
  * Bliokh  _et al._ [2015a] K. Y. Bliokh, F. J. Rodríguez-Fortuño, F. Nori, and A. V. Zayats, Nat. Photonics 9, 796 (2015a).
  * Bliokh  _et al._ [2010] K. Y. Bliokh, M. A. Alonso, E. A. Ostrovskaya, and A. Aiello, Phys. Rev. A 82, 063825 (2010).
  * Bekshaev [2010] A. Y. Bekshaev, Cent. Eur. J. Phys. 8, 947 (2010).
  * Richards and Wolf [1959] B. Richards and E. Wolf, Proceedings of the Royal Society of London. Series A. Mathematical and Physical Sciences 253, 358 (1959).
  * Novotny and Hecht [2012] L. Novotny and B. Hecht,  _Principles of nano-optics_ (Cambridge university press, 2012).
  * Born and Wolf [2013] M. Born and E. Wolf,  _Principles of optics: electromagnetic theory of propagation, interference and diffraction of light_ (Elsevier, 2013).
  * Du  _et al._ [2019] L. Du, A. Yang, A. V. Zayats, and X. Yuan, Nat. Phys. 15, 650 (2019).
  * Zhan [2006] Q. Zhan, Opt. Lett 31, 867 (2006).
  * Zhao  _et al._ [2007] Y. Zhao, J. S. Edgar, G. D. Jeffries, D. McGloin, and D. T. Chiu, Phys. Rev. Lett. 99, 073901 (2007).
  * Bliokh  _et al._ [2015b] K. Y. Bliokh, D. Smirnova, and F. Nori, Science 348, 1448 (2015b).
  * Li  _et al._ [2018] M. Li, Y. Cai, S. Yan, Y. Liang, P. Zhang, and B. Yao, Phys. Rev. A 97, 053842 (2018).
  * Noginova  _et al._ [2011] N. Noginova, A. Yakim, J. Soimo, L. Gu, and M. Noginov, Phys. Rev. B 84, 035447 (2011).