# Lancaster A at SemEval-2017 Task 5: Evaluation metrics matter: predicting
sentiment from financial news headlines

Andrew Moore and Paul Rayson
School of Computing and Communications, Lancaster University, Lancaster, UK
<EMAIL_ADDRESS>
###### Abstract

This paper describes our participation in Task 5 track 2 of SemEval 2017 to
predict the sentiment of financial news headlines for a specific company on a
continuous scale between -1 and 1. We tackled the problem using a number of
approaches, utilising a Support Vector Regression (SVR) and a Bidirectional
Long Short-Term Memory (BLSTM). We found an improvement of 4-6% using the LSTM
model over the SVR and came fourth in the track. We report a number of
different evaluations using a finance specific word embedding model and
reflect on the effects of using different evaluation metrics.

##  1 Introduction

The objective of Task 5 Track 2 of SemEval Cortis et al. (2017) was to predict
the sentiment of news headlines with respect to companies mentioned within the
headlines. This task can be seen as a finance-specific aspect-based sentiment
task Nasukawa and Yi (2003). The main motivations of this task is to find
specific features and learning algorithms that will perform better for this
domain as aspect based sentiment analysis tasks have been conducted before at
SemEval Pontiki et al. (2014).

Domain specific terminology is expected to play a key part in this task, as
reporters, investors and analysts in the financial domain will use a specific
set of terminology when discussing financial performance. Potentially, this
may also vary across different financial domains and industry sectors.
Therefore, we took an exploratory approach and investigated how various
features and learning algorithms perform differently, specifically SVR and
BLSTMs. We found that BLSTMs outperform an SVR without having any knowledge of
the company that the sentiment is with respect to. For replicability purposes,
with this paper we are releasing our source
code111https://github.com/apmoore1/semeval and the finance specific BLSTM word
embedding
model222https://github.com/apmoore1/semeval/tree/master/models/word2vec_models.

##  2 Related Work

There is a growing amount of research being carried out related to sentiment
analysis within the financial domain. This work ranges from domain-specific
lexicons Loughran and McDonald (2011) and lexicon creation Moore et al. (2016)
to stock market prediction models Peng and Jiang (2016); Kazemian et al.
(2016). Peng and Jiang (2016) used a multi layer neural network to predict the
stock market and found that incorporating textual features from financial news
can improve the accuracy of prediction. Kazemian et al. (2016) showed the
importance of tuning sentiment analysis to the task of stock market
prediction. However, much of the previous work was based on numerical
financial stock market data rather than on aspect level financial textual
data.

In aspect based sentiment analysis, there have been many different techniques
used to predict the polarity of an aspect as shown in SemEval-2016 task 5
Pontiki et al. (2014). The winning system Brun et al. (2016) used many
different linguistic features and an ensemble model, and the runner up Kumar
et al. (2016) used uni-grams, bi-grams and sentiment lexicons as features for
a Support Vector Machine (SVM). Deep learning methods have also been applied
to aspect polarity prediction. Ruder et al. (2016) created a hierarchical
BLSTM with a sentence level BLSTM inputting into a review level BLSTM thus
allowing them to take into account inter- and intra-sentence context. They
used only word embeddings making their system less dependent on extensive
feature engineering or manual feature creation. This system outperformed all
others on certain languages on the SemEval-2016 task 5 dataset Pontiki et al.
(2014) and on other languages performed close to the best systems. Wang et al.
(2016) also created an LSTM based model using word embeddings but instead of a
hierarchical model it was a one layered LSTM with attention which puts more
emphasis on learning the sentiment of words specific to a given aspect.

##  3 Data

The training data published by the organisers for this track was a set of
headline sentences from financial news articles where each sentence was tagged
with the company name (which we treat as the aspect) and the polarity of the
sentence with respect to the company. There is the possibility that the same
sentence occurs more than once if there is more than one company mentioned.
The polarity was a real value between -1 (negative sentiment) and 1 (positive
sentiment).

We additionally trained a word2vec Mikolov et al. (2013) word embedding
model333For reproducibility, the model can be downloaded, however the articles
cannot be due to copyright and licence restrictions. on a set of 189,206
financial articles containing 161,877,425 tokens, that were manually
downloaded from
Factiva444https://global.factiva.com/factivalogin/login.asp?productname=global.
The articles stem from a range of sources including the Financial Times and
relate to companies from the United States only. We trained the model on
domain specific data as it has been shown many times that the financial domain
can contain very different language.

##  4 System description

Even though we have outlined this task as an aspect based sentiment task, this
is instantiated in only one of the features in the SVR. The following two
subsections describe the two approaches, first SVR and then BLSTM. Key
implementation details are exposed here in the paper, but we have released the
source code and word embedding models to aid replicability and further
experimentation.

###  4.1 SVR

The system was created using ScitKit learn Pedregosa et al. (2011) linear
Support Vector Regression model Drucker et al. (1997). We experimented with
the following different features and parameter settings:

####  4.1.1 Tokenisation

For comparison purposes, we tested whether or not a simple whitespace
tokeniser can perform just as well as a full tokeniser, and in this case we
used Unitok555http://corpus.tools/wiki/Unitok.

####  4.1.2 N-grams

We compared word-level uni-grams and bi-grams separately and in combination.

####  4.1.3 SVR parameters

We tested different penalty parameters C and different epsilon parameters of
the SVR.

####  4.1.4 Word Replacements

We tested replacements to see if generalising words by inserting special
tokens would help to reduce the sparsity problem. We placed the word
replacements into three separate groups:

  1. 1.

Company - When a company was mentioned in the input headline from the list of
companies in the training data marked up as aspects, it was replaced by a
company special token.

  2. 2.

Positive - When a positive word was mentioned in the input headline from a
list of positive words (which was created using the N most similar words based
on cosine distance) to ‘excellent’ using the pre-trained word2vec model.

  3. 3.

Negative - The same as the positive group however the word used was ‘poor’
instead of ‘excellent’.

In the positive and negative groups, we chose the words ‘excellent’ and ‘poor’
following Turney (2002) to group the terms together under non-domain specific
sentiment words.

####  4.1.5 Target aspect

In order to incorporated the company as an aspect, we employed a boolean
vector to represent the sentiment of the sentence. This was done in order to
see if the system could better differentiate the sentiment when the sentence
was the same but the company was different.

###  4.2 BLSTM

We created two different Bidirectional Graves and Schmidhuber (2005) Long
Short-Term Memory Hochreiter and Schmidhuber (1997) using the Python Keras
library Chollet (2015) with tensor flow backend Abadi et al. (2016). We choose
an LSTM model as it solves the vanishing gradients problem of Recurrent Neural
Networks. We used a bidirectional model as it allows us to capture information
that came before and after instead of just before, thereby allowing us to
capture more relevant context within the model. Practically, a BLSTM is two
LSTMs one going forward through the tokens the other in reverse order and in
our models concatenating the resulting output vectors together at each time
step.

The BLSTM models take as input a headline sentence of size L
tokens666Tokenised by Unitok where L is the length of the longest sentence in
the training texts. Each word is converted into a 300 dimension vector using
the word2vec model trained over the financial text777See the following link
for detailed implementation details
https://github.com/apmoore1/semeval#finance-word2vec-model. Any text that is
not recognised by the word2vec model is represented as a vector of zeros; this
is also used to pad out the sentence if it is shorter than L.

Figure 1: Left hand side is the ELSTM model architecture and the right hand
side shows the SLSTM. The numbers in the parenthesis represent the size of the
output dimension where L is the length of the longest sentence.

Both BLSTM models have the following similar properties:

  1. 1.

Gradient clipping value of 5 - This was to help with the exploding gradients
problem.

  2. 2.

Minimised the Mean Square Error (MSE) loss using RMSprop with a mini batch
size of 32.

  3. 3.

The output activation function is linear.

The main difference between the two models is the use of drop out and when
they stop training over the data (epoch). Both models architectures can be
seen in figure 1.

####  4.2.1 Standard LSTM (SLSTM)

The BLSTMs do contain drop out in both the input and between the connections
of 0.2 each. Finally the epoch is fixed at 25.

####  4.2.2 Early LSTM (ELSTM)

As can be seen from figure 1, the drop out of 0.5 only happens between the
layers and not the connections as in the SLSTM. Also the epoch is not fixed,
it uses early stopping with a patience of 10. We expect that this model can
generalise better than the standard one due to the higher drop out and that
the epoch is based on early stopping which relies on a validation set to know
when to stop training.

##  5 Results

We first present our findings on the best performing parameters and features
for the SVRs. These were determined by cross validation (CV) scores on the
provided training data set using cosine similarity as the evaluation
metric.888All the cross validation results can be found here
https://github.com/apmoore1/semeval/tree/master/results We found that using
uni-grams and bi-grams performs best and using only bi-grams to be the worst.
Using the Unitok tokeniser always performed better than simple whitespace
tokenisation. The binary presence of tokens over frequency did not alter
performance. The C parameter was tested for three values; 0.01, 0.1 and 1. We
found very little difference between 0.1 and 1, but 0.01 produced much poorer
results. The eplison parameter was tested for 0.001, 0.01 and 0.1 the
performance did not differ much but the lower the higher the performance but
the more likely to overfit. Using word replacements was effective for all
three types (company, positive and negative) but using a value N=10 performed
best for both positive and negative words. Using target aspects also improved
results. Therefore, the best SVR model comprised of: Unitok tokenisation, uni-
and bi- grams, word representation, C=0.1, eplison=0.01, company, positive,
and negative word replacements and target aspects.

$\begin{gathered}\frac{\sum_{n=1}^{N}\text{Cosine
similarity}(\hat{y}_{n},y_{n})}{N}\end{gathered}$ (1)

The main evaluation over the test data is based on the best performing SVR and
the two BLSTM models once trained on all of the training data. The result
table 1 shows three columns based on the three evaluation metrics that the
organisers have used. Metric 1 is the original metric, weighted cosine
similarity (the metric used to evaluate the final version of the results,
where we were ranked 5th; metric provided on the task
website999http://alt.qcri.org/semeval2017/task5/index.php?id=evaluation). This
was then changed after the evaluation deadline to equation 1101010Where N is
the number of unique sentences, $\hat{y}_{n}$ is the predicted and $y_{n}$ are
the true sentiment value(s) of all sentiments in sentence $n$. (which we term
metric 2; this is what the first version of the results were actually based
on, where we were ranked 4th), which then changed by the organisers to their
equation as presented in Cortis et al. (2017) (which we term metric 3 and what
the second version of the results were based on, where we were ranked 5th).

Model | Metric 1 | Metric 2 | Metric 3
---|---|---|---
SVR | 62.14 | 54.59 | 62.34
SLSTM | 72.89 | 61.55 | 68.64
ELSTM | 73.20 | 61.98 | 69.24
Table 1: Results

As you can see from the results table 1, the difference between the metrics is
quite substantial. This is due to the system’s optimisation being based on
metric 1 rather than 2. Metric 2 is a classification metric for sentences with
one aspect as it penalises values that are of opposite sign (giving -1 score)
and rewards values with the same sign (giving +1 score). Our systems are not
optimised for this because it would predict scores of -0.01 and true value of
0.01 as very close (within vector of other results) with low error whereas
metric 2 would give this the highest error rating of -1 as they are not the
same sign. Metric 3 is more similar to metric 1 as shown by the results,
however the crucial difference is that again if you get opposite signs it will
penalise more.

We analysed the top 50 errors based on Mean Absolute Error (MAE) in the test
dataset specifically to examine the number of sentences containing more than
one aspect. Our investigation shows that no one system is better at predicting
the sentiment of sentences that have more than one aspect (i.e. company)
within them. Within those top 50 errors we found that the BLSTM systems do not
know which parts of the sentence are associated to the company the sentiment
is with respect to. Also they do not know the strength/existence of certain
sentiment words.

##  6 Conclusion and Future Work

In this short paper, we have described our implemented solutions to SemEval
Task 5 track 2, utilising both SVR and BLSTM approaches. Our results show an
improvement of around 5% when using LSTM models relative to SVR. We have shown
that this task can be partially represented as an aspect based sentiment task
on a domain specific problem. In general, our approaches acted as sentence
level classifiers as they take no target company into consideration. As our
results show, the choice of evaluation metric makes a great deal of difference
to system training and testing. Future work will be to implement aspect
specific information into an LSTM model as it has been shown to be useful in
other work Wang et al. (2016).

## Acknowledgements

We are grateful to Nikolaos Tsileponis (University of Manchester) and Mahmoud
El-Haj (Lancaster University) for access to headlines in the corpus of
financial news articles collected from Factiva. This research was supported at
Lancaster University by an EPSRC PhD studentship.

## References

  * Abadi et al. (2016) Martín Abadi, Ashish Agarwal, Paul Barham, Eugene Brevdo, Zhifeng Chen, Craig Citro, Greg S Corrado, Andy Davis, Jeffrey Dean, Matthieu Devin, et al. 2016\.  Tensorflow: Large-scale machine learning on heterogeneous distributed systems.  arXiv preprint arXiv:1603.04467 .
  * Brun et al. (2016) Caroline Brun, Julien Perez, and Claude Roux. 2016.  Xrce at semeval-2016 task 5: Feedbacked ensemble modelling on syntactico-semantic knowledge for aspect based sentiment analysis.  Proceedings of SemEval pages 277–281.
  * Chollet (2015) François Chollet. 2015.  Keras.  https://github.com/fchollet/keras.
  * Cortis et al. (2017) Keith Cortis, Andre Freitas, Tobias Daudert, Manuela Huerlimann, Manel Zarrouk, and Brian Davis. 2017.  Semeval-2017 task 5: Fine-grained sentiment analysis on financial microblogs and news.  Proceedings of SemEval .
  * Drucker et al. (1997) Harris Drucker, Christopher JC Burges, Linda Kaufman, Alex Smola, Vladimir Vapnik, et al. 1997.  Support vector regression machines.  Advances in neural information processing systems 9:155–161.
  * Graves and Schmidhuber (2005) Alex Graves and Jürgen Schmidhuber. 2005.  Framewise phoneme classification with bidirectional lstm and other neural network architectures.  Neural Networks 18(5):602–610.
  * Hochreiter and Schmidhuber (1997) Sepp Hochreiter and Jürgen Schmidhuber. 1997.  Long short-term memory.  Neural computation 9(8):1735–1780.
  * Kazemian et al. (2016) Siavash Kazemian, Shunan Zhao, and Gerald Penn. 2016.  Evaluating sentiment analysis in the context of securities trading.  In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, pages 2094–2103.  https://doi.org/10.18653/v1/P16-1197.
  * Kumar et al. (2016) Ayush Kumar, Sarah Kohail, Amit Kumar, Asif Ekbal, and Chris Biemann. 2016.  Iit-tuda at semeval-2016 task 5: Beyond sentiment lexicon: Combining domain dependency and distributional semantics features for aspect based sentiment analysis.  Proceedings of SemEval pages 1129–1135.
  * Loughran and McDonald (2011) Tim Loughran and Bill McDonald. 2011.  When is a liability not a liability? textual analysis, dictionaries, and 10-ks.  The Journal of Finance 66(1):35–65.
  * Mikolov et al. (2013) Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013.  Efficient estimation of word representations in vector space.  arXiv preprint arXiv:1301.3781 .
  * Moore et al. (2016) Andrew Moore, Paul Rayson, and Steven Young. 2016.  Domain adaptation using stock market prices to refine sentiment dictionaries.  In Proceedings of the 10th edition of Language Resources and Evaluation Conference (LREC2016). European Language Resources Association (ELRA).
  * Nasukawa and Yi (2003) Tetsuya Nasukawa and Jeonghee Yi. 2003.  Sentiment analysis: Capturing favorability using natural language processing.  In Proceedings of the 2nd international conference on Knowledge capture. ACM, pages 70–77.
  * Pedregosa et al. (2011) Fabian Pedregosa, Gaël Varoquaux, Alexandre Gramfort, Vincent Michel, Bertrand Thirion, Olivier Grisel, Mathieu Blondel, Peter Prettenhofer, Ron Weiss, Vincent Dubourg, et al. 2011.  Scikit-learn: Machine learning in python.  Journal of Machine Learning Research 12(Oct):2825–2830.
  * Peng and Jiang (2016) Yangtuo Peng and Hui Jiang. 2016.  Leverage financial news to predict stock price movements using word embeddings and deep neural networks.  In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Association for Computational Linguistics, pages 374–379.  https://doi.org/10.18653/v1/N16-1041.
  * Pontiki et al. (2014) Maria Pontiki, Dimitris Galanis, John Pavlopoulos, Harris Papageorgiou, Ion Androutsopoulos, and Suresh Manandhar. 2014.  Semeval-2014 task 4: Aspect based sentiment analysis.  Proceedings of SemEval pages 27–35.
  * Ruder et al. (2016) Sebastian Ruder, Parsa Ghaffari, and G. John Breslin. 2016.  A hierarchical model of reviews for aspect-based sentiment analysis.  In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, pages 999–1005.  http://aclweb.org/anthology/D16-1103.
  * Turney (2002) Peter D Turney. 2002.  Thumbs up or thumbs down?: semantic orientation applied to unsupervised classification of reviews.  In Proceedings of the 40th annual meeting on association for computational linguistics. Association for Computational Linguistics, pages 417–424.
  * Wang et al. (2016) Yequan Wang, Minlie Huang, xiaoyan zhu, and Li Zhao. 2016.  Attention-based lstm for aspect-level sentiment classification.  In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, pages 606–615.  http://aclweb.org/anthology/D16-1058.


# Refined TMD gluon density in a proton from the HERA and LHC data

A.V. Lipatov1,2, G.I. Lykasov2, M.A. Malyshev1,3

1Skobeltsyn Institute of Nuclear Physics, Lomonosov Moscow State University,
119991, Moscow, Russia
2Joint Institute for Nuclear Research, 141980, Dubna, Moscow region, Russia
3Moscow Aviation Institute, 125993, Moscow, Russia

Abstract

We update the phenomenological parameters of the Transverse Momentum Dependent
(TMD, or unintegrated) gluon density in a proton proposed in our previous
studies. This analysis is based on the analytical expression for starting
gluon distribution which provides a self-consistent simultaneous description
of HERA data on proton structure function $F_{2}(x,Q^{2})$, reduced cross
section for the electron-proton deep inelastic scattering at low $Q^{2}$ and
soft hadron production in $pp$ collisions at the LHC conditions. We extend it
to the whole kinematical region using the Catani-Ciafaloni-Fiorani-Marchesini
(CCFM) evolution equation. Exploiting our previous results on a number of
semihard QCD processes, we performed a combined fit to an extended set of LHC
and HERA data, comprising a total of $509$ points from $16$ data sets. We
illustrate our fit by applying the derived TMD gluon density in a proton to
inclusive prompt photon photoproduction at HERA.

Keywords: small-$x$ physics, high energy factorization, CCFM evolution, TMD
gluon density in the proton, deep inelastic scattering

It is well known that parton distribution functions in a proton (PDFs),
$f_{a}(x,\mu^{2})$ with $a=q$ or $g$, are an essential ingredient of any
description of hard scattering at modern colliders energies. If only one scale
is present in the process, $\mu\sim\sqrt{s}\gg\Lambda_{\text{QCD}}$, then the
PDFs can be described in Quantum Chromodynamics (QCD) via the Dokshitzer-
Gribov-Lipatov-Altarelli-Parisi (DGLAP) equations[1]. However, in case of a
two-scale process, $\sqrt{s}\gg\mu\gg\Lambda_{\text{QCD}}$, the gluon dynamics
can be described by the Balitsky-Fadin-Kuraev-Lipatov (BFKL)[2] or Catani-
Ciafaloni-Fiorani-Marchesini (CCFM)[3] equations. It leads to Transverse
Momentum Dependent (TMD, or unintegrated) gluon densities in a proton and high
energy factorization[4], or $k_{T}$-factorization[5] approach (see review[6]
for more information). The $k_{T}$-factorization approach is quite widely used
in phenomenological applications and implemented in several Monte-Carlo event
generators, such as pegasus[7] and cascade[8]. A comprehensive set of TMD
gluon distributions in a proton is available in the tmdlib library[9].

In general, TMD gluon densities can be calculated within some approaches, such
as popular Kimber-Martin-Ryskin formalism[10, 11], Parton Branching
approach[12, 13] or obtained from the analytical or numerical solutions of
BFKL-like QCD evolution equations111There are also investigations within the
non-linear evolution in QCD (see, for example,[14, 15, 16, 17, 18, 19, 20, 21,
22] and references therein).. The CCFM equation, which resumes large
logarithmic terms proportional to $\alpha_{s}^{n}\ln^{n}1/x$ and
$\alpha_{s}^{n}\ln^{n}1/(1-x)$ and therefore valid at both low and large $x$,
has been applied[23]. In these calculations, the empirical expression for
initial gluon density at some starting scale $\mu_{0}$ (which is of the order
of the hadron scale) with factorized Gauss smearing in transverse momentum was
applied. In contrast, in our previous study[24] a more physically motivated
expression for the input distribution was chosen. It is based on the
description of the LHC data on soft hadron transverse momenta spectra in the
framework of the modified soft quark gluon string model[25, 26] (see also[27,
28]) with taking into account gluon saturation effects important at small $x$
and low scales[29]. The obtained CCFM-evolved gluon density in a proton (LLM)
was successfully tested later on a number of collider data, in particular, on
latest HERA data on longitudinal structure function of proton222A comparison
of LLM predictions on $F_{L}(x,Q^{2})$ with other models can be found[30].
$F_{L}(x,Q^{2})$[31] and associated photoproduction of prompt photons and
jets[32].

Very recently it was shown[33] that some phenomenological parameters of the
starting LLM gluon density need to be corrected in order to provide a good
description (within the color dipole approach[34, 35, 36, 37]) of the low
$Q^{2}$ data on proton structure function $F_{2}(x,Q^{2})$ and reduced deep
inelastic cross sections taken by H1 and ZEUS Collaborations. These parameters
are related mainly with the small $x$ behaviour of the TMD gluon density at
low scales, as it will demostrated below. At the same time, such an adjustment
does not spoil the quality of soft hadron spectra data, provided the
corresponding parameters of fragmentation of quarks and diquarks to the
hadrons are altered moderately (see[33] for more details). Of course, in this
case essential parameters, which cannot be determined from these data, have to
be fitted from other measurements. It is the aim of this short Letter to
perform such a fit using an extended set of experimental data for several
processes known to be sensitive to the gluon content of the proton and then
complete our study by applying the CCFM evolution to the newly fitted initial
density. As a main result, we present a more universal TMD gluon distribution
function in a proton, which is already available in the Monte-Carlo event
generator pegasus and tmdlib tool.

So, we ended up with the following form (non-factorized with respect to $x$
and ${\mathbf{k}}_{T}^{2}$) of the initial LLM gluon density in a proton:

$\displaystyle
f_{g}(x,{\mathbf{k}}_{T}^{2})=c_{g}(1-x)^{b_{g}}\sum_{n=1}^{3}c_{n}\left[R_{0}(x)|{\mathbf{k}}_{T}|\right]^{n}\exp\left(-R_{0}(x)|{\mathbf{k}}_{T}|\right),$
(1)

where

$\displaystyle R_{0}^{2}(x)={1\over Q_{0}^{2}}\left(x\over
x_{0}\right)^{\lambda},\quad
b_{g}=b_{g}(0)+{4C_{A}\over\beta_{0}}\ln{\alpha_{s}(Q_{0}^{2})\over\alpha_{s}({\mathbf{k}}_{T}^{2})},$
(2)

with $C_{A}=N_{C}$, $\beta_{0}=11-2N_{f}/3$ and $Q_{0}=2.2$ GeV. Here $b_{g}$
parameter is treated to be running at ${\mathbf{k}}_{T}^{2}>Q_{0}^{2}$ only,
whereas the fixed value $b_{g}=b_{g}(0)$ at ${\mathbf{k}}_{T}^{2}\leq
Q_{0}^{2}$ is used. Simultaneous best fit to the HERA data on reduced deep
inelastic cross section $\sigma_{r}(x,Q^{2})$ and proton structure function
$F_{2}(x,Q^{2})$ measured at low $Q^{2}<4.5$ GeV2 and LHC data on charged
hadron (pion and kaon) production at small transverse momenta $p_{T}$ in the
mid-rapidity region leads to $c_{1}=5$, $c_{2}=3$, $c_{3}=2$, $x_{0}=1.3\cdot
10^{-11}$ and $\lambda=0.22$[33]. The experimental data on charged hadron
production involved into the fit are compared with our predictions in Fig. 1
(left panel). One can see that good agreement is achieved in a wide range of
energies. For a comparison we also show the results obtained within the
popular GBW model[36, 37]. It was argued[33] that the latter gives a somewhat
worse description of the data.

Figure 1: Left panel: transverse momentum distributions of multiplicities of
soft charged hadrons produced in $pp$ collisions at different LHC energies in
the mid-rapidity region. Shaded bands correspond to the uncertainties of our
calculations connected with the uncertainties in determination of $x_{0}$ and
parton-to-hadron fragmentation parameters. The experimental data are from
ATLAS[38, 39] and CMS[40]. Right panel: $\chi^{2}/d.o.f.$ dependence of our
fit for $b_{g}(0)$ parameter.

Our procedure to determine remaining parameters, $b_{g}(0)$ and $c_{g}$,
follows then the investigation[24]. As usual, they are extracted by minimizing

$\displaystyle\chi^{2}=\sum_{i}\left[{{\cal C}_{i}^{\rm data}-{\cal
C}_{i}^{\rm theory}\over\Delta{\cal C}_{i}^{\rm data}}\right]^{2}$ (3)

where $i$ runs over all experimental data points for observables ${{\cal
C}_{i}}$ from analyses listed in Table 1. Exploiting our previous results,
theoretical predictions for each of ${{\cal C}_{i}}$ are generated for large
number of fixed guessed $b_{g}(0)$ values in the range $1\leq b_{g}(0)\leq 9$
after applying the CCFM evolution to input density (1). At this step we used
updfevolv routine[41] to solve numerically the CCFM equation. Note that the
data set is extended now compared to the previous analysis[24] by including
the ATLAS data[42] on inclusive $b$-jet production in $pp$ collisions at
$\sqrt{s}=7$ TeV, CMS data on $c$-jet production in $pp$ interactions at
$\sqrt{s}=2.76$ and $5.02$ GeV[43] and HERA data[44, 45] on inclusive prompt
photon photoproduction in $ep$ collisions at $\sqrt{s}=319$ GeV. Our
simultaneous fit to all data sets leads to $b_{g}(0)=5.109$ and $c_{g}=1.2708$
with goodness $\chi^{2}/d.o.f.=1.773$ for $509$ data points, see Fig. 1 (right
panel). Then, to obtain the TMD gluon density in the whole kinematical range,
we solved the CCFM equation numerically using the fitted values of the
parameters above. In this way, the gluon distribution function (LLM) is
tabulated in a commonly recognized format (namely, grid of $50\times 50\times
50$ bins in $x$, ${\mathbf{k}}_{T}^{2}$ and $\mu^{2}$) which is used in the
tmdlib package. It is already available for public usage from there and
implemented also into the Monte-Carlo event generator pegasus.

Experiment | Collaboration | Year | Reference | Collision | $\sqrt{s}/{\rm GeV}$ | Number of points
---|---|---|---|---|---|---
incl. $c$-jet | CMS | $2017$ | [43] | $pp$ | $2.76$ | $5$
incl. $c$-jet | CMS | $2017$ | [43] | $pp$ | $5.02$ | $5$
incl. $b$-jet | ATLAS | $2011$ | [42] | $pp$ | $7$ | $46$
incl. $b$-jet | CMS | $2012$ | [46] | $pp$ | $7$ | $98$
$F_{2}^{c}(x,Q^{2})$ | H1 |  $2010$, $2011$ | [47, 48] | $ep$ | $0.319$ | $25$
$F_{2}^{c}(x,Q^{2})$ | ZEUS | $2014$ | [49] | $ep$ | $0.319$ | $18$
$F_{2}^{b}(x,Q^{2})$ | H1 | $2014$ | [47] | $ep$ | $0.319$ | $12$
$F_{2}^{b}(x,Q^{2})$ | ZEUS | $2014$ | [49] | $ep$ | $0.319$ | $17$
$\sigma_{\rm red}^{c}(x,Q^{2})$ | H1, ZEUS | $2018$ | [50] | $ep$ | $0.319$ | $51$
$\sigma_{\rm red}^{b}(x,Q^{2})$ | H1, ZEUS | $2018$ | [50] | $ep$ | $0.319$ | $27$
incl. $H\to\gamma\gamma$ | CMS | $2023$ | [51] | $pp$ | $13$ | $37$
incl. $H\to\gamma\gamma$ | ATLAS | $2018$ | [51] | $pp$ | $13$ | $27$
incl. $H\to ZZ^{*}$ | CMS | $2023$ | [52] | $pp$ | $13$ | $54$
incl. $H\to ZZ^{*}$ | ATLAS | $2020$ | [53] | $pp$ | $13$ | $54$
incl. $\gamma$ | H1 | $2010$ | [44] |  $ep$, low $Q^{2}$ | $0.319$ | $25$
incl. $\gamma$ | ZEUS | $2014$ | [45] |  $ep$, low $Q^{2}$ | $0.319$ | $8$
Table 1: The list of experimental data used for the fitting procedure and
number of data points for each experiment.

Figure 2: The inclusive prompt photon photoproduction cross section as
functions of the photon transverse energy and pseudorapidity. The green
(yellow) histograms and shaded bands correspond to the predictions obtained
with the LLM (JH’2013 set 2) gluon density and estimated scale uncertainties
of these calculations. The experimental data are from H1 [54] and ZEUS [55].

Let us now illustrate our fit with the inclusive prompt photon photoproduction
in $ep$ collisions at HERA, where sensitivity to the gluon density in a proton
can be seen clearly. Here, in the photoproduction regime of DIS, the colliding
electron emits a quasi-real photon, which then interacts with a proton. Here
we completely follow our previous study[32], where we have investigated the
prompt photon photoproduction with associated hadronic jets. The consideration
is based on two leading off-shell (depending on the non-zero virtualities of
incoming gluons) photon-gluon subprocesses, namely $\gamma g^{*}\to\gamma
q\bar{q}$ and ’box’ subprocess $\gamma g^{*}\to\gamma g$, currently
implemented into the Monte-Carlo event generator pegasus. The sea quark
contribution is then incorporated in the former subprocess, while the small
contribution of valence quarks can be taken into account in the conventional
QCD factorization (see[32] for more details). Numerically, we use the massless
limit for light ($u$, $d$ and $s$) quarks, set the charm and beauty masses to
$m_{c}=1.4$ GeV and $m_{b}=4.78$ GeV and implement certain kinematical cut
applied in the experimental analyses333All kinematic quantities are given in
the laboratory frame with positive $OZ$ axis directed as the proton beam.:
$6<E_{T}^{\gamma}<15$ GeV, $-1.0<\eta^{\gamma}<2.4$, $0.1<y<0.7$ for H1
measurement[44] and $6<E_{T}^{\gamma}<15$ GeV, $-0.7<\eta^{\gamma}<0.9$ and
$0.2<y<0.7$ for ZEUS[45] one, where $y$ is the fraction of the electron energy
transferred to the photon (inelasticity). Both these data were obtained with
the electron energy $E_{e}=27.6$ GeV and proton energy $E_{p}=920$ GeV.
Results of our calculations for produced photon transverse energy and
pseudorapidity distributions are shown in Fig. 2. One can see that our
predictions (represented by the green histograms) are consistent with the
latest HERA data within the experimental and theoretical uncertainties. The
latter are estimated in a traditional way, by varying the renormalization
scale444In the CCFM-based approach, the factorization scale $\mu_{F}$ is
related with the evolution variable and therefore should not be varied. See[3,
23] for more details. around its default value $\mu_{R}=E_{T}^{\gamma}$ as
$\mu_{R}\to\xi\mu_{R}$ with $\xi=1/2$ or $2$. In contrast with newly fitted
LLM set, using the JH’2013 set 2 gluon derived previously[23] leads to
systematic overestimation of the HERA data in most of the bins, that coincides
with observations made earlier[31, 32]. As it was argued[24], better agreement
achieved with the LLM distribution is a consequence of using a physically
motivated expression (1) for the initial gluon density for subsequent CCFM
evolution. Thus, our calculations demonstrate that considered HERA data could
help to distinguish the different approaches to evaluate the TMD gluon density
in a proton. Exact determination of the latter, of course, is important for
experiments at future electron-proton or electron-ion colliders, such as NICA,
LHeC, FCC-eh, EiC and EiCC.

To summarize, we have refined phenomenological parameters of the TMD gluon
density, which we obtained earlier (LLM). The analytical expression for
initial distribution provides a self-consistent simultaneous description of
HERA data on the proton structure function $F_{2}(x,Q^{2})$, reduced cross
section for the electron-proton deep inelastic scattering at low $Q^{2}$ and
soft hadron production in $pp$ collisions at the LHC conditions. It was
evolved according to the CCFM equation to expand it to the whole kinematical
region. Then our procedure was based on a fit to number of LHC and HERA data
for processes sensitive to the gluon content of a proton. The resulting fit
quality ($\chi^{2}/d.o.f.=1.773$) shows that the obtained gluon density does
not contradict experimental data. We illustrate it additionally with latest
HERA data on inclusive prompt photon photoproduction. Our results together
with the ones [33] represent a self-consistent approach for the TMD gluon
density in a proton valid in a wide kinematical region. The updated LLM gluon
density supersedes previous version and can be used in different
phenomenological applications for $pp$, $p\bar{p}$ and $ep$ processes at
modern and future colliders. It is available now in the tmdlib library and
Monte-Carlo event generator pegasus.

Acknowledgements. We thank S.P. Baranov, A.A. Prokhorov and H. Jung for their
interest, important comments and remarks. A.V.L. would like to thank School of
Physics and Astronomy, Sun Yat-sen University (Zhuhai, China) for warm
hospitality.

## References

  * [1] V.N. Gribov, L.N. Lipatov, Sov. J. Nucl. Phys. 15, 438 (1972);
L.N. Lipatov, Sov. J. Nucl. Phys. 20, 94 (1975);
G. Altarelli, G. Parisi, Nucl. Phys. B 126, 298 (1977);
Yu.L. Dokshitzer, Sov. Phys. JETP 46, 641 (1977).

  * [2] E.A. Kuraev, L.N. Lipatov, V.S. Fadin, Sov. Phys. JETP 44, 443 (1976);
E.A. Kuraev, L.N. Lipatov, V.S. Fadin, Sov. Phys. JETP 45, 199 (1977);
I.I. Balitsky, L.N. Lipatov, Sov. J. Nucl. Phys. 28, 822 (1978).

  * [3] M. Ciafaloni, Nucl. Phys. B 296, 49 (1988);
S. Catani, F. Fiorani, G. Marchesini, Phys. Lett. B 234, 339 (1990);
S. Catani, F. Fiorani, G. Marchesini, Nucl. Phys. B 336, 18 (1990);
G. Marchesini, Nucl. Phys. B 445, 49 (1995).

  * [4] S. Catani, M. Ciafaloni, F. Hautmann, Nucl. Phys. B 366, 135 (1991);
J.C. Collins, R.K. Ellis, Nucl. Phys. B 360, 3 (1991).

  * [5] L.V. Gribov, E.M. Levin, M.G. Ryskin, Phys. Rep. 100, 1 (1983);
E.M. Levin, M.G. Ryskin, Yu.M. Shabelsky, A.G. Shuvaev, Sov. J. Nucl. Phys.
53, 657 (1991).

  * [6] R. Angeles-Martinez, A. Bacchetta, I.I. Balitsky, D. Boer, M. Boglione, R. Boussarie, F.A. Ceccopieri, I.O. Cherednikov, P. Connor, M.G. Echevarria, G. Ferrera, J. Grados Luyando, F. Hautmann, H. Jung, T. Kasemets, K. Kutak, J.P. Lansberg, A. Lelek, G.I. Lykasov, J.D. Madrigal Martinez, P.J. Mulders, E.R. Nocera, E. Petreska, C. Pisano, R. Placakyte, V. Radescu, M. Radici, G. Schnell, I. Scimemi, A. Signori, L. Szymanowski, S. Taheri Monfared, F.F. Van der Veken, H.J. van Haevermaet, P. Van Mechelen, A.A. Vladimirov, S. Wallon, Acta Phys. Polon. B 46, 2501 (2015).
  * [7] A.V. Lipatov, M.A. Malyshev, S.P. Baranov, Eur. Phys. J. C 80, 330 (2020).
  * [8] H.Jung, S.P. Baranov, A. Bermudez Martinez, L.I. Estevez Banos, F. Guzman, F. Hautmann, A. Lelek, J. Lidrych, A.V. Lipatov, M.A. Malyshev, M. Mendizabal, S. Taheri Monfared, A.M. van Kampen, Q. Wang, H. Yang, Eur. Phys. J. C 81, 425 (2021).
  * [9] N.A. Abdulov, A. Bacchetta, S.P. Baranov, A. Bermudez Martinez, V. Bertone, C. Bissolotti, V. Candelise, L.I. Estevez Banos, M. Bury, P.L.S. Connor, L. Favart, F. Guzman, F. Hautmann, M. Hentschinski, H. Jung, L. Keersmaekers, A.V. Kotikov, A. Kusina, K. Kutak, A. Lelek, J. Lidrych, A.V. Lipatov, G.I. Lykasov, M.A. Malyshev, M. Mendizabal, S. Prestel, S. Sadeghi Barzani, S. Sapeta, M. Schmitz, A. Signori, G. Sorrentino, S. Taheri Monfared, A. van Hameren, A.M. van Kampen, M. Vanden Bemden, A. Vladimirov, Q. Wang, H. Yang, Eur. Phys. J. C 81, 752 (2021).
  * [10] M.A. Kimber, A.D. Martin, M.G. Ryskin, Phys. Rev. D 63, 114027 (2001);
A.D. Martin, M.G. Ryskin, G. Watt, Eur. Phys. J. C 31, 73 (2003).

  * [11] A.D. Martin, M.G. Ryskin, G. Watt, Eur. Phys. J. C 66, 163 (2010).
  * [12] F. Hautmann, H. Jung, A. Lelek, V. Radescu, R. Zlebcik, Phys. Lett. B 772, 446 (2017).
  * [13] F. Hautmann, H. Jung, A. Lelek, V. Radescu, R. Zlebcik, JHEP 1801, 070 (2018).
  * [14] J. Jalilian-Marian, A. Kovner, A. Leonidov, H. Weigert, Phys. Rev. D 59, 014014 (1998).
  * [15] J. Jalilian-Marian, A. Kovner, A. Leonidov, H. Weigert, Phys. Rev. D 59, 014015 (1998).
  * [16] A. Kovner, J.G. Milhano, H. Weigert, Phys. Rev. D 62, 114005 (2000).
  * [17] E. Iancu, A. Leonidov, L.D. McLerran, Nucl. Phys. A 692, 583 (2001).
  * [18] H. Weigert, Nucl. Phys. A 703, 823 (2002).
  * [19] E. Ferreiro, E. Iancu, A. Leonidov, L. McLerran, Nucl. Phys. A 703, 489 (2002).
  * [20] I. Balitsky, Nucl. Phys. B 463, 99 (1996);
Y.V. Kovchegov, Phys. Rev. D 60, 034008 (1999).

  * [21] K. Kutak, J. Kwiecinski, Eur. Phys. J. C 29, 521 (2003).
  * [22] K. Kutak, S. Sapeta, Phys. Rev. D 86, 094043 (2012).
  * [23] F. Hautmann, H. Jung, Nucl. Phys. B 883, 1 (2014).
  * [24] A.V. Lipatov, G.I. Lykasov, M.A. Malyshev, Phys. Rev. D 107, 014022 (2023).
  * [25] V.A. Bednyakov, G.I. Lykasov, V.V. Lyubushkin, Europhys. Lett. 92, 31001 (2010).
  * [26] V.A. Bednyakov, A.A. Grinyuk, G.I. Lykasov, M. Poghosyan, Int. J. Mod. Phys. A 27, 1250042 (2012).
  * [27] A.B. Kaidalov, Z. Phys. C 12, 63 (1982);
A.B. Kaidalov, Surveys High Energy Phys. 13, 265 (1999);
A.B. Kaidalov, O.I. Piskunova, Z. Phys. C 30, 145 (1986).

  * [28] G.I. Lykasov, M.N. Sergeenko, Z. Phys. C 52, 635 (1991);
G.I. Lykasov, M.N. Sergeenko, Z. Phys. C 56, 697 (1992);
G.I. Lykasov, M.N. Sergeenko, Z. Phys. C 70, 455 (1996).

  * [29] A.H. Mueller, Nucl. Phys. B 335, 115 (1990).
  * [30] G.R. Boroun, Eur. Phys. J. A 60, 48 (2024).
  * [31] A.V. Lipatov, G.I. Lykasov, M.A. Malyshev, Phys. Lett. B 839, 137780 (2023).
  * [32] A.V. Lipatov, M.A. Malyshev, Phys. Rev. D 108, 014025 (2023).
  * [33] A.V. Lipatov, G.I. Lykasov, M.A. Malyshev, Phys. Lett. B 848, 138390 (2024).
  * [34] N. Nikolaev, B.G. Zakharov, Z. Phys. C 49, 607 (1990).
  * [35] N. Nikolaev, E.P. Predazzi, B.G. Zakharov, Phys. Lett. D 326, 161 (1994).
  * [36] K. Golec-Biernat, M. Wüsthoff, Phys. Rev. D 59, 014017 (1998).
  * [37] K. Golec-Biernat, M. Wüsthoff, Phys. Rev. D 60, 114023 (1999).
  * [38] ATLAS Collaboration, New J. Phys. 13, 053033 (2011).
  * [39] ATLAS Collaboration, Eur. Phys. J. C 76, 502 (2016).
  * [40] CMS Collaboration, Phys. Rev. Lett. 105, 022002 (2010).
  * [41] F. Hautmann, H. Jung, S. Taheri Monfared, Eur. Phys. J. C 74, 3082 (2014).
  * [42] ATLAS Collaboration, Eur. Phys. J. C 71, 1846 (2011).
  * [43] CMS Collaboration, Phys. Lett. B 772, 306 (2017).
  * [44] H1 Collaboration, Eur. Phys. J. C 66, 17 (2010).
  * [45] ZEUS Collaboration, Phys. Lett. B 730, 293 (2014).
  * [46] CMS Collaboration, JHEP 1204, 084 (2012).
  * [47] H1 Collaboration, Eur. Phys. J. C 65, 89 (2009).
  * [48] H1 Collaboration, Eur. Phys. J. C 71, 1769 (2011).
  * [49] ZEUS Collaboration, JHEP 1409, 127 (2014).
  * [50] ZEUS and H1 Collaborations, Eur. Phys. J. C 78, 473 (2018).
  * [51] CMS Collaboration, JHEP 2307, 091 (2023).
  * [52] CMS Collaboration, JHEP 2308, 40 (2023).
  * [53] ATLAS Collaboration, Eur. Phys. J. C 80, 941 (2020).
  * [54] H1 Collaboration, Eur. Phys. J. C 66, 17 (2010).
  * [55] ZEUS Collaboration, Phys. Lett. B 730, 293 (2014).