# Percolating Cosmic String loops from evaporating primordial black holes

Ajit M. Srivastava<EMAIL_ADDRESS>Institute of Physics, Sachivalaya Marg,
Bhubaneswar 751005, India

###### Abstract

The Pulsar timing data from NANOGrav Collaboration has regenerated interest in
the possibility of observing stochastic gravitational wave background arising
from cosmic strings. Standard theory of formation of cosmic strings is based
on a spontaneous symmetry breaking (SSB) phase transition in the early
universe, with a string network forming via the so called Kibble mechanism.
This string network rapidly evolves and reaches a scaling solution with a
given spectrum of string loops and long strings at any given time. This
scenario necessarily requires that the entire observable Universe goes through
the SSB phase transition. This would not be possible, e.g., in models of low
energy inflation, where the reheat temperature is much lower than the energy
scale of cosmic strings. We point out a very different possibility, where a
network of even high energy scale cosmic strings can form when the temperature
of the Universe is much lower. We consider local heating of plasma in the
early universe by evaporating primordial black holes (PBHs). It is known that
for suitable masses of PBHs, Hawking radiation of evaporating primordial black
holes may re-heat the surrounding plasma to high temperatures, restoring
certain symmetries locally which are broken at the ambient temperature of the
Universe at that stage. Expansion of the hot plasma cools it so that the
locally restored symmetry is spontaneously broken again. If this SSB supports
formation of cosmic strings, then string loops will form in this region around
the PBH. Further, resulting temperature gradients lead to large pressure
gradients such that plasma will develop radial flow with the string loops
getting stretched as they get dragged by the flow. For a finite density of
PBHs of suitable masses, one will get local hot spots, each one contributing
to expanding cosmic string loops. For suitable PBH density, the loops from
different regions may intersect. If that happens, then intercommutation of
strings can lead to percolation, leading to the possibility of formation of
infinite string network, even when the entire universe never goes through the
respective SSB phase transition.

There is a renewed interest in cosmic strings in view of the Pulsar timing
array (PTA) data from NANOGrav Collaboration raising the possibility of
observing stochastic gravitational wave background arising from cosmic strings
pta1  (see, also, ref.pta2 ). With PTA, and upcoming Laser Interferometer
Space Antenna (LISA) lisa , search for cosmic strings using their
gravitational wave signatures has reached new levels of excitement cosmstrgw .
Formation of cosmic strings, and their possible observation, is of great
importance as they can provide a direct possible window to the physics of
ultra-high energy scales. Apart from the standard spontaneous symmetry
breaking related cosmic strings, these also arise in superstring theories
superst , making a compelling case for the existence of cosmic strings in the
Universe. In the conventional picture, strings are produced via the so called
Kibble mechanism kbl  when the universe goes through a symmetry breaking phase
transition. This string network rapidly evolves and reaches a scaling solution
with a given spectrum of string loops and long strings at any given time. This
necessarily requires that the entire observable Universe goes through the SSB
phase transition. This scenario may not always be possible, for example, in
models of inflation which have low reheat temperature, or in models of TeV
scale gravity tev .

Here we propose a very different possibility involving evaporating primordial
black holes (PBHs). There have been a host of investigations exploring the
possibility that small black holes can form in the early universe. (See, ref.
pbh0  for a recent review. Also, see refs. pbh  for early discussions.) Black
holes evaporate by emitting Hawking radiation, and for sufficiently small
black hole masses, they can completely evaporate away during early stages of
the universe. There has been extensive discussion in the literature regarding
the possibility of formation of a hot plasma surrounding primordial black
holes (see heckler1 ; heckler2 ; cline ; kapusta ). In these references the
plasma consists of black hole radiation and particles produced by the
interaction of the black hole radiation with itself. However, for PBH embedded
in an ambient thermalized plasma in the early universe, it is rather
straightforward to see that the plasma near the black hole will be heated up
as the black hole radiation propagates through the ambient plasma and loses
energy. A detailed investigation of this was carried out in ewb  (see, also
ewb2 ) utilizing known results about the energy loss of quarks and gluons
traversing a region of quark-gluon plasma eloss . (We mention here that the
results of ref.eloss  are certainly applicable for high energy partons, with
energies of several 100 GeV, it is not clear whether they can be extrapolated
to the case of Hawking radiation with energies almost near the GUT scale.
Still, the overall picture of PBH heating the local plasma to very high
temperatures may remain valid even for such large Hawking temperatures). It
was shown in ewb  that this energy loss leads to rapid heating of the plasma
near the black hole. Further, resulting temperature gradients lead to large
pressure gradients such that plasma develops radial flow. One therefore gets
the situation that evaporating primordial black holes in the early universe
lead to radially expanding hot plasma regions

For suitable masses of PBHs, Hawking radiation of evaporating primordial black
holes may re-heat the surrounding plasma to high temperatures, restoring
certain symmetries locally which are broken at the ambient temperature of the
Universe at that stage. Expansion of the hot plasma cools it so that the
locally restored symmetry is spontaneously broken again ewb . If this SSB
supports formation of cosmic strings, then string loops will form in this
region around the PBH. Further, resulting temperature gradients lead to large
pressure gradients such that plasma will develop radial flow. String loops can
get stretched as they flow with this expanding plasma. For a finite density of
PBHs of similar masses, one will get local hot spots, each one contributing to
expanding cosmic string loops. For suitable PBH density, the loops from
different regions may intersect. If that happens, then intercommutation of
strings can lead to percolation, leading to the possibility of formation of
infinite string network, even when the entire universe never goes through the
respective SSB phase transition.

Possibility of formation of primordial black holes, and various observational
constraints on them are well discussed in the literature pbh0 ; pbh . In first
order phase transitions primordial black holes can be produced by collapsing
regions of false vacuum kodamaetal  or due to inhomogeneities formed during
bubble wall collisions hawkingetal . They could also form by large amplitude
density perturbations produced due to fluctuations in the inflaton field
carrlidsey , or by shrinking cosmic string loops hawking2 .

For primordial black holes produced from large amplitude density fluctuations
produced by the inflation, one might ask whether CMBR data puts any
constraints. However, as discussed in ref. green , one should note that CMBR
data imposes restrictions on the power spectrum at large scales (of order 1 -
$10^{3}$ Mpc), whereas, the primordial black holes form due to large amplitude
and much smaller scale density fluctuations produced by the inflation. Hence
they do not affect CMBR results at all but imposes constraints on the spectral
index and consequently puts restrictions on certain models of inflation. The
constraints on primordial black hole formation come from other sources like
nucleosynthesis, gamma ray background etc. green . However, all these
phenomena essentially impose constraints on primordial black holes with a mass
range which is much larger than the ones relevant here. We consider primordial
black holes with masses at most few tonnes and such black holes evaporate away
much above the QCD scale, without causing any conflict with current
observations. Our main aim is to illustrate a completely novel possibility of
forming infinite string networks via black holes. Hence, we will simply assume
the required masses and number densities of black holes, without discussing
any specific mechanism which could give rise to the formation of such black
holes.

Consider the case when the plasma heated by the black hole leads to local
restoration of a gauge symmetry (with an energy scale and associated phase
transition temperature of order $\eta$) which allows for the existence of
cosmic strings kbl . As the plasma flows out radially away from the black
hole, its temperature will decrease and fall below $T_{c}(=\eta)$ at some
distance $r_{\eta}$ where the symmetry will be broken. Near that region,
again, string defects will form either via the Kibble mechanism, or due to
turbulence. (For internal symmetries, spatial variations of the order
parameter could lead to formation of non-trivial windings when turbulent
motion of the plasma folds up extended spatial regions, essentially
compactifying parts of spatial regions, i.e. plasma).

If the friction forces dominate over the string tension so that strings are
effectively frozen locally in the plasma then the string loops will be carried
out by the radially expanding plasma. They will then stretch out to large
sizes instead of shrinking due to tension. Note that cosmic string loops
shrink when energy can be dissipated in other modes, such as gravitational
waves, in particle emission, or dissipation in background plasma. For strong
friction case (which is likely in the early stages of hot plasma) strings can
expand, with the plasma flow increasing the energy of the expanding string
loop.

If there is a uniform density of similar primordial black holes, then each
black hole will emit string loops which will be stretched to large sizes as
they are carried away by convective flow away from each black hole. We now
recall a very crucial property of string-string interaction. It is well
established that string defects when crossing each other, intercommute. This
always happens as long as strings are not crossing with ultra-relativistic
velocities crs . Thus intercommutation will almost always occur here as string
velocities are not expected to increase beyond the sound velocity. If a string
loop emitted by one black hole intersects the string loop emitted by a nearby
black hole, this intercommutation will lead to formation of a much larger
string loop. Iteration of this process where string loops of different black
holes start intercommuting with each other leads to the remarkable possibility
that these string loops may percolate and lead to formation of an infinite
string network. This possibility is remarkable because such an infinite string
network would normally only arise when the entire universe undergoes phase
transition.

It is interesting to note here that this picture is quite the opposite of the
standard evolution picture of cosmic string network which are formed in an
overall SSB phase transition. In that case, large string loops fold on
themselves, and intercommutation at the intersecting point leads to the large
string loop breaking into smaller loops, which decay away by emitting
radiation (gravitational waves, or particles). This is the most crucial aspect
of the scaling behavior of the cosmic string network evolution. Completely
opposite to this, in our model, string loops first expand (due to expanding
plasma). Intercommutation of these expanding smaller loops leads to formation
of much larger loops, eventually forming infinite string network. However,
This reverse evolution only happens during the period when black hole
evaporation fuels expanding plasma shells. After the black holes evaporate
away (within one Hubble period of the beginning of this process), one may
expect that the final infinite string network will be similar to the
conventional string network, and may undergo same evolution, achieving scaling
solution eventually. However, there is a subtle point here refr . In the
conventional scenario, the strings form due to random variation of the order
parameter field (for U(1) case, the phase of the scalar field), in
uncorrelated domains leading to non-trivial windings. In contrast, in the
present scenario, expanding hot plasma produces string loops (which have zero
net winding for regions outside the loop) in the background of a relatively
cold Universe which initially has topologically trivial order parameter
configuration (ignoring any previously present strings). Thus, on global
scales, order parameter field behaves very differently in the two cases. It
may then be possible that there is some difference in the nature of the string
network in the two cases. We will discuss this issue further in the following,
after discussing the details of the model.

Let us discuss the conditions for percolation of strings. A black hole of mass
$M_{bh}$ evaporates by emitting Hawking radiation with an associated
temperature

$T_{bh}={M_{Pl}^{2}\over 8\pi M_{bh}}$ (1)

Here $M_{Pl}=1.2\times 10^{19}GeV$ is the Planck mass. We use natural units
with $\hbar$ = c = 1. The rate of loss of mass by the evaporating black hole
is given by

${dM_{bh}\over dt}=-{\alpha M_{Pl}^{4}\over M_{bh}^{2}}~{}.$ (2)

Here, $\alpha$ accounts for the scattering of emitted particles by the
curvature and depends on $T_{bh}$. For different values of $T_{bh}$ values of
$\alpha$ have been tabulated in scat . For $T_{bh}=1,~{}200$, and 1015 MeV,
the corresponding values of $\alpha$ are $3.6\times 10^{-4}$, $2.3\times
10^{-3}$ and 4.5 $\times 10^{-3}$ respectively. For the range of black hole
temperatures for our model, we will set $\alpha$ to be $3\times 10^{-3}$. The
lifetime $\tau_{bh}$ of the black hole can be obtained by integrating Eq.(2).
We get

$\tau_{bh}\simeq 10^{2}M_{Pl}^{-4}M_{0}^{3}.$ (3)

where $M_{0}$ is the initial mass of the black hole. Eq.(2) implies that very
little energy is emitted until time of the order of $\tau_{bh}$ which is when
most of the energy of the black hole gets emitted. Thus for a black hole
formed early in the Universe, it is reasonable to assume that the black hole
essentially evaporates only when the age of the Universe is of order
$\tau_{bh}$.

Let us assume that black holes formed in the early Universe have masses so
that they evaporate when the temperature of the Universe is $T_{U}$. The age
of the Universe $t_{U}$ when its temperature is $T_{U}$ is

$t_{U}\simeq 0.3g_{*}^{-1/2}M_{Pl}T_{U}^{-2},$ (4)

where $g_{*}(\simeq 100$) is the number of relevant degrees of freedom kolb .
By equating $\tau_{bh}=t_{U}$, we can get the mass $M_{0}$ of the black hole
such that its evaporation becomes effective when the temperature of the
Universe is $T_{U}$,

$M_{0}=0.07M_{Pl}^{5/3}T_{U}^{-2/3}~{}.$ (5)

Black hole with this mass will have the temperature,

$T_{bh}=0.6M_{Pl}^{1/3}T_{U}^{2/3}\,.$ (6)

For example, with $T_{U}$ = 1 GeV we get $M_{0}=4\times 10^{11}M_{Pl}$,
$T_{bh}=10^{6}$ GeV and $\tau_{bh}=5\times 10^{17}GeV^{-1}=3\times 10^{-7}$ s.
The picture then is that these black holes emit particles with energies
roughly equal to $T_{bh}$ (= $10^{6}$ GeV for this sample case) into the
background plasma which is at a temperature $T_{U}$ ($\sim$ 1 GeV) to start
with. These $10^{6}$ GeV particles will scatter with the particles in the
background plasma and will heat it up through their energy loss. For the black
hole masses considered here only elementary particles will be emitted, such as
quarks, gluons, photons, leptons, etc. (We mention here that the mass of a
primordial black hole can also increases due to accretion of background plasma
particles. However, for relevant ranges of black hole masses here, all the
dominant particle species are ultra relativistic, so only the geometric cross-
section of black hole is relevant which is not very effective acrt . Some slow
growth of the mass of the black hole could occur because of this, and the
black hole masses we use here should be taken to be the final mass of the
black hole when its evaporation becomes effective.)

In ref. ewb , conditions for the equilibration of the Hawking radiation in the
ambient plasma were discussed. Further, estimates of plasma velocity were made
in ewb  using the Euler equation for a relativistic fluid and it was shown
that plasma velocity rapidly becomes relativistic. This is natural to expect
because of sharp temperature gradients close to the evaporating black hole.
Assuming a steady state situation where the luminosity $L(r)$ is independent
of $r$ and equal to $-dM_{bh}/dt$, we can get the relation between the
temperature $T$ of the expanding plasma shell and the radius $r$ of the shell
(see, ref.ewb  for details).

$r=\Biggl{[}{L\over\gamma^{2}(8\pi^{3}g_{*}/45)T^{4}v}\Biggr{]}^{1\over 2}\,.$
(7)

where $\gamma$ is the Lorentz factor for the plasma velocity. This expression
is valid for distances where bulk plasma flow dominates over diffusion of
particles. As shown in ewb , this will be applicable for the entire range of
distances of relevance to our scenario in the present paper. We will take the
plasma velocity to be of order sound velocity $v\simeq 1/\sqrt{3}$. Though it
is important to realize that much larger velocities will be expected during
last explosive stages of black hole evaporation, and shocks should develop
during those stages. We will not consider shocks to keep calculations simple.
(Hence, for estimates, we ignore the Lorentz factor in the above equation.)

Eq.(6) gives the temperature of the black hole when its evaporation becomes
significant. Within one Hubble time the entire black hole evaporates away.
However, Eqs.(1),(2) show that as the black hole mass decreases, its
temperature, and the rate of energy loss, also increase. Thus it is the last
stages of black hole evaporation which will lead to highest temperatures for
the plasma in the nearby region. As our interest is in determining how far the
string loops can be dragged by expanding hot shells, we need to consider the
plasma flow during these last stages of black hole evaporation. Initial mass
$M_{0}$ of the black hole is determined by the ambient temperature $T_{U}$
when black hole evaporation becomes significant. However, we consider the
situation only when its mass has reduced to a value $M_{x}$, given by

$M_{x}={M_{0}\over x},~{}~{}~{}x>1$ (8)

$M_{x}$ is determined by the following considerations. Let us assume that the
cosmic strings under consideration correspond to a symmetry breaking scale
$\eta$. (For simplicity we will consider gauge strings for the estimation of
string tension, but ignore the length scales associated with gauge fields.)
Thus black holes which are relevant to our scenario must have temperatures
larger than $\eta$. Strings will be produced in expanding shells as they cross
the critical distance $r_{\eta}$ where temperature drops to a value below the
critical temperature, which we take to be of order $\eta$. For the self
consistency of this picture, we must have the thickness $\Delta r$ of the
plasma shell to be at least of the order of $\eta^{-1}$ which is roughly the
thickness of the string. With a spherical shell around the black hole, with
this thickness, several correlation domains will then form, leading to string
formation via the Kibble mechanism kbl . Thickness of the shell is determined
by the life time of the remaining black hole with mass $M_{x}$. We thus
require (using Eq.(3)),

$100M_{pl}^{-4}{M_{0}^{3}\over x^{3}}\geq\eta^{-1}$ (9)

Initial temperature $T_{x}$ of this remaining black hole is given by Eq.(1)
with $M_{bh}=M_{x}$ (Eq.(8)). We take this temperature to be effectively at a
distance of order of the Schwarzschild radius $r_{x}={2\over
M_{pl}^{2}}{M_{0}\over x}$. (This is reasonable within an order of magnitude,
see discussion in ewb .) Using the temperature distance relationship from
Eq.(7), we can get the temperature of the plasma from this remaining (last
stage) of the black hole as,

$T(r)\simeq 0.06M_{pl}~{}\sqrt{{x\over rM_{0}}}$ (10)

We mention here that though Eq.(7) (and hence Eq.(10)) was derived in ref.ewb
with considerations of bulk plasma and particle diffusion, essentially similar
profile is obtained if we take $T=T(r_{x})$ effectively at a distance of order
of the Schwarzschild radius $r_{x}={2\over M_{pl}^{2}}{M_{0}\over x}$ along
with the condition $T^{4}r^{2}=$ constant. (Which essentially means
conservation of energy-momentum of the expanding shells). We will be
considering black hole masses such that $T_{x}>\eta$. Temperature will drop to
the value $\eta$ at a distance $r_{\eta}$ which is obtained by setting
$T(r_{\eta})=\eta$ in the above equation. We get,

$r_{\eta}=({0.06M_{pl}\over\eta})^{2}({x\over M_{0}})$ (11)

String loops will be produced at this distance. We then expect these string
loops to be dragged (and consequently stretched) by expanding plasma. At
certain distance $r_{U}$, the temperature will drop to the ambient value
$T_{U}$ and plasma flow should stop roughly at that distance. String
stretching is not possible beyond distance of this order. $r_{U}$, therefore,
sets an upper limit on the distance $R_{stretch}$ up to which strings can be
carried out by plasma flow. Using Eqs.(5),(10), we can get $r_{U}$ by setting
$T(r_{U})=T_{U}$. we get,

$r_{U}\simeq 0.05x{M_{pl}^{1/3}\over T_{U}^{4/3}}$ (12)

It should be clear that $r_{U}$ sets an upper limit on the distance
$R_{stretch}$ up to which strings can be carried out by plasma flow. To
determine $R_{stretch}$ we need to find the condition when the string motion
is dominated by friction forces. Friction forces on strings arise from the
scattering of plasma particles from the string. It has been discussed in the
literature that the dominant contribution to this comes from Aharanov-Bohm
scattering (of particles with appropriate). For simplicity, we will assume
this to be the case. Friction force per unit length on the cosmic string is
then given by frict ,

$F_{frict}\sim\beta T^{3}{v\over\sqrt{1-v^{2}}}~{},$ (13)

where $v$ is the string velocity through the plasma, $T$ is the plasma
temperature, and $\beta$ is a numerical parameter related to the number of
relevant particle species frict . In specific grand unified theory models
fracq  it could be of order one, and we will assume that to be the case here.
(See, also, ref. evrt  for discussions of scattering of particles from
strings.) We mention here that stretching of string due to plasma flow has
been discussed by Chudnovsky and Vilenkin in ref. curv  where they considered
light superconducting cosmic strings getting stretched due to turbulent plasma
flow in the galactic disc and even in stellar interiors. One could also
consider formation and stretching of such strings around primordial black
holes. This will require considerations of magnetic fields in the cosmic
plasma etc.

We consider a simple situation when a string encircling the black hole is
stretched symmetrically by radial flow of the plasma. (Order of magnitudes of
our estimates should remain same for other geometries of strings, e.g. string
ends may lie on the boundary of the symmetry restored region around the black
hole) Such large strings should easily form during string formation on the
surface of the sphere with radius $r_{\eta}$ near the black hole. String tries
to collapse due to its tension. For a string loop of radius $R$, one can
estimate the tension force as curv ,

$F_{tension}\sim{\mu\over R}~{}=~{}{\eta^{2}\over R}$ (14)

At the formation stage, near $r\simeq r_{\eta}$, the string will be at rest
w.r.t local plasma frame. In such a situation $v$ in Eq.(13) will be zero and
there will be no friction force on the string. This will lead to collapse of
the string under its tension. As the string loop starts collapsing, it will
develop non-zero velocity w.r.t. the local plasma frame, and consequently a
non-zero friction force via Eq.(13). As the plasma flow velocity is taken to
be the sound velocity (= $1/\sqrt{3}$) w.r.t. the black hole rest frame, we
can deduce that the friction force on the string will be large as the string
collapses, and will decrease as string collapse decreases. When string is also
at rest w.r.t. the black hole, then the friction force will be given by
Eq.(13) with $v=1/\sqrt{3}$, and should balance the string tension force
(Eq.(14)). String loop will expand because of plasma drag as long as
$F_{frict}(v=1/\sqrt{3})>F_{tension}$. Using $r$ dependence of the temperature
from Eq.(10), this condition gives the upper limit for $R$ as,

$R<5\times 10^{-8}{\beta^{2}x^{3}M_{pl}^{6}\over M_{0}^{3}\eta^{4}}\equiv
R_{max}$ (15)

As plasma flow stops beyond a distance of order $r_{U}$, we conclude that
plasma flow can stretch string loops to sizes of order $R_{stretch}\simeq
min(R_{max},r_{U})$. We note that the largest values of $R_{max}$, (as well as
$r_{U}$ (Eq.(12)), and $r_{\eta}$ (Eq.(11)), correspond to the largest value
$x_{max}$ of $x$ obtained by taking equality sign in Eq.(9), and we will use
this.

We should also consider the effect of black hole gravity on the string loops.
Gravitational force per unit length on the string due to the black hole of
mass $M_{bh}$ can be roughly estimated as ${GM_{bh}\mu\over R^{2}}$ where R is
the separation of the string segment from the black hole. (We are taking
string as a simple gravitating system, as for a string loop. For special
geometries, such as for a straighter string the gravitational force will be
different, and will be typically smaller). We see that, per unit length, this
gravitational force becomes much less than the force $f_{tension}\sim{\mu\over
R}$ due to string tension when $R>GM_{bh}$. Thus for any distances much larger
than the Schwarzschild radius of the black hole, gravity of the black hole is
sub-dominant compared to string tension forces. We are considering the
situation when plasma drag forces completely dominate over the string tension
forces, leading to stretching of string to large distances (where black hole
gravity becomes even more negligible). Thus black hole gravity is not relevant
for string dynamics in our case.

If black hole number density is such that inter-black hole separation $d_{bh}$
is smaller than $R_{stretch}$ then loop intersections will be frequent. As
string velocity is not expected to be ultra-relativistic here (neglecting
shocks), this will lead to intercommutation of strings and hence percolation
of strings resulting in an infinite string network. By assuming that energy
density of these black holes (with mass $M_{0}$) contributes only a fraction
$f$ of the total energy density of the universe, we get (with $g_{*}\simeq
100$),

$d_{bh}\simeq 0.13{M_{pl}^{5/9}\over f^{1/3}T_{U}^{14/9}}$ (16)

Percolation of loops (leading to infinite string network formation) will
happen when $d_{bh}<R_{stretch}$. We can now determine conditions on various
parameters which satisfy all the constraints for string percolation. For given
values of the fraction $f$, and string scale $\eta$, the condition for
percolation ($d_{bh}<R_{stretch}$) shows that percolation can happen if the
Universe temperature $T_{U}$ lies in the range
$T^{min}_{U}<T_{U}<T^{max}_{U}$, which is determined as follows (using largest
value of $x$ from Eq.(9), as we discussed above). Note that as
$R_{stretch}\simeq min(R_{max},r_{U})$, condition for percolation requires
that $d_{bh}$ should be smaller than both $r_{U}$ and $R_{max}$.

If $R_{stretch}=r_{U}$, then the condition $d_{bh}<r_{U}$ implies

$T_{U}<3\times 10^{-2}f^{3/4}M_{pl}^{1/4}\eta^{3/4}\equiv T^{max}_{U}$ (17)

If $R_{stretch}=R_{max}$, then the condition $d_{bh}<R_{max}$ implies

$T_{U}>{1.6\times 10^{2}\eta^{27/14}\over M_{pl}^{13/14}f^{3/14}}\equiv
T^{min}_{U}$ (18)

Table 1: Allowed parameters (in GeV units) for percolation of string loops f | $\eta$ | $T_{U}$ | $r_{\eta}$ | $r_{U}$ | $R_{max}$ | $d_{bh}$ | $M_{0}/M_{pl}$ | $M_{x}/M_{pl}$
---|---|---|---|---|---|---|---|---
1.0 | $10^{10}$ | $10^{5}$ | $1.9\times 10^{-6}$ | $1.9\times 10^{4}$ | $7.4\times 10^{2}$ | 87.0 | $1.7\times 10^{8}$ | 230.0
$10^{-7}$ | $10^{10}$ | $10^{6}$ | $1.9\times 10^{-6}$ | 190.0 | 740.0 | 520.0 | $3.7\times 10^{7}$ | 230.0
$10^{-6}$ | $10^{10}$ | $5\times 10^{5}$ | $1.9\times 10^{-6}$ | 740.0 | 740.0 | 710.0 | $5.9\times 10^{7}$ | 230.0
1.0 | $10^{7}$ | 1.0 | 0.19 | $1.9\times 10^{13}$ | $7.4\times 10^{11}$ | $5.2\times 10^{9}$ | $3.7\times 10^{11}$ | $2.3\times 10^{3}$
$10^{-3}$ | $10^{7}$ | 1.0 | 0.19 | $1.9\times 10^{13}$ | $7.4\times 10^{11}$ | $5.2\times 10^{10}$ | $3.7\times 10^{11}$ | $2.3\times 10^{3}$
$10^{-7}$ | $10^{7}$ | 10.0 | 0.19 | $1.9\times 10^{11}$ | $7.4\times 10^{11}$ | $3.1\times 10^{10}$ | $8.0\times 10^{10}$ | $2.3\times 10^{3}$
1.0 | $10^{4}$ | 1.0 | $1.9\times 10^{4}$ | $1.9\times 10^{12}$ | $7.4\times 10^{20}$ | $5.2\times 10^{9}$ | $3.7\times 10^{11}$ | $2.3\times 10^{4}$
$10^{-3}$ | $10^{4}$ | 1.0 | $1.9\times 10^{4}$ | $1.9\times 10^{12}$ | $7.4\times 10^{20}$ | $5.2\times 10^{10}$ | $3.7\times 10^{11}$ | $2.3\times 10^{4}$
$10^{-7}$ | $10^{4}$ | 1.0 | $1.9\times 10^{4}$ | $1.9\times 10^{12}$ | $7.4\times 10^{20}$ | $1.1\times 10^{12}$ | $3.7\times 10^{11}$ | $2.3\times 10^{4}$
$10^{-10}$ | $10^{7}$ | 7.0 | 0.19 | $3.8\times 10^{11}$ | $7.4\times 10^{11}$ | ${\bf 5.4\times 10^{11}}$ | $1.0\times 10^{11}$ | $2.3\times 10^{3}$
$10^{-10}$ | $10^{5}$ | 0.1 | 410.0 | $4.0\times 10^{14}$ | $7.4\times 10^{17}$ | $4.0\times 10^{14}$ | $1.7\times 10^{12}$ | $1.1\times 10^{4}$
$10^{-10}$ | $10^{5}$ | 0.05 | 410.0 | $1.6\times 10^{15}$ | $7.4\times 10^{17}$ | $1.2\times 10^{15}$ | $2.7\times 10^{12}$ | $1.1\times 10^{4}$

In Table 1 we have given several sets of values for various parameters for
which string percolation happens. We see that for all these cases,
$R_{stretch}>>r_{\eta}$, (where $R_{stretch}\simeq min(R_{max},r_{U})$). This
means that, after getting produced near $r_{\eta}$, the string loops are
dragged (via friction forces) and stretched by the plasma flow up to very
large distances. There are several points to be noted from numbers given in
table I. We have included two cases, represented by boldfaced digits in the
column of $d_{bh}$. For $d_{bh}=520.0$ GeV-1 and $5.4\times 10^{11}$ GeV-1,
string loops will not percolate. This is because in these cases, though
$d_{bh}$ is smaller than $R_{max}$, but it is larger than $r_{U}$. As string
loops can get stretched only up to a distance $R_{stretch}\simeq
min(R_{max},r_{U})$), in these cases, loops emanating from different black
holes will not intersect, hence no possibility of percolation. We have also
chosen parameters to show cases such that in some case $r_{U}>R_{max}$, while
in other cases $R_{max}>r_{U}$, (both lengths being larger than $d_{bh}$
allowing for string percolation).

We also see that the percolation of strings is not a rare occurrence, but
happens for rather generic set of parameter values. It is important to see
that strings with energy scale $\eta=10^{7}$ GeV can percolate and form an
infinite network when the ambient temperature of the Universe is as low as
$T_{U}=1$ GeV. Similarly, strings with $\eta=10^{5}$ GeV scale can form with
$T_{U}=50$MeV. (Though, for such low temperatures, one should change value of
$g_{*}$ in Eq.(4) etc.). Further, even with almost negligible fraction of
energy density in black holes, $f=10^{-10}$, string percolation is achieved.

However, for large values of $\eta$, percolation is achieved only during very
late stages of black hole evaporation when its mass has become very small
(i.e. $x$ becomes very large, with mass of black hole in this late stage being
$M_{0}/x$), only few 100 times the Planck mass. Eqs.(1),(2) describe black
hole evaporation process when quantum gravity effects have been neglected. It
can then be of concern whether these results can be used for such late stages
of black hole evaporation when its mass is not very much larger than the
Planck mass. We, therefore, consider smaller values of $\eta$ also, for which
much larger black holes masses can also lead to string percolation. Though, we
mention that the numbers given in table I do not represent any optimized set
of values of various parameters. These are just taken as sample examples,
showing that string percolation in this model can happen for a range of
parameter values. For example, small values of $M_{x}$ here are due to the
fact that we take maximum allowed value of $x$ from the inequality in Eq.(9).
We do that so that we get largest values of $r_{U}$ and $R_{max}$ (as we
mentioned above). However, even with smaller values of $x$ string percolation
will be possible, if suitable values of other parameters (e.g. $T_{U}$) are
chosen. We have also ignored shock formation in the expanding plasma. During
last stages of exploding black hole, shocks will be expected and they can
significantly add to the possibility of expanding string loops attaining large
sizes. A more detailed investigation is needed for all these considerations to
better optimize these parameter values.

We mention that we have considered here black holes of similar masses as an
example. Clearly, the basic physics of this model will remain applicable for
black holes with a range of masses. Though, simplest situation for this model
will be to have a black hole mass spectrum which is peaked at some specific
value, for example, if the black holes form in a first order phase transition
hawkingetal .

We now come back to the issue we mentioned above in the beginning relating to
possible differences between the final infinite string network in the present
model, and the conventional string network formed via the Kibble mechanism.
First, we clarify that we have not actually demonstrated that an infinite
string network will necessarily form in this scenario. Even if all
requirements for overlapping string loops (arising from different black holes)
are satisfied, including intercommutation at string intersection points, one
still needs to check that intersecting loops do percolate to give rise to
infinite string network. Proper answer for this question can be given by a
detailed numerical simulation of this scenario. Even if we accept that an
infinite string network will form in this scenario, there are reasons to
expect that there may be important differences in the nature of string network
on large scales refr . In the conventional scenario of Kibble mechanism, the
starting point is the existence of uncorrelated domains after an overall phase
transition in the Universe, with random variation of the order parameter field
(say, the phase of scalar field for U(1) strings) across different domains.
This, with the use of geodesic rule, leads to non-trivial windings, and hence
the formation of strings. We may think of this conventional scenario as top to
bottom model, i.e. ”from order parameter field variations to strings” model. A
very important feature of this scenario is that non-trivial windings are
present at all scales larger than the correlation domains, and in particular
they are necessarily present at scales larger than the horizon (Hubble) scale
refr . Indeed, precisely this causality argument was the one which led to the
monopole problem in the Universe, and which strongly constrains the domain
wall models. The situation, in this respect, is very different in the scenario
we have discussed in this paper. Here, PBHs evaporate in a relatively cold
Universe where the order parameter field has settled to a roughly uniform,
topologically trivial, configuration (ignoring any previously present
strings). Starting point here is small string loops forming around evaporating
black holes, which have topologically trivial winding numbers on scales larger
than the loop size. Even during the period when these loops expand, due to
plasma expansion, order parameter field remains topologically trivial on
larger scales, and this remains true even if string loops from different black
holes intersect and percolate. This scenario may thus be taken as bottom to
top model, i.e. ”from string loops to order parameter field variations” model.
Even if an infinite string network forms in this scenario, it is no longer
clear if actual distribution of order parameter field on large scales, in
particular on Hubble scales, is the same here as in the conventional case. If
there are important differences, it will raise important questions whether the
nature of the infinite string network in the two cases has any crucial
differences. Though, approach to scaling solution does not retain much memory
of the initial string network, as it involves robust steps, like folding of
large loops to self intersection, and small loops radiating away. Thus one may
expect that such a scaling solution may be expected in the present scenario
also. Still, it will be very interesting to explore this issue in detail
whether the final string networks in the two cases, even in the scaling
regime, differ in any qualitative manner. It is possible that the cases of
gauge strings and global strings may behave very differently in this respect.
One may expect stronger differences in the two scenarios of string formation
for the global string case, where order parameter field variation plays a more
direct role in the string network evolution. In comparison, for gauge string
case, the string tension is the primary relevant factor for late stages, thus
making it more likely that the string networks in these two scenarios may have
similar late time evolution for the gauge string case.

We conclude by emphasizing that our results present a possible scenario for
the formation of an extended (infinite) cosmic string network in the Universe
without the entire Universe undergoing a spontaneous symmetry breaking phase
transition. Importantly, this happens when the ambient temperature of the
Universe is much below the relevant spontaneous symmetry breaking scale.
String loops are generated by evaporating primordial black holes leading to
local symmetry restoration. Expansion of locally heated plasma cools it,
leading to symmetry breaking (again locally), producing cosmic string loops.
Expanding plasma stretches these loops, and intersection of loops originating
from different regions (heated by different PBHs) can lead to percolation,
hence the formation of an extended (infinite) string network without an
overall phase transition. Though there appear to be some important differences
in the distribution of the order parameter field on large scales in this
scenario, as compared to the conventional Kibble mechanism (as discussed
above), the robust mechanisms responsible for achieving a scaling solution for
the string network may hold out here also leading to the standard scaling
solution. It needs to be further explored if, even in the scaling regime,
there are any important differences between the string networks in the two
cases.

I would like to thank R. Rangarajan, S. Digal, S. Sengupta, R. Ray, B. Layek,
and A.P. Mishra for useful discussions and comments.

## References

  * (1) J. Ellis and M. Lewicki, Phys. Rev. Lett. 126, 041304 (2021).
  * (2) L. Bian, et al. Phys. Rev. D 106, L101301 (2022).
  * (3) P. Auclair et al, JCAP04, 034 (2020).
  * (4) Y. Cui, M. Lewicki, D.E. Morrissey, and J.D. Wells, Phys. Rev. D 97, 123505 (2018).
  * (5) A.C. Davis and T.W.B. Kibble, Contemp. Phys. 46, 313 (2005); E. J. Copeland, R. C. Myers, and J. Polchinski, JHEP 0406, 013 (2004); S. Sarangi and S.H. Henry Tye, Phys. Lett. B 536, 185 (2002).
  * (6) T.W.B. Kibble, J. Phys. A9, 1387 (1976); Phys. Rept. 67, 183 (1980).
  * (7) N. Arkani-Hamed, S. Dimopoulos, and G.R. Dvali, Phys.Rev. D 59, 086004 (1999).
  * (8) E. Bagui et al. [LISA Cosmology Working Group], arXiv: astro-ph.CO/2310.19857.
  * (9) S.W. Hawking, Nature 248, 30 (1974); S. W. Hawking, Mon. Not. R. Astr. Soc. 152 (1971) 75; B.J. Carr, Astrophys. J. 201, 1 (1975); I. G. Moss, Phys. Rev. D 50 (1994) 676.
  * (10) A. F. Heckler, Phys. Rev. D 55 (1997) 480.
  * (11) A. F. Heckler, Phys. Rev. Lett. 78 (1997) 3430.
  * (12) J. M. Cline, M. Mostoslavsky and G. Servant, Phys. Rev. D 59 (1999) 063009.
  * (13) J. Kapusta, astro-ph/9911309 (1999).
  * (14) R. Rangarajan, S. Sengupta, and A.M. Srivastava, Astropart. Phys. 17, 167 (2002).
  * (15) Y. Nagatani, Phys. Rev. D 59, 041301 (1999).
  * (16) M.H. Thoma, Phys. Lett. B 273, 128 (1991).
  * (17) H. Kodama, M. Sasaki and K. Sato, Prog. Theor. Phys. 68, 1979 (1982), and references therein.
  * (18) S. W. Hawking, I. G. Moss and J. M. Stewart, Phys. Rev. D 26, 2681 (1982); J. D. Barrow, E. J. Copeland, E. W. Kolb and A. R. Liddle, Phys. Rev. D43, 984 (1991).
  * (19) B. J. Carr and J. E. Lidsey, Phys. Rev. D48, 543 (1993).
  * (20) S. W. Hawking, Phys. Lett. B231, 237 (1989); A. Polnarev and R. Zembowicz, Phys. Rev. D43, 1106 (1991).
  * (21) A.M. Green and A.R. Liddle, Phys. Rev. D56, 6166 (1997).
  * (22) E.P.S. Shellard, Nucl. Phys. B 283, 624 (1987).
  * (23) We thank an anonymous referee for pointing out this important difference between our scenario and the conventional case.
  * (24) I.G. Moss, Phys. Rev.D 32 (1985) 1333 (see Table 1, based on D. Page, Phys. Rev. D 13 (1976) 198 on emission of neutral particles by a black hole. The emission rates are not very different for massless charged particles, as shown in D. Page, Phys. Rev. D 16 (1977) 2402).
  * (25) E.W. Kolb and M.S. Turner, “The Early Universe”, (Addison-Wesley, Redwood City, California, 1990).
  * (26) N. Upadhyay, P. Das Gupta and R. P. Saxena, Phys. Rev. D60 (1999) 063513.
  * (27) A. Vilenkin, Phys. Rev. D 43, 1060 (1991); C.J.A.P. Martins, and E.P.S. Shellard, Phys. Rev. D 53 R575 (1996).
  * (28) M.G. Alford and F. Wilczek, Phys. Rev. Lett. 62, 1071 (1989).
  * (29) A.E. Everett, Phys. Rev. D 24, 858 (1981).
  * (30) E. Chudnovsky, and A. Vilenkin, Phys. Rev. Lett. 61, 1043 (1988).


#  Concrete: Improving Cross-lingual Fact-checking with
Cross-lingual Retrieval

Kung-Hsiang Huang ChengXiang Zhai Heng Ji
Department of Computer Science, University of Illinois Urbana-Champaign
{khhuang3, czhai<EMAIL_ADDRESS>

###### Abstract

Fact-checking has gained increasing attention due to the widespread of
falsified information. Most fact-checking approaches focus on claims made in
English only due to the data scarcity issue in other languages. The lack of
fact-checking datasets in low-resource languages calls for an effective cross-
lingual transfer technique for fact-checking. Additionally, trustworthy
information in different languages can be complementary and helpful in
verifying facts. To this end, we present the first fact-checking framework
augmented with cross-lingual retrieval that aggregates evidence retrieved from
multiple languages through a cross-lingual retriever. Given the absence of
cross-lingual information retrieval datasets with claim-like queries, we train
the retriever with our proposed Cross-lingual Inverse Cloze Task (X-ICT), a
self-supervised algorithm that creates training instances by translating the
title of a passage. The goal for X-ICT is to learn cross-lingual retrieval in
which the model learns to identify the passage corresponding to a given
translated title. On the X-Fact dataset, our approach achieves 2.23% absolute
F1 improvement in the zero-shot cross-lingual setup over prior systems. The
source code and data are publicly available at
https://github.com/khuangaf/CONCRETE.

##  1 Introduction

Fact-checking is an important task that assesses the veracity of a claim. This
task has gained increasing attention due to the widespread mis- and dis-
information that has a significant socioeconomic impact on our society
Scheufele and Krause (2019); Pate et al. (2019); Fung et al. (2021); Guo et
al. (2022); Wu et al. (2022); Fung et al. (2022); Huang et al. (2022). While
fact-checking is mainly conducted manually, especially in the journalism
industry, with more than hundred of millions of social media posts and
millions of blog posts published per day Hoang and Mothe (2018); Djuraskovic
(2022), manual fact-checking is no longer feasible. Hence, we are urgently in
need of reliable automated fact-checking approaches.

Figure 1: An overview of the proposed framework. Given a claim in arbitrary
language, a cross-lingual retriever, Concrete retrieves relevant passages in
any languages. The top-$k$ relevant passages and the claim are then passed to
our multilingual reader, mBERT, to predict the veracity of the claim.

Most existing work develops fact-checking approaches on English-only corpus
Wang (2017); Thorne et al. (2018); Augenstein et al. (2019); Wadden et al.
(2020). One reason for this is the scarcity of fact-checking websites in other
languages for constructing large enough non-English fact-checking datasets.
Therefore, it is even more challenging to build fact-checkers for low-resource
languages. A solution is to leverage high-resource languages with zero-shot
cross-lingual transfer, where the model is trained on source languages of
richer resources and directly tested on target languages of lower resources.
Note that the sets of languages in the training set and the test sets are
disjoint. With this technique, ground-truth labels for claims in low-resource
languages are no longer required.

Few studies in the fact-checking literature have explored the cross-lingual
setup. One line of work attempts to match an input claim with claims in other
languages that have been verified Kazemi et al. (2021, 2022). However, this
approach fails when the claims have not been fact-checked in any language.
Another line of work builds classifiers with multilingual language models. For
example, Gupta and Srikumar (2021) utilize Google Search to obtain snippets
that are relevant to the input claim and train a model based on mBERT Devlin
et al. (2019) in a cross-lingual setting. Although Google Search excels at
retrieving relevant information from a given claim, it disregards the
trustworthiness of the information being retrieved. When a claim is erroneous,
the search results often contradict each other, which likely leads to
incorrect predictions, as shown later in Section 5.3.

Motivated by these challenges, we propose Concrete, a Claim-oriented Coss-
lingual Retriever that retrieves evidence from a trustworthy multilingual
passage collection for fact-checking. This approach can handle region-specific
claims as long as relevant evidence is presented in the passage collection,
which is much more accessible compared to similar claims in other languages.
In addition, since it does not rely on a black-box retrieval system, our
approach provides the flexibility to include only trustworthy information in
the passage collection. One major challenge for training such a retriever is
the lack of multilingual information retrieval (IR) dataset with claim-like
queries. To this end, we propose a self-supervised cross-lingual learning
algorithm, Cross-lingual Inverse Cloze Task (X-ICT), to learn the retriever
based on pseudo-feedback. To mimic cross-lingual retrieval, we construct a
pseudo-query for a given passage by translating its title into a randomly
selected language. The objective for X-ICT is to identify the passage
corresponding to a given translated title among all candidate passages. Since
the title of news articles can often be regarded as a claim, this approach
mitigates the domain discrepancy issue. As shown in Figure 1, our framework
first performs cross-lingual retrieval to obtain evidence relevant to the
input claim. Then, a multilingual reader takes in the retrieved evidence and
the input claim to classify the veracity of the claim.

Our contributions can be summarized as follows:

  * •

To the best of our knowledge, we present the first fact-checking framework
augmented with cross-lingual retrieval that achieves state-of-the-art cross-
lingual transfer performance on the X-Fact fact-checking task.

  * •

We propose Concrete, a cross-lingual retriever with a bi-encoder architecture
learned through a proposed self-supervised learning algorithm.

  * •

Our experiments reveal that the distance between the input claim and the
retrieved passages is strongly correlated with the performance.

  * •

We collected a multilingual passage collection composed of reliable news
articles in seven different languages. We have demonstrated that this corpus
is effective for retrieval-augmented fact-checking.

##  2 Task Definitions

The input is a claim $c$ in an arbitrary language and the corresponding
metadata, such as claimer and claim date. Based on $c$, the retriever
component of our model retrieves relevant passages $p$ from a multilingual
passage collection $P$ where $p$ can be in any language. Then, the reader
component takes in the claim $c$, the corresponding metadata, and relevant
passages $p$ to predict the veracity of $c$. Note that the use of passage
collection $P$ to aid in fact-checking is a modeling choice, as no grounded
evidence is available.

We aim to build a passage collection that does not contain erroneous
information. Hence, we construct $P$ by crawling 49,000 articles published in
7 languages111We consider Arabic, Russian, Indonesian, Persian, French, and
Portuguese. between September 2016 and December 2022 from a trustworthy news
media, bbc.com, and split each article into passages. Each passage contains at
most 100 tokens, following Karpukhin et al. (2020). This results in a total of
347,557 passages. The collected passage collection has been made publicly
available in the link mentioned in the Abstract.

##  3 Proposed Method

Our framework is a pipeline consisting of two components: (1) Concrete, a
claim-oriented cross-lingual retriever that retrieves relevant passages from a
multilingual passage collection, and (2) a multilingual reader that determines
the veracity of a claim based on the compatibility of the claim and the
passages retrieved. Figure 1 shows an overview of the proposed method. The
following sections describe each component in detail.

###  3.1 Concrete

Concrete takes in a claim $c$ in an arbitrary language as input and retrieves
$k$ relevant passages $p=\\{p_{0},...,p_{k}\\}$ from a multilingual passage
collection $\mathcal{P}$, where the retrieved passages can be in any language.
We adapt mDPR Asai et al. (2021b), a multilingual retriever, with the proposed
self-supervised learning algorithm, Cross-lingual Inverse Cloze Task, for
claim-like queries.

#### mDPR

mDPR is a multilingual version of the Dense Passage Retriever (DPR) Karpukhin
et al. (2020). Similar to DPR, mDPR is a bi-encoder architecture that computes
the relevance score between a query and a passage with the inner product of
the corresponding representations. In addition to being trained on existing
multilingual question answering datasets, mDPR also learns from additional
samples mined from Wikipedia and labeled with the proposed answer generator
discussed in Asai et al. (2021b). However, since mDPR was trained on datasets
where queries are questions instead of claims, domain mismatch becomes an
issue if we directly apply mDPR to our task.

#### Cross-lingual Inverse Cloze Task

To address the domain discrepancy problem discussed in the previous paragraph,
a naive solution is to fine-tune mDPR on multilingual IR datasets with claim-
like queries. Unfortunately, such datasets are not available. Motivated by the
Inverse Cloze Task Lee et al. (2019), which was used to warm start the
retriever by tasking a retriever to predict the context given a randomly
sampled sentence, we propose Cross-lingual Inverse Cloze Task (X-ICT) by
extending ICT to a cross-lingual setup. Specifically, we made two major
modifications to the original ICT.

First, instead of randomly sampling a sentence as the pseudo query, we treat
the title of each passage as the pseudo query as these titles are often claim-
like sentences. The positive passages for a given title are the passages
derived from the same article. The biggest advantage of this approach is that
there is very little domain mismatch between using claims as queries and using
titles as queries. Therefore, the retriever optimized with our proposed X-ICT
can be directly applied to claim-oriented downstream tasks without further
fine-tuning, indicating that the optimization for the downstream tasks can be
more efficient. Second, to mimic cross-lingual retrieval, we use mBART-50 Tang
et al. (2020), a machine translation model, to translate the title into a
target language.

Formally, in Concrete, two dense encoders, $E_{C}(\cdot)$ and $E_{P}(\cdot)$,
are used to represent claims and passages as $d$-dimensional vectors. The
similarity between a claim and a passage is defined as the dot product between
their dense vectors,

$\displaystyle sim(c,p_{i})=E_{C}(c)^{\top}E_{P}(p_{i}).$ (1)

In X-ICT, we form positive claim-passage pairs by constructing the translation
of the corresponding title $T_{p_{i}}$ of a given passage $p_{i}$. i.e.
($T^{\prime}_{p_{i}}$, $p_{i}$) where $T^{\prime}_{p_{i}}$ is the original
title $T_{p_{i}}$ translated into a different language using mBART-50. We
treat other passages in the same batch as negative samples. The retriever is
trained by optimizing the negative log likelihood,

Figure 2: An illustration of X-ICT. Given a passage $p_{1}$, we find its title
$T_{p_{1}}$ and translates it into a different language $T^{\prime}_{p_{1}}$
to mimic cross-lingual retrieval. The goal for the cross-lingual retriever,
Concrete, is to select the correct passage $p_{1}$ based on the translated
title $T^{\prime}_{p_{1}}$ among all the passages in the same batch.

$\displaystyle P(p_{i}|T^{\prime}_{p_{i}})$
$\displaystyle=\frac{\exp(sim(T^{\prime}_{p_{i}},p_{i}))}{\sum_{p_{j}\in\textsc{batch}}\exp(sim(T^{\prime}_{p_{i}},p_{j}))}$
(2) $\displaystyle\mathcal{L_{\textsc{X-ICT}}}$
$\displaystyle=-\log\sum_{p_{i}\in\mathcal{P}}P(p_{i}|T^{\prime}_{p_{i}})$ (3)

Figure 2 demonstrates a graphical illustration.

However, if the translation is performed on every claim, the model would be
discouraged from retrieving passages in the same language as the query, which
is not a desirable property of the retriever. Therefore, we set an equal
probability for a claim to be translated to any language or not being
translated (i.e. the probability of not doing translation is $\frac{1}{7}$).
We repurposed the passage collection as the training corpus for learning
X-ICT.

###  3.2 Multilingual Reader

We use mBERT Devlin et al. (2019), a multilingual version of BERT, as the
encoder for our multilingual reader. The claim $c$ and the corresponding
metadata are encoded jointly with a template $T$: [Claim made by Claimer on
Claim-Date, reported in Language: Claim], where Claimer, Claim-Date, and
Language are placeholders. The claim template $T$ and each of the
$k$222Empirically, we set $k$ to 5 for best overall performance. retrieved
passages $p_{i}$ are first encoded independently

$\displaystyle h_{T}$ $\displaystyle=\textrm{mBERT}(T)\texttt{[CLS]}$ (4)
$\displaystyle h_{p_{i}}$ $\displaystyle=\textrm{mBERT}(p_{i})\texttt{[CLS]}.$
(5)

The final prediction is then made by feeding the concatenated [CLS] embeddings
into a multi-layer perceptron
$\hat{y}=\textrm{MLP}([h_{T};h_{p_{0}};\leavevmode\nobreak\
...\leavevmode\nobreak\ ;h_{p_{k}}])$. The model is optimized with the cross-
entropy loss

$\displaystyle\mathcal{L}$
$\displaystyle=\frac{1}{N}\sum_{i=1}^{N}y_{i}\log\hat{y}_{i},$ (6)

where $y_{i}$ and $\hat{y}_{i}$ denote the ground truth label and the
predicted label of the $i$-th sample respectively, and $N$ denotes the total
number of samples.

Split | # claims | # languages
---|---|---
Train | 19079 | 13
Development | 2535 | 12
In-domain | 3826 | 12
Out-of-domain | 2368 | 4
Zero-shot | 3381 | 12
Table 1: Dataset statistics of X-Fact.

##  4 Experimental Setup

###  4.1 Dataset and Evaluation Metric

Our experiments are conducted on a multilingual fact-checking dataset: X-Fact
Gupta and Srikumar (2021). X-Fact contains 31,189 claims in 25 languages
collected from fact-checking websites via Google’s Fact Check Explorer. Each
claim is annotated with one of the following seven labels: True, Mostly-True,
Partly-True, Mostly-False, False, Unverifiable, and Other. Gupta and Srikumar
(2021) split the data in a way that allows evaluation in various settings, as
described in Table 1. In-domain and out-of-domain test sets contain claims in
the same languages as those in the training set, except that the claims in the
out-of-domain split are from different websites. Our main focus is the zero-
shot setup, where there is no overlap between the languages in the zero-shot
split and those in the training set. We use macro F1 as the evaluation metric,
following Gupta and Srikumar (2021).

| Reader | Retrieval Method | Zero-shot F1 (%) | In-domain F1 (%)
---|---|---|---|---
Prior Gupta and Srikumar (2021) | Majority | None | 7.6 | 6.9
mBERT | None | 16.7 | 39.4
mBERT | Google Search | 16.0 | 41.9
Ours | mBERT | None | 17.25 | 36.91
mBERT | Google Search | 16.02 | 42.61
mBERT | MT+DPR | 15.01 | 35.29
mBERT | BM25 | 17.43 | 38.29
mBERT | mDPR | 17.60 | 36.79
mBERT | Concrete | $\leavevmode\nobreak\ \leavevmode\nobreak\ \textbf{19.83}^{*}$ | 40.53
Table 2: Performance comparison in macro F1 (%) of various models on the
X-Fact test sets. None retrieval method means not using any retrieval
component, while Majority means predicting the majority label, False, for all
samples. When no retrieval method is used, the reader performs fact checking
simply based on the claim template $T$ described in Section 3. Statistical
significance over the second best models computed using the paired bootstrap
procedure Berg-Kirkpatrick et al. (2012) is indicated with ∗ ($p<.05$).

###  4.2 Baselines

We compare the following competitive retrieval systems using different
retrieval components but with the same multilingual reader.

#### MT + DPR

A common approach to cross-lingual tasks is translating inputs from target
languages to source languages of richer resources so that stronger monolingual
models can be utilized Ahmad et al. (2021); Asai et al. (2021a). We translate
all claims and all passages into English with the Helsinki-NLP neural machine
translation models333https://huggingface.co/Helsinki-NLP for its comprehensive
language coverage and decent performance. For languages not covered by
Helsinki-NLP, we use Google Translate instead. Then, we use DPR Karpukhin et
al. (2020) to perform retrieval based on the translated claims and passages.

#### BM25

BM25 Robertson and Zaragoza (2009) has demonstrated advantages over dense
vector representation approaches in monolingual retrieval tasks Lee et al.
(2019). Since BM25 only works in a monolingual setup, we create dummy empty
passages for claims whose languages are not presented in the passage
collection $P$. Our implementation is based on the Rank-BM25
package444https://pypi.org/project/rank-bm25/.

#### mDPR

mDPR is a multilingual retriever based on DPR. It was trained on multilingual
question answering datasets, as detailed in Section 3.

#### Google Search

As demonstrated in previous work Augenstein et al. (2019); Gupta and Srikumar
(2021), snippets from Google Search results can serve as evidence for fact-
checking. We directly take the snippets obtained by Gupta and Srikumar (2021)
as inputs.

###  4.3 Implementation Details

When trained with X-ICT on the passage collection, the retriever is optimized
using AdamW Loshchilov and Hutter (2019) with a learning rate of 2e-5 over 30
epochs. When fine-tuning the multilingual reader, we set the learning rate to
5e-5 for parameters in mBERT and 1e-3 for all other parameters. The maximum
input sequence length for X-ICT and fine-tuning on X-Fact are set to 256 and
512, respectively. We use the pre-trained mBERT checkpoints on
HuggingFace555https://huggingface.co/bert-base-multilingual-cased.

##  5 Results

###  5.1 Main results

Table 2 summarizes the fact-checking performance on the X-Fact dataset. Our
framework establishes a new state-of-the-art in zero-shot cross-lingual fact-
checking, outperforming the previous best models by an absolute macro F1 of
2.23%. The improvements demonstrate the effectiveness of our approach in
retrieving relevant passages from multiple languages to assist in fact-
checking. Furthermore, we found that the use of vanilla mDPR does not improve
performance compared to the absence of any retrieval component. This can be
explained by the domain discrepancy issue for mDPR, as it was trained on
question-like queries instead of claim-like queries. Furthermore, although
Concrete is the most advantageous in the zero-shot setup, it trails behind
Google Search in-domain setup. The significantly increased gap between in-
domain F1 score and zero-shot F1 score for using Google Search suggests that
the reader may exploit biases or patterns presented in Google Search’s results
that are not transferrable across languages. To validate this hypothesis, we
analyzed the relationship between the snippets returned by Google Search and
the ground-truth labels. We found that for claims of richer resources in the
training set, Google Search is able to retrieve evidence that strongly
indicates the veracity of the claims due to the abundance of fact-checking
websites. For example, among Indonesian claims where Google Search results
contain the string “SALAH” (WRONG), 50% of them are Partly True and 45% of
them are False. Such patterns can also be found in the in-domain split, but
not in the zero-shot split. This finding explains the increased gap between
the performance on these two splits when using Google Search as the retrieval
method. It also implies that our approach is more generalizable to lower-
resource languages and more applicable when no fact-checking websites have
debunked the input claims.

###  5.2 Performance Analysis

Figure 3: Performance difference in macro F1 when Indonesian passages are
removed from the passage collection. On the x-axis, languages are sorted in
ascending order by distance to Indonesian (i.e. Bengali is the closest, while
Marathi is the furthest). We compute the distance between two languages based
on word ordering, following Ahmad et al. (2019).

#### Language Choice in Passage Collection

Throughout our experiments, we found that the language of the retrieved
passage can affect the fact-checking performance. We hypothesize that it may
be more challenging for readers to reason through passages when the passage
language is distant from the claim language. To verify such a hypothesis, we
first compute the distance between each pair of languages based on word
ordering, following Ahmad et al. (2019). Then, we remove passages of a
particular language from the passage collection. With this subset of selected
passage collection, we use Concrete to retrieve relevant passages for fact-
checking and train another fact-checker based on newly retrieved passages.
Finally, we compare the performance difference for different languages between
this system and the model discussed in Section 3 in the zero-shot setup.
Figure 3 shows the results for each language when Indonesian passages are
removed from the passage collection. We observe that the performance drop and
the distance between each language and Indonesian are highly correlated. In
fact, we see that removing Indonesian is beneficial for some distant languages
such as Dutch and Marathi.

Given these results, the following question arises: is this phenomenon caused
by the poor capability of our multilingual reader to reason with passages and
claims in distant languages, or by the higher information overlap between
Indonesian passages and claims in closer languages? To better understand the
results, we translate the retrieved passages into three languages of distinct
language families: Indonesian, Portuguese, and Arabic. With the three sets of
translated passages, we train three fact-checkers. Then, we compute the
difference between the original performance and the performance achieved using
the translated passages. In Figure 4, we observe that the performance
difference is negatively correlated with the distance between the languages of
the claim and the passage. The correlation coefficient is -0.490 per Kendall’s
Tau Kendall (1938). This confirms that passages in distant languages are
indeed harder for our multilingual reader to reason with. The trend is
consistent with the findings of Asai et al. (2021b).

#### Impact of the Amount of Training Data

To test the data efficiency of our approach, we compared Concrete with mDPR
and Google Search in the zero-shot setup using different numbers of languages
for training. As shown in Figure 5, Concrete consistently outperforms the
other two methods across all settings. This indicates the strength of Concrete
in aiding fact-checking in low- and high-resource scenarios.

#### Impact of Retrieving from Multiple Languages

We conducted a case study on cross-lingual retrieval versus monolingual
retrieval to understand whether retrieving passages from multiple languages
actually helps the performance. In particular, we train and evaluate the
models on samples whose languages are in the passage collection666We train the
models on Portuguese, Indonesian, and Arabic, and evaluate them on French,
Persian, and Russian.. For the monolingual retrieval setting, the model is
restricted to retrieving passage in the same language as the input claim,
while the cross-lingual setting does not have such a restriction. We found
that for both mDPR and Concrete, cross-lingual retrieval setting outperforms
their monolingual retrieval counterparts, as shown in Figure 6. This finding
confirms that retrieving evidence in multiple languages helps cross-lingual
transfer for fact-checking.

Figure 4: Performance difference in macro F1 when the retrieved passages are
translated to Indonesian, Portuguese, and Arabic, respectively. Overall, the
performance difference has a negative correlation with the distance between
the passage’s language and the claim’s language.

###  5.3 Qualitative Analysis

The following qualitative analysis provides an intuition for our model’s
advantage in cross-lingual fact-checking.

#### Impact of X-ICT

To validate the effectiveness of X-ICT in retrieving claims that are more
relevant to the topic of the claim, we compared 50 predictions between models
using Concrete and mDPR as retriever in the split zero-shot. The results show
that 23 errors made by mDPR are corrected by Concrete, while only 6 new errors
are introduced. We found that mDPR can often retrieve passages that are
relevant to a part of a claim, but the topic of the retrieved passages may not
align well with that of the claim, likely due to the domain mismatch issue
discussed in Section 3. An example is shown in Figure 8. This reflects that
X-ICT is able to improve the retrieval quality even though only pseudo
feedback is used for training.

Figure 5: Zero-shot cross-lingual performance on X-Fact with regard to various
numbers of languages used for training.

#### Importance of Passages’ Trustworthiness

Comparing the predictions and the retrieved passages between Concrete and
Google Search on the zero-shot split, we observe that our approach is better
at identifying False and Mostly False claims. As demonstrated in Figure 8,
when a claim is False or Mostly False, the snippets returned by Google Search
are often contradicted with each other, which usually leads to incorrect
predictions. The inconsistency in the snippets from Google Search is caused by
the fact that Google Search retrieves information from the entire Web without
considering the trustworthiness of the source. On the contrary, our approach
offers the flexibility to include only trustworthy information in the passage
collection. In the zero-shot split, we found that this property of Google
Search leads to 44 and 67 more errors in identifying False and Mostly False
claims, respectively.

###  5.4 Remaining Challenges

To identify the remaining challenges, we compare 50 errors made by our model
with ground-truth labels and analyze the sources of errors, as illustrated in
Figure 7. The following paragraphs will discuss these categories with
examples.

#### Evidence cannot be retrieved.

The most common error is caused by the absence of supporting or refuting
evidence in the passage collection. For the majority of such errors, the
claims are country specific. For example, Mann med tre koner får tre
leiligheter i Sverige. (Husband with three wives gets three apartments in
Sweden.) We can address this issue by adding trustworthy news articles from
more countries into the passage collection.

Figure 6: Performance comparison between using monolingual and multilingual
retrieval on cross-lingual fact-checking. For both mDPR and Concrete,
retrieving passages in multiple languages improves the overall performance.

#### Under-specified context.

Another major source of errors is the underspecification of the input claim.
An example claim is:

> “Nuk besoj që janë të informuar as partnerët tanë, SHBA dhe NATO, sepse do
> të isha i informuar edhe unë.” (I do not believe that our partners, the US
> and NATO, are informed either, because I would be informed as well.)

In this claim, it does not specify who is “I” and what the US and NATO are not
informed. Therefore, the given information is too little to determine the
veracity of the claim. This problem could potentially be solved by mining the
original context from the Internet.

Figure 7: Distribution of the remaining errors. Figure 8: An example showing
how relevant trustworthy passages retrieved by Concrete lead to correct
predictions. Concrete retrieves two passages that prove Tesla cars are not
slow or struggling on the road. Since the passages are from trustworthy
sources, our reader can correctly predict the claim as False. On the other
hand, the second passage that mDPR retrieves is slightly related to the claim
(Tesla) but not directly relevant to the topic (the poor performance of Tesla
cars). For Google Search’s results, these two passages contradict each other
as one is from a fact-checking website and the other is from an unreliable
source. Hence, the model cannot predict correctly based on the retrieved
passages from mDPR or Google Search.

#### Require intent identification.

Some of the claims are correct but contain misleading information. For
instance,

“Писатель Дин Кунц предсказал появление коронавируса в своей книге в 1981 году
и называл его Ухань-400.” (Writer Dean Koontz predicted the emergence of the
coronavirus in his book in 1981 and called it Wuhan-400.) This claim is
correct, but the claimer attempts to mislead the audience by linking the
coronavirus in the book with COVID-19, which is false. To correctly predict
this claim, the model should be capable of identifying the intent behind the
claims.

#### Reader failure.

In some cases, the reader fails to predict the correct veracity even though
the supporting evidence is successfully retrieved. This is mainly due to the
long distances between passages and claims, as discussed in Section 5.2.

#### Annotation error.

The dataset was created by matching the rating of each claim on fact-checking
websites with its label definition. Annotation errors could be caused by (1)
the misalignment between the rating and the label definition, and (2) the
inaccurate ratings listed on the fact-checking websites. We found that each
case accounts for about half of the annotation errors.

##  6 Related Work

###  6.1 Fact-checking

Previous fact-checking approaches can be roughly divided into two categories
based on task formulations. The first type of formulation assumes that
evidence candidates are given, such as the FEVER dataset Thorne et al. (2018)
and the SciFact dataset Wadden et al. (2020). Previous approaches for this
category of fact-checking tasks often involve a retrieval module to retrieve
relevant evidence from the given candidate pool followed by a reasoning
component that determines the compatibility between a piece of evidence and
the input claim Yin and Roth (2018); Pradeep et al. (2021). The second
category is the open-retrieval setting777We borrow the term open-retrieval
from the field of question answering., where evidence candidates are not
provided, such as the Liar dataset Wang (2017) and the X-Fact dataset Gupta
and Srikumar (2021). For this task formulation, one of the main challenges is
where and how to retrieve evidence. Some work determines the veracity of a
claim based solely on the claim itself and the information learned by language
models during the pre-training stage Lee et al. (2021). However, such an
approach is tied to the period of time in which the pre-training data is
collected and does not generalize well to new claims. Other studies devise
hand-crafted linguistic features as input to the fact-checking models Mihalcea
and Strapparava (2009); Choudhary and Arora (2021). However, these approaches
are language-specific, whereas our approach is language-agnostic since it
consists of a cross-lingual retriever and a multilingual language model. Gupta
and Srikumar (2021) use a similar approach, which retrieves relevant snippets
using Google Search instead of a cross-lingual retriever. Our experimental
results show that the proposed cross-lingual retriever is more effective than
Google Search in the zero-shot setting due to the fact that Google Search
ignores the trustworthiness of the retrieved information and that downstream
models tend to exploit the biased patterns in Google Search results that are
not transferrable across languages, as shown in Section 5.

###  6.2 Cross-lingual Retrieval

Early attempts on cross-lingual retrieval adopt a pipeline consisting of a
statistical machine translation system and a monolingual retrieval model
Hiemstra and de Jong (1999); Ture and Lin (2013). These methods do not perform
well due to the poor performance of statistical machine translation systems.
Later work addresses this issue with bilingual embeddings Vulic and Moens
(2015); Litschko et al. (2018). More recently, large pre-trained multilingual
language models demonstrate significant advantages in constructing
multilingual representations Jiang et al. (2020). Yu et al. (2021) pre-train a
cross-lingual language model tailored for the retrieval tasks. Yet, the
computation complexity is relatively high due to the cross-encoder
architecture of the model. Namely, it takes a pair of query and evidence as
inputs, instead of encoding the query and evidence independently. The mDPR
model presented in Asai et al. (2021b) is the most favorable for our task due
to its high efficiency and performance. However, mDPR was trained on datasets
where queries are questions instead of claims. Therefore, domain adaptation is
needed for mDPR to be applied to our task. Concrete extends mDPR by adapting
it to fact-checking using a self-supervised cross-lingual retrieval algorithm
to mitigate the domain discrepancy problem while maintaining high efficiency.

##  7 Conclusions and Future Work

We have proposed Concrete, a claim-oriented cross-lingual retriever that
retrieves trustworthy passages from a multilingual passage collection. To
overcome the lack of IR training data with claim-like queries, we present the
Cross-lingual Inverse Cloze Task (X-ICT) that leverages pseudo feedback to
train the retriever. Experimental results on X-Fact showed that our approach
outperforms all previous systems in the zero-shot cross-lingual setting. For
future work, we plan to investigate the adaptive selection mechanism for
passages based on distances and develop more robust readers for reasoning
through passages of longer distances via representation learning.

##  8 Ethical Considerations

Although our framework has significant advantages over the previous state of
the art, the proposed model is still far from being a reliable cross-lingual
fact checker given its great potential for improvement. If such a system is
deployed for public use, the general public could lose trust in automatic
fact-checking systems, and the situation of infodemic can exacerbate.
Therefore, at the current stage, our system should be served as an assistant
for human fact-checkers to validate the veracity of claims instead of directly
applying for public use, especially in the zero-shot setting. With our
framework, the efficiency of manual fact-checking can be significantly
improved thanks to its ability to retrieve relevant information across
multiple languages and produce a reasonably good preliminary judgement on the
veracity of the input claim.

In addition, we also acknowledge that the use of large multilingual language
models pre-trained on the Web could lead to biased outputs. Fortunately, after
a close inspection into the X-Fact dataset, we do not find such biased
patterns in it. This means that fine-tuning our proposed framework on X-Fact
should alleviate the problem of biased predictions.

## Acknowledgement

This research is based upon work supported by U.S. DARPA SemaFor Program No.
HR001120C0123. The views and conclusions contained herein are those of the
authors and should not be interpreted as necessarily representing the official
policies, either expressed or implied, of DARPA, or the U.S. Government. The
U.S. Government is authorized to reproduce and distribute reprints for
governmental purposes notwithstanding any copyright annotation therein.

## References

  * Ahmad et al. (2019) Wasi Ahmad, Zhisong Zhang, Xuezhe Ma, Eduard Hovy, Kai-Wei Chang, and Nanyun Peng. 2019.  On difficulties of cross-lingual transfer with order differences: A case study on dependency parsing.  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_ , pages 2440–2452, Minneapolis, Minnesota. Association for Computational Linguistics.
  * Ahmad et al. (2021) Wasi Uddin Ahmad, Nanyun Peng, and Kai-Wei Chang. 2021.  Gate: Graph attention transformer encoder for cross-lingual relation and event extraction.  In _Proceedings of the AAAI Conference on Artificial Intelligence_.
  * Asai et al. (2021a) Akari Asai, Jungo Kasai, Jonathan Clark, Kenton Lee, Eunsol Choi, and Hannaneh Hajishirzi. 2021a.  XOR QA: Cross-lingual open-retrieval question answering.  In _Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 547–564, Online. Association for Computational Linguistics.
  * Asai et al. (2021b) Akari Asai, Xinyan Yu, Jungo Kasai, and Hannaneh Hajishirzi. 2021b.  One question answering model for many languages with cross-lingual dense passage retrieval.  In _NeurIPS_.
  * Augenstein et al. (2019) Isabelle Augenstein, Christina Lioma, Dongsheng Wang, Lucas Chaves Lima, Casper Hansen, Christian Hansen, and Jakob Grue Simonsen. 2019.  MultiFC: A real-world multi-domain dataset for evidence-based fact checking of claims.  In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_ , pages 4685–4697, Hong Kong, China. Association for Computational Linguistics.
  * Berg-Kirkpatrick et al. (2012) Taylor Berg-Kirkpatrick, David Burkett, and Dan Klein. 2012.  An empirical investigation of statistical significance in NLP.  In _Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning_ , pages 995–1005, Jeju Island, Korea. Association for Computational Linguistics.
  * Choudhary and Arora (2021) Anshika Choudhary and Anuja Arora. 2021.  Linguistic feature based learning model for fake news detection and classification.  _Expert Systems with Applications_ , 169:114171.
  * Devlin et al. (2019) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019.  BERT: Pre-training of deep bidirectional transformers for language understanding.  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_ , pages 4171–4186, Minneapolis, Minnesota. Association for Computational Linguistics.
  * Djuraskovic (2022) Ogi Djuraskovic. 2022.  47 blogging statistics (2022): How many blogs are there?
  * Fung et al. (2021) Yi Fung, Christopher Thomas, Revanth Gangi Reddy, Sandeep Polisetty, Heng Ji, Shih-Fu Chang, Kathleen McKeown, Mohit Bansal, and Avi Sil. 2021.  InfoSurgeon: Cross-media fine-grained information consistency checking for fake news detection.  In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_ , pages 1683–1698, Online. Association for Computational Linguistics.
  * Fung et al. (2022) Yi R. Fung, Kung-Hsiang Huang, Preslav Nakov, and Heng Ji. 2022.  The battlefront of combating misinformation and coping with media bias.  In _Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining_ , KDD ’22, page 4790–4791, New York, NY, USA. Association for Computing Machinery.
  * Guo et al. (2022) Zhijiang Guo, Michael Schlichtkrull, and Andreas Vlachos. 2022.  A Survey on Automated Fact-Checking.  _Transactions of the Association for Computational Linguistics_ , 10:178–206.
  * Gupta and Srikumar (2021) Ashim Gupta and Vivek Srikumar. 2021.  X-fact: A new benchmark dataset for multilingual fact checking.  In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 2: Short Papers)_ , pages 675–682, Online. Association for Computational Linguistics.
  * Hiemstra and de Jong (1999) Djoerd Hiemstra and Franciska de Jong. 1999.  Disambiguation Strategies for Cross-Language Information Retrieval.  In _Proceedings of the Third European Conference on Research and Advanced Technology for Digital Libraries_ , volume 1696 of _Lecture Notes in Computer Science_ , pages 274–293. Springer.
  * Hoang and Mothe (2018) Thi Bich Ngoc Hoang and Josiane Mothe. 2018.  Location extraction from tweets.  _Information Processing & Management_, 54(2):129–144.
  * Huang et al. (2022) Kung-Hsiang Huang, Kathleen McKeown, Preslav Nakov, Yejin Choi, and Heng Ji. 2022\.  Faking fake news for real fake news detection: Propaganda-loaded training data generation.  _arXiv preprint arXiv:2203.05386_.
  * Jiang et al. (2020) Zhuolin Jiang, Amro El-Jaroudi, William Hartmann, Damianos Karakos, and Lingjun Zhao. 2020.  Cross-lingual information retrieval with BERT.  In _Proceedings of the workshop on Cross-Language Search and Summarization of Text and Speech (CLSSTS2020)_ , pages 26–31, Marseille, France. European Language Resources Association.
  * Karpukhin et al. (2020) Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. 2020.  Dense passage retrieval for open-domain question answering.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 6769–6781, Online. Association for Computational Linguistics.
  * Kazemi et al. (2021) Ashkan Kazemi, Kiran Garimella, Devin Gaffney, and Scott Hale. 2021.  Claim matching beyond English to scale global fact-checking.  In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_ , pages 4504–4517, Online. Association for Computational Linguistics.
  * Kazemi et al. (2022) Ashkan Kazemi, Zehua Li, Verónica Pérez-Rosas, Scott A Hale, and Rada Mihalcea. 2022.  Matching tweets with applicable fact-checks across languages.  _arXiv preprint arXiv:2202.07094_.
  * Kendall (1938) Maurice G Kendall. 1938.  A new measure of rank correlation.  _Biometrika_ , 30(1/2):81–93.
  * Lee et al. (2019) Kenton Lee, Ming-Wei Chang, and Kristina Toutanova. 2019.  Latent retrieval for weakly supervised open domain question answering.  In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_ , pages 6086–6096, Florence, Italy. Association for Computational Linguistics.
  * Lee et al. (2021) Nayeon Lee, Yejin Bang, Andrea Madotto, and Pascale Fung. 2021.  Towards few-shot fact-checking via perplexity.  In _Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 1971–1981, Online. Association for Computational Linguistics.
  * Litschko et al. (2018) Robert Litschko, Goran Glavas, Simone Paolo Ponzetto, and Ivan Vulic. 2018.  Unsupervised cross-lingual information retrieval using monolingual data only.  _The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval_.
  * Loshchilov and Hutter (2019) Ilya Loshchilov and Frank Hutter. 2019.  Decoupled weight decay regularization.  In _International Conference on Learning Representations_.
  * Mihalcea and Strapparava (2009) Rada Mihalcea and Carlo Strapparava. 2009.  The lie detector: Explorations in the automatic recognition of deceptive language.  In _Proceedings of the ACL-IJCNLP 2009 Conference Short Papers_ , pages 309–312, Suntec, Singapore. Association for Computational Linguistics.
  * Pate et al. (2019) Umaru A Pate, Danjuma Gambo, Adamkolo Mohammed Ibrahim, et al. 2019.  The impact of fake news and the emerging post-truth political era on nigerian polity: A review of literature.  _Studies in Media and Communication_ , 7(1):21–29.
  * Pradeep et al. (2021) Ronak Pradeep, Xueguang Ma, Rodrigo Nogueira, and Jimmy Lin. 2021.  Scientific claim verification with VerT5erini.  In _Proceedings of the 12th International Workshop on Health Text Mining and Information Analysis_ , pages 94–103, online. Association for Computational Linguistics.
  * Robertson and Zaragoza (2009) Stephen Robertson and Hugo Zaragoza. 2009.  _The probabilistic relevance framework: BM25 and beyond_.  Now Publishers Inc.
  * Scheufele and Krause (2019) Dietram A Scheufele and Nicole M Krause. 2019.  Science audiences, misinformation, and fake news.  _Proceedings of the National Academy of Sciences_ , 116(16):7662–7669.
  * Tang et al. (2020) Yuqing Tang, Chau Tran, Xian Li, Peng-Jen Chen, Naman Goyal, Vishrav Chaudhary, Jiatao Gu, and Angela Fan. 2020.  Multilingual translation with extensible multilingual pretraining and finetuning.
  * Thorne et al. (2018) James Thorne, Andreas Vlachos, Christos Christodoulopoulos, and Arpit Mittal. 2018\.  FEVER: a large-scale dataset for fact extraction and VERification.  In _Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)_ , pages 809–819, New Orleans, Louisiana. Association for Computational Linguistics.
  * Ture and Lin (2013) Ferhan Ture and Jimmy Lin. 2013.  Flat vs. hierarchical phrase-based translation models for cross-language information retrieval.  In _Proceedings of the 36th international ACM SIGIR conference on Research and development in information retrieval_ , pages 813–816.
  * Vulic and Moens (2015) Ivan Vulic and Marie-Francine Moens. 2015.  Monolingual and cross-lingual information retrieval models based on (bilingual) word embeddings.  _Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval_.
  * Wadden et al. (2020) David Wadden, Shanchuan Lin, Kyle Lo, Lucy Lu Wang, Madeleine van Zuylen, Arman Cohan, and Hannaneh Hajishirzi. 2020.  Fact or fiction: Verifying scientific claims.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 7534–7550, Online. Association for Computational Linguistics.
  * Wang (2017) William Yang Wang. 2017.  “liar, liar pants on fire”: A new benchmark dataset for fake news detection.  In _Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)_ , pages 422–426, Vancouver, Canada. Association for Computational Linguistics.
  * Wu et al. (2022) Xueqing Wu, Kung-Hsiang Huang, Yi Fung, and Heng Ji. 2022.  Cross-document misinformation detection based on event graph reasoning.  In _Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 543–558, Seattle, United States. Association for Computational Linguistics.
  * Yin and Roth (2018) Wenpeng Yin and Dan Roth. 2018.  TwoWingOS: A two-wing optimization strategy for evidential claim verification.  In _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing_ , pages 105–114, Brussels, Belgium. Association for Computational Linguistics.
  * Yu et al. (2021) Puxuan Yu, Hongliang Fei, and P. Li. 2021.  Cross-lingual language model pretraining for retrieval.  _Proceedings of the Web Conference 2021_.