11institutetext:  MPIfR, Auf dem Hügel 69, 53121 Bonn, Germany
22institutetext: Purple Mountain Observatory, Chinese Academy of Sciences,
Nanjing 210008, China 33institutetext: National Radio Astronomy Observatory,
520 Edgemont Rd., Charlottesville, VA 22903, USA 44institutetext: Joint
Institute for VLBI in Europe, Postbus 2, 7990 AA Dwingeloo, The Netherlands

# Ammonia ($J$,$K$) = (1,1) to (4,4) and (6,6) inversion lines detected in the
Seyfert 2 galaxy NGC 1068

Y. Ao email<EMAIL_ADDRESS>C. Henkel 11 J.A. Braatz 33 A. Weiß 11
K. M. Menten & S. Mühle 1144

We present the detection of the ammonia (NH3) ($J$,$K$) = (1,1) to (4,4) and
(6,6) inversion lines toward the prototypical Seyfert 2 galaxy NGC 1068, made
with the Green Bank Telescope (GBT). This is the first detection of ammonia in
a Seyfert galaxy. The ortho-to-para-NH3 abundance ratio suggests that the
molecule was formed in a warm medium of at least 20 K. For the NH3 column
density and fractional abundance, we find (1.09$\pm$0.23)$\times$1014 cm-2 and
(2.9$\pm$0.6)$\times$10-8, respectively, from the inner $\sim$1.2 kpc of NGC
1068. The kinetic temperature can be constrained to 80$\pm$20 K for the bulk
of the molecular gas, while some fraction has an even higher temperature of
140$\pm$30 K.

###### Key Words.:

galaxies:individual: NGC 1068 – galaxies: Seyfert – galaxies: ISM – ISM:
molecules – radio lines: galaxies

##  1 Introduction

Studies of molecular gas provide information about the gas density, the
temperature, and kinematics within galaxies, help us to understand chemical
evolution over cosmic time and allow us to study the triggering and fueling
mechanisms of star formation and active galactic nuclei (AGN). Most stars are
formed in dense gas cores, which are embedded in giant molecular clouds
(GMCs). The star-forming activity is related to the dense gas and not to the
bulk of the GMC’s material (Gao & Solomon 2004). Determining the physical
properties of the dense gas in galaxies is therefore of fundamental importance
for our understanding of star formation and the evolution of galaxies. Among
the most commonly observed species are CO, CS, HCN, and HCO+. In local dark
clouds, the temperature can be constrained by observations of the $J$ = 1$-$0
transition of CO, both because this transition is opaque and thermalized and
because the emission is often extended enough to fill the beam of a single-
dish telescope. However, in external galaxies, filling factors are much less
than unity. Furthermore, CO, CS, HCN, and HCO+ suffer from a coupled
sensitivity to the kinetic temperature and spatial density, making an observed
line ratio consistent with both a high density at a low temperature and a low
density at a high temperature. Specific information about the individual
physical parameters therefore requires a molecular tracer that possesses a
singular sensitivity to either density or temperature. Ammonia (NH3) is such a
molecule.

Ammonia is widespread and its level scheme contains inversion doublets owing
to the tunneling of the nitrogen atom through the plane defined by the three
hydrogen atoms. The metastable ($J,K\thinspace=\thinspace J$) rotational
levels, for which the total angular momentum quantum number $J$ is equal to
its projection on the molecule’s symmetry axis, are not radiatively but
collisionally coupled. NH3 is therefore a good tracer of the gas kinetic
temperature. Another advantage is that the inversion lines are quite close in
frequency, thus allowing us to measure sources with similar beam sizes and to
use the same telescope-receiver combination, which minimizes calibration
uncertainties of the measured line ratios. As a consequence, NH3 is widely
studied to investigate the physical properties of dark clouds and massive
star-forming regions in our Galaxy (e.g., Ho & Townes 1983; Walmsley &
Ungerechts 1983; Bourke et al. 1995; Ceccarelli et al. 2002; Pillai et al.
2006; Wilson et al. 2006). However, in spite of the high sensitivity and
stable baselines of present state-of-the-art facilities, ammonia multilevel
studies in extragalactic sources are still rare and are limited to the Large
Magellanic Cloud, IC 342, NGC 253, M 51, M 82, Maffei 2, and Arp 220 (Martin &
Ho 1986; Henkel et al. 2000; Weiß et al. 2001; Takano et al. 2002;
Mauersberger et al. 2003; Ott et al. 2005, 2010; Takano et al. 2005 ) in the
local Universe, and in absorption to the gravitationally lensed B0218+357 and
PKS 1830$-$211 at redshifts of $z\sim$ 0.7 and 0.9 (Henkel et al. 2005, 2008).

NGC 1068, a prototypical Seyfert 2 galaxy (Antonucci & Miller 1985), is
located at a distance of 15.5 Mpc (a heliocentric systemic velocity $cz$ =
1137 km s-1 is used throughout this paper, adopted from the NASA/IPA
Extragalactic Database), making it one of the nearest Seyfert galaxies and
thus an ideal target to investigate the physical properties of the molecular
gas in the vicinity of an AGN. Here we report the first detection of ammonia
in this prominent Seyfert 2 galaxy to evaluate the kinetic temperature of its
dense molecular gas.

##  2 Observations

We observed ammonia inversion lines toward the nucleus of NGC 1068 with the
Green Bank Telescope (GBT) of the National Radio Astronomy Observatory111The
National Radio Astronomy Observatory is a facility of the National Science
Foundation operated under cooperative agreement with Associated Universities,
Inc. on 2005 October 14 and October 19. We configured the telescope to observe
both circular polarizations simultaneously using a total-power nodding mode in
which the target position was placed alternately in one of the two beams of
the 22$-$26 GHz K-band receiver. The full-width-at-half-maximum (FWHM) beam
size was approximately 31$\arcsec$ in the observed frequency range, between
23.6 and 25.0 GHz. The pointing accuracy was $\sim$ 5$\arcsec$. To observe the
NH3(1,1) through (4,4) lines, we configured the spectrometer with an 800 MHz
spectral window centered on the NH3(3,3) line. The channel spacing was 390
kHz, corresponding to $\sim$ 5 km s-1. We averaged the data from the two dates
to produce the final spectrum, which has an on-source integration time of 4
hours. We observed the NH3(6,6) line only on the second date, 2005 October 19.
Here we used a 200 MHz spectral window observed with a channel spacing of 24
kHz, and then averaged channels in post-processing to achieve 390 kHz channel
spacing. The total integration time for the NH3(6,6) line was 1 hour.

The data were calibrated and averaged in GBTIDL. For calibration, we used an
estimate of the atmospheric opacity obtained from a weather model. We
subtracted a polynomial baseline fitted to the line-free channels adjacent to
each ammonia line. For the (1,1) to (4,4) lines observed simultaneously, the
relative calibration is very good and is limited primarily by the uncertainty
in the baseline fits, which is 10% for the (1,1) to (3,3) lines, 15% for the
(4,4) line, and 27% for the (6,6) line, respectively. The (1,1) and (2,2) line
profiles are close to each other but are marginally separated and the Gaussian
fit for each line does not suffer from this effect. With the calibration error
of 15% for the GBT telescope itself, we estimate the absolute calibration
uncertainties, including the error from the baseline fits, to be $\sim$ 18-31%
for the ammonia lines.

##  3 Results

###  3.1 NH3 lines

This is the first time that ammonia is detected in NGC 1068, making it the
first Seyfert galaxy observed in this molecule. Also, because Arp 220,
B0218+357, and PKS 1830-211 were detected in absorption, NGC 1068 is so far
the most distant galaxy where ammonia is detected in emission. Three para,
($J,K$) = (1,1), (2,2), and (4,4), and two ortho, ($J,K$) = (3,3) and (6,6),
transitions are covered by the observations, and all five are detected. Figure
1 shows the spectra before and after subtracting a polynomial baseline to the
line free channels. The line parameters are listed in Table 1, where
integrated flux, $I$, peak flux density, $S$, central velocity, $V$, and line
width, $\Delta V_{\rm 1/2}$, are obtained from the results of Gaussian
fitting.

The line profiles are similar for the (1,1), (2,2) and (3,3) lines, which
share a similar central velocity of around $-$35 km s-1, relative to a
heliocentric systemic velocity of $cz$ = 1137 km s-1, and a FWHM line width of
220$-$274 km s-1. The similar central velocities and the comparable widths of
the (1,1), (2,2) and (3,3) lines suggest that the line emission comes from the
same region. The (4,4) and (6,6) lines have different central velocities and a
narrower line width. The (4,4) line is located at a frequency where the
baseline is particularly steep. The (6,6) line is weak and falls into a window
with a slightly distorted baseline, making it difficult to determine accurate
line parameters. The comparatively narrow line widths of the (4,4) and (6,6)
lines may suggest that their emission originates from a less extended region
than that of the lower excitation lines.

Our measured continuum flux density of NGC 1068 ranges from 0.33 to 0.37 Jy at
frequencies of 23.4 to 25.1 GHz. This is consistent with the value of
0.342$\pm$0.034 at 22 GHz reported by Ricci et al. (2006).

Figure 1: Metastable ammonia inversion lines observed toward NGC 1068
($\alpha_{\rm 2000}$ = 02h42m40.7s, $\delta_{\rm 2000}$ =
$-$00o00$\arcmin$48$\arcsec$). Vertical lines mark a heliocentric systemic
velocity of $cz$ = 1137 km s-1. Velocities are displayed relative to this
value (in the upper left panel, the velocity scale refers to the (1,1) line).
Left: Spectra without baseline subtraction, overlaid with a polynomial
baseline fit to the line-free channels. Right: Final baseline-subtracted
spectra overlaid with Gaussian fits. Table 1: NH3 line parameters

(J,K) |  $I$a |  $S$a | $V^{a}$ |  $\Delta V_{\rm 1/2}$a |  $N(J,K)$b
---|---|---|---|---|---
| (Jy km s-1) | (mJy) | ( km s-1) | ( km s-1) | (1013 cm-2)
(1,1) | 0.89( 0.07) | 3.26( 0.13) |  $-$30(6) | 274(17) | 2.52(0.24)
(2,2) | 0.79( 0.06) | 3.10( 0.14) |  $-$37(6) | 254(19) | 1.67(0.19)
(3,3) | 1.19( 0.10) | 5.40( 0.14) |  $-$37(3) | 220( 8) | 2.22(0.13)
(4,4) | 0.55( 0.05) | 3.72( 0.14) |  $-$2(3) | 149( 7) | 0.96(0.07)
(6,6) | 0.28( 0.04) | 1.88( 0.23) |  $-$56(9) | 147(27) | 0.43(0.12)
(5,5) |  |  |  |  | 0.53
(0,0) |  |  |  |  | 2.55
$N_{\rm total}$ |  |  |  |  | 10.9(2.3)

  * a The values are derived from Gaussian fits to the spectra. Given velocities are relative to a heliocentric velocity of $cz$ = 1137 km s-1.

  * b The column density for the (5,5) state is extrapolated from the (2,2) and (4,4) states using $T_{\rm rot}$ = 119 K, while the value for the (0,0) state is extrapolated from the (3,3) state by adopting $T_{\rm rot}$ = 44 K. The total column density, $N_{\rm total}$, includes the populations of the levels from (0,0) to (6,6). Because relative calibration is excellent up to at least the (4,4) transition, the errors given in parenthesis refer exclusively to the Gaussian fits. For the cumulative column density, $N_{\rm total}$, however, the given error accounts for the absolute calibration uncertainty ($\S$ 2).

###  3.2 NH3 column density and rotation temperature

Figure 2: Rotation diagram of metastable ammonia transitions toward NGC 1068
(see § 3.2). The open squares show the normalized column densities determined
from the integrated line intensities. The numbers mark the rotational
temperatures in K. The absolute calibration uncertainties, including the
dominant contribution from the baseline fits as well as uncertainties in the
Gaussian fits and in the overall calibration uncertainty, have been taken as
error bars ($\S$ 2). Note, however, that relative calibration is excellent up
to at least the (4,4) transition.

Assuming that the line emission is optically thin and the contribution from
the cosmic background is negligible, the sum of the beam-averaged column
densities of the two states of an inversion doublet can be calculated using

$N(J,K)\thinspace=\thinspace\frac{\rm 1.55\times
10^{14}}{\nu}\frac{J(J+1)}{K^{2}}\int T_{\rm mb}{\rm dv}$ (1)

(e.g., Mauersberger et al. 2003), where the column density $N(J,K)$, the
frequency $\nu$, and the integrated line intensity $\int T_{\rm mb}{\rm dv}$,
based on the main beam brightness temperature, $T_{\rm mb}$, are in units of
cm-2, GHz, and K km s-1, respectively. The calculated column densities (1 Jy
corresponds to 2.16 K on a $T_{\rm mb}$ scale222for the details see the memo
of “Calibration of GBT Spectral Line Data in GBTIDL”
$\rm{http://www.gb.nrao.edu/GBT/DA/gbtidl/gbtidl\\_calibration.pdf}$) are
given in Table 1.

Following the analysis described by Henkel et al. (2000), the rotational
temperature ($T_{\rm rot}$) between different energy levels can be determined
from the slope, $a$, of a linear fit in the rotation diagram (i.e., Boltzmann
plot, which is normalized column density against energy above the ground state
expressed in $E/k$) by $T_{\rm rot}$ = $-{\rm log}\thinspace
e/a$$\thinspace\approx\thinspace-0.434/a$. Figure 2 shows the rotation diagram
including the five measured metastable NH3 inversion lines. Ignoring
differences in line shape, we obtain $T_{\rm rot}$ = 92${}^{+29}_{-18}$ K by
fitting the para-NH3 species, i.e. the (1,1), (2,2) and (4,4) transitions, and
$T_{\rm rot}$ = 125${}^{+11}_{-8}$ K for the (3,3) and (6,6) ortho-NH3
transitions. As discussed in $\S\leavevmode\nobreak\ \ref{ammonia}$, the
(1,1), (2,2) and (3,3) line emission may originate in a region that is
different from that of the (4,4) and (6,6) lines. The former three lines can
be fitted by $T_{\rm rot}$ = 61${}^{+12}_{-9}$ K, and the latter two by
$T_{\rm rot}$ = 111${}^{+12}_{-9}$ K. The rotation temperature between the
lowest inversion doublets of para-ammonia, (1,1) and (2,2), is $T_{\rm rot}$ =
44${}^{+6}_{-4}$ K. Without the (1,1) state, the other two para-NH3
transitions, (2,2) and (4,4), yield $T_{\rm rot}$ = 119${}^{+15}_{-11}$ K.

###  3.3 NH3 abundance

To estimate the populations of the $(J,K)$ = (0,0) and (5,5) states, we assume
the rotation temperature $T_{\rm rot}$ = 44 K, derived from the two lowest
inversion doublets, for the (0,0) ground state, and $T_{\rm rot}$ = 119 K from
the fit to the (2,2) and (4,4) lines, for the (5,5) state. The derived column
densities are 2.55$\times$1013 cm-2 for the (0,0) state, which is not a
doublet, and 0.53$\times$1013 cm-2 for the (5,5) state, respectively. The
former is extrapolated from the (3,3) state and the latter from the (2,2) and
(4,4) states. The resulting total column density of ammonia is
(1.09$\pm$0.23)$\times$1014 cm-2 if only considering the populations in the
metastable states up to ($J,K$) = (6,6). This yields an ammonia gas mass of
68$\pm$14 M⊙ within a beam size of 31$\arcsec$, which corresponds to a
physical size of 2.3 kpc. To estimate the mass of molecular gas in NGC 1068,
we convolved the 17$\arcsec$ resolution map in CO $J$ = 1$-$0 by Kaneko et al.
(1989) to the GBT’s resolution of 31$\arcsec$ and derived an integrated
intensity of 103 K km s-1. Adopting a conversion factor of 0.8 M⊙ ${(\rm
K\thinspace km\thinspace s^{-1}\thinspace pc^{2})^{-1}}$, which describes
ultraluminous galaxies (Downes & Solomon 1998) and less conspicuous nuclear
starbursts (Mauersberger et al. 2003), we obtain a molecular gas mass of
3.5$\times$108 M⊙. Therefore, the ammonia abundance relative to molecular
hydrogen is estimated to be (2.9$\pm$0.6)$\times$10-8 within a radius of
$\sim$1.2 kpc around the center of NGC 1068.

##  4 Discussion

Figure 3:  NH3 inversion line ratios as a function of the H2 density ($n_{\rm
H_{2}}$) and kinetic temperature ($T_{\rm kin}$). The red and blue lines mark
the (2,2)/(1,1) and (6,6)/(4,4) line ratios, respectively, obtained from the
large velocity gradient model outlined in § 4. Dashed lines show the line
ratios measured with the GBT. [NH3]/(d$v$/d$r$) = 10-8 pc (km s-1)-1 ([NH3] =
N(NH3)/N(H2) is the fractional abundance of ammonia). At high densities the
(1,1) and (2,2) lines saturate, which causes the steeply declining $T_{\rm
kin}$ values with rising density for a given (2,2)/(1,1) line ratio.

The ammonia abundance in NGC 1068 is consistent with the values of
(1.3$-$2.0)$\times$10-8 reported by Mauersberger et al. (2003) for the central
regions of several nearby galaxies such as NGC 253, Maffei 2 and IC 342. It is
lower than that of a few $\times$10-7 for the central $\sim$ 500 pc of the
Milky Way (Rodríguez-Fernández et al. 2001), but higher than the extremely low
value of 5$\times$10-10 determined for M 82 by Weiß et al. (2001). M 82 is a
special case, because it contains a starburst in a late stage of its evolution
leaving NH3, which is a molecule particularly sensitive to UV radiation (e.g.,
Suto & Lee 1983), in only a few molecular cores that are still sufficiently
shielded against the intense radiation field.

With the three para- and two ortho-NH3 lines observed, we can estimate the
ortho-to-para abundance ratio to be $R$ $\sim$ 0.9 (we do not give an error
here because of the extrapolated (0,0) column density) as an upper limit
because the (0,0) column density might be overestimated by extrapolating from
the (3,3) column density, which could be affected by maser activity (e.g.,
Walmsley & Ungerechts 1983; Ott et al. 2005). A value near or below unity is
expected, if ammonia was formed in a warm medium at a gas kinetic temperature
of at least 20 K and only $R$ values above $\sim$1.5 would hint at a cool
formation temperature of $\sim$ 20 K (e.g., Takano et al. 2002).

The rotation temperatures from the multilevel study of the ammonia inversion
lines are a good approximation to the kinetic gas temperature only for low
($T_{\rm kin}$ $\leq$ 20 K) temperatures (Walmsley & Ungerechts 1983; Danby et
al. 1988). At higher temperatures, the rotation temperature provides a robust
lower limit to the kinetic gas temperature as long as saturation effects do
not play a role.

To determine the kinetic temperature itself, we here use a one-component large
velocity gradient (LVG) analysis adopting a spherical cloud geometry as
described by Ott et al. (2005). Dahmen et al. (1998) estimated that the
velocity gradient ranges from 3 to 6 km s-1 pc-1 for Galactic center clouds
and Meier et al. (2008) found a typical value of 1 to 2 km s-1 pc-1 for GMCs.
Here we adopt a median value of 3 km s-1 pc-1 for the LVG models presented in
this paper. The ammonia abundance derived in $\S$ 3.3 is adopted, which yields
an NH3 abundance per velocity gradient of [NH3]/(d$v$/d$r$) $\sim$ 10-8 pc (km
s-1)-1 ([NH3] = N(NH3)/N(H2) is the fractional abundance of ammonia). Changing
the velocity gradient by a factor of 3 will affect the derived kinetic
temperature by less than 10 K. Figure 3 shows NH3 inversion line ratios as a
function of the H2 density ($n_{\rm H_{2}}$) and kinetic temperature ($T_{\rm
kin}$). As long as the NH3 lines are optically thin, the ratios are almost
independent of the gas density and are therefore a good indicator for gas
kinetic temperature. At high densities the (1,1) and (2,2) lines saturate,
which causes the steeply declining $T_{\rm kin}$ values with rising density
for a given (2,2)/(1,1) line ratio. While the optical depth may be well above
unity in molecular cores (see, Güsten et al. 1981 for the Galactic center
region), the bulk of the ammonia emission should be optically thin and arises
from gas densities $n_{\rm H_{2}}$ $<$ 105 cm-3 (e.g., Tieftrunk et al. 1998).
For a typical gas density, $n_{\rm H_{2}}$, of 103.0-4.8 cm-3, the observed
(2,2)/(1,1) line ratio constrains the kinetic temperature $T_{\rm kin}$ to
80$\pm$20 K, which characterizes the bulk of the molecular gas. The
(6,6)/(4,4) line ratio yields a gas temperature of 140$\pm$30 K for a gas
density within the range of 103.0-5.5 cm-3. The latter indicates the existence
of a hotter component of the gas, as is also suggested by the high rotational
temperature revealed by the rotation diagram (Fig. 2). Here the (6,6)/(3,3)
line ratio is not used to estimate the temperature because the (3,3) line
emission is partly affected by the maser activity in some parameter ranges
(Walmsley & Ungerechts 1983), and because the (6,6) line was not observed
simultaneously as the (3,3) line and the uncertainty of the baseline fit of
the (6,6) line is largest among all spectra as described in
$\S\leavevmode\nobreak\ \ref{observation}$.

The GBT beam size covers the two inner spiral arms and the circumnuclear disk
(CND) of NGC 1068 (see, e.g., Schinnerer et al. 2000). Our observed ammonia
line emission likely has contributions from the spiral arms and the CND. The
CND may be warmer because it is heated not only by young stars but also by the
AGN. There are observations that directly support higher temperatures in the
CND. Infrared rotational transitions of molecular hydrogen (Lutz et al. 1997)
indicate a wide range of gas temperatures between 100 and 800 K. The dust near
the AGN has been particularly thoroughly studied (e.g., Tomono et al. 2006;
Poncelet et al. 2007; Raban et al. 2009). Because densities may be high within
the central arcsecond ($\ga$ 105 cm-3), the dust and the gas phase may be
coupled and thus these results may be relevant for the kinetic temperature of
the gas component. With mid-infrared (MIR) multi-filter data, Tomono et al.
(2006) obtained a dust temperature of $\sim$200 K within a 1.0$\arcsec$-sized
region ($\sim$80 pc).

Our data provide a direct estimate of the gas properties for molecular gas in
the inner 1.2 kpc of the galaxy. However, higher angular resolution data are
needed to separate the different components and to isolate the CND in this
galaxy.

##  5 Conclusions

Our main results from the ammonia observations toward the prototypical Seyfert
2 galaxy NGC 1068 are:

(1) The metastable NH3 ($J$,$K$) = (1,1) to (4,4) and (6,6) inversion lines
are for the first time detected in emission toward NGC 1068. This opens up a
new avenue to determine kinetic temperatures of the dense gas in nearby highly
obscured AGN.

(2) For the NH3 column density and fractional abundance, we find
(1.09$\pm$0.23)$\times$1014 cm-2 and (2.9$\pm$0.6)$\times$10-8 in the inner
$\sim$1.2 kpc of NGC 1068.

(3) With an ortho-to-para-NH3 abundance ratio of $\sim$0.9, the ammonia should
have been formed in a warm medium of at least 20 K.

(4) The kinetic temperature can be constrained to 80$\pm$20 K for the bulk of
the molecular gas, while some gas fraction has an even higher temperature of
140$\pm$30 K.

###### Acknowledgements.

We thank the referee for thoughtful comments that improved this paper and the
staff at the GBT for their supporting during the observations. Y.A.
acknowledges the supports by CAS/SAFEA International Partnership Program for
Creative Research Teams (No. KTCX2-YW-T14), grant 11003044 from the National
Natural Science Foundation of China, and 2009’s President Excellent Thesis
Award of the Chinese Academy of Sciences. This research has made use of NASA’s
Astrophysical Data System (ADS).

## References

  * (1) Antonucci, R. R. J., & Miller, J. S. 1985, ApJ, 297, 621
  * (2) Bourke, T. L., Hyland, A. R., Robinson, G., James, S. D., & Wright, C. M. 1995, MNRAS, 276, 1067
  * (3) Ceccarelli, C., Baluteau, J.-P., Walmsley, M., et al. 2002, A&A, 383, 603
  * (4) Dahmen, G., Huttemeister, S., Wilson, T. L., & Mauersberger, R. 1998, A&A, 331, 959
  * (5) Danby, G., Flower, D. R., Valiron, P., Schilke, P., & Walmsley, C. M. 1988, MNRAS, 235, 229
  * (6) Downes, D., & Solomon, P. M. 1998, ApJ, 507, 615
  * (7) Gao, Y., & Solomon, P. M. 2004, ApJ, 606, 271
  * (8) Guesten, R., Walmsley, C. M., & Pauls, T. 1981, A&A, 103, 197
  * (9) Henkel, C., Braatz, J. A., Menten, K. M., & Ott, J. 2008, A&A, 485, 451
  * (10) Henkel, C., Jethava, N., Kraus, A., et al. 2005, A&A, 440, 893
  * (11) Henkel, C., Mauersberger, R., Peck, A. B., Falcke, H., & Hagiwara, Y. 2000, A&A, 361, L45
  * (12) Ho, P. T. P., & Townes, C. H. 1983, ARA&A, 21, 239
  * (13) Kaneko, N., Morita, K., Fukui, Y., et al. 1989, ApJ, 337, 691
  * (14) Lutz, D., Sturm, E., Genzel, R., Moorwood, A. F. M., & Sternberg, A. 1997, Ap&SS, 248, 217
  * (15) Martin, R. N., & Ho, P. T. P. 1986, ApJ, 308, L7
  * (16) Mauersberger, R., Henkel, C., Weiß, A., Peck, A. B., & Hagiwara, Y. 2003, A&A, 403, 561
  * (17) Meier, D. S., Turner, J. L., & Hurt, R. L. 2008, ApJ, 675, 281
  * (18) Ott, J., Henkel, C., Staveley-Smith, L., & Weiß, A. 2010, ApJ, 710, 105
  * (19) Ott, J., Weiss, A., Henkel, C., & Walter, F. 2005, ApJ, 629, 767
  * (20) Pillai, T., Wyrowski, F., Carey, S. J., & Menten, K. M. 2006, A&A, 450, 569
  * (21) Poncelet, A., Doucet, C., Perrin, G., Sol, H., & Lagage, P. O. 2007, A&A, 472, 823
  * (22) Raban, D., Jaffe, W., Röttgering, H., Meisenheimer, K., & Tristram, K. R. W. 2009, MNRAS, 394, 1325
  * (23) Ricci, R., Prandoni, I., Gruppioni, C., Sault, R. J., & de Zotti, G. 2006, A&A, 445, 465
  * (24) Schinnerer, E., Eckart, A., Tacconi, L. J., Genzel, R., & Downes, D. 2000, ApJ, 533, 850
  * Suto & (1983) Suto, M., & Lee, L. C. 1983, J. Chem. Phys., 78, 4515
  * (26) Takano, S., Hofner, P., Winnewisser, G., Nakai, N., & Kawaguchi, K. 2005, PASJ, 57, 549
  * (27) Takano, S., Nakai, N., & Kawaguchi, K. 2002, PASJ, 54, 195
  * (28) Tieftrunk, A. R., Megeath, S. T., Wilson, T. L., & Rayner, J. T. 1998, A&A, 336, 991
  * (29) Tomono, D., Terada, H., & Kobayashi, N. 2006, ApJ, 646, 774
  * (30) Walmsley, C. M., & Ungerechts, H. 1983, A&A, 122, 164
  * (31) Weiß, A., Neininger, N., Henkel, C., Stutzki, J., & Klein, U. 2001, ApJ, 554, L143
  * (32) Wilson, T. L., Henkel, C., & Hüttemeister, S. 2006, A&A, 460, 533


* [180] X. Lan and M. Schwager, “Rapidly-exploring random cycles: Persistent estimation of spatio-temporal fields with multiple sensing robots,” _IEEE Trans. on Robotics_ , vol. 32, no. 5, pp. 1230–1244, 2016.
  * [181] Y. Kantaros, B. Schlotfeldt, N. Atanasov, and G. J. Pappas, “Asymptotically optimal planning for non-myopic multi-robot information gathering,” in _Robotics: Science and Systems_ , 2019, pp. 22–26.
  * [182] S. Koga, A. Asgharivaskasi, and N. Atanasov, “Active SLAM over continuous trajectory and control: A covariance-feedback approach,” in _American Control Conf._ , 2022.
  * [183] S. Rahman and S. L. Waslander, “Uncertainty-constrained differential dynamic programming in belief space for vision based robots,” _IEEE Robotics and Automation L._ , vol. 6, no. 2, pp. 3112–3119, 2021.
  * [184] V. Murali, I. Spasojevic, W. Guerra, and S. Karaman, “Perception-aware trajectory generation for aggressive quadrotor flight using differential flatness,” in _American Control Conf._ , 2019, pp. 3936––3943.
  * [185] L. Carlone and D. Lyons, “Uncertainty-constrained robot exploration: a mixed-integer linear programming approach,” in _IEEE Int. Conf. on Robotics and Automation_ , 2014, pp. 1140–1147.
  * [186] C. Leung, S. Huang, and G. Dissanayake, “Active SLAM in structured environments,” in _IEEE Int. Conf. on Robotics and Automation_ , 2008, pp. 1898–1903.
  * [187] M. Xu, Y. Song, Y. Chen, S. Huang, and Q. Hao, “Invariant EKF based 2D active SLAM with exploration task,” in _IEEE Int. Conf. on Robotics and Automation_ , 2021, pp. 5350–5356.
  * [188] R. Martinez-Cantin, N. De Freitas, E. Brochu, J. Castellanos, and A. Doucet, “A bayesian exploration-exploitation approach for optimal online sensing and planning with a visually guided mobile robot,” _Autonomous Robots_ , vol. 27, no. 2, pp. 93–103, 2009.
  * [189] P. Karkus, D. Hsu, and W. S. Lee, “Qmdp-net: Deep learning for planning under partial observability,” _Advances in Neural Information Processing Systems_ , vol. 30, 2017.
  * [190] S. Bai, F. Chen, and B. Englot, “Toward autonomous mapping and exploration for mobile robots through deep supervised learning,” in _IEEE/RSJ Int. Conf. on Intelligent Robots and Systems_ , 2017, pp. 2379–2384.
  * [191] F. Chen, J. Wang, T. Shan, and B. Englot, “Autonomous exploration under uncertainty via graph convolutional networks,” in _Int. Symp. on Robotics Research_ , 2019.
  * [192] V. Mnih, K. Kavukcuoglu, D. Silver, A. A. Rusu, J. Veness, M. G. Bellemare, A. Graves, M. Riedmiller, A. K. Fidjeland, G. Ostrovski _et al._ , “Human-level control through deep reinforcement learning,” _Nature_ , vol. 518, no. 7540, p. 529, 2015.
  * [193] H. Van Hasselt, A. Guez, and D. Silver, “Deep reinforcement learning with double Q-learning,” in _30th AAAI Conf. on Artificial Intelligence_ , 2016, pp. 2094–2100.
  * [194] M. Hessel, J. Modayil, H. Van Hasselt, T. Schaul, G. Ostrovski, W. Dabney, D. Horgan, B. Piot, M. Azar, and D. Silver, “Rainbow: Combining improvements in deep reinforcement learning,” in _32nd AAAI Conf. on Artificial Intelligence_ , 2018, pp. 3215–3222.
  * [195] T. P. Lillicrap, J. J. Hunt, A. Pritzel, N. Heess, T. Erez, Y. Tassa, D. Silver, and D. Wierstra, “Continuous control with deep reinforcement learning,” _arXiv e-prints:1509.02971_ , 2015.
  * [196] V. Mnih, A. P. Badia, M. Mirza, A. Graves, T. Lillicrap, T. Harley, D. Silver, and K. Kavukcuoglu, “Asynchronous methods for deep reinforcement learning,” in _Int. Conf. on Machine Learning_ , 2016, pp. 1928–1937.
  * [197] K. Arulkumaran, M. P. Deisenroth, M. Brundage, and A. A. Bharath, “Deep reinforcement learning: A brief survey,” _IEEE Signal Processing Mag._ , vol. 34, no. 6, pp. 26–38, 2017.
  * [198] F. Zeng, C. Wang, and S. S. Ge, “A survey on visual navigation for artificial agents with deep reinforcement learning,” _IEEE Access_ , vol. 8, pp. 135 426–135 442, 2020.
  * [199] L. Tai and M. Liu, “Mobile robots exploration through CNN-based reinforcement learning,” _Robotics and Biomimetics_ , vol. 3, no. 1, p. 24, 2016.
  * [200] P. Mirowski, R. Pascanu, F. Viola, H. Soyer, A. J. Ballard, A. Banino, M. Denil, R. Goroshin, L. Sifre, K. Kavukcuoglu, D. Kumaran, and R. Hadsell, “Learning to navigate in complex environments,” in _Int. Conf. on Learning Representations_ , 2017.
  * [201] R. M. Ryan and E. L. Deci, “Intrinsic and extrinsic motivations: Classic definitions and new directions,” _Contemporary Educational Psychology_ , vol. 25, no. 1, pp. 54–67, 2000.
  * [202] M. Bellemare, S. Srinivasan, G. Ostrovski, T. Schaul, D. Saxton, and R. Munos, “Unifying count-based exploration and intrinsic motivation,” in _30th Int. Conf. on Neural Information Processing Systems_ , 2016, pp. 1471–1479.
  * [203] T. Chen, S. Gupta, and A. Gupta, “Learning exploration policies for navigation,” in _Int. Conf. on Learning Representations_ , 2019.
  * [204] O. Zhelo, J. Zhang, L. Tai, M. Liu, and W. Burgard, “Curiosity-driven exploration for mapless navigation with deep reinforcement learning,” in _IEEE Int. Conf. on Robotics and Automation Workshop in Machine Learning in the Planning and Control of Robot Motion_ , 2018.
  * [205] J. Hu, H. Niu, J. Carrasco, B. Lennox, and F. Arvin, “Voronoi-based multi-robot autonomous exploration in unknown environments via deep reinforcement learning,” _IEEE Trans. on Vehicular Technology_ , vol. 69, no. 12, pp. 14 413–14 423, 2020.
  * [206] M. Lodel, B. Brito, A. Serra-Gómez, L. Ferranti, R. Babuška, and J. Alonso-Mora, “Where to look next: Learning viewpoint recommendations for informative trajectory planning,” in _IEEE Int. Conf. on Robotics and Automation_ , 2022, pp. 4466–4472.
  * [207] K. Zhu and T. Zhang, “Deep reinforcement learning based mobile robot navigation: A review,” _Tsinghua Science and Technology_ , vol. 26, no. 5, pp. 674–691, 2021.
  * [208] M. Hausknecht and P. Stone, “Deep recurrent Q-learning for partially observable mdps,” in _AAAI Fall Symp. Series_ , 2015.
  * [209] H. Hu, K. Zhang, A. H. Tan, M. Ruan, C. Agia, and G. Nejat, “A sim-to-real pipeline for deep reinforcement learning for autonomous robot navigation in cluttered rough terrain,” _IEEE Robotics and Automation L._ , vol. 6, no. 4, pp. 6569–6576, 2021.
  * [210] K. Yokoyama and K. Morioka, “Autonomous mobile robot with simple navigation system based on deep reinforcement learning and a monocular camera,” in _IEEE/SICE Int. Symp. on System Integration_ , 2020, pp. 525–530.
  * [211] H. Shi, L. Shi, M. Xu, and K.-S. Hwang, “End-to-end navigation strategy with deep reinforcement learning for mobile robots,” _IEEE Trans. on Industrial Informatics_ , vol. 16, no. 4, pp. 2393–2402, 2019.
  * [212] B. Gerkey, R. T. Vaughan, and A. Howard, “The player/stage project: Tools for multi-robot and distributed sensor systems,” in _11th Int. Conf. on Advanced Robotics_ , vol. 1, 2003, pp. 317–323.
  * [213] N. Koenig and A. Howard, “Design and use paradigms for Gazebo, an open-source multi-robot simulator,” in _IEEE/RSJ Int. Conf. on Intelligent Robots and Systems_ , vol. 3, 2004, pp. 2149–2154.
  * [214] E. Rohmer, S. P. Singh, and M. Freese, “V-REP: A versatile and scalable robot simulation framework,” in _IEEE/RSJ Int. Conf. on Intelligent Robots and Systems_ , 2013, pp. 1321–1326.
  * [215] I. Zamora, N. G. Lopez, V. M. Vilches, and A. H. Cordero, “Extending the OpenAI gym for robotics: A toolkit for reinforcement learning using ROS and Gazebo,” _arXiv e-prints:1608.05742_ , 2016.
  * [216] G. Brockman, V. Cheung, L. Pettersson, J. Schneider, J. Schulman, J. Tang, and W. Zaremba, “OpenAI gym,” _arXiv e-prints:1606.01540_ , 2016.
  * [217] C. Beattie, J. Z. Leibo, D. Teplyashin, T. Ward, M. Wainwright, H. Küttler, A. Lefrancq, S. Green, V. Valdés, A. Sadik _et al._ , “Deepmind lab,” _arXiv e-prints:1612.03801_ , 2016.
  * [218] M. Savva, A. Kadian, O. Maksymets, Y. Zhao, E. Wijmans, B. Jain, J. Straub, J. Liu, V. Koltun, J. Malik _et al._ , “Habitat: A platform for embodied AI research,” in _IEEE/CVF Int. Conf. on Computer Vision_ , 2019, pp. 9339–9347.
  * [219] J. Straub, T. Whelan, L. Ma, Y. Chen, E. Wijmans, S. Green, J. J. Engel, R. Mur-Artal, C. Ren, S. Verma _et al._ , “The Replica dataset: A digital replica of indoor spaces,” _arXiv e-prints:1906.05797_ , 2019.
  * [220] C. Li, F. Xia, R. Martín-Martín, M. Lingelbach, S. Srivastava, B. Shen, K. Vainio, C. Gokmen, G. Dharan, T. Jain, A. Kurenkov, K. Liu, H. Gweon, J. Wu, L. Fei-Fei, and S. Savarese, “iGibson 2.0: Object-centric simulation for robot learning of everyday household tasks,” in _5th Conf. on Robot Learning_ , vol. 164, 2022, pp. 455–465.
  * [221] B. Charrow, V. Kumar, and N. Michael, “Approximate representations for multi-robot control policies that maximize mutual information,” _Autonomous Robots_ , vol. 37, pp. 383–400, 2014.
  * [222] Y. Kantaros, B. Schlotfeldt, N. Atanasov, and G. J. Pappas, “Sampling-based planning for non-myopic multi-robot information gathering,” _Autonomous Robots_ , vol. 45, pp. 1029–1046, 2021.
  * [223] W. Burgard, M. Moors, D. Fox, R. Simmons, and S. Thrun, “Collaborative multi-robot exploration,” in _IEEE Int. Conf. on Robotics and Automation_ , vol. 1, 2000, pp. 476–481.
  * [224] B. Schlotfeldt, D. Thakur, N. Atanasov, V. Kumar, and G. J. Pappas, “Anytime planning for decentralized multi-robot active information gathering,” _IEEE Robotics and Automation L._ , vol. 3, no. 2, pp. 1025–1032, 2018.
  * [225] M. Ossenkopf, G. Castro, F. Pessacg, K. Geihs, and P. De Cristóforis, “Long-horizon active SLAM system for multi-agent coordinated exploration,” in _European Conf. on Mobile Robots_ , 2019, pp. 1–6.
  * [226] M. Lauri, E. Heinänen, and S. Frintrop, “Multi-robot active information gathering with periodic communication,” in _IEEE Int. Conf. on Robotics and Automation_ , 2017, pp. 851–856.
  * [227] Z. I. Botev, D. P. Kroese, R. Y. Rubinstein, and P. L’Ecuyer, “The cross-entropy method for optimization,” in _Handbook of Statistics_. Elsevier, 2013, vol. 31, pp. 35–59.
  * [228] V. Indelman, “Cooperative multi-robot belief space planning for autonomous navigation in unknown environments,” _Autonomous Robots_ , vol. 42, no. 2, pp. 353–373, 2018.
  * [229] T. Regev and V. Indelman, “Multi-robot decentralized belief space planning in unknown environments via efficient re-evaluation of impacted paths,” in _IEEE/RSJ Int. Conf. on Intelligent Robots and Systems_ , 2016, pp. 5591–5598.
  * [230] G. Best, J. Faigl, and R. Fitch, “Multi-robot path planning for budgeted active perception with self-organising maps,” in _IEEE/RSJ Int. Conf. on Intelligent Robots and Systems_ , 2016, pp. 3164–3171.
  * [231] M. Fehr, T. Taubner, Y. Liu, R. Siegwart, and C. Cadena, “Predicting unobserved space for planning via depth map augmentation,” in _19th Int. Conf. on Advanced Robotics_ , 2019, pp. 30–36.
  * [232] S. K. Ramakrishnan, Z. Al-Halah, and K. Grauman, “Occupancy anticipation for efficient exploration and navigation,” in _16th European Conf. on Computer Vision_. Springer, 2020, pp. 400–418.
  * [233] K. D. Katyal, A. Polevoy, J. Moore, C. Knuth, and K. M. Popek, “High-speed robot navigation using predicted occupancy maps,” in _IEEE Int. Conf. on Robotics and Automation_ , 2021, pp. 5476–5482.
  * [234] S. Y. Hayoun, E. Zwecher, E. Iceland, A. Revivo, S. R. Levy, and A. Barel, “Integrating deep-learning-based image completion and motion planning to expedite indoor mapping,” _arXiv e-prints:2011.02043_ , 2020.
  * [235] R. Shrestha, F.-P. Tian, W. Feng, P. Tan, and R. Vaughan, “Learned map prediction for enhanced mobile robot exploration,” in _IEEE Int. Conf. on Robotics and Automation_ , 2019, pp. 1197–1204.
  * [236] A. Dai, D. Ritchie, M. Bokeloh, S. Reed, J. Sturm, and M. Nießner, “Scancomplete: Large-scale scene completion and semantic segmentation for 3D scans,” in _IEEE Conf. on Computer Vision and Pattern Recognition_ , 2018, pp. 4578–4587.
  * [237] C. Richter and N. Roy, “Safe visual navigation via deep learning and novelty detection,” in _Robotics: Science and Systems_ , 2017.
  * [238] C. Richter, W. Vega-Brown, and N. Roy, “Bayesian learning for safe high-speed navigation in unknown environments,” in _Robotics Research_. Springer, 2018, pp. 325–341.
  * [239] O. Asraf and V. Indelman, “Experience-based prediction of unknown environments for enhanced belief space planning,” in _IEEE/RSJ Int. Conf. on Intelligent Robots and Systems_ , 2020, pp. 6781–6788.
  * [240] N. Fairfield and D. Wettergreen, “Active SLAM and loop prediction with the segmented map using simplified models,” in _Field and Service Robotics_. Springer, 2010, pp. 173–182.
  * [241] I. Armeni, Z. He, J. Gwak, A. Zamir, M. Fischer, J. Malik, and S. Savarese, “3D scene graph: A structure for unified semantics, 3D space, and camera,” in _Int. Conf. on Computer Vision_ , 2019, pp. 5664–5673.
  * [242] N. Hughes, Y. Chang, and L. Carlone, “Hydra: a real-time spatial perception system for 3D scene graph construction and optimization,” _Robotics: Science and Systems_ , 2022.
  * [243] Z. Ravichandran, L. Peng, N. Hughes, J. Griffith, and L. Carlone, “Hierarchical representations and explicit memory: Learning effective navigation policies on 3D scene graphs using graph neural networks,” in _IEEE Int. Conf. on Robotics and Automation_ , 2022, pp. 9272–9279.
  * [244] N. Sunderhauf and P. Protzel, “Towards a robust back-end for pose graph SLAM,” in _IEEE Int. Conf. on Robotics and Automation_ , 2012, pp. 1254–1261.
  * [245] E. Olson and P. Agarwal, “Inference on networks of mixtures for robust robot mapping,” _The Int. J. of Robotics Research_ , vol. 32, no. 7, pp. 826–840, 2013.
  * [246] H. Yang, P. Antonante, V. Tzoumas, and L. Carlone, “Graduated non-convexity for robust spatial perception: From non-minimal solvers to global outlier rejection,” _IEEE Robotics and Automation L._ , vol. 5, no. 2, pp. 1127–1134, 2020.
  * [247] V. Indelman, E. Nelson, J. Dong, N. Michael, and F. Dellaert, “Incremental distributed inference from arbitrary poses and unknown data association: Using collaborating robots to establish a common reference,” _IEEE Control Systems Mag._ , vol. 36, no. 2, pp. 41–74, 2016.
  * [248] M. Hsiao and M. Kaess, “MH-iSAM2: Multi-hypothesis iSAM using bayes tree and hypo-tree,” in _IEEE Int. Conf. on Robotics and Automation_ , 2019, pp. 1274–1280.
  * [249] O. Shelly and V. Indelman, “Hypotheses disambiguation in retrospective,” _IEEE Robotics and Automation L._ , vol. 7, no. 2, pp. 2321–2328, 2022.
  * [250] S. Pathak, A. Thomas, and V. Indelman, “A unified framework for data association aware robust belief space planning and perception,” _The Int. J. of Robotics Research_ , vol. 32, no. 2-3, pp. 287–315, 2018.
  * [251] M. Hsiao, J. G. Mangelson, S. Suresh, C. Debrunner, and M. Kaess, “ARAS: Ambiguity-aware robust active SLAM based on multi-hypothesis state and map estimations,” in _IEEE/RSJ Int. Conf. on Intelligent Robots and Systems_ , 2020, pp. 5037–5044.
  * [252] P. M. Dames, “Distributed multi-target search and tracking using the PHD filter,” _Autonomous Robots_ , vol. 44, pp. 673–689, 2020.
  * [253] M. Shienman and V. Indelman, “D2A-BSP: Distilled data association belief space planning with performance guarantees under budget constraints,” in _IEEE Int. Conf. on Robotics and Automation_ , 2022.
  * [254] V. Indelman, “No correlations involved: Decision making under uncertainty in a conservative sparse information space,” _IEEE Robotics and Automation L._ , vol. 1, no. 1, pp. 407–414, 2016.
  * [255] K. Elimelech and V. Indelman, “Simplified decision making in the belief space using belief sparsification,” _The Int. J. of Robotics Research_ , vol. 41, no. 5, pp. 470–496, 2022.
  * [256] M. Barenboim and V. Indelman, “Adaptive information belief space planning,” in _31st Int. Joint Conf. on Artificial Intelligence and 25th European Conf. on Artificial Intelligence_ , 2022.
  * [257] J. P. Van Den Berg and M. H. Overmars, “Roadmap-based motion planning in dynamic environments,” _IEEE Trans. on Robotics_ , vol. 21, no. 5, pp. 885–897, 2005.
  * [258] M. R. U. Saputra, A. Markham, and N. Trigoni, “Visual SLAM and structure from motion in dynamic environments: A survey,” _ACM Computing Surveys_ , vol. 51, no. 2, pp. 1–36, 2018.
  * [259] D. Trivun, E. Šalaka, D. Osmanković, J. Velagić, and N. Osmić, “Active SLAM-based algorithm for autonomous exploration with mobile robot,” in _IEEE Int. Conf. on Industrial Technology_ , 2015, pp. 74–79.
  * [260] I. Maurović, M. Seder, K. Lenac, and I. Petrović, “Path planning for active SLAM based on the D* algorithm with negative edge weights,” _IEEE Trans. on Systems, Man, and Cybernetics: Systems_ , vol. 48, no. 8, pp. 1321–1331, 2017.
  * [261] E. Anshelevich, S. Owens, F. Lamiraux, and L. E. Kavraki, “Deformable volumes in path planning applications,” in _IEEE Int. Conf. on Robotics and Automation_ , vol. 3, 2000, pp. 2290–2295.
  * [262] S. Rodriguez, J.-M. Lien, and N. M. Amato, “Planning motion in completely deformable environments,” in _IEEE Int. Conf. on Robotics and Automation_ , 2006, pp. 2466–2471.
  * [263] R. A. Newcombe, D. Fox, and S. M. Seitz, “Dynamicfusion: Reconstruction and tracking of non-rigid scenes in real-time,” in _IEEE Conf. on Computer Vision and Pattern Recognition_ , 2015, pp. 343–352.
  * [264] J. Lamarca, S. Parashar, A. Bartoli, and J. Montiel, “Defslam: Tracking and mapping of deforming scenes from monocular sequences,” _IEEE Trans. on Robotics_ , 2020.
  * [265] Y. Xu, J. Yu, J. Tang, J. Qiu, J. Wang, Y. Shen, Y. Wang, and H. Yang, “Explore-Bench: Data sets, metrics and evaluations for frontier-based and deep-reinforcement-learning-based autonomous exploration,” in _IEEE Int. Conf. on Robotics and Automation_ , 2022, pp. 6225–6231.
  * [266] A. P. del Pobil, R. Madhavan, and E. Messina, “Benchmarks in robotics research,” in _IEEE/RSJ Int. Conf. on Intelligent Robots and Systems Workshop_ , 2006.
  * [267] B. Calli, A. Walsman, A. Singh, S. Srinivasa, P. Abbeel, and A. M. Dollar, “Benchmarking in manipulation research: Using the Yale-CMU-Berkeley object and model set,” _IEEE Robotics & Automation Mag._, vol. 22, no. 3, pp. 36–52, 2015.
  * [268] J. Pineau, P. Vincent-Lamarre, K. Sinha, V. Larivière, A. Beygelzimer, F. d’Alché Buc, E. Fox, and H. Larochelle, “Improving reproducibility in machine learning research: A report from the NeurIPS 2019 reproducibility program,” _J. of Machine Learning Research_ , vol. 22, 2021\.
  * [269] P. Ammirato, A. C. Berg, and J. Kosecka, “Active vision dataset benchmark,” in _IEEE Conf. on Computer Vision and Pattern Recognition Workshops_ , 2018, pp. 2046–2049.
  * [270] D. Hall, B. Talbot, S. R. Bista, H. Zhang, R. Smith, F. Dayoub, and N. Sünderhauf, “Benchbot environments for active robotics (BEAR): Simulated data for active scene understanding research,” _The Int. J. of Robotics Research_ , 2022.
  * [271] M. Walter, F. Hover, and J. Leonard, “Slam for ship hull inspection using exactly sparse extended information filters,” in _IEEE Int. Conf. on Robotics and Automation_ , 2008, pp. 1463–1470.
  * [272] J. Serafin, M. Di Cicco, T. Bonanni, C. Stachniss, and V. Ziparo, “Robots for exploration, digital preservation and visualization of archaeological sites,” _Artificial Intelligence for Cultural Heritage_ , p. 121, 2016.
  * [273] P. Li, C.-y. Yang, R. Wang, and S. Wang, “A high-efficiency, information-based exploration path planning method for active simultaneous localization and mapping,” _Int. J. of Advanced Robotic Systems_ , vol. 17, no. 1, 2020.


# ArCovidVac: Analyzing Arabic Tweets About COVID-19 Vaccination

###### Abstract

The emergence of the COVID-19 pandemic and the first global infodemic have
changed our lives in many different ways. We relied on social media to get the
latest information about COVID-19 pandemic and at the same time to disseminate
information. The content in social media consisted not only health related
advise, plans, and informative news from policymakers, but also contains
conspiracies and rumors. It became important to identify such information as
soon as they are posted to make an actionable decision (e.g., debunking
rumors, or taking certain measures for traveling). To address this challenge,
we develop and publicly release the first largest manually annotated Arabic
tweet dataset, ArCovidVac, for the COVID-19 vaccination campaign, covering
many countries in the Arab region. The dataset is enriched with different
layers of annotation, including, (i) Informativeness (more vs. less importance
of the tweets); (ii) fine-grained tweet content types (e.g., advice, rumors,
restriction, authenticate news/information); and (iii) stance towards
vaccination (pro-vaccination, neutral, anti-vaccination). Further, we
performed in-depth analysis of the data, exploring the popularity of different
vaccines, trending hashtags, topics and presence of offensiveness in the
tweets. We studied the data for individual types of tweets and temporal
changes in stance towards vaccine. We benchmarked the ArCovidVac dataset using
transformer architectures for informativeness, content types, and stance
detection.

Keywords: COVID-19, Vaccination, Stance Detection

languageresourceLanguage Resources utf8

ArCovidVac: Analyzing Arabic Tweets About COVID-19 Vaccination

Hamdy Mubarak1, Sabit Hassan2, Shammur Absar Chowdhury1 and Firoj Alam1
---
1Qatar Computing Research Institute, HBKU, Qatar
2University of Pittsburgh, USA
{hmubarak, shchowdhury<EMAIL_ADDRESS><EMAIL_ADDRESS>

Abstract content

## 1\. Introduction

Social media are integrated with our daily life. We share and access
information through social media platforms making it the most prominent form
of communication. Due to its reach to a larger and international population,
many organizations and individuals use them to circulate their contents. Thus
also making these platforms a constitutive part of online news distribution
and consumption [Mitchell and Page, 2014].

In Figure 1 and 2, we demonstrate how online users share information (rumors,
plan, travel restriction, personal experience). The post containing advice is
important to reduce the spread as vaccinated people can be a carrier.
Identifying such types of content from social media become important to the
government, international and local organisation for understanding
psychological and physical well being along with public reactions to every
taken actions. Such an understanding can (i) aid decision making by
governments; and (ii) prevent rumours and fake cures that can bring harm to
the society. Research studies have been conducted using numbers of COVID-19
datasets collected from Twitter. The research focused on: unlabeled [Chen et
al., 2020, Banda et al., 2020a, Alqurashi et al., 2020, Haouari et al., 2020],
automatically labeled [Abdul-Mageed et al., 2020, Qazi et al., 2020b], labeled
using distant supervision [Cinelli et al., 2020, Zhou et al., 2020], and small
manually annotated [Song et al., 2020, Vidgen et al., 2020, Shahi and Nandini,
2020, Pulido et al., 2020, Alam et al., 2020, Alam et al., 2021] datasets.

Despite Arabic being one of the dominant languages on Twitter [Alshaabi et
al., 2020], a very few research targeted toward aiding governments and
international organisations in their decision making and understanding public
perspective towards the vaccine, in the Arab region.

To aid such decision making process, in this study, we designed and publicly
released the largest manually annotated COVID-19 tweets regarding its vaccine
and vaccination campaigns in the Arab region. Our contributions can be
summarized as follows:

  * •

We develop a large manually annotated COVID-19 vaccine infodemic, covering
different countries in the Arab region, targeted to aid the policymakers and
the society as a whole. To the best of our knowledge, this is the first
dataset about COVID-19 vaccine in Arabic with diverse type of annotations.

  * •

We categorise the tweets for multiple classes (10 classes) including: plan,
request, advice, restrictions, rumors, authenticate news or information,
personal experience among others.

  * •

We annotate the tweets, specifying their stance towards vaccine/vaccination
process. We classify them into positive (pro-vaccination), negative (against
vaccination) or as neutral stance.

  * •

We analyse the tweets for different annotated classes and explore what topics
the content covers, top hashtags in each country, common sources that users
post their tweets in different countries, etc. Moreover, we analyse the
temporal changes in public stance on vaccination over time.

  * •

We benchmark the released dataset for several tasks. The classification tasks
includes (i) discriminating informative tweets from not-informative ones; (ii)
fine-grained multi-class tweet type categorisation; and (iii) stance detection
using transformer architectures.

  * •

We make our annotation guidelines, data and code freely
available.111https://alt.qcri.org/resources/ArCovidVac.zip

Figure 1: Examples for classes: Celebrity, Info-news, Plan, Requests and
Advice Figure 2: Examples for classes: Rumors, Requests, Others, Unrelated and
Personal

## 2\. Related Work

Research studies on COVID-19 focused on sentiment analysis [Yang et al.,
2020], propagation of misinformation [Huang and Carley, 2020, Shahi et al.,
2020], credibility check [Cinelli et al., 2020, Pulido et al., 2020, Zhou et
al., 2020], detecting racial prejudices and fear [Medford et al., 2020, Vidgen
et al., 2020] along with situational information, e.g., caution and advice [Li
et al., 2020]. Moreover, studies also include detecting mentions and stance
with respect to known misconceptions [Hossain et al., 2020].

These studies relies mostly on the social media dataset – mainly Twitter using
queries or some distant supervision. Most of the large-scale COVID-19 datasets
are unlabeled tweet collection, including multi-lingual dataset of 123M tweets
[Chen et al., 2020], 152M tweets [Banda et al., 2020b], a billion multilingual
tweets [Abdul-Mageed et al., 2020] and GeoCoV19 [Qazi et al., 2020a]
containing 524M tweets with their location information. In addition to
unlabeled data, some dataset are created using distant supervision [Cinelli et
al., 2020, Zhou et al., 2020] and some manually annotated [Song et al., 2020,
Vidgen et al., 2020, Shahi and Nandini, 2020, Pulido et al., 2020].

For Arabic, we see a similar trend in developing datasets. The Arabic dataset,
studied in [Alqurashi et al., 2020] provide a large dataset of Arabic tweets
containing keywords related to COVID-19. Similarly, ArCOV-19 proposed in
[Haouari et al., 2020], contains 750K tweets obtained by querying Twitter. The
manually labeled datasets are relatively fewer and also diversity of annotated
labels is little to none. Authors in [Alam et al., 2020, Alam et al., 2021]
manually annotated tweets in multiple languages for fact-checking, harmfulness
to society, and the relevance of the tweets to governments or policy makers.
Another study in [Alsudias and Rayson, 2020] collected 1M unique Arabic tweets
related to COVID-19 in the early 2020, among which a random 2000 tweets are
annotated for rumor detection based on the tweets posted by the Ministry of
Health in Saudi Arabia. Authors in [Mubarak and Hassan, 2020] annotated 8K
tweets and labeled them for different types of content. In [Yang et al.,
2020], author also annotated 10K Arabic and English tweets for the task of
fine-grained sentiment analysis.

#### Our Dataset:

Prior studies are mainly focused on one or two aspects of actionable
information (e.g., factuality, rumor detection). In comparison, our work is
focused on Informativeness, fine-grained content types and their stance
towards vaccination with manual annotation of ($10K$) Arabic tweets. Such a
diversity of labels enables the community to design and develop models in a
multitask learning setup (i.e., fine-grained content types and stance in one
model).

In addition, we specifically developed the dataset targeting the vaccination
campaigns in the Arab region, covering many countries. Unlike [Mubarak and
Hassan, 2020], we annotated the stance of the tweets. We also manually
analysed different annotated classes, explored topics and temporal changes in
stance regarding vaccines. We also studied the classification errors for the
stance and tweet type classification.

## 3\. Data Collection

Fighting the pandemic as well as infodemic requires to identify and understand
the content shared on social media either to reduce the spread of harmful
content, health related disinformation or to make an actionable decision
(attention worthy content for policymakers) [Alam et al., 2020, Alam et al.,
2021]. Such an understanding can help in identifying concerns and rumors about
vaccination, public sentiment, requests from governments and health
organizations, etc, while facilitating policymakers. This is a challenging
given that manually annotated language specific (e.g., Arabic) datasets are
scares.

To address this challenge, we collected Arabic tweets and manually annotated
them. To collect the tweets we used the following keywords: تطعيم، لقاح،
مطعوم¿ (vaccine, vaccination) between Jan 5th and Feb 3rd 2021.222Words used
in different countries in the Arab World. We used twarc search
API333https://github.com/DocNow/twarc to collect these tweets specifying
Arabic language. Our data collection timeline coincides with the phase where
many Arab countries already started their COVID-19 vaccination
campaigns.444https://tinyurl.com/mtm4wtrh For example, Saudi Arabic (SA)555ISO
3166-1 alpha-2 for country codes: https://tinyurl.com/mubpbjx6 started vaccine
rollout in the middle of Dec 2020.

We collected 550K unique tweets in total. After considering only tweets that
were liked or retweeted at least 10 times, we ended up with 14K tweets. We
assume that tweets with large number of likes or retweets are the most
important ones as they get highest attention from Twitter users. Out of them,
10K tweets were randomly chosen for manual annotation.

## 4\. Data Annotation

### 4.1. Annotation Task and Labels

We manually analyzed the random samples of selected tweets to understand the
data at hand and to design and define the annotation task and class labels.
Note that, we identified different types of class labels based on our
engagement with the ministry of public health and policymakers. For the three
types of labels, we manually annotate two types fine-grained content types and
their stance, and the informativeness type labels are inferred from fine-
grained content types. Below, we define the class labels with examples, which
are given to the annotators as instructions. We asked the annotators to follow
these definitions while annotating the tweets.

#### Fine-grained Content Types:

  1. 1.

Info-news: Information and news about vaccine and conditions of taking

  2. 2.

Celebrity: Vaccination of celebrities such as politicians, artists, and public
figures

  3. 3.

Plan: Governments’ vaccination plans, vaccination progress and reports

  4. 4.

Requests: Requests from governments, ex: speedup vaccination process

  5. 5.

Rumors: Rumors and refute rumors

  6. 6.

Advice Advice or instructions related to the virus or its vaccination

  7. 7.

Restrictions: Restrictions and issues that will be affected by taking vaccine,
ex: travel

  8. 8.

Personal: Personal story or opinion about the vaccine, ex: thank government

  9. 9.

Unrelated: Unrelated to vaccination process. This includes also spam and ads

  10. 10.

Others Related to vaccine but not listed in the above classes

#### Informativeness:

For informativeness, the former seven class labels are considered as more
informative and the later three class labels are considered as less
informative.

#### Stance:

For identifying stance we use the following labels:

  * •

Positive: Support vaccination, encourage people to take vaccine, and remove
their fears.
Example: متحدث الصحة: المشككون في فعالية لقاح ¿
كورونا سوف يأتون لأخذ اللقاح¿
Health spokesperson: Those who doubt the effectiveness of the Corona vaccine
will come and get it

  * •

Negative: Oppose vaccination and fear people from vaccine
Example: قلق بالغ في النرويج بسبب وفاة ¿
23 شخصا بعد تلقيهم لقاح فايزر¿
Extreme concern in Norway because 23 people have died after receiving the
Pfizer vaccine

  * •

Neutral/Unclear: Neither clearly support nor oppose vaccination
Example: توتر العلاقات بعد رفض بريطانيا ¿
تسليم فرنسا 15 مليون من لقاح كورونا¿
Relations are strained after Britain refused to deliver 15 million doses of
the Corona vaccine to France

### 4.2. Manual Annotation

For the manual annotation, we opted to use Appen crowdsourcing
platform666www.appen.com. One of the challenges with crowdsourced annotation
is to find a large number of qualified workers while filtering out low-quality
workers or spammers [Chowdhury et al., 2015, Chowdhury et al., 2014]. To deal
with this problem and to ensure the quality of the annotation we followed
standard evaluation [Chowdhury et al., 2020b], i.e., we used 150 gold standard
test tweets. Based on these gold standard test tweets, each annotator needed
to pass at least 70% of the tweets to participate in the annotation task.
Given that the content of the tweet is in Arabic, therefore, we only allowed
Arabic speaking participants from all Arab countries. While annotators needed
pass such criteria to annotate each tweet, we also designed the annotation
task to label each tweet by three annotators so that final label can be
selected based on the majority agreement.

In total, 245 annotators participated in the annotation task from different
Arab countries.777We paid more than $15 per hour of work to conform to the
minimum wage rate in US.

We selected the final label for each tweet based on the label agreement
score888https://success.appen.com/hc/en-us/articles/360038386492-How-to-
Calculate-Overall-Unit-Agreement greater than or equal to 70%. In Table 1, we
report the distribution of the dataset. As mentioned earlier, the class labels
for Informative are inferred from fine-grained labels.

#### Annotation agreement:

We compute the annotation agreement using Cohen’s kappa coefficient, and found
an agreement score of 0.82, which indicates high annotation quality.

Class | Count | Class | Count
---|---|---|---
Fine-grained | Informative
Info-news | 5,225 | More Informative | 7,891
Celebrity | 1,398 | Less Informative | 2,109
Plan | 860 | Total | 10,000
Requests | 172 | Stance
Rumors | 118 | Positive | 7,968
Advice | 94 | Negative | 638
Restrictions | 24 | Neutral/Unclear | 1,396
Personal | 1,430 | Total | 10,002
Unrelated | 450 |  |
Others | 229 |  |
Total | 10,000 |  |

Table 1: Distribution of the annotated class labels.

## 5\. Analysis

We present in-depth analysis of the ArCovidVac dataset, highlighting the
vaccine popularity, the most common trending hashtags in the dataset for
different countries. Moreover, we studied the popularity of mobile
applications used to fight spreading of the virus. We also analyze common
rumors and requests from governments followed by topic and country
distribution of tweets and the most common sources in each country. To our
believe, this analysis gives a broad understanding how people are reacting the
vaccine campaigns in each individual countries.

#### Vaccine Popularity:

Table 2 shows the list of vaccine hashtags mentioned in our dataset. This can
give a rough estimate about vaccine popularity in the Arab countries during
the period of our study. More details about these vaccines can be found at:
https://en.wikipedia.org/wiki/COVID-19˙vaccine

#### Trending Hashtags:

The most frequent hashtags in different countries are listed in Table 3. The
main messages in these hashtags show worry from vaccination, advice to take
precautionary measures, and reassure people that vaccine is safe.

Vaccine | Top Hashtags | # | CC
---|---|---|---
Pfizer | Pfizer, فايزر، لقاح_فايزر، بيونتيك، باينوتيك¿ | 184 | US
AstraZeneca |  استرازينيكا، اوكسفورد، استرازينكا¿ | 94 | UK
Sputnik V | 5سبوتنيك، سبوتنيك¿ | 65 | RU
Moderna | Moderna, موديرنا، مودرنا¿ | 43 | US
BBIBP-CorV | Sinopharm, سينوفارم¿ | 24 | CN
CoronaVac (Sinovac) |  سينوفاك، كورونافاك¿ | 10 | CN
Johnson & Johnson |  جونسون، جونسون_اند_جونسون¿ | 5 | US
Novavax |  نوفافاكس¿ | 2 | US

Table 2: Vaccine hashtag frequencies. Arabic hashtags are mainly different
transliterations of vaccine names. CC: Country Code of the manufacturing
company.

Country | Hashtags | Translation | #
---|---|---|---
IQ |  نريد_لقاح_آمن¿ | We want a safe vaccine | 288
SA |  الملك_يتلقي_لقاح_كورونا، نعود_بحذر¿ | The king takes COVID vaccine, We return cautiously | 174
LB |  لقاح_آمن، خليك_بالبيت¿ | Safe vaccine, Stay home | 157
AE |  يدا_بيد_نتعافى، اخترت_التطعيم¿ | Hand in hand we recover, I chose vaccination | 151
EG |  معا_نطمئن¿ | Together we can rest assured | 7
MA |  نبقاو_على_بال¿ | We remain alert | 7
OM |  عمان_تواجه_كورونا، التحصين_وقاية¿ | Oman fights Corona, Vaccination is protection | 6
JO |  المطعوم_وقاية، صحتك_بتهمنا¿ | Vaccine is protection, Your health is important to us | 5

Table 3: Most frequent hashtags in some Arab countries.

#### Rumors:

Rumors are very important class that needs more attention from governments and
policy makers. False claims about vaccines can negatively affect public trust
in vaccination campaigns. This may cause a threat to global public health. We
analyzed all rumors in our dataset and classified them into the following main
topics:

  * •

Vaccine is unsafe and ineffective: i) causes death and has side effects
especially on elderly; ii) manipulates genes; iii) causes infertility in
women.

  * •

Conspiracy theory: i) big countries or companies created the virus and its
vaccine for commercial purposes; ii) vaccine has chips to monitor and control
people; iii) vaccine is a biological weapon; iv) question about finding
vaccines within a year. Figure 3 shows the most retweeted and targeted tweet
in this category.

  * •

Doubts about government statistics, plans, and vaccination process.

Figure 3: An example of a tweet with conspiracy theory. Figure 4: An example
of a tweet with a request to give priority to the teaching professional.

#### Requests from Governments:

We analyzed all requests from governments and classified their main topics
into the following classes:

  * •

Safe vaccine: i) wait until studies and other countries prove vaccine
effectiveness and safety; ii) prefer US vaccines over their Chinese
counterparts; iii) refuse vaccine from the US (especially in Iraq).

  * •

Fair access to vaccine: i) rich and poor countries and people; ii) males and
females; iii) citizens, expats and refugees; iv) cities and regions in the
same country; v) politicians and common people; vi) Israel and Palestinians.

  * •

Vaccination process: i) speedup; ii) transparency in plans and contract
details; iii) finding alternative companies and cheaper vaccines; iv) allow
private sector to sell vaccines.

  * •

Give priority to some professionals such as doctors, teachers, players, and
natives. Figure 4 shows one of the most common tweets that asks to give
priority to the teaching professional.

#### Vaccine Announcements:

We spotted many news, posted in Jan 2020, about successful vaccines coming
from research labs in different countries in the MENA region, but in reality
none of those vaccines was used in any Arab countries until the date of our
study. Examples of such announcements are shown in Figure 5. We suspect these
news were posted for political or social purposes.

Figure 5: Vaccine announcements in Turkey, Egypt, Saudi Arabia and Iran.

#### Topic and Country Distribution:

We took a random sample of 1000 tweets and manually categorized them by expert
annotators for their main topics, such as health, politics, society and
economy. Additionally for all tweets, we use ASAD [Hassan et al., 2021], which
achieves 88.1% F1 score on the UL2C dataset [Mubarak and Hassan, 2021] for
country prediction of original tweet authors based on their user locations in
their profiles. Figure 6 shows that in addition to the health topics in most
of the tweets, one third of tweets talk about the vaccine from different
aspects (e.g., attacking politicians or countries). We also found that 7% of
tweets have hate speech or offensive language.

Figure 6: Distribution of topics labeled on a random sample of 1000 tweets.

Country distribution of tweets and top accounts that users share their posts
the most in each country are shown in Table 4. Analysis of such accounts shows
that people retweet posts mainly from online news agencies and newspapers in
their countries, and less from some journalists or activists. Most of those
accounts are verified. Surprisingly, accounts of ministries of health were not
among the top four sources in the listed countries. We anticipate one reason
for that might be due to the less amount of posts from ministries of health
compared to the large volumes of tweets that come from news agencies and
newspapers.

CC | % | Top Accounts
---|---|---
SA | 25 | sabqorg, Akhbaar24, KSA24, ajlnews
AE | 14 | cnnarabic, AlArabiya_Brk, skynewsarabia, AlHadath
LB | 11 |  | AlMayadeenNews, ALJADEEDNEWS, JamalCheaib
---
EG | 8 | youm7, AlMasryAlYoum, RassdNewsN, Extranewstv
GB | 5 | aawsat_News, AlarabyTV, IndyArabia, Mhd_AlObaidi
KW | 5 | liferdefempire, WhistleBlowerQ8, gucciya234, TfTeeeSH
JO | 4 | AlMamlakaTV, alrai, khaberni, RoyaTV
TR | 4 | TRTArabi, aa_arabic, TurkPressMedia, YeniSafakArabic
DZ | 3 | ennaharonline, El_Bilade, radioalgerie_ar, elkhabarlive
RU | 3 | RTarabic, RTarabic_Bn

Table 4: Distribution of top accounts across different countries. CC: Country
Code.

#### Distribution of Stance:

Figure 7 shows timeline of stance towards vaccine during the period of our
study. We observe a big increase of positive stance (pro vaccine) in Jan 8th
when media announced that the king of Saudi Arabia took the vaccine. This can
show the effect of sharing news about celebrity vaccination on public opinion.
On the opposite side, we found an increase of negative stance (anti vaccine)
in Jan 12th than other days due to wide adoption of a hashatg against US
vaccines among activists especially in Iraq.

Figure 7: Distribution of stance towards vaccine over time. pro: positive
stance, anti: negative stance.

#### Uses of Mobile Application:

We spotted also discussions about the mobile applications listed in Table 5
that help fighting the spread of COVID-19 virus. The purposes of these
applications vary from showing the health status of application users,
reporting violations of precautionary measures, booking and following-up
medical services, tracking medicines, and facilitating travel/visa process. It
is worth to mention that there are other applications used in different Arab
countries but didn’t appear in our dataset, e.g., احتراز¿ Ehterhaz
“Precaution” (QA, released in Apr’20), أمان¿ Aman “Safety” (JO, Aug’20), etc.

Application (and meaning) | Arabic Name | CC | Date | # | DL
---|---|---|---|---|---
Tawakkalna |  توكلنا¿ | SA | May’20 | 35 | 10M
(We Trust in God) |  |  |  |  |
Sehhaty |  صحتي¿ | SA | Dec’20 | 32 | 5M
(My Health) |  |  |  |  |
Kuwait Mosafer |  كويت مسافر¿ | KW | Feb’21 | 3 | 5K
(Kuwait Traveller) |  |  |  |  |
DHA |  صحة دبي¿ | AE | Dec’20 | 2 | 500K
(Dubai Health Authority) |  |  |  |  |
Al Hosn UAE |  الحصن¿ | AE | Apr’20 | 1 | 1M
(The Fort) |  |  |  |  |

Table 5: Applications used to fight COVID-19 in some Arab countries. DL:
Downloads at Google Store in May’20.

## 6\. Experiments

Class | Train | Dev | Test
---|---|---|---
Informativeness
More informative | 5482 | 819 | 1590
Less informative | 1518 | 181 | 410
Fine-grained categorization
Info-news | 3623 | 545 | 1057
Celebrity | 977 | 145 | 276
Plan | 606 | 82 | 172
Requests | 112 | 20 | 40
Rumors | 79 | 15 | 24
Advice | 67 | 10 | 17
Restrictions | 18 | 2 | 4
Personal | 1027 | 128 | 275
Unrelated | 324 | 36 | 90
Others | 167 | 17 | 45
Stance
Negative | 439 | 70 | 127
Neutral | 1017 | 126 | 253
Positive | 5544 | 804 | 1620

Table 6: Distribution of labels for different tasks.

For the experiments, we randomly split the data into train, dev and test sets
with 7000, 1000 and 2000 tweets, respectively. Table 6 shows the distribution
of labels across the three label sets, defined as three tasks, which include
(i) Task1: distinguish important tweets from less important ones, (ii) Task2:
fine-grained classification of important tweets, and (iii) Task3 stance of the
tweets.

We train several models, SVM with different features combinations, and
different tranformers models as discussed below.

To measure the performance of the models we compute and report macro-averaged
Precision (P), Recall (R) and F1 score along with Accuracy (Acc) on test set.
We use F1 score as the primary metric for comparison.

### 6.1. Classification Models

#### Support Vector Machines (SVMs)

SVMs are known to perform decently for Arabic text classification tasks, with
imbalanced class distribution, in tasks such as offensiveness detection
[Hassan et al., 2020, Chowdhury et al., 2020b], text categorisation [Chowdhury
et al., 2020a] or dialect identification [Abdelali et al., 2020]. Due to its
popularity and efficacy among machine learning algorithms, this is one of the
algorithm we explored in this study for the aforementioned classification
tasks.

Using SVM, we experimented with character and word n-gram features weighted by
term frequency-inverse term document frequency (tf-idf). We report results for
only the most significant ranges, namely, word [1-3] and character [2-7]. As
for the classifier training, we used LinearSVC implementation by scikit-learn
999https://scikit-learn.org/. We use default scikit-learn parameters.

#### Deep Contextualized Transformer Models (BERT)

Transformer-based pre-trained contextual embeddings, such as BERT [Devlin et
al., 2019], have outperformed other classifiers in many NLP tasks. We used
AraBERT [Antoun et al., 2020], a BERT-based model trained on Arabic news and
QARiB [Abdelali et al., 2021], another BERT-model trained on Arabic Wikipedia
and Twitter data. We used ktrain library [Maiya, 2020] that utilizes
Huggingface101010https://huggingface.co/ implementation to fine-tune AraBERT
and QARiB. We used learning rate of 8e-5, truncating length of 54 and fine-
tuned for 3 epochs.

### 6.2. Results

#### Task1: Informativeness

For discriminating between more vs. less informative tweets, we designed
binary classifiers and reported the results in Table 7. For baseline, we used
majority approach where we assign the label of most frequent class. We
observed the fine-tuned BERT models, AraBERT and QARiB outperform the SVMs
significantly. We noticed AraBERT achieves the highest macro F1 score of 80%.

#### Task2: Fine-grained Tweet Categorization

We experiment with fine-grained labels using the multiclass classification
setting. Due to skewed class distribution, we merged scarce classes (see Table
1) and use the hierarchical representation for further classification. For
this, we merge Restrict and Request classes with Plan, whereas we integrated
Advice tweets with Info-news due to their similarity in nature. We exclude
Rumor class since detecting rumors is difficult without any fact-checking or
other contextual features.

We end up with four classes: i) Info-news, ii) Celebrity, iii) Plan, and iv)
Less Informative. From Table 7, we noticed all classifiers outperform the
majority baseline. Moreover, we noticed that once again, the fine-tuned BERT
models, AraBERT and QARiB outperform the simple SVMs. With F1 score of 67.1,
QARiB outperforms AraBERT (F1 score of 64.3) by 2.8%. From the confusion
matrix (see Figure 8), we observe a confusion for the class Plan with
Info_News. Such confusion is indeed expected due to the similarity in nature
of the tweets. For example, plans introduced by the government are very much
similar to the tweets that are discussing the vaccine news or condition to
take it.

Model | Features | Acc. | P | R | F1
---|---|---|---|---|---
Informativeness
Majority |  | 79.5 | 39.8 | 50.0 | 44.3
SVM | W[1-3] | 84.0 | 75.7 | 73.1 | 74.3
SVM | C[2-7] | 84.9 | 77.6 | 72.9 | 74.8
SVM | C[2-7] + W[1-3] | 84.6 | 76.8 | 73.0 | 74.6
QARiB |  | 86.0 | 78.4 | 80 | 79.1
AraBERT |  | 86.4 | 78.9 | 81.3 | 80.0
Fine-grained categorization (multiclass)
Majority |  | 54.4 | 13.6 | 25.0 | 17.6
SVM | W[1-3] | 70.2 | 66.4 | 57.9 | 59.0
SVM | C[2-7] | 71.6 | 66.7 | 58.0 | 58.8
SVM | C[2-7] + W[1-3] | 72.0 | 68.7 | 59.3 | 60.5
QARiB |  | 72.1 | 66.2 | 68.2 | 67.1
AraBERT |  | 75.4 | 69.2 | 65.1 | 64.3
Stance Detection (multiclass)
Majority |  | 81.0 | 27.0 | 33.3 | 29.8
SVM | W[1-3] | 81.6 | 60.8 | 48.6 | 52.1
SVM | C[2-7] | 82.5 | 65.8 | 47.9 | 52.3
SVM | C[2-7] + W[1-3] | 82.5 | 62.6 | 47.7 | 51.4
QARiB |  | 81.6 | 64.3 | 62.7 | 63.1
AraBERT |  | 82.2 | 61.0 | 65.1 | 62.5

Table 7: Results for different classification tasks.

#### Task3: Stance Detection

For predicting the stance of the user (tweet), we designed a multiclass
classifier using the aforementioned algorithms. To identify the stance of the
tweets, we designed the classifier using 3 classes: positive, negative and
neural. From our results, in Table 7, we observe a similar pattern to Task2,
where transformers beat SVMs by a significant margin of about 10%. QARiB
achieves the best results with F1 score of 63.1%. Relatively high error
percentage suggests that stance detection is a difficult task for classifiers.
From the per class performance (see Figure 9), we noticed that both neutral
and negative stances are confused with the positive ones (the major class).

## 7\. Error Analysis

To understand the designed model behaviour, we analyze the errors and
confusion made by our best classifier, fine-tuned QARiB for Task2 (fine-
grained tweet categorization) and Task3 (stance detection).

Figure 8: Confusion matrix of fine-grained classification normalized over true
labels. Figure 9: Confusion matrix of stance detection normalized over true
labels.

Figure 8 shows the confusion matrix for fine grained classification by QARiB.
From the confusion matrix, we can see that most errors stem from Plan class
misclassified as Info-news. Figure 9 shows the confusion matrix for stance
detection. The confusion matrix shows that most errors stem from Anti-vaccine
and Neutral being tagged as Pro-vaccine due to high class imbalance.

### 7.1. Classification Errors: Task2

We picked 200 errors from our best classifier and analyzed them manually. We
can summarize most important cases in the following categories:

  * •

Confusion between classes: Info-news (vaccine) and Plan (vaccination process)
in the reference or system prediction.

  * •

Annotation errors: In some cases personal opinions about the vaccine are
labeled as informative.

  * •

Multilabel: Some tweets can have more than one class label. For example,
announcement from government about the vaccine followed by details about
vaccination plan. We plan to allow multiple labels in the future. We found
this case in 10% of the errors.

  * •

Contextual information: Need to consider associated multimedia posted with
tweet to get the correct prediction. For example, a question about the vaccine
and the answer is in an associated video.

### 7.2. Stance Errors: Task3

Similarly, for random 200 errors in stance prediction, we found the following
issues:

  * •

Full context: Need to understand full context including questions and
associated multimedia. This includes also considering sarcasm and negation.
For example, Is vaccine unsafe? Answer: No.

  * •

Annotation errors: Labelling a question about taking the vaccine or not as
positive stance.

  * •

Ambiguous content: Errors are due to spam, unrelated or unclear content.

  * •

Mixed/Targeted stance: For example, refusing vaccines from a certain country
but want a safer vaccine.

## 8\. Key Observations

In this study, we show the popularity of different vaccines, the common
hashtags, e.g., ‘safe vaccine’, present in the data, indicating the main
concern of the public towards the vaccine. We also observed different types of
rumors spreading the doubts on the safety of vaccination, conspiracy theory
and doubts in government assessments and plans. Meanwhile, we also noticed
informative tweets confirming vaccine safety, promising fair access and
priority and importance of front-liners vaccination. We observed the topics
covered in the tweets are mainly health, politics and society centred. From
stance timeline, we observed the reliability on the vaccine (pro- stance)
increase when leaders/influencers (e.g., kings) takes the vaccine, to set
examples.

As for the classification performance, for all the three tasks we noticed
transformer architecture outperforms SVMs and present a high performance
classifier even with imbalanced class levels. Such performance indicated the
efficacy of this data to aid automation of such process.

## 9\. Conclusion

We presented and publicly released the first large manually annotated Arabic
tweet dataset, ArCovidVac, for the COVID-19 vaccination campaign. The dataset
of 10k tweets, covering many countries in Arab region, is enriched with
different types of annotation, including, (i) informativeness of the tweets;
(ii) fine-grained tweet content types with 10 classes; and (iii) stance
towards vaccination identifying tweets with pro-vaccination (positive),
neutral, anti-vaccination content (negative). We performed an in-depth
analysis of the dataset considering diverse aspects and presented
classification results, which can be used as a benchmark in future studies and
aid policymakers in decision making process. In the future, we plan to study
the dynamics and changes in types/topics of the content and stance towards
vaccination in long run.

## 10\. Bibliographical References

## References

  * Abdelali et al., 2020 Abdelali, A., Mubarak, H., Samih, Y., Hassan, S., and Darwish, K.  (2020).  Arabic dialect identification in the wild.  ArXiv, abs/2005.06557.
  * Abdelali et al., 2021 Abdelali, A., Hassan, S., Mubarak, H., Darwish, K., and Samih, Y.  (2021).  Pre-training bert on arabic tweets: Practical considerations.
  * Abdul-Mageed et al., 2020 Abdul-Mageed, M., Elmadany, A., Pabbi, D., Verma, K., and Lin, R.  (2020).  Mega-COV: A billion-scale dataset of 65 languages for COVID-19.  arXiv:2005.06012.
  * Alam et al., 2020 Alam, F., Shaar, S., Dalvi, F., Sajjad, H., Nikolov, A., Mubarak, H., Martino, G. D. S., Abdelali, A., Durrani, N., Darwish, K., and Nakov, P.  (2020).  Fighting the covid-19 infodemic: Modeling the perspective of journalists, fact-checkers, social media platforms, policy makers, and the society.  arXiv:2005.00033.
  * Alam et al., 2021 Alam, F., Dalvi, F., Shaar, S., Durrani, N., Mubarak, H., Nikolov, A., Martino, G. D. S., Abdelali, A., Sajjad, H., Darwish, K., and Nakov, P.  (2021).  Fighting the COVID-19 infodemic in social media: A holistic perspective and a call to arms.  In Proceedings of the International AAAI Conference on Web and Social Media, ICWSM ’21.
  * Alqurashi et al., 2020 Alqurashi, S., Alhindi, A., and Alanazi, E.  (2020).  Large arabic twitter dataset on covid-19.  arXiv preprint arXiv:2004.04315.
  * Alshaabi et al., 2020 Alshaabi, T., Dewhurst, D. R., Minot, J. R., Arnold, M. V., Adams, J. L., Danforth, C. M., and Dodds, P. S.  (2020).  The growing amplification of social media: Measuring temporal and social contagion dynamics for over 150 languages on twitter for 2009–2020.  Available online at http://arxiv.org/abs/2003.03667.
  * Alsudias and Rayson, 2020 Alsudias, L. and Rayson, P.  (2020).  COVID-19 and Arabic Twitter: How can Arab world governments and public health organizations learn from social media?  In Proceedings of the 1st Workshop on NLP for COVID-19 at ACL 2020, Online, July. Association for Computational Linguistics.
  * Antoun et al., 2020 Antoun, W., Baly, F., and Hajj, H. M.  (2020).  Arabert: Transformer-based model for arabic language understanding.  ArXiv, abs/2003.00104.
  * Banda et al., 2020a Banda, J. M., Tekumalla, R., Wang, G., Yu, J., Liu, T., Ding, Y., and Chowell, G.  (2020a).  A large-scale COVID-19 Twitter chatter dataset for open scientific research – an international collaboration.  arXiv:2004.03688.
  * Banda et al., 2020b Banda, J. M., Tekumalla, R., Wang, G., Yu, J., Liu, T., Ding, Y., and Chowell, G.  (2020b).  A large-scale covid-19 twitter chatter dataset for open scientific research–an international collaboration.  arXiv preprint arXiv:2004.03688.
  * Chen et al., 2020 Chen, E., Lerman, K., and Ferrara, E.  (2020).  Tracking social media discourse about the COVID-19 pandemic: Development of a public coronavirus Twitter data set.  JMIR Public Health Surveill, 6(2):e19273, May.
  * Chowdhury et al., 2014 Chowdhury, S. A., Ghosh, A., Stepanov, E. A., Bayer, A. O., Riccardi, G., and Klasinas, I.  (2014).  Cross-language transfer of semantic annotation via targeted crowdsourcing.  In ISCA.
  * Chowdhury et al., 2015 Chowdhury, S. A., Calvo, M., Ghosh, A., Stepanov, E. A., Bayer, A. O., Riccardi, G., García, F., and Sanchis, E.  (2015).  Selection and aggregation techniques for crowdsourced semantic annotation task.  In Proc. of 16th ISCA.
  * Chowdhury et al., 2020a Chowdhury, S. A., Abdelali, A., Darwish, K., Soon-Gyo, J., Salminen, J., and Jansen, B. J.  (2020a).  Improving arabic text categorization using transformer training diversification.  In Proceedings of the Fifth Arabic Natural Language Processing Workshop, pages 226–236.
  * Chowdhury et al., 2020b Chowdhury, S. A., Mubarak, H., Abdelali, A., Jung, S.-g., Jansen, B. J., and Salminen, J.  (2020b).  A multi-platform arabic news comment dataset for offensive language detection.  In Proceedings of The 12th Language Resources and Evaluation Conference, pages 6203–6212.
  * Cinelli et al., 2020 Cinelli, M., Quattrociocchi, W., Galeazzi, A., Valensise, C. M., Brugnoli, E., Schmidt, A. L., Zola, P., Zollo, F., and Scala, A.  (2020).  The covid-19 social media infodemic.  arXiv preprint arXiv:2003.05004.
  * Devlin et al., 2019 Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K.  (2019).  BERT: Pre-training of deep bidirectional transformers for language understanding.  In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171–4186, Minneapolis, Minnesota, June. Association for Computational Linguistics.
  * Haouari et al., 2020 Haouari, F., Hasanain, M., Suwaileh, R., and Elsayed, T.  (2020).  ArCOV-19: The first Arabic COVID-19 twitter dataset with propagation networks.  arXiv:2004.05861.
  * Hassan et al., 2020 Hassan, S., Samih, Y., Mubarak, H., and Abdelali, A.  (2020).  ALT at SemEval-2020 task 12: Arabic and English offensive language identification in social media.  In Proceedings of the Fourteenth Workshop on Semantic Evaluation, pages 1891–1897, Barcelona (online), December. International Committee for Computational Linguistics.
  * Hassan et al., 2021 Hassan, S., Mubarak, H., Abdelali, A., and Darwish, K.  (2021).  ASAD: Arabic social media analytics and unDerstanding.  In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: System Demonstrations, pages 113–118, Online, April. Association for Computational Linguistics.
  * Hossain et al., 2020 Hossain, T., Logan IV, R. L., Ugarte, A., Matsubara, Y., Young, S., and Singh, S.  (2020).  COVIDLies: Detecting COVID-19 misinformation on social media.  In Proceedings of the 1st Workshop on NLP for COVID-19 (Part 2) at EMNLP 2020, Online, December. Association for Computational Linguistics.
  * Huang and Carley, 2020 Huang, B. and Carley, K. M.  (2020).  Disinformation and misinformation on twitter during the novel coronavirus outbreak.  arXiv preprint arXiv:2006.04278.
  * Li et al., 2020 Li, L., Zhang, Q., Wang, X., Zhang, J., Wang, T., Gao, T., Duan, W., Tsoi, K. K., and Wang, F.  (2020).  Characterizing the propagation of situational information in social media during covid-19 epidemic: A case study on weibo.  IEEE Transactions on Computational Social Systems, 7(2):556–562.
  * Maiya, 2020 Maiya, A. S.  (2020).  ktrain: A low-code library for augmented machine learning.  arXiv preprint arXiv:2004.10703.
  * Medford et al., 2020 Medford, R. J., Saleh, S. N., Sumarsono, A., Perl, T. M., and Lehmann, C. U.  (2020).  An ”infodemic”: Leveraging high-volume twitter data to understand public sentiment for the COVID-19 outbreak.  medRxiv 2020.04.03.20052936.
  * Mitchell and Page, 2014 Mitchell, A. and Page, D.  (2014).  State of the news media 2014: Overview.  Pew Research Center.
  * Mubarak and Hassan, 2020 Mubarak, H. and Hassan, S.  (2020).  Arcorona: Analyzing arabic tweets in the early days of coronavirus (covid-19) pandemic.  arXiv preprint arXiv:2012.01462.
  * Mubarak and Hassan, 2021 Mubarak, H. and Hassan, S.  (2021).  UL2C: Mapping user locations to countries on Arabic Twitter.  In Proceedings of the Sixth Arabic Natural Language Processing Workshop, pages 145–153, Kyiv, Ukraine (Virtual), April. Association for Computational Linguistics.
  * Pulido et al., 2020 Pulido, C. M., Villarejo-Carballido, B., Redondo-Sama, G., and Gómez, A.  (2020).  Covid-19 infodemic: More retweets for science-based information on coronavirus than for false information.  International Sociology, page 0268580920914755.
  * Qazi et al., 2020a Qazi, U., Imran, M., and Ofli, F.  (2020a).  Geocov19.  SIGSPATIAL Special, 12(1):6–15, Jun.
  * Qazi et al., 2020b Qazi, U., Imran, M., and Ofli, F.  (2020b).  GeoCoV19: A dataset of hundreds of millions of multilingual COVID-19 tweets with location information.  SIGSPATIAL Special, 12(1):6–15, June.
  * Shahi and Nandini, 2020 Shahi, G. K. and Nandini, D.  (2020).  Fakecovid–a multilingual cross-domain fact check news dataset for covid-19.  arXiv preprint arXiv:2006.11343.
  * Shahi et al., 2020 Shahi, G. K., Dirkson, A., and Majchrzak, T. A.  (2020).  An exploratory study of covid-19 misinformation on twitter.  arXiv preprint arXiv:2005.05710.
  * Song et al., 2020 Song, X., Petrak, J., Jiang, Y., Singh, I., Maynard, D., and Bontcheva, K.  (2020).  Classification aware neural topic model and its application on a new COVID-19 disinformation corpus.  arXiv:2006.03354.
  * Vidgen et al., 2020 Vidgen, B., Botelho, A., Broniatowski, D., Guest, E., Hall, M., Margetts, H., Tromble, R., Waseem, Z., and Hale, S.  (2020).  Detecting east asian prejudice on social media.
  * Yang et al., 2020 Yang, Q., Alamro, H., Albaradei, S., Salhi, A., Lv, X., Ma, C., Alshehri, M., Jaber, I., Tifratene, F., Wang, W., et al.  (2020).  Senwave: Monitoring the global sentiments under the covid-19 pandemic.  arXiv preprint arXiv:2006.10842.
  * Zhou et al., 2020 Zhou, X., Mulay, A., Ferrara, E., and Zafarani, R.  (2020).  ReCOVery: A multimodal repository for COVID-19 news credibility research.  arXiv:2006.05557.

## Appendix

## Ethics and Broader Impact

### Dataset Collection

We collected the dataset using the Twitter
API111111http://developer.twitter.com/en/docs with keywords that only use
terms related to COVID-19 vaccine, without other biases. We followed the terms
of use outlined by Twitter.121212http://developer.twitter.com/en/developer-
terms/agreement-and-policy Specifically, we only downloaded public tweets, and
we only distribute dehydrated Twitter IDs. We release the dataset by
maintaining Twitter data redistribution policy.

### Biases

We note that some of the annotations are subjective. Thus, it is inevitable
that there would be biases in our dataset. Yet, we have a very clear
instructions, which should reduce biases.

### Misuse Potential

Most datasets compiled from social media present some risk of misuse. We,
therefore, ask researchers to be aware that our dataset can be maliciously
used to unfairly moderate text (e.g., a tweet) that may not be malicious based
on biases that may or may not be related to demographics and other information
within the text. Intervention with human moderation would be required in order
to ensure this does not occur.

### Intended Use

Our dataset can enable automatic systems for analysis of social media content,
which could be of interest to practitioners, social media platforms, and
policymakers. Such systems can be used to alleviate the burden for social
media moderators, but human supervision would be required for more intricate
cases and in order to ensure that the system does not cause harm.