#  $\beta^{2}$ corrections to spherical EDF calculations for root-mean-square
charge radii

B. Alex Brown and Kei Minamisono  Department of Physics and Astronomy, FRIB
Laboratory, Michigan State University, East Lansing, MI 42284-1321, USA

###### Abstract

Root-mean-square charge radii are discussed in terms of spherical Energy
Density Functional (EDF) models corrected for quadrupole deformations.
Comparisons between experiment and theory are made for the absolute radii of
all even-even nuclei, for the isotonic shift between cadmium and tin isotopes,
the isotopic shifts of the calcium isotopes and the isotonic shift for nuclei
with $N=28$. We conclude that the data are well described in this approach,
except for the sharp rise just after the neutron magic numbers.

##  I Introduction

Root-mean-square (rms) charge radii of nuclei provide one of the most precise
insights into nuclear structure. New experiments are being carried out on long
chains of isotopes, and theoretical models are being improved, see ko22 , pe21
, re17  and references therein. Fig. (1) shows the measured rms charge radii
for even-even nuclei from calcium to tellurium. One observes kinks in the
isotopic trends at neutron numbers $N=28$ and $N=50$, as well as kinks in the
isotonic trends at the proton numbers $Z=28$ and $Z=50$. In particular, the
isotonic kink at $Z=50$ results in a close spacing between the rms charge
radii of cadmium and tin isotopes which is one of the topics of this paper.

We start with a comparison of the data to spherical energy-density functional
(EDF) calculations using the 12 Skyrme functionals from Table I of br14 . As
shown in Fig. 4 of ro15 , the main difference in the results for the rms
charge radii for these 12 functionals is correlated with the effective mass.
One group has $m^{*}/m\approx 1.0$. The results for $s8$ parameter set which
is representative of this group are shown in Fig. (2). The other group has
$m^{*}/m\approx 0.7-0.8$. The results for $s17$ parameter set which is
representative of this group are shown in Fig. (3). These are carried out in a
spherical basis with proton and neutron occupation numbers obtained from a
self-consistent exact-pairing calculation vo01  based on the $(J,T)=(0,1)$
two-body matrix elements from configuration-interaction Hamiltonians used for
the various mass regions. The calculations include the proton and neutron
finite sizes and the relativistic spin-orbit corrections as described in br84
.

The difference between experiment and the spherical calculations shows the
well-known effect of nuclear deformation in between the magic numbers of 28,
50, 82 and 126. To account for deformation one can carry out deformed EDF
calculations. Many such calculations give results at the energy minimum for a
given $\beta$ deformation. More advanced calculations include fluctuations
around the minimum de10 .

Figure 1: Measured rms charge radii for even-even nuclei from $Z=20$ (calcium)
to $Z=52$ (tellurium). The data are taken from compilations an13  and fr04
with updated results for Ca ga16 , Fe mi16 , Ni so22 , Cd cd18 . and Sn sn19 .

A different approach is to use the information contained in the reduced
transition probability $B(E\lambda)$ for the excitation of low-lying states to
deduce an effective defomation parameter $\beta$ (this includes the
fluctuations) and then use the Bohr collective model to evaluate the change in
the rms radius taking into account volume conservation as a function of
deformation bo69 . To order $\beta^{2}$ the mean-square radius is

$<r^{2}>=\,<r^{2}>_{0}\Big{[}1+\frac{5\beta^{2}}{4\pi}\Big{]},$ $None$

where

$<r^{2}>_{0}=\frac{3R_{0}^{2}}{5}$ $None$

is the mean-square radius with no deformation, and

$\beta^{2}=\displaystyle\sum_{\lambda\geq 2}\beta_{\lambda}^{2}.$ $None$

The proton $\beta^{2}$ is related to the $B(E\lambda,\uparrow)$ for 0+ to
$\lambda^{+}$ transitions (in units of e2) by

$\beta_{\lambda}^{2}=\frac{B(E\lambda,\uparrow)}{[\frac{3}{4\pi}Z\,R_{0}^{\lambda}]^{2}}.$
$None$

The operator for $E2$ is

$\displaystyle\sum_{i}r_{i}^{2}Y^{(2)}(\hat{r}_{i})e_{iq}{\rm e}$ $None$

where $e_{q}$ is the effective charge for protons ($q=p$) or neutrons ($q=n$).
Fluctuations of the nuclear surface give corrections to the rms charge radii
that have the same form as those of the Bohr deformed model es83 .

We will assume equal (isoscalar) proton and neutron deformations. A more
general expression for Eq. (1) with unequal proton and neutron deformations is
given in pi21 . Results for higher-order terms in Eq. (1) are given do84 ,
ah88 , be19 . For $\lambda$=2, the next most important term is proportional to
$\beta_{2}^{3}$. For $\mid\beta\mid$= 0.6 this term gives about a 10% change
to Eq. (1). This term depends on the sign of $\beta$ which is often not
measured, and it will not be included here.

Figure 2: (a) Difference between the experimental rms charge radii and those
obtained in a spherical EDF calculation with the $S8$ Skyrme functional. (b)
The increase in the rms charge radii calculated with Eq. (1) using
experimental $B(E2)$ values. (c) The difference between (a) and (b).  Figure
3: (a) Difference between the experimental rms charge radii and those obtained
in a spherical EDF calculation with the $s17$ Skyrme functional. (b) The
increase in the rms charge radii calculated with Eq. (1) using experimental
$B(E2)$ values. (c) The difference between (a) and (b).

The strength functions for $B(E2,\uparrow)$ are dominated by the low-lying
excited 2+ state and the high-lying giant quadrupole state. Low-lying excited
states fall into three groups, (a) states in well deformed nuclei, (b) states
in semi-magic nuclei formed from valence configurations below 2-3 MeV, and (c)
states in doubly-magic nuclei formed by particle-hole excitations.

In this paper we compare experimental rms charge radii for nuclei on or near
the semi-magic numbers with the results from three types of EDF calculations.
The first is for the spherical EDF with parameters determined from bulk
properties of doubly-magic nuclei skx , br13 , br14 , or from a large sample
of nuclear bulk properties together with other observables resulting in the
Sv-min set of EDF paramters kl09 . The second is the same spherical
calculations but supplemented with $\beta_{2}^{2}$ determined from
experimental and/or calculated $B(E2)$ values of type (a) and (b) above. The
third is the Fayans-type EDF fa00 , sa11  with pairing terms added to provide
an improved description of rms charge radii resulting in the Fy($\Delta$r) set
of parameters re17 .

In the following we first consider the global comparison for even-even nuclei
using experimental data for the $B(E2)$ values. We will call this the
$\beta_{2}^{2}$ correction. We also look at the relative size of contributions
to the rms radii coming from $\lambda$=2, 3 and 4. Next we look at the
specific application to the isotonic shift betwen cadmium and tin to compare
various approaches. Then we consider the isotopic shifts for $Z=20$ (calcium)
and the isotonic shifts for $N=28$ using experimental $B(E2)$ values. We also
use configuration-mixing calculations for $Z=20$ and $N=28$ to calculate
$<Q\cdot Q>$ for both even-even and odd-even isotopes in order to explore the
odd-even oscillations in the rms charge radii. At the end we discuss the
problem for the increase in charge radii after the magic numbers and suggest
possible solutions.

##  II Global Comparisons

Using the compiled experimental $B(E2,0^{+}\rightarrow 2^{+}_{1})$ values be2
the correction to the rms radii implied by Eq. (1) are shown in Figs. (2) and
(3). The bottom panel of these figures show difference $R_{ch}$(experiment) -
$R_{ch}$(theory) - $\Delta R_{ch}$($\beta_{2}$). $\Delta R_{ch}$($\beta_{2}$)
is the calculated correction based on $\beta_{2}^{2}$. The increase away from
the magic numbers accounts for part of the kinks in the experimental radii at
the magic numbers. There are some remaining deviations just after the magic
numbers. For the region of the mass number $A=100$, the $B(E2)$ for higher 2+
states are typically less than about 5% of those for the first 2+ state, see
for example li76 . The contribution for the higher 2+ states for the calcium
isotopes and the $Z=28$ isotones will be discussed below.

Figure 4:  Increase in the rms charge radii calculated with Eq. (1) using
experimental $B(E3)$ be3 .

The next most important multiple is $\lambda$=3. The increase in the rms
charge radii obtained from the experimental $B(E3,0^{+}\rightarrow 3^{-}_{1})$
values be3  are shown in Fig. (4). In contrast to the dramatic shell effects
observed for the $B(E2)$ in Figs. (2) and (3), the radius increase from the
$B(E3)$ are rather flat. For the doubly-magic nuclei the $B(E3)$ are particle-
hole excitation of the type (c) above. To the extent that the EDF parameters
are determined to reproduce the rms radii of these doubly-magic nuclei, the
effects of the $B(E3)$ contributions are contained in the EDF calculations.
For nuclei in between the magic numbers, the $B(E3)$ is typically split over
many low-lying states and their total contribution to the rms radii may be
larger than that shown in Fig. (4). The largest $\Delta R_{ch}$ are observed
for 40Ca and 96Zr. The implications of the increase in $\Delta R_{ch}$(E3) for
96Zr and nuclei just after the magic numbers will be discussed at the end.

Results for the radii using the $\beta_{2}$ and $\beta_{4}$ obtained from the
BSk27 EDF calculations go13  are shown in Fig. (5). $\beta_{4}$ accounts for
about a third of the difference just after $N=82$ shown in the bottom panels
of Fig. (2) and (3).

##  III Isotonic Shifts for the Cadmium and Tin Isotopes

Modifications have been made to the Skyrme functionals to account for odd-even
oscillators in the rms charge radii and the increase in radii after the magic
numbers. The Fayans functionals add a pairing-type term to spherical EDF fa00
, sa11 . The M3Y-P6a functional adds density-dependent spin-orbit and pairing
terms to the M3Y-type functional na15 .

As an example for isotonic shift in the rms charge radii, we consider the
results for the cadmium and tin isotopes. These are shown in Fig. (6) using
the experimental data and calculations given in cd18  for cadmium and sn19
for tin. The experimental data are compared to the spherical calculations with
the Sv-min kl09  EDF calculations given in the experimental papers cd18 , sn19
. All 12 Skyrme functionals given in br14  give isotonic shifts similar to
those shown for Sv-min. The experimental isotonic shift is closer to zero than
to the standard Skyrme results.

In Fig. (6) The isotonic shifts are also compared to the results from the
Fy($\Delta$r,HFB) functional given in cd18 , sn19 , and to the results
obtained by adding the $\beta_{2}^{2}$ corrections using the experimental
$B(E2)$ values from pr17  to the Sv-min values. The error bars shown in Fig.
(6) are from those given in pr17  for the $B(E2)$ and from those given in cd18
and sn19  for the rms radii. The Sv-min plus $\beta_{2}^{2}$ corrections
provide a reasonbaly good description of the data. The remaining differences
between theory and experiment may be due to: (a) systematic uncertainties in
the charge radii, (b) systematic uncertainties in the $B(E2)$ extracted from
Coulex and/or lifetime measurements, (c) deficiencies in the Sv-min EDF, or
(d) $\beta_{4}^{2}$ corrections.

##  IV Isotopic Shifts for the Calcium Isotopes

One of the most famous and challenging data for rms charge radii is that for
the calcium isotopes where there is a strong odd-even oscillation in the rms
charge radii with 42,44,46Ca being relatively large compared to those for the
”closed-shell” nuclei 40Ca and 48Ca. It is notable that the experimental rms
charge radii of 40Ca and 48Ca are nearly the same ga16 . This data has led to
many theoretical ideas ta84 , za10 , ho94 , za87 , ca01 , za85 , ba85 , fa00 ,
sa11 . It was noted by Talmi ta84  that a two-body effective operator for a
correction to rms radii contains odd-even oscillations. The increase in the
rms charge radii is correlated with an increase in the matter rms radii gi92 .
The increase in the rms radii for 42,44,46Ca relative to 40Ca and 48Ca are in
relatively good agreement with $\beta_{2}^{2}$ corrections obtained from the
experimental $B(E2)$ values as shown in panels (a) and (b) of Fig. (7). The
data are for 42Ca ca42 , 44Ca ca44 , 46Ca and 50Ca ca50 . For 42Ca we include
the the $B(E2)$ for to the 2${}^{+}_{2}$ state for which
$B(E2,0^{+}_{1}\rightarrow 2^{+}_{2})/B(E2,0^{+}_{1}\rightarrow
2^{+}_{1})\approx 0.1$ ca42 . The theoretical results from the
Fy($\Delta$r,HFB) calculations given in mi19  are shown in panel (c) of Fig.
(7).

Figure 5:  (a) Increase in the rms charge radii calculated with Eq. (1) using
the $\beta_{2}$ values obtained in a deformed EDF calculation with the HFB27
Skyrme functional. (b) Increase in the rms charge radii calculated with Eq.
(1) using the b4 values obtained in a deformed EDF calculations with the HFB27
Skyrme functional.  Figure 6: Cd-Sn isotonic shifts. The experimental results
from cd18 , and sn19  are compared to the Sv-min and Fr($\Delta$r) EDF
calculations given in cd18  and sn19 .

These large $B(E2)$ cannot be described by calculations in the $fp$ model
space lo21 . They are a result of admixtures from configurations with proton
excitations from the $sd$ shell to the $pf$ shell. The configuration
interaction (CI) calculations that include these cross shell excitation are
challenging. In ca01  the ZBM2 Hamiltonian for the
$(1s_{1/2},0d_{3/2},0f_{7/2},1p_{3/2})$ model space was used to calculate the
rms charge radii of the calcium isotopes using harmonic-oscllator radial
wavefunctions. The number of protons excited from $(1s_{1/2},0d_{3/2})$ to
$(0f_{7/2},1p_{3/2})$ showed an odd-even effect. When these orbital
occupations were used with harmonic-oscllator radial wavefunctions one
obtained an increase in the rms charge radii with odd-even oscillations that
were in qualitative agreement with experiment. However, in ro15  when the
orbital occupation numbers from these calculations were used to constrain the
spherical EDF calculations, the increase in the rms charge radii was small
compared to experiment.

To explore the $\beta_{2}^{2}$ contributions to the charge radii we will use
the ZBM2-modified Hamiltonian for the $(1s_{1/2},0d_{3/2},0f_{7/2},1p_{3/2})$
model space as described in bi14 . The corrections for the rms charge radii
using the $B(E2)$ values from these calculations are shown by the blue line in
panel (b) of Fig. (7). We use effective charges of $e_{p}=1.22$ and
$e_{n}=0.78$ from pi21 . There is a disagreement with the calculated and
experimental $B(E2)$ for 46Ca. This can be traced to the location of the
$2p-2h$ proton intruder state that comes at 1.8 MeV in the calculations.
Experimentally it is observed at 2.4 MeV ca46 . ZBM2-modified Hamiltonian was
designed for the region of 40Ca. When it is used for the region of 48Ca the
proton shell gap is too small. This can be fixed by adding a monopole term to
the Hamiltonian that moves the proton $2p-2h$ state in 46Ca up to 2.4 MeV. The
proton $2p-2h$ state in 48Ca is suggested to be at 4.28 MeV br98  compared to
the calculated excitation energy of 4.45 MeV. The results for the
$\beta_{2}^{2}$ correction are shown by the green line in panel (a) of Fig.
(7). EDF results for 49,50Ca are discussed on the end of this paper.

In addition to the results for even-even nuclei obtained with
$B(E\lambda,\uparrow)$, we include results for the odd-even nuclei obtained
from

$<i\mid Q\cdot Q\mid i>\,=\frac{1}{(2J_{i}+1)}\displaystyle\sum_{c\neq
i}<i||Q||c><c||Q||i>,$ $None$

where $Q$ is the $E2$ operator. For even-even nuclei this is $B(E2,\uparrow)$.
The sum for odd-even nuclei is typically over many excited states. For
example, in 43Ca, there are 11 excited states below 3.5 MeV that contribute.

The odd-even effects in the orbital occupations and $\beta_{2}^{2}$ values are
implicitly connected to the pairing interaction. But the specific connections
between the pairing oscillations in the nuclear binding energies and the rms
charge radii depends upon details of the local nuclear structure. There are
many contributions to consider for rms radii: the properties of the EDF
functional, orbital occupations, $\beta_{2}^{2}$ corrections, and possible
pairing-type additions to the EDF. Our results suggest that the spherical EDF
together with the $\beta_{2}^{2}$ corrections are the most important. But at a
quantitative level of detail, all must be considered.

Figure 7: $\Delta R_{ch}$ for the calcium isotopes. The black circles are the
experimental results from ga16  with error bars about the size of the circles.
The calculations shown in panel (c) are the results from the Fy($\Delta$r,HFB)
calculations given in mi19 . The lines shown in panels (a) and (b) are based
on the $B(E2)$ calculations discussed in the text. The red crosses in panels
(a) and (b) are based on experimental $B(E2)$ values in the text.

##  V Isotonic Shifts for the $N=28$ Isotones

The results for the $N=28$ isotones obtained with the GXPF1A Hamiltonian for
$N=28$ are shown by the green line in Fig. (8). We use effective charges of
$e_{p}=1.22$ and $e_{n}=0.78$ from pi21  and the $\beta_{2}^{2}$ model of pi21
that includes the isovector term. One observes odd-even oscillations in the
calculated results. The $\beta_{2}^{2}$ contribution only includes the
2${}^{+}_{1}$ state. These are the calculations that were used to make the
$\beta_{2}^{2}$ corrections for the connecting the mirror charge radii of 54Ni
and 54Fe to the slope parameter $L$ of neutron equation of state in pi21 . As
noted in pi21 , for 54Fe there is additional $E2$ strength at 2.8 MeV in the
$fp$ calculation. This is not included because it comes from coupling of two-
proton holes in 56Ni to the 2+ particle-hole excited state in 56Ni at 2.7 MeV.

Charge radii calculated by the spherical EDF for $N=28$ are linear as a
function $Z$. Thus, to obtain the $\beta^{2}$ correction from experiment we
can take the difference between the rms charge radius for each isotope and
that obtained from a linear extrapolation between the experimental values for
48Ca ga16  and 56Ni so22 . These results are shown by the black points in Fig.
(8). More complete and improved experimental results are needed for the odd-
even isotopes.

Figure 8: $\Delta R_{ch}$ obtained from $\beta_{2}^{2}$ corrections for the
isotones with $N=28$ isotopes. The red crosses are based on experimental
$B(E2,0^{+}_{1}\rightarrow 2^{+}_{1})$ values. The calculations shown by the
green lines are discussed in the text. The experimental data differences
between the given isotope and a a linear extrapolation between 48Ca ga16  and
56Ni so22 . The data for Z = 22, 23, 24 and 26 are from fr04  using combined
analyses of the muonic-X ray and electron scattering measurements, where they
are available. Otherwise only the electron scattering data is used. The charge
radius for Z = 25 is from an13 . Charge radii for Z = 21 and 27 are not
experimentally known yet.  Figure 9: Nuclear matter densities obtained with
the $s17$ EDF parameters.

The correlation between the difference in mirror charge radii and the symmetry
energy parameter $L$ is obtained with spherical EDF br17  and CODF ya18
calculations that are linear in between 48Ca and 56Ni. In addition, as
discussed in pi21 , one has to add the $\beta_{2}^{2}$ corrections for 54Fe as
shown in Fig. (8) and for 54Ni. The oscillations in the rms charge radii are
treated in a different way in the Fayans-type functional by adding pairing-
type functional with new global parameters cd18 , sn19 , re17 . In the Fayans
method some of the correlation between the mirror charge radii and $L$ is lost
re22 .

##  VI Increase in Charge Radii After the Magic Numbers

Spherical EDF with $\beta_{2}^{2}$ corrections do not account for the increase
in the rms charge radii just after the magic numbers shown in the bottom of
Figs. (2), (3) and (7). All of these increases are associated with the sudden
occupation of the orbitals with an addition node given in Table I. The
experimental difference in the rms charge radii between 49Ca and 48Ca is given
in Table II and compared with results of some calculations. The result for the
covariant density functional (CODF) model are taken as one quarter of the
increase between 52Ca and 48Ca shown in Fig. 24 of pe21 . Nuclear matter
densities for 40Ca, 48Ca and 52Ca obtained with the $s17$ EDF parameter set
are shown in Fig. (9). The interior densities show a large variation. In
particular, the matter density for 52Ca has a larger interior density due to
the node in the radial wavefunctions. The ”size” of the nucleus is determined
by the interior saturation conditions imposed by the EDF functional. Perhaps
the present generation of EDF functionals are not complex enough to correctly
take into account these variations in the interior shape.

It is noted in na84  that the orbitals given in Table I are all associated
with a sudden change in the octupole instabilities via their large $B(E3)$
values with higher orbitals that differ by $\Delta j=3$, $0g_{9/2}$,
$0h_{11/2}$, $0i_{13/2}$, and $0j_{15/2}$, respectively (see Fig. 2 in na84 ).
This results in a sudden increase in the $B(E3)$ just after the magic numbers.
This may contribute to the increase in the $\Delta R_{ch}$ observed in the
bottom of Figs. (2) and (3). The $E3$ contribution from the $(0p_{3/2})^{n}$
configurations for 48+nCa would increase linearly with $n$. The $E3$
contribution from the $(0d_{5/2})^{n}$ configurations for 90+nZr would be
maximized in 96Zr where a peak in the $E3$ contribution to $\Delta R_{ch}$ is
observed in Fig. (4). The octupole contributions to the rms radii in an EDF
model that includes octupole degrees of freedom na84 , ca20  needs to be
explored.

Table 1: Experimental increase in the rms charge radii after the magic numbers. $N$ | orbital | Nuclei | $\Delta R_{ch}$
---|---|---|---
|  |  | (fm)
28 | 1p3/2 |  49-48Ca | 0.015(3)
50 | 1d5/2 |  91-90Zr | 0.015(2)
82 | 1f7/2 |  133-132Sn |
132 | 1g9/2 |  209-208Pb | 0.009(2)
Table 2: Charge rms difference between 49Ca and 48Ca in units of (fm) $\Delta R_{ch}$(exp) | Ref. | 0.015(3)
---|---|---
Sv-min | mi19  | 0.008
$s8$ | br14  | 0.009
$s17$ | br14  | 0.010
CODF | pe21  | 0.011
M3Y-P6a | na15  | 0.011
Fy($\Delta$r,BCS) | mi19  | 0.004
Fy($\Delta$r,HFB) | mi19  | 0.014

##  VII Conclusions

In this paper we considered the $\beta_{\lambda}^{2}$ corrections to rms radii
provided by the Bohr model. We showed results for $\lambda$=2, 3 and 4. The
most important $\beta_{2}^{2}$ contribution can be obtained from experimental
$B(E2)$ value. When these $\beta_{2}^{2}$ corrections are added to spherical
EDF calculations, the rms charge radii for even-even nuclei are in reasonable
agreement with experiment. The major deviations between theory and experiment
appear just after the magic numbers. We also used the experimental $B(E2)$ to
calculate the isotonic shift between the cadmium and tin isotopes, the
isotopic shifts of the calcium isotopes and the isotonic shifts of the $N=28$
isotones. The results are all in reasonable agreement with the data.

For the calcium isotopes and the $N=28$ isotones, we used configuration-mixing
calculations to obtain the $<Q\cdot Q>$ corrections for both even-even and
odd-even nuclei. The calculated $B(E2)$ for even-even nuclei are in reasonable
agreement with experiment. The evaluation of $<Q\cdot Q>$ for the odd-even
nuclei leads to odd-even oscillations in the rms charge radii that are also in
reasonably good agreement with the data.

For the $\beta_{2}^{2}$ correction model, the primary remaining deviation
between experimental and theoretical rms charge radii is for the rapid
increase just after the magic numbers. We suggest that this may come form
octupole deformations and/or a limitation in the standard form of the Skyrme-
type EDF functionals.

This work was supported by NSF grant PHY-2110365 and NSF PHY 2111185. We thank
H. Nakada and W. Nazarwicz for discussions and suggestions.

## References

  * (1) M. Kortelainen, Z. Sun, G. Hagen, W.Nazarewicz, T. Papenbrock, and P.-G. Reinhard, Phys. Rev. C 105, L021303 (2022).
  * (2) U. C. Perera, A. V. Afanasjev, and P. Ring, Phys. Rev. C 104, 064313 (2021).
  * (3) P.-G. Reinhard and W. Nazarewicz, Phys. Rev. C 95, 064328 (2017).
  * (4) B. A. Brown and A. Schwenk, Phys. Rev. C 89, 011307(R) (2014).
  * (5) D. M. Rossi et al., Phys. Rev. C 92, 014305 (2015).
  * (6) A. Volya, B. A. Brown and V. Zelevinsky, Phys. Lett. B 509, 37 (2001).
  * (7) B. A. Brown, C. Bronk and P. E. Hodgson, Jour. Phys. G 10, 1683 (1984).
  * (8) J.-P. Delaroche, M. Girod, J. Libert, H. Goutt e, S. Hilaire, S. Peru, N. Pillet, and G. F. Bertsch, Phys. Rev. C 81, 014303 (2010).
  * (9) I. Angeli and K. P. Marinova, Atomic Data and Nucl. Data Tables 99, 69 (2013).
  * (10) G. Fricke and K. Heilig, Nuclear Charge Radii (Spr inger, Berlin Heidelberg, 2004).
  * (11) R. F. Garcia Ruiz, Nature Physics 12, 594 (2016).
  * (12) K. Minamisono et al., Phys. Rev. Lett. 117, 252501 (2016).
  * (13) F. Sommer et al., to be published.
  * (14) M. Hammen et al., Phys. Lev. Lett. 121, 102501 (2018).
  * (15) C. Gorges et al., Phys. Rev. Lett. 122, 192502 (2019).
  * (16) A. Bohr and B. R. Mottelson, Nuclear Structure vol. I, (New York: Benjamin. 1969).
  * (17) H. Esbensen and G. F. Bertsch, Phys. Rev. C 28, 355 (1983).
  * (18) S. V. Pineda et al., Phys. Rev. Lett. 127, 182503 (2021).
  * (19) J. Dobaczewski, P. Vogel and A. Winter, Phys. Rev. C 29, 1540 (1984).
  * (20) S. A. Ahmad, W. Klempt, R. Neugart, E. W. Otten, P.-G. Reinhard, G. Ulm, and K. Wendt, Nucl. Phys. A 483, 244 (1988).
  * (21) G. F. Bertsch, Eur. Phys. J. A 55, 248 (2019).
  * (22) B. A. Brown, Phys. Rev. C 58, 220 (1998).
  * (23) B. A. Brown, Phys. Rev. Lett. 111, 232502 (2013).
  * (24) P. Kluepfel, P.-G. Reinhard, T. J. Buervenich, and J. A. Maruhn Phys. Rev. C 79, 034310 (2009).
  * (25) S. Fayans, S. Tolokonnikov, E. Trykov, and D. Zawischa, Nucl. Phys. A 676, 49 (2000).
  * (26) E. E. Saperstein and S. V. Tolokonnikov, Phys. At. Nucl. 74, 1277 (2011).
  * (27) B. Pritychenko, M. Birch, B. Singh, and M. Horoi, Atomic Data and Nucl. Data Sheets 107, 1 (2016).
  * (28) J. W. Lightbody, S. Penner, and S. P. Fivozinsky, P. L. Hallowell and H. Cranne, Phys. Rev. C 14, 952 (1976).
  * (29) R. H. Spear and W. N. Catford, Phys. Rev. C 41, R1351 (1990).
  * (30) S. Goriely, N. Chamel and J. M. Pearson, Phys. Rev. C 88, 061302(R) (2013).
  * (31) H. Nakada, Phys. Rev. C 92, 044307 (2015), and privite communication.
  * (32) B. Pritychenko, M. Birch, and B.Singh, Nucl. Phys. A 962, 73 (2017)
  * (33) I. Talmi, Nucl. Phys. A 423, 189 (1984).
  * (34) L. Zamick, Phys. Rev. C 82, 057304 (2010).
  * (35) M. Horoi, Phys. Rev. C 50, 2384 (1994).
  * (36) L. Zamick, Zeit. fur Phys. A 327, 409 (1987).
  * (37) E. Caurier, K. Langanke, G. Martinez-Pinedo, F. Nowacki and P. Vogel, Phys. .Lett. B 522, 240 (2001).
  * (38) D. Zawischa, Phys. Legg. B 155, 309 (1985).
  * (39) F. Barranco and R. A. Broglia, Phye. Lett. B 15 1, 90 (1985).
  * (40) W. R. Gibbs and J. P. Dedonder, Phys. Rev. C 46, 1825 (1992). pion scattering and matter radii for Ca
  * (41) K. Hadynska-Klek, Phys. Rev. C 97, 024326 (2018).
  * (42) C. E. Towsley, D. Cline and R. N. Horoshko, Nucl. Phys. A 204, 574 (1973).
  * (43) J. J. Valiente-Dobon et al., Phys. Rev. Lett. 102, 242502 (2009).
  * (44) A. J. Miller et al., Nat. Phys. 15, 432 (2019).
  * (45) B. Longfellow, D. Weisshaar, A. Gade, B. A. Bro wn, D. Bazin, K. W. Brown, B. Elman, J. Pereira, D. Rhodes, and M. Spieker, Phys. Rev. C 103, 054309 (2021).
  * (46) M. L. Bissell et al., Phys. Lev. Lett. 113, 052502 (2014).
  * (47) W. Kutschera, B. A. Brown, H. Ikezoe, G. D. Sprouse, Y. Yamazaki, Y. Yoshida, T. Nomura and H. Ohnuma, Phys. Rev. C 12, 813 (1975).
  * (48) B. A. Brown and W. A. Richter, Phys. Rev. C 58, 2099 (1998).
  * (49) B. A. Brown, Phys. Rev. Lett. 119, 122502 (2017).
  * (50) J. Yang and J. Piekarewicz, Phys. Rev. C 97, 014314 (2018).
  * (51) P.-G. Reinhard and W. Nazarewicz, Phys. Rev. C 105, L021301 (2022).
  * (52) W. Nazarewicz, P. Oleadners, I. Ragnarsson, J. Dudkek, G. A. Leander, P. Moeller, R. Ruchowka, Nucl. Phys. A 429, 269 (1984).
  * (53) Y. Cao, S. E. Agbemava, A. V. Afanasjev, W. Nazarewicz, and E. Olsen, Phys. Rev. C 102, 024311 (2020).


# Recommendations for Systematic Research on Emergent Language

Brendon Boldt  David Mortensen

###### Abstract

Emergent language is unique among fields within the discipline of machine
learning for its open-endedness, not obviously presenting well-defined
problems to be solved. As a result, the current research in the field has
largely been exploratory: focusing on establishing new problems, techniques,
and phenomena. Yet after these problems have been established, subsequent
progress requires research which can measurably demonstrate how it improves on
prior approaches. This type of research is what we call _systematic research_
; in this paper, we illustrate this mode of research specifically for emergent
language. We first identify the overarching goals of emergent language
research, categorizing them as either science or engineering. Using this
distinction, we present core methodological elements of science and
engineering, analyze their role in current emergent language research, and
recommend how to apply these elements.

Machine Learning, ICML

##  1 Introduction

Emergent language research is characterized by the use of the neural networks
and reinforcement learning to simulate the evolution of “natural” language
from scratch (see Lazaridou & Baroni (2020) for a comprehensive survey of
emergent language research). Both the range of potential applications of
emergent language and the technique itself differ significantly from pre-
existing methods in reinforcement learning and natural language processing.
Such novelty means that a large proportion of the work on the topic is
exploratory research: focusing on establishing a baseline level of knowledge
of new problems, techniques, and phenomena in light of the unique challenges
of the field.

Yet after this foundation has been laid, subsequent long-term progress
requires research which measurably demonstrates how it improves on prior
approaches rather than tackling a new problem entirely. This type of research
is what we call _systematic research_. This systematicity is necessary for
research to be effectively applied to the real problems and questions which
motivate it in the first place. The goal of this paper is to provide concrete
recommendations for moving work on emergent language toward systematic
research.

In order to develop these recommendations, we start in Section 2 by clarifying
the goals of emergent language research; drawing attention to how they fit
into the categories of science and engineering. In Section 3 we illustrate the
prototypical methods of science and engineering research which facilitate
systematic research. Finally, in Section 4 we analyze to what extent current
emergent language research uses these methods and recommend how they could be
more fully employed. The specific contributions of this work are:

  1. 1.

Recommending methods which move emergent language research toward systematic
research, working towards overarching goals via measurable progress.

  2. 2.

Distinguishing between the capacities in which emergent language research can
be science or engineering via the goals of a given project.

###  1.1 Exploratory and Systematic Research

Exploratory research is research which focuses on innovative approaches and
problems rather than pursuing a well-established goal. Exploratory research
can take place either in a new field, such as emergent language, or on the
frontier of a more established research area. The primary purpose of
exploratory research is to build up a baseline level of knowledge of new
approaches which differ significantly from well-studied approaches. Once a
critical mass of knowledge is acquired, a more structured and systematic
research program is pursued in order to advance the knowledge and expertise of
the field on that topic.

Systematic research is characterized by research contributions which
measurably demonstrates how they improve on prior approaches; this entails
adhering to established research problems. The long-term success of any given
research field is, in fact, predicated on systematic research; the direct
comparison between two approaches to the same problem is what allows for
significant progress to be made in an incremental way. At times, “incremental”
is considered a negative attribute, but this is far from the case in our own
usage as the incremental nature of science and engineering are integral to
their success as disciplines. In the end, both exploratory and systematic
research have a time and place where they are appropriate.

The ideas of exploratory and systematic research parallel the concepts of
“pre-paradigmatic science” and “normal science” from Kuhn (1962). Our account,
though, is intended to apply to science _and engineering_ and is targeted
towards practitioners of these fields rather philosophers of science.

##  2 Goals of Emergent Language Research

The goals of a pursuit dictate the methods employed to achieve it.
Accordingly, we begin by enumerating the overarching goals of emergent
language research. Furthermore, these goals will also make the discussion of
methods more concrete and easier to apply.

For our later discussion on methodology, it will prove fruitful to divide the
goals of emergent language research into the categories of science and
engineering.111The inspiration for this science-engineering distinction comes
from the brief mention it was given in the abstract of Lazaridou & Baroni
(2020, p. 1). In the following subsections we enumerate first the scientific
goals of emergent language research followed by the engineering goals. This
list is intended to be comprehensive breadth-wise although, on account of
space, we are only able to offer a brief description of each goal. For each
goal, we present a handful of papers which have _mentioned_ the corresponding
goal as a motivation for emergent language research. Since most of the papers
address relatively foundational topics, it is not as informative to look at
what they directly address.

###  2.1 Scientific Goals in Emergent Language

Prototypically speaking, science is a descriptive pursuit seeking explanation
and understanding of phenomena in the natural world (Bunge, 1966, p. 330).
Such explanations are to be found in scientific _theories_ built through
empirical investigation (Winther, 2021, abstract). The scientific goals of
emergent language, then, center on its unique ability to support precise,
controllable experiments on the formation, evolution, and use of language.
Since it is almost always impossible to run in situ experiments with these
phenomena, current investigation relies on observing historical data, making
limited observations of real people, and small-scale laboratory studies. The
techniques of emergent language, then, are the sole hope to build theories of
these aspects of human language with the typical methods of experimental
science.

##### Evolution of Language

Better understanding the origin and evolution of language is a straightforward
line of scientific inquiry for emergent language because simulating these
processes is, in fact, the distinguishing technique of this field. The initial
steps toward this goal are characterizing the rules governing the evolution of
language abstractly from controllable emergent language experiments.
Subsequently the experiments can be made more realistic and adapted more
directly to the evolution of human language. Mentioned in Lazaridou & Baroni
(2020, p. 1); Lazaridou et al. (2018, p. 1); Mordatch & Abbeel (2018, p. 1);
Havrylov & Titov (2017, p. 1).

##### Building Blocks of Language

The foundation of the structure and meaning of language include the linguistic
disciplines of phonology, morphology, syntax, and semantics. Emergent language
can potentially serve as a testbed for disentangling the various aspects of
language; something difficult to do via more traditional means like corpus-
based or small-scale human studies. Furthermore, if emergent language agents
become fully competent language-users, it will an allow an unmatched degree of
observability and controllability for the mechanics of language usage.
Mentioned in Lazaridou et al. (2018, p. 1); Bouchacourt & Baroni (2018, p. 1).

##### Interactive Language

Linguistic disciplines such as pragmatics and sociolinguistics focus on the
external factors of language surrounding its use among multiple language
users. The interactive aspects of language are even more difficult to capture
by static means (e.g., in corpora) than the aforementioned aspects, so the
inherently interactive experiments within emergent language serve as an
important tool for investigation here. Language acquisition research, in
particular, serves to greatly benefit from emergent language techniques due to
the ability to both precisely observe pre-linguistic properties of language
agents as well as easily test counterfactual hypotheses. Mentioned in Harding
Graesser et al. (2019, p. 1); Eccles et al. (2019, p. 2).

###  2.2 Engineering Goals in Emergent Language

Engineering, in contrast to science, is a prescriptive pursuit as it
recommends the best solutions to a problem rather than describing things as
they are (Bunge, 1966, p. 330). Specifically, engineering addresses problems
which are well-defined insofar as it is possible to evaluate and compare
competing solutions. The engineering goals of emergent language research
center around creating a language agent which is competent with an emergent
language of comparable sophistication, complexity, or expressive power to
human languages. The end result is effectively having a computerized version
of a human’s language faculty which can be scaled, tweaked, and employed as
needed for relevant machine learning applications.

##### Replication of Natural Language

The core goal of emergent language research as engineering is to simulate the
emergence of a language as complex and expressive as human language. Most
other scientific and engineering pursuits within emergent language research
are predicated on making significant progress toward this goal. The initial
steps towards this goal subsume much of the current work which studies human
language-like properties, like compositionality, in emergent languages.
Mentioned in Evtimova et al. (2018, p. 1); other papers discuss imitating
aspects of human communication without explicitly mentioning the full
replication of human language.

##### Alternative Data Paradigm

The language agents developed through emergent language could serve as
alternative paradigm for generating data for many other areas of natural
language processing. Since the language-competent agents are easily scalable
and controllable (compared to humans), they could generate sophisticated
synthetic data for the (pre)training and evaluation of machine learning
models. Li et al. (2020) exemplify this by using synthetic data generated from
an emergent language to improve the performance of a few-shot machine
translation model. Mentioned in Lazaridou et al. (2017, p. 2); Li et al.
(2020).

Embodied NLP is an area of NLP that would reap an outsized benefit from
emergent language as an alternative data source. This is primarily because of
the relative paucity of multi-modal—let alone interactive—data that the
techniques of embodied NLP require. The necessity of such multi-modal and
interactive data for the ultimate goals of NLP are discussed in Bisk et al.
(2020, p. 8722).

##### Communicating with Humans

Simulating interactive and embodied language use facilitates improvements to
the direct interaction between humans and artificial language agents. Even
outside of direct interaction, emergent language could augment the
explainability of machine learning models. Mentioned in Lazaridou & Baroni
(2020, p. 3); Chaabouni et al. (2019, p. 1); Lowe et al. (2020, p. 1);
Lazaridou et al. (2017, p. 1); Havrylov & Titov (2017, p. 1); Kottur et al.
(2017, p. 1).

##### Robust Multi-Agent Communication

Apart from communicating directly with humans, there is a desire to imitate
the generality and robustness which humans’ use of language exhibits. While
emergent language is not the only path to this goal, its techniques
potentially provide a more “organic” or ground-up way of developing such
communication protocols rather than relying on hand-crafted methods of
communication. Mentioned in Lazaridou & Baroni (2020, p. 3); Evtimova et al.
(2018, p. 1); Chaabouni et al. (2020, p. 1); Lazaridou et al. (2017, p. 2);
Havrylov & Titov (2017, p. 1); Kottur et al. (2017, p. 1).

##### Improving Emergent Language Techniques

Finally, an essential part of achieving all other goals of emergent language
research is simply improving its own techniques. This could come in a variety
of forms including finding better ways to measure and analyze emergent
languages, making certain properties more robust in emergent language
experiments (e.g., compositionality), or comparing multiple approaches against
each other. Mentioned in Lazaridou & Baroni (2020, p. 3); Chaabouni et al.
(2019, p. 1); Chaabouni et al. (2020, p. 1); Kharitonov & Baroni (2020, p. 1);
Cao et al. (2018, p. 1); Eccles et al. (2019, p. 1); Resnick et al. (2020, p.
1); Mordatch & Abbeel (2018, p. 1).

###  2.3 Beyond Science and Engineering

It is important to note that there are highly relevant _philosophical_
question surrounding emergent language, but since these fall outside the
typical purview of the machine learning and computational linguistics
communities, they are not incorporated into the analysis of this paper. Such
issues include providing a more rigorous and concrete definition for what
experimental factors make an emergent language truly _emergent_ (as opposed to
constrained or pre-planned) or how to quantify the expressivity or complexity
of an emergent language vis-à-vis a human language. As the scientific and
engineering aspects of the field progress it will become clearer to what
degree philosophically rigorous accounts are necessary, beyond pragmatic
informal accounts determined by consensus.

##  3 Methodological Elements

With the distinction between the scientific and engineering goals of emergent
language in mind, we now address the _methodological elements_ of science and
engineering. These elements describe the concepts that are central to
performing systematic research in the given discipline because they allow for
research to measurably demonstrate how it improves over prior work. They will
serve as the structure of our analysis of current and future methods in
emergent language research. These elements are not intended to be necessary
and/or sufficient conditions for science and engineering but rather
prototypical elements which serve as a template.

###  3.1 Elements of Science

_Theory_ and _theory-building_ are the most pertinent elements of scientific
research because they are both the means for achieving the explanatory goals
as well as the measure of progress. Thus, we present a collection of
scientific elements which support theory-building, closely following
definitions given in Giere (1984). This account of science seeks to avoid
naïve, unphilosophical accounts as well as philosophically nuanced but
unwieldy accounts of science. Accordingly, we choose not to introduce science
as a well-defined, five-step method because this is an oversimplification
which fails to represent science as it is actually practiced (Hepburn &
Andersen, 2021, Sec. 6.1).

Two preliminary terms that we will use throughout this discussion are _system_
and _object of study_. We use system as a very general term denoting a
collection of things which relate to each other in some reasonable way.
Systems range from natural systems like the solar system or an ecosystem to
artificial systems like a car engine or an economy to conceptual systems like
Euclidean geometry. Secondly, when we talk about an object of study, we are
referring to the particular system which a given line of inquiry seeks to
understand, which may be different from the system which is the means of
study.

##### Theoretical Model

A _theoretical model_ (or _theoretical framework_) is a conceptual system
which is defined by a set of precise set of axioms or stipulations (Giere,
1984, p. 80). For example, Newton’s three laws of motion define a system of
moving and interacting physical bodies, and Mendel’s theoretical model of
inheritance defines a system of trait inheritance in sexually reproducing
organisms. These models have a tractable222“Tractable” in the sense of being
able to be understood by a human as opposed to some programmatically generated
set of thousands or millions of axioms, suitable only for automated
interpretation. number of stipulations which can be used both for explanation
as well as for predicting the behavior of the system which they describe.
Although theoretical models need not be formal in the strictest sense, there
should be a precision about them that yields unambiguous conclusions given the
axioms.

The core desideratum of a model is its accuracy: that its predictions are, in
fact, borne out by experiments. Nevertheless, there are a number of other
characteristics which are important to models. For example, the scope,
simplicity, and even (controversially) beauty of a theoretical model can
factor into how scientists evaluate it.

##### Computational Model

_Computational models_ are similar to theoretical models insofar as both are
systems _defined_ by a given set of stipulations.333As opposed something like
a scale model which is not defined solely by a set of axioms Rather than
providing a general-purpose description which can be used to reason and make
predictions about a system, computational models define a procedure for
simulating particular instances of some system. Computational models, then,
are often derived from theoretical models as a way of understanding the
complex instances of the system which could not otherwise be reasoned about
mentally or by hand.

##### Hypothesis and Experiment

A _hypothesis_ is a statement about a system which can be either true or false
(Giere, 1984, p. 81). A hypothesis is always derived from a model of some
sort. This is straightforward in the case of a theoretical model which
predicated on being able to develop predictions and hypotheses which can then
be tested, but even when no explicit model is present, a hypothesis comes from
some sort of implicit or intuitive model from which one makes an educated
guess.

A good (not necessarily correct) hypothesis must be empirically testable, that
is, one can formulate an _experiment_ which verifies or refutes the hypothesis
by observing causes and effects in the actual system in question. It is
important to note that not all _research questions_ qualify as hypotheses;
this does not make them inadequate _as questions_ but they cannot fulfill the
role that hypotheses do in scientific research and theory-building in
particular.

##### Theory-Building

A _theory_ is a general hypothesis which states that the studied system is
exactly or approximately described by a theoretical model (Giere, 1984, p.
84). Since such a general hypothesis cannot be tested directly, we resort to
using the theoretical model to generate smaller, more tractable hypotheses
which can be directly tested. These hypotheses are empirically tested; when a
hypothesis is refuted, the model must be revised in order to account for this
discrepancy with the system it claims to describe. The modified model, then,
is used to generate further hypotheses, forming an iterative process which is
constitutive of _theory-building_. The measurable progress is scientific
research is marked by theoretical models becoming more accurate, accounting
for more phenomena, and better understood through this iterative process.

It is important to note that _hypotheses which are not generated from
theoretical models do not contribute to the process of building a theory_.
This is because when such a hypothesis is confirmed or rejected, that result
does not, then, have anything to reflect back on. Thus, since theory-building
is critical to scientific research, it is necessary for hypotheses to be
derived from theoretical models.

###  3.2 Elements of Engineering

Engineering research is directed towards solving well-defined problems.
Accordingly, one of the key elements of such research is the determining the
quality of a solution to a problem. The measurability of progress in
engineering research is best illustrated by _benchmarks_ since they are the
culmination of its other methodological elements. The elements of engineering
given here are selected and illustrated with computer vision serving as a
prototype.

##### Problem Definition

In engineering, it is integral to have a clear statement of the task to be
accomplished as problem-solving is precisely the aim of the discipline; this
we term the _problem definition_. A problem definition should prescribe the
inputs and desired outputs of a solution and any constraints to be put on the
solution (e.g., a fully self-driving car cannot require human input).

While a good problem definition is as precise as possible, many problems
include aspects which are inherently difficult to define precisely. For
example, we could define the problem of image classification as, “given an
image $x$ depicting an instance of class $k$, the model should recover class
$k$ solely from observing $x$.” The definition of “$x$ depicting $k$” is
inherently fuzzy and not formalizable. Does a low-light, blurry image of a
cheetah truly _depict_ a cheetah? We argue that there is no precise answer to
this question, yet the definition of image classification is still clear
enough to serve as a productive problem definition.

##### Evaluation Metric

A _metric_ is a quantitative measure of some property of a system. An
_evaluation metric_ is a metric which has a value judgment attached to it;
that is, a metric which ought to be maximized (or minimized). This valuation
is rooted in the fact that it measures how well a proposed system solves the
problem in question. This is not to say that evaluation metrics always assign
a better a score to a better solution to the problem, but insofar as the
metric fails to do this, it is an imperfect evaluation metric. Within computer
vision, evaluation metrics can range from the straightforward, like accuracy
for classification, to the highly engineered, like Frèchet inception distance
for GAN image synthesis; in either case, the quality of a given solution to
classification or synthesis is assessed through the use of these and other
such evaluation metrics. Without an evaluation metric, it is only possibly to
qualitatively compare the goodness of competing solutions which is inherently
less rigorous and repeatable than a quantitative assessment.

Metrics more generally do not necessarily have a value judgment attached to
them. For example, the slope of a linear regression is a metric but has no
value in and of itself—it would not make sense to say the slope of one model
is “better” than another without further context.

##### Standardization

By _standardization_ , we mean the consistent use of a set of common problems,
datasets, methods, evaluation metrics, etc. within a field. Within machine
learning, standardization often occurs through consensus of the research
community rather than via formal organizations.

Evaluating and comparing two different approaches to the same problem becomes
difficult if not impossible unless most of the variables between the two are
held constant. In an extreme example, it would futile to attempt to compare
two image classification architectures if one were trained on ImageNet and the
other on CIFAR-10. Nevertheless, even small differences in problem
formulations can become confounding factors when comparing different
approaches, necessitating some degree of standardization.

Although not integral to making comparisons between solutions, the
standardization of tooling is also of great importance to engineering. For
example, within machine learning, the widespread use of neural network
toolkits like TensorFlow or PyTorch make the development and evaluation of
novel algorithms far easier as components and expertise are applicable across
tasks and research groups. Thus, standardized tooling pragmatically
facilitates the rapid and wide diffusion of new techniques, greatly improving
the quality and speed of engineering research.

##### Benchmark

_Benchmarks_ are the result of tying together the standardization of a number
of different aspects of a given engineering problem. For machine learning,
this typically includes the task formulation (e.g., model inputs and outputs),
training and testing datasets, and evaluation metric. Conversely benchmarks
also _encourage_ standardization insofar as they provide pre-defined problem
which is established as being relevant often with the attendant resources
(e.g., dataset) needed to tackle it. For example, the ImageNet (Russakovsky et
al., 2015) benchmark standardizes a dataset, problem formulation (image
classification), and evaluation metric (top-$k$ accuracy) allowing for the
direct comparison of image classifiers.

Due to the standardization of many facets of a given problem, benchmarks are a
straightforward method of demonstrating how one approach measurably improves
on a previous approach. A measurable comparison may also be made without a
formal benchmark, but the necessity of holding constant as many aspects of the
problem as possible still applies.

##  4 Elements of Emergent Language Research

In this section, we will identify the role the aforementioned methodological
elements play in emergent language research (according to its category) in the
current body of research. Building on this analysis of primarily exploratory
research, we give recommendations on how to work towards more systematic
research on emergent language.

###  4.1 Science

####  4.1.1 Theoretical Model

Theoretical models are employed only occasionally in current emergent language
papers, although Kottur et al. (2017); Resnick et al. (2020); Kharitonov et
al. (2020) do employ theoretical models to some degree. As an example we will
look at the theoretical model presented by Resnick et al. (2020) which
addresses relationship between model capacity and compositionality of emergent
languages.

Specifically, an autoencoder with a discrete bottleneck layer is trained on a
(finite) language $L$ generated by context-free grammar $G$. The
compositionality of the model is measured at the encoding yielded by the
discrete bottleneck. We could present a simplified444The original theoretical
model does provide enough detail to apply directly to experiments. version of
the model as,

$\begin{split}|X|&\in[|G|,|L|)\wedge|Y|\not\in[|G|,|L|)\\\ &\Rightarrow
C(X)>C(Y),\end{split}$ (1)

where autoencoders $X$ and $Y$ have capacities $|X|$ and $|Y|$ (roughly
parameter count), $C(\cdot)$ measures a trained model’s bottleneck’s
compositionality, $|G|$ is the underfitting threshold, $|L|$ is the
overfitting threshold, and $[|G|,|L|)$ is an interval. If the autoencoder
model’s capacity is below $|G|$ (i.e., the sum of the terminal and non-
terminal symbols and the production rules of $G$), the model will not be able
to encode the language and underfit. If the model’s capacity is above $|L|$
(i.e., number of strings in the language), the model can simply memorize the
strings (overfit) which will lead to a non-compositional protocol. Thus, any
model between the under- and over-fitting regimes achieves optimal performance
by learning the underlying rules of the grammar, resulting in a higher
compositionality than a model in one of those regimes.

The primary benefit to such a theoretical model is generating clear, testable
hypotheses which is the basis for building a theory. For example, the above
model yields an easily testable hypothesis such as: for autoencoder $X$ with
capacity $|X|=|G|+1$ and $Y$ with capacity $|Y|=|G|-1$, $C(X)>C(Y)$ (assuming
$|G|+1<|L|$).

Looking forward in scientific emergent language research, the first step is to
incorporate theoretical models more consistently in the research process.
Although not all claims can be neatly reduced to a handful of mathematical
axioms, explicitly stating the model with as much specificity as possible is
integral to generating informative hypotheses. The second step is expanding
the range of applicability or scope of theoretical models. The model presented
above is relatively limited with respect to the range of environments and
agent architectures it can describe. This reflects one of the greatest
challenges for theoretical model-building in emergent language: the range of
environments whence language can emerge is diverse, resisting simple
formalization. There are further ways in which to develop theoretical models,
but addressing these two steps is sufficient for advancing beyond exploratory
research on this front.

####  4.1.2 Computational Model

As a science, emergent language research typically has one of two systems as
the object of study: either multi-agent communication in the abstract or human
language as a part of nature. It is important to be aware of this distinction
due to the ubiquity of computational models in machine learning-based emergent
language research.

When the studying multi-agent communication in the abstract, the computational
model directly instantiates some theoretical system which is the object of
study. That is to say, the theoretical models and hypotheses concern the
behavior of the computational model itself. Using computational models in this
way happens analogously in game theory, where we might search for Nash
equilibria in a game by running simulations or search algorithms with a
computer program. This use of computational models encompasses a majority of
current machine learning-based emergent language research.

If the ultimate object of study is human language itself, then a computational
model is used as an approximation of real system of human language. In this
case, the theoretical model and hypotheses are _actually_ about the human use
of language, and the computational model is only a proxy due to the inability
to run experiments in the actual system555In contrast, in sciences like
biology, chemistry, and physics, researchers can perform experiments directly
in the system being studied (that is, nature).. In this mode of research, the
computational models of emergent language have to imitate the particular
circumstances of human language use as closely as possible in order for the
computational model to serve as a good proxy, whereas this is not necessary if
communication is only being studied in the abstract.

####  4.1.3 Hypothesis and Experiment

Hypotheses, in one form or another, show up about roughly half of the time in
recent emergent language papers. Exploratory research often contains research
questions which are not hypotheses. For example, Evtimova et al. (2018, p. 7)
phrase their research question as “analyz[ing] the impact of changing the
message dimensionality, and the effect of applying visual and linguistic
attention mechanisms.” Although this is an informative research question, it
itself is not a prediction which can be shown to be true or false which is a
necessary trait of a hypothesis factoring into the process of developing a
theory.

When a hypothesis is present, it is typically derived from one of three
sources: (1) from intuitions, (2) by analogy from a related field, or (3) from
a theoretical model. To say that a hypothesis comes from intuition is not to
say it is random or entirely arbitrary, but it does mean that there is no set
of explicit underlying assumptions that serve as the foundation for the
hypothesis. In other cases, a hypothesis is derived from an analogous field;
for example, Chaabouni et al. (2021, p. 1) study information content in a
color naming game and derive their hypothesis (in part) from color-naming
conventions observed in human languages.

Finally, hypotheses can be generated from a theoretical model, as illustrated
in Section 4.1.2. By combining the axioms, the model makes certain claims
about how a given system will behave, which forms the core of a hypothesis. A
theoretical model facilitates deriving clear, testable hypotheses as the
predictions and their preconditions are often evident looking at the
theoretical model. In most cases, if the aim is stated in the form of
“investigating the effect of $X$ on $Y$” or “seeing whether or not $X$”
occurs, it is not a testable hypothesis. These aims are adequate for gaining a
baseline sense of the stated topic, but since is not possible to confirm or
refute such statements they cannot contribute to developing an underlying
model.

####  4.1.4 Theory

Theories are not yet present in emergent language research primarily because
there has only been sparse use of theoretical models, which form the core of a
theory. To move forward, then, it is necessary first to normalize the use of
theoretical models in forming hypotheses and designing experiments. After
basic theoretical models are developed, it is then necessary to build on top
of them by both amending them when their hypotheses are refuted and expanding
their scope to account for a greater range of situations and phenomena.
Without taking an accumulative approach by building on top of prior
theoretical models, it is difficult to determine if findings from the prior
work applies to the subsequent approaches since any differences not accounted
for by the model are confounding factors.

Expecting the construction of precise and broad-reaching theories in emergent
language research analogous to what we see in the natural sciences like
physics, biology, and chemistry is likely unreasonable. First and foremost,
this is because such a theory would entail a precise way to theoretically
model reinforcement learning and neural networks, two fields dealing with
complex systems undergoing rapid change and innovation. The second difficulty
is that when building theoretical models describing human language evolution,
we do not even have access to the system we are trying to describe since we
cannot directly observe or experiment with how human language evolves. Both of
these facts put emergent language research in a similar boat to scientific
approaches to economics, first because economics fundamentally concerns the
behavior of humans which is too complex to model precisely and second because
it is impossible to run fully-fledged economic experiments (although economics
is in a better position than emergent language research here). Thus, it would
be more reasonable to expect emergent language theories to develop more along
the lines of scientific theories in economics, which do not provide natural
science-level precision but still represent a rational and effective method of
research.

###  4.2 Engineering

####  4.2.1 Problem Definition

Well-defined problems are occasionally given in the current body of emergent
language literature and co-occur with topics that very clearly fall into the
realm of engineering. For example, Bullard et al. (2021, p. 1) define a zero-
shot communication class of problems which are defined by the following
constraints in a cooperative game: (1) the agents train against themselves
(self-play), (2) the agents are evaluated by playing with a new partner
(cross-play), and (3) the symbols used to communicate have a variable cost
assigned by the environment. Further examples include Eccles et al. (2019) and
Li et al. (2020) which offer formulations to particular, measurable problem.

To a certain extent, exploratory research is characterized by _not_ precisely
defining problems because it is attempting to find that base level of
knowledge which is necessary for a well-defined problem. Thus, as more
emergent language research moves to tackling real applications, it should
become easier to precisely define problem a research project is addressing.
Nevertheless, being more intentional with defining and illustrating the
problem addressed by the paper will catalyze this process.

####  4.2.2 Evaluation Metric

Similarly to problem definitions, evaluation metrics only appear occasionally
in the current literature. One group of metrics that has received considerable
attention in the emergent language literature are metrics for measuring the
compositionality of an emergent language. In most cases, these are not true
_evaluation_ metrics since they are not being optimized for as part of an
engineering problem. If such a metric were being optimized for, one could
simply introduce strong inductive biases (e.g., hard-coding the language)
towards a fully compositional protocol; although this may seem inappropriate,
the constraints for the solution space must be clear from the problem
definition.

Clear problem definitions go hand-in-hand with metrics which are truly
evaluative as the criteria for success are clear in a well-defined problem.
For example, in the aforementioned papers in Section 4.2.1, we find that each
problem has a clear evaluation metric appropriate for their problem
definition: Bullard et al. (2021) study zero-shot coordination and evaluate
with success rate, Eccles et al. (2019) study the beneficial effect positive
signalling between agents and evaluate with success rate and environment
reward, and Li et al. (2020) study emergent language for pretraining in few-
shot translation and evaluate with BLEU.

####  4.2.3 Standardization

Standard problems and approaches to problems are largely not present in
emergent language research. This is largely inherent to exploratory research
since one of its primary purposes is to find approaches which are worthy of
becoming standard approaches.

To a certain extent, consensus of a set of problem definitions will happen
without needing explicit efforts. Researchers will naturally choose the most
salient problems to work on, and additionally, problem selection has a self-
reinforcing effect where the more a given problem is researched, the more
visibility it receives which in turn increases the likelihood of it being
selected for subsequent studies. On the other hand, novelty is often treated
as inherently valuable in the publication process, which pushes against
consensus and standardization. Nevertheless, since other areas of machine
learning have naturally standardized certain problems, it is reasonable to
think that emergent language research can follow a similar path.

The best instance of standardization more generally in emergent language
research is in the area of tooling: in particular, the EGG toolkit (Kharitonov
et al., 2019). This toolkit provides a simple way to write Python code for
emergent language experiments. Such a resource can be especially helpful for
emergent language researchers who do not have a strong background in computer
programming. It has been used in the implementation in a handful of papers
(beyond the authors’ own) including Słowik et al. (2020); Korbak et al.
(2019); Guo et al. (2020).

####  4.2.4 Benchmark

Benchmarks have not yet appeared in emergent language research. This is due to
the fact that benchmarks, in part, rely on a commonly agreed upon problem
definition which is absent. On the other hand, benchmarks can also be an
impetus for standardization if the problem formulation and method of
evaluation is of sufficient quality to convince a number of research groups to
put their efforts towards this. Nevertheless, it would be likely be difficult
to build a convincing benchmark until some degree of consensus on worthwhile
problem formulations is reached.

###  4.3 Summary

For the scientific goals of emergent language, the critical step towards
systematic research is incorporating theoretical models into the standard
research process. Theoretical models will yield hypotheses and empirical
evaluations which for the basis for building theories. These theories are
integral to understanding a number of questions about the nature language as
well measurably gauging progress towards answering these questions.

For the engineering goals of emergent language, the critical step towards
systematic research is precisely defining problems. Well-defined problems form
the basis for effective evaluation metrics and eventually standardization, the
key factor in producing research contributions which measurably improve on
prior approaches.

##  5 Conclusion

The above methodological recommendations offer a way to move work on emergent
language toward systematic research, where research projects build off each
other in a way that shows measurable improvement. The exploratory research in
the field thus far aids in recognizing the unique challenges of emergent
language research which is necessary to implement these recommendations. The
distinction we have made between the capacities in which emergent language
research can be science or engineering is critical—while the fields are
closely related and intertwined, their paths to comparable, iterative research
differ significantly. Moving towards systematic research on emergent language
will, then, facilitate the achievement of the unique and important goals of
this field.

#### Acknowledgements

This material is based on research sponsored in part by the Air Force Research
Laboratory under agreement number FA8750-19-2-0200. The U.S. Government is
authorized to reproduce and distribute reprints for Governmental purposes
notwithstanding any copyright notation thereon. The views and conclusions
contained herein are those of the authors and should not be interpreted as
necessarily representing the official policies or endorsements, either
expressed or implied, of the Air Force Research Laboratory or the U.S.
Government.

## References

  * Bisk et al. (2020) Bisk, Y., Holtzman, A., Thomason, J., Andreas, J., Bengio, Y., Chai, J., Lapata, M., Lazaridou, A., May, J., Nisnevich, A., Pinto, N., and Turian, J.  Experience grounds language.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pp. 8718–8735, Online, November 2020\. Association for Computational Linguistics.  doi: 10.18653/v1/2020.emnlp-main.703.  URL https://www.aclweb.org/anthology/2020.emnlp-main.703.
  * Bouchacourt & Baroni (2018) Bouchacourt, D. and Baroni, M.  How agents see things: On visual representations in an emergent language game.  In _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing_ , pp. 981–985, Brussels, Belgium, October-November 2018. Association for Computational Linguistics.  doi: 10.18653/v1/D18-1119.  URL https://aclanthology.org/D18-1119.
  * Bullard et al. (2021) Bullard, K., Kiela, D., Meier, F., Pineau, J., and Foerster, J.  Quasi-equivalence discovery for zero-shot emergent communication, 2021\.
  * Bunge (1966) Bunge, M.  Technology as applied science.  _Technology and Culture_ , 7(3):329–347, 1966\.  ISSN 0040-165X.  doi: 10.2307/3101932.
  * Cao et al. (2018) Cao, K., Lazaridou, A., Lanctot, M., Leibo, J. Z., Tuyls, K., and Clark, S.  Emergent Communication through Negotiation.  _ICLR_ , 2018.
  * Chaabouni et al. (2019) Chaabouni, R., Kharitonov, E., Dupoux, E., and Baroni, M.  Anti-efficient encoding in emergent communication.  In Wallach, H., Larochelle, H., Beygelzimer, A., Alché-Buc, F. d., Fox, E., and Garnett, R. (eds.), _Advances in Neural Information Processing Systems 32_ , pp. 6293–6303. Curran Associates, Inc., 2019.  URL http://papers.nips.cc/paper/8859-anti-efficient-encoding-in-emergent-communication.pdf.
  * Chaabouni et al. (2020) Chaabouni, R., Kharitonov, E., Bouchacourt, D., Dupoux, E., and Baroni, M.  Compositionality and generalization in emergent languages.  _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , 2020.  doi: 10.18653/v1/2020.acl-main.407.  URL http://dx.doi.org/10.18653/v1/2020.acl-main.407.
  * Chaabouni et al. (2021) Chaabouni, R., Kharitonov, E., Dupoux, E., and Baroni, M.  Communicating artificial neural networks develop efficient color-naming systems.  _Proceedings of the National Academy of Sciences_ , 118(12), Mar 2021.  ISSN 0027-8424, 1091-6490.  doi: 10.1073/pnas.2016569118.  URL https://www.pnas.org/content/118/12/e2016569118.
  * Eccles et al. (2019) Eccles, T., Bachrach, Y., Lever, G., Lazaridou, A., and Graepel, T.  Biases for Emergent Communication in Multi-agent Reinforcement Learning.  In _Advances in Neural Information Processing Systems_ , volume 32. Curran Associates, Inc., 2019.  URL https://papers.nips.cc/paper/2019/hash/fe5e7cb609bdbe6d62449d61849c38b0-Abstract.html.
  * Evtimova et al. (2018) Evtimova, K., Drozdov, A., Kiela, D., and Cho, K.  Emergent communication in a multi-modal, multi-step referential game.  In _International Conference on Learning Representations_ , 2018.  URL https://openreview.net/forum?id=rJGZq6g0-.
  * Giere (1984) Giere, R. N.  _Understanding scientific reasoning_.  Dryden Press, Fort Worth, second edition, 1984.
  * Guo et al. (2020) Guo, S., Ren, Y., Slowik, A., and Mathewson, K.  Inductive bias and language expressivity in emergent communication.  _ArXiv_ , abs/2012.02875, 2020.
  * Harding Graesser et al. (2019) Harding Graesser, L., Cho, K., and Kiela, D.  Emergent linguistic phenomena in multi-agent communication games.  In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_ , pp. 3700–3710. Association for Computational Linguistics, Nov 2019.  doi: 10.18653/v1/D19-1384.  URL https://aclanthology.org/D19-1384.
  * Havrylov & Titov (2017) Havrylov, S. and Titov, I.  Emergence of language with multi-agent games: Learning to communicate with sequences of symbols.  In _Proceedings of the 31st International Conference on Neural Information Processing Systems_ , NIPS’17, pp. 2146–2156, Red Hook, NY, USA, 2017. Curran Associates Inc.  ISBN 9781510860964.
  * Hepburn & Andersen (2021) Hepburn, B. and Andersen, H.  _Scientific Method_.  Metaphysics Research Lab, Stanford University, summer 2021 edition, 2021\.  URL https://plato.stanford.edu/archives/sum2021/entries/scientific-method/.
  * Kharitonov & Baroni (2020) Kharitonov, E. and Baroni, M.  Emergent language generalization and acquisition speed are not tied to compositionality.  In _Proceedings of the Third BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP_ , pp. 11–15, Online, November 2020\. Association for Computational Linguistics.  doi: 10.18653/v1/2020.blackboxnlp-1.2.  URL https://aclanthology.org/2020.blackboxnlp-1.2.
  * Kharitonov et al. (2019) Kharitonov, E., Chaabouni, R., Bouchacourt, D., and Baroni, M.  EGG: a toolkit for research on emergence of lanGuage in games.  In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP): System Demonstrations_ , pp. 55–60, Hong Kong, China, November 2019. Association for Computational Linguistics.  doi: 10.18653/v1/D19-3010.  URL https://aclanthology.org/D19-3010.
  * Kharitonov et al. (2020) Kharitonov, E., Chaabouni, R., Bouchacourt, D., and Baroni, M.  Entropy minimization in emergent languages.  In III, H. D. and Singh, A. (eds.), _Proceedings of the 37th International Conference on Machine Learning_ , volume 119 of _Proceedings of Machine Learning Research_ , pp. 5220–5230. PMLR, 13–18 Jul 2020.  URL http://proceedings.mlr.press/v119/kharitonov20a.html.
  * Korbak et al. (2019) Korbak, T., Zubek, J., Kucinski, L., Milos, P., and Rączaszek-Leonardi, J.  Developmentally motivated emergence of compositional communication via template transfer.  Emergent Communication Workshop, NeurIPS, 2019.
  * Kottur et al. (2017) Kottur, S., Moura, J., Lee, S., and Batra, D.  Natural language does not emerge “naturally” in multi-agent dialog.  _Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing_ , 2017.  doi: 10.18653/v1/d17-1321.  URL http://dx.doi.org/10.18653/v1/D17-1321.
  * Kuhn (1962) Kuhn, T. S.  _The Structures of Scientific Revolutions_.  University of Chicago Press, Chicago, fourth edition, 1962.
  * Lazaridou & Baroni (2020) Lazaridou, A. and Baroni, M.  Emergent multi-agent communication in the deep learning era.  _arXiv:2006.02419 [cs]_ , Jul 2020.  URL http://arxiv.org/abs/2006.02419.  arXiv: 2006.02419.
  * Lazaridou et al. (2017) Lazaridou, A., Peysakhovich, A., and Baroni, M.  Multi-agent cooperation and the emergence of (natural) language.  _International Conference on Learning Representations_ , 2017.  URL https://openreview.net/forum?id=Hk8N3Sclg.
  * Lazaridou et al. (2018) Lazaridou, A., Hermann, K. M., Tuyls, K., and Clark, S.  Emergence of linguistic communication from referential games with symbolic and pixel input.  In _International Conference on Learning Representations_ , 2018.  URL https://openreview.net/forum?id=HJGv1Z-AW.
  * Li et al. (2020) Li, Y., Ponti, E. M., Vulić, I., and Korhonen, A.  Emergent communication pretraining for few-shot machine translation.  In _Proceedings of the 28th International Conference on Computational Linguistics_ , pp. 4716–4731, Barcelona, Spain (Online), December 2020. International Committee on Computational Linguistics.  doi: 10.18653/v1/2020.coling-main.416.  URL https://aclanthology.org/2020.coling-main.416.
  * Lowe et al. (2020) Lowe, R., Gupta, A., Foerster, J., Kiela, D., and Pineau, J.  On the interaction between supervision and self-play in emergent communication.  In _International Conference on Learning Representations_ , 2020.  URL https://openreview.net/forum?id=rJxGLlBtwH.
  * Mordatch & Abbeel (2018) Mordatch, I. and Abbeel, P.  Emergence of grounded compositional language in multi-agent populations, 2018.
  * Resnick et al. (2020) Resnick, C., Gupta, A., Foerster, J., Dai, A. M., and Cho, K.  Capacity, Bandwidth, and Compositionality in Emergent Language Learning.  _International Conference on Autonomous Agents and Multi-Agent Systems_ , April 2020.  URL http://arxiv.org/abs/1910.11424.
  * Russakovsky et al. (2015) Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M. S., Berg, A. C., and Fei-Fei, L.  Imagenet large scale visual recognition challenge.  _International Journal of Computer Vision_ , 115:211–252, 2015.
  * Słowik et al. (2020) Słowik, A., Gupta, A., Hamilton, W. L., Jamnik, M., Holden, S., and Pal, C. J.  Exploring structural inductive biases in emergent communication.  _ArXiv_ , abs/2002.01335, 2020.
  * Winther (2021) Winther, R. G.  _The Structure of Scientific Theories_.  Metaphysics Research Lab, Stanford University, spring 2021 edition, 2021\.  URL https://plato.stanford.edu/archives/spr2021/entries/structure-scientific-theories/.