11institutetext:  Instituto de Radioastronomía y Astrofísica, Universidad
Nacional Autónoma de México, P.O. Box 3-72, 58090, Morelia, Michoacán, Mexico
11email<EMAIL_ADDRESS>22institutetext: Univ. Grenoble Alpes, CNRS,
Institut de Planétologie et d’Astrophysique de Grenoble (IPAG), 38000
Grenoble, France 33institutetext: Laboratoire d’Astrophysique de Bordeaux,
Univ. Bordeaux, CNRS, B18N, allée Geoffroy Saint-Hilaire, 33615 Pessac, France
44institutetext: Laboratoire de Physique de l’ENS, ENS, Université PSL, CNRS,
Sorbonne Université, Université de Paris, 75005 Paris, France 55institutetext:
Observatoire de Paris, PSL University, Sorbonne Université, LERMA, 75014
Paris, France

# Modeling the early mass ejection in jet-driven protostellar outflows.
Lessons from Cep E.

P. R. Rivera-Ortiz 1122 A. de A. Schutzer 22 B. Lefloch 2233 A. Gusdorf 4455

(Accepted January 12, 2022)

###### Abstract

Context.  Protostellar jets and outflows are an important agent of star
formation as they carry away a fraction of momentum and energy, which is
needed for gravitational collapse and protostellar mass accretion to occur.

Aims.  Our goal is to provide constraints on the dynamics of the inner
protostellar environment from the study of the outflow-jet propagation away
from the launch region.

Methods.  We have used the axisymmetric chemo-hydrodynamical code Walkimya-2D
to numerically model and reproduce the physical and CO emission properties of
the jet-driven outflow from the intermediate-mass protostar CepE-mm, which was
observed at $\sim 800$ au resolution in the CO $J$=2–1 line with the IRAM
interferometer. Our simulations take into account the observational
constraints available on the physical structure of the protostellar envelope.

Results.  Walkimya-2D successfully reproduces the main qualitative and
quantitative features of the Cep E outflow and the jet kinematics, naturally
accounting for their time variability. Signatures of internal shocks are
detected as knots along the jet. In the early times of the ejection process,
the young emitted knots interact with the dense circumstellar envelope through
high-velocity, dissociative shocks, which strongly decrease the CO gas
abundance in the jet. As time proceeds, the knots propagate more smoothly
through the envelope and dissociative shocks disappear after $\sim
10^{3}\hbox{\kern 1.99997ptyr}$. The distribution of CO abundance along the
jet shows that the latter bears memory of the early dissociative phase in the
course of its propagation. Analysis of the velocity field shows that the jet
material mainly consists of gas entrained from the circumstellar envelope and
accelerated away from the protostar at 700 au scale. As a result, the overall
jet mass-loss rate appears higher than the actual mass-ejection rate by a
factor $\sim 3$.

Conclusions. Numerical modeling of the Cep E jet-driven outflow and comparison
with the CO observations have allowed us to peer into the outflow formation
mechanism with unprecedented detail and to retrieve the history of the mass-
loss events that have shaped the outflow.

###### Key Words.:

stars: formation — ISM: jets and outflows — astrochemistry

##  1 Introduction

Protostellar jets and outflows are an ubiquitous phenomenon of the star
formation process from the early Class 0 to the late Class I phase, when the
parental envelope is dissipated. They are an important agent of star formation
feedback, which affects the gas’s physical and chemical properties from cloud
scale down to the central parental cocoon, where mass accretion is occurring.
On the one hand, they act as a source of energy and momentum into the parental
cloud, and they disperse the material of the parental core, directly impacting
the star formation efficiency and the final stellar mass (see Frank et al.
(2014) for a review); on the other hand, they are thought to remove a
significant fraction of the angular momentum from the star–disk system,
enabling the gas in the accretion disk to reach the central protostar (Konigl
& Pudritz, 2000).

A natural consequence of the propagation of such high velocity outflows
through the protostellar envelope and the ambient molecular medium are shock
fronts (Reipurth & Raga, 1999). These shocks both heat and compress the gas,
which favors chemical reactions in the gas phase while they modify the dust
grain properties and release a fraction of their material in the gas phase
through, for example, sputtering and shattering, thereby leading to a
different chemical composition than observed in the ambient, preshock gas.
Commonly, the high relative abundance of CO and its low energy J transitions
makes it a good tracer for outflows in the conditions of cold molecular
clouds. Even more, the morphology, velocity, and sizes of a molecular outflow
depend on the observed tracer, the luminosity, mass, and age of the outflow,
which indicates its evolution and its inner structure (Bally, 2016).

Over the years, there have been intense numerical efforts to simulate the
chemical and dynamical structure of protostellar outflows and their evolution
in the ambient interstellar gas. Smith (1994)and Smith et al. (1997) carried
out 3D numerical simulations of dense molecular jets drilling through a
molecular environment. They succeeded in reproducing the morphology of the
“classical” CO bipolar outflows, which image the accumulated and accelerated
cool gas. They showed that the simulations predict infrared shock structures
remarkably similar to those found in highly collimated Class 0 outflows. They
proved that in spite of the limitations in their description of the physical
conditions (dust, magnetic field), these hydrodynamical models could provide a
meaningful description of the dynamics of bipolar outflows from young stars,
accurate enough images for comparison with observations, and the outflow mass
and energy contribution to the interstellar medium.

Raga et al. (1990) showed that simulations with an arbitrarily imposed
ejection velocity variability lead to the formation of chains of internal
working surfaces traveling down the jet flow. Such features strikingly
resemble Herbig-Haro (HH) flows, with a chain of aligned knots close to the
outflow source, and a large “head,” resulting from the “turning on” of the jet
flow at larger distances. Later on, the authors showed that a two-mode
ejection velocity variability leads to the formation of chains of “short
period knots”, which catch up with each other to form “long period knots”.
This class of models has successfully accounted for the properties of HH flows
and they nowadays dominate the literature on the theory of astrophysical jets
(Canto, 1985; Raga et al., 1990, 2003; Noriega-Crespo et al., 2014; Frank et
al., 2014; Rabenanahary et al., 2022). On the other hand, most of the
hydrodynamical codes developed so far have not included a chemical network and
thereby do not address the molecular gas composition, in contradiction with
the observational evidence that outflows from Class 0 protostars are often
chemically active (Bachiller et al., 2001; Arce et al., 2007; Lefloch et al.,
2017; Ospina-Zamudio et al., 2018; De Simone et al., 2020).

It is only with the advent of the first generation of chemo-hydrodynamical
codes that it is now possible to accurately model the physical and chemical
outflowing gas evolution, and to build spectroscopic diagnostics which can be
tested against the observational constraints provided by the large
(sub)millimeter arrays such as the Northern Extended Millimetre Array (NOEMA)
and the Atacama Large Millimeter/submillimeter Array (ALMA), and to
investigate in detail the processes that determine the jet-outflow properties
in order to quantify the energetic and chemical feedback of newly born stars
on their environment and eventually on galaxies.

In this work, we use Walkimya-2D, a new 2D hydrodynamical code coupled to a
reduced gas phase chemical network allowing for the evolution of the CO
chemistry in molecular outflows to be followed (Castellanos-Ramírez et al.,
2018). The advantage of this code is that it computes the time evolution of
chemical species in a full gas dynamic simulation. The new functionalities of
Walkimya-2D allow us to investigate more thoroughly the dynamics and the
physical structure of protostellar outflows.

We have selected the high-velocity outflow associated with the intermediate-
mass Class 0 protostellar system Cep E-mm, located in the Cepheus OB3
association at a distance of $819\pm 16\hbox{\kern 1.99997ptpc}$ (Karnath et
al., 2019), whose CO emission observed at $1\arcsec$ with the Institut de
Radioastronomie Millimétrique (IRAM) interferometer is displayed in Fig. 1.
This outflow has a luminosity of $\approx 100\hbox{\kern
1.99997pt$L_{\odot}$}$ (Lefloch, Eisloeffel & Lazareff, 1996) and a core mass
of $35\hbox{\kern 1.99997pt$M_{\odot}$}$ (Crimier et al., 2010). The source
has been the subject of several detailed studies at arcsec angular resolution,
in particular in the CO rotational transitions in the (sub)millimeter domain
(Lefloch, Eisloeffel & Lazareff, 1996; Lefloch et al., 2015; Ospina-Zamudio et
al., 2019), which have constrained the jet and outflow dynamical parameters
(mass, mass-loss rate, momentum, density, temperature). The recent study by
Schutzer et al. (2022) has brought a detailed view of the jet structure and
its time variability, showing a complex interaction with the ambient
protostellar material. It is also one of the few protostellar jets for which a
full 3D picture of the gas kinematics is available. Overall, this outflow
appears as an excellent testbed for chemo-hydrodynamical codes, with the
prospect of getting more insight into the jet and outflow formation process.

The goal of this study is to obtain an accurate numerical model of the Cep E
molecular outflow in agreement with the constraints provided by the large
(sub)millimeter observatories. In particular, we aim to understand the
dynamics of the jet and envelope interaction near the source. The synergy
between the Walkimya-2D numerical simulations and the observations with the
IRAM interferometer has allowed us to peer into the outflow formation
mechanism with unprecedented detail and to retrieve the history of the mass-
loss events that have shaped the outflow.

This paper is organized as follows: in Section 2, we describe the numerical
setup and the boundary conditions, which were chosen in agreement with the
observational constraints on Cep E and present the best fitting model. The
following sections present our main results: the outflow formation in Section
3, the gas acceleration mechanism in Section 4, the mass-loss history in
Section 5, and the CO emission in Section 6. Finally, we summarize our
conclusions in Section 7.

##  2 Simulations

###  2.1 Walkimya-2D

In order to study the temporal evolution of the molecular composition of a
gas, one needs to solve the rate of change of the abundances of the different
species contained in it. It is required to construct a chemical network
gathering the creation and destruction reactions for the different species. We
have used the 2D chemo-hydrodynamical Walkimya-2D that solves the hydrodynamic
equations and a chemical network, on an axisymmetric numerical adaptive mesh.
A complete description of the code is presented in Castellanos-Ramírez et al.
(2018). Initially, both the jet and the surrounding quiescent protostellar gas
have a CO abundance of 1.6$\times 10^{-4}$ relative to H2. The chemical
network is designed mainly to follow the evolution of the CO chemistry, as
induced by the heating-cooling processes in the jet propagation and the shock
interaction(s) with the ambient gas. For computational reasons (to keep
computational times within reach), the network is reduced to 14 chemical
species, including C, O, H2O, OH, and CO (Castellanos-Ramírez et al., 2018).
The reaction rates were obtained from the UMIST database (McElroy et al.,
2013). Walkimya-2D and its chemical network were successfully benchmarked
against laboratory experiments to study the NO formation by electrical
discharges using both a zero-dimensional and a gas dynamic model approach
Castellanos-Ramírez et al. (2018). Also, in the astrochemical context, the
code was successfully benchmarked against the model of dark molecular cloud
presented in McElroy et al. (2013), and has been used to explain the CO
emission produced by the Orion Fingers in the Orion KL region (Rodríguez-
González et al., 2022).

The energy loss rate is calculated adopting the same prescription as in
Rivera-Ortiz et al. (2019) (see references therein): for temperatures larger
than 5800 K the cooling function considers the atomic contribution while in
the lower temperature range, it uses a parametric molecular cooling function
based on CO and H2.

The adaptative mesh uses seven levels of refinement, yielding $4096\times
1024$ cells, in a computational domain ($5\times 1.25$)$\times 10^{4}$ au,
allowing a resolution as high as 12.2 au per cell side. We used a reflective
boundary condition for the symmetry axis and a free outflow boundary condition
for all the other borders. The size of the mesh is large enough that the outer
boundaries do not affect the simulation. The jet is injected on the left side
of the simulation box with the physical conditions indicated in Table 1.

###  2.2 Initial conditions: the protostellar core

In order to describe the protostellar core, we have adopted the physical
conditions derived by Crimier et al. (2010) from a simple 1D modeling of the
dust spectral energy distribution between $24\mu m$ and $1300\mu m$, using the
radiative transfer code 1D DUSTY (Ivezic & Elitzur, 1997). The temperature is
fitted by a broken radial power law $T\propto r^{\beta}$ with $\beta$= $-0.8$
in the range 50–$300\hbox{\kern 1.99997ptK}$ and $\beta$= $-0.4$ in the range
$7\hbox{\kern 1.99997ptK}$–$50\hbox{\kern 1.99997ptK}$. In Crimier’s model,
the density profile is assumed to start at a radius $r_{in}$= 70 au and to
follow a single radial power law distribution $n(r)$=
$n_{0}\times(r_{0}/r)^{\alpha}$.

In order to avoid density singularities in the limit $r\to 0$ in the numerical
modeling, we have adopted the slightly modified density profile

$n(r)=\frac{n_{0}}{1+\left(r/r_{0}\right)^{\alpha}},$ (1)

where $n_{0}=10^{9}$ cm-3, $r_{0}=100$ au and $\alpha=1.9$.

The simulation takes into account the role of gravity, in agreement with the
source density stratification evidenced from Crimier’s analysis. Our model
includes the resulting gravitational force as a source term, which reads:

$g=-\frac{2\;c_{0}^{2}\;r}{r_{0}^{2}\left(1+\left(r/r_{0}\right)^{\alpha}\right)}$
(2)

where $c_{0}$ is the local sound speed. The inclusion of this gravity term
ensures hydrostatic equilibrium close to the source.

###  2.3 Initial conditions: the jet

Figure 1: CO $2$–$1$ line emission in the Cep E-mm outflow as observed with
the PdBI at $1\arcsec$ resolution (Lefloch et al., 2015). Four main velocity
components are detected: a) the outflow cavity walls (black) emitting at low
velocities, in the range [$-8$;$-3$]kms-1 and [$-19$;$-14$]kms-1 in the
northern and southern lobe, respectively; b) the jet, emitting at high
velocities in the range [$-135$;$-110$]kms-1 in the southern lobe (blue) and
in the range [$+40$;$+80$]kms-1 in the northern lobe (red); c) the southern
terminal bow shock HH377, integrated in the range [$-77$,$-64$]kms-1 (green);
d) the northern terminal bullet NB integrated in the velocity range
[$+84$,$+97$]kms-1 (magenta). First contour and contour interval are $20\%$
and $10\%$ of the peak intensity in each component, respectively. The
synthesized beam ($1\aas@@fstack{\prime\prime}07\times
0\aas@@fstack{\prime\prime}87$, HPBW) is shown in the bottom left corner. The
main axis of the northern and southern lobes of the jet are shown with black
arrows. (Schutzer et al., 2022)

.

The jet is bipolar and consists of a narrow (diameter ¡ 400 au) central
component surrounded by a collimated layer with a radial extent up to 1000 au,
as shown by Schutzer et al. (2022). The jet appears young, with a dynamical
age of $1400\hbox{\kern 1.99997ptyr}$, and compact, with a length of $\sim
0.1$–$0.14\hbox{\kern 1.99997ptpc}$ (18000 – 29000 au) for the southern and
northern lobes, respectively. The physical conditions (temperature, H2 gas
density) in the outflow were estimated by Lefloch et al. (2015) and Ospina-
Zamudio et al. (2019) from a CO multi-rotational line study complemented with
CO $J$=2–1 observations at $1\arcsec$ angular resolution with the IRAM
interferometer. In the southern lobe, the jet appears to consist of a warm
(T=$80$–$100\hbox{\kern 1.99997ptK}$) gas component of H2 density
$n(\hbox{H${}_{2}$})$= (0.5–1.0)$\times 10^{5}\hbox{\kern
1.99997ptcm${}^{-3}$}$ component and a higher-excitation component of $n(H)$=
(0.5–1.0)$\times 10^{6}\hbox{\kern 1.99997ptcm${}^{-3}$}$ and temperature
(T=$400$–$750\hbox{\kern 1.99997ptK}$), which the authors associated with the
high-velocity knots. Similar physical conditions are found in the northern
lobe, with a kinetic temperature T=$180$–$300\hbox{\kern 1.99997ptK}$ and gas
density $n(\hbox{H${}_{2}$})$= (0.6–2.0)$\times 10^{5}\hbox{\kern
1.99997ptcm${}^{-3}$}$. Therefore, the jet is rather massive, with a total
mass of $0.03\hbox{\kern 1.99997pt$M_{\odot}$}$ ($0.09\hbox{\kern
1.99997pt$M_{\odot}$}$) in the southern (northern) lobe, after taking into
account the revised distance to the source (Karnath et al., 2019).

The jet asymptotic radial velocities were determined from the CO 2–1 line
profiles as $+65\hbox{\kern 1.99997ptkm\kern 1.99997pts${}^{-1}$}$ and
$-125\hbox{\kern 1.99997ptkm\kern 1.99997pts${}^{-1}$}$ in the northern and
southern lobe, respectively (Lefloch et al., 2015). The jet proper motions in
both lobes were measured by Noriega-Crespo et al. (2014) from combining
multiple mid-infrared IRAC $4.5\mu m$ observations and H2 $2.12\mu m$ images
at a difference of 16 years. We note that the revised distance of Cep E-mm
(820 pc instead of 730 pc previously) does not significantly affect the
tangential velocities, nor the inclination angle of the jet with respect to
the plane of the sky derived by Lefloch et al. (2015), which is now
$47\hbox{\kern 1.99997pt${}^{\rm o}$}$ ($40\hbox{\kern 1.99997pt${}^{\rm o}$}$
previously estimated) and the estimated dynamical age of the jet (about
$10^{3}\hbox{\kern 1.99997ptyr}$). Based on these observations, the molecular
jet velocity is estimated $\sim 100\hbox{\kern 1.99997ptkm\kern
1.99997pts${}^{-1}$}$ ($150\hbox{\kern 1.99997ptkm\kern 1.99997pts${}^{-1}$}$)
in the northern (southern) lobe, respectively. Kinematical analysis of the
molecular gas knots in the jet shows evidence for time and velocity
variability of the mass-ejection process in Cep E (Schutzer et al., 2022).

In the simulation, the jet is launched at z=0, with a radius $r_{j}$, a gas
density $n_{j}(H)$= $10^{6}\hbox{\kern 1.99997ptcm${}^{-3}$}$, and a
temperature $T_{j}$= $300\hbox{\kern 1.99997ptK}$. In order to account for
knot formation inside the jet, we introduced variability in the gas injection,
which we modeled following the equation

$V_{j}=V_{j,0}\left[1+\delta_{v}\cos(2\pi t/\tau)\right]$ (3)

where $\tau=130$ yr is the injection mass period and $t$ is the evolutionary
time. The jet injection velocity $V_{j,0}$ and the relative amplitude
variability $\delta_{v}$ are free parameters of the simulation. We adopted
$\delta_{v}$= 0.05–0.08, which implies velocity variations of
10–$15\hbox{\kern 1.99997ptkm\kern 1.99997pts${}^{-1}$}$, consistent with the
typical variations reported by Schutzer et al. (2022) in Cep E and in other
outflows such as IRAS04166+2706 (Santiago-García et al., 2009).

###  2.4 Comparison with observations

Parameter | M1 | M2 | M3 | M4
---|---|---|---|---
$r_{j}$[au] | 100 | 50 | 50 | 150
$v_{j,0}$ [km s-1] | 200 | 165 | 200 | 200
$\delta_{v}$ | 0.08 | 0.08 | 0.05 | 0.08
Table 1: Initial parameters of the four models M1–M4 computed with
Walkimya-2D: jet radius $r_{j}$, injection velocity $v_{j,0}$, relative
velocity variability amplitude $\delta_{v}$. M1 is the model which best fits
the Cep E outflow.

We have run four models M1–M4, whose initial jet parameters are listed in
Table 1. For the sake of simplicity, our Walkimya-2D simulations do not
include the effect of jet precession. Comparison between numerical simulations
and observations are made with the northern outflow lobe of Cep E, whose
entrained gas dynamics is better revealed in CO interferometric observations
(Schutzer et al., 2022): morphology, gas acceleration, time-variability, and
knot formation.

The simulations give us access to the physical structure of the outflow and
its chemical properties. Filtering the emission of the high- and low-velocity
components allows direct comparison between the simulations and the
observational signatures of the distinct outflow components and therefore it
is possible to quantify the physical processes involved in the outflow
formation. In order to compare our numerical simulations with the CO
observations, the emissivity has been computed directly from the
hydrodynamical simulations assuming local thermodynamic equilibrium (LTE). We
have constructed synthetic CO J= 2–1 maps of the outflow cavity and the jet,
by integrating the emission in the velocity intervals [3;$7\hbox{\kern
1.99997ptkm\kern 1.99997pts${}^{-1}$}$] and [50;$150\hbox{\kern
1.99997ptkm\kern 1.99997pts${}^{-1}$}$], respectively. Those maps were
subsequently convolved with a gaussian profile whose size (FWHM) corresponds
to the synthetic beamsize of the interferometer ($1\arcsec$ or 820 au).

We first explored the parameter space and searched for the model which best
reproduces the qualitative and quantitative properties of the Cep E northern
outflow lobe. Overvall, we found that all four models M1 – M4 show similar
results. Therefore, in what follows we present the results of model M1, that
is, the simulation that best accounts for the molecular gas observations of
Cep E, whose initial jet parameters are summarized in Table 1. The initial jet
velocity is $200\hbox{\kern 1.99997ptkm\kern 1.99997pts${}^{-1}$}$, with a
velocity fluctuation amplitude of 0.08, and an ejection radius of 100 au.

Parameter | M1 | M2 | M3 | M4 | Cep E
---|---|---|---|---|---
|  |  |  |  | North South
$M_{j}\,[10^{-2}$M${}_{\odot}]$ | 2.7 | 0.75 | 1.5 | 5 | 3.6 | 2.6
$M_{o}[$M${}_{\odot}]$ | 2.3 | 1.3 | 1.2 | 2.9 | 2.0 | 0.4
$z_{bs}[\,10^{3}$au$]$ | 28 | 23 | 29 | 31 | 28 | 21
$r_{max}[\,10^{3}$au$]$ | 4.8 | 1.5 | 3.6 | 5.3 | 5 | 4
Table 2: Outflow physical properties derived from CO emissivity maps at 1500
yr in the four numerical simulations (M1– M4) and comparison with the
observational values of the Cep E outflow: jet mass $M_{j}$, outflow mass
$M_{o}$, jet length $z_{bs}$, maximum outflow radius $r_{max}$. These values
are taken from Lefloch et al. (2015) and Ospina-Zamudio et al. (2019) and have
been corrected for the revised distance to the source. Figure 2: Outflow
formation and impact of the jet on the gas density distribution as a function
of time. Snapshots for 0, 500, 1000, 1500, and 2000 yr. The injection velocity
variability forms internal working surfaces, or knots, and a structured
cavity. The leading bowshock accumulates the mass of several knots.

We compare the numerical results to the main observational features of the
northern outflow lobe, identified in the recent study by Schutzer et al.
(2022): morphology, gas acceleration, time-variability, and knot formation.

##  3 Outflow formation

Figure 2 displays the molecular gas distribution at five timesteps of the
outflow formation, from $t$=0 to $t$= $2000\hbox{\kern 1.99997ptyr}$ by steps
of $500\hbox{\kern 1.99997ptyr}$. (A video is available at Fig. 2).

The jet is launched at $t$=0 into gas of density $10^{9}\hbox{\kern
1.99997ptcm${}^{-3}$}$. The density contrast between the jet
($10^{6}\hbox{\kern 1.99997ptcm${}^{-3}$}$) and the ambient gas
($10^{9}\hbox{\kern 1.99997ptcm${}^{-3}$}$) makes the shock propagate into the
ambient gas at a velocity much lower than the injection velocity $v_{j,0}$, an
effect that has been studied in detail already by Raga et al. (1990). Turning
on the jet creates a high-velocity, collimated narrow component which
propagates in the ejection direction. It is surrounded by a cocoon of gas that
expands laterally, creating a low-velocity (0–$10\hbox{\kern 1.99997ptkm\kern
1.99997pts${}^{-1}$}$) “cavity” component, with dense walls surrounding the
jet. The first working surface interacts with the very dense gas close to the
jet source, which makes it decelerate, causing the second ejected knot to
catch up with the first bowshock. As shown in Fig. 2, this process repeats a
few times until the bowshock leaves the inner envelope and propagates into the
protostellar gas, reaching its terminal velocity. It is only after $\sim
200\hbox{\kern 1.99997ptyr}$ and the launch of 3 knots that the jet manages to
drill out the inner 1000 au of the protostellar envelope and to propagate into
the surrounding gas (second panel from top). As a consequence the overpressure
resulting from the accumulation of knots pushes the protostellar envelope
laterally away from the jet, causing the formation of a wide-angle outflow
cavity, and a density increase in the low-velocity cavity walls at the base of
the jet. This implies that a kinematic age computed from the bowshock position
and velocity is actually a lower limit to the real age of the outflow.

As time proceeds, the model shows a leading jet head and a series of traveling
“internal working surfaces” (IWS). These structures form as the result of the
ejection time-variability and are observed as ”knots”, or overdensity regions
with a small size of a few $\sim 100$ au. At $t$= $1000\hbox{\kern
1.99997ptyr}$, 3 knots are easily detected along the jet, while at $t$=
$2000\hbox{\kern 1.99997ptyr}$, 8 eight knots are easily identified. It is
worth noticing they tend to expand radially as they propagate along the jet,
tracing the wings of inner bowshocks, since the slower material interacts with
the faster material ejected at later times. These wings appear to drive the
formation of complex structures inside the outflow cavity as can be seen in
the bottom panel of Fig. 2. Fast moving knots tend to accumulate at the head
of the bow, increasing the density at the tip. After $1500\hbox{\kern
1.99997ptyr}$, the knots located at distance $>15000$ au display wings with a
typical size of 1000 au.

One expects the gas density to decrease as a result of its radial expansion in
the course of the jet propagation. However, looking in detail at the jet
density field at e. g. $t$= $1000\hbox{\kern 1.99997ptyr}$ (second panel in
Fig. 2), it appears that the gas density does not decrease monotonically along
the narrow jet. The density distribution of fast material along the
propagation direction is analyzed in Section 5.2.

At $t$= $1500\hbox{\kern 1.99997ptyr}$, a second outflow cavity has now
formed. The first outflow cavity has reached a size of $\approx$ 15000 au and
expanded up to a radius of about 5000 au ($6\arcsec$ at the distance of Cep
E-mm), and the head of the outflow has reached a distance of $2.8\times
10^{4}$ au ($34\arcsec$). At later times, the second cavity expands to reach a
similar radius of $\approx 5000$ au.

Inspection of Figure 2 reveals that a complex network of relatively dense
structures ($n\sim 10^{5}$–$10^{6}\hbox{\kern 1.99997ptcm${}^{-3}$}$) forms
inside the outflow cavity very early. One can note the presence of ”filaments”
along the jet in the first $1000\hbox{\kern 1.99997ptyr}$. As can be seen in
the panel at $t$= $1000\hbox{\kern 1.99997ptyr}$, a filament has formed along
the jet. Located at $R=500$ au, it is detected up to $z=2000$ au and appears
as a thin, yellow, wiggling structure parallel to the jet. We also note the
presence of a ”shell”, or a bow, which formed in the dense protostellar
envelope and moves slowly ($\sim 20\hbox{\kern 1.99997ptkm\kern
1.99997pts${}^{-1}$}$) as it propagates over 8000au in $2000\hbox{\kern
1.99997ptyr}$, whereas the jet reached $40\times 10^{3}$ au. This dense shell
(or bow) connects the jet envelope of entrained gas to the cavity walls.

Our numerical results on the outflow morphology appear in good agreement with
the NOEMA observations of the Cep E northern lobe (Fig. 1). As can be seen in
Table 2, both the length and the radius of the outflow cavity are correctly
reproduced; this good match between observations and simulations is obtained
for a computational timescale of $1500\hbox{\kern 1.99997ptyr}$, similar to
the estimated Cep E dynamical age ($1400\hbox{\kern 1.99997ptyr}$). Even more,
the jet momentum obtained from the simulation is $3.78$ M⊙ km s-1, comparable
to the momentum obtained by Lefloch et al. (2015) of $2.5$ M⊙ km s-1, taking
into account the inclination angle.

In order to better understand the role of the core initial conditions in
shaping the outflow morphology, we also carried out simulations without the
inclusion of the high-density core given by Crimier nor the related
gravitational term. In these cases, it turned out that the low-velocity
outflows formed in the ambient medium of the jet propagation were actually far
too collimated with respect to what is observed, in particular close to the
protostar and the jet launch region. Therefore, it appears that the wide
opening of the low-velocity outflow cavity depends on the density of the dense
inner region and the contribution of the latter to the gravitational field,
two parameters that are related to the evolutionary stage of the system. The
importance of including a density distribution and a gravitational term is in
agreement with (Raga & Cabrit, 1993) and (Cabrit et al., 1997) who proposed
that a steep radial density decrease would produce a wider opening angle for
jet-driven outflows.

##  4 Gas acceleration

Figure 3: Cep E northern outflow lobe. Position-Velocity diagram of the CO J=
2–1 line emission along the jet main axis. Positions are in arcsec offset
relative to the location of the driving protostar Cep E-A. First contour and
contour interval are 10% of the peak intensity. We have superimposed in yellow
the best fitting solution $(V-V_{lsr})$/$V_{0}$= $\exp(-\delta_{0}/\delta)$
where $\delta_{0}$= 690 au (Schutzer et al., 2022).

Figure 4:  Best fitting model (M1) to the Cep E northern outflow. (top)
Density distribution at $1250\hbox{\kern 1.99997ptyr}$. The jet has propagated
over a distance of $2.4\times 10^{4}$ au and formed a low-velocity cavity of
3000 au radius. Five knots and their associated wings are detected along the
jet.(bottom) Position-Velocity diagram of the CO emission along the jet main
axis obtained using the $47^{\circ}$ outflow inclination angle with respect to
the plane of the sky.  Figure 5: Cep E northern outflow lobe. Position-
Velocity diagram of the CO J= 2–1 line emission across the jet main axis at
$\delta$= $0\arcsec$. Positions are in arcsec offset relative to the location
of the driving protostar CepE-A. First contour and contour interval are 10% of
the peak intensity (Schutzer et al., 2022).

Cep E displays evidence of gas acceleration along the jet over a distance of
1000 au away from the protostar (Fig. 3). This effect was reported and
observationally characterized by Schutzer et al. (2022). We show in the bottom
panel of Fig. 4, the synthetic Position-Velocity diagram of the CO gas along
the main axis of the jet. Two kinematical components are clearly identified:
i) a low velocity component, between $+0$ and $+10\hbox{\kern 1.99997ptkm\kern
1.99997pts${}^{-1}$}$ up to 8000 au ($10\arcsec$) from the source, associated
with the outflow cavity walls; ii) a high velocity component, that displays a
periodical behavior, reflecting the mass injection variability and the
internal shocks, where the fast material catches up with the slower material.
The jet is detected at a radial velocity close to $V_{j}$= $65\hbox{\kern
1.99997ptkm\kern 1.99997pts${}^{-1}$}$ at $z>2000$ au from the source.

Our simulations show that the effective velocity of the outflow is
significantly lower than the injection jet velocity. This appears as a result
of the jet interaction with the dense central envelope. In this case, the
Position-Velocity diagram in Fig. 4 shows that the CO gas emission accelerates
from ambient velocity ($V$= $0\hbox{\kern 1.99997ptkm\kern
1.99997pts${}^{-1}$}$) to the terminal jet velocity on a short scale $<5\times
10^{3}$ au. Close to the source, gas structures accelerated up to $V\sim
50\hbox{\kern 1.99997ptkm\kern 1.99997pts${}^{-1}$}$ are also detected along
the jet axis up to 1000–2000 au (bottom panel of Fig. 4).

An observational signature of envelope gas can also be found in the Position-
Velocity diagram of the CO $J$=2–1 emission across the jet main axis, as can
be seen in Fig. 5. The CO emission contours show how the ambient material
initially at rest at $V_{lsr}$= $-11\hbox{\kern 1.99997ptkm\kern
1.99997pts${}^{-1}$}$ is gradually accelerated as one gets closer to the
location of the protostar at $\alpha$= $0.0\arcsec$. In addition to the jet
mainly detected up to $V\sim 60\hbox{\kern 1.99997ptkm\kern
1.99997pts${}^{-1}$}$, signatures of high-velocity knots (up $+90\hbox{\kern
1.99997ptkm\kern 1.99997pts${}^{-1}$}$) are also identified close to the
protostar (Fig. 5).

It is worth remembering that the synthetic Position-Velocity diagrams use a
wide slit in the direction of the projected jet axis and the contribution of
all the possible velocity projections along the line of sight, using an angle
of $47^{\circ}$ with respect to the plane of the sky, whereas the Cep E
observational plots (Fig. 3) display the radial component of the jet velocity.
For the sake of comparison with the Cep E jet, we have checked whether the
solution found by Schutzer et al. (2022) ($V-V_{lsr})/V_{j}$=
$\exp(-\delta_{0}/\delta)$ could provide a reasonable fit to the synthetic jet
velocity profile along the main axis. The yellow curve drawn in the bottom
panel of Fig. 4 traces the best fitting solution of Schutzer et al. obtained
for a length scale $\delta_{0}$= 690 au, applied to the radial jet velocity
$V_{r}$=$65\hbox{\kern 1.99997ptkm\kern 1.99997pts${}^{-1}$}$. Taking into
account the inclination of the jet with respect to the line of sight, the
agreement between numerical and observational jet velocity profiles is very
satisfying both qualitatively and quantitatively.

To conclude, our simulation satisfyingly accounts for the acceleration of
material from the protostellar envelope by the Cep E jet, from ambient
velocity up to reaching the radial terminal jet velocity $V_{j}\sim
90\hbox{\kern 1.99997ptkm\kern 1.99997pts${}^{-1}$}$. This process occurs over
a scale of 5000 au.

##  5 Mass-loss history

###  5.1 The jet

Figure 6: Model M1. Outflow mass evolution as a function of time.

We first measured the mass of outflowing material as a function of time. The
mass integration was performed over the whole velocity range. The result is
displayed in Fig. 6 and shows that the outflow mass increases linearly with
time at a rate of $1.6\times 10^{-3}\hbox{\kern
1.99997pt$M_{\odot}$}\hbox{\kern 1.99997ptyr${}^{-1}$}$. At a time of
$1500\hbox{\kern 1.99997ptyr}$ in the simulation (similar to the dynamical age
of Cep E) the mass of outflowing gas amounts to $2.3\hbox{\kern
1.99997pt$M_{\odot}$}$, a value in good agreement with the observational
determination in the northern lobe (see Table 2).

Figure 7: Model M1. Jet mass evolution as a function of time. The points are
obtained from the simulation every 100 yr. The black continous line is a
linear fit of the points starting from 300 yr, with a slope of 2.3 $\times
10^{-5}M_{\odot}$ yr-1 and the dashed line corresponds to the material
injection rate of $\dot{M}=7.3\times 10^{-6}$ used in the simulation.

In a second step, we have studied the variations of the jet mass as a function
of time. We took into account all the gas at velocity $V>50\hbox{\kern
1.99997ptkm\kern 1.99997pts${}^{-1}$}$ to estimate the jet mass and follow its
variations. The variations of the jet mass as a function of time are displayed
in Fig. 7. After an initial delay of $\sim 200\hbox{\kern 1.99997ptyr}$, which
corresponds to the time needed for the knots to drill the envelope, the jet
mass increases linearly with time, at a rate $\dot{M}$= $2.3\times
10^{-5}\hbox{\kern 1.99997pt$M_{\odot}$}\hbox{\kern 1.99997ptyr${}^{-1}$}$ in
very good agreement with the observational determination by Schutzer et al.
(2022) ($2.7\times 10^{-5}\hbox{\kern 1.99997pt$M_{\odot}$}\hbox{\kern
1.99997ptyr${}^{-1}$}$). As discussed previously in Sect. 3.2, the
protostellar material appears to feed the high-velocity jet through an
entrainment process. An observational signature of this effect is found in the
Position-Velocity diagram of the CO $J$=2–1 emission across the jet main axis,
displayed in Fig. 5. The CO emission contours show how the ambient material
initially at rest at $V_{lsr}$= $-11\hbox{\kern 1.99997ptkm\kern
1.99997pts${}^{-1}$}$ is gradually accelerated as one gets closer to the
location of the protostar at $\alpha$= $0.0\arcsec$. In addition to the jet
mainly detected up to $V\sim 60\hbox{\kern 1.99997ptkm\kern
1.99997pts${}^{-1}$}$, signatures of higher-velocity knots (up $+90\hbox{\kern
1.99997ptkm\kern 1.99997pts${}^{-1}$}$) are also identified close to the
protostar (Fig. 5).

Third, in order to quantify the magnitude of the entrainment effect to the jet
mass, we have disentangled the contributions of the injected material from the
entrained material. Theoretically, the jet mass injection rate per unit of
time can be estimated as a function of the injected material density $n_{j}$,
the jet velocity $v_{j}$ and the jet injection radius $r_{j}$:

$\left[\frac{\dot{M}}{{\rm
M}_{\odot}/\rm{yr}}\right]=1.5\times{10^{-6}}\left[\frac{r_{j}}{50\rm{au}}\right]^{2}\left[\frac{n_{j}}{10^{6}\hbox{\kern
1.99997ptcm${}^{-3}$}}\right]\left[\frac{v_{j}}{165\hbox{\kern
1.99997ptkm\kern 1.99997pts${}^{-1}$}}\right].$ (4)

Numerically, the jet mass injection rate in the computational domain is in
reasonable agreement with the simple, theoretical description from Eq. 4
($\dot{M}=7.3\times 10^{-6}\hbox{\kern 1.99997pt$M_{\odot}$}\hbox{\kern
1.99997ptyr${}^{-1}$}$ with the conditions for model M1). In both numerical
and theoretical cases, the jet mass injection rate is lower than the jet mass
rate $\dot{M}$ of the simulation by a factor of $\approx 3$. We consider the
difference between both values as very significant as it highlights the
importance of the entrainment process of circumstellar material in the
formation of molecular jets.

###  5.2 The knots

We now examine the physical properties of the knots which form along the jet
in the numerical simulation and their evolution with time. We then compare
them with their observational counterparts in the Cep E jet.

####  5.2.1 Formation and evolution

Figure 8: Gas density distribution along the jet main axis for different
snapshots from 1450 to 1575 yr (dashed lines). Knots can be identified as
overdensities in the density distribution (thick lines) in the order of
$10^{5}$ cm-3. To follow their behavior some knots have been labeled as Ka,
Kb, Kc and Kd.

We have studied the gas density distribution along the jet as a function of
time in the simulation up to $2000\hbox{\kern 1.99997ptyr}$. We found that the
evolution is characterized by very short timescales on the order of
$25\hbox{\kern 1.99997ptyr}$. In order to illustrate and better explain the
process at work, we present here in Fig. 8 a montage of the jet density
profiles at intervals of $25\hbox{\kern 1.99997ptyr}$ over a time of
$125\hbox{\kern 1.99997ptyr}$, between $1450\hbox{\kern 1.99997ptyr}$ and
$1575\hbox{\kern 1.99997ptyr}$. The mean density distribution appears steady
and constant at distances further than $10^{4}$ au, while at shorter
distances, there is a decreasing density profile. This is a fossil signature
of the steep initial density profile of the envelope, since it is expected to
be difficult to remove the dense envelope material close to the source, even
when it has been processed by the countinuous injection of the jet. The peaks,
which appear on top of the mean density distribution, result from the ejection
velocity variability in the jet source (Sect. 2.3), which causes fast material
to shock with lower velocity gas. Each of these peaks corresponds to a local
overdensity structure in the jet, whose properties were derived from a
gaussian profile fitting, leading to a typical size of 1000 au, a density of
about $10^{5}\hbox{\kern 1.99997ptcm${}^{-3}$}$, and a typical mass of
$10^{-3}\hbox{\kern 1.99997pt$M_{\odot}$}$. The physical properties of these
peaks are actually very similar to those of the CO knots reported by Schutzer
et al. (2022) in the Cep E jet, and, for that reason, they are probably their
counterparts in the numerical simulation. Based on Fig. 8, it appears that at
distances larger than $10^{4}$ au the density and the knot distributions are
almost steady, which reflects the fact that the knots are propagating into the
low-density material of the jet at a speed of about 525 au in $25\hbox{\kern
1.99997ptyr}$. The high density in the inner $10^{4}$ au makes the situation
drastically different. Rivera-Ortiz et al. (2019) have shown that gas flowing
into an environment of similar density can create reverse shocks that
propagates at a significant fraction of the original velocity relatively to
the shock velocity. This is precisely the present situation with a jet of
density $10^{6}\hbox{\kern 1.99997ptcm${}^{-3}$}$ propagating into gas of
density 2–$4\times 10^{5}\hbox{\kern 1.99997ptcm${}^{-3}$}$. This effect is
seen in Fig. 8, where the number of peaks varies in the inner $10^{4}$au. As
an example we follow the formation of the knot labeled Kbc: at
$t=1450\hbox{\kern 1.99997ptyr}$ Kc looks as a short and wide peak, that gets
taller and thiner up to $1500\hbox{\kern 1.99997ptyr}$, when it appears to
have merged with Kc. Immediatly after this, a small peak Kd forms behind it at
$1550\hbox{\kern 1.99997ptyr}$, which moves forward at a slightly lower
velocity, creating a very wide peak and giving a similar density distribution
at 1450 yr and 1575 yr, an interval of $125\hbox{\kern 1.99997ptyr}$, which is
approximately the period of variability used in the simulation. This process
gets repeated several times, accelerating the fossil envelope material, as
described in Sec. 4, relating the process of gas acceleration with the
formation of knots.

We have reported in Fig. 9 the mass of the knots as a function of their
location along the jet at $t$= $1500\hbox{\kern 1.99997ptyr}$. It appears that
the knots tend to increase their mass by a factor of 2 as they move away from
the injection site. The simulation shows that the mass increase occurs on a
very short length scale as the second knot at $z\sim 3000$ au has already a
mass of $1.7\times 10^{-3}\hbox{\kern 1.99997pt$M_{\odot}$}$. Beyond 8000 au,
all knots display rather similar and steady masses, $\sim 1.7\times
10^{-3}\hbox{\kern 1.99997pt$M_{\odot}$}$. The terminal knot is an exception
due to its higher mass ($>3\times 10^{-3}\hbox{\kern 1.99997pt$M_{\odot}$}$).
This high value is consistent with the fact that several knots have already
reached the tip of the jet and are accumulating there, as can be seen in
panels at t=1500 and $2000\hbox{\kern 1.99997ptyr}$ in Fig. 2.

####  5.2.2 Observational comparison

In the later times of the simulation, it appears that the spatial separation
between the knots located close to the source is actually shorter by about a
factor 0.5 than the separation between those located in the older region of
the jet. This is consistent with the bimodal distribution of dynamical ages
reported by Schutzer et al. (2022).

Figure 9: Mass of the knots detected at 1500 yr in Model M1 as a function of
their position $z$ along the jet, projected by 47∘, which corresponds to the
angle between the Cep E jet main axis and the plane of the sky. The dashed
line corresponds to the mass injected to the simulation in a single period.

The physical parameters of the knots (size, mass, density) such as derived
from the numerical simulation, are therefore in good agreement with the
observations. Schutzer et al. (2022) noticed that the distribution of knot
dynamical ages appeared to be bimodal, with the presence of two timescales: a
short time interval of 50–$80\hbox{\kern 1.99997ptyr}$ close to the protostar
(8000 au), corresponding to the first 4–5 knots, and a longer timescale
(150–-$200\hbox{\kern 1.99997ptyr}$) at larger distances from the protostar.

Interestingly, the numerical simulation reports the same trend, as can be seen
in Fig. 10, in which the ages of all the knots identified along the jet have
been reported. The first four younger knots are separated by a timescale of
about $100\hbox{\kern 1.99997ptyr}$, whereas the following knots are separated
by a timescale of about $200\hbox{\kern 1.99997ptyr}$, or twice the value
measured close to the jet launch region. Our simulations show that a high-
velocity knot can overcome the spatial separation to the previous ejecta,
which was slown down as a result of the interaction with the dense gas of the
protostellar envelope. The typical timescale for the collision is
$500\hbox{\kern 1.99997ptyr}$ (Fig. 10).

We propose that the bimodal distribution is actually the signature of the
interaction of the knots with the high-density gas inside the shell. As soon
as the knots run out of the shell and propagate into lower density gas, they
encounter free motion conditions.

Figure 10: Dynamical age of the knots identified in model M1 at $t$=
$1500\hbox{\kern 1.99997ptyr}$ in terms of a multiple number $n_{i}$ of the
shorter dynamical age.

##  6 CO emission

Thanks to the detailed modeling of the CO chemistry by Walkimya-2D, we could
produce synthetic CO integrated emissivity maps of the outflow jet and cavity,
assuming LTE and optically thin emission. As an example, Fig. 11 displays the
CO emissivity of the low-velocity outflow cavity integrated between 3 and
$7\hbox{\kern 1.99997ptkm\kern 1.99997pts${}^{-1}$}$ (black contours) and of
the high-velocity jet, integrated between 50 and $150\hbox{\kern
1.99997ptkm\kern 1.99997pts${}^{-1}$}$ (blue contours), as calculated by Model
M1 at $1500\hbox{\kern 1.99997ptyr}$. The CO image reveals the previously
identified highly collimated structure of the jet with a typical width of
$\sim 1000$ au, which slightly increases with distance from the source. An
important result is that the CO emission both from the high-velocity
(50-$150\hbox{\kern 1.99997ptkm\kern 1.99997pts${}^{-1}$}$) jet (blue
contours) and the low-velocity (3-$7\hbox{\kern 1.99997ptkm\kern
1.99997pts${}^{-1}$}$) cavity (black contours) drops $\sim 2000$ au before the
terminal bowshock, whose location is marked by a red spot in the plot. The
lack of low-velocity CO emission over a few 1000 au near the apex of the
outflow cavity is consistent with the terminal bowshock being very efficient
at dissociating the ambient molecular gas.

Figure 11: Synthetic map of Model M1 at $t$= $1500\hbox{\kern 1.99997ptyr}$
considering a projection angle with respect to the plane of the sky of 47∘, as
observed in Cep E-mm . CO J= 2–1 integrated emissivity contour maps of the
cavity (Black contours, 3 to 7 km s-1) and the jet (Blue contours, 50 to 150
km s-1) components. Scaling is logarithmic. The response of the IRAM
interferometer was modeled by a gaussian of 830 au diameter (FWHM),
corresponding to a beam size of $1\arcsec$ (FWHP) at the distance of Cep E-mm,
which is drawn with a gray disk (bottom left). The location of the frontal
shock is marked with a red point. Figure 12: Model M1. Variations of the CO/H2
abundance relative to its initial value ($10^{-4}$) along the high-velocity
jet at 4 different times of the simulation: 0, 500 (dotted), 1000 (solid
black), $1500\hbox{\kern 1.99997ptyr}$ (solid red).

In order to gain more insight into the time variations of the CO abundance
(relative to H2) in the outflow, we have computed the CO abundance
distribution along the high-velocity jet at 4 different times of the
simulation in model M1: 0, $500\hbox{\kern 1.99997ptyr}$, $1000\hbox{\kern
1.99997ptyr}$, and $1500\hbox{\kern 1.99997ptyr}$. The distributions are shown
in Fig. 12. For the sake of simplicity, we have reported the ratio $\rm
X_{CO}$ of the CO abundance to its initial value, taken equal to $10^{-4}$, a
standard value in dense interstellar clouds (Lacy et al., 1994).

In the very early stages of the simulation (t ¡ $500\hbox{\kern
1.99997ptyr}$), when the jet has run only a few $1000$ au, XCO is well below
its canonical value of 1.0. For instance, we measure $\rm X_{CO}\approx 0.5$
at $5000$ au from the protostar at $t=500\hbox{\kern 1.99997ptyr}$. At later
times in the simulation ($1000\hbox{\kern 1.99997ptyr}$ and $1500\hbox{\kern
1.99997ptyr}$ in model M1, see Fig. 12), two regimes are identified in the
$\rm X_{CO}$ distribution observed along the jet: The first regime is close to
the jet launch region $\rm X_{CO}$= 1. The length of this region increases
with time (2000 au at t=$1000\hbox{\kern 1.99997ptyr}$, 10000 au at
t=$1500\hbox{\kern 1.99997ptyr}$) as the jet propagates at velocity Vj=
$140\hbox{\kern 1.99997ptkm\kern 1.99997pts${}^{-1}$}$. The second regime, at
a larger distance, $\rm X_{CO}$ decreases from 1 to a few $10^{-1}$ with
increasing distance to the protostar. The decrease is rather smooth and does
not display abrupt variations along the jet. The minimum value is reached at
the terminal bowshock before $\rm X_{CO}$ jumps abruptly again to 1, which
stands as the canonical value in the ambient, quiescent gas in which the jet
propagates.

To summarize, our simulation shows that CO is partly destroyed in the early
times of the jet launch, close to the protostar ($\rm X_{CO}$ ¡ $10^{-1}$).
The destruction process appears to stop $\approx 900\hbox{\kern 1.99997ptyr}$
after the beginning of the simulation and the CO abundance of the ejected
material remains steady, equal to its initial value. We propose that CO
dissociation occurs as a consequence of the violent shocks caused by the first
high-velocity knots (and the jet) when they are launched and impact the dense,
protostellar envelope at the beginning of the simulation. It is only when the
knots finally drill out the envelope and escape out of the inner protostellar
region, entraining a fraction of the ambient protostellar envelope, that the
local value of $\rm X_{CO}$ tends to return to its initial value. This process
can also be influenced by the gas entrainment process.

Interestingly, we note that the slope of the $\rm X_{CO}$ distribution along
the jet at $t$= $1000\hbox{\kern 1.99997ptyr}$ and $t$= $1500\hbox{\kern
1.99997ptyr}$ is similar, suggesting that it does not vary with time. Also,
the velocity variations between subsequent knots ($\sim 10\hbox{\kern
1.99997ptkm\kern 1.99997pts${}^{-1}$}$) do not appear to significantly affect
the $\rm X_{CO}$ distribution. Therefore it seems that the jet kept the memory
of the initial launch process in the $\rm X_{CO}$ axial distribution.

This numerical result has several implications on the observational side.
First, lower values of $\rm X_{CO}$ may actually reflect the conditions of the
jet formation process. Of course, precession and knot interaction with the
ambient gas (or the cavity) may alter this conclusion. We speculate that in
the latter case, the $\rm X_{CO}$ variations occur on a much smaller length
scale, corresponding to the size of the knots, i.e. 1000 au typically. Our
model suggests that the distribution of $\rm X_{CO}$ along the jet could
probably provide constraints robust enough to discriminate between the
processes at work.

On the observational side, Gusdorf et al. (2017) studied the emission of the
OI $63\mu m$ line at $\sim 6\arcsec$ resolution with the Stratospheric
Observatory For Infrared Astronomy (SOFIA) and detected both the signature of
the Cep E southern jet and the terminal bowshock HH377. A column density ratio
N(OI)/N(CO)$\sim 2.7$ was measured in the jet, indicating that that the jet is
essentially atomic in the region of HH377. The authors proposed that the OI
emission could arise from dissociative J-type shocks or with a radiative
precursor, caused by the knots propagating at a different velocity in the jet
(Lehmann et al., 2020). Interestingly, no signature of OI was detected in the
jet toward shock position BI, located halfway between the protostar and HH377.

As discussed above, our numerical simulations propose an alternative
explanation for the origin of OI in the Cep E jet. A detailed map of the OI
emission along the southern jet with SOFIA, would help to confirm whether the
CO dissociation is localized only toward HH377 or if it is also present along
the jet, hence could trace the history of the early stages of the mass-
ejection process.

##  7 Conclusions

Using the reactive hydrodynamical code Walkimya-2D, which includes a chemical
network based on CO(Castellanos-Ramírez et al., 2018), we have carried out a
set of simulations in order to reproduce the morphology and the physical
properties of the molecular jet-driven outflow of the intermediate-mass
protostellar source Cep E, as derived from previous observational studies
(Lefloch et al., 2015; Gusdorf et al., 2017; Ospina-Zamudio et al., 2019;
Schutzer et al., 2022). Outflow precession was not considered in this work. We
have obtained a very satisfying agreement, both qualitatively and
quantitatively, with the observations when modeling a time-variable jet with
initial density $n(H)$= $10^{6}\hbox{\kern 1.99997ptcm${}^{-3}$}$, radius
$r_{j}$= 100 au, temperature $T_{j}$= $300\hbox{\kern 1.99997ptK}$, velocity
$V_{j}$= $200\hbox{\kern 1.99997ptkm\kern 1.99997pts${}^{-1}$}$ propagating
into the density stratified protostellar envelope as modeled by Crimier et al.
(2010). Our main results are as follows:

  * •

The jet and cavity morphologies (width and length) are consistent with the
observations and the expected kinematics using a variable jet ejection $\delta
V/V_{j}$= $0.08$. The best fitting solution is obtained at a numerical
timescale $t_{dyn}\simeq 1500\hbox{\kern 1.99997ptyr}$, consistent with the
jet dynamical timescale estimated observationally (Schutzer et al., 2022,
$\sim 1400\hbox{\kern 1.99997ptyr}$).

  * •

The jet terminal velocity ($\simeq 90\hbox{\kern 1.99997ptkm\kern
1.99997pts${}^{-1}$}$) is different from the injection velocity $V_{j}$
($200\hbox{\kern 1.99997ptkm\kern 1.99997pts${}^{-1}$}$), as a result of the
first ejections being decelerated by the dense envelope and the entrainment of
a layer of ambient protostellar material. It implies that the jet dynamical
timescale is actually lower than the duration of the ejection phase.

  * •

The jet acceleration reported observationally by Schutzer et al. (2022) is
consistently reproduced in the simulations. It appears to be the result of
ambient material entrainment by the knots on a length scale of $\sim 700$ au,
in agreement with the observational data.

  * •

We reproduce the properties of the knots along the jet assuming a periodic
ejection. We found evidence for knot interactions in the dense inner
protostellar region, where the densities of the local gas and the jet are
comparable, which lead to the formation of secondary shocks in the close
protostellar environment. At larger distances from the protostar, the lower
ambient gas density allows the knots to propagate freely, without significant
interaction. We propose that this process could account for the bimodal
distribution of knots observed along the Cep E jet. Knots have a typical size
of 1000-2000 au, with a mass of $\sim 1.5\times 10^{-3}\hbox{\kern
1.99997pt$M_{\odot}$}$, and density of $10^{5}\hbox{\kern
1.99997ptcm${}^{-3}$}$. The mass carried away by the knots in the jet
translates into a steady ejection mass rate of $2.3\times 10^{-5}\hbox{\kern
1.99997pt$M_{\odot}$}\hbox{\kern 1.99997ptyr${}^{-1}$}$, a factor of 3 higher
than the mass injection rate into the jet ($7.3\times 10^{-6}\hbox{\kern
1.99997pt$M_{\odot}$}\hbox{\kern 1.99997ptyr${}^{-1}$}$). This difference is
the signature of the entrainment and the subsequent acceleration of ambient
material by the jet.

  * •

The shock interaction of the jet knots with the protostellar envelope in the
early times of the simulation lead to the dissociation of CO. The destruction
process appears to stop after $\approx 900\hbox{\kern 1.99997ptyr}$, time from
which the CO abundance of the ejected material remains steady and equal to its
initial (canonical) value. As a consequence, the older part of the jet is
characterized by a lower CO gas abundance, which gradually decreases as one
moves closer to the jet head.

Our simulations underline the importance of mass-ejection time variability in
the molecular outflow formation process and its interaction with the
protostellar envelope. More work should be done, both observationally and
numerically, in order to investigate the role of knots and their importance in
the dynamical evolution of other young protostellar systems. Interferometric
observations at subarcsec angular resolution of Cep E and other young
protostellar jets should be undertaken as they would allow to make a
significant step forward by bringing extremely useful constraints on the knot
internal structure and the dynamical processes at work in the jet. In
parallel, the high spatial resolution accessible in the Walkimya-2D numerical
simulations ($\sim 10$au) gives access to a novel view on the structure of
protostellar jets and their dynamics, which will help interpret a new harvest
of observations.

## Acknowledgments

PR-RO, AS, BL acknowledge support from a) the European Union’s Horizon 2020
research and innovation program under the Marie Skłodowska-Curie grant
agreement No 811312 for the project “Astro-Chemical Origins” (ACO); b) the
European Research Council (ERC) under the European Union’s Horizon 2020
research and innovation program for the Project “The Dawn of Organic
Chemistry” (DOC) grant agreement No 741002; c) the UNAMPAPIIT grant IN110722.
Some of the computations presented in this paper were performed using the
GRICAD infrastructure (https://gricad.univ-grenoble-alpes.fr), which is partly
supported by the Equip@Meso project (reference ANR-10-EQPX-29-01) of the
programme Investissements d’Avenir supervised by the Agence Nationale de la
Recherche and the Miztli-UNAM supercomputer, project LANCADUNAM-DGTIC-123
2022-1.

## References

  * Arce et al. (2007) Arce, H. G., Shepherd, D., Gueth, F., et al. Protostars and Planets V, B. Reipurth, D. Jewitt, and K. Keil (eds.), University of Arizona Press, Tucson, 2007, p.245-260
  * Bachiller & Pérez-Gutiérrez (1997) Bachiller, R., & Pérez-Gutiérrez, M. 1997, ApJ, 487, L93
  * Bachiller et al. (2001) Bachiller, R., Pérez-Gutiérrez, M., Kumar, M. S. N., et al. 2001, A&A, 372, 899
  * Bachiller (1996) Bachiller, R., 1996, ARA&A, 34, 111
  * Bally, Reipurth, & Davis (2007) Bally J., Reipurth B., Davis C. J., 2007, prpl.conf, 215
  * Bally (2016) Bally J., 2016, ARA&A, 54, 491
  * Cabrit et al. (1997) Cabrit, S., Raga, A., Gueth, F., in Herbig-Haro Flows and the Birth of Stars; IAU Symposium No. 182, Eds. Bo Reipurth and Claude Bertout. Kluwer Academic Publishers, 1997, p. 163-180
  * Canto (1985) Canto, J. 1985, Cosmical Gas Dynamics, 267
  * Castellanos-Ramírez et al. (2018) Castellanos-Ramírez, A., Rodríguez-González, A., Rivera-Ortíz, P. R., et al., 2018, RMxAA, 54, 409.
  * Crimier et al. (2010) Crimier, N., Ceccarelli, C., Alonso-Albi, T., et al., 2010, A&A, 516, 102
  * De Simone et al. (2020) De Simone, M., Codella, C., Ceccarelli, C., et al., 2020, A&A, 640, 75
  * Eisloffel et al. (1996) Eisloeffel, J., Smith, M. D., Davis, C. J., et al., 1996, AJ, 112, 2086.
  * Frank et al. (2014) Frank, A., Ray, T., Cabrit, S., et al., 2014, Protostars and Planets VI, eds H. Beuther, R.S. Klessen, C.P. Dullemond & T. Henning Tucson, AZ: University of Arizona Press, 451
  * Gusdorf et al. (2017) Gusdorf A., Anderl S., Lefloch B., et al., 2017, A&A, 602, 8
  * Hughes (1997) Hughes, V. A. 1997, Herbig-Haro Flows and the Birth of Stars, 182, 141
  * Ivezic & Elitzur (1997) Ivezić, Z., Elitzur, M., 1997, MNRAS, 287, 799
  * Jin et al. (2015) Jin, M., Lee, J.-E.E, Kim, K.-T., 2015, ApJS, 219, 2
  * Karnath et al. (2019) Karnath N., Prchlik J. J., Gutermuth R. A., et al., 2019, ApJ, 871, 46.
  * Konigl & Pudritz (2000) Konigl, A. & Pudritz, R. E. 2000, Protostars and Planets IV, 759
  * Lacy et al. (1994) Lacy, J. H., Knacke, R., Geballe, T. R., et al., 1994, ApJ, 428, L69
  * Lefloch, Eisloeffel & Lazareff (1996) Lefloch B., Eisloeffel J., Lazareff B., 1996, A&A, 313, L17
  * Lefloch et al. (2015) Lefloch, B., Gusdorf, A., Codella, C., et al. 2015, A&A, 581, A4
  * Lefloch et al. (2017) Lefloch, B., Ceccarelli, C., Codella, C., et al., 2017, MNRAS, 469, L73
  * Lehmann et al. (2020) Lehmann, A., Godard, B., Pineau des Forêts, G., et al. 2020, Uncovering Early Galaxy Evolution in the ALMA and JWST Era, 352, 73. doi:10.1017/S1743921320000654
  * Masciadri & Raga (2002) Masciadri, E., Raga, A. C., 2002, ApJ, 568, 733
  * McElroy et al. (2013) McElroy, D., Walsh, C., Markwick, A.J., et al., 2013, A&A, 550, 36
  * Milam et al. (2005) Milam, S. N., Savage, C., Brewster, M. A., et al., 2005 ApJ, 634, 1126
  * Nony et al. (2020) Nony, T., Motte, F., Louvet, F., et al., 2020, 636, A&A, 38
  * Noriega-Crespo et al. (2014) Noriega-Crespo, A., Raga, A. C., Moro-Martín, A., et al., 2014, New Journal of Physics, 16, 10
  * Ospina-Zamudio et al. (2018) Ospina-Zamudio, J., Lefloch, B., Ceccarelli, C., et al., 2019, A&A, 618, 145
  * Ospina-Zamudio et al. (2019) Ospina-Zamudio, J., Lefloch, B., Favre, C., et al., 2019, MNRAS, 490, 2679
  * Plunkett et al. (2015) Plunkett A. L., Arce H. G., Mardones D., et al., 2015, Nature, 527, 70
  * Rabenanahary et al. (2022) Rabenanahary, M., Cabrit, S., Meliani, Z., et al. 2022, A&A, 664, A118. doi:10.1051/0004-6361/202243139
  * Raga et al. (1990) Raga, A.C., Cantó, J., Binette, Calvet, N., 1990, ApJ, 364, 601
  * Raga & Cabrit (1993) Raga A., Cabrit S., 1993, A&A, 278, 267
  * Raga et al. (2003) Raga, A.C., Cantó, J., 2003, A&A, 412, 745
  * Reipurth & Raga (1999) Reipurth, B. & Raga, A. C. 1999, The Origin of Stars and Planetary Systems, 540, 267
  * Rodríguez-González et al. (2022) Rodríguez-González, A., Rivera-Ortiz, P. R., Castellanos-Ramírez, A., et al. 2023, MNRAS, 519, 4818. doi:10.1093/mnras/stac3793
  * Rivera-Ortiz et al. (2019) Rivera-Ortiz, P. R., Rodríguez-González, A., Hernández-Martínez, L., et al., 2019, ApJ, 874, 38
  * Santiago-García et al. (2009) Santiago-García, J., Tafalla, M., Johnstone, D., et al. 2009, A&A, 495, 169
  * Schutzer et al. (2022) Schutzer, A. de A., Rivera-Ortiz, P. R., Lefloch, B., et al. 2022, A&A, 662, 104
  * Smith (1994) Smith, M. D., 1994, A&A, 289, 256
  * Smith et al. (1997) Smith, M. D., Suttner, G., & Yorke, H. W. 1997, A&A, 323, 223
  * Zier et al. (2021) Zier, O., Burkert, A., Alig, C., 2021, ApJ, 915, 7
  * Zucker et al. (2019) Zucker, C., Speagle, J. S., Schlafly, E. F. et al. 2019, ApJ, 879, 125


# ED-FAITH: Evaluating Dialogue Summarization on Faithfulness

Sicong Huang Asli Celikyilmaz Haoran Li
Meta AI
<EMAIL_ADDRESS>
{aslic<EMAIL_ADDRESS>
Work done during AI Residency at Meta

###### Abstract

Abstractive summarization models typically generate content unfaithful to the
input, thus highlighting the significance of evaluating the faithfulness of
generated summaries. Most faithfulness metrics are only evaluated on news
domain, can they be transferred to other summarization tasks? In this work, we
first present a systematic study of faithfulness metrics for dialogue
summarization. We evaluate common faithfulness metrics on dialogue datasets
and observe that most metrics correlate poorly with human judgements despite
performing well on news datasets. Given these findings, to improve existing
metrics’ performance on dialogue summarization, we first finetune on in-domain
dataset, then apply unlikelihood training on negative samples, and show that
they can successfully improve metric performance on dialogue data. Inspired by
the strong zero-shot performance of the T0 language model, we further propose
T0-Score – a new metric for faithfulness evaluation, which shows consistent
improvement against baseline metrics across multiple domains.

##  1 Introduction

Abstrative text summarization aims to condense a piece of text into a shorter
version by distilling the information in the source text and rewriting it in a
concise manner. Recent advancements in pretrained language models (Vaswani et
al., 2017; Brown et al., 2020; Raffel et al., 2020; Lewis et al., 2020; Zhang
et al., 2020) have enabled summarization systems to generate highly fluent and
coherent summaries on common summarization datasets such as CNN/DailyMail
(Nallapati et al., 2016), XSum (Narayan et al., 2018).

Although news domain has long been the focus of summarization research,
dialogue summarization, with many practical applications has gained more
research attention lately. SAMSum (Gliwa et al., 2019) is the first large-
scale dataset with human annotated abstractive summaries for chat-dialogue
conversations. ConvoSumm (Fabbri et al., 2021a) created the first
comprehensive benchmark for conversational summarization across diverse
domains of news comments, discussion forums, community question answering
forums, and email threads.

Success in a summarization system’s application is crucially dependent on its
faithfulness: the factual alignment between the generated summary and the
source. Kryscinski et al. (2020) found that up to 30% of generated summaries
are affected by factual inconsistencies. Tang et al. (2022) studied types of
factual errors generated by current models on popular dialogue summarization
dataset and revealed hallucination issues. Thus having metrics that can
reliably identify hallucinations and source-contradicting information becomes
a critical step in summarization research.

Although commonly used to evalaute summarization models, metrics based on
n-gram overlaps, such as ROUGE (Lin, 2004), BLEU (Papineni et al., 2002) and
METEOR (Lavie and Agarwal, 2007) are inadequate to measure a summary’s
faithfulness, because of their low correlation with human judgements (Yuan et
al., 2021; Deng et al., 2021; Maynez et al., 2020). Even though many newly
proposed metrics specifically target faithfulness (Kryscinski et al., 2020;
Wang et al., 2020; Yuan et al., 2021; Deng et al., 2021) they are only
evaluated on the news domain. Because of the unique challenges inherent to
dialogue data, i.e. spoken terms, special discourse structures, coreferences
and ellipsis, etc. (Chen et al., 2021), we suspect they will not be as
effective in evaluating dialogue summaries right out-of-the-box.

In this paper, we evaluate the faithfulness aspect of common summarization
metrics in the dialogue domain and investigate various approaches to improve
their reliabilities. Our contributions are:

  1. 1.

We analyze faithfulness metrics for dialogue summarization and find that
metrics that perform well on one domain don’t transfer directly to another
domain.

  2. 2.

We improve existing faithfulness evaluation metrics for dialogue summarization
by finetuning on in-domain data and applying unlikelihood training with
negative samples.

  3. 3.

We propose a more general faithfulness metric, T0-Score, and show that it
outperforms many metrics across multiple domains.

##  2 Related Work

###  2.1 Summarization Metrics

Common summarization metrics ROUGE, BLEU, and METEOR (Lin, 2004; Papineni et
al., 2002; Banerjee and Lavie, 2005) are based on n-gram overlaps, which were
shown to not correlate well with human judgements of factual consistency
(Falke et al., 2019; Kryscinski et al., 2020). Newly proposed metrics aim to
solve this problems with various approaches: FactCC (Kryscinski et al., 2020)
generates weakly supervised training data and treat consistency evaluation as
an NLI task. QAGS and FEQA (Wang et al., 2020; Durmus et al., 2020) model
consistency evaluation as question generation and answering. BERTScore (Zhang*
et al., 2020) uses contextualized embeddings to compute similarities between
the generated and the reference summaries. CTC (Deng et al., 2021) introduces
information alignment that measure consistency at the token level. BARTScore
(Yuan et al., 2021) formulates evaluation of generated text as a text
generation problem.

###  2.2 Meta-Evaluation of Summarization

Maynez et al. (2020) conducted a human evaluation of hallucinated content in
system generated summaries on XSum data and found textual entailment scores
are best correlated with summary faithfulness. Fabbri et al. (2021b) assembled
SummEval, a collection of model-generated CNNDM summaries and their human
judgement scores along four axes including factual consistency. They found, on
system-level, metrics using higher-order n-gram overlap such as ROUGE-3 are
more effective. Similarly, Pagnoni et al. (2021) composed FRANK, consists of
generated summaries from CNNDM and XSum and corresponding human annotations of
factuality based on a typology of factual errors. They observe all tested
metrics exhibit low correlations with human judgements with the best metric
FactCC achieving 0.3 Spearman correlation. Gabriel et al. (2021) proposed a
meta-evaluation framework, GO Figure, which evaluates the sensitivity and
validity of factual consistency metrics with only reference summaries. They
found SummaQA, ROUGE-(2/3), and BERTScore perform better than ROUGE(1/L).

##  3 Metrics and Data

#### Baseline Metrics

We compare 4 summarization metrics that are either widely used or shown to
correlate well with human judgement in terms of faithfulness in news domain
and seek further improvement of automatic metrics for faithfulness on dialogue
summarization.

ROUGE (Lin, 2004) measures summarization quality by counting n-gram overlaps
between the hypothesis and the human written reference summary.

BERTScore (Zhang* et al., 2020) compares the similarity between the hypothesis
and the reference by measuring the cosine similarities between each of their
token’s contextualized embedding.

CTC (Deng et al., 2021), when evaluating consistency with its discriminative
model, works by predicting the probability of each token in the hypothesis
being consistent with the source. Training requires constructing negative
training samples.

BARTScore (Yuan et al., 2021) when used to evaluate factuality, works by
conditioning on the source and predicting the probability of a hypothesis
being generated with a seq2seq language model

#### Datasets

We use SAMSum (Gliwa et al., 2019) dataset to study faithfulness metrics for
dialogue summarization. The SAMSum corpus is a large-scale dialogue
summarization dataset that contains 16k English daily conversations with
corresponding summaries written by linguists. We use the human annotation of
SAMSum summaries in ConFiT (Tang et al., 2022) as our meta-evaluation dataset,
where they generate summaries from six summarization models and collected
faithfulness score on a scale of 1-10. We refer to this dataset as MetaSAMSum.

##  4 Methods

During evaluation, we found that off-the-shelf CTC and BARTScore perform
poorly on MetaSAMSum (see "Vanilla" column of Table 1). We hypothesize that
the low performance of CTC and BARTScore on the SAMSum dataset is twofold. 1\.
Neither of them are trained on dialogue data and their performance transfer
poorly with shifted domain. 2\. BARTScore has never seen negative samples, so
it struggles to recognize unfaithful summaries. Thus, we design two approaches
to improve them.

###  4.1 In-domain Training

Yuan et al. (2021) have shown when evaluating the factuality of generated text
BARTScore finetuned on CNNDM performs the best. CTC is also trained on data
constructed from CNNDM and XSum. However, since we find that they don’t
transfer well to evaluating dialogue summarization, based on hypothesis 1, we
experiment with adapting CTC and BARTScore to the dialogue domain by training
on in-domain data.

We use conversations from Pushshift Reddit (Baumgartner et al., 2020) 111an
existing dataset extracted and obtained by a third party and made available on
pushshift.io as unsupervised in-domain training data. Since Reddit
conversations have no associated reference summaries, and results from CTC (in
Table 4) have shown using extractive summaries on conversation data hurts
performance, to train BARTScore, we generate fake references with T0 (Sanh et
al., 2022), a multi-task seq2seq LM exhibiting strong zero-shot generalization
abilities. For each conversation, we generate multiple summaries with
different prompts and pick the summary with the highest ROUGE-L score
comparing against the source as the fake reference to prevent it from
deviating too much from the source. In addition, We also investigate training
directly on SAMSum, the task data.

###  4.2 Unlikelihood Training with Negative Samples

BARTScore works by predicting the probability of a hypothesis being generated
given a source text. In the finetuning stage, standard MLE training aims to
maximize the probability of positive tokens but does not explicitly minimize
those of negative tokens. Based on hypothesis 2, i.e. without explicit
guidance, it is difficult for the model to discern the unfaithful content, we
remedy this difficulty by leveraging unlikelihood loss (Welleck et al., 2020)
to penalize negative tokens, teaching the model to assign low probabilities to
unfaithful tokens.

Consider the sequence $S=(x_{1},...,x_{T})$, $N$ is the set of indices of its
negative tokens. The loss for $S$ is given as

$L_{S}=\left\\{\begin{array}[]{lr}\\!\\!-\sum\limits_{t}^{T}\log
p_{\theta}(x_{t}|x_{<t}),&\\!\\!\text{if }S\text{ is pos}\\\
\\!\\!-\alpha\sum\limits_{t\in
N}\log(1-p_{\theta}(x_{t}|x_{<t})),&\\!\\!\text{if }S\text{ is
neg}\end{array}\right.$

Where $p_{\theta}$ is the model parameterized by $\theta$, $\alpha$ is a
hyperparameter representing the weight of the unlikelihood loss.

Inspired by Cao and Wang (2021), we generate three types of negative samples
from SAMSum reference summaries. Swapent shuffles entities of the same type,
simulating wrong reference error which is a common error type for dialogue
summarization. Maskent randomly masks one entity of each type, and fill the
masks with BART, simulating summaries with incorrect subjects or objects.
Hallu generates SAMSum summaries using BART trained on XSum with top p
sampling where p = 1.0, simulating complete hallucinations. We show examples
of negative samples generated by our system in Appendix A.

##  5 Experiments & Analysis

We report and discuss experiment results and new findings in this section.
Each model is trained on 4 A100 GPUs using the AdamW optimizer with learning
rate set to 2e-5. Trained with DDP, the effective batch size is 64. All
reported numbers are the average of 3 runs using different random seeds.

###  5.1 In-domain Results

| Vanilla | w. Reddit | w. SAMSum
---|---|---|---
BScore | 0.1844 | 0.2667 | 0.3275
CTC | 0.1245 | 0.0552 | 0.0692
Table 1: Spearman correlations of BARTScore (BScore) and CTC on SAMSum data
with in-domain training. "Vanilla" uses bart-large-cnn and D-XSum for
BARTScore and CTC respectively.

Table 1 shows training on Reddit conversations with fake references increases
BARTScore’s correlation with human judgements by 45%. However, training on
CTC’s negative samples results in decreases in correlation. This suggests
fine-tuning BARTScore on in-domain data even with weakly supervised summaries
can boost its performance at evaluating the faithfulness of dialogue
summaries. However, CTC did not generalize well to the new domain. We suspect
this is due to CTC’s negative generation pipeline being not suitable for
dialogues, producing low-quality negative samples.

Nonetheless, training directly on task data i.e. SAMSum with human written
references yields better performance, despite SAMSum being 10 times smaller
than unsupervised Reddit data.

###  5.2 BARTScore with Negatives Ablation Results

| Types of Negatives |
---|---|---
ref | swapent | maskent | hallu | correlations
|  |  |  | 0.3275
|  |  |  | 0.3414
|  |  |  | 0.3400
|  |  |  | 0.3302
|  |  |  | 0.3349
|  |  |  | 0.3441
Table 2: Spearman correlations of BARTScore on SAMSum data. BART models are
trained on SAMSum’s human written references and different combinations of
negative examples.

We report ablation results of BARTScore with additional unlikelihood training
on negative samples in Table 2, in which the weight of unlikelihood loss
$\alpha$ = 0.1 and all models start from bart-large-cnn. All models trained
with unlikelihood loss on negative samples in addition to MLE show performance
improvements. When training with one type of negative, swapent contributes the
most improvement among the three types of negative samples, and hallu
contributes the least. Using all three types of negatives yields the best
performance.

###  5.3 T0-Score

Metrics | SAMSum | CNN | XSum
---|---|---|---
CTC | 0.1129 | 0.4293 | 0.3149
BARTScore | 0.3441 | 0.3820 | 0.1705
T0-Score (3B) | 0.3049 | 0.4141 | 0.1829
T0-Score (11B) | 0.3780 | 0.4573 | 0.1862
Table 3: Spearman correlations of CTC, BARTScore, and T0-Score, on SAMSum,
CNNDM, and XSum.

From previous ablations, we found that bart-large-cnn performs better than
bart-large when finetuned on SAMSum for evaluation, which suggests that a more
generalized model could do better as an evaluator. Because T0 is known for its
strong zero-shot generalization abilities, we experiment with using its
generation probability of summary conditioned on source as the faithfulness
metric, which we call T0-Score. Table 3 shows results of 2 versions of
T0-Score on meta evaluation datasets from multiple domains compared to other
metrics. T0-Score shows performance gains even over the best performing
BARTScore on SAMSum. It also outperforms CTC on CNNDM, setting state-of-the-
art performance. However, it falls short on XSum compared to CTC.

###  5.4 Discussion

Baseline Metrics
---
ROUGE-1 | 0.2462
ROUGE-2 | 0.2626
ROUGE-3 | 0.2415
ROUGE-L | 0.2461
BERTScore (roberta-large-mnli) | 0.2957
BERTScore (deberta-v2-xxlarge-mnli) | 0.2974
CTC (D-XSUM) | 0.1245
CTC (extractive on SAMSum) | 0.0253
BARTScore (bart-large) | 0.1315
BARTScore (bart-large-cnn) | 0.1844
Our Metrics
BARTScore (reddit) | 0.2667
BARTScore (unlikelihood) | 0.3441
T0Score (11B) | 0.3780
Table 4: Spearman correlation of all evaluated metrics

Table 4 summarizes results from all metrics together with our improvements. We
see that most existing summarization faithfulness metrics perform poorly on
dialogue data out-of-the-box (with BERTScore being an exception). BARTScore
can be significantly improved when finetuned on in-domain or task data, and
additional unlikelihood training with negative samples can further boost its
performance. Since a metric’s performance can be highly dependant on the
domain that it is applied to, e.g. CTC is competitive on news but falls short
on dialogues, we call for researchers to evaluate on multiple domains when
proposing general automatic metrics for summarization evaluation.

##  6 Conclusion

We evaluate common faithfulness metrics on dialogue summarization and find
that most of them exhibit low correlations with human judgements. We
experiment with ways to improve them by training on in-domain data with
unlikelihood loss on negative samples, and we show both of them can bring
significant improvements. We call for more domain-aware use of evaluation
metrics and more comprehensive evaluation over multiple domains when proposing
new metrics.

##  7 Limitations

The negative samples we generate from handcrafted rules are limited to three
types of common errors in auto-generated summaries. To obtain more accurate
faithfulness evaluations, more error types have to be proposed. Or as an
alternative, obtaining token-level human annotations is another expensive but
viable option.

This study is conducted using SAMSum as a proxy to dialogue data. While our
approaches are shown to perform well on SAMSum as a more casual chit-chat
dialogue dataset, because of the lack of annotated data, we have not tested
our approaches on datasets with formal and longer conversations. When more
annotated data is available, evaluating on a more diverse set of dialogue
datasets could make our arguments stronger.

## References

  * Banerjee and Lavie (2005) Satanjeev Banerjee and Alon Lavie. 2005.  METEOR: An automatic metric for MT evaluation with improved correlation with human judgments.  In _Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization_ , pages 65–72, Ann Arbor, Michigan. Association for Computational Linguistics.
  * Baumgartner et al. (2020) Jason Baumgartner, Savvas Zannettou, Brian Keegan, Megan Squire, and Jeremy Blackburn. 2020.  The pushshift reddit dataset.  _Proceedings of the International AAAI Conference on Web and Social Media_ , 14(1).
  * Brown et al. (2020) Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020.  Language models are few-shot learners.  In _Advances in Neural Information Processing Systems_ , volume 33, pages 1877–1901. Curran Associates, Inc.
  * Cao and Wang (2021) Shuyang Cao and Lu Wang. 2021.  CLIFF: Contrastive learning for improving faithfulness and factuality in abstractive summarization.  In _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing_ , pages 6633–6649, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
  * Chen et al. (2021) Yulong Chen, Yang Liu, Liang Chen, and Yue Zhang. 2021.  DialogSum: A real-life scenario dialogue summarization dataset.  In _Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021_ , pages 5062–5074, Online. Association for Computational Linguistics.
  * Deng et al. (2021) Mingkai Deng, Bowen Tan, Zhengzhong Liu, Eric Xing, and Zhiting Hu. 2021.  Compression, transduction, and creation: A unified framework for evaluating natural language generation.  In _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing_ , pages 7580–7605, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
  * Durmus et al. (2020) Esin Durmus, He He, and Mona Diab. 2020.  FEQA: A question answering evaluation framework for faithfulness assessment in abstractive summarization.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 5055–5070, Online. Association for Computational Linguistics.
  * Fabbri et al. (2021a) Alexander Fabbri, Faiaz Rahman, Imad Rizvi, Borui Wang, Haoran Li, Yashar Mehdad, and Dragomir Radev. 2021a.  ConvoSumm: Conversation summarization benchmark and improved abstractive summarization with argument mining.  In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_ , pages 6866–6880, Online. Association for Computational Linguistics.
  * Fabbri et al. (2021b) Alexander R. Fabbri, Wojciech Kryściński, Bryan McCann, Caiming Xiong, Richard Socher, and Dragomir Radev. 2021b.  SummEval: Re-evaluating summarization evaluation.  _Transactions of the Association for Computational Linguistics_ , 9:391–409.
  * Falke et al. (2019) Tobias Falke, Leonardo F. R. Ribeiro, Prasetya Ajie Utama, Ido Dagan, and Iryna Gurevych. 2019.  Ranking generated summaries by correctness: An interesting but challenging application for natural language inference.  In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_ , pages 2214–2220, Florence, Italy. Association for Computational Linguistics.
  * Gabriel et al. (2021) Saadia Gabriel, Asli Celikyilmaz, Rahul Jha, Yejin Choi, and Jianfeng Gao. 2021\.  GO FIGURE: A meta evaluation of factuality in summarization.  In _Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021_ , pages 478–487, Online. Association for Computational Linguistics.
  * Gliwa et al. (2019) Bogdan Gliwa, Iwona Mochol, Maciej Biesek, and Aleksander Wawer. 2019.  SAMSum corpus: A human-annotated dialogue dataset for abstractive summarization.  In _Proceedings of the 2nd Workshop on New Frontiers in Summarization_ , pages 70–79, Hong Kong, China. Association for Computational Linguistics.
  * Kryscinski et al. (2020) Wojciech Kryscinski, Bryan McCann, Caiming Xiong, and Richard Socher. 2020.  Evaluating the factual consistency of abstractive text summarization.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 9332–9346, Online. Association for Computational Linguistics.
  * Lavie and Agarwal (2007) Alon Lavie and Abhaya Agarwal. 2007.  METEOR: An automatic metric for MT evaluation with high levels of correlation with human judgments.  In _Proceedings of the Second Workshop on Statistical Machine Translation_ , pages 228–231, Prague, Czech Republic. Association for Computational Linguistics.
  * Lewis et al. (2020) Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. 2020.  BART: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 7871–7880, Online. Association for Computational Linguistics.
  * Lin (2004) Chin-Yew Lin. 2004.  ROUGE: A package for automatic evaluation of summaries.  In _Text Summarization Branches Out_ , pages 74–81, Barcelona, Spain. Association for Computational Linguistics.
  * Maynez et al. (2020) Joshua Maynez, Shashi Narayan, Bernd Bohnet, and Ryan McDonald. 2020.  On faithfulness and factuality in abstractive summarization.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 1906–1919, Online. Association for Computational Linguistics.
  * Nallapati et al. (2016) Ramesh Nallapati, Bowen Zhou, Cicero dos Santos, Çağlar Gulçehre, and Bing Xiang. 2016.  Abstractive text summarization using sequence-to-sequence RNNs and beyond.  In _Proceedings of The 20th SIGNLL Conference on Computational Natural Language Learning_ , pages 280–290, Berlin, Germany. Association for Computational Linguistics.
  * Narayan et al. (2018) Shashi Narayan, Shay B. Cohen, and Mirella Lapata. 2018.  Don’t give me the details, just the summary! topic-aware convolutional neural networks for extreme summarization.  In _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing_ , pages 1797–1807, Brussels, Belgium. Association for Computational Linguistics.
  * Pagnoni et al. (2021) Artidoro Pagnoni, Vidhisha Balachandran, and Yulia Tsvetkov. 2021.  Understanding factuality in abstractive summarization with FRANK: A benchmark for factuality metrics.  In _Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 4812–4829, Online. Association for Computational Linguistics.
  * Papineni et al. (2002) Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002.  Bleu: a method for automatic evaluation of machine translation.  In _Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics_ , pages 311–318, Philadelphia, Pennsylvania, USA. Association for Computational Linguistics.
  * Raffel et al. (2020) Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020.  Exploring the limits of transfer learning with a unified text-to-text transformer.  _Journal of Machine Learning Research_ , 21(140):1–67.
  * Sanh et al. (2022) Victor Sanh, Albert Webson, Colin Raffel, Stephen Bach, Lintang Sutawika, Zaid Alyafeai, Antoine Chaffin, Arnaud Stiegler, Arun Raja, Manan Dey, M Saiful Bari, Canwen Xu, Urmish Thakker, Shanya Sharma Sharma, Eliza Szczechla, Taewoon Kim, Gunjan Chhablani, Nihal Nayak, Debajyoti Datta, Jonathan Chang, Mike Tian-Jian Jiang, Han Wang, Matteo Manica, Sheng Shen, Zheng Xin Yong, Harshit Pandey, Rachel Bawden, Thomas Wang, Trishala Neeraj, Jos Rozen, Abheesht Sharma, Andrea Santilli, Thibault Fevry, Jason Alan Fries, Ryan Teehan, Teven Le Scao, Stella Biderman, Leo Gao, Thomas Wolf, and Alexander M Rush. 2022.  Multitask prompted training enables zero-shot task generalization.  In _International Conference on Learning Representations_.
  * Tang et al. (2022) Xiangru Tang, Arjun Nair, Borui Wang, Bingyao Wang, Jai Desai, Aaron Wade, Haoran Li, Asli Celikyilmaz, Yashar Mehdad, and Dragomir Radev. 2022.  CONFIT: Toward faithful dialogue summarization with linguistically-informed contrastive fine-tuning.  In _Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 5657–5668, Seattle, United States. Association for Computational Linguistics.
  * Vaswani et al. (2017) Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Ł ukasz Kaiser, and Illia Polosukhin. 2017.  Attention is all you need.  In _Advances in Neural Information Processing Systems_ , volume 30. Curran Associates, Inc.
  * Wang et al. (2020) Alex Wang, Kyunghyun Cho, and Mike Lewis. 2020.  Asking and answering questions to evaluate the factual consistency of summaries.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 5008–5020, Online. Association for Computational Linguistics.
  * Welleck et al. (2020) Sean Welleck, Ilia Kulikov, Stephen Roller, Emily Dinan, Kyunghyun Cho, and Jason Weston. 2020.  Neural text generation with unlikelihood training.  In _International Conference on Learning Representations_.
  * Yuan et al. (2021) Weizhe Yuan, Graham Neubig, and Pengfei Liu. 2021.  Bartscore: Evaluating generated text as text generation.  In _Advances in Neural Information Processing Systems_ , volume 34, pages 27263–27277. Curran Associates, Inc.
  * Zhang et al. (2020) Jingqing Zhang, Yao Zhao, Mohammad Saleh, and Peter J. Liu. 2020.  Pegasus: Pre-training with extracted gap-sentences for abstractive summarization.  _ArXiv_ , abs/1912.08777.
  * Zhang* et al. (2020) Tianyi Zhang*, Varsha Kishore*, Felix Wu*, Kilian Q. Weinberger, and Yoav Artzi. 2020.  Bertscore: Evaluating text generation with bert.  In _International Conference on Learning Representations_.

##  Appendix A Generated Negative Samples

Table 5 shows examples of negative samples generated by our system.

Dialogue |  Amanda: I baked cookies. Do you want some? Jerry: Sure! Amanda: I’ll bring you tomorrow :-)
---|---
Reference |  Amanda baked cookies and will bring Jerry some tomorrow.
Swapent |  Jerry baked cookies and will bring Amanda some tomorrow.
Maskent |  I have baked cookies and will bring Jerry some tomorrow.
Hallu |  Amanda: I baked cookies, and I want to bring them to your house so you can eat them!
Table 5: An example SAMSum dialogue and its three types of negative summaries.
Highlighted parts are negative tokens.