11institutetext: 1415 Granvia Altamira, Palos Verdes Estates CA, USA 90274:
ICRANet, Piazza della Repubblica 10 Pescara 65100, Italy and ICRA, Physics
Department, University La Sapienza, Roma, Italy
11email<EMAIL_ADDRESS>

# First image of a jet launching from a black hole accretion system:
Kinematics

B. Punsly

(Received March 12, 2023; )

Jets are endemic to both Galactic solar mass and extragalactic supermassive
black holes. A recent 86 GHz image of M 87 shows a jet emerging from the
accretion ring around a black hole, providing the first direct observational
constraint on the kinematics of the jet-launching region in any black hole
jetted system. The very wide ($\sim 280\mu\rm{as}$), highly collimated, limb-
brightened cylindrical jet base is not predicted in current numerical
simulations. The emission was shown to be consistent with that of a thick-
walled cylindrical source that apparently feeds the flow that produces the
bright limbs of the outer jet at an axial distance downstream of
$0.4\,\rm{mas}<z<0.65\,\rm{mas}$. The analysis here applies the conservation
laws of energy, angular momentum, and magnetic flux to the combined system of
the outer jet, the cylindrical jet, and the launch region. It also uses the
brightness asymmetries of the jet and counterjet to constrain the Doppler
factor. The only global solutions have a source that is located
$<34\mu\rm{as}$ from the event horizon. This includes the Event Horizon
Telescope annulus of emission and the regions interior to this annulus. The
axial jet begins as a magnetically dominated flow that spreads laterally from
the launch radius ($<34\mu\rm{as}$). It becomes super-magnetosonic before it
reaches the base of the cylindrical jet. The flow is ostensibly redirected and
collimated by a cylindrical nozzle formed in a thick accretion disk. The flow
emerges from the nozzle as a mildly relativistic ($0.3c<v<0.4c$) jet with a
significant protonic kinetic energy flux.

###### Key Words.:

black hole physics — galaxies: jets—galaxies: active — accretion, accretion
disks—(galaxies:) quasars: general

##  1 Introduction

The detection of a jet connected to the accretion flow in the radio galaxy M
87 ($\approx 16.8$ Mpc distant) at 86 GHz with the Global Millimeter VLBI
Array, the phased Atacama Large Millimeter/submillimeter Array, and the
Greenland Telescope in 2018 provides the first direct evidence of a jet
launching from a black hole accretion system (Lu et al., 2023). A limb-
brightened cylindrical jet was found, $25\mu\rm{as}<z<100\mu\rm{as}$, where
$z$ is the axial displacement from the midline (see Figure 1) of the accretion
ring (Lu et al., 2023). It is too wide and too collimated to be consistent
with numerical simulations of Blandford-Znajek jets (Lu et al., 2023). The
plasma source for the observed cylindrical jet was approximated by a thick-
walled, uniform cylinder with a radius $R\approx 144\mu\rm{as}\approx 38M$,
and the tubular jet walls have a width $W\approx 36\mu\rm{as}\approx 9.5M$
(where $M$ is the geometrized mass of the black hole), and it is apparently
the source for the thick-walled tubular jet observed
$0.405\,\rm{mas}<z<0.65\,\rm{mas}$ downstream (see the left panel of Figure ).
We call this the outer jet hereafter (Punsly, 2022, 2023). The jet is detected
within $z^{\prime}\approx 20M$ above the midline of the accretion ring, where
$z^{{}^{\prime}}$ is the deprojected (intrinsic) distance. A line of sight
(LOS) to the jet axis of $18^{\circ}$ and $3.8\mu\rm{as}=M\approx 9.5\times
10^{14}\rm{cm}$ is assumed throughout (EHT Collaboration et al., 2019). This
detection of a jet launching is a robust discovery that is of fundamental
importance to the field of black hole jet-launching, regardless of future
Event Horizon Telescope (EHT) imaging campaigns (Johnson et al., 2023).
Furthermore, if the cylindrical jet tapers to a launch point at smaller radii
and the disk (beneath it) also radiates, then it might be impossible to
unambiguously segregate the launch region from the disk proper due to the
nearly face-on nature of the disk. Hence, a kinematic analysis is likely
necessary to achieve a deep understanding. This preliminary attempt is based
on the currently existing data.

The main physical questions raised by this detection are: 1. why it was
launched axially so far from the black hole, 2. how it attained such a high
velocity, which was at least mildly relativistic, so close to the accretion
plane when we assume bilateral symmetry and the lack of a counterjet
detection, and 3. why the jet is so collimated. The motivation of this study
is to answer these questions. The analysis proceeds as follows. Section 2
provides an estimate of the cylindrical jet flux density. Sections 3 and 4
explain the strategy of the analysis and the solutions, respectively. The last
section discusses the results.

Figure 1: Relating the cylindrical source to the cylindrical and outer jets.
The left panel is a wide angle view from (Punsly, 2023). This shows the
cylindrical source as an extension of the ridges of the outer jet. The nucleus
is saturated in this format, which accentuates the jet. The sky plane is the
w-z plane and the midline of the putative accretion disk is the w-axis. The
right panel is a close-up of the cylindrical source location. The location of
the cylindrical source projected onto the sky plane is denoted by the red
rectangle.

##  2 Flux density of the cylindrical jet base

The right panel of Figure 1 is a reformatted version of the image FITS file
used to create Figure 1a of Lu et al. (2023) that appeared in (Punsly, 2023).
The uniform cylinder source approximation that reproduced the cross sections
of the cylindrical jet in Punsly (2023) has a flux density of
$26.2^{+8.8}_{-7.1}$ mJy for $20\mu\rm{as}<z<100\mu\rm{as}$. No uncertainty
was provided by Lu et al. (2023). Thus, various crude, liberal uncertainty
estimates were added in quadrature to create an overall liberal uncertainty
estimate: the flux calibration (20%), statistical noise (15%), and the model
(10%) (Lee et al., 2008; Fomalont, 1999). The positive uncertainty includes
the possibility that some emission (20%) is resolved out by the interferometer
beam. The average brightness ratio of the south to north ridge is $R(S/N)\sim
1.1$. However, the observations are not sensitive enough to determine small
differences from unity like these due to the uncertainties associated with the
statistical noise.

##  3 Assumptions and solution strategy

The estimation of cylindrical jet properties and launch point is predicated on
the assumptions of axisymmetry, bilaterally symmetry, perfect
magnetohydrodynamics (MHD), and approximate time stationarity, combined with
conservation of energy, $Q$, angular momentum, $L$, and magnetic flux, $\Phi$,
with the outer jet. The cylindrical jet plasma was chosen to have uniform
properties in the absence of a detection at higher resolution.

Approximate time stationarity is critical since the properties of the outer
jet are required for the analysis. However, there are no observations at or
near the epoch shown in Figure 1, April 14, 2018, with sufficient sensitivity
for these purposes. The details of the outer jet in 2013 are reviewed in
Appendix A. In order to justify the assumption of approximate time
stationarity, we considered the nuclear light curves in Figure 2, whic h are
an augmentation of the light curve in (Punsly, 2021). The light curve combines
the peak intensities from 43 GHz VLBA (with a 0.4 mas x 0.2 mas restoring
beam: $-16^{\circ}\leq\rm{PA}\leq 12^{\circ}$), fitted nuclear flux densities
from 22 GHz VLBI Exploration Radio Astrometry (VERA) observations, and the 86
GHz Korean VLBI Network (KVN) nuclear flux densities, with an uncertainty
$<20\%$ (Walker et al., 2018; Hada et al., 2013, 2014, 2016; Kim et al., 2018;
Algaba et al., 2021). The 43 GHz VLBA observations have the highest
resolution, but are not sampled densely enough, and the other measurements
were designed to fill the gaps. The date of jet ejection is based on the
average propagation velocity found in the next section. The light curve
indicates that the M 87 nucleus is typically steady within about $\pm 25\%$,
and the nucleus might have been $\sim 20\%$ brighter when the 2018 outer jet
was launched compared to the 2013 launch epoch (Kim et al., 2018). The nucleus
receives a large contribution from the accretion disk, but theoretical models
have a strong correlation between disk luminosity (accretion rate) and jet
power (Punsly, 2008; EHT Collaboration et al., 2021). We concluded that using
an observation in 2013 to estimate the outer jet $Q$ is probably not grossly
inaccurate. To minimize this modest uncertainty, the estimates for the outer
jet properties were biased toward the high side by choosing the $W/R\approx
0.35$ solution in Figure A.1. The assumption was not that $W/R=0.35$ was valid
in 2018. The assumption was that the 2013 acceleration in the outer jet (see
the upper right panel of Figure A.1) is qualitatively typical, and that the
jet power is slightly stronger in 2018. Even though the exact jet properties
are not known (i.e., the exact W/R, $Q$, and $L$), the choices are close
enough for the results in Section 4 to be representative of M 87 in 2018.

Viable solutions obey seven constraints on the global flow from the launch
point, or anchor, for the magnetic field to the outer jet solution: 1\. The
flux density of the cylindrical jet, $S_{\nu}=26.2^{+8.8}_{-7.1}$ mJy; 2.
$R(J/CJ)>5$ (see Equation 1); 3. $Q$ conservation; 4.$L$ conservation; 5. the
jet requires an energy source. The cylindrical jet Poynting flux power of the
cylindrical jet, $S^{P}$, exceeds that of the outer jet (magnetic energy
converted into kinetic energy); 6. the mass flux does not decrease along the
jet. Only entrainment is allowed during propagation, not the loss of mass
flux; 7. $\Phi$ conservation.

Figure 2: Time variability of the nucleus. The figure shows the modest
variability of the nucleus and motivates approximate time stationarity. Figure
3: Constraining the counterjet. The putative location of the counterjet
cylindrical source. For $z<-70\mu\rm{as}$, the dilution from the bright
nucleus is negligible at the ridge locations.

##  4 Solution space

Three different reference frames were needed for the calculations, the
emission and observer’s frame (subscripts e and o), and the cosmological rest
frame of M87 (M87 frame hereafter). The calculations were performed in the
emission frame and were then transformed to the observer’s frame. The relevant
equations are reviewed in Appendix B.

The upper limit on surface brightness of the counterjet “within 0.1 - 0.3 mas”
is $\sim 1$ mJy/beam (Lu et al., 2023). Nuclear dilution affects our ability
to resolve a weak counter jet. Figure 3 indicates that a 1 mJy/beam counter
jet ridge would be cleanly resolved from the nucleus for $z<-70\mu\rm{as}$
(0.14 mas from the black hole). Therefore, the $z=70\mu\rm{as}$ peak
intensities on the ridges that are indicated in Figure 2 motivate a lower
bound of the brightness ratio, $R(J/CJ)>5$. From Equation (B.3),

$R(a/b)=\frac{[1+\@vec{\beta_{a}}\cdot\mathbf{\hat{n}}]^{2+\alpha}}{[1+\@vec{\beta_{b}}\cdot\mathbf{\hat{n}}]^{2+\alpha}}\;,$
(1)

where $R(a/b)$ is the flux density ratio, $\mathbf{\hat{n}}$ is the unit
vector along the LOS, and $a$ and $b$ are two identical flows with different
angles to the LOS. If $\@vec{\beta_{a}}=-\@vec{\beta_{b}}$ and $R(J/CJ)>5$,
the axial velocity of the cylindrical jet in the M87 frame is
$\beta^{z^{\prime}}>0.3$.

If the jet is launched from the disk, the field line angular velocity in the M
87 frame, $\Omega_{F}$, would approximately be the local Keplerain angular
velocity at which the poloidal field lines are anchored. In the M 87 frame
(Boyer-Lindquist coordinates, which are used throughout, unless otherwise
stated), (t, r, $\theta$, $\phi$) this is (Lightman et al., 1975)

$\Omega_{\rm{kep}}(r)=\frac{M^{0.5}}{r^{1.5}+aM^{0.5}}\;,$ (2)

where a is the angular momentum per unit mass of the black hole.

A method of graphically displaying conformance to conservation laws in the jet
was developed in Punsly and Chen (2021) and repeated in Appendix A. These
methods are applied to various potential global solutions of the cylindrical
jet and the outer jet in Appendix C and summarized here. Figure C.1 shows
conformance plots for the outer jet with various launch points. The only
effect on the outer jet is the imposition of the $\Omega_{F}$ associated with
the anchor. The top left example assumes that the jet emerges vertically from
the local disk at $r=35$M, $\Omega_{F}=1.62\times 10^{-7}\rm{sec}^{-1}$ from
Equation (2). Constraint (6) from Section 3 is grossly violated, and there is
no global solution. The middle row shows improvement as the anchor point moves
inward toward r =15 M. The outer jet fit is improved enough that it is now
practical to expand the plot to include the cylindrical jet and $\Phi$
conservation. The result is nonconforming. $\dot{M}$ in the cylinder is below
the plot, but still satisfies constraint (6). In the bottom left panel, anchor
at r = 10M and a cylindrical jet velocity of v = 0.32c, the system conforms
better. The standard of $\pm 5\%$ conformance is set by the outer jet in
Figure A.1, it is not a pass or fail condition. The conformance plots are a
diagnostic of the tension between conserved quantities induced by the
assumptions of the solution. For v = 0.4c in the bottom panel, there is a
significant $\Phi$ nonconformance. The fit deteriorates as the anchor point
moves out because $\Omega_{F}$ decreases. From Equations (B.5), (B.7), and
(B.9), this requires a higher azimuthal velocity, $\beta^{\phi}$, to conserve
$L$. A small $\Omega_{F}$ combined with a significant $\beta^{\phi}$ modifies
the right-hand side of the last expression in Equation (B.4). The fact that
$B^{P}/B^{\phi}\ll 1$ is inconsistent with the frozen-in condition in the
outer jet induces large changes to the system of equations.

Figure C.2 shows the conformance plots as the footpoint (modeled as rings with
constant plasma properties with $a/M=0.99$, as described in Punsly and Chen
(2021)) moves inward. A model is required near the black hole because the
gradients in $\Omega_{\rm{kep}}$ are large. $Q$ at the foorpoint is
predominantly electromagnetic. Averaging over the ring, $\Omega_{F}=7.04\times
10^{-6}\rm{s}^{-1}=\Omega_{H}/2$ for r =1.7M -2.2M, where $\Omega_{H}$ is the
angular velocity of the event horizon (Lightman et al., 1975). Anchors with
$r<10$M improve the $\Phi$ conformance for v=0.32c. For v=0.4c, $\Phi$
conservation is improved, but its conformance is just marginal. We conclude
that v=0.4c is an upper bound on the velocity.

The top left panel of Figure 4 considers Doppler asymmetry. $R(S/N)<1.2$ for
launch points $<10\rm{M}$, consistent with the findings of Section 2. This is
an independent corroboration of the conformance scatter plot analysis
indicating $r<10$M. The plasma angular momentum vector points antiparallel to
the z-axis, the same as the EHT accretion ring (EHT Collaboration et al.,
2019). Note that based on the jet velocity and the finite size of the regions
used to compute the flux density ($\approx$ beam full width at half maximum),
the cylindrical (outer) jet data points in Figures C.1, C.2, and A.1 represent
time averages with a duration of $\sim 3$ months ($\sim 6$ months) that smooth
out abrupt fluctuations (Punsly, 2022). The top right panel of Figure 4 shows
the MHD wind properties in the cylindrical jet for different launch regions.
The fastest MHD wave is the fast wave. In this analysis, the fast four-speed,
$U_{F}^{2}\gtrsim(B^{p})^{2}/(4\pi m_{p}c^{2})$ (Punsly, 2008). All the viable
solutions (those launched inside of 10M) are superfast (Punsly, 2008). Thus,
the interaction of the magnetized flow with the disk at r =35M cannot be
transmitted back to the launch region (it is causally disconnected). The next
two rows plot the dependence on the physical properties of the cylindrical jet
as a function of launch point and bulk velocity. In the bottom row, $B^{P}$
and $\beta^{\phi}$ are much larger with a launch point at r = 10M. This is a
reflection of the tension among the MHD conservation laws, indicating that a
viable solution breaks down at these larger radii.

Figure 4: Physical properties of the cylindrical jet. The top left panel
indicates that the observed brightness asymmetries are consistent with the
conforming solutions. The top right panel shows that all viable solutions are
super-magnetosonic in the cylindrical jet.The physical properties of the
cylindrical jet as a function of launch point and bulk velocity are given in
the last two rows.

##  5 Discussion and conclusion

This study explored the kinematics of the cylindrical jet base of M 87 that
was discovered in a 2018 observation. The analysis considered the global
conservation of $Q$, $L$, and $\Phi$ in an attempt to answer the three
questions posed in the Introduction. The global solutions indicate the
following:

1\. The jet is launched axially so far from the black hole: The conformance
plots in Appendix C show that i originates from small radii, $r<10$M, and
expands laterally to r = 29.5M -39M. At this point, it transitions to an axial
flow aligned with the outer jet. Apparently, plasma flows over the top of the
disk and feeds the vertical jet. However, the inclination to the equator of
the streamlines is likely never to be discerned due to the nearly face-on view
of the accretion disk and lateral flow.

2\. It attained its high velocity so close to the accretion plane: The middle
left panel of Figure 4 and Figure A.1 show that magnetic energy is converted
into kinetic energy in the axial jet. When we extend this trend back toward
the launch point, plasma is accelerated by magnetic forces in the laterally
expanding flow above the disk.

3\. The jet is so collimated: The strong collimation is much closer to the
launch point than can be achieved in MHD jet models (Rohoza et al., 2024;
Blandford and Payne, 1982). It is almost axisymmetric, which also rules out
the possibility of that it is a manifestation of sporadic nonaxisymmetric
large-scale vertical flux ejection from reconnection at the interface of the
disk with the event horizon magnetosphere (Ripperda et al., 2022).
Furthermore, these flux tubes are not twisted, indicating that they transport
negligible Poynting flux. By process of elimination, the straightforward
explanation is that the axial jet appears to emerge from a nozzle in a thick
accretion disk (half height to radius ratio, $h/r\approx
70\rm{M}/35\rm{M}=2$). This seems to indicate a structured disk with different
physical properties and $h/r$, an inner and outer disk.

The main uncertainty in this analysis is the flux density of the cylindrical
jet. Appendix D shows that the results are unchanged for any flux density
within the very liberal assignment of an uncertainty in Section 2. Another
uncertainty is the outer jet properties. However, the solution is tightly
constrained by observations with one unique solution for every $W/R$. Since
$Q\sim(W/R)^{0.46}$, plausible $W/R$ values cannot generate a significant
variation in $Q$ (Punsly and Chen, 2021). These limitations mean that the
analysis must rely on the light curve in Figure 2 to justify approximate time
stationarity.

The mass flow rate is $\sim 25\%$ that of the outer jet in Figure 4. There
must be entrainment of protonic plasma before it reaches the outer jet,
probably by Kelvin-Helmholttz instabilities (Bicknell, 1995). The entrainment
damps the electromagnetic acceleration of the plasma. The jet apparently
strongly interacts with a non-negligible environment for the first $\sim 0.2$
light years of its propagation.

###### Acknowledgements.

I am indebted to Thomas Krichbaum for explaining the uncertainties inherent to
this remarkable image. I would like to thank Rusen Lu for the image FITS file.
This manuscript benefitted from the improvements suggested by the supportive
referee.

## References

  * Algaba et al. (2021) Algaba,, J., Anczarski, J. Asada, K. et al. 2021, ApJL, 911, L11
  * Bicknell (1995) Bicknell, G. 1995, ApJSS, 101, 29
  * Blandford and Payne (1982) Blandford, R. D., & Payne, D. G. 1982, MNRAS, 199, 883
  * EHT Collaboration et al. (2019) Event Horizon Telescope Collaboration, Akiyama, K., Alberdi, A., et al. 2019, ApJL, 875, L5
  * EHT Collaboration et al. (2021) Event Horizon Telescope Collaboration, Akiyama, K., Algaba, J. C., et al. 2021, ApJL, 910, L13
  * Fomalont (1999) Fomalont, E. 1999, in Synthesis Imaging in Radio Astronomy II, ASP Conference Series, eds. Taylor, G., Carilli, C., Perley, R. 180, 301
  * Ginzburg and Syrovatskii (1965) Ginzburg, V. and Syrovatskii, S. 1965, Annu. Rev. Astron. Astrophys. 3 297
  * Hada et al. (2013) Hada, K., Kino, M., Niinuma, K., et al. 2013, PASJ, 65, 24
  * Hada et al. (2014) Hada, K., Giroletti, M., Kino, M., et al. 2014, ApJ, 788, 165
  * Hada et al. (2016) Hada, K., Kino, M., Doi, A., et al. 2016, ApJ, 817, 131
  * Johnson et al. (2023) Johnson, M.., Akiyama, K., Blackburn, L. et al 2023 Galaxies, 11(3), 61
  * Kim et al. (2018) Kim, J.-Y., Lee, S.-S.., Hogsdon, J., et al. 2018, A&A, 610, L5
  * Lee et al. (2008) Lee, S.-S., Lobanov, A., Krichbaum, T. P. et al. 2008, ApJ, 136, 59
  * Lightman et al. (1975) Lightman, A., Press, W., Price, R. and Teukolsky, S. 1975, _Problem Book in Relativity and Gravitation_ (Princeton University Press, Princeton)
  * Lind and Blandford (1985) Lind, K., Blandford, R.1985 ApJ 295 358
  * Lu et al. (2023) Lu, RS., Asada, K., Krichbaum, T.P. et al. Nature 616, 686–690 (2023). https://doi.org/10.1038/s41586-023-05843-w
  * Punsly (2008) Punsly, B. 2008, _Black Hole Gravitohydromagnetics_ , second edition (Springer-Verlag, New York)
  * Punsly (2021) Punsly, B. 2021, ApJ 918, 4
  * Punsly (2022) Punsly, B. 2022, ApJ 936, 79
  * Punsly (2023) Punsly, B. 2023, A&A 679, L1
  * Punsly and Chen (2021) Punsly, B. and Chen, S. 2021, ApJL 921 L38
  * Rohoza et al. (2024) Rohoza, V., Lalakos, A., Paik, M. et al. 2024, arXiv231100018R
  * Ripperda et al. (2022) Ripperda, B., Liska, M., Chatterjee, K. et al. 2022, ApJ, 924, 32
  * Tucker (1975) Tucker, W. 1975, _Radiation Processes in Astrophysics_ (MIT Press, Cambridge).
  * Walker et al. (2018) Walker R. C., Hardee P., Davies, F., Ly, C., Junor, W., 2018, ApJ 855 12 __

##  Appendix A Review of the details of the outer jet

The only observations for which a reliable estimate of the outer jet
kinematics can be obtained are the high-sensitivity 43 GHz VLBA observations
from January 12, 2013 (Walker et al., 2018). This image alone clearly defines
the limb-brightened counterjet (top left panel in Figure A.1). A high-
sensitivity depiction of the counterjet is essential for the kinematic
analysis. Doppler-beaming calculations (assuming intrinsic bilateral symmetry)
in Punsly (2021) indicate consistency with a uniformly accelerating jet from
0.27c to 0.38c in the range $0.4\,\rm{mas}<z<0.65\,\rm{mas}$ (top right panel
in Figure A.1). This bulk velocity is more relevant for defining the large-
scale kinematics than anecdotal pattern speeds (e.g., MHD wave speeds or local
inhomogeneities).

An analysis of the outer jet based on the bulk velocity field, flux density
distribution, and the conservation laws of mass, energy, and angular momentum
appeared in (Punsly and Chen, 2021). Various cases were considered. The first
case was a minimum-energy plasma in which the magnetic field was turbulent and
the plasma protonic. The jet power, $Q$, increased significantly and the mass
flow rate, $\dot{\mathcal{M}}$, decreased significantly along the jet. Thus,
this assumption did not satisfy energy or mass conservation and was
disfavored. The same results were found for a positronic plasma instead of a
protonic plasma. The outer jet plasma is not in a minimum-energy state.

Next, a nearly pure Poynting flux outer jet was assumed. Even though $Q$ was
conserved by assumption, $\dot{\mathcal{M}}$ decreased significantly along the
jet. Thus, this assumption did not satisfy mass conservation. The result holds
for a positronic or a protonic plasma. The outer jet is not a Poynting jet.

The only mechanism that can ensure mass conservation (or any a nondecreasing
mass flux as the jet propagates, for that matter) with the other constraints
satisfied is a significant conversion of magnetic into mechanical energy in
which the kinetic energy flux is non-negligible in comparison with the
Poynting flux. This solution requires a protonic plasma. The approaching jet
kinematic properties are shown in the middle panels of Figure A.1, originally
from (Punsly and Chen, 2021). In summary, the tubular jet was determined to be
a mildly relativistic protonic jet.

The solution is completely constrained (kinematics and composition) by the
observation for each value of $W/R$. The tubular jet wall was estimated at
$W\approx 0.25R$, same as the cylindrical jet (Punsly, 2023). It was pointed
out by (Walker et al., 2018) that the LOSs that graze the inner wall of the
tubular jet (the longest LOSs through the jet wall) should be near the peak
intensity of the ridges. This was associated with the inner wall of the
tubular jet. The width, $W$, of the tubular jet wall was crudely estimated by
fitting a Gaussian line source, after convolution with the restoring beam, to
the cross-sections of the approaching jet in the image in the top left panel
of Figure A.1. The half width at half maximum of the Gaussian source was
assumed to be a decent approximation to $W$.

Based on the above, an accurate estimate of $W/R$ would improve the estimate
of the outer jet properties. Thus motivated, numerous axisymmeric jet models
were investigated in a detailed analysis of high-sensitivity high-resolution
jet images in 2013 and 2014 (Punsly, 2022). It was found that $W/R=0.35$ in
the axisymmetric models provided the best fit to the cross-sections of the
images (Punsly, 2022). The estimates in the original solution for $W/R$ range
from 0.24-0.27. They are all increased by a factor of 1.4 =0.35/0.25, and the
solution is recalculated in the bottom panels of Figure A.1. The two solutions
behave almost identically. The conformance plots on the right are the main
tool we used to assess the fit quality in the body of the manuscript.

Figure 5: Properties of the outer jet in 2013. The top row is reprinted from
(Punsly, 2021). The left panel is a high-sensitivity 43 GHz VLBA image from
January 12, 2013, that is designed to highlight the counterjet. The white
linear contours levels are (1, 1.64, 2.29 ….. 10) mJy/beam. The red contours
are at -0.5 mJy/beam. The right panel is the constant acceleration jet
velocity profile associated with the Doppler boosts that convert the sum of
the ridge peak intensities of the counterjet into those of the approaching jet
(see Equation 1). The middle row is reprinted from (Punsly and Chen, 2021).
Poynting flux is converted into kinetic energy along the jet, and the
conformance to the conservation laws is tight. Increasing W/R from 0.25 and
0.35 in the bottom row has little effect on the nature of the solution, just
the normalization.

##  Appendix B Review of relevant equations

The synchrotron emissivity is given in (Tucker, 1975),

$\displaystyle j_{\nu}(\nu_{e})=1.7\times 10^{-21}[4\pi
N_{E}]a(n)B^{(1+\alpha)}\left(\frac{4\times
10^{6}}{\nu_{e}}\right)^{\alpha}\;,$ (3) $\displaystyle
a(n)=\frac{\left(2^{\frac{n-1}{2}}\sqrt{3}\right)\Gamma\left(\frac{3n-1}{12}\right)\Gamma\left(\frac{3n+19}{12}\right)\Gamma\left(\frac{n+5}{4}\right)}{8\sqrt{\pi}(n+1)\Gamma\left(\frac{n+7}{4}\right)}\;,$
(4)

where the coefficient $a(n)$ separates the pure dependence on $n$ (Ginzburg
and Syrovatskii, 1965). A power-law energy spectrum for the leptons with
normalization $N_{E}$ and number index $n$ was assumed. The total magnetic
field is $B$. A value of $\alpha=0.7$ was chosen for the outer jet because it
was previously found that for $z<1$ mas, $\alpha\approx 0.6$ from 22 GHz to 43
GHz and $\alpha\approx 0.8$ from 43 GHz to 86 GHz (Hada et al., 2016; Punsly,
2021). This can be transformed into the observed flux density,
$S(\nu_{\mathrm{o}})$, for an optically thin jet using the relativistic
transformation relations from Lind and Blandford (1985),

$\displaystyle S(\nu_{\mathrm{o}})=\frac{\delta^{(2+\alpha)}}{4\pi
D^{2}}\int{j_{\nu}^{{}^{\prime}}(\nu_{o})dV{{}^{\prime}}}\;,$ (5)

where $D$ is the distance. In this expression, the primed frame is the rest
frame of the plasma. $\beta$ is the three-velocity of the moving plasma and
the Doppler factor, $\delta=1/[\gamma(1-\beta\cos{\theta})]$, with $\theta$
being the LOS, and $\gamma=1/\sqrt{1-\beta^{2}}$.

Consider the time stationary, axisymmetric approximation to the frozen-in
conditions in the M 87 frame (Punsly, 2008),

$E^{\perp}=-\beta_{F}B^{P}\;,\;\beta_{F}\equiv(\Omega_{F}R_{\perp})/c\;,\quad
B^{\phi}=\frac{\beta^{\phi}-\beta_{F}}{\beta^{P}}B^{P}\;,$ (6)

where P indicates the poloidal direction of $B$, $\perp$ is the orthogonal
poloidal direction, $\beta^{i}$ are components of the bulk velocity, and
$R_{\perp}$ is the cylindrical radius. The kinematic quantities,
$\mathcal{K}(\mathrm{protonic})$, $\mathcal{L}(\mathrm{protonic})$, and
$\mathcal{M}$ are the kinetic energy, angular momentum, and mass flux of a
protonic plasma, respectively,

$\displaystyle\mathcal{K}(\mathrm{protonic})=\mathcal{M}c^{2}(\gamma-1)\;,\;\mathcal{L}(\mathrm{protonic})=\mathcal{M}c\gamma\beta^{\phi}R_{\perp}\;,$
$\displaystyle\mathcal{M}=Nm_{p}c\gamma\beta^{z}\;,$ (7)

where $m_{p}$ is the mass of the proton, and $N$ is the proper number density.
The lepto-magnetic energy, $E(\mathrm{lm})$, is the volume integral of the
leptonic internal energy density, $U_{e}$, and the magnetic field energy
density, $U_{B}$,

$\displaystyle E(\mathrm{lm})=\int{(U_{B}+U_{e})}\,dV=$
$\displaystyle\int{\left[\frac{B^{2}}{8\pi}+\int_{E_{min}}^{E_{max}}(m_{e}c^{2})(N_{E}E^{-n+1})\,d\,E\right]dV},\,$
(8)

where $dV$ is the volume element. The leptonic contributions are negligible to
the solutions relevant to this study. For consistency with the outer jet
solutions, $E_{min}\approx m_{e}c^{2}$ in the cylinder (Punsly and Chen,
2021). $E_{max}$ has very little effect on the solutions.

The electromagnetic angular momentum, $S_{L}$, and the poloidal Poynting
power, $S^{P}$, are (Punsly, 2008)

$\displaystyle
S_{L}=\frac{1}{4\pi}\int{B^{T}B^{P}dA_{\perp}}\;,S^{P}=\frac{1}{4\pi}\int{\Omega_{F}B^{T}B^{P}dA_{\perp}}\;,$
$\displaystyle\Phi\equiv\int{B^{P}dA_{\perp}}\;,$ (9)

where

$\displaystyle
B^{T}=[\sqrt{r^{2}-2Mr+a^{2}}\sin{\theta}]\sqrt{F^{r\,\theta}F_{r\,\theta}}\;,$
(10) $\displaystyle\;-M\leq a\leq M\;,$

where $F^{\mu\,\nu}$ is the Faraday tensor. Combining Equations (7) and (9) in
the M87 frame,

$\displaystyle
Q\approx\int{\mathcal{K}(\mathrm{protonic})dA_{\perp}}+S^{P}\;,$
$\displaystyle
L\approx\int{\mathcal{L}(\mathrm{protonic})dA_{\perp}}+S_{L}\,\quad\dot{M}=\int{\mathcal{M}(\mathrm{protonic})dA_{\perp}}\;.$

##  Appendix C Conformance plots

This appendix provides conservation law conformance plots analogous to those
used to analyze solution viability in Punsly and Chen (2021) and Appendix A.

Figure 6: Finding a viable solution. The four panels in the first two rows
show that mass flux conservation in the outer jet improves as the launch point
moves inward. At r=10M, the mass flux is conserved in the outer jet and the
magnetic flux is marginally conserved globally for v=0.32c, but not for v=0.4c
in the bottom panel.

Figure 7: Moving the launch point inward. The first row shows improvement by
an annulus of plasma between 4M and 5M in the $\Phi$ conservation compared to
$r=10M$ in Figure C.1. The bottom row shows no noticeable change in
conformance at smaller radii.

##  Appendix D Uncertainty analysis of the flux density

In this appendix, the variation in the conformance plots as a function of the
cylindrical jet flux density is explored, $26.2^{+8.8}_{-7.1}$ mJy.

Figure 8: Flux density variation for an anchor at r =10M. The fit to $\Phi$
conformance is improved with a higher flux density. r = 10 M conforms
marginally even at 35 mJy and $v=0.32$c, however. The identification of r =
10M as the outermost anchor point that is viable is not a consequence of the
flux density uncertainty.

.

Figure 9: Flux density variation for an annular anchor at r =4M-5M. The fit to
$\Phi$ conformance is improved with a higher flux density. An actual model of
the anchor, the annular region, is required to capture gradients in the Kerr
geometry as the event horizon is approached.

Figure 10: Flux density variation for an annular anchor at r =1.7M-2.7M. The
fit to $\Phi$ conformance is improved with a higher flux density.


#  : Sales Script Extraction and Analysis from Sales Chatlog

Hua Liang∗1 Tianyu Liu∗1 Peiyi Wang${}^{{}^{\dagger}2}$ Mengliang Rao1 &Yunbo
Cao1 ∗Equal Contribution †Work during Internship at Tencent
1Tencent Cloud Xiaowei 2Institute of Computational Linguistics, Peking
University
{suvedoliang, rogertyliu, peiyiwang, sekarao<EMAIL_ADDRESS>
###### Abstract

In modern sales applications, automatic script extraction and management
greatly decrease the need for human labor to collect the winning sales
scripts, which largely boost the success rate for sales and can be shared
across the sales teams. In this work, we present the system to serve both the
sales representatives and managers to attain the sales insights from the
large-scale sales chatlog. consists of three modules: 1) Customer frequently
asked questions (FAQ) extraction aims to enrich the FAQ knowledge base by
harvesting high quality customer question-answer pairs from the chatlog. 2)
Customer objection response assists the salespeople to figure out the typical
customer objections and corresponding winning sales scripts, as well as search
for proper sales responses for a certain customer objection. 3) Sales manager
dashboard helps sales managers to monitor whether a specific sales
representative or team follows the sales standard operating procedures (SOP).
The proposed prototype system is empowered by the state-of-the-art
conversational intelligence techniques and has been running on the Tencent
Cloud to serve the sales teams from several different areas.

##  1 Introduction

In the sales teams, some sales representatives consistently attain or exceed
the sales goals while others do not. The conventional routine of escalating
the performance of sales teams is to figure out what makes good sales stand
out by manually analyzing hundreds or thousands of human-to-human conversation
chatlog between salespeople and customers, and then summarize the winning
sales script or other sales tips for sales team training Leong et al. (1989);
Leigh and McGraw (1989); Humphrey and Ashforth (1994). However, this labor
intensive procedure requires the participation of sales experts and tedious
human analysis, which makes it undesirable for modern fast-paced, high-growth
sales teams.

Recent technological advances in natural language processing (NLP) have made
it possible to induce the skeleton or key semantics of the human conversations
in an automatic fashion Yu et al. (2020); Gliwa et al. (2019); Sun et al.
(2019). Compared with recognizing the winning sales scripts and analyzing the
large-scale sales chatlog through human experts, the automatic winning sales
scripts mining and analyses empowered by advanced NLP techniques would largely
accelerate the performance improvement of sales teams. To this end, we build
an easy-to-use system named that is capable of handling hundreds of thousands
or even millions of sales chatlog and assists the both sales representatives
and managers to figure out the best practises to increase the sales success
rate. is a web application that consists of three modules: Customer FAQ
extraction, Customer objection response mining and search, and Sales manager
dashboard.

has been deployed on the Tencent Cloud and can also be seamlessly embedded in
the existing customer relationship management (CRM) systems. We verify the
performance of according to the feedback from an online education sales teams
and an automobile sales teams.

##  2 System Architecture

Figure 1: The illustration for the user interface of (the UI elements are
translated to English from Chinese). The users start from an entrance page
(\small1⃝) that includes file uploading and task selection
(\small2⃝-\small5⃝). The selected tasks correspond to the different modules of
, which are elaborated in Sec 2.2.

###  2.1 Modules

#### Customer FAQ Extraction

As a replacement for manual FAQ accumulation, this module offers a much easier
and more efficient way for sales teams. The question-answer pairs retrieval
followed by human post-checking would be one of the cornerstones for knowledge
acquisition as new customers emerge. According to the real customer feedback,
this module reduce the human cost by up to 97%, 87% during knowledge base
cold-start for the sales teams of an international express company and an
online education company respectively.

#### Customer Objection Response

Customer objections are the concerns that a prospect has which cause them to
hesitate (at best) and abandon (at worst) a purchase. Even professionals might
feel nervous while they find themselves facing the customer objections. One of
the common resolutions is to highlight the typical customer objections that
come up again and again by going through the chatlog and then create a plan to
answer them. However, the sales chatlog would grow rapidly for a large sales
group which has vast prospective customers, and the customer objections might
change in the different stages of the sales circle. The proposed module would
serve as an assistant for the salespeople to determine typical customer
objections and gain actionable insights from good responses in the periodical
sales retrospectives or performance reviews. As illustrated in Fig 3, after
filtering trivial customer messages, semantically similar customer queries and
corresponding responses from sales are assembled in the same cluster. The
clusters of customer queries are ordered by frequency and semantic relevance.
For each cluster, we offer keywords that highlights the core semantics of
customer queries. The sales representatives could figure out the typical
customer objections and the best practises to respond, or search for a
successful sales response for a particular customer objection with the
proposed module.

#### Sales Manager Dashboard

In order to increase the sales performance, the sales manager should
understand the pros and cons of their crew in order to develop the training
programs for the sales staffs. The proposed module helps the managers to
overview the sales performance with a “query trigger-response spotlight”
paradigm. The paradigm consists of a set of pre-defined rules that specify the
sales standard operating procedures (SOP), i.e. “what the sales should do in
the given situation”. For example,

  * •

When a customer hesitates due to the affordability, the sales should mention
payment policy such as “pay by installments”.

  * •

While facing a new customer, the sales should advertise for the best-seller
productions.

The customer query triggers and sales response spotlights are formulated as a
set of rules, keyword matching and query intention classification models,
which are tailored for different sales groups.

We verify the module functionality and effectiveness of customer objection
response and sales manager dashboard with the chatlog from an automobile sales
team and an online education sales team.

###  2.2 Interface

system is a web application which is deployed on the Tencent cloud and can be
easily integrated in the customers’ CRM system. 111The demonstration video for
can be viewed in https://www.youtube.com/watch?v=87rxct_HUjU. We show the user
interface and user interaction of in Fig 1. The users start from an entrance
page that includes chatlog file uploading and task selection. After uploading
the chatlog in the csv format, the users could select a specific task with the
“task starts” operation (\small1⃝), and view the details of the finished tasks
in the “task list” section. The selected task in the entrance page corresponds
to different modules of the system (\small2⃝-\small5⃝). For the customer FAQ
extraction module (\small2⃝), the extracted FAQs would be plainly shown in a
new webpage that does not need further user interaction. For the customer
objection response module, the response mining sub-module corresponds to the
information table of the clustered customer messages (\small3⃝), the users
could refer to the clustered sales responses by clicking on a specific message
cluster in a popup window (\small6⃝). The user could also gain sales insights
by searching for the existing sales responses in the searchbar (\small4⃝). On
the sales manager dashboard page (\small5⃝), the sales managers could overview
the execution ratio of the sales SOP from different viewpoints, i.e. trigger
view, team view and staff view, by interacting with the tabbar.

Figure 2: The workflow of the customer FAQ extraction module. Figure 3: The
NLP pipeline for customer objection response module.

##  3 AI Engine

#### QA pair mining

As depicted in Fig 2, the QA pairs mining module is comprised of a question
extractor and an answer extractor. The question extractor aims to determine
the meaningful customer queries and is formulated as a multi-label classifier
for semantic integrity, chitchat filtering, legal customer inquiry222A valid
question should be labeled as “yes” for all three aspects., which is modeled
as a one-layer MLP classifier over the CLS vector of a syntax-enhanced BERT Li
et al. (2021); Xu et al. (2021) encoder. We model the answer extractor with a
prompt-based BERT matching model after recognizing the valid customer queries.
The input of the answer extractor is a dialog snippet
$\mathcal{S}=\\{u_{1}^{q},u_{2},\dots,u_{r}\\}$ with $r$ utterance, in which
$u_{1}^{q}$ denotes the valid customer query recognized by the question
extractor while $\\{u_{2},\dots,u_{r}\\}$ denotes the subsequent utterances
that includes the messages from both customer and sales. We feed the dialog
snippet $\mathcal{S}$ with soft prompt templates Qin and Eisner (2021);
Hambardzumyan et al. (2021); Schick and Schütze (2021) into a BERT encoder.
For simplicity, suppose there only exists 4 utterances in $\mathcal{S}$, the
input format is ${\rm
BERT}(\\{{\rm[C]}\,u_{1}^{q}\,{\rm[S]}\,{\rm[V_{1}]}\,u_{2}^{a}\,{\rm[S]}\,u_{3}^{q}\,[S]\,{\rm[V_{2}]}\,u_{4}^{a}\,{\rm[S]}\\})$,
where ${\rm[C]}$, ${\rm[S]}$, ${\rm[V_{1}]}$, ${\rm[V_{2}]}$ represents the
CLS token, the SEP token, and the prompt template tokens that are inserted
before the candidate answers {$u_{2}^{a}$, $u_{4}^{a}$}, respectively. Then we
derive the answer scores $s_{k}^{a}={\rm sigmoid}({\rm
MLP}(h^{\rm[V_{k}]},h^{\rm[C]}))(k=1,2)$ for {$u_{2}^{a}$, $u_{4}^{a}$}. We
select the answer with the highest score for $u_{1}^{q}$ only if the score is
larger than a threshold, i.e. 0.75, otherwise we posit no valid answer exists
in the chatlog for $u_{1}^{q}$. Offline experiments show that the proposed QA
extractor outperforms multiple QA extraction baselines Devlin et al. (2019);
Jia et al. (2020). The training data for QA extraction are annotated by the
crowdsource workers on an internal platform of Tencent comparable to AMT.

#### Utterance Semantic Clustering

After filtering trivial information in the dialog (Fig 3), we represent the
utterances with the dense vectors using the pretrained sentence encoders
Reimers and Gurevych (2019) which are compatible with efficient query
retrieval on GPUs Johnson et al. (2019). Then we assemble the utterances that
are similar in the semantics with the K-means algorithm. In each assembled
cluster, we set the semantic centroid query as the anchor and use a dedicated
BERT sentence-pair matching model Devlin et al. (2019) that is pretrained on
an internal query-query matching corpus to filter the utterances with the low
relevance to the anchor query in the cluster.

#### Keywords and Semantic Classifier

We use the open-source topmine El-Kishky et al. (2014) toolkit to extract
keywords from customer or sales utterances in the customer objection response
module. For the trigger and spotlight detection in the sales manager dashboard
module, the intent classifier for a specific sales groups is tailored and has
a domain-specific intent vocabulary. For example, for the sales teams in the
online education domain, the pre-defined intents include “affordability”,
“competitors”, “eagerness to learn”, “curriculum” , “lack of time”, etc. The
intent classifier also supports domain-specific keyword matching, e.g. “new
energy car”, “driver asisstance”, “intelligent vehicle” for the car sales
teams.

##  4 Conclusion

We propose the system to automatically extract and analyze winning sales
scripts from the enormous customer-sales chatlog using advanced NLP
technologies for both sales representatives and managers. consists of three
modules: 1) customer FAQ extraction aims to enrich the FAQ knowledge base by
mining question-answer pairs from chatlog; 2) customer objection response
helps the sales to figure out the typical customer objections and the best
practises to respond; 3) sales manager dashboard depicts the performances for
different sales staffs or teams. The system has been running on the Tencent
Cloud for several sales teams from different domains.

## References

  * Devlin et al. (2019) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova.  BERT: Pre-training of deep bidirectional transformers for language understanding.  In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171–4186, Minneapolis, Minnesota, June 2019. Association for Computational Linguistics.
  * El-Kishky et al. (2014) Ahmed El-Kishky, Yanglei Song, Chi Wang, Clare R Voss, and Jiawei Han.  Scalable topical phrase mining from text corpora.  Proceedings of the VLDB Endowment, 8(3), 2014.
  * Gliwa et al. (2019) Bogdan Gliwa, Iwona Mochol, Maciej Biesek, and Aleksander Wawer.  SAMSum corpus: A human-annotated dialogue dataset for abstractive summarization.  In Proceedings of the 2nd Workshop on New Frontiers in Summarization, pages 70–79, Hong Kong, China, November 2019. Association for Computational Linguistics.
  * Hambardzumyan et al. (2021) Karen Hambardzumyan, Hrant Khachatrian, and Jonathan May.  WARP: Word-level Adversarial ReProgramming.  In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 4921–4933, Online, August 2021. Association for Computational Linguistics.
  * Humphrey and Ashforth (1994) Ronald H Humphrey and Blake Ashforth.  Cognitive scripts and prototypes in service encounters.  Advances in services marketing and management, 3(C):175–199, 1994\.
  * Jia et al. (2020) Qi Jia, Mengxue Zhang, Shengyao Zhang, and Kenny Q Zhu.  Matching questions and answers in dialogues from online forums.  In ECAI 2020, pages 2046–2053. IOS Press, 2020.
  * Johnson et al. (2019) Jeff Johnson, Matthijs Douze, and Hervé Jégou.  Billion-scale similarity search with gpus.  IEEE Transactions on Big Data, 7(3):535–547, 2019.
  * Leigh and McGraw (1989) Thomas W Leigh and Patrick F McGraw.  Mapping the procedural knowledge of industrial sales personnel: A script-theoretic investigation.  Journal of Marketing, 53(1):16–34, 1989.
  * Leong et al. (1989) Siew Meng Leong, Paul S Busch, and Deborah Roedder John.  Knowledge bases and salesperson effectiveness: A script-theoretic analysis.  Journal of Marketing Research, 26(2):164–178, 1989.
  * Li et al. (2021) Zhongli Li, Qingyu Zhou, Chao Li, Ke Xu, and Yunbo Cao.  Improving BERT with syntax-aware local attention.  In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 645–653, Online, August 2021. Association for Computational Linguistics.
  * Qin and Eisner (2021) Guanghui Qin and Jason Eisner.  Learning how to ask: Querying LMs with mixtures of soft prompts.  In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 5203–5212, Online, June 2021. Association for Computational Linguistics.
  * Reimers and Gurevych (2019) Nils Reimers and Iryna Gurevych.  Sentence-BERT: Sentence embeddings using Siamese BERT-networks.  In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3982–3992, Hong Kong, China, November 2019. Association for Computational Linguistics.
  * Schick and Schütze (2021) Timo Schick and Hinrich Schütze.  Exploiting cloze-questions for few-shot text classification and natural language inference.  In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pages 255–269, Online, April 2021. Association for Computational Linguistics.
  * Sun et al. (2019) Kai Sun, Dian Yu, Jianshu Chen, Dong Yu, Yejin Choi, and Claire Cardie.  DREAM: A challenge data set and models for dialogue-based reading comprehension.  Transactions of the Association for Computational Linguistics, 7:217–231, 2019.
  * Xu et al. (2021) Zenan Xu, Daya Guo, Duyu Tang, Qinliang Su, Linjun Shou, Ming Gong, Wanjun Zhong, Xiaojun Quan, Daxin Jiang, and Nan Duan.  Syntax-enhanced pre-trained model.  In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 5412–5422, Online, August 2021. Association for Computational Linguistics.
  * Yu et al. (2020) Dian Yu, Kai Sun, Claire Cardie, and Dong Yu.  Dialogue-based relation extraction.  In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4927–4940, Online, July 2020. Association for Computational Linguistics.


# PySBD: Pragmatic Sentence Boundary Disambiguation

Nipun Sadvilkar
Episource LLC
<EMAIL_ADDRESS>
Mark Neumann
Allen Institute for Artificial Intelligence
<EMAIL_ADDRESS>

###### Abstract

In this paper, we present a rule-based sentence boundary disambiguation Python
package that works out-of-the-box for 22 languages. We aim to provide a
realistic segmenter which can provide logical sentences even when the format
and domain of the input text is unknown. In our work, we adapt the _Golden
Rules Set_ (a language specific set of sentence boundary exemplars) originally
implemented as a ruby gem _pragmatic_segmenter_
111https://github.com/diasks2/pragmatic˙segmenter which we ported to Python
with additional improvements and functionality. PySBD passes 97.92% of the
Golden Rule Set examplars for English, an improvement of 25% over the next
best open source Python tool.

##  1 Introduction

Sentence Boundary Disambiguation (SBD), also known as sentence boundary
detection, is a key underlying task for natural language processing. In many
NLP pipelines, gold standard SBD often assumed, and acts as a primary input to
downstream NLP tasks such as machine translation, named entity recognition and
coreference resolution. However, in real world scenarios, text occurs in a
variety of input modalities, such as HTML forms, PDFs and word processing
doccument formats.

Although SBD is considered to be a simple problem, it becomes more complex in
other domains due to unorthodox use of punctuation symbols. For example, drug
names in medical documents, case citations in legal text and references in
academic articles all use punctuation in ways which are uncommon in newswire
documents. Simple SBD approaches for English web text (treating - “?!:;.” - as
end of sentence markers) covers a majority of cases, but an ideal SBD system
should be able to disambiguate these edge case scenarios and be robust to
known textual variation.

Our contributions in this paper are describing an open-source, freely
available tool for pragmatic sentence boundary disambiguation. In particular,
we describe the implementation details of PySBD, evaluate it in comparison to
other open source SBD tools and discuss it’s natural extensibility due to it’s
rule based nature.

##  2 Related Work

Sentence segmentation methods can be broadly divided into 3 approaches: i)
rules-based, requiring hand crafted rules/heuristics; ii) Supervised machine
learning, requiring annotated datasets, and iii) Unsupervised machine
learning, requiring distributional statistics derived from raw text.

Palmer and Hearst (1997) use decision trees and neural networks in a
supervised, feature based SBD model, requiring part of speech information and
training data. Kiss and Strunk (2006) design punkt, an unsupervised SBD model
centered around the observation that abbreviations are the main confounders
for rule based sentence boundary models. Although it is unsupervised, punkt
requires the computation of various co-occurence and distributional statistics
of a relevant corpora; as PySBD is rule-based, it does not require a initial
corpus of text. Evang et al. (2013) cast SBD as a character sequence labelling
problem and use features from a recurrent neural network language model to
train a CRF labelling model.

Many SBD papers reject rule-based approaches due to non-robustness,
maintainability and performance. We reject these conclusions, and instead
focus on the positive features of rule-based systems - namely that their
errors are interpretable, rules can be adjusted incrementally and their
performance is often on-par with learnt statistical models.

#### Issues with benchmarks on PTB/WSJ corpora

SBD systems have historically been benchmarked on the Wall Street Journal/Penn
Treebank corpora (Read et al., 2012). The majority of the sentences found in
the Penn Tree Bank are sentences that end with a regular word followed by a
period, testing the same sentence boundary cases repeatedly. In the Brown
Corpus 90% of potential sentence boundaries come after a regular word.
Although the Wall Street Journal corpus is richer with numerical values,
abbreviations and only 53% according to (Gale and Church, 1993) of sentences
end with a regular word followed by a period (Mikheev, 2002).

Given that commonly used training/evaluation corpora do not contain a
particularly large amount of sentence marker variation, we use a _Golden Rule
Set_ to enumerate edge cases observed in sentence boundaries. The _Golden Rule
Set_ contains 48 hand-constructed rules, designed to cover sentence boundaries
across a variety of domains. The GRS is interpretable (each rule targets a
specific type of sentence boundary) and easy to extend with new examples of
particular sentence boundary markers.

##  3 Implementation

PySBD is divided into four high level components: The Segmenter, Processor,
Language and Cleaner sub-modules.

The _Segmenter_ class is the public API to PySBD. It allows a user to set up a
_Segmenter_ in their language of choice, as well as specify additional options
such as text cleaning and char_span functionality. The _Segmenter_ requires a
two character ISO 639-1
code222https://en.wikipedia.org/wiki/List˙of˙ISO˙639-1˙codes to process input
text. Text extracted from a PDF or obtained from OCR systems typically
contains unusually formatted text, such as line breaks in the middle of
sentences. This can be handled with the _doc_type_ option, or for more
aggressive text cleaning, the _clean_ functionality performs additional pre-
filtering of the input text, removing repeated and unnecessary punctuation.

The _Processor_ contains the sentence segmentation logic, using rules to
segment the input text. The _Processor_ contains several groups of sentence
segmentation rules, some of which are universal across languages, and some of
which are language specific. These are grouped as follows:

  * •

Common

  * •

Standard

  * •

ListItemReplacer

  * •

AbbreviationReplacer

  * •

ExclamationWords

  * •

BetweenPunctuation

The _Processor_ identifies sentence boundaries by manipulating input text in 3
stages. Firstly, rules are applied to alter the input text by adding
intermediate unicode characters as placeholders to signify that particular
pieces of punctuation are not sentence boundaries. The segment stage
identifies true sentence boundaries by bypassing unicode characters and
segments text into sentences using a much simpler regex rule. Finally, the
manipulated text is transformed into original text form by replacing the
unicode placeholders with their original characters.

The _Language_ holds all the languages supported by PySBD. Each language is
built on top of two sub-components - _Common_ and _Standard_ \- involving
basic rules prevalent across languages. _Common_ rules encompass the main
sentence boundary regexes; AM-PM regexes handle numerically expressed time
periods; number regexes handle period/newline characters before or after
single/multi-digit numbers and additional rules handle quotation, parenthesis,
and numerical references within the input text. The _Standard_ rule set
contains regex patterns to handle single/double punctuation, geolocation
references, fileformat mentions and ellipsis in input text. The
_ListItemReplacer_ rule set handles itemized, ordered/unordered lists; the
_AbbreviationReplacer_ contains language specific common abbreviations.
Finally, the _ExclaimationWords_ and _BetweenPunctuation_ rules handle
language specific exclamations and more complicated punctuation cases.

In practice, text encountered in the wild is noisy, containing extraneous line
breaks, unicode characters, uncommon spacing and hangovers from document
structure. In order to handle this, PySBD provides an optional component to
handle such texts. The _Cleaner_ is passed as an option through the top-level
_Segmenter_ component and provides text cleaning rules for cases like
irregular newline characters, tables of contents, URLs, HTML tags and text
involving no space between sentences. As the text cleaning rules perform a
destructive operation, this feature is incompatible with the _char_span_
functionality, as mapping back to character indices within the original text
is no longer possible.

##  4 Experimental Setup

#### Data

Contrary to the WSJ, Brown and GENIA datasets mentioned in (Read et al.,
2012), we use language specific _Golden Rules Sets_ for our experiments. There
are total 22 Golden Rules sets for following languages - English, Marathi,
Hindi, Bulgarian, Español (Spanish), Russian, Arabic, Amharic, Armenian,
Persian, Urdu, Polish, Chinese, Dutch, Danish, French, Italian, Greek,
Burmese, Japanese, Deutsch (Germen), Kazakh.

The _Golden Rules Sets_ are devised by considering possible sentence
boundaries per language as well as considering different domains. For example
the English language GRS is comprised of 48 golden rules 333Golden Rules from
pragmatic_segmenter from formal and informal domains to cover a wide variety
of phenomena. For example, news articles are grammatically and punctually
correct; scientific literature often involves numbers, abbreviations and
bibliography references, and Informal domain like Web Text - E-mail, Social
media text involves irregular punctuation and ellipses. To ensure our rules-
based system built with respect to the _Golden Rules Set_ generalizes well in
the real world, we have also performed a benchmark comparison on the GENIA
corpus Kim et al. (2003), a dataset of linguistic annotations on top of the
abstracts of biomedical papers. The GENIA corpus provides both raw and
segmented abstracts, which we use as natural data for our evaluation.

#### Setup

We evaluate PySBD and other segmentation tools on two corpora - the _English
Golden Rules Set_ and GENIA corpus.

##  5 Comparison to alternatives

Table 1 summarizes accuracy of PySBD and the alternative Python SBD modules on
_English Golden Rules Set_ and the GENIA corpus. The supervised machine
learning based sentence segmenters, stanza Qi et al. (2020) and spacy
dependency parsing (spacy dep) Honnibal and Montani (2017) are slower compared
to other python modules and seem to segment incorrectly when text contains
mixed case words or abrupt punctuation within words, which is prevalent in
biomedical domain. Inability to generalise on out of domain corpora is a main
drawback of using supervised learning for SBD.

NLTK’s Bird (2006) _PunktSentenceTokenizer_ is based on an unsupervised
algorithm Kiss and Strunk (2006) and fails to segment text containing
brackets, itemized text and abbreviations as sentence boundaries. In contrast,
practically all the modules following a rules-based approach - blingfire Bling
Team (2020), syntok Leitner (2020) and PySBD - appear to be faster and more
accurate on both corpora. Blingfire and syntok modules struggle when text has
decimal numbers, abbreviations, brackets and mixed cased words prior to or
following the true sentence boundary. Lastly, the 3% drop in PySBD’s accuracy
on the GENIA corpus is caused by splitting itemized text into segments,
whereas the GENIA abstracts contain single sentences with multiple bulleted
lists. We feel that this segmentation choice comes down to preference, and
both are equally valid.

Table 2 shows the runtime performance of each module on the entire text of
“The Adventures of Sherlock Holmes”
444http://www.gutenberg.org/files/1661/1661-0.txt, which contains 10K lines
and 100K words. The experiment was performed on Intel Core i5 processor
running at 2.9 GHz clock speed. Blingfire module is the fastest since it is
extremely performance oriented (at cost of reduced
maintainability/extensibility). Although PySBD is slower than several
alternatives, it is considerably faster than running full pipelines and is a
good choice for users who require high accuracy segmentations. Our comparisons
cover a variety of different implementations (C++, Cython, Pytorch),
approaches (model-based, distributional, rule-based) and are representative of
practical choices for an NLP practitioner.

Tool | GRS | GENIA
---|---|---
blingfire | 75.00 | 86.95
syntok | 68.75 | 80.90
spaCy | 52.08 | 76.80
spacy dep | 54.17 | 39.20
stanza | 72.92 | 63.40
NLTK | 56.25 | 87.95
PySBD | 97.92 | 97.00
Table 1: Accuracy (%) of PySBD compared to other open source SBD packages with respect to the English _Golden Rule Set_ and the GENIA corpus.  Tool | Speed(ms)
---|---
blingfire | 85.24
syntok | 1764.11
spaCy | 1523.20
spacy dep | 26850.69
stanza | 48383.46
NLTK | 780.49
PySBD | 9483.96
Table 2: Speed benchmark on the entire text of “The adventures of Sherlock
Holmes” for PySBD compared to other open source SBD packages.

##  6 Discussion

###  6.1 Package Development

We use Test-driven Development (TDD) whilst developing, first write a test for
one of the rule from _Golden Rules Set_ that fails intentionally, before
writing functional code to pass the test. The approach used by our python
module is rules-based. We employ Python’s standard library module called
re555https://docs.python.org/3/library/re.html which provides regular
expression (regex) matching operations for text processing.

###  6.2 Non-destructive Segmentation

Our module does non-destructive sentence tokenization, as when dealing with
noisy text, character offsets into the original documents are often desirable.
The indices are obtained after postprocess stage by mapping post-processed
sentence start indices into the original input text. Upon enabling this
feature, the output format is a list of _TextSpan_ objects containing
sentence, start and end character indices. The character spans makes it easy
to navigate to any sentence within the unaltered original text.

###  6.3 Multilingual Implementation

NLP research predominantly focuses on developing dataset and methods for
English language despite the many benefits of working on other languages Ruder
(2020). An advantage of PySBD’s rule-based approach is straightforward
extension to new languages. PySBD has support for 22 languages spanning many
language families, each having its own _Golden Rules Set_. Adding support for
a new language involves adding language specific rules to the Golden Rules Set
and adding language specific punctuation markers to a new language module.
This modular language support allows PySBD to be maintained in a community
driven way by open source NLP practitioners. For example, we extended the
original Ruby pragmatic segmeter by adding support for Marathi by forming the
Golden Rules and identifying Marathi language specific sentence syntax,
punctuations and abbreviations resulting in a usable module within 1 hour of
work. If you would like to contribute to the PySBD module by updating existing
GRS or by adding support for new language then refer to our contributing
guidelines666PySBD Contributing Guidelines to get you started.

Language | Accuracy (%)
---|---
Amharic | 80.95%
Arabic | 70.40%
Armenian | 63.75%
Bulgarian | 93.35%
Burmese | 48.05%
Chinese | 85.35%
Danish | 91.40%
Deutsch | 80.95%
Dutch | 91.40%
French | 91.90%
Greek | 91.05%
Hindi | 88.50%
Italian | 90.55%
Japanese | 96.45%
Kazakh | 63.20%
Marathi | 92.60%
Persian | 84.95%
Polish | 55.48%
Russian | 88.55%
Spanish | 92.65%
Urdu | 77.55%
Table 3: Accuracy of PySBD’s multilingual modules on the OPUS-100 multilingual
corpus test sets, containing 2000 sentences per language. Each language module
build with respect to its own GRS.

We benchmarked PySBD on the OPUS-100 parallel multilingual corpus, covering
100 languages Tiedemann (2012). We used the test sets of 21 of the languages
excluding English which contain 2000 sentences per language (Due to
unavailability of the test set for Armenian, we used its train set, containing
7000 sentences). Due to noisy nature of OPUS (on inspection, multiple
sentences were present on individual lines in the test sets) and lack of
language specific knowledge to form rules and abbreviation list we observed
weak performance in a few languages like Burmese, Polish, Kazakh, Armenian,
etc. Shortcomings of such languages can be improved in community driven way by
collaborating with multilingual NLP practitioners.

##  7 Conclusion

In this paper, we have described PySBD, a pragmatic sentence boundary
disambiguation model. PySBD is open source, has over 98% test coverage and
integrates easily with existing natural language processing pipelines. PySBD
currently supports 22 languages, and is easily extensible, with 57 projects
depending on it at the time of writing. Although slower than some alternatives
implemented in low level languages such as C++, PySBD successfully
disambiguates 97% of sentence boundaries in a Golden Rule Set and is robust
across domains and noisy text.

## References

  * Bird (2006) Steven Bird. 2006.  Nltk: The natural language toolkit.  _ArXiv_ , cs.CL/0205028.
  * Bling Team (2020) Beyond Language Understanding Bling Team, Microsoft. 2020.  Blingfire : A lightning fast finite state machine and regular expression manipulation library.  Github: BlingFire sbd repo.
  * Evang et al. (2013) Kilian Evang, Valerio Basile, Grzegorz Chrupała, and Johan Bos. 2013.  Elephant: Sequence labeling for word and sentence segmentation.  In _Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing_ , pages 1422–1426, Seattle, Washington, USA. Association for Computational Linguistics.
  * Gale and Church (1993) William A. Gale and Kenneth W. Church. 1993.  A program for aligning sentences in bilingual corpora.  _Computational Linguistics_ , 19(1):75–102.
  * Honnibal and Montani (2017) Matthew Honnibal and Ines Montani. 2017.  spaCy 2: Natural language understanding with Bloom embeddings, convolutional neural networks and incremental parsing.  https://github.com/explosion/spaCy.  To appear.
  * Kim et al. (2003) Jin-Dong Kim, Tomoko Ohta, Yuka Tateisi, and Jun’ichi Tsujii. 2003.  Genia corpus - a semantically annotated corpus for bio-textmining.  _Bioinformatics_ , 19 Suppl 1:i180–2.
  * Kiss and Strunk (2006) Tibor Kiss and Jan Strunk. 2006.  Unsupervised multilingual sentence boundary detection.  _Computational Linguistics_ , 32(4):485–525.
  * Leitner (2020) Florian Leitner. 2020.  Syntok : Text tokenization and sentence segmentation (segtok v2).  https://github.com/fnl/syntok.
  * Mikheev (2002) Andrei Mikheev. 2002.  Periods, capitalized words, etc. _Computational Linguistics_ , 28(3):289–318.
  * Palmer and Hearst (1997) David D. Palmer and Marti A. Hearst. 1997.  Adaptive multilingual sentence boundary disambiguation.  _Computational Linguistics_ , 23(2):241–267.
  * Qi et al. (2020) Peng Qi, Yuhao Zhang, Yuhui Zhang, Jason Bolton, and Christopher D. Manning. 2020\.  Stanza: A python natural language processing toolkit for many human languages.  In _ACL_.
  * Read et al. (2012) Jonathon Read, Rebecca Dridan, Stephan Oepen, and Lars Jørgen Solberg. 2012.  Sentence boundary detection: A long solved problem? _Computational Linguistics_ , pages 985–994.
  * Ruder (2020) Sebastian Ruder. 2020.  Why You Should Do NLP Beyond English.  http://ruder.io/nlp-beyond-english.
  * Tiedemann (2012) Jörg Tiedemann. 2012.  Parallel data, tools and interfaces in opus.  In _LREC_.