The Bel-Robinson tensor and the classical pseudotensors

Lau Loi So

###### Abstract

Describing the gravitational energy and momentum, the Bel-Robinson tensor is
the best tensor. However, the classical pseudotensors can also manage the the
same job. As Deser mentioned in 1999, a certain linear combination between
Einstein pseudotensor and Landau-Lifshitz pseudoetnsor give a pure Bel-
Robinson tensor. Here we used the same idea but adapted the harmonic gauge, we
found that all the classical pseudotensor alone cannot give a multiple of the
Bel-Robinson tensor. But under a modification, all of them contribute the same
energy-momentum and stress as the Bel-Robinson tensor does.

##  1 Introduction

The Bel-Robinson tensor is the best tensor to describe the energy-momentum and
stress in general relativity. Senovilla had written a very nice article about
this tensor [1]. This tensor has many very nice mathematical properties:
completely symmetric, trace free and divergence free. In 1999, Deser [2]
explained how to make a certain linear combination between Einstein [3]
pseudotensor and Landau-Lifshitz [4] pseudotensor to produce the pure Bel-
Robinson tensor, under the Riemann normal coordinates systems. In fact, we had
examined that all the classical pseudotensor alone cannot generate a multiple
of the Bel-Robinson tensor [5]. Our present paper used the same idea, but
using another angle, apply the harmonic gauge and using the physical detail of
the metric. As expected, we found none of the classical pseudotensor can have
a multiple of the pure Bel-Robinson tensor. But after adding some
modification, the results indicate that each classical pseudotensor does have
the pure Bel-Robinson tensor, restricted in the energy-momentum and stress.
When we say the classical pseudotensors which means Einstein, Landau-Lifshitz,
Bergmann-Thomson [6], Papapetrou [7] and Weinberg [8].

##  2 Technical background

Throughout this work, we use the same spacetime signature and notation in MTW
[9], including the geometrical units $G=c=1$, where $G$ and $c$ are the
Newtonian constant and the speed of light. The Greek letters denote the
spacetime and Latin letters refer to spatial. In vacuum, the Bel-Robinson
tensor can be defined as follows

$\displaystyle
B_{\alpha\beta\mu\nu}:=R_{\alpha\lambda\mu\sigma}R_{\beta}{}^{\lambda}{}_{\nu}{}^{\sigma}+R_{\alpha\lambda\nu\sigma}R_{\beta}{}^{\lambda}{}_{\mu}{}^{\sigma}-\frac{1}{8}g_{\alpha\beta}g_{\mu\nu}R^{2}_{\lambda\sigma\rho\tau},$
(1)

where
$R^{2}_{\lambda\sigma\rho\tau}=R_{\lambda\sigma\rho\tau}R^{\lambda\sigma\rho\tau}$.
In order to extract the energy-momentum and the stress, one can use the analog
of the “electric” $E_{ab}$ and “magnetic” $B_{ab}$ parts of the Weyl tensor
[10],

$\displaystyle E_{ab}:=C_{0a0b},\quad B_{ab}:=\ast{C_{0a0b}},$ (2)

where $C_{\alpha\beta\mu\nu}$ is the Weyl conformal tensor and
$\ast{C_{\alpha\beta\mu\nu}}$ is its dual,

$\ast{C_{\alpha\beta\mu\nu}}=\frac{1}{2}\epsilon_{\alpha\beta\lambda\sigma}C^{\lambda\sigma}{}_{\mu\nu}.$
(3)

(Here $\epsilon_{\alpha\beta\mu\nu}=\epsilon_{[\alpha\beta\mu\nu]}$ with
$\epsilon_{0123}=\sqrt{-g}$ is the totally anti-symmetric Levi-Civita tensor,
see [9], in particular Eq. 8.10 and Ex. 8.3.) In vacuum using the Riemann
tensor

$E_{ab}=R_{0a0b},\quad B_{ab}=*R_{0a0b}.$ (4)

Certain commonly occurring quadratic combinations of the Riemann tensor
components in terms of the electric $E_{ab}$ and magnetic $B_{ab}$ parts in
vacuum are

$R_{0a0b}R_{0}{}^{a}{}_{0}{}^{b}=E_{ab}E^{ab},\quad
R_{0abc}R_{0}{}^{abc}=2B_{ab}B^{ab},\quad R_{abcd}R^{abcd}=4E_{ab}E^{ab}.$ (5)

In particular, the Riemann squared tensor can then be written

$R^{2}_{\lambda\sigma\rho\tau}=8(E_{ab}E^{ab}-B_{ab}B^{ab}).$ (6)

The energy-momentum and stress for the Bel-Robinson tensor are

$\displaystyle B_{0000}$ $\displaystyle=$ $\displaystyle
E^{2}_{ab}+B^{2}_{ab},$ (7) $\displaystyle B_{000j}$ $\displaystyle=$
$\displaystyle 2\epsilon_{jab}E^{ac}B^{b}{}_{c},$ (8) $\displaystyle B_{00ij}$
$\displaystyle=$
$\displaystyle\eta_{ij}(E^{2}_{ab}+B^{2}_{ab})-2(E^{c}{}_{i}E_{cj}+B^{c}{}_{i}B_{cj}),$
(9)

where $E^{2}_{ab}:=E_{ab}E^{ab}$ and similarly for $B^{2}_{ab}$.

Within a weak field the metric tensor can be decomposed as
$g_{\alpha\beta}=\eta_{\alpha\beta}+h_{\alpha\beta}$, and its inverse is
$g^{\alpha\beta}=\eta^{\alpha\beta}-h^{\alpha\beta}$. Here we only consider
the lowest order (see (8) at [11]), the metric components can be written as

$\displaystyle
h^{00}=-E_{ab}x^{a}x^{b},\quad{}h^{0j}=\frac{2}{3}\epsilon^{j}{}_{pq}B^{p}{}_{l}x^{q}x^{l},\quad
h^{ij}=\eta^{ij}h^{00}.$ (10)

The lowest order for the harmonic gauge is

$\displaystyle\Gamma^{\alpha\beta}{}_{\beta}=\partial_{\beta}\bar{h}^{\alpha\beta}=0,$
(11)

where $\bar{h}^{\alpha\beta}=h^{\alpha\beta}-\frac{1}{2}\eta^{\alpha\beta}h$.
Thus, the energy-momentum and the stress for the Bel-Robinson tensor can be
written as the following form

$\displaystyle B^{0000}$ $\displaystyle=$
$\displaystyle\frac{1}{4}\vec{\nabla}^{2}\left[\frac{\alpha}{2}-\frac{A}{2}+\frac{B}{2}\right],$
(12) $\displaystyle B^{000j}$ $\displaystyle=$
$\displaystyle\frac{1}{4}\vec{\nabla}^{2}(C-D),$ (13) $\displaystyle B^{00ij}$
$\displaystyle=$
$\displaystyle\frac{1}{4}\vec{\nabla}^{2}\left[-X-Y+Z-U-V+\eta^{ij}\left(\frac{\alpha}{2}+\frac{A}{2}-\frac{B}{2}\right)\right],$
(14)

where

$\displaystyle\alpha=(\partial_{c}h_{00})(\partial^{c}h_{00}),\quad{}A=(\partial_{c}h_{0d})(\partial^{c}h^{0d}),\quad{}B=(\partial_{c}h_{0d})(\partial^{d}h^{0c}),$
(15) $\displaystyle
C=(\partial_{c}h_{00})(\partial^{c}h^{0j}),\quad{}D=(\partial_{c}h_{00})(\partial^{j}h^{0c}),$
(16) $\displaystyle
X=(\partial^{k}h_{00})(\partial^{f}h_{00}),\quad{}Y=(\partial^{k}h^{0c})(\partial^{f}h_{0c}),\quad{}Z=(\partial_{c}h^{0k})(\partial^{c}h^{0f}),$
(17) $\displaystyle
U=(\partial^{k}h^{0c})(\partial_{c}h^{0f}),\quad{}V=(\partial_{c}h^{0k})(\partial^{f}h^{0c}).$
(18)

On the other hand, generally speaking, regarding the classical pseudotensor
[5] which can be obtained from a rearrangement of the Einstein field equation:
$G^{\mu\nu}=\kappa{}T^{\mu\nu}$, where the constant $\kappa=8\pi{}G/c^{4}$,
$G^{\mu\nu}$ and $T^{\mu\nu}$ are the Einstein and stress tensors. We define
the gravitational energy-momentum density pseudotensor in terms of a suitable
superpotential $U_{\alpha}{}^{[\mu\nu]}$:

$\displaystyle
2\kappa\sqrt{-g}\,t_{\alpha}{}^{\mu}:=\partial_{\nu}U_{\alpha}{}^{[\mu\nu]}-2\sqrt{-g}\,G_{\alpha}{}^{\mu}.$
(19)

Alternatively, one can rewrite (19) as
$\partial_{\nu}U_{\alpha}{}^{[\mu\nu]}=\sqrt{-g}(2G_{\alpha}{}^{\mu}+2\kappa{}t_{\alpha}{}^{\mu})$.
Using the Einstein equation, the total energy-momentum density complex can be
defined as

$\displaystyle{\cal{T}}_{\alpha}{}^{\mu}:=\sqrt{-g}(T_{\alpha}{}^{\mu}+t_{\alpha}{}^{\mu})=(2\kappa)^{-1}\partial_{\nu}U_{\alpha}{}^{[\mu\nu]}.$
(20)

This total energy-momentum density is automatically conserved as
$\partial_{\mu}{\cal{T}}_{\alpha}{}^{\mu}\equiv{}0$, which can be split into
two parts:

$\displaystyle(\partial_{\nu}U_{\alpha}{}^{[\mu\nu]})_{1}=2\sqrt{-g}\,G_{\alpha}{}^{\mu},\quad{}(\partial_{\nu}U_{\alpha}{}^{[\mu\nu]})_{2}=2\sqrt{-g}\kappa\,t_{\alpha}{}^{\mu}.$
(21)

The first part indicates the contribution of matter and the second piece
refers to vacuum gravity [9]. For the equivalence principle small limit at a
point within matter, it is known that the Einstin, Landau-Lifshitz, Bergmann-
Thomson, Papapetrou and Weinberg pseudotensors all satisfy the standard result
$2G_{\alpha}{}^{\mu}$ at the origin in Riemann normal coordinates, while
$t_{\alpha}{}^{\mu}$ vanishes in this limit [5].

Actually there are three kinds of superpotentials: $U_{\alpha}{}^{[\mu\nu]}$,
$U^{\alpha[\mu\nu]}$ and $H^{[\alpha\beta][\mu\nu]}$. The latter also must
satisfy some further symmetries, under the interchange of the pairs, and the
total antisymmetry of 3 indices (i.e., the same as the symmetries of the
Riemann tensor). From them the total energy-momentum complex can be obtained
as follows:

$\displaystyle
2\kappa{\cal{T}}_{\alpha}{}^{\mu}=\partial_{\nu}U_{\alpha}{}^{[\mu\nu]},\quad{}2\kappa{\cal{T}}^{\alpha\mu}=\partial_{\nu}U^{\alpha[\mu\nu]},\quad{}2\kappa{\cal{T}}^{\alpha\mu}=\partial^{2}_{\beta\nu}H^{[\alpha\beta][\mu\nu]}.$
(22)

In vacuum, $t^{00}$ is the gravitational energy density, $t^{0j}$ is the
momentum and $t^{ij}$ is the stress. In short: we use the following for the
representation

$\displaystyle t^{\alpha\mu}=(t^{00},~{}t^{0j},~{}t^{ij})$ (23)

##  3 The classical pseudotensors

###  3.1 Einstein pseudotensor

The following is the Freud (F) superpotential [3]:

${}_{F}U_{\alpha}{}^{[\mu\nu]}:=-\sqrt{-g}g^{\beta\sigma}\Gamma^{\tau}{}_{\lambda\beta}\delta^{\lambda\mu\nu}_{\tau\sigma\alpha}.$
(24)

The corresponding Einstein pseudotensor is

$\displaystyle t^{\alpha\mu}_{E}$ $\displaystyle=$
$\displaystyle-2\Gamma^{\beta\lambda\alpha}\Gamma^{\mu}{}_{\beta\lambda}+\Gamma^{\mu\alpha\lambda}\Gamma^{\beta}{}_{\beta\lambda}+\Gamma^{\beta\alpha}{}_{\beta}\Gamma^{\mu\lambda}{}_{\lambda}$
(25)
$\displaystyle+\eta^{\alpha\mu}(\Gamma^{\beta\lambda}{}_{\nu}\Gamma^{\nu}{}_{\beta\lambda}-\Gamma^{\beta}{}_{\beta\nu}\Gamma^{\nu\lambda}{}_{\lambda})+\Gamma^{\nu\alpha\mu}\Gamma^{\beta}{}_{\beta\nu}-\Gamma^{\beta\alpha}{}_{\beta}\Gamma^{\lambda\mu}{}_{\lambda}.$

Apply the harmonic gauge, the energy-momentum and the stress are

$\displaystyle
t^{\alpha\mu}_{E}=-\left[\left(-\frac{\alpha}{2}-\frac{A}{2}+\frac{B}{2}\right),~{}0,~{}\eta^{kf}\left(\frac{\alpha}{2}+\frac{A}{2}-\frac{B}{2}\right)-X-Y-U\right].$
(26)

According to Table 1, we add the extra piece

$\displaystyle t^{\alpha\mu}_{\rm{New}}$ $\displaystyle=$ $\displaystyle
t^{\alpha\mu}_{E}+2t_{1}+2t_{2}-\frac{t_{3}}{2}+\frac{t_{5}}{2}-t_{9}-\frac{t_{11}}{2}$
(27) $\displaystyle=$
$\displaystyle-\left[\frac{\alpha}{2}-\frac{A}{2}+\frac{B}{2},~{}C-D,~{}\eta^{kf}\left(\frac{\alpha}{2}+\frac{A}{2}-\frac{B}{2}\right)-X-Y+Z-U-V\right].$

Then

$\displaystyle-\frac{1}{4}\vec{\nabla}^{2}t^{\alpha\mu}_{\rm{New}}=(B^{0000},B^{000j},B^{00ij}).$
(28)

###  3.2 Landau-Lifshitz pseudotensor

The Landau-Lifshitz (LL) superpotential [4]

$\displaystyle
U^{\alpha[\mu\nu]}_{LL}:=-(-g)g^{\alpha\beta}g^{\pi\sigma}\Gamma^{\tau}{}_{\lambda\pi}\delta^{\lambda\mu\nu}_{\tau\sigma\beta},$
(29)

and the pseudotensor becomes

$\displaystyle t^{\alpha\mu}_{LL}$ $\displaystyle=$ $\displaystyle
g^{\alpha\mu}(\Gamma^{\gamma}{}_{\gamma\nu}\Gamma^{\beta\nu}{}_{\beta}+\Gamma^{\nu\beta}{}_{\rho}\Gamma^{\rho}{}_{\nu\beta}-2\Gamma^{\gamma}{}_{\gamma\nu}\Gamma^{\nu\beta}{}_{\beta})+2\Gamma^{\nu\alpha\mu}\Gamma^{\beta}{}_{\beta\nu}-\Gamma^{\rho\alpha}{}_{\rho}\Gamma^{\beta\mu}{}_{\beta}$
(30)
$\displaystyle+\Gamma^{\alpha\rho}{}_{\beta}\Gamma^{\mu\beta}{}_{\rho}-\Gamma^{\rho\alpha}{}_{\beta}\Gamma^{\beta\mu}{}_{\rho}-\Gamma^{\alpha\rho}{}_{\beta}\Gamma^{\beta\mu}{}_{\rho}-\Gamma^{\rho\alpha}{}_{\beta}\Gamma^{\mu\beta}{}_{\rho}-(\Gamma^{\alpha\mu}{}_{\nu}+\Gamma^{\mu\alpha}{}_{\nu})\,\Gamma^{\beta\nu}{}_{\beta}$
$\displaystyle+\Gamma^{\alpha\rho}{}_{\rho}\Gamma^{\beta\mu}{}_{\beta}+\Gamma^{\rho\alpha}{}_{\rho}\Gamma^{\mu\beta}{}_{\beta}-\Gamma^{\alpha\rho}{}_{\rho}\Gamma^{\mu\beta}{}_{\beta}+(\Gamma^{\alpha\mu}{}_{\rho}+\Gamma^{\mu\alpha}{}_{\rho})\,\Gamma^{\rho\beta}{}_{\beta},$

Apply the harmonic gauge, we have the following results

$\displaystyle
t^{\alpha\mu}_{LL}=-\left[\frac{7\alpha}{2}+\frac{A}{2}+\frac{B}{2},~{}2C-2D,~{}\eta^{kf}\left(\frac{\alpha}{2}+\frac{A}{2}-\frac{B}{2}\right)-X-Y+Z-U-V\right].$
(31)

This result is not good, then consider the modification

$\displaystyle t^{\alpha\mu}_{\rm{New}}$ $\displaystyle=$ $\displaystyle
t^{\alpha\beta}_{LL}+2t_{1}+2t_{2}-t_{3}+\frac{3}{2}t_{5}-t_{11}$ (32)
$\displaystyle=$
$\displaystyle-\left[\frac{\alpha}{2}-\frac{A}{2}+\frac{B}{2},~{}C-D,~{}\eta^{kf}\left(\frac{\alpha}{2}+\frac{A}{2}-\frac{B}{2}\right)-X-Y+Z-U-V\right].$

Then we have the expected result

$\displaystyle-\frac{1}{4}\vec{\nabla}^{2}t^{\alpha\mu}_{\rm{New}}=(B^{0000},B^{000j},B^{00ij}).$
(33)

###  3.3 Bergmann-Thomson pseudotensor

The Bergmann-Thomson (BT) superpotential [6] is defined as

$\displaystyle
U^{\alpha[\mu\nu]}_{BT}:=-\sqrt{-g}g^{\alpha\beta}g^{\pi\sigma}\Gamma^{\tau}{}_{\lambda\pi}\delta^{\lambda\mu\nu}_{\tau\sigma\beta},$
(34)

and the associated pseudotensor is

$\displaystyle t^{\alpha\mu}_{BT}$ $\displaystyle=$ $\displaystyle
g^{\alpha\mu}\Gamma^{\beta\lambda}{}_{\rho}\Gamma^{\rho}{}_{\beta\lambda}-(g^{\alpha\mu}\Gamma^{\beta}{}_{\beta\rho}-\Gamma^{\alpha\mu}{}_{\rho}-\Gamma^{\mu\alpha}{}_{\rho})\Gamma^{\rho\nu}{}_{\nu}+\Gamma^{\alpha\nu}{}_{\nu}(\Gamma^{\beta\mu}{}_{\beta}-\Gamma^{\mu\beta}{}_{\beta})$
(35)
$\displaystyle+(\Gamma^{\rho\alpha\mu}-\Gamma^{\alpha\mu\rho})\Gamma^{\beta}{}_{\beta\rho}+\Gamma^{\alpha\rho}{}_{\beta}(\Gamma^{\mu\beta}{}_{\rho}-\Gamma^{\beta\mu}{}_{\rho})-\Gamma^{\rho\alpha}{}_{\beta}(\Gamma^{\mu\beta}{}_{\rho}+\Gamma^{\beta\mu}{}_{\rho}).$

Apply the harmonic gauge, the results are

$\displaystyle
t^{\alpha\mu}_{\rm{New}}=-\left[\frac{3\alpha}{2}+\frac{A}{2}+\frac{B}{2},C-D,\eta^{kf}\left(\frac{\alpha}{2}+\frac{A}{2}-\frac{B}{2}\right)-X-Y+Z-U-V\right].$
(36)

This result requires some modification

$\displaystyle t^{\alpha\mu}_{\rm{New}}$ $\displaystyle=$ $\displaystyle
t^{\alpha\mu}_{BT}+2t_{1}+2t_{2}-\frac{t_{3}}{2}+\frac{3}{2}t_{5}-\frac{t_{11}}{2}$
(37) $\displaystyle=$
$\displaystyle-\left[\frac{\alpha}{2}-\frac{A}{2}+\frac{B}{2},C-D,\eta^{kf}\left(\frac{\alpha}{2}+\frac{A}{2}-\frac{B}{2}\right)-X-Y+Z-U-V\right].$

Then we have the expected result

$\displaystyle-\frac{1}{4}\vec{\nabla}^{2}t^{\alpha\mu}_{\rm{New}}=(B^{0000},B^{000j},B^{00ij}).$
(38)

###  3.4 Papapetrou pseudotensor

The Papapertrou superpotential [7] is defined as

$\displaystyle
H^{[\mu\nu][\alpha\beta]}_{P}:=-\sqrt{-g}g^{\rho\pi}\eta^{\tau\gamma}\delta_{\pi\gamma}^{\nu\mu}\delta_{\rho\tau}^{\alpha\beta},$
(39)

equivalently one can use
$U^{\alpha[\mu\nu]}_{P}\equiv\partial_{\beta}H^{[\mu\nu][\alpha\beta]}$. The
corresponding pseudotensor is

$\displaystyle t^{\alpha\mu}_{P}$ $\displaystyle=$
$\displaystyle\eta^{\alpha\mu}(\Gamma^{\beta\lambda}{}_{\rho}\Gamma^{\rho}{}_{\beta\lambda}+\Gamma^{\nu\beta}{}_{\nu}\Gamma^{\lambda}{}_{\lambda\beta}-\Gamma^{\beta}{}_{\beta\rho}\Gamma^{\rho\lambda}{}_{\lambda}+h^{\beta\nu}\Gamma^{\lambda}{}_{\lambda\beta,\nu})$
(40)
$\displaystyle+2\Gamma^{\alpha}{}_{\beta\lambda}\Gamma^{\mu\beta\lambda}+\Gamma^{\alpha\beta}{}_{\beta}\Gamma^{\lambda\mu}{}_{\lambda}+\Gamma^{\beta\alpha}{}_{\beta}\Gamma^{\mu\lambda}{}_{\lambda}-(\Gamma^{\alpha\mu}{}_{\beta}+\Gamma^{\mu\alpha}{}_{\beta})\,(2\Gamma^{\lambda\beta}{}_{\lambda}+\Gamma^{\beta\lambda}{}_{\lambda})$
$\displaystyle+h^{\alpha\beta}\partial_{\beta}\Gamma^{\mu\lambda}{}_{\lambda}+h^{\beta\mu}\partial_{\beta}\Gamma^{\alpha\lambda}{}_{\lambda}-h^{\alpha\mu}\partial_{\beta}\Gamma^{\beta\lambda}{}_{\lambda}-h^{\beta\lambda}(\Gamma^{\alpha\mu}{}_{\beta,\lambda}+\Gamma^{\mu\alpha}{}_{\beta,\lambda}).$

Apply the harmonic gauge, we have the following results

$\displaystyle
t^{\alpha\mu}_{P}=-\left[\frac{7\alpha}{2}+\frac{A}{2}+\frac{3B}{2},~{}2C-2D,~{}\eta^{kf}\left(\frac{\alpha}{2}+\frac{A}{2}-\frac{B}{2}\right)-X-Y+Z-U-V\right].$
(41)

Adding some extra pieces

$\displaystyle t^{\alpha\mu}_{\rm{New}}$ $\displaystyle=$ $\displaystyle
t^{\alpha\mu}_{P}+t_{1}+t_{2}-\frac{3}{2}t_{3}+\frac{3}{2}t_{5}-\frac{t_{11}}{2}$
(42) $\displaystyle=$
$\displaystyle-\left[\frac{\alpha}{2}-\frac{A}{2}+\frac{B}{2},C-D,\eta^{kf}\left(\frac{\alpha}{2}+\frac{A}{2}-\frac{B}{2}\right)-X-Y+Z-U-V\right].$

Then we have desired result

$\displaystyle-\frac{1}{4}\vec{\nabla}^{2}t^{\alpha\mu}_{\rm{New}}=(B^{0000},B^{000j},B^{00ij}).$
(43)

###  3.5 Weinberg pseudotensor

The superpotential for Weinberg(W) [8] is

$\displaystyle
H^{[\mu\nu][\alpha\beta]}_{W}:=\sqrt{-\eta}\,\eta^{\alpha\pi}\eta^{\beta\xi}\eta^{\lambda\kappa}\delta^{\sigma\mu\nu}_{\pi\xi\kappa}g_{\lambda\sigma}.$
(44)

The associated Weinberg pseudotensor becomes

$\displaystyle
t^{\alpha\mu}_{W}=(2\Gamma^{\beta\alpha\mu}-g^{\alpha\mu}\Gamma^{\beta\pi}{}_{\pi})\Gamma_{\beta}{}^{\nu}{}_{\nu}+g^{\alpha\mu}\Gamma^{\rho\beta\nu}\Gamma_{\rho\beta\nu}-2\Gamma^{\beta\nu\alpha}\Gamma_{\beta\nu}{}^{\mu}-2h^{\beta\lambda}(\partial_{\lambda}\Gamma^{\alpha\mu}{}_{\beta}-\partial^{\mu}\Gamma^{\alpha}{}_{\beta\lambda}).$
(45)

Apply the harmonic gauge, we have the following results

$\displaystyle
t^{\alpha\mu}_{W}=-\left[\left(\frac{3\alpha}{2}+\frac{A}{2}+\frac{B}{2}\right),~{}D,~{}\eta^{kf}\left(-\frac{3\alpha}{2}-\frac{3A}{2}+\frac{B}{2}\right)+X+Y-Z\right].$
(46)

Adding some modification

$\displaystyle t^{\alpha\mu}_{\rm{New}}$ $\displaystyle=$ $\displaystyle
t^{\alpha\mu}_{W}+2t_{1}+t_{2}-\frac{t_{3}}{2}+\frac{5}{2}t_{5}-t_{6}-t_{8}-t_{9}-t_{10}+t_{11}$
(47) $\displaystyle=$
$\displaystyle-\left[\frac{\alpha}{2}-\frac{A}{2}+\frac{B}{2},C-D,\eta^{kf}\left(\frac{\alpha}{2}+\frac{A}{2}-\frac{B}{2}\right)-X-Y+Z-U-V\right].$

Then we have desired result

$\displaystyle-\frac{1}{4}\vec{\nabla}^{2}t^{\alpha\mu}_{\rm{New}}=(B^{0000},B^{000j},B^{00ij}).$
(48)

Table 1: Extra superpotential for $U_{i}$. Superpotential |  |
---|---|---
$U_{1}=h^{\mu\pi}\Gamma^{\alpha\nu}{}_{\pi}-(\mu\leftrightarrow\nu)$, |  $U_{2}=h^{\mu\pi}\Gamma^{\nu\alpha}{}_{\pi}-(\mu\leftrightarrow\nu)$, |  $U_{3}=g^{\alpha\mu}h^{\beta\nu}\Gamma^{\lambda}{}_{\lambda\beta}-(\mu\leftrightarrow\nu)$,
$U_{4}=h^{\alpha\nu}\Gamma^{\mu\lambda}{}_{\lambda}-(\mu\leftrightarrow\nu)$, |  $U_{5}=h^{\alpha\mu}\Gamma^{\lambda\nu}{}_{\lambda}-(\mu\leftrightarrow\nu)$, |  $U_{6}=g^{\alpha\mu}h^{\beta\lambda}\Gamma_{\beta\lambda}{}^{\nu}-(\mu\leftrightarrow\nu)$,
$U_{7}=g^{\alpha\nu}h^{\beta\mu}\Gamma_{\beta}{}^{\lambda}{}_{\lambda}-(\mu\leftrightarrow\nu)$, |  $U_{8}=g^{\alpha\nu}h^{\beta\lambda}\Gamma^{\mu}{}_{\beta\lambda}-(\mu\leftrightarrow\nu)$, |  $U_{9}=h^{\alpha\lambda}\Gamma^{\nu\mu}{}_{\lambda}-(\mu\leftrightarrow\nu)$,
$U_{10}=h^{\nu\pi}\Gamma_{\pi}{}^{\alpha\mu}-(\mu\leftrightarrow\nu),$ |  $U_{11}=h\Gamma^{\nu\mu\alpha}-(\mu\leftrightarrow\nu)$, |  $U_{12}=g^{\alpha\mu}h\Gamma^{\lambda\nu}{}_{\lambda}-(\mu\leftrightarrow\nu)$,
$U_{13}=g^{\alpha\mu}h\Gamma^{\nu\lambda}{}_{\lambda}-(\mu\leftrightarrow\nu)$. |  |

##  4 Conclusion

In 1999, Deser proposed a certain linear combination between Einstein and
Landau Lifshitz pseudotensors to generate a multiple of the pure Bel-Robinson
tensor using the Riemann normal coordinates. Here we use a similar idea but
just try to obtain the energy-momentum and the stress, under a harmonic gauge
condition. We found that none of the classical pseudotensor can have the
expected result by itself. But after adding some modification, all of them
have the desired results.

## References

  * [1] Senovilla J M M 2000 Class. Quantum Grav. 17 2799
  * [2] Deser S, Franklin J S and Seminaea D 1999 Class. Quantum Grav. 16 2815
  * [3] Freud Ph 1939 Ann. Math. 40 417
  * [4] Landau L D and Lifshitz E M 1975 The classical theory of fields (Oxford: Pergamon)
  * [5] So L L, Nester J M and Chen H 2009 Class. Quantum Grav. 26 085004
  * [6] Bergmann P G and Thomson R 1953 Phys. Rev. 89 400
  * [7] Papapetrou A 1948 Proc. R. Irish. Acad. A52 11
  * [8] Weinberg S 1972 Gravitation and Cosmology (New York: Wiley) p371
  * [9] Misner C W, Thorne K S and Wheeler J A 1973 Gravitation (San Francisco, CA: Freeman)
  * [10] Carmeli M “Classical Fields General relativuty and Gauge Theory” (John Wiley $\&$ Sons 1982)
  * [11] Zhang X H 1985 Phys. Rev. D 31 3130


# How to Write Summaries with Patterns?
Learning towards Abstractive Summarization through Prototype Editing

Shen Gao ♠, Xiuying Chen ♠♣11footnotemark: 1, Piji Li ♢, Zhangming Chan ♠♣,
Dongyan Zhao ♠♣, Rui Yan ♠♣
♠Wangxuan Institute of Computer Technology, Peking University, Beijing, China,
♢Tencent AI Lab, Shenzhen, China
♣Center for Data Science, Peking University, Beijing, China
{shengao, xy-chen, zhangming.chan, zhaody<EMAIL_ADDRESS>
<EMAIL_ADDRESS>Equal contribution. Ordering is decided by a coin flip.
Corresponding author.

###### Abstract

Under special circumstances, summaries should conform to a particular style
with patterns, such as court judgments and abstracts in academic papers. To
this end, the prototype document-summary pairs can be utilized to generate
better summaries. There are two main challenges in this task: (1) the model
needs to incorporate learned patterns from the prototype, but (2) should avoid
copying contents other than the patternized words—such as irrelevant
facts—into the generated summaries. To tackle these challenges, we design a
model named Prototype Editing based Summary Generator (PESG). PESG first
learns summary patterns and prototype facts by analyzing the correlation
between a prototype document and its summary. Prototype facts are then
utilized to help extract facts from the input document. Next, an editing
generator generates new summary based on the summary pattern or extracted
facts. Finally, to address the second challenge, a fact checker is used to
estimate mutual information between the input document and generated summary,
providing an additional signal for the generator. Extensive experiments
conducted on a large-scale real-world text summarization
dataset111https://github.com/gsh199449/proto-summ show that PESG achieves the
state-of-the-art performance in terms of both automatic metrics and human
evaluations.

##  1 Introduction

Abstractive summarization can be regarded as a sequence mapping task that maps
the source text to the target summary Rush et al. (2015); Li et al. (2017);
Cao et al. (2018); Gao et al. (2019a). It has drawn significant attention
since the introduction of deep neural networks to natural language processing.
Under special circumstances, the generated summaries are required to conform
to a specific pattern, such as court judgments, diagnosis certificates,
abstracts in academic papers, etc. Take the court judgments for example, there
is always a statement of the crime committed by the accused, followed by the
motives and the results of the judgment. An example case is shown in Table 1,
where the summary shares the same writing style and has words in common with
the prototype summary (retrieved from the training dataset).

prototype summary |  The court held that the defendant Wang had  stolen the property of others for the purpose of illegal possession. The amount was large, and his behavior constituted the crime of theft. The accusation of the public prosecution agency was established. The defendant Wang has a criminal record and will be considered when sentencing. Since the defendant Wang did not succeed because of reasons other than his will, he could be punished lightly. After the defendant confessed his crimes to the case, he was given a lighter punishment according to law.
---|---
summary |  The court held that the accused Zhang and Fan stole property and the amount was large. Their actions constituted the crime of theft. The accusation of the public prosecution agency was established and supported. This crime was committed within two years after the release of the defendants Zhang and Fan. Thus they are recidivists and this situation will be considered when sentencing. The fact that defendants Zhang and Fan surrendered themselves and pleaded guilty in court gives a lighter punishment according to law.
Table 1: An example of patternized summary generation. The text in red denotes
patternized words shared in different summaries, and text in blue denotes
specific facts.

Existing prototype based generation models such as Wu et al. (2018) are all
applied on short text, thus, cannot handle long documents summarization task.
Another series of works focus on template-based methods such as Oya et al.
(2014). However, template-based methods are too rigid for our patternized
summary generation task. Hence, in this paper, we propose a summarization
framework named _Prototype Editing based Summary Generator_ (PESG) that
incorporates prototype document-summary pairs to improve summarization
performance when generating summaries with pattern. First, we calculate the
cross dependency between the prototype document-summary pair to obtain a
summary pattern and prototype facts (explained in § 4.2). Then, we extract
facts from the input document with the help of the prototype facts (explained
in § 4.3). Next, a recurrent neural network (RNN) based decoder is used to
generate a new summary, incorporating both the summary pattern and extracted
facts (explained in § 4.4). Finally, a fact checker is designed to provide
mutual information between the generated summary and the input document to
prevent the generator from copying irrelevant facts from the prototype
(explained in § 4.5). To evaluate PESG, we collect a large-scale court
judgment dataset, where each judgment is a summary of the case description
with a patternized style. Extensive experiments conducted on this dataset show
that PESG outperforms the state-of-the-art summarization baselines in terms of
ROUGE metrics and human evaluations by a large margin.

Our contributions can be summarized as follows:

$\bullet$ We propose to use prototype information to help generate better
summaries with patterns.

$\bullet$ Specifically, we propose to generate the summary incorporating the
prototype summary pattern and extracted facts from input document.

$\bullet$ We provide mutual information signal for the generator to prevent
copying irrelevant facts from the prototype.

$\bullet$ We release a large-scale prototype based summarization dataset that
is beneficial for the community.

##  2 Related Work

We detail related work on text summarization and prototype editing.

Text summarization can be classified into extractive and abstractive methods.
Extractive methods Narayan et al. (2018b); Chen et al. (2018) directly select
salient sentences from an article to compose a summary. One shortcoming of
these models is that they tend to suffer from redundancy. Recently, with the
emergence of neural network models for text generation, a vast majority of the
literature on summarization Ma et al. (2018); Zhou et al. (2018); Gao et al.
(2019a); Chen et al. (2019) is dedicated to abstractive summarization, which
aims to generate new content that concisely paraphrases a document from
scratch.

Another line of research focuses on prototype editing. Guu et al. (2018)
proposed the first prototype editing model, which samples a prototype sentence
from training data and then edits it into a new sentence. Following this work,
Wu et al. (2018) proposed a new paradigm for response generation, which first
retrieves a prototype response from a pre-defined index and then edits the
prototype response. Cao et al. (2018) applied this method on summarization,
where they employed existing summaries as soft templates to generate new
summary without modeling the dependency between the prototype document,
summary and input document. Different from these soft attention methods, Cai
et al. (2018) proposed a hard-editing skeleton-based model to promote the
coherence of generated stories. Template-based summarization is also a hard-
editing method Oya et al. (2014), where a multi-sentence fusion algorithm is
extended in order to generate summary templates.

Different from all above works, our model focuses on patternized summary
generation, which is more challenging than traditional news summarization and
short sentence prototype editing.

##  3 Problem Formulation

For an input document $X=\\{x_{1},x_{2},\dots,x_{T_{m}}\\}$, we assume there
is a ground truth summary $Y=\\{y_{1},y_{2},\dots,y_{T_{n}}\\}$. In our
prototype summarization task, a retrieved prototype document
$\hat{X}=\\{\hat{x}_{1},\hat{x}_{2},\dots,\hat{x}_{T_{m}}\\}$ with a
corresponding prototype summary
$\hat{Y}=\\{\hat{y}_{1},\hat{y}_{2},\dots,\hat{y}_{T_{n}}\\}$ is also attached
according to their similarities with $X$.

For a given document $X$, our model extracts salient facts from $X$ guided by
a prototype document $\hat{X}$, and then generates the summary
$Y^{{}^{\prime}}$ by referring to the prototype summary $\hat{Y}$. The goal is
to generate a summary $Y^{{}^{\prime}}$ that not only follows a patternized
style (as defined by prototype summary $\hat{Y}$) but also is consistent with
the facts in document $X$.

##  4 Model

###  4.1 Overview

Figure 1:  Overview of PESG. We divide our model into four parts: (1)
Prototype Reader; (2) Fact Extraction; (3) Editing Generator; (4) Fact
Checker.

In this section, we propose our _prototype editing based summary generator_ ,
which can be split into two main parts, as shown in Figure 1:

$\bullet$ Summary Generator. (1) Prototype Reader analyzes the dependency
between $\hat{X}$ and $\hat{Y}$ to determine the summary pattern and prototype
facts. (2) Fact Extraction module extracts facts from the input document under
the guidance of the prototype facts. (3) Editing Generator module generates
the summary $Y^{{}^{\prime}}$ of document $X$ by incorporating summary pattern
and facts.

$\bullet$ Fact Checker estimates the mutual information between the generated
summary $Y^{{}^{\prime}}$ and input document $X$. This information provides an
additional signal for the generation process, preventing irrelevant facts from
being copied from the prototype document.

###  4.2 Prototype Reader

To begin with, we use an embedding matrix $e$ to map a one-hot representation
of each word in $X$, $\hat{X}$, $\hat{Y}$ into a high-dimensional vector
space. We then employ a bi-directional recurrent neural network (Bi-RNN) to
model the temporal interactions between words:

$\displaystyle h^{x}_{t}$ $\displaystyle=\text{Bi-
RNN}_{x}(e(x_{t}),h^{x}_{t-1}),$ (1) $\displaystyle\hat{h}^{x}_{t}$
$\displaystyle=\text{Bi-RNN}_{x}(e(\hat{x}_{t}),\hat{h}^{x}_{t-1}),$ (2)
$\displaystyle\hat{h}^{y}_{t}$ $\displaystyle=\text{Bi-
RNN}_{y}(e(\hat{y}_{t}),\hat{h}^{y}_{t-1}),$ (3)

where $h^{x}_{t}$, $\hat{h}^{x}_{t}$ and $\hat{h}^{y}_{t}$ denote the hidden
state of $t$-th step in Bi-RNN for $X$, $\hat{X}$ and $\hat{Y}$, respectively.
Following Tao et al. (2018); Gao et al. (2019b); Hu et al. (2019), we choose
long short-term memory (LSTM) as the cell for Bi-RNN.

On one hand, the sections in the prototype summary that are not highly related
to the prototype document are the universal patternized words and should be
emphasized when generating the new summary. On the other hand, the sections in
the prototype document that are highly related to the prototype summary are
useful facts that can guide the process of extracting facts from input
document. Hence, we employ a bi-directional attention mechanism between a
prototype document and summary to analyze the cross-dependency, that is, from
document to summary and from summary to document. Both of these attentions are
derived from a shared similarity matrix, $S\in\mathbb{R}^{T_{m}\times T_{n}}$,
calculated by the hidden states of prototype document $\hat{X}$ and prototype
summary $\hat{Y}$. $S_{ij}$ indicates the similarity between the $i$-th
document word $\hat{x}_{i}$ and $j$-th summary word $\hat{y}_{j}$ and is
computed as:

$\displaystyle S_{ij}$
$\displaystyle=\alpha(\hat{h}^{x}_{i},\hat{h}^{y}_{j}),$ (4)
$\displaystyle\alpha(x,y)$ $\displaystyle=w^{\intercal}[x\oplus
y\oplus(x\otimes y)],$

where $\alpha$ is a trainable scalar function that calculates the similarity
between two input vectors. $\oplus$ denotes a concatenation operation and
$\otimes$ is an element-wise multiplication.

We use $a_{t}^{s}=\text{mean}(S_{:t})\in\mathbb{R}$ to represent the attention
weight on the $t$-th prototype summary word by document words, which will
learn to assign high weights to highly related universal patternized words
when generating a summary. From $a_{t}^{s}$, we obtain the weighted sum of the
hidden states of prototype summary as “summary pattern”
$l=\\{l_{1},\dots,l_{T_{n}}\\}$, where $l_{i}$ is:

$l_{i}=a^{s}_{i}\hat{h}^{y}_{i}.$ (5)

Similarly, $a_{t}^{d}=\text{mean}(S_{t:})\in\mathbb{R}$ assigns high weights
to the words in a prototype document that are relevant to the prototype
summary. A convolutional layer is then applied to extract “prototype facts”
$\hat{r}_{t}$ from the prototype document:

$\hat{r}_{t}=\text{CNN}(a^{d}_{t}\hat{h}^{x}_{t}).$ (6)

We sum the prototype facts to obtain the overall representation of these
facts:

$q=\textstyle\sum_{t}^{T_{m}}\hat{r}_{t}.$ (7)

###  4.3 Fact Extraction

In this section, we discuss how to extract useful facts from an input document
with the help of prototype facts.

We first extract the facts from an input document by calculating their
relevance to prototype facts. The similarity matrix $E$ is then calculated
between the weighted prototype document $a^{d}_{i}\hat{h}^{x}_{i}$ and input
document representation $h^{x}_{j}$:

$E_{ij}=\alpha(a^{d}_{i}\hat{h}^{x}_{i},h^{x}_{j}),$ (8)

where $\alpha$ is the similarity function introduced in Equation 4. Then, we
sum up $E_{ij}$ along the length of the prototype document to obtain the
weight $E_{j}=\sum_{ti}^{T_{m}}E_{tj}$ for $j$-th word in the document. Next,
similar to Equation 6, a convolutional layer is applied on the weighted hidden
states $E_{t}h^{x}_{t}$ to obtain the fact representation $r_{t}$ from the
input document:

$r_{t}=\text{CNN}(E_{t}h^{x}_{t}).$ (9)

Figure 2: Framework of fact extraction module.

Inspired by the polishing strategy in extractive summarization Chen et al.
(2018), we propose to use the prototype facts to polish the extracted facts
$r_{t}$ and obtain the final fact representation $m_{.}$, as shown in Figure
2. Generally, the polishing process consists of two hierarchical recurrent
layers. The first recurrent layer is made up of Selective Recurrent Units
(SRUs), which take facts $r_{\cdot}$ and polished fact $q_{k}$ as input,
outputting the hidden state $h^{k}_{T_{m}}$. The second recurrent layer
consists of regular Gated Recurrent Units (GRUs), which are used to update the
polished fact from $q_{k}$ to $q_{k+1}$ using $h^{k}_{T_{m}}$.

SRU is a modified version of the original GRU introduced in Chen et al.
(2018), details of which can be found in Appendix A. Its difference from GRU
lies in that the update gate in SRU is decided by both the polished fact
$q_{k}$ and original fact $r_{t}$ together. The $t$-th hidden state of SRU is
calculated as:

$\displaystyle h^{k}_{t}$ $\displaystyle=\text{SRU}(r_{t},q_{k}).$ (10)

We take $h^{k}_{T_{m}}$ as the overall representation of all input facts
$r_{\cdot}$. In this way, SRU can decide to which degree each unit should be
updated based on its relationship with the polished fact $q_{k}$.

Next, $h^{k}_{T_{m}}$ is used to update the polished fact $q_{k}$ using the
second recurrent layer, consisting of GRUs:

$m_{k+1},q_{k+1}=\text{GRU}(h^{k}_{T_{m}},q_{k}),$ (11)

where $q_{k}$ is the cell state, $h^{k}_{T_{m}}$ is the input and $m_{k+1}$ is
the output hidden state. $q_{0}$ is initialized using $q$ in Equation 7. This
iterative process is conducted $K$ times, and each output $m_{k}$ is stored as
extracted facts $M=\\{m_{1},m_{2},\dots,m_{K}\\}$. In this way, $M$ stores
facts with different polished levels.

###  4.4 Editing Generator

The editing generator aims to generate a summary based on the input document,
prototype summary and extracted facts. As with the settings of prototype
reader, we use LSTM as the RNN cell. We first apply a linear transformation on
the summation of the summary pattern
$l^{{}^{\prime}}=\textstyle\sum_{i}^{T_{n}}l_{i}$ and input document
representations $h^{x}_{T_{m}}$, and then employ this vector as the initial
state $d_{0}$ of the RNN generator as shown in Equation 12. The procedure of
$t$-th generation is shown in Equation 13:

$\displaystyle d_{0}$ $\displaystyle=W_{e}[h^{x}_{T_{m}}\oplus
l^{{}^{\prime}}]+b_{e},$ (12) $\displaystyle d_{t}$
$\displaystyle=\text{LSTM}(d_{t-1},[g^{i}_{t-1}\oplus e(y_{t-1})]),$ (13)

where $W_{e},b_{e}$ are trainable parameters, $d_{t}$ is the hidden state of
the $t$-th generating step, and $g_{t-1}^{i}$ is the context vector produced
by the standard attention mechanism Bahdanau et al. (2015).

To take advantage of the extracted facts $M$ and prototype summary $l$, we
incorporate them both into summary generation using a dynamic attention. More
specifically, we utilize a matching function $f$ to model the relationship
between the current decoding state $d_{t}$ and each $v_{i}$ ($v_{i}$ can be a
extracted fact $m_{i}$ or summary pattern $l_{i}$):

$\displaystyle\delta_{it}$
$\displaystyle=\frac{\exp(f(v_{i},d_{t}))}{\sum_{j}^{K}\exp(f(v_{j},d_{t}))},$
(14) $\displaystyle g^{*}_{t}$
$\displaystyle=\textstyle\sum_{i}^{K}\delta_{it}v_{i},$ (15)

where $g^{*}_{t}$ can be $g^{m}_{t}$ or $g^{s}_{t}$ for attending to extracted
facts or a summary pattern, respectively. We use a simple but efficient bi-
linear layer as the matching function $f=m_{i}W_{f}d_{t}$. As for combining
$g^{m}_{t}$ and $g^{s}_{t}$, we propose to use an “editing gate” $\gamma$,
which is determined by the decoder state $d_{t}$, to decide the importance of
the summary pattern and extracted facts at each decoding step.

$\gamma=\sigma\left(W_{g}d_{t}+b_{g}\right),$ (16)

where $\sigma$ denotes the sigmoid function. Using the editing gate, we obtain
$g^{h}_{t}$ which dynamically combines information from the extracted facts
and summary pattern with the editing gate $\gamma$, as:

$g^{h}_{t}=\left[\gamma g^{m}_{t}\oplus\left(1-\gamma\right)g^{s}_{t}\right].$
(17)

Finally, the context vector $g^{h}_{t}$ is concatenated with the decoder state
$d_{t}$ and fed into a linear layer to obtain the generated word distribution
$P_{v}$:

$\displaystyle d^{o}_{t}$ $\displaystyle=W_{o}[d_{t}\oplus g^{h}_{t}]+b_{o},$
(18) $\displaystyle P_{v}$
$\displaystyle=\text{softmax}\left(W_{v}d^{o}_{t}+b_{v}\right).$ (19)

The loss is the negative log likelihood of the target word $y_{t}$:

$\mathcal{L}_{s}=-\textstyle\sum^{T_{n}}_{t=1}\log P_{v}(y_{t}).$ (20)

In order to handle the out-of-vocabulary (OOV) problem, we equip our decoder
with a pointer network Gu et al. (2016); Vinyals et al. (2015); See et al.
(2017). This process is the same as the model described in See et al. (2017),
thus, is omit here due to limited space.

What’s more, previous work Holtzman et al. (2018) has found that using a cross
entropy loss alone is not enough for generating coherent text. Similarly, in
our task, using $\mathcal{L}_{s}$ alone is not enough to distinguish a good
summary with accurate facts from a bad summary with detailed facts from the
prototype document (see § 6.2). Thus, we propose a fact checker to determine
whether the generated summary is highly related to the input document.

###  4.5 Fact Checker

To generate accurate summaries that are consistent with the detailed facts
from the input document rather than facts from the prototype document, we add
a fact checker to provide additional training signals for the generator.
Following Hjelm et al. (2019), we employ the neural mutual information
estimator to estimate the mutual information between the generated summary
$Y^{{}^{\prime}}$ and its corresponding document $X$, as well as the prototype
document $\hat{X}$. Generally, mutual information is estimated from a local
and global level, and we expect the matching degree to be higher between the
generated summary and input document than the prototype document. An overview
of the fact checker is shown in Figure 3.

Figure 3: Framework of fact checker module.

To begin, we use a local matching network to calculate the matching degree,
for local features, between the generated summary and the input, as well as
prototype document. Remember that, in § 4.3, we obtain the fact representation
of an input document $r_{i}$ and prototype facts $\hat{r}_{i}$. Combining
these with the final hidden state $d_{T_{n}}$ of the generator RNN (in
Equation 13), yields the local features of input extracted facts and the
prototype facts:

$\displaystyle C^{r}=\\{d_{T_{n}}\oplus r_{1},\dots,d_{T_{n}}\oplus
r_{T_{m}}\\},$ (21) $\displaystyle
C^{f}=\\{d_{T_{n}}\oplus\hat{r}_{1},\dots,d_{T_{n}}\oplus\hat{r}_{T_{m}}\\}.$
(22)

A $1\times 1$ convolutional layer and a fully-connected layer are applied to
score these two features:

$\tau^{r}_{l}=\text{CNN}_{l}(C^{r}),\quad\tau^{f}_{l}=\text{CNN}_{l}(C^{f}),$
(23)

where $\tau^{r}_{l}\in\mathbb{R}$, $\tau^{f}_{l}\in\mathbb{R}$ represent the
local matching degree between the generated summary and input document and
prototype document, respectively. We want the generated summary to be more
similar to the input document than the prototype document. Thus, the
optimization objective of the local matching network is to minimize
$\mathcal{L}_{l}$:

$\mathcal{L}_{l}=-\left(\log(\tau^{r}_{l})+\log(1-\tau^{f}_{l})\right).$ (24)

We also have a global matching network to measure the matching degree, for
global features, between the generated summary and the input document, as well
as prototype document. To do so, we concatenate the representation of the
generated summary with the final hidden state of the input document
$h^{x}_{T_{m}}$ and final state of the prototype document
$\hat{h}^{x}_{T_{m}}$, respectively, and apply a linear layer to these:

$\displaystyle\tau^{r}_{g}$ $\displaystyle=\text{relu}(W_{m}[d_{T_{n}}\oplus
h^{x}_{T_{m}}]+b_{m}),$ (25) $\displaystyle\tau^{f}_{g}$
$\displaystyle=\text{relu}(W_{m}[d_{T_{n}}\oplus\hat{h}^{x}_{T_{m}}]+b_{m}),$
(26)

where $W_{m},b_{m}$ are trainable parameters and $\tau^{r}_{g}\in\mathbb{R}$
and $\tau^{f}_{g}\in\mathbb{R}$ represent the matching degree between the
generated summary and the input document, and prototype document,
respectively. The objective of this global matching network, similar to the
local matching network, is to minimize:

$\mathcal{L}_{g}=-\left(\log(\tau^{r}_{g})+\log(1-\tau^{f}_{g})\right).$ (27)

Finally, we combine the local and global loss functions to obtain the final
loss $\mathcal{L}$, which we use $\mathcal{L}$ to calculate the gradients for
all parameters:

$\mathcal{L}=\epsilon\mathcal{L}_{g}+\eta\mathcal{L}_{l}+\mathcal{L}_{s},$
(28)

where $\epsilon,\eta$ are both hyper parameters. To optimize the trainable
parameters, we employ the gradient descent method Adagrad Duchi et al. (2010)
to update all parameters.

##  5 Experimental Setup

###  5.1 Dataset

We collect a large-scale prototype based summarization
dataset222https://github.com/gsh199449/proto-summ, which contains 2,003,390
court judgment documents. In this dataset, we use a case description as an
input document and the court judgment as the summary. The average lengths of
the input documents and summaries are 595.15 words and 273.57 words
respectively. The percentage of words common to a prototype summary and the
reference summary is 80.66%, which confirms the feasibility and necessity of
prototype summarization. Following other summarization datasets Grusky et al.
(2018); Kim et al. (2019); Narayan et al. (2018a), we also count the novel
n-grams in a summary compared with the n-grams in the original document, and
the percentage of novel n-grams are 51.21%, 84.59%, 91.48%, 94.83% for novel
1-grams to 4-grams respectively. The coverage, compression and density Grusky
et al. (2018) are commonly used as metrics to evaluate the abstractness of a
summary. For the summaries in our dataset, the coverage percentage is 48.78%,
compression is 2.28 and density is 1.31. We anonymize entity tokens into
special tags, such as using “PERS” to replace a person’s name.

(a)  Figure 4: Visualizations of editing gate.

###  5.2 Comparisons

Acronym | Gloss
---|---
PESG-FC |  PESG w/o Fact Checker
PESG-PR |  PESG w/o Prototype Reader
PESG-SS |  PESG w/o Summary Pattern
PESG-FG |  PESG w/o FGRU
Table 2: Ablation models for comparison.

In order to prove the effectiveness of each module of PESG, we conduct several
ablation studies, shown in Table 2. We also compare our model with the
following baselines: (1) Lead-3 is a commonly used summarization baseline
Nallapati et al. (2017); See et al. (2017), which selects the first three
sentences of document as the summary. (2) S2S is a sequence-to-sequence
framework with a pointer network, proposed by See et al. (2017). (3) Proto is
a context-aware prototype editing dialog response generation model proposed by
Wu et al. (2018). (4) Re3Sum, proposed by Cao et al. (2018), uses an IR
platform to retrieve proper summaries and extends the seq2seq framework to
jointly conduct template-aware summary generation. (5) Uni-model was proposed
by Hsu et al. (2018), and is the current state-of-the-art abstractive
summarization approach on the CNN/DailyMail dataset. (6) We also directly
concatenate the prototype summary with the original document as input for S2S
and Uni-model, named as Concat-S2S and Concat-Uni, respectively.

###  5.3 Evaluation Metrics

For the court judgment dataset, we evaluate standard ROUGE-1, ROUGE-2 and
ROUGE-L Lin (2004) on full-length F1 following previous works Nallapati et al.
(2017); See et al. (2017); Paulus et al. (2018), where ROUGE-1 (R1), ROUGE-2
(R2), and ROUGE-L (RL) refer to the matches of unigram, bigrams, and the
longest common subsequence respectively.

Schluter (2017) notes that only using the ROUGE metric to evaluate
summarization quality can be misleading. Therefore, we also evaluate our model
by human evaluation. Three highly educated participants are asked to score 100
randomly sampled summaries generated by three models: Uni-model, Re3Sum and
PESG. The statistical significance of observed differences between the
performance of two runs is tested using a two-tailed paired t-test and is
denoted using ▲(or ▼) for strong (or weak) significance for $\alpha=0.01$.

###  5.4 Implementation Details

We implement our experiments in TensorFlow Abadi et al. (2016) on an NVIDIA
GTX 1080 Ti GPU. The word embedding dimension is 256 and the number of hidden
units is 256. The batch size is set to 64. We padded or cut input document to
contain exactly 250 words, and the decoding length is set to 100. $\epsilon$
and $\eta$ from the Equation 28 are both set to 1.0. We initialize all of the
parameters randomly using a Gaussian distribution. We use Adagrad optimizer
Duchi et al. (2010) as our optimizing algorithm and employ beam search with
size 5 to generate more fluency summary sentence. We also apply gradient
clipping Pascanu et al. (2013) with range $[-5,5]$ during training. We use
dropout Srivastava et al. (2014) as regularization with keep probability
$p=0.7$.

##  6 Experimental Result

###  6.1 Overall Performance

| R1 | R2 | RL
---|---|---|---
Lead-3 | 32.9 | 13.3 | 30.0
Re3Sum | 36.3 | 24.0 | 36.0
S2S | 37.6 | 24.6 | 37.3
Uni-model | 37.9 | 25.0 | 37.6
Concat-S2S | 34.2 | 20.3 | 34.3
Concat-Uni | 37.4 | 24.3 | 36.9
PESG | 40.2 | 28.1 | 39.9
Proto | – | – | –
Table 3: ROUGE scores comparison with baselines. Proto directly copies from the prototype summary as generated summary. | Fluency | Consistency
---|---|---
Uni-model | 1.61 | 1.53
Re3Sum | 1.53 | 1.14
PESG | 1.86▲ | 1.73▲
Table 4: Fluency and consistency comparison by human evaluation.

We compare our model with the baselines listed in Table 3. Our model performs
consistently better than other summarization models including the state-of-
the-art model with improvements of 6%, 12% and 6% in terms of ROUGE-1, ROUGE-2
and ROUGE-L. This demonstrates that prototype document-summary pair provides
strong guidance for summary generation that cannot be replaced by other
complicated baselines without prototype information. Meanwhile, directly
concatenating the prototype summary with the original input does not increase
performance, instead leading to drops of 9%, 17%, 8% and 1%, 3%, 2% in terms
of ROUGE 1,2,L on the S2S and Unified models, respectively. As for the
baseline model Proto, we found that it directly copies from the prototype
summary as generated summary, which leads to a totally useless and incorrect
summary.

For the human evaluation, we asked annotators to rate each summary according
to its consistency and fluency. The rating score ranges from 1 to 3, with 3
being the best. Table 4 lists the average scores of each model, showing that
PESG outperforms the other baseline models in both fluency and consistency.
The kappa statistics are 0.33 and 0.29 for fluency and consistency
respectively, and that indicates the moderate agreement between annotators. To
prove the significance of these results, we also conduct the paired student
t-test between our model and Re3Sum (row with shaded background). We obtain a
p-value of $2\times 10^{-7}$ and $9\times 10^{-12}$ for fluency and
consistency, respectively.

We also analyze the effectiveness of performance by the two hyper-parameters:
$\eta$ and $\epsilon$. It turns out that our model has a consistently good
performance, with ROUGE-1, ROUGE-2, ROUGE-L scores above 39.5, 27.5, 39.4,
which demonstrates that our model is very robust.

###  6.2 Ablation Study

Figure 5: Visualizations of ROUGE score with different hop numbers.

| R1 | R2 | RL
---|---|---|---
PESG-FC | 38.7 | 26.2 | 38.6
PESG-PR | 37.3 | 24.6 | 37.0
PESG-SS | 38.6 | 25.6 | 38.3
PESG-FG | 38.8 | 25.9 | 38.5
Table 5: ROUGE scores of different ablation models of PESG.

The ROUGE scores of different ablation models are shown in Table 5. All
ablation models perform worse than PESG in terms of all metrics, which
demonstrates the preeminence of PESG. More importantly, by this controlled
experiment, we can verify the contributions of each modules in PESG.

###  6.3 Analysis of Editing Generator

We visualize the editing gate (illustrated in Equation 16) of two randomly
sampled examples, shown in Figure 4. A lower weight (lighter color) means that
the word is more likely to be copied from the summary pattern; that is to say,
this word is a universal patternized word. We can see that the phrase 本院认为
(the court held that) has a lower weight than the name of the defendant
(PERS), which is consistent with the fact that (the court held that) is a
patternized word and the name of the defendant is closely related to the input
document.

We also show a case study in Table 6, which includes the input document and
reference summary with the generated summaries. Underlined text denotes a
grammar error and a strike-through line denotes a fact contrary to the input
document. We only show part of the document and summary due to limited space;
the full version is shown in Appendix. As can be seen, the summary generated
by Uni-model faces an inconsistency problem and the summary generated by
Re3Sum is contrary to the facts described in the input document. However, PESG
overcomes both of these problems and generates an accurate summary with good
grammar and logic.

###  6.4 Analysis of Fact Extraction Module

We investigate the influence of the iteration number when facts are extracted.
Figure 5 illustrates the relationship between iteration number and the f-value
of the ROUGE score. The results show that the ROUGE scores first increases
with the number of hops. After reaching an upper limit it then begins to drop.
This phenomenon demonstrates that the fact extraction module is effective by
polishing the facts representation.

安徽省 人民法院 作出 刑事 判决 ， 以 斗殴 罪 判处 被告人 PERS 有期徒刑 YEARS 。 合肥市 监狱 提出 减刑 意见 。 执行 机关 以
PERS 在 服刑 期间 确 有 悔改 表现 ， 已 获得 奖励 二 个 ， 建议 对 其 减刑 。 经 审理 查明 ， 罪犯 PERS 在 服刑 期间 ，
能 认罪 服法 , 努力 参加 技术 学习 。(Anhui Provincial People’s Court made a criminal
judgment and sentenced the defendant PERS to YEARS imprisonment for the crime
of fighting. Hefei Prison proposed a commutation sentence since PERS had
repentance and received two awards during his sentence. It was found through
trial that the criminal PERS pleaded guilty and strived to participate in
technical learning while serving his sentence.)
---
reference |  本院 认为， 罪犯 PERS 在 服刑 期间 确 有 悔改，表现符合 法定 减刑 条件。依据 《 中华人民共和国 刑法 》 第 NUM 款 之 规定，对 罪犯 PERS 减去 自 本 裁定 送达 之 日 起 至 DATE 止 尚未 执行 完毕 的 刑罚。本 裁定 送达 后 即 发生 法律 效力。 (The court believed that the criminal PERS did have repentance during his sentence, and his performance was in line with the statutory commutation conditions. According to the provisions of Section NUM of the Criminal Law of the People’s Republic of China, the court decided to exempt PERS from penalty that had not been executed since the date of delivery of this ruling until DATE. Legal effect would occur upon the delivery of this ruling.)
Uni-model |  本院 认为 ， 被告人 PERS 以 非法 占有 为 目的 ， 秘密 窃取 他人 财物 ， 数额 较 大 ， 其 行为 已 构成 盗窃罪。 公诉 机关 指控 被告人 PERS勋 斗殴 罪 的 事实 清楚 ， 证据 确实 充分 ， 本院 予以 支持。 (The court held that the defendant PERS secretly stole the property of others for the purpose of illegal possession, the stolen amount was relatively large, and his behavior had constituted theft. The fact that defendant PERS committed the crime of fighting was clear, the evidence was indeed sufficient, and the court supported it.)
Re3Sum |  本院 认为 , PERS 在 道路 上 驾驶 机动车,其 行为 已 构成 上诉人 服刑 确 系 罪. 公诉 机关 指控 的 犯罪 事实 清楚，证据 确实 充分。 (The court held that PERS was driving a motor vehicle on the road, and his behavior constituted that the appellant was guilty of serving a sentence. The criminal facts accused by the public prosecution agency were clear and the evidence was indeed sufficient.)
PESG |  本院 认为，罪犯 PERS 在 服刑 期间，确 有 悔改 表现。符合 法定 减刑 条件，依照 《 中华人民共和国 刑法 》 第 NUM 条 、 第 NUM 条 之 规定，裁定 如下 对 罪犯 PERS 减去 有期徒刑 MONTHS。本 裁定 送达 后 即 发生 法律 效力。 (The court held that the criminal PERS did have repentance during his sentence. In accordance with the provisions of NUM and NUM of the Criminal Law of the People’s Republic of China, the ruling was as follows: exempted PERS from the MONTHS penalty. Legal effect would occur upon the delivery of this ruling.)
Table 6: Examples of the generated natural answers by PESG and other models.

##  7 Conclusion

In this paper, we propose a framework named _Prototype Editing based Summary
Generator_ (PESG), which aims to generate summaries in formal writing
scenarios, where summaries should conform to a patternized style. Given a
prototype document-summary pair, our model first calculates the cross
dependency between the prototype document-summary pair. Next, a fact
extraction module is employed to extract facts from the document, which are
then polished. Finally, we design an editing-based generator to produce a
summary by incorporating the polished fact and summary pattern. To ensure that
the generated summary is consistent with the input document, we propose a fact
checker to estimate the mutual information between the input document and
generated summary. Our model outperforms state-of-the-art methods in terms of
ROUGE scores and human evaluations by a large margin, which demonstrates the
effectiveness of PESG.

## Acknowledgments

We would like to thank the anonymous reviewers for their constructive
comments. We would also like to thank Anna Hennig in Inception Institute of
Artificial Intelligence for her help on this paper. This work was supported by
the National Key Research and Development Program of China (No.
2017YFC0804001), the National Science Foundation of China (NSFC No. 61876196
and NSFC No. 61672058)

##  Appendix A SRU Cell

Gated recurrent unit (GRU) Cho et al. (2014) is a gating mechanism in
recurrent neural networks, which incorporate an update gate in an RNN. We
first give the details of the original GRU here.

$\displaystyle u_{i}$
$\displaystyle=\sigma(W^{(u)}x_{i}+U^{(u)}h_{i-1}+b^{(u)}),$ (29)
$\displaystyle r_{i}$
$\displaystyle=\sigma(W^{(r)}x_{i}+U^{(r)}h_{i-1}+b^{(r)}),$ (30)
$\displaystyle\tilde{h_{i}}$
$\displaystyle=\text{tanh}(W^{(h)}x_{i}+r_{i}\circ Uh_{i-1}+b^{(h)}),$ (31)
$\displaystyle h_{i}$ $\displaystyle=u_{i}\circ\tilde{h_{i}}+(1-u_{i})\circ
h_{i-1},$ (32)

where $\sigma$ is the sigmoid activation function,
$W^{(u)},W^{(r)},W^{(h)}\in\mathbb{R}^{n_{H}\times
n_{I}},U^{(u)},U^{(r)},U\in\mathbb{R}^{n_{H}\times n_{H}},n_{H}$ is the hidden
size, and $n_{I}$ is the size of input $x_{i}$. In the original version of
GRU, the update gate $u_{i}$ in Equation 29 is used to decide how much of the
hidden state should be retained and how much should be updated. In our case,
we want to decide which facts are salient according to the polished facts
$q_{k-1}$ at the $k$-th hop. To achieve this, we replace the calculation of
$u_{i}$ with the newly computed update gate $g_{i}$:

$\displaystyle f_{i}$ $\displaystyle=[x_{i}\circ q_{k-1};x_{i};q_{k-1}],$ (33)
$\displaystyle z_{i}$
$\displaystyle=W^{(2)}\tanh(W^{(1)}f_{i}+b^{(1)})+b^{(2)},$ (34)
$\displaystyle g_{i}$
$\displaystyle=\frac{\exp(z_{i})}{\sum^{n_{s}}_{j=1}\exp(z_{j})},$ (35)

where $W^{(2)},W^{(1)},b^{(1)},b^{(2)}$ are all trainable parameters and $k$
is the hop number in the multi-hop situation which is a hyper-parameter
manually set. The effectiveness of this hyper-parameter is verified in the
experimental results shown in § 6.4. Equation 32 now becomes:

$\displaystyle h_{i}$ $\displaystyle=g_{i}\circ\tilde{{h_{i}}}+(1-g_{i})\circ
h_{i-1}.$ (36)

We use the name “SRU” to denote this modified version of an GRU cell.

##  Appendix B Case Study

安徽省 合肥市 包河区 人民法院 作出 刑事 判决 ， 以 斗殴 罪 判处 被告人 PERS 有期徒刑 YEARS 。 案经 本院 二审 ， 准许 上诉人
撤回 上诉 。 判决 生效 后 交付 执行 。 执行 机关 合肥市 义城 监狱 于 DATE 提出 减刑 意见 ， 报送 本院 审理 。 本院 依法 组成
合议庭 进行 了 审理 ， 现 已 审理 终结 。 执行 机关 以 罪犯 PERS 在 服刑 期间 确 有 悔改 表现 ， 已 获得 奖励 二 个 ， 建议
对 其 减刑 。 经 审理 查明 ， 罪犯 PERS 在 服刑 期间 ， 能 认罪 服法 ， 努力 参加 技术 学习 ， 积极 完成 生产 任务 。
(The People’s Court of Baohe District, Hefei City, Anhui Province made a
criminal judgment and sentenced the defendant PERS to YEARS imprisonment for
the crime of fighting. In the second instance of the court, the appellant was
allowed to withdraw the appeal. The judgment takes effect after being
delivered. The enforcement agency Hefei City Yicheng Prison put forward a
commutation opinion on DATE and submitted it to the court for trial. The court
had formed a collegiate bench in accordance with the law and had now end the
trial. Criminal PERS had repentance during his sentence and received two
awards, thus the enforcement agency suggested he should be commuted.
Investigation confirmed that the criminal PERS was able to plead guilty while
serving his sentence, strived to participate in technical studies, and
completed production tasks.)
---
reference |  本院 认为， 罪犯 PERS 在 服刑 期间 确 有 悔改 表现符合 法定 减刑 条件。依据 《 中华人民共和国 刑法 》 第 NUM 款 之 规定，裁定 如下对 罪犯 PERS 减去 自 本 裁定 送达 之 日 起 至 DATE 止 尚未 执行 完毕 的 刑罚。本 裁定 送达 后 即 发生 法律 效力。 (The court held that the criminal PERS did have repentance during the period of serving his sentence in accordance with the statutory commutation conditions. According to the provisions of NUM Section of the Criminal Law of the People’s Republic of China, the court decided to exempt PERS from penalty that had not been executed since the date of delivery of this ruling until DATE. Legal effect would occur upon the delivery of this ruling.)
Unified |  本院 认为 ， 被告人 PERS 以 非法 占有 为 目的 ， 秘密 窃取 他人 财物 ， 数额 较 大 ， 其 行为 已 构成 盗窃罪 ， 依法 应 予 惩处。 公诉 机关 指控 被告人 PERS 聚众斗殴 罪 的 事实 清楚 ， 证据 确实 充分 ， 指控 罪名 成立 ， 本院 予以 支持。 被告人 PERS 曾 因 故意 犯罪 被 判处 有期徒刑 ， 在 刑罚 执行 完毕 后 YEARS 内 再 犯， 应当 判处 有期徒刑 以上 刑罚 之 罪 ， 是 累犯 ， 依法 应当 从重 处罚 。 (The court held that the defendant PERS secretly stole the property of others for the purpose of illegal possession, and the amount was relatively large. His behavior had constituted theft and should be punished according to law. The fact that the defendant PERS commiting the crimes of the fighting was clear, the evidence was indeed sufficient, thus the charges were established and the court supported it. The defendant PERS was sentenced to fixed-term imprisonment for intentional crimes. If he committed another crime in YEARS after the execution of the penalty, he should be sentenced to a fixed-term imprisonment or more. He is a recidivist and should be severely punished according to law.)
Re3Sum |  本院 认为NUM PERS 在 道路 上 驾驶 机动车其 行为 已 构成 上诉人 服刑 确 系 罪公诉 机关 指控 的 犯罪 事实 清楚证据 确实 充分指控 的 罪名 成立本院 予以 支持被告人 PERS 归案 后 如实 供述 自己 的 罪行依法 予以 从轻 处罚公诉 机关 的 量刑 建议 适当本院 予以 采纳 此依照 《 中华人民共和国 刑法 》 第 NUM 款 之 规定判决 如下被告人 PERS 犯 服刑判处 有期徒刑 MONTHS并 处 罚金 人民币 MONEY (The court held that NUM PERS was driving a motor vehicle on the road, and his behavior constituted that the appellant was guilty of serving a sentence. The criminal facts accused by the public prosecution agency were clear, the evidence was indeed sufficient, and the charges were supported by the court. After confessing his crimes, he should be given a lighter punishment according to law. The public prosecution agency’s sentencing recommendations were appropriate and adopted by the court. According to the provisions of Section NUM of the Criminal Law of the People’s Republic of China, the judgment was as follows, the defendant PERS was sentenced to imprisonment and sentenced to fixed-term MONTHS imprisonment, and fined the penalty RMB MONEY.)
PESG |  本院 认为，罪犯 PERS 在 服刑 期间，确 有 悔改 表现。符合 法定 减刑 条件，依照 《 中华人民共和国 刑法 》 第 NUM 条 、 第 NUM 条 之 规定，裁定 如下：对 罪犯 PERS 减去 有期徒刑 MONTHS。本 裁定 送达 后 即 发生 法律 效力。 (The court believed that the criminal PERS did have repentance during his sentence. In accordance with the statutory commutation conditions and NUM and NUM of the Criminal Law of the People’s Republic of China, the ruling was as follows: exempted PERS from the MONTHS penalty. Legal effect would occur upon the delivery of this ruling.)
Table 7: Examples of the generated natural answers by PESG and other models.

## References

  * Abadi et al. (2016) Martín Abadi, Paul Barham, Jianmin Chen, Zhifeng Chen, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Geoffrey Irving, Michael Isard, et al. 2016\.  Tensorflow: a system for large-scale machine learning.  In _OSDI_ , volume 16, pages 265–283.
  * Bahdanau et al. (2015) Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2015.  Neural machine translation by jointly learning to align and translate.  In _ICLR_.
  * Cai et al. (2018) Deng Cai, Yan Wang, Victoria Bi, Zhaopeng Tu, Xiaojiang Liu, Wai Lam, and Shuming Shi. 2018.  Skeleton-to-response: Dialogue generation guided by retrieval memory.  _arXiv preprint arXiv:1809.05296_.
  * Cao et al. (2018) Ziqiang Cao, Wenjie Li, Sujian Li, and Furu Wei. 2018.  Retrieve, rerank and rewrite: Soft template based neural summarization.  In _ACL_ , pages 152–161. Association for Computational Linguistics.
  * Chen et al. (2019) Xiuying Chen, Zhangming Chan, Shen Gao, Meng-Hsuan Yu, Dongyan Zhao, and Rui Yan. 2019.  Learning towards abstractive timeline summarization.  In _Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI-19_ , pages 4939–4945. International Joint Conferences on Artificial Intelligence Organization.
  * Chen et al. (2018) Xiuying Chen, Shen Gao, Chongyang Tao, Yan Song, Dongyan Zhao, and Rui Yan. 2018\.  Iterative document representation learning towards summarization with polishing.  In _EMNLP_ , pages 4088–4097.
  * Cho et al. (2014) Kyunghyun Cho, Bart van Merrienboer, Çaglar Gülçehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2014.  Learning phrase representations using rnn encoder-decoder for statistical machine translation.  In _EMNLP_.
  * Duchi et al. (2010) John C. Duchi, Elad Hazan, and Yoram Singer. 2010.  Adaptive subgradient methods for online learning and stochastic optimization.  _JMLR_ , 12:2121–2159.
  * Gao et al. (2019a) Shen Gao, Xiuying Chen, Piji Li, Zhaochun Ren, Lidong Bing, Dongyan Zhao, and Rui Yan. 2019a.  Abstractive text summarization by incorporating reader comments.  In _AAAI_.
  * Gao et al. (2019b) Shen Gao, Zhaochun Ren, Yihong Eric Zhao, Dongyan Zhao, Dawei Yin, and Rui Yan. 2019b.  Product-aware answer generation in e-commerce question-answering.  In _WSDM_.
  * Grusky et al. (2018) Max Grusky, Mor Naaman, and Yoav Artzi. 2018.  Newsroom: A dataset of 1.3 million summaries with diverse extractive strategies.  In _NAACL-HLT_.
  * Gu et al. (2016) Jiatao Gu, Zhengdong Lu, Hang Li, and Victor O. K. Li. 2016.  Incorporating copying mechanism in sequence-to-sequence learning.  _CoRR_ , abs/1603.06393.
  * Guu et al. (2018) Kelvin Guu, Tatsunori B. Hashimoto, Yonatan Oren, and Percy Liang. 2018.  Generating sentences by editing prototypes.  _TACL_ , 6:437–450.
  * Hjelm et al. (2019) R Devon Hjelm, Alex Fedorov, Samuel Lavoie-Marchildon, Karan Grewal, Phil Bachman, Adam Trischler, and Yoshua Bengio. 2019.  Learning deep representations by mutual information estimation and maximization.  In _International Conference on Learning Representations_.
  * Holtzman et al. (2018) Ari Holtzman, Jan Buys, Maxwell Forbes, Antoine Bosselut, David Golub, and Yejin Choi. 2018.  Learning to write with cooperative discriminators.  In _ACL_.
  * Hsu et al. (2018) Wan-Ting Hsu, Chieh-Kai Lin, Ming-Ying Lee, Kerui Min, Jing Tang, and Min Sun. 2018\.  A unified model for extractive and abstractive summarization using inconsistency loss.  In _ACL_.
  * Hu et al. (2019) Wenpeng Hu, Zhangming Chan, Bing Liu, Dongyan Zhao, Jinwen Ma, and Rui Yan. 2019\.  Gsn: A graph-structured network for multi-party dialogues.  In _Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI-19_ , pages 5010–5016. International Joint Conferences on Artificial Intelligence Organization.
  * Kim et al. (2019) Byeongchang Kim, Hyunwoo Kim, and Gunhee Kim. 2019.  Abstractive Summarization of Reddit Posts with Multi-level Memory Networks.  In _NAACL-HLT_.
  * Li et al. (2017) Piji Li, Wai Lam, Lidong Bing, and Zihao Wang. 2017.  Deep recurrent generative decoder for abstractive text summarization.  In _EMNLP_ , pages 2091–2100.
  * Lin (2004) Chin-Yew Lin. 2004.  Rouge: A package for automatic evaluation of summaries.  _Text Summarization Branches Out_.
  * Ma et al. (2018) Shuming Ma, Xu Sun, Junyang Lin, and Houfeng Wang. 2018.  Autoencoder as assistant supervisor: Improving text representation for chinese social media text summarization.  In _ACL_.
  * Nallapati et al. (2017) Ramesh Nallapati, Feifei Zhai, and Bowen Zhou. 2017.  Summarunner: A recurrent neural network based sequence model for extractive summarization of documents.  In _AAAI_.
  * Narayan et al. (2018a) Shashi Narayan, Shay B. Cohen, and Mirella Lapata. 2018a.  Don’t give me the details, just the summary! topic-aware convolutional neural networks for extreme summarization.  In _EMNLP_.
  * Narayan et al. (2018b) Shashi Narayan, Shay B. Cohen, and Mirella Lapata. 2018b.  Ranking sentences for extractive summarization with reinforcement learning.  In _NAACL-HLT_.
  * Oya et al. (2014) Tatsuro Oya, Yashar Mehdad, Giuseppe Carenini, and Raymond Ng. 2014.  A template-based abstractive meeting summarization: Leveraging summary and source text relationships.  In _INLG_ , pages 45–53.
  * Pascanu et al. (2013) Razvan Pascanu, Tomas Mikolov, and Yoshua Bengio. 2013.  On the difficulty of training recurrent neural networks.  In _ICML_.
  * Paulus et al. (2018) Romain Paulus, Caiming Xiong, and Richard Socher. 2018.  A deep reinforced model for abstractive summarization.  In _ICLR_.
  * Rush et al. (2015) Alexander M Rush, Sumit Chopra, and Jason Weston. 2015.  A neural attention model for abstractive sentence summarization.  In _EMNLP_ , pages 379–389.
  * Schluter (2017) Natalie Schluter. 2017.  The limits of automatic summarisation according to rouge.  In _Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers_ , pages 41–45. Association for Computational Linguistics.
  * See et al. (2017) Abigail See, Peter J. Liu, and Christopher D. Manning. 2017.  Get to the point: Summarization with pointer-generator networks.  In _ACL_.
  * Srivastava et al. (2014) Nitish Srivastava, Geoffrey E. Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. 2014.  Dropout: a simple way to prevent neural networks from overfitting.  _Journal of Machine Learning Research_ , 15:1929–1958.
  * Tao et al. (2018) Chongyang Tao, Shen Gao, Mingyue Shang, Wei Chung Wu, Dongyan Zhao, and Rui Yan. 2018.  Get the point of my utterance! learning towards effective responses with multi-head attention mechanism.  In _IJCAI_.
  * Vinyals et al. (2015) Oriol Vinyals, Meire Fortunato, and Navdeep Jaitly. 2015.  Pointer networks.  In _NIPS_ , pages 2692–2700.
  * Wu et al. (2018) Yu Ping Wu, Furu Wei, Shaohan Huang, Zhoujun Li, and Ming Zhou. 2018.  Response generation by context-aware prototype editing.  _CoRR_ , abs/1806.07042.
  * Zhou et al. (2018) Qingyu Zhou, Nan Yang, Furu Wei, and Ming Zhou. 2018.  Sequential copying networks.  In _AAAI_.