# Optimal social security timing

A.Y. Aydemir<EMAIL_ADDRESS>
###### Abstract

The optimal age that a retiree claims social security retirement benefits is
in general a complicated function of many factors. However, if the
beneficiary’s finances and health are not the constraining factors, it is
possible to formally derive mathematical models that maximize a well-defined
measure of his total benefits. A model that takes into account various factors
such as the increase in the benefits for delayed claims and the penalties for
early retirement, the advantages of investing some of the benefits in the
financial markets, and the effects of cost-of-living adjustments shows that
not waiting until age 70 is almost always the better option. The optimal
claiming age that maximizes the total benefits, however, depends on the
expected market returns and the rate of cost-of-living adjustments, with the
higher market rates in general pushing the optimal age lower. The models
presented here can be easily tailored to address the particular circumstances
and goals of any individual.

Social Security; optimal timing

##  Introduction

Social Security retirement benefits are an essential part of the “Social
Safety Net” in the US and provide much needed income to many retirees. Thus,
at what age they are claimed is usually not an option but dictated by
financial necessity; a beneficiary files for benefits as soon as or soon after
he is qualified. However, there are undoubtably many cases where the
beneficiary can afford to wait for an optimal age for claiming his benefits.
Optimization criteria can vary greatly, of course, depending on the
circumstances and goals of the individual. In general, maximizing some measure
of the total lifetime benefits can be an appropriate goal when health issues
and financial stress are not important concerns.

A number of authors over the years have studied the question of the “optimal
age for claiming Social Security benefits” from various points of view, with
differing methodologies. Here we give only a very brief summary and refer the
reader to the extensive references in the works cited here. Coile et al.[1]
establish the benefits of delayed claims using financial simulations and find
that their results are in general consistent with the behavior observed in
various cohort groups. Friedman and Phillips [2] take what they refer to as a
“sequential approach” in which a decision to claim or postpone is made each
year. Perhaps not surprisingly, they find that a particular decision that is
appropriate for a given year may not be the correct one for the following
year. Alleva[3] uses a range of discount rates to calculate the present value
of a future benefits stream to find an optimal age for claiming benefits. As
expected, higher discount rates lead to a correspondingly lower optimal age.
Ali et al.[4] do a similar optimization of the discounted present value of the
future benefits but make use of numerical simulations to take into account
various factors such as interest rate and life expectancy risks. A model based
on income and wealth-level changes that includes asset risks and mortality by
Diamond et al.[5] seems to suggest that delaying until age 70 is usually the
better option unless market returns of over $7\%$ can be guaranteed, which
differs from our conclusions. However, we are unable to pinpoint the source of
this difference due to an apparent lack of transparency in their methodology.

The problem of determining an optimal age for claiming benefits, stripped of
concerns about longevity, market risks and general economic fluctuations,
essentially boils down to an interesting optimization exercise in a multi-
dimensional parameter space. In our view, with some simplifying assumptions,
this problem can be attacked essentially analytically, yielding expressions
immediately useful for most individuals. They can also be used in future
stochastic simulations or other studies to explore the effects of various risk
factors.

The opportunity for optimization arises from the way the benefits stream is
structured. Briefly, the benefits are reduced if claimed before the full
retirement age (FRA) at 66 or 67[6]; similarly they increase for delayed
retirement, until age 70[7]. However, there are opportunity costs associated
with delayed claims, since the benefits claimed early can be invested in the
financial markets. These factors all combine to make the task of determining
the optimal timing of Social Security retirement benefits a challenging
problem, even without longevity or market risk concerns.

Section II is a review of the usual break even analysis, the simplest
calculation that a potential beneficiary can do to determine the age he would
break even if he were to delay his claim. This Section also examines how the
cost-of-living adjustments (COLAs) would modify these calculations. Section
III defines a gain function, a measure of the relative merits of claiming
early versus late (at age 70) and studies how it varies with various
parameters such as the claiming age, an assumed average market return rate and
COLAs. Section IV discusses two possible optimization paths for the claiming
age. Finally Sect. V presents a summary and conclusions of this study.

##  A break-even analysis

Here we introduce some of the parameters and terminology, all in the context
of a commonly-performed break-even analysis. Assume that $S_{0}$ is the yearly
benefit amount that would be available at age 70, i.e., the maximum benefit
allowed under the Social Security rules for a given prospective retiree. Since
the benefits can be claimed as early as age 62, we will always be comparing
two cases:

  * •

The early scenario: The beneficiary claims his benefits starting $K$ number of
years before age 70, where $1\leq K\leq 8$, or

  * •

The late scenario: He waits until age $70$ when his benefits reach the maximum
amount $S_{0}$.

Presently the social security benefits increase by $8\%$ for every year they
are delayed beyond the full retirement age (FRA); the increases stop at age
70[7]. We will use the parameter $p$ to denote this rate of increase, which,
unless otherwise noted, will have the value $p=0.08$ throughout this work. The
benefits are reduced, but at a smaller rate, if claimed before the full-
retirement age[6]. In order to simplify the calculations we will assume
symmetry about FRA and use $p$ for the rate of reduction also. This
simplification will make our results somewhat more pessimistic, from the point
of view of an early claimer, for claims made before FRA.

With this assumption, if the benefits are claimed $K$ years before age 70, the
annual amount is reduced to

$S_{K}=\frac{S_{0}}{(1+p)^{K}},$ (1)

where $p=0.08$. As stated earlier, the comparison is always with the “late
scenario” where the beneficiary claims at age 70 when his annual benefit
amount is $S_{0}$.

Since $S_{0}>S_{K}$, the total earnings with the late scenario, $T_{L}$, will
eventually exceed those from the early scenario, $T_{E}$. After $n$ number of
years past age 70, the total benefits for the early and late scenarios are

$\displaystyle T_{E}(K,n,p)$ $\displaystyle=$
$\displaystyle(K+n)S_{K}=\frac{(K+n)S_{0}}{(1+p)^{K}},\text{~{}and}$ (2)
$\displaystyle T_{L}(n)$ $\displaystyle=$ $\displaystyle nS_{0}.$ (3)

Note that we find it convenient to count the years before and after age 70
with different variables, $K$ and $n$, respectively. Thus, $K=2$ will refer to
age $70-K=68$. Similarly $n=10$ will refer to age $70+n=80$.

Let $n=n_{1}(K,p)$ be the solution of the equation $T_{E}=T_{L}$, the break-
even point. We can easily show that

$n_{1}(K,p)=\frac{K}{(1+p)^{K}-1},~{}~{}K\geq 1.$ (4)

A beneficiary who opts for the early scenario $(1\leq K\leq 8)$ starts falling
behind at age $70+n_{1}$; beyond that point, his cumulative benefits ($T_{E}$
in Eq. 2) will be less than $T_{L}$ that he would have enjoyed if he had
waited until age 70. In other words, for given values of the parameters $K,p$,
we have

$T_{E}(K,n,p)<T_{L}(n)\text{~{}for~{}}n>n_{1}.$ (5)

Equivalently, viewed from the point of view of a late claimer, the break-even
point comes at age $70+n_{1}$, after which he gets further ahead every year.
(In this work, we will view the results mostly from an early claimer’s point
of view.)

For some numerical examples, we have (with $p=0.08$):

$\displaystyle K=1$ $\displaystyle:$
$\displaystyle~{}n_{1}=\frac{1}{p}=12.5\text{~{}years},$ $\displaystyle K=8$
$\displaystyle:$ $\displaystyle~{}n_{1}=9.4\text{~{}years.}$ (6)

Thus, someone who claims benefits at age 69 ($K=1$) will start falling behind
at age $70+12.5\simeq 83$. A more impatient beneficiary who starts at age 62
($K=8$) will be behind by the time he reaches his $80^{th}$ birthday. Under
these conditions, and excluding all other factors (market gains, poor health,
spousal benefits, etc.) it is better to wait until age 70 to claim your
benefits, assuming that you will live beyond age 83. Note that this analysis
is independent of $S_{0}$, the benefit amount at age 70, and depends only on
$p$, the annual rate of increase in the benefits before age 70. Further
results for $1\leq K\leq 8$ are shown in the column labeled $n_{1}(q=0)$ of
Table 1.

Next we examine how these results are modified by the cost-of-living
adjustments (COLAs).

###  Effects of the cost-of-living adjustments

The benefits increase in general every year due to cost-of-living adjustments
implemented to compensate for inflation. The data for these annual increases
is available for the years 1975-2022 at the Social Security Administration
website[8].

Future COLA’s are of course unknown. For the purposes of these calculations we
choose to use an average COLA $q$ based on $N$ years of historical data; here
the average $q$ is defined by

$(1+q)^{N}=\prod_{i=1}^{N}(1+q_{i}),$ (7)

where $q_{i}$ is the COLA for the $i^{th}$ year. More explicitly, $q$ can be
put in the form

$q=\exp\left\\{\frac{1}{N}\sum_{i=1}^{N}\ln(1+q_{i})\right\\}-1.$ (8)

Using the full set of data for 1975-2022 in Eq. 8 results in an average COLA
of $q=0.03745$ or $3.7\%$. If we exclude the high-inflation period of the late
1970’s and average only over the last 40 years (1983-2022), we find
$q=0.02508$ or $2.5\%$.

Since the cumulative effect of these adjustments can be substantial over time,
here we examine how they modify our earlier results, in particular Eqs. 1-4.
We again assume $S_{0}$ is the annual benefit amount at age 70. As in the
“early scenario” above, if the beneficiary claims benefits $K$ years earlier,
$S_{K}$ of Eq. 1 has to be discounted by $K$ number of future COLA increases;
thus, the starting year benefits will now be

$S_{K}^{C}=\frac{S_{K}}{(1+q)^{K}}=\frac{S_{0}}{(1+p)^{K}(1+q)^{K}},$ (9)

where we use the superscript $C$ to denote COLA-adjusted values. Then, $n$
years after age 70, the total benefits received will be

$T_{E}^{C}=S_{K}^{C}\sum_{i=0}^{K+n-1}(1+q)^{i}=\frac{S_{0}}{(1+p)^{K}(1+q)^{K}}\frac{(1+q)^{K+n}-1}{q}.$
(10)

Since $\lim_{q\rightarrow 0}[(1+q)^{K+n}-1]/q=(K+n)$, $T_{E}^{C}$ reduces to
$T_{E}$ of Eq. 2 for $q\to 0$ as expected. In the “late scenario” where the
benefits start at age 70, the total after $n$ years will now be

$T_{L}^{C}=S_{0}\sum_{i=0}^{n-1}(1+q)^{i}=S_{0}\frac{(1+q)^{n}-1}{q}.$ (11)

Again, $T_{L}^{C}$ reduces to $T_{L}$ of Eq. 3 in the limit $q\rightarrow 0$.
In order to find when an early claimer starts to fall behind, for given
$K,p,q$, we need to solve the equation

$T_{E}^{C}(K,n,p,q)=T_{L}^{C}(n,q)$ (12)

for $n$. We can easily show that the solution is (for $q>0$) $n=n_{1}$, where

$\displaystyle n_{1}$ $\displaystyle=$
$\displaystyle\frac{a_{1}(K,p,q)}{a_{2}(q)},$ (13) $\displaystyle a_{1}$
$\displaystyle=$
$\displaystyle\ln\left\\{\frac{(1+p)^{K}(1+q)^{K}-1}{(1+q)^{K}[(1+p)^{K}-1]}\right\\},$
$\displaystyle a_{2}$ $\displaystyle=$ $\displaystyle\ln{(1+q)}.$

$~{}K~{}$ | $~{}n_{1}(q=0)~{}$ | $~{}n_{1}(q=0.025)~{}$ | $~{}n_{1}(q=0.037)~{}$
---|---|---|---
1 | 12.5 | 10.78 | 10.15
2 | 12.02 | 10.30 | 9.67
3 | 11.55 | 9.84 | 9.21
4 | 11.10 | 9.39 | 8.77
5 | 10.65 | 8.95 | 8.34
6 | 10.22 | 8.54 | 7.93
7 | 9.81 | 8.13 | 7.53
8 | 9.40 | 7.74 | 7.15
Table 1: The break-even point $n_{1}(q)$ for three different values of the
average COLA parameter $q$ is shown: $q=0$ (no COLA), $q=2.5\%$ (average for
the years 1983-2022), and $q=3.7\%$ (average for the years 1975-2022).
$p=0.08$ is assumed. Recall that $n_{1}$ measures the number of years after
age 70.

Note that Eq. 13 is valid in the limit $q\rightarrow 0$ but is not defined at
$q=0$. Recall that $n_{1}$ is the break-even point: For given $K,p,q$, and
early claimer starts falling behind at age $70+n_{1}$, i.e.,
$T_{E}^{C}(n)<T_{L}^{C}(n)$ for $n>n_{1}$. Table 1 shows that $n_{1}$
decreases as the COLA rate $q$ increases. For example, if the beneficiary
claims at age 66 (K=4), he starts falling behind at age 81 ($n_{1}=11.1$) with
no COLA ($q=0$). If the average COLA is $q=3.7\%$, he falls behind by age 79
($n_{1}=8.77$), two years earlier. This general decrease in $n_{1}$ with COLA
is mostly due to the decrease in the initial benefits:
$S_{K}^{C}=S_{K}/(1+q)^{K}<S_{K}$ (All scenarios assume a fixed yearly benefit
$S_{0}$ at age 70). In summary, taking into account the cost-of-living
adjustments makes early claims less attractive. Below we investigate another
scenario where claiming early may lead to a more positive outcome.

##  Taking advantage of possible market gains

In the previous section, we assumed that, if the beneficiary claimed his
Social Security benefits early (before age 70), he would be using them to meet
his financial needs. The analysis in this section is geared towards those
beneficiaries who can afford to wait until age 70 but may nevertheless want to
claim earlier, with the intention of investing the payments they receive in
the financial markets. Clearly there is no unique way to proceed in this
scenario. Investments can be made in myriad different ways. In order to obtain
some clear, quantitative results, we assume a rather simple scheme that can be
improved with more realistic details at a later date if necessary.

###  “Early claim with market gains” scenario–no COLAs

We consider a rather straightforward scheme and, for the time being, we ignore
the cost-of-living adjustments. The assumptions we make are as follows: As
before, the beneficiary files for benefits $K$ years before age 70, where
$1\leq K\leq 8$. But now, at the end of each year, until he is 70, he invests
the benefits received that year in a financial instrument with an average
annual return rate of $r$. Starting at 70, he stops diverting his benefits to
the financial markets but allows the already-accumulated sum to grow
indefinitely at the assumed annual rate of $r$.

Because of the exponential growth of the funds in the market (earnings
compounded annually), it is possible for this scenario to generate more total
income (integrated over some relevant period) than the late scenario of the
previous section. Here we determine a range of values for the parameters $K,r$
for which such a positive outcome is possible.

The average rate of return $r$, averaged over some number of years $N$ (for
our purposes, $N\simeq 30$ years) is defined similarly to the COLA parameter
$q$ of Eqs. 7, 8:

$r=\exp\left\\{\frac{1}{N}\sum_{i=1}^{N}\ln(1+r_{i})\right\\}-1,$ (14)

where $r_{i}$ is the rate for the $i^{th}$ year. In order to explore the
possible advantages of this scenario, we start with a calculation of the total
sum, $T_{M}$, that the beneficiary accumulates in the market before he reaches
age 70, i.e., the total value of his social security investments in the market
when he turns 70. Recalling Eq. 1 and using the assumptions outlined above,
this sum is given by

$T_{M}=S_{K}\sum_{i=0}^{K-1}(1+r)^{i}=\frac{S_{0}}{(1+p)^{K}}\frac{(1+r)^{K}-1}{r},~{}~{}K\geq
1.$ (15)

By assumption, this sum will be left in the market to grow; therefore, $n$
years after age 70, the total benefits for the early claim with market gains
scenario will be

$T_{E}^{M}=T_{M}(1+r)^{n}+nS_{K}.$ (16)

The first term on the right-hand side represents the growing amount in the
markets. The second term is the total received (and not invested) after age
70. $T_{E}^{M}$ can be written explicitly as

$T_{E}^{M}=\frac{S_{0}}{(1+p)^{K}}\left\\{\frac{(1+r)^{n}\left[(1+r)^{K}-1\right]}{r}+n\right\\}.$
(17)

This equation replaces Eq. 2 of the previous section. Note that in the limit
$r\rightarrow 0$ (thus removing the market gains), Eq. 17 reduces to Eq. 2 as
expected, since

$\lim_{r\rightarrow 0}\frac{(1+r)^{n}\left[(1+r)^{K}-1\right]}{r}=K.$ (18)

At this point it will be useful to define a relative gain function $g$ that
will measure the early claimer’s gains (or losses) as a function of time:

$g(K,n,p,r)\equiv\frac{T_{E}^{M}(K,n,p,r)-T_{L}(n)}{T_{L}(n)},$ (19)

where $T_{L}(n)$ is given by Eq. 3. For fixed $K,p,r$, a positive gain
function implies that the beneficiary is ahead ($n$ years after 70) with
respect to the “late scenario” where he waits until age 70 to start collecting
his benefits.

Using Eqs. 3, 17, the gain function can be written explicitly in the form

$g(K,n,p,r)=\frac{1}{(1+p)^{K}}\left\\{\left[\frac{(1+r)^{K}-1}{r}\right]\left[\frac{(1+r)^{n}}{n}\right]+1\right\\}-1.$
(20)

For fixed $K,p,r$ we have

$\lim_{n\to 0}g(K,n,p,r)=\infty,~{}~{}\lim_{n\to\infty}g(K,n,p,r)=\infty.$
(21)

Thus, $g(K,n,p,r)$ is a non-monotonic, convex function of $n$ with a minimum
at some $n=n^{\ast}$. The point where the gain function attains its minimum
value (again for fixed $K,p,r$) can be calculated easily. Treating $n$ as a
continuous time variable and letting

$\left.\frac{\partial g}{\partial n}\right|_{n=n^{\ast}}=0$ (22)

leads to

$n^{\ast}(r)=\frac{1}{\ln{(1+r)}},$ (23)

which is independent of the parameters $K,p$. Making use of $n^{\ast}$, we
find the minimum gain

$g_{min}(K,p,r)=g[K,n^{\ast}(r),p,r].$ (24)

In the following analysis, the return rate $r=r^{\ast}$ for which $g_{min}=0$
will play an important role; it can be calculated by solving the equation

$g[K,n^{\ast}(r^{\ast}),p,r^{\ast}]=0$ (25)

for $r^{\ast}$ with given $K,p$. In general this step requires a numerical
solution, but the case $K=1$ can be treated analytically because of
simplifications in the function $g(K,n,p,r)$. For fixed $p$ and $K=1$, Eqs. 23
and 25 lead to

$n^{\ast}=\frac{e}{p},~{}~{}~{}~{}r^{\ast}=e^{p/e}-1~{}~{}~{}~{}(K=1),$ (26)

where $e$ is the base of the natural logarithm. For $K>1$ the analytic results
are not very illuminating, and we use Mathematica[9] for numerical solutions.
Values of $n^{\ast},~{}r^{\ast}$ for $1\leq K\leq 8$ without the COLA
modifications are tabulated in the columns labeled $n^{\ast}(q=0)$ and
$r^{\ast}(q=0)$, respectively, of Table 2.

An important property of the gain function that will be useful below is

$r_{1}>r_{2}\Rightarrow g(K,n,p,r_{1})>g(K,n,p,r_{2}),$ (27)

which follows directly from Eq. 20. It is also intuitively obvious, since
higher market yields will naturally lead to higher relative gains.

Figure 1: The gain function $g(K,n,p,r)$, with no COLAs, for various values of
$r$, and $K=1,~{}p=0.08$. For $r=r^{\ast}=0.02987$, $g(K,n,p,r)\geq 0$; the
minimum (zero) occurs at $n=n^{\ast}=33.98$ (see the text). The horizontal
axis shows the number of years after age 70.

As an example, behavior of $g(K,n,p,r)$ is shown in Fig. 1 for various values
of $r$ and $K=1,~{}p=0.08$. As seen in the figure, for given $K,p,r$, the
equation $g(K,n,p,r)=0$ can have zero, one, or two solutions for $n$:

  * •

No solutions (the solid green curve); here $g(K,n,p,r)>0$ for all $n$,
implying that the early-claimer is always ahead for this value of $r$, the
average market return rate.

  * •

Only one solution, at $n=n^{\ast}$ for $r=r^{\ast}$ (the dotted red curve);
here the beneficiary is always ahead except at $n=n^{\ast}$, where his
position temporarily becomes neutral ($g=0$). The parameters
$n^{\ast},~{}r^{\ast}$ will play a critical role in our analysis, since
$r>r^{\ast}$ implies $g(K,n,p,r)>g(K,n,p,r^{\ast})\geq
g(K,n^{\ast},p,r^{\ast})=0$ for all $n$.

  * •

Two distinct solutions, $n=n_{1},~{}n_{2}$ (the dashed blue curve). For this
value of $r$, the beneficiary starts falling behind at $n=n_{1}$ (age
$70+n_{1}$) since $g<0$ for $n_{1}~{}<~{}n~{}<~{}n_{2}$. The gain function is
positive again for $n>n_{2}$; however, $70+n_{2}$ is too large a number to be
relevant for most cases. For the parameters used in Fig. 1, we have
$n_{1}~{}=~{}20.87$ and $n_{2}~{}=~{}70.32$, again found numerically with
Mathematica[9].

$~{}K~{}$ | $~{}~{}n^{\ast}(q=0)~{}$ | $~{}~{}r^{\ast}(q=0)~{}$ | $~{}~{}n^{\ast}(q=0.025)~{}$ | $~{}~{}r^{\ast}(q=0.025)~{}$ | $~{}~{}n^{\ast}(q=0.037)~{}$ | $~{}~{}r^{\ast}(q=0.037)~{}$
---|---|---|---|---|---|---
1 | 33.98 | 0.02987 | 34.58 | 0.04394 | 35.69 | 0.05128
2 | 33.17 | 0.03061 | 33.53 | 0.04483 | 34.45 | 0.05221
3 | 32.39 | 0.03135 | 32.54 | 0.04573 | 33.30 | 0.05314
4 | 31.65 | 0.03210 | 31.62 | 0.04662 | 32.24 | 0.05406
5 | 30.93 | 0.03286 | 30.75 | 0.04751 | 31.26 | 0.05498
6 | 30.23 | 0.03363 | 29.93 | 0.04840 | 30.35 | 0.05589
7 | 29.57 | 0.03440 | 29.16 | 0.04928 | 29.52 | 0.05679
8 | 28.93 | 0.03517 | 28.44 | 0.05016 | 28.74 | 0.05767
Table 2: The critical parameters $n^{\ast}$ and $r^{\ast}$ for the “early
claim with market gains” scenario for various values of $K$, and $p=0.08$. The
columns with $q=0$ (no COLAs) are solutions of Eqs. 23, 25. The remaining
columns with $q>0$ (with COLAs) are solutions of Eqs. 36, 37.

In order to emphasize the meaning of these important parameters, we will go
through an example. Making use of Table 2, assume that a beneficiary claims
his benefits at age 62 ($K=70-62=8$) and invests his Social Security income in
some market instrument (as detailed previously) with an average annual return
rate of $r^{\ast}\simeq 3.5\%$. With these assumptions, his cumulative
benefits will continue to exceed those from the “late scenario,”
$T_{E}^{M}>T_{L}$, until he is $70+n^{\ast}\simeq 99$ years old. At that time
his position becomes neutral ($T_{E}^{M}=T_{L}$) briefly (the break-even point
from the point of view of the “late scenario”), but he gets ahead again beyond
that age. And a final crucial point is that, for any given $K$, if
$r>r^{\ast}$ the beneficiary remains always ahead: $T_{E}^{M}>T_{L}$ for all
$n$.

Next we examine how these results are modified when the cost-of-living
adjustments are included in the analysis.

###  “Early claim with market gains” scenario–with COLAs

Making use of some of our earlier results, here we look at the effects of
cost-of-living adjustments. With COLA’s, the starting year benefit $S_{K}^{C}$
(see Eq. 9) will grow both due to market gains and COLAs; thus, the total sum
the beneficiary accumulates in the market before age 70 will now be

$T_{M}^{C}=S_{K}^{C}\sum_{i=0}^{K-1}(1+q)^{i}(1+r)^{i}=\frac{S_{0}}{(1+p)^{K}(1+q)^{K}}\frac{(1+q)^{K}(1+r)^{K}-1}{(1+q)(1+r)-1},$
(28)

which replaces $T_{M}$ of Eq. 15. (Recall that the COLA-modified terms have a
superscript $C$). By assumption, $T_{M}^{C}$ is left in the market after
reaching 70 and continues to grow at the rate $r$. After 70, however, the new
COLA-modified benefits are not invested but presumably used for other
purposes. Thus, $T_{E}^{M}$ of Eq. 17 representing the accumulated benefits at
age $70+n$ now becomes

$\displaystyle T_{E}^{MC}$ $\displaystyle=$ $\displaystyle
T_{M}^{C}(1+r)^{n}+S_{K}\sum_{i=0}^{n-1}(1+q)^{i},\text{~{}or}$ $\displaystyle
T_{E}^{MC}$ $\displaystyle=$
$\displaystyle\frac{S_{0}(1+r)^{n}}{(1+p)^{K}(1+q)^{K}}\frac{(1+q)^{K}(1+r)^{K}-1}{(1+q)(1+r)-1}+\frac{S_{0}}{(1+p)^{K}}\frac{(1+q)^{n}-1}{q}.$
(29)

The total benefits at age $70+n$ with the “late scenario” and including COLAs
is still given by Eq. 11:

$T_{L}^{C}=S_{0}\frac{(1+q)^{n}-1}{q}.$ (30)

At this point we can do a couple of consistency checks: Since $\lim_{q\to
0}[(1+q)^{n}-1]/q=n$, $T_{E}^{MC}$ and $T_{L}^{C}$ reduce to $T_{E}^{M}$ and
$T_{L}$ of Eqs. 17, 3, respectively, when the COLAs are ignored ($q\rightarrow
0$). Similarly, when the COLAs are retained but the market gains are ignored
($r\to 0$), $T_{E}^{MC}$ reduces to $T_{E}^{C}$ of Eq. 10 as expected.

As we saw earlier in Sect. II.1 on the effects of COLAs without the market
gains, we intuitively expect that the benefits of claiming early will be
reduced with the COLAs, even when the market gains are taken into account. In
particular, we expect that the ratio of the average market return to the
average COLA, $r/q$, will play an important role in determining how useful an
early claim will be. In order to understand these issues better, we again look
at the gain function and its time derivative. The new gain function with the
COLAs can be written as

$\displaystyle g^{c}(K,n,p,q,r)$ $\displaystyle\equiv$
$\displaystyle\frac{T_{E}^{MC}-T_{L}^{C}}{T_{L}^{C}},\text{~{}~{}or}$ (31)
$\displaystyle g^{c}(K,n,p,q,r)$ $\displaystyle=$
$\displaystyle\frac{1}{(1+p)^{K}}\left\\{\left[\frac{(1+q)^{K}(1+r)^{K}-1}{(1+q)^{K}[(1+q)(1+r)-1]}\right]\left[\frac{q(1+r)^{n}}{(1+q)^{n}-1}\right]+1\right\\}-1,$

which reduces to $g(K,n,p,r)$ of Eq. 20 in the limit $q\to 0$, as expected.
Letting

$A(K,p,q,r)\equiv\frac{q}{(1+p)^{K}}\frac{(1+q)^{K}(1+r)^{K}-1}{(1+q)^{K}[(1+q)(1+r)-1]},$
(32)

we can write the time derivative $\partial g^{c}/\partial n$ in the form

$\frac{\partial g^{c}}{\partial
n}=A(1+r)^{n}\left\\{\frac{(1+q)^{n}\ln[(1+r)/(1+q)]-\ln(1+r)}{[(1+q)^{n}-1]^{2}}\right\\}.$
(33)

By simple inspection, we can draw some general conclusions from Eqs. 31-33:

  * •

The coefficient $A(K,p,q,r)>0$ for $r,q>0$. Thus,

$\frac{\partial g^{c}}{\partial n}<0\text{~{}~{}for~{}}0<r\leq q.$ (34)

In other words, when the market return rate is less than the average COLA, the
relative gain will be a monotonically decreasing function of time (for fixed
$K,p,q,r$). In fact, for strict inequality, $r<q$, we have

$\lim_{n\to\infty}g^{c}=\frac{1}{(1+p)^{K}}-1<0.$ (35)

Under these circumstances, the equation $g^{c}=0$ will have only one solution
($n=n_{1}$). An early-claiming beneficiary will fall behind at age $70+n_{1}$
and will never recover ($g^{c}(K,n,p,q,r)\leq 0$ for $n\geq n_{1}$).

  * •

For $r>q>0$, $\lim_{n\to\infty}g^{c}=+\infty$, and the gain function displays
a behavior similar to what we observed earlier when we considered the market
gains without the COLAs. It will have a minimum at $n=n^{\ast}$, where
$n^{\ast}$ is a solution of the equation $\partial g^{c}/\partial n=0$. We can
easily show that

$n^{\ast}(q,r)=\frac{1}{\ln{(1+q)}}\ln\left\\{\frac{\ln{(1+r)}}{\ln{[(1+r)/(1+q)]}}\right\\}.$
(36)

Note that $\lim_{r\to q}n^{\ast}=\infty$, consistent with the discussion
above. Now the equation $g^{c}=0$ may again have zero, one or two solutions,
depending on the parameters $K,p,q,r$. As in Sect. III.1, the parameter
$r^{\ast}$ will be a solution, for fixed $K,p,q$, of the equation

$g^{c}[K,n^{\ast}(q,r^{\ast}),p,q,r^{\ast}]=0.$ (37)

In order to re-emphasize the significance of the parameters
$n^{\ast},~{}r^{\ast}$, we summarize here some of their important properties:

$\displaystyle g_{min}(K,p,q,r)$ $\displaystyle=$ $\displaystyle
g^{c}[K,n^{\ast}(q,r),p,q,r],$ (38) $\displaystyle g_{min}(K,p,q,r^{\ast})$
$\displaystyle=$ $\displaystyle g^{c}[K,n^{\ast}(q,r^{\ast}),p,q,r^{\ast}]=0.$
(39)

In other words, the gain function has a minimum at $n=n^{\ast}$, and that
minimum is zero for $r=r^{\ast}$. A consequence of Eqs. 27, 38, 39, for fixed
$K,p,q$, is

$g^{c}(K,n,p,q,r>r^{\ast})>0\text{~{}~{}for~{}all~{}}n.$ (40)

Thus, an early-claiming beneficiary will always be ahead if he can guarantee
an average market return rate of $r>r^{\ast}$, even with the COLAs. However,
we will find that, with everything else fixed, $r^{\ast}$ will be higher with
the COLAs.

These different types of behavior for the gain function $g^{c}$ are
illustrated in Fig. 2 for $K=~{}1,~{}p=0.08$ and $q=0.025$, the average COLA
for the years $1983-2022$. The dashed blue curve has $r=0.02<q$; hence the
negative slope and a zero at $n_{1}=13.6$. The early claimer falls behind at
$70+n_{1}\simeq 84$ and never recovers. The dotted red curve is for
$r=r^{\ast}=0.04394>q$. It has a minimum (zero) at
$n^{\ast}=34.58:~{}g^{c}(n^{\ast},r^{\ast})=0$. The solid green curve has
$r=0.05>r^{\ast}>q$; hence $g^{c}>0$ for all $n$.

Figure 2: The gain function $g^{c}(n)$ (with COLAs) for three different values
of $r$, and $K=1,~{}p=0.08,~{}q=0.025$. For these parameters we have
$r^{\ast}=0.04394,~{}n^{\ast}=34.58$ (Table 2). The dashed blue curve:
$r=0.02<q$. The dotted red curve: $r=r^{\ast}>q$; it has a minimum at
$n=n^{\ast}$ where $g^{c}(n^{\ast},r^{\ast})=0$. The solid green curve:
$r=0.05>r^{\ast}>q$.

In Table 2, columns with $q>0$ are solutions of the coupled Eqs. 36 and 37.
Since they are both complicated functions of the parameters, they are solved
iteratively for $\\{n^{\star},r\\}$, with the converged solution yielding
$\\{n^{\ast}(q,r^{\ast}),r^{\ast}=r\\}$. Note that the solutions of Eqs. 36,
37 ($q>0$, with COLA) will agree with those of Eqs. 23, 25 ($q=0$, no COLA) in
the limit $q\to 0$. We find that we are able to reproduce the $q=0$ results in
the Table (obtained using Eqs. 23, 25) from Eqs. 36, 37 with $0<q<10^{-5}$.
However, $q=0$ is a singular limit for the equations with COLA, since they are
undefined at $q=0$.

Some general comments regarding the $\\{n^{\ast},r^{\ast}\\}$ values in Table
2 may be helpful. Recall that the gain function (Eq. 31) has a minimum at
$n=n^{\ast}(q,r)$. The minimum value is zero for $r=r^{\ast}$, i.e.,
$g^{c}(n^{\ast},r^{\ast})=0$, where we assume the other parameters, $K,p,q$
are held fixed. Recalling Eqs. 38-40, for $r>r^{\ast}$ we have

$g^{c}(n,r)\geq g^{c}(n^{\ast},r)>g^{c}(n^{\ast},r^{\ast})=0.$ (41)

Thus, a beneficiary who files early will have a positive gain throughout his
life, if he can maintain an average market return rate of $r>r^{\ast}$. The
values of $n^{\ast}$ in Table 2 are approximately in the range $28-36$; thus,
the minimum of the gain function occurs when the beneficiary is in his late
90’s $(70+28)$ at the earliest (for $r\geq r^{\ast}$). In order to maintain
positive gains during this time, he has to ensure an average market return
rate of approximately $3-3.5\%$ (depending on how early he claims), if the
COLAs are not taken into account ($q=0$ columns in the table). With COLAs, the
required rate of return is approximately $4.4-5.0\%$ for $q=2.5\%$, the
average COLA for the years $1983-2022$. The required rate may be as high as
$5.8\%$ if $q=3.7\%$, the average for the years $1975-2022$ (the entire data
set at [8]), is used.

##  Optimization

In the previous section we were concerned with finding parameter regimes where
an early claimer can have positive gains with respect to the “late scenario,”
possibly for the rest of his life, when both the cost-of-living adjustments
and possible market gains are taken into account. It is possible to go a step
further and find an optimal time for claiming benefits as a function of the
remaining parameters in the problem. Mathematically, the problem reduces to
finding a maximum for the gain function $g^{c}=g^{c}(K,n,p,q,r)$ in a
5-dimensional space. Clearly the problem is quite unwieldy in this form; by
choosing $p=0.08$, as we have done throughout this work, and using $q=0.025$
for the average COLA parameter, the dimension can be reduced to a more
manageable three. Then it is possible to find an optimal time for claiming
benefits, or a $K_{opt}(n,r)$, as a function of an assumed average market
return rate, $r$, and a time, $n$, in the future. Below we examine a couple of
different optimization paths.

###  Maximizing the minimum gain

Here instead of choosing a specific time in the future, we let $n=n^{\ast}$
(see Eq. 36) where the gain function has its minimum, i.e., we seek a
$K_{opt}$ that maximizes $g^{c}_{min}$. Recall that for the range of
parameters $q,~{}r$ we have been considering, $n^{\ast}\simeq 28-36$ (Table
2), corresponding to an age range of 98-106.

The mathematical problem can now be stated as follows:

  * •

Find $K=K_{opt}(n^{\ast},r)$ that maximizes $g^{c}(K,n^{\ast},p,q,r)$ for
$p=0.08,~{}q=0.025$ and an arbitrary $r>q$ (Recall that $g^{c}$ is
monotonically decreasing for $r\leq q$, a regime we try to avoid).

We start by slightly rewriting $g^{c}$ of Eq. 31:

$\displaystyle g^{c}(K,n,p,q,r)$ $\displaystyle=$ $\displaystyle
B(n,q,r)\left\\{[(1+r)/(1+p)]^{K}-\frac{1}{[(1+p)(1+q)]^{K}}\right\\}+\frac{1}{(1+p)^{K}}-1,$
$\displaystyle\text{where~{}~{}}B(n,q,r)$ $\displaystyle\equiv$
$\displaystyle\frac{q(1+r)^{n}}{[(1+q)(1+r)-1][(1+q)^{n}-1]}.$ (42)

Then treating $K$ as a continuous variable and setting $\partial
g^{c}/\partial K=0$ leads to

$B\left\\{(1+r)^{K}\ln\left(\frac{1+r}{1+p}\right)+\frac{\ln[(1+p)(1+q)]}{(1+q)^{K}}\right\\}-\ln(1+p)=0.$
(43)

Although Eq. 43 can be solved for $K=K_{opt}$ analytically in certain limits
(e.g., $r=p$), in general it requires a numerical solution. Figure 3 shows
$K_{opt}(n^{\ast},r)$ (from Eq. 43), $n^{\ast}(q,r)$ (from Eq. 36) and
$g^{c}_{min}=g^{c}(K_{opt},n^{\ast},q,p,r)$ as functions of $r$.

Figure 3: (a) $K_{opt}$ that maximizes the minimum of the relative gain,
$g^{c}_{min}$, as a function of the assumed average market return rate $r$.
Recall that the optimal age is given by $70-K_{opt}$. Also shown is the
parameter $n^{\ast}(q,r)$ where the minimum occurs. Again recall that $n$
measures years after age $70$; thus, $g^{c}_{min}$ occurs at age
$70+n^{\ast}$. (b) $g^{c}_{min}$ as a function of $r$.

The limits for the average market return rate, $0.0440\leq r\leq 0.0598$, are
chosen to keep $K_{opt}$ within the allowed arrange of $0<K_{opt}\leq 8$ (the
optimal age has to be within $62-70$). A number of observations follows from
Fig. 3:

  * •

$K_{opt}$ increases approximately linearly with the market return rate (Fig.
3(a)); thus, the higher the market rate, the earlier is the optimal claiming
age, $70-K_{opt}$, which is intuitively obvious.

  * •

The optimized minimum gain, $g^{c}_{min}$, increases faster than linearly with
$r$ (Fig. 3(b)). At $r=6\%$, the optimal claiming age is 62 ($K_{opt}=7.99$).
The beneficiary’s minimum gain with respect to the late scenario is $14\%$,
which occurs at age $70+n^{\ast}=92.4$. At any other age, his relative gain is
higher since $g^{c}(K_{opt},n,p,q,r)>g_{min}$ for $n\neq n^{\ast}$.

  * •

The sum $K_{opt}+n^{\ast}$, representing the time span from the claiming age
to when the gain function attains its minimum, is relatively constant:
$30.4\leq K_{opt}+n^{\ast}\leq 35.0$, which has two implications:

    * –

When calculating the average COLA rate $q$ (Eq. 8) or the expected average
market return rate $r$ (Eq. 14), the time span to consider is approximately
30-35 years from the start of the benefits. This is a fairly long period that
should be enough to smooth out the market or economic fluctuations. Thus
$r=5-6\%$ and $q=2.5\%$ are not unreasonable assumptions.

    * –

Regardless of where the optimal claiming age happens to be, the optimized
minimum gain is approximately 30-35 years in the future from that point. And
neither ill-health nor extreme longevity will leave the beneficiary worse off
than what is implied by $g^{c}_{min}$. Therefore, finding a claiming age that
maximizes the minimum of the relative gain is a reasonable strategy.

###  Optimizing for a particular age

At this point a complication that we should consider is the following:
Optimizing $g^{c}_{min}$ does not imply that the gain function itself has been
optimized for all ages. In other words, we may still have
$g^{c}(K_{opt},n,p,q,r)<g^{c}(K,n,p,q,r)$ for some values of $K\neq
K_{opt},~{}n\neq n^{\ast}$. In fact, the equation

$g^{c}(K_{opt},n,p,q,r)=g^{c}(K,n,p,q,r),$ (44)

with fixed $p,q,r$ tends to have two distinct solutions, $n=n_{1},n_{2}$, for
$K>K_{opt}$. Thus the gain function for $K>K_{opt}$ will be higher for
$n<n_{1}$ and $n>n_{2}$. This point is illustrated in Fig. 4 where we plot
$g^{c}$ as a function of $n$ for various values of $K$, and
$p=0.08,~{}q=0.025,~{}r=0.0525$. The minima for all three curves are at
$n=n^{\ast}(q,r)=26.7$, and clearly the minimum for $K=K_{opt}=4.69$ curve is
higher than those of the $K=2,7$ curves. However, the $K=7$ curve (dotted red)
crosses above the $K=K_{opt}$ curve (solid green) at $n_{1}=17.28$ and
$n_{2}=39.13$. In other words, although $g^{c}_{min}$ is optimized with
$K=K_{opt}=4.69$, for the years corresponding to $n<n_{1}$ (earlier than age
87), or $n>n_{2}$ (later than age 109), the beneficiary may be better off with
$K=7$, i.e., claiming at age 63 instead of $70-K_{opt}\simeq 65$.

Figure 4: The relative gain function $g^{c}(K,n,p,q,r)$ for three different
values of $K$, with $p=0.08,q=0.025$ and $r=0.0525$. The solid green curve is
for $K=K_{opt}$, optimized for $n=n^{\ast}$. The dashed blue curve for
$K=2<K_{opt}$ always remains below the optimized curve; however, the dotted
red curve for $K=7>K_{opt}$ crosses above the optimal curve at $n=n_{1}=17.28$
and $n=n_{2}=39.13$.

In order to address this particular complication, the gain function itself
(not its minimum) can be optimized for any given age using the analytic
expressions given above. The mathematical problem we need to solve now is the
following:

  * •

Find solutions of Eq. 43 for $K=K_{opt}(n,r)$, which maximizes
$g^{c}(K,n,p,q,r)$ with given $p,q,r$, and for various values of $n$ while
ensuring that $0<K_{opt}\leq 8$.

Figure 5: $K_{opt}$ and the optimized value of the gain function $g^{c}$ as a
function of $n$, the number of years after age 70. Recall that the optimal
claiming age is given by $70-K_{opt}$, and $g^{c}$ measures the gain with
respect to the “late scenario” where the beneficiary waits until age 70 to
claim. (a) Average market return rate $r=0.045$. (b) $r=0.0575$. Note that in
(b) although the two curves nearly coincide, the scales are different.

A couple of examples are shown in Fig. 5, again for $p=0.08,~{}q=0.025$.
Recall that $g^{c}_{min}(K,p,q,r^{\ast})=g^{c}(K,n^{\ast},p,q,r^{\ast})=0$.
For the first example, in order to ensure $g^{c}_{min}>0$, we choose
$r=0.045$, which is just above the minimum $r^{\ast}$ for $q=0.025$ in Table
2. The results are shown in Fig. 5(a). For any given $n$, the figure shows
$K_{opt}$ and the resulting optimized gain. For example, for $n=10$ (age 80),
we have $K_{opt}=7.29,~{}g^{c}=0.175$, a again of 17.5% over the late
scenario. For $n=20$ (age 90), we get $K_{opt}=2.70$ and $g^{c}=0.018$, a gain
of only $1.8\%$ over the late scenario. The minimum gain is at
$n=n^{\ast}=33.34$ (age 103), where it is a mere $0.3\%$. In Fig. 5(b) where
we assume a slightly higher return rate of $5.75\%$, the gains are markedly
higher. For $15<n<35$ (ages 85-105), the optimized $g^{c}$ varies between a
minimum of $10.8\%$ at $n=n^{\ast}=23.6$ and $15.2\%$ at the end points of the
range. Similarly, $K_{opt}$ varies little: $6.94\leq K_{opt}\leq 8$ for this
value of $r$. Thus, with a rather modest market return of $5.8\%$, a
beneficiary who claims at age 62-63 would indeed be well ahead for the rest of
his life.

##  Summary and conclusions

The Social Security benefits provide a financial lifeline to most retirees,
and a retiree typically claims his benefits as soon as he reaches his full-
retirement age (FRA), or as early as age 62 if he is willing to accept some
penalties. If a retiree is not financially constrained, however, there is an
incentive for waiting until age 70 because of the built-in yearly increases to
benefits (presently $8\%$) between his FRA and age 70. Ironically, precisely
in those cases where filing at FRA or earlier is not a financial necessity, it
may be beneficial to file early without waiting until age 70. This work
focused on the decision of whether to do so and when the optimal time might
be.

Starting with a simple “break-even” analysis that retirees typically do to
find out when they would break even if they delayed retirement beyond FRA, we
developed a series of successively more comprehensive analytic models that
compare early-claiming scenarios with the “late scenario” where the
beneficiary waits until age 70. First improvement on the break-even analysis
was the inclusion of inflation through the cost-of-living adjustments (COLAs)
in Sect. II. From the point of view of an early-claimer (a view adopted
throughout this work) COLAs make early claims less attractive. Conversely,
from the point of view of a late claimer, they bring the break-even point
closer (Table 1). Results of filing early and investing the benefits received
in the financial markets were examined in Sect. III. As summarized in Table 2,
if the COLAs are ignored, even a modest market return of $3.5\%$ would be
sufficient for a beneficiary who files at age 62 to be ahead of the late
scenario all his life. If the COLAs are taken into account, that return rate
would have to be $5\%$ if the average COLA is $2.5\%$, and $5.8\%$ if the COLA
rate is at its historical average of $3.7\%$. Since the timespan relevant to
our discussion is approximately three decades from the time of claiming
benefits, these market rates, averaged over 30-35 years, are not unreasonable.

If the beneficiary wants to be not just ahead of the late scenario but to
optimize his gains, then he has a number of options, of which we examined only
two. First, we found an optimal filing age that maximizes his minimum gain,
$g^{c}_{min}$, as a function of the assumed market return rate (Fig. 3). For
example, if the average market rate $r=5.8\%$, we find $K_{opt}=6.9$, implying
an optimal age of approximately 63 ($70-K_{opt}$) for claiming; the minimum
gain of $11\%$ occurs at $n^{\ast}=23.6$, corresponding to an age of over 93
($70+n^{\ast}$). Both before and after that age, the gain is higher (on either
side of the minimum).

The second option we examined was finding an optimal claiming age that
maximizes the relative gain at a specific point in time, for two different
market rates (Fig. 5). We found that the claiming age is rather insensitive to
the age for which the gain is maximized ($K_{opt}\simeq 7-8$) if $r=5.8\%$,
although there is considerable variation in both $K_{opt}$ and the maximized
gain for $r=4.5\%$.

The short conclusion from these analyses is that, under the conditions
explained at the beginning of Sect. III, there is no financial incentive for
waiting until age 70 for claiming Social Security benefits. A beneficiary
would be almost always ahead if he were to claim quite early (age 62-63), if
the market return rate on his investments over the next 30-35 years is
approximately $5-6\%$.

Real-life scenarios are usually more complex than the simplified mathematical
models discussed in this work. We made some assumptions in order to make
analytical progress, and we did not consider important factors such as taxes
and spousal benefits. It would also be beneficial to examine optimization that
takes into account a probabilistic survival function. We hope to address these
issues in a future publication. It is important to note that, while we
provided examples in our figures and tables for a variety of parameters, the
analytic expressions derived here can be useful more generally. An individual
can customize the choice of parameters to his own circumstances and use these
models to make a timing decision that aligns with his goals better.

###### Acknowledgements.

The author gratefully acknowledges useful comments by Gary Hallock that helped
improve the manuscript. This research did not receive any specific grant from
funding agencies in the public, commercial, or not-for-profit sectors.

## References

  * Coile et al. [2002] C. Coile, P. Diamond, J. Gruber, and A. Jousten, Journal of Public Economics 84, 357 (2002).
  * Friedman and Phillips [2008] J. Friedman and H. Phillips, Financial Services Review 17, 155 (2008).
  * Alleva [2016] B. J. Alleva, Social Security Bulletin 76, 1 (2016).
  * Ali et al. [2019] Y. Ali, M. Fang, P. A. A. Sota, S. Taylor, and X. Wang, Risks 7, 124 (2019).
  * Diamond et al. [2021] S. Diamond, S. Boyd, D. Greenberg, M. Kochenderfer, and A. Ang, https://doi.org/10.48550/arXiv.2106.00125 (2021).
  * [6] https://www.ssa.gov/benefits/retirement/planner/agereduction.html.
  * [7] https://www.ssa.gov/benefits/retirement/planner/delayret.html.
  * [8] https://www.ssa.gov/oact/cola/colaseries.html.
  * [9] Wolfram Research, Inc., _Mathematica, Version 13.1_ , Champaign, IL, 2022, URL https://www.wolfram.com/mathematica.


# AraBART: a Pretrained Arabic Sequence-to-Sequence Model for Abstractive
Summarization

Moussa Kamal Eddine  École Polytechnique, 2Université Sorbonne Paris Nord,
3New York University Abu Dhabi, 4AUEB  Nadi Tomeh  Nizar Habash  Joseph Le
Roux  Michalis Vazirgiannis  École Polytechnique, 2Université Sorbonne Paris
Nord, 3New York University Abu Dhabi, 4AUEB

###### Abstract

##  1 Introduction

Summarization is the task of transforming a text into a shorter representation
of its essential meaning in natural language. Extractive approaches Nallapati
et al. (2017); Narayan et al. (2018b); Zhou et al. (2018); See et al. (2017)
identify informative spans in the original text and stitch them together to
generate the summary. Abstractive approaches on the other hand are not
restricted to the input Rush et al. (2015); Chopra et al. (2016); Dou et al.
(2021). While the vast majority of published models in both categories focused
on English, some tackled other languages including Chinese Hu et al. (2015)
and French Kamal Eddine et al. (2021b), while Arabic remained understudied. In
fact, most Arabic summarization models are extractive Qassem et al. (2019);
Alshanqiti et al. (2021). They generate explainable and factual summaries but
tend to be verbose and lack fluency. Addressing this problem, abstractive
models are flexible in their word choices, resorting to paraphrasing and
generalization to obtain more fluent and coherent summaries. Sequence-to-
sequence (seq2seq) is the architecture of choice for abstractive models. Al-
Maleh and Desouki (2020), for instance, apply the pointer-generator network
See et al. (2017) to Arabic, while Khalil et al. (2022) propose a more generic
RNN-based model. There are, however, two main issues with abstractive models
as applied to Arabic. First, they are trained and evaluated either on
extractive datasets such as KALIMAT El-Haj and Koulali (2013) and ANT Corpus
Chouigui et al. (2021), or on headline generation datasets such as AHS Al-
Maleh and Desouki (2020), which only contains short and rather extractive
headlines. Second, despite their state-of-the-art performance, abstractive
models frequently generate content that is nonfactual or unfaithful to the
original text. Maynez et al. (2020) showed that English models that are based
on the Transformer architecture such as Bert2Bert Rothe et al. (2020)
efficiently mitigate this phenomenon thanks to pretraining on large corpora.
Therefore, Elmadani et al. (2020) finetuned a pretrained BERT using the
encoder-decoder architecture of BertSum Liu and Lapata (2019). However, only
the encoder is pretrained, the decoder and the connection weights between the
encoder and the decoder are initialized randomly which is sub-optimal. To
address these two problems, we propose AraBART, the first sequence-to-sequence
Arabic model in which the encoder, the decoder and their connection weights
are pretrained end-to-end using BART’s denoising autoencoder objective Lewis
et al. (2020). While the encoder is bidirectional, the decoder is auto-
regressive and thus more suitable for summarization than BERT-based decoders.
We finetuned and evaluate our model on two abstractive datasets. The first is
Arabic Gigaword Parker et al. (2011), a newswire headline-generation dataset,
not previously exploited in Arabic abstractive summarization; the second is
XL-Sum, a multilingual text summarization dataset for 44 languages including
Arabic Hasan et al. (2021). AraBART achieves state-of-the-art results
outperforming pretrained BERT-based models as well as a much larger model,
mBART25 Liu et al. (2020), a multilingual denoising auto-encoder pretrained on
25 different languages using the BART objective. In section 2 we present the
architecture and the pretraining settings of AraBART. In section 3 we evaluate
and compare AraBART against three strong baselines on a wide range of
abstractive summarization datasets. Finally, we discuss related work in
section 4.

|  | Datasets
---|---|---
|  | AAW | AHR | AFP | HYT | NHR | QDS | XIN | MIX | XL-S | XL-T
Average #tokens | document | 453.3 | 394.2 | 232.8 | 474.0 | 455.9 | 450.6 | 187.2 | 364.5 | 428.7 | 428.7
summary | 15.5 | 9.2 | 8.3 | 11.2 | 10.4 | 8.0 | 8.2 | 9.4 | 25.6 | 9.4
%novel n-grams in summary | unigrams | 44.2 | 46.5 | 30.7 | 42.4 | 46.5 | 24.9 | 26.4 | 40.0 | 53.5 | 44.3
bigrams | 78.5 | 78.4 | 63.6 | 78.6 | 80.7 | 46.9 | 48.5 | 72.2 | 85.8 | 81.2
trigrams | 91.2 | 91.3 | 81.9 | 92.0 | 92.8 | 57.5 | 60.8 | 86.3 | 95.2 | 94.1

Table 1: Statistics of Gigaword subsets an XL-Sum summaries (XL-S) and titles
(XL-T). The first two lines show the average document and summary lengths. The
percentage of n-grams in the summary that do not occur in the input article is
used as a measure of abstractiveness Narayan et al. (2018a).

##  2 AraBART

AraBART follows the architecture of BART Base Lewis et al. (2020), which has 6
encoder and 6 decoder layers and 768 hidden dimensions. In total AraBART has
139M parameters. We add one additional layer-normalization layer on top of the
encoder and the decoder to stabilize training at FP16 precision, following Liu
et al. (2020). We use sentencepiece Kudo and Richardson (2018) to create the
vocabulary of AraBART. We train the sentencepiece model on a randomly sampled
subset of the pretraining corpus (without any preprocessing) with size 20GB.
We fix the vocabulary size to 50K tokens and the character coverage to 99.99%
to avoid a high rate of unknown tokens.

###  2.1 Pretraining

We adopt the same corpus used to pretrain AraBERT Antoun et al. (2020). While
Antoun et al. (2020) use a preprocessed version of the corpus, we opted to
reverse the preprocessing by using a script that removes added spaces around
non alphabetical characters, and also undo some words segmentation. The use of
a corpus with no preprocessing, makes the text generation more natural. The
size of the pretraining corpus before/after sentencepiece tokenization is
73/96 GB.

#### Pretraining Objective

AraBART is a denoising autoencoder i.e. it learns to reconstruct a corrupted
text. The noise function applied to the input text are the same as in Lewis et
al. (2020). The first noise function is text infilling, where 30% of the text
is masked by replacing a number of text spans with a [MASK] token. The length
of the spans is sampled from a Poisson distribution with $\lambda=3.5$. The
second noise function is sentence permutation, where the sentences of the
input text are shuffled based on the full stops.

#### Pretraining Settings

AraBART pretraining took approximately 60h. The pretraining was carried out on
128 Nvidia V100 GPUs which allowed for 25 full passes over the pretraining
corpus. We used the Adam optimizer with $\epsilon=10^{-6}$, $\beta_{1}=0.9$,
and $\beta_{2}=0.98$ following Liu et al. (2019). We use a warm up for 6% of
the pretraining were the learning rate linearly increases from 0 to 0.0006,
then decreases linearly to reach 0 at the end of the pretraining. We fixed the
update frequency to 2 and we use a dropout 0.1 in the first 20 epochs and we
changed it to 0 in the last 5 epochs. Finally we used FP16 to speed-up the
pretraining. The pretraining is done using Fairseq Ott et al. (2019).

##  3 Experiments

###  3.1 Datasets

To evaluate our model, we use several datasets that consist mostly of news
articles annotated with summaries with different level of abstractivness. The
first 7 datasets (AAW, AFP, AHR, HYT, NHR, QDS and XIN) are subsets of the
Arabic Gigaword Parker et al. (2011) corpus. Each one is a different news
source, composed of document-headline pairs. In all these datasets we use a
train set of 50K examples, a validation set of size 5K examples and a test set
of size 5K examples, selected randomly. The MIX dataset consists of 60K
examples uniformly sampled from the union of the 7 different sources. In
addition the Arabic Gigaword corpus, we use XL-Sum Hasan et al. (2021). The
news articles in XL-sum are annotated with summaries and titles, thus creating
two tasks: summary and title generation. Table 1 shows that the different
datasets used in our experiments cover a wide range of article/summary lengths
and levels of abstractivness.

###  3.2 Baselines

We compare our model to three types of state-of-the-art baselines. The first,
called C2C, is a monolingual seq2seq model based on Bert2Bert Rothe et al.
(2020). The encoder and decoder are initialized using CamelBert Inoue et al.
(2021) weights while the cross-attention weights are randomly
initialized.222We experimented with AraBERT Antoun et al. (2020) which was
slower to converge and didn’t achieve better performance. C2C has 246M
parameters in total. The second baseline is mBART25 Liu et al. (2020) which is
a multilingual BART pretrained on 25 different languages including Arabic.
Although mBART25 was initially pretrained for neural machine translation, it
was shown that it can be used in monolingual generative tasks such as
abstractive summarization Kamal Eddine et al. (2021b). mBART25 has 610M
parameters in total. While mBART25 is pretrained on multilingual corpora, we
finetuned it on Arabic data only. We therefore, include a third multilingual
baseline pretrained and finetuned on multilingual data. We use the
checkpoint333https://huggingface.co/csebuetnlp/mT5_multilingual_XLSum of
mT5base in the comparison on XL-S (summary). This checkpoint was finetuned on
the training set of the 45 different languages included in the corpus. The
total training size is 1M multilingual examples shuffled together Hasan et al.
(2021). mT5base has 582M parameters in total.

###  3.3 Training and Evaluation

We finetuned each model for three epochs, using the Adam optimizer and
$5\times 10^{-5}$ maximum learning rate with linear decay scheduling. In the
generation phase we use beam-search with beam size of 3. For evaluation, we
first normalize the output summaries as is standard practice in Arabic: we
removed Tatweel and diacritization, we normalized Alef/Yaa and separated
punctuations. We report ROUGE-1, ROUGE-2 and ROUGE-L f1-scores Lin (2004).
However, these metrics are solely based on surface-form matching and have
limited sense of semantic similarity Kamal Eddine et al. (2021a). Thus we
opted for using BERTScore Zhang et al. (2020), a metric based on the
similarity of the contextual embeddings of the reference and candidate
summaries, produced by a BERT-like model.444We use the official implementation
(https://github.com/Tiiiger/bert_score) with the following options: -m UBC-
NLP/ARBERT -l 9 Chiang et al. (2020)

Source | Model | R1 | R2 | RL | BS
---|---|---|---|---|---
AAW | AraBART | 30.7 | 15.3 | 27.4 | 62.5
| mBART25 | 29.5 | 14.4 | 26.0 | 61.5
| C2C | 24.6 | 9.87 | 21.7 | 58.3
AFP | AraBART | 55.0 | 37.9 | 53.4 | 77.5
| mBART25 | 54.8 | 37.3 | 52.8 | 77.2
| C2C | 50.0 | 32.2 | 48.4 | 74.8
AHR | AraBART | 39.1 | 25.4 | 37.7 | 68.2
| mBART25 | 39.1 | 26.1 | 37.5 | 68.1
| C2C | 33.0 | 19.7 | 31.8 | 63.5
HYT | AraBART | 33.1 | 17.5 | 30.7 | 63.8
| mBART25 | 32.0 | 16.2 | 29.3 | 63.1
| C2C | 27.4 | 11.5 | 25.2 | 59.6
NHR | AraBART | 32.0 | 17.2 | 30.3 | 61.2
| mBART25 | 31.0 | 16.2 | 29.2 | 60.3
| C2C | 24.1 | 10.0 | 22.9 | 53.0
QDS | AraBART | 62.1 | 53.9 | 61.4 | 80.3
| mBART25 | 62.4 | 54.1 | 61.7 | 80.4
| C2C | 57.9 | 48.9 | 57.4 | 77.3
XIN | AraBART | 66.0 | 53.9 | 65.1 | 84.4
| mBART25 | 65.1 | 53.4 | 64.2 | 84.0
| C2C | 62.4 | 50.1 | 61.6 | 82.5
MIX | AraBART | 39.2 | 25.5 | 37.6 | 67.6
| mBART25 | 39.0 | 25.6 | 37.1 | 67.2
| C2C | 32.8 | 19.1 | 31.4 | 62.5
XL-S | AraBART | 34.5 | 14.6 | 30.5 | 67.0
| mBART25 | 32.1 | 12.5 | 27.6 | 65.3
| C2C | 26.9 | 8.7 | 23.1 | 61.6
| mT5base | 32.8 | 12.7 | 28.7 | 65.8
XL-T | AraBART | 32.0 | 13.7 | 29.4 | 65.8
| mBART25 | 29.8 | 11.7 | 26.9 | 64.3
| C2C | 25.2 | 7.9 | 22.9 | 61.1
Macro | AraBART | 42.4 | 28.8 | 40.3 | 69.8
Averages | mBART25 | 41.5 | 28.1 | 39.2 | 69.1
| C2C | 36.4 | 23.1 | 34.6 | 65.4
Table 2: The performance of AraBART, mBART25 and C2C (CamelBert2CamelBert) on
all datasets in terms of ROUGE-1 (R1), ROUGE-2 (R2), ROUGE-L (RL) and
BERTScore (BS). Macro averages are computed over all datasets.

###  3.4 Results

We observe in Table 2 that AraBART outperforms C2C on all datasets with a
clear margin. This is probably a direct consequence of pretraining the seq2seq
architecture end-to-end. AraBART also outperforms mBART25 on XL-Sum which is
the most abstractive dataset. On Gigawords, AraBART is best everywhere except
on AHR with mitigated results. On QDS, however, it falls clearly behind
mBART25 on all metrics. In fact, we notice that the gap between AraBART and
the baselines is greater on the XL-Sum dataset than Gigaword. For instance,
our model’s ROUGE-L score is 2.9 absolute points higher that mBART25 on XL-S
while the maximum margin obtained on a Gigaword subset is 1.4 points on AAW
and HYT. We observe a tendency for AraBART to outperform mBART on more
abstractive datasets. In fact, the margin between their BERTScores is
positively correlated with abstractiveness as measures by the percentage of
novel trigrams.555With a Pearson R score of 0.6625 and $p$-value¡0.05. On the
XL-Sum dataset, AraBART also outperforms mT5 which was finetuned in
multilingual setup using more data Hasan et al. (2021). Figure 1 presents some
examples of the output of the various systems we studied.

##  4 Related Work

#### Arabic Summarization

The overwhelming majority of past Arabic models are extractive Douzidia and
Lapalme (2004); Azmi and Al-thanyyan (2009); El-Haj et al. (2011); El-
Shishtawy and El-Ghannam (2012); Haboush et al. (2012); Belkebir and Guessoum
(2015); Qaroush et al. (2021); Ayed et al. (2021). Recently, seq2seq
abstractive models for Arabic have been proposed in the literature Al-Maleh
and Desouki (2020); Suleiman and Awajan (2020); Khalil et al. (2022), but none
of them used pretraining. Fine-tuning Transformer-based language models like
BERT Devlin et al. (2019) has been shown to help Arabic abstractive Elmadani
et al. (2020) and extractive Helmy et al. (2018) summarization, but unlike
AraBART, not all components of the model are pretrained. Readily-available
multilingual pretrained seq2seq models have been applied to Arabic
summarization. Kahla et al. (2021) uses mBART25 Liu et al. (2020) in cross-
lingual transfer setup on an unpublished dataset, while Hasan et al. (2021)
experiment with mT5 Xue et al. (2021) on XL-Sum. Our model, tailored
specifically for Arabic, outperform mBART25 and mT5 for almost all datasets
despite having a smaller architecture with less parameters.

#### Arabic Datasets

Most available datasets for Arabic are extractive El-Haj et al. (2010);
Chouigui et al. (2021), use short headlines that are designed to attract the
reader Webz.io ; Al-Maleh and Desouki (2020), or contain machine-generated El-
Haj and Koulali (2013) or translated El-Haj et al. (2011) summaries. Notable
exceptions we choose for our experiments are Gigaword Parker et al. (2011) and
XL-Sum Hasan et al. (2021) because they cover both headline and summary
generation, contains multiple sources, and manifest variable levels of
abstractivness as shown in Table 1.

#### Pretrained seq2seq models

BART-based models have been developed for multiple language including English
Lewis et al. (2020), French Kamal Eddine et al. (2021b) and Chinese Shao et
al. (2021) in addition to multilingual models Liu et al. (2020). While they
can be finetuned to perform any language understanding or generation tasks, we
focus on summarization in this work.

##  5 Conclusion and Future Work

We release AraBART, the first sequenece-to-sequence pretrained Arabic model.
We evaluated our model on a set of abstractive summarization tasks, with
different level of abstractiveness. We compared AraBART to two state-of-the-
art models and we showed that it outperforms them almost everywhere despite
the fact that it is smaller in terms of parameters. In future work, we are
planning to extend the model to multitask setups to take advantage of
availability of both titles and summaries in some datasets including XL-Sum,
and use external knowledge sources to improve faithfulness. We will also
explore new directions for evaluating summarization on morphologically rich
languages like Arabic.

## Ethical Considerations

#### Limitations

Our models are optimized for news text summarization; we do not expect
compararble performance on other summarization tasks without additional
training data.

#### Risks

We acknowledge that our models sometimes produce incorrect non-factual and
non-grammatical output, which can be misleading to general users.

#### Data

All of the data we used comes from reputable news agencies and do not contain
unanonymized private information or malicious social media content.

#### Models

We will make our pretrained and finetuned models available on the well known
Hugging Face models hub666https://huggingface.co/models, so it can be easily
used and distributed for research or production purposes.

## References

  * Al-Maleh and Desouki (2020) Molham Al-Maleh and Said Desouki. 2020.  Arabic text summarization using deep learning approach.  _Journal of Big Data_ , 7:1–17.
  * Alshanqiti et al. (2021) Abdullah Alshanqiti, Abdallah Namoun, Aeshah Alsughayyir, Aisha Mousa Mashraqi, Abdul Rehman Gilal, and Sami Saad Albouq. 2021.  Leveraging distilbert for summarizing arabic text: An extractive dual-stage approach.  _IEEE Access_ , 9:135594–135607.
  * Antoun et al. (2020) Wissam Antoun, Fady Baly, and Hazem Hajj. 2020.  AraBERT: Transformer-based model for Arabic language understanding.  In _Proceedings of the 4th Workshop on Open-Source Arabic Corpora and Processing Tools, with a Shared Task on Offensive Language Detection_ , pages 9–15, Marseille, France. European Language Resource Association.
  * Ayed et al. (2021) Alaidine Ben Ayed, Ismaïl Biskri, and Jean-Guy Meunier. 2021.  Arabic text summarization via knapsack balancing of effective retention.  _Procedia Computer Science_ , 189:312–319.  AI in Computational Linguistics.
  * Azmi and Al-thanyyan (2009) Aqil Azmi and Suha Al-thanyyan. 2009.  Ikhtasir — a user selected compression ratio arabic text summarization system.  In _2009 International Conference on Natural Language Processing and Knowledge Engineering_ , pages 1–7.
  * Belkebir and Guessoum (2015) Riadh Belkebir and Ahmed Guessoum. 2015.  A supervised approach to arabic text summarization using adaboost.  In _New Contributions in Information Systems and Technologies_ , pages 227–236, Cham. Springer International Publishing.
  * Chiang et al. (2020) Cheng-Han Chiang, Sung-Feng Huang, and Hung-yi Lee. 2020.  Pretrained language model embryology: The birth of ALBERT.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 6813–6828, Online. Association for Computational Linguistics.
  * Chopra et al. (2016) Sumit Chopra, Michael Auli, and Alexander M. Rush. 2016.  Abstractive sentence summarization with attentive recurrent neural networks.  In _Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 93–98, San Diego, California. Association for Computational Linguistics.
  * Chouigui et al. (2021) Amina Chouigui, Oussama Ben Khiroun, and Bilel Elayeb. 2021.  An arabic multi-source news corpus: Experimenting on single-document extractive summarization.  _Arabian Journal for Science and Engineering_ , 46(4):3925–3938.
  * Devlin et al. (2019) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019.  BERT: Pre-training of deep bidirectional transformers for language understanding.  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_ , pages 4171–4186, Minneapolis, Minnesota. Association for Computational Linguistics.
  * Dou et al. (2021) Zi-Yi Dou, Pengfei Liu, Hiroaki Hayashi, Zhengbao Jiang, and Graham Neubig. 2021\.  GSum: A general framework for guided neural abstractive summarization.  In _Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 4830–4842, Online. Association for Computational Linguistics.
  * Douzidia and Lapalme (2004) Fouad Douzidia and Guy Lapalme. 2004.  Lakhas, an arabic summarization system.  In _Proceedings of DUC’04_ , pages 128–135, Boston. NIST, NIST.
  * El-Haj et al. (2010) M. El-Haj, Udo Kruschwitz, and C. Fox. 2010.  Using mechanical turk to create a corpus of arabic summaries.  In _Proceedings of the 7th International Conference on Language Resources and Evaluation : Workshops & Tutorials May 17-18, May 22-23, Main Conference May 19-21, Valletta_. ELRA, Paris.
  * El-Haj and Koulali (2013) Mahmoud El-Haj and Rim Koulali. 2013.  Kalimat a multipurpose arabic corpus.  pages 22–25.  Second Workshop on Arabic Corpus Linguistics (WACL-2).
  * El-Haj et al. (2011) Mahmoud El-Haj, Udo Kruschwitz, and Chris Fox. 2011.  Exploring clustering for multi-document arabic summarisation.  In _Information Retrieval Technology - 7th Asia Information Retrieval Societies Conference, AIRS 2011, Dubai, United Arab Emirates, December 18-20, 2011. Proceedings_ , volume 7097 of _Lecture Notes in Computer Science_ , pages 550–561. Springer.
  * El-Haj et al. (2011) Mahmoud El-Haj, Udo Kruschwitz, and Chris Fox. 2011.  Multi-document arabic text summarisation.  _2011 3rd Computer Science and Electronic Engineering Conference (CEEC)_ , pages 40–44.
  * El-Shishtawy and El-Ghannam (2012) Tarek El-Shishtawy and Fatma El-Ghannam. 2012.  Keyphrase based arabic summarizer (kpas).  In _2012 8th International Conference on Informatics and Systems (INFOS)_ , pages NLP–7–NLP–14.
  * Elmadani et al. (2020) Khalid N. Elmadani, Mukhtar Elgezouli, and Anas Showk. 2020.  BERT fine-tuning for arabic text summarization.  _CoRR_ , abs/2004.14135.
  * Haboush et al. (2012) Ahmad Haboush, Ahmed Momani, Maryam Al-Zoubi, and Motassem Al-Tarazi. 2012.  Arabic text summerization model using clustering techniques.  _World Comput Sci Inf Technol J_ , 2.
  * Hasan et al. (2021) Tahmid Hasan, Abhik Bhattacharjee, Md. Saiful Islam, Kazi Mubasshir, Yuan-Fang Li, Yong-Bin Kang, M. Sohel Rahman, and Rifat Shahriyar. 2021.  XL-sum: Large-scale multilingual abstractive summarization for 44 languages.  In _Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021_ , pages 4693–4703, Online. Association for Computational Linguistics.
  * Helmy et al. (2018) Muhammad Helmy, R. M. Vigneshram, Giuseppe Serra, and Carlo Tasso. 2018.  Applying deep learning for arabic keyphrase extraction.  In _Fourth International Conference On Arabic Computational Linguistics, ACLING 2018, November 17-19, 2018, Dubai, United Arab Emirates_ , volume 142 of _Procedia Computer Science_ , pages 254–261. Elsevier.
  * Hu et al. (2015) Baotian Hu, Qingcai Chen, and Fangze Zhu. 2015.  LCSTS: A large scale Chinese short text summarization dataset.  In _Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing_ , pages 1967–1972, Lisbon, Portugal. Association for Computational Linguistics.
  * Inoue et al. (2021) Go Inoue, Bashar Alhafni, Nurpeiis Baimukan, Houda Bouamor, and Nizar Habash. 2021\.  The interplay of variant, size, and task type in Arabic pre-trained language models.  In _Proceedings of the Sixth Arabic Natural Language Processing Workshop_ , pages 92–104, Kyiv, Ukraine (Virtual). Association for Computational Linguistics.
  * Kahla et al. (2021) Mram Kahla, Zijian Győző Yang, and Attila Novák. 2021.  Cross-lingual fine-tuning for abstractive Arabic text summarization.  In _Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP 2021)_ , pages 655–663, Held Online. INCOMA Ltd.
  * Kamal Eddine et al. (2021a) Moussa Kamal Eddine, Guokan Shang, Antoine J-P Tixier, and Michalis Vazirgiannis. 2021a.  Frugalscore: Learning cheaper, lighter and faster evaluation metricsfor automatic text generation.  _arXiv preprint arXiv:2110.08559_.
  * Kamal Eddine et al. (2021b) Moussa Kamal Eddine, Antoine Tixier, and Michalis Vazirgiannis. 2021b.  BARThez: a skilled pretrained French sequence-to-sequence model.  In _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing_ , pages 9369–9390, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
  * Khalil et al. (2022) Ahmed Mostafa Khalil, Y. M. Wazery, Marwa E. Saleh, Abdullah Alharbi, and Abdelmgeid A. Ali. 2022.  Abstractive arabic text summarization based on deep learning.  _Computational Intelligence and Neuroscience_ , 2022:1566890.
  * Kudo and Richardson (2018) Taku Kudo and John Richardson. 2018.  SentencePiece: A simple and language independent subword tokenizer and detokenizer for neural text processing.  In _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing: System Demonstrations_ , pages 66–71, Brussels, Belgium. Association for Computational Linguistics.
  * Lewis et al. (2020) Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. 2020.  BART: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 7871–7880, Online. Association for Computational Linguistics.
  * Lin (2004) Chin-Yew Lin. 2004.  ROUGE: A package for automatic evaluation of summaries.  In _Text Summarization Branches Out_ , pages 74–81, Barcelona, Spain. Association for Computational Linguistics.
  * Liu and Lapata (2019) Yang Liu and Mirella Lapata. 2019.  Text summarization with pretrained encoders.  In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_ , pages 3730–3740, Hong Kong, China. Association for Computational Linguistics.
  * Liu et al. (2020) Yinhan Liu, Jiatao Gu, Naman Goyal, Xian Li, Sergey Edunov, Marjan Ghazvininejad, Mike Lewis, and Luke Zettlemoyer. 2020.  Multilingual denoising pre-training for neural machine translation.  _Transactions of the Association for Computational Linguistics_ , 8:726–742.
  * Liu et al. (2019) Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019.  Roberta: A robustly optimized bert pretraining approach.  _arXiv preprint arXiv:1907.11692_.
  * Maynez et al. (2020) Joshua Maynez, Shashi Narayan, Bernd Bohnet, and Ryan McDonald. 2020.  On faithfulness and factuality in abstractive summarization.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 1906–1919, Online. Association for Computational Linguistics.
  * Nallapati et al. (2017) Ramesh Nallapati, Feifei Zhai, and Bowen Zhou. 2017.  Summarunner: A recurrent neural network based sequence model for extractive summarization of documents.  In _Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence_ , AAAI’17, page 3075–3081. AAAI Press.
  * Narayan et al. (2018a) Shashi Narayan, Shay B. Cohen, and Mirella Lapata. 2018a.  Don’t give me the details, just the summary! topic-aware convolutional neural networks for extreme summarization.  In _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing_ , pages 1797–1807, Brussels, Belgium. Association for Computational Linguistics.
  * Narayan et al. (2018b) Shashi Narayan, Shay B. Cohen, and Mirella Lapata. 2018b.  Ranking sentences for extractive summarization with reinforcement learning.  In _Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)_ , pages 1747–1759, New Orleans, Louisiana. Association for Computational Linguistics.
  * Ott et al. (2019) Myle Ott, Sergey Edunov, Alexei Baevski, Angela Fan, Sam Gross, Nathan Ng, David Grangier, and Michael Auli. 2019.  fairseq: A fast, extensible toolkit for sequence modeling.  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics (Demonstrations)_ , pages 48–53, Minneapolis, Minnesota. Association for Computational Linguistics.
  * Parker et al. (2011) Robert Parker, David Graff, Ke Chen, Junbo Kong, and Kazuaki Maeda. 2011.  Arabic gigaword fifth edition.  https://doi.org/10.35111/p02g-rw14.
  * Qaroush et al. (2021) Aziz Qaroush, Ibrahim Abu Farha, Wasel T. Ghanem, Mahdi Washaha, and Eman Maali. 2021.  An efficient single document arabic text summarization using a combination of statistical and semantic features.  _J. King Saud Univ. Comput. Inf. Sci._ , 33:677–692.
  * Qassem et al. (2019) Lamees Al Qassem, Di Wang, Hassan Barada, Ahmad Al-Rubaie, and Nawaf Almoosa. 2019\.  Automatic Arabic text summarization based on fuzzy logic.  In _Proceedings of the 3rd International Conference on Natural Language and Speech Processing_ , pages 42–48, Trento, Italy. Association for Computational Linguistics.
  * Rothe et al. (2020) Sascha Rothe, Shashi Narayan, and Aliaksei Severyn. 2020.  Leveraging pre-trained checkpoints for sequence generation tasks.  _Transactions of the Association for Computational Linguistics_ , 8:264–280.
  * Rush et al. (2015) Alexander M. Rush, Sumit Chopra, and Jason Weston. 2015.  A neural attention model for abstractive sentence summarization.  In _Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing_ , pages 379–389, Lisbon, Portugal. Association for Computational Linguistics.
  * See et al. (2017) Abigail See, Peter J. Liu, and Christopher D. Manning. 2017.  Get to the point: Summarization with pointer-generator networks.  In _Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 1073–1083, Vancouver, Canada. Association for Computational Linguistics.
  * Shao et al. (2021) Yunfan Shao, Zhichao Geng, Yitao Liu, Junqi Dai, Fei Yang, Li Zhe, Hujun Bao, and Xipeng Qiu. 2021.  Cpt: A pre-trained unbalanced transformer for both chinese language understanding and generation.
  * Suleiman and Awajan (2020) Dima Suleiman and Arafat Awajan. 2020.  Deep learning based abstractive arabic text summarization using two layers encoder and one layer decoder.  _Journal of Theoretical and Applied Information Technology_ , 98:3233.
  * (47) Webz.io.  Webz.io’s arabic news articles.  https://webz.io/free-datasets/arabic-news-articles/.
  * Xue et al. (2021) Linting Xue, Noah Constant, Adam Roberts, Mihir Kale, Rami Al-Rfou, Aditya Siddhant, Aditya Barua, and Colin Raffel. 2021.  mT5: A massively multilingual pre-trained text-to-text transformer.  In _Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 483–498, Online. Association for Computational Linguistics.
  * Zhang et al. (2020) Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger, and Yoav Artzi. 2020\.  Bertscore: Evaluating text generation with bert.  In _International Conference on Learning Representations_.
  * Zhou et al. (2018) Qingyu Zhou, Nan Yang, Furu Wei, Shaohan Huang, Ming Zhou, and Tiejun Zhao. 2018\.  Neural document summarization by jointly learning to score and select sentences.  In _Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 654–663, Melbourne, Australia. Association for Computational Linguistics.

##  Appendix A Example Appendix

Figure 1 presents some examples of the output of the various systems we
studied.

Figure 1: Three selected examples contrasting the output of the various
systems we studied. All examples are from the XL-Sum summaries test set. We
provide English translations to provide context for the general readers.