# Cellular automata in the light of COVID-19

Sourav Chowdhury Suparna Roychowdhury Indranath Chaudhuri email:
<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>Department of Physics, St. Xavier’s College (Autonomous)
30 Mother Teresa Sarani, Kolkata-700016, West Bengal, India

###### Abstract

Currently, the world has been facing the brunt of a pandemic due to a diseases
called COVID-19 for the last two years. To study the spread of such infectious
diseases it is important to not only understand their temporal evolution but
also the spatial evolution. In this work, the spread of this disease has been
studied with a cellular automata (CA) model to find the temporal and the
spatial behavior of it. Here, we have proposed a neighborhood criteria which
will help us to measure the social confinement at the time of the disease
spread. The two main parameters of our model are (i) disease transmission
probability ($q$) which helps us to measure the infectivity of a disease and
(ii) exponent ($n$) which helps us to measure the degree of the social
confinement. Here, we have studied various spatial growths of the disease by
simulating this CA model. Finally we have tried to fit our model with the
COVID-19 data of India for various waves and have attempted to match our model
predictions with regards to each wave to see how the different parameters vary
with respect to infectivity and restrictions in social interaction.

##  1 Introduction

Epidemics and pandemics have a long story throughout human history. Recently
human civilization has faced another pandemic named COVID-19. This pandemic
has affected many countries through multiple waves. Total of 504,451,689
people have been infected worldwide and 6,222,430 people have died due to
COVID-19 till 17 April 2022. In India 43,042,097 people have suffered and
521,781 have died as of 17/04/2022 due to this disease [1]. COVID-19 is caused
by the virus which is named SARS COV-2. Multiple variants of this virus, like
delta, omicron and many others makes it harder to control and predict its
behavior. Recently another variant of COVID-19 named XE has been found [2].
Mathematical modeling helps us to understand the behavior of disease spread
such that prevention and control strategies can be built. Also, mathematical
models can help us to find some inherent properties of the disease and nature
of its spread.

There are many different types of models that have been used in the past to
study various diseases. These models are mainly modified versions of the
Kermack McKendrick SIR model which is based on a system of coupled ordinary
differential equations [3]. Currently, the ODE-based models and statistical
models are widely used in literature to model the temporal behavior of the
spread of COVID-19 from different aspects. Most of these models have tried to
analyze the spread of this disease and tried to predict its future behavior
[4, 5, 6, 7]. There are models which have proposed various intervention and
vaccination strategies to prevent and control the spread of the disease [8, 9,
10, 11, 12]. Also, some authors have tried to predict different inherent
properties of this pandemic like herd immunity and its chaotic nature [13, 14,
15, 16, 17]. These temporal models can give us much valuable information,
however most of these models assume that a population is homogeneously mixed
and cannot describe any spatial behavior. To incorporate this spatial
behavior, deterministic and probabilistic Spatio-temporal models have been
used in recent studies. Cellular automata (CA) is one such kind of spatio-
temporal model.

Cellular automata (CA) has been used in many studies to model different
aspects of epidemics. It has been widely used to model the disease spread of
influenza and various vector-borne diseases like dengue [18, 19, 20, 21, 22,
23, 24, 25, 26]. A neighborhood condition is an important aspect in the CA.
The most used neighborhood conditions are (i) Neumann’s neighborhood
condition, (ii) Moore’s neighborhood condition, (iii) Extended neighborhood
condition, and (iv) Random interactions. Coupled with these neighborhood
conditions, various models like SEIR, SEIRS, SEIRD, and SEIRQD have been
studied with the help of CA to model the spatial growth of epidemics [19, 22,
27, 28, 29, 30, 31, 32]. Currently, CA has gained a lot of momentum in the
studies of COVID-19. Various advanced studies with Genetic algorithms and
network models have been for COVID-19 data. [33, 34, 35, 36, 37, 38]. Also
there are models where COVID-19 has studied from different aspects.

In this paper, we have tried to model COVID-19 using cellular automata (CA) to
find spatio-temporal behavior of it. We have also made some analysis to
understand its behavior in different waves of the disease. A cellular automata
(CA) model is represented in a square lattice and defined by some neighborhood
and boundary conditions, the details of which are given in following sections.

This paper is arranged as follows: Section 2 consists of a detailed discussion
of the model, neighborhood conditions, probability of infection, and the
algorithm of the model. The result of the simulation has been shown in section
3 and the data analysis is shown in section 4. Finally, Section 5 consists of
the conclusions of our model.

##  2 Mathematical Model

In this article, we have illustrated a Cellular automata (CA) model for
epidemics and assumed the SEIR model as the base model. SEIR model stands for
Susceptible-Exposed-Infectious-Removed. Here we have considered a $N\times N$
square lattice, where each cell of the lattice is assumed to be a person. Each
cell of the lattice can have the set of states,
$\mathcal{S}=\left\\{S,E,I,R\right\\}$ and these states are represented by the
values $\mathcal{V}=\left\\{0,1,2,3\right\\}$. The updation of a cell’s state
depends upon various conditions like, (i) the current state of the cell, (ii)
the amount of time spent in the current state, and (iii) the current states of
the neighbors. The main assumptions of our model are given below:

  * •

Every cell represents a person.

  * •

Only susceptible persons can interact with the other cells.

  * •

A removed person cannot be infected again.

  * •

For this CA model, we have assumed a periodic boundary condition. If a cell of
$i$th row and $j$th column of a $N\times N$ lattice is denoted by $(i,j)$
then,

$\begin{split}(N+1,j)\equiv(1,j)\qquad j=0,1,...N.\\\ (i,N+1)\equiv(i,1)\qquad
i=0,1,...N.\\\ \end{split}$ (1)

  * •

One susceptible person can interact with a single person in each time step.

###  2.1 Neighborhood condition

Nearest neighborhood condition is a widely used concept in the literature.
Here, it has been assumed that a particular cell can only interact with its
nearest neighborhood cells. Such two famous neighborhood conditions are: (i)
Neumann’s neighborhood condition and (ii) Moore’s neighborhood condition.

(a) Neumann’s neighborhood condition.

(b) Moore’s neighborhood condition.

Figure 1: Different neighborhood conditions. Blue cells represent the nearest
neighborhoods of $(i,j)$ cell.

Fig. 1 shows the two neighborhood conditions. Fig. 1(a) shows Neumann’s
neighborhood condition, where the nearest neighborhoods of any chosen $(i,j)$
cell are the first neighborhood cells with respect to the chosen cell.
Similarly, Fig. 1(b) shows Moore’s neighborhood condition. In this case, all
first and second neighborhoods are treated as the nearest neighborhoods of the
chosen $(i,j)$ cell.

In this work, we have assumed that a cell can interact with any other cells
depending on the probability of interaction ($p_{int}$) between them. Here we
have assumed that the probability of interaction ($p_{int}$) of a cell $(i,j)$
to any other cell varies inversely as a function of $d$ (distance between two
cells) in the form of a power law. Hence,

$p_{int}(d)\propto\frac{1}{d^{n}}$ (2)

where $n$ is the degree exponent and can have a value greater than zero. Here,
we have assumed that the distance between two cells is not just the
geometrical distance between two. It depends on the layer number ($l$). In
Fig. 2, we have shown how layers are defined. Also, it shows that a layer $l$
contains the $8l$ number of cells. If we choose a lattice of size $N\times N$
then the total number of layers in this lattice is $L=\frac{N-1}{2}$, when $N$
is odd.

Figure 2: Different layers of a lattice with respect to $(i,j)$ cell.

Hence, we can write,

$d\propto l.$ (3)

For mathematical simplicity, we can assume $d=l$. Hence, from Eq. 2 we can
write,

$p_{int}(d)=p_{int}(l)\propto\frac{1}{l^{n}}$ (4)

If there are $L$ number of layers then the above equation can be written as,

$p_{int}(l)=\frac{\frac{1}{l^{n}}}{\sum_{l=1}^{L}\frac{1}{l^{n}}}=\frac{1}{A_{n}l^{n}}$
(5)

where, $A_{n}=\sum_{l=1}^{L}\frac{1}{l^{n}}$. Hence, a person at the $(i,j)$
cell can interact with any other cell of layer $l$ with a probability
$p_{int}(l)$. Thus, average interaction distance ($\langle d\rangle$) can be
defined as,

$\langle
d\rangle=\sum_{l=1}^{L}lp_{int}(l)=\frac{1}{A_{n}}\sum_{l=1}^{L}l\frac{1}{l^{n}}=\frac{1}{A_{n}}\sum_{l=1}^{L}\frac{1}{l^{n-1}}$
(6)

(a)

(b)

Figure 3: Plots of average interaction distance ($\langle d\rangle$) with $n$
and the total number of layers ($L$). (a) Plot of average interaction distance
($\langle d\rangle$) with $n$ by considering the total number of layers ($L$)
= 50. (b) Plot of average interaction distance ($\langle d\rangle$) with the
total number of layers ($L$) for $n$=3,4, and 5.

Fig. 3(a) shows the variation of average interaction distance ($\langle
d\rangle$) with the degree exponent ($n$). From this figure, it can be found
that the average interaction distance ($\langle d\rangle$) is quickly
decreases and saturates to unity as $n$ increases. Also, from Fig. 3(b) it can
be seen that the average interaction distance ($\langle d\rangle$) is
approximately $\sim 1$ for exponents $n>3$. Hence for $n\gg 3$ the
neighborhood condition is approximately similar to Moore’s neighborhood
condition as discussed earlier and does not give any significantly different
results.

###  2.2 Probability of infection ($Q_{I}$)

Let, $q$ denote the disease transmission probability when a susceptible and an
infectious person interact. The probability that a susceptible person will
interact with any person at the layer $l$ is $p_{int}(l)$. If the probability
of finding an infectious person in that layer is $p_{I}(l)$ then the
probability that the susceptible person will be infected is
$qp_{int}(l)p_{I}(l)$. Hence, the probability of infection ($Q_{I}$) of a
susceptible person is,

$Q_{I}=q\sum_{l=1}^{L}p_{int}(l)p_{I}(l).$ (7)

As, $p_{int}(l)=\frac{1}{A_{n}l^{n}}$, from the above equation (Eq. 7) we can
write,

$Q_{I}=\frac{q}{A_{n}}\sum_{l=1}^{L}\frac{p_{I}(l)}{l^{n}}$ (8)

From the above equation (Eq. 8) we can say that the terms with small layer
number ($l$) dominate the summation. Hence, the infection possibility of a
susceptible person mainly depends on the infection situation around the
person.

Thus in our model, instead of choosing a traditional neighborhood condition
where the degree of the interaction is fixed, we have assumed a model where we
can vary the degree of the social confinement by changing $n$ (degree
exponent). Also we have calculated the probability of infection ($Q_{I}$) for
this modified model.

##  3 Algorithm and Simulations

###  3.1 Algorithm

Here, we have discussed the state updation algorithm of the SEIR model. As we
have mentioned earlier, every cell’s state is denoted by a value ($v$) which
is present in this set $\left\\{0,1,2,3\right\\}$. The algorithm is given
below:

  * •

Let, at time $t$ there is a susceptible person at $(i,j)$ cell. So, the value
of the $(i,j)$ cell is $v(i,j,t)=0$ and the probability of infection is
$Q_{I}(i,j,t)$. To find the infection possibility of the susceptible person,
we will generate a uniform random number $u$ between 0 and 1.

If, $u\leq Q_{I}(i,j,t)$ then the susceptible person is exposed and at time
$t+1$ the state of the $(i,j)$ cell will be changed from $v=0$ to $v=1$.

else, at time $t+1$ the state of the $(i,j)$ cell will be unchanged.

  * •

An exposed person ($v=1$) will remain exposed for $\tau_{I}$ number of days.
After that, the person will be infectious and the state of the corresponding
cell will be changed from $v=1$ to $v=2$.

  * •

An infectious person will remain in this state for $\tau_{R}$ number of days.
After that, the person will be removed (recovered or dead) and the state of
the cell will be changed from $v=2$ to $v=3$.

###  3.2 Simulation

In this part, we have done simulation of our model with $n=1,2,3$. The values
of the parameters and initial conditions that are used in the simulations are
listed in the tables (Table 1 and Table 2) below:

Description of the parameters | Parameters | Values of the parameters
---|---|---
Lattice size | $N\times N$ | $101\times 101$
Disease transmission probability | $q$ | 0.3
Latency period of the disease | $\tau_{I}$ | $8~{}days$
Removal period | $\tau_{R}$ | $18~{}days$
Table 1: Table for the parameter values that are used in the simulations. States | Initial values
---|---
$S(0)$ | 10200
$I(0)$ | 1
$E(0)$ | 0
$R(0)$ | 0
Table 2: Table for the initial conditions of the simulations.  |  |
---|---|---
|  |
|  |
|  |
Figure 4: Plots of the temporal and spatial behavior of the disease spread for
$n=1$.

In Fig. 4, the first plot shows the temporal behavior of the exposed cases
($E(t)$) and the infectious cases ($I(t)$) of the epidemic. These temporal
plots are averaged on 50 simulation samples. The rest plots of Fig. 4 are CA
plots that represent the spatial evolution of disease spread. From these CA
plots, we can hardly detect any clustering of the infected cases. This happens
because the average interaction distance, $\langle d\rangle\approx 11.11$.
Thus a susceptible person can be infected by an infectious person who is far
away from the susceptible one.

|  |
---|---|---
|  |
|  |
|  |
Figure 5: Plots of the temporal and spatial behavior of the disease spread for
$n=2$.

In Fig. 5, the first plot again shows the temporal growth of the epidemic. It
can be seen from the CA plots that for $n=2$, clusters are formed. The reason
behind this is the short average interaction distance. For $n=2$, the average
interaction distance, $\langle d\rangle\approx 2.77$. Also from the temporal
plot, we can see that the infection spread time is increased than in the $n=1$
case. This is also because of the short average interaction distance. For a
short average interaction distance, only a few susceptible persons can
interact with the infectious person. So, if most of those susceptible persons
become infected then the infectious person cannot spread the disease further.
Whereas, for $n=1$ case, an infectious person can interact with many
susceptible persons. Hence, an infectious person can infect more people during
the infectious period.

|  |
---|---|---
|  |
|  |
|  |
Figure 6: Plots of the temporal and spatial behavior of the disease spread for
$n=3$.

Fig. 6 shows the evolution of the disease for $n=3$. Here we can also clearly
find the clusters. These clusters are more prominent than the $n=2$ case
because of the less average interaction distance ($\langle d\rangle$). Value
of the average interaction distance for $n=3$ is $\langle d\rangle\approx
1.35$. Here we can see that the disease takes a longer time to fall for $n=3$
than for $n=1$ and $n=2$. The reason behind this is the lower value of the
average interaction distance which is discussed earlier.

Hence, from the above discussions, we can conclude that the clustering
behavior of the disease spread depends on the average interaction distance
($\langle d\rangle$) as well as on degree exponent $n$. Also, the average
interaction distance ($\langle d\rangle$), gives an average estimation of the
number of susceptible persons who can interact with an infectious person which
is represented by $8\langle d\rangle$. So, for a large $\langle d\rangle$ (or
small $n$) an infectious person can spread the disease to distanced region.
Thus the infection period depends on the average interaction distance
($\langle d\rangle$) and also on $n$.

##  4 Comparison with data

In this section, we have tried to fit our model with current COVID-19 data.
Our model has four free parameters which are, (i) $q$ : disease transmission
probability, (ii) $n$ : degree exponent, (iii) $\tau_{I}$ : mean latency
period, (iv) $\tau_{R}$ : mean infectious period. We have optimized these free
parameters for different waves of the COVID-19 pandemic in India. Here, we
have considered each wave separately and normalized the active cases of each
wave with the total number of infected cases in the respective wave. The data
is taken from covid19india.org [39]. The date range of the different waves
that we have considered here are given below:

Waves | Start date | End date
---|---|---
First wave | 30-Jan-2020 | 16-Feb-2021
Second wave | 17-Feb-2021 | 31-Oct-2021
Table 3: Date ranges for different waves.

To fit the model with the data we have optimized the sum of squared errors
($SSE$)

$SSE=\sum_{k}\left(i_{k}^{d}-i_{k}\right)^{2}$ (9)

$i_{k}^{d}$ : fraction of the active cases from the data. $i_{k}$ : fraction
of the infectious cases from the model.

The results of the best fit parameter values are given below.

Waves | q | n | $\tau_{I}~{}\left(days\right)$ | $\tau_{R}~{}\left(days\right)$
---|---|---|---|---
First wave | 0.1950 | 1.9310 | 5 | 11
Second wave | 0.2406 | 1.3449 | 8 | 10
Table 4: Fitting parameter values for different waves.

We have optimized COVID-19 data with a $101\times 101$ lattice space. Fig 7(a)
and 7(b) shows the fitted model along with data for the first and the second
wave.

(a)

(b)

Figure 7: Model fitting of the COVID-19 data of India (a) Model fitting of the
first wave. (b) Model fitting of the second wave.

Here we can see that the model fits reasonably well with the data. Also from
the table 4, we see that the disease transmission probability ($q$) increases
and the degree exponent ($n$) decreases in the second wave as compared to the
first wave. Increment of $q$ represents that in the second wave of the disease
it was possibly more infectious and spread faster than in the first wave. Also
decrement of $n$ indicates that the interaction between infectious and the
susceptible population spread out to a larger distances in the second wave as
compared to the first wave. This is possibly because of the COVID protocols
which is relaxed much more in the initial phase of the second wave as compared
to the first wave.

##  5 Conclusions

In this section we have summarized the main features and results of our model.
The cellular automata (CA) is a very common tool to model a disease spread and
has been used extensively in literature for studying different systems. In
this paper, we have modeled the CA by proposing a new neighborhood criteria.
Usually in earlier studies, neighborhood condition is such that the
neighborhood of a lattice cell is always fixed. Whereas in our model, rather
than choosing a specific neighborhood condition, we assume that a lattice cell
can interact with any other cell at distance $d$ with a certain probability
which is called interaction probability ($p_{int}(d)$) (Eq.5). We have assumed
that the interaction probability ($p_{int}(d)$) is a function of the distance
($d$) and has a form of inverse power law with degree exponent $n$. Here,
exponent $n$ is a very important parameter as it enables us to tune the social
confinement of our model. With this newly defined neighborhood criteria, we
have calculated various relations like average interaction distance ($\langle
d\rangle$) and the probability of infection ($Q_{I}$) to understand and
represent our model properly.

From Fig. 3(a), we can see that the average interaction distance ($\langle
d\rangle$) deceases and saturation is reached to $\sim 1$ as $n$ increases.
So, for a higher $n$, a person can mainly interact with nearest neighbors.
However, for a smaller value of $n$, a person can interact with the distant
neighbors. Hence higher values of $n$ represents higher social confinement and
vice-versa. Also we want to mention that for exponents, $n>3$, the average
interaction distance, $\langle d\rangle\approx 1$. Thus the values $n\gg 3$ do
not give us any new results.

In the simulation section, we have studied the temporal and spatial behavior
of our model for different degree exponents ($n$). As $n$ increases, the
disease spread becomes slower and it is more clustered. This happens because
of the decrease in the average interaction distance ($\langle d\rangle$) or in
the other words increase in the social confinement with increasing values of
$n$. Thus the disease transmission probability ($q$) and the degree exponent
$n$ regulates the speed of the disease spread.

Also, we have compared our model with COVID-19 data of India for different
waves. We have first normalized the active cases of a wave with the total
number of infected cases in that wave. Then we have optimized the sum of
squared errors of the infectious cases (Eq. 9) to get the best fit result with
the data. Here all simulations are done on a $101\times 101$ lattice. We have
found that the disease transmission probability ($q$) increases in the second
wave than the first wave. This means that the disease is more infectious in
the second wave than the first wave. Also the degree exponent ($n$) decreases
in the second wave. This implies that the decrement in the COVID-19
restrictions (or decrement in the degree of social confinement) at the initial
time of the second wave played a significant role to the faster spread of the
disease. Our model fits the peak of the waves well, however fall in the data
at the end of both waves and plateauing at the end of the second wave has some
matches with our fitted model. This possibly indicates that our model needs to
be modified to incorporate these aspects which we will look at in our future
works.

Modeling spread of a disease is a very complicated process since several
factors have to be considered. Non-uniform distribution of the population and
economic situation of the regions are two major factors which affects the
disease spread. In future we want to look at these complex aspects by refining
this model to find the behavior of the disease spreading more accurately. We
would also like to study these possibilities in the context of COVID-19.

## Acknowledgment

The authors would like to thank the Department of Physics, St. Xavier’s
College, Kolkata for providing support during this work. One of the authors
(S. C.) acknowledges the financial support provided from the University Grant
Commission (UGC) of the Government of India, in the form of CSIR-UGC NET-JRF.
Finally, the authors would also like to express their gratitude to the
anonymous referee for his/her valuable comments and suggestions.

## References

  * [1] Live update of COVID-19 situation in different countries- Worldometers.  https://www.worldometers.info/coronavirus/.
  * [2] Tracking SARS-CoV-2 variants (WHO).  https://www.who.int/en/activities/tracking-SARS-CoV-2-variants/.
  * [3] William Ogilvy Kermack, A. G. McKendrick, and Gilbert Thomas Walker.  A contribution to the mathematical theory of epidemics.  Proceedings of the Royal Society of London. Series A, Containing Papers of a Mathematical and Physical Character, 115(772):700, 1927.  DOI: 10.1098/rspa.1927.0118.
  * [4] Supriya Mondal and Sabyasachi Ghosh.  Mapping first to second wave transition of covid19 indian data via sigmoid function and prediction of third wave.  medRxiv, 2021.  DOI: 10.1101/2021.07.11.21260325.
  * [5] Sandip Mandal, Nimalan Arinaminpathy, Balram Bhargava, and Samiran Panda.  Plausibility of a third wave of covid-19 in india: A mathematical modelling based analysis.  The Indian journal of medical research, 153(5):522, 2021.
  * [6] C Kavitha, A Gowrisankar, and Santo Banerjee.  The second and third waves in india: when will the pandemic be culminated?  The European Physical Journal Plus, 136(5):596, 2021.
  * [7] A Gowrisankar, TMC Priyanka, and Santo Banerjee.  Omicron: a mysterious variant of concern.  The European Physical Journal Plus, 137(1):100, 2022.
  * [8] Brody H. Foy, Brian Wahl, Kayur Mehta, Anita Shet, Gautam I. Menon, and Carl Britto.  Comparing covid-19 vaccine allocation strategies in india: A mathematical modelling study.  International Journal of Infectious Diseases, 103:431, 2021.  DOI: 10.1016/j.ijid.2020.12.075.
  * [9] Namrata Soni, Jyoti Bhola, Ashutosh Yadav, Ishita Srivastva, and Utcarsh Mathur.  A mathematical reflection of covid-19 and vaccination acceptance in india.  8:150, 2021.  DOI: 10.21276/apjhs.2021.8.3.27.
  * [10] Ting-Yu Lin, Sih-Han Liao, Chao-Chih Lai, Eugenio Paci, and Shao-Yuan Chuang.  Effectiveness of non-pharmaceutical interventions and vaccine for containing the spread of covid-19: Three illustrations before and after vaccination periods.  Journal of the Formosan Medical Association, 120:S46, 2021.  DOI: 10.1016/j.jfma.2021.05.015.
  * [11] Simon A Rella, Yuliya A Kulikova, Emmanouil T Dermitzakis, and Fyodor A Kondrashov.  Rates of sars-cov-2 transmission and vaccination impact the fate of vaccine-resistant strains.  Scientific Reports, 11(1):15729, 2021.  DOI: 10.1038/s41598-021-95025-3.
  * [12] Sourav Chowdhury, Suparna Roychowdhury, and Indranath Chaudhuri.  A robust prediction from a minimal model of covid-19–can we avoid the third wave?  arXiv preprint arXiv:2112.08794, 2021.
  * [13] Sourav Chowdhury, Suparna Roychowdhury, and Indranath Chaudhuri.  Universality and herd immunity threshold: Revisiting the sir model for covid-19.  International Journal of Modern Physics C, 32(10):2150128, 2021\.  DOI: 10.1142/S012918312150128X.
  * [14] A Gowrisankar, Lamberto Rondoni, and Santo Banerjee.  Can india develop herd immunity against covid-19?  The European Physical Journal Plus, 135(6):526, 2020.
  * [15] André F. Steklain, Ahmed Al-Ghamdi, and Euaggelos E. Zotos.  Using chaos indicators to determine vaccine influence on epidemic stabilization.  Phys. Rev. E, 103:032212, Mar 2021.  DOI: 10.1103/PhysRevE.103.032212.
  * [16] Jorge Duarte, Cristina Januário, Nuno Martins, Jesús Seoane, and Miguel AF Sanjuán.  Controlling infectious diseases: the decisive phase effect on a seasonal vaccination strategy.  arXiv preprint arXiv:2102.08284, 2021.
  * [17] D Easwaramoorthy, A Gowrisankar, A Manimaran, S Nandhini, Lamberto Rondoni, and Santo Banerjee.  An exploration of fractal-based prognostic model and comparative analysis for second wave of covid-19 diffusion.  Nonlinear Dynamics, 106(2):1375–1395, 2021.
  * [18] M. J Keeling and C. A Gilligan.  Bubonic plague: a metapopulation model of a zoonosis.  Proceedings of the Royal Society of London. Series B: Biological Sciences, 267(1458):2219–2230, 2000.  DOI: 10.1098/rspb.2000.1272.
  * [19] A. Holko, M. Medrek, Z. Pastuszak, and K. Phusavat.  Epidemiological modeling with a population density map-based cellular automata simulation system.  Expert Systems with Applications, 48:1–8, 2016.  DOI: 10.1016/j.eswa.2015.08.018.
  * [20] Khaled M Khalil, M Abdel-Aziz, Taymour T Nazmy, and Abdel-Badeeh M Salem.  An agent-based modeling for pandemic influenza in egypt.  In Handbook on Decision Making, volume 33, pages 205–218. Springer, 2012.  DOI: 10.1007/978-3-642-25755-1_11.
  * [21] Senthil Athithan, Vidya Prasad Shukla, and Sangappa Ramachandra Biradar.  Epidemic spread modeling with time variant infective population using pushdown cellular automata.  Journal of Computational Environmental Sciences, 2014:769064, 2014\.  DOI: 10.1155/2014/769064.
  * [22] Sheng Bin, Gengxin Sun, and Chih-Cheng Chen.  Spread of infectious disease modeling and analysis of different factors on spread of infectious disease based on cellular automata.  International Journal of Environmental Research and Public Health, 16(23), 2019.  DOI: 10.3390/ijerph16234683.
  * [23] Gerardo Ortigoza, Fred Brauer, and Iris Neri.  Modelling and simulating chikungunya spread with an unstructured triangular cellular automata.  Infectious Disease Modelling, 5:197–220, 2020.  DOI: 10.1016/j.idm.2019.12.005.
  * [24] Murali Krishna Enduri and Shivakumar Jolad.  Dynamics of dengue disease with human and vector mobility.  Spatial and Spatio-temporal Epidemiology, 25:57–66, 2018.  DOI: 10.1016/j.sste.2018.03.001.
  * [25] Puspa Eosina, Taufik Djatna, and Helda Khusun.  A cellular automata modeling for visualizing and predicting spreading patterns of dengue fever.  TELKOMNIKA, 14(1):228, 2016.  DOI: 10.12928/TELKOMNIKA.v14i1.2404.
  * [26] Emily Burkhead and Jane Hawkins.  A cellular automata model of ebola virus dynamics.  Physica A: Statistical Mechanics and its Applications, 438:424–435, 2015.  DOI: 10.1016/j.physa.2015.06.049.
  * [27] Armin R. Mikler, Sangeeta Venkatachalam, and Kaja Abbas.  Modeling infectious diseases using global stochastic cellular automata.  Journal of Biological Systems, 13(04):421–439, 2005.  DOI: 10.1142/S0218339005001604.
  * [28] Henrique Fabricio Gagliardi and Domingos Alves.  Small-world effect in epidemics using cellular automata.  Mathematical Population Studies, 17(2):79–90, 2010.  DOI: 10.1080/08898481003689486.
  * [29] Quan-Xing Liu, Zhen Jin, and Mao-Xing Liu.  Spatial organization and evolution period of the epidemic model using cellular automata.  Phys. Rev. E, 74:031110, 2006.  DOI: 10.1103/PhysRevE.74.031110.
  * [30] Liu Quan-Xing and Jin Zhen.  Cellular automata modelling of SEIRS.  Chinese Physics, 14(7):1370–1377, 2005.  DOI: 10.1088/1009-1963/14/7/018.
  * [31] Leonardo López, Germán Burguerner, and Leonardo Giovanini.  Addressing population heterogeneity and distribution in epidemics models using a cellular automata approach.  BMC research notes, 7(1):234, 2014.  DOI: 10.1186/1756-0500-7-234.
  * [32] Senthil Athithan, Vidya Prasad Shukla, and Sangappa Ramachandra Biradar.  Dynamic cellular automata based epidemic spread model for population in patches with movement.  Journal of Computational Environmental Sciences, 2014:518053, 2014\.  DOI: 10.1155/2014/518053.
  * [33] P.H.T. Schimit.  A model based on cellular automata to estimate the social isolation impact on covid-19 spreading in brazil.  Computer Methods and Programs in Biomedicine, 200:105832, 2021.  DOI: 10.1016/j.cmpb.2020.105832.
  * [34] P.K. Jithesh.  A model based on cellular automata for investigating the impact of lockdown, migration and vaccination on covid-19 dynamics.  Computer Methods and Programs in Biomedicine, 211:106402, 2021.  DOI: 10.1016/j.cmpb.2021.106402.
  * [35] Sayantari Ghosh and Saumik Bhattacharya.  A data-driven understanding of covid-19 dynamics using sequential genetic algorithm based probabilistic cellular automata.  Applied Soft Computing, 96:106692, 2020.  DOI: 10.1016/j.asoc.2020.106692.
  * [36] Larissa M. Fraga, Gina M. B. de Oliveira, and Luiz G. A. Martins.  Multistage evolutionary strategies for adjusting a cellular automata-based epidemiological model.  In 2021 IEEE Congress on Evolutionary Computation (CEC), pages 466–473, 2021.  DOI: 10.1109/CEC45853.2021.9504738.
  * [37] L. L. Lima and A. P. F. Atman.  Impact of mobility restriction in covid-19 superspreading events using agent-based model.  PLOS ONE, 16(3):1–17, 03 2021.  DOI: 10.1371/journal.pone.0248708.
  * [38] Anand Sahasranaman and Henrik Jeldtoft Jensen.  Spread of covid-19 in urban neighbourhoods and slums of the developing world.  Journal of The Royal Society Interface, 18(174):20200599, 2021.  DOI: 10.1098/rsif.2020.0599.
  * [39] COVID-19 data source.  https://www.covid19india.org/.


#  TimeDial: Temporal Commonsense Reasoning in Dialog

Lianhui Qin♢ Aditya Gupta♠ Shyam Upadhyay♠
Luheng He♠ Yejin Choi♢ Manaal Faruqui♠
♠Google Assistant
♢Paul G. Allen School of Computer Science & Engineering, University of
Washington
{gaditya, shyamupa, luheng<EMAIL_ADDRESS>
{lianhuiq<EMAIL_ADDRESS>
Work done during an internship at Google.

###### Abstract

Everyday conversations require understanding everyday events, which in turn,
requires understanding temporal commonsense concepts interwoven with those
events. Despite recent progress with massive pre-trained language models (LMs)
such as T5 and GPT-3, their capability of temporal reasoning in dialogs
remains largely under-explored. In this paper, we present the first study to
investigate pre-trained LMs for their temporal reasoning capabilities in
dialogs by introducing a new task and a crowd-sourced English challenge set,
TimeDial. We formulate TimeDial as a multiple choice cloze task with over 1.1K
carefully curated dialogs. Empirical results demonstrate that even the best
performing models struggle on this task compared to humans, with $23$ absolute
points of gap in accuracy. Furthermore, our analysis reveals that the models
fail to reason about dialog context correctly; instead, they rely on shallow
cues based on existing temporal patterns in context, motivating future
research for modeling temporal concepts in text and robust contextual
reasoning about them. The dataset is publicly available at:
https://github.com/google-research-datasets/timedial.

##  1 Introduction

A: May we see the wine list please.
---
B: Sure. Our special wine today is a 1989 Chardonnay.
A: I’d like a bottle please.
B: I’ll need to see your ID please.
A: Here you go.
B: Sorry about the inconvenience, you look so young. I had to make sure you
are over  .
a) 21 years old ✓ b) 30 years old ✗
c) 4 years old ✗ d) 18 years old ✓
A: Good morning! May I help you?
B: Yes. My wife and I are interested in renting a house for the summer.
A: Very well. How long do you want the house? All summer?
B: No, not all summer. Just for six weeks.
A: I am afraid I can only rent it for two months.
B: My holiday is only  , but I think my brother and his family would take it
for the other two weeks.
a) six decades ✗ b) 45 days ✓
c) six weeks ✓ d) two months ✗
Table 1: Examples from our TimeDial challenge set, demonstrating the need for
commonsense knowledge and arithmetic reasoning over the context to infer the
correct answers. Key contextual information for reasoning success is
highlighted.

Humans can effortlessly reason about temporal concepts of everyday events such
as their duration, frequency, or relative ordering Allen (1984); Radvansky and
Zacks (2014) based on rich commonsense knowledge about how the world works,
especially in relation to time. However, reasoning about such concepts has
been challenging for machines Kahn and Gorry (1977); Kozareva and Hovy (2011)
since it requires both understanding the local temporal expressions and
reasoning about their global contexts such as their relative ordering and
relations UzZaman et al. (2013); Ning et al. (2018b); Pustejovsky (2017). The
problem becomes even more challenging in dialogs, where explicit and implicit
inter-dependencies among temporal concepts can appear across conversation
turns.

For instance, for the first dialog in Table 1, one must understand the
context, i.e., selling wine, and use world knowledge of minimum legal drinking
age in order to reason about correct answers to fill in the blank. Similarly,
in the second conversation, commonsense about the durations _summer_ , _month_
, _week_ , _day_ and their relations, plus numerical reasoning, are necessary
to make the inference.

Although previous works have studied temporal reasoning in natural language,
they have either focused on specific time-related concepts in isolation, such
as temporal ordering and relation extraction Leeuwenberg and Moens (2018);
Ning et al. (2018a), and/or dealt with limited context, such as single-
sentence-based question answering Zhou et al. (2019) and natural language
inference Vashishtha et al. (2020); Mostafazadeh et al. (2016).

In this work, we make the first systematic study of temporal commonsense
reasoning in a multi-turn dialog setting. The task involves complex reasoning
that requires operations like comparison and arithmetic reasoning over
temporal expressions and the need for commonsense and world knowledge.

We design a new task for dialog-based temporal reasoning and present a new
challenge set in English, called TimeDial, to evaluate language understanding
models on the task. We formulate the problem as a crowd-sourced cloze task
with multiple choices based on dialogs in the DailyDialog dataset Li et al.
(2017). Given a dialog with one temporal span masked out, the model is asked
to find _all_ correct answers from a list of four options to fill in the blank
(Table 1).

The challenge set requires the models to demonstrate understanding of the
context and use temporal commonsense to make right choices. Our final
challenge set consists of $1.1$K carefully curated dialog instances.

We then study the performance of several state-of-the-art pre-trained language
models on TimeDial along several dimensions including modeling paradigms
(classification, mask filling, and generation), the scope of dialog contexts,
in-domain vs. out-of-domain training, dependence on shallow text matching for
reasoning, and the types of reasoning required. Our experiments demonstrate
that off-the-shelf, pre-trained language models cannot effectively reason
about temporal aspects in a dialog, even with domain-specific finetuning. Our
findings indicate that large-scale pre-trained models even after fine-tuning
may not be sufficient for robust temporal reasoning in dialogs, and motivate
future research toward modeling temporal concepts over diverse everyday
events, and contextual reasoning about them.

##  2 Task: Temporal Reasoning in Dialog

We formulate the dialog-based temporal commonsense reasoning problem as a
cloze task Taylor (1953). Formally, given a multi-turn dialog context of $n$
conversational turns between two speakers A and B, where a temporal words span
within the context is masked out, the task is to predict the suitable temporal
expression(s) for the masked-out span from a list of options. That is, we want
the conversation model to select all the correct answers from the options
based on the dialog context. Following similar cloze-style challenge datasets,
we use accuracy as the evaluation metric Mostafazadeh et al. (2016); Onishi et
al. (2016); Mihaylov and Frank (2018).

Having a non-trivial set of options is crucial to build a challenge set and to
avoid accidental spurious biases Geirhos et al. (2020); Gururangan et al.
(2018); Le Bras et al. (2020). We ensure this via the following filtering
process. (1) For each masked span, there is more than one correct answer in
the options. This makes the task more challenging for models since more
comprehensive understanding of the context is required to recognize all the
correct choices. In our dataset (§3) we guarantee two incorrect answers for
each masked span. (2) Some incorrect options are selected to be spuriously
correlated with the dialog context. For example, we include temporal spans in
the dialog context as negative options, which will challenge models that rely
primarily only on shallow pattern matching without correct temporal reasoning.
We present more information in §3 about how the negative options were created
by human annotators.

Category |  Dialog |  Options
---|---|---
World Knowledge (5%) |  A: May we see the wine list ? B: Sure . Our special wine today is a 1989 Chardonnay . A: That sounds pretty good! How much is it ? B: It’s $\$4.25$ cents by the glass . The whole bottle is $\$22.25$ . A: I’d like a bottle please . B: I’ll need to see your ID please . A: Here you go . B: Sorry about the inconvenience, I had make sure you are over  . |  ✓ 21 years old ✗ 30 years old ✗ 4 years old ✓ 18 years old
Comparison (24%) |  A: Yes , sir. May I help you? B: Please I’d like a ticket to New York. A: For today? B: No, early Saturday morning. A: We have a flight that we’ll put you there at  . Is that ok? B: Nothing earlier? I prefer flight at 9 thirty. A: I’m afraid not , unless you want a night flight. B: No, exactly not. |  ✓ ten AM ✗ 9:30 PM ✓ eleven AM ✗ four AM
Arithmetic (5%) |  A: How long do you want the house ? All summer ? B: No , just for six weeks. A: I’m afraid I can only rent it for two months. B: My holiday is only  , but I think my brother and his family would take it for the other two weeks. |  ✗ six decades ✓ 45 days ✓ six weeks ✗ two months
General Commonsense (60%) |  A: Do you get up early every morning ? B: About 6 in the morning. I like to walk to the office. A: Good habit. How long does it take ? B: . Do you live alone ? A: No , my little sister lives with me … |  ✓ 20 minutes ✗ 10 seconds ✓ 15 minutes ✗ 20 hours
Others (6%) |  A: How long does a facial service take? B: We have half-hour and one-hour treatments. A: What’s the regular price? B: Well , the half-hour facial costs $\$50$ and the one-hour costs $\$80$. A: Good , I will take  facial. B: That’s fine , madam. |  ✓ the one hour ✗ the 20 hour ✗ the 80 second ✓ the half hour
Table 2:  Example dialogs and answer options from the TimeDial dataset,
categorized by the nature of reasoning required to correctly answer them,
along with the percentage of each reasoning category in the set of 100 sampled
examples. The relevant key information in the dialog context is highlighted.

##  3 Dataset: TimeDial

The TimeDial dataset is derived from DailyDialog data Li et al. (2017), which
is a multi-turn dialog corpus containing over $13$K English dialogs. Dialogs
in this dataset consist of turn-taking between two people on topics over $10$
broad categories, ranging from daily lives to financial topics.

###  3.1 Data Collection

Our data collection process involves two steps: (1) identifying dialogs that
are rich in temporal expressions, and (2) asking human annotators to provide
correct and incorrect options for cloze instances derived from these dialogs.
We now describe these steps in detail.

##### Temporal expression identification.

Here, we select dialogs that are rich with temporal information, in order to
focus on complex temporal reasoning that arises in natural dialogs. Temporal
expressions are automatically identified with SUTime Chang and Manning (2012),
an off-the-shelf temporal expression
detector.111https://nlp.stanford.edu/software/sutime.shtml We keep only the
dialogs with more than 3 temporal expressions and at least one expression that
contains numerals like “two weeks” (as opposed to non-numeric spans, like
“summer”, “right now”, and “later”). In our initial experiment, we observe
that language models can often correctly predict these non-numerical temporal
phrases.

We note that temporal expressions containing numerals serve as more
challenging sets of options than non-numerical ones. This filtering step
results in 1,127 unique dialogs for further processing.

##### Human annotated options.

Next, we make spans in the dialogs. For a dialog, we mask out each temporal
expression that contains numerals, each resulting in a cloze question that is
then sent for human annotation.

This resulted in 1,526 instances for annotation. For each masked span in each
dialog, we obtain human annotation to derive a fixed set of correct and
incorrect options given the context. Concretely, given a masked dialog and a
seed correct answer (i.e., the original text) for the masked span, the
annotators222who are English linguists. were asked to (1) come up with an
alternative correct answer that makes sense in the dialog adhering to
commonsense, and (2) formulate two incorrect answers that have no possibility
of making sense in the dialog context. We highlight all time expressions in
the context to make it easier for annotators to select reasonable time
expressions.

To ensure that the annotated incorrect options are not too trivially
distinguishable by the models (as discussed in §2), we define three rules for
the annotators to follow.

  * •

Rule 1: Phrase Matching. The rater should first try to pick another temporal
span from the dialog context that makes syntactic/semantic sense (e.g., when
the span is of the appropriate type, such as duration, for the masked span)
but is still incorrect according to commonsense.

  * •

Rule 2: Numeral Matching. If Rule 1 does not apply, raters should follow a
relaxed version of Rule 1, whereby the incorrect option should contain any
numeral occurring in the dialog context.

  * •

Rule 3: Open-ended. If neither of the above rules is applicable, then raters
can come up with an incorrect option using their own judgment. The two
incorrect options are required to differ from each other as much as possible.

Rules-1&2 are designed to confuse models that rely on shallow pattern
matching. Finally, to ensure the quality of the human-annotated options, we
perform a subsequent round of human validation on the gathered data. The
validators identify and fix issues such as duplicate options, unreasonable or
obscure annotations w.r.t natural usage, or ungrammatical annotations that do
not fit in the context.

###  3.2 Properties of TimeDial

# Dialog instances | $1,104$
---|---
# Temporal Expressions | $1,985$
# Avg. Turns Per Dialog | $11.7$
# Avg. Words Per Turn | $16.5$
# Avg. Time Spans Per Dialog | $3.0$
_Incorrect Options_
% Phrase Matching |  $16.3$ %
% Numeral Matching |  $49.6$ %
% Open-ended |  $45.4$ %
Table 3: Statistics of our TimeDial challenge set.

Table 3 shows statistics of TimeDial. The dataset contains over 1.1K test
instances. Each dialog contains 11.7 turns and 3 temporal expressions on
average, presenting richer and more complex context compared to the recent
single-sentence-based temporal question answering benchmarks (e.g., Zhou et
al., 2019; Vashishtha et al., 2020). As above, each test instance contains two
correct answers and two incorrect ones.333We also collected 342 extra
instances for which the annotators deem there is only one unique correct
answer for the context. Thus, each of those instances contains one correct
option and two incorrect ones. We release those instances along with the
dataset, though we did not include them in empirical study in this paper. Over
half of the incorrect options are annotated based on phrase and numeral
matching from context, which pose a significant challenge for models relying
on shallow text matching, as we show in our experimental analysis (§5).

Answering different instances in the dataset requires different types of core
reasoning abilities, such as comparison, arithmetic inference, or reasoning
based on world knowledge or general commonsense. To facilitate fine-grained
analysis, we also annotate the reasoning categories for a randomly sampled set
of 100 dialogs. Though each instance can involve multiple reasoning types, we
associate it with one predefined category label that indicates the primary
type of reasoning it requires. Table 2 shows the category distribution and
examples in each of the category. We observe that the dataset requires general
commonsense for $60\%$ of the dialogs, making it the most common reasoning
type.

##  4 Modeling

Figure 1:  We study three modeling paradigms for the task, based on BERT and
T5, including (1) Classification, (2) Mask Filling, and (3) Generation (§4.1).
The models are finetuned with various training data, as discussed in §4.3.

We consider a broad set of methods and evaluate their performance on our
challenge TimeDial dataset. These methods vary in terms of the modeling
paradigms, the scope of the dialog contexts, and training settings. In
particular, they encompass the major ways pre-trained LMs are currently used
in downstream tasks (§4.1) which often outperform earlier specialized non-
pretrained models. We also consider different lengths of context used in
reasoning, varying by their vicinity to the masked span (§4.2). Finally, we
study different training settings, including zero-shot, in-domain, and out-of-
domain training (§4.3).

###  4.1 Modeling Paradigms

We experiment across three major modeling paradigms: (i) Binary
Classification, (ii) Mask Filling, and (iii) Generation. Figure 1 shows the
different architectures. For each test instance, the model takes as input a
pair of (masked dialog context, candidate), and outputs a score measuring how
likely the candidate being a correct answer. Based on the prediction scores of
all options, the model then chooses the top two positive candidates as the
predicted answer for the instance. Each paradigm of models is finetuned using
training data from different domains, as discussed in §4.3.

####  4.1.1 Binary Classification

In this setting, we formulate the task as a binary classification problem,
i.e., we use a classifier to measure the probability of the candidate in the
(masked dialog context, candidate) pair being a correct answer. Any powerful
LM — e.g., BERT Devlin et al. (2019), ALBERT Lan et al. (2019), RoBERTa Liu et
al. (2019), etc. can be used to build the classifier.

This method’s key challenge is the lack of annotated training data for direct
supervision. We generate weak supervision training data as follows. In an
unlabeled corpus, we use the SUTime tool to annotate temporal spans. We mask
each temporal span in this corpus and use the masked text as one positive
example for binary classification. To generate negative example, we randomly
sample another temporal span from the dialog context and use it as a negative
example for the masked temporal span. The resulting data is noisy because the
randomly sampled temporal span can also logically fit in the masked span in
the given context; however, we assume the likelihood of that happening is low.
We leave drawing harder negative instances using heuristics to future work.

####  4.1.2 Mask Filling

We also use the mask filling approach of BERT-like mask language models
(MLMs). For each dialog context and a candidate temporal span of $m$ tokens,
we replace the blank in the dialog context with $m$ masked tokens. We then
evaluate the likelihood of predicting the temporal span tokens for those
masked positions, and make average across the positions. A key advantage of
this method is that we can directly apply a BERT model in the zero-shot manner
since the model was pre-trained in the same way, as for accommodating for
[MASK] fillings. Additionally, we also finetune BERT’s MLM for learning task
specific properties.

####  4.1.3 Generation

The third method is a fully generative approach using the _text-to-text_
paradigm of T5 Raffel et al. (2020). Given a masked dialog context, the model
is trained to generate the masked text in an encoder-decoder framework. As a
result, evaluating the likelihood of generating the given temporal span
(normalized with the length of the span) is used as the probability of it
being correct. Similar to mask filling, we use T5 either in a zero-shot manner
or with additional fine-tuning.

###  4.2 Dialog Context

We aim to study the influence of context on a model’s temporal reasoning in
dialog by incorporating varying scopes of dialog context based on their
vicinity to the target span. Since the dialogs in TimeDial are rich in
temporal concepts, we want to evaluate LMs’ dependence on shallow text
matching vs. the ability to accurately understand the causal relations between
those concepts (see Table 6). We use the following three settings:

  * •

Full context, where the model is presented with the complete available dialog
to reason on. Due to our design of challenging negatives, the full context can
often confuse models that rely on shallow cues.

  * •

Local context, where we provide only with the utterances that immediately
precede and follow the target utterance.

  * •

Target context, where the context is restricted to only the particular
utterance that contains the masked span.

_Mask Filling_ and _Generation_
---
| # Train | # Dev |
In-domain (Daily) | 14.5K | 2.4K |
Out-domain (Meena) | 1.26M | 23K |
_Classification_
| # Train | # Dev | # Spans
In-domain (Daily) | 58.0K | 9.6K | 2,153
Out-domain (Meena) | 5.04M | 92K | 38,750
Table 4:  Number of training and development instances for different settings.
An instance is derived by masking one temporal span of a dialog. For
classification, we draw 3 negative samples per positive sample. “# Spans” is
the size of temporal span pool from which negative samples are drawn for weak
supervision.

###  4.3 Training Details

For all models, we consider two common training settings, e.g., in-domain
data, which is typically small, and out-of-domain training where a large
amount of data is available. Table 4 shows training data statistics. For mask-
filling and generation, we also evaluate in a zero-shot setup with no
finetuning.

##### In-domain training.

Our challenge TimeDial test set is derived from contextually rich dialogs from
the DailyDialog dataset, based on the number of temporal spans. However, this
still leaves remaining data with less than 3 temporal spans or with no numeric
span. By masking each temporal span in each dialog, we obtain 14.5K training
instances to use in our domain specific fine-tuning.

##### Out-of-domain training.

In this setting, we consider a much larger corpus from a general domain.
Specifically, we use the large scale training set based on the Meena dataset
Adiwardana et al. (2020), which is mined and filtered from public domain
social media conversations over 341GB of text (40B words).444We acquired a
trimmed down version of the Meena dataset by contacting the authors. Compared
to the above in-domain data from DailyDialog which were manually written by
human annotators in a clean and consistent way, the dialogs in the Meena
corpus tend to be noisy, casual, and usually short. Like our DailyDialog
processing, we identify all temporal expressions for dialogs in Meena using
SUTime.

##  5 Experiments and Analyses

Size-Train |  $2$-best Acc (%)
---|---
_Classification (BERT)_
Base-Out |  $43.1$
Base-In |  $51.1$
Large-Out |  $48.7$
Large-In |  $53.2$
_Mask Filling (BERT)_
Base-Zero |  $44.8$
Base-Out |  $47.4$
Base-In |  $67.4$
Large-Zero |  $47.7$
Large-Out |  $54.8$
Large-In |  $70.0$
_Generation (T5)_
Base-Zero |  $39.8$
Base-Out |  $50.6$
Base-In |  $59.2$
Large-Zero |  $39.1$
Large-Out |  $61.9$
Large-In |  $\mathbf{74.8}$
Human |  $\mathbf{97.8}$
Table 5:  Model and human performance on TimeDial. Base and Large denote the
size of the pre-trained BERT and T5; Zero, In, and Out denote that the model
is zero-shot (with no finetuning), fintuned using the in-domain DailyDialog
data, or finetuned using the out-of-domain Meena data, respectively. THe full
dialog context is used for all models.

Using the proposed TimeDial challenge set, we next conduct extensive
experiments and analyses on the different model variants and context settings.
We use either 4x4 or 8x8 Cloud TPUs V3 pod
slices555https://cloud.google.com/tpu for fine-tuning and one V100 GPU for
inference. We provide more details of the experiment configurations in the
appendix.

##### Evaluation.

Since each example of TimeDial contains two correct answers, we report the
metric $2$-best accuracy, which measures whether both of the model’s top-
ranked answers are correct. In other words, if the model erroneously ranks an
incorrect answer over a correct one, we consider it to be an error case. Note
that we use the ranking-based metric as opposed to classification-based ones
(for example, by asking the model to classify whether each individual
candidate answer is correct or not (e.g., Zhou et al., 2019)) and because it
presents a stricter measure that penalizes any incorrect answers being ranked
over correct answers, and the ranking metric is not influenced by specific
choices of the threshold hyperparameter that cuts off positive and negative
predictions.

###  5.1 Model Performance

Dialog Context |  Options |  Gold |  CLS |  MF |  GEN
---|---|---|---|---|---
A: What’s the date today? B: Today is September 28th, 2007. A: I have a meeting this afternoon. B: When will it begin? A: It will begin at three o’clock. What’s the time now? B: It is  . A: I have to go now. I don’t want to be late. B: Don’t worry, time is enough. |  half past one quarter to two half past three half past nine |  ✓ ✓ ✗ ✗ |  ✓ ✗ ✓ ✗ |  ✗ ✗ ✓ ✓ |  ✓ ✗ ✓ ✗
A: Doctor, I feel much better now. Will I be able to go home some time this week? B: That’s good to hear. You’ve had an ideal recovery from your operation. We’re going to send you home tomorrow. A: Do you think I can get back to work very soon? B:Don’t be in such a hurry. I’m confident that you’ll be completely recovered in  . A: Is there anything I should do? B: You’d better have a good rest for a week. |  4 to 6 weeks 5 to 7 weeks a week a day |  ✓ ✓ ✗ ✗ |  ✗ ✓ ✓ ✗ |  ✗ ✗ ✓ ✓ |  ✓ ✗ ✓ ✗
Table 6: Example prediction errors made by different models for cases with
challenging options, based on the phrase and numeral matching rules (§3). Gold
denotes the true labels. The model predictions show that the models get
confused by learning shallow text matching in terms of pre-existing temporal
concepts (marked by bold faced text) in the context.

Table 5 shows model results and human performance. Human performance achieves
a near-perfect level ($97.80$, with Cohen’s kappa score of 0.86 showing almost
perfect inter-rater agreement Landis and Koch (1977)).

##### Overall.

The generation model based on T5-Large and finetuned on the in-domain
DailyDialog data achieves the best performance. However, its $2$-best accuracy
($74.8$) lagged far behind the human performance, demonstrating the difficulty
of the TimeDial challenge set.

##### Zero-shot vs. out-of-domain vs. in-domain.

When comparing the different training data setup, we observe that models with
in-domain training using the DailyDialog data (e.g., Large-In) consistently
outperforms those trained on the large out-of-domain Meena dataset (e.g.,
Large-Out). Both setups outperform the zero-shot models (without any fine-
tuning) (e.g., Large-Zero). The results show that the large LMs still highly
depend on in-domain or at least dialog data to grasp and enhance their
temporal reasoning ability in dialog context. Further, we see increasing
performance with increasing model size, which is not unexpected given the
complexity of the task.

###  5.2 Error Analysis

Next, we analyze the different types of errors based on different rules for
negative option creation in the annotation process. In particular, the _phrase
matching_ rule picks an exact time span from the dialog context, and _numeral
matching_ picks numerals from the dialog context. Thus, models picking those
incorrect options imply reliance on spurious shallow text matching features.

Figure 2:  Percentage of errors on options created by different rules. CLS,
MF, and GEN represent classification, mask-filling, and generation models,
respectively; and In and Out denote in-domain and out-of-domain training. All
models are of large size.

Figure 2 shows the percentage of errors in terms of the different rules. For
example, the BERT-based classification model CLS-In erroneously picks 52% of
negative options created by the phrase matching rule as correct answers (i.e.,
by ranking those negative options over the true correct options). We observe
that the various models are all most vulnerable to the phrase matching options
compared to other types of negative options, showing that they rely on
spurious text matching to a significant extent. Between BERT and T5, we find
T5 being more robust to shallow text matching.

Size | BASE | LARGE
---|---|---
Training | In | Out | In | Out
_Classification (BERT)_
Target | 50.5 | 40.0 | 50.5 | 47.5
Local |  $+$ 3.4 |  $+$ 3.3 |  $+$ 7.5 |  $+$ 2.0
Full |  $-$ 0.6 |  $-$ 0.1 |  $+$ 2.7 |  $+$ 1.2
_Mask Filling (BERT)_
Target | 57.8 | 44.3 | 60.3 | 46.8
Local |  $+$ 5.4 |  $+$ 3.0 |  $+$ 8.1 |  $+$ 4.9
Full |  $+$ 9.6 |  $+$ 3.1 |  $+$ 9.6 |  $+$ 8.0
_Generation (T5)_
Target | 55.5 | 45.9 | 66.7 | 56.1
Local |  $+$ 3.7 |  $+$ 2.7 |  $+$ 6.1 |  $+$ 3.7
Full |  $+$ 3.7 |  $+$ 4.7 |  $+$ 8.2 |  $+$ 5.8
Table 7:  Impact of dialog context on reasoning accuracy. In and Out denote
in-domain and out-of-domain training, respectively. We use _2-best accuracy_
of _target_ context as reference and report the absolute changes in
performance of _local_ and _full_ context, respectively. Local dialog context
results in better performance to full dialog context on $5$ of the $12$ cases,
which are highlighted in the table.

Table 6 provides further examples of prediction errors, illustrating
confusions due to shallow text matching. In the first dialog, both incorrect
answers already partially occur in the context or are related to preexisting
concepts (i.e., _“three”_ to _“three o’clock”_ , and _“nine”_ to
_“September”_). All the three models were confused and chose either of the two
as the top prediction for the blank, even though the options clearly violate
the context. Interestingly, the mask filling model was completely confused and
ranked both incorrect answers over the correct ones. Similarly in the second
example, the models fail to capture the contextual semantics.

###  5.3 Influence of Dialog Context

Table 7 shows how different scopes of dialog context (§4.2) affect model
performance. First, the most restrictive target-only context is insufficient
for accurate reasoning, by producing the weakest performance of most models.
This highlights the importance of context information for temporal commonsense
reasoning in dialog, which differs from previous temporal reasoning studies
based on limited context (e.g., single-sentence question answering). Second,
we note that the full dialog context does not always lead to the best
performance. In $5$ out of the $12$ cases, using the local context yields
equal or higher reasoning accuracy. The results show that the LMs still fall
short of properly modeling the rich dialog contexts and making effective use
of all information to do reasoning.

Figure 3:  Percentage of errors on different reasoning types. CLS, MF, and GEN
represent classification, mask-filling, and generation models, respectively.
All models are of large size.

###  5.4 Errors of Reasoning Categories

Figure 3 shows the percentage of errors in each reasoning category. We observe
that the models tend to make non-trivial portions of errors on
commonsense/world knowledge questions. For example, the strongest model, T5
Gen-In, failed on 18% of the instances that require commonsense or world
knowledge, while BERT CLS-In made errors on 48% of such instances. The
performance on comparison-based instances seems similar.

##  6 Related Work

##### Temporal commonsense reasoning.

Early studies related to temporal analysis define time in the context of sets
and relations Bruce (1972); Allen (1983). More recent works often associate
time with events and focus on identifying time expressions Chang and Manning
(2012); Angeli et al. (2012); Lee et al. (2014), extracting temporal relations
among events Setzer and Gaizauskas (2000); Pustejovsky et al. (2005); Lapata
and Lascarides (2006); Chambers et al. (2007); Ning et al. (2018b), and
timeline construction Do et al. (2012); Leeuwenberg and Moens (2018).

Some recent work has focused on building challenging benchmarks for temporal
commonsense reasoning. Story Cloze Test focuses on stereotypical causal
temporal and causal relations between events Mostafazadeh et al. (2016).
Vashishtha et al. (2020) recast temporal reasoning datasets for event duration
and event ordering into the natural language inference (NLI) format. Turque
Ning et al. (2020) is an reading comprehension dataset where the model needs
to answer questions such as “what happens before/after [event]”. Most related
to our work is McTaco Zhou et al. (2019), a dataset for evaluating temporal
commonsense in the form of multiple-choice reading comprehension, where the
context usually consists of a single sentence. Our work instead studies
temporal commonsense reasoning in dialogs which often require significant
commonsense and world knowledge to reason over rich context (Qin et al.,
2019b; Dinan et al., 2018).

##### Commonsense reasoning with LMs.

With the recent success of large pre-trained language models (LMs) Devlin et
al. (2019); Brown et al. (2020), it is an open question whether these models,
pretrained on large amounts of data, capture commonsense knowledge. Several
works have been proposed to assess the ability of LMs for commonsense or
numerical reasoning Zhang et al. (2020); Bouraoui et al. (2020), or to mine
commonsense knowledge from LMs Davison et al. (2019). Lin et al. (2020) showed
that state-of-the-art LMs such as BERT and RoBERTa performs poorly on
numerical reasoning tasks without any finetuning. Works have also been
proposed to improve language model’s commonsense reasoning Qin et al. (2020,
2019a); Zhou et al. (2020) and numerical reasoning abilities Geva et al.
(2020). In our work, we study several modeling approaches and finetuning
settings of large LMs, and establish strong baselines for temporal commonsense
reasoning in dialogs.

##  7 Conclusions

We introduced TimeDial, a challenge set consistting of 1.1K multiple-choice
cloze questions for temporal commonsense reasoning in dialog. The dataset is
carefully curated to evaluate a models’ ability to do temporal
commonsense/numerical reasoning over dialog context. In order to establish
strong baselines and provide information on future model development, we
conducted extensive experiments with state-of-the-art language models with
different settings: the scope of context, weak supervision strategies, and
learning objectives. While humans can easily answer these questions (97.8%
accuracy), even our best model variant (T5-large with in-domain training)
struggles on this challenge set (73%). Moreover, our qualitative error
analyses show that these large language models often rely on shallow, spurious
features (particularly text matching) when answering these questions, instead
of truly doing reasoning over the context.

## References

  * Adiwardana et al. (2020) Daniel Adiwardana, Minh-Thang Luong, David R So, Jamie Hall, Noah Fiedel, Romal Thoppilan, Zi Yang, Apoorv Kulshreshtha, Gaurav Nemade, Yifeng Lu, et al. 2020\.  Towards a human-like open-domain chatbot.  _arXiv preprint arXiv:2001.09977_.
  * Allen (1983) James F Allen. 1983.  Maintaining knowledge about temporal intervals.  _Communications of the ACM_ , 26(11):832–843.
  * Allen (1984) James F Allen. 1984.  Towards a general theory of action and time.  _Artificial intelligence_ , 23(2):123–154.
  * Angeli et al. (2012) Gabor Angeli, Christopher D Manning, and Dan Jurafsky. 2012.  Parsing time: Learning to interpret time expressions.  In _Proc. of NAACL_.
  * Bouraoui et al. (2020) Zied Bouraoui, José Camacho-Collados, and S. Schockaert. 2020.  Inducing relational knowledge from bert.  In _Proc. of AAAI_.
  * Brown et al. (2020) Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020.  Language models are few-shot learners.
  * Bruce (1972) Bertram C Bruce. 1972.  A model for temporal references and its application in a question answering program.  _Artificial intelligence_ , 3:1–25.
  * Chambers et al. (2007) Nathanael Chambers, Shan Wang, and Dan Jurafsky. 2007.  Classifying temporal relations between events.  In _Proc. of ACL_.
  * Chang and Manning (2012) Angel X Chang and Christopher D Manning. 2012.  Sutime: A library for recognizing and normalizing time expressions.  In _Proc. of LREC_.
  * Davison et al. (2019) Joe Davison, Joshua Feldman, and Alexander Rush. 2019.  Commonsense knowledge mining from pretrained models.  In _Proc. of EMNLP_.
  * Devlin et al. (2019) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019.  BERT: Pre-training of deep bidirectional transformers for language understanding.  In _Proc. of NAACL_.
  * Dinan et al. (2018) Emily Dinan, Stephen Roller, Kurt Shuster, Angela Fan, Michael Auli, and Jason Weston. 2018.  Wizard of wikipedia: Knowledge-powered conversational agents.  In _Proc. of ICLR_.
  * Do et al. (2012) Quang Do, Wei Lu, and Dan Roth. 2012.  Joint inference for event timeline construction.  In _Proc. of EMNLP_.
  * Geirhos et al. (2020) Robert Geirhos, Jörn-Henrik Jacobsen, Claudio Michaelis, Richard Zemel, Wieland Brendel, Matthias Bethge, and Felix A Wichmann. 2020.  Shortcut learning in deep neural networks.  _Nature Machine Intelligence_ , 2(11):665–673.
  * Geva et al. (2020) Mor Geva, Ankit Gupta, and Jonathan Berant. 2020.  Injecting numerical reasoning skills into language models.  In _Proc. of ACL_.
  * Gururangan et al. (2018) Suchin Gururangan, Swabha Swayamdipta, Omer Levy, Roy Schwartz, Samuel Bowman, and Noah A Smith. 2018.  Annotation artifacts in natural language inference data.  In _Proc. of NAACL_ , pages 107–112.
  * Kahn and Gorry (1977) Kenneth Kahn and G.Anthony Gorry. 1977.  Mechanizing temporal knowledge.  _Artificial Intelligence_ , 9(1):87 – 108.
  * Kozareva and Hovy (2011) Z. Kozareva and E. Hovy. 2011.  Learning temporal information for states and events.  In _2011 IEEE Fifth International Conference on Semantic Computing_ , pages 424–429.
  * Lan et al. (2019) Zhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel, Piyush Sharma, and Radu Soricut. 2019.  Albert: A lite bert for self-supervised learning of language representations.  _arXiv preprint arXiv:1909.11942_.
  * Landis and Koch (1977) J Richard Landis and Gary G Koch. 1977.  The measurement of observer agreement for categorical data.  _biometrics_ , pages 159–174.
  * Lapata and Lascarides (2006) Mirella Lapata and Alex Lascarides. 2006.  Learning sentence-internal temporal relations.  _Journal of Artificial Intelligence Research_ , 27:85–117.
  * Le Bras et al. (2020) Ronan Le Bras, Swabha Swayamdipta, Chandra Bhagavatula, Rowan Zellers, Matthew Peters, Ashish Sabharwal, and Yejin Choi. 2020.  Adversarial filters of dataset biases.  In _Proc. of ICML_ , pages 1078–1088. PMLR.
  * Lee et al. (2014) Kenton Lee, Yoav Artzi, Jesse Dodge, and Luke Zettlemoyer. 2014.  Context-dependent semantic parsing for time expressions.  In _Proc. of ACL_.
  * Leeuwenberg and Moens (2018) Artuur Leeuwenberg and Marie Francine Moens. 2018.  Temporal information extraction by predicting relative time-lines.  In _Proc. of EMNLP_.
  * Li et al. (2017) Yanran Li, Hui Su, Xiaoyu Shen, Wenjie Li, Ziqiang Cao, and Shuzi Niu. 2017.  Dailydialog: A manually labelled multi-turn dialogue dataset.  _arXiv preprint arXiv:1710.03957_.
  * Lin et al. (2020) Bill Yuchen Lin, Seyeon Lee, Rahul Khanna, and Xiang Ren. 2020.  Birds have four legs?! NumerSense: Probing Numerical Commonsense Knowledge of Pre-Trained Language Models.  In _Proc. of EMNLP_.
  * Liu et al. (2019) Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019.  Roberta: A robustly optimized bert pretraining approach.  _arXiv preprint arXiv:1907.11692_.
  * Mihaylov and Frank (2018) Todor Mihaylov and Anette Frank. 2018.  Knowledgeable reader: Enhancing cloze-style reading comprehension with external commonsense knowledge.  In _Proc. of ACL_.
  * Mostafazadeh et al. (2016) Nasrin Mostafazadeh, Nathanael Chambers, Xiaodong He, Devi Parikh, Dhruv Batra, Lucy Vanderwende, Pushmeet Kohli, and James Allen. 2016.  A corpus and cloze evaluation for deeper understanding of commonsense stories.  In _Proc. of NAACL_.
  * Ning et al. (2018a) Qiang Ning, Zhili Feng, Hao Wu, and Dan Roth. 2018a.  Joint reasoning for temporal and causal relations.  In _Proc. of ACL_.
  * Ning et al. (2020) Qiang Ning, Hao Wu, Rujun Han, Nanyun Peng, Matt Gardner, and Dan Roth. 2020.  TORQUE: A Reading Comprehension Dataset of Temporal Ordering Questions.  In _Proc. of EMNLP_.
  * Ning et al. (2018b) Qiang Ning, Hao Wu, Haoruo Peng, and Dan Roth. 2018b.  Improving temporal relation extraction with a globally acquired statistical resource.  In _Proc. of NAACL_.
  * Onishi et al. (2016) Takeshi Onishi, Hai Wang, Mohit Bansal, Kevin Gimpel, and David McAllester. 2016\.  Who did what: A large-scale person-centered cloze dataset.  In _Proc. of EMNLP_.
  * Pustejovsky (2017) James Pustejovsky. 2017.  Iso-timeml and the annotation of temporal information.  In _Handbook of Linguistic Annotation_ , pages 941–968. Springer.
  * Pustejovsky et al. (2005) James Pustejovsky, Robert Knippen, Jessica Littman, and Roser Saurí. 2005.  Temporal and event information in natural language text.  _Language resources and evaluation_ , 39(2):123–164.
  * Qin et al. (2019a) Lianhui Qin, Antoine Bosselut, Ari Holtzman, Chandra Bhagavatula, Elizabeth Clark, and Yejin Choi. 2019a.  Counterfactual story reasoning and generation.  In _Proc. of EMNLP_.
  * Qin et al. (2019b) Lianhui Qin, Michel Galley, Chris Brockett, Xiaodong Liu, Xiang Gao, William B Dolan, Yejin Choi, and Jianfeng Gao. 2019b.  Conversing by reading: Contentful neural conversation with on-demand machine reading.  In _Proc. of ACL_.
  * Qin et al. (2020) Lianhui Qin, Vered Shwartz, Peter West, Chandra Bhagavatula, Jena D Hwang, Ronan Le Bras, Antoine Bosselut, and Yejin Choi. 2020.  Back to the future: Backpropagation-based decoding for unsupervised counterfactual and abductive reasoning.  In _Proc. of EMNLP_.
  * Radvansky and Zacks (2014) Gabriel A Radvansky and Jeffrey M Zacks. 2014.  _Event cognition_.  Oxford University Press.
  * Raffel et al. (2020) Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020.  Exploring the limits of transfer learning with a unified text-to-text transformer.  _Journal of Machine Learning Research_ , 21(140):1–67.
  * Setzer and Gaizauskas (2000) Andrea Setzer and Robert J Gaizauskas. 2000.  Annotating events and temporal information in newswire texts.  In _Proc. of LREC_.
  * Taylor (1953) Wilson L Taylor. 1953.  “cloze procedure”: A new tool for measuring readability.  _Journalism quarterly_ , 30(4):415–433.
  * UzZaman et al. (2013) Naushad UzZaman, Hector Llorens, Leon Derczynski, James Allen, Marc Verhagen, and James Pustejovsky. 2013.  SemEval-2013 task 1: TempEval-3: Evaluating time expressions, events, and temporal relations.  In _Proc. of SemEval_.
  * Vashishtha et al. (2020) Siddharth Vashishtha, Adam Poliak, Yash Kumar Lal, Benjamin Van Durme, and Aaron Steven White. 2020.  Temporal reasoning in natural language inference.  In _Proc. of Findings of EMNLP_.
  * Zhang et al. (2020) Xikun Zhang, Deepak Ramachandran, Ian Tenney, Yanai Elazar, and Dan Roth. 2020.  Do language embeddings capture scales? In _Proc. of Findings of EMNLP_.
  * Zhou et al. (2019) Ben Zhou, Daniel Khashabi, Qiang Ning, and Dan Roth. 2019.  “going on a vacation” takes longer than “going for a walk”: A study of temporal commonsense understanding.  In _Proc. of EMNLP_.
  * Zhou et al. (2020) Ben Zhou, Qiang Ning, Daniel Khashabi, and Dan Roth. 2020.  Temporal common sense acquisition with minimal supervision.  In _Proc. of ACL_.

##  Appendix A Configurations

We provide all model and training configurations used across our experiments:

###  A.1 BERT Experiments for Classification and Mask-Filling

  * •

Model configuration for BERT-Base classification and mask-filling:

            attention_dropout_rate: 0.1
        dropout_rate: 0.1
        hidden_activation: gelu
        hidden_size: 768
        initializer_range: 0.02
        intermediate_size: 3072
        max_position_embeddings: 512
        num_attention_heads: 12
        num_layers: 12
        type_vocab_size: 2
        vocab_size: 30522

  * •

Model configuration for BERT-Large classification and mask-filling:

            attention_dropout_rate: 0.1
        dropout_rate: 0.1
        hidden_activation: gelu
        hidden_size: 1024
        initializer_range: 0.02
        intermediate_size: 4096
        max_position_embeddings: 512
        num_attention_heads: 16
        num_layers: 24
        type_vocab_size: 2
        vocab_size: 30522

  * •

Training configuration for classification with BERT-Base and in-domain data:

            num_classes: 2
        train_data:
          global_batch_size: 128
          seq_length: 512
        validation_data:
          global_batch_size: 32
          seq_length: 512
        trainer:
          max_to_keep: 3
          checkpoint_interval: 1000
          decay_steps: 30000
          end_learning_rate: 0.0
          initial_learning_rate: 1.0e-5
          power: 1.0
          optimizer: adam
          warmup_steps: 5000
          steps_per_loop: 1000
          train_steps: 30000
          validation_steps: 256

  * •

Training configuration for classification with BERT-Large and in-domain data:

            num_classes: 2
        train_data:
          global_batch_size: 128
          seq_length: 512
        validation_data:
          global_batch_size: 32
          seq_length: 512
        trainer:
          max_to_keep: 3
          checkpoint_interval: 1000
          decay_steps: 100000
          end_learning_rate: 0.0
          initial_learning_rate: 1.0e-6
          power: 1.0
          optimizer: adam
          warmup_steps: 10000
          steps_per_loop: 1000
          train_steps: 100000
          validation_steps: 3000

  * •

Training configuration for classification with BERT-Base and out-domain data:

            num_classes: 2
        train_data:
          global_batch_size: 128
          seq_length: 512
        validation_data:
          global_batch_size: 128
          seq_length: 512
        trainer:
          max_to_keep: 3
          checkpoint_interval: 5000
          decay_steps: 500000
          end_learning_rate: 0.0
          initial_learning_rate: 1.0e-6
          power: 1.0
          optimizer: adam
          warmup_steps: 10000
          steps_per_loop: 1000
          train_steps: 500000
          validation_steps: 512

  * •

Training configuration for classification with BERT-Large and out-domain data:

            num_classes: 2
        train_data:
          global_batch_size: 128
          seq_length: 512
        validation_data:
          global_batch_size: 128
          seq_length: 512
        trainer:
          max_to_keep: 3
          checkpoint_interval: 5000
          decay_steps: 500000
          end_learning_rate: 0.0
          initial_learning_rate: 1.0e-6
          power: 1.0
          optimizer: adam
          warmup_steps: 10000
          steps_per_loop: 1000
          train_steps: 500000
          validation_steps: 512

  * •

Training configuration for mask-filling with BERT-Base and in-domain data:

            train_data:
          global_batch_size: 128
          seq_length: 512
          max_predictions_per_seq: 20
        validation_data:
          global_batch_size: 128
          seq_length: 512
          max_predictions_per_seq: 20
        trainer:
          checkpoint_interval: 2000
          max_to_keep: 30
          decay_steps: 30000
          end_learning_rate: 0.0
          initial_learning_rate: 1.0e-8
          power: 1.0
          optimizer: adam
          warmup_steps: 5000
          steps_per_loop: 1000
          train_steps: 30000
          validation_interval: 1000

  * •

Training configuration for mask-filling with BERT-Large and in-domain data:

            train_data:
          global_batch_size: 128
          seq_length: 512
          max_predictions_per_seq: 20
        validation_data:
          global_batch_size: 128
          seq_length: 512
          max_predictions_per_seq: 20
        trainer:
          checkpoint_interval: 2000
          max_to_keep: 30
          decay_steps: 30000
          end_learning_rate: 0.0
          initial_learning_rate: 1.0e-8
          power: 1.0
          optimizer: adam
          warmup_steps: 5000
          steps_per_loop: 1000
          train_steps: 30000
          validation_interval: 1000

  * •

Training configuration for mask-filling with BERT-Base and out-domain data:

            train_data:
          global_batch_size: 512
          seq_length: 512
        validation_data:
          global_batch_size: 512
          seq_length: 512
        trainer:
          checkpoint_interval: 5000
          max_to_keep: 10
          decay_steps: 300000
          end_learning_rate: 0.0
          initial_learning_rate: 1.0e-6
          power: 1.0
          optimizer: adam
          warmup_steps: 10000
          steps_per_loop: 1000
          train_steps: 300000
          validation_steps: 1000

  * •

Training configuration for mask-filling with BERT-Large and out-domain data:

            train_data:
          global_batch_size: 512
          seq_length: 512
        validation_data:
          global_batch_size: 512
          seq_length: 512
        trainer:
          checkpoint_interval: 5000
          max_to_keep: 10
          decay_steps: 300000
          end_learning_rate: 0.0
          initial_learning_rate: 1.0e-6
          power: 1.0
          optimizer: adam
          warmup_steps: 10000
          steps_per_loop: 1000
          train_steps: 300000
          validation_steps: 1000

###  A.2 T5 Experiments for Generation

  * •

The training configuration for generation with T5-Base and in-domain data:

          encoder_seq_length: 512
      decoder_max_length: 128
      train_batch_size: 128
      max_train_steps: 100000
      valid_batch_size: 128
      dropout_rate: 0.2
      optimizer: adam
      learning_rate: 1.0e-6

  * •

The training configurations for generation with T5-Base/Large and in-
domain/out-domain data are similar as above, except that the learning rate is
set to 5.0e-6 for T5-Large in-domain data, 5.0e-4 for T5-Base out-domain data,
and 1.0e-4 for T5-Large out-domain data.