# Automated Story Generation as Question-Answering

Louis Castricato, Spencer Frazier, Jonathan Balloch, Nitya Tarakad, and Mark
O. Riedl

###### Abstract

Neural language model-based approaches to automated story generation suffer
from two important limitations. First, language model-based story generators
generally do not work toward a given goal or ending. Second, they often lose
coherence as the story gets longer. We propose a novel approach to automated
story generation that treats the problem as one of generative question-
answering. Our proposed story generation system starts with sentences
encapsulating the final event of the story. The system then iteratively (1)
analyzes the text describing the most recent event, (2) generates a question
about “why” a character is doing the thing they are doing in the event, and
then (3) attempts to generate another, preceding event that answers this
question. We evaluate the coherence of generated stories using human
participant studies that show that stories generated by our system are 15%
more coherent ($p<0.013$) than those generated by BART fine-tuned on a story
corpus to generate backward from a given ending.

##  1 Introduction

Consider a story, at the ending of which a princess is reunited with her lover
thought to be lost at sea, a swordsman has enacted revenge on the man who
killed his father, and a giant becomes a pirate. One might reasonably wonder
how this situation came to pass. Aristotle writes in Poetics that the events
of the story serve the plot and the end. Under this interpretation,
storytelling is explanation—every event answers the question of “how did the
next event come to pass?”. In this paper, we propose an automated story
generation system using the principles of question-answering and show how it
can improve automated story generation capabilities.

Automated Story Generation is the challenge of designing an artificial
intelligence system that can generate a story from a minimal number of
inputs—often just a prompt and some storytelling knowledge. Symbolic story and
plot generation systems have traditionally relied on planning or case-based
reasoning (see Gervás (2009) for an overview of symbolic story generation
systems). Some of these systems start with an end state—the state the
fictional world should be in at the end of the story—and work backward,
determining what must have happened to transform an initial world state into
the goal. These systems often generate coherent stories guaranteed to end in a
given state. Their drawback is that they require significant hand-authored
domain knowledge.

Machine learning-based story generation systems acquire or learn story domain
knowledge from data, often corpora of human-authored stories. Most machine
learning-based story generation systems have relied on neural network-based
language models. Auto-regressive neural language models trained on a corpus of
stories learn a probability distribution over tokens
$p(t_{n}|t_{n-1},t_{n-2},...,t_{n-k})$ based on the tokens that occur in the
training corpus. This distribution can then be sampled to create new texts
that emulate the training corpus. Training a neural language model on story
corpora results in a generative model that produces texts that look like
stories (Roemmele 2016; Khalifa, Barros, and Togelius 2017; Martin et al.
2018). However, language model based approaches are unable to bring stories to
a particular conclusion or goal state. Stories generated by language models
also tend to lose coherence over time as they rely on probabilistic sampling
and do not learn a richer model of the story world.

We consider how neural story generation systems can be induced to generate
more coherent narratives that also end in a pre-determined, desirable way.
Narratives are perceived to be coherent when events are related to each other
in a way that is comprehensible by the reader (Trabasso and Van Den Broek
1985; Graesser, Lang, and Roberts 1991). There are many relations between
events which fit this need, the most important are: (1) causal relations—one
event cannot happen if another event had not happened prior to it—and (2)
character goal hierarchies—an action is in service of a goal or another action
that is in service of a goal.

Our insight is that if each event in the story is generated to explicitly
answer the question of “why” the next event in the story happens, then readers
will perceive the story as more coherent. To generate a story that will be
perceived as a coherent and build up to a pre-determined ending, we propose to
generate the story backward. This is achieved by starting from a textual
description of the final event; each event added best answering the question
of what must have proceeded it. Our system, EDGAR, repeats this process for a
specified number of iterations. Questions are generated using a commonsense
inference model, Para-COMET (Gabriel et al. 2020), to predict what readers are
likely to believe about a story event; the inferences are transformed into
questions using templates. EDGAR then attempts to answer each question using a
generative question-answering model.

We evaluate our system against a baseline neural transformer-based language
model approach that is fine-tuned to generate story events backward, matching
the backward process of EDGAR We measure story coherence with two human-
participant studies. In the first, perceived coherence is measured as the
entropy in participant responses to true/false questions about the story
(Castricato et al. 2021b, a); a story that is more comprehensible results in
less random guessing by human readers. We find that EDGAR generates more
coherent stories than the baseline as evidenced by the entropy of answers
about stories generated by EDGAR had 15.9% lower entropy than those of the
baseline. The second evaluation is subjective—we qualitatively measure
coherency via subjective questionnaire about coherence. Participants consider
stories written by EDGAR twice as coherent as those written by the baseline.

##  2 Related Work

Gervás (2009) overviews early symbolic story generation systems. Story
generation systems that use symbolic story planners utilize logic-like domain
representations that provide knowledge about available actions, their
preconditions, and their effects. A search process—such as that by Riedl and
Young (2010)—selects a goal condition or a precondition of an action in the
plan and attempts to find another, preceding action that has an effect that
establishes the condition. This process iterates, creating chains of
preconditions and effects until everything is grounded in the initial world
state. However, the chaining can be done forward from the initial state to the
goal as well (Ware and Young 2010; Ware and Siler 2021).

Neural networks have the potential to generate a greater range of stories by
learning model for how to tell stories from a corpus of exemplar stories.
Neural language models learn the probability that one or more tokens will
occur given a history of one or more prior tokens,
$P_{\theta}(t_{n+1},...,t_{n+m}|t_{n-k},...,t_{n-1},t_{n})$, according to
token occurrence patterns in a corpus. Neural language models can be induced
to generate text that can be read as a story by sampling from the learned
distribution over tokens and appending them to a prompt. Some neural language
model based story generation techniques include (Roemmele 2016; Martin et al.
2018; Khalifa, Barros, and Togelius 2017). However, a neural language model
alone is incapable of achieving a specific end state or event. Sampling from a
distribution over tokens only considers the most likely successive tokens
given a window of prior tokens. Neural language models also tend to lose story
coherence over time. This is due to the fact that a language model only models
a distribution over tokens in the training set. Additionally, the hidden
parameters of current neural networks are unlikely to encode the state of a
fictional world, as human readers would understand.

Tambwekar et al. (2018) attempt to train a neural language model to generate
toward a given goal. They fine-tune a neural language model with a policy-
gradient reinforcement learning technique that rewards the language model for
generating events progressively closer to the goal event. This has the benefit
of improving readers’ perceptions of coherence, but—being based on a language
model—does not ensure that any transition from one event to the next will
always be perceived as related.

Other neural language model approaches to story generation using neural
networks use hierarchical conditioning, in which a high-level guidance
specification is given either periodically or per sentence in the story (Fan,
Lewis, and Dauphin 2018; Rashkin et al. 2020; Ammanabrolu et al. 2020b). These
high-level guidance specifications turn the generation problem into a
supervised learning problem. We do not consider these approaches further in
this paper because we do not assume the existence of a guidance specification.
A high level plan can be generated (Yao et al. 2019; Fan, Lewis, and Dauphin
2019; Ammanabrolu et al. 2020b), which elevates the challenges of maintaining
coherence to a higher level of abstraction but cannot reliably achieve a given
story ending.

Our work is most similar to that of Peng et al. (2021), which uses a
commonsense inference model to match inferences at time step $t-1$ to those at
time $t$. In contrast our approach works backwards and uses a commonsense
inference model to generate a “why” question about a character, which is then
used to prompt generation of story events that are preceding. The C2PO system
(Ammanabrolu et al. 2020a) also uses the COMET (Bosselut et al. 2019)
commonsense inference engine. However, this work uses COMET to generate
successor and predecessor events, performing a bi-directional search from a
given start event and a given end event. It is relevant to our work in that it
does partially chain backward from a given end event, and also uses a
commonsense inference engine. However, C2PO generates plots made up of short
statements of character intentions, whereas our system generates stories with
more natural-looking language.

##  3 The EDGAR System

The Explanatory Drama Generation And Recall (EDGAR) system constructs a story
backwards from a given sentence describing the end of the story. The system
contains three major components. The first component is a question generator.
Given a story context—the sequence of text describing the earliest event in
the ending context—a set of questions about the event is generated. Second, a
question answering component attempts to generate text describing one or more
events that answer that question. A number of candidate answers are generated
for each question. Finally, the answers are iteratively pre-pended to the
context and a ranker chooses the best sequence. The best sequence is added to
the story and the process iterates. See the pipeline in Figure 1.

Figure 1: EDGAR generates stories backward. Given the end of the story where
$S_{0}$ is the earliest event sequence and $S^{\prime}$ is the remainder,
Para-COMET generates a set of $n$ inferences. Each inference is converted a
question and the ELI5 QA model generates $k+1$ answers. The answers are
concatenated to the beginning of the story and the ranker selects the best
scoring story. This process is repeated.

###  3.1 Question Generation

We use Para-COMET (Gabriel et al. 2020) to generate questions. Para-COMET is a
commonsense inference model trained to generate potential commonsense
explanations about one or more sentences of input text. Inferences made by
Para-COMET have types. In particular, xIntent explains what a character in the
sentences might have intended in performing any actions in the sentences.
These correspond to goal relations in reader comprehension (Graesser, Lang,
and Roberts 1991). xNeed explains what a character might have needed to
perform any actions in the sentences. These provide precondition-like
inferences, corresponding to causal relations in reader comprehension
(Trabasso and Van Den Broek 1985). We discard all other relation types.
Because Para-COMET works on multi-sentence sequences, we extract a rolling
window of the last 5 xIntent and xNeed inferences.

Unfortunately, Para-COMET does not identify which character is associated with
each xIntent and xNeed, which is problematic for stories with more than one
character. To associate the xNeed and xIntent clauses with a character, we
generate the following templates:

  * •

“Who needs to $xIntent$”

  * •

“Who needs $xNeed$”

filling in the details of the inferences. These filled templates are provided
as input to RoBERTa (Liu et al. 2019), a question-answering model that, when
provided the context and the template will output the names of the characters
most likely to have had these needs and intents.

Finally, we use a second set of templates to assemble the final set of
questions:

  * •

“Why does $character$ do $xIntent$?”

  * •

“What does $character$ do to need $xNeed$?”

This process generates a total of 8 questions.

###  3.2 Question Answering

Once we have a set of questions, EDGAR generates candidate answers, such that
each candidate can be added to the beginning of the story context so far. To
generate sentences describing the preceding event that answers the questions
generated, we feed the questions into the ELI5 QA model (Fan et al. 2019). The
ELI5 QA model is a long-form, question-answering model trained on the Explain
Like I’m Five Reddit corpus,111https://www.reddit.com/r/explainlikeimfive/ in
which people give long, yet easily comprehensible answers to open-ended
questions as one might give to a five-year old. ELI5 QA requires a reference
document from which to abstract answers. The reference document is the source
material—in this case a story—that ELI5 QA uses to generate an answer. Because
EDGAR is an unsupervised technique designed to generate novel stories, there
is no one reference document that should be used; using a single reference
document would run the risk of accidentally recreating a human-written story.
For every iteration we randomly select a reference document from the Flash
Fiction Online repository.222https://www.flashfictiononline.com The question
templates above were constructed to induce relatively short answers from ELI5,
which has a tendency to generate very long explanations.

We use beam search to generate 15 candidate answers for each question. As
another measure to prevent ELI5 from providing overly verbose explanations, we
have accumulated a list of over 700 unique banned phrases, which occur when
ELI5 commentators point out “facts” or likening a character’s action to mental
disability. This blocked phrases list was accumulated iteratively, by
rerunning the model repeatedly and adding any toxic phrases to this excluded
list. The result is $n\times k$ story continuations where $n$ is the number of
questions, $k$ is the number of beams per question on ELI5.

###  3.3 Ranking

Once EDGAR has generated a set of candidates, the final step in the process is
to select the best candidate for pre-pending to the context (the end of the
story). We prepend each answer to the context and rate each resulting text
sequence using GPT-2 (Radford et al. 2019) to assess the probability of the
sequence. GPT2 was fine-tuned on the science fiction summary corpus
(Ammanabrolu et al. 2020b) dataset, which consists of 2,276 high-quality plot
summaries from science fiction TV and movie wikis. We fine-tune on the science
fiction summary corpus because wiki plots do not include descriptive details
or dialogue; our ranker thus prefers more plot-like narrative content.
Candidates are ranked by perplexity of the GPT2 model. The normalized
perplexity distribution over the beams outputted by ELI5 refers to the
$1-\textit{probability}$ distribution of a body of text existing within the
distribution of science fiction summaries.

Ranking is an important step because of the numerous processes involved; as a
consequence the ranking of ELI5 beam distribution does not necessarily
correlate with the final ranking, which roughly measures fluency. The best
scoring candidate is added to the overall story. The process repeats with the
new, longer story, attempting to determine what happened just before the new
context.

##  4 Objective Evaluation

We hypothesize that EDGAR, by virtue of question-answering, can generate more
coherent stories than a pure language modeling technique. We define coherence
as any perceivable relationship between events in a story. Research on reading
comprehension (Trabasso and Van Den Broek 1985; Graesser, Lang, and Roberts
1991) suggest that causal and goal relationships are particularly important.

Stories generated by EDGAR | Stories generated by bBART
---|---
He needs to find a way out of the house. If he wants to go to the kitchen, he must first find a way to escape from the house. This means that he must either run away or fight his way past the lock. He goes to the bathroom. The only way to get to the kitchen is through a locked door, which Hansel has never been able to do before. However, when he opens the door, it doesn’t take him very long at all. Hansel’s hand still trembles as he pushes open the twice-cooked door. The last time he saw the house he was glancing back over his shoulder as he and his sister fled into the trees. |  A young boy named Hansel has been walking through town recently looking at some strange things that are going on outside their walls. In fact, they were just playing cards before anyone noticed them. A young boy called Hansel has Been walking through town lately looking at some odd things that are gone missing outside their walls – maybe even someone noticing them. In reality, they were probably playing cards before everyone noticed them. He wants to learn more about it though, but can always tell for sure. He wants to know more about it, but can never tell for sure. After all, this is a city of humans who have no interest in living here.He wants to know what happened to him, but can only tell for sure if this is a City of Humans who have little interest in living there. Hansel’s hand still feels like an old man with no interest whatsoever. His mind wanders frantically around trying to figure out how to live here. Hansel’s hand still trembles as he pushes open the twice-cooked door. The last time he saw the house he was glancing back over his shoulder as he and his sister fled into the trees.
If it does not have a pulse then it is dead. The heart rate of an animal tells
them if it is alive or dead. There is no such thing as absolute death. He is
able to tell the difference between something being alive and something being
dead, so when he looks at the house, he feels like he’s seeing things that
aren’t there. This makes him feel uncomfortable because he doesn’t want to be
in that situation. It’s similar to how people can see ghosts or monsters from
inside their head but they don’t know what those things are. Hansel’s hand
still trembles as he pushes open the twice-cooked door. The last time he saw
the house he was glancing back over his shoulder as he and his sister fled
into the trees.
Table 1: Example stories generated by EDGARnd backwards BART (bBART). Bold
text indicates the given ending prompt.

Common automated evaluation metrics for story generation such as perplexity
and BLEU are insufficient as they only measure whether a generator can
recreate the ground truth corpus. A story may deviate from the ground truth
and be considered a good story—indeed this is a desirable property of an
automated story generator. Furthermore, systems such as ours may be
unsupervised and have many components that intentionally push a language model
away from any one corpus, thus making perplexity less meaningful. For these
reasons, story generation research often relies on human participant studies
with subjective questions.

We assert human participant studies are the best way to assess the coherence
of generated stories. Question-answering protocols, wherein questions are
asked about a story, have been proposed as a means to make human-participant
evaluations more objective (Riedl and Young 2010; Cardona-Rivera et al. 2016).
We conduct a human-participant evaluation using a metric based on the Fabula
Entropy Indices (Castricato et al. 2021b), a set of objective measures of
story coherence grounded in narratological formalisms (Castricato et al.
2021a). The Fabula Entropy Indices are validated to be correlated with more
common, subjective evaluation metrics, which we also replicate in this paper
to provide further experimental evidence.

###  4.1 Baselines

Because EDGAR generates stories from a given ending, the system cannot be
evaluated relative to other systems that generate forward and are thus not
guaranteed to reach a given goal. The BART (Lewis et al. 2019) neural language
model was used as a baseline, but fine-tuned to generate events backward to
conform to EDGAR and guarantee the presence of a given end event. The dataset
used to fine-tune consisted of $2276$ narratives from a science fiction
summary corpus (Ammanabrolu et al. 2020b). The narratives are preprocessed to
create our dataset. From every narrative, $2+2k$ sequential sentences are
obtained, where $k$ is a random integer less than $5$. The $2+2k$ sentences
are split apart into $2$ sentences and $2k$ sentences, creating the source and
target of the dataset respectively. The $2$ sentences generated in the $2+2k$
sentence chunk always precede the $2k$ sentences, establishing a relationship
between sequential sentences. We preprocess this data to this format because
an attribute found in most narrative summaries within our dataset is that
preceding sentences to any given sentences gives some notion of causality.
BART utilizes seq2seq as its translation architecture. As a consequence of the
input data format, our fine-tuned Backward-BART—which we refer to as bBART—can
generate narratives backwards by assessing the causality between sequentially
sentences.

Human-written stories from the ROCStories corpus (Mostafazadeh et al. 2016)
were are also included in our evaluation as a point of comparison. These
stories have a definitive causality between sequential sentences.

###  4.2 Method

We detail the Fabula Entropy Index methodology we use below. To evaluate the
objective coherence of stories, we turn to cognitive psychology. Cognitive
psychology research suggests that recall is strongly correlated with narrative
causal coherence (Trabasso and Van Den Broek 1985). The cognitive load of
inferring entailments about a story is strongly correlated with how well the
story conveys information about its fabula333A story’s fabula denotes the
chronological sequence of events in a narrative. (Carney 2019). We devise a
new evaluation methodology wherein we ask participants to read stories and
then answer true/false questions about how the events of the story relate to
each other. We measure the amount of agreement between readers’ answers in
terms of entropy. If the story is coherent, readers will come to the same
conclusions about the truth or falseness of the questions, and entropy will be
low. If the story is incoherent, readers—forced to choose between true and
false—will choose more randomly, resulting in higher entropy. We do not
require a ground truth “correct” answer to each question in order to compute
the entropy; this is a desirable property of our methodology given (1) there
are no algorithmically produced ground truth answers to the true/false
questions and (2) obtaining a ground truth answer from humans can be noisy.
Our index method is inspired by the evaluation used in Li et al. (2012) where
human participants were asked to choose event orderings and participant
agreement was assessed as entropy.

We generated 11 stories using EDGAR, 11 stories using backward-BART, and
randomly selected 11 stories from the ROCStories corpus. Stories were
generated by running the respective systems 3 iterations. Stories ranged from
5 sentences to 20 sentences in length. See Table 1 for examples from EDGAR and
bBART. The Appendix gives the entire set of stories used in the evaluation.
For the 33 stories, we produced 7 true/false questions for each story using
the technique described in Section 4.3. To avoid bias, a non-computer-science
graduate student not affiliated with the research project was paid to write
the questions using the above template.

We recruited 180 human-subject participants from Mechanical Turk. Participants
were recruited from countries that speak English as the national language.
Each participant was asked to read 3 stories and answer the 7 true/false
questions after each story. Participants were paid $7 for a 15 minute task.
The first story is a “screener” story, an uncommon fable that is easy to
understand. If a participant did not answer the questions how we expect, we
eliminated the participant from the pool. Participants were also eliminated
from the pool if they resorted to marking all questions true or false or
marked questions in some otherwise visually obvious repeating pattern; we
eliminated 26 participants.

###  4.3 Computing the Entropy Index

We define the entropy index of each story as follows. For a given story
generation system, we randomly selected 11 generated stories. For each story,
we then produced 7 entailment questions about each story. Entailment questions
are of the form of implications. By asking the reader to answer true or false
we are asking the reader to prove or disprove the statement within the realm
of what has been presented about the story world.

In order to ensure our questions were not biased, we provide annotators the
following templates, two of which are given as examples here:

  * •

$E_{i}$ depends on $E_{j}$

  * •

$E_{i}$ could be removed and the story would still make sense.

$i<j$ and $E$ refers to an event within the story. The full set of templates
can be found in the appendix. The questions themselves were manually written
to ensure grammatically correctness and readability.

The answers to the entailment questions give us a measure of entropy. When
participants disagree, it can be determined how ambiguous their model of the
story world is, such that they must rely heavily on external bias.

Consider that we have some story, $S$, composed of an event chain
$E=\\{E_{i}\\}_{n}$. An event chain being a sequence of events discussed in a
story, one path in a fabula. Generate two events, one that could be inserted
into $E$ and preserve coherence and its negation. We’ll refer to these events
as $A$ and $B$. Refer to their insertions as $E^{A}$ and $E^{B}$. Assume that
we had some function $f(\cdot)$ that could take either $E^{A}$ or $E^{B}$ and
rank all of the explanations for $A$ and $B$ respectively by mental load
induced on the reader. Then, if $E^{B}$ is coherent, consider what mental
leaps are required by the reader for justification. Let $D(A)$ and $D(B)$
refer to these normalized distributions respectively. Measure the following:

$\begin{split}&\text{KL}_{A}=\text{KL}(D(A)\|U)\text{ and
}\text{KL}_{B}=\text{KL}(D(B)\|U)\end{split}$ (1)

Where KL is Kullback–Leibler divergence and $U$ is a uniform distribution.
Inductively if $A$ and $B$ are in direct contradiction of each other, we can
collapse the above statement to

$\text{KL}_{A,B}=\text{KL}\Big{(}D\Big{(}\sum f\big{(}E^{A}\big{)},\sum
f\big{(}E^{B}\big{)}\Big{)}\|U\Big{)}$ (2)

In this case, since $U$ is of dimension two, simplify the above to entropy. We
can conclude that measuring the coherence of such an insertion is equivalent
to measuring the entropy over the answers to a similarly constructed T/F
statement about a causal relationship within a story. Over a large number of
questions and stories per model, the above serves as a sound proxy for
coherence. Consider a coherent story and a set of T/F questions concerning
this story. It is often easier to disprove a statement about a coherent story
than it is to prove a statement about an incoherent story (O’Brien and
Albrecht 1992; Albrecht and O’Brien 1993). By utilizing the format of T/F
questions, the above will tend to converge to zero on a coherent story as
there will always be one option that is disprovable. To get a large enough
sample, we used 77 questions per model over 11 stories.

###  4.4 Results

The evaluation results are plotted in Figure 2. The evaluation shows that
EDGAR scores a median of 0.427 on the entropy index, compared to bBART’s
median of 0.508. Human written stories from the ROCStories corpus scored a
median entropy of 0.26.

From these results we can draw a number of conclusions. First, the median
entropy of human-authored stories is over 95% better than bBART and over 63%
better than EDGAR. This implies that human-authored stories are much more
coherent than computer-generated stories according to our Entropy Index
metric. This is the expected result and shows that our Entropy Index metric is
operating as expected. The human story entropy index is a lower bound.
Importantly, the median entropy EDGAR is 15.9% lower than that of the bBART
baseline, indicating that our technique has improved the coherence of
generated stories when generating backwards in order to ensure a given ending.

##  5 Subjective Evaluation

We conducted a second human-participant evaluation in which participants read
stories and answered subjective questions about the coherence of the stories.
We would expect the results of this experiment to concur with the results of
the previous experiment.

Purdy et al. (2018) proposes a number of questions to be used to evaluate
story generation systems. They have been used in a number of story generation
system evaluations (cf. (Tambwekar et al. 2018; Ammanabrolu et al. 2020b, a;
Peng et al. 2021)) as well as to validate the Fabula Entropy Indices. We use a
subset of the questions and adapt them to rank-order choice between stories
from two systems:

  * •

Which story’s events occur in a more PLAUSIBLE ORDER?

  * •

Which story’s sentences MAKE MORE SENSE given sentences before and after them?

  * •

Which story better follows a SINGLE PLOT?

  * •

Which story is of HIGHER QUALITY?

  * •

Which story is more ENJOYABLE?

The first three questions ask about different aspects of perceived story
coherence.

Figure 2: The entropy indices for human-written stories, EDGAR, and backwards BART (bBART). Lower is better.  Question | EDGAR | bBART |  $p$-value
---|---|---|---
Plausible | 31 | 15 | 0.013
Single plot | 32 | 14 | 0.005
Makes sense | 29 | 17 | 0.052
Quality | 31 | 15 | 0.013
Enjoyable | 31 | 15 | 0.013
Table 2: Total counts of times per question in the subjective evaluation that
participants selected a story generated by each system. $P$-tests were
determined to ensure that the chance of EDGAR winning a pairing was greater
than 50/50.

###  5.1 Method

We used the same stories from the first evaluation and the same baselines.
Participants read two stories from two different sources back-to-back. Then
for that pair of stories, the participant was asked to answer the subjective
questions above, picking between the two stories.

We recruited 48 human-subject participants from Mechanical Turk. Participants
were recruited from countries that speak English as the national language.
Each participant was asked to read 4 stories, presented in pairs of two, and
answer the 5 questions after each story. Participants were paid $5 for a 10
minute task. We screened participants by asking them similar questions about
human written stories but inserted the answers to the questions in the
directions, to determine their attentiveness. Participants that were
considered inattentive where disqualified.

###  5.2 Results

The results are summarized in Table 2, which shows the number of times, per
question, a participant selected the story from each system. When forced to
pick between stories generated by EDGAR and stories generated by Backward-
BART, participants chose stories generated by EDGAR twice as often for every
question asked. A one-tailed binomial $p$-test for the results of each
question determines EDGAR was significantly preferred above the baseline for
every dimension at $p<=0.013$ except the “Makes sense” dimension, which was
significant at $p=0.052$. These results suggest that EDGAR generates more
coherent and overall better quality stories than Backward-BART. These results
are consistent with the Entropy Index metric, confirming that the metric is
also measuring coherence.

##  6 Conclusions

We propose a new approach to neural story generation that treats story
generation as question-answering problem—given an ending, the story must
answer the question of how the ending comes about. Our proposed EDGAR system
generates backward from the ending event to ensure the presence of the desired
ending. It decomposes the generation process into distinct processes for using
human commonsense to produce questions and then to answer them. These
processes are grounded in reader narrative comprehension. We show that stories
generated by EDGAR are more coherent than stories generated in a more
conventional language modeling approach based on subjective and objective
measures of perceived coherence. The EDGAR technique is a significant
departure from techniques that sample from a language model that opens up new
avenues for improving neural story generation in ways that are inspired by the
comprehension needs of the human reader.

## References

  * Albrecht and O’Brien (1993) Albrecht, J. E.; and O’Brien, E. J. 1993.  Updating a mental model: Maintaining both local and global coherence.  _Journal of experimental psychology: Learning, memory, and cognition_ , 19(5): 1061.
  * Ammanabrolu et al. (2020a) Ammanabrolu, P.; Cheung, W.; Broniec, W.; and Riedl, M. O. 2020a.  Automated Storytelling via Causal, Commonsense Plot Ordering.  _arXiv preprint arXiv:2009.00829_.
  * Ammanabrolu et al. (2020b) Ammanabrolu, P.; Tien, E.; Cheung, W.; Luo, Z.; Ma, W.; Martin, L. J.; and Riedl, M. O. 2020b.  Story Realization: Expanding Plot Events into Sentences.  In _AAAI Conference on Artificial Intelligence_ , 8.
  * Bosselut et al. (2019) Bosselut, A.; Rashkin, H.; Sap, M.; Malaviya, C.; Celikyilmaz, A.; and Choi, Y. 2019\.  COMET: Commonsense transformers for automatic knowledge graph construction.  _arXiv preprint arXiv:1906.05317_.
  * Cardona-Rivera et al. (2016) Cardona-Rivera, R. E.; Price, T. W.; Winer, D. R.; and Young, R. M. 2016.  Question Answering in the Context of StoriesGenerated by Computers.  _Advances in Cognitive Systems_ , 4: 227–245.
  * Carney (2019) Carney, J. 2019.  _Necessary Fictions: Supernormal Cues, Complex Cognition, and the Nature of Fictional Narrative_ , 391–413.  University of Nebraska Press.
  * Castricato et al. (2021a) Castricato, L.; Biderman, S.; Cardona-Rivera, R. E.; and Thue, D. 2021a.  Towards a Formal Model of Narratives.  In _Proceedings of the 3rd Workshop on Narrative Understanding_.
  * Castricato et al. (2021b) Castricato, L.; Frazier, S.; Balloch, J.; and Riedl, M. O. 2021b.  Fabula Entropy Indexing: Objective Measures of Story Coherence.  In _Proceedings of the 3rd Workshop on Narrative Understanding_.
  * Fan et al. (2019) Fan, A.; Jernite, Y.; Perez, E.; Grangier, D.; Weston, J.; and Auli, M. 2019.  Eli5: Long form question answering.  _arXiv preprint arXiv:1907.09190_.
  * Fan, Lewis, and Dauphin (2018) Fan, A.; Lewis, M.; and Dauphin, Y. 2018.  Hierarchical neural story generation.  _arXiv preprint arXiv:1805.04833_.
  * Fan, Lewis, and Dauphin (2019) Fan, A.; Lewis, M.; and Dauphin, Y. 2019.  Strategies for structuring story generation.  _arXiv preprint arXiv:1902.01109_.
  * Gabriel et al. (2020) Gabriel, S.; Bhagavatula, C.; Shwartz, V.; Bras, R. L.; Forbes, M.; and Choi, Y. 2020.  Paragraph-Level Commonsense Transformers with Recurrent Memory.  _arXiv preprint arXiv:2010.01486_.
  * Gervás (2009) Gervás, P. 2009.  Computational Approaches to Storytelling and Creativity.  _AI Mag._ , 30: 49–62.
  * Graesser, Lang, and Roberts (1991) Graesser, A.; Lang, K. L.; and Roberts, R. M. 1991.  Question Answering in the Context of Stories.  _Journal of Experimental Psychology: General_ , 120(3): 254–277.
  * Khalifa, Barros, and Togelius (2017) Khalifa, A.; Barros, G. A.; and Togelius, J. 2017.  Deeptingle.  _arXiv preprint arXiv:1705.03557_.
  * Lewis et al. (2019) Lewis, M.; Liu, Y.; Goyal, N.; Ghazvininejad, M.; Mohamed, A.; Levy, O.; Stoyanov, V.; and Zettlemoyer, L. 2019.  Bart: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension.  _arXiv preprint arXiv:1910.13461_.
  * Li et al. (2012) Li, B.; Lee-Urban, S.; Appling, D. S.; and Riedl, M. O. 2012.  Crowdsourcing Narrative Intelligence.  _Advances in Cognitive Systems_ , 2: 25–42.
  * Liu et al. (2019) Liu, Y.; Ott, M.; Goyal, N.; Du, J.; Joshi, M.; Chen, D.; Levy, O.; Lewis, M.; Zettlemoyer, L.; and Stoyanov, V. 2019.  Roberta: A robustly optimized bert pretraining approach.  _arXiv preprint arXiv:1907.11692_.
  * Martin et al. (2018) Martin, L.; Ammanabrolu, P.; Wang, X.; Hancock, W.; Singh, S.; Harrison, B.; and Riedl, M. 2018.  Event representations for automated story generation with deep neural nets.  In _Proceedings of the AAAI Conference on Artificial Intelligence_ , volume 32.
  * Mostafazadeh et al. (2016) Mostafazadeh, N.; Chambers, N.; He, X.; Parikh, D.; Batra, D.; Vanderwende, L.; Kohli, P.; and Allen, J. F. 2016.  A Corpus and Evaluation Framework for Deeper Understanding of Commonsense Stories.  _CoRR_ , abs/1604.01696.
  * O’Brien and Albrecht (1992) O’Brien, E. J.; and Albrecht, J. E. 1992.  Comprehension strategies in the development of a mental model.  _Journal of experimental psychology: learning, memory, and cognition_ , 18(4): 777.
  * Peng et al. (2021) Peng, X.; Li, S.; Wiegreffe, S.; and Riedl, M. O. 2021.  Inferring the Reader: Guiding Automated Story Generation with Commonsense Reasoning.  In _Proceedings of the 3rd Workshop on Narrative Understanding_.
  * Purdy et al. (2018) Purdy, C.; Wang, X.; He, L.; and Riedl, M. 2018.  Predicting generated story quality with quantitative measures.  In _Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment_ , volume 14.
  * Radford et al. (2019) Radford, A.; Wu, J.; Child, R.; Luan, D.; Amodei, D.; and Sutskever, I. 2019.  Language models are unsupervised multitask learners.  _OpenAI blog_ , 1(8): 9.
  * Rashkin et al. (2020) Rashkin, H.; Celikyilmaz, A.; Choi, Y.; and Gao, J. 2020.  PlotMachines: Outline-Conditioned Generation with Dynamic Plot State Tracking.  _arXiv preprint arXiv:2004.14967_.
  * Riedl and Young (2010) Riedl, M. O.; and Young, R. M. 2010.  Narrative planning: Balancing plot and character.  _Journal of Artificial Intelligence Research_ , 39: 217–268.
  * Roemmele (2016) Roemmele, M. 2016.  Writing stories with help from recurrent neural networks.  In _Proceedings of the AAAI Conference on Artificial Intelligence_.
  * Tambwekar et al. (2018) Tambwekar, P.; Dhuliawala, M.; Martin, L. J.; Mehta, A.; Harrison, B.; and Riedl, M. O. 2018.  Controllable Neural Story Plot Generation via Reinforcement Learning.  _arXiv preprint arXiv:1809.10736_.
  * Trabasso and Van Den Broek (1985) Trabasso, T.; and Van Den Broek, P. 1985.  Causal thinking and the representation of narrative events.  _Journal of memory and language_ , 24(5): 612–630.
  * Ware and Siler (2021) Ware, S.; and Siler, C. 2021.  Sabre: A Narrative Planner Supporting Intention and Deep Theory of Mind.  In _Proceedings of the 17th AAAI International Conference on Artificial Intelligence and Interactive Digital Entertainment_.
  * Ware and Young (2010) Ware, S. G.; and Young, R. M. 2010.  Modeling Narrative Conflict to Generate Interesting Stories.  In _AIIDE_.
  * Yao et al. (2019) Yao, L.; Peng, N.; Weischedel, R.; Knight, K.; Zhao, D.; and Yan, R. 2019.  Plan-and-write: Towards better automatic storytelling.  In _Proceedings of the AAAI Conference on AI_.

##  Appendix A Appendix

###  A.1 Horizon Regularization

Repetition penalty accumulates penalty over tokens that the language model has
already written. Similarly, n-gram penalties prevent a language model from
repeating a string of tokens longer than n. Both of these methods rely on the
language model having generated the tokens given the current prompt. Therefore
to avoid repetition between prompts, we accumulate error from previous
iterations by passing previously generated text as a horizon. That is to say,
it can influence the penalties but cannot be used as part of the prompt.
Horizon regularization was applied to both backwards BART and EDGAR.

###  A.2 Context Documents

Context was pulled from flash fiction online.
444https://www.flashfictiononline.com Scripts for scraping plus the dataset
will be released upon publication. Flash fiction online was chosen as the
stories are roughly the same length as thought found in the science fiction
summary corpus, but require much higher reading comprehension skills. By
comparison, the science fiction corpus is only marginally harder to follow
than ROC.

###  A.3 Experiment details

The only training that was performed during the completion of this project was
that of the ranker and the baseline. No hyper parameter search was performed,
all values were the default values provided by Huggingface’s Transformer
library.

For inference, we chose a high repetition penalty of 10.0 for both bBART and
EDGAR. Similarly, a length penalty of 3.0 and max_length of $150$ was used.

###  A.4 Release of code

We will release our code publicly upon paper decision, but currently it uses
internal code and models that are not ready for release as they relate to
separate unreleased projects.

###  A.5 T/F templates

  * •

$E_{j}$ depends on $E_{i}$.

  * •

$E_{i}$ happening after $E_{j}$ would prevent $E_{k}$.

  * •

Without $E_{i}$, $E_{k}$ would have happened before $E_{2}$.

  * •

Is object $O$ mentioned explicitly or implicitly before $E_{2}$.

  * •

$E_{j}$ contradicts assertions in $E_{i}$.

  * •

$E_{i}$ depends on $E_{j}$.

  * •

Character $C$ could be removed and the story would still make sense.

  * •

$E_{i}$ could be removed and the story would still make sense.

  * •

$E_{i},\ldots,E_{k}$ can be removed and the story would still make sense.

  * •

$E_{i}$ could be removed and the story would still make sense.

  * •

$E_{i}$ changes some property of object $O$/character $C$.

where $i<j<k$, $E$ refers to an event, $O$ refers to an object, and $C$ refers
to a character.

###  A.6 Story Example: EDGAR

The first bar is tied to the end of a rope running up the side of the tower
while the second is tied to a pulley system connected to the elevator shaft.
So when Prince Albert climbs down, he uses one of the rails to get to the
platform where the elevator shaft connects to the rest of the building.
When he comes back down, he takes the ropes and ties them to the ends of the
pulley system instead of using the entire length of the staircase.
When he went to the top of the tower, he climbed down from the roof onto the
steps at the base of the stairs leading to the main stairway.
He then took off his helmet and set himself on the second floor.
From there, he had to climb all the way back down to the bottom of the stairs
before he could take off again.
She doesn’t know how to find it, so she can’t even make a phone call to ask
for help.
It’s possible that there are people who have access to the tower but they
don’t want to tell anyone because they don’t think it’s worth their time or
money.
There was a princess with long hair locked in a tower far away.
A prince came to save her by climbing her hair and taking her out of the
tower.

If it does not have a pulse then it is dead.
The heart rate of an animal tells them if it is alive or dead.
There is no such thing as absolute death.
He is able to tell the difference between something being alive and something
being dead, so when he looks at the house, he feels
like he’s seeing things that aren’t there.
This makes him feel uncomfortable because he doesn’t want to be in that
situation.
It’s similar to how people can see ghosts or monsters from inside their head
but they don’t know what those things are.
Hansel’s hand still trembles as he pushes open the twice-cooked door.
The last time he saw the house he was glancing back over his shoulder as he
and his sister fled into the trees.

The sparrow is watching him.
It’s perched on the barrow fence, a tousle of feathers and watchful birds and
watch their prey.
He’s worried that the sparrow is listening to him.
Hansel’s hand still trembles as he pushes open the twice-cooked door.
The last time he saw the house he was glancing back over his shoulder as he
and his sister fled into the trees.

“She’s dead.”
He doesn’t know how to get out of the house because he isn’t aware of anything
else happening.
However, if someone were to come up to him and say “You should go home,” he
would have no choice but to run away from them.
He wants to get out of that creepy house.
The only way for him to do this is to push open the door once it’s been
cooked.
This would be a terrible thing to do, so he shudders in anticipation of doing
it.
Hansel’s hand still trembles as he pushes open the twice-cooked door.
The last time he saw the house he was glancing back over his shoulder as he
and his sister fled into the trees.

He doesn’t seem to be able to control his emotions, but that isn’t always the
case.
The knife may have been used by him in a momentary panic or just because he
felt like pushing through the frosted gingerbread girders on the other side of
the door.
It might have been one of those moments where he didn’t know what else to do
with his hands.
But then again, there are plenty of people who think they can handle things
without any kind of mental breakdown.
Hansel’s hand still trembles as he pushes open the twice-cooked door.
The last time he saw the house he was glancing back over his shoulder as he
and his sister fled into the trees.

She would say things like “I’m tired!” or “It’s been a long day!” when she
wasn’t really tired.
It’s kind of hard to imagine what being able to do both at once feels like.
The Princess didn’t have time to see the world, she had no time to see it at
all.
In order to experience the world there needs to be some sort of barrier
between herself and everything else.
There are many ways to do this without going too far into detail, but that’s
the general gist of it.
There was a princess with long hair locked in a tower far away.
A prince came to save her by climbing her hair and taking her out of the
tower.

It made sense that he’d have a strong sense of unease about his mother’s
safety.
It was a very strange experience for a young child to have, and it made him
think about how he’d felt before, which he was very conscious of.
He was raised in a home where his mother was the only person who could give a
damn.
The only thing Hansel had to do was escape.
Hansel’s hand still trembles as he pushes open the twice-cooked door.
The last time he saw the house he was glancing back over his shoulder as he
and his sister fled into the trees.

They want a job to provide for them and their family.
He has no other option.
Hansel’s hand still trembles as he pushes open the twice-cooked door.
The last time he saw the house he was glancing back over his shoulder as he
and his sister fled into the trees.

They just don’t want to admit it.
Because they feel the urge to.
So they go somewhere else.
He has no other option.
Hansel’s hand still trembles as he pushes open the twice-cooked door.
The last time he saw the house he was glancing back over his shoulder as he
and his sister fled into the trees.

If someone says they’re going to use their teeth to open a door, then they
probably aren’t going to be able to do it very well.
But if they say they’ll use their fingers instead, then they will be able to
get the door open better.
And since people have been doing it for so long, it becomes second nature.
In order to enter the room, Hansel has to pass through a gate which is much
more difficult to open than the main door.
There are many different ways to open the door, but the most common method is
to push against the handle with one hand while using the other hand to open
the gap between the handle and the door.
This allows Hansel to walk right through the gap without having to put any
effort into opening the door.
There’s no reason for him to see it in person.
He doesn’t need to know anything about it other than that it’s there.
The only thing he needs to know is that he can’t live there anymore.
Hansel’s hand still trembles as he pushes open the twice-cooked door.
The last time he saw the house he was glancing back over his shoulder as he
and his sister fled into the trees.

###  A.7 Story Example: Backwards BART

In this universe, humans are forced to deal with one another by creating
monsters that can destroy everything they touch or even kill themselves. It’s
all about them being human instead of just living on planet Earth.In this
universe, Humans are forced to decide between two different worlds by creating
monsters which can destroy everything you touch or even die themselves.
It’s all about them be human rather than just living on Planet Earth. A battle
has been fought between two monstrous creatures who have come. A battle has
broken out between two monstrous beings who have come together to fight each
other for their freedom.
On Earth, a powerful creature called the Nymph emerges from its hiding place
in an open forest. A battle has broke out between two huge beings who have
came together to fight against each other for Their freedom.
On earth, a powerful monster called the Nymph emerges from his hiding place in
An open forest.
The hero charged at it, both disappearing Into the void. On Earth The hero
charged at the dragon, both disappearing into the void. The world is saved and
people will not forget the sacrifice of the nameless hero.

A little while ago, I got home from work having just finished reading one of
my books. As usual, I didn’t sleep well.
A little bit ago, I get home from work has just finished reading another book.
As always, I don’t sleep well either. The night Before, I was sleeping
peacefully.
They were eating breakfast together. This morning, I was sitting quietly. The
night before – I was playing Chess with my friends In their apartment.They
were eating breakfast.
The night before, I was playing chess with my friends in their apartment. They
were watching me play chess on TV.The night before, they watched me play chess
with my friend in their apartment building.
It was very strange to watch them play chess with each other. I woke up
feeling like an idiot. I thought it was because of some unknown reason.I woke
up feeling as though an alien had come into my apartment building.
They were viewing me playing chess with eachother. I woke up realizing that I
became a cat. I caught a mouse and at the next moment I realized that I’m a
human again!

In 1989, two women who have been married since they were 5 years old wrestled
together into one team. One man is now famous for his ability to beat men -
even if he can’t do it all themselves.
In 1989, two female wrestlers whom have been married throughout their lives
wrestled together onto one team.One man is now known for his abilities to
defeat men - even though he can’t win them all himself.
A young girl has been married throughout both worlds. A A young girl from
China was born and brought up on an orphanage whose parents were forced to let
her go by their husbands’ demands.
She grew up watching sports at home with her mother before becoming a wrestler
herself.A young girl from Chinese was born and raised on an orphanages whose
parents had turned against her fathers’ demands.
She grew up watching games at home without her mother’s demands. She became a
wrestler herself after becoming a wrestler himself. A woman broke through-
stereotype
A woman broke through the stereotype of not being able to wrestle. With her
father as a strict coach, she won a gold medal for wrestling in India,
fulfilling her father’s dream.

The house is derelict and crumbling now, but that doesn’t mean it isn’t still
full of ghosts!
That’s just not true at all. The fact that he can imagine himself doing it
makes him feel more comfortable with what he’s doing
If he didn’t enjoy it, he wouldn’t have done it.
The first thing is that he had to do it because he was afraid of being kicked
out by his brother. And the second thing is that he wanted to see Jemim’s room
as much as possible before going to bed. So he took a moment to check for any
signs of danger.
He turned out the light and went into Jem’s room. He would be there all night,
and he would be there when Jem waked up in the morning. But little did Travis
know, Jem knew about this all along.

A woman named Stella Smith comes into their house in Puddle Creek on New
Year’s Eve 2012. They have just had lunch together.
A woman named StellaSmith comes into their home in Puddle Rapids on New Year’s
Eve 2012. We have just having lunch together.
The peanut farmer will be selling peanuts for About three years now which he
wants her willing to eat some of his goods as well.We have just had breakfast
together. The Peanut farmer has already sold peanuts for. The peanut farmer
has been selling peanuts for about three years now.
He wants her to buy some of his own food as soon as she can.The peanut farmer
has started selling peanuts for around three years now.. he wants her to
purchase some of his other food as well.
Stella loves peanuts because she knows what kind of animal they are
eating.Stella loves peppers for about three weeks now. He want her to buy a
few of his own foods as soon as possible. Stella loves.
Stella loves peanuts! She goes to the market every day, and is best friends
with the owner.

A young boy named Hansel has been walking through town recently looking at
some strange things that are going on outside their walls. In fact, they were
just playing cards before anyone noticed them.
A young boy called Hansel has Been walking through town lately looking at some
odd things that are gone missing outside their walls – maybe even someone
noticing them. In reality, they were probably playing cards before everyone
noticed them. He wants to learn more about it though, but can always tell for
sure. He wants to know more about it, but can never tell for sure.
After all, this is a city of humans who have no interest in living here.He
wants to know what happened to him, but can only tell for sure if this is a
City of Humans who have little interest in living there. Hansel’s hand still
feels like an old man with no interest whatsoever. His mind wanders
frantically around trying to figure out how to live here.
Hansel’s hand still trembles as he pushes open the twice-cooked door. The last
time he saw the house he was glancing back over his shoulder as he and his
sister fled into the trees.

In another world, there were many stories about humans who have lost their
lives while trying to survive on this planet.
One man told him that they all know each other’s names.In another universe,
there were several stories about humans Who have lost their life while trying
to escape on this continent. One man tells him that they both know each
other’s names.
The story end when an old women had died from dehydration, leaving only his
dead body which is not reaching without her help. The story ended when an old
woman had died from starvation, leaving only his dead body which she could not
reach for. She tried to hide it but couldn’t find it.
The story ends when an old lady had died from hunger, leaving only His dead
body which he could not reach without her help. She tried hiding it herself
but couldn’t get it anywhere else.
There was a Princess with long hair tied in a tower near her. A prince brought
to save her through climbing her.  There was a princess with long hair locked
in a tower far away. A prince came to save her by climbing her hair and taking
her out of the tower.

.It was late December 22nd – another day of quiet for them to enjoy each
other’s company. The two men0 sat down by the fireplace drinking hot cocoa
after dinner.It was late December 23rd – another day during quiet for us to
enjoy each others company. The pair sat down by one firelight after dinner.
A man named Randy had just finished an conversation with his neighbor Janice
on New Year’s Eve. They’d been having breakfast together since midnight.They
sat down by A man named Travis had just finished a conversation with his wife
Janice on Christmas Eve.
They were having breakfast together at her house.A man named Travis has just
finished a discussion with his wifeJanice had just finished talking with his
wife Jenice on Christmas eve.
They were eating breakfast together at their house. He turned out the lights
and went into Jake’s room. There would be no night sleep, and he wouldn’t be
there when they waked up next morning.
He turned out the light and went into Jem’s room. He would be there all night,
and he would be there when Jem waked up in the morning. But little did Travis
know, Jem knew about this all along.

In this universe, humans are forced to deal with one another by creating
monsters that can destroy everything they touch or even kill themselves.
It’s all about them being human instead of just living on planet Earth.In this
universe, Humans are forced to decide between two different worlds by creating
monsters which can destroy everything you touch or even die themselves.
It’s all about them be human rather than just living on Planet Earth. A battle
has been fought between two monstrous creatures who have come.
A battle has broken out between two monstrous beings who have come together to
fight each other for their freedom.
On Earth, a powerful creature called the Nymph emerges from its hiding place
in an open forest.
A battle has broke out between two huge beings who have came together to fight
against each other for Their freedom.
On earth, a powerful monster called the Nymph emerges from his hiding place in
An open forest.
The hero charged at it, both disappearing Into the void.On Earth The hero
charged at the dragon, both disappearing into the void. The world is saved and
people will not forget the sacrifice of the nameless hero.

The last time I saw my grandfather before leaving home had been during an
argument between two people over money. His parents were very angry with him
because they didn’t want him to leave them alone.
The last time I see my grandfather before going away had been during another
argument between two men over money. My parents were very upset with him
because we didn’t want anyone else to leave them together.
A young man called after him is telling me how much better their relationship
was than theirs. A young man named after him is trying to kill his father for
being too strong of a woman.
He has no idea who he is or what he really does and that he doesn’t like about
it. A young man named After him is attempting to kill his dad for being too
weak of a woman’s strength and that he shouldn’t like about this.
He has nothing knows who he really is and that he don’t like about anything.
An evil queen wanted at least one thing from An evil queen wanted to take
revenge on the prettiest girl in the land. She gave her a poison apple, but
the girl was saved by a prince.

###  A.8 Story Example: ROC

I got Charlie Horse when I was four years old.
He’s a brown stuffed horse, and at 35 I still sleep with him at night.
He was my best friend, and always laid at the head of my bed.
I laid him next to me, smelling his soft fur every night.
I liked to listen to my radio as I fell asleep cuddling him.


# Gradient Mask: Lateral Inhibition Mechanism Improves Performance in
Artificial Neural Networks

Lei Jiang
Lomonosov Moscow State University
<EMAIL_ADDRESS>These authors contributed equally to this work. Yongqing
Liu 11footnotemark: 1
Lomonosov Moscow State University
<EMAIL_ADDRESS>Shihai Xiao
Huawei Technologies Co., Ltd.
<EMAIL_ADDRESS>Yansong Chua
China Nanhu Academy of Electronics and Information Technology
<EMAIL_ADDRESS>Corresponding author

###### Abstract

Lateral inhibitory connections have been observed in the cortex of the
biological brain, and has been extensively studied in terms of its role in
cognitive functions. However, in the vanilla version of backpropagation in
deep learning, all gradients (which can be understood to comprise of both
signal and noise gradients) flow through the network during weight updates.
This may lead to overfitting. In this work, inspired by biological lateral
inhibition, we propose Gradient Mask, which effectively filters out noise
gradients in the process of backpropagation. This allows the learned feature
information to be more intensively stored in the network while filtering out
noisy or unimportant features. Furthermore, we demonstrate analytically how
lateral inhibition in artificial neural networks improves the quality of
propagated gradients. A new criterion for gradient quality is proposed which
can be used as a measure during training of various convolutional neural
networks (CNNs). Finally, we conduct several different experiments to study
how Gradient Mask improves the performance of the network both quantitatively
and qualitatively. Quantitatively, accuracy in the original CNN architecture,
accuracy after pruning, and accuracy after adversarial attacks have shown
improvements. Qualitatively, the CNN trained using Gradient Mask has developed
saliency maps that focus primarily on the object of interest, which is useful
for data augmentation and network interpretability.

##  1 Introduction

One may largely divide the discussions on the connection between neuroscience
and artificial neural networks (ANN) into two different school of thoughts:
ANN has achieved significant results, many even beyond human performance, with
minimal contribution from neuroscience and should proceed as such; another
being that, there is a wealth of knowledge in neuroscience that ANN or AI in
general can learn from.

On the one hand, one may argue that ANNs were inspired by neuroscience,
notable examples could be listed as such: inspired by neuronal working
mechanism, the early neural network model was developed [30, 38]; inspired by
biological memory storage, Hopfield networks were proposed [22]; the working
mechanism of convolutional neural networks (CNNs) is often considered to be
similar to the visual receptive field [26]; based on the firing pattern of
biological neurons, spiking neural networks (SNNs) were proposed [8, 41, 34];
the continuous-attractor neural network (CANN) was inspired by neuronal
network dynamics [3, 5]; the cortical minicolumn influenced the design of
CapsuleNet [21, 39]; predictive coding mechanism [37] was applied to ANNs in
many practical problems [29, 48, 50], and work on contrastive predictive
coding [33] has led to current trend of unsupervised contrastive learning[20,
18, 10].

On the other hand, researchers have used neuro-scientific findings to
corroborate and explain the effectiveness of ANNs: [35, 27] argue for the
biological plausibility of back-propagation in the biological brain. In [4],
researchers identified a number of neurons sharing grid cell activation
characteristics in LSTM networks. OpenAI has also discovered multimodal
neurons in ANNs in its recent work [36, 16].

We have referred to several brain-inspired mechanisms that enhance the
performance of ANNs, our work falls in this category.

Research in neuroscience has shown that inhibitory circuitry may play an
important role in associative LTD (Long-term depression) by ensuring that a
postsynaptic cell is inhibited when certain inputs to that cell are active
[31]. And long-lasting changes in the efficacy of synaptic connections between
two neurons can involve the making and breaking of synaptic contacts [42],
which is an integral part in the ability of an organism to learn [11]. As a
kind of inhibitory mechanism, lateral inhibition (LI) affects the distribution
of the attention field by amplifying the contrast between strong and weak
stimuli (see Figure 1(a)). This effect can be described by a Mexican hat
function (see Figure 1(b)). In our model, we use the Laplacian of Gaussian
(LoG) operator to approximate this function - highlighting the signal and
reducing the noise.

During training of ANN, we hope to inhibit non-critical features to highlight
key features. Therefore, we introduce the lateral inhibition (LI) mechanism in
ANN training to calculate the importance distribution of the gradients on
feature maps, which leads to a new training method Gradient Mask — filtering
out the noise gradients, so that only the important gradients could be passed
in the backpropagation. The rationality of this method is not only
theoretically intuitive (see Section 2.2) but also demonstrated in several
experiments (see Section 3.2 and 3.3.3).

The effectiveness of the Gradient Mask method is reflected in the following:
Firstly, better classification accuracy is obtained by training the network
with Gradient Mask (see Section 3.1). Secondly, the training in which the
weights are updated by the most relevant gradients leads to a more robust
network. On one hand, this method reduces the impact of the unimportant
features (as demonstrated by adversarial experiments, see Section 3.3.2), on
the other hand, the signals are packed into a smaller subnet which is sparser
but more decisive for learning. (as demonstrated by resistance to pruning, see
Section 3.3.1). In addition, a new data enhancement method by Gradient Mask is
proposed, in which we detect the background of the image in real time, and
generate new training data by some operations (Gaussian blur, etc.) (see
Section 3.4).

(a) Lateral inhibition process

(b) Mexican-hat like distribution

Figure 1: In figure 1(a), the contrast between strong and weak stimuli is
amplified after lateral inhibition; In figure 1(b), The Mexican-hat like
distribution reflects the effects of LI.

##  2 Methods

During backpropagation, gradients are generated for all elements the feature
map, these then work together to update the corresponding weights. However,
previous work [25] showed that not every gradient is critical for training, so
the importance of the gradient needs to be measured. In order to achieve this
goal, we implement the LI mechanism with Laplacian of Gaussian (LoG) operator
in the backpropagation process of ANN, because the LI mechanism makes the
signal distributed like a Mexican hat, and the LoG operator functions
similarly. The operator assigns a value to each feature gradient as a measure
of its importance, and through theoretical intuition and experiments we show
that this measure of importance is reasonable and effective. During training
based on this metric Gradient Mask is generated to filter out unimportant
feature gradients, so that back propagation is performed in a subnet.

###  2.1 Gradient Mask

For convolutional layer $l$ with feature maps of width $u$ and height $v$, we
divide all the feature maps into $K$ sets evenly. For each set, in order to
reduce the computational complexity, the gradients at the same coordinate are
composed into a vector called minicolumn, which constitutes a fundamental
computational unit of the cerebral cortex [45, 44, 46, 32, 7, 7]. We denote
the minicolumn on the coordinates $(i,j)$ of the $k$-th set as
$M_{ij}^{l}(k)$. We calculate its $l_{2}$ norm $||M_{ij}^{l}(k)||_{2}$ to
represent the magnitude of the gradients in this minicolumn.

Then for each $k$ we apply the $LoG$ operator to the matrix composed of
$||M_{ij}^{l}(k)||_{2}$ for all $0\leq i\leq u,0\leq j\leq v$. This process is
done with $LoG$ convolution kernel,

$\displaystyle LoG(x,y)$
$\displaystyle=\frac{\partial^{2}G_{\sigma}(x,y)}{\partial
x^{2}}+\frac{\partial^{2}G_{\sigma}(x,y)}{\partial y^{2}}$ (1)
$\displaystyle=-\frac{1}{\pi\sigma^{4}}[1-\frac{x^{2}+y^{2}}{2\sigma^{2}}]e^{-\frac{x^{2}+y^{2}}{2\sigma^{2}}}$
(2)

where $(x,y)$ refers to the coordinates of the $LoG$ convolution kernel and
$G_{\sigma}(x,y)$ is the Gaussian convolution with standard deviation
$\sigma$. Let $\delta^{l}_{ij}(k)$ denote the result of the $LoG$ convolution
on the corresponding part.

By setting the threshold to $\epsilon$, we can define the set of coordinates
on which the gradient is not important,

$A^{l}(k)=\\{(i,j):|\delta^{l}_{ij}(k)|<\epsilon\\}$ (3)

Therefor we can generate the Gradient Mask: $Mask^{l}(k)=[a_{ij}]_{u\times
v}$, where

$\displaystyle a_{ij}=\left\\{\begin{array}[]{rcl}0&&,{(i,j)\in A^{l}(k)}\\\
1&&,{(i,j)\not\in A^{l}(k)}\\\ \end{array}\right.$ (6)

Since gradient mask $Mask^{l}(k)$ corresponds to the $k$-th set of the feature
maps, the filters corresponding to this set of feature maps share the same
mask $Mask^{l}(k)$. During backpropagation, each gradient passes through the
gradient mask, i.e. multiplied by the mask in an element-wise fashion, and
then continues to propagate. Let $L$ denote the loss, the gradient of the
weight $w_{mn}^{l}$ on a filter may be written as

$\displaystyle\frac{\partial L}{\partial w^{l}_{mn}}$
$\displaystyle=\frac{\partial L}{\partial a^{l}_{11}}\frac{\partial
a^{l}_{11}}{\partial w^{l}_{mn}}+\dots+\frac{\partial L}{\partial
a^{l}_{NN}}\frac{\partial a^{l}_{NN}}{\partial w^{l}_{mn}}$ (7)
$\displaystyle=\underbrace{\sum_{(i,j)\in A^{l}}\frac{\partial L}{\partial
a^{l}_{ij}}\frac{\partial a^{l}_{ij}}{\partial
w^{l}_{mn}}}_{inhibited~{}part}+\underbrace{\sum_{(i^{\prime},j^{\prime})\not\in
A^{l}}\frac{\partial L}{\partial a^{l}_{i^{\prime}j^{\prime}}}\frac{\partial
a^{l}_{i^{\prime}j^{\prime}}}{\partial w^{l}_{mn}}}_{important~{}part}$ (8)
$\displaystyle=0+\underbrace{\sum_{(i^{\prime},j^{\prime})\not\in
A^{l}}\frac{\partial L}{\partial a^{l}_{i^{\prime}j^{\prime}}}\frac{\partial
a^{l}_{i^{\prime}j^{\prime}}}{\partial w^{l}_{mn}}}_{important~{}part}$ (9)

The unimportant feature gradients are set to 0; therefore, the weight only
changes in the direction indicated by the important feature gradients, which
helps to denoise the feature information retained in the weight [2], improving
robustness of the network. This is illustrated in experiments which test the
performance of our model against adversarial attacks 3.3.2. The denoising
effect is also illustrated in the GSNR (Gradient Signal to Noise Ratio) (see
Section 3.3.3).

With gradients masked during back-propagation, given the same accuracy, one
can intuitively understand that most features are packed into a smaller subset
of useful weights. This is illustrated in the network loss of accuracy during
pruning, as discussed in section 3.3.1. Figure 2 describes how Gradient Mask
is generated and affects the backpropagation. Algorithm 1 illustrates the
entire process of our method.

Figure 2: Lateral Inhibition in neural networks. The white square represents
the activated neuron while the gray square is inhibited, and the gradient is
only flowing through the activated neurons.

###  2.2 Gradient Flux Sensitivity

In this section, we provide the theoretical intuition of the Gradient Mask,
which is to show the rationality of using $\delta^{l}_{ij}$ to measure the
importance of the feature gradients.

In backpropagation, as shown in equation (5), a weight gradient can be
decomposed into multiple terms, and each term corresponds to a coordinate on
the feature map. In order to express how significant a point on the feature
map is to a weight gradient $g^{l}_{mn}$, we calculate its laplacian along the
coordinates $(i,j)$ on the feature map:

$\Delta g_{mn}^{l}=\frac{\partial^{2}g_{mn}^{l}}{\partial
i^{2}}+\frac{\partial^{2}g_{mn}^{l}}{\partial j^{2}}$ (10)

Physically speaking, it is the divergence of the change of the gradient
$g^{l}_{mn}$ in space, and its absolute value indicates to what extent the
point $(i,j)$ is the ”source” (positive or negative source) of the gradient
$g^{l}_{mn}$ [23]. A larger source means that it is more significant and
critical for the gradient $g^{l}_{mn}$. We define it as Gradient Flux
Sensitivity $s^{l}_{mn}=|\Delta g_{mn}^{l}|$.

Let $\overline{\frac{\partial L}{\partial a^{l}_{ij}}}$ be the feature
gradient after Gaussian smoothing. So that the gradient flux sensitivity can
be rewritten as follows:

$\displaystyle s_{mn}^{l}$
$\displaystyle=\left|\Delta\sum_{i,j}(\overline{\frac{\partial L}{\partial
a^{l}_{ij}}}\frac{\partial a^{l}_{ij}}{\partial w^{l}_{mn}})\right|$ (11)
$\displaystyle=\frac{\partial a^{l}_{ij}}{\partial
w^{l}_{mn}}\left|\Delta\overline{\frac{\partial L}{\partial
a^{l}_{ij}}}\right|$ (12)

The coefficient $\frac{\partial a^{l}_{ij}}{\partial w^{l}_{mn}}$ is the
derivative of the activation function given the input of the neuron, which can
be regarded as a constant, and is non-negative since ReLU is the activation
function. So the element of the Gradient Mask
$\delta^{l}_{ij}=\left|\Delta\overline{\frac{\partial L}{\partial
a^{l}_{ij}}}\right|$ is positively related to $s_{mn}^{l}$, therefore
$\delta^{l}_{ij}$ can measure the importance of feature gradients to a weight
gradient.

Algorithm 1 Generation of Gradient Masks for one sample in layer $l$

1:Tensor of gradients in layer $l$, Quantile, Sigma, Kernel size, Number of
channels, Number of sets.

2:Initialize $G^{l}$ = Tensor of gradients in layer $l$, $q$ = Quantile,
$\sigma$ = Sigma, $s$ = Kernel size, $C$ = Number of channels , $K$ = Number
of sets. Let $l_{c}$ be the $c$-th feature map of layer $l$,  $\displaystyle
G^{l}=[\frac{\partial{L}}{\partial{a_{ij}^{l_{c}}}}]_{C\times u\times v}$ (13)

3:Divide gradients into minicolumns:  $\displaystyle
M^{l}_{ij}(k)=[\frac{\partial{L}}{\partial{a_{ij}^{l_{c^{\prime}}}}}]_{\frac{C}{K}\times
1}$ (14) and compute $l_{2}$ norm of each minicolumn: $||M^{l}_{i,j}(k)||_{2}$
to get $K$ matrices:  $\displaystyle D^{l}(k)=[||M^{l}_{ij}(k)||_{2}]_{u\times
v},$ (15)

4:Perform $LoG$ convolution on each matrix $D^{l}(k)$ with given parameters
$\sigma$ and $s$:  $\displaystyle\delta_{ij}^{l}(k)=LoG_{\sigma,s}(D^{l}(k))$
(16)

5:Calculate threshold value $Q_{k}^{l}$ with given parameter quantile $q$:
$\displaystyle Q^{l}_{k}=Threshold({[\delta_{ij}^{l}(k)]_{u\times v},q)}$ (17)

6:Generate Gradient Masks:  $\displaystyle
Mask^{l}(k)=[\mathbb{I}_{|\delta^{l}_{ij}(k)|>Q_{k}}]_{u\times v}$ (18)

7:Update the gradients on each feature map $c$ with the Gradient Mask:
$\displaystyle\overline{G_{c}^{l}}=[\frac{\partial{L}}{\partial{a_{ij}^{l_{c}}}}]_{u\times
v}\circ Mask^{l}(k)|_{k\in\\{k|l_{c}\in set(k)\\}}$ (19)

8:Continue backpropagation with
$\overline{G^{l}}=[\overline{G^{l}_{c}}]_{C\times 1}$.

##  3 Experiments

Training with Gradient Mask, our model achieved higher accuracy on ImageNet
[12] and CIFAR-100 [24] datasets. And In order to visualize the effect of the
Gradient Mask method, we use the LoG operator to generate saliency maps, and
compare them with the saliency maps generated by other methods like Grad-
CAM[40], verifying that this method is able to more accurately capture key
features.

The network trained with Gradient Mask has also improved in robustness and
generalization ability, which comes from the accumulation of signals and the
inhibition of noise:

  * •

In terms of signal, the pruning experiment shows that the model can be pruned
more while maintaining accuracy, indicating that the key gradients are indeed
packed into a sparser sub-network;

  * •

In terms of noise, adversarial experiments show that the noise gradients are
less learned by the model.

  * •

Furthermore, we quantitatively demonstrate the relative enhancement of the
signal and the reduction of the noise by calculating the Gradient Signal to
Noise Ratio (GSNR) [28].

Finally, applying Lateral Inhibition, we propose a new data enhancement scheme
that can detect and blur unimportant parts of the images in real time.

###  3.1 Training with Gradient Mask

To fairly compare the performance of the original network and the network
trained with Gradient Mask, we use ResNet[19] as the experimental framework
and apply the same hyperparameters and data enhancement strategy. In order to
generate the Gradient Mask, we set additional hyperparameters $\sigma=11$,
$K=16$, and the quantile(inhibition rate) at $50\%$. We apply Gradient Mask on
the output layer of every bottleneck except the last two bottlenecks, which
contain rich semantic information, and too much inhibition would lead to
degradation of network performance. Although using the same learning rate as
the original network is not the optimal choice for the Gradient Masked
network, as the latter only updates the weights in its sub-network, but in
order to make a fair comparison, we use the same hyperparameters as the
original network. We conducted classification experiments on ImageNet (with 8
Tesla V100 GPUs) and CIFAR-100 (with 1 Tesla V100 GPU) respectively, and the
experiments showe that using Gradient Mask improves classification accuracy.
Table 1 shows the result of ResNet with/without gradient masks in the two
datasets.

Model | CIFAR-100 | ImageNet
---|---|---
Normal ResNet-18/50 | 78.2 | 75.5
Masked ResNet-18/50 | 80.26 | 76.01

Table 1: The top-1 accuracy (%) of ResNet-18 with/without masks on CIFAR-100
and ResNet-50 with/without mask on ImageNet

In order to investigate whether LI and minicolumns are crucial for Gradient
Mask to work, two experiments are conducted: in the first experiment, after
slicing the pixel channels into minicolumns, we perform an $l_{2}$ norm
operation on each minicolumn, and we took the quantile directly from the norm
results to decide which minicolumns should be inhibited/activated (without
LI); in the second experiment, we do not aggregate neurons as minicolumn, we
take the absolute value of each pixel on each feature map, and then perform
$LoG$ on the feature map (without minicolumn).

Model | Masked ResNet-18 | Without LI | Without Minicolumn
---|---|---|---
Accuracy | 80.26 | 75.86 | 78.97

Table 2: The top-1 accuracy (%) of ResNet-18 with and without LI and
Minicolumn on CIFAR-100

We use ResNet-18 and the same hyperparameters to conduct the experiment in
CIFAR-100. The results are shown in Table 2 and Figure 3, we show that $LoG$
is crucial for the generation of Gradient Mask - Without LI reduces the
accuracy significantly. The accuracy of Without Minicolumn is lower than
Masked ResNet-18, and its training speed is 3 times slower than Masked
ResNet-18 (more masks need to be generated).

Figure 3: Training with and without LoG and Minicolumn in CIFAR-100 dataset.

###  3.2 Saliency Map

Saliency map highlights portion of the image that contributes to a
classification decision, hence it is an effective tool for the interpretation
of convolutional neural networks (CNN). In recent years, various saliency
detection methods have been proposed: Guided Backpropagation(GBP) [43], Class
Activation Mapping(CAM)[52], Grad-CAM[40], Grad-CAM++[9] and so on. Although
some saliency detection methods [43] are able to generate the saliency map,
they are independent of the model and data generation process, and may not
best explain the relationship between the inputs and outputs of the model
during learning or to debug the model [1].

We used the LI method on a CNN to generate better saliency maps, which we
named as LI-Map. Additional experiments were conducted to show that the
saliency map generated with LI pass the Cascading Randomization test [1] (See
Appendix A), confirming that the generated saliency map is helpful for
interpretability of the network.

For every activation layer $A^{l}$, we apply step 1-3 from Algorithm 1 to get
$\delta^{l}$. Notice that: 1) Replace loss $L$ with $P$, where $P$ is the
prediction of interest; 2) Set $K=1$, i.e. each pixel channel can be seen as
one minicolumn. Then resize $\delta^{l}$ from $u\times v$ to $H\times W$,
where $H$ and $W$ are the height and width of the input image:

$\displaystyle\delta^{l}=LoG_{\sigma,s}(D^{l})$ (20)
$\displaystyle\delta^{l}\in\mathbb{R}^{u\times
v}\rightarrow\delta^{l}\in\mathbb{R}^{H\times W}$ (21)

Finally, in order to combine information from each layer, we sum the
$\delta^{l}$ of all activation layers to get the saliency map $F$:

$\displaystyle F=\sum_{l=1}^{t}\delta^{l},\quad l\in[1,t]$ (22)

where $t$ is the number of activation layers. Figure 4 shows our results
compared with Grad-CAM. We can see that the saliency maps generated by LI-Map
are more accurately focused on the target objects.

Figure 4: Saliency maps generated by LI-Map and Grad-CAM. Note that the
saliency maps generated by LI-Map are more accurately focused on the target
objects.

With the aim of quantitatively comparing the object detection ability of
various saliency map methods, we propose a measure based on Intersection over
Union (IoU) scores. We select 15% of pixels with the largest F values (F is
calculated by Equation 22) in the saliency map generated by the different
methods as the region of interest. The smallest and largest coordinate points
of these pixels are used to form a rectangular bounding box and the IoU score
is calculated with the real bounding box where the target is located. As shown
in Table 3, we verify the superiority of our method on the PASCAL VOC 2007
[15] and ImageNet datasets.

Dataset | Grad-CAM | Grad-CAM++ | LI-Map
---|---|---|---
PASCAL VOC 2007 | 0.44 | 0.45 | 0.5
ImageNet | 0.46 | 0.46 | 0.49

Table 3: The IoU results on PASCAL VOC 2007 and ImageNet

In order to further understand the impact of using Gradient Mask, we visualize
different saliency maps trained with no mask, up to one mask per image, see
Appendix B.

###  3.3 Robustness and Generalization performance

####  3.3.1 Network Pruning

To confirm that the network trained with Gradient Mask has the ability to
intensively learn features into a sparser subnet, we trained a GM-ResNet-50
using Gradient Mask, pruned it with L1 (we pruned a certain percentage of the
weights with the smallest absolute value between layers or globally to observe
the performance of the network) and compared it with a normally trained
ResNet-50. As shown in Figure 5, the network with Gradient Mask has better
performance after pruning.

Figure 5: The LI-based network has better performance after pruning

####  3.3.2 Adversarial Attack

In order to test the robustness of the network trained with Gradient Mask, we
conducted adversarial attacks on it. We apply Fast Gradient Sign Attack (FGSM)
[17] on both models (GM-ResNet-50 and normal ResNet-50). The dataset comprises
of images from ImageNet validation set that can be correctly predicted by both
models. As shown in Figure 6, model trained with Gradient Mask is more robust
against adversarial attacks.

Figure 6: The LI-based network has better capabilities against adversarial
attacks compared with ordinary networks.

####  3.3.3 Gradient Mask leads to better GSNR

In order to test the performance of Gradient Mask in filtering noise gradient,
we use the Gradient Signal to Noise Ratio (GSNR) [28] that has been shown to
quantify network generalizability. The GSNR of a model parameter $\theta_{j}$
is defined as:

$r\left(\theta_{j}\right):=\frac{\tilde{\mathbf{g}}^{2}\left(\theta_{j}\right)}{\rho^{2}\left(\theta_{j}\right)}$

where $\tilde{\mathbf{g}}\left(\theta_{j}\right)$,
$\rho\left(\theta_{j}\right)$ are mean and variance of $\theta_{j}$ in the
iteration, respectively. In our experiment, we compare the GSNR of
convolutional layers between normal training and training with Gradient Mask
in CIFAR-10 [24]. Figure LABEL:gsnr shows that Gradient Mask can improve GSNR
of the model during training, indicating training with Gradient Mask can
improve the generalization of the network.

###  3.4 Data Enhancement

Data enhancement is an effective method to improve the generalization and
robustness of a neural network. Previous works [51, 49] demonstrated that
context and background will greatly affect the performance of the network and
the network can easily overfit to the background during training. As Section
3.2 shown, LI can help to locate the target of interest in real time, so that
we can insert noise in the non-target area (background or other objects) for
data enhancement.

Since no annotation was found for the ImageNet validation set, we selected
100,000 images with bounding box from 1.28 million images as the validation
set, and retrained the ResNet-50 on the rest 1.18 million images. Figure 8
shows the examples from the validation dataset, in which sigma is the blur
ratio. Based on the retrained ResNet-50, we continued to train the model using
data augmentation for 30 epochs, tested it on 100,000 images with bounding box
and Gaussian blur, and compared it with the original pre-trained model. The
algorithm is described in Algorithm 2. Examples of enhanced images are shown
in Figure 9, we can observe that the irrelevant information of the target
object in the image is well covered.

Figure 8: Examples from validation dataset Algorithm 2 Data Enhancement with
Gradient Mask

1:Input tensor with shape $N\times 3\times H\times W$, Quantile, Sigma, Kernel
size, Set of activation layers

2:Output tensor with shape $N\times 3\times H\times W$.

3:Initialize $I$ = Input tensor, $q$ = Quantile, $\sigma$ = Sigma, $s$ =
Kernel size, $K$ = Number of Sets = 1, $A=\\{A^{1},A^{2},\ldots,A^{t}\\}$ =
Set of activation layers.

4:For every activation layer $A^{l}$, $1\leq l\leq t$ in $A$, apply steps 1-3
in algorithm 1. Then Use linear interpolation to scale $\delta^{l}$ to input
size $(H,W)$.  $\displaystyle\delta^{l}$ $\displaystyle=LoG_{\sigma,s}(D^{l})$
(23) $\displaystyle\delta^{l}\in\mathbb{R}^{N\times 1\times u\times v}$
$\displaystyle\rightarrow\delta^{l}\in\mathbb{R}^{N\times 1\times H\times W}$
(24)

5:Summary all $\delta^{l}$ on second dimession:  $\displaystyle
F=\sum_{l=1}^{t}\delta^{l},\quad F\in\mathbb{R}^{N\times 1\times H\times W}$
(25)

6:Calculate threshold value $Q$ with given parameter quantile $q$:
$\displaystyle Q=Threshold({[F_{ij}]_{H\times W},q)},\quad Q\in\mathbb{R}^{N}$
(26)

7:Generate input Mask for every sample:  $\displaystyle
Mask=[\mathbb{I}_{|F_{ij}|>Q}]_{H\times W},Mask\in\\{0,1\\}^{N\times 1\times
H\times W}$ (27)

8:Random sampling $r\%$ images from the input, then apply Gaussian blur on
their inhibited areas (i.e where mask[i,j]= 0) to obtain the enhanced image.

Figure 9: Generation of enhanced images with lateral inhibition.

Figure LABEL:enhancement shows the result of ResNet-50 and enhanced ResNet-50
on the validation set with different blur ratio applied to their ”background”.
It can be seen that the data-enhanced network has better robustness in the
dataset with blurred background.

##  4 Conclusion and Discussion

In this work, we propose a novel CNN training technique inspired by biological
lateral inhibition - Gradient Mask - for filtering noise gradients during the
backpropagation process so as to improve its performance. The experimental
results show that, accuracy of the original CNN architecture, after pruning,
and under adversarial attacks, improves. CNN trained with Gradient Mask also
generates better saliency maps, which is used for data enhancement which
achieves better network interpretability. Furthermore, we provide an
analytical explanation for these improvements based on a new criterion for
gradient quality: the gradient flux sensitivity.

For future work, we would consider making some of the hyper-parameters, such
as the inhibition rate of each layer/bottleneck, $\sigma$ in LoG, the number
of groups of channel division, learnable. Gradient Mask applies the notion of
LI during back-propagation. How would one implement LI in the forward-
propagation, and what purpose would it serve? With the recent advances made by
Transformers in CV and NLP [47, 13, 6, 14, 53, 36], and its potential to unify
different neural network architectures [36], and to form minicolumn naturally,
one may contemplate how LI maybe introduce to the transformer.

## References

  * [1] Julius Adebayo, Justin Gilmer, Michael Muelly, Ian Goodfellow, Moritz Hardt, and Been Kim.  Sanity checks for saliency maps.  arXiv preprint arXiv:1810.03292, 2018.
  * [2] Zeyuan Allen-Zhu and Yuanzhi Li.  Feature purification: How adversarial training performs robust deep learning.  arXiv preprint arXiv:2005.10190, 2020.
  * [3] Shun-ichi Amari.  Dynamics of pattern formation in lateral-inhibition type neural fields.  Biological cybernetics, 27(2):77–87, 1977.
  * [4] Andrea Banino, Caswell Barry, Benigno Uria, Charles Blundell, Timothy Lillicrap, Piotr Mirowski, Alexander Pritzel, Martin J Chadwick, Thomas Degris, Joseph Modayil, et al.  Vector-based navigation using grid-like representations in artificial agents.  Nature, 557(7705):429–433, 2018.
  * [5] Francesco P Battaglia and Alessandro Treves.  Attractor neural networks storing multiple space representations: a model for hippocampal place fields.  Physical Review E, 58(6):7738, 1998.
  * [6] Tom B Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al.  Language models are few-shot learners.  arXiv preprint arXiv:2005.14165, 2020.
  * [7] Daniel P Buxhoeveden and Manuel F Casanova.  The minicolumn and evolution of the brain.  Brain, Behavior and Evolution, 60(3):125–151, 2002.
  * [8] Yongqiang Cao, Yang Chen, and Deepak Khosla.  Spiking deep convolutional neural networks for energy-efficient object recognition.  International Journal of Computer Vision, 113(1):54–66, 2015.
  * [9] Aditya Chattopadhay, Anirban Sarkar, Prantik Howlader, and Vineeth N Balasubramanian.  Grad-cam++: Generalized gradient-based visual explanations for deep convolutional networks.  In 2018 IEEE Winter Conference on Applications of Computer Vision (WACV), pages 839–847. IEEE, 2018.
  * [10] Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton.  A simple framework for contrastive learning of visual representations.  In International conference on machine learning, pages 1597–1607. PMLR, 2020.
  * [11] Steven J Cooper.  Donald o. hebb’s synapse and learning rule: a history and commentary.  Neuroscience & Biobehavioral Reviews, 28(8):851–874, 2005.
  * [12] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei.  Imagenet: A large-scale hierarchical image database.  In 2009 IEEE conference on computer vision and pattern recognition, pages 248–255. Ieee, 2009.
  * [13] Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova.  Bert: Pre-training of deep bidirectional transformers for language understanding.  arXiv preprint arXiv:1810.04805, 2018.
  * [14] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al.  An image is worth 16x16 words: Transformers for image recognition at scale.  arXiv preprint arXiv:2010.11929, 2020.
  * [15] M. Everingham, L. Van Gool, C. K. I. Williams, J. Winn, and A. Zisserman.  The PASCAL Visual Object Classes Challenge 2007 (VOC2007) Results.  http://www.pascal-network.org/challenges/VOC/voc2007/workshop/index.html.
  * [16] Gabriel Goh, Nick Cammarata, Chelsea Voss, Shan Carter, Michael Petrov, Ludwig Schubert, Alec Radford, and Chris Olah.  Multimodal neurons in artificial neural networks.  Distill, 6(3):e30, 2021.
  * [17] Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy.  Explaining and harnessing adversarial examples.  arXiv preprint arXiv:1412.6572, 2014.
  * [18] Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick.  Momentum contrast for unsupervised visual representation learning.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9729–9738, 2020.
  * [19] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun.  Deep residual learning for image recognition.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016.
  * [20] Olivier Henaff.  Data-efficient image recognition with contrastive predictive coding.  In International Conference on Machine Learning, pages 4182–4192. PMLR, 2020.
  * [21] Geoffrey E Hinton, Alex Krizhevsky, and Sida D Wang.  Transforming auto-encoders.  In International conference on artificial neural networks, pages 44–51. Springer, 2011.
  * [22] John J Hopfield.  Neural networks and physical systems with emergent collective computational abilities.  Proceedings of the national academy of sciences, 79(8):2554–2558, 1982.
  * [23] Victor J Katz.  The history of stokes’ theorem.  Mathematics Magazine, 52(3):146–156, 1979.
  * [24] Alex Krizhevsky, Geoffrey Hinton, et al.  Learning multiple layers of features from tiny images.  2009\.
  * [25] Janice Lan, Rosanne Liu, Hattie Zhou, and Jason Yosinski.  Lca: Loss change allocation for neural network training.  arXiv preprint arXiv:1909.01440, 2019.
  * [26] Yann LeCun, Bernhard Boser, John S Denker, Donnie Henderson, Richard E Howard, Wayne Hubbard, and Lawrence D Jackel.  Backpropagation applied to handwritten zip code recognition.  Neural computation, 1(4):541–551, 1989.
  * [27] Timothy P Lillicrap, Adam Santoro, Luke Marris, Colin J Akerman, and Geoffrey Hinton.  Backpropagation and the brain.  Nature Reviews Neuroscience, 21(6):335–346, 2020.
  * [28] Jinlong Liu, Guoqing Jiang, Yunzhi Bai, Ting Chen, and Huayan Wang.  Understanding why neural networks generalize well through gsnr of parameters.  arXiv preprint arXiv:2001.07384, 2020.
  * [29] William Lotter, Gabriel Kreiman, and David Cox.  Deep predictive coding networks for video prediction and unsupervised learning.  arXiv preprint arXiv:1605.08104, 2016.
  * [30] Warren S McCulloch and Walter Pitts.  A logical calculus of the ideas immanent in nervous activity.  The bulletin of mathematical biophysics, 5(4):115–133, 1943.
  * [31] Kenneth D Miller.  Synaptic economics: competition and cooperation in synaptic plasticity.  Neuron, 17(3):371–374, 1996.
  * [32] Vernon B Mountcastle.  The columnar organization of the neocortex.  Brain: a journal of neurology, 120(4):701–722, 1997.
  * [33] Aaron van den Oord, Yazhe Li, and Oriol Vinyals.  Representation learning with contrastive predictive coding.  arXiv preprint arXiv:1807.03748, 2018.
  * [34] Michael Pfeiffer and Thomas Pfeil.  Deep learning with spiking neurons: opportunities and challenges.  Frontiers in neuroscience, 12:774, 2018.
  * [35] Isabella Pozzi, Sander Bohte, and Pieter Roelfsema.  Attention-gated brain propagation: How the brain can implement reward-based error backpropagation.  Advances in Neural Information Processing Systems, 33, 2020.
  * [36] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al.  Learning transferable visual models from natural language supervision.  arXiv preprint arXiv:2103.00020, 2021.
  * [37] Rajesh PN Rao and Dana H Ballard.  Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects.  Nature neuroscience, 2(1):79–87, 1999.
  * [38] Frank Rosenblatt.  The perceptron: a probabilistic model for information storage and organization in the brain.  Psychological review, 65(6):386, 1958.
  * [39] Sara Sabour, Nicholas Frosst, and Geoffrey E Hinton.  Dynamic routing between capsules.  arXiv preprint arXiv:1710.09829, 2017.
  * [40] Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra.  Grad-cam: Visual explanations from deep networks via gradient-based localization.  In Proceedings of the IEEE international conference on computer vision, pages 618–626, 2017.
  * [41] Abhronil Sengupta, Yuting Ye, Robert Wang, Chiao Liu, and Kaushik Roy.  Going deeper in spiking neural networks: Vgg and residual architectures.  Frontiers in neuroscience, 13:95, 2019.
  * [42] Yoko Shoji-Kasai, Hiroshi Ageta, Yoshihisa Hasegawa, Kunihiro Tsuchida, Hiromu Sugino, and Kaoru Inokuchi.  Activin increases the number of synaptic contacts and the length of dendritic spine necks by modulating spinal actin dynamics.  Journal of Cell Science, 120(21):3830–3837, 2007.
  * [43] Jost Tobias Springenberg, Alexey Dosovitskiy, Thomas Brox, and Martin Riedmiller.  Striving for simplicity: The all convolutional net.  arXiv preprint arXiv:1412.6806, 2014.
  * [44] Nicholas V Swindale.  Is the cerebral cortex modular?  Trends in Neurosciences, 13(12):487–492, 1990.
  * [45] J Szentagothai.  The ‘module-concept’in cerebral cortex architecture.  Brain research, 95(2-3):475–496, 1975.
  * [46] Gary W Van Hoesen and Ana Solodkin.  Some modular features of temporal cortex in humans as revealed by pathological changes in alzheimer’s disease.  Cerebral Cortex, 3(5):465–475, 1993.
  * [47] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin.  Attention is all you need.  arXiv preprint arXiv:1706.03762, 2017.
  * [48] Haiguang Wen, Kuan Han, Junxing Shi, Yizhen Zhang, Eugenio Culurciello, and Zhongming Liu.  Deep predictive coding network for object recognition.  In International Conference on Machine Learning, pages 5266–5275. PMLR, 2018.
  * [49] Kai Xiao, Logan Engstrom, Andrew Ilyas, and Aleksander Madry.  Noise or signal: The role of image backgrounds in object recognition.  arXiv preprint arXiv:2006.09994, 2020.
  * [50] Muchao Ye, Xiaojiang Peng, Weihao Gan, Wei Wu, and Yu Qiao.  Anopcn: Video anomaly detection via deep predictive coding network.  In Proceedings of the 27th ACM International Conference on Multimedia, pages 1805–1813, 2019.
  * [51] Mengmi Zhang, Claire Tseng, and Gabriel Kreiman.  Putting visual object recognition in context.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12985–12994, 2020.
  * [52] Bolei Zhou, Aditya Khosla, Agata Lapedriza, Aude Oliva, and Antonio Torralba.  Learning deep features for discriminative localization.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2921–2929, 2016.
  * [53] Xizhou Zhu, Weijie Su, Lewei Lu, Bin Li, Xiaogang Wang, and Jifeng Dai.  Deformable detr: Deformable transformers for end-to-end object detection.  arXiv preprint arXiv:2010.04159, 2020.