# Complex Reading Comprehension Through Question Decomposition

Xiao-Yu Guo  Yuan-Fang Li  Gholamreza Haffari

###### Abstract

Multi-hop reading comprehension requires not only the ability to reason over
raw text but also the ability to combine multiple evidence. We propose a novel
learning approach that helps language models better understand difficult
multi-hop questions and perform “complex, compositional” reasoning. Our model
first learns to decompose each multi-hop question into several sub-questions
by a trainable _question decomposer_. Instead of answering these sub-
questions, we directly concatenate them with the original question and
context, and leverage a _reading comprehension_ model to predict the answer in
a sequence-to-sequence manner. By using the same language model for these two
components, our best _seperate_ /_unified_ t5-base variants outperform the
baseline by 7.2/6.1 absolute F1 points on a hard subset of DROP dataset.

##  1 Introduction

Multi-hop Reading Comprehension (RC) is a challenging problem that requires
compositional, symbolic and arithmetic reasoning capabilities. Facing a
difficult question, humans tend to first decompose it into several sub-
questions whose answers can be more easily identified. The final answer to the
overall question can then be concluded from the aggregation of all sub-
questions’ answers. For instance, for the question in Table 1, we can
naturally decompose it into three simpler sub-questions (1) “return the
touchdown yards”, (2) “return the fewest of $\\#1$”, and (3) “return who
caught $\\#2$”. The tokens $\\#1$ and $\\#2$ are the answers to the first and
second sub-questions respectively. Finally, the player with the touchdown of
$\\#2$ is returned as the final answer.

$\mathbf{C}$ | First, Detroit’s Calvin Johnson caught
---|---
| a 1-yard pass in the third quarter. The
| game’s final points came when Mike
| Williams of Tampa Bay caught a 5-yard.
$\mathbf{Q}$ | Who caught the touchdown for the
| fewest yards?
$\mathbf{Q_{1}}$ | return the touchdown yards
$\mathbf{Q_{2}}$ | return the fewest of $\\#1$
$\mathbf{Q_{3}}$ | return who caught $\\#2$
$\mathbf{A}$ | Calvin Johnson
Table 1: An example for reading comprehension. $\mathbf{C}$ is the context,
$\mathbf{Q}$ is a hard multi-hop question, and $\mathbf{Q_{1}}$,
$\mathbf{Q_{2}}$, $\mathbf{Q_{3}}$ are sub-questions annotated in Break
dataset. $\mathbf{A}$ is the answer to $\mathbf{Q}$.

State-of-the-art RC techniques employ large-scale pre-trained language models
(LMs) such as GPT-3 Brown et al. (2020) for their superior representation and
reasoning capablities. Chain of thought prompting Wei et al. (2022) elicits
strong reasoning capability of LMs by providing intermediate reasoning steps.
Least-to-most prompting Zhou et al. (2022) further shows the feasibility of
conducting decomposition and multi-hop reasoning, which happen on the decoder
side together with the answer prediction procedure. However, compared to
supervised learning models, both of these methods rely on extremely large LMs
with tens and hundreds of billions of parameters to achieve competitive
performance, thus requiring expensive hardware and incurring a large
computation footprint.

Despite significant research on RC Dua et al. (2019); Perez et al. (2020),
those questions that require strong compositional generalisability and
numerical reasoning abilities are still challenging to even the state-of-the-
art models Ran et al. (2019); Chen et al. (2020a, b); Wei et al. (2022); Zhou
et al. (2022). While decomposition is a natural approach to tackle this
problem, the lack of sufficient ground-truth sub-questions limits our ability
to train RC models based on large LMs.

In this paper, we propose a novel low-budget (only 1‰ parameters of GPT-3)
learning approach to improve LMs’ performance on hard multi-hop RC such as the
Break subset of DROP Dua et al. (2019). Our model consists of two main
modules: (1) an encoder-decoder LM as a _question decomposer_ and (2) another
encoder-decoder LM as the _reading comprehension_ model. First, we train the
question decomposer to decompose a difficult multi-hop question to sub-
questions from a limited amount of annotated data. Next, instead of solving
these sub-questions, we train the reading comprehension model to predict the
final answer by directly concatenating the sub-questions with the original
question. We further propose a _unified_ model that utilizes the same LM for
both question decomposition and reading comprehension with task-specific
prompts. With 9$\times$ weakly supervised data, we design a Hard EM-style
algorithm to iteratively optimise the _unified_ model.

To prove the effectiveness of our approach, we leverage two different types of
LMs: T5 Raffel et al. (2020) and Bart Lewis et al. (2020) to build baselines
and our variants. The experimental results show that without changing the
model structure, our proposed variant outperforms the end-to-end baseline. By
adding ground-truth sub-questions, gains on the F1 metric are 1.7 and 0.7
using T5 and Bart separately. Introducing weakly supervised training data can
help improve the performance of both _separate_ and _unified_ variants by at
least 4.4 point on F1. And our method beats the state-of-the-art model GPT-3
by a large margin.

##  2 Related Work

##### Multi-hop Reading Comprehension

mentioned in this paper requires more than one reasoning or inference step to
answer a question. For example, multi-hop RC in DROP Dua et al. (2019)
requires numerical reasoning such as addition, subtraction. To address this
problem, Dua et al. proposed a number-aware model NAQANet that can deal with
such questions for which the answer cannot be directly extracted. NumNet Ran
et al. (2019) leveraged Graph Neural Network to design a number-aware deep
learning model. QDGAT Chen et al. (2020a) distinguished number types more
precisely by adding the connection with entities and obtained better
performance. Nerd Chen et al. (2020b) searched possible programs exhaustively
based on the ground-truth and employed these programs as weak supervision to
train the whole model.

##### Question Decomposition

is the approach that given a complex question, break it into several simple
sub-questions. These sub-questions can also be Question Decomposition Meaning
Representation (QDMR) Wolfson et al. (2020) for complex questions. Many
researchers Perez et al. (2020); Geva et al. (2021) have been trying to solve
the problem by incorporating decomposition procedures. For example, Perez et
al. (2020) propose a model that can break hard questions into easier sub-
questions. Then, simple QA systems provide answers of these sub-questions for
downstream complex QA systems to produce the final answer corresponding to the
original complex question. Fu et al. (2021) propose a three-stage framework
called Relation Extractor Reader and Comparator (RERC), based on complex
question decomposition. Different from these approaches, we aim to improve the
multi-hop capability of current encoder-decoder models without dedicated pre-
designing the architecture.

##### Language Models

like BERT Devlin et al. (2019), GPT families Radford et al. (2018, 2019);
Brown et al. (2020), BART Lewis et al. (2020) and T5 Raffel et al. (2020) are
demonstrated to be effective on many NLP tasks, base on either fine-tuning or
few-shot learning Wei et al. (2022); Zhou et al. (2022), even zero-shot
learning. However, LMs suffer a lot from solving multi-hop questions and logic
reasoning and numerical reasoning problems. Although some research Nye et al.
(2021); Wei et al. (2022) has conducted experiments on either simple or
synthetic datasets and shown the effectiveness, Razeghi et al. (2022)
indicates that the model reasoning is not robust enough.

Recently, Dohan et al. (2022) points out that prompted models can be regarded
as employing a unified framework a _language model cascade_. From the
perspective view of probabilistic programming, several recent literature Wei
et al. (2022); Zhou et al. (2022) are formalized. In this paper, we also treat
our whole process as a probabilistic model that is consistent to Dohan et al.
(2022).

Figure 1: Our model structure on complex reading comprehension through
question decomposition. Step 1: Question Decomposer generates a sequence of
sub-questions; Step 2: RC component predicts the answer based on question,
sub-questions and the given context. The context of this given example is
truncated.

##  3 Complex Question Answering Through Decomposition

Our focus in this work is on complex questions requiring multi-hop reasoning.
As such, our approach consists of the following two steps:

  1. 1.

The complex question is decomposed to a sequence of sub-questions. The
decomposition of the question is performed by the _question decomposer_
component of our system.

  2. 2.

The model produces the answer to the complex question leveraging the generated
subquestions to provide guidance to the reasoning of the system. This is
performed by the _reading comprehension_ component.

We use LMs such as T5 and Bart as the backbone111Our approach is general, and
it can be used with other pre-trained seq2seq models and language models as
well. for both question decomposer and the reading comprehension (Figure 1).
We present several variants of our model, depending whether the models for the
above two steps are either separate or unified using multitask learning. As we
have the ground truth question decomposition for only a subset of the training
data, we treat the missing decompositions as latent variables. We then propose
an algorithm based on Hard-EM Neal and Hinton (1998) for learning the model.
The rest of this section provides more details.

##### Probabilistic Model.

Given a question $Q$ and a $C$ context pair, our system generates the answer
$A$ according to the following probabilistic model:

$\displaystyle P_{\theta}(A|Q,C)=\sum_{Z}P_{\theta}(A,Z|Q,C)$ (1)
$\displaystyle=\sum_{Z}P^{\text{dc}}_{\text{LM}}(Z|Q)\times
P^{\text{rc}}_{\text{LM}}(A|Q,C,Z)$ (2)

where $Z$ denotes the unobserved decomposition of the question,
$P^{\text{dc}}_{\text{LM}}(Z|Q)$ 222We have made the following independence
assumption: $P^{\text{dc}}_{\text{LM}}(Z|Q)\approx
P^{\text{dc}}_{\text{LM}}(Z|Q,C)$. denotes the question decomposer
(operationalised based on one specific LM), and
$P^{\text{rc}}_{\text{LM}}(A|Q,C,Z)$ denotes the reading comprehension
component. In principle, the $P^{\text{dc}}_{\text{LM}}$ and
$P^{\text{rc}}_{\text{LM}}$ components can be constructed using different
models, so the parameters $\theta$ of the whole probabilistic model consists
of those for these two models. This is denoted by the _separate_ variant.

We further investigate using the same LM for both the question decomposer and
reading comprehension component, which we denote by the _unified_ variant in
the experiments. In this case, the probabilistic model parameter $\theta$
consists of only one set of parameters corresponding to the underlying model.

##### Question Decomposer.

To obtain high-quality sub-questions, we first train a question decomposer
$P^{\text{dc}}_{\text{LM}}$ to break down difficult multi-hop questions, i.e.,
the first term in Equation 2. It learns the decomposition based on QDMRs
Wolfson et al. (2020). We only use the specific partition on the DROP dataset
Dua et al. (2019) and treat QDMRs as sub-questions. These sub-questions only
cover around 10% QA pairs in DROP. Therefore, we need to predict
decompositions for the rest of the dataset. More details will be revealed in
Section 4.

Formally, given a multi-hop question $Q$, the question decomposer
$P^{\text{dc}}_{\text{LM}}$ generates the sub-questions
$Z:=\\{Q^{1},Q^{2},...,Q^{s}\\}$. Intuitively, We treat it as a seq2seq
learning problem: our input to the encoder is “$\texttt{<PARSE>}Q$”, where
<PARSE> is a special token. The decoder then generates tokens of the sub-
questions in auto-regressive way
“$\texttt{<subQ>}Q^{1}\texttt{<subQ>}Q^{2}\texttt{<subQ>}\ldots Q^{s}$”, where
<subQ> is a special token 333We employ the greedy search algorithm to generate
the sub-questions $Z$. However, one can leverage other strategies like beam
search to make more than one predictions..

##### Reading Comprehension Component.

To further obtain answers based on the question and generated sub-questions,
the reading comprehension component $P^{\text{rc}}_{\text{LM}}$ generates the
answer $A$, i.e., the second term in Equation 2. In stead of directly
answering all the sub-questions given by the trained question decomposer, we
train our RC component to predict the final answer in a sequence-to-sequence
way.

Formally, given a multi-hop complex question $Q$ and the corresponding sub-
questions $Z:=\\{Q1,Q2,...,Q^{s}\\}$ generated by a trained question
decomposer, our input to the RC encoder is
“$\texttt{<QUESTION>}Q\texttt{<subQ>}Q^{1}\ldots\texttt{<subQ>}Q^{s}\texttt{<CONTEXT>}C$”,
where <QUESTION> and <CONTEXT> are special tokens. In other words, we
concatenate the multi-hop question and all the sub-questions, together with
the context as the input to our RC component. The decoder then generates the
tokens of the answer autoregressively.

Algorithm 1 Learning with Hard-EM

1:an initial pre-trained LM $M$; the full reading comprehesion dataset
$\mathcal{D}_{1}$; the subset with sub-question annotations $\mathcal{D}_{2}$.

2:Train $M$ on $\mathcal{D}_{2}$ to get $M^{0}$

3:for iter in N_iters do

4: For all $\mathcal{D}=\mathcal{D}_{1}\setminus\mathcal{D}_{2}$ employ
$M^{iter-1}$ to predict sub-questions and get $\mathcal{D}^{iter}$

5: Retrain $M^{iter-1}$ on all examples:
$\mathcal{D}_{2}\cup\mathcal{D}^{iter}$, get updated model $M^{iter}$

6:end for

##### Training and Inference.

The training objective of our model is

$\begin{split}\mathcal{L}=&\sum_{(Q,C,A)\in\mathcal{D}_{1}\setminus\mathcal{D}_{2}}\log
P_{\theta}(A|Q,C)+\\\ &\sum_{(Q,C,Z^{*},A)\in\mathcal{D}_{2}}\log
P_{\theta}(A,Z^{*}|Q,C),\end{split}$ (3)

where $Z^{*}$ denotes the ground truth decomposition available only for the
subset of the training data referred to by $\mathcal{D}_{2}$. The first term
of the training objective involves enumerating over all possible latent
decompositions, which is computationally intractable. Therefore, we resort to
Hard-EM for learning the parameters of our model (see Algorithm 1) for the
unified variant. We found taking 10 iterations of the Hard-EM algorithm to be
mostly sufficient for learning model parameters in our experiments.

For the separate variant, i.e., using two different LMs for
$P^{\text{dc}}_{\text{LM}}$ and $P^{\text{rc}}_{\text{LM}}$, we train the
question decomposer on $\mathcal{D}_{2}$, and then train the reading
comprehension component on $\mathcal{D}_{2}$ as well as
$\mathcal{D}_{1}\setminus\mathcal{D}_{2}$ augmented with the generated
decomposition $Z$. We also compare with training the reading comprehension
component on $\mathcal{D}_{2}$ only, in the experiments. During inference
time, we first generate the question decomposition $\tilde{Z}$ according to
$P^{\text{dc}}_{\text{LM}}$, and then use $\tilde{Z}$ in
$P^{\text{rc}}_{\text{LM}}$ to generate the answer.

##  4 Experiments

Proportions | 1% | 5% | 10% | 50% | 100%
---|---|---|---|---|---
BLEU | 39.08 | 44.76 | 47.74 | 50.12 | 54.69
Rouge-1 | 77.49 | 81.75 | 83.12 | 84.76 | 85.67
Rouge-2 | 57.00 | 62.83 | 64.97 | 66.94 | 68.61
RougeL | 67.78 | 72.65 | 74.37 | 76.55 | 77.43
RC | EM | 26.0 | 26.5 | 27.0 | 27.8 | 27.2
F1 | 31.3 | 31.3 | 31.6 | 32.2 | 32.0

Table 2: Experimental results of the Bart based question decomposer: (1) Row 1-4 show intrinsic metrics for the question decomposition by using different proportions of training instances. (2) Row 5-6 show extrinsic metrics of the RC model by using the corresponding decomposer generated sub-questions. LMs | t5-small | t5-base
---|---|---
Proportions | 1% 444Trained as a question decomposer, the t5-small model and cannot be further evaluated on downstream RC task, as the generated sub-questions are poor-quality. | 5% | 10% | 50% | 100% | 1% | 5% | 10% | 50% | 100%
BLEU | 11.21 | 44.50 | 50.44 | 60.15 | 62.73 | 34.86 | 52.98 | 57.3 | 62.18 | 64.40
Rouge-1 | 43.00 | 76.93 | 81.53 | 87.25 | 88.59 | 70.66 | 84.16 | 85.77 | 88.50 | 89.27
Rouge-2 | 28.18 | 59.13 | 64.33 | 72.60 | 74.76 | 50.57 | 66.86 | 70.24 | 74.24 | 75.72
RougeL | 39.22 | 68.92 | 73.66 | 79.99 | 81.57 | 62.10 | 75.49 | 78.07 | 81.20 | 82.53
RC | EM | - | 28.9 | 29.9 | 29.0 | 29.0 | 33.7 | 34.3 | 34.3 | 34.6 | 34.8
F1 | - | 33.0 | 34.0 | 33.2 | 33.1 | 37.8 | 38.4 | 38.5 | 38.5 | 38.6
Table 3: Results of the T5 based question decomposer (left-half: t5-small,
right-half: t5-base): (1) Row 1-4 show all intrinsic metrics to evaluate the
question decomposer by using different proportions of training instances. (2)
Row 5-6 show extrinsic metrics of the RC component by using the corresponding
decomposer generated sub-questions.

###  4.1 Dataset

We consistently use the same notations as in Algorithm 1.

  * •

$\mathcal{D}_{1}$: the DROP dataset Dua et al. (2019) that contains
77,400/9,536 question ($Q$) answer ($A$) training/testing pairs for the
reading comprehension component.

  * •

$\mathcal{D}_{2}$: the Break dataset Wolfson et al. (2020) 555The full Break
dataset Wolfson et al. (2020) annotated is a combination of many datasets
including DROP. In this paper, we only use the DROP partition of the original
Break. that contains 7,683/1,268 question ($Q$) decomposition ($Z^{*}$)
training/testing pairs for the question decomposer 666This subset of DROP
contains the corresponding answers for each question. Therefore, we also use
it to evaluate the RC component in our experiments..

  * •

$\mathcal{D}=\mathcal{D}_{1}\setminus\mathcal{D}_{2}$: the difference set
between $\mathcal{D}_{1}$ and $\mathcal{D}_{2}$ that contains only question
answer pairs without ground-truth decomposition.

  * •

$\mathcal{D}^{iter}$: $\mathcal{D}$ with decomposition ($Z$) generated by the
trained question decomposer.

Note that every question ($Q$) is associated with a specific context ($C$).
With all question decomposition labelled, $\mathcal{D}_{2}$ is actually a
subset of $\mathcal{D}_{1}$ and is more challenging.

###  4.2 Backbone and Evaluation Metric

There are three LMs of different types and sizes we employ as backbones in
this paper: (1) t5-small (60M parameters), (2) t5-base (220M parameters), (3)
bart-base (140M parameters). We also employ GPT-3 (175B parameters) as it is
the current state-of-the-art language model in a various of natural language
processing tasks.

Sub-question Decomposition We train and evaluate our question decomposer using
$\mathcal{D}_{2}$, which was proposed to better understand difficult multi-hop
questions. We report BLEU Papineni et al. (2002) and Rouge Lin (2004) scores
to show the intrinsic performance of the decomposer.

Reading Comprehension We evaluate our RC model on $\mathcal{D}_{2}$. For the
Hard-EM approach, we have $\mathcal{D}_{1}\setminus\mathcal{D}_{2}$ as weakly
supervised data. We report F1 and Exact Match(EM) Dua et al. (2019) scores in
the following experiments.

Backbone | Variant | Training Set | F1 | EM
---|---|---|---|---
baselines |  |  |  |
bart-base Lewis et al. (2020) | - | $\mathcal{D}_{2}$ | 30.9 | 27.1
t5-base Raffel et al. (2020) | - | $\mathcal{D}_{2}$ | 37.9 | 33.9
our bart-base variants |  |  |  |
w/ predicted sub-questions | _separate_ | $\mathcal{D}_{2}$ | 32.0 | 27.2
w/ ground-truth sub-questions | _separate_ | $\mathcal{D}_{2}$ | 33.2 | 29.0
w/ ground-truth sub-questions | _separate_ | $\mathcal{D}_{2},\mathcal{D}^{1}$ | 45.0 | 40.5
w/o Hard-EM | _unified_ | $\mathcal{D}_{2},\mathcal{D}^{1}$ | 44.2 | 39.9
w/ Hard-EM | _unified_ | $\mathcal{D}_{2},\mathcal{D}^{iter}$ | 44.3 | 40.0
our t5-base variants |  |  |  |
w/ predicted sub-questions | _separate_ | $\mathcal{D}_{2}$ | 38.6 | 34.8
w/ ground-truth sub-questions | _separate_ | $\mathcal{D}_{2}$ | 39.6 | 35.6
w/ ground-truth sub-questions | _separate_ | $\mathcal{D}_{2},\mathcal{D}^{1}$ | 45.1 | 40.8
w/o Hard-EM | _unified_ | $\mathcal{D}_{2},\mathcal{D}^{1}$ | 38.8 | 34.9
w/ Hard-EM | _unified_ | $\mathcal{D}_{2},\mathcal{D}^{iter}$ | 44.0 | 40.1
GPT-3 (zero-shot) | - | - | 15.7 | 4.6
GPT-3 (few-shot) | - | - | 34.9 | 27.0
Table 4: Overall results for baselines, our separate and unified variants. All
models are evaluated on the same test set from $\mathcal{D}_{2}$.

###  4.3 Results on Decomposition

Based on Bart and T5, Table 2 and Table 3 respectively show the experimental
results of the question decomposers. To comprehensively show their
performance, we conducted two aspects of experiments including intrinsic
decomposition evaluation and extrinsic RC evaluation.

##### Intrinsic Evaluation

We first evaluate the quality of sub-questions generated by different question
decomposers. In this part, intrinsic metrics, BLEU and Rouge scores, are shown
in the first four rows of Table 2 and Table 3. And also we show the results of
five decomposers trained on different proportions (1%, 5%, 10%, 50%, 100%) of
the Break dataset $\mathcal{D}_{2}$’s training data. All these evaluations are
conducted on the same validation set of $\mathcal{D}_{2}$.

Comparing column-by-column, we find that with more training data, both
question decomposers achieve a better performance for both BLEU and Rouge. We
also note that the rate of improvement of these metrics becomes slower when
more data is added (e.g. 1% to 5% and 10% to 50%). Therefore, we posit that
with more training data, the performance of the decomposer will not improve
due to the capability of the LM model.

##### Extrinsic Evaluation

Since the eventual usage of the generated sub-questions is to improve the RC
component, we conduct a RC performance comparison experiments to see how can
the quality of these sub-questions influence the downstream RC task. Also like
the intrinsic evaluation, we show the results based on decomposers trained on
different proportions of $\mathcal{D}_{2}$ by using two extrinsic metrics: EM
and F1. All the evaluations are conducted on the same validation set of
$\mathcal{D}_{2}$.

To clarify our settings in this part, we don’t employ the ground-truth sub-
questions from $\mathcal{D}_{2}$. Instead, we employ the sub-questions
generated by five question decomposers for the RC component to predict
answers. As the last two rows of both Table 2 and Table 3 show, both EM and F1
scores show a gradually increasing trend when more training instances are used
to train the question decomposer. With more parameters, t5-base tends to have
a better performance than t5-small.

###  4.4 Results on Reading Comprehension

Table 4 shows the experimental results for the downstream RC task. We show two
baselines in the first place: “bart-base” and “t5-base”. Without taking sub-
questions as input, both are trained on the Break dataset $\mathcal{D}_{2}$.
Based on these vanilla models, we show our _separate_ and _unified_ approaches
that use “bart-base” and “t5-base” as backbones separately in Table 4.

####  4.4.1 Separate Variant

Our _separate_ variants are base on the architecture in Figure 1. In Table 4,
we have three _separate_ variants based on each backbone for comparison.
Taking t5-base as one example, comparing to the t5-base, using predicted sub-
questions achieves a 0.7-point gain of F1 score. Meanwhile using ground-truth
sub-questions, our model outperforms the t5-base by 1.7 points of F1 score.
The same improvement can be also concluded from the bart-base model. They
employ $\mathcal{D}_{2}$ for training but their testing sets are different:
predicted one use generated sub-questions while ground-truth one use sub-
questions from $\emph{D}_{2}$. The reason why our approach is more effective
than the baseline model is that concatenating sub-questions can give LMs hints
on the reasoning procedure, which helps LMs produce step-by-step thoughts
implicitly.

Furthermore, we add $\mathcal{D}^{1}$ as the training set to train our
seperate model. As it shows in Table 4, this kind of _separate_ variants show
the overall best performance since we have two sets of parameters separately
learning question decomposition and reading comprehension. Compared to
t5-base, the bart-base variant shows a higher performance gain that proves the
effectiveness of our method.

Context |  Question |  GPT-3 (few-shot) |  bart-base _separate_ (best) |  ground-truth answer
---|---|---|---|---
… notably striking out Julio Franco, at the time the oldest player in the MLB at 47 years old; Clemens was himself 43. In the bottom of the eighteenth inning, Clemens came to bat again… |  Which player playing in the 2005 National League Division Series was older, Julio Franco or Roger Clemens? |  Julio Franco (✓) |  Julio Franco (✓) |  Julio Franco
… Nyaungyan then systematically reacquired nearer Shan states. He captured Nyaungshwe in February 1601, and the large strategic Shan state of Mone in July 1603, bringing his realm to the border of Siamese Lan Na. In response, Naresuan of Siam marched in early 1605 to … |  How many years after capturing Nyaungshwe did Nyaungyan capture the large strategic Shan state of Mone? |  3 years (✗) |  2 (✓) |  2
Kannada language is the official language of Karnataka and spoken as a native language by about 66.54% of the people as of 2011. Other linguistic minorities in the state were Urdu (10.83%), Telugu language (5.84%), Tamil language (3.45%), … |  How many in percent of people for Karnataka don’t speak Telugu? |  66.54% (✗) |  94.04% (✗) |  94.16%
A 2013 analysis of the National Assessment of Educational Progress found that from 1971 to 2008, the size of the black-white IQ gap in the United States decreased from 16.33 to 9.94 IQ points. It has also concluded however that, … |  How many IQ points did the black-white IQ gap decrease between 1971 and 2008? |  16.33 (✗) |  0.9 (✗) |  6.39
Table 5: Correct and incorrect outputs from GPT-3 and our _separate_ variant. Correct and Wrong supporting facts are annotated in the context using the corresponding color. Correct and wrong answer predictions are also marked with ✓and ✗ (the table is best seen in colours). overlaps | $0\sim 25\%$ | $25\%\sim 50\%$ | $50\%\sim 75\%$ | $75\%\sim 100\%$
---|---|---|---|---
uni-grams | bart-base | - | 0 | 25.7 | 27.4
_unified_ | - | 0 | 32.9 | 40.2
_separate_ | - | 0 | 35.7 | 41.3
GPT-3 | - | 100.0 | 35.7 | 26.4
bi-grams | bart-base | - | 16.7 | 23.6 | 28.2
_unified_ | - | 33.3 | 29.1 | 41.9
_separate_ | - | 50.0 | 28.6 | 43.2
GPT-3 | - | 44.4 | 29.1 | 26.2
tri-grams | bart-base | 22.2 | 20.5 | 25.5 | 29.3
_unified_ | 38.9 | 26.2 | 32.3 | 45.1
_separate_ | 50.0 | 30.0 | 33.4 | 45.9
GPT-3 | 50.0 | 28.0 | 25.8 | 26.8
Table 6: EM scores separately computed based on overlaps of sub-questions
n-grams between training set and testing set on $\mathcal{D}_{2}$. Four models
listed in this table are: the bart-base baseline, the best performed
_separate_ model, the best performed _unified_ model

####  4.4.2 Unified Variant

Our _unified_ variants are base on the architecture in Figure 1 and one single
model is used to train on both steps. In Table 4, the last two rows of each
variants show the performance of our _unified_ variant. Without the Hard-EM
algorithm, performing multi-task learning achieves a 0.9 point improve over
the T5 baseline. However, it shows a performance drop when compared to the
_separate_ variant with ground-truth sub-questions. This can be caused by the
enlarged dataset and the additional decomposition work the _unified_ variant
need to handle.

When more training data is provided (i.e. $\mathcal{D}^{1}$ and
$\mathcal{D}^{iter}$), though without ground-truth sub-questions, the
_unified_ variants substantially outperforms the baselines by 10.1 and 6.1
points over bart-base and t5-base model. Furthermore, when compared with the
best _separate_ variants, our _unified_ models also show comparable
performance on both F1 and EM metrics. Based on the observations of the last
three rows of each backbone, it can be concluded that introducing more weakly-
supervised training data can significantly help our model address the original
difficult multi-hop RC task.

We also include another evaluation of employing GPT-3, which is the state-of-
the-art language model on many tasks and also in a large parameter scale
(175B). The results are shown by last two rows in Table 4. Based on the
experimental results, GPT-3 cannot even beat two baseline models under the
zero-shot learning paradigm, which again shows the complexity and challenging
of the task. When provided with several exemplars, it can easily outperform
the bart-base model by 2.4 points on F1 score. However, even with $\times
1000$ parameters, GPT-3 is still far behind to our best variants by 10.2 F1
points.

##  5 Analysis and Discussions

###  5.1 Qualitative Analysis

In this section, we will further discuss some real-life cases generated by our
proposed variants from the dataset. In Table 5, the first row shows a
comparison question and both GPT-3 and our bart-base _separate_ model can
produce the correct answer. However, when the question requires some
arithmetic operations, such as addition or subtraction, the GPT-3 model would
fail to answer correctly. Our model can handle this as shown by the second
row.

There are two types of failures from our variants: one is that our model
cannot handle unseen numbers, and the other is arithmetic between float
numbers. The unseen number case happens in the third row of Table 5. Asking
for the number of a complement set, though the number 94.04% is wrongly
predicted by our model, it is more close to the ground-truth (94.16%) when
compared to the GPT-3, which directly predict an wrong evidence annotated with
red color. Furthermore, the last row shows a subtraction question between two
float numbers. Different from integer number subtraction in the second row, it
is much harder to compute this arithmetic for language models. Traditionally,
some symbolic methods can handle this problem very well. Tackling these
problems can be interesting future work directions.

###  5.2 Quantitative Analysis

We look into details of $\mathcal{D}_{2}$ from the perspective of sub-question
n-grams for both training and testing data. Intuitively, given one instance
from the test set, more n-grams overlap it shows with the training set, higher
the EM and F1 scores. Therefore, we further conducted the analysis and list
all the statistics in Table 6.

We calculate for uni-grams, bi-grams and tri-grams for four models: bart-base
baseline, the best-performed _separate_ and _unified_ variants proposed in
Section 3 and GPT-3 with few-shot learning. The overlaps we choose is four
intervals using percentages to represent. For example, $0\sim 25\%$
overlapping on bi-grams means that the test instance have this proportion of
bi-grams overlaps with all the training instances. Note that there is no
overlapping for uni-grams and bi-grams in $0\sim 25\%$.

In Table 6, we report the EM score (F1 score shows the similar results). The
bart-base model show a tendency that with more overlaps across all n-grams,
the performance will increase, which is consistent with our assumption.
However, on the contrary, GPT-3 model show a reverse tendency that is probably
due to the pre-trained corpus that shares far less n-grams with the test set.
This characteristic improves the compositional generalisation ability as it
outperforms the baseline model on the low-overlapping part of test set. Both
of our _separate_ and _unified_ variants show overall improvements over the
bart-base baseline. In particular, the first and second columns also show our
model can better handle the low-overlapping questions, even without
performance drop on the high-overlapping questions ($50\%\sim 100\%$). This
experiment can further prove the compositional generalisation of our method is
comparable to GPT-3.

##  6 Conclusion

We propose a two-step process for multi-hop reading comprehension task. The
first step involves a question decomposer that maps a difficult multi-hop
question into several sub-questions. The second step is to train a reading
comprehension model based on (question, sub-questions, paragraph, answer)
tuples. With the addition of sub-questions, our bart-/t5-base variants
outperform the baseline model by 2.3/1.7 using ground truth sub-questions and
1.1/0.7 using generated ones on F1 score. Based on the hard-EM paradigm, large
positive gains of another 11.1/4.4 point on F1 by the unified multi-task
learning bart-/t5-base models shows the effectiveness of introducing weakly
supervised training data. By further analysing the predicted examples and
dataset, we also found our model can make a more comprehensive improvement
compared with the SOTA GPT-3 model. But some problems like handling unseen
numbers still exist and will be our future research directions.

## References

  * Brown et al. (2020) Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020.  Language models are few-shot learners.  In _Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual_.
  * Chen et al. (2020a) Kunlong Chen, Weidi Xu, Xingyi Cheng, Zou Xiaochuan, Yuyu Zhang, Le Song, Taifeng Wang, Yuan Qi, and Wei Chu. 2020a.  Question directed graph attention network for numerical reasoning over text.  In _Proceedings of EMNLP_ , pages 6759–6768.
  * Chen et al. (2020b) Xinyun Chen, Chen Liang, Adams Wei Yu, Denny Zhou, Dawn Song, and Quoc V. Le. 2020b.  Neural symbolic reader: Scalable integration of distributed and symbolic representations for reading comprehension.  In _Proceedings of ICLR_.
  * Devlin et al. (2019) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019.  BERT: pre-training of deep bidirectional transformers for language understanding.  In _Proceedings of the NAACL HLT 2019_ , pages 4171–4186.
  * Dohan et al. (2022) David Dohan, Winnie Xu, Aitor Lewkowycz, Jacob Austin, David Bieber, Raphael Gontijo Lopes, Yuhuai Wu, Henryk Michalewski, Rif A. Saurous, Jascha Sohl-Dickstein, Kevin Murphy, and Charles Sutton. 2022.  Language model cascades.  _CoRR_ , abs/2207.10342.
  * Dua et al. (2019) Dheeru Dua, Yizhong Wang, Pradeep Dasigi, Gabriel Stanovsky, Sameer Singh, and Matt Gardner. 2019.  DROP: A reading comprehension benchmark requiring discrete reasoning over paragraphs.  In _Proceedings of the NAACL HLT 2019_ , pages 2368–2378.
  * Fu et al. (2021) Ruiliu Fu, Han Wang, Xuejun Zhang, Jun Zhou, and Yonghong Yan. 2021.  Decomposing complex questions makes multi-hop QA easier and more interpretable.  In _Findings of the Association for Computational Linguistics: EMNLP 2021_ , pages 169–180, Punta Cana, Dominican Republic. Association for Computational Linguistics.
  * Geva et al. (2021) Mor Geva, Tomer Wolfson, and Jonathan Berant. 2021.  Break, perturb, build: Automatic perturbation of reasoning paths through question decomposition.  _CoRR_ , abs/2107.13935.
  * Lewis et al. (2020) Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. 2020.  BART: denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020, Online, July 5-10, 2020_ , pages 7871–7880. Association for Computational Linguistics.
  * Lin (2004) Chin-Yew Lin. 2004.  ROUGE: A package for automatic evaluation of summaries.  In _Text Summarization Branches Out_ , pages 74–81, Barcelona, Spain. Association for Computational Linguistics.
  * Neal and Hinton (1998) Radford Neal and Geoffrey E. Hinton. 1998.  A view of the em algorithm that justifies incremental, sparse, and other variants.  In _Learning in Graphical Models_ , pages 355–368. Kluwer Academic Publishers.
  * Nye et al. (2021) Maxwell Nye, Anders Johan Andreassen, Guy Gur-Ari, Henryk Michalewski, Jacob Austin, David Bieber, David Dohan, Aitor Lewkowycz, Maarten Bosma, David Luan, Charles Sutton, and Augustus Odena. 2021.  Show your work: Scratchpads for intermediate computation with language models.  _CoRR_ , abs/2112.00114.
  * Papineni et al. (2002) Kishore Papineni, Salim Roukos, Todd Ward, and Wei jing Zhu. 2002.  Bleu: a method for automatic evaluation of machine translation.  pages 311–318.
  * Perez et al. (2020) Ethan Perez, Patrick Lewis, Wen-tau Yih, Kyunghyun Cho, and Douwe Kiela. 2020\.  Unsupervised question decomposition for question answering.  _CoRR_ , abs/2002.09758.
  * Radford et al. (2018) Alec Radford, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever. 2018.  Improving language understanding by generative pre-training.
  * Radford et al. (2019) Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019.  Language models are unsupervised multitask learners.  _OpenAI blog_ , 1(8):9.
  * Raffel et al. (2020) Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020.  Exploring the limits of transfer learning with a unified text-to-text transformer.  _J. Mach. Learn. Res._ , 21:140:1–140:67.
  * Ran et al. (2019) Qiu Ran, Yankai Lin, Peng Li, Jie Zhou, and Zhiyuan Liu. 2019.  NumNet: Machine reading comprehension with numerical reasoning.  In _Proceedings of EMNLP-IJCNLP_ , pages 2474–2484.
  * Razeghi et al. (2022) Yasaman Razeghi, Robert L. Logan IV, Matt Gardner, and Sameer Singh. 2022.  Impact of pretraining term frequencies on few-shot reasoning.  _CoRR_ , abs/2202.07206.
  * Wei et al. (2022) Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Ed Chi, Quoc Le, and Denny Zhou. 2022.  Chain of thought prompting elicits reasoning in large language models.  _CoRR_ , abs/2201.11903.
  * Wolfson et al. (2020) Tomer Wolfson, Mor Geva, Ankit Gupta, Yoav Goldberg, Matt Gardner, Daniel Deutch, and Jonathan Berant. 2020.  Break it down: A question understanding benchmark.  _Trans. Assoc. Comput. Linguistics_ , 8:183–198.
  * Zhou et al. (2022) Denny Zhou, Nathanael Schärli, Le Hou, Jason Wei, Nathan Scales, Xuezhi Wang, Dale Schuurmans, Olivier Bousquet, Quoc Le, and Ed Chi. 2022.  Least-to-most prompting enables complex reasoning in large language models.  _CoRR_ , abs/2205.10625.


# Production and suppression of delayed light in NaI

F. Sutanto<EMAIL_ADDRESS>J. Xu  S. Pereverzev  A. Bernstein  Lawrence
Livermore National Laboratory, Livermore, CA 94550, USA

###### Abstract

We investigate a hypothesis that energy accumulation and the subsequent
release in NaI(Tl) may lead to pulse-like events in the few-keV energy regime,
a phenomenon suggested by the crystal manufacturing company Saint-Gobain, who
provided the crystals for DAMA/LIBRA . While we observed delayed long-lasting
light emission in a 3” NaI(Tl) crystal after exposing it to UV light, the
delayed light consists primarily of single photons that are uncorrelated with
each other. We also observe delayed light emission in NaI(Tl) following gamma
radiation and large ionization events like cosmic-ray muons. We found that
irradiating the crystal with red light after UV exposure significantly
suppressed delayed photon emissions.

###### pacs:

Valid PACS appear here

##  I Introduction

In the last few decades, the quest for dark matter (DM) has given a null
result but for a few exceptions Aalseth  _et al._ (2011a); Davis  _et al._
(2014); Angloher  _et al._ (2012); Bernabei  _et al._ (2019). To this date,
the DAMA/LIBRA collaboration maintains the sole remaining claim Bernabei  _et
al._ (2019, 1998, 2008a, 2013, 2018), which has been in strong tension with
other experiments Undagoitia and Rauch (2016); The COSINE-100 Collaboration
(2018); Adhikari  _et al._ (2019, 2021); Amare  _et al._ (2016, 2021, 2020);
de Souza  _et al._ (2017). The DAMA/LIBRA detector is situated in the
Laboratori Nazionali del Gran Sasso (LNGS) in Italy at an overburden of 1400 m
Bernabei  _et al._ (2008b). The active volume consists of an array of ultra-
high purity NaI(Tl) crystals with a total active mass of 250 kg Bernabei  _et
al._ (2008a, b), and passively shielded with high-purity copper, lead,
polyethylene, and concrete layers. The dark matter claim was made based on an
event rate modulation observed in the low energy region of the experimental
spectrum. A model-independent analysis of twenty years of DAMA/LIBRA data
reports a modulation of 2–6 keVee event rate over 14 annual cycles with 9.3
sigma detection level Bernabei  _et al._ (2013). Reduction of the energy
threshold to 1 keVee in the latest six annual cycles improves the confirmation
of the annual modulation at a 12.9 sigma detection level Bernabei  _et al._
(2019). Recently, combined data of DAMA/NaI, DAMA/LIBRA-phase1, and
DAMA/LIBRA-phase2 favor the presence of a modulated behavior of 2–6 keVee
event rate at a 13.7 sigma confidence level Belli (2022). A few other NaI-
based experiments have been launched with the goal of performing a model-
independent test of the DAMA/LIBRA claim The COSINE-100 Collaboration (2018);
Adhikari  _et al._ (2019, 2021); Amare  _et al._ (2016, 2021, 2020); de Souza
_et al._ (2017). Among all presently running NaI-based experiments except
DAMA, the SABRE detector demonstrates the lowest background rate
(1.20$\pm$0.05 cpd/kg/keV) in the 1–6 keV energy range Calaprice  _et al._
(2021). However, no experiments have obtained a background rate as low as that
in DAMA/LIBRA yet. As a result, the DAMA/LIBRA annual modulation has not been
definitively confirmed or refuted up to date.

Other authors have sought to explain the DAMA/LIBRA results via alternative
hypotheses based on known physics processes. For example, it has been
demonstrated that a slowly varying time-dependent background can result in an
annual modulation when data are treated in terms of residuals obtained by
subtracting the yearly average background such as that used in the DAMA/LIBRA
analysis Buttazzo  _et al._ (2020). Analysis of COSINE-100 data ignoring
information from the veto detectors and using DAMA/LIBRA analysis methods
shows a slowly decreasing background partly due to the decay of cosmogenically
activated nuclides. However, the residual rate obtained by subtracting the
yearly average background shows an opposite modulation phase to DAMA/LIBRA
Adhikari  _et al._ (2022). It was also proposed that the seasonal modulation
of cosmogenic background could explain the observed DAMA/LIBRA signal Davis
(2014a); Ralston (2010); Zhitnitsky (2020). However, the direct interaction
rates of such processes in the few-keV energy region are estimated to fall
short by orders of magnitude compared to the observed DAMA/LIBRA low-energy
event rate Klinger and Kudryavtsev (2016, 2015); Cherwinka  _et al._ (2016).
The presence of radioactive impurities has also been proposed to explain the
observed low-energy event rate and may have the potential to modulate over
time from event selection criteria. However, DAMA/LIBRA claimed very low
levels of radioactive contaminants Bernabei  _et al._ (2016) that are not
sufficient to offer a viable explanation for the observed low-energy event
rate Kudryavtsev  _et al._ (2010). The claimed impurity contamination level
also rules out the potential of DM scattering on impurities as an explanation
for the observed modulation Profumo and Queiroz1 (2014); McKinsey (2018);
Chang  _et al._ (2011).

One hypothesis suggests that non-radioactive processes such as delayed photon
emission as a result of accumulated energy discharge may account for the
claimed DM signal in the DAMA/LIBRA experiment Nygren (2011). While no
specific mechanism leading to such relaxation was offered, the authors noted
that long-lasting photon production in NaI(Tl) has been observed previously
Nygren (2011). Saint-Gobain, the supplier of NaI(Tl) crystals for the
DAMA/LIBRA experiment, reports that stG (a):

> With mild UV exposure, several pulses per second can be seen in the 6-10 keV
> region of a spectrum. If the crystal is stored in a dark area, this mild UV
> exposure will eventually disappear, although it may take from several hours
> to several days for the effects to stop.

In order for energy discharge following accumulation and storage to explain
the DAMA/LIBRA observation, the photons emitted need to match that of NaI
scintillation to effectively pass the DAMA/LIBRA event selection criteria. A
mechanism known as Self Organized Criticality (SOC) Bak  _et al._ (1988)
offers a feasible model for how such pulses might develop following a gradual
accumulation of energy Pereverzev (2022). In this hypothesis, small amounts of
energy gradually accumulate and are stored in excitonic or other states within
the crystal, which can collectively relax and release the energy in an
avalanche-like process on a time scale much shorter than that required for
energy accumulation. If this hypothesis can be confirmed in NaI(Tl), it may
also provide hints on the recurring background patterns seen in many dark
matter detectors, i.e., the sharp rise of event rate at the energy region
approaching those of excitations in the detector materials Aalseth  _et al._
(2011b); Agnes  _et al._ (2018); Crisler  _et al._ (2018); Sorensen  _et al._
(2011); Aprile  _et al._ (2016, 2019); Pereverzev (2022); Adari  _et al._
(2022).

Consistency with the DAMA/LIBRA data also requires the release of the stored
energy to exhibit seasonal modulation aligned with the observed annual peak in
the DAMA/LIBRA data. If muons were to be responsible for the storage and
release of energy in DAMA/LIBRA , an as-yet-unknown mechanism to shift the
maximum production of such pulses forward from the seasonal maximum in muon
flux by approximately 45 days would be needed Cherwinka  _et al._ (2016);
Hubbard (2015). Environmental factors, such as changes in pressure,
temperature, electric/magnetic fields, mechanical stress, or vibration, might
lead to the annual modulation of delayed light emission, but these effects
will be out of the scope of the current study.

Recognizing that a careful experimental study of delayed light emission
processes in NaI is required, we sought to test the Saint-Gobain observations
in a more controlled fashion than we could find in published literature. While
additional assumptions would be needed for this delayed emission effect to
explain the DAMA/LIBRA low-energy signal modulation, the possibility of pulsed
events with the characteristics of keV-scale recoils produced by energy
accumulation would itself be a significant result. In this work, a 3” NaI(Tl)
crystal was exposed to UV light, MeV-scale gamma-rays, and muons. The detector
response was then monitored for approximately two days following UV and gamma-
ray exposure and a few hundred milliseconds following a muon trigger Cherwinka
_et al._ (2016); Hubbard (2015); Emigh  _et al._ (1954); Kim  _et al._ (2015);
Robertson  _et al._ (1961), we observed delayed light emission following the
different forms of irradiation of the NaI(Tl) crystal. However, our analysis
suggests that the majority of the photons are uncorrelated with each other,
and do not form correlated pulse-like events. Lacking the characteristic time
structure of a typical NaI(Tl)pulse (a correlated emission of light with a
decay time constant of 200–300 ns), these uncorrelated events can largely be
suppressed with pulse-shape-based cuts. However, leakage of these events into
the DAMA/LIBRA signal region remains a possibility. This experiment also does
not rule out the presence of keV-scale pulses whose rate is too low to be seen
in the presence of the strong uncorrelated afterglow signal. We further
demonstrated that exposure of the NaI(Tl) crystal after UV radiation to red
light reduces the intensity of the delayed photon emissions, providing an
opportunity for NaI(Tl) experiments and other low energy-oriented experiments
to suppress potential parasitic backgrounds.

The remainder of this article is organized as follows. In Section II, we
describe the experimental setup and the data acquisition scheme. The temporal
characteristics of the long-lived photon signals are explained in Section
III.1. The possible underlying process leading to such relaxation is
discussed, followed by a discussion on a technique to suppress the delayed
light discovered in this work in Section III.2. The relevance of our
observation to the DAMA/LIBRA experiment is discussed in Section III.3. We
present our conclusion and propose future work in Section IV.

##  II Experimental setup

Figure 1: Experimental setup. The 3 in NaI(Tl) crystal was coupled to two
Hamamatsu R6091 PMTs. The support structure was used to keep the detector and
PMTs concentric. We used two-inch thick lead bricks to passively shield the
target volume against ambient gamma backgrounds.

A 3” $\times$ 3” NaI(Tl) crystal fabricated by Saint-Gobain stG (b) was used
in this experiment. The crystal size was chosen to be sufficiently large
compared to the few-centimeter UV absorption length Mao  _et al._ (2007) of
NaI(Tl) crystals. Saint-Gobain also reported that the effect of UV exposure on
NaI(Tl) is most noticeable in large crystals stG (a). The crystal was
encapsulated in a 0.02-inch thick cylindrical aluminum container with two
0.25-inch thick commercial-grade quartz glass optical windows to view the
crystal. The quartz glass has $>$90% transmission within [300,2000] nm range,
allowing passage of both scintillation and UV light with minimal losses. The
inner cylindrical wall was coated with a proprietary white reflector to
improve light collection. To ensure high light-collection efficiency,
Eljen-550 optical silicone grease was used to couple the crystal module to two
R6091 Hamamatsu head-on photomultiplier tubes that have high gain values
(5$\times 10^{6}$ at 1500 V) and good quantum efficiency ($\sim$26 % at 420
nm) at the scintillation peak of NaI(Tl) crystal r (60). The light yield of
our detector was estimated to be 9.89 photoelectron/keVee (PE/keVee) from the
30.85 keV peak observed in the measured 133Ba spectrum (Fig. 3).

Figure 2: Coincidence trigger scheme used in this experiment. Each PMT signal
was amplified by $\times$10 and split into two copies, with one copy digitized
and the other used to generate a 1$\mu$s coincidence trigger between single
photoelectron pulses in each PMT.

Figure 1 shows the support structure used to keep the assembly concentric and
maintain good optical contact between the crystal and the PMTs. The PMTs were
wrapped in polytetrafluoroethylene (PTFE) tape (POLY-TEMP PN-16050, $>$98%
reflectivity) to maintain the consistency of the light collection. The crystal
and the PMTs were supported by Delrin-based holders, which were secured to an
aluminum base plate. Elastomer bands were used as an additional safety
measure, holding the crystal in place. Two Delrin-based alignment plates,
fiberglass rods, fiberglass flange nuts, and compression springs were used to
apply moderate pressure between the detector and the PMT faceplates (Fig. 1).
To suppress gamma-ray and other ambient backgrounds in low-energy regions, we
placed two-inch-thick lead bricks on the side and below the detector and one-
inch-thick lead bricks on the top of the detector to passively shield the
active volume. The support structure and the surrounding lead shielding were
installed in a dark box.

Full-waveform digitization of the PMT outputs is chosen to provide an accurate
determination of event energy, timing, and pulse shape. An ORTEC 556 high
voltage power supply provides -1700 V and -1600 V to PMT A and PMT B,
respectively. Each PMT signal was amplified using a CAEN V975 unit
(10$\times$-gain), producing two sets of outputs. One set was sent to a
16-channel 250-MHz 14-bit SIS3316 Struck digitizer to be recorded, and the
other was sent to a series of discriminators and logic units to produce a
trigger for the data acquisition system.

In this experiment, we used two different trigger schemes: a coincidence
trigger commonly used in NaI(Tl) detectors and a periodic trigger mode. The
coincidence trigger scheme is illustrated in Fig. 2. The amplified PMT outputs
were first sent to an 8-channel CAEN N840 low-threshold leading-edge
discriminator, which uses a threshold of 5 mV to trigger on nearly all single
photoelectron pulses while rejecting a large fraction of random noise. The
low-threshold discriminator outputs were then sent to LeCroy 821 high-level
discriminator (used as a fast gate generator) to produce 1 $\mu$s coincidence
windows and an LRS 122 logic unit was used to generate a coincidence trigger.
For each coincidence trigger, waveforms were digitized 2 $\mu$s before and 5
$\mu$s after the trigger time. To suppress repetitive triggers within the same
event window, we also use one copy of the LRS 122 coincidence outputs to
produce a 10-$\mu$s pulse in GG 8000 octal gate generator to veto the LRS 122
unit after each trigger. This arrangement ensures pulses recorded by the
digitizer channels are well synchronized but also causes 3 $\mu$s of deadtime
for each trigger. This trigger logic can detect kHz level of keV-like pulses
with high efficiency, but has severe efficiency loss for recording
uncorrelated photoelectron pulses following UV radiation.

In the periodic trigger mode, a forced periodic trigger was provided by a
Tektronik AFG31000 pulse generator operating at 3600 Hz (278 $\mu$s period),
with a 99.9% duty cycle. For each trigger, we recorded waveforms from 6 $\mu$s
before the trigger time to 244 $\mu$s after. The 28 $\mu$s time gap between
recorded waveforms is needed for the digitizer to store the previous waveform.
Both PMT waveforms are recorded separately, and a single p.e. in either is
defined as a valid event. This time window is sufficiently wide to allow for
full recording of NaI(Tl)-like scintillation lights ($\tau$=200–300 ns) and
also enables us to study large time structures for the photons emitted
following UV exposure.

Figure 3: The detector response to 133Ba source with a normalized area under
the curve. For each set of data, the support structure was disassembled and
reassembled to test the reproducibility of our energy response.

All measurements were performed with the crystal module enclosed in the dark
box in a darkened room. When handling of the crystal and PMTs was needed, a
red light (640-700 nm) source was used for room illumination, i.e., the source
light was not directed to the detector assembly. To expose the crystal to the
UV light source (Hg vapor lamp, 365 nm) , we opened the dark box and removed
the top layer of the lead shielding. Parts of the support structure were
disassembled, allowing PMT B to decouple from the NaI(Tl) target (Fig. 1). The
optical grease was wiped clean from the PMT B faceplate and the NaI(Tl)
target’s quartz window prior to the UV exposure. After UV exposure the optical
grease was then reapplied to optically couple the PMT B and the target. The
support structure was reassembled, and the lead shielding was placed back to
cover the top of the detector. We maintained a similar PMT-detector
orientation before and after UV exposure by indexing PMT B and the detector,
as shown in Fig. 1. After UV exposure, the dark box was closed, and the high
voltage power supply was turned on. Data acquisition was then started. To
study the reproducibility of the detector response using the assembly and
disassembly procedure, we measured the energy spectra of 133Ba after repeating
this procedure. As shown in Fig. 3, variation in the detector response is
negligible. We comment that we monitor single PE response during the data-
taking period to ensure PMT response stability.

Figure 4: Examples of identified peaks within the 250 $\mu$s time window 15
minutes, one hour, and two days since the UV exposure.

##  III Results and discussion

After exposing the NaI(Tl) crystal to a 4-W UV light source (365 nm) from a 20
cm distance for 4 minutes (roughly 1.6$\times 10^{19}$ photons), we observed
delayed light emission in the detector that lasted for days. The intensity of
light emission was around 1 PE/$\mu$s shortly after the UV radiation, and it
decreased monotonically over time. Similar long-lived delayed light emission
was also observed after the NaI(Tl) crystal was irradiated by a $\sim$20
$\mu$Ci 60Co source from a 3 cm distance for three hours, and after large
energy depositions in the crystal ($>$10,000 PE) consistent with passage of
muons. The observed light emission consists primarily of single photoelectron
pulses and does not appear to contain pulse-like structures. These
observations are consistent with previous reports of light emission in NaI
scintillating crystals following particle interactions such as cosmic ray
Cuesta  _et al._ (2014, 2014); Koiki  _et al._ (1973), radioactive sources
Koiki  _et al._ (1973), atmospheric muon Kim  _et al._ (2015), and visible
lights Fronka (1971); Delaney  _et al._ (1968) with decay time ranging from
$\mu$s to days.

We rule out the PMTs from being a leading cause of this photoelectron emission
because only PMT A is exposed to the UV light (through the NaI(Tl) crystal
module), but both PMT A and PMT B observed comparable amounts of photoelectron
pulses in this experiment. The quartz windows of the crystal module were
exposed to UV light in this experiment, and quartz/fused quartz has been
previously reported to emit delayed light lasting from minutes to hours
following UV irradiation or X-ray/beta ray exposure in room temperature Coop
and Hammond (1962); Webb and Messenger (1929); M.Martini  _et al._ (1994);
Bailey (1926). However, a report on the afterglow measurement following UV
irradiation on alkali halide crystal (KCl) through a quartz window at room
temperature suggests that the dominant component in the afterglow is
consistent with a known process in the crystal rather than the quartz window
de Carcer  _et al._ (1998).

###  III.1 Waveform characterization

For the characterization of this delayed photon emission, we use the periodic
trigger scheme to avoid trigger biases. Peaks were first identified in the
recorded raw data containing waveforms from PMT A and PMT B, and the charge
integrals within each peak were scaled based on the PMT’s single photoelectron
response. The scaled PMT responses were then combined to produce a summed
detector event waveform. Fig. 4 shows examples of the resulting combined
response 15 minutes, one hour, and two days since the UV exposure. The width
of the bin is set to be 20 ns, which is finer compared to NaI(Tl) pulse decay
time. We comment that 15 minutes is the approximate time required to
reassemble the detector support structure after UV radiation is stopped.

(a)

(b)

Figure 5: An example of a simulated waveform with a 2 keV ($\sim$20 PE)
NaI(Tl) pulse among relatively intense uncorrelated pulses (1 PE/$\mu$s) (a)
still shows a noticeable Lorentzian-like feature in the frequency domain (b).
The frequency domain is shown in (b). Smaller simulated pulses produce less
noticeable features. The corresponding response in the frequency domain is
zoomed from 0 to 10 MHz to show the Lorentzian-like feature. The Lorentzian
profile is shown in red.

(a)

(b)

Figure 6: An example of a simulated waveform populated with uncorrelated
pulses (1 PE/$\mu$s) (a) shows a uniform response in the frequency domain. The
analysis is applied to ten such a waveform, and the average response in the
frequency domain is shown in (b). The corresponding response in the frequency
domain is zoomed from 0 to 10 MHz to show the absence of the Lorentzian-like
feature (red).

We apply Fast Fourier transform (FFT) analysis to the acquired event waveforms
to identify their frequency components. FFT is a mathematical operation that
decomposes a signal in the time domain into an alternate representation in the
frequency domain Korener (1988); Morin . A random flow of uncorrelated
photoelectron pulses would produce a flat frequency spectrum (aka white noise
spectrum), while clustered photoelectron structures may demonstrate themselves
as nontrivial features at characteristic frequencies. For the case of NaI(Tl)
scintillation pulses, which can be modeled as an exponential decay with a
characteristic time constant, the FFT should produce a Lorentzian-like feature
in the frequency domain Morin ; Korener (1988).

To test the power of the FFT analysis to identify NaI(Tl) pulse-like features
in a random background-dominated waveform, we performed a toy Monte Carlo
simulation to mimic the situation when one 2 keV ($\sim$20 PE) NaI(Tl)
scintillation pulse is present in a 250 $\mu$s time window with the highest
observed photoelectron background rate ($\sim$1PE/$\mu$s around 15 minutes
after UV radiation). First, we generated 250 event times randomly distributed
within 250 $\mu$s time window. Next, one event is treated as a NaI(Tl)
scintillation pulse containing 20 single-photoelectron (SPE) peaks following
an exponential decay with 250 ns time constant, and the rest of 249 events are
simply SPE pulses. For each simulated PE pulse, the height was sampled from
the measured SPE Gaussian distribution with a mean of 211 ADC counts and sigma
of 128 ADC counts. Figure 5 shows an example waveform generated by the
simulation with 20 ns time bins and the corresponding FFT response in the
frequency domain. Note that the amplitude in the frequency domain is the
magnitude of the transform ($\sqrt{\text{Re}^{2}+\text{Im}^{2}}$). The red
line in Fig. 5(a) corresponds to the Lorentzian profile fitted into the
frequency distribution. In contrast, when the 2 keV pulse is removed from the
simulations, the FFT spectrum becomes a flat white noise response, as
illustrated in Figure 6. The Lorentzian profile fitted to the frequency
distribution (Fig. 6(b), red line) does not show the expected shape.

Figure 7: Fast Fourier Transform of NaI(Tl) energy response 15 minutes, one
hour, and two days since the UV exposure shows uniform distribution,
suggesting that most of the photoelectrons are uncorrelated with each other.

The FFT spectra measured at 15 minutes, one hour, and two days after the UV
exposure (example waveforms are given in Figure 4) are shown in Fig. 7. The
absence of features in the frequency domain, in combination with the toy Monte
Carlo simulation study, suggests that the observed photoelectrons following UV
exposure are dominated by uncorrelated single-photoelectron pulses.

We acknowledge that the FFT analysis has a finite sensitivity and can be
obscured if the pulse-like events have average energy significantly below 2
keV or if the pulse occurrence is substantially below one per 250 $\mu$s as we
assumed in the toy simulation. Therefore, this analysis does not rule out the
Saint Gobain report of few-keV pulse-like events (contain time-correlated
photons by definition) produced by UV light. Request to Saint Gobain for more
technical information on their definition of pulse-like events in their
measurement and how the rate of these events was calculated was unfruitful. A
more in-depth analysis of our recorded data may be able to identify occasional
few-keV events that are in excess to radiation interactions in the NaI(Tl)
crystal and are masked by the high rate of uncorrelated single photoelectron
events but is beyond the scope of this study. For this work, we simply report
that the substantial and dominant emission of single-photoelectron events
observed following UV exposure are uncorrelated.

###  III.2 The delayed emission rate and its suppression

Figure 8: Example of the distribution of the number of observed photoelectron
peaks 45 minutes to one hour after the UV exposure.

In this section, we characterize the rate of the delayed photon emission from
the NaI(Tl) crystal following various forms of radiation. For this study, the
coincidence trigger, as detailed in Section. II, was used to reduce the data
rate. To mitigate bias from the coincidence trigger requirement, we exclude
the waveform segments both before and around the trigger time in the analysis
and only focus on the time window 1.5 $\mu$s to 4.0 $\mu$s after the trigger
time. Because the waveforms are dominated by the uncorrelated flow of
photoelectron pulses, recorded activities in this post-trigger time window
approximate that in a triggerless scenario. We comment that the number of
observed photoelectron peaks in this post-trigger time window approximately
follows a Poisson distribution, which further confirms the uncorrelated
feature of these photoelectron features. Fig. 8 shows an example of the
distribution of the number of observed photoelectron peaks 45 minutes to one
hour after the UV exposure. The Poisson profile (Fig. 8, red line) used to fit
the distribution shows respectable $\chi^{2}$/NDF.

Fig. 9(a) shows the mean number of observed photoelectron peaks in the post-
trigger window from Poisson fits as a function of time. The intensity of light
emission in the NaI(Tl) crystal decreases monotonically over time, with an
observed photoelectron rate of $\sim$1 PE/$\mu$s shortly after the UV
radiation, which decreases to less than 0.1 PE/$\mu$s after a day. We comment
that the UV experiment was repeated and resulted in a broadly similar trend,
demonstrating the reproducibility of the result. A control experiment, in
which the same detector disassembly and reassembly procedure was exercised but
without the UV light source turned on, was also performed. In this test, no
increase in photoelectron rate was observed (Fig. 9, a), confirming that the
experimental procedure does not contribute to the delayed light emission.

Following 60Co radiation, the photon background rate decreases faster than
that observed for UV radiation, and the rate returns to the baseline after
several hours. Between subsequent measurements, the NaI(Tl) detector is left
in the dark box for at least a week to allow the delayed light emission rate
to return to the baseline and not contaminate later measurements. In the case
of single muons passing the crystal, we search for large ionization events
($>$10,000 PE) in the background data (Fig. 9(a), dotted black line) and
monitor the intensity of the number of observed photoelectrons before and
after the large ionization event. The result is shown in Fig. 9(b). Note that
we exclude time zero, which corresponds to the time of the large ionization
event, resulting in the observed discontinuity observed in Fig. 9(b). It has
been reported that properties of delayed light emission in alkali halide, such
as intensity, spectral composition, and decay time, are affected by the purity
of raw material, heat treatment, and imparted radiation doses Rodnyi (1997).
Hence, the intensity, irradiation time, and radiation modes may alter the
observed decay time of the luminescence.

(a)

(b)

Figure 9: The delayed light intensity was monitored over time following UV
exposure (a, solid black line) and Co-60 irradiation (a, solid blue line).
Three hours after the UV exposure, the crystal was exposed to red light (a,
solid red line); and a pronounced decrease in the afterglow intensity was
observed. The shaded green box corresponds to the red light irradiation time.
Control runs were performed, in which exposure with the UV or red light
sources was omitted, but the same procedure was used for source placement as
in the runs with exposure. This check confirmed that the experimental
procedure had a negligible contribution to the observed trend (a, dashed
lines). We also observed delayed light following large energy depositions in
the crystal (b). Note that we omit the data point at the zero time, which is
the time of the large ionization event (b).

While the precise mechanisms of delayed light production by NaI(Tl) following
UV radiation are not well understood at present, long-lived metastable
activator states and trapping due to UV-induced crystal defects can be partly
responsible Rodnyi (1997). The former refers to an activator center (Tl)
excited to a forbidden state, in which case additional energy is required to
promote the activator to another state that is allowed to decay. The
additional energy is typically gained from thermal excitation. The resulting
delayed light emission hence follow the metastable activator state lifetime
and are postulated to range from $\mu$s to days. The latter refers to excitons
that are immobilized by lattice distortion field William and Song (1990) and
also require additional energy to hop from site to site. In both cases,
external energy injection into the system can expedite the decay of the
metastable and trapped states and thus suppress long-lived light emission.

As suggested in Pereverzev (2022), we next tested injecting energy into the
NaI(Tl) crystal by radiating it with red light after UV exposure to study how
it may change the delayed photon emission. First, we exposed the NaI(Tl)
crystal to UV light (365 nm, 4 W, 4 min, 20 cm distance) as described in Sec.
III.1 and monitored the light emission for approximately 3 hours. Then the PMT
high voltage supplies were turned off, the detector was disassembled, and the
crystal was bathed in red light (640–700 nm) from a Dewalt DCL043 unit (20 W)
for 30 minutes. The absorption length of red light in NaI(Tl) is calculated to
be at the order of tens of cm Bondarenko  _et al._ (2002), so the entire
crystal volume (3 in.) was effectively exposed to the red light in this
process. Afterward, the detector was assembled again, and the crystal
scintillation response continued to be monitored by the DAQ. Such a procedure
means that the measured intensity would drop to zero during the red light
irradiation (Fig. 9 (a), shaded green region).

Fig. 9 (a) shows the delayed photon intensity following UV radiation with and
without red light exposure; a large drop in the light intensity is observed
after the red light exposure, and the residual light emission rate also
decreases at a faster pace. A control experiment was carried out, in which the
identical procedure was followed only without the red light turned on after
the detector was disassembled. In the controlled experiment, the delayed light
emission rate appears to follow a continuous decreasing trend with no abrupt
change following the detector disassembly and reassembly procedure.
Simultaneous exposure to red light and 60Co irradiation shows a similar
decrease in the delayed light emission intensity, as shown in Fig. 9(a) (solid
blue line). We note that the idea of energy injection via longer-wavelength
photon irradiation to force the release of trapped states also has been
proposed for other detectors. For example, the effect of infrared light
irradiation to release trapped electrons by electronegative impurities in a
liquid Xenon TPC detector was investigated but the effect was found to be
minimal Kopec  _et al._ (2021). To the best of our knowledge, our result is
the first demonstration of luminescence reduction following UV and gamma
radiation via red light exposure in NaI(Tl) detector.

###  III.3 Relevance to NaI(Tl)-based dark matter experiments

The DM-Ice17 collaboration extensively studied the muon-induced delayed light
emission and whether it can mimic the DAMA/LIBRA modulation Cherwinka  _et
al._ (2016); Hubbard (2015). They concluded that this process is unlikely to
be the sole source of the DAMA/LIBRA modulation for two reasons. First, the
event rate is too low compared to the DAMA/LIBRA observation (even neglecting
noise suppression cuts in the analysis), and second, the phase of the muon
flux annual pattern is off by approximately 45 days compared to the DAMA/LIBRA
modulation. However, it is possible that other forms of energy storage in the
DAMA/LIBRA crystals from both internal and external radiation (such as alpha
decays from U/Th chains), in combination with different sources of annual
variation Davis (2014b) include temperature, pressure, vibration, electric
field hayashi  _et al._ (1972) in addition to muon flux, may still play a role
in the DAMA/LIBRA experiment. We also comment that delayed light emission may
appear as a single-hit event or a multiple-hit event depending on the type of
particles inducing the delayed light production. For example, muon passage
across an array of NaI(Tl) crystals may result in multiple-hit events. In
contrast, short-ranged particles like alpha likely produce single-hit events.

(a)

(b)

Figure 10: 133Ba data reveals the position of traditional NaI(Tl) pulse (red
circle) in the $X_{1}$-$X_{2}$ parameter space (a). UV-induced delayed light
emission has the potential to leak into the $X_{1}$-$X_{2}$ region of genuine
NaI(Tl) pulse (b).

Here we consider the possibility of radiation-induced delayed light emission
leaking through some DAMA/LIBRA -like pulse shape-based event selection cuts.
A train of uncorrelated single photoelectrons can pass the trigger coincidence
requirement commonly employed by dark matter experiments, and these events may
further pass analysis cuts designed to select NaI(Tl) scintillation pulses.
DAMA/LIBRA defines pulse shape parameters, $X_{1}$ and $X_{2}$, or the
fraction of delayed and prompt pulse integrals, respectively, to discriminate
NaI(Tl) scintillation events from noise-like events Bernabei  _et al._
(2008b).

$X_{1}=\frac{A[100,600]~{}ns}{A[0,600]~{}ns},X_{2}=\frac{A[0,50]~{}ns}{A[0,600]~{}ns}$
(1)

Because NaI(Tl) scintillation pulses have a relatively long decay time, while
PMT noise events that pass the coincidence requirement are more likely to have
short durations, a genuine NaI(Tl) scintillation event is expected to have a
large $X_{1}$ and a relatively low $X_{2}$ value.

In this exercise, we choose an event window of 2048 ns after the trigger time,
which is the same as the DAMA/LIBRA ’s time recording window. To simulate the
effect of additional pulse-selection cuts, we also require a candidate event
to have an integrated charge less than 10% of the total event energy in the
pre-trigger region [-1,-0.5] $\mu$s and at a later time window [1,1.5] $\mu$s.
Additionally, we only accept events within 1–6 keV energy regions. We took a
133Ba data to show the expected $X_{1}$-$X_{2}$ response to genuine NaI(Tl)
pulses (Fig. 10(a)). The red circle corresponds to the region where genuine
NaI(Tl) pulses are located. The heavy population in the
($X_{1}$,$X_{2}$)$=$(1,0), the horizontal line at $X_{2}=$0, and the diagonal
line extending from ($X_{1}$,$X_{2}$)$=$(1,0) to ($X_{1}$,$X_{2}$)$=$(0,1) are
expected from dark counts. Since the UV-induced delayed light emission has a
dominant uncorrelated component, the resulting $X_{1}$-$X_{2}$ response mimics
that of dark counts, as shown in Fig. 10(b). Although the $X_{1}$-$X_{2}$
responses are significantly different for genuine NaI(Tl) pulses produced by
gamma-ray interactions and for delayed light emission, the UV-induced
background is capable of contaminating the region where genuine NaI(Tl) pulses
are located, as shown in Fig. 10(b). Note that this is simply an illustrative
analysis, and the contamination level may vary from one experiment to another
due to the variations in the crystal properties, data acquisition methods, and
analysis cuts. If a NaI(Tl)-based dark matter experiment observes leakages of
backgrounds in this manner, they would need to characterize the leakage
fraction for each crystal and consider suppressing this parasitic background
by injecting small amounts of energy into the crystal to force energy
relaxation, as demonstrated in Sec. III.2.

##  IV Conclusions

We exposed a 3” NaI(Tl) crystal to a 365 nm UV light source and observed
strong light emission that lasted for multiple days, which may be explained by
energy accumulation and delayed release in the detector. Gamma-ray irradiation
and muon events are also observed to produce similar phenomena. We found that
uncorrelated photons dominate such light emission, but this background could
still pass a coincidence trigger and be mistakenly categorized as keV-scale
NaI(Tl) scintillation events by DAMA/LIBRA -style pulse shape selection cuts.
These observations may prompt reconsideration of seasonal-modulated external
sources as a possible explanation for the DAMA/LIBRA signal. For example, muon
flux, changes in temperature, vibration due to the nearby traffic, pressure
variation, and electric/magnetic field hayashi  _et al._ (1972) exhibit yearly
patterns, and a combination of external energy inputs may yield apparent
modulation similar to that observed in DAMA/LIBRA .

We found that red light exposure suppresses the intensity of delayed light
emission in NaI(Tl) by stimulating the dissipation of stored energy in the
system. However, the use of red light in an actual experiment may be difficult
due to the finite sensitivities of common photocathodes in the red light
region. Future work should investigate the potential of longer-wavelength (IR
or mid-IR) sources to suppress the production of delayed light. Other stimuli,
such as small but well-controlled temperature or pressure changes, or
vibrations, may also impact the accumulation and dissipation of stored energy
in NaI(Tl) detectors or other systems.

##  V Acknowledgements

We thank D. Nygren and F. Calaprice for their helpful discussion on the
potential contribution of accumulated energy discharge to the observed DM
signal in the DAMA/LIBRA experiment. We also thank Antonia Hubbard for
insightful discussions on the muon-induced delayed light emission in NaI(Tl)
crystal. This project is supported by the U.S. Department of Energy (DOE)
Office of Science/High Energy Physics under Work Proposal Number SCW1508
awarded to Lawrence Livermore National Laboratory (LLNL). LLNL is operated by
Lawrence Livermore National Security, LLC, for the DOE, National Nuclear
Security Administration (NNSA) under Contract DE-AC52-07NA27344 [LLNL release
number LLNL-JRNL-837382].

## References

  * Aalseth  _et al._ (2011a) C. E. Aalseth _et al._ , Phys. Rev. Lett. 106, 131301 (2011a).
  * Davis  _et al._ (2014) J. H. Davis, C. McCabe, and C. Boehm, J. Cosmol. Astropart. Phys 08, 014 (2014).
  * Angloher  _et al._ (2012) G. Angloher _et al._ , Eur. Phys. J. C 72, 1971 (2012).
  * Bernabei  _et al._ (2019) R. Bernabei, P. Belli, F. Cappella, V. Caracciolo, R. Cerulli, C. J. Dai, A. d’Angelo, A. D. Marco, H. L. He, A. Incicchitti, X. H. Ma, V. Merlo, F. Montecchia, X. D. Sheng, and Z. P. Ye, Nuclear Physics and Atomic Energy 20, 317 (2019).
  * Bernabei  _et al._ (1998) R. Bernabei, P. Belli, F. Montecchia, W. di Nicolantonio, A. Incicchitti, D. Prosperi, C. Bacci, C. J. Dai, L. K. Ding, H. H. Kuang, and J. M. Ma, Phys. Lett. B 424, 195 (1998).
  * Bernabei  _et al._ (2008a) R. Bernabei, P. Belli, F. Cappella, R. Cerulli, C. J. Dai, A. d’Angelo, H. L. He, A. Incicchitti, H. H. Kuang, J. M. Ma, F. Montecchia, F. Nozzoli, D. Prosperi, X. D. Sheng, and Z. P. Ye, Eur. Phys. J. C 56, 333– (2008a).
  * Bernabei  _et al._ (2013) R. Bernabei, P. Belli, F. Cappella, V. Caracciolo, S. Castellano, R. Cerulli, C. J. Dai, A. d’Angelo, S. d’Angelo, A. D. Marco, H. L. He, A. Incicchitti, H. H. Kuang, X. H. Ma, F. Montecchia, D. Prosperi, X. D. Sheng, R. G. Wang, and Z. P. Ye, Eur. Phys. J. C 73, 2648 (2013).
  * Bernabei  _et al._ (2018) R. Bernabei, P. Belli, A. Bussolotti, F. Cappella, V. Caracciolo, R. Cerulli, C. J. Dai, A. d’Angelo, A. D. Marco, H. L. He, A. Incicchitti, X. H. Ma, A. Mattei, V. Merlo, F. Montecchia, X. D. Sheng, and Z. P. Ye, Nuclear Physics and Atomic Energy 19, 307 (2018).
  * Undagoitia and Rauch (2016) T. M. Undagoitia and L. Rauch, J. Phys. G: Nucl. Part. Phys. 43, 013001 (2016).
  * The COSINE-100 Collaboration (2018) The COSINE-100 Collaboration, Nature 564, 83 (2018).
  * Adhikari  _et al._ (2019) G. Adhikari _et al._ , Phys. Rev. Lett. 123, 031302 (2019).
  * Adhikari  _et al._ (2021) G. Adhikari _et al._ , Scince Advances 7 (2021).
  * Amare  _et al._ (2016) J. Amare _et al._ , J. Phys. Conf. Ser. 718, 042052 (2016).
  * Amare  _et al._ (2021) J. Amare _et al._ , Phys. Rev. D 103, 102005 (2021).
  * Amare  _et al._ (2020) J. Amare _et al._ , J. Phys.: Conf. Ser. 1468, 012014 (2020).
  * de Souza  _et al._ (2017) E. B. de Souza _et al._ , Phys. Rev. D 95, 032006 (2017).
  * Bernabei  _et al._ (2008b) R. Bernabei, P. Belli, A. Bussolotti, F. Cappella, R. Cerulli, C. J. Dai, A. d’Angelo, H. L. He, A. Incicchitti, H. H. Kuang, J. M. Ma, A. Mattei, F. Montecchia, F. Nozzoli, D. Prosperi, X. D. Sheng, and Z. P. Ye, Nucl. Instrum. Methods Phys. Res. A592, 297 (2008b).
  * Belli (2022) P. Belli, The 14th conference on the identification of dark matter (2022).
  * Calaprice  _et al._ (2021) F. Calaprice _et al._ , Phys. Rev. D 104, L021302 (2021).
  * Buttazzo  _et al._ (2020) D. Buttazzo, P. Panci, N. Rossi, and A. Strumia, JHEP 2020, 137 (2020).
  * Adhikari  _et al._ (2022) G. Adhikari _et al._ , (2022), arXiv:2208.05158 [hep-ex] .
  * Davis (2014a) J. H. Davis, Phys. Rev. Lett. 113, 081302 (2014a).
  * Ralston (2010) J. P. Ralston, (2010), arXiv:1006.5255 [hep-ph] .
  * Zhitnitsky (2020) A. Zhitnitsky, Phys. Rev. D 101, 083020 (2020).
  * Klinger and Kudryavtsev (2016) J. Klinger and V. A. Kudryavtsev, J. Phys.: Conf. Ser. 718, 0042033 (2016).
  * Klinger and Kudryavtsev (2015) J. Klinger and V. A. Kudryavtsev, Phys. Rev. Lett. 114, 151301 (2015).
  * Cherwinka  _et al._ (2016) J. Cherwinka _et al._ , Phys. Rev. D 93, 042001 (2016).
  * Bernabei  _et al._ (2016) R. Bernabei, P. Belli, A. Incicchitti, and C. J. Dai, Int. J. Mod. Phys. A 31, 1642003 (2016).
  * Kudryavtsev  _et al._ (2010) V. Kudryavtsev, M. Robinson, and N. Spooner, Astroparticle Physics 33, 91 (2010).
  * Profumo and Queiroz1 (2014) S. Profumo and F. S. Queiroz1, J. Cosmol. Astropart. Phys. 2014, 038 (2014).
  * McKinsey (2018) D. N. McKinsey, (2018), arXiv:1803.10110 [hep-ex] .
  * Chang  _et al._ (2011) S. Chang, R. F. Lang, and N. Weiner, Phys.Rev.Lett. 106, 011301 (2011).
  * Nygren (2011) D. Nygren, (2011), arXiv:1102.0815 [astro-ph.IM] .
  * stG (a) Saint Gobain Technical Note https://www.crystals.saint-gobain.com/sites/hps-mac3-cma-crystals/files/2021-11/Effects-of-UV-Light-on-NaI-CsI-BGO.pdf (a).
  * Bak  _et al._ (1988) P. Bak, C. Tang, and K. Wiesenfeld, Phys. Rev. A 38, 364 (1988).
  * Pereverzev (2022) S. Pereverzev, Phys. Rev. D 105, 063002 (2022).
  * Aalseth  _et al._ (2011b) C. E. Aalseth _et al._ , Phys. Rev. Lett. 106, 131301v (2011b).
  * Agnes  _et al._ (2018) P. Agnes _et al._ , Phys. Rev. Lett. 121, 081307 (2018).
  * Crisler  _et al._ (2018) M. Crisler _et al._ , Phys. Rev. Lett. 121, 061803 (2018).
  * Sorensen  _et al._ (2011) P. Sorensen _et al._ , PoS IDM 2010, 017 (2011).
  * Aprile  _et al._ (2016) E. Aprile _et al._ , Phys. Rev. D 94, 092001 (2016).
  * Aprile  _et al._ (2019) E. Aprile _et al._ , Phys. Rev. Lett. 123, 251801 (2019).
  * Adari  _et al._ (2022) P. Adari _et al._ , (2022), arXiv:arXiv:2202.05097v2 [astro-ph.IM] .
  * Hubbard (2015) A. J. F. Hubbard,  _Muon-Induced Backgrounds in the DM-Ice17 NaI(Tl) Dark Matter Detector_ , Ph.D. thesis, University of Wisconsin–Madison (2015).
  * Emigh  _et al._ (1954) C. R. Emigh _et al._ , Phys. Rev. 93, 1190 (1954).
  * Kim  _et al._ (2015) K. W. Kim _et al._ , Astroparticle Physics 62, 249 (2015).
  * Robertson  _et al._ (1961) J. C. Robertson _et al._ , Proceedings of the Physical Society 77, 751 (1961).
  * stG (b) Saint Gobain Technical Note https://www.crystals.saint-gobain.com/radiation-detection-scintillators/crystal-scintillators/naitl-scintillation-crystals (b).
  * Mao  _et al._ (2007) R. Mao, L. Zhang, and R.-Y. Zhu, IEEE Nuclear Science Symposium Conference (2007).
  * r (60) Hamamatsu Specification Sheet https://www.hamamatsu.com/resources/pdf/etd/R6091_TPMH1114E.pdf .
  * Cuesta  _et al._ (2014) C. Cuesta _et al._ , (2014), arXiv:1407.5125 [astro-ph] .
  * Koiki  _et al._ (1973) S. Koiki _et al._ , Nuclear Instruments and Methods A108, 297 (1973).
  * Fronka (1971) O. Fronka, physica status solidi 6, K89 (1971).
  * Delaney  _et al._ (1968) C. Delaney _et al._ , The International Journal of Applied Radiation and Isotopes 19, 169 (1968).
  * Coop and Hammond (1962) W. H. Coop and J. A. Hammond, J. Opt. Soc. Am. 52, 835 (1962).
  * Webb and Messenger (1929) H. W. Webb and H. A. Messenger, Phys. Rev. D 34, 1463 (1929).
  * M.Martini  _et al._ (1994) M.Martini, G.Spinolo, A.Vedda, and C.Arena, Solid State Communications 91, 751 (1994).
  * Bailey (1926) A. C. Bailey, Proceedings of the Iowa Academy of Science 33, 242 (1926).
  * de Carcer  _et al._ (1998) I. A. de Carcer, A. P. Rowlands, F. Jaque, and P. D. Townsend, Radiation Measurements 29, 203 (1998).
  * Korener (1988) T. W. Korener, (1988).
  * (61) D. Morin, http://www.phy.pmf.unizg.hr/~npoljak/files/vjezbe/2016_OF3/Morin_Waves.pdf .
  * Rodnyi (1997) P. A. Rodnyi,  _Physical Processes in Inorganic Scintillators_ (Taylor & Francis, 1997).
  * William and Song (1990) R. T. William and K. S. Song, J Phys Chem Solids 51, 679 (1990).
  * Bondarenko  _et al._ (2002) S. K. Bondarenko, L. V. Udovichenko, A. I. Mitichkin, N. Kosinov, and I. V. Khromaya, Journal of Applied Spectroscopy 69, 901 (2002).
  * Kopec  _et al._ (2021) A. Kopec, A. L. Baxter, M. Clark, R. F. Lang, S. Li, J. Qin, and R. Singh, JINST 16, P07014 (2021).
  * Davis (2014b) J. H. Davis, Phys. Rev. Lett. 113, 081302 (2014b).
  * hayashi  _et al._ (1972) T. hayashi, S. Koshino, Y. Kawai, and T. Ohata, J. Phys. Soc. Jpn. 33, 1018 (1972).