∎

corrauthor0e-mail<EMAIL_ADDRESS>corrauthor1e-mail<EMAIL_ADDRESS>
11institutetext:  Department of Physics and Astronomy, Seoul National
University, Seoul 08826, Republic of Korea22institutetext: Department of
Physics and Wright Laboratory, Yale University, New Haven, CT 06520,
USA33institutetext: Physics Institute, University of São Paulo, 05508-090, São
Paulo, Brazil44institutetext: Center for Underground Physics, Institute for
Basic Science (IBS), Daejeon 34126, Republic of Korea55institutetext:
Department of Physics and Astronomy, University of Sheffield, Sheffield S3
7RH, United Kingdom66institutetext: Department of Physics, Chung-Ang
University, Seoul 06973, Republic of Korea77institutetext: Center for Exotic
Nuclear Studies, Institute for Basic Science (IBS), Daejeon 34126, Republic of
Korea88institutetext: Department of Science Education, Ewha Womans University,
Seoul 03760, Republic of Korea99institutetext: IBS School, University of
Science and Technology (UST), Daejeon 34113, Republic of
Korea1010institutetext: Department of Physics and Wisconsin IceCube Particle
Astrophysics Center, University of Wisconsin-Madison, Madison, WI 53706,
USA1111institutetext: Department of Physics, Kyungpook National University,
Daegu 41566, Republic of Korea1212institutetext: Department of Physics, Sejong
University, Seoul 05006, Republic of Korea1313institutetext: Korea Research
Institute of Standards and Science, Daejeon 34113, Republic of
Korea1414institutetext: Department of Accelerator Science, Korea University,
Sejong 30019, Republic of Korea1515institutetext: Department of Physics and
IQS, Chungnam National University, Daejeon 34134, Republic of
Korea1616institutetext: Department of Physics, Universitas Negeri Malang,
Malang 65145, Indonesia1717institutetext: Department of Physics, Sungkyunkwan
University, Suwon 16419, Republic of Korea1818institutetext: Department of
Physics and Astronomy, University of Utah, Salt Lake City, UT 84112, USA

# Nonproportionality of NaI(Tl) Scintillation Detector for Dark Matter Search
Experiments

S. M. Leecorrauthor0,addr:snu  0000-0002-7326-3317 G. Adhikariaddr:yale  N.
Carlinaddr:saopaulo  J. Y. Choaddr:cup  J. J. Choiaddr:snu,addr:cup  S.
Choiaddr:snu  A. C. Ezeribeaddr:sheffield  L. E. Françaaddr:saopaulo  C.
Haaddr:cau  I. S. Hahnaddr:cens,addr:ewu,addr:ibs  S. J. Hollickaddr:yale  E.
J. Jeonaddr:cup,addr:ibs  H. W. Jooaddr:snu  W. G. Kangaddr:cup  M.
Kaueraddr:wismad  B. H. Kimaddr:cup  H. J. Kimaddr:knu  J. Kimaddr:cau  K. W.
Kimaddr:cup  S. H. Kimaddr:cup  S. K. Kimaddr:snu  S. W. Kimaddr:cup  W. K.
Kimaddr:ibs,addr:cup  Y. D. Kimaddr:cup,addr:sejong,addr:ibs  Y. H.
Kimaddr:cup,addr:kriss,addr:ibs  Y. J. Kocorrauthor1,addr:cup
0000-0002-5055-8745 D. H. Leeaddr:knu  E. K. Leeaddr:cup  H.
Leeaddr:ibs,addr:cup  H. S. Leeaddr:cup,addr:ibs  H. Y. Leeaddr:cens  I. S.
Leeaddr:cup  J. Leeaddr:cup  J. Y. Leeaddr:knu  M. H. Leeaddr:cup,addr:ibs  S.
H. Leeaddr:ibs,addr:cup  Y. J. Leeaddr:cau  D. S. Leonardaddr:cup  N. T.
Luanaddr:knu  B. B. Manzatoaddr:saopaulo  R. H. Maruyamaaddr:yale  R. J.
Nealaddr:sheffield  J. A. Nikkeladdr:yale  S. L. Olsenaddr:cup  B. J.
Parkaddr:ibs,addr:cup  H. K. Parkaddr:koreau  H. S. Parkaddr:kriss  J. C.
Parkaddr:cnu  K. S. Parkaddr:cup  S. D. Parkaddr:knu  R. L. C.
Pittaaddr:saopaulo  H. Prihtiadiaddr:negerimalang  S. J. Raaddr:cup  C.
Rottaddr:skku,addr:utah  K. A. Shinaddr:cup  D. F. F. S.
Cavalcanteaddr:saopaulo  A. Scarffaddr:sheffield  M. K. Sonaddr:cnu  N. J. C.
Spooneraddr:sheffield  L. T. Trucaddr:knu  L. Yangaddr:saopaulo  G. H.
Yuaddr:skku,addr:cup
(COSINE-100 Collaboration)

(Received: / Accepted: N/A)

###### Abstract

We present a comprehensive study of the nonproportionality of NaI(Tl)
scintillation detectors within the context of dark matter search experiments.
Our investigation, which integrates COSINE-100 data with supplementary
$\gamma$ spectroscopy, measures light yields across diverse energy levels from
full-energy $\gamma$ peaks produced by the decays of various isotopes. These
$\gamma$ peaks of interest were produced by decays supported by both long and
short-lived isotopes. Analyzing peaks from decays supported only by short-
lived isotopes presented a unique challenge due to their limited statistics
and overlapping energies, which was overcome by long-term data collection and
a time-dependent analysis. A key achievement is the direct measurement of the
0.87 keV light yield, resulting from the cascade following electron capture
decay of $\rm{{}^{22}Na}$ from internal contamination. This measurement,
previously accessible only indirectly, deepens our understanding of NaI(Tl)
scintillator behavior in the region of interest for dark matter searches. This
study holds substantial implications for background modeling and the
interpretation of dark matter signals in NaI(Tl) experiments.

††journal: Eur. Phys. J. C

##  1 Introduction

In 1950, Robert W. Pringle observed that the light output of NaI(Tl)
scintillation detector is not proportional to the incident photon energy
pringle_gamma-rays_1950 . This discovery sparked numerous studies on
nonproportionality (nPR) between light output and the incident
electron$/\gamma$ energy using various scintillation detectors
engelkemeir_nonlinear_1956 ; aitken_fluorescent_1967 ;
leutz_scintillation_1997 ; wayne_response_1998 ; swiderski_response_2013 ;
devare_effect_1963 ; collinson_fluorescent_1963 ; jones_nonproportional_1962 .
The nPR phenomenon has been observed in various scintillators, including
NaI(Tl), but the detailed values were published differently in each
experiment. To understand the physics behind the phenomenon and the
discrepancies observed, numerous remarkable studies have been published over
the decades. These studies include the improvement of measurement techniques
khodyuk_nonproportional_2010 ; choong_design_2008 ; choong_performance_2008 ;
laplace_simultaneous_2021  as well as theoretical interpretations
moses_origins_2012 ; payne_nonproportionality_2009 ;
payne_nonproportionality_2011 ; payne_nonproportionality_2014 ;
payne_nonproportionality_2015 ; beck_nonproportionality_2015 ;
shi_precise_2002 ; dietrich_kinetics_1972 ; rooney_scintillator_1997 ;
bizarri_role_2009 ; vasilev_luminescence_2008 ; murray_scintillation_1961 ;
li_transport-based_2011 ; li_role_2011 ; kerisit_computer_2009 ; gao_electron-
hole_2008 ; kerisit_kinetic_2008 ; bizarri_analytical_2009 , with a subset
focusing on simulations.

In recent years, the dark matter (DM) community has shown increasing interest
in the nPR of NaI(Tl), particularly in the low-energy region. It is driven by
experiments searching for DM using NaI(Tl) detectors, inspired by the
observations made by DAMA/LIBRA bernabei_dark_2004 ; bernabei_final_2013 ;
b_bernabei_first_2018 . DAMA/LIBRA claimed to have detected an annual
modulation signal of DM below 6 keV, which has motivated other experiments to
analyze low-energy scintillation events in an attempt to replicate their
results adhikari_lowering_2021 ; coarasa_improving_2022 . Moreover, the search
for low-mass DM signal in the low-energy region has intensified, with
consideration of interactions such as weakly interacting massive particle
(WIMP) scattering on the nucleus lee_cosmological_1977 ;
goodman_detectability_1985 ; the_cosine-100_collaboration_experiment_2018 ;
adhikari_strong_2021 ; adhikari_three-year_2022 ; amare_annual_2021 , the
Migdal effect migdal_ionization_1941 ; ibe_migdal_2018 ;
cosine-100_collaboration_searching_2022 , and DM-electron scattering
cosine-100_collaboration_search_2023 ;
the_darkside_collaboration_constraints_2018 ; lee_modulation_2015 ;
griffin_extended_2021 ; pandax-ii_collaboration_search_2021 ;
essig_direct_2012 ; graham_semiconductor_2012 . Another phenomenon that relies
on the analysis of low-energy scintillation is coherent elastic neutrino-
nucleus scattering (CE$\nu$NS). Ongoing efforts aim to observe CE$\nu$NS of
reactor neutrinos using NaI(Tl) detectors choi_exploring_2023 . Additionally,
a feasibility test of future NaI(Tl) detectors for CE$\nu$NS observations of
solar and supernova neutrinos has been conducted ko_sensitivities_2023 . These
endeavors crucially require understanding extremely low-energy events.

While substantial efforts have been dedicated to lowering the energy threshold
through hardware and software upgrades choi_improving_2020 ;
lee_scintillation_2022 ; adhikari_lowering_2021 , there has been a relative
neglect of the study on nPR of scintillation response. However, misconceptions
regarding scintillation response could lead to incorrect interpretations of DM
and $\nu$ signals. This is particularly relevant in the low-energy region,
where light yield rapidly changes and can introduce significant biases.
Unfortunately, previously measured nPR curves have not adequately covered the
low-energy region, resulting in a scarcity of reported light-yield
measurements at these energies, with only a few indirect or obsolete values
available khodyuk_nonproportional_2010 ; aitken_fluorescent_1967 .

To address the aforementioned challenges, this paper presents the results of
our $\gamma$-ray calibration of NaI(Tl) crystals used in the COSINE-100
experiment. We utilized various mono-energetic peaks from internally
contaminated isotopes, covering an energy range from 0.87 to 88 keV. Given the
long-term nature of the DM search experiment, we extracted these energy peaks
by simultaneously examining their decay times and peak positions. To
complement the data, we also conducted $\gamma$ spectroscopy using another
NaI(Tl) crystal from the same manufacturer.

In Sec. 2, we describe the experimental setup, encompassing the COSINE-100
experiment, as well as the additional $\gamma$ spectroscopy. COSINE-100 data,
which forms the basis of this study, is summarized in Sec. 3, highlighting the
observations and fitting procedures. Sec. 4 focuses on the $\gamma$
spectroscopy methodology, outlining the measurements and analysis techniques
to supplement COSINE-100 data. In Sec. 5, we consolidate the results and
present the measured nPR. Additionally, we provide the resolution and validate
it using a waveform simulation. Finally, in Sec. 6, we conclude by summarizing
the key findings of our research and emphasizing the implication for future
research.

##  2 Experimental Setup

The COSINE-100 experiment employed eight high-purity NaI(Tl) crystals,
collectively constituting a target for direct DM investigations. These
crystals had a total mass of 106 kg and were equipped with one 3-inch
photomultiplier tube (PMT) at each of the two ends, enabling the detection of
scintillation light. Each crystal was housed within a copper encapsulation.
The setup featured eight such encapsulations submerged in a 2,200 L volume of
LAB-based liquid scintillator (LS), which served as an active veto system
adhikari_cosine-100_2021 . An acrylic box contained the LS, with eighteen PMTs
attached to its walls for scintillation light detection, respectively. Events
were categorized as multiple-hit or single-hit events based on whether
particles left signals in other detectors (LS or crystal). The acrylic box is
surrounded by copper and lead, which provide passive shielding layers, while
an outermost layer consists of plastic scintillator panels used as a muon veto
prihtiadi_muon_2018 .

Signals collected by the crystal PMTs were recorded, as waveforms, through a
data acquisition (DAQ) system. The DAQ system comprised pre-amplifiers,
analog-to-digital converters (ADC), and a trigger control board (TCB), with
detailed specifications outlined in Sec. 7 of Ref. adhikari_initial_2018 . The
charge deposition of an event, serving as an indicator of energy, is
determined through the integration of the waveform over a 5 $\mu$s time window
following noise removal. Data taking for physics purposes commenced on October
20, 2016, at the Yangyang Underground Laboratory adhikari_initial_2018 , and
this study utilized data collected through June 9, 2022. Notably, three
crystals were excluded from the analysis due to their low light yields or high
noise levels the_cosine-100_collaboration_experiment_2018 ; adhikari_three-
year_2022 .

Despite the high purity of these NaI(Tl) crystals, radioactive decay signals
were observed inside them adhikari_background_2021 . Specifically, in the
region of interest of this study, three long-lived isotopes—$\rm{{}^{210}Pb}$,
$\rm{{}^{22}Na}$, and $\rm{{}^{40}K}$—were clearly identified as contamination
inside the crystals. Moreover, several short-lived isotopes111Precisely to
say, short-lived isotopes without long-lived supporting parent isotopes. were
generated by cosmic rays prior to installation, including $\rm{{}^{109}Cd}$,
$\rm{{}^{113}Sn}$, $\rm{{}^{125}I}$, $\rm{{}^{121m}Te}$, and
$\rm{{}^{127m}Te}$ barbosa_de_souza_study_2020 . Signal rates from these
isotopes were characterized by the isotopes’ decay times and effectively
modeled using Monte Carlo simulations. This paper reanalyzes the energy peaks
associated with these isotopes, which serve as calibration points.

Apart from the crystals used in COSINE-100, a smaller-sized NaI(Tl) crystal
from the same manufacturer was utilized. The crystal had a cylindrical shape
with a diameter of 3 inches and a height of 7 inches. It was enveloped in 2–3
mm of PTFE, enclosed by 2 mm of aluminum, and fitted with a quartz window. It
featured a 3-inch PMT identical to those used in COSINE-100. The crystal was
further shielded using 5-cm-thick lead bricks. The DAQ system for this crystal
also included a pre-amplifier, an ADC module, and a TCB, all with the same
specifications as those in COSINE-100. Notably, a more stringent trigger
condition was applied to reduce the high trigger rate arising from noise and
background events at the low-energy region. This setup facilitated
measurements of energy responses to external $\gamma$ sources, such as
$\rm{{}^{241}Am}$, $\rm{{}^{133}Ba}$, $\rm{{}^{109}Cd}$, and
$\rm{{}^{137}Cs}$. Importantly, the peaks used for analysis remained
independent of the trigger condition.

##  3 Extracting Internal Peaks

###  3.1 Long-Lived Isotopes

The charge distribution of single-hit events in COSINE-100 revealed a distinct
peak from $\rm{{}^{210}Pb}$. This peak was modeled with the function
$f_{\mathrm{Pb}}(q)$, defined as

$\begin{split}f_{\mathrm{Pb}}(q)=&\mathrm{Q}\left(q;~{}A,~{}q_{\mathrm{min}},~{}q_{0}\right)\ast\mathrm{N}\left(q;~{}\mu=0,~{}\sigma^{2}\right)\\\
&+\mathrm{AG}\left(q;~{}B,~{}\mu,~{}\sigma^{2}_{\mathrm{l}},~{}\sigma^{2}_{\mathrm{r}}\right)+Cq+D,\end{split}$
(1)

where $q$ is the integrated charge from the signal waveform, while all the
other parameters are free fitting parameters. In this equation, the first part
($\mathrm{Q}\ast\mathrm{N}$) represents the 49 keV peak from
$\rm{{}^{210}Pb}$, where $\mathrm{Q}$ is the quadratic distribution, defined
as

$\begin{split}\mathrm{Q}&\left(q;~{}A,~{}q_{\mathrm{min}},~{}q_{0}\right)\\\
&~{}~{}\equiv\begin{cases}A\left(q-q_{0}\right)^{2},&\text{for
}q_{\mathrm{min}}\leq q\leq q_{0}\\\
0,&\text{otherwise}\end{cases},\end{split}$ (2)

where $A$ is the normalization factor, $q_{0}$ is the vertex position, and
$q_{\mathrm{min}}$ is the minimum range of the distribution. This distribution
models the sum of $\gamma$ and $\beta$ rays emitted from the $\rm{{}^{210}Pb}$
decay. In this context, $q_{\mathrm{min}}$ represents the charge deposition
associated with the energy of the $\gamma$ ray, while $q_{0}$ corresponds to
the charge deposition at the maximum total energy. The symbol $\ast$ denotes
the convolution operator, and $\mathrm{N}$ is the normal distribution centered
at 0 with its standard deviation given by the parameter $\sigma$. The
convolution of this normal distribution mimics the resolution-smearing effect
in the charge distribution.

Figure 1:  The charge distribution around the 49 keV peak of
$\rm{{}^{210}Pb}$, as measured by a NaI(Tl) crystal in COSINE-100 (depicted by
the black crosses). The solid red curve represents the fitted model described
in Eq. 1. The $\rm{{}^{210}Pb}$ peak is drawn as the dashed yellow curve. The
vertical yellow line indicates the position of the $\rm{{}^{210}Pb}$ peak,
along with its associated error. The dotted blue curve represents the summed
spectrum of background components.

For the background distribution around this peak, we adopted a sum of a linear
component ($Cq+D$) and an asymmetric Gaussian distribution $\mathrm{AG}$
defined as

$\begin{split}\mathrm{AG}&\left(q;~{}B,~{}\mu,~{}\sigma^{2}_{\mathrm{l}},~{}\sigma^{2}_{\mathrm{r}}\right)\\\
&~{}\equiv\begin{cases}B\exp{\left[-\left(q-\mu\right)^{2}\,/\,2\sigma^{2}_{l}\right]},&\text{for
}q<\mu\\\ \\\
B\exp{\left[-\left(q-\mu\right)^{2}\,/\,2\sigma^{2}_{r}\right]},&\text{for
}q\geq\mu\\\ \end{cases},\end{split}$ (3)

where $B$ is the normalization constant, $\mu$ is the peak position, and
$\sigma_{\mathrm{l}}$ and $\sigma_{\mathrm{r}}$ are the standard deviations
impacting the left and right tails. These distributions could accurately model
the background components. A Monte Carlo simulation was used to verify the
fitting procedure, yielding a bias of less than 0.1%. The data distribution
and the fitted model are depicted in Fig. 1.

The position of the $\rm{{}^{210}Pb}$ peak was determined as the position
where the peak component ($\mathrm{Q}\ast\mathrm{N}$) is maximized. To
interpret the position as the light amount for the energy, we assign a 1%
systematic uncertainty to account for the unknown spatial distribution of
$\rm{{}^{210}Pb}$ within the crystal and for the poorly known spatial
dependence of the detector response. The total uncertainty is visually
depicted as a yellow band around the vertical line representing the peak
position in Fig. 1.

Figure 2:  The charge distribution of multiple hit events at the low-energy
region, as recorded by a NaI(Tl) crystal in COSINE-100 (indicated by the black
crosses). The solid red curve corresponds to the fitted model. Additionally,
the dashed yellow curve represents the scaled Poisson distribution for the
0.87 keV from $\rm{{}^{22}Na}$, while the dashed pink curve illustrates the
normal distribution for the 3.2 keV from $\rm{{}^{40}K}$. The vertical lines
and bands depict their peak positions and associated errors. The dotted blue
curve represents the linear-shape model for the background components.

In the charge distribution of multiple-hit events, we observed two distinct
peaks associated with isotopes from internal contamination, specifically
$\rm{{}^{22}Na}$ and $\rm{{}^{40}K}$. Approximately 10% of $\rm{{}^{22}Na}$
decays to the 1275 keV level of $\rm{{}^{22}Ne}$ through electron capture
(EC), while a similar proportion of $\rm{{}^{40}K}$ decays via EC to the 1461
keV level of $\rm{{}^{40}Ar}$ BASUNIA201569 ; CHEN20171 . The daughter atoms
undergo cascade processes, leaving characteristic peaks at their respective
K-shell edges within the crystal where $\rm{{}^{22}Na}$ and $\rm{{}^{40}K}$
originally resided. Concurrently, high-energy $\gamma$ rays are emitted due to
the deexcitation of these daughter nuclei. These energetic $\gamma$ rays can
escape the crystal, potentially triggering the neighboring crystals or the LS
veto system, leading to coincidences. Consequently, we clearly identified two
low-energy peaks in the charge distribution of multiple-hit events,
corresponding to 0.87 keV and 3.2 keV. These values represent the K-shell
binding energies of $\rm{{}^{22}Ne}$ and $\rm{{}^{40}Ar}$ NISTXray2005 . The
charge distribution is shown in Fig. 2, where the observed peak positions and
shapes align with our expectations.

For the 0.87 keV peak, a scaled Poisson distribution was used for modeling.
The distribution is represented as

$f_{\mathrm{Na}}\left(q;~{}A,~{}\lambda,~{}s\right)\equiv
A\,\frac{\lambda^{q/s}\,e^{-\lambda}}{\Gamma\left(q/s\,+1\right)},$ (4)

where $A$ is the normalization constant, $\lambda$ is the expected number of
photo-electrons ($N_{\mathrm{pe}}$), and $s$ denotes the scaling factor
originating from the PMTs. The 3.2 keV peak produced sufficient photo-
electrons such that we modeled it using a normal distribution. The background
components were attributed to Compton scattering of high-energy external
$\gamma$ rays, resulting in relatively smooth shapes. To account for the
uncertainty in the background distribution’s shape, flat or linear-shaped
distributions were adopted as background models, and the difference between
results from these models was considered as a systematic uncertainty. All the
parameters denoting the normalization, position, and width of the peaks, and
the shape of background distribution are free in the fitting process. The
charge distribution, fitted model, and the extracted peak positions are shown
in Fig. 2, and the peak positions are summarized in Table 1.

Table 1:  List of peak positions extracted from COSINE-100 data.  |  | Peak Position (pC)
---|---|---
Isotope | Energy (keV) | Crystal 2 | Crystal 3 | Crystal 4 | Crystal 6 | Crystal 7
$\rm{{}^{22}Na}$ | $0.87$ | $5.29\,\pm 0.16$ | $5.33\,\pm 0.17$ | $4.80\,\pm 0.14$ | $5.00\,\pm 0.09$ | $5.23\,\pm 0.11$
$\rm{{}^{40}K}$ | $3.2$ | $21.53\,\pm 0.36$ | $21.70\,\pm 0.39$ | $20.29\,\pm 0.58$ | $19.10\,\pm 0.64$ | $20.61\,\pm 0.37$
$\rm{{}^{109}Cd}$ | $25$ | $192.5\,\pm 2.6$ | $198.3\,\pm 2.1$ | $183.3\,\pm 1.9$ | $165.8\,\pm 2.4$ | $177.5\,\pm 2.6$
$\rm{{}^{113}Sn}$ | $28$ |  | $223.0\,\pm 3.7$ | $202.3\,\pm 2.1$ | $190.1\,\pm 2.2$ | $201.7\,\pm 2.4$
$\rm{{}^{125}I}$ | $39$ |  | $291.7\,\pm 5.4$ | $274.1\,\pm 2.8$ | $265.6\,\pm 2.8$ | $276.5\,\pm 2.9$
$\rm{{}^{210}Pb}$ | $49$ | $370.7\,\pm 3.7$ | $380.1\,\pm 3.8$ | $349.2\,\pm 3.5$ | $337.0\,\pm 3.4$ | $351.7\,\pm 3.5$
$\rm{{}^{125}I}$ | $67$ |  | $536.1\,\pm 7.0$ | $489.0\,\pm 4.9$ | $467.8\,\pm 4.7$ | $487.7\,\pm 4.9$
$\rm{{}^{109}Cd}$ | $88$ | $674.5\,\pm 8.4$ | $685.9\,\pm 7.0$ | $627.2\,\pm 6.3$ | $605.6\,\pm 6.2$ | $632.3\,\pm 6.5$

###  3.2 Short-Lived Isotopes

Short-lived isotopes, such as $\rm{{}^{109}Cd}$, $\rm{{}^{113}Sn}$,
$\rm{{}^{125}I}$, $\rm{{}^{121m}Te}$, and $\rm{{}^{127m}Te}$, were initially
activated by cosmic rays while the crystals were exposed on the ground and
disappeared quickly due to their fast decay times. Their decay processes emit
$\gamma$ rays and X-rays at various energy levels, making them valuable for
this study. Unfortunately, their energy levels overlap, and their activities
are scarce, posing challenges for extracting their peaks using the method
outlined in Sec. 3.1. To overcome this issue, we leveraged constraints based
on their decay characteristics.

To observe the decays of these isotopes, we accumulated charge distributions
over 475 days from the beginning of data acquisition, dividing it into 19
periods, each spanning 25 days. Fig. 3 shows the distributions for three
periods: $1^{\mathrm{st}}$, $5^{\mathrm{th}}$, and $9^{\mathrm{th}}$. While
decaying and constant components are visually identifiable, determining the
position of each peak directly from the spectral shape is challenging. We
built a model to extract the peaks from these spectra, with consideration of
the decay characteristics of the short-lived isotopes.

Figure 3:  The charge distributions from the initial COSINE-100 data and the
extracted peaks attributed to short-lived isotopes. The data points, indicated
by black crosses, represent measurements from each data collection period and
have been simultaneously fitted by using Eq. 8. The time-normalized late
distribution $L(q)$ is depicted as gray histograms in each data collection
period. Each isotope component is visualized as a stacked histogram in its own
color.

The time-dependent charge distributions for each short-lived isotope $i$ can
be expressed as

$D_{i}\left(q,~{}t;~{}A_{i},~{}\vec{\mu}_{i},~{}\vec{\sigma}_{i}\right)=A_{i}\,e^{-t/\tau_{i}}\sum_{j}f_{ij}\,\mathrm{N}\left(q;~{}\mu_{ij},~{}\sigma^{2}_{ij}\right),$
(5)

where $t$ denotes time, $A_{i}$ and $\tau_{i}$ are the initial activity and
lifetime of isotope $i$, and $f_{ij}$ is the branching fraction for the $j$-th
peak of isotope $i$. Each peak is modeled as a normal distribution
$\mathrm{N}(q)$ with peak position $\mu_{ij}$ and resolution $\sigma_{ij}$.
For the $\rm{{}^{210}Pb}$ component, the normal distribution in Eq. 5 was
replaced with $\mathrm{Q}\ast\mathrm{N}$, as used in Sec. 3.1.

The charge distribution accumulated between times $t_{s}$ and $t_{e}$ can be
modeled as

$\begin{split}S&\left(q,~{}t_{s},~{}t_{e};~{}\left\\{A_{i},~{}\vec{\mu}_{i},~{}\vec{\sigma}_{i}\right\\}\right)\\\
&=\int_{t_{s}}^{t_{e}}\left[\sum_{i}D_{i}\left(q,~{}t;~{}A_{i},~{}\vec{\mu}_{i},~{}\vec{\sigma}_{i}\right)+C\left(q\right)\right]s\left(t\right)dt,\end{split}$
(6)

where $s(t)$ denotes the detector operational state (1 or 0) at time $t$. The
term $C(q)$ represents the time-independent component in the spectrum,
consisting of isotopes with decay times orders of magnitude longer than the
experiment’s duration. Instead of constructing a complex analytic model for
the uninteresting $C(q)$ term, we attempted to eliminate its explicit
dependence. Since the term is common to all time periods, subtracting a charge
distribution from another period with proper time-normalization could
effectively remove the $C(q)$ dependency.

For the subtrahend, a late distribution $L(q)$ was accumulated, expressed as
$L\left(q\right)=S\left(q,~{}t_{0},~{}t_{1}\right)$ for $t_{0}$ and $t_{1}$
defined as 650 days and 1,150 days after the start of data acquisition. Due to
its long time period, $L(q)$ has a sufficiently small error to be treated in
the model as a known distribution rather than fitted data. Replacing the
$C(q)$ term with $L(q)$ from Eq. 6 yields

$\begin{split}S&\left(q,~{}t_{s},~{}t_{e};~{}\left\\{A_{i},~{}\vec{\mu}_{i},~{}\vec{\sigma}_{i}\right\\}\right)\\\
&=\int_{t_{s}}^{t_{e}}\sum_{i}D_{i}\left(q,~{}t\right)\,s\left(t\right)\,dt\\\
&~{}~{}~{}~{}+\frac{\int_{t_{s}}^{t_{e}}s(t)\,dt}{\int_{t_{0}}^{t_{1}}s(t)\,dt}\left[L\left(q\right)-\int_{t_{0}}^{t_{1}}\sum_{i}D_{i}\left(q,~{}t\right)\,s\left(t\right)\,dt\right],\end{split}$
(7)

which depends only on the set of parameters related to the peaks.
Consequently, the number of events for the $k$-th 25-days-period spectrum in
the $l$-th charge bin is expected as $S_{kl}=S\left(q_{l},~{}(k-1)\Delta
T,~{}k\Delta T\right)$, where $q_{l}$ is the central value of the $l$-th
charge bin, and $\Delta T$ is the period interval, 25 days.

To extract peaks from the short-lived isotopes, we conducted a simultaneous
fit on 19 distributions using a binned likelihood,

$\begin{split}\mathcal{L}&\left(\left\\{A_{i},\vec{\mu}_{i},~{}\vec{\sigma}_{i}\right\\};~{}\mathbf{O},~{}\vec{\mu}^{A},~{}\vec{\sigma}^{A}\right)\\\
&=\prod_{k}^{19}\prod_{l}^{N_{\mathrm{bin}}}\frac{S_{kl}^{O_{kl}}e^{-S_{kl}}}{O_{kl}!}\times\prod_{i}\exp\left[-\frac{1}{2}\left(\frac{A_{i}-\mu_{i}^{A}}{\sigma_{i}^{A}}\right)^{2}\right],\end{split}$
(8)

where $O_{kl}$ is the observed number of events in the $k$-th period spectrum
at the $l$-th charge bin. The initial activity $A_{i}$, peak positions
$\mu_{ij}$, and resolutions $\sigma_{ij}$ were treated as free parameters,
while those for $\rm{{}^{210}Pb}$ were kept constant at the values obtained in
Sec. 3.1. Constraints for the initial activities, $\mu^{A}_{i}$ and
$\sigma^{A}_{i}$ were obtained from our previous measurement
barbosa_de_souza_study_2020 . As seen in Fig. 3, the extracted peaks
effectively account for the charge distributions in different time domains
simultaneously. For verification, we applied the same fitting procedure to
simulated distributions, allowing us to obtain the true mean energy value for
each peak. Through this process, light yields were obtained for several points
ranging from 25 to 88 keV, as summarized in Table 1.

##  4 Gamma Spectroscopy

An independent experiment was conducted for $\gamma$ spectroscopy to acquire
complementary data and broaden the energy range of our analysis. Charge
distributions were measured using four $\gamma$ sources, $\rm{{}^{241}Am}$,
$\rm{{}^{133}Ba}$, $\rm{{}^{109}Cd}$, and $\rm{{}^{137}Cs}$. These sources
were placed near the crystal and aligned at its center during data taking. The
energies of the $\gamma$ rays and X-rays emitted from these sources, along
with their exposure times, are listed in Table 2. Data without any sources was
also collected over an extended period to obtain the background distributions
before and after acquiring data with the sources. This allowed us to verify
the stability of the PMT gain and background distribution with time. The pure
peak for each source was identified by subtracting the background
distribution, as illustrated in Fig 4.

Table 2:  List of sources used for $\gamma$ spectroscopy, with measured peak positions.  $\gamma$ Source | Time | Energy | Peak Position
---|---|---|---
| (Hours) | (keV) | (pC)
No Source | 22 |  |
$\rm{{}^{241}Am}$ | 14 | $59.25\,\pm 0.16$ | $453.4\,\pm 4.4$
$\rm{{}^{133}Ba}$ | 6 | $29.17\,\pm 0.08$ | $211.5\,\pm 2.2$
$30.82\,\pm 0.08$ | $224.1\,\pm 2.2$
$35.73\,\pm 0.10$ | $251.4\,\pm 2.6$
$51.65\,\pm 0.14$ | $375.7\,\pm 3.8$
$80.00\,\pm 0.22$ | $588.8\,\pm 5.9$
$\rm{{}^{109}Cd}$ | 8 | $21.98\,\pm 0.06$ | $157.5\,\pm 1.6$
$24.63\,\pm 0.07$ | $177.8\,\pm 1.8$
$87.54\,\pm 0.24$ | $641.3\,\pm 6.8$
$\rm{{}^{137}Cs}$ | 11 | $32.11\,\pm 0.10$ | $234.5\,\pm 2.4$
$37.54\,\pm 0.30$ | $268.9\,\pm 4.2$

The peak positions were determined by fitting the charge distributions. The
charge distribution for isotope $i$ can be modeled as

$S_{i}\left(q\right)=\sum_{j}N_{ij}\,\mathrm{CB}\left(q;~{}\mu_{ij},~{}\sigma_{ij},~{}\alpha_{ij},~{}n_{ij}\right),$
(9)

where $N_{ij}$ represents the normalization constant of the $j$-th peak, and
$\mathrm{CB}(q)$ denotes a Crystal Ball function commonly used to model peaks
measured by scintillation crystals with energy loss processes
gaiser_charmonium_1982 . The function includes parameters for peak position
$\mu_{ij}$ and resolution $\sigma_{ij}$, which are free during the fitting
process. The $\alpha_{ij}$ and $n_{ij}$ parameters account for the peak shape.
Flat and linear background components were added to the $\rm{{}^{133}Ba}$ and
$\rm{{}^{137}Cs}$ distributions, respectively, to model Compton scattering of
high-energy $\gamma$ rays.

Figure 4:  Charge distributions of external sources after subtraction of the
background distribution, recorded by the sample crystal for $\gamma$
spectroscopy. Black crosses represent the data, and red curves depict the
fitting results used to extract the peak positions. The peak positions
extracted are shown as vertical lines with error bands.

The model was fitted to data by minimizing the following chi-squared:

$\chi^{2}_{i}\equiv\sum_{l}^{N_{\mathrm{bin}}}\dfrac{\left[D_{il}-\left(t_{i}/t_{b}\right)B_{l}-S_{i}\left(q_{l}\right)\right]^{2}}{D_{il}+\left(t_{i}/t_{b}\right)^{2}B_{l}}.$
(10)

In this equation, $D_{i}l$ and $B_{l}$ are the measured number of events in
$i$-th source and background data, respectively, at the $l$-th charge bin,
centered at $q_{l}$. Exposure times for the $i$-th source and background data
are represented by $t_{i}$ and $t_{b}$, respectively. The extracted peak
positions are displayed in Fig. 4 as vertical lines with their uncertainties.

The same fitting procedure was conducted as in Sec. 3, with distributions
obtained by GEANT4 Monte Carlo simulation agostinelli_geant4simulation_2003 .
All dimensions of the detector geometry were implemented based on
measurements, except for the thickness of the PTFE wrapped inside the aluminum
case. To account for the uncertainty, two different conditions were simulated
with this thicknesses set to 2 mm and 3 mm. The simulation results were fitted
separately, and the difference in peak positions was assigned to the
systematic error of the true energy. One can see those errors and the
extracted peak positions in Table 2.

##  5 Scintillation Response of NaI(Tl) Crystal

###  5.1 Photon Response Measurements

The relative light yield obtained via peak extraction is depicted in Fig. 5.
Both the response function and the data points are normalized to the value at
50 keV to highlight the nonproportional behavior of the light yield. The error
bars include systematic errors resulting from the radioactive source’s
position and the peak extraction method. These measurements exhibited
consistency across the crystals and the measuring techniques within the
associated uncertainties. The well-known K-shell dip of NaI(Tl) at 33 keV is
also identifiable.

Figure 5:  The relative light yield as a function of incident $\gamma$-ray
energy, normalized to the value at 50 keV. The measurements are represented by
colored dots with error bars, and the fitted function is drawn as the red
curve. The translucent red band represents the uncertainty from the modeling
at 67% confidence level.

The data was fitted using an empirical function developed based on the Payne’s
model payne_nonproportionality_2009 ; payne_nonproportionality_2011 ;
payne_nonproportionality_2014 ; payne_nonproportionality_2015 ;
beck_nonproportionality_2015 , as shown by the red curve in Fig. 5. An
expectation arises for the presence of the L-shell dip structure around 4.5
keV, although there is an insufficient number of data points below 20 keV to
constrain the detailed shape of the dip. We have checked the consistency of
our curve with measurements from two references comparing quantities, such as
the light-yield ratio between 4.5 and 20 keV, the ratio between 0.87 and 20
keV, and the slope around 10 keV. These were found to be consistent with
previous reports khodyuk_nonproportional_2010 ; aitken_fluorescent_1967 , as
summarized in Table 3. Despite this consistency check, we acknowledge that our
curve between 4.5 and 20 keV is less convincing, resulting in a relatively
wider error band, with its width dependent on the selected model
parameterization. The measurement of light yield in this energy region is left
for future investigation.

Table 3:  Comparison of $\gamma$ nPR measurements khodyuk_nonproportional_2010 ; aitken_fluorescent_1967 . $\mathrm{LY}_{\gamma}\left(E\right)$ is the light yield for a $\gamma$ ray with incident energy $E$, and $\mathrm{LY}_{\gamma,~{}50}\left(E\right)\equiv\mathrm{LY}_{\gamma}\left(E\right)/\mathrm{LY}_{\gamma}\left(50\,\mathrm{keV}\right)$.  | This Work | Khodyuk | Aitken
---|---|---|---
$\dfrac{d\,\mathrm{LY}_{\gamma,~{}50}}{dE}\left(10\,\mathrm{keV}\right)$ $\left(\dfrac{\%}{\mathrm{keV}}\right)$ | 1.12 | 1.18 | 1.16
$\dfrac{\mathrm{LY}_{\gamma}\left(4.5\,\mathrm{keV}\right)}{\mathrm{LY}_{\gamma}\left(20\,\mathrm{keV}\right)}$ (%) | 93.0 | 93.3 | 91.8
$\dfrac{\mathrm{LY}_{\gamma}\left(0.89\,\mathrm{keV}\right)}{\mathrm{LY}_{\gamma}\left(20\,\mathrm{keV}\right)}$ (%) | 83.7 | 89.3 |

To evaluate the nPR in the high-energy region above 200 keV, we examined its
consistency with the data reported in Ref. devare_effect_1963 . We measured
the light yields by fitting the prominent $\gamma$ peaks from the high-energy
spectrum, including 295 keV from $\rm{{}^{214}Pb}$, 511 and 1274 keV from
$\rm{{}^{22}Na}$, 609 and 1764 keV from $\rm{{}^{214}Bi}$, 1461 keV from
$\rm{{}^{40}K}$, and 2615 keV from $\rm{{}^{208}Tl}$. The measured light
yields demonstrate agreement with the nPR curve reported in Ref.
devare_effect_1963 , as illustrated in Fig. 6.

Figure 6:  The relative light yield (normalized at 662 keV) for the high-
energy $\gamma$ peaks is represented by black dots with error bars. The red
dots and curve represent the nPR measured from Ref. devare_effect_1963 , where
our data agrees well with the reference.

###  5.2 Resolution and Waveform Simulation

Once a peak is extracted, the energy resolution $\sigma_{ij}$ is naturally
obtained along with the peak position $\mu_{ij}$. As shown in the dashed gray
curve in Fig. 7, the characteristic $1/\sqrt{E}$-like shape, originating from
the Poisson fluctuations in $N_{\mathrm{pe}}$ for the PMT, is clearly visible.
Also resolution effects due to the detector geometry and the distribution of
the number of amplified electrons in the PMT are observed.

Figure 7:  The relative resolution as a function of incident $\gamma$-ray
energy for a NaI(Tl) crystal. The black dots with error bars represent the
measurements, and the red curve represents the model of the resolution curve,
as described in Eq. 11, with an error band at 67% confidence level. The solid
pink curve illustrates the $\sqrt{a/q}$ component from the resolution model,
which closely matches the waveform simulation (solid blue curve). The dashed
gray curve represents the Poisson fluctuation in $N_{\mathrm{pe}}$.

To account for these effects, the relative resolution as a function of charge
was modeled using the following equation,

$\frac{\sigma(q)}{q}=\sqrt{\frac{a}{q}+b}.$ (11)

Here, $a$ and $b$ are fitting parameters, where $a$ scales the Poisson
resolution contribution and $b$ typically arises from non-uniformity of
detector response, which is not included in the simulation. In Fig. 7, the red
line represents the model fitted to the measurements, with the uncertainty
shown as a band. The resolution $\sigma(q)$ is transformed to $\sigma(E)$ via
the nPR curve shown in Fig. 5, leading to the emergence of non-analytic points
at the 4 keV L-shell edge and the 33 keV K-shell edge. The pink line
represents the function when $b=0$.

Given the focus on DM detection experiments, where the region of interest
typically lies below 6 keV, it is crucial to investigate the resolution in the
sub-keV range, particularly for future upgrades. Since the effect of electron
amplification in PMTs and DAQ system is most dominant at low energies, except
for that due to the Poisson fluctuations with $N_{\mathrm{pe}}$, a waveform
simulation developed for the COSINE-100 experiment was utilized to validate
the resolution in this region. This simulation incorporates factors such as
rise and decay times of NaI(Tl) scintillation, fluctuations in the single
photo-electron shape, DAQ electronics, and charge accumulation processes.
Therefore, by inputting $N_{\mathrm{pe}}$ that takes into account Poisson
fluctuations, we can estimate the resolution with these effects included.

The blue line in Fig. 7 shows that the simulation successfully replicates the
data. Despite not accounting for the intricate details of detector geometry
and other mechanisms, the simulation exhibits good agreement with the
measurements at the low-energy region. In addition, the agreement between the
simulation and the pink curve, which is the measurement with the constant-term
factor excluded, confirms the possibility of extrapolating the resolution
curve to the low-energy direction. These findings hold particular significance
for scintillator experiments focusing on low-energy ranges.

##  6 Summary and Conclusion

In conclusion, this study has thoroughly examined the nPR characteristics of
NaI(Tl) scintillators within the context of DM search experiments. Through a
comprehensive calibration process using COSINE-100 data, complemented by
$\gamma$ spectroscopy, we have gained valuable insights into the behavior of
NaI(Tl) crystals.

The measurement of light yields at various energies was successfully achieved
by analyzing internal peaks originating from both long-lived and short-lived
isotopes. Extraction of peaks from long-lived isotopes was easily accomplished
due to stable experimental data acquisition. A significant milestone was
reached by directly measuring the light yield at 0.87 keV, generated by the
cascade following $\rm{{}^{22}Na}$ EC decay, facilitated by the active veto
system. This direct measurement serves as validation of an indirect
measurement obtained by analyzing X-rays near the Iodine K-shell using
synchrotron radiation khodyuk_nonproportional_2010 . Overcoming the challenge
of extracting multiple overlaid peaks from short-lived isotopes was made
possible through long-term data collection and time-dependent analysis.

In the nPR curve measured here, the distinctive K-shell dip structure was
observed. The nPR curve was fitted using an empirical curve. The resulting
fitted nPR curve exhibits consistency with previously reported measurements,
particularly in the energy range below 20 keV. However, the lack of data
points around the 10 keV region limits the precision of the calibration curve,
leaving the error band relatively wider. This aspect remains a focus for
future investigation.

The resolution analysis of NaI(Tl) scintillators revealed a characteristic
resolution curve that follows the $1/\sqrt{N_{\mathrm{pe}}}$ trend, as
expected for Poisson fluctuations in $N_{\mathrm{pe}}$. Waveform simulation
provided compelling confirmation of the resolution at low energies, allowing
for the extension of the resolution curve to lower energy ranges. This
enhancement in our ability to detect low-energy DM recoil signals is very
important.

The findings of this study hold significant implications for background
modeling in NaI(Tl) experiments. Precise calibration and characterization of
the nPR effect are essential for accurately modeling the background spectra
resulting from radioactive interactions involving electrons and photons. To
facilitate more detailed simulations, the integration of the nonproportional
light production of NaI(Tl) into the GEANT4 toolkit is under development,
building on similar efforts reported previously rasco_nonlinear_2015 ; cano-
ott_monte_1999 .

Furthermore, it is imperative to consider nPR calibration when interpreting
signals that may come from DM interactions with electrons or photons. As the
scientific community explores diverse scenarios involving boosted DM and light
DM, the significance of these interactions is poised to grow. This study lays
the foundation for more robust and accurate DM search experiments in the
future.

###### Acknowledgements.

We thank the Korea Hydro and Nuclear Power (KHNP) Company for providing
underground laboratory space at Yangyang and the IBS Research Solution Center
(RSC) for providing high-performance computing resources. This work is
supported by the Institute for Basic Science (IBS) under project code
IBS-R016-A1, NRF-2019R1C1C1005073, NRF-2021R1A2C3010989, NRF-2021R1A2C1013761
and Hyundai Motor Chung Mong-Koo Foundation, Republic of Korea; NSF Grants No.
PHY-1913742, DGE-1122492, WIPAC, the Wisconsin Alumni Research Foundation,
United States; STFC Grant ST/N000277/1 and ST/K001337/1, United Kingdom; Grant
No. 2021/06743-1, 2022/12002-7 and 2022/13293-5 FAPESP, CAPES Finance Code
001, CNPq 303122/2020-0, Brazil.

## References

  * (1) R. W. Pringle and S. Standil, The Gamma-Rays from Neutron-Activated Gold, Phys. Rev. 80, 762–763 (1950).
  * (2) D. Engelkemeir, Nonlinear Response of NaI(Tl) to Photons, Rev. Sci. Instrum. 27, 589–591 (1956).
  * (3) D. W. Aitken et al., The Fluorescent Response of NaI(Tl), CsI(Tl), CsI(Na) and CaF2(Eu) to X-Rays and Low Energy Gamma Rays, IEEE Trans. Nucl. Sci. 14, 468–477 (1967).
  * (4) H. Leutz and C. D’Ambrosio, On the scintillation response of NaI(TI)-crystals, IEEE Trans. Nucl. Sci. 44, 190–193 (1997).
  * (5) L. R. Wayne et al., Response of NaI(Tl) to X-rays and electrons, Nucl. Instrum. Methods Phys. Res. Sect. A 411, 351–364 (1998).
  * (6) L. Swiderski et al., Response of doped alkali iodides measured with gamma-ray absorption and Compton electrons, Nucl. Instrum. Methods Phys. Res. Sect. A 705, 42–46 (2013).
  * (7) H. G. Devare and P. N. Tandon, Effect of the non-linear response of NaI(Tl) on the single crystal summing spectra, Nucl. Instrum. Methods 22, 253–255 (1963).
  * (8) A. J. L. Collinson and R. Hill, The Fluorescent Response of NaI(Tl) and CsI(Tl) to X Rays and $\gamma$ Rays, Proc. Phys. Soc. 81, 883 (1963).
  * (9) T. H. Jones, The nonproportional response of a NaI(Tl) crystal to diffracted X rays, Nucl. Instrum. Methods 15, 55–58 (1962).
  * (10) I. V. Khodyuk, P. A. Rodnyi, and P. Dorenbos, Nonproportional scintillation response of NaI:Tl to low energy x-ray photons and electrons, J. Appl. Phys. 107, 113513 (2010).
  * (11) W.-S. Choong et al., Design of a Facility for Measuring Scintillator Non-Proportionality, IEEE Trans. Nucl. Sci. 55, 1753–1758 (2008).
  * (12) W.-S. Choong et al., Performance of a Facility for Measuring Scintillator Non-Proportionality, IEEE Trans. Nucl. Sci. 55, 1073–1078 (2008).
  * (13) T. A. Laplace et al., Simultaneous measurement of organic scintillator response to carbon and proton recoils, Phys. Rev. C 104, 014609 (2021).
  * (14) W. W. Moses et al., The Origins of Scintillator Non-Proportionality, IEEE Trans. Nucl. Sci. 59, 2038–2044 (2012).
  * (15) S. A. Payne et al., Nonproportionality of Scintillator Detectors: Theory and Experiment, IEEE Trans. Nucl. Sci. 56, 2506–2512 (2009).
  * (16) S. A. Payne et al., Nonproportionality of Scintillator Detectors: Theory and Experiment. II, IEEE Trans. Nucl. Sci. 58, 3392–3402 (2011).
  * (17) S. A. Payne et al., Nonproportionality of Scintillator Detectors. III. Temperature Dependence Studies, IEEE Trans. Nucl. Sci. 61, 2771–2777 (2014).
  * (18) S. A. Payne, Nonproportionality of Scintillator Detectors. IV. Resolution Contribution from Delta-Rays, IEEE Trans. Nucl. Sci. 62, 372–380 (2015).
  * (19) P. R. Beck et al., Nonproportionality of Scintillator Detectors. V. Comparing the Gamma and Electron Response, IEEE Trans. Nucl. Sci. 62, 1429–1436 (2015).
  * (20) H.-X. Shi et al., Precise Monte Carlo simulation of gamma-ray response functions for an NaI(Tl) detector, Appl. Radiat. Isot. 57, 517–524 (2002).
  * (21) H. B. Dietrich and R. B. Murray, Kinetics of the diffusion of self-trapped holes in alkali halide scintillators, J. Luminescence 5, 155–170 (1972).
  * (22) B. Rooney and J. Valentine, Scintillator light yield nonproportionality: calculating photon response using measured electron response, IEEE Trans. Nucl. Sci. 44, 509–516 (1997).
  * (23) G. Bizarri et al., The role of different linear and non-linear channels of relaxation in scintillator non-proportionality, J. Luminescence 129, 1790–1793 (2009).
  * (24) A. N. Vasil’ev, From Luminescence Non-Linearity to Scintillation Non-Proportionality, IEEE Trans. Nucl. Sci. 55, 1054–1061 (2008).
  * (25) R. B. Murray and A. Meyer, Scintillation Response of Activated Inorganic Crystals to Various Charged Particles, Phys. Rev. 122, 815–826 (1961).
  * (26) Q. Li et al., A transport-based model of material trends in nonproportionality of scintillators, J. Appl. Phys. 109, 123716 (2011).
  * (27) Q. Li et al., The role of hole mobility in scintillator proportionality, Nucl. Instrum. Methods Phys. Res. Sect. A 652, 288–291 (2011).
  * (28) S. Kerisit et al., Computer simulation of the light yield nonlinearity of inorganic scintillators, J. Appl. Phys. 105, 114915 (2009).
  * (29) F. Gao et al., Electron-Hole Pairs Created by Photons and Intrinsic Properties in Detector Materials, IEEE Trans. Nucl. Sci. 55, 1079–1085 (2008).
  * (30) S. Kerisit, K. M. Rosso, and B. D. Cannon, Kinetic Monte Carlo Model of Scintillation Mechanisms in CsI and CsI(Tl), IEEE Trans. Nucl. Sci. 55, 1251–1258 (2008).
  * (31) G. Bizarri et al., An analytical model of nonproportional scintillator light yield in terms of recombination rates, J. Appl. Phys. 105, 044507 (2009).
  * (32) R. Bernabei et al., (DAMA/NaI), Dark Matter Particles in the Galactic Halo: Results and Implications from dama/nai, Int. J. Mod. Phys. D 13, 2127–2159 (2004).
  * (33) R. Bernabei et al., (DAMA/LIBRA), Final model independent result of DAMA/LIBRA–phase1, Eur. Phys. J. C 73, 2648 (2013).
  * (34) B. Bernabei et al., (DAMA/LIBRA), First model independent results from DAMA/LIBRA-phase2, Nucl. Phys. At. Energy 19, 307–325 (2018).
  * (35) G. Adhikari et al., (COSINE-100), Lowering the energy threshold in COSINE-100 dark matter searches, Astropart. Phys. 130, 102581 (2021).
  * (36) I. Coarasa et al., (ANAIS-112), Improving ANAIS-112 sensitivity to DAMA/LIBRA signal with machine learning techniques, J. Cosmol. Astropart. Phys. 2022, 048 (2022).
  * (37) B. W. Lee and S. Weinberg, Cosmological Lower Bound on Heavy-Neutrino Masses, Phys. Rev. Lett. 39, 165–168 (1977).
  * (38) M. W. Goodman and E. Witten, Detectability of certain dark-matter candidates, Phys. Rev. D 31, 3059–3063 (1985).
  * (39) G. Adhikari et al., (COSINE-100), An experiment to search for dark-matter interactions using sodium iodide detectors, Nature 564, 83–86 (2018).
  * (40) G. Adhikari et al., (COSINE-100), Strong constraints from COSINE-100 on the DAMA dark matter results using the same sodium iodide target, Sci. Adv. 7, eabk2699 (2021).
  * (41) G. Adhikari et al., (COSINE-100), Three-year annual modulation search with COSINE-100, Phys. Rev. D 106, 052005 (2022).
  * (42) J. Amaré et al., (ANAIS-112), Annual modulation results from three-year exposure of ANAIS-112, Phys. Rev. D 103, 102005 (2021).
  * (43) A. Migdal, Ionization of atoms accompanying $\alpha$-and $\beta$-decay, J. Phys. USSR 4, 449 (1941).
  * (44) M. Ibe et al., Migdal effect in dark matter direct detection experiments, J. High Energ. Phys. 2018, 194 (2018).
  * (45) G. Adhikari et al., (COSINE-100), Searching for low-mass dark matter via the Migdal effect in COSINE-100, Phys. Rev. D 105, 042006 (2022).
  * (46) G. Adhikari et al., (COSINE-100), Search for solar bosonic dark matter annual modulation with COSINE-100, Phys. Rev. D 107, 122004 (2023).
  * (47) P. Agnes et al., (DarkSide), Constraints on Sub-GeV Dark-Matter–Electron Scattering from the DarkSide-50 Experiment, Phys. Rev. Lett. 121, 111303 (2018).
  * (48) S. K. Lee et al., Modulation effects in dark matter-electron scattering experiments, Phys. Rev. D 92, 083517 (2015).
  * (49) S. M. Griffin et al., Extended calculation of dark matter-electron scattering in crystal targets, Phys. Rev. D 104, 095015 (2021).
  * (50) C. Cheng et al., (PandaX-II), Search for Light Dark Matter–Electron Scattering in the PandaX-II Experiment, Phys. Rev. Lett. 126, 211803 (2021).
  * (51) R. Essig, J. Mardon, and T. Volansky, Direct detection of sub-GeV dark matter, Phys. Rev. D 85, 076007 (2012).
  * (52) P. W. Graham et al., Semiconductor probes of light dark matter, Phys. Dark Universe 1, 32–49 (2012).
  * (53) J. J. Choi et al., (NEON), Exploring coherent elastic neutrino-nucleus scattering using reactor electron antineutrinos in the NEON experiment, Eur. Phys. J. C 83, 226 (2023).
  * (54) Y. J. Ko and H. S. Lee, Sensitivities for coherent elastic scattering of solar and supernova neutrinos with future NaI(Tl) dark matter search detectors of COSINE-200/1T, Astropart. Phys. 153, 102890 (2023).
  * (55) J. J. Choi et al., (NEON), Improving the light collection using a new NaI(Tl) crystal encapsulation, Nucl. Instrum. Methods Phys. Res. Sect. A 981, 164556 (2020).
  * (56) H. Y. Lee et al., Scintillation characteristics of a NaI(Tl) crystal at low-temperature with silicon photomultiplier, J. Inst. 17, P02027 (2022).
  * (57) G. Adhikari et al., (COSINE-100), The COSINE-100 liquid scintillator veto system, Nucl. Instrum. Methods Phys. Res. Sect. A 1006, 165431 (2021).
  * (58) H. Prihtiadi et al., (COSINE-100), Muon detector for the COSINE-100 experiment, J. Inst. 13, T02007 (2018).
  * (59) G. Adhikari et al., (COSINE-100), Initial performance of the COSINE-100 experiment, Eur. Phys. J. C 78, 107 (2018).
  * (60) G. Adhikari et al., (COSINE-100), Background modeling for dark matter search with 1.7 years of COSINE-100 data, Eur. Phys. J. C 81, 837 (2021).
  * (61) E. Barbosa De Souza et al., (COSINE-100), Study of cosmogenic radionuclides in the COSINE-100 NaI(Tl) detectors, Astropart. Phys. 115, 102390 (2020).
  * (62) M. S. Basunia, Nuclear data sheets for a = 22, Nuclear Data Sheets 127, 69–190 (2015).
  * (63) J. Chen, Nuclear data sheets for a=40, Nuclear Data Sheets 140, 1–376 (2017).
  * (64) R. Deslattes et al., X-ray transition energies, 2005.  Accessed on 2023 Nov. 23.
  * (65) J. E. Gaiser, Charmonium Spectroscopy from Radiative Decays of the J/Psi and Psi-Prime,  ph.D. Thesis, Stanford Linear Accelerator Center, Stanford University, (1982).  The function is defined on p.178.
  * (66) S. Agostinelli et al., (GEANT4), Geant4—a simulation toolkit, Nucl. Instrum. Methods Phys. Res. Sect. A 506, 250–303 (2003).
  * (67) B. C. Rasco et al., The nonlinear light output of NaI(Tl) detectors in the Modular Total Absorption Spectrometer, Nucl. Instrum. Methods Phys. Res. Sect. A 788, 137–145 (2015).
  * (68) D. Cano-Ott et al., Monte Carlo simulation of the response of a large NaI(Tl)total absorption spectrometer for $\beta$-decay studies, Nucl. Instrum. Methods Phys. Res. Sect. A 430, 333–347 (1999).


# “Sometimes You Just Gotta Risk It for the Biscuit ”:
A Portrait of Student Risk-Taking

Juho Leinonen  Aalto UniversityEspooFinland<EMAIL_ADDRESS>0000-0001-6829-9449 and  Paul Denny  University of AucklandAucklandNew Zealand
<EMAIL_ADDRESS>
###### Abstract.

Understanding how individuals, including students, make decisions involving
risk is a fundamental aspect of behavioral research. Despite the ubiquity of
risk in various aspects of life, limited empirical work has explored student
risk-taking behavior in computing education. This study aims to partially
replicate prior research on risk-taking behavior in software engineers while
focusing on students, shedding light on the factors that affect their risk-
taking choices. In our work, students were presented with a hypothetical
scenario related to meeting a course project deadline, where they had to
choose between a risky option and a safer alternative. We examined several
factors that might influence these choices, including the framing of the
decision (as a potential gain or loss), students’ enjoyment of programming,
perceived difficulty of programming, and their academic performance in the
course. Our findings reveal intriguing insights into student risk-taking
behavior. First, similar to software engineers in prior work, the framing of
the decision significantly impacted the choices students made, with the loss
framing leading to a higher likelihood for risky choices. Surprisingly,
students displayed a greater inclination towards risk-taking compared to their
professional counterparts in prior research. Furthermore, we observed that
students’ prior academic performance in the course and their enjoyment of
programming had a subtle influence on their risk-taking tendencies, with
better-performing students and those who enjoyed programming being marginally
more prone to taking risks. Notably, we did not find statistically significant
correlations between perceived difficulty of programming and risk-taking
behavior among students.

risk-taking behavior, student decision-making, framing effects, academic
performance

††ccs: Social and professional topics Computing education

##  1\. Introduction

Risk-taking is an inherent aspect of human decision-making, and part of daily
life. From minor choices such as trying a new cuisine to more significant
professional decisions such as how to best implement a new software feature,
many factors contribute to how decisions are made. More complex decisions
often involve carefully weighing up potential benefits against potential
losses, taking into account an individual’s appetite for risk. Quite subtle
factors can also play an important role, including how the choices that one is
deciding between are presented. Indeed, the way that options are framed has
been shown to significantly influence people’s choices, demonstrating that
decision-making is as much about perception as it is about logic. In
educational contexts, if the framing of decisions can impact the choices that
students make, we believe this has important implications for educators.

While there exists a multitude of research on risk-taking in other fields
(Yates, 1992), it has received little attention in computing education
research. Our work is inspired by the very recent paper of Graf-Vlachy in the
field of software engineering, published at ICSE in 2023 (Graf-Vlachy, 2023).
Graf-Vlachy studied the risk-taking behavior of professional software
engineers through a survey experiment with 124 participants. The primary
finding was that the way decisions were presented (framed as gains or losses)
significantly affected software engineers’ risk-taking behaviors. However,
when adjusting for multiple testing, the study did not find a significant
relationship between risk-taking and the Big Five personality traits. This
prior work highlights that engineers are generally risk-averse, but framing
can lead to substantial changes in their level of risk-taking. The results
imply that both individual developers and project managers should be cognizant
of framing effects when making decisions in software projects.

In our work, we partially replicate Graf-Vlachy’s study, but with students
taking an introductory programming course. Firstly, we are interested in
seeing if, similar to professionals, the framing of the risk-taking scenario
correlates with students’ decisions. As the original study did not find
significant correlations between personality traits and risk-taking, we
analyze other attributes that might correlate with risk-taking behavior. More
specifically, we examine the correlation between risk-taking and students’
performance in the course, their enjoyment of programming, and whether they
find programming difficult. Lastly, we asked students to provide a rationale
for their choice, and are interested in examining what rationales students
give for their risk-taking choices. Our research questions for this work are
as follows.

  * •

RQ1. Do framing and order of choices correlate with students’ risk-taking?

  * •

RQ2. What factors correlate with risk-taking?

  * •

RQ3. What rationales do students give for their risk-taking choices?

##  2\. Related Work

Understanding the impact of cognitive biases and individual differences on
decision-making has been studied extensively across a variety of domains,
including medicine and healthcare (Saposnik et al., 2016), work management
(Acciarini et al., 2021), and public policy and governance (DeCaro et al.,
2017). One robust finding is that the decisions people make can often be
heavily influenced by the way that choices are presented to them. For
instance, Tversky and Kahneman describe ‘framing’ effects in which people’s
choices can be influenced by whether the outcomes are presented as gains or as
losses (Tversky and Kahneman, 1981).

###  2.1. Exploring Framing Effects

The classic example used to study framing effects involves presenting someone
with two different options for how a new disease could be managed. If the
options are worded in terms of the number of lives that might be saved (which
is a positive frame), people tend to to be more likely to select the option
that provides certainty. In other words, they are less likely to take a
‘risky’ option where the outcome depends on chance, even if there is a
possibility that more people would be saved. Conversely, if the options are
presented in terms of the number of lives that might be lost (a negative
frame) then people tend to be more likely to select the risky option – even if
the outcomes are statistically equivalent to those in the positively framed
scenario.

This phenomenon – that a choice between a “sure thing” and a risky option of
equal expected value is affected by option phrasing – has been widely
replicated (Fagley and Miller, 1990; Kühberger, 1998; DeKay et al., 2022). A
very recent study by Graf-Vlachy, published at the International Conference on
Software Engineering (ICSE) in 2023, explored this in the domain of software
engineering (Graf-Vlachy, 2023). Recognizing that risk-taking is inherent in
daily software engineering decisions, Graf-Vlachy sought to provide empirical
data on the subject, which is often overlooked in favor of risk management at
the project or organizational level (Masso et al., 2022). A survey experiment
was conducted with a sample of 124 software engineers. Participants were
presented with a software engineering scenario and given two options: a risk-
averse and a risk-seeking option. The framing manipulation involved presenting
these options in two distinct ways: as “gains” and as “losses”.

The scenario asked participants to imagine they were working on a software
project with a deadline, and that some requirements were wrongly implemented
that will result in a missed deadline of 6 weeks. For participants in the
first condition the options were framed as “gains”, i.e., there was a chance
of recovering time. They were presented with the following two options: (A) If
you reduce non-essential features, you will recover 2 weeks or (B) If you
simplify the software architecture, there is a 1/3 chance that you will
recover the full 6 weeks, and there is a 2/3 chance that the simplified
architecture will lead to performance problems and you will not recover any
time at all.

In the second condition, the options were described in terms of “losses”,
i.e., the delay with which they would finish the project, and participants
were asked to choose between: (A) If you reduce non-essential features, you
will finish with a delay of 4 weeks or (B) If you simplify the software
architecture, there is a 1/3 chance that you will finish the project with no
delay at all, and there is a 2/3 chance that the simplified architecture will
lead to performance problems and you will finish with a delay of 6 weeks.

These subtle differences in phrasing were used to study the framing effect on
risk-taking decisions. No correlations were found between personality traits
and risk-taking after multiple testing adjustments, suggesting that individual
predispositions might not be as influential as situational framing. The key
finding was the substantial effect of framing: software engineers exhibited
higher risk-taking behaviors (i.e. favoring options involving an element of
chance) when decisions were framed in terms of losses. The results imply that
both individual developers and project managers should be cognizant of framing
effects when making decisions in software projects.

###  2.2. Framing Effects in Education

In educational contexts, framing effects have been found to significantly sway
student decision-making processes. For instance, Smith and Smith (Smith and
Smith, 2009) found that the framing of grading systems can impact students’
motivation and perceptions. Similarly, Bies-Hernandez (Bies-Hernandez, 2012)
highlight that negatively framed academic evaluations can adversely affect
students’ preferences and performance, and Tansley et al. (Tansley et al.,
2007) show that the framing of messages notably affects students’ career-
related behaviors, particularly when loss-framed. Personal attributes can also
impact framing effects, for example Dunegan (Dunegan, 2010) found that
students with higher GPAs are more sensitive to framing. Other personal
attributes may impact decision making depending on context. For example, when
decisions relate to performance in a course, they may be heavily influenced by
fear of failure which varies significantly between individuals (Coreia et al.,
2017). This body of work on framing and decision making underscores the
critical role of framing in shaping student perceptions and their actions.

To the best of our knowledge, there is no research on how framing effects
might impact students in computing courses. We can see numerous scenarios
where students might be faced with complex decisions as they are studying, and
insights into the factors that might influence risk-taking behaviors for
students can inform pedagogical strategies. In this work we seek to replicate
the study described by Graf-Vlachy, exploring framing effects in the context
of a scenario common to computing students – submitting assignments on time.

##  3\. Methods

###  3.1. Course

Our data was collected from a large first-year programming course taught in
the winter term of 2023 at a large public university in New Zealand. A total
of 889 students were enrolled in the course that term, and all were invited to
participate in our study for a very small amount of course credit
(approximately 0.25% of the final grade).

The size of the course is relevant because a sufficiently large number of
participants is needed to achieve the statistical power necessary for
detecting an effect, if one truly exists, with minimal risk of encountering
false positives or negatives. In the study we are replicating, Graf-Vlachy
conduct a detailed power analysis to determine that 124 is the minimum sample
size in order to detect a medium effect size with a desired power of 0.8
(Graf-Vlachy, 2023) and, in fact, their email-based recruitment resulted in
exactly 124 participants. The much larger cohort in our study (853 students
participated) allows us to account for additional variability in the data, and
detect potentially smaller effect sizes.

The course ran over a total of 12 weeks, and data relevant to this study was
collected in weeks 7 and 10. Given that the scenario we presented students was
related to making a decision about a course project and its deadline, this
timing ensured that all students had prior experience working on and
submitting a project in the first half of the course. We also used data from
the mid-term test, which was conducted in week 6 of the course, to provide a
measure of individual student performance.

At the start of week 7, the mid-point of the course, students were beginning a
new module and were asked to reflect on their prior experiences learning to
program up to that point. Responses were collected online through the course
learning management tool. The following prompt was shown: “Now that you have
finished your first programming module, read the following statements and
reflect on how much you agree with them – respond on a scale from _Strongly
disagree_ to _Strongly agree_ ”, followed by the two statements:

  1. (1)

I find programming enjoyable

  2. (2)

I find programming difficult

###  3.2. Experiment

A total of 853 students participated in the experiment. Students were randomly
presented one of four scenario options. Two groups had the gain framing, and
two had the loss framing for the scenario. Within each condition, the order of
the safe and risky choice was randomly selected, leading to a total of four
groups. For each condition, there was a “safe” option where the outcome was
guaranteed, and a “risky” option where the outcome was based on chance,
randomly leading to a more positive or negative outcome compared to the “safe”
option. The expected value was exactly the same for all choices. The number of
students in each experiment group is outlined in Table 1. The exact phrasing
of the scenario is presented next.

Framing/Order | Risky first | Safe first
---|---|---
Gains | 189 | 192
Losses | 235 | 237
Table 1. Number of students in randomly-assigned experimental groups.

###  3.3. Scenario

Imagine that you are working on a large software project for a university
course with an upcoming deadline on the 10th of the month. You just realize
that you have implemented some of the requirements incorrectly, and you
estimate that this will make you miss the deadline by 6 days (i.e. you will
submit on the 16th). Each day lost will reduce the marks that you will receive
for the project by 10% per day (= 60% in total if you submit on the 16th). You
think about how this could be fixed, and you come up with two options. You can
only choose one.

####  3.3.1. Gain framing

Here, the options are framed as “gains”, or recovering time on the assignment.

  * Option A:

If you start all over again from scratch, you will recover 2 days (i.e. submit
on the 14th).

  * Option B:

If you modify the current implementation, there is a 1/3 chance that you will
recover the full 6 days (i.e. submit on the 10th), and there is a 2/3 chance
that modifying the current implementation does not work and you will not
recover any time at all (i.e. submit on the 16th).

####  3.3.2. Loss Framing

Here, the options are framed as “losses”, or losing time on the assignment so
that a submission will be late.

  * Option A:

If you start all over again from scratch, you will finish with a delay of 4
days (i.e. submit on the 14th).

  * Option B:

If you modify the current implementation, there is a 1/3 chance that you will
finish the project with no delay at all (i.e. submit on the 10th), and there
is a 2/3 chance that modifying the current implementation does not work and
you will finish with a delay of 6 days (i.e. submit on the 16th).

After reading the scenario and selecting their choice, students were prompted
to provide a rationale by answering the question: _Please explain why you
chose that option in a sentence or two_.

###  3.4. Variables

Our study involves a total of six variables: (1) order of the questions, (2)
framing of the scenario, (3) enjoyment of programming, (4) finding programming
difficult, (5) performance in the course, and (6) choice of risk. The detailed
description of the variables and their use is as follows.

  * •

(1) Order of the questions: The order in which the choices were presented to
the students was random. This allows examining if the ordering of the choices
affects which option is selected. Coded as 0 = risky first, 1 = safe first.

  * •

(2) Framing of the scenario: The scenario was framed as gains or losses,
similar to the study by Graf-Vlachy. Coded as 0 = losses, 1 = gains.

  * •

(3) Enjoyment of programming: Halfway through the course, students were asked
to express their level of agreement with the statement _I find programming
enjoyable_. Likert-answer between 1 and 5 (1 = “Strongly disagree”, 2 =
“Disagree”, 3 = “Neutral”, 4 = “Agree”, 5 = “Strongly agree”).

  * •

(4) Finding programming difficult: Halfway through the course, students were
asked to express their level of agreement with the statement _I find
programming difficult_. Likert-answer between 1 and 5 (1 = “Strongly
disagree”, 2 = “Disagree”, 3 = “Neutral”, 4 = “Agree”, 5 = “Strongly agree”).

  * •

(5) Performance in the course: We use students’ performance in the mid-term
examination. Score ranged from 0 to 48 (which was the maximum score
available).

  * •

(6) Choice of risk: Whether students chose the risky (uncertain) outcome or
the safe (guaranteed) outcome.

(a) Distribution of student responses to the question of whether they find
programming enjoyable.

(b) Distribution of student responses to the question of whether they find
programming difficult.

(c) Distribution of student scores in the midterm examination.

Figure 1. Distributions of variables.

Figure 1 shows the distribution of student responses to the questions on
enjoyment and difficulty of programming, and scores in the midterm
examination.

###  3.5. Analysis

To answer RQ1, “Do framing and order of choices correlate with students’ risk-
taking?”, we conduct a Barnard’s exact test111Using the SciPy (Virtanen et
al., 2020) implementation:
https://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.barnard_exact.html
(Barnard, 1947) to analyze whether there are statistically significant
differences between the gain and loss framing (i.e., whether the framing
correlates with the choice), and separately if there are differences between
the order of choices (i.e., whether the order in which the choices are
presented correlates with the choice).

To answer RQ2, “What factors correlate with risk-taking?”, we follow the
method of Graf-Vlachy’s original study (Graf-Vlachy, 2023), and run probit
regression222Using the statsmodels (Seabold and Perktold, 2010) Python
package:
https://www.statsmodels.org/dev/generated/statsmodels.discrete.discrete_model.Probit.html
to analyze how the different variables (framing, order, enjoyment, difficulty,
performance) correlate with the choice students make in the scenario.

In all statistical analyses, a p-value threshold of 0.05 was set to indicate
statistical significance.

To answer RQ3, “What rationales do students give for their risk-taking
choices?”, we randomly sampled 100 responses to the question “Please explain
why you chose that option in a sentence or two”. Two researchers jointly coded
the data for commonly occurring themes. One response could potentially include
multiple themes.

##  4\. Results

###  4.1. RQ1: Do Framing and Order of Choices Correlate With Students’ Risk-
Taking?

Framing/Choice | Risky | Safe
---|---|---
Gain | 190 | 191
Loss | 290 | 182
Table 2. The choices students made with the two different framings (gain or
loss) of the scenario.

Table 2 shows the choices students made with the two different framings of the
scenario. For the gain framing, students were almost evenly split between the
risky (190) and the safe (191) option. For the loss framing, more students
chose the risky option (290 versus 182). The results of a Barnard’s exact test
confirm that the difference between the framings is statistically significant
(statistic = -3.39, p-value = 0.0007).

Order/Choice | Risky | Safe
---|---|---
Risky-first | 243 | 181
Safe-first | 237 | 192
Table 3. The choices students made with the two different orders of the
choices (risky first and safe first).

Table 3 shows the choices students made with the two different orders of the
choices. As can be seen in the table, there seem to be no differences between
the group where the risky option was presented first and the group where the
safe option was presented first. This is confirmed by the results of a
Barnard’s exact test (statistic = 0.61, p-value = 0.55).

###  4.2. RQ2: What Factors Correlate With Risk-Taking?

| Probit model | Marginal effects
---|---|---
| Coeff. | Std. err. | z | p-value | [0.025 | 0.975] | dy/dx | Std. err. | z | P¿—z— | [0.025 | 0.975]
Framing | -0.3206 | 0.091 | -3.513 | ¡0.001 | -0.500 | -0.142 | -0.1218 | 0.034 | -3.600 | ¡0.001 | -0.188 | -0.055
Order | -0.0406 | 0.091 | -0.446 | 0.656 | -0.219 | 0.138 | -0.0154 | 0.035 | -0.446 | 0.655 | -0.083 | 0.052
Enjoys | 0.1172 | 0.049 | 2.384 | 0.017 | 0.021 | 0.213 | 0.0445 | 0.018 | 2.410 | 0.016 | 0.008 | 0.081
Difficult | 0.0139 | 0.054 | 0.255 | 0.799 | -0.093 | 0.120 | 0.0053 | 0.021 | 0.255 | 0.799 | -0.035 | 0.046
Performance | 0.0146 | 0.006 | 2.616 | 0.009 | 0.004 | 0.025 | 0.0055 | 0.002 | 2.650 | 0.008 | 0.001 | 0.010
const | -0.4190 | 0.322 | -1.301 | 0.193 | -1.050 | 0.212 | - | - | - | - | - | -
Table 4. Results of the probit regression, showing the coefficient, standard
error, z-value, p-value, and 95% confidence interval. The dependant variable
was the choice of risky or safe option, which were coded as 0 (safe) and 1
(risky). The independent variables were framing (0 = losses, 1 = gains), order
(0 = risky first, 1 = safe first), whether students enjoy programming, whether
students find programming difficult, and performance in the course. The
marginal effects show the “effect size” of the result, where the dy/dx column
shows the change in the probability of making the risky choice with a one unit
increase in the variable.

Table 4 shows the results of the probit regression. From the table, it can be
seen that two factors do not statistically significantly correlate with the
choice of risky or safe option: order (p-value = 0.656) and whether the
student found programming difficult (p-value = 0.799). However, the other
three factors show a statistically significant correlation with the choice:
framing (p-value ¡ 0.001), enjoyment of programming (p-value = 0.017), and
performance in the coursee (p-value = 0.009). The marginal effects of the
variables on the dependent variable can be seen in the dy/dx column. For
example, one point increase in performance (score in the midterm exam) will
increase the likelihood of making the risky choice by 0.0055.

###  4.3. RQ3: What Rationales Do Students Give for Their Risk-Taking Choices?

Our qualitative analysis of the rationales that students gave for their
choices revealed five common themes. We titled the themes “Risk Tolerance and
Aversion” (46 responses), “Strategic Calculations” (37 responses), “Quality of
Work and Professional Work Ethics” (15 responses), “Mental and Emotional
Factors” (12 responses), and “Optimism versus Pessimism” (9 responses).

The theme of Risk Tolerance and Aversion emerged as the most commonly
mentioned theme, with students weighing the pros and cons of taking risks
versus playing it safe. Those who were averse to risk gravitated towards
ensuring a known outcome and minimizing any potential negative repercussions.
For instance, one student expressed a clear preference for predictability: “I
would much rather play it safe and have a guaranteed lesser penalty”.
Conversely, those who had a higher risk tolerance recognized a unique
opportunity to potentially achieve a perfect score, undeterred by the
possibility of incurring a higher penalty, such as the student who declared,
“Sometimes you just gotta risk it for the biscuit .”

The complexity of decision-making was articulated through the lens of
Strategic Calculations, which was the second most common theme. A subset of
students approached their decisions analytically, dissecting the probabilities
and outcomes at a granular level. One student’s response encapsulated this: “I
would rather have a 100% chance of having 40% grade reduction than a 66.667%
chance of a 60% reduction. Since it means that my max score is 60% and if I
get 50% I can still pass where as with the 60% I cannot pass.” Their
calculations served as the main deciding factor, seemingly offering them a
semblance of control in an otherwise uncertain scenario. This also included
students who made mistakes in their calculations, incorrectly deducing that
one option would have a better expected value than the other: “I calculated
the probability mathematically. OptionA) Percentage lost = 2 * 10% = 20%
OptionB) Percentage lost = 1/3 * 0% + 2/3 * 60% = 40% The Percentage Lost is
less for option A so I choose option A.”

The priority for Quality of Work and Professional Work Ethics underscored many
students’ rationales, influencing them to favor a late submission of higher-
quality work over a rushed, on-time submission with potential errors. For
many, the potential for high-caliber work, reflective of their skills and
effort, outweighed the immediate penalty of delays: “Code that works properly
is better than broken code in production. I rather something works well than
‘just works’ if not at all.” A few students echoed sentiments of completing
projects to the best of their ability in line with professional standards or
even viewing the scenario as a real-world situation influencing their
professional reputation. Their commitment to upholding standards of timeliness
and responsibility was captured by one student: “I’d rather have a chance to
complete the assignment within the deadline as it practices real-world work
experiences and by staying calm under immense stress, I can improve my working
ethics and efforts.”

There were also those who included Mental and Emotional Factors in their
decision-making process. Stress, anxiety, and emotional well-being were
contemplated by students who sought to balance these internal states with the
external pressures of their project deadlines. They made decisions based on
personal emotional management strategies, with one student lamenting: “the
amount of time and effort involved in completing a project that is supposed to
take 6 full days to complete is not negligible, and to reach the conclusion
that you might have to start again from scratch only a couple of days before
the deadline is a crushing realization. for this reason i just know i would
not have the mental fortitude to scrap all of my work and start again.”

The theme of Optimism versus Pessimism was also prominent. Optimists relied on
the potential promise of perfect project completion within the original time
frame, holding onto a hopeful outlook such as, “I chose option B as I am
simply built different and the odds are always in my favour.” Pessimists,
however, assumed the likelihood of negative outcomes, expressing doubts in
statements like, “I’ve played many luck based games, and I know my luck is
usually pretty bad so I would rather option A.”

##  5\. Discussion

We were able to replicate earlier findings (Graf-Vlachy, 2023; Fagley and
Miller, 1990; Kühberger, 1998; DeKay et al., 2022) that have suggested the
framing of choices correlates with risk-taking behavior. Interestingly,
compared to previous results derived from professional software engineers
using a similar scenario (i.e. working on a software project), student
participants in our study reported being more willing to take risks. Graf-
Vlachy (Graf-Vlachy, 2023) reported 7 out of 63 (11.1%) participants in the
‘gain’ condition and 19 out of 61 (31.1%) participants in the ‘loss’ condition
chose the risky option. In contrast, in our case 190 out of 381 (49.9%)
participants in the ‘gain’ condition and 290 out of 472 (61.4%) participants
in the ‘loss’ condition chose the risky option. On the surface, this suggests
that students might be more willing to take risks than professional software
engineers, however this could also be explained by the fact that the stakes
are lower in the context of course projects compared to professional contexts,
where there may be impacts on employment or serious consequences for end users
of production systems.

Related to the factors that correlate with risk-taking behavior, we were
surprised at the result that better performing students were more likely to
take risks. Our initial hypothesis was that better performing students would
be more similar to professional software engineers, i.e. less inclined to take
risks. Conversely, poor performance may be explained by regular risk-taking
involving studying and coursework. However, there are potential explanations
for this. First, the study was conducted after the halfway point of the
course, and we used the mid-term score as the measure of student performance.
Students who did well in the midterm exam had therefore received positive
feedback, and thus may have developed higher self-efficacy which may explain
their confidence in taking risks to earn a higher mark. Better self-efficacy
has been linked with better work-related performance (Stajkovic and Luthans,
1998), and is one of the strongest predictors of performance in introductory
programming as well (Watson et al., 2014; Lishinski et al., 2016). We also
found that students who enjoy programming more were more likely to take risks
compared to those who do not. One possible explanation may be that such
students are more likely to be excited by the prospect of modifying their
current implementation (which was the “risky” option in our experiment).

There has been some interest in computing education research on “nudging”
strategies, where students are gently encouraged towards adopting better
learning habits, often through various forms of messaging (Brown and Parnin,
2021; Zamprogno et al., 2020; Fouh et al., 2021; Edwards and Li, 2020). Given
the quite large effect we have observed in our work, which simply involved
framing options differently, future work should explore how better framing of
‘nudges’ could lead to more impactful results.

###  5.1. Limitations

Although our aim was to replicate Graf-Vlachy’s study (Graf-Vlachy, 2023) as
closely as possible, we necessarily made some modifications. Firstly, we
modified the scenario to involve the deadline for a software project in a
university course, to better fit our educational context. This involved, for
example, changing the time frame in the scenario from weeks to days which may
have had an impact on the results. Research has found that humans typically
take more risks when scenarios involve longer time frames (Björkman, 1984),
and thus a change from weeks to days should have reduced risk-taking behavior.
However, we found that the students in our study were more likely to take
risks overall compared to the software engineers in Graf-Vlachy’s work.
Another change that we made is to more clearly explain the consequences of the
decision to help students understand the scenario (i.e., we stated precisely
how many marks would be lost as penalty for submitting late). In contrast, the
scenario in Graf-Vlachy’s work did not concretely specify the consequences for
a project delay.

Perhaps the most significant limitation of all relates to the construct
validity of using reported decisions on hypothetical scenarios to infer actual
risk-taking behavior. Although there is no reason to believe that students
would deliberately provide dishonest answers, if put in the situation where
real course outcomes were on the line, they may have acted differently. This
is a difficult limitation to overcome as it would be hard, if not impossible,
to conduct a large-scale study of similar nature involving real scenarios.

There may also be other context-dependent factors influencing our results
limiting how well they would generalize. For example, the course in which we
collected our data is highly competitive, given that it is a required course
in an Engineering program where entry into preferred second year
specializations is determined by GPA. The competitive nature of the course may
affect student risk-taking choices, and a different pattern of responses may
be observed in contexts where failing an assignment has lower stakes.

Lastly, there are factors that could affect risk-taking that we did not
collect that would have been interesting to study. For example, Graf-Vlachy
(Graf-Vlachy, 2023) collected information on participants’ programming
experience, while we did not. While Graf-Vlachy did not find a statistically
significant correlation between programming experience and risk-taking, it is
possible that this would be different for students in an introductory
programming course. Similarly, it would have been interesting to look at the
relationship between gender and risk-taking, as prior research has found that
gender correlates with risk-taking behavior (Byrnes et al., 1999). However, we
did not have this data available to us.

##  6\. Conclusions

We conducted an experiment looking at the risk-taking behavior of students in
an introductory programming course inspired by an earlier study by Graf-Vlachy
(Graf-Vlachy, 2023) that looked at the risk-taking behavior of professional
software engineers. We were able to replicate earlier findings suggesting that
the framing of choices correlates with risk-taking, and that framing the
choices as “losses” as opposed to “gains” correlates with making riskier
choices – choices where the outcome is based on chance as opposed to a
guaranteed outcome. Another interesting finding was that students were in
general a lot more prone to taking risks than professional software engineers.
Interestingly, students who performed better in the course, and who enjoyed
programming more, were somewhat more likely to make risky choices, while
whether students found programming difficult or not did not correlate
statistically significantly with risk-taking. Lastly, we found that students
justified their risk-related choices in different ways, such as based on their
risk tolerance, strategic calculations, and emotional factors. Altogether, our
study sheds light on risk-taking behavior of students in computing, which has
been underexplored in previous work.

###### Acknowledgements.

We are grateful for the grants from the Ulla Tuominen Foundation and the
Research Council of Finland (Academy Research Fellow grant number 356114) to
Juho Leinonen.

## References

  * (1)
  * Acciarini et al. (2021) Chiara Acciarini, Federica Brunetta, and Paolo Boccardelli. 2021.  Cognitive biases and decision-making strategies in times of change: a systematic literature review.  _Management Decision_ 59, 3 (2021), 638–652.
  * Barnard (1947) GA Barnard. 1947.  Significance tests for 2$\times$ 2 tables.  _Biometrika_ 34, 1/2 (1947), 123–138.
  * Bies-Hernandez (2012) Nicole J. Bies-Hernandez. 2012.  The Effects of Framing Grades on Student Learning and Preferences.  _Teaching of Psychology_ 39, 3 (2012), 176–180.  https://doi.org/10.1177/0098628312450429 arXiv:https://doi.org/10.1177/0098628312450429
  * Björkman (1984) Mats Björkman. 1984.  Decision making, risk taking and psychological time: Review of empirical findings and psychological theory.  _Scandinavian Journal of Psychology_ 25, 1 (1984), 31–49.
  * Brown and Parnin (2021) Chris Brown and Chris Parnin. 2021.  Nudging Students Toward Better Software Engineering Behaviors. In _2021 IEEE/ACM Third International Workshop on Bots in Software Engineering (BotSE)_. 11–15.  https://doi.org/10.1109/BotSE52550.2021.00010
  * Byrnes et al. (1999) James P Byrnes, David C Miller, and William D Schafer. 1999.  Gender differences in risk taking: A meta-analysis.  _Psychological bulletin_ 125, 3 (1999), 367.
  * Coreia et al. (2017) Marco Estêvão Coreia, António Rosado, Sidónio Serpa, and Vítor Ferreira. 2017.  Fear of failure in athletes: Gender, age and type of sport differences.  _Revista iberoamericana de psicología del ejercicio y el deporte_ 12, 2 (2017), 185–193.
  * DeCaro et al. (2017) Daniel A DeCaro, Craig Anthony Tony Arnol, Emmanuel Frimpong Boama, and Ahjond S Garmestani. 2017.  Understanding and applying principles of social cognition and decision making in adaptive environmental governance.  _Ecology and society: a journal of integrative science for resilience and sustainability_ 22, 1 (2017), 1.
  * DeKay et al. (2022) Michael L DeKay, Nataliya Rubinchik, Zhaojun Li, and Paul De Boeck. 2022.  Accelerating psychological science with metastudies: A demonstration using the risky-choice framing effect.  _Perspectives on Psychological Science_ 17, 6 (2022), 1704–1736.
  * Dunegan (2010) Ken Dunegan. 2010.  GPA and Attribute Framing Effects: Are Better Students More Sensitive or More Susceptible?  _Journal of Education for Business_ 85, 4 (2010), 239–247.  https://doi.org/10.1080/08832320903449444 arXiv:https://doi.org/10.1080/08832320903449444
  * Edwards and Li (2020) Stephen H. Edwards and Zhiyi Li. 2020.  A Proposal to Use Gamification Systematically to Nudge Students Toward Productive Behaviors. In _Proceedings of the 20th Koli Calling International Conference on Computing Education Research_ (Koli, Finland) _(Koli Calling ’20)_. Association for Computing Machinery, New York, NY, USA, Article 28, 8 pages.  https://doi.org/10.1145/3428029.3428057
  * Fagley and Miller (1990) Nancy S Fagley and Paul M Miller. 1990.  The effect of framing on choice: Interactions with risk-taking propensity, cognitive style, and sex.  _Personality and social psychology bulletin_ 16, 3 (1990), 496–510.
  * Fouh et al. (2021) Eric Fouh, Wellington Lee, and Ryan S. Baker. 2021.  Nudging students to reduce procrastination in office hours and forums. In _2021 25th International Conference Information Visualisation (IV)_. 248–254.  https://doi.org/10.1109/IV53921.2021.00047
  * Graf-Vlachy (2023) Lorenz Graf-Vlachy. 2023.  The Risk-Taking Software Engineer: A Framed Portrait. In _2023 IEEE/ACM 45th International Conference on Software Engineering: New Ideas and Emerging Results (ICSE-NIER)_. 25–30.  https://doi.org/10.1109/ICSE-NIER58687.2023.00011
  * Kühberger (1998) Anton Kühberger. 1998.  The influence of framing on risky decisions: A meta-analysis.  _Organizational behavior and human decision processes_ 75, 1 (1998), 23–55.
  * Lishinski et al. (2016) Alex Lishinski, Aman Yadav, Jon Good, and Richard Enbody. 2016.  Learning to program: Gender differences and interactive effects of students’ motivation, goals, and self-efficacy on performance. In _Proceedings of the 2016 ACM conference on international computing education research_. 211–220.
  * Masso et al. (2022) Jhon Masso, Félix García, César Pardo, Francisco J. Pino, and Mario Piattini. 2022.  A Common Terminology for Software Risk Management.  _ACM Trans. Softw. Eng. Methodol._ 31, 4, Article 59 (jul 2022), 47 pages.  https://doi.org/10.1145/3498539
  * Saposnik et al. (2016) Gustavo Saposnik, Donald Redelmeier, Christian C Ruff, and Philippe N Tobler. 2016.  Cognitive biases associated with medical decisions: a systematic review.  _BMC medical informatics and decision making_ 16, 1 (2016), 1–14.
  * Seabold and Perktold (2010) Skipper Seabold and Josef Perktold. 2010.  statsmodels: Econometric and statistical modeling with python. In _9th Python in Science Conference_.
  * Smith and Smith (2009) Jeffrey K. Smith and Lisa F. Smith. 2009.  The impact of framing effect on student preferences for university grading systems.  _Studies in Educational Evaluation_ 35, 4 (2009), 160–167.  https://doi.org/10.1016/j.stueduc.2009.11.001
  * Stajkovic and Luthans (1998) Alexander D Stajkovic and Fred Luthans. 1998.  Self-efficacy and work-related performance: A meta-analysis.  _Psychological bulletin_ 124, 2 (1998), 240.
  * Tansley et al. (2007) Denny P. Tansley, LaRae M. Jome, Richard F. Haase, and Matthew P. Martens. 2007.  The Effects of Message Framing on College Students’ Career Decision Making.  _Journal of Career Assessment_ 15, 3 (2007), 301–316.  https://doi.org/10.1177/1069072707301204 arXiv:https://doi.org/10.1177/1069072707301204
  * Tversky and Kahneman (1981) Amos Tversky and Daniel Kahneman. 1981.  The framing of decisions and the psychology of choice.  _science_ 211, 4481 (1981), 453–458.
  * Virtanen et al. (2020) Pauli Virtanen, Ralf Gommers, Travis E. Oliphant, Matt Haberland, Tyler Reddy, David Cournapeau, Evgeni Burovski, Pearu Peterson, Warren Weckesser, Jonathan Bright, Stéfan J. van der Walt, Matthew Brett, Joshua Wilson, K. Jarrod Millman, Nikolay Mayorov, Andrew R. J. Nelson, Eric Jones, Robert Kern, Eric Larson, C J Carey, İlhan Polat, Yu Feng, Eric W. Moore, Jake VanderPlas, Denis Laxalde, Josef Perktold, Robert Cimrman, Ian Henriksen, E. A. Quintero, Charles R. Harris, Anne M. Archibald, Antônio H. Ribeiro, Fabian Pedregosa, Paul van Mulbregt, and SciPy 1.0 Contributors. 2020.  SciPy 1.0: Fundamental Algorithms for Scientific Computing in Python.  _Nature Methods_ 17 (2020), 261–272.  https://doi.org/10.1038/s41592-019-0686-2
  * Watson et al. (2014) Christopher Watson, Frederick WB Li, and Jamie L Godwin. 2014.  No tests required: comparing traditional and dynamic predictors of programming success. In _Proceedings of the 45th ACM technical symposium on Computer science education_. 469–474.
  * Yates (1992) Jacques Yates. 1992.  _Risk-taking behavior_.  John Wiley & Sons.
  * Zamprogno et al. (2020) Lucas Zamprogno, Reid Holmes, and Elisa Baniassad. 2020.  Nudging student learning strategies using formative feedback in automatically graded assessments. In _Proceedings of the 2020 ACM SIGPLAN Symposium on SPLASH-E_ (Virtual, USA) _(SPLASH-E 2020)_. Association for Computing Machinery, New York, NY, USA, 1–11.  https://doi.org/10.1145/3426431.3428654