#  LLM Internal States Reveal Hallucination Risk Faced With a Query

Ziwei Ji, Delong Chen, Etsuko Ishii, Samuel Cahyawijaya,
Yejin Bang, Bryan Wilie, Pascale Fung
Center for Artificial Intelligence Research (CAiRE)
Hong Kong University of Science and Technology
<EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract

The hallucination problem of Large Language Models (LLMs) significantly limits
their reliability and trustworthiness. Humans have a self-awareness process
that allows us to recognize what we don’t know when faced with queries.
Inspired by this, our paper investigates whether LLMs can estimate their own
hallucination risk before response generation. We analyze the internal
mechanisms of LLMs broadly both in terms of training data sources and across
15 diverse Natural Language Generation (NLG) tasks, spanning over 700
datasets. Our empirical analysis reveals two key insights: (1) LLM internal
states indicate whether they have seen the query in training data or not; and
(2) LLM internal states show they are likely to hallucinate or not regarding
the query. Our study explores particular neurons, activation layers, and
tokens that play a crucial role in the LLM perception of uncertainty and
hallucination risk. By a probing estimator, we leverage LLM self-assessment,
achieving an average hallucination estimation accuracy of 84.32% at run time.

##  1 Introduction

Humans have an awareness of the scope and limit of their own knowledge Fleming
and Dolan (2012); Koriat (1997); Hart (1965), as illustrated in Fig. 1. This
cognitive self-awareness ability in humans introduces hesitation in us before
we respond to queries or make decisions in scenarios where we know we don’t
know Yeung and Summerfield (2012); Nelson (1990); Bland and Schaefer (2012).
However, LLM-based AI assistants lack this cognitive uncertainty estimation.
Consequently, they tend to be overconfident and may produce plausible-sounding
but unfaithful or nonsensical contents called hallucination or confabulation
Ji et al. (2022); Xiao and Wang (2021); Bang et al. (2023); Xiong et al.
(2023). This problem limits their applications in numerous real-world
scenarios and undermines user trustworthiness.

Figure 1: Humans have self-awareness and recognize uncertainties when
confronted with unknown questions. LLM internal states reveal uncertainty even
before responding. Pink dots are the internal LLM states associated with
hallucinated responses, whereas Blue dots are those of faithful responses. The
queries leading to those LLM responses are colored accordingly.

Previous research Bricken et al. (2023); Templeton et al. (2024); Bills et al.
(2023); Wu et al. (2024) have explored the internal states of language models
that capture contextual and semantic information learned from training data
Liu et al. (2023); Chen et al. (2024); Gurnee and Tegmark (2023).
Nevertheless, internal states of language models sometimes exhibit limited
generalization on unseen data and their representation effectiveness can be
undermined by flawed training data or modeling issues Wang et al. (2022a);
Belinkov and Glass (2019); Meng et al. (2021); Xie et al. (2022); Carlini et
al. (2021); Yin et al. (2023a). Notably, recent works have shown that the
LLM’s internal states can potentially detect hallucinations in texts Azaria
and Mitchell (2023); Chen et al. (2024); Su et al. (2024). However, these
works examine texts not exclusively produced by the same LLMs whose internal
states are analyzed, highlighting the necessity for further investigation into
the LLM self-awareness and how their internal states correlate with their
uncertainty and own hallucination occurrence.

Our work takes a step further by investigating whether LLM internal states
have some indication of hallucination risk given queries and whether it can be
reliably estimated even before the actual response generation (Fig. 1). We
conduct a comprehensive analysis of the LLMs internal mechanisms both in terms
of training data sources and across 15 diverse NLG tasks, spanning over 700
datasets through an in-depth analysis. We explore particular neurons,
different activation layers, and tokens that play a crucial role in the LLM
perception of uncertainty and hallucination risk. Employing a probing
estimator Belinkov (2022) on the internal states associated with the queries,
we validate their self-awareness and ability to indicate uncertainty in two
aspects: (1) Whether they have seen the query in training data, achieving an
accuracy of 80.28%. (2) Whether they are likely to hallucinate regarding the
query, achieving an average estimation accuracy of 84.32% across 15 NLG tasks.
We propose that understanding these representations could offer a proactive
approach to estimating uncertainty, potentially serving as an early indicator
for the necessity of retrieval augmentation Wang et al. (2023) or as an early
warning system.

##  2 Hallucination and Training Data

The sources of hallucination in LLMs can be traced back to data and modeling
Ji et al. (2022). Factors tied to data encompass unseen knowledge, task-
specific innate divergence, noisy training data, etc. From the modeling
perspective, hallucinations can be traced to the model architecture, alignment
tax, teacher-forced maximum likelihood estimation (MLE) training, etc.

A common situation where hallucinations from data occur is when LLMs attempt
to provide information on unseen queries that are not included in their
training set, rather than refusing to reply. Previous works Kadavath et al.
(2022); Rajpurkar et al. (2018); Onoe et al. (2022); Yin et al. (2023b)
explore to identify the unseen data based on various indicators such as text
similarity, perplexity. We investigate the capability of LLMs to recognize
whether they have seen the query in training data via novel analysis of their
internal states. To facilitate analysis, we craft two sets of queries by
collecting news from periods before and after the release of the LLM we
analyze to represent unseen and seen data, respectively.

However, in the real-world scenario, it’s impractical to definitively
categorize data as entirely seen or unseen due to the inability to access the
vast training data of LLM. Thus, we expand the preliminary insights and
further investigate LLMs’ self-awareness of recognizing whether models are
likely to hallucinate regarding the query. It’s important to note that the
hallucinations are source-agnostic, meaning they can result from both unseen
and seen data. The latter can still trigger hallucinations due to deficiencies
in modeling. To facilitate analysis, we construct data by using LLM to
directly generate responses to queries across diverse NLG tasks and then label
the hallucination level in the responses.

##  3 Methodology

Figure 2: Visualization of the Neurons for Hallucination Perception in various
NLG tasks. Pink dots represent Unknown Queries triggering hallucinations and
Blue dots represent Known Queries.

This section begins with an introduction to the problem formulation of
uncertainty estimation faced with queries in § 3.1. We construct datasets in §
3.2 focusing on two dimensions: (1) the distinction between queries seen and
unseen in the training data; (2) the likelihood of hallucination risk faced
with the queries. To validate the efficacy of internal state representation in
hallucination estimation, we visualize the neurons for perception extracted
from a specified LLM layer (§ 3.3) and then leverage the probing classifier
technique Belinkov (2022) on top of internal states associated with the last
token of queries (§ 3.4).

###  3.1 Problem Formulation

Suppose we have an LLM $f$ parameterized by $\theta$. It is able to gain
internal states $I$ and generate response $r$ given user query $q$ represented
as $I,r=f_{\theta}(q)$. We aim to investigate the self-awareness of LLM,
specifically how their internal states $I$ relate to their level of
hallucination risk $h$ when faced with a query $q$. This correlation is
formulated by $u=\mathds{E}(I_{\theta,q})$, where $\mathds{E}$ signifies an
estimator function. The combination of model-specific attribute via
$f_{\theta}$, and query-only attribute via $q$, allows it to accurately
capture the characteristics of individual LLMs and fosters a more efficient
prediction mechanism that mirrors human cognitive processes.

We employ a dataset $\mathcal{D}_{\theta}=\\{\langle
I_{\theta,q,i}^{\text{train}},h^{\text{train}}_{i}\rangle\\}_{i=1}^{N}$
consisting of $N$ query-label pairs. These pairs serve to represent the
behavior of $f_{\theta}$ Here, $h^{\text{train}}_{i}$ denotes the
hallucination risk label, which is determined based on (1) the query’s
presence in the training data or (2) the degree of hallucination in the
response $r$ to $q$. Thus, our objective is mathematically expressed as:

$h=\mathds{E}(I_{\theta,q};\mathcal{D}_{\theta})$ (1)

###  3.2 Data Construction

As introduced in § 1 and 2, we investigate LLM internal states’ self-awareness
and ability to indicate uncertainty in two aspects: (1) whether they have seen
the query in training data; and (2) whether they are likely to hallucinate
when faced with the query.

#### (1) Seen/Unseen Query in Training Data

The unseen queries will trigger hallucinations due to the lack of information
within the model’s training data when the model doesn’t refuse to respond. In
other words, hallucinations triggered by unseen queries are data-related. To
investigate the distinguishability between seen and unseen queries, we
construct a compact dataset consisting of two distinct sets of queries. For
the seen group, we collect historical BBC news in 2020 likely exposed during
LLM’s training. For the unseen group, we collect recent BBC news in 2024 after
the release of the LLM we analyze. To ensure comparability, we ensure these
two sets share similar length distributions and semantic information via
sentence embeddings 111https://huggingface.co/sentence-transformers/all-mpnet-
base-v2.

#### (2) Hallucination Risk faced with the Query

We first construct data using LLM to directly generate responses to queries in
diverse NLG tasks. Subsequently, for labeling the responses and corresponding
queries, a comprehensive integration of NLG metrics assesses the levels of
hallucination. This generation only uses the parametric knowledge of LLM which
is a proxy of performance in real-world applications. This generation process
can also be in other settings, such as retrieval-augmented generation, to
explore whether the internal states can estimate hallucination risk or other
aspects’ performance in these settings.

To evaluate the generated responses, we implement a multi-faceted evaluation
approach. We employ classical Rouge-L (Lin, 2004), which compares the
generated response with gold-standard reference. To measure hallucination
level, we also utilize Natural Language Inference (NLI)
222https://huggingface.co/MoritzLaurer/mDeBERTa-v3-base-xnli-multilingual-
nli-2mil7 and Questeval Scialom et al. (2021). NLI is a common metric for
hallucination evaluation Ji et al. (2023a, b) which assesses the logical
consistency/entailment of generated text with the provided context or the
reference. QuestEval is a QA-based metric for evaluating the faithfulness of
the output in generation tasks. This work adopts its reference-dependent mode
depending both on the input source and golden reference.

To make up for deficiencies of single automatic metrics, we integrate these
three metrics comprehensively. If NLI predicts entailment and both Rouge-L and
Questeval exceed their respective median values, we assign a label of 1.
Conversely, if NLI predicts contradiction or neutrality, and both Rouge-L and
Questeval fall below their median values, we assign a label of 0. This
labeling strategy not only provides a binary quality assessment but also
reflects a multi-dimensional evaluation of the text, capturing the
hallucination level of the generated responses.

###  3.3 Preliminary Analysis: Neurons for Hallucination Perception from
Internal States

Internal states play a crucial role in language models, encapsulating rich
contextual and semantic information learned from predicting tokens. They are
adept at recognizing complex patterns and relationships pertinent to various
NLP tasks, which positions them as potentially powerful tools for estimating
the risk of hallucinations Azaria and Mitchell (2023); Liu et al. (2023); Chen
et al. (2024). To analyze the self-assess sense of internal states and the
role of specific neurons in the uncertainty and hallucination estimation, we
employ a feature selection method based on Mutual Information Kraskov et al.
(2004). This technique measures the relevance of different features for
distinguishing between the categories in a dataset.

In the context of NLG tasks including dialogue, QA, and translation, we select
the eight most significant neurons from the last activation layer and
visualize them in Fig. 2. We observe that these neurons exhibit sensitivity to
uncertainty, allowing them to distinguish between different hallucination
levels given known and unknown queries. In other words, there exist individual
neurons within LLM that can fairly perceive uncertainty and predict future
hallucinations. This approach not only enhances our understanding of the
neural correlates of hallucinations but also paves the way for developing
targeted interventions that mitigate the effects of hallucinations.

###  3.4 Internal State-based Estimator

Based on the above preliminary analysis, we use internal states corresponding
to the last token of queries, denoted as $x_{q}$. These representations, taken
from a specified layer within the LLM, serve as the input to our estimation
model. The accessibility and ease of obtaining these states further underscore
their practicality for such applications.

For the architecture of our estimator, we employ a variant of the multilayer
perceptron (MLP) adapted from the Llama Touvron et al. (2023). The estimator
is mathematically formulated as:

$H=\mathrm{down}(\mathrm{up}(x_{q})\times\mathrm{SiLU}(\mathrm{gate}(x_{q})))$
(2)

where $\mathrm{SiLU}$ is the activation function. $\mathrm{down}$,
$\mathrm{up}$, and $\mathrm{gate}$ are linear layers for down-projection, up-
projection, and gate mechanisms, respectively. The combination of internal
states and the Llama MLP structure handles the complexity of hallucination
risk estimation in NLG tasks.

##  4 Experiments

Figure 3: Automatic evaluation results for our method and baselines including
Perplexity (PPL), Zero-shot Prompt, and In Context Learning (ICL) Prompt.

###  4.1 Dataset

#### (1) Seen/Unseen Query in Training Data

LatestEval Li et al. (2024) is a benchmark designed to tackle data
contamination in evaluation through dynamic and time-sensitive construction.
It includes data from various time periods. In our study, we utilize BBC news
collected in Jan. 2020 and 2024, which correspond to periods before and after
the release of the LLM. The selected dataset includes 2.8k samples allowing us
to represent both seen and unseen data in our analysis.

#### (2) Hallucination Risk faced with the Query

Super-Natural Instructions Wang et al. (2022b) is a large and diverse
benchmark including 1,616 diverse NLP datasets and their expert-written
instructions. Among this benchmark, we select 15 NLG task categories including
QA, Summarization, Translation, etc consisting of over 700 datasets. Please
find the full list of NLG task categories in Fig. 3 or 4 and the full list of
tasks in Tab. D in § A. We also leverage ANAH Ji et al. (2024) to test our
estimator’s performance in the hallucination aspect and its generalization.
ANAH is a bilingual dataset that offers analytical annotation of
Hallucinations in LLMs within Generative QA. Our work uses English samples and
treats the hallucination type as the label in the testing stage.

###  4.2 LLM

In this work, we primarily use Llama2-7B Touvron et al. (2023) as our
generative model and delve into its internal states to access hallucination
risk estimation. In addition, we explore the impact of different internal
states in the Mistral-7B Jiang et al. (2023) in § 5.

###  4.3 Baselines

To explore query-only uncertainty estimation, we involve straightforward
prompt-based approaches as baselines.

#### Zero-shot Prompt

We directly ask the LLM whether it can accurately respond to the query via the
following prompt: "Query: {Query}$\backslash n\backslash n$Are you capable of
providing an accurate response to the query given above? Respond only to this
question with ’yes’ or ’no’ and do not address the content of the query
itself."

#### In-Context-Learning (ICL) Prompt

We ask the LLM whether it can accurately respond to the query and give some
examples: "Are you capable of providing an accurate response to the following
query? Respond only to this question with ’yes’ or ’no’ and do not address the
content of the query itself.$\backslash n\backslash n$Query: {Example Query
0}$\backslash n$Answer: no$\backslash n\backslash n$Query: {Example Query
1}$\backslash n$Answer: yes...$\backslash n\backslash n$Query:
{Query}$\backslash n$Answer:"

#### Perplexity (PPL)

Considering the prompt-based methods only use the model’s inner knowledge, we
also incorporate the distribution of the training dataset and employ a
Perplexity (PPL)-based baseline. Assume LLMs are trained on a hypothetical
large dataset that perfectly contains every possible query-response pair,
where the responses are guaranteed to be faithful. Then, the hallucination
estimation can be simply done by checking whether the given query appears in
the training corpus Lee et al. (2021); Kandpal et al. (2023). To determine
this threshold, we first calculate the PPL for each query. Subsequently, we
identify the optimal PPL threshold that yields the maximum accuracy on our
training dataset. This optimal threshold is then applied to the test dataset
to gauge the accuracy of our hallucination risk estimation method.

Figure 4: F1 scores of Internal-State from Different Layers for Hallucination
Estimation.

###  4.4 Estimator Evaluation Protocols

For the classification task with the discrete type predicted, we utilize F1
and Accuracy to measure the quality of predicted categorization.

##  5 Results and Analysis

| Training Task
---
| Testing Task
---
F1 | ACC
QA | Unseen QA | 64.79 | 73.32
Translation | 51.34 | 65.10
Translation | Unseen Translation | 74.03 | 73.81
QA | 20.45 | 37.50

Table 1: Zero-Shot Automatic Evaluation Results in the Same Task and across
Different Tasks.

###  5.1 Results for Internal State-based Estimator

#### (1) Seen/Unseen Query in Training Data

We evaluate our internal state-based estimator trained to distinguish unseen
and seen questions. The F1 and accuracy scores reach 80.28% and 80.24% These
high results shed light on the effectiveness of our internal state-based
method in identifying unseen queries. This phenomenon is aligned with the
previous works Kadavath et al. (2022); Yin et al. (2023b) which find the model
can distinguish answerable and unanswerable questions that include future
information.

#### (2) Hallucination Risk faced with the Query

For estimating hallucination risk, as depicted in Fig. 3, our methods exhibit
superior performance in both F1 and ACC. Notably, its performance remains
stable across different tasks. It performs less effectively in the translation
task (F1 and ACC 76.90%) while excelling in the Number Conversion task (F1
94.04%, ACC 96.00%). Zero-shot prompt and ICL yield similar results, with ICL
slightly outperforming zero-shot prompt. Both methods tend to be overconfident
and predict LLM can accurately respond to the query (Recall 99%), which is
aligned with the observation of Xiong et al. (2023). PPL is better than the
prompt methods while exhibiting varying performance across tasks. It performs
poorly in the translation task (F1 33.73%, ACC 50.36%) but achieves its best
performance in the data-to-text task (F1 88.28%, ACC 92.08%).

Task |  | Internal State
---
F1 | ACC
Dialogue | Llama2 | 74.33 | 74.22
Mistral | 72.39 | 72.55
QA | Llama2 | 82.37 | 82.55
Mistral | 80.46 | 81.00
Summarization | Llama2 | 88.08 | 88.95
Mistral | 83.63 | 85.42
Translation | Llama2 | 76.90 | 76.90
Mistral | 73.10 | 73.14

Table 2: Automatic Evaluation Results of Internal States from Different
Models.

###  5.2 Analysis

#### Layer Depth Positively Correlates with its Prediction Performance.

We systematically dissect the contribution of each layer to the overall
hallucination risk estimation. We hypothesize that certain layers may be more
indicative of hallucinatory propensities than others, and our analysis seeks
to validate this hypothesis. As shown in Fig. 4, early Layers perform poorly
since they often capture basic syntactic information. Intermediate layers
perform better since these layers typically encode more complex semantic
relationships. Deep layers perform best and learn hallucination patterns with
high-level presentation. This observation is different from Azaria and
Mitchell (2023) where middle-layer hidden states of statements perform best in
recognizing lying.

#### Consistency of Internal States across Different LLMs

To evaluate the impact of the LLM’s Internal State, we use Mistral-7B’s
internal state to assess Llama2’s hallucination risk. As shown in Tab. 2, the
results for four common NLG tasks exhibit a decrease compared to Llama2’s own
internal states. Since different LLMs share a similarity in model architecture
and data, there is a potential for zero-shot transfer. Nonetheless, the most
effective predictor of LLM’s generative performance is still its own internal
state, which underscores the importance of considering model-specific
assessments rather than universal ones.

#### Internal States Share Features inner-task but do not Cross-task.

As shown in Tab. 1, we evaluate the generalization across different NLG tasks
and within the same NLG task. Specifically, we examine zero-shot performance
in QA and translation. While the zero-shot performance within these individual
tasks is acceptable, the cross-task generalization remains relatively weak,
aligned with the findings reported by Kadavath et al. (2022). In addition, we
evaluate our estimator trained in QA on the out-of-domain hallucination QA
dataset ANAH. The F1 score reaches 78.56% and the accuracy is 78.83%. These
relatively high results shed light on the effectiveness of our internal state
estimator in handling hallucination challenges and further show our
generalization capabilities. Therefore, the features in internal states are
shared with OOD data within the same task but not shared across tasks.

Figure 5: Inference time of various estimation methods.

#### Internal State as an Efficient Hallucination Estimator

Our estimator has three linear layers which requires minimal computing power.
As shown in Fig. 5, our estimator demonstrates impressive efficiency.
Specifically, likelihood-based costs 1.36s per sample, while internal-state-
based costs only 0.05s per sample. This rapid inference speed is essential for
real-world applications. The generation time, with a maximum token length of
50 and a batch size of 1, is 3.37 seconds. Notably, Questeval costs the most
time 10.25s in total.

#### Hallucination Rate

During the labeling process mentioned in § 3.2, we obtain the hallucination
rate in the responses of each task. As illustrated in Fig. 6, the
hallucination rate fluctuates significantly across NLG tasks. Among them,
Title Generation exhibits the highest rate since its divergent nature and
there is no unique and standard answer. In contrast, Number Conversion gains
the lowest rate since the task is relatively easy and the answer is fixed
leaving less room for hallucination.

Figure 6: Hallucination Rate for each NLG Task.

Query |  | Generated
---
Reply
| Golden
---
Answer
| Trainig
---
Label
| Predicted
---
Label
Given a scientific question, generate a correct answer to it. What is the term for tough, flexible connective tissue that contains the protein collagen? |  Ligaments are fibrous connective tissue structures that connect bones to other bones and provide support and stability to the joint… | cartilage | hallucinated | faithful
Given a scientific question, generate a correct answer to it. Since 1900, four of the five earthquakes of the greatest magnitude occurred near what appropriately nicknamed pacific location? |  Since 1900, four of the five earthquakes of the greatest magnitude have occurred near the Pacific location of the Ring of Fire. The Ring of Fire is an area of… | ring of fire | faithful | hallucinated

Table 3: Negative Samples. The hallucinated context is marked in pink.

#### Visualizing Tokens Triggering Hallucination

To further understand the mechanisms behind hallucination, we dissect the
process of the queries triggering hallucinations at a fine-grain level.
Inspired by the Gradient-weighted Class Activation Mapping (Grad-CAM)
technique Selvaraju et al. (2017), we quantify the average gradients of input
embedding associated with each token in the joint operation of the LLM and
estimator. Specifically, we focused on how these tokens influence the LLM’s
internal state and the subsequent estimation of hallucinations based on this
internal state.

Fig. 7 indicates that tokens within an unknown query contribute unequally to
the occurrence of hallucinations. We observe that the tokens that are part of
unfamiliar named entities or carry critical information exhibit a higher
impact. For instance, “amniotes” in the QA task and “麻雀” in the translation
task gain higher gradients and significantly impact hallucination estimation.
This could be attributed to the system’s attempts to generate fluent responses
despite gaps in its understanding or knowledge about these entities.

Figure 7: Visualization of Token Contributions to hallucinations in unknown
queries for QA (top) and translation (bottom) tasks. Deeper background color
means higher contributions and the hallucinated content in the generated reply
is marked in pink.

#### Error Analysis

Although our method performs better than the baselines in the estimation task,
it still generates a few cases of failure. To gain more insight into our
model, we present a failure example in Tab. 3 and conduct an error analysis.
For example, the estimator predicts that LLM can correctly answer the query
“What is the term for tough, flexible connective tissue that contains the
protein collagen?” But LLM replies “ligaments” instead of “cartilage”, which
is hallucinated. the estimator predicts that LLM will hallucinate when faced
with “what appropriately nicknamed pacific location?” But LLM replies without
hallucination.

More analysis is described in Appendix B including treating separate metrics
as continuous regression labels and different estimator backbones.

##  6 Related Work

#### Knowledge Boundary

Researchers investigate the boundary of parametric knowledge in LLMs which aim
to uncover what models know and where their capabilities end. Current research
on the knowledge boundary predominantly narrows down this investigation to
specialized tasks, charting a map of knowns and unknowns within these defined
borders. In the representative QA field, Rajpurkar et al. (2018) and Yin et
al. (2023b) collect datasets containing known and unknown questions and
develop classifiers to differentiate them. The concept of unanswerability in
these works is universal and model-agnostic, such as philosophical questions
and unsolved mysteries. In contrast, Kadavath et al. (2022) focuses on model-
specific prediction of which questions the model will be able to answer
correctly. This approach requires direct fine-tuning of the LLM to make it a
classifier in a few-shot setting. Out-of-domain or out-of-distribution
detection Zhou et al. (2023); Ryu et al. (2018); Tan et al. (2019); Yang et
al. (2021); Zheng et al. (2020) are also relevant areas dealing with the
differentiation of unknown/unseen from training data, with main focus on
classification tasks. Our method is versatile across various NLG tasks without
requiring fine-tuning of LLMs.

#### Hallucination Detection

The phenomenon of hallucination in NLG encourages a variety of detection
methods Min et al. (2023); Ji et al. (2024); Li et al. (2023); Scialom et al.
(2021). Some of these methods delve into the internal states for detection.
Azaria and Mitchell (2023), for example, collect a true-false statement
dataset with artificial guidance and the classification results indicate that
the LLMs’ internal state can reveal the truthfulness of statements. INSIDE
Chen et al. (2024) also leverages LLMs’ internal states and proposes
EigenScore for evaluating the self-consistency of responses, thereby serving
as a proxy for hallucination levels. MIND Su et al. (2024), an unsupervised
training approach, distinguishes the hallucinated continuation text from the
original Wikipedia content based on internal states. On the other hand, Xiao
and Wang (2021) shows evidence that higher uncertainty corresponds to a higher
hallucination probability. Uncertainty estimation methods, such as Xiao et al.
(2022); Xiong et al. (2023); Kadavath et al. (2022), predict the reliability
of their natural language outputs and can also serve as a tool for
hallucination detection. Previous works leverage the LLM’s internal states for
the text to be measured which is not necessary from the same LLM. Differently,
this work focuses on self-awareness corresponding to the queries.

##  7 Conclusion

Inspired by human self-awareness, this work demonstrates the latent capacity
of LLMs to self-assess and estimate hallucination risks prior to response
generation. We conduct a comprehensive analysis of the internal states of LLMs
both in terms of training data sources and across 15 NLG tasks with over 700
datasets. Employing a probing estimator on the internal states associated with
the queries, we assess their self-awareness and ability to indicate
uncertainty in two aspects: (1) recognizing whether they have seen the query
in training data, achieving an accuracy of 80.28%333Please refer to the first
parts of § 3.2, § 4.1, and § 5.1.. (2) recognizing whether they are likely to
hallucinate when faced with the query. The results demonstrate that internal
state-based self-assessment outperforms PPL-based and prompting-based
baselines, with an average estimation accuracy of 84.32% across all tested
datasets. In addition, we explore the role of particular neurons in
uncertainty and hallucination perception and reveal a positive correlation
between the depth of activation layers in an LLM and its predictive accuracy.
The consistency of internal states across different models suggests a
potential for zero-shot transfer, but model-specific estimation is the optimal
strategy. Challenges of generalizing these findings across different tasks are
noted, despite observing promising generalizations within the same NLG tasks.

For future work, we aim to refine our methodology to enhance the robustness
and generalization across various NLG tasks in the field of hallucination risk
assessment. In addition, we will involve a broader spectrum of LLMs to extend
the applicability of our findings.

##  8 Limitation

#### Model Coverage

This work primarily investigate the widely used LLM, Llama2, due to its
prevalence in current NLG applications. However, it does not encompass other
LLMs. In the future, we will extend the scope of LLMs to enhance the
robustness and applicability of our results.

#### Human Evaluation in Data Construction

Human judgment is extremely resource-intensive for hallucination judgment. The
extensive time commitment and financial expenditure required are beyond the
scope of this study, particularly given the large scale of the datasets.
Consequently, this research did not include human evaluation in the data
labeling process in § 3.2.

#### Comparative Performance

Our method predicts the risk in advance of generation and depends solely on
the query. It may lead to a trade-off in performance compared to other
existing approaches that consider both the query and the response.

##  9 Ethical Considerations

In our experiments, we utilized datasets that are either publicly accessible
or synthetically generated, thereby circumventing any potential adverse
effects on individuals or communities. The datasets employed in this
investigation were meticulously curated and processed to uphold the principles
of privacy and confidentiality. We ensured the exclusion of any personally
identifiable information, with all data undergoing anonymization before any
analysis was conducted.

When contemplating the deployment of our research outcomes, we recognize the
inherent risks and ethical dilemmas involved. The tendency of LLMs to produce
hallucinations could disproportionately affect various demographic groups, a
consequence of the inherent biases in the training datasets. We are committed
to the identification and rectification of such biases to forestall the
continuation of stereotypes or the inequitable treatment of any demographic.

By adhering to these ethical considerations, we aim to contribute positively
to the field of NLP and ensure that advancements in understanding and
mitigating hallucinations in LLMs are achieved responsibly and with
consideration for the broader societal impact.

## References

  * Azaria and Mitchell (2023) Amos Azaria and Tom Mitchell. 2023.  The internal state of an llm knows when it’s lying.  In _Findings of the Association for Computational Linguistics: EMNLP 2023_ , pages 967–976.
  * Bang et al. (2023) Yejin Bang, Samuel Cahyawijaya, Nayeon Lee, Wenliang Dai, Dan Su, Bryan Wilie, Holy Lovenia, Ziwei Ji, Tiezheng Yu, Willy Chung, Quyet V. Do, Yan Xu, and Pascale Fung. 2023.  A multitask, multilingual, multimodal evaluation of chatgpt on reasoning, hallucination, and interactivity.  _AACL_.
  * Belinkov (2022) Yonatan Belinkov. 2022.  Probing classifiers: Promises, shortcomings, and advances.  _Computational Linguistics_ , 48(1):207–219.
  * Belinkov and Glass (2019) Yonatan Belinkov and James Glass. 2019.  Analysis methods in neural language processing: A survey.  _Transactions of the Association for Computational Linguistics_ , 7:49–72.
  * Bills et al. (2023) Steven Bills, Nick Cammarata, Dan Mossing, Henk Tillman, Leo Gao, Gabriel Goh, Ilya Sutskever, Jan Leike, Jeff Wu, and William Saunders. 2023.  Language models can explain neurons in language models.  https://openaipublic.blob.core.windows.net/neuron-explainer/paper/index.html.
  * Bland and Schaefer (2012) Amy R Bland and Alexandre Schaefer. 2012.  Different varieties of uncertainty in human decision-making.  _Frontiers in neuroscience_ , 6:85.
  * Bricken et al. (2023) Trenton Bricken, Adly Templeton, Joshua Batson, Brian Chen, Adam Jermyn, Tom Conerly, Nick Turner, Cem Anil, Carson Denison, Amanda Askell, Robert Lasenby, Yifan Wu, Shauna Kravec, Nicholas Schiefer, Tim Maxwell, Nicholas Joseph, Zac Hatfield-Dodds, Alex Tamkin, Karina Nguyen, Brayden McLean, Josiah E Burke, Tristan Hume, Shan Carter, Tom Henighan, and Christopher Olah. 2023.  Towards monosemanticity: Decomposing language models with dictionary learning.  _Transformer Circuits Thread_.  Https://transformer-circuits.pub/2023/monosemantic-features/index.html.
  * Carlini et al. (2021) Nicholas Carlini, Florian Tramer, Eric Wallace, Matthew Jagielski, Ariel Herbert-Voss, Katherine Lee, Adam Roberts, Tom Brown, Dawn Song, Ulfar Erlingsson, et al. 2021.  Extracting training data from large language models.  In _30th USENIX Security Symposium (USENIX Security 21)_ , pages 2633–2650.
  * Chen et al. (2024) Chao Chen, Kai Liu, Ze Chen, Yi Gu, Yue Wu, Mingyuan Tao, Zhihang Fu, and Jieping Ye. 2024.  INSIDE: LLMs’ internal states retain the power of hallucination detection.  In _The Twelfth International Conference on Learning Representations_.
  * Fleming and Dolan (2012) Stephen M Fleming and Raymond J Dolan. 2012.  The neural basis of metacognitive ability.  _Philosophical Transactions of the Royal Society B: Biological Sciences_ , 367(1594):1338–1349.
  * Gurnee and Tegmark (2023) Wes Gurnee and Max Tegmark. 2023.  Language models represent space and time.  In _The Twelfth International Conference on Learning Representations_.
  * Hart (1965) Julian T Hart. 1965.  Memory and the feeling-of-knowing experience.  _Journal of educational psychology_ , 56(4):208.
  * Ji et al. (2024) Ziwei Ji, Yuzhe Gu, Wenwei Zhang, Chengqi Lyu, Dahua Lin, and Kai Chen. 2024.  Anah: Analytical annotation of hallucinations in large language models.  In _ACL_.
  * Ji et al. (2022) Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Yejin Bang, Andrea Madotto, and Pascale Fung. 2022.  Survey of hallucination in natural language generation.  _ACM Computing Surveys_.
  * Ji et al. (2023a) Ziwei Ji, Zihan Liu, Nayeon Lee, Tiezheng Yu, Bryan Wilie, Min Zeng, and Pascale Fung. 2023a.  RHO: Reducing hallucination in open-domain dialogues with knowledge grounding.  In _Findings of the Association for Computational Linguistics: ACL 2023_ , pages 4504–4522, Toronto, Canada. Association for Computational Linguistics.
  * Ji et al. (2023b) Ziwei Ji, Tiezheng Yu, Yan Xu, Nayeon Lee, Etsuko Ishii, and Pascale Fung. 2023b.  Towards mitigating hallucination in large language models via self-reflection.  _EMNLP Findings_.
  * Jiang et al. (2023) Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al. 2023.  Mistral 7b.  _arXiv preprint arXiv:2310.06825_.
  * Kadavath et al. (2022) Saurav Kadavath, Tom Conerly, Amanda Askell, Tom Henighan, Dawn Drain, Ethan Perez, Nicholas Schiefer, Zac Hatfield-Dodds, Nova DasSarma, Eli Tran-Johnson, et al. 2022.  Language models (mostly) know what they know.  _arXiv preprint arXiv:2207.05221_.
  * Kandpal et al. (2023) Nikhil Kandpal, Haikang Deng, Adam Roberts, Eric Wallace, and Colin Raffel. 2023.  Large language models struggle to learn long-tail knowledge.  In _International Conference on Machine Learning_ , pages 15696–15707. PMLR.
  * Koriat (1997) Asher Koriat. 1997.  Monitoring one’s own knowledge during study: A cue-utilization approach to judgments of learning.  _Journal of experimental psychology: General_ , 126(4):349.
  * Kraskov et al. (2004) Alexander Kraskov, Harald Stögbauer, and Peter Grassberger. 2004.  Estimating mutual information.  _Physical review E_ , 69(6):066138.
  * Lee et al. (2021) Nayeon Lee, Yejin Bang, Andrea Madotto, and Pascale Fung. 2021.  Towards few-shot fact-checking via perplexity.  In _Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 1971–1981.
  * Li et al. (2023) Junyi Li, Xiaoxue Cheng, Wayne Xin Zhao, Jian-Yun Nie, and Ji-Rong Wen. 2023.  Halueval: A large-scale hallucination evaluation benchmark for large language models.  In _Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing_ , pages 6449–6464.
  * Li et al. (2024) Yucheng Li, Frank Guerin, and Chenghua Lin. 2024.  Latesteval: Addressing data contamination in language model evaluation through dynamic and time-sensitive test construction.  In _Proceedings of the AAAI Conference on Artificial Intelligence_ , volume 38, pages 18600–18607.
  * Lin (2004) Chin-Yew Lin. 2004.  Rouge: A package for automatic evaluation of summaries.  In _Text summarization branches out_ , pages 74–81.
  * Liu et al. (2023) Kevin Liu, Stephen Casper, Dylan Hadfield-Menell, and Jacob Andreas. 2023.  Cognitive dissonance: Why do language model outputs disagree with internal representations of truthfulness?  In _Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing_ , pages 4791–4797.
  * Meng et al. (2021) Zhong Meng, Sarangarajan Parthasarathy, Eric Sun, Yashesh Gaur, Naoyuki Kanda, Liang Lu, Xie Chen, Rui Zhao, Jinyu Li, and Yifan Gong. 2021.  Internal language model estimation for domain-adaptive end-to-end speech recognition.  In _2021 IEEE Spoken Language Technology Workshop (SLT)_ , pages 243–250. IEEE.
  * Min et al. (2023) Sewon Min, Kalpesh Krishna, Xinxi Lyu, Mike Lewis, Wen-tau Yih, Pang Koh, Mohit Iyyer, Luke Zettlemoyer, and Hannaneh Hajishirzi. 2023.  Factscore: Fine-grained atomic evaluation of factual precision in long form text generation.  In _Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing_ , pages 12076–12100.
  * Nelson (1990) Thomas O Nelson. 1990.  Metamemory: A theoretical framework and new findings.  In _Psychology of learning and motivation_ , volume 26, pages 125–173. Elsevier.
  * Onoe et al. (2022) Yasumasa Onoe, Michael Zhang, Eunsol Choi, and Greg Durrett. 2022.  Entity cloze by date: What LMs know about unseen entities.  In _Findings of the Association for Computational Linguistics: NAACL 2022_ , pages 693–702, Seattle, United States. Association for Computational Linguistics.
  * Rajpurkar et al. (2018) Pranav Rajpurkar, Robin Jia, and Percy Liang. 2018.  Know what you don’t know: Unanswerable questions for squad.  In _Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)_. Association for Computational Linguistics.
  * Ryu et al. (2018) Seonghan Ryu, Sangjun Koo, Hwanjo Yu, and Gary Geunbae Lee. 2018.  Out-of-domain detection based on generative adversarial network.  In _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing_ , pages 714–718, Brussels, Belgium. Association for Computational Linguistics.
  * Scialom et al. (2021) Thomas Scialom, Paul-Alexis Dray, Sylvain Lamprier, Benjamin Piwowarski, Jacopo Staiano, Alex Wang, and Patrick Gallinari. 2021.  Questeval: Summarization asks for fact-based evaluation.  In _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing_ , pages 6594–6604.
  * Selvaraju et al. (2017) Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. 2017.  Grad-cam: Visual explanations from deep networks via gradient-based localization.  In _Proceedings of the IEEE international conference on computer vision_ , pages 618–626.
  * Su et al. (2024) Weihang Su, Changyue Wang, Qingyao Ai, Yiran Hu, Zhijing Wu, Yujia Zhou, and Yiqun Liu. 2024.  Unsupervised real-time hallucination detection based on the internal states of large language models.  _ACL 2024 Findings_.
  * Tan et al. (2019) Ming Tan, Yang Yu, Haoyu Wang, Dakuo Wang, Saloni Potdar, Shiyu Chang, and Mo Yu. 2019.  Out-of-domain detection for low-resource text classification tasks.  In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_ , pages 3566–3572, Hong Kong, China. Association for Computational Linguistics.
  * Templeton et al. (2024) Adly Templeton, Tom Conerly, Jonathan Marcus, Jack Lindsey, Trenton Bricken, Brian Chen, Adam Pearce, Craig Citro, Emmanuel Ameisen, Andy Jones, Hoagy Cunningham, Nicholas L Turner, Callum McDougall, Monte MacDiarmid, C. Daniel Freeman, Theodore R. Sumers, Edward Rees, Joshua Batson, Adam Jermyn, Shan Carter, Chris Olah, and Tom Henighan. 2024.  Scaling monosemanticity: Extracting interpretable features from claude 3 sonnet.  _Transformer Circuits Thread_.
  * Touvron et al. (2023) Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. 2023.  Llama 2: Open foundation and fine-tuned chat models.  _arXiv preprint arXiv:2307.09288_.
  * Wang et al. (2022a) Jindong Wang, Cuiling Lan, Chang Liu, Yidong Ouyang, Tao Qin, Wang Lu, Yiqiang Chen, Wenjun Zeng, and S Yu Philip. 2022a.  Generalizing to unseen domains: A survey on domain generalization.  _IEEE transactions on knowledge and data engineering_ , 35(8):8052–8072.
  * Wang et al. (2023) Yile Wang, Peng Li, Maosong Sun, and Yang Liu. 2023.  Self-knowledge guided retrieval augmentation for large language models.  In _Findings of the Association for Computational Linguistics: EMNLP 2023_ , pages 10303–10315.
  * Wang et al. (2022b) Yizhong Wang, Swaroop Mishra, Pegah Alipoormolabashi, Yeganeh Kordi, Amirreza Mirzaei, Anjana Arunkumar, Arjun Ashok, Arut Selvan Dhanasekaran, Atharva Naik, David Stap, et al. 2022b.  Super-naturalinstructions:generalization via declarative instructions on 1600+ tasks.  In _EMNLP_.
  * Wu et al. (2024) Jeffrey Wu, Leo Gao, Tom Dupre la Tour, and Henk Tillman. 2024.  Extracting concepts from gpt-4.  https://openai.com/index/extracting-concepts-from-gpt-4/.
  * Xiao and Wang (2021) Yijun Xiao and William Yang Wang. 2021.  On hallucination and predictive uncertainty in conditional language generation.  In _Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume_ , pages 2734–2744.
  * Xiao et al. (2022) Yuxin Xiao, Paul Pu Liang, Umang Bhatt, Willie Neiswanger, Ruslan Salakhutdinov, and Louis-Philippe Morency. 2022.  Uncertainty quantification with pre-trained language models: A large-scale empirical analysis.  In _Findings of the Association for Computational Linguistics: EMNLP 2022_ , pages 7273–7284.
  * Xie et al. (2022) Shuo Xie, Jiahao Qiu, Ankita Pasad, Li Du, Qing Qu, and Hongyuan Mei. 2022.  Hidden state variability of pretrained language models can guide computation reduction for transfer learning.  In _Findings of the Association for Computational Linguistics: EMNLP 2022_ , pages 5750–5768.
  * Xiong et al. (2023) Miao Xiong, Zhiyuan Hu, Xinyang Lu, YIFEI LI, Jie Fu, Junxian He, and Bryan Hooi. 2023.  Can llms express their uncertainty? an empirical evaluation of confidence elicitation in llms.  In _The Twelfth International Conference on Learning Representations_.
  * Yang et al. (2021) Jingkang Yang, Kaiyang Zhou, Yixuan Li, and Ziwei Liu. 2021.  Generalized out-of-distribution detection: A survey.  _arXiv preprint arXiv:2110.11334_.
  * Yeung and Summerfield (2012) Nick Yeung and Christopher Summerfield. 2012.  Metacognition in human decision-making: confidence and error monitoring.  _Philosophical Transactions of the Royal Society B: Biological Sciences_ , 367(1594):1310–1321.
  * Yin et al. (2023a) Xunjian Yin, Baizhou Huang, and Xiaojun Wan. 2023a.  Alcuna: Large language models meet new knowledge.  In _Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing_ , pages 1397–1414.
  * Yin et al. (2023b) Zhangyue Yin, Qiushi Sun, Qipeng Guo, Jiawen Wu, Xipeng Qiu, and Xuanjing Huang. 2023b.  Do large language models know what they don’t know? In _Findings of the Association for Computational Linguistics: ACL 2023_ , pages 8653–8665, Toronto, Canada. Association for Computational Linguistics.
  * Zheng et al. (2020) Yinhe Zheng, Guanyi Chen, and Minlie Huang. 2020.  Out-of-domain detection for natural language understanding in dialog systems.  _IEEE/ACM Transactions on Audio, Speech, and Language Processing_ , 28:1198–1209.
  * Zhou et al. (2023) Yunhua Zhou, Jianqiang Yang, Pengyu Wang, and Xipeng Qiu. 2023.  Two birds one stone: Dynamic ensemble for ood intent classification.  In _Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 10659–10673.

##  Appendix A Dataset

This work uses benchmark Super-Natural Instructions Wang et al. (2022b) which
includes 1,616 diverse NLP datasets covering 76 distinct task types. We select
15 NLG task types and list all datasets included in each NLG task in Tab. D.

##  Appendix B Results and Analysis

#### Separate Metric as Continuous Regression Label

In addition to the comprehensive integration of all metrics (i.e. NLI,
Rouge-L, Questeval) described in § 3.2, we analyze our internal state-based
method’s performances when treating each metric as the label, separately.

We consider three forms of “golden score” for each metric. First, the absolute
values, which serve as the target for regression, with higher scores
indicating fewer hallucinations. We consider the probability of entailment as
the absolute value of the NLI metric. Second, we standardize these absolute
values using the minimum and maximum values from the training dataset to
obtain normalized “golden scores”. Third, we use the relative rankings of
these scores within the training dataset as an alternative regression target.

For the regression task with continuous score predicted, we utilize Root Mean
Squared Error (RMSE) to measure the average difference between the values
predicted by our estimator and the actual values.

As shown in Fig. A1, our method’s prediction performance varies across the
form of the “golden score”. For each metric, the RMSE is the smallest when
predicting absolute value, which indicates that the hidden state performs best
in predicting the absolute value of the metric. Conversely, the highest RMSE
occurs when the model attempts to predict the relative rankings, implying that
predicting the precise ordering of the metrics is more challenging for the
hidden state representation.

Figure A1: RMSE Scores of Internal State-based Estimator with Labels: (a)
Rouge-L (b) NLI (c) QuestEval

#### Estimator Backbone

Instead of Llama MLP, we employ a standard MLP as the backbone of the
estimator. The results in Tab. A1 demonstrate that Llama MLP outperforms the
standard MLP.

Task |  | Internal State
---
F1 | ACC
Dialogue | LlamaMLP | 74.33 | 74.22
MLP | 70.22 | 71.12
QA | LlamaMLP | 82.37 | 82.55
MLP | 81.57 | 81.84
Summarization | LlamaMLP | 88.08 | 88.95
MLP | 87.59 | 87.59
Translation | LlamaMLP | 76.90 | 76.90
MLP | 74.23 | 74.90

Table A1: Automatic Evaluation Results for Different Classifier Backbone

##  Appendix C Implementation Details

The input dimension of our classifier is 4096 and the hidden dimension is
11008, which are aligned with Llama2-7B. We train our classifier with the
following settings and hyper-parameters: the epoch is 10, the batch size is
128, the learning rate is 1e-5, and the AdamW optimizer has a linear
scheduler. Our model is trained on 1 NVIDIA A800 GPU.

##  Appendix D AI Assistants Using

In this paper, we use ChatGPT to improve the writing at grammar level.

Task | No. | Dataset
---|---|---
Table A1 – continued from previous page |  |
Task | No. | Dataset
Code to Text | 4 |  task110 logic2text sentence generation, task129 scan long text generation action command short, task127 scan long text generation action command all, task131 scan long text generation action command long
| Data
---
to
Text
9 |  task1728 web nlg data to text, task1598 nyc long text generation, task1631 openpi answer generation, task677 ollie sentence answer generation, task957 e2e nlg text generation generate, task760 msr sqa long text generation, task1407 dart question generation, task102 commongen sentence generation, task1409 dart text generation
| Dialogue
---
Generation
13 |  task574 air dialogue sentence generation, task361 spolin yesand prompt response classification, task576 curiosity dialogs answer generation, task1603 smcalflow sentence generation, task1714 convai3 sentence generation, task1730 personachat choose next, task565 circa answer generation, task611 mutual multi turn dialogue, task1729 personachat generate next, task1600 smcalflow sentence generation, task639 multi woz user utterance generation, task1590 diplomacy text generation, task360 spolin yesand response generation
| Explanation
---
6 |  task295 semeval 2020 task4 commonsense reasoning, task192 hotpotqa sentence generation, task593 sciq explanation generation, task1369 healthfact sentence generation, task223 quartz explanation generation, task134 winowhy reason generation
| Grammar
---
Error
Correction
2 |  task1415 youtube caption corrections grammar correction, task1557 jfleg answer generation
| Number
---
Conversion
2 |  task1703 ljspeech textmodification, task1704 ljspeech textmodification
| Overlap
---
Extraction
2 |  task039 qasc find overlapping words, task281 points of correspondence
| Paraphrasing
---
12 |  task776 pawsx japanese text modification, task045 miscellaneous sentence paraphrasing, task770 pawsx english text modification, task771 pawsx korean text modification, task774 pawsx german text modification, task177 para-nmt paraphrasing, task466 parsinlu qqp text modification, task775 pawsx chinese text modification, task1614 sick text modify, task773 pawsx spanish text modification, task132 dais text modification, task772 pawsx french text modification
| Program
---
Execution
90 |  task113 count frequency of letter, task1151 swap max min, task509 collate of all alphabetical and numerical elements in list separately, task100 concatenate all elements from index i to j, task096 conala list index subtraction, task365 synthetic remove vowels, task622 replace alphabets in a list by their position in english alphabet, task852 synthetic multiply odds, task1088 array of products, task1405 find median, task637 extract and sort unique digits in a list, task1446 farthest integers, task506 position of all alphabetical elements in list, task378 reverse words of given length, task093 conala normalize lists, task1404 date conversion, task097 conala remove duplicates, task372 synthetic palindrome numbers, task755 find longest substring and replace its sorted lowercase version in both lists, task636 extract and sort unique alphabets in a list, task267 concatenate and reverse all elements from index i to j, task162 count words starting with letter, task159 check frequency of words in sentence pair, task208 combinations of list, task1316 remove duplicates string, task504 count all alphabetical elements in list, task079 conala concat strings, task158 count frequency of words, task507 position of all numerical elements in list, task374 synthetic pos or neg calculation, task1087 two number sum, task163 count words ending with letter, task756 find longert substring and return all unique alphabets in it, task101 reverse and concatenate all elements from index i to j, task1551 every ith element from kth element, task606 sum of all numbers in list between positions i and j, task368 synthetic even or odd calculation, task1150 delete max min, task851 synthetic multiply evens, task377 remove words of given length, task063 first i elements, task064 all elements except first i, task245 check presence in set intersection, task161 count words containing letter, task605 find the longest common subsequence in two lists, task850 synthetic longest palindrome, task157 count vowels and consonants, task373 synthetic round tens place, task206 collatz conjecture, task1443 string to number, task123 conala sort dictionary, task244 count elements in set union, task499 extract and add all numbers from list, task124 conala pair averages, task1444 round power of two, task099 reverse elements between index i and j, task1089 check monotonic array, task1188 count max freq char, task125 conala pair differences, task488 extract all alphabetical elements from list in order, task1542 every ith element from starting, task1194 kth largest element, task371 synthetic product of list, task1406 kth smallest element, task095 conala max absolute value, task1315 find range array, task243 count elements in set intersection, task1331 reverse array, task062 bigbench repeat copy logic, task122 conala list index addition, task091 all elements from index i to j, task369 synthetic remove odds, task497 extract all numbers from list in order, task505 count all numerical elements in list, task205 remove even elements, task1189 check char in string, task1445 closest integers, task094 conala calculate mean, task160 replace letter in a sentence, task1148 maximum ascii value, task098 conala list intersection, task078 all elements except last i, task523 find if numbers or alphabets are more in list, task370 synthetic remove divisible by 3, task367 synthetic remove floats, task1190 add integer to list, task376 reverse order of words, task600 find the longest common substring in two strings, task207 max element lists, task366 synthetic return primes
| Question
---
Answering
207 |  task837 viquiquad answer generation, task701 mmmlu answer generation high school computer science, task1399 obqa answer generation, task075 squad1.1 answer generation, task724 mmmlu answer generation moral scenarios, task666 mmmlu answer generation astronomy, task742 lhoestq answer generation frequency, task1438 doqa cooking answer generation, task863 asdiv multiop question answering, task864 asdiv singleop question answering, task058 multirc question answering, task669 ambigqa answer generation, task704 mmmlu answer generation high school government and politics, task728 mmmlu answer generation professional accounting, task740 lhoestq answer generation quantity, task1293 kilt tasks hotpotqa question answering, task849 pubmedqa answer generation, task1424 mathqa probability, task1625 disfl qa asnwer generation, task858 inquisitive span detection, task723 mmmlu answer generation moral disputes, task083 babi t1 single supporting fact answer generation, task118 semeval 2019 task10 open vocabulary mathematical answer generation, task582 naturalquestion answer generation, task237 iirc answer from subtext answer generation, task714 mmmlu answer generation human sexuality, task444 com qa question paraphrases answer generation, task720 mmmlu answer generation marketing, task332 tellmewhy answer generation, task119 semeval 2019 task10 geometric mathematical answer generation, task310 race classification, task1132 xcsr ur commonsense mc classification, task702 mmmlu answer generation high school european history, task710 mmmlu answer generation high school statistics, task870 msmarco answer generation, task047 miscellaneous answering science questions, task711 mmmlu answer generation high school us history, task1286 openbookqa question answering, task598 cuad answer generation, task685 mmmlu answer generation clinical knowledge, task084 babi t1 single supporting fact identify relevant fact, task1420 mathqa general, task1520 qa srl answer generation, task868 mawps singleop question answering, task768 qed text span selection, task061 ropes answer generation, task041 qasc answer generation, task144 subjqa question answering, task1570 cmrc2018 answer generation, task1610 xquad es answer generation, task164 mcscript question answering text, task703 mmmlu answer generation high school geography, task705 mmmlu answer generation high school macroeconomics, task1131 xcsr es commonsense mc classification, task1130 xcsr vi commonsense mc classification, task750 aqua multiple choice answering, task473 parsinlu mc classification, task385 socialiqa incorrect answer generation, task691 mmmlu answer generation college physics, task719 mmmlu answer generation management, task1327 qa zre answer generation from question, task715 mmmlu answer generation international law, task737 mmmlu answer generation world religions, task010 mctaco answer generation event ordering, task741 lhoestq answer generation place, task028 drop answer generation, task730 mmmlu answer generation professional medicine, task491 mwsc answer generation, task716 mmmlu answer generation jurisprudence, task732 mmmlu answer generation public relations, task735 mmmlu answer generation us foreign policy, task898 freebase qa answer generation, task887 quail answer generation, task024 cosmosqa answer generation, task1140 xcsr pl commonsense mc classification, task225 english language answer generation, task1608 xquad en answer generation, task170 hotpotqa answer generation, task667 mmmlu answer generation business ethics, task699 mmmlu answer generation high school biology, task595 mocha answer generation, task751 svamp subtraction question answering, task1656 gooaq answer generation, task1431 head qa answer generation, task1296 wiki hop question answering, task490 mwsc options generation, task867 mawps multiop question answering, task865 mawps addsub question answering, task1133 xcsr nl commonsense mc classification, task1422 mathqa physics, task1135 xcsr en commonsense mc classification, task054 multirc write correct answer, task1661 super glue classification, task708 mmmlu answer generation high school physics, task1726 mathqa correct answer generation, task664 mmmlu answer generation abstract algebra, task1412 web questions question answering, task002 quoref answer generation, task752 svamp multiplication question answering, task1297 qasc question answering, task692 mmmlu answer generation computer security, task1136 xcsr fr commonsense mc classification, task727 mmmlu answer generation prehistory, task725 mmmlu answer generation nutrition, task104 semeval 2019 task10 closed vocabulary mathematical answer generation, task694 mmmlu answer generation econometrics, task820 protoqa answer generation, task700 mmmlu answer generation high school chemistry, task390 torque text span selection, task1421 mathqa other, task918 coqa answer generation, task309 race answer generation, task247 dream answer generation, task695 mmmlu answer generation electrical engineering, task230 iirc passage classification, task712 mmmlu answer generation high school world history, task731 mmmlu answer generation professional psychology, task596 mocha question generation, task698 mmmlu answer generation global facts, task718 mmmlu answer generation machine learning, task395 persianqa answer generation, task597 cuad answer generation, task339 record answer generation, task835 mathdataset answer generation, task238 iirc answer from passage answer generation, task228 arc answer generation easy, task380 boolq yes no question, task152 tomqa find location easy noise, task754 svamp common-division question answering, task713 mmmlu answer generation human aging, task665 mmmlu answer generation anatomy, task706 mmmlu answer generation high school mathematics, task697 mmmlu answer generation formal logic, task753 svamp addition question answering, task1727 wiqa what is the effect, task1139 xcsr ru commonsense mc classification, task1134 xcsr hi commonsense mc classification, task344 hybridqa answer generation, task165 mcscript question answering commonsense, task1145 xcsr jap commonsense mc classification, task1295 adversarial qa question answering, task239 tweetqa answer generation, task1382 quarel write correct answer, task866 mawps multidiv question answering, task151 tomqa find location easy clean, task591 sciq answer generation, task717 mmmlu answer generation logical fallacies, task726 mmmlu answer generation philosophy, task1419 mathqa gain, task1423 mathqa geometry, task231 iirc link classification, task049 multirc questions needed to answer, task734 mmmlu answer generation sociology, task580 socialiqa answer generation, task736 mmmlu answer generation virology, task729 …
| Translation
---
394 |  task808 pawsx chinese korean translation, task254 spl translation fi en, task1111 ted translation he it, task988 pib translation oriya english, task650 opus100 ar en translation, task763 emea es lt translation, task1648 opus books en-sv translation, task1263 ted translation pl fa, task1020 pib translation telugu oriya, task913 bianet translation, task1060 pib translation urdu malayalam, task1676 xquad-ca translation, task1098 ted translation ja fa, task984 pib translation marathi gujarati, task1086 pib translation marathi english, task789 pawsx french english translation, task1110 ted translation he gl, task1689 qed amara translation, task787 pawsx korean chinese translation, task1071 pib translation malayalam marathi, task548 alt translation en ch, task1373 newscomm translation, task1023 pib translation english hindi, task1271 ted translation fa it, task1274 ted translation pt en, task552 alt translation en bu, task1040 pib translation punjabi oriya, task1323 open subtitles hi en translation, task1058 pib translation urdu english, task1105 ted translation ar gl, task1353 hind encorp translation en hi, task1085 pib translation english marathi, task1103 ted translation es fa, task784 pawsx korean french translation, task811 pawsx chinese german translation, task1365 opustedtalks translation, task1278 ted translation pt he, task1115 alt ja id translation, task538 alt translation bu en, task786 pawsx korean german translation, task805 pawsx german chinese translation, task1692 qed amara translation, task1690 qed amara translation, task655 bible en fa translation, task1256 ted translation pl en, task977 pib translation oriya urdu, task841 para pdt de en translation, task996 pib translation english bengali, task531 europarl es en translation, task452 opus paracrawl en ig translation, task1250 ted translation it ar, task644 refresd translation, task1248 ted translation it ja, task1034 pib translation hindi gujarati, task1225 ted translation ja he, task997 pib translation bengali oriya, task1127 alt ja th translation, task783 pawsx korean english translation, task1031 pib translation bengali telugu, task560 alt translation en entk, task1000 pib translation tamil malayalam, task252 spl translation en tr, task1650 opus books en-fi translation, task654 bible fa en translation, task802 pawsx german korean translation, task1025 pib translation bengali punjabi, task262 spl translation ja en, task785 pawsx korean spanish translation, task530 europarl en es translation, task1232 ted translation ar es, task799 pawsx spanish chinese translation, task1119 alt fil ja translation, task260 spl translation zh en, task1686 menyo20k translation, task448 opus paracrawl en tl translation, task994 pib translation tamil hindi, task1065 pib translation punjabi telugu, task557 alt translation en ba, task1072 pib translation marathi malayalam, task535 alt translation ch en, task762 emea fr sk translation, task1024 pib translation hindi english, task914 bianet translation, task779 pawsx english spanish translation, task547 alt translation entk en, task1128 alt th ja translation, task537 alt translation th en, task1277 ted translation pt ar, task1124 alt ja lo translation, task1514 flores translation entone, task435 alt en ja translation, task425 hindienglish corpora en hi translation, task1371 newscomm translation, task818 pawsx japanese chinese translation, task873 opus xhosanavy translation xhosa eng, task1240 ted translation gl es, task553 alt translation en ma, task1351 opus100 translation gu en, task999 pib translation malayalam tamil, task438 eng guj parallel corpus en gu translation, task541 alt translation kh en, task1329 open subtitles en hi translation, task1102 ted translation es pl, task661 mizan en fa translation, task1259 ted translation pl ar, task424 hindienglish corpora hi en translation, task793 pawsx french chinese translation, task1005 pib translation malayalam punjabi, task1262 ted translation pl it, task1367 opustedtalks translation, task117 spl translation en de, task1237 ted translation he ar, task1122 alt khm ja translation, task1230 ted translation ar en, task790 pawsx french korean translation, task433 alt hi en translation, task253 spl translation en zh, task1037 pib translation telugu urdu, task840 para pdt en es translation, task982 pib translation tamil bengali, task1009 pib translation bengali hindi, task1062 pib translation marathi bengali, task1218 ted translation en ja, task1113 ted translation he fa, task1691 qed amara translation, task1276 ted translation pt es, task1108 ted translation ar fa, task1070 pib translation urdu bengali, task1244 ted translation gl pl, task1239 ted translation gl ja, task1055 pib translation marathi oriya, task794 pawsx french japanese translation, task1004 pib translation malayalam bengali, task1049 pib translation malayalam telugu, task989 pib translation marathi urdu, task450 opus paracrawl so en translation, task815 pawsx japanese french translation, task1066 pib translation telugu punjabi, task777 pawsx english korean translation, task542 alt translation ja en, task830 poleval2019 mt translation, task1655 mkb translation, task313 europarl en sv translation, task1044 pib translation punjabi gujarati, task1038 pib translation urdu telugu, task1057 pib translation english urdu, task1047 pib translation english telugu, task1258 ted translation pl es, task1001 pib translation gujarati urdu, task1063 pib translation gujarati tamil, task1649 opus books en-no translation, task1282 ted translation pt fa, task983 pib translation gujarati marathi, task261 spl translation es en, task439 eng guj parallel corpus gu en translation, task795 pawsx spanish english translation, task1046 pib translation telugu hindi, task1233 ted translation ar he, task1112 ted translation he pl, task663 global voices en fa translation, task662 global voices fa en translation, task1376 newscomm translation, task258 spl translation fa en, task1029 pib translation marathi punjabi, task986 pib translation oriya hindi, task1067 pib translation bengali gujarati, task604 flores translation entosn, task1224 ted translation ja ar, task250 spl translation en ar, task1242 ted translation gl he, task559 alt translation en fi, task1015 pib translation punjabi tamil, task259 spl translation tr en, task1269 ted translation fa he, task807 pawsx chinese english translation, task809 pawsx chinese french translation, task995 pib translation bengali english, task1093 ted translation en fa, task174 spl translation en ja, task1036 pib translation urdu tamil, task1245 ted translation gl fa, task1081 pib translation hindi marathi, task1249 ted translation it es, task842 para pdt cs en translation, task1006 pib translation punjabi malayalam, task1091 ted translation en it, task1022 pib translation malayalam english, task656 quran en fa translation, task1435 ro sts parallel language translation ro to en, task1219 ted translation en es, task782 pawsx english japanese translation, task1054 pib translation urdu hindi, task1035 pib translation tamil urdu, task554 alt translation en la, task993 pib translation hindi tamil, task540 alt translation la en, task556 alt translation en ja, task1324 open subtitles te en…
| Sentence
---
Compression
1 |  task1340 msr text compression compression
| Summarization
---
16 |  task1357 xlsum summary generation, task672 amazon and yelp summarization dataset summarization, task1579 gigaword incorrect summarization, task1658 billsum summarization, task618 amazonreview summary text generation, task522 news editorial summary, task1355 sent comp summarization, task589 amazonfood summary text generation, task1553 cnn dailymail summarization, task1572 samsum summary, task1291 multi news summarization, task668 extreme abstract summarization, task1309 amazonreview summary classification, task1499 dstc3 summarization, task1290 xsum summarization, task511 reddit tifu long text summarization
| Text
---
to
Code
12 |  task210 logic2text structured text generation, task107 splash question to sql, task077 splash explanation to sql, task076 splash correcting sql mistake, task130 scan structured text generation command action long, task869 cfq mcd1 sql to explanation, task212 logic2text classification, task126 scan structured text generation command action all, task211 logic2text classification, task128 scan structured text generation command action short, task868 cfq mcd1 explanation to sql, task956 leetcode 420 strong password check
| Title
---
Generation
19 |  task1540 parsed pdfs summarization, task1561 clickbait new bg summarization, task769 qed summarization, task1342 amazon us reviews title, task1356 xlsum title generation, task569 recipe nlg text generation, task1161 coda19 title generation, task220 rocstories title classification, task219 rocstories title answer generation, task1586 scifact title generation, task602 wikitext-103 answer generation, task1358 xlsum title generation, task1659 title generation, task418 persent title generation, task743 eurlex summarization, task288 gigaword summarization, task500 scruples anecdotes title generation, task619 ohsumed abstract title generation, task510 reddit tifu title summarization
Dataset list for each NLG task from Super-Natural Instructions. |  |