# Evaluating the Deductive Competence of Large Language Models

S.M. Seals123, Valerie L. Shalin34

###### Abstract

The development of highly fluent large language models (LLMs) has prompted
increased interest in assessing their reasoning and problem-solving
capabilities. We investigate whether several LLMs can solve a classic type of
deductive reasoning problem from the cognitive science literature. The tested
LLMs have limited abilities to solve these problems in their conventional
form. We performed follow up experiments to investigate if changes to the
presentation format and content improve model performance. We do find
performance differences between conditions; however, they do not improve
overall performance. Moreover, we find that performance interacts with
presentation format and content in unexpected ways that differ from human
performance. Overall, our results suggest that LLMs have unique reasoning
biases that are only partially predicted from human reasoning performance.

## Introduction and Related Work

111The views expressed are those of the authors and do not necessarily reflect
the official policy or position of the Department of the Air Force, the
Department of Defense, or the U.S. government. Approved for public release,
case number: AFRL-2023-4238

The development and availability of highly fluent large language models (LLMs)
(i.e., (Brown et al. 2020; Devlin et al. 2019; Ouyang et al. 2022; Zhang et
al. 2022)) has increased interest in assessing their reasoning and problem
solving abilities (Bhargava and Ng 2022; Geva, Gupta, and Berant 2020;
Jumelet, Zuidema, and Hupkes 2019; Mitchell 2021; Trinh and Le 2019; Webb,
Holyoak, and Lu 2022). Despite considerable performance improvements on
benchmark tasks, LLMs exhibit mixed results on reasoning tasks. Some research
has suggested that LLMs may have emergent reasoning abilities that enables
better performance levels than those of human subjects (Webb, Holyoak, and Lu
2022). Other research has suggested that LLM reasoning performance is more
mixed and task dependent. Such research has suggested that some tasks, such as
four term analogy problems (Mikolov, Yih, and Zweig 2013) and different
natural language inference tasks (Bowman et al. 2015; Williams, Nangia, and
Bowman 2018), are more straightforward to solve. Other types of reasoning
tasks such as analogy generation (Bhavya, Xiong, and Zhai 2022) and deductive
competence (Dasgupta et al. 2022) are more challenging.

(Dasgupta et al. 2022) has investigated deductive competence in LLMs with
characteristically mixed results. They demonstrated that one LLM, Chinchilla
(Hoffmann et al. 2022), showed content effects on reasoning similar to human
behavior documented in the cognitive science literature. For zero-shot
performance, they found 50% accuracy for what they call realistic problems but
chance accuracy for unrealistic problems. A 5-shot presentation resulted in
some performance improvement for realistic problems, but unrealistic
performance remained low.

In this paper, we extend this previous research in several ways. First, we
investigate the extent to which limited performance may be due to how the task
was formatted. Prior research has demonstrated that overall performance can
vary according to how a particular task is formatted (Gao, Fisch, and Chen
2021; Jiang et al. 2020; Li and Liang 2021; Shin et al. 2020). Research on
analogy generation, (Bhavya, Xiong, and Zhai 2022) demonstrates that
performance depends on the specific prompt that models were given. Thus the
inability of a model to solve one particular format of a task only provides a
lower limit for assessing whether a model can successfully solve that task
(Jiang et al. 2020).

Second, limited performance on this task may be due to how the researchers
employed content familiarity, of direct relevance to the distribution of
content in the training corpus. For familiar content, they cover several
different types of problems including social rules (i.e. If people are
driving, they must have a license) and other relationships (i.e., If the
plants are flowers, they must be fertilized).

However, content familiarity does not fully capture the content benefit seen
in human subjects (Griggs and Cox 1982; Manktelow and Evans 1979). Instead,
previous research indicates that people perform substantially better on
problems that involve social rules than those that do not (Cosmides 1989),
even when the problems contain other types of familiar content (Griggs and Cox
1982).

We expand the research base on the reasoning capabilities of LLMs by: 1)
examining the role of specifically social-rules in reasoning about realistic
content, 2) investigating the role of alternative presentation formats in
deductive reasoning performance, and 3) expanding the set of candidate

LLMs to evaluate potential algorithmic effects. Our results show that social
content does benefit LLM performance, but not to the extent that might be
expected based on training corpus content. While presentation formats do
influence performance, they interact with content in a surprising (non-human)
fashion. These findings are independent of the LLM examined.

## Evaluating Deductive Competence

The Wason selection task is a reasoning task from the cognitive science
literature that evaluates deductive competence (Wason 1968). Participants are
presented with a rule of the form If p, then q and four cards with p status on
one side and q status on the other that correspond to the options $P,\neg
P,Q,$ and $\neg Q$. Participants are asked to determine which card or cards
must be flipped over to validate whether the rule holds for this set of cards.

Inference | Definition
---|---
Modus ponens | If $p$ then $q$, if $q$ therefore $p$
Deny the antecedent | If $p$ then $q$, $\neg p$, therefore $\neg q$
Affirm the consequent | If $p$ then $q$, $q$ therefore $p$
Modus tollens | If $p$ then $q$, $\neg q$ therefore $\neg p$
Table 1: Four conditional inferences in the Wason Task

In the traditional abstract version of the task, participants are given rules
about letters and numbers (i.e., if there is an odd number on one side of the
card, there is a vowel on the other side of the card). The correct response
requires the identification of two cards. Typical human accuracy for these
problems is 10-20% with common errors consistent with confirmation bias. In
contrast, problems that deal with a social rule (i.e., if a person is drinking
beer, they must be at least 21 years old) are easy to solve- most participants
(70%+) select the correct cards (Griggs and Cox 1982).

There are four potential conditional inferences in task: modus ponens, denial
of the antecedent, affirmation of the consequent, and modus tollens (Evans
2013). Of these four inferences, only modus ponens and modus tollens are
logically valid. These inferences and their logical forms are illustrated in
Table 1.

The Wason task makes a good candidate task for evaluating the reasoning
performance of LLMs for several reasons.

First, the task is relatively close to certain language modeling objectives.
While the task involves a reasoning component, it can be formatted as a
completion task, where the objective is to predict the answer given the
problem text. This suggests that prior training, particularly for LLMs with
high numbers of parameters that have been trained on large text corpora,
should provide sufficient information for performing the Wason task. Moreover,
the construction of the task minimizes the potential for confounds that may
artificially inflate performance (Hovy and Yang 2021; Mitchell and Krakauer
2023; Rudinger, May, and Van Durme 2017). Previous work has demonstrated that
high performance on some natural language inference tasks (Bowman et al. 2015;
Williams, Nangia, and Bowman 2018) can be due to exploitable properties of the
training data (Gururangan et al. 2018). The standardized format of the Wason
task allows for the creation of a large number of carefully constructed
examples without the risk that some answers may be easily determined from the
original prompt alone.

Second, the problem examines both straightforward and challenging aspects of
deductive competence. As noted above, a correct answer to a Wason problem
involves two logical processes: modus ponens and modus tollens. Results from
the cognitive science literature indicate that these rules are not equally
difficult- applying modus ponens is considerably easier than apply modus
tollens. The vast majority of people correctly apply modus ponens, even for
difficult problems (i.e., (Wason 1968; Griggs and Cox 1982; Manktelow and
Evans 1979). In contrast, people fail to apply modus tollens, unless the
problem has a particular form of semantic content associated with it.
Similarly, we might expect presentation format to assist LLMs.

Third, the format of the task allows for careful examination of how problem
content influences reasoning performance. Because LLMs are built on co-
occurring content, they should benefit from problems that contain familiar
relationships. This should be especially true for problems where the
relationship between the antecedent and the consequent is highly familiar. For
example, in the rule If a person is driving, they must have a driver’s
license, the antecedent driving a car, and the consequent driver’s license
have a familiar (and commonly occurring) relationship. In comparison, the
antecedent and the consequent in a rule such as If a person is driving, they
must have a book bag do not have the same familiar relationship. While it is
likely that some problems may be more difficult than others (i.e., because
some completions are more probable) we control for this experimentally. We
create sets of problems where both arguments are contain realistic content,
but the relationship between them is unfamiliar (see 2).

Lastly, there is a large body of previous research in cognitive science on
this task. This literature provides a comparison point for evaluating the
performance of LLMs.

## Experiments

In this section, we discuss the task formatting, models, implementation
details, and evaluation metrics associated with our experiments.

### Task Conditions

We evaluate a total of 350 problems, 325 of which we created for this project.
The remaining 25 were drawn from recent work (Dasgupta et al. 2022) and sorted
into our problem categories. To facilitate comparison between content
conditions, our problems are created as matched sets. For each condition
(except the arbitrary condition), we create problems that are nearly identical
in content except for the feature at issue and minor grammatical corrections.
We evaluate three different types of problem content: realistic, shuffled, and
arbitrary. A diagram of the different problems we evaluate is in Figure 1.
Example problems for each condition are in Table 2.

For the realistic condition, we evaluate a total 140 problems. Of these 140,
70 take the form of social rules and 70 take the form of non-social rules. Of
the social rule problems, 35 problems take the form of familiar social rules.
These problems are designed to take the form of social rules governing human
behavior and are designed to be familiar such that they reflect social rules
that are consistent with the real world. The other 35 social rule problems
take the form of unfamiliar social rules. These problems have the form of
social rules but do not contain familiar relationships.

Of the 70 non-social rule problems, 35 are designed to be familiar non-social
rules and 35 are designed to be unfamiliar non-social rules. For the familiar
non-social rule condition, we evaluate 35 problems that are not social rules
and are familiar such that the antecedent and the consequent have a
relationship that is consistent with real-world expectations. For the
unfamiliar non-social rule condition, we evaluate 35 problems that do not take
the form of social rules and are designed such that the antecedent and the
consequent do not have a familiar real-world relationship.

The realistic grouping is intended to capture the same types of realistic
problems that have been used in previous work (Dasgupta et al. 2022). For some
of our analyses, we compare these problems as a group.

Figure 1: Breakdown of the different types of problems we examine.

For the shuffled condition, we evaluate problems where the antecedent and the
consequent are switched. We create shuffled versions of each of the different
categories of realistic problems. The purpose of the shuffled condition is
twofold. First, the creation of shuffled non-social rules allows for the
ability to stress the semantics of plausibility beyond mere co-occurrence.
Second, the creation of shuffled social rules allows evaluating evaluating
sensitivity to the cost-benefit structure of social rules. Standard social
rules typically have an if-then format such that if a person receives a
benefit, then they must pay the cost for that benefit, per (Cosmides 1989). In
comparison, switched social rules occur in past tense- if a person has paid
the cost, then they may receive the benefit. We create shuffled prompts for
both the familiar non-social rule and the familiar social rule conditions. We
make syntactic corrections to make problems grammatically correct.

For the arbitrary condition, we evaluate 70 problems where there is no
particular relationship between the antecedent and the consequent. For
example, in the problem The rule is that if the cards have a type of food then
they must have an outdoor activity, there is no particular pre-supposed
relationship between types of food and outdoor activities.

### Task Format

A complete prompt for each problem consists of the instruction sentence, a
context sentence, the rule, and a list of cards formatted as a multiple choice
question. The instruction sentence was the same for all questions. The context
sentences were consistent with the content type. For the arbitrary problems, a
neutral context sentence was used to prevent a potential length confounds. The
instruction sentence and a sample context sentence are in Table 2.

Instruction sentence: Pick two cards that are required to determine if the
rule is true:
---
Sample Context Sentence: An attendant needs to make sure that customers are
following the rules.
Familiar Social Rule: The rule is that if the customer is over 25 they can
drive a rental car.
Unfamiliar Social Rule: The rule is that if the customer is over 25 they must
be in elementary school.
Familiar Non-social Rule: The rule is that if the equipment is a laptop then
it must have a plastic keyboard.
Unfamiliar Non-social Rule: The rule is that if the equipment is a laptop then
it must have a grass keyboard.
Shuffled Unfamiliar Non-social Rule: The rule is that if the equipment has a
grass keyboard then it must be a laptop.
Arbitrary: The rule is that if the cards have a type of food then they must
have an outdoor activity.
Table 2: Example Problems

### Models

We test four recently released large language models with approximately 7
billion parameters: Guanaco, MPT, BLOOM, and Falcon. Guanaco is a family of
LLMs fine-tuned with QLoRa, a fine tuning approach designed to reduce memory
demands while preserving model performance (Dettmers et al. 2023). We use the
16-bit version. MPT is an open-source family of LLMs released by Mosaic that
are designed to support fast inference (Team 2023; Dao et al. 2022). BLOOM was
trained on a large multilingual corpus (Laurençon et al. 2022) and has a
decoder only transformer architecture (Workshop et al. 2023). Falcon is an
open source LLM designed to support fast inference (Almazrouei et al. 2023;
Dao et al. 2022; Shazeer 2019).

### Implementation Details

We run our experiments on a A100 GPU with 12 vCPUs and 85GB of RAM, running
Debian 10. Experiments were conducted in Python 3.10. A complete list of
libraries is available in the supplementary materials. Total run time for all
experiments was approximately 3 hours.

### Evaluation Metrics

Previous work has proposed various methods to correct for interactions between
the specific form of a prompt and the answer generated by a language model
(Brown et al. 2020; Holtzman et al. 2021; Zhou et al. 2019). We use one of
these metrics, domain conditional PMI, as our scoring metric (Holtzman et al.
2021). Domain conditional PMI measures how much information a particular
instruction domain provides about a particular answer. Formally, a correct
answer is equivalent to

$argmax\frac{P(y_{i}|x)}{P(y_{i}|baseline)}$

where $y_{i}$ is the $i$th answer choice, $x$ is the input prompt, and
$baseline$ is the probability associated with a task-specific premise.
Candidate answers are evaluated independently. Chance performance is $1/6$.

To facilitate comparison between content conditions, our problems are created
as matched sets. We model this shared variance statistically via random
effects terms for sets of stimuli. We use a mixed-effects approach, which
allows for modeling the hierarchical structure of the data (Gelman 2006).
Mixed-effects models are commonly used to analyze linguistic data (i.e.,
(Matuschek et al. 2017; Baayen, Davidson, and Bates 2008)) and allow the
ability to generalize performance beyond a specific set of problems (Clark
1973).

## Results

### Analysis 1 Results

For our first analysis, we evaluate two different crossed factors: social rule
status and content type, using domain conditional PMI as our scoring metric.
For content type, we evaluate arbitrary, shuffled, and realistic rules. The
realistic group contains social rules and non-social rules. We find a
significant beneficial main effect for realistic rules compared to shuffled
rules (odds ratio (OR) = 1.30, 95% CI=[1.05 - 1.62], $z=2.29$, $p<0.05$). We
find support for an interaction between social rule status and content type
(OR = 1.59, 95% CI=[1.26 - 2.02], $Z=3.97,p<0.001$).

Factors for LLM and familiarity do not improve model fit, suggesting that the
overall pattern of results does not significantly differ between LLMs or
between familiar and unfamiliar problems. Overall performance is illustrated
in Figure 2; see Figure 3 for the interaction. Performance remains rather low
overall.

Figure 2: Model performance by content type for arbitrary (AR), shuffled (SH),
and realistic (RE) rules. RE contains both social and non-social rules. We do
not find effects for LLM or familiarity, thus performance is collapsed.
Relative to arbitrary content, most models result in a benefit for realistic
rules, with mixed influences of shuffling.

Follow-up tests for the interaction find significant differences between
realistic non-social rules and realistic social rules (Odds ratio (OR) = 0.33,
95% CI=[0.22 - 0.44], $z=-3.34$, $p<0.001$), between shuffled social rules and
shuffled non-social rules (OR = 2.25, 95% CI=[1.47 - 3.03], $z=2.35$,
$p<0.05$), and between realistic social rules and shuffled social rules (OR =
0.22, 95% CI=[0.15 - 0.29], $z=-4.382$, $p<0.001$). The interaction is plotted
in Figure 3.

Figure 3: Interaction between content type and social rule status for Analysis
1. Content type: arbitrary (AR), shuffled (SH), or realistic (RE) rules. The
realistic category contains social rules and non-social rules. Social rule
status: social rule or non- social rule problems. We do not find effects for
LLM or familiarity, thus performance is collapsed. Figure 4: Performance
across all models for arbitrary (AR), shuffled (SH), and realistic (RE) rules.
The realistic category contains both social rules and non-social rules. We
collapse across LLM and familiarity.

### Analysis 1 Discussion

Initial results from analysis 1 do seem to replicate the general pattern of
results demonstrated for human subjects: performance is better for social rule
problems than non-social rule problems. We also find an effect for switched
social rules such that switched social rules have considerably lower
performance than standard social rule problems. This effect is similar to one
reported for human subjects (Cosmides 1989).

However, we do not replicate the magnitude of the content effect. While humans
find abstract problems quite difficult, they successfully solve social rule
problems 7̃0% of the time (Griggs and Cox 1982). In comparison, the LLMs solve
social rule problems approximately 30% of the time when using domain
conditional PMI scoring. Comparison with traditional highest probability
scoring indicates that this pattern of results is not due to domain
conditional scoring; highest probability scoring produces worse overall
performance and the overall pattern of results is similar.

Additionally, we find an interaction between social rule status and content
type. As observed here, the pattern of human subject responses predicts that
performance should be low for shuffled social rules compared to standard
social rules (Cosmides 1989).However, human subject responses do not predict
the observed differences between shuffled social rules and shuffled non-social
rules. Performance for LLMs is influenced by problem content in a manner that
is not parallel to human subject results.

### Analysis 2 Results

A follow-up experiment examines whether alternative presentation formats may
improve performance. In addition to the standard presentation format
(classic), we investigate three additional formats. In the front condition,
the problems include descriptions of the front of each card. In the back
condition, the problems include descriptions of the hypothetical category of
the item on the back of the card. In the both condition, the problems include
descriptions of both the front and the hypothetical category on the back of
the card. We created alternative formats for all the content types: realistic
social rules, realistic non-social rules, shuffled social rules, and shuffled
non-social rules. We use domain conditional PMI as our scoring metric.

The best fitting statistical model includes an interaction between
presentation format, social rule status, and content type plus a random effect
for item instances. Adding factors for LLM and problem familiarity did not
improve overall model fit, suggesting that performance does not vary
substantially by model or by familiarity. We find significant main effects for
classic versus front presentation format (OR = 1.19, 95% CI: [1.06 - 1.34],
$z=2.79$, $p<0.01$), front versus back presentation format (OR = 1.28, 95% CI:
[1.09 - 1.50], $z=3.17$, $p<0.01$) and back versus both presentation format
(OR = 1.15, 95% CI: [1.00 - 1.31], $z=1.98$, $p<0.05$).

Focusing further on presentation formats, we also find significant
interactions, between the front versus back presentation format and social
rule status (OR = 1.22, 95% CI: [1.06-1.40], $z=2.60$, $p<0.01$) and back
versus both presentation format and social rule status (OR=1.25, 95% CI: [1.09
- 1.44], $z=3.30$, $p<0.001$). For shuffled versus realistic rules, we find
significant interactions between classic and front presentation formats
(OR=1.22, 95% CI: [1.08 - 1.37], $z=2.96$, $p<0.01$) and front versus back
presentation formats (OR=1.17, 95% CI: [1.02 - 1.34], $z=2.04$, $p<0.05$). We
find a significant interaction between social rule status and shuffled versus
realistic rules (OR = 1.32, 95% CI: [1.17 - 1.48], $z=4.66$, $p<0.001$). No
three way interactions were significant ($z$-values $=(1.92,1.88,1.50)$, all
$p>0.05$). See Figures 5 and 6 for plots of the interactions.

Figure 5: Interaction between presentation format (classic, front, back, or
both), content type (shuffled (SH) or realistic (RE)), and social rule status
(social rule or non social rule) broken out by presentation format.

For the interaction between front versus back presentation format and social
rule status, none of the follow up tests were significant (ORs=[1.35, 1.01,
1.41, 1.06], $z$-values = [1.94, 0.06, 1.90, 0.33], all $p>0.05$]). For the
interaction between back versus both and social rule status, none of the
follow up tests were significant (ORs=[1.33, 1.28,1.21, 1.26], $z$-values
$=[1.70,1.25,1.04,1.43]$, all $p>0.05$. For the classic versus front
presentation format and shuffled versus realistic interaction, there was a
significant difference between shuffled and realistic rules for the classic
presentation format (OR = 1.56, 95% CI=[1.29-1.83], $z=2.5$, $p<0.05$). Other
tests were non-significant (ORs=[1.08, 0.97, 1.39], $z$-values = [0.46, -0.16,
1.89], all $p>0.05$). For the front versus back presentation format and
shuffled versus realistic interaction, all follow up tests were non-
significant (ORs=[1.03, 1.08, 1.33, 1.19], $z$-values = [0.13,0.41,1.53,0.95],
all $p>0.05$).

For the interaction between social rule status and shuffled versus realistic
rules, the follow up test between shuffled and realistic non-social rules was
significant (OR = 1.68, 95% CI=[1.39 - 1.96], $z=3.049$ $p<0.001$). The follow
up test between realistic social rules and shuffled social rules was
significant (OR = 1.84, 95% CI=[1.53-2.15], $z=3.551$, $p<0.001$). The other
two tests were not significant.

We conduct a follow-up set of comparisons to examine differences within
content types (shuffled or realistic). For shuffled problems, only the effect
for social rules with the both presentation format was significant (OR = 0.60,
$z=-2.78,p<0.05$). For realistic problems, effects for front non-social rules
(OR = 0.65, $z=-2.40,p<0.05$), back non-social rules (OR = 0.61,
$z=-2.73,p<0.05$), classic social rules (OR = 2.35, $z=5.95,p<0.001$), and
front social rules were significant (OR = 1.61, $z=3.20,p<0.01$).

Figure 6: Interaction between presentation format (classic, front, back, or
both), content type (shuffled (SH) or realistic (RE)), and social rule status
(social rule or non social rule) broken out by content type.

### Analysis 2 Discussion

In general, we do not find performance improvements for different presentation
formats. Results suggest that there are some interactions between presentation
format and problem content; however, most of our follow up tests were not
significant. A follow up analysis of treatment effects broken out by content
type suggests that presentation formats have more of an effect on realistic
rules than shuffled rules. Such interactions are not anticipated in human
performance data.

### Analysis 3 Results

For analysis 3, we examine incorrect answers, scored with domain conditional
PMI, across all conditions. Specifically, we evaluate whether the selected
answer contains any antecedent card varies according to content type,
presentation format, or social rule status.

The best-fitting statistical model contains a main effect for social rule
status and an interaction between presentation format and content type, plus a
random effect for overall item. A term for LLM did not improve model fit. For
content type, we find significant differences between arbitrary versus non-
arbitrary problems (OR = 1.53, 95% CI=[1.23 - 1.90], $z=3.74$, $p<0.001$) and
between shuffled and realistic problems (OR = 1.30, 95% CI=[1.14 - 1.50],
$z=3.59$, $p<0.001$). For presentation formats, we find significant main
effects for classic versus front presentation formats (OR = 1.34, 95% CI=[1.15
- 1.57], $z=3.76$, $p<0.001$), front versus back presentation formats (OR =
1.29, 95% CI=[1.10 - 1.51], $z=3.10$, $p<0.01$), and back versus both
presentation formats (OR = 1.23, 95% CI=[1.07 - 1.41], $z=3.06$, $p<0.01$).
The main effect for social rule status was not significant.

We find significant interactions between arbitrary and non-arbitrary problems
and classic versus front presentation formats (OR=1.28, 95% CI=[1.03 - 1.59],
$z=2.17$, $p<0.05$), shuffled versus realistic problems and classic versus
front presentation formats (OR = 1.29, 95% CI=[1.08 - 1.54], $z=2.74$,
$p<0.01$), shuffled versus realistic problems and front versus back formats
(OR=1.28, 95% CI=[1.05 - 1.56], $z=2.54$, $p<0.05$), shuffled versus realistic
problems and back versus both formats (OR=1.33, 95% CI=[1.14-1.56], $z=3.61$,
$p<0.001$).

For the interaction between arbitrary and non-arbitrary problems and classic
versus front presentation formats, follow up tests between arbitrary classic
versus non-arbitrary classic and non-arbitrary classic versus non-arbitrary
front were significant (ORs=[2.73, 0.57], 95% CIs: [[1.49-5.19], [0.39-0.83]],
$z$-values: [4.09, -3.68], all $p<0.001$). For the interaction between
shuffled and realistic problems and classic versus front presentation formats,
the tests between shuffled classic and realistic classic, shuffled front and
realistic front, and shuffled front and realistic classic were significant
(ORs=[0.33,0.59,0.25], 95% CIs=[[0.17-0.65], [0.34-1.0], [0.13-0.49]],
$z$-values $=[-4.091,-2.35,-5.23]$, $p$-values $<[0.001,0.05,0.001]$). For the
shuffled and realistic problems and front versus back presentation format
interaction, all four follow up tests were significant (shuffled front versus
realistic front: OR = 0.59, 95% CI=[0.34-1.0], $z$ = -2.35, $p<0.05$, shuffled
back versus realistic back: OR = 0.53, 95% CI=[0.30-0.93], $z$ = -2.77,
$p<0.05$), shuffled back versus realistic front: OR = 0.62, 95% CI=[0.36-1.0],
$z$ = -2.09, $p<0.05$, shuffled front versus realistic back: OR = 0.50, 95%
CI=[0.29-0.88], $z$ = -3.03, $p<0.01$). For the shuffled versus realistic and
back versus both interaction, follow up tests between shuffled back versus
realistic back and shuffled both versus realistic back were significant
(ORs=[0.53, 0.58], 95% CIs: [[0.30-0.93], [0.33-1.0]], $z$-values: [-2.77,
-2.41], all $p<0.05$). Interaction plots are in Figure 7.

Figure 7: Evaluation of whether the LLMs select an antecedent card. Content
type: arbitrary (AR), shuffled (SH), and familiar (FM). Presentation formats:
classic, front, back, and both. Social rule status: non-social rule, social
rule. Collapsed over LLM.

### Analysis 3 Discussion

Results from analysis 3 are particularly interesting because there is limited
variance in human performance according to this metric. Regardless of content
type, human subjects select at least one antecedent card (Johnson-Laird,
Legrenzi, and Legrenzi 1972; Griggs and Cox 1982). Even for conditions that
influence antecedent card selection, the overwhelming tendency is for
participants to select the alternative antecedent card (Cosmides 1989). In
contrast, our results suggest that whether LLMs select antecedent cards varies
significantly according to content type and presentation format.

Despite differences in training datasets and tasks and model architecture, we
do not find any effects for LLM. Additionally, the interaction between content
type and presentation format suggests that different presentation formats have
differential effects on antecedent selection for different content types. LLMs
may benefit from different types of formatting depending on the content of the
reasoning task.

## Discussion

We set out to expand the research base on evaluating the reasoning
capabilities of LLMs with a classic experiment contrasting a priori
conditions. We do replicate some effects found in the literature for human
subjects. Performance is higher for familiar problems than arbitrary ones and
social rule have higher performance than non-social rules. However, we do not
replicate the magnitude of the effects; social content does not benefit LLM
performance as might be expected based on training corpus content.

We do find effects for different presentation formats; however, they do not
improve performance. For shuffled problems, the specific presentation format
does not make a difference. For realistic problems, we find presentation type
does make a difference-the classic presentation type has the highest
performance.

However, our systematic, content and format controlled experimentation and
performance measurement has also revealed a number of inexplicable
interactions that appear to be consistent across different LLMs. Given the
literature on human performance, an interaction between social rule status and
content type is expected. However, many of the other interactions are not
expected and are inconsistent with human performance.

We find some evidence that LLMs benefit from different types of presentation
formats, (depending on the specific content of the problem), as might be
expected from popular compensatory prompt engineering efforts. However, it is
not immediately clear what types of information facilitate overall reasoning
performance. This limits the ability to make general predictions about the
conditions under which LLM reasoning is accurate.

In addition to content and presentation interactions, we find that LLMs do not
pick antecedent cards at the same rate that human subjects do. Moreover, this
behavior is influenced by the the task condition- LLMs are less likely to
select antecedent cards for arbitrary and realistic problems than for social
rules.

Overall low performance is particularly surprising for realistic social and
non-social rules as the relationships for solving these problems are plausibly
available in the training data of LLMs. Yet we find that all LLMs diverge from
documented human performance.

Overall performance is also remarkably consistent across models despite
different training data, objectives, and model architectures. Moreover, we
find that the interaction results are also independent of the LLM examined.
Some consistency between LLMs is to be expected, given that LLMs are trained
on human-generated text corpora. However, the commonalities are not consistent
with human reasoning. This suggests a common emergent reasoning bias.

Fine-tuning the models for this task would likely improve task performance. We
did not fine-tune the models for several reasons. First, the Wason task is
intended to be a general task for evaluating reasoning performance. These tap
into general knowledge and a set of reasoning skills that transfers to new
tasks. Thus, the position that networks specifically trained on large
reasoning task corpora is the best way to evaluate the reasoning performance
of models is questionable. This is particularly true given that models often
have high performance on the specific training task and limited performance
related reasoning tasks (Mitchell 2021).

Second, several researchers have proposed that the Wason task can be solved
via linguistic and real world knowledge (Pollard 1982; Tversky and Kahneman
1973; Wason 1983). Human participants achieve high performance on problems
that deal with familiar social rules with no experience with the task, using
prior knowledge. However, the knowledge that this task requires is plausibly
available in the training data for LLMs. The words used in this task are all
common English words. Moreover, many of the relationships between items are
plausibly available in training text, particularly for problems that deal with
familiar social rules. Yet, performance remained quite low.

Previous work has proposed that ideal tasks for evaluating the reasoning of
computational algorithms are those that do not require task-specific training
(Chollet 2019; Mitchell 2021). We concur with this position and suggest that
the Wason task is an ideal task in this regard.

## Conclusion

Despite substantial performance improvements on standard benchmark datasets,
existing LLMs have considerable room for improvement with regards to many
aspects of human intelligence (Lake et al. 2017; Mitchell 2021). In these
experiments, we specifically investigate two of these aspects: generalized
performance on related tasks and generation of answers at the limits of
available knowledge.

Overall, our results replicate some of the same patterns found in the
cognitive science literature. However, we find inexplicable interactions
between problem content and overall performance. Additionally, we find that
LLMs are sensitive to different sets of task criteria than human subjects.
These criteria are not predictable across conditions and suggest areas where
the reasoning of LLMs is not consistent with that of human capability.

## References

  * Almazrouei et al. (2023) Almazrouei, E.; Alobeidli, H.; Alshamsi, A.; Cappelli, A.; Cojocaru, R.; Debbah, M.; Goffinet, E.; Heslow, D.; Launay, J.; Malartic, Q.; Noune, B.; Pannier, B.; and Penedo, G. 2023.  Falcon-40B: an open large language model with state-of-the-art performance.
  * Baayen, Davidson, and Bates (2008) Baayen, R. H.; Davidson, D. J.; and Bates, D. M. 2008.  Mixed-effects modeling with crossed random effects for subjects and items.  _Journal of Memory and Language_ , 59(4): 390–412.  Publisher: Elsevier Inc.
  * Bhargava and Ng (2022) Bhargava, P.; and Ng, V. 2022.  Commonsense Knowledge Reasoning and Generation with Pre-trained Language Models: A Survey.  _Proceedings of the AAAI Conference on Artificial Intelligence_ , 36(11): 12317–12325.  Number: 11.
  * Bhavya, Xiong, and Zhai (2022) Bhavya, B.; Xiong, J.; and Zhai, C. 2022.  Analogy Generation by Prompting Large Language Models: A Case Study of InstructGPT.  ArXiv:2210.04186 [cs].
  * Bowman et al. (2015) Bowman, S. R.; Angeli, G.; Potts, C.; and Manning, C. D. 2015.  A large annotated corpus for learning natural language inference.  In _Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing_ , 632–642. Lisbon, Portugal: Association for Computational Linguistics.
  * Brown et al. (2020) Brown, T.; Mann, B.; Ryder, N.; Subbiah, M.; Kaplan, J. D.; Dhariwal, P.; Neelakantan, A.; Shyam, P.; Sastry, G.; Askell, A.; Agarwal, S.; Herbert-Voss, A.; Krueger, G.; Henighan, T.; Child, R.; Ramesh, A.; Ziegler, D.; Wu, J.; Winter, C.; Hesse, C.; Chen, M.; Sigler, E.; Litwin, M.; Gray, S.; Chess, B.; Clark, J.; Berner, C.; McCandlish, S.; Radford, A.; Sutskever, I.; and Amodei, D. 2020.  Language models are few-shot learners.  In Larochelle, H.; Ranzato, M.; Hadsell, R.; Balcan, M.; and Lin, H., eds., _Advances in neural information processing systems_ , volume 33, 1877–1901. Curran Associates, Inc.
  * Chollet (2019) Chollet, F. 2019.  On the Measure of Intelligence.  ArXiv:1911.01547 [cs].
  * Clark (1973) Clark, H. H. 1973.  The language-as-fixed-effect fallacy: A critique of language statistics in psychological research.  _Journal of Verbal Learning and Verbal Behavior_ , 12(4): 335–359.
  * Cosmides (1989) Cosmides, L. 1989.  The logic of social exchange: Has natural selection shaped how humans reason? Studies with the Wason selection task.  _Cognition_ , 31(3): 187–276.
  * Dao et al. (2022) Dao, T.; Fu, D. Y.; Ermon, S.; Rudra, A.; and Ré, C. 2022.  FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness.  ArXiv:2205.14135 [cs].
  * Dasgupta et al. (2022) Dasgupta, I.; Lampinen, A. K.; Chan, S. C. Y.; Creswell, A.; Kumaran, D.; McClelland, J. L.; and Hill, F. 2022.  Language models show human-like content effects on reasoning.  ArXiv:2207.07051 [cs].
  * Dettmers et al. (2023) Dettmers, T.; Pagnoni, A.; Holtzman, A.; and Zettlemoyer, L. 2023.  QLoRA: Efficient Finetuning of Quantized LLMs.  ArXiv:2305.14314 [cs].
  * Devlin et al. (2019) Devlin, J.; Chang, M. W.; Lee, K.; and Toutanova, K. 2019.  BERT: Pre-training of deep bidirectional transformers for language understanding.  _NAACL HLT 2019 - 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies \- Proceedings of the Conference_ , 1(Mlm): 4171–4186.  ArXiv: 1810.04805 ISBN: 9781950737130.
  * Evans (2013) Evans, J. S. B. 2013.  Reasoning.  In _Oxford Handbook of Cognitive Psychology_ , 635–649.
  * Gao, Fisch, and Chen (2021) Gao, T.; Fisch, A.; and Chen, D. 2021.  Making Pre-trained Language Models Better Few-shot Learners.  In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_ , 3816–3830. Online: Association for Computational Linguistics.
  * Gelman (2006) Gelman, A. 2006.  Multilevel (hierarchical) modeling: What It can and cannot do.  _Technometrics_ , 48(3): 432–435.
  * Geva, Gupta, and Berant (2020) Geva, M.; Gupta, A.; and Berant, J. 2020.  Injecting Numerical Reasoning Skills into Language Models.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , 946–958. Online: Association for Computational Linguistics.
  * Griggs and Cox (1982) Griggs, R. A.; and Cox, J. R. 1982.  The elusive thematic-materials effect in Wason’s selection task.  _British Journal of Psychology_ , 73(3): 407–420.  _eprint: https://onlinelibrary.wiley.com/doi/pdf/10.1111/j.2044-8295.1982.tb01823.x.
  * Gururangan et al. (2018) Gururangan, S.; Swayamdipta, S.; Levy, O.; Schwartz, R.; Bowman, S.; and Smith, N. A. 2018.  Annotation Artifacts in Natural Language Inference Data.  In _Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers)_ , 107–112. New Orleans, Louisiana: Association for Computational Linguistics.
  * Hoffmann et al. (2022) Hoffmann, J.; Borgeaud, S.; Mensch, A.; Buchatskaya, E.; Cai, T.; Rutherford, E.; Casas, D. d. L.; Hendricks, L. A.; Welbl, J.; Clark, A.; Hennigan, T.; Noland, E.; Millican, K.; Driessche, G. v. d.; Damoc, B.; Guy, A.; Osindero, S.; Simonyan, K.; Elsen, E.; Rae, J. W.; Vinyals, O.; and Sifre, L. 2022.  Training Compute-Optimal Large Language Models.  ArXiv:2203.15556 [cs].
  * Holtzman et al. (2021) Holtzman, A.; West, P.; Shwartz, V.; Choi, Y.; and Zettlemoyer, L. 2021.  Surface Form Competition: Why the Highest Probability Answer Isn’t Always Right.  In _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing_ , 7038–7051. Online and Punta Cana, Dominican Republic: Association for Computational Linguistics.
  * Hovy and Yang (2021) Hovy, D.; and Yang, D. 2021.  The Importance of Modeling Social Factors of Language: Theory and Practice.  In _Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , 588–602. Online: Association for Computational Linguistics.
  * Jiang et al. (2020) Jiang, Z.; Xu, F. F.; Araki, J.; and Neubig, G. 2020.  How Can We Know What Language Models Know?  _Transactions of the Association for Computational Linguistics_ , 8: 423–438.
  * Johnson-Laird, Legrenzi, and Legrenzi (1972) Johnson-Laird, P. N.; Legrenzi, P.; and Legrenzi, M. S. 1972.  Reasoning and a sense of reality.  _British journal of psychology_ , 63(3): 395–400.  Publisher: Wiley-Blackwell.
  * Jumelet, Zuidema, and Hupkes (2019) Jumelet, J.; Zuidema, W.; and Hupkes, D. 2019.  Analysing Neural Language Models: Contextual Decomposition Reveals Default Reasoning in Number and Gender Assignment.  In _Proceedings of the 23rd Conference on Computational Natural Language Learning (CoNLL)_ , 1–11. Hong Kong, China: Association for Computational Linguistics.
  * Lake et al. (2017) Lake, B. M.; Ullman, T. D.; Tenenbaum, J. B.; and Gershman, S. J. 2017.  Building machines that learn and think like people.  _Behavioral and Brain Sciences_ , 40: e253.
  * Laurençon et al. (2022) Laurençon, H.; Saulnier, L.; Wang, T.; Akiki, C.; Villanova del Moral, A.; Le Scao, T.; Von Werra, L.; Mou, C.; González Ponferrada, E.; Nguyen, H.; Frohberg, J.; Šaško, M.; Lhoest, Q.; McMillan-Major, A.; Dupont, G.; Biderman, S.; Rogers, A.; Ben allal, L.; De Toni, F.; Pistilli, G.; Nguyen, O.; Nikpoor, S.; Masoud, M.; Colombo, P.; de la Rosa, J.; Villegas, P.; Thrush, T.; Longpre, S.; Nagel, S.; Weber, L.; Muñoz, M.; Zhu, J.; Van Strien, D.; Alyafeai, Z.; Almubarak, K.; Vu, M. C.; Gonzalez-Dios, I.; Soroa, A.; Lo, K.; Dey, M.; Ortiz Suarez, P.; Gokaslan, A.; Bose, S.; Adelani, D.; Phan, L.; Tran, H.; Yu, I.; Pai, S.; Chim, J.; Lepercq, V.; Ilic, S.; Mitchell, M.; Luccioni, S. A.; and Jernite, Y. 2022.  The BigScience ROOTS Corpus: A 1.6TB Composite Multilingual Dataset.  _Advances in Neural Information Processing Systems_ , 35: 31809–31826.
  * Li and Liang (2021) Li, X. L.; and Liang, P. 2021.  Prefix-Tuning: Optimizing Continuous Prompts for Generation.  In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_ , 4582–4597. Online: Association for Computational Linguistics.
  * Manktelow and Evans (1979) Manktelow, K. I.; and Evans, J. S. B. T. 1979.  Facilitation of reasoning by realism: Effect or non-effect?  _British Journal of Psychology_ , 70(4): 477–488.  _eprint: https://onlinelibrary.wiley.com/doi/pdf/10.1111/j.2044-8295.1979.tb01720.x.
  * Matuschek et al. (2017) Matuschek, H.; Kliegl, R.; Vasishth, S.; Baayen, H.; and Bates, D. 2017.  Balancing Type I error and power in linear mixed models.  _Journal of Memory and Language_ , 94: 305–315.  ArXiv: 1511.01864 Publisher: The Authors.
  * Mikolov, Yih, and Zweig (2013) Mikolov, T.; Yih, W.-t.; and Zweig, G. 2013.  Linguistic Regularities in Continuous Space Word Representations.  In _Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , 746–751. Atlanta, Georgia: Association for Computational Linguistics.
  * Mitchell (2021) Mitchell, M. 2021.  Abstraction and analogy-making in artificial intelligence.  _Annals of the New York Academy of Sciences_ , 1505(1): 79–101.  _eprint: https://onlinelibrary.wiley.com/doi/pdf/10.1111/nyas.14619.
  * Mitchell and Krakauer (2023) Mitchell, M.; and Krakauer, D. C. 2023.  The debate over understanding in AI’s large language models.  _Proceedings of the National Academy of Sciences_ , 120(13): e2215907120.
  * Ouyang et al. (2022) Ouyang, L.; Wu, J.; Jiang, X.; Almeida, D.; Wainwright, C. L.; Mishkin, P.; Zhang, C.; Agarwal, S.; Slama, K.; Ray, A.; Schulman, J.; Hilton, J.; Kelton, F.; Miller, L.; Simens, M.; Askell, A.; Welinder, P.; Christiano, P.; Leike, J.; and Lowe, R. 2022.  Training language models to follow instructions with human feedback.  ArXiv:2203.02155 [cs].
  * Pollard (1982) Pollard, P. 1982.  Human reasoning: Some possible effects of availability.  _Cognition_ , 12(1): 65–96.
  * Rudinger, May, and Van Durme (2017) Rudinger, R.; May, C.; and Van Durme, B. 2017.  Social Bias in Elicited Natural Language Inferences.  In _Proceedings of the First ACL Workshop on Ethics in Natural Language Processing_ , 74–79. Valencia, Spain: Association for Computational Linguistics.
  * Shazeer (2019) Shazeer, N. 2019.  Fast Transformer Decoding: One Write-Head is All You Need.  ArXiv:1911.02150 [cs].
  * Shin et al. (2020) Shin, T.; Razeghi, Y.; Logan IV, R. L.; Wallace, E.; and Singh, S. 2020.  AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Prompts.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , 4222–4235. Online: Association for Computational Linguistics.
  * Team (2023) Team, M. N. 2023.  Introducing MPT-7B: A new standard for open-source, commercially usable LLMs.
  * Trinh and Le (2019) Trinh, T. H.; and Le, Q. V. 2019.  A Simple Method for Commonsense Reasoning.  ArXiv:1806.02847 [cs].
  * Tversky and Kahneman (1973) Tversky, A.; and Kahneman, D. 1973.  Availability: A heuristic for judging frequency and probability.  _Cognitive Psychology_ , 5(2): 207–232.
  * Wason (1968) Wason, P. C. 1968.  Reasoning about a Rule.  _Quarterly Journal of Experimental Psychology_ , 20(3): 273–281.  Publisher: SAGE Publications.
  * Wason (1983) Wason, P. C. 1983.  Realism and rationality in the selection task.  In _Thinking and Reasoning (Psychology Revivals)_. Psychology Press.  ISBN 978-1-315-81961-7.  Num Pages: 32.
  * Webb, Holyoak, and Lu (2022) Webb, T.; Holyoak, K. J.; and Lu, H. 2022.  Emergent Analogical Reasoning in Large Language Models.  ArXiv:2212.09196 [cs].
  * Williams, Nangia, and Bowman (2018) Williams, A.; Nangia, N.; and Bowman, S. R. 2018.  A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference.  ArXiv:1704.05426 [cs].
  * Workshop et al. (2023) Workshop, B.; Scao, T. L.; Fan, A.; Akiki, C.; Pavlick, E.; Ilić, S.; Hesslow, D.; Castagné, R.; Luccioni, A. S.; Yvon, F.; Gallé, M.; Tow, J.; Rush, A. M.; Biderman, S.; Webson, A.; Ammanamanchi, P. S.; Wang, T.; Sagot, B.; Muennighoff, N.; del Moral, A. V.; Ruwase, O.; Bawden, R.; Bekman, S.; McMillan-Major, A.; Beltagy, I.; Nguyen, H.; Saulnier, L.; Tan, S.; Suarez, P. O.; Sanh, V.; Laurençon, H.; Jernite, Y.; Launay, J.; Mitchell, M.; Raffel, C.; Gokaslan, A.; Simhi, A.; Soroa, A.; Aji, A. F.; Alfassy, A.; Rogers, A.; Nitzav, A. K.; Xu, C.; Mou, C.; Emezue, C.; Klamm, C.; Leong, C.; van Strien, D.; Adelani, D. I.; Radev, D.; Ponferrada, E. G.; Levkovizh, E.; Kim, E.; Natan, E. B.; De Toni, F.; Dupont, G.; Kruszewski, G.; Pistilli, G.; Elsahar, H.; Benyamina, H.; Tran, H.; Yu, I.; Abdulmumin, I.; Johnson, I.; Gonzalez-Dios, I.; de la Rosa, J.; Chim, J.; Dodge, J.; Zhu, J.; Chang, J.; Frohberg, J.; Tobing, J.; Bhattacharjee, J.; Almubarak, K.; Chen, K.; Lo, K.; Von Werra, L.; Weber, L.; Phan, L.; allal, L. B.; Tanguy, L.; Dey, M.; Muñoz, M. R.; Masoud, M.; Grandury, M.; Šaško, M.; Huang, M.; Coavoux, M.; Singh, M.; Jiang, M. T.-J.; Vu, M. C.; Jauhar, M. A.; Ghaleb, M.; Subramani, N.; Kassner, N.; Khamis, N.; Nguyen, O.; Espejel, O.; de Gibert, O.; Villegas, P.; Henderson, P.; Colombo, P.; Amuok, P.; Lhoest, Q.; Harliman, R.; Bommasani, R.; López, R. L.; Ribeiro, R.; Osei, S.; Pyysalo, S.; Nagel, S.; Bose, S.; Muhammad, S. H.; Sharma, S.; Longpre, S.; Nikpoor, S.; Silberberg, S.; Pai, S.; Zink, S.; Torrent, T. T.; Schick, T.; Thrush, T.; Danchev, V.; Nikoulina, V.; Laippala, V.; Lepercq, V.; Prabhu, V.; Alyafeai, Z.; Talat, Z.; Raja, A.; Heinzerling, B.; Si, C.; Taşar, D. E.; Salesky, E.; Mielke, S. J.; Lee, W. Y.; Sharma, A.; Santilli, A.; Chaffin, A.; Stiegler, A.; Datta, D.; Szczechla, E.; Chhablani, G.; Wang, H.; Pandey, H.; Strobelt, H.; Fries, J. A.; Rozen, J.; Gao, L.; Sutawika, L.; Bari, M. S.; Al-shaibani, M. S.; Manica, M.; Nayak, N.; Teehan, R.; Albanie, S.; Shen, S.; Ben-David, S.; Bach, S. H.; Kim, T.; Bers, T.; Fevry, T.; Neeraj, T.; Thakker, U.; Raunak, V.; Tang, X.; Yong, Z.-X.; Sun, Z.; Brody, S.; Uri, Y.; Tojarieh, H.; Roberts, A.; Chung, H. W.; Tae, J.; Phang, J.; Press, O.; Li, C.; Narayanan, D.; Bourfoune, H.; Casper, J.; Rasley, J.; Ryabinin, M.; Mishra, M.; Zhang, M.; Shoeybi, M.; Peyrounette, M.; Patry, N.; Tazi, N.; Sanseviero, O.; von Platen, P.; Cornette, P.; Lavallée, P. F.; Lacroix, R.; Rajbhandari, S.; Gandhi, S.; Smith, S.; Requena, S.; Patil, S.; Dettmers, T.; Baruwa, A.; Singh, A.; Cheveleva, A.; Ligozat, A.-L.; Subramonian, A.; Névéol, A.; Lovering, C.; Garrette, D.; Tunuguntla, D.; Reiter, E.; Taktasheva, E.; Voloshina, E.; Bogdanov, E.; Winata, G. I.; Schoelkopf, H.; Kalo, J.-C.; Novikova, J.; Forde, J. Z.; Clive, J.; Kasai, J.; Kawamura, K.; Hazan, L.; Carpuat, M.; Clinciu, M.; Kim, N.; Cheng, N.; Serikov, O.; Antverg, O.; van der Wal, O.; Zhang, R.; Zhang, R.; Gehrmann, S.; Mirkin, S.; Pais, S.; Shavrina, T.; Scialom, T.; Yun, T.; Limisiewicz, T.; Rieser, V.; Protasov, V.; Mikhailov, V.; Pruksachatkun, Y.; Belinkov, Y.; Bamberger, Z.; Kasner, Z.; Rueda, A.; Pestana, A.; Feizpour, A.; Khan, A.; Faranak, A.; Santos, A.; Hevia, A.; Unldreaj, A.; Aghagol, A.; Abdollahi, A.; Tammour, A.; HajiHosseini, A.; Behroozi, B.; Ajibade, B.; Saxena, B.; Ferrandis, C. M.; McDuff, D.; Contractor, D.; Lansky, D.; David, D.; Kiela, D.; Nguyen, D. A.; Tan, E.; Baylor, E.; Ozoani, E.; Mirza, F.; Ononiwu, F.; Rezanejad, H.; Jones, H.; Bhattacharya, I.; Solaiman, I.; Sedenko, I.; Nejadgholi, I.; Passmore, J.; Seltzer, J.; Sanz, J. B.; Dutra, L.; Samagaio, M.; Elbadri, M.; Mieskes, M.; Gerchick, M.; Akinlolu, M.; McKenna, M.; Qiu, M.; Ghauri, M.; Burynok, M.; Abrar, N.; Rajani, N.; Elkott, N.; Fahmy, N.; Samuel, O.; An, R.; Kromann, R.; Hao, R.; Alizadeh, S.; Shubber, S.; Wang, S.; Roy, S.; Viguier, S.; Le, T.; Oyebade, T.; Le, T.; Yang, Y.; Nguyen, Z.; Kashyap, A. R.; Palasciano, A.; Callahan, A.; Shukla, A.; Miranda-Escalada, A.; Singh, A.; Beilharz, B.; Wang, B.; Brito, C.; Zhou, C.; Jain, C.; Xu, C.; Fourrier, C.; Periñán, D. L.; Molano, D.; Yu, D.; Manjavacas, E.; Barth, F.; Fuhrimann, F.; Altay, G.; Bayrak, G.; Burns, G.; Vrabec, H. U.; Bello, I.; Dash, I.; Kang, J.; Giorgi, J.; Golde, J.; Posada, J. D.; Sivaraman, K. R.; Bulchandani, L.; Liu, L.; Shinzato, L.; de Bykhovetz, M. H.; Takeuchi, M.; Pàmies, M.; Castillo, M. A.; Nezhurina, M.; Sänger, M.; Samwald, M.; Cullan, M.; Weinberg, M.; De Wolf, M.; Mihaljcic, M.; Liu, M.; Freidank, M.; Kang, M.; Seelam, N.; Dahlberg, N.; Broad, N. M.; Muellner, N.; Fung, P.; Haller, P.; Chandrasekhar, R.; Eisenberg, R.; Martin, R.; Canalli, R.; Su, R.; Su, R.; Cahyawijaya, S.; Garda, S.; Deshmukh, S. S.; Mishra, S.; Kiblawi, S.; Ott, S.; Sang-aroonsiri, S.; Kumar, S.; Schweter, S.; Bharati, S.; Laud, T.; Gigant, T.; Kainuma, T.; Kusa, W.; Labrak, Y.; Bajaj, Y. S.; Venkatraman, Y.; Xu, Y.; Xu, Y.; Xu, Y.; Tan, Z.; Xie, Z.; Ye, Z.; Bras, M.; Belkada, Y.; and Wolf, T. 2023.  BLOOM: A 176B-Parameter Open-Access Multilingual Language Model.  ArXiv:2211.05100 [cs].
  * Zhang et al. (2022) Zhang, S.; Roller, S.; Goyal, N.; Artetxe, M.; Chen, M.; Chen, S.; Dewan, C.; Diab, M.; Li, X.; Lin, X. V.; Mihaylov, T.; Ott, M.; Shleifer, S.; Shuster, K.; Simig, D.; Koura, P. S.; Sridhar, A.; Wang, T.; and Zettlemoyer, L. 2022.  OPT: Open Pre-trained Transformer Language Models.  ArXiv:2205.01068 [cs].
  * Zhou et al. (2019) Zhou, K.; Zhang, K.; Wu, Y.; Liu, S.; and Yu, J. 2019.  Unsupervised Context Rewriting for Open Domain Conversation.  In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_ , 1834–1844. Hong Kong, China: Association for Computational Linguistics.