# Measurement in the Age of LLMs:
An Application to Ideological Scaling

Sean O’Hagan  Department of Statistics, University of Chicago  Aaron Schein
Department of Statistics, University of Chicago

###### Abstract

Much of social science is centered around terms like “ideology” or “power”,
which generally elude precise definition, and whose contextual meanings are
trapped in surrounding language. This paper explores the use of large language
models (LLMs) to flexibly navigate the conceptual clutter inherent to social
scientific measurement tasks. We rely on LLMs’ remarkable linguistic fluency
to elicit ideological scales of both legislators and text, which accord
closely to established methods and our own judgement. A key aspect of our
approach is that we elicit such scores directly, instructing the LLM to
furnish numeric scores itself. This approach affords a great deal of
flexibility, which we showcase through a variety of different case studies.
Our results suggest that LLMs can be used to characterize highly subtle and
diffuse manifestations of political ideology in text.

Last update: August 28, 2024111This work was presented at the 4th
International Conference of Social Computing in Beijing, China, September
2023, the New Directions in Analyzing Text as Data (TADA) meeting in Amherst,
MA, USA, November 2023, and the NeurIPS workshop titled “I Can’t Believe It’s
Not Better!” Failure Modes in the Age of Foundation Models in New Orleans, LA,
December 2023.

##  1 Introduction

Social science pertains to complex constructs denoted by terms like
“ideology”, “power”, or “culture”, whose meanings are contextual and generally
hard to pin down precisely. Although slippery and subjective, such terms are
routinely used in conversation, among experts and non-experts alike, without
anyone (except the occasional pedant) demanding formal definitions from their
conversational partners. It is indeed a feature of natural language discourse
that such terms are assumed to wear many hats, and that conversational
partners must cooperate to arrive at mutually intelligible meanings. This
cooperation is typically tacit, and speakers coordinate on a shared meaning by
offering examples, reformulations, and engaging generally in an elaborative
process that builds upon shared context and common knowledge. In so doing
however, speakers inevitably introduce new terms requiring their own processes
of disambiguation. Such processes are therefore never quite finished, and
merely halted for expediency. Complex constructs referred to by terms like
“ideology” are thus in a sense trapped in language, and their captivity in
linguistic form has long been the chief bugaboo of social scientists, stymying
attempts to match the physical and natural sciences’ success at deploying
mathematical and data-intensive approaches.

Generations of social scientists have battled this chief bugaboo, fighting to
extract precise truths about complex constructs despite the inherent
imprecision wrought by constructs’ linguistic captivity. The widely-adopted
(though crude) distinction between qualitative versus quantitative social
science can be understood as distinguishing two different fronts in this
battle. On the one hand, qual researchers accept the inherently linguistic
nature of the battlefield, and do not attempt to fight it elsewhere. On the
other, quant researchers seek to airlift constructs out of the linguistic
realm and into the numeric realm wherein their properties and relations can be
precisely defined and analyzed using formal mathematical and scalable
computational methods. The latter’s attempt to liberate constructs from their
linguistic captivity is known variably by such terms as quantification,
operationalization, or measurement.

There is perhaps no better example of such battles than the one social
scientists have long fought to extract precise truths about “ideology”, as
Gerring [13] says:

> > Few concepts in the social science lexicon have occasioned so much
> discussion, so much disagreement, and so much selfconscious discussion of
> the disagreement, as “ideology”. Condemned time and again for its semantic
> excesses, for its bulbous unclarity, the concept of ideology remains,
> against all odds, a central term of social science discourse.

The lack of consensus on what social science means by “ideology” (Gerring
catalogues literally hundreds of different definitions) stands in glaring
opposition to the sheer volume of literature on the matter. One might think
that such conceptual clutter suggests the absence of any real underlying
phenomena, and point to famous examples in the physical and natural sciences
of theories like geocentrism or phlogiston that became increasingly muddled,
prior to being ultimately falsified, in attempting to explain accumulating
anomalies [24]. However, the clutter with “ideology”, we argue, simply
reflects the term’s linguistic promiscuity. As with all such terms,
cooperative speakers can arrive at a roughly mutually-intelligible meaning
without ever agreeing on a formal definition. The construct’s meaning to
speakers is thus largely driven by its conversational context. Constructs that
manifest in many settings prompt many conversations, each of which contribute
a slightly different meaning to the broader discourse, and thus exacerbate the
clutter.

Quants have traditionally dealt with this clutter by operationalizing
“ideology” simply as a scale, along which individuals (or other entities) can
be placed, in a manner predictive of some relevant quantified behavior. The
pioneering work of Poole and Rosenthal [38] sought to measure the “ideology”
of US legislators using roll call data, where a data point
$y_{ij}\in\\{\textsc{Yea},\textsc{Nay},\textsc{Abstain}\\}$ records how
legislator $i$ voted on bill $j$. Their statistical model for such data posits
an ideal point $x_{i}\in\mathbb{R}^{2}$ in some 2-dimensional latent space,
and assumes the probability that legislator $i$ votes Yea on bill $j$ is
proportional to the distance $|x_{i}-z_{j}|$, where $z_{j}\in\mathbb{R}^{2}$
is the position of the bill in that same space. Both $x_{i}$ and $z_{j}$ are
unknown but estimated by fitting the model to the observed roll call data.
Doing so often yields fitted values of $x_{i}$ that accord reasonably well
with expert intuition about the relative positions of legislators along two
axes that are reasonably easy to label post-hoc.

Many approaches for learning “ideological” scales from quantified behavioral
data have since been proposed. The original model of Poole and Rosenthal [38]
was first extended to allow for time-varying ideal points [39] and then to
allow bills to have varying degrees of relevance [40]. These approaches were
generalized to measure “ideological” scales from roll call vote data of
lawmakers more broadly, including Supreme Court Justices [26]. Alternate
statistical frameworks were quickly introduced, like item-response theory [9]
or Bayesian methodology [20, 4], as have models that incorporate the actual
text of bills [14, 25, 17]. Beyond roll call votes, researchers have
introduced approaches to measure lawmakers’ ideal points from campaign
contributions [7, 8], from their Twitter network [5], and from the text of
their floor speeches and Tweets [42], among other quantified behaviors. The
literature is vast, and these represent only an illustrative subset.

Underlying these approaches is often the attitude that what “ideology” means
is so hopelessly mired in conflicting verbiage, then rather than contributing
more of our own, we ought to “let the data speak”. However, critics reply that
the data is only permitted to speak through the tin can telephone of a
stylized statistical model, whose assumptions encode a meaning of their own,
just more obscurely and less intentionally. This then simply adds to all the
clutter; again, Gerring [13]:

> > When concepts are defined “backwards”—by working out methods of
> measurement first—it may only complicate the task of social science inquiry
> since this encourages a rather facile approach to definition (slapping a
> term onto a set of empirical findings without much consideration of the
> term’s previous definition, or alternative labels that might be more
> appropriate). […] My hunch is that behavioralists may have more to learn
> from a close examination of the term—including its usage in other corners
> […]

At the heart of this debate is the reliance of data-intensive methodology on
quantification—i.e., the act of recording of data as numbers—and the (related)
problem of measurement—i.e., the problem of representing (linguistically-
captive) constructs by numeric systems [18, 1]. Quantification naturally
begets measurement: if observations are numerically encoded, inference on
their basis must necessarily relate numbers to the broader constructs of
study.

However, constructs can be measured without data being quantified. Although,
“quantitative”, “data-driven”, and “empirical” are often used interchangeably,
there is nothing inherently non-empirical about non-numeric data, like notably
the verbiage that qual researchers pore over. Indeed, language data is data,
and inference on the basis of that is empirical. For instance, a researcher
interested in the conservative slant of talk radio [28], might seek to measure
that slant over time by reading 10,000 radio show transcripts herself and
assigning each a score from -1 to 1 on a conservative–liberal axis based on
her own background knowledge and qualitative assessment of the text. Such an
approach, known as content analysis [23], would yield a measurement of
“conservative slant” without requiring that the data (her transcripts) be
first quantified. We contrast this with the “text as data” [16] approaches
that have become widespread in the data-intensive social sciences. Such
approaches typically begin the same, by collecting relevant language data, but
then proceed to form inferences on the basis of only word counts or other
derived quantities, which preserve a mere (numeric) semblance of the
information conveyed in the text.

So, why quantify? We offer a rather boring answer: statistical inference has
historically been the only form of inference that could be faithfully
implemented on computers and scaled to large data sets. Thus, incorporating
large data sets into the scientific process, as any good empiricist should
want, has necessitated the translation of scientific reasoning into
statistical reasoning, which operates fundamentally on numbers. By this view,
the qual-quant divide is an accident of history. Both are data-driven—but in
the battle to extract truths about linguistically-captive constructs, qual
researchers have prioritized linguistic inference, of which only humans have
been traditionally capable.

The perspective of this paper is that the recent emergence of large language
models (LLMs) opens a third front in this battle that is meaningfully distinct
from either qual or quant. An autoregressive language model is a conditional
probability distribution over the next word given context $P(w_{n+1}\mid
w_{n},w_{n-1},\dots)$. Sampling from a trained LM sequentially, adding each
sampled token into the context for the next, generates utterances that look,
in some sense, like those in its training corpus. Although LMs are ostensibly
dumb—e.g., “stochastic parrots” [6]—their recent “large” variants—which have
an enormous number (e.g., billions) of parameters and are trained an enormous
numbers (e.g., trillions) of word tokens—exhibit an undeniably remarkable
degree of linguistic fluency. It is our view that machines with this
unprecedented degree of fluency allow social scientists, for the first time,
to study complex constructs with the neatness and scalability of the
traditional quantitative approach without the attendant requirement that
constructs be ripped prematurely from their natural linguistic habitat.

This paper explores the use of LLMs to synthesize a variety of interpretive
judgements relating to “ideology”. Specifically, we elicit ideological scales
from LLMs directly, through conversations like the one below (with gpt-4 on
November 2, 2023):

Prompt: On a Left–Right ideological scale from -10 to +10, how would you rate
Bernie Sanders? GPT-4: […] Sanders could be seen as quite far to the left,
potentially somewhere between -6 to -8, given his advocacy for policies such
as Medicare for All, the Green New Deal […] Prompt: How about Elizabeth
Warren? Only respond with the score. GPT-4: Around -5 to -6.

Any self-respecting methodologist might cringe at such a seemingly unmoored
approach to measurement222We originally began this work to prove to a
colleague, who off-handedly proposed it, that this was a terrible idea., and
reject it outright for any number of reasons. Among them might be concerns
about factuality—e.g.,

  1. 1.

LLMs often “hallucinate” false claims [49, 48],

  2. 2.

LLMs are politically slanted [41, 31, 32, 27].

Or, concerns related to a lack of transparency—e.g.,

  1. 3.

LLMs’ are inscrutable “black boxes”,

  2. 4.

LLMs’ training data is (typically) not open source,

  3. 5.

LLMs’ training data may be “contaminated” [33].

One might worry about numerical pathologies that elicited scores might
exhibit, since they no longer come from a well-understood statistical
model—e.g.,

  1. 6.

LLMs’ provided scores may be on different scales,

  2. 7.

LLMs exhibit flawed numeric reasoning [36, 50].

Or, one might worry about a lack of reliability—e.g.,

  1. 8.

LLMs’ responses are inherently stochastic,

  2. 9.

LLMs can be “distracted” by spurious details [35],

  3. 10.

LLMs are not robust to minor prompt changes [44].

More fundamentally, all of these concerns (and others), when taken together,
make it difficult for the researcher to even know what meaningful constructs
(if any) they can measure with such an approach, and whether those
measurements are reliable or have any validity.

Figure 1: Kernel density estimates of the distribution of GPT-elicited ideal
points for Donald Trump and Bernie Sanders. Two prompts are used: one which
leaves open the interpretation of “authoritarian-libertarian”, and one which
elaborates informally.

While clear-eyed about the perils, our experience suggests LLMs nevertheless
hold promise for social scientific measurement. The purpose of this paper is
to convey that perspective, by reporting ways we have learned to mitigate
against some of the pitfalls listed above, and by sharing a variety of both
remarkable and curious results that showcase this approach and illuminate
paths for future development.

We begin in § 2 with a set of experiments using LLMs to measure the ideal
points of Senators in the $114^{\textrm{th}}$ U.S. Congress. Many well-
understood scaling methods have been previously applied to this task, whose
results we regard loosely as a form of “ground truth”. We find that the ideal
points directly elicited from gpt-3.5-turbo are highly correlated with those
of DW-NOMINATE [40], which are based on roll-call votes, CFScores [7], which
are based on campaign contributions, and text-based ideal points (TBIP) [42],
which are based on word counts of Senators’ Tweets and speeches.

These results are consistent with the findings in a recent preprint by Wu et
al. [47], whose concurrent work also develops an LLM-based approach for
ideological scaling, and finds that its ideal points are highly correlated
with established scaling methods. Their approach restricts prompts to elicit
only pairwise comparisons—e.g., “Which Senator is more liberal: Bernie Sanders
or Elizabeth Warren?”—and then fits a Bradley-Terry model to infer the scaling
of legislators implied by the set of pairwise judgements. Their approach is
complementary to ours—by restricting to pairwise comparisons, theirs trades
flexibility and scalability for reliability.

The main benefit of the direct approach is how flexibly and easily it can be
tailored to different measurement tasks. In § 3, we tailor this approach for
scaling the ideology of text in a variety of different settings. First, we
explore an LLM scale the tweets of Senators in the $114^{\textrm{th}}$
Congress using conversations like the one below (which we had with gpt-4 via
ChatGPT on November 3, 2023):

Prompt: On a Left–Right ideological scale from -10 to +10, how would you rate
the following tweet: The devastating effects of the climate crisis are here.
Now is the time to act boldly. GPT-4: […] On a Left-Right scale, where -10
represents the far left (advocating for systemic change and possibly anti-
capitalist sentiments that associate environmental degradation with
capitalism) […] this tweet could be rated around a -3 to -5 […] Prompt: How
about this one: It’s official: I’m running for reelection! GPT-4: […] It’s a
neutral announcement that could be made by any political figure […] it would
not be assigned a rating as it does not provide any information about policy
positions […]

We draw attention to two notable aspects of this example. First, our prompt
does not provide any elaboration of what we mean by “a Left-Right ideological
scale”. However, in its response, gpt-4 offers a specific interpretation of
“far left” which is appropriately tailored to the context. (We include its
full response in the appendix, which also offers an appropriately tailored
interpretation of the “far right”.) We find that LLMs gracefully handle the
inherent ambiguity of terms like “left” or “right” and produce text that is
consistent with the sensible in-context inferences a knowledgeable human
interlocutor might make. This echoes the perspective of Andreas [2] that LLMs
perform inference over the prompter’s intent. We find that elaborating
informally often leads to a sharp decrease in elicited scores’ variability
which is qualitatively consistent with a reduction in uncertainty about our
intended meaning of ambiguous terms—an example of this is given in Figure 1.

Second, in the conversation above, gpt-4’s response does not assign a score to
the second tweet which it says is non-ideological. We find that both
gpt-3.5-turbo and gpt-4 are remarkably good at detecting very subtle
manifestations of political ideology in text, or lack thereof, and provide a
few demonstrations of that in § 3, which includes detecting a far-right “dog
whistle” [30], characterizing a character’s political beliefs from scant
details in a constructed vignette, and the measurement of Donald Trump’s
political ambitions over time via the ideological content of his tweets.

As a final series of experiments, we connect the text scaling approach back to
the ideal points by having gpt-3.5-turbo scale a subsets of tweets of the
Senators in the $114^{\textrm{th}}$ Congress. We do so by first asking whether
a tweet is ideological or not, and eliciting an ideological score, if so. For
Democratic Senators, we find a close correspondence between the ideal points
gpt-3.5-turbo assigns and the scores it gives to their (ideological) tweets.
For Republican Senators we do not find such a correspondence. This echoes the
finding of Vafa et al. [42], based only on word counts, that tweets are more
effective for scaling Democrats than Republicans. We then add to our prompts
an encouragement for the LLM to first “show its work” [22] before providing
its score. With this adjustment, a correspondence for Republican Senators
emerges, suggesting that manifestations of ideology in Republicans’ tweets are
subtle but still present.

##  2 Eliciting Ideal Points of Legislators

This section reports on our attempts to directly elicit an ideological scaling
of Senators in the U.S. $114^{\textrm{th}}$ Congress from OpenAI’s
gpt-3.5-turbo. The prompts we use have the following the form:

Prompt: Assign a numerical ideal point score to each member of the following
list of United States politicians. This score should be a single real number
that reflects their ideological position on a left/right spectrum. Use
negative values for the left, and positive values for the right. […]

Self-anchoring. The prompt here does not specify any constraints on the
numeric scale, other than 0 being its midpoint. One might consider providing
anchors—e.g., “Bernie Sanders is -9 and Mitch McConnell is +8”—whose values
are prescribed or otherwise selected from some “ground truth”. This would
correspond to a few-shot task. To avoid arbitrary prescriptions, we instead
embrace this zero-shot task and rely on the autoregressive nature of
gpt-3.5-turbo to anchor itself by asking it to score all the Senators in a
single generated response. In so doing, the score it assigns to one Senator
will be anchored by the scores it assigned to previous Senators in the
provided list.

Marginalizing over order. We may worry that self-anchoring will make the
elicited scores sensitive to the order in which Senators appear in the
provided list. To mitigate against a lack of robustness to ordering, we
randomly sample permutations of the list and repeat the same prompt for each.
For each Senator $i$, we thus elicit ideal points $x_{i,1},\dots,x_{i,S}$ for
all $S$ sampled permutations, and report the average
$x_{i}=\tfrac{1}{S}\sum_{s=1}^{S}x_{i,s}$ as their elicited ideal point, while
maintaining the full distribution to characterize uncertainty.

Normalization. Although we expect self-anchoring to ensure the elicited scores
have a meaningful ordering (e.g., Bernie Sanders is left of Mitch McConnell),
we do not expect it to promote any specific scale. We found that slight
variations in the prompt (e.g., ordering) could lead to changes in the scale
of elicited scores—e.g., sometimes ranging from -5 to +5, or -10 to
+10—although much less than we initially expected. To ensure a standardized
scale, we $Z$-normalize all scores from a single run, which thus ensures that
all batches of scores across runs (e.g., using different permutations) are on
comparable numeric scales.

Spectrum. The above prompt leaves the meaning of “left/right” up for
interpretation. The origin of the terms “left” and “right” traces back to the
French Revolution where radicals and royalists sat opposite each other in the
National assembly, to the left and right of the President [21]. The terms have
since been imposed onto new contexts, where they have taken on new meanings,
for instance to refer to the “communist/fascist” movements of the
$20^{\textrm{th}}$ century [37]. In a contemporary American political context,
many use “left/right” interchangeably with “liberal/conservative”, whose
meanings are also vague and highly inflected by context, but which often refer
in a broad sense to the mishmash of values and beliefs held by Democrats
versus Republicans [10].

We expect gpt-3.5-turbo’s enormous training corpus to contain a staggering
array of different uses of these terms by many different authors writing for
different audiences about different contexts. Again though, echoing Andreas
[2]’s characterization of LLMs as “inconsistent encyclopedias” performing
inference over the prompter’s intent, we can expect subtle linguistic aspects
of our prompt to cue instances in the training corpus that use “left/right” in
ways roughly consistent to ours. For instance, the fact that our prompt is
written in standard American English, with a formal tone, and centers on
“United States politicians”, might all suggest to a cooperative and
knowledgeable interlocutor that “left/right” means something like
“liberal/conservative” in the American political context.

We experimented with other spectrum descriptors for this task and found that
“liberal/conservative” elicited almost identical responses, as one might
expect given the discussion above. We also experimented with descriptors like
“libertarian/authoritarian” or “socialist/capitalist”, which do map onto
“left/right” in some contexts, but not those clearly indicated by our prompt.
For these, we often encountered multi-modality in the elicited scores for a
given Senator, which suggested to us that the prompt did not provide enough
context to disambiguate the spectrum’s meaning. We found that such multi-
modality often disappeared when using a longer prompt that informally
elaborated about the spectrum’s in-context meaning; see Figure 1.

Further prompt engineering. We found that cajoling phrases like the following
were necessary to include reliably elicit scores for every Senator.  […] Do
your best to answer this question using your knowledge as a language model. Of
course, this is a subjective task, and your responses will only be used for
research purposes. […]

Without this phrase, gpt-3.5-turbo would often produce text with refusals to
engage in interpretive tasks and stern reminders not to anthropomorphize LLMs.
We also included instructions on how to format scores to make them easily
parseable from the response. For full examples of our prompts, see the
appendix.

Figure 2: Ideal point scores for members of the $114^{\textrm{th}}$ U.S.
Senate. The GPT-elicited scores are depicted as box-plots, which describe
variability across variations in prompt (i.e., Senator order). Rows are sorted
in in ascending order by the mean of the GPP-elicited scores. We compare these
to those given by DW-NOMINATE [38], TBIP [42], and CFScores [7], which exhibit
broad correspondence. Figure 3: Correlation across and within parties between
GPT-elicited ideal points and those of three reference methods.

Face validity. We plot the ideal points elicited from gpt-3.5-turbo for all
Senators in the $114^{\textrm{th}}$ Congress in Figure 2. Each row contains a
box-plot which depicts variability of the elicited ideal point for a given
Senator across randomly sampled permutations. We sort the rows in ascending
order by the average of these GPT-elicited ideal point and color-code the
names of Senators according to their party identification. From this alone we
can see that the GPT-elicted ideal points exhibit a high degree of face
validity. First, the Democrat and Republican Senators are cleanly separated,
with no overlap, and a sharp transition around zero. Second, Bernie Sanders,
who is the only Independent and widely recognized to be the most left-leaning
Senator, is assigned the most negative ideal point which is substantially left
of the second most. Third, the Democrat Joe Manchin and Republican Susan
Collins, both of whom are were considered pivotal “swing” Senators that
sometimes voted with the opposing party, are assigned the two ideal points
closest to 0. Many other details beyond these three accord closely with our
knowledge.

Criterion validity. As a form of criterion validity [1], we compared the GPT-
elicited ideal points to three well-established models for ideal point
estimation. We consider DW-NOMINATE [40], which is based on roll call vote
data, CFScores [7], which is based on campaign contributions, and text-based
ideal points (TBIP) [42], which is based on word counts derived from Senators’
floor speeches. We regard the ideal points measured by these methods as a form
of “ground truth”, though only loosely—we expect a valid LLM-based measurement
approach to not differ too wildly, but still differ in certain meaningful
ways, just as we expect these three methods to differ meaningfully among
themselves.

We overlay the ideal points of these reference methods on Figure 2. Visual
inspection suggests broad correspondence between the GPT-elicted ideal points
and those of the three references. We analyze this correspondence in Figure 3,
where we give a breakdown of the correlation both within and across Democrats
and Republicans. The total correlation of GPT-elicted ideal points with all
three is remarkably high—0.97 with DW-NOMINATE, 0.88 with TBIP, and 0.94
CFscores—though this is substantially driven by how all methods easily
separate Democrats and Republicans. The within-party correlation with DW-
NOMINATE is also high for both parties—0.83 and 0.72—while lower for the other
two, with the lowest being TBIP within Republicans. We surmise this relates to
Vafa et al. [42]’s finding that word counts were generally less effective for
scaling Republicans, who tend to use fewer ideologically-inflected words than
Democrats.

Discussion. It is remarkable to us that an LLM can be used so directly, with
simple natural language instruction, to elicit an ideological scaling of
legislators that has such face validity and correspondence with other
measures. Our findings are consistent with those of Wu et al. [47] who also
find a similar correspondence, and persuasively show that LLMs are not merely
“parroting” known ideal-point measures. Their approach is more controlled,
eliciting only comparisons between pairs of legislators, and then fitting a
Bradley-Terry model to infer a scaling implied by the binary outcomes of those
comparisons. As such, their approach avoids the issues of scale, anchoring,
and sensitivity to ordering that our direct approach must contend with. The
trade-off for this reliability is then scalability—while the direct
elicitation approach requires $\mathcal{O}(n)$ queries for $n$ legislators,
the pairwise one requires $\mathcal{O}(n^{2})$. By extension, the direct
approach is more flexible, allowing for a much broader range measurements to
be elicited. We explore this flexibility in the next section, where we deploy
direct elicitation to scale the ideology of text in a variety of settings.

##  3 Eliciting Ideological Scales of Text

One might choose to be unimpressed that LLMs encode an association between the
names of Senators and their ideological leanings. There are undoubtedly many
contexts in the training corpus that directly associate politician’s names
with their political beliefs, and recapitulating such associations might not
require a great degree of synthesis. People too are often capable of making
vague associations about political leaders—e.g., Bernie Sanders is a
socialist—without possessing a sophisticated or well-informed viewpoint.

Ideological slant manifests in language in often subtle and complex ways that
require substantial background knowledge and linguistic inference to detect.
Recent work in the past two years has begun investigating LLMs’ abilities in
handling ideological text, like detecting “dog-whistles” [30], rewriting
ideological text [19], and responding coherently to political surveys [3],
among others. Most notably, a recent preprint by Wu et al. [46] extends the
pairwise elicitation approach of Wu et al. [47] to scale tweets, by eliciting
judgements from an LLM whether one of two provided tweets is more liberal.
This section contributes to this emerging area by showcasing a variety of ways
that a direct elicitation can be flexibly used to scale text and by providing
a close-up exploration of the results.

Figure 4: The rate of Trump’s tweets being classified by gpt-3.5-turbo as
“ideological” from 2009 to 2017. A dramatic increase in his rate of
ideological tweeting immediately following the 2011 White House
Correspondents’ dinner is consistent with a long-alleged story about his
motivation to run for President in 2016.

###  3.1 Measuring Trump’s political ambitions

A natural first question is whether LLMs can reliably distinguish between
ideological and non-ideological text. We study this by directly eliciting
binary judgements from gpt-3.5-turbo—e.g.  Prompt: Is the following text
ideological? Answer yes or no. […] Our anecdotal experience is that
gpt-3.5-turbo’s judgements are remarkably consistent with our own, even in
very subtle instances, and we highly encourage the (skeptical) reader to
experiment themselves.

This subsection attempts to validate our impression more systematically, by
checking for a form of construct validity [1]. In lieu of “ground truth”
labels, which would be labor-intensive and time-consuming to produce, we
instead check whether patterns in gpt-3.5-turbo’s judgements make sense in
relation to other known patterns.

We use a large sample of Donald Trump’s tweets from 2009–2017, and ask
gpt-3.5-turbo in a few-shot manner whether each one is ideological or not. We
then examine whether temporal patterns in Trump’s rate of ideological tweeting
(as judged by gpt-3.5-turbo) correspond meaningfully to well-known events in
Trump’s life. The time window begins substantially before 2015, when Trump was
not yet in politics (officially), and extends after he took office as
President in 2017. It is therefore reasonable to assume that his rate of
ideological tweeting changed substantially over that time frame, as he
transitioned from business to politics, and that a good method for scaling
text will show that.

Figure 4 depicts the rate at which gpt-3.5-turbo classified Trump’s tweets as
ideological over the time frame 2009–2017. To obtain a smooth rate function
from the time-stamped binary judgements, we fit a Gaussian process with a
squared exponential kernel and logit link function; its mean is visualized.

The dramatic increase in his rate of ideological tweeting around April 2011
seems to corroborate a widely-alleged story about Trump’s decision to run for
President. The story, as presented in the PBS documentary “The Choice 2016”
[11], was that:

> > Donald Trump was the focus of President Obama’s jokes at the 2011 White
> House Correspondents’ Dinner. It was there that Trump resolved to run for
> president […]

This story was also presented in the New Yorker [15], National Review [29],
and TIME [12], among several other major media outlets. (We also note that
some journalists have cast doubt on the story [43], and Trump has denied it.)
We are unaware of other measurements of Trump’s political fixation over time,
but this one does appear to show that the time of the dinner was indeed a
turning point. Other aspects of Figure 4 also appear to have high validity,
like peaks at the times he announced his candidacy and was nominated for
President.

###  3.2 Detecting a neo-Nazi dog-whistle

Figure 5: Results on gpt-3.5-turbo’s sensitivity to the dog whistle “1488”,
with and without zero-shot CoT, when scoring the ideology of a vaguely
political but otherwise innocuous tweet.

Although aggregate trends (and anecdotal experience) suggest that
gpt-3.5-turbo’s classification of whether text is ideological has high
validity, this leaves open the question of whether it can reliably place
ideological text on meaningful scales. This and the next sub-section zoom in
on specific hand-crafted examples to build intuition for that question.

Inspired by Mendelsohn et al. [30], we construct a simple experiment to check
if gpt-3.5-turbo picks up on a neo-Nazi “dog whistle”—i.e., the number “1488”
[34]—in otherwise innocuous contexts. We construct three tweet-like examples
that are vaguely political. Each is written from the perspective of an
organizer who is mobilizing their friends into some form of political activity
(e.g., attending a march), and includes reference to some number $n$—e.g.,

Prompt: Consider the following tweet: I made a playlist of n songs for the
march today, check it out Assign a numerical score which reflects its
ideological position on a left/right spectrum […]

We try this for $n\in\\{1480,\dots,1490\\}$, and check whether the scores
assigned by gpt-3.5-turbo are substantially different at $n=1488$, which would
be consistent with a coded neo-Nazi message.

We visualize these results as light-blue bars in Figure 5, where pronounced
spikes at $n=1488$ are clearly evident for the first two examples. While the
first two are consistent with a right-leaning or moderate interpretation for
all numbers, the last is judged by gpt-3.5-turbo to be left-leaning, which we
surmise further obscures the neo-Nazi interpretation.

We further experiment by including an encouragement to “think step-by-step”
[22]—i.e.,  Prompt: Please first give your analysis of the tweet and reasoning
for the score […] This kind of prompt engineering is known as “chain-of-
thought” (CoT) prompting [45], which has been shown to improve performance in
a broad array of tasks, including by Wu et al. [46] for scaling the ideology
of tweets. More specifically, this is a form of zero-shot CoT, as the LLM is
instructed to provide reasoning and answer within the same response.

The results using zero-shot CoT are visualized in dark-blue in Figure 5, where
we see that the gpt-3.5-turbo is in fact capable of detecting the dog whistle,
even in the harder example. More details on the prompts are provided in the
appendix.

###  3.3 Dinner table politics

Here we continue with another hand-crafted example that encodes ideology in a
much more subtle and diffuse manner. Unlike in the previous example, where
knowledge of a single ideologically-inflected word “1488” is sufficient, here
we construct examples where no individual words or phrases are inherently
ideological, but we (the authors) can still form judgements about ideological
slant.

In this case, all prompts begin the same by describing a dinner table
conversation between Kyle and his parents:

Prompt: Consider the following scenario: Kyle is home from college, eating
dinner with his parents. He mentions that on his walk home, he noticed the new
community gardens where people were planting vegetables and herbs. […]

From here the scenario continues on one of 24 possible paths. In each, one of
Kyle’s two parents mutters one of three possible exasperated phrases, and then
offers one of four explanations. One such path is:  His father rolls his eyes
and mutters “these people”. Kyle asks his father what he means. His father
responds: “I go to work everyday, and this is what I see.” From this vignette,
we (the authors) feel the father is behaving in a way commonly stereotyped as
conservative. We have this feeling despite being unable to point to any
specific ideologically-inflected words or other unambiguous indicators. The
father’s vague appeal to “work ethic”, a commonly cited conservative value, is
perhaps the strongest, but is still rather subtle.

Compare this to another possible path:  His mother rolls her eyes and mutters
“these people”. Kyle asks his mother what she means. His mother responds: “A
bunch of folks used to sleep in that lot, and now they can’t because someone
wanted to grow heirloom tomatoes.” From this alternate vignette, we (the
authors) interpret the mother to be expressing strong empathy for the
homeless, as well as a disdain for the yuppies who displaced them, all of
which suggest generally left-leaning political beliefs. Again though, there
are no simple words or phrases that are clearly ideologically inflected, and
our interpretation requires a combination of linguistic inference and world
knowledge.

After describing one of these 24 scenarios, we complete the prompt by
instructing the LLM to describe of the parent’s political beliefs and assign
them an ideal point:

[…] Use your knowledge as a language model to interpret the father’s political
beliefs […] After detailing your reasoning, provide a score that describes his
beliefs on a Left/Right scale […]

In the appendix, we provide a description of all 24 scenarios. We visualize
the scores assigned to each by gpt-4 in Figure 6, which accord remarkably
closely to our (the authors’) impressions. In the appendix, we also include
those assigned by gpt-3.5-turbo, which were accordant, but less so.

###  3.4 Scaling the tweets of the $114^{\textrm{th}}$ Senate

In this last case study, we connect back to § 2 and investigate whether the
scores that gpt-3.5-turbo assigns to a Senator’s tweets agrees overall with
the ideal point it assigned them based only their name. The absence of such a
correspondence would suggest an unsophisticated synthesis of the training
corpus, and undermine the validity of both sets of scores.

Data set. The authors Vafa et al. [42] shared with us a data set of tweets of
U.S. legislators from 2009–2017 which is open-source and similar in scope to
the one used in their paper on text-based ideal points. For each Senator in
the $114^{\textrm{th}}$ Congress, we sub-sampled 250 of their tweets uniformly
at random.

Basic prompt. For each tweet in the sub-sampled corpus, we directly elicit an
ideological score from gpt-3.5-turbo using prompts of the same flavor as those
in § 3.2 and 3.3, which ask it to assign a numeric score to the provided tweet
using negative numbers for “left” and positive for “right”.

Dynamic self-anchoring with human feedback. As with eliciting ideal points (§
2), the prompts do not contrain a scale, beyond specifying zero as its
midpoint. In the previous section we had the LLM score all Senators at once
and relied on its autoregressive nature to ensure that the scores were
comparable; we referred to this strategy as self-anchoring.

Figure 6: Politics at the dinner table, as elicited from gpt-4. Each subplot
corresponds a parent, each column an exasperated phrases, and each row an
explanations, combinations of which subtly alter the interpretation of the
parent’s ideological slant.

We cannot rely on the same strategy in this instance, since fitting all tweets
into a single prompt would greatly exceed gpt-3.5-turbo’s context limit.
Instead, for each tweet, we include as context the five previous tweets and
scores it assigned to them, a strategy we call dynamic self-anchoring. In
addition, we also include four hand-selected tweets and corresponding scores,
a form of human feedback. More details are contained in the appendix.

Eliciting binary judgements. We do not expect all text tweeted by a Senator to
be intrinsically ideological. We therefore also experiment with first
eliciting a binary judgement about whether a tweet is ideological or not (like
in § 3.1), and proceeding to elicit a numeric score only if gpt-3.5-turbo’s
response is yes.

Figure 7: Kernel density estimates of ideal point scalings of sets of tweets
by selected senators in the 114th U.S. Senate. Tweets are scored using three
different prompts: one that scores each tweet, one that scores only tweets
that are deemed ideological, and one that scores tweets that are deemed
ideological after explaining its reasoning for giving the score.

Zero-shot chain-of-thought. As in § 3.3, we also experiment with encouraging
the LLM to “think step-by-step” before providing responses, both to the binary
judgements and to the numeric scores.

Overall approaches. We test three overall approaches for eliciting ideological
scores, which all use dynamic self-anchoring with human feedback, but differ
in their adoption of the other two strategies:

  1. 1.

“All tweets”, which does not use any initial binary judgements, or zero-shot
CoT in its prompt,

  2. 2.

“Ideological tweets”, which introduces initial binary judgements to only score
ideological tweets,

  3. 3.

“Ideological tweets with zero-CoT”, which adds zero-shot CoT into both
elicitation stages.

Figure 8: Correlations between average scores of Senators’ tweets and GPT-
elicited ideal points, as well as with those from the three reference methods.
See appendix for within-Democrats results.

Results. We first examine the correlation between the average score assigned
to all of a Senator’s tweets, with the GPT-elicited ideal point assigned to
them in § 2. We examine this correlation for all three text-scoring methods,
and also include the correlations with the three reference methods DW-
NOMINATE, TBIP, and CFScores. We further break down correlations across and
within parties.

Figure 8 visualizes the overall scorrelations and those within Republicans
(see appendix for those within Democrats). The overall correlations are
strikingly high, including 0.94 between the third text-scaling method and the
GPT-elicited ideal points, which suggests a remarkable degree of internal
consistency.

The correlations are smaller when broken down by party, which is true between
all methods. Intriguingly, among Republicans, the correlations with GPT-
elicted ideal points for the three methods are 0.29, 0.0 and 0.47,
respectively, which suggests that asking first whether a tweet is ideological
or not actually degrades performance if not coupled with zero-shot CoT. This
seems consistent with overall picture that ideological slant in Republicans’
tweets is diffuse and requires more sophisticated linguistic inference to
surface.

To build more intuition, we zoom in on the results for five Senators in Figure
7, which we found to be generally representative. In each sub-plot, we
visualize kernel-density estimates of the distributions of tweet scores by the
three different methods. We compare these to the GPT-elicited ideal point and
the one assigned by TBIP, which appera as vertical lines.

From these plots we can clearly see the effect of the initial binary
judgements—while the grey densities are all centered at 0, the light blue and
dark blue densities are shifted toward the ideal points. We also see that the
light blue densities, corresponding to the second method without zero-shot
CoT, are peaked around the ideal points for the three left-most Senators, but
not for the two right-most. Meanwhile the dark blue densities, which
correspond to the third method with zero-shot CoT, appear to concentrate in
the area of the ideal points for all Senators. These results seem to
corroborate the aggregate correlations, that the most sophisticated tweet-
scaling method has a high degree of internal consistency with the GPT-elicited
ideal points.

##  4 Conclusion

Traditional quantitative approaches to measuring ideology rely on the analysis
of tangible behavioral data, like roll-call votes or campaign contributions.
These methodologies encode a definition of “ideology” that is constrained by
the scope of the behavioral data and obscured by modeling assumptions. LLMs
may provide social scientists, for the first time, with a scalable approach
for studying complex constructs without ripping them from their natural
linguistic habitat. This paper contributes to the recent area of LLM-based
measurement by exploring direct elicitation, which affords flexibility to the
researcher by regarding the LLM as a cooperative and knowledgeable
interlocutor. Our results suggest that such an approach can measure highly
subtle and diffuse manifestations of ideology in a variety of settings, and
that continued efforts to develop it are hold great promise.

## Acknowledgment

We thank Shun Kamaya, Brandon Stewart, Keyon Vafa, and Victor Veitch for
helpful discussions and useful feedback.

## References

##

  * Adcock and Collier [2001] Robert Adcock and David Collier.  Measurement Validity: A Shared Standard for Qualitative and Quantitative Research.  _American Political Science Review_ , 95(3):529–546, September 2001.
  * Andreas [2022] Jacob Andreas.  Language Models as Agent Models, December 2022.  URL http://arxiv.org/abs/2212.01681.  arXiv:2212.01681 [cs].
  * Argyle et al. [2022] Lisa P. Argyle, Ethan C. Busby, Nancy Fulda, Joshua Gubler, Christopher Rytting, and David Wingate.  Out of One, Many: Using Language Models to Simulate Human Samples.  In _Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 819–862, 2022.  doi: 10.18653/v1/2022.acl-long.60.  URL http://arxiv.org/abs/2209.06899.  arXiv:2209.06899 [cs].
  * Bafumi et al. [2005] Joseph Bafumi, Andrew Gelman, David K. Park, and Noah Kaplan.  Practical Issues in Implementing and Understanding Bayesian Ideal Point Estimation.  _Political Analysis_ , 13(2):171–187, 2005.  ISSN 1047-1987, 1476-4989.  doi: 10.1093/pan/mpi010.  URL https://www.cambridge.org/core/product/identifier/S1047198700001054/type/journal_article.
  * Barberá [2015] Pablo Barberá.  Birds of the Same Feather Tweet Together: Bayesian Ideal Point Estimation Using Twitter Data.  _Political Analysis_ , 23(1):76–91, 2015.  ISSN 1047-1987, 1476-4989.  doi: 10.1093/pan/mpu011.  URL https://www.cambridge.org/core/product/identifier/S104719870001161X/type/journal_article.
  * Bender et al. [2021] Emily M Bender, Timnit Gebru, Angelina McMillan-Major, and Shmargaret Shmitchell.  On the dangers of stochastic parrots: Can language models be too big?  In _Proceedings of the 2021 ACM conference on fairness, accountability, and transparency_ , pages 610–623, 2021.
  * Bonica [2014] Adam Bonica.  Mapping the Ideological Marketplace.  _American Journal of Political Science_ , 58(2), April 2014\.
  * Bonica [2019] Adam Bonica.  Are Donation-Based Measures of Ideology Valid Predictors of Individual-Level Policy Preferences?  _The Journal of Politics_ , 81(1):327–333, January 2019.  ISSN 0022-3816, 1468-2508.  doi: 10.1086/700722.  URL https://www.journals.uchicago.edu/doi/10.1086/700722.
  * Clinton et al. [2004] Joshua Clinton, Simon Jackman, and Douglas Rivers.  The Statistical Analysis of Roll Call Data.  _American Political Science Review_ , 98(2):355–370, May 2004.  ISSN 0003-0554, 1537-5943.  doi: 10.1017/S0003055404001194.  URL https://www.cambridge.org/core/product/identifier/S0003055404001194/type/journal_article.
  * Converse [2006] Philip E. Converse.  The nature of belief systems in mass publics (1964).  _Critical Review_ , 18(1-3):1–74, January 2006\.  ISSN 0891-3811, 1933-8007.  doi: 10.1080/08913810608443650.  URL http://www.tandfonline.com/doi/abs/10.1080/08913810608443650.
  * Frontline [2016] PBS Frontline.  The choice 2016.  Frontline, 2016.  URL https://www.pbs.org/wgbh/frontline/documentary/the-choice-2016/? PBS.
  * Gajanan [2017] Mahita Gajanan.  A history of president trump being trolled at the white house correspondents’ dinner.  _TIME_ , April 2017.  URL https://time.com/4756751/donald-trump-white-house-correspondents-dinner/.
  * Gerring [1997] John Gerring.  Ideology: A Definitional Analysis.  _Political Research Quarterly_ , 50(4):957–994, December 1997.
  * Gerrish and Blei [2012] Sean M. Gerrish and David M. Blei.  The Issue-Adjusted Ideal Point Model, September 2012.  URL http://arxiv.org/abs/1209.6004.  arXiv:1209.6004 [cs, stat].
  * Gopnik [2015] Adam Gopnik.  A history of president trump being trolled at the white house correspondents’ dinner.  _The New Yorker_ , September 2015.  URL https://www.newyorker.com/news/daily-comment/trump-and-obama-a-night-to-remember.
  * Grimmer and Stewart [2013] Justin Grimmer and Brandon M Stewart.  Text as data: The promise and pitfalls of automatic content analysis methods for political texts.  _Political analysis_ , 21(3):267–297, 2013.
  * Gu et al. [2014] Yupeng Gu, Yizhou Sun, Ning Jiang, Bingyu Wang, and Ting Chen.  Topic-factorized ideal point estimation model for legislative voting network.  In _Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining_ , pages 183–192, New York New York USA, August 2014. ACM.  ISBN 978-1-4503-2956-9.  doi: 10.1145/2623330.2623700.  URL https://dl.acm.org/doi/10.1145/2623330.2623700.
  * Hand [1996] D. J. Hand.  Statistics and the Theory of Measurement.  _Journal of the Royal Statistical Society. Series A (Statistics in Society)_ , 159(3):445, 1996.  ISSN 09641998.  doi: 10.2307/2983326.  URL https://www.jstor.org/stable/10.2307/2983326?origin=crossref.
  * Hoyle et al. [2023] Alexander Hoyle, Rupak Sarkar, Pranav Goel, and Philip Resnik.  Making the Implicit Explicit: Implicit Content as a First Class Citizen in NLP, May 2023.  URL http://arxiv.org/abs/2305.14583.  arXiv:2305.14583 [cs].
  * Jackman [2001] Simon Jackman.  Multidimensional Analysis of Roll Call Data via Bayesian Simulation: Identification, Estimation, Inference, and Model Checking.  _Political Analysis_ , 9(3):227–241, January 2001\.  ISSN 1047-1987, 1476-4989.  doi: 10.1093/polana/9.3.227.  URL https://www.cambridge.org/core/product/identifier/S1047198700003818/type/journal_article.
  * Kinder and Kalmoe [2017] Donald R. Kinder and Nathan P. Kalmoe.  _Neither Liberal Nor Conservative: Ideological Innocence in the American Public_.  Chicago studies in American politics. The University of Chicago Press, Chicago ; London, 2017.  ISBN 978-0-226-45231-9 978-0-226-45245-6.
  * Kojima et al. [2022] Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa.  Large Language Models are Zero-Shot Reasoners.  2022\.
  * Krippendorff [2018] Klaus Krippendorff.  _Content analysis: An introduction to its methodology_.  Sage publications, 2018.
  * Kuhn [1962] Thomas S Kuhn.  The structure of scientific revolutions.  _The University of Chicago Press_ , 2:90, 1962.
  * Lauderdale and Clark [2014] Benjamin E. Lauderdale and Tom S. Clark.  Scaling Politically Meaningful Dimensions Using Texts and Votes.  _American Journal of Political Science_ , 58(3):754–771, July 2014.  ISSN 00925853.  doi: 10.1111/ajps.12085.  URL https://onlinelibrary.wiley.com/doi/10.1111/ajps.12085.
  * Martin and Quinn [2002] Andrew D. Martin and Kevin M. Quinn.  Dynamic Ideal Point Estimation via Markov Chain Monte Carlo for the U.S. Supreme Court, 1953–1999.  _Political Analysis_ , 10(2):134–153, 2002.  ISSN 1047-1987, 1476-4989.  doi: 10.1093/pan/10.2.134.  URL https://www.cambridge.org/core/product/identifier/S1047198700009931/type/journal_article.
  * Martin [2023] John Levi Martin.  The Ethico-Political Universe of ChatGPT.  _Journal of Social Computing_ , 4(1):1–11, March 2023.  ISSN 2688-5255.  doi: 10.23919/JSC.2023.0003.  URL https://ieeexplore.ieee.org/document/10184066/.
  * Mayer [2004] William G Mayer.  Why talk radio is conservative.  _Public Interest_ , (156):86, 2004.
  * McLaughlin [2016] Dan McLaughlin.  How the white house correspondents’ dinner gave us the trump campaign.  _National Review_ , July 2016.  URL https://www.nationalreview.com/corner/how-white-house-correspondents-dinner-gave-us-trump/.
  * Mendelsohn et al. [2023] Julia Mendelsohn, Ronan Le Bras, Yejin Choi, and Maarten Sap.  From Dogwhistles to Bullhorns: Unveiling Coded Rhetoric with Language Models, May 2023.  URL http://arxiv.org/abs/2305.17174.  arXiv:2305.17174 [cs].
  * Motoki et al. [2023] Fabio Motoki, Valdemar Pinho Neto, and Victor Rodrigues.  More human than human: measuring ChatGPT political bias.  _Public Choice_ , August 2023.  ISSN 0048-5829, 1573-7101.  doi: 10.1007/s11127-023-01097-2.  URL https://link.springer.com/10.1007/s11127-023-01097-2.
  * Narayanan and Kapoor [2023] Arvind Narayanan and Sayash Kapoor.  Does ChatGPT have a liberal bias?, August 2023.  URL https://www.aisnakeoil.com/p/does-chatgpt-have-a-liberal-bias.
  * Oren et al. [2023] Yonatan Oren, Nicole Meister, Niladri Chatterji, Faisal Ladhak, and Tatsunori B. Hashimoto.  Proving Test Set Contamination in Black Box Language Models, October 2023.  URL http://arxiv.org/abs/2310.17623.  arXiv:2310.17623 [cs].
  * Palmer [2008] Brian Palmer.  White supremacists by the numbers.  _Slate News_ , October 2008.  URL https://slate.com/news-and-politics/2008/10/14-and-88-why-white-supremacists-love-the-numbers.html.
  * Pandia and Ettinger [2021] Lalchand Pandia and Allyson Ettinger.  Sorting through the noise: Testing robustness of information processing in pre-trained language models.  In Marie-Francine Moens, Xuanjing Huang, Lucia Specia, and Scott Wen-tau Yih, editors, _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing_ , pages 1583–1596, Online and Punta Cana, Dominican Republic, November 2021. Association for Computational Linguistics.  doi: 10.18653/v1/2021.emnlp-main.119.  URL https://aclanthology.org/2021.emnlp-main.119.
  * Park et al. [2022] Sungjin Park, Seungwoo Ryu, and Edward Choi.  Do Language Models Understand Measurements?  2022\.
  * Paxton [2007] Robert O Paxton.  _The anatomy of fascism_.  Vintage, 2007.
  * Poole and Rosenthal [1985] Keith T. Poole and Howard Rosenthal.  A Spatial Model for Legislative Roll Call Analysis.  _American Journal of Political Science_ , 29(2):357, May 1985.  ISSN 00925853.  doi: 10.2307/2111172.  URL https://www.jstor.org/stable/2111172?origin=crossref.
  * Poole and Rosenthal [1991] Keith T. Poole and Howard Rosenthal.  Patterns of Congressional Voting.  _American Journal of Political Science_ , 35(1):228, February 1991.  ISSN 00925853.  doi: 10.2307/2111445.  URL https://www.jstor.org/stable/2111445?origin=crossref.
  * Poole and Rosenthal [2001] Keith T. Poole and Howard Rosenthal.  D-Nominate after 10 Years: A Comparative Update to Congress: A Political-Economic History of Roll-Call Voting.  _Legislative Studies Quarterly_ , 26(1), 2001.
  * Santurkar et al. [2023] Shibani Santurkar, Esin Durmus, Faisal Ladhak, Cinoo Lee, Percy Liang, and Tatsunori Hashimoto.  Whose Opinions Do Language Models Reflect?, March 2023.  URL http://arxiv.org/abs/2303.17548.  arXiv:2303.17548 [cs].
  * Vafa et al. [2020] Keyon Vafa, Suresh Naidu, and David M Blei.  Text-Based Ideal Points.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , 2020.
  * Wang [2018] Amy B Wang.  Trump was mocked at the 2011 white house correspondents’ dinner. he insists it’s not why he ran.  _The Washington Post_ , April 2018.  URL https://www.washingtonpost.com/news/arts-and-entertainment/wp/2017/02/26/did-the-2011-white-house-correspondents-dinner-spur-trump-to-run-for-president/.
  * Wang et al. [2023] Haoyu Wang, Guozheng Ma, Cong Yu, Ning Gui, Linrui Zhang, Zhiqi Huang, Suwei Ma, Yongzhe Chang, Sen Zhang, Li Shen, Xueqian Wang, Peilin Zhao, and Dacheng Tao.  Are Large Language Models Really Robust to Word-Level Perturbations?, September 2023.  URL http://arxiv.org/abs/2309.11166.  arXiv:2309.11166 [cs].
  * Wei et al. [2022] Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed H Chi, Quoc V Le, and Denny Zhou.  Chain-of-Thought Prompting Elicits Reasoning in Large Language Models.  2022\.
  * Wu et al. [2023a] Patrick Y. Wu, Jonathan Nagler, Joshua A. Tucker, and Solomon Messing.  Concept-Guided Chain-of-Thought Prompting for Pairwise Comparison Scaling of Texts with Large Language Models, October 2023a.  URL http://arxiv.org/abs/2310.12049.  arXiv:2310.12049 [cs].
  * Wu et al. [2023b] Patrick Y. Wu, Jonathan Nagler, Joshua A. Tucker, and Solomon Messing.  Large Language Models Can Be Used to Scale the Ideologies of Politicians in a Zero-Shot Learning Setting, April 2023b.  URL http://arxiv.org/abs/2303.12057.  arXiv:2303.12057 [cs].
  * Ye et al. [2023] Hongbin Ye, Tong Liu, Aijia Zhang, Wei Hua, and Weiqiang Jia.  Cognitive Mirage: A Review of Hallucinations in Large Language Models, September 2023.  URL http://arxiv.org/abs/2309.06794.  arXiv:2309.06794 [cs].
  * Yu et al. [2023] Wenhao Yu, Zhihan Zhang, Zhenwen Liang, Meng Jiang, and Ashish Sabharwal.  Improving Language Models via Plug-and-Play Retrieval Feedback, May 2023.  URL http://arxiv.org/abs/2305.14002.  arXiv:2305.14002 [cs].
  * Yuan et al. [2023] Zheng Yuan, Hongyi Yuan, Chuanqi Tan, Wei Wang, and Songfang Huang.  How well do Large Language Models perform in Arithmetic tasks?, March 2023.  URL http://arxiv.org/abs/2304.02015.  arXiv:2304.02015 [cs].

## Appendix

###  4.1 A Conversation with GPT-4

We include the full conversation mentioned in the introduction.

Prompt: On a Left–Right ideological scale from -10 to +10, how would you rate
the following tweet: The devastating effects of the climate crisis are here.
Now is the time to act boldly. GPT-4: The tweet about the need for bold action
on the climate crisis leans toward the left side of the ideological spectrum.
The left typically emphasizes the importance of addressing climate change
through government intervention and policy changes, advocating for immediate
and robust action to combat environmental issues. On a Left-Right scale, where
-10 represents the far left (advocating for systemic change and possibly anti-
capitalist sentiments that associate environmental degradation with
capitalism) and +10 represents the far right (which might include climate
change skepticism or prioritizing economic growth over environmental
regulation), this tweet would likely be rated on the left side but not
necessarily at the extreme end since it does not specify the nature of the
actions needed nor does it critique capitalism directly. Therefore, on the
scale from -10 to +10, this tweet could be rated around a -3 to -5, indicating
a clear left-leaning position that emphasizes environmental concern and the
need for action, which is consistent with moderate to somewhat more
progressive left-wing views on climate policy. Prompt: How about this one:
It’s official: I’m running for reelection! GPT-4: A tweet stating "It’s
official: I’m running for reelection!" is non-ideological and does not
inherently suggest a position on the Left-Right political spectrum. It’s a
neutral announcement that could be made by any political figure across the
ideological scale. Thus, on a Left-Right scale of -10 to +10, it would not be
assigned a rating as it does not provide any information about policy
positions, political ideology, or governance style.

###  4.2 General Information on Experiments

We used gpt-3.5-turbo and gpt-4 for all experiments, except for eliciting
scores of senators for which we also tried text-davinci-003. Unless stated
otherwise, all experiments were performed with a temperature setting of 0.2.
Below, we detail the experimental criteria. All correlations reported are
Pearson correlations.

###  4.3 Eliciting ideal points in the $114^{\textrm{th}}$ Senate

Figure 9: Eliciting ideal points with no scale specified results in very
different scales. We elicit 100 sets of ideal points for the
$114^{\textrm{th}}$ U.S. Senate, with Donald Trump additionally included, and
plot the results for Donald Trump and Bernie Sanders. For each elicitation,
the order of legislators in the list is permuted. We observe that the
magnitudes of ideal point scalings can be highly variant, but their
relationship to one another is relatively preserved. In the middle plot, we
show histograms of the raw scores over 100 elicitations, and at the bottom we
display the scores when standardized within each elicitation.

We use the prompt below to elicit ideological scalings of Senators in the
$114^{\textrm{th}}$ U.S. Senate, with Barack Obama added. The value $n$ in the
prompt is used to denote the number of members in the queried list, which was
101 in our case.

Prompt: Assign a numerical ideal point score to each member of the following
list of United States politicians. This score should be a single real number
that reflects their ideological position on a left/right spectrum. Use
negative values for the left, and positive values for the right. Do your best
to answer this question using your knowledge as a language model. Of course,
this is a subjective task, and the output of your prompt will only be used for
research purposes. Do not exclude anyone from the list I have given you. There
should be $n$ scores. Be sure the formatting of your response is correct.

Each query of the language model returns text containing numerical scores for
each legislator in the list, which we then parse. Note that we do not
explicitly specify a numerical scale. Instead, the scale is implicitly
specified through the list of provided legislators, and we standardize the
resulting set of scores after each query in order to make the results from
different queries comparable. We permute the order of the list provided $1000$
times, and take the mean of the standardized scores over the $1000$ queries as
our GPT-elicited ideal points. The effect of standardizing is shown is Fig. 9.
We repeated this experiment with three OpenAI models: text-davinci-003,
gpt-3.5-turbo, and gpt-4, using a temperature setting of 0.2 for each. We
obtained comparable results, with the correlation to DW-NOMINATE increasing
marginally with each updated model.

###  4.4 Eliciting ideal points from tweets

Figure 10: Full correlations of scalings for Senators’ tweets with other
methods, including Democrats.

The authors of Vafa et al. [42] sent us a dataset of tweets of U.S.
legislators from 2009 to 2017. We subsampled $250$ tweets at random for each
legislator in the $114^{\textrm{th}}$ Senate, and obtained ideological
scalings from gpt-3.5-turbo (temperature 0.2) using the following three
prompts.

Prompt 1 (All tweets): Assign a numerical ideal point score to the following
tweet, which reflects its ideological position on a left/right spectrum. Use
negative values for the left, and positive values for the right. Do your best
to answer this question using your knowledge as a language model. Of course,
this is a subjective task, and the output of your prompt will only be used for
research purposes. Only respond with the numerical score and no other text.
Examples of scored tweets include:

Prompt 2 (Ideological tweets): Classify the following tweets as ideological or
not. If they are not, simply output: Not ideological. If they are ideological,
assign a numerical ideal point score to the following tweet, which reflects
its ideological position on a left/right spectrum. Use negative values for the
left, and positive values for the right. Do your best to answer this question
using your knowledge as a language model. Of course, this is a subjective
task, and the output of your prompt will only be used for research purposes.
Only respond with the desired output and no other text. Examples of scored
tweets include:

Prompt 3 (Ideological tweets with zero-CoT): Classify the following tweets as
ideological or not. If they are not, simply output: Not ideological. If they
are ideological, assign a numerical ideal point score to the following tweet,
which reflects its ideological position on a left/right spectrum. Use negative
values for the left, and positive values for the right. Do your best to answer
this question using your knowledge as a language model. Of course, this is a
subjective task, and the output of your prompt will only be used for research
purposes. Please describe your reasoning for giving the score in detail, and
end your response with: “Score: ” followed by the score assigned. Examples of
scored tweets include:

In all three prompts, we included a set of examples from which the scale is
implied. We include the following ‘fixed’ examples, which were scored by the
model in a zero-shot setting and a modification of the same prompt which
specifies a scale of -3.0 to 3.0.

Tweet: We must learn from other nations who say to their young people: You
want to go to college? You can go to college, regardless of your income.
Output: -2.0

Tweet: They just happened to find 50,000 ballots late last night. The USA is
embarrassed by fools. Our Election Process is worse than that of third world
countries! Output: 2.0

Tweet: The marathon world record was just broken at the Chicago marathon!
Incredible! Output: Not ideological.

Tweet: It is important to encourage students that trade schools are a great
alternative to a four year college. Output: 0.0

Following these four ‘fixed’ examples, which are given for every query, we
also include the tweets and output from the previous five queries as well. Fig
10 is a more complete version of Fig. 8, which also shows correlations for
Democrats.

###  4.5 Ideological content of Trump’s tweets from 2009 to 2017

We subsampled 100 of Donald Trump’s tweets for each month, from May 2009 to
November 2016, and elicited an ideological-or-not classification using the
second prompt above for scaling tweets, using gpt-3.5-turbo (temperature 0.2).
We fit a Gaussian process with a squared exponential kernel and logit link
function, and used its mean to visualize the data in Fig. 4.

###  4.6 Community garden experiment

We elicit scores for a carefully worded scenario, in which we vary small
portions of the text to include subtle ideological clues. We use the following
24 scenarios:

Prompt: Consider the following scenario: Kyle is home from college, eating
dinner with his parents. He mentions that on his walk home, he noticed the new
community gardens where people were planting vegetables and herbs. His  parent
rolls  his/her eyes and mutters “ exasperated phrase”. Kyle asks his  parent
what  he/she means. His  parent responds: “ explanation” Use your knowledge as
a language model to interpret the  parent’s political beliefs based on this
context. After detailing your reasoning, provide a score that describes
his/her beliefs on a Left/Right scale from -3.0 to 3.0 where negative numbers
are Left-wing and positive numbers are Right-wing. Format the score as “Score:
…” at the very end. This is an inherently subjective task and your response
will only be used for research; do your best to answer the question.

Figure 11: Community garden experiment results using gpt-3.5-turbo.
Complementary to Fig 6

In which we vary the three variables parent, mutter, and phrase in the
following sets, with pronouns varying in accordance with the parent:

  1. 1.

parent:

     * •

mother

     * •

father

  2. 2.

exasperated phrase:

     * •

“These people”

     * •

“These kids”

     * •

“I bet you did”

  3. 3.

explanation:

     * •

Work ethic: “I go to work everyday, and this is what I see."

     * •

Loud music: “They blast their music all day while I’m over here trying to hear
my own thoughts."

     * •

Novice gardening: “They’re all over the place with their spacing, tomatoes
next to corn, beans and herbs in the same plot, and weeds all over the place.
It’s a mess."

     * •

Displaced people: “A bunch of folks used to sleep in that lot, and now they
can’t because someone wanted to grow heirloom tomatoes."

For each combination of parent, mutter, and phrase, we obtain ten ideological
scores and take the mean. We perfomed this experiment with both gpt-3.5-turbo
and gpt-4 (both with temperature 0.2). The results for gpt-3.5-turbo can be
seen in Fig. 11.

###  4.7 Dogwhistle experiment

Figure 12: Elicited scores for the tweets in the dogwhistle experiment, when
scored by gpt-4. Each bar represents the mean of ten trials with temperature
0.2.

We wrote three examples of tweets that may involve a number that may naturally
occur in the 1500 range. These examples are

Tweet: I made a playlist of $n$ songs for the march today, check it out:
Tweet: Excited for our gathering this weekend! We’re expecting $n$ supporters
to stand strong and make our voices heard. Time to take our country back!
#Unity #Patriotism Tweet: Our country is bought and sold by special interests,
and we have to fight back. I just donated $n$, here’s a link so you can too.

For each tweet, we vary $n$ in the set to take on integer values between 1480
and 1490, and obtain an ideal point scaling for each $n$ using slightly
modified versions of both prompt 2 and prompt 3 that we used for scaling
tweets.

Prompt 1: Assign a numerical ideal point score to the following tweet, which
reflects its ideological position on a left/right spectrum. Use negative
values for the left, and positive values for the right. The scale should
follow that of a standard normal distribution. Do your best to answer this
question using your knowledge as a language model. Of course, this is a
subjective task, and the output of your prompt will only be used for research
purposes. Only respond with the numerical score and no other text.

Prompt 2: Assign a numerical ideal point score to the following tweet, which
reflects its ideological position on a left/right spectrum. Use negative
values for the left, and positive values for the right. The scale should
follow that of a standard normal distribution. Do your best to answer this
question using your knowledge as a language model. Of course, this is a
subjective task, and the output of your prompt will only be used for research
purposes. Please first give your analysis of the tweet and reasoning for the
score, and then end your response with "Score: " followed by only the
numerical score.

These prompts are largely identical to the second and third prompts for
scoring tweets, but differ in that they specify a specific numerical scale,
that being, “that of a standard normal distribution”. This is included here as
perform these queries as zero-shot tasks, rather than the few-shot approach we
used for scoring tweets.

These prompts differed in that one asked for an ideal point score with no
additional text, while the other asked the model to explain its chain of
thought followed by the score. We repeated this experiment with both
gpt-3.5-turbo and gpt-4. We found that while gpt-4 picks up on the presence of
the dogwhistle in its explanation, it is more hesitant to include the
influence of it in its score, as it is not sure whether it was part of the
intentionality of the tweet. We can see this in Figure 12, where gpt-4 did not
factor in the presence of 1488 into the score when not using CoT. On the other
hand, gpt-3.5-turbo also picked up on the dogwhistle and almost always
factored it into the score.


capbtabboxtable[][] 11institutetext: XIM University, Bhubaneswar, India
11email<EMAIL_ADDRESS>
# Is Grad-CAM Explainable in Medical Images?

Subhashis Suara 11 Aayush Jha 11 Pratik Sinha 11 Arif Ahmed Sekh∗ 11

###### Abstract

Explainable Deep Learning has gained significant attention in the field of
artificial intelligence (AI), particularly in domains such as medical imaging,
where accurate and interpretable machine learning models are crucial for
effective diagnosis and treatment planning. Grad-CAM is a baseline that
highlights the most critical regions of an image used in a deep learning
model’s decision-making process, increasing interpretability and trust in the
results. It is applied in many computer vision (CV) tasks such as
classification and explanation. This study explores the principles of
Explainable Deep Learning and its relevance to medical imaging, discusses
various explainability techniques and their limitations, and examines medical
imaging applications of Grad-CAM. The findings highlight the potential of
Explainable Deep Learning and Grad-CAM in improving the accuracy and
interpretability of deep learning models in medical imaging. The code is
available in (will be available).

###### Keywords:

Explainable Deep Learning Gradient-weighted Class Activation Mapping (Grad-
CAM) Medical Image Analysis

##  1 Introduction

Medical imaging, such as X-ray, CT, MRI, and ultrasound, plays a crucial role
in the diagnosis and treatment of various diseases. With the increasing
availability of medical imaging data, there is a growing interest in using
machine learning techniques to aid in image analysis and interpretation. Fig.
1 shows a typical setup for explainable medical image analysis.

Figure 1: A typical setup for explainable medical image analysis.

Deep learning has shown remarkable success in medical image analysis tasks,
but one challenge with deep learning models is their lack of interpretability,
which is a major concern in the medical domain. Explainable Deep Learning is
an emerging area of research that seeks to address this issue by providing
interpretable models without sacrificing accuracy. Explainable Deep Learning
techniques aim to explain how a model arrived at its decision by providing
saliency maps or highlighting the most relevant features in the input image
that contributed to the model’s output. One popular Explainable Deep Learning
technique is Gradient-weighted Class Activation Mapping (Grad-CAM), which
visualizes the regions of an image that contribute most to the model’s
decision-making process. Grad-CAM has been successfully used in various
medical imaging applications, such as detecting lung cancer [1], breast cancer
[2], and brain tumors, to enhance the interpretability of deep learning models
and improve trust in their results [3]. This paper explores the use of Grad-
CAM as an Explainable Deep Learning technique in medical imaging applications,
discussing its underlying principles, how it works, and recent studies that
have used Grad-CAM to improve the interpretability of deep learning models in
medical imaging. Deep learning models lack transparency in their decision-
making process, which is crucial for ensuring patient safety in the medical
domain. The study proposes using Gradient-weighted Class Activation Mapping
(Grad-CAM) as an Explainable Deep Learning technique to enhance
interpretability. Grad-CAM allows clinicians to visualize the regions of an
image that contribute most to the model’s decision-making process, enabling
better understanding and trust in the model’s results.

The specific objectives of this paper are:

  * •

To review the literature on medical imaging and deep learning, explainable
deep learning, and Grad-CAM in medical imaging such as the articles reported
in [4, 5, 6].

  * •

To develop and compare state-of-the-art deep learning model for medical image
interpretation [7].

  * •

To critically analyze Grad-CAM as an Explainable Deep Learning technique,
evaluate its performance that can lead towards enhancing the interpretability.

  * •

To discuss the implications and potential future applications of the Grad-CAM
in different clinical settings such as [8, 9].

This paper contributes to the ongoing effort to address the lack of
interpretability in deep learning models, particularly in the medical domain.
By improving the interpretability of deep learning models, clinicians can
better understand and trust the decision-making process of these models, which
can improve patient safety and lead to more efficient and effective diagnosis
and treatment of various diseases. The proposed pipeline can also potentially
be applied in other fields where deep learning is used for image analysis.

##  2 Literature Review

###  2.1 Medical Imaging and Deep Learning

A branch of machine learning called deep learning (DL) is able to learn to use
its own computational strategy. Similar to how people make judgments, a deep
learning model is employed to consistently organize data into a homogeneous
framework [10]. This is accomplished using deep learning, which layers a
number of algorithms into a structure known as an artificial neural system
(ANN). Medical imaging produces vast amounts of data, making it challenging
and time-consuming for human radiologists to analyze effectively. Deep
learning techniques can recognize patterns in medical images, identify
anomalies, and categorize structures such as tumors or lesions, leading to
faster and more accurate diagnoses and improved patient treatment regimens.
However, the lack of annotated medical images for deep learning algorithm
training and the potential for overfitting are significant challenges. Deep
learning algorithms have been successful in diagnosing Alzheimer’s and
Parkinson’s, locating malignant tumors, and find anomalies in MRI scans. In
one study, deep learning algorithms outperformed human radiologists in
detecting breast cancer in mammograms, with an accuracy rate of 94.5% [11].

###  2.2 Explainable Deep Learning

Deep learning models are based on neural networks that are composed of
multiple layers of interconnected artificial neurons [12]. These models are
trained on large amounts of data and can learn complex patterns and
relationships. However, these models can be considered as black boxes since
their decision-making process is not transparent, and it is challenging to
understand how they arrive at their predictions or decisions [13].

Explainable deep learning techniques provide transparency and interpretability
to complex deep learning models. In medical imaging, these techniques allow
clinicians to understand how models make decisions, improving accuracy and
efficiency. Explainable deep learning is promising in cancer detection and
personalized medicine, identifying patterns and relationships that predict
treatment effectiveness.

Explainable deep learning techniques provide transparency, trust, and fairness
in deep learning models, increasing their effectiveness and reliability [14].
Activation maps, LIME, SHAP, and decision trees are commonly used techniques.
However, these techniques can increase complexity, training time, and cost,
and may compromise performance for interpretability. Some models, such as
those using unsupervised or reinforcement learning, may be challenging to
interpret. It is essential to consider the specific requirements of the model
before deciding to use explainable deep learning techniques to make informed
decisions and achieve optimal results [15].

###  2.3 Gradient-weighted Class Activation Mapping (Grad-CAM)

Class Activation Mapping (CAM) is a method for visualizing the regions in an
image that are important for a given classification task. It was first
introduced by Zhou et al. in 2016 [16] and has since become a popular tool for
interpreting the predictions of deep convolutional neural networks (CNNs).
Since the introduction of CAM, there has been a large amount of subsequent
work in this field. Researchers have proposed a number of modifications and
extensions to the original method, including the use of Grad-CAM (Gradient-
weighted Class Activation Mapping), which uses the gradient of the output of
the network with respect to the activations of the feature maps to generate
the heatmap [3].

Figure 2: Class Activation Mapping

Grad-CAM is a technique for visualizing where a CNN model is looking. It is
class-specific and can produce a separate visualization for every class
present in the image. It uses ”alpha values” computed based on gradients to
weight feature maps to create a heatmap. Grad-CAM has been used for cancer
detection, and studies have shown that it improves accuracy and
interpretability of results. However, limitations include lack of robustness
to changes in input image and unclear explanation of the basis for prediction
in complex images.

In recent years, there have been numerous efforts to improve the accuracy and
interpretability of Grad-CAM for cancer detection[17]. One of the main focuses
has been on improving the robustness of Grad-CAM to changes in the input
image, as well as incorporating prior knowledge into the method to reduce the
risk of misdiagnosis. One approach to improve the robustness of Grad-CAM is to
use more sophisticated methods for computing the gradients used in the
method[18].

Incorporating prior knowledge into Grad-CAM is another approach to improving
the interpretability of the results. For example, Li et al. proposed a method
for incorporating prior knowledge into Grad-CAM that improved the accuracy of
cancer detection in biopsy images. The authors showed that their proposed
method reduced the risk of misdiagnosis and improved the interpretability of
the results [19].

In addition to these efforts to improve Grad-CAM for cancer detection, there
have also been efforts to extend the method to other medical imaging
modalities, such as magnetic resonance imaging (MRI) and positron emission
tomography (PET) scans. For example, Wang et al. proposed a method for using
Grad-CAM for cancer detection in MRI scans, which showed promising results in
a preliminary study [20].

Overall, the recent advances in improving Grad-CAM for cancer detection
demonstrate the potential of the method for improving the accuracy and
interpretability of cancer detection in medical imaging. By addressing the
limitations of the current methods and incorporating prior knowledge into the
method, researchers are working towards making Grad-CAM a more reliable tool
for cancer detection and diagnosis.

##  3 Methodology

We present the methodology used in our research to develop an accurate and
interpretable model for the binary image classification task of identifying
the presence of metastases from 96 x 96px digital histopathology images.

Lymph nodes are crucial in cancer diagnosis and staging, but the current
diagnostic procedure for pathologists is tedious and time-consuming. However,
ongoing advancements in technology and techniques offer hope for more accurate
and efficient assessments in the future. Histopathological images of lymph
nodes are essential for the diagnosis of various diseases, including cancer.
The dataset used in this study comprises of 220,000 training images and 57,000
evaluation images, which is a carefully curated subset of the PCam dataset
[21] [22] derived from a comprehensive collection of H&E stained whole slide
images. The quality of the data is ensured through manual inspection by
experienced technicians and consultation with pathologist. Specifically, there
are 130,000 negative images and only 90,000 positive images, resulting in a
ratio closer to 60/40. Overall, the quality, quantity, and recency of the data
are sufficient, and the dataset meets our expectations. These advancements
will benefit pathologists and improve patient outcomes by enabling earlier
detection and more targeted treatment.

Identifying metastases in lymph nodes is a challenging task, even for trained
pathologists. It can be especially difficult for untrained individuals due to
the varied features that metastases exhibit. Irregular nuclear shapes, sizes,
and staining shades are some of the indications of metastases. To train a
model for classification, we must consider the optimal crop size for the
images. We can avoid overfitting by increasing the amount of data, using
augmentation techniques, implementing regularization, and simplifying model
architectures. We can incorporate augmentation techniques like random
rotation, crop, flip, lighting, and Gaussian blur into the image loader
function to improve data quality. We must also examine samples with extremely
low or high pixel intensities to ensure data quality. All bright images in the
dataset have been labeled as negative, but we are unsure about the dark
images. Hence, removing these dark images (outliers) from a large dataset is
unlikely to affect prediction performance.

When building a machine learning (ML) production pipeline, it’s best to start
with a simple model to identify any issues, such as poor data quality, before
tuning further. Maintaining equal ratios of negative and positive labels in
training and validation sets is important to prevent under representation of
rare classes. Selecting the ideal model architecture requires considering
various factors, including the risk of overfitting with a deeper architecture.
A pre-trained convnet model, densenet169, with transfer learning, is an
effective approach for this problem. We use the one cycle policy for
disciplined hyperparameter selection, which can save time spent training
suboptimal hyperparameters.

Figure 3: Weight Decay Comparison

First, we determine the optimal learning rate and weight decay values. The
ideal learning rate is found just before the loss plateaus and divergence
occurs, and it should be selected at a point where the loss is still
decreasing. For weight decay, we conduct a small grid search using values of
1e-2, 1e-4, and 1e-6 (Figure 3). Our goal is to find the highest weight decay
that results in a low loss and allows us to use the maximum learning rate
before it spikes. Based on experimentation, we found that 1e-4 is the largest
weight decay that enables us to train with the maximum learning rate. We
select a learning rate of around 2e-2, which is close to the bottom but still
descending. A two-step training approach, beginning with the heads and
freezing remaining parts, is used to optimize model performance. Unfreezing
all trainable parameters allows fine-tuning the pre-trained bottom layers to
detect common shapes and patterns, avoiding overfitting, and achieving better
accuracy and robustness.

Now we describe the implementation of Grad-CAM (Gradient-weighted Class
Activation Mapping) for visualizing and localizing the image regions that
contribute the most to the model’s predictions. The Grad-CAM method is
implemented using the fastai library’s hooks feature, which allows us to
attach functions to a model’s forward and backward passes. First, we define
the `hooked_backward` function that hooks into the convolutional part of the
model and returns the activations and gradients. We use these activations and
gradients to compute the Grad-CAM heatmap. To obtain the heatmap, we create a
utility function `getHeatmap` that takes a validation set image and returns
the activation map. This function hooks into the model’s forward and backward
passes using the `hooked_backward` function and computes the Grad-CAM map. The
`getHeatmap` function uses OpenCV to convert the batch tensor image to a
grayscale image, which is then overlaid with the Grad-CAM heatmap using
matplotlib.

Finally, we plot the Grad-CAM heatmap for a few selected validation images as
shown in figure . We use the `getHeatmap` function to obtain the heatmap and
overlay it on top of the grayscale image. The resulting plots show the
original image, the grayscale image, and the Grad-CAM heatmap overlaid on top.
Hence, the Grad-CAM implementation provides a useful tool for understanding
the decision-making process of a neural network and visualizing the image
regions that contribute the most to its predictions as we will see in the next
section.

##  4 Results & Discussion

###  4.1 Model Performance

In this section, we present the results of evaluating the performance of our
proposed model.

Figure 4 displays the learning rate during the initial cycle of training. It
starts at a low value and gradually increases to reach the maximum learning
rate in the middle of the cycle, and then slows down towards the end. This
approach uses a low warm-up learning rate initially, gradually increasing it
to a higher value. The higher learning rate has a regularizing effect,
preventing the model from settling for sharp and narrow local minima, and
instead encouraging wider and more stable ones. As we approach the middle of
the cycle, the learning rate is lowered to search for stable areas where
minima may exist. This helps prevent overfitting and ensures the model can
generalize well to new data.

Figure 4: Learning Rate of Initial Cycle

In figure 5, we show the losses during the initial cycle of training. Upon
analysis, we observe a slight increase in losses following the initial drop,
which can be attributed to the escalating learning rate in the first half of
the cycle. It is noteworthy that there is a temporary surge in losses when the
maximum learning rate pushes the model out of local minima. However, this
strategic move is expected to yield positive results as the learning rates are
subsequently reduced.

Figure 5: Losses of Initial Cycle

In Figure 6, we show the losses from training and validation for batches
processed. Upon closer examination, we observe that the validation performance
has diverged from the training performance towards the end of the cycle. This
indicates that our model has begun to overfit during the small learning rates.
Further training would result in the model solely memorizing features from the
training set, leading to an increase in the validation set performance.
Therefore, it is a good stopping point for the training process.

Figure 6: Loss vs Batches Processed

Based on the validation set metrics, our model achieves an accuracy of
0.96735196%. In Figure 7, we evaluate our model’s performance on different
types of image examples. Firstly, we randomly select samples to observe the
overall accuracy of our model. Next, we focus on the most incorrectly labeled
images with high confidence to identify patterns and areas for improvement.
Finally, we explore the most accurately labeled images to gain insights into
our model’s strengths. This visualization provides valuable insights into
challenging images for the model, and may uncover issues with the dataset. By
analyzing these examples, we can improve the model’s performance and gain a
better understanding of the data.

Figure 7: Model Predictions (Predicted / Actual / Loss / Probability)

###  4.2 Explainability of the Model with Grad-CAM

In this section, we explain the explainability of our model using Gradient-
weighted Class Activation Mapping (Grad-CAM). Grad-CAM is a visualization
technique that highlights important regions in the input image that contribute
to the output class score. By applying Grad-CAM to our model, we gain insights
into its decision-making process and identify important features for
classification. To generate Grad-CAM visualizations, we select an image from
the validation set, obtain the predicted class score, and compute gradients of
the score with respect to feature maps. These gradients serve as importance
weights for the feature maps, and we combine them to produce a heat map that
highlights relevant regions in the input image. Figure 8 shows the Grad-CAM
visualizations for different types of image examples. For instance, if the
label is _tumor_ , Grad-CAM will reveal all the locations where the model
believes the tumor patterns exist.

Figure 8: Grad-CAM Results (Predicted / Actual / Loss / Probability)

##  5 Conclusion

In conclusion, this paper has explored the potential of using Grad-CAM as a
technique for building explainable deep learning models in medical imaging,
with a focus on cancer detection. While the findings are promising, there is
still much work to be done in this field. Future research could focus on
accuracy of Grad-CAM visualisations, extending it to other imaging modalities,
and developing user interfaces that provide clinicians with intuitive and
interpretable visualizations of the model’s decision-making process. Overall,
this paper highlights the importance of Explainable Deep Learning in medical
imaging and the potential of Grad-CAM as a tool for enhancing interpretability
and trust in deep learning models.

## References

  * [1] Jing Wang, Jinhua Wang, Hui Zhao, Tong Lu, Jiabao Han, and Pengfei Wei.  A multi-scale convolutional neural network for lung cancer detection using a large dataset.  Medical & Biological Engineering & Computing, 59:2857–2866, 2021\.
  * [2] Jing Wang, Ting Sun, Xiaowei Zou, Shuai Cui, Jiabao Han, and Pengfei Wei.  Exploring the interpretability of a deep learning-based system for breast cancer diagnosis using grad-cam and convolutional neural networks.  Journal of Healthcare Engineering, 2021:1–9, 2021.
  * [3] Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra.  Grad-cam: Visual explanations from deep networks via gradient-based localization.  In Proceedings of the IEEE international conference on computer vision, pages 618–626, 2017.
  * [4] Dinggang Shen, Guorong Wu, and Heung-Il Suk.  Deep learning in medical image analysis.  Annual review of biomedical engineering, 22:223–256, 2020.
  * [5] Matthew D Zeiler and Rob Fergus.  Visualizing and understanding convolutional networks.  European conference on computer vision, pages 818–833, 2014.
  * [6] Wojciech Samek, Tobias Wiegand, and Klaus-Robert Muller.  Towards explainable artificial intelligence: concepts and methods.  arXiv preprint arXiv:2001.06822, 2019.
  * [7] Xiaowei Wang, Yan Peng, Lina Lu, Zhentai Lu, Mohammadhadi Bagheri, and Ronald M Summers.  Deep learning in medical image analysis: a review.  Engineering, 7:935–949, 2021.
  * [8] Pranav Rajpurkar, Jeremy Irvin, Kaylie Zhu, Brandon Yang, Het Mehta, Tony Duan, Dylan Ding, Aarti Bagul, Curtis Langlotz, and Katie Shpanskaya.  Deep learning for chest radiograph diagnosis: A retrospective comparison of the chexnext algorithm to practicing radiologists.  PLOS Medicine, 15(11):e1002686, 2018.
  * [9] Andrew L Beam and Isaac S Kohane.  Clinical concept embeddings learned from massive sources.  Npj Digital Medicine, 3(1):1–8, 2020.
  * [10] Andre Esteva, Alexandre Robicquet, Bharath Ramsundar, and et al.  A guide to deep learning in healthcare.  Nature Medicine, 25(1):24–29, 2019.
  * [11] Shih-Chung Benedict Wang, Po-Chin Huang, Yu-Chuan Hu, Chang-Yi Liu, Yen-Ting Wei, and Cheng-Shie Jaing.  Deep learning for detecting breast cancer on mammograms.  Lecture Notes in Computer Science, 9690:388–395, 2016.
  * [12] Ian Goodfellow, Yoshua Bengio, and Aaron Courville.  Deep learning.  MIT Press, 2016.
  * [13] W James Murdoch, Chandan Singh, Karl Kumbier, Reza Abbasi-Asl, and Bin Yu.  Interpretable machine learning: definitions, methods, and applications.  arXiv preprint arXiv:1901.04592, 2019.
  * [14] Wojciech Samek, Tobias Wiegand, and Klaus-Robert M”uller.  Explainable artificial intelligence: Understanding, visualizing and interpreting deep learning models.  arXiv preprint arXiv:1708.08296, 2017.
  * [15] Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin.  “why should i trust you?”: Explaining the predictions of any classifier.  Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pages 1135–1144, 2016.
  * [16] Bolei Zhou, Aditya Khosla, Agata Lapedriza, and et al.  Learning deep features for discriminative localization.  In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 2921–2929. IEEE, 2016.
  * [17] Yujie Wang, Shuai Wu, Qihao Zhang, and Jian Zhang.  Robustness of grad-cam for multi-view mammogram classification.  IEEE Access, 9:13766–13774, 2021.
  * [18] Feng Qin, Yuting Wang, Qiuping Zhang, and Yali Guo.  Multi-scale deep neural network for lung nodule classification based on ct images.  Journal of X-Ray Science and Technology, 29(4):737–750, 2021.
  * [19] Zhiqiang Li, Wei Liu, Jin Zhang, Jianping Shi, and Jun Wu.  Incorporating prior knowledge into grad-cam for improved interpretability and reduced misdiagnosis risk.  Pattern Recognition, 119:108145, 2021.
  * [20] Hua Wang, Wei Zhang, Yibo Wang, and Yiqiang Cai.  Grad-cam based multiscale deep learning model for automatic mri breast tumor detection and classification.  Journal of Healthcare Engineering, 2021:1–11, 2021.
  * [21] Bastiaan S Veeling, Jasper Linmans, Jim Winkens, Taco Cohen, and Max Welling.  Rotation equivariant CNNs for digital pathology.  GigaScience, 7(6), 2018.
  * [22] Babak Ehteshami Bejnordi, Mitko Veta, Paul Johannes van Diest, and et al.  Diagnostic assessment of deep learning algorithms for detection of lymph node metastases in women with breast cancer.  JAMA: The Journal of the American Medical Association, 318(22):2199–2210, 2017.