# Uncovering Latent Arguments in Social Media Messaging by Employing LLMs-in-
the-Loop Strategy

Tunazzina Islam & Dan Goldwasser
Department of Computer Science
Purdue University, West Lafayette
Indiana 47907, USA
<EMAIL_ADDRESS>🖂 indicates the corresponding author.

###### Abstract

The widespread use of social media has led to a surge in popularity for
automated methods of analyzing public opinion. Supervised methods are adept at
text categorization, yet the dynamic nature of social media discussions poses
a continual challenge for these techniques due to the constant shifting of the
focus. On the other hand, traditional unsupervised methods for extracting
themes from public discourse, such as topic modeling, often reveal overarching
patterns that might not capture specific nuances. Consequently, a significant
portion of research into social media discourse still depends on labor-
intensive manual coding techniques and a human-in-the-loop approach, which are
both time-consuming and costly. In this work, we study the problem of
discovering arguments associated with a specific theme. We propose a generic
LLMs-in-the-Loop strategy that leverages the advanced capabilities of Large
Language Models (LLMs) to extract latent arguments from social media
messaging. To demonstrate our approach, we apply our framework to contentious
topics. We use two publicly available datasets: (1) the climate campaigns
dataset of $14k$ Facebook ads with $25$ themes and (2) the COVID-19 vaccine
campaigns dataset of $9k$ Facebook ads with $14$ themes. Furthermore, we
analyze demographic targeting and the adaptation of messaging based on real-
world events.

##  1 Introduction

Public opinion is crucial for ensuring responsive governance, policy alignment
with public interests, and societal harmony by facilitating a feedback loop
for continuous policy refinement (Glynn & Huge, 2008; Price, 1988). The advent
of social media has transformed the landscape of public discourse and opinion
formation, enabling the rapid exchange of information and ideas, facilitating
instant communication, and promoting a participatory approach to addressing
societal issues (McGregor, 2019; Xiong & Liu, 2014; Murphy et al., 2014). This
shift has led to an increased focus on automatically analyzing public opinion
on social media platforms (Liu, 2022; Pacheco et al., 2022a; Yousefinaghani et
al., 2021; Han et al., 2020; Islam & Goldwasser, 2020; Liang & Dai, 2013;
Maynard et al., 2012; Sobkowicz et al., 2012). In response to this, argument
mining has emerged as a critical technique that automatically extracts the
reasons, claims, and talking points (arguments)111We will use arguments and
talking points interchangeably in this paper., shedding light on how and why
specific opinions are formed (Wawrzuta et al., 2021; Sowa et al., 2021;
Skeppstedt et al., 2018; Habernal et al., 2018; Stab & Gurevych, 2014; Kim &
Hovy, 2006). In these cases, the variables of interest are well defined, and
substantial efforts are invested in developing manually annotated resources,
thereby allowing the problems to be structured as supervised learning tasks.

Figure 1: LLMs-in-the-Loop framework. TP: Talking point.

Unsupervised text analysis methods like topic modeling (Latent Dirichlet
Allocation (LDA) (Blei et al., 2003), non-negative matrix factorization (NMF)
(Lee & Seung, 1999)) can discover topics within data without requiring pre-
labeled datasets. However, they fall short of extracting meaningful themes
beneath those topics. Therefore, a non-trivial number of recent studies on
social media discourse rely on manual and qualitative coding methods (Hagen et
al., 2022; Nguyen et al., 2021; Del Valle et al., 2020).

In recent years, efforts have been made to delve into the subtleties of topics
in social media content by discovering themes (Islam & Goldwasser, 2024;
Pacheco et al., 2023; Islam et al., 2023b; Islam & Goldwasser, 2022; Pacheco
et al., 2022a). Yet, to fully comprehend why an opinion is formed, it’s
crucial to identify the specific arguments within these broader themes. For
example, under the ‘Climate Change’ topic, a theme might be ‘Alternative
Energy’, and under this theme, there might be conflicting arguments, such as
Argument 1: “Alternative energy will create more jobs.” versus Argument 2:
“Alternative energy will take away our jobs.”. Identifying arguments within a
theme reveals the granular nuances.

Pacheco et al. (2022b; a) has developed a human-in-the-loop approach for
discovering arguments by balancing unsupervised text analysis techniques and
manual coding. While this method enhances accuracy, it suffers from
scalability issues, is time-consuming, and demands high resources. In this
paper, we introduce an LLMs-in-the-Loop approach that integrates the
capabilities of Large Language Models (LLMs) (Brown et al., 2020) and
incorporates them into an efficient algorithmic framework to uncover latent
arguments under a theme in social media messaging.

We define large textual collections as repositories of textual instances
(e.g., ads, tweets, posts, documents) where each instance is associated with a
theme (the set of themes derived from previous studies (Islam & Goldwasser,
2024; Islam et al., 2023b; Pacheco et al., 2023; Islam & Goldwasser, 2022;
Pacheco et al., 2022a)). An overview of our framework is illustrated in
Figure. 1. Our framework provides a way of clustering talking points motivated
by themes. We have assigned instances with themes. For each theme cluster, we
cluster the associated instances using a clustering algorithm. We characterize
the sub-clusters by performing zero-shot multi-document summarization of the
top-K instances assigned to each sub-cluster. Then, we prompt LLMs in a zero-
shot manner to generate the talking point advocating in the sub-cluster
summary in the context of the given theme. For redundancy check, we assess the
similarity between talking point pairs and merge overlapping talking points
based on threshold. To generate a common talking point from the most similar
talking points, we prompt LLMs in a zero-shot setting. We have an optional
human evaluation phase for newly generated talking points. Finally, the
assignment phase maps instances to their most likely talking points.

Our framework is designed to address three main challenges: 1) Given a large
collection of instances with a known space of high-level themes, how can we
effectively identify a set of repeating arguments, 2) how can we refine those
arguments, and 3) how can we map instances to corresponding arguments
efficiently. To demonstrate our method, we look at the task of characterizing
social media messaging on contentious topics. Various interest groups,
including politicians, advertisers, and stakeholders, have used the highly
distributed social media landscape to microtarget audiences (Hersh, 2015;
Barbu, 2014). While effective in enhancing content relevance, microtargeting
also presents risks such as manipulating user behavior (Ribeiro et al., 2019;
Etudo et al., 2019; Kruikemeier et al., 2016), creating echo chambers
(Garimella et al., 2018; Lima et al., 2018; Del Vicario et al., 2016;
Quattrociocchi et al., 2016; Jamieson & Cappella, 2008), and fostering
polarization (Jiang et al., 2020; Zuiderveen Borgesius et al., 2018; Bostrom
et al., 2013).

In this paper, we examine two distinct case studies: the climate campaigns and
the COVID-19 vaccine campaigns in the United States. In each case, the
qualitative researchers apply different theories to identify themes. For
climate campaigns, the identification of themes is based on the energy
industry and climate change-related stances (Islam et al., 2023b). For the
COVID-19 vaccine campaigns (Islam & Goldwasser, 2022), the theme discovery is
grounded using Moral Foundation Theory (MFT) (Haidt & Graham, 2007; Haidt &
Joseph, 2004). These case studies are selected for their relevance to the
computational social science (CSS) community, illustrating the complexities of
targeted messaging in a digital age. From a machine learning viewpoint, these
case studies represent distinct challenges because of the data, context, and
target themes.

Our experiments show that our framework can be used to uncover a set of
arguments that cover a a large portion of the discussion about the climate and
COVID-19 vaccine campaigns on Facebook and the resulting mapping from ads to
arguments is fairly accurate with respect to human judgments. Additionally, we
provide an in-depth analysis of how messaging is tailored to specific
demographics and how these talking points evolve in response to real-world
events.

##  2 Related Work

Previous works for understanding messaging on social media (Islam et al.,
2023a; b; Islam & Goldwasser, 2022; Capozzi et al., 2021) used a predefined
set of labels, themes, and arguments to analyze messaging. These were fixed
and established based on existing topics or theoretical frameworks. In this
work, we focus on gaining a deeper understanding of messaging, especially at
the argument level.

Argument mining involves identifying and extracting argument components such
as claims, evidence, and conclusions and understanding how they are logically
connected to form coherent arguments. Several works have been done on
analyzing arguments on social media (Islam et al., 2023b; Pacheco et al.,
2022b; a; Islam & Goldwasser, 2022; Wawrzuta et al., 2021; Bhatti et al.,
2021; Malagoli et al., 2021; Kotelnikov et al., 2022; Freire-Vidal & Graells-
Garrido, 2019; Skeppstedt et al., 2018; Torsi & Morante, 2018; Dusmanu et al.,
2017). Bhatti et al. (2021) formulated argument mining on Twitter as a text
classification task to identify tweets serving as premises for hashtags that
represent claims to reveal prominent arguments for and against funding Planned
Parenthood. Wawrzuta et al. (2021) studied the arguments made by Twitter users
in Poland when discussing the COVID-19 vaccine. Pacheco et al. (2022b; a)
proposed interactive human-in-the-loop protocol to analyze COVID-19 vaccine
debate in Twitter. Miller & Lellis (2016) characterized the arguments
supporting the oil and gas industries. Islam et al. (2023b) manually developed
relevant arguments for climate change and energy industry-related stances. In
this paper, instead of using manual coding and human feedback to discover
arguments, we rely on a machine-in-the-loop approach.

Recently, large language models (LLMs) have achieved promising progress in
learning from prompts via in-context learning (ICL) (Chowdhery et al., 2023;
Kojima et al., 2022; Le Scao et al., 2022; Brown et al., 2020). Several
studies indicate that LLMs exhibit better performance in tasks traditionally
completed by humans (Gilardi et al., 2023; De Paoli, 2023; Dai et al., 2023;
Chiang & Lee, 2023; Ziems et al., 2024), highlighting a potential to leverage
LLMs effectively in our task. A recent approach has introduced the concept of
an LLM-in-the-loop for thematic analysis (Dai et al., 2023). In contrast, our
framework employs LLMs-in-the-Loop to uncover the latent arguments in
messaging. At a high level, our work connects with prior work on interactive
clustering (Pacheco et al., 2023; 2022a; Lund et al., 2017; Hu et al., 2014;
Bernstein et al., 2010), but instead of using human feedback to shape the
emergent clusters, we rely on LLM inference.

##  3 LLMs-in-the-Loop Framework

We propose an iterative LLMs-in-the-Loop framework that combines Natural
Language Processing (NLP) techniques and inference capabilities of LLMs to
automate the process of discovering latent arguments within themes. Our
framework enhances the ability to analyze vast amounts of social media content
by identifying nuanced arguments within predefined themes. This section
outlines the methodological steps, including clustering instances by themes,
identifying arguments, refining those arguments, and mapping instances to
corresponding arguments.

###  3.1 Theme-Specific Clustering

Initially, the framework categorizes textual instances into clusters based on
their associated themes. These instances can range from social media posts to
comprehensive documents. We divide each theme-based cluster into sub-clusters
using a clustering algorithm. This allows for a more granular analysis of the
thematic content, revealing the nuances of arguments present within each
theme. We embed the instances using Sentence BERT (Reimers & Gurevych, 2019).
The embedded instances are clustered using K-means (Jin & Han, 2010). To
determine the optimal value of $k$ in k-Means, we follow both the Elbow method
and the Silhouette method.

###  3.2 Summarizing Sub-clusters

To articulate the arguments found within each sub-cluster, we employ zero-shot
multi-document summarization using GPT-4 (Achiam et al., 2023) on the top-k
instances. The prompts are engineered to generate short theme-specific
summaries in a zero-shot setting. The five closest instances to each centroid
are used in the prompt to generate the summaries. This summarization process
highlights the key points and arguments without the need for pre-labeled data,
showcasing the framework’s unsupervised capabilities.

###  3.3 Generating and Refining Arguments

Subsequently, each sub-cluster summaries serve as a prompt for LLMs in a zero-
shot manner to generate specific talking points advocating for the arguments
implied in the summaries. This approach ensures that the extracted talking
points are relevant and coherent within the context of their respective
themes.

####  3.3.1 Refinement

To refine the generated arguments, we implement a redundancy check to identify
and merge similar talking points. This process involves assessing the
similarity between pairs of talking points and consolidating those that exceed
a predefined similarity threshold. We embed the talking points using Sentence
BERT and compute the cosine similarity score between the embedded talking
point pairs. Based on a threshold value ($\geq 0.70$), we decide to merge two
arguments. Here are the two examples of talking points where we decide to
merge them based on the threshold value- TP1: ”The Build Back Better Act is
crucial for economic growth, job creation, and addressing the climate
crisis.”; TP2: ”Legislative support for the Build Back Better Act, emphasizing
its benefits for clean energy jobs and climate action.”

In cases of high similarity, LLMs are prompted again in a zero-shot setting,
this time to generate a common talking point that encapsulates the essence of
the merged inputs. This step not only streamlines the argument discovery
process but also eliminates repetitive or overlapping content.

###  3.4 Human Evaluation

An optional human evaluation phase is incorporated to ensure the quality and
relevance of the generated talking points. This phase allows for the
verification of generated and refined talking points, ensuring their alignment
with human understanding and interpretation. The annotators are asked to
verify if the generated talking points and the merging decision are correct or
not. They are asked to provide a score of $1$ if the talking points are
correct and a score of $0$ if they are not.

###  3.5 Mapping Instances to Arguments

In this stage of our framework, we use a simple distance-based approach for
mapping. To measure the closeness between an instance and an argument, we
compute the cosine distance between the instance and all arguments for each
theme and take the minimum distance score among them. We embed the arguments
and instances using Sentence BERT. By thresholding, we determine if the
instance can be mapped to an argument.

Case Study | Iter. | # Args | Coverage | $\leq Q_{1}$ | $\leq Q_{2}$ | $\leq Q_{3}$ | All
---|---|---|---|---|---|---|---
Climate | 1 | 113 | 37.38% | 76.00% | 70.67% | 58.67% | 57.33%
2 | 213 | 44.40% | 88.00% | 74.67% | 70.67% | 64.00%
COVID-19 | 1 | 47 | 36.18% | 78.57% | 61.90% | 61.90% | 52.38%
2 | 78 | 40.47% | 82.93% | 73.81% | 64.29% | 57.14%
Table 1: Coverage and mapping quality w.r.t. Human Judgements.  Case | Iter. | Arguments
---|---|---
Study
Climate | 1 | (1) Prioritizing domestic energy production as a patriotic duty to
ensure America’s independence, security, and economic prosperity.
(2) Maintaining local oil and gas production to ensure economic and
energy security, highlighting patriotism through self-reliance.
(3) Supporting the oil and gas industry as a patriotic duty,
emphasizing its critical role in national security and economic health.
(4) Supporting natural gas to ensure energy independence and
consumer choice, highlighting patriotism through energy security
and freedom.
2 | (5) Rallying around patriotism, energy independence, and
conservative values to protect and promote American interests
and security.
(6) Maintaining local oil and gas production to ensure California’s
energy independence and protect consumers from rising fuel costs.
(7) Energy independence for Israel through faith-inspired and
scientifically supported oil and gas exploration and diverse
leadership in community engagement.
COVID | 1 | (1) Against price control legislation, emphasizing its potential to
-19 | hinder vaccine development and equitable access to life-saving
COVID-19 vaccinations.
(2) Global vaccine equity, emphasizing free and universal access
to COVID-19 vaccines to end the pandemic for all.
(3) Overcoming barriers to ensure equitable COVID-19 vaccine
access for underserved communities and prioritizing vulnerable
populations.
(4) Equitable access to COVID-19 vaccines, addressing eligibility,
distribution fairness.
2 | (5) Donating funds that will be matched to multiply impact,
promoting fair global access to vaccines.
(6) Efforts and challenges in equitable vaccine distribution and
access for seniors across various counties.
Table 2: Example of resulting arguments after $1^{st}$ and $2^{nd}$ iteration. For climate campaigns, arguments are shown under the ‘Patriotism’ theme. For COVID-19 vaccine campaigns, arguments are shown under the ‘VaccineEquity’ theme. All themes derived by previous studies are shown in Appendix A.1 (Table 5) Age Group | State |  Entity |  Talking Points
---|---|---|---
13-24 | CA |  Young People, Climate Change Denier, Climate Movement. |  (1) Climate change’s health impacts through informed initiatives and equity-focused efforts to protect vulnerable communities. (2) Collective action and education to combat climate change, emphasizing community involvement and urgent policy measures.
TX |  Natural Gas Systems, Coal Energy Systems, Nuclear Energy Systems, Wind Turbines, Solar Panels. |  (1) Ensure energy reliability and affordability, learning from Texas’ deregulation disaster.
25-54 | CA |  Carbon Emissions, Sacramento Municipal Utility District. |  (1) Public engagement to support and expand carbon-free energy as a crucial step against climate change. (2) Transitioning from coal to cleaner energy sources to combat climate change and promote sustainability.
TX |  Climate Change, Renewable Energy Plan. |  (1) Accelerating the transition to renewable energy to combat climate change and support economic growth through job creation.
55+ | CA |  Western Burrowing Owls, Aramis Industrial Solar Power Plant. |  (1) Continued investment in climate and clean energy solutions to protect birds from climate change impacts.
TX |  U.S. Senators or Congressman, Citizens’ Climate Lobby. |  (1) Urgent legislative action on climate change to enhance resiliency against extreme weather events and secure a sustainable future.
Table 3: Most mentioned entities and talking points of targeted ads for young, working-age, older population of CA and TX from climate campaigns dataset. Age Group | State |  Entity |  Talking Points
---|---|---|---
13-24 | TX |  Children, Parents, reproductive health. |  Advocates for the safety of COVID-19 vaccines for children, emphasizing mild side effects and community protection through vaccination.
25-54 | FL |  Ron DeSantis, Dr. Joseph Ladapo, Surgeon General. |  Advocates for building and restoring public trust in the COVID-19 vaccine and the medical community.
55+ | TX |  seniors, Pfizer, who have passed away or are hospitalized due to Covid, I. |  Strongly advocates for COVID-19 vaccination, highlighting its safety, efficacy, and crucial role in preventing severe illness and ending the pandemic.
FL |  Governor Ron DeSantis, seniors, loved one, Johnson & Johnson. |  Efforts and challenges in equitable vaccine distribution and access for seniors across various counties.
Table 4: Most mentioned entities and talking points of targeted ads for young,
working-age, older population of TX and FL from COVID-19 vaccine campaigns
dataset.

##  4 Case Studies

We explore two case studies involving discussions on social media: (1) climate
campaigns and (2) COVID-19 vaccine campaigns. For climate campaigns, we work
on the corpus of $14k$ ads released by Islam et al. (2023b). The dataset
focuses on climate-related English ads on Facebook based on the US from
January $2021$ to January $2022$. All ads in this corpus contain predictions
for stance (e.g., pro-energy, clean-energy) and theme (e.g., support climate
policy.) (Islam & Goldwasser, 2024). For COVID-19 vaccine campaigns, we use
the corpus of $9k$ ads released by Islam & Goldwasser (2022) focusing on
COVID-19 vaccine-related English ads on Facebook based on US from December
$2020$ to January $2022$. All ads in this corpus contain predictions for moral
foundation (e.g., care/harm) (Haidt & Graham, 2007) and theme (e.g., vaccine
equity.) (Islam & Goldwasser, 2022). For each ad of both corpora, there are
the following attributes: ad ID, title, ad description, ad body, funding
entity, spend, impressions, distribution over impressions broken down by
gender (male, female, unknown), age (7 groups), and location down to states in
the USA. Derived themes are shown in Table 5 in Appendix A.1. Additional
details about the dataset can be found in the original publication.

Our main goal is to use our LLMs-in-the-Loop framework to identify prominent
arguments in the corpora introduced above. At the beginning, we have ads and
their associated themes. Those themes are derived by Islam & Goldwasser (2024;
2022). At the first iteration, we follow the step introduced in Section 3 to
obtain the list of talking points under each theme and the mapping
$ads\rightarrow arguments$. After iteration $1$, we identify $113$ arguments
for Climate campaigns and $47$ arguments for COVID-19 vaccine campaigns (Table
1). In $2^{nd}$ iteration, we repeat this process for the rest of the ads
remaining unassigned from iteration $1$ (by applying a threshold of $<0.5$),
resulting in the discovery of $100$ and $31$ additional arguments for climate
and COVID-19 vaccine campaigns respectively (Table 1). An example of a
resulting set of arguments under the ‘Patriotism’ (one of the themes from
climate campaigns) and ‘VaccineEquity’ (one of the themes from COVID-19
vaccine campaigns) themes after two rounds of iterations of our LLMs-in-the-
Loop are shown in Table 2. The prompt templates are provided in Figure 3 in
Appendix A.2. The complete set of final arguments is provided in the Appendix
A.3 in Table LABEL:arg_climate and LABEL:arg_covid.

###  4.1 Evaluation

To evaluate the performance of our ad-to-argument mapping, we sort the ads
according to their semantic distance from their assigned arguments. We then
compute the three quartiles and sample a set of $12$ ads per theme, such that
$3$ ads are randomly sampled from each quartile. Subsequently, we manually
annotate whether the mapping is correct or not (detailed in Appendix A.4).
This process results in $300$ ads in the $1^{st}$ iteration and another $300$
ads from the $2^{nd}$ iteration from the climate case study. On the other
hand, from the COVID-19 vaccine case study, we obtain $168$ ads in the
$1^{st}$ iteration and another $168$ ads from the $2^{nd}$ iteration.

To assess the performance across varying degrees of semantic closeness to the
argument embeddings, we conduct evaluations within each quartile. Results for
the first quartile (Q1) correspond to the $25\%$ closest examples. For the
second quartile (Q2), they correspond to the $50\%$ closest examples, and for
the third quartile (Q3), to the $75\%$ closest examples. Intuitively, we
expect better performance in the lower distance between the ad and the
argument. Results are outlined in Table 1. We notice that we obtain higher
macro average F1-scores for ads that are the closest to the arguments in the
embedding space. In addition to the macro average F1-score, we also look at
the percentage of ads that are covered by the set of arguments uncovered by
our LLMs-in-the-Loop framework after each iteration. We do not enforce the
idea that all ads need to be mapped to arguments, and therefore, some ads
remain unassigned. There is an improvement in performance both in coverage and
mapping quality after subsequent iterations as we increase both the number of
arguments and the number of ads mapped (Table 1).

##  5 Analysis

This section provides a detailed analysis of how specific campaigns target
demographics and adjust messaging based on real event occurrences.

###  5.1 Demographic Targeting

Understanding how messages are tailored to target particular demographic
groups is crucial for opinion analysis. To analyze the variation in talking
points on climate campaigns across different age groups, we examine targeted
ads within three age categories, i.e., young people (ages $13$-$24$), working-
age people (ages $25$-$54$), and the older population (age $55+$), focusing on
California (CA) and Texas (TX). We choose these states because TX is
recognized as the world’s energy
capital222https://www.eia.gov/todayinenergy/detail.php?id=49356, while CA has
recently positioned itself as a leading advocate in the battle against climate
change333https://www.pewtrusts.org/en/research-and-
analysis/blogs/stateline/2022/10/06/california-takes-leading-edge-on-climate-
laws-others-could-follow. To identify the top-k most frequently mentioned
entities in the texts, we prompt LLMs in a zero-shot setting. We show the
results in Table 3. We observe that talking points vary by age group in TX.
For young people, the focus is on energy reliability and affordability,
learning from Texas’ deregulation disaster. For the working-age population,
the emphasis is on accelerating the transition to renewable energy to combat
climate change and support economic growth through job creation. Meanwhile,
for the older population, the discussion centers on the need for urgent
legislative action on climate change (Table 3).

(a) Event: Hurricane Ida hits on $29$ August $2021$.

(b) Event: Biden announced COVID-19 vaccine mandates for federal workers on
$09$ September $2021$.

Figure 2: The top-$4$ talking points used by advertisers before and after the
events.

We adopt a similar approach to analyze talking points across three different
age groups in COVID-19 vaccine campaigns. We select Florida (FL) because the
coronavirus surged to record levels there in August
$2021$444https://en.wikipedia.org/wiki/COVID-19_pandemic_in_Florida, and Texas
(TX) because, as of April $3^{rd}$, $2021$, its vaccination rates were lagging
behind the US
average555https://en.wikipedia.org/wiki/COVID-19_pandemic_in_Texas. Results
are detailed in Table 4. For the working age group in FL, the ad’s talking
point advocates for building and restoring public trust in the COVID-19
vaccine and medical community by criticizing FL Governor Ron DeSantis’s
appointment of Dr. Joseph Ladapo as Surgeon General, characterizing Ladapo as
an anti-masker and vaccine skeptic. The critique suggests that this
appointment is insensitive or disrespectful and emphasizes the need to appoint
someone who believes in science, not conspiracy theories. While targeting an
older population of FL, the talking point evolves around equitable vaccine
distribution and access for senior citizens (Table 4). On the other hand,
while targeting a young population of TX, the talking point focuses on
COVID-19 vaccine safety for children (Table 4).

###  5.2 Shifts in Talking Points Following Significant Events

To investigate how talking points change in response to real-world events, we
pick $2$ defining events ($1$ event per case study) from our case studies. The
events are as follows:

  * •

Hurricane Ida: On August $29$, $2021$, Hurricane Ida made landfall near Port
Fourchon, Louisiana, as a Category 4
hurricane666https://en.wikipedia.org/wiki/Hurricane_Ida.

  * •

Federal Vaccine Mandate: On September $09$, $2021$, President Biden announced
an executive order on COVID-19 vaccine mandates for federal workers, large
employers, and health care staff777https://www.whitehouse.gov/briefing-
room/presidential-actions/2021/09/09/executive-order-on-requiring-coronavirus-
disease-2019-vaccination-for-federal-employees/.

Figure 2(a) displays the top-four shifts in talking points in sponsored
advertisements over the five months before and after Hurricane Ida struck the
USA. There is a noticeable presence of arguments for ”urgent legislative
action on climate change to enhance resiliency against extreme weather events
and secure a sustainable future.” as well as advocacy for ”embracing solar
energy for its broad benefits, focusing on economic growth, environmental
protection, and enhanced accessibility.” following the hurricane (see Figure
2(a)). Figure 2(b) illustrates the talking points used in sponsored ads three
days before and after President Biden announced the federal vaccine mandate.
In the aftermath, three new predominant arguments emerged: ”Resistance against
federal vaccine mandates, emphasizing personal choice and legal action to
uphold individual rights.”, ”COVID-19 vaccine mandates, including recent
Supreme Court rulings and public opinion on federal mandates.”, ”Against
Biden’s vaccine mandate, portraying it as government tyranny and urging
resistance and financial support.” (Figure 2(b)). These shifts suggest that
sponsored content is highly responsive to real-world events, quickly
reflecting and potentially shaping public discourse on pivotal issues.

##  6 Conclusion

We introduce a simple yet effective iterative LLMs-in-the-Loop framework for
uncovering latent arguments from social media messaging. Our quantitative
results show that newly discovered talking points can cover a larger portion
of messaging and map instances to associated arguments accurately with respect
to human judgment. Additionally, our analysis highlights how these talking
points are specifically tailored for demographic targeting and dynamically
shift in response to real-world events.

##  7 Ethics Statement

To our knowledge, our research work, as described in this paper, adhered to
all ethical guidelines. Instead of introducing a new dataset, we conducted
experiments using two existing datasets, which have been properly cited. In
this work, we iterated the LLMs-in-the-Loop approach twice. In the future, we
aim to determine the optimal number of iterations and identify when to decide
to stop exploring unassigned instances. Our approach relies on GPT-4. We use
GPT-4 because it requires fewer computational resources compared to open-
source alternatives. The author’s personal views are not represented in any
qualitative result that we report.

## References

  * Achiam et al. (2023) Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al.  Gpt-4 technical report.  _arXiv preprint arXiv:2303.08774_ , 2023.
  * Barbu (2014) Oana Barbu.  Advertising, microtargeting and social media.  _Procedia-Social and Behavioral Sciences_ , 163:44–49, 2014.
  * Bernstein et al. (2010) Michael S Bernstein, Bongwon Suh, Lichan Hong, Jilin Chen, Sanjay Kairam, and Ed H Chi.  Eddi: interactive topic-based browsing of social status streams.  In _Proceedings of the 23nd annual ACM symposium on User interface software and technology_ , pp. 303–312, 2010.
  * Bhatti et al. (2021) Muhammad Mahad Afzal Bhatti, Ahsan Suheer Ahmad, and Joonsuk Park.  Argument mining on twitter: A case study on the planned parenthood debate.  In _Proceedings of the 8th Workshop on Argument Mining_ , pp. 1–11, 2021.
  * Blei et al. (2003) David M Blei, Andrew Y Ng, and Michael I Jordan.  Latent dirichlet allocation.  _the Journal of machine Learning research_ , 3:993–1022, 2003.
  * Bostrom et al. (2013) Ann Bostrom, Gisela Böhm, and Robert E O’Connor.  Targeting and tailoring climate change communications.  _Wiley Interdisciplinary Reviews: Climate Change_ , 4(5):447–455, 2013.
  * Brown et al. (2020) Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al.  Language models are few-shot learners.  _Advances in neural information processing systems_ , 33:1877–1901, 2020.
  * Capozzi et al. (2021) Arthur Capozzi, Gianmarco De Francisci Morales, Yelena Mejova, Corrado Monti, André Panisson, and Daniela Paolotti.  Clandestino or rifugiato? anti-immigration facebook ad targeting in italy.  In _CHI_ , 2021.
  * Chiang & Lee (2023) Cheng-Han Chiang and Hung-yi Lee.  Can large language models be an alternative to human evaluations?  In _The 61st Annual Meeting Of The Association For Computational Linguistics_ , 2023.
  * Chowdhery et al. (2023) Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, et al.  Palm: Scaling language modeling with pathways.  _Journal of Machine Learning Research_ , 24(240):1–113, 2023.
  * Dai et al. (2023) Shih-Chieh Dai, Aiping Xiong, and Lun-Wei Ku.  Llm-in-the-loop: Leveraging large language model for thematic analysis.  _arXiv preprint arXiv:2310.15100_ , 2023.
  * De Paoli (2023) Stefano De Paoli.  Can large language models emulate an inductive thematic analysis of semi-structured interviews? an exploration and provocation on the limits of the approach and the model.  _arXiv preprint arXiv:2305.13014_ , 2023.
  * Del Valle et al. (2020) Marc Esteve Del Valle, Rimmert Sijtsma, Hanne Stegeman, and Rosa Borge.  Online deliberation and the public sphere: Developing a coding manual to assess deliberation in twitter political networks.  _Javnost-The Public_ , 27(3):211–229, 2020.
  * Del Vicario et al. (2016) Michela Del Vicario, Alessandro Bessi, Fabiana Zollo, Fabio Petroni, Antonio Scala, Guido Caldarelli, H Eugene Stanley, and Walter Quattrociocchi.  The spreading of misinformation online.  _Proceedings of the national academy of Sciences_ , 113(3):554–559, 2016.
  * Dusmanu et al. (2017) Mihai Dusmanu, Elena Cabrio, and Serena Villata.  Argument mining on twitter: Arguments, facts and sources.  In _Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing_ , pp. 2317–2322, 2017.
  * Etudo et al. (2019) Ugo Etudo, Victoria Y Yoon, and Niam Yaraghi.  From facebook to the streets: Russian troll ads and black lives matter protests.  In _52nd Hawaii International Conference on System Sciences_ , 2019.
  * Freire-Vidal & Graells-Garrido (2019) Yerka Freire-Vidal and Eduardo Graells-Garrido.  Characterization of local attitudes toward immigration using social media.  In _Companion Proceedings of The 2019 World Wide Web Conference_ , pp. 783–790, 2019.
  * Garimella et al. (2018) Kiran Garimella, Gianmarco De Francisci Morales, Aristides Gionis, and Michael Mathioudakis.  Political discourse on social media: Echo chambers, gatekeepers, and the price of bipartisanship.  In _Proceedings of the 2018 World Wide Web Conference_ , pp. 913–922. International World Wide Web Conferences Steering Committee, 2018.
  * Gilardi et al. (2023) Fabrizio Gilardi, Meysam Alizadeh, and Maël Kubli.  Chatgpt outperforms crowd workers for text-annotation tasks.  _Proceedings of the National Academy of Sciences_ , 120(30):e2305016120, 2023.
  * Glynn & Huge (2008) Carroll J Glynn and Michael E Huge.  Public opinion.  _The international encyclopedia of communication_ , 2008.
  * Habernal et al. (2018) Ivan Habernal, Henning Wachsmuth, Iryna Gurevych, and Benno Stein.  The argument reasoning comprehension task: Identification and reconstruction of implicit warrants.  In _Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)_ , pp. 1930–1940, 2018.
  * Hagen et al. (2022) Loni Hagen, Ashley Fox, Heather O’Leary, DeAndre Dyson, Kimberly Walker, Cecile A Lengacher, and Raquel Hernandez.  The role of influential actors in fostering the polarized covid-19 vaccine discourse on twitter: Mixed methods of machine learning and inductive coding.  _Jmir Infodemiology_ , 2(1):e34231, 2022.
  * Haidt & Graham (2007) Jonathan Haidt and Jesse Graham.  When morality opposes justice: Conservatives have moral intuitions that liberals may not recognize.  _Social Justice Research_ , 20(1):98–116, 2007.
  * Haidt & Joseph (2004) Jonathan Haidt and Craig Joseph.  Intuitive ethics: How innately prepared intuitions generate culturally variable virtues.  _Daedalus_ , 133(4):55–66, 2004.
  * Han et al. (2020) Xuehua Han, Juanle Wang, Min Zhang, and Xiaojie Wang.  Using social media to mine and analyze public opinion related to covid-19 in china.  _International journal of environmental research and public health_ , 17(8):2788, 2020.
  * Hersh (2015) Eitan D Hersh.  _Hacking the electorate: How campaigns perceive voters_.  Cambridge University Press, 2015.
  * Hu et al. (2014) Yuening Hu, Jordan Boyd-Graber, Brianna Satinoff, and Alison Smith.  Interactive topic modeling.  _Machine learning_ , 95:423–469, 2014.
  * Islam & Goldwasser (2020) Tunazzina Islam and Dan Goldwasser.  Does yoga make you happy? analyzing twitter user happiness using textual and temporal information.  In _2020 IEEE International Conference on Big Data (Big Data)_ , pp. 4241–4249. IEEE, 2020.
  * Islam & Goldwasser (2022) Tunazzina Islam and Dan Goldwasser.  Understanding covid-19 vaccine campaign on facebook using minimal supervision.  In _2022 IEEE International Conference on Big Data (Big Data)_ , pp. 585–595. IEEE, 2022.
  * Islam & Goldwasser (2024) Tunazzina Islam and Dan Goldwasser.  Uncovering latent themes of messaging on social media by integrating llms: A case study on climate campaigns.  _arXiv preprint arXiv:2403.10707_ , 2024.
  * Islam et al. (2023a) Tunazzina Islam, Shamik Roy, and Dan Goldwasser.  Weakly supervised learning for analyzing political campaigns on facebook.  In _Proceedings of the International AAAI Conference on Web and Social Media_ , volume 17, pp. 411–422, 2023a.
  * Islam et al. (2023b) Tunazzina Islam, Ruqi Zhang, and Dan Goldwasser.  Analysis of climate campaigns on social media using bayesian model averaging.  In _Proceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and Society_ , AIES ’23, pp. 15–25, New York, NY, USA, 2023b. Association for Computing Machinery.  ISBN 9798400702310.  doi: 10.1145/3600211.3604665.  URL https://doi.org/10.1145/3600211.3604665.
  * Jamieson & Cappella (2008) Kathleen Hall Jamieson and Joseph N Cappella.  _Echo chamber: Rush Limbaugh and the conservative media establishment_.  Oxford University Press, 2008.
  * Jiang et al. (2020) Julie Jiang, Emily Chen, Shen Yan, Kristina Lerman, and Emilio Ferrara.  Political polarization drives online conversations about covid-19 in the united states.  _Human Behavior and Emerging Technologies_ , 2(3):200–211, 2020.
  * Jin & Han (2010) Xin Jin and Jiawei Han.  _K-Means Clustering_ , pp. 563–564.  Springer US, Boston, MA, 2010.  ISBN 978-0-387-30164-8.  doi: 10.1007/978-0-387-30164-8˙425.  URL https://doi.org/10.1007/978-0-387-30164-8_425.
  * Kim & Hovy (2006) Soo-Min Kim and Eduard Hovy.  Automatic identification of pro and con reasons in online reviews.  In _Proceedings of the COLING/ACL 2006 main conference poster sessions_ , pp. 483–490, 2006.
  * Kojima et al. (2022) Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa.  Large language models are zero-shot reasoners.  _Advances in neural information processing systems_ , 35:22199–22213, 2022.
  * Kotelnikov et al. (2022) Evgeny Kotelnikov, Natalia Loukachevitch, Irina Nikishina, and Alexander Panchenko.  Ruarg-2022: Argument mining evaluation.  _arXiv preprint arXiv:2206.09249_ , 2022.
  * Kruikemeier et al. (2016) Sanne Kruikemeier, Minem Sezgin, and Sophie C Boerman.  Political microtargeting: relationship between personalized advertising on facebook and voters’ responses.  _Cyberpsychology, Behavior, and Social Networking_ , 19(6):367–372, 2016.
  * Le Scao et al. (2022) Teven Le Scao, Angela Fan, Christopher Akiki, Ellie Pavlick, Suzana Ilić, Daniel Hesslow, Roman Castagné, Alexandra Sasha Luccioni, François Yvon, Matthias Gallé, et al.  Bloom: A 176b-parameter open-access multilingual language model.  2022\.
  * Lee & Seung (1999) Daniel D Lee and H Sebastian Seung.  Learning the parts of objects by non-negative matrix factorization.  _Nature_ , 401(6755):788–791, 1999.
  * Liang & Dai (2013) Po-Wei Liang and Bi-Ru Dai.  Opinion mining on social media data.  In _2013 IEEE 14th international conference on mobile data management_ , volume 2, pp. 91–96. IEEE, 2013.
  * Lima et al. (2018) Lucas Lima, Julio CS Reis, Philipe Melo, Fabricio Murai, Leandro Araujo, Pantelis Vikatos, and Fabricio Benevenuto.  Inside the right-leaning echo chambers: Characterizing gab, an unmoderated social system.  In _2018 ieee/acm international conference on advances in social networks analysis and mining (asonam)_ , pp. 515–522. IEEE, 2018.
  * Liu (2022) Bing Liu.  _Sentiment analysis and opinion mining_.  Springer Nature, 2022.
  * Lund et al. (2017) Jeffrey Lund, Connor Cook, Kevin Seppi, and Jordan Boyd-Graber.  Tandem anchoring: A multiword anchor approach for interactive topic modeling.  In _Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pp. 896–905, 2017.
  * Malagoli et al. (2021) Larissa G Malagoli, Julia Stancioli, Carlos HG Ferreira, Marisa Vasconcelos, Ana Paula Couto da Silva, and Jussara M Almeida.  A look into covid-19 vaccination debate on twitter.  In _Proceedings of the 13th ACM Web Science Conference 2021_ , pp. 225–233, 2021.
  * Maynard et al. (2012) Diana Maynard, Kalina Bontcheva, and Dominic Rout.  Challenges in developing opinion mining tools for social media.  _Proceedings of@ NLP can u tag# usergeneratedcontent_ , pp. 15–22, 2012.
  * McGregor (2019) Shannon C McGregor.  Social media as public opinion: How journalists use social media to represent public opinion.  _Journalism_ , 20(8):1070–1086, 2019.
  * Miller & Lellis (2016) Barbara M Miller and Julie Lellis.  Audience response to values-based marketplace advocacy by the fossil fuel industries.  _Environmental Communication_ , 10(2):249–268, 2016.
  * Murphy et al. (2014) Joe Murphy, Michael W Link, Jennifer Hunter Childs, Casey Langer Tesfaye, Elizabeth Dean, Michael Stern, Josh Pasek, Jon Cohen, Mario Callegaro, and Paul Harwood.  Social media in public opinion research: Executive summary of the aapor task force on emerging technologies in public opinion research.  _Public Opinion Quarterly_ , 78(4):788–794, 2014.
  * Nguyen et al. (2021) Thu T. Nguyen, Shaniece Criss, Eli K. Michaels, Rebekah I. Cross, Jackson S. Michaels, Pallavi Dwivedi, Dina Huang, Erica Hsu, Krishay Mukhija, Leah H. Nguyen, Isha Yardi, Amani M. Allen, Quynh C. Nguyen, and Gilbert C. Gee.  Progress and push-back: How the killings of ahmaud arbery, breonna taylor, and george floyd impacted public discourse on race and racism on twitter.  _SSM - Population Health_ , 15:100922, 2021.  ISSN 2352-8273.  doi: https://doi.org/10.1016/j.ssmph.2021.100922.  URL https://www.sciencedirect.com/science/article/pii/S235282732100197X.
  * Pacheco et al. (2022a) María Leonor Pacheco, Tunazzina Islam, Monal Mahajan, Andrey Shor, Ming Yin, Lyle Ungar, and Dan Goldwasser.  A holistic framework for analyzing the covid-19 vaccine debate.  In _Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pp. 5821–5839, 2022a.
  * Pacheco et al. (2022b) Maria Leonor Pacheco, Tunazzina Islam, Lyle Ungar, Ming Yin, and Dan Goldwasser.  Interactively uncovering latent arguments in social media platforms: A case study on the covid-19 vaccine debate.  In _Proceedings of the Fourth Workshop on Data Science with Human-in-the-Loop (Language Advances)_ , pp. 94–111, 2022b.
  * Pacheco et al. (2023) Maria Leonor Pacheco, Tunazzina Islam, Lyle Ungar, Ming Yin, and Dan Goldwasser.  Interactive concept learning for uncovering latent themes in large text collections.  In Anna Rogers, Jordan Boyd-Graber, and Naoaki Okazaki (eds.), _Findings of the Association for Computational Linguistics: ACL 2023_ , pp. 5059–5080, Toronto, Canada, July 2023. Association for Computational Linguistics.  doi: 10.18653/v1/2023.findings-acl.313.  URL https://aclanthology.org/2023.findings-acl.313.
  * Price (1988) Vincent Price.  On the public aspects of opinion: Linking levels of analysis in public opinion research.  _Communication research_ , 15(6):659–679, 1988.
  * Quattrociocchi et al. (2016) Walter Quattrociocchi, Antonio Scala, and Cass R Sunstein.  Echo chambers on facebook.  _Available at SSRN 2795110_ , 2016.
  * Reimers & Gurevych (2019) Nils Reimers and Iryna Gurevych.  Sentence-BERT: Sentence embeddings using Siamese BERT-networks.  In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_ , pp. 3982–3992, Hong Kong, China, November 2019. Association for Computational Linguistics.  doi: 10.18653/v1/D19-1410.  URL https://aclanthology.org/D19-1410.
  * Ribeiro et al. (2019) Filipe N Ribeiro, Koustuv Saha, Mahmoudreza Babaei, Lucas Henrique, Johnnatan Messias, Fabricio Benevenuto, Oana Goga, Krishna P Gummadi, and Elissa M Redmiles.  On microtargeting socially divisive ads: A case study of russia-linked ad campaigns on facebook.  In _ACM FAccT_ , 2019.
  * Skeppstedt et al. (2018) Maria Skeppstedt, Andreas Kerren, and Manfred Stede.  Vaccine hesitancy in discussion forums: computer-assisted argument mining with topic models.  In _Building Continents of Knowledge in Oceans of Data: The Future of Co-Created eHealth_ , pp. 366–370. IOS Press, 2018.
  * Sobkowicz et al. (2012) Pawel Sobkowicz, Michael Kaschesky, and Guillaume Bouchard.  Opinion mining in social media: Modeling, simulating, and forecasting political opinions in the web.  _Government information quarterly_ , 29(4):470–479, 2012.
  * Sowa et al. (2021) Paweł Sowa, Łukasz Kiszkiel, Piotr Paweł Laskowski, Maciej Alimowski, Łukasz Szczerbiński, Marlena Paniczko, Anna Moniuszko-Malinowska, and Karol Kamiński.  Covid-19 vaccine hesitancy in poland—multifactorial impact trajectories.  _Vaccines_ , 9(8):876, 2021.
  * Stab & Gurevych (2014) Christian Stab and Iryna Gurevych.  Identifying argumentative discourse structures in persuasive essays.  In _Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP)_ , pp. 46–56, 2014.
  * Torsi & Morante (2018) Benedetta Torsi and Roser Morante.  Annotating claims in the vaccination debate.  In _Proceedings of the 5th Workshop on Argument Mining_ , pp. 47–56, 2018.
  * Wawrzuta et al. (2021) Dominik Wawrzuta, Mariusz Jaworski, Joanna Gotlib, and Mariusz Panczyk.  What arguments against covid-19 vaccines run on facebook in poland: Content analysis of comments.  _Vaccines_ , 9(5):481, 2021.
  * Xiong & Liu (2014) Fei Xiong and Yun Liu.  Opinion formation on social media: an empirical approach.  _Chaos: An Interdisciplinary Journal of Nonlinear Science_ , 24(1), 2014.
  * Yousefinaghani et al. (2021) Samira Yousefinaghani, Rozita Dara, Samira Mubareka, Andrew Papadopoulos, and Shayan Sharif.  An analysis of covid-19 vaccine sentiments and opinions on twitter.  _International Journal of Infectious Diseases_ , 108:256–262, 2021.
  * Ziems et al. (2024) Caleb Ziems, William Held, Omar Shaikh, Jiaao Chen, Zhehao Zhang, and Diyi Yang.  Can large language models transform computational social science?  _Computational Linguistics_ , pp. 1–55, 2024.
  * Zuiderveen Borgesius et al. (2018) Frederik Zuiderveen Borgesius, Judith Möller, Sanne Kruikemeier, Ronan Ó Fathaigh, Kristina Irion, Tom Dobber, Balazs Bodo, and Claes H de Vreese.  Online political microtargeting: Promises and threats for democracy.  _Utrecht Law Review_ , 14(1):82–96, 2018.

##  Appendix A Appendix

###  A.1 Themes

Table 5 shows the derived themes from the climate campaign (Islam &
Goldwasser, 2024; Islam et al., 2023b) and COVID-19 vaccine campaign (Islam &
Goldwasser, 2022) datasets.

Climate |  Economy_pro, ClimateSolution, Pragmatism, Patriotism, AgainstClimatePolicy, Economy_clean, FutureGeneration, Environmental, HumanHealth, Animals, SupportClimatePolicy, AltEnergy, PoliticalAffiliation. BidenGasPriceIncrease, AgainstCorporateInterests, GasTax, Deforestation, Carbon, CustomerBasedAltEnergy, EnergyAffordabilityandSustainabilityLegislation, EcofriendlyConsumerChoices, PlasticWasteandEnvironmentalImpact, PromoteSustainableTransportation, WaterManagementandSustainability, FoodSecurity.
---|---
COVID-19 |  GovDistrust, GovTrust, VaccineRollout, VaccineSymptom, VaccineEquity, VaccineStatus, EncourageVaccination, VaccineMandate, VaccineReligion, VaccineEfficacy, VaccineDevelopment, CovidPlan, VaccineMisinformation, NaturalImmunity.

Table 5: Derived themes from previous studies: Climate campaigns (Islam &
Goldwasser, 2024; Islam et al., 2023b), COVID-19 vaccine campaigns (Islam &
Goldwasser, 2022)

###  A.2 Prompt Templates

In this section, we show the prompt templates used in our work to generate a
sub-cluster summary (Figure. 3 (a)), a talking point from each summarized sub-
cluster (Figure. 3 (b)), and a refined talking point (Figure. 3 (c)).

Figure 3: Prompt templates (shown as zero-shot).

###  A.3 Resulting Arguments

In this section, we provide the resulting arguments using our LLMs-in-the-Loop
approach for both datasets.

####  A.3.1 Climate Campaigns

Table LABEL:arg_climate provides the resulting arguments related to climate
campaigns after two rounds of iterations of our LLMs-in-the-Loop approach.

Table 6: Resulting arguments related to Climate campaigns after two rounds of iterations. Themes |  Talking Points
---|---
Economy_pro |  ”Against natural gas bans due to their potential economic harm to families and workers.”, ”Joining Energy Citizens to support the natural gas and oil industry’s economic impact and sustainability efforts.”, ”Acknowledging and supporting the oil and natural gas industry’s critical role in economic growth and job creation.”, ”Permian Basin’s significant contribution to the energy sector’s growth and the positive economic impact of increased drilling activity.”, ”Considering the economic and community impact of taxation on the energy sector, emphasizing support for energy-related jobs and services.”, ”The Line 3 Replacement project’s role in boosting Minnesota’s economy through job creation, supporting Tribal communities, and enhancing safety, while positively impacting the environment.”, ”Recognizing the oil and gas industry’s vital role in economic growth, while addressing challenges like automation and training the future workforce.”
ClimateSolution |  ”Adopting propane as a versatile and environmentally friendly energy solution to support climate change mitigation efforts.”, ”Leveraging innovative technologies and the synergy between natural gas and renewables as key strategies for climate change mitigation.”, ”Leveraging diverse, innovative, and market-based strategies to effectively address climate change and promote environmental sustainability.”, ”Adoption of RGGI in Pennsylvania, international cooperation on climate action, and exploring market-based, conservative solutions for sustainable economic and environmental health.”, ”Investing in innovative technologies like ICHOR’s RKI Injector and AT&T’s initiatives to drastically reduce emissions and improve fuel efficiency.”
Pragmatism |  ”Pragmatic, multi-tiered action on climate change, emphasizing renewable energy investments and policy for sustainable development and job growth.”, ”Practical investments in sustainable infrastructure and energy diversity to address climate change and ensure future resilience.”, ”Pragmatic approach to energy transition, emphasizing the need to harmonize environmental sustainability, economic growth, and job preservation.”, ”Recognizing and utilizing natural gas as a pragmatic, efficient energy solution for reliability and affordability.”, ”Pragmatic approach to energy policy that balances clean energy transition with economic growth and job preservation.”, ”Pragmatic approach to energy, balancing economic growth and environmental protection through Colorado’s oil and gas industry.”, ”Pragmatic approach to energy, emphasizing the need for oil and safe pipeline infrastructure for global energy security.”, ”Pragmatic policy changes and discussions that address workers’ rights, environmental sustainability, and societal challenges for improvement and growth.”, ”Pragmatic solutions to environmental challenges through technological innovation, collaboration, and strategic policy efforts to ensure sustainability and security.”
Patriotism |  ”Prioritizing domestic energy production as a patriotic duty to ensure America’s independence, security, and economic prosperity.”, ”Maintaining local oil and gas production to ensure economic and energy security, highlighting patriotism through self-reliance.”, ”Supporting the oil and gas industry as a patriotic duty, emphasizing its critical role in national security and economic health.”, ”Supporting natural gas to ensure energy independence and consumer choice, highlighting patriotism through energy security and freedom.”, ”Rallying around patriotism, energy independence, and conservative values to protect and promote American interests and security.”, ”Maintaining local oil and gas production to ensure California’s energy independence and protect consumers from rising fuel costs.”, ”Energy independence for Israel through faith-inspired and scientifically supported oil and gas exploration, and diverse leadership in community engagement.”
AgainstClimatePolicy |  ”Renewable energy investment as a strategy for economic growth and job creation while addressing climate change.”, ”Transitioning to a clean energy economy, emphasizing job creation, sustainability, and combating climate change for economic recovery.”, ”Significant investment in clean energy infrastructure to create jobs and power homes sustainably, driving economic and environmental progress.”, ”Passage of the bipartisan infrastructure deal to enhance clean water, transit, and create numerous good jobs.”, ”American Jobs Plan as a catalyst for economic revitalization through clean energy, creating jobs, and improving infrastructure.”, ”Enhancing sustainability through clean energy initiatives, education, infrastructure improvement, and worker health and safety in the clean energy sector.”, ”Local and state-level initiatives that promote clean energy, sustainability, and environmental protection to build a more sustainable future.”, ”Comprehensive community and environmental betterment through sustainable economic practices, fair wages, and innovative climate solutions.”, ”Rigorous scrutiny and equitable policies in clean-energy projects to ensure they genuinely benefit the environment and communities.”
Economy_clean |  ”Renewable energy investment as a strategy for economic growth and job creation while addressing climate change.”, ”Transitioning to a clean energy economy, emphasizing job creation, sustainability, and combating climate change for economic recovery.”, ”Significant investment in clean energy infrastructure to create jobs and power homes sustainably, driving economic and environmental progress.”, ”Passage of the bipartisan infrastructure deal to enhance clean water, transit, and create numerous good jobs.”, ”American Jobs Plan as a catalyst for economic revitalization through clean energy, creating jobs, and improving infrastructure.”, ”Enhancing sustainability through clean energy initiatives, education, infrastructure improvement, and worker health and safety in the clean energy sector.”, ”Local and state-level initiatives that promote clean energy, sustainability, and environmental protection to build a more sustainable future.”, ”Comprehensive community and environmental betterment through sustainable economic practices, fair wages, and innovative climate solutions.”, ”Rigorous scrutiny and equitable policies in clean-energy projects to ensure they genuinely benefit the environment and communities.”
HumanHealth |  ”Reducing air pollution from oil and gas industries to protect children’s health and ensure environmental justice.”, ”Electing knowledgeable leaders and supporting organizations to address climate change’s significant health and humanitarian impacts.”, ”Climate change’s health impacts through informed initiatives and equity-focused efforts to protect vulnerable communities.”, ”Leadership that understands and addresses the health impacts of climate change, emphasizing immediate action for global health and stability.”, ”Environmental health risks, from fracking and gas stoves to substance abuse, to protect and improve public health.”, ”Combining environmental activism with health equity to combat pollution and support sustainable, just communities.”, ”Awareness of gas stoves’ health risks, promoting safer alternatives, and implementing preventive measures to protect indoor air quality and prevent asthma.”, ”Informed, compassionate leadership and global medical support to address healthcare challenges, ensuring dignity and comprehensive care across life stages and crises.”, ”Addressing and mitigating air pollution sources to protect public health and promote environmental sustainability.”, ”Proactive engagement and informed responses to health and safety challenges exacerbated by environmental changes and societal issues.”
FutureGeneration |  ”Urgent environmental action and policy change to protect future generations from the health impacts of climate change.”, ”Educating children about climate change through open discussions to inspire future generations to take action for their planet’s health.”, ”Collective action led by motivated parents to combat climate change for a healthier future for children.”, ”Empowering the youth to lead climate change solutions, emphasizing education, activism, and support for vulnerable communities.”, ”Civic engagement and local policy reforms to secure a safer, fairer, and sustainable future for the next generation.”, ”Sustainable and renewable energy initiatives led by Indigenous and community efforts to ensure a healthier planet for future generations.”, ”Integrated solutions to climate change that also address social, racial, and economic inequities for future generations’ benefit.”, ”Sustainable energy and community initiatives that create educational, economic, and environmental benefits for current and future generations.”
Environmental |  ”Ending fossil fuel use, implementing clean energy transitions, and enacting protective regulations to combat climate change and pollution.”, ”Urgent legislative action on climate change to enhance resiliency against extreme weather events and secure a sustainable future.”, ”Collective action and education to combat climate change, emphasizing community involvement and urgent policy measures.”, ”Community resilience through local initiatives, transitioning to clean energy, sustainable practices, and comprehensive support for businesses impacted by environmental disasters.”, ”Environmental protection and Indigenous rights by opposing oil pipelines that threaten natural resources and exacerbate the MMIW epidemic.”, ”Vigilance and community action against organized crime targeting green energy resources, and promotes sustainable living practices.”, ”Proactive climate change solutions through innovative forestry, ecosystem restoration projects, clean energy investment, and urgent policy actions to safeguard the future.”, ”Climate change’s role in exacerbating droughts, fires, and heatwaves, underscored by scientific consensus and observable impacts like Minnesota’s rising temperatures.”, ”Active participation in environmental conservation efforts led by Derek Kilmer to restore Puget Sound, protect wilderness, and combat climate change.”
Animals |  ”Re-protection of gray wolves, recognizing their vital ecological role, and calls for increased conservation efforts and public support.”, ”Urgent financial contributions, leveraging donation matches to combat threats to endangered wildlife like habitat loss and climate change.”, ”Urgent climate action to protect Alaska’s fisheries, crucial for both the economy and marine ecosystems.”, ”Continued investment in climate and clean energy solutions to protect birds from climate change impacts.”, ”Immediate public action to conserve wildlife and address climate change’s role in accelerating species extinction.”, ”Habitat restoration and conservation to support wildlife, enhance outdoor activities, and address climate change for future ecological balance.”, ”Protecting and restoring ocean ecosystems to preserve marine biodiversity and combat climate change’s impacts on animals.”, ”Awareness and action against the environmental and ethical harms of animal agriculture, emphasizing sustainable choices and animal rights.”, ”Habitat conservation, creating pollinator gardens, and innovative habitat restoration to protect bats, bees, and wildlife crucial for ecosystem health and biodiversity.”, ”Promoting renewable energy projects as a means to generate clean power, support local economies, and protect marine biodiversity for future generations.”
AltEnergy |  ”Accelerating the transition to renewable energy to combat climate change and support economic growth through job creation.”, ”Promoting clean and renewable energy to foster economic growth, create jobs, and address climate change effectively.”, ”Public engagement to support and expand carbon-free energy as a crucial step against climate change.”, ”The adoption of wind energy as a crucial, effective solution for reducing CO2 emissions and combating climate change.”, ”Diversified renewable energy solutions to ensure sustainability, grid reliability, and meet increasing demand for clean power across different regions.”, ”Expansion and innovation in bioethanol production and solar energy to support a sustainable, clean energy future.”, ”The positive impact of wind energy, emphasizing job creation, economic growth, environmental benefits, and community engagement.”, ”Political support and action towards clean energy, job creation, climate action, and sustainability initiatives in local communities.”, ”Investment in ICHOR’s revolutionary fuel injection technology to reduce emissions and fuel consumption.”, ”Transitioning from coal to cleaner energy sources to combat climate change and promote sustainability.”, ”The adoption and celebration of renewable energy initiatives while balancing environmental preservation.”
SupportClimatePolicy |  ”Passing the Build Back Better Act, highlighting its importance for economic growth, job creation, clean energy employment, and combating climate change.”, ”Supporting the Growing Climate Solutions Act and the senators who champion this bipartisan environmental policy.”, ”Supporting the Growing Climate Solutions Act as a means to boost farmer income and combat climate change.”, ”Legislative action and public support for policies that address climate change through clean energy investment and promote social justice.”, ”The passage of the Build Back Better Act, emphasizing its investments in clean energy and climate action.”, ”Halting the Line 3 Pipeline construction to protect the climate, water resources, and Indigenous rights.”, ”Voting for candidates who prioritize climate change action and sustainable policies in their platforms.”, ”The inclusion of bold climate action in legislation, simultaneous with job creation and equity.”
PoliticalAffiliation |  ”Active political participation and advocacy to influence policies on environmental protection and social justice across various states.”, ”Exposing and combating the corrupt influence of the fossil fuel industry on politics to promote climate action.”, ”Appointments that prioritize environmental protection and resist the fossil fuel industry’s influence in government and finance.”, ”Holding fossil fuel companies accountable through political, public action, and legislative reforms for their environmental impact and misinformation.”, ”Greater scrutiny of political ties to the oil and gas industry and calls for fair regulation.”, ”Understanding and challenging perceived extremism in politics, scrutinizing shifts in political power and ideology, and recognizing ongoing political efforts on issues like infrastructure and climate change.”, ”Supporting candidates whose platforms align with individual voter values and community needs.”, ”Political appointments and decisions that prioritize climate action and reduce the influence of fossil fuels.”
BidenGasPriceIncrease |  ”Reversing President Biden’s energy policies to support American energy production and reduce gas prices for families.”, ”Against perceived financial mismanagement by ’DC liberals’, linking it to inflation and higher gas prices under Biden’s presidency.”, ”Mobilizing against government mandates perceived to increase gas prices, emphasizing grassroots engagement to protect consumer interests.”, ”Opposing a spending plan criticized for contributing to higher gas prices and economic burdens.”, ”Criticizes President Biden’s economic policies for significantly increasing gas prices and living costs, urging a shift in leadership or policy.”, ”A return to energy independence to control gas prices and criticizes current policies for rising fuel costs.”
AgainstCorpInter |  ”Supporting Sarah, who prioritizes climate action, honesty, and fiscal responsibility over corporate interests and misinformation.”, ”Grassroots support over corporate interests, emphasizing financial contributions to champion climate, democracy, and economic issues in Iowa’s race.”, ”Grassroots-supported candidates promoting progressive policies over those funded by corporate interests and lobbyists in Congressional races.”, ”Dismantling financial and political support for the fossil fuel industry to favor sustainable, clean energy solutions and accountability.”, ”Electing progressive candidates who prioritize public welfare and grassroots support over corporate interests and profits.”, ”Using the power of voting to challenge corporate interests by supporting policies and candidates focused on public welfare and environmental sustainability.”, ”Empowering grassroots movements over corporate interests, emphasizing small donations to drive significant policy changes for social and environmental justice.”, ”Active resistance against corporate influence in politics and society to achieve environmental, social, and economic reforms.”, ”Grassroots, community-focused leadership challenging established politics and corporate dominance to achieve progressive social and environmental reforms.”, ”Grassroots funding to counteract corporate-funded campaigns, emphasizing the power of collective small donations against big money interests.”
GasTax |  ”Reassessment of gas tax reliance, emphasizing modernizing revenue models to sustainably fund road maintenance amidst electric vehicle challenges.”, ”Halting proposed gas tax hikes to alleviate financial burdens on residents amidst concerns over rising living costs.”, ”Repealing the CPI portion of the gas tax law to alleviate economic burdens on citizens.”, ”Against implementing a per-mile tax for drivers, citing concerns over increased burdens and privacy issues.”, ”Opposing gas tax increases in Lake and Kane Counties, Illinois, citing high current gas taxes and economic challenges.”, ”Against a proposed fuel mandate in Iowa, framed as a tax on consumers leading to increased gas prices.”, ”Against the implementation of a new gas tax in Connecticut due to its potential financial burdens on families.”, ”The burden of state gas taxes on Californians amid inflation, advocating for measures to ease financial strain on residents.”, ”Lowering gas taxes as part of broader fiscal reforms to reduce the financial burden on citizens.”, ”Emphasizes the need for transparency, accountability, and community involvement in decisions impacting gas station operations and related policies.”
Deforestation |  ”Roadless protections in the Tongass National Forest to preserve its ecosystem, combat climate change, and support local economies and cultures.”, ”Leveraging consumer purchases for tree planting to combat deforestation and climate change, emphasizing individual contributions to collective environmental efforts.”, ”Global reforestation and individual action against climate change through tree planting and support for sustainable community practices to combat deforestation.”, ”Permanent legislative protection of the Arctic National Wildlife Refuge to preserve biodiversity and prevent environmental degradation from oil and gas extraction.”, ”Preserving urban green spaces and mature trees to maintain air quality, carbon sequestration, and historical landscapes against deforestation for construction projects.”, ”Combating deforestation and climate change through community-driven reforestation efforts and financial support in California.”, ”Stopping the use of forests as fuel for electricity and urging significant climate action to address and mitigate deforestation.”, ”Halting energy projects that cause deforestation, respecting treaties, and protecting ecosystems against corporate environmental damage.”, ”Eliminating conflict palm oil from products to protect rainforests and wildlife, promoting transparency and sustainable sourcing.”, ”Improved forest management and proactive measures like controlled burns to mitigate wildfire risks and prevent deforestation.”, ”Conservation, restoration efforts, and responsible management policies to combat climate change and prevent deforestation.”, ”Restoration and protection of critical habitats to combat deforestation and support ecosystem resilience and biodiversity.”, ”Indigenous-led efforts to protect natural resources and uphold sovereignty as crucial in combating deforestation and environmental degradation.”
Carbon |  ”The cement and concrete industry’s commitment to environmental sustainability and carbon neutrality to combat climate change by 2050.”, ”Accessible carbon offset solutions, emphasizing affordability, simplicity, and the importance of taking action for Earth Day.”, ”Using Ecologi’s platform to offset carbon footprint, plant trees, and collectively combat climate change.”, ”Implementing carbon pricing strategies through legislative action, communication, and collective efforts to address climate change.”, ”Collaborative efforts across various sectors to reduce carbon emissions and transition towards a sustainable, low-carbon economy.”, ”Innovative technologies and solutions to capture, reduce, and offset carbon emissions for a more sustainable and carbon-neutral future.”, ”Reducing carbon footprints through lifestyle changes, sustainable eating habits, and supporting initiatives aimed at carbon neutrality and environmental sustainability.”
CustBasedAltEnergy |  ”Customer engagement in alternative energy solutions, emphasizing renewable and nuclear options to combat climate change and promote sustainability.”, ”Public support and awareness of nuclear energy as a sustainable, carbon-free alternative energy solution in the U.S.”, ”Community engagement and support for wind energy, highlighting its benefits for jobs, rural economies, and sustainable development.”, ”Community engagement in renewable energy through a cost-effective, equipment-free Community Solar program with enrollment incentives.”, ”Homeowner participation in solar energy trials to foster renewable energy adoption and reduce individual energy costs.”, ”Embracing solar energy for its broad benefits, focusing on economic growth, environmental protection, and enhanced accessibility.”, ”The adoption and support of utility-scale solar projects to supply clean energy to the grid for consumers.”, ”Embracing wind and nuclear power as key to Illinois’ energy strategy, highlighting job creation and environmental benefits.”, ”Adopting biofuels and innovative fuel technologies to reduce emissions and promote sustainability in transportation for consumers.”, ”Community engagement and legal actions to promote sustainable development and energy practices at the local level.”, ”Adopting geothermal heating and cooling as a cost-effective, efficient, and environmentally friendly alternative energy solution for homeowners.”
FoodSecurity |  ”Addressing climate change’s impact on agriculture, social equity, and women’s rights to ensure food security for all.”, ”Cross-sector collaboration, addressing food loss, engaging communities, and prioritizing food security initiatives to combat global hunger.”, ”Participation in the Ration Challenge to support emergency food aid and attending a forum to explore food security issues.”, ”Engagement with the Fruitful Communities initiative to combat climate change, promote sustainable agriculture, and enhance food security.”, ”Urgent humanitarian aid and community support to address immediate and long-term food security challenges in crisis-affected areas.”, ”Sustainable dietary changes, particularly reducing meat consumption, to improve environmental health and ensure long-term food security.”, ”Community support and mobilization to provide essential resources and aid, ensuring food security and health during crises.”, ”Combining sustainable agriculture, traditional knowledge, and community efforts to improve food security and environmental sustainability.”
EnrgyAffrd&SustainLeg |  ”Legislative action in Virginia to ensure energy reliability and affordability, learning from Texas’ deregulation disaster.”, ”Legislative action to significantly lower energy costs, promoting cleaner and more reliable energy solutions in North Carolina.”, ”Legislative and policy efforts to enhance energy efficiency, grid reliability, and sustainability across various regions.”, ”Legislative supporting utility bill assistance, pollution reduction, energy-efficient building codes, and transition to pollution-free buildings for sustainability.”, ”Policy-driven support for renewable energy, affordability, emissions reduction, and sustainable development, emphasizing community involvement and legislative backing.”, ”Legislative action to ensure utility service protection, improve energy infrastructure resilience, and address affordability amid extreme weather and economic challenges.”, ”Transitioning to sustainable energy sources, enhancing security for clean energy businesses, and supporting policies for cleaner, affordable energy solutions.”, ”Leveraging legislation and community programs to provide financial assistance for utility bills, ensuring energy affordability for all residents.”
EcofrndlyConsmChoice |  ”Making eco-friendly consumer choices by purchasing sustainable apparel and actively contributing to environmental conservation efforts.”, ”Choosing eco-friendly, sustainable laundry solutions like Earthbreeze to reduce environmental impact and support ethical consumption.”, ”Adopting home water treatment systems to reduce reliance on single-use plastic bottles, promoting environmental sustainability and health.”, ”Making eco-friendly consumer choices by using Cleanyst’s plant-based products to reduce plastic waste and carbon emissions.”, ”Empowering communities and individuals through education and involvement in sustainable practices to combat climate change effectively.”, ”Embracing sustainable living through eco-friendly consumer choices, corporate sustainability efforts, and supporting green energy and conservation initiatives.”, ”Choosing sustainable, skin-friendly products like bamboo bandages as a simple yet effective way to support eco-friendly consumer choices.”, ”Supporting local leadership and community initiatives that promote sustainability, equitable growth, and eco-friendly consumer practices in urban development.”
PlasticWste&EnvImpact |  ”Recognizing plastic’s sustainability benefits, urging scientific studies and informed policies to optimize its use and minimize environmental impact.”, ”Reducing plastic waste through reusable products, supporting cleanup efforts, and embracing innovative solutions to protect marine ecosystems.”, ”Using pyrolysis technology to recycle waste plastic, supporting a circular economy and environmental protection efforts.”, ”Urgent collective action, habit changes, and support for organizations fighting plastic pollution to protect oceans and marine life.”, ”Eengaging brands towards Plastic Neutrality and supports innovative recycling technologies to combat the global plastic waste crisis.”, ”Daily action against plastic waste, supporting legislation, and making conscious choices in food consumption and waste management.”, ”Responsible waste management, recycling efforts, and addressing illegal activities to mitigate environmental contamination and promote sustainability.”
PromtSustainTrnsport |  ”Embracing sustainable transportation and clean energy through EV test drives and educational events on innovative energy solutions.”, ”Attending Drive Electric Earth Day events to learn about and embrace the benefits of electric vehicles for a sustainable future.”, ”Adopting cycling and supporting policies for electric school buses and a bold climate bill for sustainable transportation.”, ”Urgent investment in electric vehicles and legislative support to reduce emissions and create union jobs.”, ”Adopting electric vehicles for their environmental benefits, cost savings, and the expanding availability of models.”, ”Adopting clean car standards and investing in EV infrastructure to improve air quality, health, and combat climate change.”, ”Significant investment in public transportation electrification and infrastructure to support climate goals, community benefits, and union job creation.”, ”Adopting and promoting sustainable transportation solutions and technologies to improve air quality and address environmental challenges.”, ”Infrastructure investments and community initiatives that reduce emissions and promote sustainable transportation as essential to combating climate change.”, ”Adopting biking, walking, electric vehicles, and public transit as key strategies to reduce emissions and improve air quality.”, ”Shifting to public and active transportation, electric vehicles, and renewable energy to reduce emissions and enhance sustainability.”
WaterMngmnt&Sustain |  ”Stricter regulations on coal ash disposal to protect water quality and public health in Georgia.”, ”Supporting legislative efforts to protect and restore Colorado’s rivers for economic growth, climate resilience, and agricultural preservation.”, ”Exploring and adopting successful global strategies to address water scarcity and ensure sustainable water management in rapidly growing states.”, ”Sustainable water management through refurbishing existing facilities and adopting eco-friendly water solutions to protect health and the environment.”, ”Proactive water management strategies, infrastructure investment, and community involvement to ensure sustainable water access and conservation efforts.”, ”Responsible disposal of hazardous wastes like oils and grease to protect water sources and promote environmental sustainability.”, ”Innovative financing, policy reforms, and sustainable practices to ensure water security and resilience against climate change impacts.”, ”Active engagement and informed leadership in water board governance to ensure sustainable water management and protection.”

####  A.3.2 COVID-19 Vaccine Campaigns

Table LABEL:arg_covid provides the resulting arguments related to COVID-19
vaccine campaigns after two rounds of iterations of our LLMs-in-the-Loop
approach.

Table 7: Resulting arguments related to COVID-19 vaccine campaigns after two rounds of iterations. Themes |  Talking Points
---|---
GovDistrust |  ”Accountability and reform in the Biden administration, criticizing its policies for undermining national stability and public trust.”, ”Against government control of the prescription drug market, warning of hindered medical innovation and reduced access to cures.”, ”Skepticism towards government actions and legislation, emphasizing the need for transparency and resistance to perceived overreach.”, ”Skepticism towards the Biden administration’s handling of COVID-19, immigration, and economic policies.”, ”Questions the efficacy and motives of government policies on immigration, economy, and public health measures.”
GovTrust |  ”Recognizing and appreciating government efforts and collaboration in pandemic response, emphasizing trust in legislative support for public health.”, ”Recognizing the Biden administration’s efforts to address key national and global issues, building trust through proactive policies and actions.”, ”Political engagement and trust through transparency, community involvement, and proactive governance initiatives.”, ”Building trust through community engagement, open dialogue, and collaboration between citizens and their representatives.”, ”President Biden’s commitment to international accords, public health, economic recovery, and progressive policy achievements to build government trust.”
VaccineRollout |  ”The widespread and accessible COVID-19 vaccination and testing services, emphasizing no-appointment-necessary clinics to increase public participation.”, ”Utilizing digital platforms to access up-to-date information on COVID-19 vaccine availability and registration across different locations.”, ”Easy and accessible COVID-19 vaccination for all eligible individuals, emphasizing convenience and inclusivity in public health efforts.”, ”The organized distribution of COVID-19 vaccines across Texas, highlighting state efforts to enhance accessibility and efficiency in vaccination.”, ”The importance of local news in keeping communities informed about the evolving COVID-19 vaccine rollout and registration processes.”, ”Accessible vaccine clinics, incentive programs, and safety protocols to enhance vaccine rollout at the local level.”
VaccineSymptom |  ”Emphasizes the safety and effectiveness of COVID-19 vaccines despite mild side effects, advocating for vaccination to protect against the virus.”, ”Vigilance and thorough investigation into the rare but serious side effects like blood clots associated with the J&J vaccine.”, ”Careful monitoring and transparent communication about potential vaccine side effects to ensure public safety and confidence.”, ”Informed vaccine choices, awareness of rare side effects, and the importance of continued vaccination for public health safety.”, ”Awareness and transparency regarding the range and prevalence of COVID-19 vaccine-related symptoms and effects.”, ”Advocates for booster shots to enhance immunity, addressing vaccine safety and mild side effects experienced post-vaccination.”, ”Advocates for the safety of COVID-19 vaccines for children, emphasizing mild side effects and community protection through vaccination.”
VaccineEquity |  ”Against price control legislation, emphasizing its potential to hinder vaccine development and equitable access to life-saving COVID-19 vaccinations.”, ”Global vaccine equity, emphasizing free and universal access to COVID-19 vaccines to end the pandemic for all.”, ”Overcoming barriers to ensure equitable COVID-19 vaccine access for underserved communities and prioritizing vulnerable populations.”, ”Investigating and improving the distribution and administration process to ensure equitable vaccine access in Illinois.”, ”Equitable access to COVID-19 vaccines, addressing eligibility, distribution fairness.”, ”Donating funds that will be matched to multiply impact, promoting fair global access to vaccines.”, ”Efforts and challenges in equitable vaccine distribution and access for seniors across various counties.”
VaccineStatus |  ”Raising awareness about the low vaccination rates in Central Texas counties, as indicated by state data.”, ”Staying informed about COVID-19 vaccination updates and initiatives through trusted local news apps in respective counties.”, ”Accessing COVID-19 vaccine information and locations for vaccination through the SmartNews app for free.”, ”Highlights the concern over the high number of unvaccinated individuals in Central Texas counties according to state data.”
EncourageVaccination |  ”Promoting vaccination by providing information, guidance, and incentives to encourage people to get vaccinated against COVID-19.”, ”Importance of COVID-19 vaccination by emphasizing its safety, efficacy, availability, and role in ending the pandemic.”, ”Engaging in community discussions and raising awareness about vaccination, especially within the LatinX community.”, ”Encourages COVID-19 vaccination and boosters for all ages to protect against the virus and maintain community health.”, ”Promoting COVID-19 vaccination for children and the community, emphasizing safety, accessibility, and the benefits of vaccination.”
VaccineMandate |  ”Against COVID vaccine mandates and passports, urging support for personal choice and constitutional rights regarding vaccination.”, ”Awareness and updates on issues related to COVID-19 vaccine mandates and refusals in various states.”, ”COVID-19 vaccine mandates, including recent Supreme Court rulings and public opinion on federal mandates.”, ”Against Biden’s vaccine mandate, portraying it as government tyranny and urging resistance and financial support.”, ”Advocating for Covid-19 vaccine mandates for customers by Snowy Range Taxi in Wyoming to prioritize community safety.”, ”Awareness and preparation among employers for the impending COVID-19 vaccine mandate.”, ”Against federal COVID-19 vaccine and mask mandates, framing them as overreach and questioning their constitutionality.”, ”Resistance against federal vaccine mandates, emphasizing personal choice and legal action to uphold individual rights.”, ”Vaccine documentation management, vaccine refusal, job consequences for non-compliance, and HIPAA implications in vaccine proof requests.”, ”Advocates for personal choice and against federal vaccine mandates, viewing them as infringements on civil liberties and rights.”
VaccineReligion |  ”Using religion as a means to exempt individuals from mandatory COVID-19 vaccination requirements.”, ”Incorporating religious teachings into actions promoting community well-being, including vaccination and pandemic safety measures.”, ”Vaccination as an act of love and protection towards neighbors, aligning it with Christian values.”, ”Seeking religious guidance and perspective on COVID, vaccination, and related moral concerns with Pastor Bunjee Garrett.”, ”Religious communities to participate in public health by supporting vaccination as an act of neighborly love and care.”
VaccineEfficacy |  ”The safety and efficacy of the Moderna COVID-19 vaccine, but acknowledges instances of post-vaccination infections.”, ”Getting vaccinated, highlighting the vaccine’s effectiveness in significantly reducing infection, severity, and mortality from the Delta variant of COVID-19.”, ”Strongly advocates for COVID-19 vaccination, highlighting its safety, efficacy, and crucial role in preventing severe illness and ending the pandemic.”, ”The prioritization strategy for COVID-19 vaccinations in certain highly infected North Carolina counties, addressing vaccine efficacy concerns.”, ”Promoting vaccination as an effective measure to protect individuals and families from COVID-19, with community endorsement.”, ”Highlights that vaccinations, including boosters, are highly effective in preventing severe illness, hospitalization, and death from COVID-19.”, ”Advocates for vaccination and continued safety measures to protect against the transmissible and potentially severe Delta variant.”
VaccineDevelopment |  ”Trust in COVID-19 vaccines, highlighting their development from decades-old, scientifically robust messenger RNA technology.”, ”Confidence in the Pfizer-BioNTech COVID-19 vaccine, emphasizing its emergency use authorization and pending full FDA approval.”, ”Understanding the distinct characteristics and allocation strategies of the newly approved Johnson & Johnson COVID-19 vaccine.”
CovidPlan |  ”Staying informed about regional COVID-19 vaccine distribution plans and understanding challenges in meeting demand.”, ”Recognizing and supporting the American Rescue Plan and lawmakers’ efforts in providing pandemic relief.”, ”Investigating and addressing the apparent inefficiencies in the COVID-19 vaccine distribution in Illinois.”, ”Commending the collaborative development of COVID-19 vaccines and treatments supported by the American Rescue Plan.”, ”Providing COVID-19 vaccinations to residents through the #VaxMobile program in different locations throughout the township.”
VaccineMisinformation |  ”Correcting vaccine misinformation and encouraging vaccine participation through expert-led educational conversations.”, ”Debunking vaccine myths and empowering people with credible and accurate information about COVID-19 vaccines.”, ”Holding news outlets accountable for spreading potentially harmful misinformation about Covid-19 vaccines.”, ”Clear communication and spreading accurate information about COVID-19 vaccines to counter misinformation.”, ”Utilizing virtual town hall meetings to combat vaccine misinformation by providing accurate, expert information.
NaturalImmunity |  ”Changes in herd immunity goals due to the Delta variant.”, ”Natural immunity against COVID-19 could be better than vaccine-induced immunity”, ”Boosting the immune system naturally with supplements as an alternative approach to handling COVID-19.”

###  A.4 Human Validation

Three NLP and CSS researchers, age range $30$-$45$, $2$ males and $1$ female,
did two $1$-hour sessions to validate whether the generated and refined
talking points as well as to annotate whether $ads\rightarrow arguments$
mapping is correct or not. The annotators included advanced graduate students
and faculty. Our study was approved by the Institutional Review Board (IRB),
and we followed their protocols.