Research

# DrawL: Understanding the Effects of Non-Mainstream Dialects in Prompted
Image Generation

Joshua N. Williams<EMAIL_ADDRESS>Carnegie Mellon
UniversityPittsburghPennsylvaniaUSA ,  Molly Fitzmorris  GoogleMountain
ViewCaliforniaUSA ,  Osman Aka  Google ResearchMountain ViewCaliforniaUSA and
Sarah Laszlo  Google ResearchMountain ViewCaliforniaUSA

###### Abstract.

Text-to-image models are now easy to use and ubiquitous. However, prior work
has found that they are prone to recapitulating harmful Western stereotypes.
For example, requesting that a model generate an "African person and their
house," may produce a person standing next to a straw hut. In this example,
the word "African" is an explicit descriptor of the person that the prompt is
seeking to depict. Here, we examine whether implicit markers, such as dialect,
can also affect the portrayal of people in text-to-image outputs. We pair
prompts in Mainstream American English with counterfactuals that express
grammatical constructions found in dialects correlated with historically
marginalized groups. We find that through minimal, syntax-only changes to
prompts, we can systematically shift the skin tone and gender of people in the
generated images. We conclude with a discussion of whether dialectic
distribution shifts like this are harmful or are expected, possibly even
desirable, model behavior.

Image Generation, Language Bias, Dialect

##  1\. Introduction

There has been a recent surge of interest in generative, multimodal machine
learning (ML) models, in many domains, such as text-to-text , image-to-image,
image-to-text, and text-to-image. Text-to-image models such as Stable
Diffusion (Rombach et al., 2022) and Dall-E (Ramesh et al., 2021) have been
especially conspicuous due to their public release. With these models
available broadly to users, individuals can generate novel high-quality images
from natural-language descriptions. Yet others, such as ChatGPT (Brown et al.,
2020) have allowed users to more freely interact with high-quality
conversational agents in daily use.

During this period of growing interest in and adoption of text-to-image
models, it is important to understand the characteristics of the images that
they produce, and how these images may be unfair or harmful– in order to
enable research that minimizes the harm of these models. It is well
established that when models are trained on large datasets that contain
harmful content, such as stereotypes or toxic imagery, such content will be
reproduced in the resultant model’s behavior (Pessach and Shmueli, 2022). As
we will discuss in Section 3, Text-to-image models are no exception (Bianchi
et al., 2023). Understanding (and eventually, mitigating) these effects in the
image generation domain is particularly pressing as prior work has shown that
repeated exposure to imagery that encodes this harmful content is a predictor
of discrimination, hostility, and justification of violence (Amodio and
Devine, 2006; Goff et al., 2008; Slusher and Anderson, 1987).

Prompt engineering (Oppenlaender, 2022) is one potential vector for
investigating and addressing harmful model behavior. Prompt engineering is a
popular approach because it can be applied post-model-training, and is
therefore less expensive and time-consuming than mitigative approaches that
require architectural or training data changes. For example, such techniques
have been used by Dall-E 2, where the company reported found that users were
12x more likely to say that the images included people of diverse backgrounds
without retraining the model (OpenAI, 2022). Several studies (Bianchi et al.,
2023; Reviriego and Merino-Gómez, 2022), have used prompt engineering to
understand how text-to-image models respond to socially salient information.
For example, Bianchi et al. (2023), have analyzed the effects of varying
identity markers in prompts provided to Stable Diffusion. They found that
prompts such as, “an African man and his house” may elicit very different
responses than, “an American man and his house”. The former showing a mud,
straw-covered hut and the latter showing an upper-middle class western
residence. On the other hand, Si et al. (2022) have found that well-engineered
prompts for conversational agents can improve their generalization to out of
distribution prompts and reduce social biases in their responses. Here, we
extend such investigations to examinations of whether prompts that do not
explicitly mention an identity group, but rather are composed in a dialect
that implicitly codes for a particular group of speakers, have a similar
effect.

As this area of study grows, we are also seeing discourse around appropriate
design considerations to account for the experiences of users among a variety
of backgrounds and communities. Much of this work focuses on the relationship
between language families and conversational agents (Harrington et al., 2022;
Garg et al., 2022; Choi et al., 2023). As such prior work has found that users
modify their speech when communicating with conversational agents, we
investigate whether these dialect changes influence the downstream performance
of the model, by both extending such work into the text-to-image domain and
providing additional color to prior work that analyzes ways in which models
may be sensitive to unique patterns in language across cultural or social
groups. We focus on determining whether image-generation models are
conditioning their output space on not only the explicit request, but also the
implicit demographic information present in the language patterns across
communities (i.e., the dialect of the users).

Such an effect, if observed, would evidence remarkable pragmatic sensitivity
in the models’ internal representations of people. For example, some English-
language dialects permit dropping the copula, transforming phrases, such as,
“a man who _is_ going to the store” to “a man who ____ going to the store”.
This construction, known as the null copula or “deletion of the copula” is
pervasive in African American English (AAE) and many English-based creoles and
pidgins (Parsard, 2016) found throughout the Caribbean and West Africa. These
languages generally originated in speech communities within former British
colonies (Mufwene, 2015), with speakers that were generally darker-skinned
than their colonizers. If image generation models have learned a correlation
between skin tone and languages like AAE and English creoles, then, that would
allow prompts without the copula to act as proxy requests for darker skin tone
in the generated image–even though the copula itself carries no explicit
demographic information.

In order to investigate potential dialectic effects, we engineer a set of
prompts for which image generation models are expected to produce images of
people. The prompts do not explicitly reference demographic, physical, or
social characteristics of the people to be generated– these characteristics
are left up to the model to decide. We examine how the model decides to depict
people when their characteristics are not explicitly specified, by focusing on
the skin tones of the people generated by the model. Given an initial set of
baseline prompts, we select nine grammatical constructions that are pervasive
in African American English (AAE), and generate a matched set of
counterfactual prompts that only differ by the minimal changes required to
express each grammatical construction, (e.g., the null copula example above).
We then generate four images with Stable Diffusion111All images were generated
with CompVis/stable-diffusion-v1-4 checkpoint
(https://github.com/CompVis/stable-diffusion) for both the baseline and
counterfactual prompts, before applying an automated skintone annotator in
order to provide a quantitative comparison of the skin tone of individuals
generated with each prompt type according to the Monk Skin Tone Scale (Monk,
2023).

Figure 1. The Monk Skin Tone Scale. The skin tones of humans generated by the
model are annotated with a score of 1 (lightest) to 10 (darkest). In this
work, we measure how the distribution of skin tones generated by the model
changes when prompting in African American English (AAE) as opposed to
Standard American English (SAE)

Our contributions are as follows: 1) We extend existing work that evaluates
the impact of language variation from marginalized groups on conversational
agents into the text-to-image domain 2) We present a method for designing
analyses to measure the impact of a user’s dialect on a model through
contrastive prompting 3) We construct a dataset of 1821 baseline and 1821
counterfactual prompts with this framework for future work to examine the
impact of AAE on the behavior of text-prompted models 222The dataset used in
this research is included at the following github repository:
https://github.com/jnwilliams/DrawL.git, and 4) We show that applications of
minimal non-mainstream constructions to otherwise semantically equivalent
prompts changes the output distribution of skin tones in people generated by
text-to-image models. This finding demonstrates that even minor, closed-class
wording changes that correlate with social information are responded to
sensitively by the models. We then discuss the implications of the effects we
observe with respect to a harm-reduction framework (Shelby et al., 2023).

##  2\. Background

Throughout this work, we use the term dialect as an umbrella term to refer to
any variety of a language, usually thought of as pertaining to some group of
speakers, like Bostonian English or Appalachian English. Linguists often
further subdivide the term “dialect” into sociolects, that describe language
varieties spoken by people in a particular social group, and ethnolects, that
describe varieties spoken by members of a particular ethnic group. Here,
African-American English is a dialect, a sociolect, and an ethnolect. Among
dialects, languages generally have a socially accepted or favored “mainstream”
dialect and all other variants that contain features that differ from the
mainstream are “non-mainstream”; linguists have long agreed unequivocally that
these “non-mainstream” dialects are fully formed language varieties with
grammar rules that govern their use.

In line with both the broader sociolinguistic literature and prior work at the
intersection of linguistics and computing (Harrington et al., 2022; Gutierrez
and Ochoa, 2016), we refer to this “mainstream” dialect as Mainstream American
English (MAE). This phrasing is not meant to convey any degree of correctness
in language; it is important to reiterate that the misguided ideas that
African American English is “incorrect” or otherwise an inferior or invalid
language variety reflect ongoing structural inequities.

Sociolinguistic variation – that is, differences between language varieties –
is inherent to human language. Dialects of a language can differ from one
another lexically (in the vocabulary), syntactically (in the sentence
structure), phonetically or phonologically (in the sound system and
pronunciation), or morphologically (in the way words are formed). The English
language comprises numerous dialects which differ in a variety of ways,
including lexicon, syntax, phonetics, phonology, morphology in both vocabulary
and syntax (Wolfram and Schilling, 2015). Sociolects– the focus of this
investigation– have been the subject of an enduring body of literature that
investigates how linguistic variation correlates with identity groups within a
speech community (Clopper, 2017; Luhman, 1990). Our work is in alignment with
prior work that has studied how identity markers can be expressed through
language data and how they interact with human biases to cause harmful
outcomes (Pessach and Shmueli, 2022).

##  3\. Related Work

In order to better frame our investigation, we first explore and situate our
work within the broader space of research that studies the impact of language
on automated systems. We then make specific connections to generative modeling
and the ways that a user’s dialect can affect their experience. Throughout our
work, we also consider potential harms associated with the effects we study,
referencing the harms taxonomy proposed in (Shelby et al., 2023). This
taxonomy classifies algorithmic system harms into five categories:
Representational Harms (stereotyping, demeaning, or erasing social groups);
Allocative Harms (opportunity or economic loss); Quality of Service Harms
(alienation or increased user labor); Interpersonal Harms (tech-facilitated
violence or diminished user well-being); and Social System Harms (cultural,
civic, political, and socio-economic harms).

###  3.1. Language and Model Interactions

As multimodal systems and conversational agents in particular have grown,
researchers have both studied the effects of users’ language on the systems
(Choi et al., 2023) and found ways to incorporate these models in the research
itself in order to support novel work (Hämäläinen et al., 2023). In doing so,
such studies have highlighted similar issues to those found in the broader
issues of bias in natural language. Namely, that models are not agnostic to
the cultural and ethnic connotations behind the language used by users.

Work at this intersection of natural language processing (NLP) and fair ML
includes the study of sociolinguistic bias, and often specifically bias along
dialects spoken by particular racial groups – for example, the finding that
harmful tweet detection does not predict harmful content equally well in
“White English” and AAE, and therefore perpetuates a racial dialect bias
(Ball-Burack et al., 2021). Commonly, work that investigates the effect of
dialect on NLP applications focuses on improving accessibility of annotated
data for non-mainstream dialects, with the assumption being that if annotated
data are more available, cross-dialectic fairness will follow. For example,
(Bouamor et al., 2014) have introduced a multi-dialectical Arabic corpus that
includes statements in a variety of dialects in parallel. Similarly,
(Hollenstein and Aepli, 2015) have introduced a large language corpus with
hand-labeled part-of-speech tags for Swiss-German dialects and (Aguilar et
al., 2017) have introduced a new corpus for the Ecuadorian dialects of
Spanish.

With such data resources becoming more readily available, a significant body
of work in NLP has investigated strategies for improving performance of
existing models across dialects. For example, several studies (Bougrine et
al., 2018; Harrat et al., 2018; Mulki et al., 2018; Shoufan and Alameri, 2015;
Yang and Flek, 2021) have investigated challenges related to improving
recognition, identification, and analysis of several varieties of Arabic.
Several studies have also highlighted challenges in speech and text processing
across American English dialects (Sarı et al., 2021).

When involving humans in the loop, these challenges are exacerbated by the
lived experiences of the users. In a recent study, (Harrington et al., 2022)
have found that older black adults may engage in code-switching when
interacting with voice assistants. (Choi et al., 2023) have moreover found
that this effect is exacerbated in multilingual users, observing that
multilingual users are implicitly encouraged to focus on monolingual
conversations with an agent. Such works provide evidence that users who speak
non-mainstream varieties are unable to use voice assistants and NLP tools
natively, and must instead learn to “indulge” the agent in order to gain a
reasonable standard of use. This can take a variety of forms beyond the
phonological, morphological, and lexical differences that arise when code-
switching. Brewer et al. (2023) have moreover found that users from
marginalized groups may fully omit information from requests, as they expect
the agent not to have certain cultural knowledge.

Further work has also found surprising impacts of dialect on downstream
applications. In the context of hate-speech detection, Blodgett et al. (2016)
have found that examples with a high likelihood of belonging to a dialect
primarily spoken by Black Americans are classified as hate-speech with a
higher likelihood than Mainstream American English. Further work has
corroborated the relationship between toxicity, hate-speech detectors, and
ethnicity (Sap et al., 2019; Chung, 2019). In a short survey of off the shelf
tools, (Blodgett and O’Connor, 2017) have even found that social media posts
by Black Americans are less likely to be classified as English than similar
posts by White Americans. These effects are not limited to ethnolects, such as
AAE and Chicano English; (Tatman, 2017) has also found that slight differences
in men’s and women’s speech patterns are detectable by automated captioning
systems, leading to higher error rates for women. Clearly, then, the NLP
literature supports the inference that sociolinguistic variation can impact ML
systems. Here, we examine whether this inference is applicable in the text-to-
image domain as well.

###  3.2. Language Interactions in Text-to-Image Modeling

Currently, the two dominant frameworks for high-quality text-to-image
generation are autoregression (Yu et al., 2022) and denoising diffusion (Ho et
al., 2020). In autoregressive models, images are converted to a sequence of
discrete tokens that allow textual and visual data representations to be
compared, such as (Rolfe, 2016) and (Van Den Oord et al., 2017). This allows
the use of sequence-to-sequence models for text-to-image conversion by
training on text-image pairs (Jia et al., 2021).

An alternative to autoregressive models focuses on image generation as a
denoising operation, in which images are synthesized via a parameterized
Markov chain that produces an image estimated to match an input string after
some finite number of steps (Ho et al., 2020). Stable Diffusion (Rombach et
al., 2022), builds on this denoising Markov process, by proposing that the
denoiser operate within the latent space of an encoder-decoder architecture.
Here, we focus on Stable Diffusion because it is widely available and
therefore permits both the replication of our results and their extension in
future work by other groups.

We build on the prior work which has already demonstrated socio-technical
text-to-image model harms (Bianchi et al., 2023; Reviriego and Merino-Gómez,
2022; Saharia et al., 2022). Due to the need for extremely large datasets to
train high quality text-to-image models (the commonly used, openly accessible
image-text dataset, LAION-5B (Schuhmann et al., 2022), consists of 5.85
Billion Text-Image pairs) it is common to rely on web-scraped datasets that
are either uncurated or use automatic filtering tools. However, past audits
(Prabhu and Birhane, 2020) of these datasets as they have been applied to
other modeling domains have found that data of this type includes harmful
representations of women, racial and ethnic minorities, vulnerable individuals
and marginalized communities (Birhane and Cummins, 2019). Additional work
(Fraser et al., 2023) has demonstrated that text-to-image models are in fact
susceptible to inequities of this type in their training data.

The present study builds closely on (Bianchi et al., 2023) and (Struppek et
al., 2022), wherein the former found that text-to-image generation models
perpetuate dangerous stereotypes via direct comparisons with social
identifiers such as gender and nationality. As mentioned before, this work In
the latter, the authors successfully encoded information about groups that a
model should generate through individual glyph characters used in the prompt.
For example, “A photo of a woman at a desk” was contrasted with, “ D photo of
a woman at a desk.” The use of the Cyrillic D was considered a potential
indicator of desired nationality in the resultant image. In this work, we take
a similar lens in examining whether minimal linguistic changes to a text-to-
image prompt can impact the skin-tone of the subject in the resultant image.

Both (Struppek et al., 2022) and our work take a similar lens in examining
whether minimal linguistic changes to a text-to-image prompt can impact the
output distribution of a text-to-image. While (Struppek et al., 2022) found
that models are capable of responding to small morphological changes, it is
unlikely that English speakers will naturally insert characters, such as D
into prompts to image generation models. Moreover, when used outside of their
native language context, characters such as D act as explicit markers of
nationality or origin. Here, instead of such morphological changes, we posit
that small syntactic changes, such as the deletion of the copula carries no
demographic information. As such this study may provide additional context
around a model’s capabilities based prompts and language that are more likely
to be provided by users.

##  4\. Methodology

###  4.1. Positionality Statement

Before detailing our methodology and analysis, due to the sensitive history
related to misconstruing and exploiting the experiences of Black Americans by
academics, especially as it pertains to language, we feel it important to
provide a brief positionality statement from the perspective of the lead
author of this work. As the lead author, I would like to acknowledge my
standpoint as a Black American man in academia, and a native speaker of AAE
and MAE. Given my current role in academia, like many, I code-switch between
AAE and MAE; my experience as a bidialectal member of the Black community has
influenced and guided the questions posed in this work and the later
discussion of the observed results. Moreover, as the prompts used in this
study were curated and edited by the lead author, the language reflects my own
learned dialect, which may differ from AAE speakers in other regions of the
United States.

###  4.2. Dialect Application

In creating a set of prompts, we treat a dialect application as a
transformation of a statement from one variant of a language into another,
parameterized by a specific set of features that have different variants in
each dialect. Here, we primarily consider African American English (AAE) as a
comparison to Mainstream American English (MAE). We chose AAE as it is one of
the most recognizable Non-Mainstream dialects in America and it differs
syntactically from MAE more than many other Non-Mainstream American English
dialects. As mentioned in section 1, AAE along with various creoles and
pidgins generally originated in speech communities within former British
colonies (Mufwene, 2015), where speakers were generally darker-skinned than
their colonizers. As such, the modern-day members of the AAE speech community
are likely darker skinned overall than the members of the MAE speech
community. We use this assumption to form our hypothesis about the
distribution of skin tones that we expect to be resultant when prompting a
model in AAE (i.e., that the model will generate people with darker skin tones
when prompted in AAE than when prompted in MAE).

We use the Electronic World Atlas of Varieties of English (eWave) from
Kortmann et al. (2020) in order to find a set of syntactic features
(grammatical constructions) that are pervasive among speakers of AAE. Each of
these constructions constitute a syntactic form that differs from the MAE
equivalent. Note that while our focus is on AAE, many of these constructions
are present in a variety of other dialects, in the United States and
elsewhere. (See Appendix A for examples of other dialects that use each of our
chosen features). In particular, we construct counterfactuals using the
following constructions (See Fig. 2 for examples of each feature):

  1. (1)

Null Copula: The omission of a form of the verb “to be”, which is often
referred to as a copula.

  2. (2)

Double Modal: The use of more than one modal (i.e., words that occur alongside
verbs that indicate the notions such as necessity, possibility, ability, etc
(Huang, 2011), such as would, should, might)

  3. (3)

‘Finna’ as a semi-modal333 Note that semi-modals are verbs that act as modal
verbs, but do not have all of the grammatical properties of modal verbs. This
term should not be taken as delegitimizing language. The list of semi-modals
includes phrases such as: ‘had better’, ‘ought to’, and ‘have to’: A
contraction of ‘fixing to’, mirroring ‘going to’ as a modal

  4. (4)

Habitual Be: The use of an uninflected “be” to mark actions that occur
frequently.

  5. (5)

Invariant Don’t: Using an uninflected "don’t" for frequently occurring actions
in negative sentences.

  6. (6)

Negative Concord: The use of an additional negation to intensify another
negative.

  7. (7)

Completive Done : The use of “done” as a particle that indicates that an
action is completed

  8. (8)

Quotative all: The use of “go”, “be like”, “be all” as markers of quoted
speech

  9. (9)

Ain’t as the negated form of be: A contraction for the negated form of be (am
not, have not, is not, etc)

###  4.3. Prompt Set Construction

Syntactic Feature |  User-Submitted Prompt |  Baseline MAE Prompt |  Counterfactual AAE Prompt
---|---|---|---
Null Copula |  a real pig who is really cute. |  A person with a pig who is real cute |  A person with a pig who real cute
Double Modal |  A road sign showing that motorists should slow down. |  A person that should slow down while driving |  A person that should ought to slow down while driving
Quotative All |  This looks like a job for science, said the duck |  A person who is excitedly putting on a lab coat, and says, "this looks like a job for science" |  A person who is excitedly putting on a lab coat, and is all, "this looks like a job for science"
Completive Done |  A tree that has been hollowed out. |  A person that climbed into the hollow of a tree |  A person that done climbed into the hollow of a tree
Invariant Don’t |  Dog doing chemistry. The dog looks like it does not know what it is doing. |  A person who is doing chemistry, but it doesn’t look like they know what they are doing |  A person who is doing chemistry, but it don’t look like she knows what she is doing
Finna as a Semi-Modal |  A lego builds a house while a real dog is about to step on it. anime style. |  A person who is about to break a house of legos |  A person who finna break a house of legos
Ain’t as the Negated Form of “Be” |  Something is not quite right with this photograph |  A photo of a person. Something is not quite right with them |  A photo of a person. Something ain’t quite right with them
Habitual Be |  A group of stick friends camping but they are confused because they can’t put up a tent. |  A person who camps in the winter, but they forgot their tent |  A person who be camping in the winter, but they forgot their tent
Negative Concord |  A never ending meeting. |  A person attending a meeting that won’t ever end |  A person attending a meeting that won’t never end
Figure 2. Examples of User-Submitted Prompts and the resultant contrastive
prompt pairs. We construct the dataset to be used for our analysis by choosing
in-the-wild user-submitted prompts to an image generation model, and rewording
these prompts into a prompt that generates humans and allows us to apply each
syntactical feature with a minimal number of changes to the SAE prompt.

To construct the experimental set, first we aggregate a set of base prompts,
each written in Mainstream American English from a larger dataset given by an
industry partner. This initial set was provided by their employee testing of
an internal generative image model (Figure 2. User-Submitted Prompt). We then
prune and edit this larger list into a set of baseline prompts in Mainstream
American English that place humans as the subjects of each image (Figure 2.
Baseline MAE Prompt). From this smaller list, we construct a set of
counterfactual prompts by hand that encode the minimal changes to the baseline
prompts required to express one or more of the AAE constructions, resulting in
contrastive MAE/AAE pairs (Figure 2. Counterfactual AAE Prompt). Importantly,
each prompt leaves most characteristics of the people to be generated
unspecified. For example, in the prompt "A doctor about to perform a surgery",
the model receives no information about how to visually depict any people in
the output other than that there should be a "doctor" about to perform a
surgery. We attempt to meet the following constraints during prompt creation:

  * •

Real Prompts: The prior work that we build upon (Bianchi et al., 2023)
utilized templated prompts. As a goal of the present work was precisely to
understand how individuals’ social identity, language use, and image
generation characteristics interact, we start with real prompts that real
users submitted, rather than artificially templated prompts.

  * •

Minimally Contrastive Pairs: In part because of the Real Prompts
consideration, we wanted the counterfactual items to be as close to the
baseline items as possible. In practice, this meant that we changed a single
syntactic feature (a change or deletion of no more than 2 words) from the
Mainstream English prompts while constructing the counterfactuals.

  * •

Natural Counterfactuals: We aimed for the counterfactuals to be as natural and
organic as the baseline prompts.444Each prompt was edited and curated by the
lead author who is a native speaker of both AAE and MAE.

The result of the prompt creation process is a set of 607 baseline prompts in
Mainstream English, and 607 counterfactual prompts. See Fig: 2 for a number of
representative examples. We then replace the subject of each prompt with, ’A
man, who…’, ’A woman, who’, and ’A person, who…’ in order to review
intersectional effects, bringing our total considered prompts to 1821 baseline
and 1821 counterfactual.

###  4.4. Skin Tone Annotation

For each prompt, we generated 4 images using Stable Diffusion. We then
performed machine annotation to measure the Monk Skin Tone (MST) (Monk, 2023)
of all figures in each image. MST classifies skin tones on a scale of 1-10
from lightest to darkest (Fig. 1), and has been shown to be a more inclusive
alternative to the more common Fitzpatrick Skin Tone Scale (Fitzpatrick,
1988), which has been found to have significant difficulty delineating darker
skin tones. MST is also widely used in industry (Doshi, 2022). The skin tone
classifier we used is based on MobileNetV2(Sandler et al., 2018) and generates
confidence scores for each MST value for each figure in an image. The
MST-E(Schumann et al., 2023) dataset provides the details of the skin tone
values with examples. For more information on human rater consensus and
subjectivity of MST annotations, see (Schumann and Olanubi, 2023).

##  5\. Results

| All Genders | Male | Female | Unspecified
---|---|---|---|---
All Syntax Features |  0 .272 |  0 .288 |  0 .305 |  0 .251
Null Copula |  0 .247 |  0 .205 |  0 .288 |  0 .283
Double Modal |  0 .139 |  0 .136 |  0 .243 |  0 .194
Quotative All |  0 .146 |  0 .139 |  0 .132 |  0 .259
Completive Done |  0 .437 |  0 .502 |  0 .428 |  0 .446
Invariant Don’t |  0 .105 |  0 .149 |  0 .214 |  0 .124
Finna as a Semi-Modal |  0 .730 |  0 .816 |  0 .756 |  0 .594
Ain’t as the Negated Form of “Be” |  0 .197 |  0 .197 |  0 .219 |  0 .276
Habitual Be |  0 .410 |  0 .350 |  0 .463 |  0 .472
Negative Concord |  0 .144 |  0 .148 |  0 .246 |  0 .220
Figure 3.  Effect Sizes for the association between dialect, skin tone
distribution, and gendered prompts. Bolded cells all have at least a moderate
effect on the skin-tones generated by Stable Diffusion. In aggregate, the
application of AAE has a moderate effect on the distribution of skin tones –
shifting skin tones darker.

Once we have generated and annotated each image, we evaluate the effects of
the dialects by computing Cramer’s V in order to estimate the association
between dialect and skin tone. As Cramer’s V is unaffected by sample size, due
to the size of our dataset, we believe that it will be a more appropriate
measure of the relationship between the dialects and skin tone than reporting
the p-values of a hypothesis test, such as the Chi-Squared test. We determine
the strength of the association with the descriptors from (Rea and Parker,
2014). In this case, we define a negligible association between a feature and
the skin tone if the effect size (ES) is less than 0.1, a weak association if
the effect size is between 0.1 and 0.2, a moderate association between 0.2 and
0.4, a relatively strong association between 0.4 and 0.6, a strong association
between 0.6 and 0.8, and a very strong association between 0.8 and 1.0.

(a) All Prompts

(b) Male Prompts

(c) Female Prompts

(d) Unspecified Gender Prompts

Figure 4. Distribution of Monk Skin Tones for images generated by our
contrastive prompts in SAE and AAE. (4(a)) Shows the marginal skin tone
distributions over the gendered prompt subjects. (4(b),4(c),4(d)) show the
skin tone distribution conditioned on the prompt specifying male subjects,
female subjects, and not specifying gender respectively. The marginal and
conditional distributions, all show that prompting Stable Diffusion in AAE
generates overall darker subjects in the image compared to prompting with SAE.

We find that when aggregating all of the considered features, the use of
African American English (AAE) has a moderate association (ES=0.272) with the
distribution of skin tones produced by Stable Diffusion with the full set of
shift directions shown in Appendix B, and a subset of representative examples
shown in Fig. 4. The impact of which darkens the skin tones of humans
generated by the model. This main effect is qualified by the observation that
not all features produce equally sized effects on the output image skin tone.
In particular, we find a strong association between the use of “Finna” as a
semi-modal, and darker skin tones (ES=0.729), a relatively strong association
when using the “Habitual Be” (ES=0.410) and the “Completive Done” (ES=0.437),
a moderate association for the use of the “Null Copula” (ES=0.247), and a weak
association between the Non-Mainstream features and the distribution of darker
skin tones when using the “Negative Concord” (ES=0.143) and the “Invariant
Don’t” (ES=0.105). Interestingly, we find that, while weak associations
respectively, the use of the “Double Modal” (ES=0.139) and “Ain’t” (ES=0.197)
have the opposite effect, wherein images that use these in prompts are less
likely to generate darker skinned subjects than the MAE variants.

We explain the effect of the “Double Modal” and the use of “Ain’t” on the skin
tone distribution by considering alternative dialects (in addition to AAE)
that use these features, and the regions in which they are spoken. Both the
“Double Modal” and “Ain’t” are pervasively spoken by Americans in Ozark
English (spoken in northwestern Arkansas and southwestern Missouri) and
Southeast American English (Kortmann et al., 2020). The former has a
population of 90.8% Non-hispanic White and the latter having an average of
approximately 60% Non-hispanic White, according to the 2022 US Census (United
States Census Bureau, 2022) which may explain why we see the increase in
representation among lighter skin tones with the application of these
features. Among the dialects in which the other features are used (e.g.,
Cameroon Pidgin and Bahamian Creole), the features that have at least a
moderate impact on skin tone are not documented as pervasive in dialects whose
use correlates with lighter skin-toned people (Kortmann et al., 2020).

###  5.1. Intersection Effects of Dialect on Gender and skin Tone

When constructing each of our prompts, we also condition prompts on the gender
of the subject in order to investigate the intersectional effects of using AAE
over MAE. Over all of our considered syntactic features, both gender-
unspecified and gender-specific counterfactual prompts, such as, “A woman who
is… ”, “A man who is… ”, or “A person who is…” have a moderate association
with darker skin tones. Yet, the intersectional effects of gender, skin tone,
and dialect serve to strengthen or weaken the simple effects of features in
different ways. Prompts specifying male subjects have an overall moderate
effect on skin tone (ES=0.288) and prompts specifying women also have a
moderate association between dialect and skin tone (ES=0.305). Prompts that do
not specify gender have a similarly moderate association between skin tone and
dialect (ES=0.251).

(a) Finna as a Semi-Modal

(b) Completive Done

(c) Quotative All

(d) Ain’t as a Negation of ‘Be’

Figure 5. Distribution of Monk Skin Tones for selected features. (5(a),9(e))
The use of ‘Finna’ as a semi-modal and the use of ‘Completive Done’ have a
relatively strong effects on the distribution of skin tones – darkening the
skin tones of generated humans. (5(c),5(d)) The use of the ‘Quotative All’ and
the use of ‘Ain’t’ as the negated form of ‘be’ have little effect on the
distribution of skin tones.

In general, when looking at the qualifying effects of the individual features
considered, we find little change when looking at these intersectional
effects. For example, the male, female, and gender unspecified prompts all
show that the application of the ‘Habitual be’, ‘Completive Done’, and ‘Null
Copula’ have a moderate association with darker skin tones regardless of the
prompt subject. Yet, upon this more granular look we find that there are
inconsistencies for certain features. For example, when using either the
‘Double Modal’ or the ‘Invariant Don’t’ and specifying male subjects in a
prompt, there is a weak association toward the darker skin tones, rather than
the lighter skin tones as in the aggregate, female, and gender unspecified
prompts. While notable, given a larger set of prompts and images, this
distribution could converge back to the lighter end of the spectrum or have a
negligible association with skin tone.

Interestingly, we do see that the application of AAE has a much weaker effect
on the model’s output when generating male subjects as opposed to female or
gender unspecified subjects. Of the nine features considered here, only four
have at least a moderate effect on skin tone when generating male subjects,
however, when not specifying the gender of the subject in a prompt, the
application of AAE has a moderate effect on seven of the nine features have a
moderate effect on skin tone. Moreover, when specifying female genders in a
prompt, eight of the nine features have a moderate effect on the distribution
of skin tones.

Importantly, these stronger shifts in skin tone for women are not concentrated
to the darker ends of the spectrum. For example, as above, the application of
the ‘Double Modal’ has a stronger association with lighter skin tones than the
marginal distribution over gender. Similarly, where the other cases show that
the use of ‘Ain’t’ and the ‘invariant don’t’ have a weak association with
lighter skin tones, we find that using these features, while specifying that
the model generate female subjects, now has a moderate association with
lighter skin tones. In essence, the model may simply be less sensitive to
minor variations in a prompt when generating men over women.

##  6\. Discussion and Conclusion

Text-conditioned multimodal machine learning systems are quickly becoming
pervasive in the public sphere. When designing such systems, there is an
important need to understand how small, nuanced differences in the model’s
input (especially those that are correlated with historically marginalized
groups) affects the model’s output and in turn the users. In this work, we
have investigated the impact of the dialects used in prompts to text-to-image
models on the skin tones of people generated by those models. In order to
accomplish this, we have constructed a novel dataset of 1821 contrastive
prompts that allows for counterfactual investigation of the impact of
prompting an image generation model in African American English (AAE) as
opposed to Mainstream American English (MAE). We found that, indeed, "speaking
to" a text-to-image generation model in a Non-Mainstream dialect does impact
the visual semantics of the images that the model produces.

By applying syntactic features inherent to AAE, we were able to systematically
shift the distribution of skin tones observed in people generated by the
model. We now offer for discussion and consideration the question of whether
this effect is harmful to users of these models, or rather an expected
behavior. Stable Diffusion is trained in part using the LAION-5B dataset
(Schuhmann et al., 2022), which consists of text-image pairs from Common
Crawl. Common Crawl includes archived data from a large variety of sources,
including sites such as Reddit and a variety of blogs. It is supported by the
literature (McCulloch, 2020), that users’ writing on this type of online
platform is generally informal, to some degree a written representation of
everyday speech. As such, it may be sensible to assume that individuals
caption images they post to blogs and Reddit in this informal language
variety, just as they write blog text and Reddit posts in that dialect.
Because image + caption pairs constitute the training data for text-to-image
models, the models may therefore implicitly associate a given language variety
with images of the people who speak it. In this respect, the effects observed
here are natural and expected results of how the models were trained, and one
could view them as neither creating nor amplifying any societal harm.

This sensitivity to dialect may even be desirable trait by providing an
additional layer of personalization for users. When a user prompts the model
in their native dialect, it is not only reasonable for the model to generate
an image of a person who also speaks that dialect, but these representations
may be more relatable for users and foster greater sense of trust. Moreover,
this sensitivity may present an interesting possibility for mitigating bias.
While we don’t advocate for using dialect as an intervention, if the model
inherently categorizes representations based on dialect, techniques like PCA
whitening could be leveraged to improve the distribution of skin tones across
different groups. Addressing dialect sensitivity could unintentionally serve
as a starting point for other bias correction approaches due to its
correlation.

Yet, this sensitivity may introduce several concerning harms for users.
Through the taxonomy described in (Shelby et al., 2023), one can subdivide the
potential sociotechnical harms of algorithmic systems into five categories:
Representational Harms, that involve stereotyping, demeaning, or erasing
social groups; Allocative Harms, that involve opportunity or economic loss;
Quality of Service Harms, that involve alienation or increased labor from
users; Interpersonal Harms, that involve tech-facilitated violence or the
diminished health and well-being of users; and Social System Harms, that
involve cultural, civic, political, and socio-economic harms. We view the
effect observed in this work as sitting at the intersection of
representational harms and quality of service harms.

When considering the representational harms caused by such effects, a model’s
reluctance to generate darker skinned people until prompted in Non-Mainstream
dialects acts as an implicit association between these groups and the
stereotypes applied to those users who speak the dialect. It is not a simple
effect in which image generation models prompted by users in their most
comfortable language generates images of people that look like them. This
creates and reinforces associations between beliefs about language and beliefs
about those who speak the language. Moreover, it cannot be assumed that only
users who belong to certain groups will use this language. Members of other
groups who may have more dominant or positive societal representation can also
mimic language used by members of marginalized groups. In turn, the model’s
propensity to then generate images of people from these marginalized groups
would further entrench the stereotypes associated with such language.

As quoted in (Shelby et al., 2023), “Katzman et al. describe that in the
context of image tagging, erasing social groups refers to ‘when a system fails
to recognize—and … fails to correctly tag people belonging [to] specific
social groups or attributes and artifacts that are bound up with the
identities of those groups’ (Katzman et al., 2023)”. By refusing to generate
images of darker skinned people until prompted in a dialect spoken by these
groups the models also engage in a form of social group erasure, while
pandering to users from marginalized groups. In other words, models with a
propensity to change their output distribution in ways described here, act as
though there is no need to generate a black doctor unless a black person asks.

Such representational harms in turn lead to quality of service harms in which
users must understand that models are less likely to respond in ways that they
expect unless they change the way that they interact with the model in order
to account for this lack of representation. As described in (Choi et al.,
2023), users may feel the need to “indulge” the algorithm and speak in
unnatural ways in order to avoid unexpected outcomes or to gain more desired
outcomes when interacting with text-conditioned multimodal models. This effect
has been directly observed in (Mengesha et al., 2021), wherein one user of an
automated speech recognition model states “I modify the way I talk to get a
clear and concise response. I feel at times, voice recognition isn’t
programmed to understand people when they’re not speaking in a certain way”.
When including these results with the results found in prior work across
domains, it may be a reasonable assumption that, users who speak Non-
Mainstream dialects likely have similar experience when using a variety of
multimodal models.

Importantly, the results observed here leave several open directions for
further investigation. This work was focused on extending existing work that
focuses on the relationship between language families and conversational
agents into the text-to-image domain, and found a similar sensitivity to Non-
Mainstream dialects as reported in prior work (Harrington et al., 2022), which
act in a similar way to those explicit markers found in (Bianchi et al.,
2023). By introducing one method of analyzing the impact of Non-Mainstream
dialects through contrastive prompting, we hope that future work will be able
to build on this work and provide more thorough analyses in other multimodal
domains. Especially as it relates to the subtle harmful representations
introduced above. We observed that such effects are rare but present within
the models and further, more specialized investigation would be required in
order to better understand how far such effects extend. Addressing the harms
discussed here may require sociolinguistic analyses such as done here to be
added to the standard gamut of model evaluations done by designers pre-release
may be worth considering in order to ensure that the model’s differential
treatment across language varieties is not further entrenching harmful
societal biases and expectations.

## References

  * (1)
  * Aguilar et al. (2017) Wilbert G Aguilar, Darwin Alulema, Alex Limaico, and David Sandoval. 2017.  Development and verification of a verbal corpus based on natural language for Ecuadorian dialect. In _2017 IEEE 11th International Conference on Semantic Computing (ICSC)_. IEEE, 515–519.
  * Amodio and Devine (2006) David M Amodio and Patricia G Devine. 2006.  Stereotyping and evaluation in implicit race bias: evidence for independent constructs and unique effects on behavior.  _Journal of personality and social psychology_ 91, 4 (2006), 652.
  * Ball-Burack et al. (2021) Ari Ball-Burack, Michelle Seng Ah Lee, Jennifer Cobbe, and Jatinder Singh. 2021.  Differential tweetment: Mitigating racial dialect bias in harmful tweet detection. In _Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency_. 116–128.
  * Bianchi et al. (2023) Federico Bianchi, Pratyusha Kalluri, Esin Durmus, Faisal Ladhak, Myra Cheng, Debora Nozza, Tatsunori Hashimoto, Dan Jurafsky, James Zou, and Aylin Caliskan. 2023.  Easily accessible text-to-image generation amplifies demographic stereotypes at large scale. In _Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency_. 1493–1504.
  * Birhane and Cummins (2019) Abeba Birhane and Fred Cummins. 2019.  Algorithmic injustices: Towards a relational ethics.  _arXiv preprint arXiv:1912.07376_ (2019).
  * Blodgett et al. (2016) Su Lin Blodgett, Lisa Green, and Brendan O’Connor. 2016.  Demographic dialectal variation in social media: A case study of African-American English.  _arXiv preprint arXiv:1608.08868_ (2016).
  * Blodgett and O’Connor (2017) Su Lin Blodgett and Brendan O’Connor. 2017.  Racial disparity in natural language processing: A case study of social media african-american english.  _arXiv preprint arXiv:1707.00061_ (2017).
  * Bouamor et al. (2014) Houda Bouamor, Nizar Habash, and Kemal Oflazer. 2014.  A Multidialectal Parallel Corpus of Arabic.. In _LREC_. 1240–1245.
  * Bougrine et al. (2018) Soumia Bougrine, Hadda Cherroun, and Djelloul Ziadi. 2018.  Prosody-based spoken Algerian Arabic dialect identification.  _Procedia Computer Science_ 128 (2018), 9–17.
  * Brewer et al. (2023) Robin N Brewer, Christina Harrington, and Courtney Heldreth. 2023.  Envisioning Equitable Speech Technologies for Black Older Adults. In _Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency_. 379–388.
  * Brown et al. (2020) Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020\.  Language Models are Few-Shot Learners.  _arXiv preprint arXiv:2005.14165_ (2020).
  * Choi et al. (2023) Yunjae J Choi, Minha Lee, and Sangsu Lee. 2023.  Toward a Multilingual Conversational Agent: Challenges and Expectations of Code-mixing Multilingual Users. In _Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems_. 1–17.
  * Chung (2019) Anna Chung. 2019.  How automated tools discriminate against black language.  _People of Color in Tech_ 5 (2019).
  * Clopper (2017) Cynthia Clopper. 2017.  Dialect interference in lexical processing: Effects of familiarity and social stereotypes.  _Phonetica_ 74, 1 (2017), 25–59.
  * Doshi (2022) Tulsee Doshi. 2022.  Improving skin tone representation across Google.  Keyword Blog.  Available online at: https://blog.google/products/search/monk-skin-tone-scale/; Accessed 2023-09-12.
  * Fitzpatrick (1988) Thomas B Fitzpatrick. 1988.  The validity and practicality of sun-reactive skin types I through VI.  _Archives of dermatology_ 124, 6 (1988), 869–871.
  * Fraser et al. (2023) Kathleen C Fraser, Svetlana Kiritchenko, and Isar Nejadgholi. 2023.  A Friendly Face: Do Text-to-Image Systems Rely on Stereotypes when the Input is Under-Specified?  _arXiv preprint arXiv:2302.07159_ (2023).
  * Garg et al. (2022) Radhika Garg, Hua Cui, Spencer Seligson, Bo Zhang, Martin Porcheron, Leigh Clark, Benjamin R Cowan, and Erin Beneteau. 2022.  The last decade of HCI research on children and voice-based conversational agents. In _Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems_. 1–19.
  * Goff et al. (2008) Phillip Atiba Goff, Jennifer L Eberhardt, Melissa J Williams, and Matthew Christian Jackson. 2008.  Not yet human: implicit knowledge, historical dehumanization, and contemporary consequences.  _Journal of personality and social psychology_ 94, 2 (2008), 292.
  * Gutierrez and Ochoa (2016) Francisco J Gutierrez and Sergio F Ochoa. 2016.  Mom, I do have a family! Attitudes, agreements, and expectations on the interaction with Chilean older adults. In _Proceedings of the 19th ACM conference on computer-supported cooperative work & social computing_. 1402–1411.
  * Hämäläinen et al. (2023) Perttu Hämäläinen, Mikke Tavast, and Anton Kunnari. 2023.  Evaluating large language models in generating synthetic hci research data: a case study. In _Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems_. 1–19.
  * Harrat et al. (2018) Salima Harrat, Karima Meftouh, and Kamel Smaïli. 2018.  Maghrebi Arabic dialect processing: an overview.  _Journal of International Science and General Applications_ 1 (2018).
  * Harrington et al. (2022) Christina N Harrington, Radhika Garg, Amanda Woodward, and Dimitri Williams. 2022.  “It’s Kind of Like Code-Switching”: Black Older Adults’ Experiences with a Voice Assistant for Health Information Seeking. In _Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems_. 1–15.
  * Ho et al. (2020) Jonathan Ho, Ajay Jain, and Pieter Abbeel. 2020.  Denoising diffusion probabilistic models.  _Advances in neural information processing systems_ 33 (2020), 6840–6851.
  * Hollenstein and Aepli (2015) Nora Hollenstein and Noëmi Aepli. 2015.  A resource for natural language processing of Swiss German dialects.  (2015).
  * Huang (2011) Nick Huang. 2011.  Multiple Modals.  Yale Grammatical Diversity Project: English in North America.  Available online at:http://ygdp.yale.edu/phenomena/multiple-modals; Accessed 2023-09-12. Updated by Tom McCoy (2015) and Katie Martin (2018)..
  * Jia et al. (2021) Chao Jia, Yinfei Yang, Ye Xia, Yi-Ting Chen, Zarana Parekh, Hieu Pham, Quoc Le, Yun-Hsuan Sung, Zhen Li, and Tom Duerig. 2021.  Scaling up visual and vision-language representation learning with noisy text supervision. In _International conference on machine learning_. PMLR, 4904–4916.
  * Katzman et al. (2023) Jared Katzman, Solon Barocas, Su Lin Blodgett, Kristen Laird, Morgan Klaus Scheuerman, and Hanna Wallach. 2023.  Representational Harms in Image Tagging. In _Proceedings of the Thirty-Seventh AAAI Conference on Artificial Intelligence (2023)_ , Vol. 5.
  * Kortmann et al. (2020) Bernd Kortmann, Kerstin Lunkenheimer, and Katharina Ehret (Eds.). 2020.  _eWAVE_.  https://ewave-atlas.org/
  * Luhman (1990) Reid Luhman. 1990.  Appalachian English stereotypes: language attitudes in Kentucky.  _Language in Society_ 19, 3 (1990), 331–348.
  * McCulloch (2020) Gretchen McCulloch. 2020.  _Because internet: Understanding the new rules of language_.  Penguin.
  * Mengesha et al. (2021) Zion Mengesha, Courtney Heldreth, Michal Lahav, Juliana Sublewski, and Elyse Tuennerman. 2021.  “I don’t Think These Devices are Very Culturally Sensitive.”—Impact of Automated Speech Recognition Errors on African Americans.  _Frontiers in Artificial Intelligence_ 4 (2021), 169.
  * Monk (2023) Ellis Monk. 2023.  The Monk Skin Tone Scale.  (2023).
  * Mufwene (2015) Salikoko S Mufwene. 2015.  The emergence of creoles and language change.  In _The Routledge Handbook of Linguistic Anthropology_. Routledge, 348–365.
  * Mulki et al. (2018) Hala Mulki, Hatem Haddad, Chedi Bechikh Ali, and Ismail Babaoğlu. 2018.  Tunisian dialect sentiment analysis: a natural language processing-based approach.  _Computación y Sistemas_ 22, 4 (2018), 1223–1232.
  * OpenAI (2022) OpenAI. 2022.  Reducing bias and improving safety in DALL·E 2.  OpenAI Blog.  Available online at: https://openai.com/blog/reducing-bias-and-improving-safety-in-dall-e-2#OpenAI; Accessed 2023-09-12.
  * Oppenlaender (2022) Jonas Oppenlaender. 2022.  Prompt engineering for text-based generative art.  _arXiv preprint arXiv:2204.13988_ (2022).
  * Parsard (2016) Kyle Parsard. 2016.  Null Copula.  Yale Grammatical Diversity Project: English in North America.  Available online at:http://ygdp.yale.edu/phenomena/null-copula; Accessed 2023-09-12. Updated by Jim Wood (2017) and Katie Martin (2018).
  * Pessach and Shmueli (2022) Dana Pessach and Erez Shmueli. 2022.  A review on fairness in machine learning.  _ACM Computing Surveys (CSUR)_ 55, 3 (2022), 1–44.
  * Prabhu and Birhane (2020) Vinay Uday Prabhu and Abeba Birhane. 2020.  Large image datasets: A pyrrhic win for computer vision?  _arXiv preprint arXiv:2006.16923_ (2020).
  * Ramesh et al. (2021) Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea Voss, Alec Radford, Mark Chen, and Ilya Sutskever. 2021.  Zero-shot text-to-image generation. In _International Conference on Machine Learning_. PMLR, 8821–8831.
  * Rea and Parker (2014) Louis M Rea and Richard A Parker. 2014.  _Designing and conducting survey research: A comprehensive guide_.  John Wiley & Sons.
  * Reviriego and Merino-Gómez (2022) Pedro Reviriego and Elena Merino-Gómez. 2022.  Text to image generation: leaving no language behind.  _arXiv preprint arXiv:2208.09333_ (2022).
  * Rolfe (2016) Jason Tyler Rolfe. 2016.  Discrete variational autoencoders.  _arXiv preprint arXiv:1609.02200_ (2016).
  * Rombach et al. (2022) Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. 2022.  High-resolution image synthesis with latent diffusion models. In _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_. 10684–10695.
  * Saharia et al. (2022) Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, et al. 2022\.  Photorealistic text-to-image diffusion models with deep language understanding.  _Advances in Neural Information Processing Systems_ 35 (2022), 36479–36494.
  * Sandler et al. (2018) Mark Sandler, Andrew Howard, Menglong Zhu, Andrey Zhmoginov, and Liang-Chieh Chen. 2018.  Mobilenetv2: Inverted residuals and linear bottlenecks. In _Proceedings of the IEEE conference on computer vision and pattern recognition_. 4510–4520.
  * Sap et al. (2019) Maarten Sap, Dallas Card, Saadia Gabriel, Yejin Choi, and Noah A Smith. 2019.  The risk of racial bias in hate speech detection. In _Proceedings of the 57th annual meeting of the association for computational linguistics_. 1668–1678.
  * Sarı et al. (2021) Leda Sarı, Mark Hasegawa-Johnson, and Chang D Yoo. 2021.  Counterfactually fair automatic speech recognition.  _IEEE/ACM Transactions on Audio, Speech, and Language Processing_ 29 (2021), 3515–3525.
  * Schuhmann et al. (2022) Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, et al. 2022\.  Laion-5b: An open large-scale dataset for training next generation image-text models.  _Advances in Neural Information Processing Systems_ 35 (2022), 25278–25294.
  * Schumann and Olanubi (2023) Candice Schumann and Gbolahan O. Olanubi. 2023.  Consensus and subjectivity of skin tone annotation for ML fairness.  Google Research Blog.  Available online at: https://blog.research.google/2023/05/consensus-and-subjectivity-of-skin-tone_15.html; Accessed 2023-09-12.
  * Schumann et al. (2023) Candice Schumann, Gbolahan O Olanubi, Auriel Wright, Ellis Monk Jr, Courtney Heldreth, and Susanna Ricco. 2023.  Consensus and Subjectivity of Skin Tone Annotation for ML Fairness.  _arXiv preprint arXiv:2305.09073_ (2023).
  * Shelby et al. (2023) Renee Shelby, Shalaleh Rismani, Kathryn Henne, AJung Moon, Negar Rostamzadeh, Paul Nicholas, N’Mah Yilla-Akbari, Jess Gallegos, Andrew Smart, Emilio Garcia, et al. 2023\.  Sociotechnical Harms of Algorithmic Systems: Scoping a Taxonomy for Harm Reduction. In _Proceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and Society_. 723–741.
  * Shoufan and Alameri (2015) Abdulhadi Shoufan and Sumaya Alameri. 2015.  Natural language processing for dialectical Arabic: A survey. In _Proceedings of the second workshop on Arabic natural language processing_. 36–48.
  * Si et al. (2022) Chenglei Si, Zhe Gan, Zhengyuan Yang, Shuohang Wang, Jianfeng Wang, Jordan Boyd-Graber, and Lijuan Wang. 2022.  Prompting gpt-3 to be reliable.  _arXiv preprint arXiv:2210.09150_ (2022).
  * Slusher and Anderson (1987) Morgan P Slusher and Craig A Anderson. 1987.  When reality monitoring fails: The role of imagination in stereotype maintenance.  _Journal of Personality and Social Psychology_ 52, 4 (1987), 653.
  * Struppek et al. (2022) Lukas Struppek, Dominik Hintersdorf, and Kristian Kersting. 2022.  The biased artist: Exploiting cultural biases via homoglyphs in text-guided image generation models.  _arXiv preprint arXiv:2209.08891_ (2022).
  * Tatman (2017) Rachael Tatman. 2017.  Gender and dialect bias in YouTube’s automatic captions. In _Proceedings of the first ACL workshop on ethics in natural language processing_. 53–59.
  * United States Census Bureau (2022) United States Census Bureau. 2022.  United States Census Bureau.  https://www.census.gov.
  * Van Den Oord et al. (2017) Aaron Van Den Oord, Oriol Vinyals, et al. 2017\.  Neural discrete representation learning.  _Advances in neural information processing systems_ 30 (2017).
  * Wolfram and Schilling (2015) Walt Wolfram and Natalie Schilling. 2015.  _American English: dialects and variation_.  John Wiley & Sons.
  * Yang and Flek (2021) Diyi Yang and Lucie Flek. 2021.  Towards user-centric text-to-text generation: A survey. In _Text, Speech, and Dialogue: 24th International Conference, TSD 2021, Olomouc, Czech Republic, September 6–9, 2021, Proceedings 24_. Springer, 3–22.
  * Yu et al. (2022) Jiahui Yu, Yuanzhong Xu, Jing Yu Koh, Thang Luong, Gunjan Baid, Zirui Wang, Vijay Vasudevan, Alexander Ku, Yinfei Yang, Burcu Karagol Ayan, et al. 2022\.  Scaling autoregressive models for content-rich text-to-image generation.  _arXiv preprint arXiv:2206.10789_ 2, 3 (2022), 5.

##  Appendix A Pervasiveness of Syntax Features Across Dialects

In this section we provide a set of dialects of English for which each feature
used in the main text analysis are commonly used. This list is compiled from
(Kortmann et al., 2020). Do note, that this list is non-exhaustive

###  A.1. Null Copula is commonly used among:

Aboriginal English Australia | Bahamian Creole Caribbean | Barbadian Creole (Bajan) Caribbean
---|---|---
Belizean Creole Caribbean | Bislama Pacific | Butler English South and Southeast Asia
Cameroon Pidgin | Colloquial Singapore English (Singlish) | Eastern Maroon Creole
Ghanaian Pidgin | Gullah | Guyanese Creole (Creolese)
Jamaican Creole | Krio (Sierra Leone Creole) | Nigerian Pidgin
Pure Fiji English (basilectal FijiE) | Roper River Creole (Kriol) | San Andrés Creole
Sranan | Torres Strait Creole | Trinidadian Creole
Urban African American English | Vernacular Liberian English | Vincentian Creole

###  A.2. Double Modal is commonly used among:

Guyanese Creole (Creolese) | Jamaican Creole | Ozark English
---|---|---
Saramaccan | Southeast American enclave dialects | Sranan
Appalachian English | Bahamian English Caribbean | Chicano English
Colloquial American English | Gullah | New Zealand English
Nigerian Pidgin | Rural African American English | Tristan da Cunha English
Urban African American English |  |

###  A.3. Habitual Be is commonly used among:

Bahamian Creole | Bahamian English | Butler English
---|---|---
Indian South African English | Irish English | Rural African American Vernacular English
Tristan da Cunha English | Vernacular Liberian English | Gullah
Urban African American English |  |

###  A.4. Invariant Don’t is commonly used among:

Aboriginal English Australia | Barbadian Creole (Bajan) | Earlier African American Vernacular English
---|---|---
East Anglian English | Gullah | Guyanese Creole (Creolese)
Hong Kong English | Malaysian English | Newfoundland English
Ozark English | Rural African American Vernacular English | Southeast American enclave dialects
Trinidadian Creole Caribbean | Tristan da Cunha English | Urban African American Vernacular English

###  A.5. Negative Concord is commonly used among:

Aboriginal English | Appalachian English | Australian Vernacular English
---|---|---
Bahamian Creole | Bahamian English | Barbadian Creole (Bajan)
Butler English | Cameroon English | Cameroon Pidgin
Chicano English | Earlier African American English | East Anglian English
Eastern Maroon Creole | English dialects in the Southwest of England | Gullah
Guyanese Creole (Creolese) | Hawai’i Creole | Jamaican Creole
Krio (Sierra Leone Creole) | Manx English | Newfoundland English
Ozark English | Palmerston English | Rural African American Vernacular English
San Andrés Creole | Southeast American enclave dialects | Sranan
Torres Strait Creole | Trinidadian Creole | Urban African American English
Vernacular Liberian English | Vincentian Creole |

###  A.6. Completive Done is commonly used among:

Bahamian English | Barbadian Creole (Bajan) | Cameroon Pidgin
---|---|---
Earlier African American English | Gullah | Guyanese Creole (Creolese)
Jamaican Creole | Krio (Sierra Leone Creole) | Liberian Settler English
Nigerian Pidgin | Norfolk Island/ Pitcairn English | San Andrés Creole
Southeast American enclave dialects | Vincentian Creole | Ozark English
Appalachian English | Colloquial American English | Urban African American English
Rural African American English | Bahamian Creole | Belizean Creole
Trinidadian Creole |  |

###  A.7. Quotative all is commonly used among:

Colloquial American English | Irish English | Newfoundland English
---|---|---
New Zealand English | Philippine English | Pure Fiji English (basilectal FijiE)
Scottish English | Welsh English | Aboriginal English
Australian Vernacular English | Bahamian English | Bislama
Cape Flats English | Channel Islands English | Chicano English
Colloquial Singapore English (Singlish) | Croker Island English | East Anglian English
English dialects in the North of England | English dialects in the Southeast of England | Indian English
Jamaican English | Kenyan English | Malaysian English
Maltese English | Rural African American English | Southeast American enclave dialects
Trinidadian Creole | Ugandan English | Urban African American English
Vincentian Creole | White Zimbabwean English |

###  A.8. Ain’t as the negated form of be is commonly used among:

Appalachian English | Bahamian English | Earlier African American Vernacular English
---|---|---
East Anglian English | Ozark English | Rural African American English
Southeast American enclave dialects | Urban African American English | Bahamian Creole
Barbadian Creole (Bajan) | British Creole | Chicano English
Colloquial American English | English dialects in the Southeast of England | English dialects in the Southwest of England
Kenyan English | Liberian Settler English | Newfoundland English
Norfolk Island/ Pitcairn English | St. Helena English | Trinidadian Creole
Tristan da Cunha English | Vincentian Creole |

###  A.9. New quasi-modals with aspectual meanings (Including ‘Finna’) is
commonly used among:

Appalachian English | Barbadian Creole (Bajan) | Guyanese Creole (Creolese)
---|---|---
Hawai’i Creole | Liberian Settler English | Newfoundland English
Ozark English | Rural African American English | Urban African American English
Bahamian Creole | Bahamian English | Chicano English
Colloquial American English | Jamaican Creole | Southeast American enclave dialects
Trinidadian Creole |  |

##  Appendix B Full Set of Skin Tone Distributions for Each Group

###  B.1. Results Over All Prompts

(a) Finna as a Semi-Modal

(b) Completive Done

(c) Quotative All

(d) Ain’t as the Negated Form of “Be”

(e) Invariant Don’t

(f) Habitual Be

(g) Double Modal

(h) Null Copula

(i) Negative Concord

Figure 6. Distribution of Monk Skin Tones for all features and all prompts.

###  B.2. Results For Prompts Conditioned on Male Gendered Subjects

(a) Finna as a Semi-Modal

(b) Completive Done

(c) Quotative All

(d) Ain’t as the Negated Form of “Be”

(e) Invariant Don’t

(f) Habitual Be

(g) Double Modal

(h) Null Copula

(i) Negative Concord

Figure 7. Distribution of Monk Skin Tones for all features, conditioned on the
prompt specifying male gendered subjects.

###  B.3. Results For Prompts Conditioned on Female Gendered Subjects

(a) Finna as a Semi-Modal

(b) Completive Done

(c) Quotative All

(d) Ain’t as the negated “Be”

(e) Invariant Don’t

(f) Habitual Be

(g) Double Modal

(h) Null Copula

(i) Negative Concord

Figure 8. Distribution of Monk Skin Tones for all features, conditioned on the
prompt specifying female gendered subjects.

###  B.4. Results For Prompts Conditioned on Unspecified Gender Subjects

(a) Finna as a Semi-Modal

(b) Completive Done

(c) Quotative All

(d) Ain’t as the Negated “Be”

(e) Invariant Don’t

(f) Habitual Be

(g) Finna as a Semi-Modal

(h) Null Copula

(i) Negative Concord

Figure 9. Distribution of Monk Skin Tones for all features, conditioned on the
prompt not specifying the subjects’ gender.