# Balancing Act: Distribution-Guided Debiasing in Diffusion Models

Rishubh Parihar ,1 Abhijnya Bhat∗,1 Saswat Mallick1 Abhipsa Basu1 Jogendra
Nath Kundu2 R. Venkatesh Babu1 Equal contribution 1Indian Institute of
Science, Bangalore 2Meta Reality Labs

###### Abstract

Diffusion Models (DMs) have emerged as powerful generative models with
unprecedented image generation capability. These models are widely used for
data augmentation and creative applications. However, DMs reflect the biases
present in the training datasets. This is especially concerning in the context
of faces, where the DM prefers one demographic subgroup vs others (eg. female
vs male). In this work, we present a method for debiasing DMs without relying
on additional data or model retraining. Specifically, we propose Distribution
Guidance, which enforces the generated images to follow the prescribed
attribute distribution. To realize this, we build on the key insight that the
latent features of denoising UNet hold rich demographic semantics, and the
same can be leveraged to guide debiased generation. We train Attribute
Distribution Predictor (ADP) - a small mlp that maps the latent features to
the distribution of attributes. ADP is trained with pseudo labels generated
from existing attribute classifiers. The proposed Distribution Guidance with
ADP enables us to do fair generation. Our method reduces bias across
single/multiple attributes and outperforms the baseline by a significant
margin for unconditional and text-conditional diffusion models. Further, we
present a downstream task of training a fair attribute classifier by
rebalancing the training set with our generated data. Project Page

##  1 Introduction

Figure 1: a) Random sampling from a pretrained DM [33] generates images with
gender imbalance. b) Proposed method takes a user-defined reference attribute
$\mathbf{p^{a}_{ref}}$ and performs distribution guidance on a pretrained DM.
c) Sampling with distribution guidance results in fair generation that follow
user define $\mathbf{p^{a}_{ref}}$.

Recent advancements in Diffusion Models (DM) [11, 33, 42] have garnered much
interest among researchers in evaluating the quality of the generated content.
These models are not only used to generate realistic content but also to
augment real datasets [45, 24] for downstream tasks. However, these popular
models have been found to synthesize biased content with respect to multiple
demographic factors like gender, race, age, _etc_., which can be detrimental
to society once these models are deployed in the real world [29, 21, 34]. The
problem is largely caused by the images used to train these models, as the
outputs of these models are governed by these training datasets [29, 22].
Effects of such harmful biases have been shown by multiple recent works
involving studies on not only DMs [29, 21, 34], but also GANs and other
generative models [22, 5, 12]. In fact, Perera et al. [29] show that
unconditional DMs–even when trained with balanced data–amplify racial biases,
leading to the generation of more white-skinned faces than dark-skinned ones.
Biases in the generated data are even more evident in large text-to-image DMs,
e.g., models mostly tend to generate a specific gender with a given profession
(like male and doctor) [23, 52, 39].

Existing works on debiasing, either require a reference dataset [5, 52] and/or
allow retraining of the model [5, 50, 51]. On the contrary, our work aims to
mitigate biases in both unconditional and conditional DMs, enabling fair
generation without model retraining. We propose a setting where we provide a
reference attribute distribution ($\mathbf{p^{a}_{ref}}$) for sensitive
attribute $a$ and query the DM to sample images following
$\mathbf{p^{a}_{ref}}$. This framework allows for adapting any existing DM to
a pre-defined distribution. _E.g_., a user can provide $\mathbf{p^{a}_{ref}}$
as a uniform histogram of sensitive attributes to generate balanced attribute
distribution. We believe defining $\mathbf{p^{a}_{ref}}$ provides just enough
information to condition the DM for fair generation. This is an extremely
practical setting for debiasing and is particularly important for large text-
to-image DMs [33, 36, 32] where retraining or fine-tuning is computationally
intensive.

One plausible approach for fair generation is to guide every generated sample
with attribute classifiers following classifier guidance [8]. However, such a
framework, though simple, is overly restrictive as it requires presetting and
enforcing attributes for each sample individually (which we call sample
guidance). Such constraints during denoising result in inferior generation
quality, as discussed in Sec.4.1. Instead, we propose to jointly denoise a
whole batch of samples and guide the process with $\mathbf{p^{a}_{ref}}$
(which we call distribution guidance). Specifically, we push the generated
batch attribute distribution $\mathbf{{p^{a}_{\theta}}}$ and
$\mathbf{p^{a}_{ref}}$ close to each other during the reverse process.
Distribution guidance provides more flexibility to each sample during
generation as it does not enforce a preset of attributes on a sample basis.
This results in fair generation without sacrificing the generation quality.

A major challenge for guidance-based conditioning is that it requires separate
image classifiers for each noise scale of the diffusion process. To overcome
this, we propose to perform guidance in semantically rich features space -
h-space [18] of DMs. Specifically, we train an Attribute Distribution
Predictor (ADP) that predicts attribute distribution directly from the h-space
features. As ADP is trained on rich and discriminative h-space features, it is
- a) implemented as a linear layer, b) requires minimal training data, and c)
fast training and inference. Finally, during inference, we steer the h-space
representation by matching the predictions from ADP to $\mathbf{p^{a}_{ref}}$.

We extensively evaluate our proposed method for the fair generation of single
and multi-attribute cases for face generation. Additionally, we present the
results of our method on Stable Diffusion [33], a large text-to-image DM.
Further, as downstream application train debiased attribute classifiers by
augmenting the training data for minority subgroups.

The major contributions of this work are the following:

  1. 1.

A novel setting for debiasing existing DMs without retraining given a
reference attribute distribution.

  2. 2.

Distribution guidance to condition the reverse diffusion process on a
reference attribute distribution.

  3. 3.

Propose guidance in the intermediate features of diffusion network (h-space),
which leads to data-efficient training and fast generation.

##  2 Related Works

Biases in Generative Models. While generative models like Generative
Adversarial Networks and Diffusion Models have become the de-facto tools for
image generation in recent times, studies show that they are not free of
biases [29, 21, 34, 22, 13]. Perera et al. [29] show that unconditional
diffusion models amplify the biases in the training data with respect to
gender, race, and age. Luccioni et al. [21] identify and quantify social
biases in images generated by popular text-to-image models like DALL-E 2, and
Stable Diffusion v1.4 and 2. Maluleke et al. [22] study racial biases in GANs
and find that GANs mimic the racial distribution in the training data.

Debiasing generative models by retraining. This line of work focuses on
mitigating biases in generative models by retraining them [27, 50, 47, 38, 51,
5, 44, 46]. Some of these works [50, 38, 47] assume knowledge of the labels of
the sensitive attribute and then debias the models such that there is no
correlation between the decision attribute and the sensitive attribute. IMLE-
GAN [51] ensures coverage of minority groups by combining GAN adversarial
training with Implicit Maximum Likelihood Estimation (IMLE) [20]. Another body
of works employs a balanced unlabelled reference dataset to ensure unbiased
generations [5, 44, 46]. Choi et al. [5] use a density-ratio based technique
to identify the bias in datasets via the reference dataset, and learn a fair
model based on importance reweighting with the help of both the original
biased and the reference dataset. To capture the distance between the small
reference data and the generated data, Um et al. [46] use the LeCam Divergence
[19]. On the other hand, Teo et al. [44] introduce a transfer learning
approach to solve this problem by training the model on the biased dataset
first and then adapting the model to the reference set.

Debiasing generative models without training. As training of GANs and DMs can
be resource-consuming, many methods prefer fair generation of images without
explicit training [31, 43, 24, 12]. MaGNET [12] aims to produce uniform
sampling on the learned manifold of any generative model like GANs or VAEs,
while Ramaswamy et al. [31] and Tan et al. [43] manipulate the latent space of
GANs to generate balanced outputs. GANDiffFace [24], on the other hand,
generates balanced synthetic for face recognition by first generating high-
quality images from different demographics using GANs and then finetuning
Stable Diffusion [33] using DreamBooth [35] to generate more images of such
identities with different poses, expressions, etc. Multiple works attempt to
mitigate biases in vision-language models and text-conditioned diffusion
models as well [6, 52, 41, 53, 3]. Chuang et al. [6] debias the text embedding
using a calibrated projection matrix and shows unbiased generations without
any additional training or data. However, debiasing the unconditional DMs has
received less attention, which is the main focus of this work.

Guidance in Diffusion Models. One of the primary techniques to condition the
diffusion model is to guide the reverse diffusion model with the gradients of
additional network [8]. GLIDE [28] used CLIP [30] based guidance for open
world caption to image generation. Guidance is used for image-to-image
translation [49], counterfactual generation [37]. However, guidance in its
original form requires retraining of guiding networks on the noisy data from
scratch. Few works overcome this by learning a mapping function from the
diffusion feature space to sketches for sketch-guidance [48] and a universal
guidance that repurposes pretrained networks for guidance [1].

##  3 Method

We assume a setting where we are given a pretrained DM trained on biased data
and a reference distribution of the sensitive attributes
$\mathbf{p^{a}_{ref}}$. Our goal is to generate data from the DM, whose
generated attribute distribution $\mathbf{p^{a}_{\theta}}$ best approximates
the reference $\mathbf{p^{a}_{ref}}$ without retraining.

The key idea is to jointly guide the denoising of a batch of samples such that
$\mathbf{p^{a}_{ref}}$ $\approx$ $\mathbf{p^{a}_{\theta}}$. Directly computing
$\mathbf{p^{a}_{\theta}}$ in a closed form is intractable. Instead, we train
an Attribute Distribution Predictor, a small network that maps the
intermediate batch features of a diffusion network to an estimate of attribute
distribution $\mathbf{\hat{p}^{a}_{\theta}}$.

Figure 2: Sample guidance vs Distribution guidance. a) After a few steps of
denoising ($t=\tau$), the generated samples have learned some discriminative
features for gender. Sample guidance enforces a preset attribute state
(male/female) to all the samples in the batch. This is stringent and can
result in enforcing samples with dominant female features to be changed.
However, in distribution guidance, only those samples are converted, which are
easy to convert and are close to the decision boundary. b) Visualization of
the generated samples with different guidance strengths. It can be observed
that sample guidance is not able to change the attribute in some cases, and
increasing guidance strength results in collapse.

###  3.1 Preliminary

Diffusion models have emerged as a powerful family of generative models
trained to learn the data distribution by gradual denoising from a Gaussian
distribution. Starting from a clean point $\mathbf{x_{0}}$, and a set of
scalar values $\\{\alpha_{t}\\}_{t=1}^{T}$, applying $\mathbf{t}$ steps of the
forward diffusion process yields a noisy data point, where $\bar{\alpha_{t}}$
= $\prod_{i=1}^{t}\alpha_{i}$

$\mathbf{x_{t}}=\sqrt{\bar{\alpha_{t}}}\mathbf{x_{0}}+(\sqrt{1-\bar{\alpha_{t}}})\epsilon,\epsilon\approx\mathcal{N}(0,I)$
(1)

A diffusion model is learned as a neural network $\epsilon_{\theta}$ that
predicts the noise from given $\mathbf{x_{t}}$ and $\mathbf{t}$. The reverse
process takes the form $q(\mathbf{x_{t-1}}|\mathbf{x_{t}},\mathbf{x_{0}})$,
which is parameterized as a Gaussian distribution. In this work, we consider
DDIM [42] sampling which first computes a clean data point
$\mathbf{\hat{x}_{0}}$ and then sample $\mathbf{x_{t-1}}$ from
$q(\mathbf{x_{t-1}}|\mathbf{x_{t}},\mathbf{\hat{x}_{0}})$.

Classifier guidance is proposed to condition a diffusion model on class labels
with the help of a pretrained classifier [8]. Specifically, a classifier
$f_{\phi}(c|\mathbf{x_{t}},t)$ is trained on noisy images at different noise
scales, where $c$ is the class label. The gradients of the classifier are then
used to guide the diffusion sampling process towards the class label $c$.
Concretely, the classifier guidance performs sampling by updating the noise
prediction $\epsilon_{\theta}(\mathbf{x_{t}},\mathbf{t})$.

$\hat{\epsilon}_{\theta}(\mathbf{x_{t}},t)=\epsilon_{\theta}(\mathbf{x_{t}},t)-\sqrt{1-\alpha_{t}}\nabla_{\mathbf{x_{t}}}\log
f_{\phi}(c|\mathbf{x_{t}},t)$ (2)

###  3.2 Classifier guidance for debiasing

A promising approach is to leverage pretrained attribute classifiers to guide
toward balanced generation. Assuming a reference attribute distribution
$\mathbf{p^{a}_{ref}}$ for a binary attribute $\mathbf{a}$ (e.g., gender) and
corresponding attribute classifier ${f}_{\phi}$. We parameterize
$\mathbf{p^{a}_{ref}}$ as a Bernoulli distribution with parameter $\mathbf{r}$
denoting fraction of males samples. To generate samples following
$\mathbf{p^{a}_{ref}}$, we can randomly select $\mathbf{Nr}$ samples from the
batch size of $\mathbf{N}$ and guide them towards male class using predictions
from ${f}_{\phi}$. Similarly, for the remaining $\mathbf{N(1-r)}$ samples, we
guide them towards female class. In practice, such a sample guidance follows
$\mathbf{p^{a}_{ref}}$ up to some extent, but results in inferior sample
quality (see Fig. 2b).

Insight. 1: Attempting to steer the generation of individual samples towards a
pre-defined attribute class is overly restrictive and leads to inferior
generation quality.

Remark: Sample guidance requires enforcing a preset attribute state (male /
female) to each sample of a batch during the denoising process, which is too
stringent and results in distorting the outputs. Given an intermediate time-
stamp $\tau$, the samples in the earlier stages ($t>\tau$) of reverse
diffusion process are close to the noise space, resulting in the poor
classifier of the corresponding stages. The guidance becomes effective only in
the later stage of denoising ($t<\tau$). However, till timestep $\tau$, some
facial features are already formed for the samples [25]. Hence, enforcing a
preset attribute state at later stages ($t<\tau$) is restrictive and results
in sample collapse (Fig. 2-b)). For e.g., for gender attribute, if a sample
has formed dominant female features till $\tau$, enforcing it to be $male$ is
overly restrictive, and classifier guidance distorts the generated sample.

###  3.3 Distribution Guidance

We propose an alternate guidance strategy termed Distribution Guidance for
fair generation, which provides more flexibility to modify the attribute
states during the denoising process. The key idea is to jointly denoise a
batch of samples $\mathbf{x_{T}^{[1:N]}}$, with the guidance from reference
attribute distribution $\mathbf{p^{a}_{ref}}$. To realize distribution
guidance, we define a differentiable distribution prediction function
$g_{\psi}$ that maps the batch samples to an estimate of generated attribute
distribution $\mathbf{\hat{p}^{a}_{\theta}}$. We learn the function $g_{\psi}$
over an intermediate feature space - h-space [18] of denoising network instead
of image space for efficiency (Sec.3.4). Hence, the batch estimate is given
by,
$\mathbf{\hat{p}^{a}_{\theta}}=g_{\psi}(\mathbf{h_{t}^{[1:N]}},\mathbf{t})$,
where $\mathbf{h_{t}^{[1:N]}}$ is the bottleneck U-Net feature representation
of the batch samples $\mathbf{x_{t}^{[1:N]}}$. Further, we define a loss
function $\mathcal{L}(.)$ that measures the similarity of two distributions.
During denoising we guide the batch of samples to bring
$\mathbf{\hat{p}^{a}_{\theta}}$ closer to the reference, i.e.,
$\mathcal{L}(\mathbf{\hat{p}^{a}_{\theta}},\mathbf{p^{A}_{ref}})\approx 0$.
This can be easily integrated into the reverse diffusion process as an
extension of classifier guidance by modifying Eq.2.

Insight.2: Distribution guidance provides flexibility to batch samples to
easily modify the attributes to match the defined attribute distribution.

Remark: As distribution guidance does not require presetting for attribute
states for each sample, it gives more flexibility during the generation as
long as $\mathbf{\hat{p}^{a}_{\theta}}$ follows $\mathbf{p^{a}_{ref}}$.
_E.g_., in our running example of gender attribute, only those samples will
change gender after $t=\tau$, which are close to the decision boundary. In
contrast to sample guidance, where random samples are forced to change the
attribute state (Fig. 2) resulting in inferior quality, distribution guidance
is able to generate fair samples with good quality. Observe that distribution
guidance covers the samples that are close to the decision boundary for gender
classification, resulting in good outputs.

A major design decision for implementing distribution guidance is the
selection of function $g_{\psi}$. The conventional approach is to train
multiple attribute classifiers at each level of noise in the image space.
Instead, we propose to leverage intermediate semantic features from the
diffusion model itself and repurpose them for the task of attribute
classification. This framework is super efficient as compared to image space
classifiers.

###  3.4 Guidance in the H-space

Diffusion models, although trained for image generation, learn semantically
rich representations in the internal features of denoising network
$\mathbf{{\epsilon_{\theta}}}$. These representations have been successfully
used for segmentation [2] and classification [26]. Motivated by this, we ask
can we use these internal features for training guidance classifiers? To
answer this, we take features from the bottleneck layer of the diffusion
U-Net, termed as the h-space [18] and attach a linear head for classification.
The trained classifiers achieve good classification performance across
multiple attributes, as shown in Fig. 3. Hence, we use the trained $h-space$
classifiers to realize $g_{\psi}$. Specifically, we train a network Attribute
Distribution Predictor that maps the batch h-space features to an estimate of
attribute distribution $\mathbf{\hat{p}^{a}_{\theta}}$.

Figure 3: Classification accuracy for linear h-space classifiers and ResNet-18
image space classifiers trained on $2K$ training examples. h-space classifiers
are data efficient and achieve superior performance even with a linear layer
Figure 4: Distribution guidance in the h-space. For a given batch
$\mathbf{x_{t}^{[1:N]}}$, we extract the intermediate h-space representation
$\mathbf{h_{t}^{[1:N]}}$ and pass it through ADP to obtain attribute
distribution $\mathbf{\hat{p^{a}_{\theta}}}$. Guidance updates
$\mathbf{h_{t}^{[1:N]}}$ by backpropagating the derivative of loss.

Attribute Distribution Predictor (ADP) is realized via a linear attribute
classifier conditioned on the diffusion time step $\mathbf{t}$. Given a batch
of generating samples $\mathbf{x_{t}^{[1:N]}}$, we extract the corresponding
h-space features $\mathbf{h_{t}^{[1:N]}}$ =
$\epsilon^{E}_{\theta}(\mathbf{x_{t}^{[1:N]}},\mathbf{t})$, where
$\epsilon^{E}_{\theta}$ is the encoder of U-Net network. Next, we pass the
batch $\mathbf{h_{t}^{[1:N]}}$ to the attribute classifier and obtain a batch
of softmax predictions. The softmax predictions are aggregated per class to
obtain the attribute distribution estimate $\mathbf{\hat{p}^{a}_{\theta}}$.
Finally, we update the intermediate h-vectors with the gradients of the
distribution matching loss $\mathcal{L}$, where $\mathbf{\gamma}$ is the
guidance strength parameter:

$\displaystyle\mathbf{\tilde{h}_{t}^{[1:N]}}=\mathbf{h_{t}^{[1:N]}}$
$\displaystyle-\gamma*\nabla_{\mathbf{h_{t}^{[1:N]}}}\mathcal{L}(\mathbf{\hat{p}^{a}_{\theta}},\mathbf{p^{a}_{ref}})$
$\displaystyle\mathbf{\hat{p}^{a}_{\theta}}=$ $\displaystyle
g_{\psi}(\mathbf{h_{t}^{[1:N]}},\mathbf{t})$

Finally, we obtain the distribution guided noise predictions
$\tilde{\epsilon}$ by passing the $\mathbf{h^{[1:N]}_{t}}$ through the U-Net
decoder $\epsilon^{D}_{\theta}$, _i.e_.,
$\tilde{\epsilon}(\mathbf{x^{[1:N]}_{t}},\mathbf{t})=\epsilon^{D}_{\theta}(\mathbf{h^{[1:N]}_{t}},\mathbf{t})$.
The predicted noise is then used to update batch $x^{[1:N]}_{t-1}$ using DDIM
[42].

Insight.3: H-space guidance is extremely effective and efficient as compared
to image space guidance.

Remark. As compared to conventional image space guidance, guidance in h-space
has multiple advantages: i) it requires only a set of linear layers to
implement the classifiers, ii) it is fast to backpropagate during guidance as
compared to image models, iii) highly data-efficient and can be trained with
only a few thousand examples due to semantically rich h-space. In the
experiments section, we compare these properties of the h-space guidance.

##  4 Experiments

In this section, we first discuss the evaluation metrics, followed by the
implementation details. We take face datasets as the subject of study, as they
are subject to very high demographic biases. We compare our method with other
inference time debiasing approaches for both single-attribute and multi-
attribute debiasing. Further, we present detailed ablations to validate design
choices. Finally, we show the generalization of our debiasing approach on
large text-to-image generation model Stable Diffusion [33].

###  4.1 Evaluation Metrics

A fair generative model is evaluated on two grounds: image quality and
fairness. Therefore, we begin by discussing the metrics we use to measure
these two attributes of the generated images.

Fairness. The primary goal of this paper is to generate images based on the
reference distribution $\mathbf{p_{ref}^{a}}$. We follow Choi et al. [5] to
define the Fairness Discrepancy (FD) metric. Given an attribute $\mathbf{a}$,
we assume access to a high-accuracy classifier for $\mathbf{a}$ (denoted as
$\mathcal{C}_{a}$), and using the predictions from the latter, we compute the
following [5, 44]:

$|\bar{p}-\mathbb{E}_{\mathbf{x}\sim p_{\theta}(\mathbf{x})}(\mathbf{y})|_{2}$

where $\mathbf{y}$ is the softmax output of the classifier
$\mathcal{C}_{a}(\mathbf{x})$, $\bar{p}$ is a uniform vector of the same
dimension as $\mathbf{y}$, $p_{\theta}$ is the distribution of the generated
images. The lower the FD score, the closer the distribution of the attribute
values is to the uniform distribution – i.e., the generated images are fairer
with respect to attribute $\mathbf{a}$.

Image Quality. To measure the quality of generation, we follow the standard
Fréchet Inception Distance (FID) [10]. We compute this metric with the help of
an attribute-wise balanced dataset sampled from the original training data.

###  4.2 Implementation Details

Training h-space classifiers. We start with creating a paired dataset
$\mathcal{D}^{h}_{clf}$ of h-vector and attribute labels. Specifically, we
take a subset $\mathcal{D}$ of the CelebA-HQ [15] dataset and obtain attribute
labels for the images using the pretrained attribute classifier
$\mathcal{C}_{a}$. Next, we embed image $\mathbf{I^{i}}\in\mathbf{D}$ to
obtain the corresponding h-space representation
$\mathbf{\mathcal{H}^{i}}=\\{\mathbf{h^{i}_{t}}\\}^{t=T}_{t=0}$ using DDIM
[42] inversion. This yields a labelled dataset $\mathcal{D}^{h}_{clf}$ with
pairs $(\mathcal{H}^{i},\mathbf{y^{i}})$, where
$\mathbf{y^{i}}=\mathcal{C}_{a}(I^{i})$ is the predicted attribute label for
image $\mathbf{I^{i}}$ (_e.g_. male / female). Next, we train h-space
attribute classifiers $\mathcal{C}^{h}_{a}(\mathbf{h_{t}},t)$ as a linear head
over $\mathbf{h_{t}}$ and conditioned on time $t$ (ignored sample index $i$
for brevity). These obtained classifiers generate high-accuracy attribute
predictions, as shown in Fig. 3. Further details about the dataset and
classifiers are provided in a supplementary document.

Figure 5: Balancing generated data across gender and eyeglass attributes with
various guidance strategies. Minority group $male$ and $eyeglasses$ are
marked. Image-space Universal Guidance and Sample Guidance results generate
imbalanced and poor-quality images. For gender, Sample Guidance is able to
generate only a few males while maintaining quality. The proposed Distribution
Guidance balances (close to the ratio of $0.50:0.50$) both the attributes and
generates high-quality images.

Distribution guidance. To perform distribution guidance, we realize the
distribution prediction function, ADP with h-space classifiers
$\mathcal{C}^{h}_{a}$. Specifically for a generating batch
$\mathbf{x_{t}^{[1:N]}}$, we obtain the corresponding $h-space$ representation
$\mathbf{h_{t}^{[1:N]}}$ and obtain a set of attribute predictions
$\mathbf{\hat{y}^{[1:N]}_{t}}$ from classifiers $\mathcal{C}^{h}_{a}$.
Finally, we add all the softmax values for all the $N$ images in the batch for
each class to obtain the estimate of $\mathbf{p^{a}_{\theta}}$. We use Chi-
square distance as the loss $\mathcal{L}$ with the reference distribution
$\mathbf{p^{a}_{ref}}$.

Diffusion Model Architecture. We evaluated our proposed debiasing method for
fair generation from two state-of-the-art pretrained DMs: a) an unconditional
DM, P2 [4], trained on CelebA-HQ dataset, and b) a text conditional DM
StableDiffusion v1.5 [33] trained on the LAION [40] dataset. Both these models
have exceptional image generation quality; however, they have significant bias
concerning the sensitive face attributes, as shown in the following
subsections.

###  4.3 Evaluation and Comparisons.

We compare our proposed method against two techniques for guidance-based
generations for DMs and a state-of-the-art sampling-based technique for
debiasing generative models, MagNet [12].

Sample guidance. We use classifier guidance as explained in Sec. 3.2 in the
h-space using the trained h-space attribute classifiers $\mathcal{C}_{a}^{h}$.
Such guidance requires presetting an attribute state for each generating image
and pushing the trajectory to change the attribute state.

Universal guidance [1] performs guidance in the image space but uses
pretrained classifiers trained on the clean image. This resolves the
additional requirement of training image classifiers on noisy images. The key
idea is to use the DDIM scheduler and predict the approximation of
$\mathbf{\hat{x_{0}}}$ from noise image $\mathbf{x_{t}}$, and pass it through
pretrained image classifier $\mathcal{C}_{A}$. However, this process has two
shortcomings. Firstly, it is slow as we have to backpropagate the gradients
from the image classifier. Experimentally, we found the h-space guidance to be
$7$ times faster than the image guidance. Secondly, it performs poorly at the
early stage due to an inaccurate approximation of $\mathbf{x_{0}}$. We used
the same training set of $|\mathcal{D}_{clf}|=2K$ to train the h-space and
image space attribute classifiers for a fair comparison.

Table 1: Evaluation of balanced generation for single attribute

Gender Race Eyeglasses Metric FD FID FD FID FD FID Random Sampling 0.178 54.59
0.334 60.01 0.251 75.21 Universal Guidance [1] 0.193 52.10 0.377 93.42 0.189
64.55 H-Sample Guidance 0.113 51.46 0.184 56.53 0.118 57.63 H-Distribution
Guidance 0.049 50.27 0.113 52.38 0.014 51.78 Random sampling* 0.307 112.28
0.463 123.97 0.276 117.83 Magnet* 0.267 91.15 0.454 97.05 0.281 106.55

Quantitative evaluation. We evaluate our debiasing method for the single
attribute case by generating balanced distributions of individual sensitive
attributes - gender, eyeglasses, and race, in Tab. 1. For the scope of this
paper, we consider all these attributes as binary, and hence expect a $1:1$
ratio of the sensitive attributes in the synthesized images (for e.g. $0.50$
fraction of males and $0.50$ fraction of females in case of gender).
Specifically, we generate $10K$ images from each method per attribute and
compute the metrics defined in Sec. 4.1. The proposed guidance method
outperforms all the methods in terms of visual quality as measured by FID and
bias generation measured by FD. Sample guidance achieves comparable FID for
gender; however, higher FD indicates inferior debiasing. The tradeoff between
FD and FID is discussed in detail in subsection 4.5. This supports our thesis
that the distribution guidance provides enough flexibility during generation,
resulting in high-quality outputs even with a high guidance scale.

Multiple attributes. Additionally, we apply our method for debiasing multiple
attributes simultaneously in Tab. 2. Specifically, given two reference
distributions $\mathbf{p^{a_{1}}_{ref}}$ and $\mathbf{p^{a_{2}}_{ref}}$, we
add guidance from two pretrained attribute distribution predictors
$g_{\psi_{1}}$ and $g_{\psi_{2}}$. For this experiment, we define both the
reference distribution as uniform for each attribute ($50\%$-$50\%$ splits).
The generated results follow the reference distribution, resulting in a
balanced generation across attributes.

Table 2: Evaluation of balanced generation for multiple attribute

Gender + Race Eyeglasses + Race Gender + Eyeglasses Metric FD $\downarrow$ FID
$\downarrow$ FD $\downarrow$ FID $\downarrow$ FD $\downarrow$ FID $\downarrow$
Random Sampling 0.256 60.68 0.292 89.14 0.214 70.97 Universal Guidance 0.283
71.84 0.264 91.54 0.157 80.57 H-Sample Guidance 0.241 59.78 0.135 67.87 0.079
52.03 H-Distribution Guidance 0.075 49.91 0.101 57.46 0.057 47.45

Comparison with MagNet [12]: MagNet is an unsupervised method enabling fair
sampling from a pretrained model. They propose a method for uniform sampling
on the image manifold to generate under-represented groups equally. We
generated results by MagNet sampling from a StyleGAN2 [17] model trained on
the FFHQ [16] dataset from their official codebase. We compare our method with
MagNet [12] for the task of fair generation in Tab. 1. We used MagNet to
generate $10K$ samples and compute the FD score. As the base model for MagNet
is StyleGAN2, we cannot compare FID directly with our DM debiased results.
However, we can observe that MagNet achieves similar FD scores to its baseline
model and is ineffective in debiasing the three sensitive attributes: gender,
race, and eyeglasses. However, our distribution guidance significantly
improves the FD metric from the baseline DM model.

Qualitative evaluation. We present qualitative results for attribute balancing
with respect to gender and eyeglasses in Fig. 5. We randomly sample $20$
starting noise and use individual guidance methods for debiasing. Without
guidance, the DM mostly generates female faces for the gender attribute.
Sample guidance and universal guidance, although increasing the number of
males, collapses some images. On the other hand, Distribution guidance
generates an almost equal number of males and females without affecting the
generation quality. Moreover, the DM hardly generates images with eyeglasses
for sample and universal guidance and also without any guidance. Our method,
on the contrary, leads to highly balanced generations.

###  4.4 Generating Imbalanced Distributions

We tested our distribution guidance for generating imbalanced attribute
distribution by providing skewed $\mathbf{p^{a}_{ref}}$ \- i) $0.20$ female
and $0.80$ male ii) $0.10$ white race and $0.90$ black race. These two
settings are extremely challenging given that male and black race are minority
groups in the dataset. We present qualitative results for both the settings in
Fig.6, where our distribution guidance is able to generate the defined
distribution with majority males. Note, we have binarized the race attribute
as black and whites for simplicity, and hence in the generation brown race is
considered under black category. We report quantitative metrics in Tab. 3,
where the proposed method can achieve good FD scores with the same FID.

Figure 6: Generating imbalanced data with two non-uniform reference
distribution - i) 0.9B-0.1W, ii) 0.8M-0.2F. Minority classes are marked.
Distribution guidance generates high-quality images and closely follows the
skewed reference distribution. Table 3: Distribution guidance for imbalanced
generation

0.20F - 0.80M 0.10W - 0.90B Method FD FID FD FID Random Sampling 0.478 72.26
0.734 77.63 H-Distribution Sampling 0.168 51.65 0.325 53.80

###  4.5 Ablations

Guidance Strength $\gamma$. We ablate over guidance strength parameter
$\gamma$ for both the distribution guidance and sample guidance in Fig. 7.
Increasing $\gamma$ reduces the bias (better FD) at the cost of inferior image
quality (increased FID). Guidance strength $1500$ achieved a good tradeoff
between image quality and balancing attributes. Further, as compared to sample
guidance, the distribution guidance achieves a better tradeoff for all the
guidance strengths.

Figure 7: Ablation over guidance strength $\gamma$

Batch size. As we approximate $\mathbf{p^{a}_{\theta}}$ with a an estimate
$\mathbf{\hat{p}^{a}_{\theta}}$ over a batch of size $N$, we ablate over
different values of $N$ in Tab. 4. Intuitively, using a larger batch size
yields a better estimate of $\mathbf{p^{a}_{\theta}}$. We have found that the
$N=100$ works best for all our experiments.

Table 4: Ablation over batch size for gender balancing.

Batch size 10 25 50 75 100 FD $\downarrow$ 0.062 0.059 0.049 0.059 0.046 FID
$\downarrow$ 50.54 51.51 50.86 50.98 51.64

Figure 8: Debiasing results on stable diffusion for gender attribute. The
proposed distribution guidance can balance the gender attributes in the
neutral prompt and generalizes well to prompts involving other professions
_e.g_. firefighter, doctor.

###  4.6 Experiments on Stable Diffusion

We implemented our distribution guidance technique for debiasing a text-to-
image generation model, Stable Diffusion (SD) v1.5 [33] concerning the gender
attribute in this subsection. First, we generate a dataset $\mathcal{D}$, of
$10K$ images from SD, with prompts ‘a photo of a male’ and ‘a photo of a
female’ to generate a labeled dataset for training h-space classifiers. If we
use the same CelebA-HQ dataset to train the h-space classifier, the guidance
is ineffective due to a significant domain shift from the SD generations.
Next, we obtain the corresponding labeled dataset $\mathcal{D}^{h}_{clf}$ in
the h-space and train a gender classifier in the h-space. We used the trained
classifier for distribution guidance following Sec. 3.4 to generate images
with balanced gender.

It is observed that SD increases the gender bias when queried to generate
certain professions (_e.g_. male and doctor) [52]. To this end, we implement
our distribution guidance along the prompts ‘a photo of a doctor’ and ‘a photo
of a firefighter’ to evaluate the effectiveness in this challenging setting.
The qualitative results are shown in Fig.8, and the quantitative results are
reported in Tab. 5. We can observe, as compared to the baseline (random
generation without guidance), our method generates fairer gender distribution
with both neutral prompts and prompts mentioning professions (_e.g_. ‘a photo
of a firefighter’). Additional results on text-to-image models are present in
supp. material.

Table 5: Balancing gender on Stable Diffusion [33] model.

Method Gender Doctor Firefighter FD $\downarrow$ FID $\downarrow$ FD
$\downarrow$ FID $\downarrow$ FD $\downarrow$ FID $\downarrow$ Random Sampling
0.317 72.37 0.355 70.11 0.235 71.86 H-Sample Guidance 0.026 70.96 0.021 68.43
0.097 70.42 H-Distribution Guidance 0.024 70.69 0.015 67.36 0.093 69.41

###  4.7 Class-imbalance in attribute classification

We explore an important downstream application of our proposed approach in
balancing minority classes by augmenting the under-represented classes with
generated data. Specifically, we train a race classifier (labels obtained from
$\mathcal{C}_{\mathcal{A}}$) on the CelebA-HQ [14] dataset. The race
classifier is an ImageNet [7]-pretrained ResNet-18 [9] encoder, followed by 2
MLP layers and a classifier. We manually oversample the Whites and undersample
the Blacks in the training dataset such that the imbalanced dataset consists
of $10k$ samples of white people and $1k$ black people (we keep the genders
balanced within a race class). Consequently, the model performs poorly on the
minority class (i.e., Black) due to under-representation. Next, we augment
(class-balanced) the training data by generating samples whose distribution is
inversely proportional to the class counts in the training set to increase
images of minority classes using our distribution guidance approach. This adds
9k images of only Black race, and the classifier trained on this balanced data
performs significantly better (Tab. 6). Even when gender is balanced in both
the classes, we observe a significant disparity in the accuracies for Male and
Female Black samples in the vanilla classifier. However, our proposed method
helps reduce the accuracy gap between Black males and Black females. This
shows a potential application in generating ’class’ balanced datasets to train
models for other downstream tasks, which can also mitigate bias.

Table 6: Group-wise accuracies of identical classifiers trained on existing
and generated(balanced) data

Black female White female Black male White male Vanilla classifier 75.76 98.96
70.29 99.93 Balanced classifier 91.04 97.33 90.72 97.68

##  5 Discussion.

Limitations. Although our method performs guidance in the h-space, which is
efficient compared to the image space guidance, it still requires additional
training of h-space classifiers. Another limitation is reliance on accurate
attribute classifiers to obtain labels for training h-space classifiers.

Future works. An important future work is extending distribution guidance
beyond de-biasing for controlled generation and data augmentation. In the
context of debiasing DMs, extending the proposed approach without needing an
attribute classifier or labeled data.

##  6 Conclusion

In this work, we aim to mitigate biases from pretrained diffusion models
without retraining - given only a desired reference attribute distribution. We
propose a novel approach leveraging distribution guidance that jointly guides
a batch of images to follow the reference attribute distribution. The proposed
method is effective and results in both high-quality and fair generations
across multiple attributes and outperforms sample guidance strategies based on
conditioning each sample individually. Extensive experiments demonstrate the
effectiveness of our method in balancing both single and multiple attributes
on unconditional DMs and conditional text-to-image diffusion models. We
believe such a setting of debiasing without retraining is practical,
especially in today’s era of large-scale generative models.

## References

  * Bansal et al. [2023] Arpit Bansal, Hong-Min Chu, Avi Schwarzschild, Soumyadip Sengupta, Micah Goldblum, Jonas Geiping, and Tom Goldstein.  Universal guidance for diffusion models.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 843–852, 2023.
  * Baranchuk et al. [2021] Dmitry Baranchuk, Ivan Rubachev, Andrey Voynov, Valentin Khrulkov, and Artem Babenko.  Label-efficient semantic segmentation with diffusion models.  _arXiv preprint arXiv:2112.03126_ , 2021.
  * Berg et al. [2022] Hugo Berg, Siobhan Hall, Yash Bhalgat, Hannah Kirk, Aleksandar Shtedritski, and Max Bain.  A prompt array keeps the bias away: Debiasing vision-language models with adversarial learning.  In _Proceedings of the 2nd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 12th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_ , pages 806–822, Online only, 2022. Association for Computational Linguistics.
  * Choi et al. [2022] Jooyoung Choi, Jungbeom Lee, Chaehun Shin, Sungwon Kim, Hyunwoo Kim, and Sungroh Yoon.  Perception prioritized training of diffusion models.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 11472–11481, 2022.
  * Choi et al. [2020] Kristy Choi, Aditya Grover, Trisha Singh, Rui Shu, and Stefano Ermon.  Fair generative modeling via weak supervision.  In _International Conference on Machine Learning_ , pages 1887–1898. PMLR, 2020.
  * Chuang et al. [2023] Ching-Yao Chuang, Varun Jampani, Yuanzhen Li, Antonio Torralba, and Stefanie Jegelka.  Debiasing vision-language models via biased prompts.  _arXiv preprint arXiv:2302.00070_ , 2023.
  * Deng et al. [2009] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei.  Imagenet: A large-scale hierarchical image database.  In _2009 IEEE conference on computer vision and pattern recognition_ , pages 248–255. Ieee, 2009.
  * Dhariwal and Nichol [2021] Prafulla Dhariwal and Alexander Nichol.  Diffusion models beat gans on image synthesis.  _Advances in neural information processing systems_ , 34:8780–8794, 2021.
  * He et al. [2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun.  Deep residual learning for image recognition.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , pages 770–778, 2016.
  * Heusel et al. [2017] Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter.  Gans trained by a two time-scale update rule converge to a local nash equilibrium.  _Advances in neural information processing systems_ , 30, 2017.
  * Ho et al. [2020] Jonathan Ho, Ajay Jain, and Pieter Abbeel.  Denoising diffusion probabilistic models.  _Advances in neural information processing systems_ , 33:6840–6851, 2020.
  * Humayun et al. [2021] Ahmed Imtiaz Humayun, Randall Balestriero, and Richard Baraniuk.  Magnet: Uniform sampling from deep generative network manifolds without retraining.  In _International Conference on Learning Representations_ , 2021.
  * Jain et al. [2020] Niharika Jain, Alberto Olmo, Sailik Sengupta, Lydia Manikonda, and Subbarao Kambhampati.  Imperfect imaganation: Implications of gans exacerbating biases on facial data augmentation and snapchat selfie lenses.  _arXiv preprint arXiv:2001.09528_ , 2020.
  * Karras et al. [2017] Tero Karras, Timo Aila, Samuli Laine, and Jaakko Lehtinen.  Progressive growing of gans for improved quality, stability, and variation.  _arXiv preprint arXiv:1710.10196_ , 2017.
  * Karras et al. [2018] Tero Karras, Timo Aila, Samuli Laine, and Jaakko Lehtinen.  Progressive growing of GANs for improved quality, stability, and variation.  In _International Conference on Learning Representations_ , 2018.
  * Karras et al. [2019] Tero Karras, Samuli Laine, and Timo Aila.  A style-based generator architecture for generative adversarial networks.  In _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , pages 4401–4410, 2019.
  * Karras et al. [2020] Tero Karras, Samuli Laine, Miika Aittala, Janne Hellsten, Jaakko Lehtinen, and Timo Aila.  Analyzing and improving the image quality of stylegan.  In _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , pages 8110–8119, 2020.
  * Kwon et al. [2022] Mingi Kwon, Jaeseok Jeong, and Youngjung Uh.  Diffusion models already have a semantic latent space.  _arXiv preprint arXiv:2210.10960_ , 2022.
  * Le Cam [2012] Lucien Le Cam.  _Asymptotic methods in statistical decision theory_.  Springer Science & Business Media, 2012.
  * Li and Malik [2018] Ke Li and Jitendra Malik.  Implicit maximum likelihood estimation.  _arXiv preprint arXiv:1809.09087_ , 2018.
  * Luccioni et al. [2023] Sasha Luccioni, Christopher Akiki, Margaret Mitchell, and Yacine Jernite.  Stable bias: Evaluating societal representations in diffusion models.  In _Thirty-seventh Conference on Neural Information Processing Systems Datasets and Benchmarks Track_ , 2023.
  * Maluleke et al. [2022] Vongani H Maluleke, Neerja Thakkar, Tim Brooks, Ethan Weber, Trevor Darrell, Alexei A Efros, Angjoo Kanazawa, and Devin Guillory.  Studying bias in gans through the lens of race.  In _European Conference on Computer Vision_ , pages 344–360. Springer, 2022.
  * Mandal et al. [2023] Abhishek Mandal, Susan Leavy, and Suzanne Little.  Measuring bias in multimodal models: Multimodal composite association score.  In _International Workshop on Algorithmic Bias in Search and Recommendation_ , pages 17–30. Springer, 2023.
  * Melzi et al. [2023] Pietro Melzi, Christian Rathgeb, Ruben Tolosana, Ruben Vera-Rodriguez, Dominik Lawatsch, Florian Domin, and Maxim Schaubert.  Gandiffface: Controllable generation of synthetic datasets for face recognition with realistic variations.  _arXiv preprint arXiv:2305.19962_ , 2023.
  * Meng et al. [2022] Chenlin Meng, Yutong He, Yang Song, Jiaming Song, Jiajun Wu, Jun-Yan Zhu, and Stefano Ermon.  SDEdit: Guided image synthesis and editing with stochastic differential equations.  In _International Conference on Learning Representations_ , 2022.
  * Mukhopadhyay et al. [2023] Soumik Mukhopadhyay, Matthew Gwilliam, Vatsal Agarwal, Namitha Padmanabhan, Archana Swaminathan, Srinidhi Hegde, Tianyi Zhou, and Abhinav Shrivastava.  Diffusion models beat gans on image classification.  _arXiv preprint arXiv:2307.08702_ , 2023.
  * Nam et al. [2023] Junhyun Nam, Sangwoo Mo, Jaeho Lee, and Jinwoo Shin.  Breaking the spurious causality of conditional generation via fairness intervention with corrective sampling.  _Transactions on Machine Learning Research_ , 2023.
  * Nichol et al. [2021] Alex Nichol, Prafulla Dhariwal, Aditya Ramesh, Pranav Shyam, Pamela Mishkin, Bob McGrew, Ilya Sutskever, and Mark Chen.  Glide: Towards photorealistic image generation and editing with text-guided diffusion models.  _arXiv preprint arXiv:2112.10741_ , 2021.
  * Perera and Patel [2023] Malsha V Perera and Vishal M Patel.  Analyzing bias in diffusion-based face generation models.  _arXiv preprint arXiv:2305.06402_ , 2023.
  * Radford et al. [2021] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al.  Learning transferable visual models from natural language supervision.  In _International conference on machine learning_ , pages 8748–8763. PMLR, 2021.
  * Ramaswamy et al. [2021] Vikram V Ramaswamy, Sunnie SY Kim, and Olga Russakovsky.  Fair attribute classification through latent space de-biasing.  In _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , pages 9301–9310, 2021.
  * Ramesh et al. [2022] Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen.  Hierarchical text-conditional image generation with clip latents.  _arXiv preprint arXiv:2204.06125_ , 2022.
  * Rombach et al. [2022] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer.  High-resolution image synthesis with latent diffusion models.  In _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , pages 10684–10695, 2022.
  * Rosenberg et al. [2023] Harrison Rosenberg, Shimaa Ahmed, Guruprasad V Ramesh, Ramya Korlakai Vinayak, and Kassem Fawaz.  Unbiased face synthesis with diffusion models: Are we there yet?  _arXiv preprint arXiv:2309.07277_ , 2023.
  * Ruiz et al. [2023] Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Yael Pritch, Michael Rubinstein, and Kfir Aberman.  Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 22500–22510, 2023.
  * Saharia et al. [2022] Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, et al.  Photorealistic text-to-image diffusion models with deep language understanding.  _Advances in Neural Information Processing Systems_ , 35:36479–36494, 2022.
  * Sanchez and Tsaftaris [2022] Pedro Sanchez and Sotirios A Tsaftaris.  Diffusion causal models for counterfactual estimation.  In _Conference on Causal Learning and Reasoning_ , pages 647–668. PMLR, 2022.
  * Sattigeri et al. [2019] Prasanna Sattigeri, Samuel C Hoffman, Vijil Chenthamarakshan, and Kush R Varshney.  Fairness gan: Generating datasets with fairness properties using a generative adversarial network.  _IBM Journal of Research and Development_ , 63(4/5):3–1, 2019.
  * Schramowski et al. [2023] Patrick Schramowski, Manuel Brack, Björn Deiseroth, and Kristian Kersting.  Safe latent diffusion: Mitigating inappropriate degeneration in diffusion models.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 22522–22531, 2023.
  * Schuhmann et al. [2022] Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, et al.  Laion-5b: An open large-scale dataset for training next generation image-text models.  _Advances in Neural Information Processing Systems_ , 35:25278–25294, 2022.
  * Seth et al. [2023] Ashish Seth, Mayur Hemani, and Chirag Agarwal.  Dear: Debiasing vision-language models with additive residuals.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 6820–6829, 2023.
  * Song et al. [2020] Jiaming Song, Chenlin Meng, and Stefano Ermon.  Denoising diffusion implicit models.  In _International Conference on Learning Representations_ , 2020.
  * Tan et al. [2020] Shuhan Tan, Yujun Shen, and Bolei Zhou.  Improving the fairness of deep generative models without retraining.  _arXiv preprint arXiv:2012.04842_ , 2020.
  * Teo et al. [2023] Christopher TH Teo, Milad Abdollahzadeh, and Ngai-Man Cheung.  Fair generative models via transfer learning.  In _Proceedings of the AAAI Conference on Artificial Intelligence_ , pages 2429–2437, 2023.
  * Trabucco et al. [2023] Brandon Trabucco, Kyle Doherty, Max Gurinas, and Ruslan Salakhutdinov.  Effective data augmentation with diffusion models.  _arXiv preprint arXiv:2302.07944_ , 2023.
  * Um and Suh [2023] Soobin Um and Changho Suh.  A fair generative model using lecam divergence.  In _Proceedings of the AAAI Conference on Artificial Intelligence_ , pages 10034–10042, 2023.
  * van Breugel et al. [2021] Boris van Breugel, Trent Kyono, Jeroen Berrevoets, and Mihaela van der Schaar.  Decaf: Generating fair synthetic data using causally-aware generative networks.  _Advances in Neural Information Processing Systems_ , 34:22221–22233, 2021.
  * Voynov et al. [2023] Andrey Voynov, Kfir Aberman, and Daniel Cohen-Or.  Sketch-guided text-to-image diffusion models.  In _ACM SIGGRAPH 2023 Conference Proceedings_ , pages 1–11, 2023.
  * Wolleb et al. [2022] Julia Wolleb, Florentin Bieder, Robin Sandkühler, and Philippe C Cattin.  Diffusion models for medical anomaly detection.  In _International Conference on Medical image computing and computer-assisted intervention_ , pages 35–45. Springer, 2022.
  * Xu et al. [2018] Depeng Xu, Shuhan Yuan, Lu Zhang, and Xintao Wu.  Fairgan: Fairness-aware generative adversarial networks.  In _2018 IEEE International Conference on Big Data (Big Data)_ , pages 570–575. IEEE, 2018.
  * Yu et al. [2020] Ning Yu, Ke Li, Peng Zhou, Jitendra Malik, Larry Davis, and Mario Fritz.  Inclusive gan: Improving data and minority coverage in generative models.  In _Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XXII 16_ , pages 377–393. Springer, 2020.
  * Zhang et al. [2023] Cheng Zhang, Xuanbai Chen, Siqi Chai, Chen Henry Wu, Dmitry Lagun, Thabo Beeler, and Fernando De la Torre.  Iti-gen: Inclusive text-to-image generation.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , pages 3969–3980, 2023.
  * Zhu et al. [2023] Beier Zhu, Yulei Niu, Saeil Lee, Minhoe Hur, and Hanwang Zhang.  Debiased fine-tuning for vision-language models by prompt regularization.  _arXiv preprint arXiv:2301.12429_ , 2023.