¡html¿¡head¿ ¡meta http-equiv=”content-type” content=”text/html;
charset=ISO-8859-1”¿

¡title¿CERN-2010-002¡/title¿

¡/head¿

¡body¿

¡h1¿¡a href=”http://cas.web.cern.ch/cas/Belgium-2009/Bruges-advert.html”¿CAS -
CERN Accelerator School: Magnets¡/a¿¡/h1¿

¡h2¿Bruges, Belgium, 16 - 25 June 2009¡/h2¿

¡h2¿Proceedings - CERN Yellow Report

¡a href=”http://cdsweb.cern.ch/record/1158462”¿CERN-2010-004¡/a¿¡/h2¿

¡h3¿editor: D. Brandt ¡/h3¿

These proceedings present the lectures given at the twenty-third specialized
course organized by the CERN Accelerator School (CAS), the topic being
’Magnets’. The course was held in Bruges, Belgium, from 16 to 25 June 2009.
This is the first time this topic has been selected for a specialized course.
Taking into account the number of related applications currently in use in
accelerators around the world, but, even more important, the worrying decrease
in the corresponding expertise in the different laboratories, it was
recognized that such a topic should definitively be incorporated into the CAS
series of specialized courses. The specific aim of the course was to introduce
the participants to the basics of resistive magnet design and its underlying
theoretical concepts. The first part of the school dealt with basic
introductory courses such as Maxwell’s equations for magnets, beam optics,
physics and measurement of magnetic materials, the different types of
resistive magnets and their respective performance, an introduction to
numerical field computation, and a core lecture on basic magnet design. The
second part of the course focused more on quality control, the different
measurement systems with their electronics, calibration techniques and
respective applications as well as the question of stability and
reproducibility. For the first time, in addition to the actual lectures, a
Case Study was proposed to the participants. This consisted of eight hours of
a guided practical exercise, where the participants had to propose their own
design for a magnet fulfilling the boundary conditions corresponding to a
combined-function magnet developed for the ALBA Synchrotron Light Source in
Barcelona, Spain. This Case Study was enthusiastically received by the
participants, who praised both the proposed approach and the amount of
practical information acquired from this exercise.

¡h2¿Lectures¡/h2¿

¡!– Maxwell’s Equations for Magnets –¿

LIST:arXiv:1103.0713¡br¿

¡!–Physics and measurements of magnetic materials –¿

LIST:arXiv:1103.1069¡br¿

¡!– Basic design and engineering of normal-conducting, iron-dominated
electromagnets –¿

LIST:arXiv:1103.1119¡br¿

¡!– Eddy currents in accelerator magnets –¿

LIST:arXiv:1103.1800¡br¿

¡!–Injection and extraction magnets: kicker magnets –¿

LIST:arXiv:1103.1583¡br¿

¡!–Injection and extraction magnets: septa –¿

LIST:arXiv:1103.1062¡br¿

¡!– Permanent magnets including undulators and wigglers –¿

LIST:arXiv:1103.1573¡br¿

¡!– Specifications, quality control, manufacturing, and testing of accelerator
magnets –¿

LIST:arXiv:1103.1815¡br¿

¡!– Dimensional metrology and positioning operations: basics for a spatial
layout analysis of measurement systems –¿

LIST:arXiv:1104.0799¡br¿

¡!–Dielectric insulation and high-voltage issues –¿

LIST:arXiv:1104.0802¡br¿

¡!– Magnetic measurement with coils and wires –¿

LIST:arXiv:1104.3784¡br¿

¡!– Fabrication and calibration of search coils –¿

LIST:arXiv:1104.0803¡br¿

¡!– Hall probes: physics and application to magnetometry –¿

LIST:arXiv:1103.1271¡br¿

¡!– Magnet stability and reproducibility –¿

LIST:arXiv:1103.1575¡br¿

¡/body¿¡/html¿


m #1[l,vc] *X *X

# Women Wearing Lipstick: Measuring the Bias
Between an Object and Its Related Gender

Ahmed Sabir Lluís Padró
Universitat Politècnica de Catalunya, TALP Research Center, Barcelona, Spain
<EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract

In this paper, we investigate the impact of objects on gender bias in image
captioning systems. Our results show that only gender-specific objects have a
strong gender bias (_e.g_. women-lipstick). In addition, we propose a visual
semantic-based gender score that measures the degree of bias and can be used
as a plug-in for any image captioning system. Our experiments demonstrate the
utility of the gender score, since we observe that our score can measure the
bias relation between a caption and its related gender; therefore, our score
can be used as an additional metric to the existing Object Gender Co-Occ
approach. Code and data are publicly available at
https://github.com/ahmedssabir/GenderScore.

##  1 Introduction

Visual understanding of image captioning is an important and rapidly evolving
research topic Karpathy and Fei-Fei (2015); Anderson et al. (2018). Recent
approaches predominantly rely on Transformer Huang et al. (2019); Cho et al.
(2022) and the BERT based pre-trained paradigm Devlin et al. (2019) to learn
cross-modal representation Li et al. (2020); Zhang et al. (2021); Li et al.
(2022, 2023).

While image captioning models achieved notable benchmark performance in
utilizing the correlation between visual and co-occurring labels to generate
an accurate image description, this often results in a gender bias that
relates to a specific gender, such as confidently identifying a woman when
there is a kitchen in the image. The work of Zhao et al. (2017) tackles the
problem of gender bias in visual semantic role labeling by balancing the
distribution. For image captioning, Hendricks et al. (2018) consider
ameliorating gender bias via a balance classifier. More recently, Hirota et
al. (2022) measured racial and gender bias amplification in image captioning
via a trainable classifier on an additional human-annotated existing caption
dataset for the gender bias task Zhao et al. (2021).

To close the full picture of gender bias in image captioning, in this work,
unlike other works, we examine the problem from a visual semantic relation
perspective. We therefore propose a Gender Score via human-inspired judgment
named Belief Revision Blok et al. (2003) which can be used to (1) discover
bias and (2) predict gender bias without training or unbalancing the dataset.
We conclude our contributions are as follows: (1) we investigate the gender
object relation for image captioning at the word level (_i.e_. object-gender)
and the sentence level with captions (_i.e_. gender-caption); (2) we propose a
Gender Score that uses gender in relation to the visual information in the
image to predict the gender bias. Our Gender Score can be used as a plug-in
for any out-of-the-box image captioning system. Figure 1 shows an overview of
the proposed gender bias measure for image captioning.

Figure 1: An overview of the proposed gender bias measure for image
captioning. (a) word level with word embedding, (b) sentence level with
semantic embedding, (c) our proposed gender score that relies on visual bias
revision (with GPT-2 for initialization
$\mathrm{P}\left(g^{\prime}_{y}\right)$ (common observation as initial bias
without visual) and with BERT for similarity score), and (d) the proposed <
MASK > gender estimation (prediction), also with visual bias revision.

##  2 Visual Context Information

In this work, we investigate the relation between gender bias and the objects
that are mainly used in image captioning systems, and more precisely, the
widely used manually annotated image caption datasets: Flickr30K Young et al.
(2014) and COCO dataset Lin et al. (2014). However, we focus mainly on the
COCO dataset as the most used dataset in gender bias evaluation. COCO Captions
is an unbalanced gender bias dataset with a 1:3 ratio bias towards men Zhao et
al. (2017); Hendricks et al. (2018); Tang et al. (2021). The dataset contains
around 120K images, and each image is annotated with five different human-
written captions.

To obtain the visual context $o$ from each image $I$, we use out-of-the-box
classifiers to extract the image context information $o(I)$. Specifically,
following Sabir et al. (2023), the objects extracted from all pre-trained
models are obtained by extracting the top-3 object class/category (excluding
person category) from each classifier after filtering out instances with (1)
the cosine distance between objects (voting between classifiers of top-$k$)
and (2) a low confidence score via a probability threshold $<0.2$. We next
describe each of these classifiers:

GloVe Gender Dist (Biased) | GloVe Gender Dist (Balanced)
---|---
\+ person | \+ man | \+ woman | \+ person | \+ man | \+ woman
face | walking | lipstick | can | police | police
hand | cowboy | dishwasher | police | white | black
ballplayer | gorilla | sleeping | go | walking | white
mailbox | teddy | horse | walking | black | car
book | ballplayer | bathtub | black | car | walking

Table 1: Most frequency count of object + gender in the COCO Captions training
dataset via Cosine Distance with GloVe and balanced Gender Neutral GN-GloVe.

ResNet-152 He et al. (2016): A residual deep network that is designed for
ImageNet classification tasks, which relies heavily on batch normalization.

CLIP Radford et al. (2021): (Contrastive Language-Image Pre-training) is a
pre-trained model with contrastive loss where the pair of image-text needs to
be distinguished from randomly selected sample pairs. CLIP uses Internet
resources without human annotation on 400M pairs.

Inception-ResNet FRCNN Huang et al. (2017): is an improved variant of Faster
R-CNN, with a trade-off of better accuracy and fast inference via high-level
features extracted from Inception-ResNet. It is a pre-trained model that is
trained on COCO categories, with 80 object categories.

##  3 Gender Object Distance

The main component of our Gender Score (next section) is the semantic relation
between the object and the gender. Therefore, in this section, we investigate
the semantic correlation between the object and the gender in the general
dataset (_e.g_. wiki). More specifically, we assume that all images contain a
gender man, woman or gender neutral person, and we aim to measure the Cosine
Distance between the object $o$, objects with context _i.e_. caption $y$ from
the image $I$ and its related gender $a^{\prime}\in\\{\text{man, woman,
{person}}\\}$. In this context, we refer to the Cosine Distance as the closest
distance (_i.e_. semantic relatedness score) between the gender-to-object via
similarity measure _e.g_. cosine similarity. We apply our analysis to the most
widely used human-annotated datasets in image captioning tasks: Flikr30K and
COCO Captions datasets.

We employ several commonly used pre-trained models: (1) word-level: GloVe
Pennington et al. (2014), fasttext Bojanowski et al. (2017) and word2vec
Mikolov et al. (2013) out-of-the-box as baselines, then we utilize Gender
Neutral GloVe Zhao et al. (2018), which balances the gender bias; (2)
sentence-level: Sentence-BERT Reimers and Gurevych (2019) tuned on Natural
Language Inference (NLI) Conneau et al. (2017), (2b) SimCSE Gao et al. (2021)
contrastive learning supervised by NLI, and an improved SimCSE version with
additional Information aggregation via masked language model objective InfoCSE
Wu et al. (2022). In particular, we measure the Cosine Distance from the
gender $a^{\prime}$ to the object $o$ or caption $y$ as shown in Figure 1 (a,
b) (_e.g_. how close the gender vector of $a^{\prime}$ is to the object $o$
umbrella). Table 1 shows the top most object-gender frequent count with the
standard and gender-balanced GloVe, respectively.

| COCO Captions | Flickr30K
---|---|---
| Avg: Gender Object Distance | Ratio | Avg: Gender Object Distance | Ratio
Model | \+ person | \+ man | \+ woman | to-m | to-w | \+ person | \+ man | \+ woman | to-m | to-w
Word2Vec Mikolov et al. (2013) | 0.101 | 0.116 | 0.124 | 0.48 | 0.51 | 0.116 | 0.142 | 0.154 | 0.47 | 0.52
GloVe Pennington et al. (2014) | 0.146 | 0.175 | 0.169 | 0.50 | 0.49 | 0.131 | 0.170 | 0.168 | 0.50 | 0.49
Fasttext Bojanowski et al. (2017) | 0.180 | 0.200 | 0.191 | 0.51 | 0.48 | 0.146 | 0.196 | 0.191 | 0.50 | 0.49
GN-GloVe Zhao et al. (2018) | 0.032 | 0.055 | 0.054 | 0.50 | 0.49 | 0.024 | 0.085 | 0.088 | 0.49 | 0.50
SBERT Reimers and Gurevych (2019) | 0.124 | 0.155 | 0.128 | 0.54 | 0.45 | 0.121 | 0.167 | 0.129 | 0.56 | 0.43
SimCSE-RoBERTa Gao et al. (2021) | 0.194 | 0.137 | 0.093 | 0.59 | 0.40 | 0.189 | 0.140 | 0.107 | 0.56 | 0.43
InfoCSE-RoBERTa Wu et al. (2022) | 0.199 | 0.222 | 0.211 | 0.51 | 0.48 | 0.228 | 0.265 | 0.241 | 0.52 | 0.47

Table 2: Result of Average Cosine Distance between gender and object in the COCO Captions and Flikr30K training datasets. The ratio is the gender bias rate towards men/women. The results show there is a slight bias toward men. GloVe and GN-GloVe (balanced) show identical results on COCO Captions, which indicate that not all objects have a strong bias toward a specific gender. In particular, regarding non-biased objects, both models exhibit a low/similar bias ratio _e.g_. bicycle-gender (GloVe: m=0.31 | w=0.27, ratio=0.53) and (GN-GloVe: m=0.15 | w=0.13, ratio=0.53).

| Avg: Gender Score | Bias Ratio | Leakage
---|---|---|---
Model | \+ person | \+ man | \+ woman | m | w | to-m | m | w
Human |  |  |  |  |  |  | 930 | 291
Gender Object Distance
Transformer Vaswani et al. (2017) | 0.075 | 0.062 | 0.033 | 0.82 | 0.44 | 0.65 | 0.85 | 1.40
AoANet Huang et al. (2019) | 0.079 | 0.061 | 0.047 | 0.77 | 0.59 | 0.56 | 0.82 | 1.29
Vilbert Lu et al. (2020) | 0.089 | 0.070 | 0.056 | 0.78 | 0.62 | 0.55 | 0.75 | 1.06
OSCAR Li et al. (2020) | 0.076 | 0.054 | 0.044 | 0.71 | 0.57 | 0.58 | 0.93 | 1.40
BLIP Li et al. (2022) | 0.068 | 0.050 | 0.040 | 0.73 | 0.58 | 0.55 | 0.83 | 1.32
TraCLIPS-Reward Cho et al. (2022) | 0.069 | 0.053 | 0.040 | 0.76 | 0.57 | 0.56 | 0.82 | 1.30
BLIP-2 Li et al. (2023) | 0.073 | 0.049 | 0.043 | 0.67 | 0.58 | 0.53 | 0.73 | 1.22
Gender Score
Transformer Vaswani et al. (2017) | 0.201 | 0.194 | 0.187 | 0.96 | 0.93 | 0.50 | 0.85 | 1.40
AoANet Huang et al. (2019) | 0.217 | 0.210 | 0.201 | 0.96 | 0.92 | 0.51 | 0.82 | 1.29
Vilbert Lu et al. (2020) | 0.225 | 0.217 | 0.207 | 0.96 | 0.92 | 0.51 | 0.75 | 1.06
OSCAR Li et al. (2020) | 0.196 | 0.188 | 0.181 | 0.95 | 0.92 | 0.50 | 0.93 | 1.40
BLIP Li et al. (2022) | 0.226 | 0.220 | 0.213 | 0.97 | 0.94 | 0.50 | 0.83 | 1.32
TraCLIPS-Reward Cho et al. (2022) | 0.198 | 0.194 | 0.185 | 0.97 | 0.93 | 0.51 | 0.82 | 1.30
BLIP-2 Li et al. (2023) | 0.224 | 0.215 | 0.210 | 0.95 | 0.93 | 0.50 | 0.73 | 1.22

Table 3: Result of the Average Cosine Distance and Gender Score, between
gender and object information, on the Karpathy test split. Our score balances
the bias better than direct Cosine Distance, primarily because not all objects
exhibit strong gender bias. The leakage is a comparison between human-
annotated caption and the classifier output that uses the gender-related
object to influence the final gender prediction, such as associating women
with food.

Gender Bias Ratio: we follow the work of Zhao et al. (2017) to calculate the
gender bias ratio towards men as:

$\text{bias}_{\text{to-$m$}}=\frac{\operatorname{s}(\text{obj,
$m$})}{\operatorname{s}(\text{obj, $m$})+\operatorname{s}(\text{obj, $w$})}$
(1)

where $m$ and $w$ refer to the gender in the image, and the $s$ is our
proposed gender-to-object semantic relation bias score. In our case, we also
use the score to compute the ratio to gender neutral person:

$\text{Ratio}_{\text{to-n}}=\frac{\operatorname{s}(\text{obj,
$m$/$w$})}{\operatorname{s}(\text{obj, \text{{person}}})}$ (2)

##  4 Gender Score

In this section, we describe the proposed Gender Score that estimates gender
based on its semantic relation with the visual information extracted from the
image. Sabir et al. (2022) proposed a caption re-ranking method that leverages
visual semantic context. This approach utilized Belief Revision Blok et al.
(2003) to convert the similarity _i.e_. Cosine Distance (Section 3) into a
probability measure.

Belief Revision. To obtain likelihood bias revisions based on similarity
scores (_i.e_. gender, object), we need three parameters: (1) Hypothesis (g):
caption $y$ with the associated gender $a\in\\{\text{man, woman}\\}$, (2)
Informativeness (c): image object information $o$ confidence and (3)
Similarities: the degree of relatedness between object and gender $sim(y,o)$.

$\text{GS}_{a}(y)=\frac{1}{|\mathcal{D}|}\sum_{(y,o)\in\mathcal{D}}\operatorname{P}\left(g_{y}\mid
c_{o}\right)=\operatorname{P}(g_{y})^{\alpha}$ (3)

where $\operatorname{P}(g_{y})$ is the hypothesis probability of $y$,
$\mathcal{D}$ is the predicted captions with the gender $a$, and
$\operatorname{P}(c_{o})$ is the probability of the evidence that causes
hypothesis probability revision _i.e_. visual bias revision via object context
$o$ from the image $I$, $o(I)$:

Hypothesis: $\operatorname{P}(g_{y})\quad({\text{caption}}\,y\,{\text{with the
gender}}\,a)$

Informativeness: $1-\operatorname{P}(c_{o})\quad({\text{object context}}\,o$)

Similarities:
$\alpha=\left[\frac{1-\operatorname{sim}(y,o)}{1+\operatorname{sim}(y,o)}\right]^{1-\operatorname{P}(c_{o})}({\text{{visual
bias}}})$

The visual context $\operatorname{P}(c_{o})$ will revise the caption with the
associated gender $\operatorname{P}(g_{y})$ (_i.e_. gender bias) if there is a
semantic relation between them $sim(y,o)$. We discuss each component next:

| Bias Ratio Toward Men | Bias Ratio Toward Women
---|---|---
Model | skateboard | kitchen | motorcycle | baseball | skateboard | kitchen | motorcycle | baseball
Object Gender Co-Occ Zhao et al. (2017)
Transformer Vaswani et al. (2017) | 0.96  | 0.50  | 0.83  | 0.75  | 0.05  | 0.50  | 0.16  | 0.25
AoANet Huang et al. (2019) | 0.97  | 0.51  | 0.85  | 0.81  | 0.02  | 0.48  | 0.14  | 0.18
Vilbert Lu et al. (2020) | 0.96  | 0.47  | 0.84  | 0.66  | 0.03  | 0.52  | 0.15  | 0.33
OSCAR Li et al. (2020) | 0.97  | 0.58  | 0.82  | 0.90  | 0.02  | 0.41  | 0.18  | 0.09
BLIP Li et al. (2022) | 0.96  | 0.52  | 0.88  | 0.97  | 0.03  | 0.47  | 0.11  | 0.02
TraCLIPS-Reward Cho et al. (2022) | 0.89  | 0.48  | 0.93  | 0.50  | 0.10  | 0.51  | 0.06  | 0.50
BLIP-2 Li et al. (2023) | 0.94  | 0.57  | 0.88  | 0.90  | 0.05  | 0.42  | 0.11  | 0.10
Gender Score
Transformer Vaswani et al. (2017) | 0.96  | 0.51  | 0.83  | 0.61  | 0.03  | 0.48  | 0.16  | 0.38
AoANet Huang et al. (2019) | 0.97  | 0.46  | 0.84  | 0.82  | 0.02  | 0.53  | 0.15  | 0.17
Vilbert Lu et al. (2020) | 0.96  | 0.53  | 0.84  | 0.65  | 0.03  | 0.46  | 0.15  | 0.34
OSCAR Li et al. (2020) | 0.98  | 0.42  | 0.78  | 0.83  | 0.01  | 0.57  | 0.21  | 0.16
BLIP Li et al. (2022) | 0.96  | 0.50  | 0.86  | 0.98  | 0.03  | 0.49  | 0.13  | 0.01
TraCLIPS-Reward Cho et al. (2022) | 0.88  | 0.43  | 0.92  | 0.50  | 0.11  | 0.56  | 0.07  | 0.49
BLIP-2 Li et al. (2023) | 0.93  | 0.56  | 0.82  | 0.89  | 0.06  | 0.43  | 0.17  | 0.10

Table 4: Example of the most common gender bias objects in COCO Captions,
Karpathy test split. The result shows that our score (bias ratio) aligns
closely with the existing Object Gender Co-Occ approach when applied to the
most gender-biased objects toward men. Note that TraCLIPS-Reward (CLIPS+CIDEr)
inherits biases from RL-CLIPS, resulting in distinct gender predictions and
generates caption w/o a specific gender _i.e_. person, baseball player, etc.

Hypothesis initial bias: In visual-based belief revision, one of the
conditions is to start with an initial hypothesis and then revise it using
visual context and a similarity score. Therefore, we initialize the caption
hypothesis $\operatorname{P}(g_{y})$ with a common observation
$\operatorname{P}(g^{\prime}_{y})$, such as a Language Model (LM) (we consider
this as an initial bias without visual). We employ (GPT-2) Radford et al.
(2019) with mean token probability since it achieves better results.

Informativeness of bias information: As the visual context probability
$\operatorname{P}(c_{o})$ approaches 1 and in consequence is less informative
(very frequent objects have no discriminative power since they may co-occur
with any gender) $1-\operatorname{P}(c_{o})$ approaches zero, causing $\alpha$
to get closer to 1, and thus, a smaller revision of
$\operatorname{P}(g^{\prime}_{y})$. Therefore, as we described in the Visual
Context Information Section 2, we leverage a threshold and semantic filter
visual context dataset from ResNet, Inception-ResNet v2 based Faster R-CNN
object detector, and CLIP to extract top-$k$ textual visual context
information from the image. The extracted object is used to measure the
gender-to-object bias direct relation.

Relatedness between hypothesis and bias information: Likelihood revision
occurs if there is a close correlation between the hypothesis and the new
information. As the $sim(y,o)$ (gender, object), gets closer to 1 (higher
relatedness) $\alpha$ gets closer to 0, and thus hypothesis probability is
revised (_i.e_. gender bias) and raised closer to 1. Therefore, the initial
hypothesis will be revised or backed off to 1 (no bias) based on the
relatedness score. In our case, we employ Sentence-BERT to compute the Cosine
Distance (Section 3) by using object $o$ as context for the caption $y$ with
associated gender $a$.

##  5 Experiments

Caption model. We examine seven of the most recent Transformer state-of-the-
art caption models: Transformer Vaswani et al. (2017) with bottom-up top-down
features Anderson et al. (2018), AoANet Huang et al. (2019), Vilbert Lu et al.
(2020), OSCAR Li et al. (2020), BLIP Li et al. (2022), Transformer with
Reinforcement Learning (RL) CLIPS+CIDEr as image+text similarity Reward
(TraCLIPS-Reward) Cho et al. (2022) and Large LM${}_{\text{2.7B}}$ based
BLIP-2 Li et al. (2023). Note that for a fair comparison with other pre-
trained models, OSCAR uses a cross-entropy evaluation score rather than the
RL-based CIDEr optimization score.

Data. Our gender bias score is performed on the (82783 $\times$ 5 human
annotations) COCO Captions dataset. For baselines (testing), the score is used
to evaluate gender-to-object bias on the standard 5K Karpathy test split
images Karpathy and Fei-Fei (2015) (GT is the average of five human bias
ratios).

Our experiments apply visual semantics between the object or object with
context _i.e_. caption and its related gender information to predict object-
to-gender related bias. The proposed visual-gender scores are (1): Cosine
Distance, which uses the similarity score to try to estimate the proper gender
as shown in Table 2; (2) Gender Score, which carries out Belief Revision
(visual bias likelihood revision) to revise the hypothesis initial bias via
similarity _e.g_. Cosine Distance (gender, object). For our baseline, we adopt
the Object Gender Co-Occ metric Zhao et al. (2017) for the image captioning
task.

In this work, we hypothesize that every image has a gender $\in\\{\text{man,
woman}\\}$ or gender neutral (_i.e_. person), and our rationale is that each
model suffers from Right for the Wrong Reasons Hendricks et al. (2018) or
leakage Wang et al. (2019) (see Table 3), such as associating all kitchens
with women. Therefore, we want to explore all the cases and let the proposed
distance/score decide which gender (_i.e_. bias) is in the image based on a
visual bias. In particular, inspired by the cloze probability last word
completion task Gonzalez-Marquez (2007), we generate two identical sentences
but with a different gender, and then we compute the likelihood revisions
between the sentence-gender and the caption using the object probability.
Table 2 shows that GloVe and GN-GloVe (balanced) have identical results on
COCO Captions dataset, which indicate that not all objects have a strong bias
toward a specific gender. In addition, Table 3 shows that our score balances
the bias of the Cosine Distance and demonstrates, on average, that not all
objects have a strong gender bias. Also, our approach detects strong specific
object-to-gender bias and has a similar result to the existing Object Gender
Co-Occ method on the most biased object toward men, as shown in Table 4.
TraCLIPS-Reward inherits biases from RL-CLIPS and thus generates caption w/o a
specific gender (_e.g_. person, guy, etc). Therefore, we adopt the combined
CLIPS+CIDEr Rewards, which suffer less gender prediction error.

|  Visual: tennis ball
---|---
Caption: a < MASK > hitting |
a tennis ball on a tennis court |
Gender Object Distance: |
man 0.44  woman 0.46 |
Gender Score: man 0.45  woman 0.45  |
|  Visual: umbrella
Caption: a < MASK > holding |
an umbrella in the rain |
Gender Object Distance: |
man 0.20  woman 0.20  |
Gender Score: man 0.12  woman 0.13 |
|  Visual: paddle
Caption: a < MASK > riding a wave |
on top of a surfboard |
Gender Object Distance: |
man 0.16 woman 0.11 |
Gender Score: man 0.33 woman 0.30  |

Figure 2: Examples of Gender Score Estimation and Cosine Distance. The result
shows that (Top) the score balances the bias (as men and women have a similar
bias for the sport tennis), (Middle) a slight bias object umbrella toward
women, and (Bottom) men strong object bias relation (paddle, surfboard), the
model adjusts the women bias while preserving the object gender score.

| Gender | Bias Ratio
---|---|---
Model | man | woman | to-m | to-w
Object Gender Co-Occ Zhao et al. (2017)
Transformer | 792 | 408 | 0.66 | 0.34
AoANet | 770 | 368 | 0.67 | 0.32
Vilbert | 702 | 311 | 0.69 | 0.30
OSCAR | 845 | 409 | 0.67 | 0.32
BLIP | 775 | 385 | 0.66 | 0.33
TraCLIPS-Reward | 769 | 381 | 0.66 | 0.33
BLIP-2 | 695 | 356 | 0.66 | 0.33
Gender Score (Gender Score Estimation)
Transformer | 616 | 217 | 0.73 | 0.26
AoANet | 527 | 213 | 0.71 | 0.28
Vilbert | 526 | 161 | 0.76 | 0.23
OSCAR | 630 | 237 | 0.72 | 0.27
BLIP | 554 | 240 | 0.69 | 0.30
TraCLIPS-Reward | 537 | 251 | 0.68 | 0.31
BLIP-2 | 498 | 239 | 0.67 | 0.32

Table 5: Comparison results between Object Gender Co-Occ (predicted gender-
object results) and our gender score estimation on Karpathy test split. The
proposed score measures gender bias more accurately, particularly when there
is a strong object to gender bias relation.

Gender Score Estimation. In this experiment, we < MASK > the gender and use
the object with context to predict the gender as shown in Figure 2. The idea
is to measure the amplified gender-to-object bias in pre-trained models. Table
5 shows that the fill-in gender score has a more bias towards men results than
object-gender pair counting. The rationale is that the model can estimate the
strong gender object bias as shown in Figure 2, including the false positive
and leakage cases by the classifier.

Discussion. Our approach measures the amplified gender bias more accurately
than the Object Gender Co-Occ Zhao et al. (2017) in the following two
scenarios: (1) where the gender is not obvious in the image and is
misclassified by the caption baseline, and (2) when there is leakage by the
classifier. In addition, unlike the Object Gender Co-Occ, our model balances
the gender to object bias and only measures a strong object to gender bias
relation as shown in Figure 2. For instance, the word “hitting” in a generated
caption (as a stand-alone without context) is associated with a bias toward
men more than women, and it will influence the final gender-to-caption bias
score. However, our gender score balances the unwanted bias and only measures
the pronounced gender to object bias relation.

##  6 Conclusion

In this work, we investigate the relation between objects and gender bias in
image captioning systems. Our results show that not all objects exhibit a
strong gender bias and only in special cases does an object have a strong
gender bias. We also propose a Gender Score that can be used as an additional
metric to the existing Object-Gender Co-Occ method.

## Limitations

The accuracy of our Gender Score heavily relies on the semantic relation
(_i.e_. Cosine Distance) between gender and object, a high or low degree of
similarity score between them can influence the final bias score negatively.
Also, our model relies on a large pre-trained model(s), which inherently
encapsulate their own latent biases that might impact the visual bias revision
behavior in gender-to-object bias scenarios. Specifically, in cases when there
are multiple strong bias contexts (_i.e_. non-objects context) with high
similarity scores toward a particular gender present within the caption. This
can imbalance the final gender-to-object score, leading to errors in gender
prediction and bias estimation. In addition, the false positive object context
information extracted by the visual classifier will result in inaccurate bias
estimation.

## Ethical Considerations

We rely upon an existing range of well-known publicly available caption
datasets crawled from the web and annotated by humans that assume a binary
conceptualization of gender. Therefore, it is important to acknowledge that
within the scope of this work, we are treating gender as strictly binary
(_i.e_. man and woman) oversimplifies a complex and multifaceted aspect of
human identity. Gender is better understood as a spectrum, with many
variations beyond just two categories, and should be addressed in future work.
Since all models are being trained on these datasets, we anticipate all models
contain other biases (racial, cultural, _etc_.). For example, an observation
in Table 1, when we remove gender bias, we notice the emergence of another
bias, such as racial bias. For example, vectors representing Black person or
women are closer together than those representing other colors, like white.
Moreover, there is another form of bias that has received limited attention in
the literature, the propagation of gender and racial bias via RL (_e.g_. RL-
CLIPS). For instance, in Figure 1 the model associates gender with race, as
“asian woman”.

## References

  * Anderson et al. (2018) Peter Anderson, Xiaodong He, Chris Buehler, Damien Teney, Mark Johnson, Stephen Gould, and Lei Zhang. 2018.  Bottom-up and top-down attention for image captioning and visual question answering.  In _CVPR_.
  * Blok et al. (2003) Sergey Blok, Douglas Medin, and Daniel Osherson. 2003.  Probability from similarity.  In _AAAI_.
  * Bojanowski et al. (2017) Piotr Bojanowski, Edouard Grave, Armand Joulin, and Tomas Mikolov. 2017.  Enriching word vectors with subword information.  _TACL_.
  * Cho et al. (2022) Jaemin Cho, Seunghyun Yoon, Ajinkya Kale, Franck Dernoncourt, Trung Bui, and Mohit Bansal. 2022.  Fine-grained image captioning with clip reward.  _arXiv preprint arXiv:2205.13115_.
  * Conneau et al. (2017) Alexis Conneau, Douwe Kiela, Holger Schwenk, Loic Barrault, and Antoine Bordes. 2017.  Supervised learning of universal sentence representations from natural language inference data.  _arXiv preprint arXiv:1705.02364_.
  * Devlin et al. (2019) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019.  Bert: Pre-training of deep bidirectional transformers for language understanding.  In _NAACL-HLT_.
  * Gao et al. (2021) Tianyu Gao, Xingcheng Yao, and Danqi Chen. 2021.  SimCSE: Simg of sentence embeddings.  _arXiv preprint arXiv:2104.08821_.
  * Gonzalez-Marquez (2007) Monica Gonzalez-Marquez. 2007.  _Methods in cognitive linguistics_ , volume 18.  John Benjamins Publishing.
  * He et al. (2016) Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016.  Deep residual learning for image recognition.  In _CVPR_.
  * Hendricks et al. (2018) Lisa Anne Hendricks, Kaylee Burns, Kate Saenko, Trevor Darrell, and Anna Rohrbach. 2018.  Women also snowboard: Overcoming bias in captioning models.  In _ECCV_.
  * Hirota et al. (2022) Yusuke Hirota, Yuta Nakashima, and Noa Garcia. 2022.  Quantifying societal bias amplification in image captioning.  In _CVPR_.
  * Huang et al. (2017) Jonathan Huang, Vivek Rathod, Chen Sun, Menglong Zhu, Anoop Korattikara, Alireza Fathi, Ian Fischer, Zbigniew Wojna, Yang Song, Sergio Guadarrama, et al. 2017.  Speed/accuracy trade-offs for modern convolutional object detectors.  In _CVPR_.
  * Huang et al. (2019) Lun Huang, Wenmin Wang, Jie Chen, and Xiao-Yong Wei. 2019.  Attention on attention for image captioning.  In _CVPR_.
  * Karpathy and Fei-Fei (2015) Andrej Karpathy and Li Fei-Fei. 2015.  Deep visual-semantic alignments for generating image descriptions.  In _CVPR_.
  * Li et al. (2023) Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. 2023.  Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models.  _arXiv preprint arXiv:2301.12597_.
  * Li et al. (2022) Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi. 2022.  Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation.  _arXiv preprint arXiv:2201.12086_.
  * Li et al. (2020) Xiujun Li, Xi Yin, Chunyuan Li, Pengchuan Zhang, Xiaowei Hu, Lei Zhang, Lijuan Wang, Houdong Hu, Li Dong, Furu Wei, et al. 2020.  Oscar: Object-semantics aligned pre-training for vision-language tasks.  In _ECCV_.
  * Lin et al. (2014) Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. 2014.  Microsoft coco: Common objects in context.  In _ECCV_.
  * Lu et al. (2020) Jiasen Lu, Vedanuj Goswami, Marcus Rohrbach, Devi Parikh, and Stefan Lee. 2020.  12-in-1: Multi-task vision and language representation learning.  In _CVPR_.
  * Mikolov et al. (2013) Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013.  Efficient estimation of word representations in vector space.  _arXiv preprint arXiv:1301.3781_.
  * Pennington et al. (2014) Jeffrey Pennington, Richard Socher, and Christopher D Manning. 2014.  Glove: Global vectors for word representation.  In _EMNLP_.
  * Radford et al. (2021) Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. 2021.  Learning transferable visual models from natural language supervision.  _arXiv preprint arXiv:2103.00020_.
  * Radford et al. (2019) Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019.  Language models are unsupervised multitask learners.  _OpenAI blog_.
  * Reimers and Gurevych (2019) Nils Reimers and Iryna Gurevych. 2019.  Sentence-bert: Sentence embeddings using siamese bert-networks.  _arXiv preprint arXiv:1908.10084_.
  * Sabir et al. (2022) Ahmed Sabir, Francesc Moreno-Noguer, Pranava Madhyastha, and Lluís Padró. 2022.  Belief revision based caption re-ranker with visual semantic information.  _arXiv preprint arXiv:2209.08163_.
  * Sabir et al. (2023) Ahmed Sabir, Francesc Moreno-Noguer, and Lluís Padró. 2023.  Visual semantic relatedness dataset for image captioning.  In _CVPRW_.
  * Tang et al. (2021) Ruixiang Tang, Mengnan Du, Yuening Li, Zirui Liu, Na Zou, and Xia Hu. 2021.  Mitigating gender bias in captioning systems.  In _Proceedings of the Web Conference 2021_.
  * Vaswani et al. (2017) Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017.  Attention is all you need.  In _NeurIPS_.
  * Wang et al. (2019) Tianlu Wang, Jieyu Zhao, Mark Yatskar, Kai-Wei Chang, and Vicente Ordonez. 2019.  Balanced datasets are not enough: Estimating and mitigating gender bias in deep image representations.  In _CVPR_.
  * Wu et al. (2022) Xing Wu, Chaochen Gao, Zijia Lin, Jizhong Han, Zhongyuan Wang, and Songlin Hu. 2022.  Infocse: Information-aggregated contrastive learning of sentence embeddings.  _arXiv preprint arXiv:2210.06432_.
  * Young et al. (2014) Peter Young, Alice Lai, Micah Hodosh, and Julia Hockenmaier. 2014.  From image descriptions to visual denotations: New similarity metrics for semantic inference over event descriptions.  _TACL_.
  * Zhang et al. (2021) Pengchuan Zhang, Xiujun Li, Xiaowei Hu, Jianwei Yang, Lei Zhang, Lijuan Wang, Yejin Choi, and Jianfeng Gao. 2021.  Vinvl: Making visual representations matter in vision-language models.  In _CVPR_.
  * Zhao et al. (2021) Dora Zhao, Angelina Wang, and Olga Russakovsky. 2021.  Understanding and evaluating racial biases in image captioning.  In _CVPR_.
  * Zhao et al. (2017) Jieyu Zhao, Tianlu Wang, Mark Yatskar, Vicente Ordonez, and Kai-Wei Chang. 2017.  Men also like shopping: Reducing gender bias amplification using corpus-level constraints.  _arXiv preprint arXiv:1707.09457_.
  * Zhao et al. (2018) Jieyu Zhao, Yichao Zhou, Zeyu Li, Wei Wang, and Kai-Wei Chang. 2018.  Learning gender-neutral word embeddings.  _arXiv preprint arXiv:1809.01496_.

##  7 Appendix

## Additional Information

If the goal is to evaluate gender bias why using pre-trained models. We
believe this is representative of real-world scenarios where we can measure
the gender bias from real data and not from a closed domain as in the human-
annotated COCO Captions dataset. Moreover, in contrast to recent previous work
Hirota et al. (2022), which heavily depends on a costly human annotation
dataset Zhao et al. (2021), limiting the scalability across various domains,
our Gender Score is not bounded by any dataset and can be adapted for any
task, including the additionally annotated existing caption dataset for the
gender bias analysis.

Table 6 shows a comparison result between our Gender Score and
BERT${}_{\text{L}}$ /GPT-2${}_{\text{L}}$ as baselines using only the word
surrounding context without any visual bias. The result shows that our gender
estimation model is similar to BERT in some cases as shown in Figures 3|4,
which are only influenced by a strong bias. However, when the generated
caption is more/less diverse, _i.e_. due to less/more context to predict the
gender, BERT Mask LM struggles with predicting the correct biased gender. In
contrast to BERT, our approach relies on additional context extracted from the
image and uses a language model and similarity score to measure the degree of
gender-based bias prediction.

| Gender | Bias Ratio
---|---|---
Model | man | woman | to-m | to-w
Baseline - GPT-2 Score (initial bias) |
Transformer | 654 | 161 | 0.80 | 0.19
AoANet | 686 | 175 | 0.79 | 0.20
Vilbert | 450 | 148 | 0.75 | 0.24
OSCAR | 645 | 215 | 0.75 | 0.25
BLIP | 545 | 220 | 0.71 | 0.28
TraCLIPS-Reward | 606 | 205 | 0.74 | 0.25
BLIP-2 | 522 | 218 | 0.70 | 0.29
Baseline - BERT MLM (no visual) |
Transformer | 524 | 187 | 0.73 | 0.26
AoANet | 531 | 172 | 0.75 | 0.24
Vilbert | 517 | 158 | 0.76 | 0.23
OSCAR | 576 | 205 | 0.73 | 0.26
BLIP | 541 | 195 | 0.73 | 0.26
TraCLIPS-Reward | 559 | 180 | 0.75 | 0.24
BLIP-2 | 536 | 181 | 0.74 | 0.25
Object Gender Co-Occ Zhao et al. (2017) |
Transformer | 792 | 408 | 0.66 | 0.34
AoANet | 770 | 368 | 0.67 | 0.32
Vilbert | 702 | 311 | 0.69 | 0.30
OSCAR | 845 | 409 | 0.67 | 0.32
BLIP | 775 | 385 | 0.66 | 0.33
TraCLIPS-Reward | 769 | 381 | 0.66 | 0.33
BLIP-2 | 695 | 356 | 0.66 | 0.33
Gender Score (Gender estimation) |
Transformer | 616 | 217 | 0.73 | 0.26
AoANet | 527 | 213 | 0.71 | 0.28
Vilbert | 526 | 161 | 0.76 | 0.23
OSCAR | 630 | 237 | 0.72 | 0.27
BLIP | 554 | 240 | 0.69 | 0.30
TraCLIPS-Reward | 537 | 251 | 0.68 | 0.31
BLIP-2 | 498 | 239 | 0.67 | 0.32

Table 6: Full amplified gender-to-object results. Comparison result between 1)
initial bias via language model GPT-2; 2) Baseline BERT Mask Language Modeling
(MLM) with MASK prediction without any visual context; 3) Object Gender Co-
Occ; 4) our Gender Score estimation with MASK prediction on the karpathy test
split. Our result (bias ratio) is similar to BERT in some cases, as shown in
Figure 3 with man instance with the pizza bias. However, when the generated
caption is more/less diverse, the MLM struggles with predicting the bias and
correct gender. Unlike BERT, our score relies on additional parameters such as
the similarity score to measure an accurate gender bias prediction.

## Baseline

Transformer Vaswani et al. (2017). We use a transformer model that is trained
on bottom-up and top-down features Anderson et al. (2018) with cross-entropy
loss with learning rate $3e-04$.

AoANet Huang et al. (2019). We utilize the model as out-of-the-box with the
best-reported result by the authors.

Vilbert${}_{\text{Large}}$ Lu et al. (2020) (12-in-1: Multi-Task Vilbert)
Since Vilbert is trained on 12 datasets, we also use the provided pre-trained
model out-of-the-box extracted captions Sabir et al. (2022).

OSCAR${}_{\text{Large}}$ Li et al. (2020). To make a fair comparison with
other pre-trained vision and language models, we fine-tuned the model on the
COCO-Caption dataset with cross-entropy evaluation score without RL with a
learning rate $5e-06$ (15 epochs), following the same paper setup.

BLIP${}_{\text{Large}}$ Li et al. (2022). We use the model with a beam search
b=3 (best-reported result by the authors with their provided checking points.

TransCLIPS-Reward Cho et al. (2022). Transformer CLIPS-Reward best-proposed
model by the authors CLIP-S+Grammar reward inherits biases from RL and thus
generates caption w/o a specific gender (_e.g_. person, guy, etc.). Therefore,
we adopt the combined CLIPS+CIDEr Rewards, which suffer less gender prediction
error.

BLIP-2 VIT-G OPT${}_{\text{2.7B}}$Li et al. (2023). We use the best image
captioning model reported by the authors in the benchmark (beam search $b$=5)
with Open Pre-trained Transformer (OPT) Large Language Models (LLMs) (with
2.7B parameters model), and scaled vision transformer 2B parameters model
(with VIT-Giant 1843M parameters).

## Case Study

Gender Bias in Twitter. In this section, we apply our proposed Gender Score to
a subset of the Twitter user gender classification
dataset111https://www.kaggle.com/crowdflower/twitter-user-gender-
classification, as shown in Table 9. We use a BERT-based keyword
extractor222https://github.com/MaartenGr/KeyBERT to extract the biased context
from the sentence (_e.g_. travel-man, woman-family), and we then employ the
cloze probability333From cognitive linguistics, cloze probability is the
probability of the target word completing that particular sentence. Gonzalez-
Marquez (2007) to extract the probability of the context. We observe that some
keywords have a strong bias: women are associated with keywords such as novel,
beauty, and hometown. Meanwhile, men are more frequently related to words such
as gaming, coffee, and inspiration. Table 7 shows a comparison result between
the frequency count method via ground truth and our score.

Model | to-m | to-w
---|---|---
Object Gender Co-Occ Zhao et al. (2017) |
Twitter Profile Data | 0.55 | 0.44
Gender Score |
Twitter Profile Data | 0.50 | 0.49

Table 7: Gender Ratio results on gender classification task via Gender Score
and estimated ground truth.

|  Visual: laptop
---|---
Caption: a < MASK > sitting on |
a couch with two laptops |
BERT [MASK]: woman |
Gender Object Distance: |
man 0.43 woman 0.42  |
Gender Score: man 0.25  woman 0.25  |
|  Visual: toilet seat
Caption: a < MASK > reading |
a paper in a bathroom |
BERT [MASK]: man Gender Object Distance: |
man 0.42 woman 0.40  |
Gender Score: man 0.15  woman 0.16 |
|  Visual: zebra ✘
Caption: a < MASK > feeding |
a giraffe through a fence |
BERT [MASK]: dog |
Gender Object Distance: |
man 0.23  woman 0.26 |
Gender Score: man 0.30  woman 0.32 |
|  Visual: walking stick
Caption: a < MASK > riding skis |
down a snow covered slop |
BERT [MASK]: man |
Cosine Distance: |
man 0  woman 0 |
Gender Score: man 0.11 woman 0.09  |
|  Visual: pizza
Caption: a < MASK > eating |
a slice of pizza |
BERT [MASK]: man |
Gender Object Distance: |
man 0.62 woman 0.57  |
Gender Score: man 0.39 woman 0.36  |
|  Visual: umbrella
Caption: a < MASK > holding |
a hot dog and a bottle ✘ BERT [MASK]: woman |
Gender Object Distance: |
man 0.15  woman 0.20 |
Gender Score: man 0.23  woman 0.24 |
|  Visual: motor scooter
Caption: a < MASK > riding a |
motorcycle on a road |
BERT [MASK]: man |
Gender Object Distance: |
man 0.54 woman 0.50 |
Gender Score: man 0.43 woman 0.40  |
|  Visual: tennis ball
Caption: a < MASK > sitting on |
a bench holding a tennis racket |
BERT [MASK]: man |
Gender Object Distance: |
man 0.45  woman 0.51 |
Gender Score: man 0.47  woman 0.50 |

Figure 3: Examples of Gender Score Estimation and Cosine Distance. In the
Laptop example, our score balances the amplified bias. However, our score is
less successful in balancing the bias in tennis instance, as there are strong
biases of non-object contexts associated with women (sitting and holding). Our
baseline is BERT as Mask Language Modeling to predict the Mask using only the
word surrounding context without any visual information. For example, for the
same Laptop instance, the word couch has a strong bias toward women, which
influences the gender output and is similar to gender to motorcycle prediction
which relies on a men bias.

|  Visual: hotdog
---|---
Caption: a < MASK > eating a hot dog |
BERT [MASK]: dog |
Gender Object Distance: |
man 0.44 woman 0.42  |
Gender Score: man 0.42 woman 0.40  |
|  Visual: spaghetti squash ✘
Caption: a < MASK > holding |
a sandwich and french fries |
BERT [MASK]: waiter (man) |
Gender Object Distance: |
man 0.17  woman 0.24 |
Gender Score: man 0.24  woman 0.39 |
|  Visual: hair spray
Caption: a < MASK > holding |
a baby in a chair ✘ |
BERT [MASK]: woman |
Cosine Distance: |
man 0.24  woman 0.39 |
Gender Score: man 0.30  woman 0.32 |
|  Visual: wine bottle
Caption: a < MASK > sitting at |
a table preparing food ✘ |
BERT [MASK]: woman |
Gender Object Distance: |
man 0  woman 0 |
Gender Score: man 0.05  woman 0.06 |
|  Visual: umbrella
Caption: a < MASK > holding |
an umbrella in the rain |
BERT [MASK]: man |
Gender Object Distance: |
man 0.20  woman 0.20  |
Gender Score: man 0.12  woman 0.13 |
|  Visual: remote control
Caption: a < MASK > playing |
a video game in a living room BERT [MASK]: man |
Gender Object Distance: |
man 0.24 woman 0.14 |
Gender Score: man 0.38 woman 0.31  |
|  Visual: shopping basket
Caption: a < MASK > standing in |
front of a fruit stand |
BERT [MASK]: man |
Gender Object Distance: |
man 0.34  woman 0.38 |
Gender Score: man 0.46  woman 0.48 |
|  Visual: dining table
Caption: a < MASK > and a baby at |
a table with a cake |
BERT [MASK]: mother (woman) |
Gender Object Distance: |
man 0.26  woman 0.40 |
Gender Score: man 0.14  woman 0.16 |

Figure 4: Examples of Gender Score Estimation and Cosine Distance. Note that
in some cases when there is a no Cosine Distance $\leq$ 0 (Gender Object
Distance) between the gender and the caption, the model will rely on general
observation language model GPT-2 _i.e_. initial bias without the visual
context as the main gender bias, as shown in the wine bottle-woman example and
in Figure 3 the ski walking stick-woman instance.

Object | \+ man | \+ woman
---|---|---
hoopskirt | 0.17  |  0.19
bobsled | 0.16  |  0.17
bagel | 0.14  | 0.14
sliding door | 0.14  | 0.14
tobacco shop | 0.26  |  0.27
pizza |  0.35 | 0.32
cellular telephone |  0.55 | 0.54
refrigerator | 0.17  |  0.23
washbasin | 0.24  | 0.24
paddle |  0.26 | 0.24
ski |  0.54 | 0.45
television |  0.35 | 0.30
restaurant | 0.28  |  0.37
wine bottle | 0.43  |  0.47
soccer ball |  0.57 | 0.49
overskirt | 0.11  |  0.16
ballplayer |  0.56 | 0.41
parking meter |  0.25 | 0.23
umbrella | 0.28  |  0.30
walking stick |  0.13 | 0.09
bathing cap | 0.27  |  0.28
tape player |  0.12 | 0.09
whiskey jug |  0.08 | 0.07
chocolate sauce | 0.07  |  0.08
dishwasher | 0.13  | 0.13
toyshop |  0.17 | 0.14
bassinet | 0.11  |  0.12
necklace | 0.19  |  0.20
ipod | 0.10  |  0.12
dutch oven | 0.24  |  0.29
moving van |  0.32 | 0.30
shower curtain | 0.31  |  0.32
park bench |  0.21 | 0.20
tennis ball | 0.47  |  0.51
granny smith | 0.18  |  0.22
passenger car | 0.05  |  0.06
sweatshirt | 0.20  | 0.20
reflex camera | 0.05  |  0.12
banana | 0.30  |  0.32
book jacket | 0.28  |  0.31
ice cream | 0.14  |  0.15
folding chair |  0.25 | 0.22
cheeseburger |  0.22 | 0.19
vending machine | 0.29  |  0.33
tricycle | 0.28  |  0.31

Table 8: CLIP-based Gender Score on randomly selected objects from the test
set. The baseline is a Transformer Vaswani et al. (2017) that is trained with
Bottom-Up and Top-Down features Anderson et al. (2018). Figure 5: Probability
distribution of our Gender Score _i.e_. object-to-gender semantic bias score
on the karpathy test split. As shown in the Figure, the distributions of
object-to-gender $\in\\{\text{man, woman}\\}$ bias are similar in most
instances, which indicate that not all object has a strong bias toward a
specific gender.

Tweets | \+ man | \+ woman | GT confidence
---|---|---|---
BSc economics graduate | 0.00  | 0.00  | 0.99
Blood makes you related. Loyalty makes you family | 0.17  |  0.27 | 0.99
I’m the author of novels filled with family drama and romance. | 0.22  |  0.34 | 0.99
You are to me all that an angel could be. | 0.12  |  0.13 | 0.99
ask no questions and you’ll get no lies |  0.15 | 0.13  | 0.99
who cares what people think | 0.17  | 0.17  | 0.68
I know I’m mad because iv always been mad |  0.10 | 0.19  | 0.99
I’m floating!!!!! I read lots of them comic book things. |  0.11 | 0.10  | 0.99
weird in a mysterious way, i’m not a psycho | 0.12  |  0.13 | 0.99
there are two kinds of people in this world And I dont like them |  0.28 | 0.27  | 0.99
nothing more important to me than my family. | 0.10  |  0.16 | 0.99
do you believe in time travel? |  0.28 | 0.24  | 0.99
fan of the latest technology | 0.05  | 0.05  | 0.99
life is not a dress rehearsal. | 0.10  |  0.11 | 0.99
I love music and watching livestreams |  0.13 | 0.12  | 0.35
I don’t get how people can get so anti-something. | 0.32  |  0.33 | 0.99
I am me and that is who I want to be I wont change for anyone | 0.21  |  0.22 | 0.99
Left my hometown in search of myself. | 0.14  |  0.21 | 0.99
My life is inspired by actual events |  0.20 | 0.14  | 0.99
do whatever makes you happy | 0.25  | 0.25  | 0.99
I write a beauty blog also love me some metal music !!! | 0.15  |  0.17 | 0.99
sometimes I’m cynical. I have no filter. |  0.13 | 0.12  | 0.99
I am a web and iOS developer | 0.17  |  0.18 | 0.99
and if you like having secret little rendezvous | 0.13  |  0.18 | 0.99
You can make great money by buying things cheaply |  0.04 | 0.03  | 0.99
You may be a sinner but your innocence is mine. |  0.11 | 0.10  | 0.99
I live off white wine and avocados | 0.24  |  0.25 | 0.99
be without fear. | 0.04  | 0.19  | 0.99
I love anything related to gaming. |  0.14 | 0.13  | 0.99
There’s such a lot of world to see |  0.28 | 0.25  | 0.99
I’m probably funnier than you. |  0.34 | 0.31  | 0.99
I’m just here for sweet and detailed everythings | 0.04  | 0.04  | 0.99
You probably think this song is about you | 0.16  |  0.20 | 0.99
go after your dream | 0.01  | 0.01  | 0.99
Let’s grab coffee |  0.15 | 0.12  | 0.99
We can dream a dream for us that no one else can touch | 0.14  |  0.15 | 0.66
One day I’ll be able to stop crying over people who aren’t worth it | 0.21  | 0.21  | 0.67
Believe in something bigger than yourself and find your purpose. |  0.28 | 0.25  | 0.99
some say I’m a photographer who loves politics | 0.06  | 0.07  | 0.99
Student with an interest in politics | 0.19  |  0.20 | 0.99
the world isn’t as cruel as you take it to be |  0.42 | 0.37  | 0.99
It’s hard to want to stay awake when everyone you meet |  0.21 | 0.19  | 0.99
You’re such a hypocrite |  0.36 | 0.30  | 0.32
Half human half dolphin and professional tea maker | 0.00  | 0.00  | 0.99
you can now purchase my book word vomit in stores everywhere |  0.03 | 0.02  | 0.99
i love watching ghost adventures | 0.00  | 0.00  | 0.36
im not a bad person but i did bad things |  0.18 | 0.14  | 0.99
You have to understand that sometimes is hard to understand me. |  0.27 | 0.25  | 0.99
I mean Im in college or something like that | 0.20  |  0.21 | 0.99
if u cant be happy u can at least be drunk |  0.17 | 0.16  | 0.99
We are all addicted to something | 0.19  |  0.22 | 0.99
It’s a hard life in the mountains. |  0.11 | 0.10  | 0.99
my heart is yet to be thawed | 0.17  |  0.18 | 0.99
I can’t find my damn left sock | 0.10  |  0.19 | 0.10
Only speak if what you have to say is more beautiful than the silence. |  0.40 | 0.39  | 0.99
left/right it doesn’t matter where you go. important is u choose a path and didn’t |  0.19 | 0.18  | 0.99
I’m a Christian. I love Jesus and pray to always glorify Him. |  0.23 | 0.18  | 0.99
Don’t have a lot to say |  0.30 | 0.25  | 0.99

Table 9: Gender Score on user gender classification task (Twitter). We observe
that some keywords have a strong bias: women are associated with keywords such
as novel, beauty, and hometown. Meanwhile, men are more frequently related to
words such as gaming, coffee, and inspiration. Highlighted in red color, the
Gender Score disagrees with human estimation confidence due to gender-context
bias.