# Centered Masking for Language-Image Pre-Training

Mingliang Liang, Martha Larson
Radboud University
Nijmegen
{m.liang<EMAIL_ADDRESS>

###### Abstract

We introduce Gaussian masking for Language-Image Pre-Training (GLIP) a novel,
straightforward, and effective technique for masking image patches during pre-
training of a vision-language model. GLIP builds on Fast Language-Image Pre-
Training (FLIP), which randomly masks image patches while training a CLIP
model. GLIP replaces random masking with centered masking, which uses a
Gaussian distribution and is inspired by the importance of image patches at
the center of the image. GLIP retains the same computational savings as FLIP,
while improving performance across a range of downstream datasets and tasks,
as demonstrated by our experimental results. We show the benefits of GLIP to
be easy to obtain, requiring no delicate tuning of the Gaussian, and also
applicable to datasets containing images without an obvious center focus.

_K_ eywords Vision-Language Model $\cdot$ Multimodal Data $\cdot$ Gaussian
Distribution Masking.

##  1 Introduction

The rise of Vision-Language Models trained on a large number of image text
pairs has been led by Contrastive Language-Image Pretraining CLIP [1]. CLIP
delivers high-quality visual representations, with strong performance on
downstream tasks and impressive transferability. However, the downside of
CLIP, is the large amount of computational resources that it requires to
train. According to [1], pre-training CLIP on 400 million image-text pairs
over 32 epochs required thousands of GPU days for completion. The high
computational cost of training Vision-Language models limits further increases
in training data size.

Recent research has proposed to accelerate the training of CLIP, with
surprising success being achieved by Fast Language-Image Pre-Training (FLIP)
[2]. FLIP randomly masks image patches during CLIP training. According to [2],
if 50% or 75% of the patches in each training image are discarded by masking,
FLIP can reduce the computation carried out during training by 2–4 times.
Interestingly, masking 50% of an image does not compromise performance. In
fact, it makes it possible to increase the global batch size given a
processor’s fixed processing power. More samples per batch are known to be
beneficial for contrastive learning [3, 4, 2, 1].

In this paper, we further improve on the performance of FLIP, while retaining
the same savings in computation. Specifically, we conjecture that patches
closer to the center of an image are more important when training a Vision-
Language Model than patches nearer the periphery. Based on this conjecture, we
propose an approach that replaces random masking of FLIP with centered
masking. Our approach, called Gaussian masking for Language-Image Pre-Training
(GLIP), uses a Gaussian distribution centered at the middle of the image in
order to select patches during training. Figure 1 contrasts the random masking
of FLIP with the centered masking of GLIP. It can be seen that the main
subject of the image is better preserved in the retained patches when masking
prioritizes patches from the center and disregards more patches at the
periphery. The inspiration for the centered masking of GLIP comes from the
behavior of human photographers, who often place the main subject of the photo
in the center. However, as will be seen from our experimental results, GLIP
also delivers improvements for image data that does not have a strong
photographic center focus, demonstrating GLIP’s promise for generality. Note
that we use masking in the same way that FLIP does, i.e., not for the purpose
of reconstruction, which was shown by [2] to actually hurt performance.

|  |
---|---|---
|  |
|  |
Figure 1:  Images from ImageNet-1K [5] (left side) and CC12M [6] (right side).
A contrast can be seen between random masking (left mask) and centered masking
(right mask). Centered masking captures more of the main subject of the image.

GLIP can be considered to be related to other approaches that do not use
random masking. A key example is A-CLIP [7], which employs an online
Exponential Moving Average (EMA) to generate masks derived from the attention
weights associated with the [CLS] token of the visual encoder. The attentive
masks attempt to include more image patches that are highly correlated with
the semantics of the text description. GLIP aims to leverage the same effect,
but instead of calculating attention, assumes that the most important part of
the image is the center. In this paper, we do not compare GLIP with A-CLIP
because even the efficient version of A-CLIP requires more computational
resources than GLIP.

This paper makes the following contributions:

  * •

We introduce a centered masking approach for language-image pre-training
called Gaussian Masking for Language-Image Pre-Training (GLIP), which enjoys
the same reduction in computational resources as FLIP.

  * •

We provide extensive experiments that show the superiority of GLIP over FLIP.
GLIP shows general performance improvements and also allows higher masking
ratios.

  * •

We show that GLIP delivers an easy-to-obtain improvement across different
datasets without requiring effort to tune the variance of the Gaussian.

Our code is available online111https://github.com/Anastasiais-ml/GLIP. In our
experiments comparing FLIP and GLIP we train on a smaller dataset than was
used in the original FLIP paper [2]. For this reason, the results of our paper
can be easily reproduced using the resources typically available in either an
academic or industry setting.

##  2 Related work

###  2.1 Vision-Language Models

Refining visual representations of images and developing foundation models to
enhance subsequent tasks is a key objective in the field of computer vision.
CLIP [1], and closely related ALIGN [8], learn visual representations using
natural language supervision. The approach involves collecting billions of
image-text pairs from the internet and pre-training the model on the data
using contrastive learning techniques. This extensive pre-training on large-
scale datasets substantially improves the transferability of the model to a
variety of downstream tasks and datasets.

However, pre-training models on billion-scale datasets require thousands of
GPU days, which is prohibitively expensive. The Fast Language-Image Pre-
training (FLIP) approach involves randomly masking a large proportion of the
patches in each training image to improve the efficiency of pre-training.
Remarkably, as mentioned above, FLIP not only outperforms models trained
without masking but also reduces computational requirements by 2–4 times by
masking 50% to 75% of the image patches.

The Attentive Mask CLIP (A-CLIP) refines the image patch masking strategy by
utilizing correlation scores between image and text [7]. The correlation
scores are computed using the attention weights to the [CLS] token of the
visual encoder. By reducing the image resolution of the Exponential Moving
Average (EMA) encoder by half and masking 50% of the image patches of the
image encoder, A-CLIP reduces 14% of pre-training time while also improving
performance. Nevertheless, selecting external patches for processing still
requires over 30% more training time compared to masking patches directly.

Masked autoencoders (MAE) develop an encoder-decoder architecture to visual
representations through self-supervision [3]. In this approach, the MAE
randomly masks most of the patches in the input image during the encoding
stage and then reconstructs these masked patches using the decoder. The pre-
trained encoders developed through this method can be effectively used in
downstream tasks, such as image classification. Although MAE appears to have
originally inspired FLIP, the original FLIP paper [2] did not find
reconstruction helpful, and GLIP also does not use reconstruction.

###  2.2 Relative Importance of Image Regions

There are several strands of literature motivating our move from random
masking to centered masking because the build on the idea that not all parts
of an image are equally important. The first that comes to mind might be the
concept of model attention in deep learning, which is a mechanism that allows
models to focus on specific parts of the input data that are more relevant for
a given task [9, 10, 11, 12, 13]. GLIP exploits the same notion that underlies
model attention but in a simple manner rooted in the assumption that the
center of the image is most important.

The importance of the center of the image is exploited for data augmentation
during training. Center cropping and random cropping are a commonly used data
augmentation technique for image preprocessing for deep learning models,
especially in tasks related to computer vision [14, 15, 16, 17] and Vision-
Language Model [2, 1]. Center cropping assumes that the most relevant
information is located in the center of the image and helps to reduce
overfitting by providing the model with slightly different views of the same
image during training [14, 15, 16]. This assumption applies to many datasets,
especially those collected from the Internet, which contain images framed by
photographers [6, 18, 19, 20]. MaskCLIP [21] and Masked Siamese Networks (MSN)
[22] augment images through the masking of image patches in Vision Transformer
(ViT) [23], subsequently aligning the representations of both masked and
unmasked images. This process contributes to the improvement of the semantic
performance of images by ensuring the semantic content of both masked and
unmasked images is coherent. In our study, we do not draw comparisons with
these methods. In our study, we do not compare our method with MaskCLIP. This
is because MaskCLIP focuses on maintaining semantic consistency between masked
and unmasked images to enhance image representation via self-supervised
learning, rather than reducing pre-training time.

###  2.3 Photographer behavior

When photographers take pictures they have a strong tendency to place the
subject of the image near the center of the image [24, 25, 26]. The placement
of the subject or object within a photograph greatly influences its
composition and aesthetic appeal [24, 25]. The Rule of Thirds and Center
Composition are two widely accepted techniques for enhancing the visual appeal
of an image [24]. The Rule of Thirds involves dividing the frame into nine
equal sections with two horizontal and two vertical lines and positioning the
subject at the intersections or along these lines to create balanced, visually
pleasing images [24]. Centre composition focuses on placing the subject at the
center of the picture, highlighting symmetry and prominence, a technique
commonly used in portraiture and symmetrical scenes [25]. A commonality across
these techniques is the avoidance of placing objects at the edges of the
image. We refer to the tendency of a dataset to contain images with the main
subject material as “center focus”. In addition to being created by a
photograph “center focus” can also arise when someone crops an image after it
has been taken so that the main subject appears close to the center.

Figure 2: Our GLIP architecture. Following CLIP [1] and FLIP [2], we use
contrastive loss to pre-train our model. Different from FLIP, we mask image
patches with a Gaussian distribution instead of random masking.

##  3 Method

The key idea of FLIP is to enhance computational efficiency and increase the
batch size of contrastive objectives by applying masks to the input image in
CLIP. GLIP differs by masking image patches according to the idea that the
center of an image is more important. This technique substantially improves
performance across various tasks almost without additional computation,
outperforming FLIP. Our image masking strategy is straightforward: we utilize
probabilities from a Gaussian distribution to mask the image patches, as
illustrated in Figure 2. Considering the three-dimensional nature of the
image, we employ a Bivariate Gaussian Distribution for this purpose:

$f(x,y)=\frac{1}{2\pi\sigma_{x}\sigma_{y}\sqrt{1-\rho^{2}}}\exp\left(-\frac{1}{2(1-\rho^{2})}\left[\frac{(x-\mu_{x})^{2}}{\sigma_{x}^{2}}+\frac{(y-\mu_{y})^{2}}{\sigma_{y}^{2}}-\frac{2\rho(x-\mu_{x})(y-\mu_{y})}{\sigma_{x}\sigma_{y}}\right]\right)$

(1)

  * •

$f(x,y)$ is the probability density function of the bivariate Gaussian
distribution.

  * •

$\mu_{x}$ and $\mu_{y}$ are the means of the two variables, $x$ and $y$,
respectively.

  * •

$\sigma_{x}$ and $\sigma_{y}$ are the standard deviations of the two
variables, $x$ and $y$, respectively.

  * •

$\rho$ is the correlation coefficient between $x$ and $y$.

when set $\mu_{x}$ and $\mu_{y}$ to $0$ and set $\rho$ to $0$:

$f(x,y)=\frac{1}{2\pi\sigma_{x}\sigma_{y}}\exp\left(-\frac{1}{2}\left[\frac{x^{2}}{\sigma_{x}^{2}}+\frac{y^{2}}{\sigma_{y}^{2}}\right]\right)$
(2)

Finally, we use Equation 2 to calculate the probability that an image patch
will be masked. In this formula, $x$ and $y$ are within the range of $[-1,1]$,
and the step size corresponds to the grid size of the image encoder. We set
the center of the image as the coordinate origin, which is the center of the
bivariate Gaussian distribution. The examples are shown in Figure 1 and 3, we
remain more patches in the center of the image.

|  |  |
---|---|---|---
(a) Random Masking | (b) $\sigma=0.1$ | (c) $\sigma=0.2$ | (d) $\sigma=0.8$
Figure 3: Comparison of Random and Gaussian Masking Strategies. Image (a)
demonstrates a random masking strategy with uniform masking probability.
Images (b), (c), and (d) illustrate Gaussian masking with increasing standard
deviations ($\sigma$), showcasing the effect of masking that is focused in the
center and gradually spreads to the edges.

Building on the CLIP [1] method, we pre-trained the model using Noise-
Contrastive Estimation (InfoNCE) loss with temperature parameter $\tau$ [27,
1] is shown in Equation 3. This function works by bringing positive image-text
pairs closer and negative image-text pairs farther apart [28]. This approach
allows the encoder to learn and recognize similar semantics in the
corresponding image-text pairs.

$\mathcal{L}_{\text{InfoNCE}}=-\log\left(\frac{\exp(\text{sim}(I_{i},T_{i})/\tau)}{\sum_{j=1}^{N}\exp(\text{sim}(I_{i},T_{j})/\tau)}\right)$
(3)

Here, $sim$ is the similarity score between image $I_{i}$ and text $T_{i}$.
Following FLIP [2], we also continue pre-training the model without masking
for a small number of steps to reduce the distribution gap between training
and testing caused by masking.

##  4 Experiments

Table 1: The details of pre-training and fine-tuning setup. Config | Pre-training Value | Fine-tuning Value
---|---|---
Optimizer | AdamW | AdamW
Learning rate | 1e-3 | 5e-6
Weight decay | 0.1 | 0.05
Optimizer momentum | $\beta 1,\beta 2=0.9,0.999$ | $\beta 1,\beta 2=0.9,0.98$
Learning rate schedule | Cosine decay | Cosine decay
Warmup steps | 10k | 10% of total steps
Epoch | 30 | 1
Numerical precision | Automatic mixed precision | Automatic mixed precision
Augmentation | RandomResizedCrop | RandomResizedCrop

###  4.1 Implementation Details

Our implementation follows CLIP [1], OpenCLIP [29] and FLIP [2].

Dataset. We pre-trained our model on CC12M [20] dataset which includes 12
million image-text pairs. To observe the performance of the model that was
pre-trained on a different dataset, we further evaluate our method on CC3M
[20]. These datasets were selected due to their diverse range of real-world
concepts and scenes. Models such as SLIP, CLIP, and BLIP [30, 31, 7] also use
these popular datasets to train their models. Despite expired URLs, we
successfully downloaded approximately 2.72 million and 9.30 million image-text
pairs for CC3M and CC12M, respectively. Notably, our model achieved similar
baseline performance levels as reported by SLIP and A-CLIP for both CC3M and
CC12M [31, 7], demonstrating its effectiveness and robustness across diverse
datasets.

Architecture. For our image encoders, we employed the ViT-B/16 [23] with a
patch size of 16, while utilizing a transformer-based model for the text
encoder [32]. According to the results reported by FLIP, patch size is not an
important contributor to performance. The maximum context length of the text
encoder used in our study is 32 [2]. Consistent with the configurations in
CLIP [1] and FLIP [2], the input image size is 224 $\times$ 224\. In our
principal experiments, we specified the values of $\sigma_{x}$ and
$\sigma_{y}$ to be 0.20 in the formula 2 as the default value, unless
otherwise mentioned. Additionally, we explored the impact of varying
$\sigma_{x}$ on the performance of GLIP.

Training and fine-tuning. Following the FLIP [2], we pre-train the model for
30 epochs with patch masking and fine-tune the model for small steps without
patch masking. Pre-training and fine-tuning configurations are detailed in
Table 1, utilizing 8 RTX A5000 GPUs with batch sizes of 160, 320, 640, and
1024 for mask ratios of 0%, 50%, 75%, and 90%, respectively.

###  4.2 Evaluation.

First, we evaluate our approach using the ImageNet dataset for the zero-shot
classification task, comparing its performance with CLIP and FLIP.
Subsequently, we extended our evaluation to include various other datasets. we
follow the prompt engineering of CLIP [1] and OpenCLIP [29]. We use the
average of a set of 80 templates embedding for each class. Then, the cosine
similarity between the caption and the image embeddings is calculated to
obtain the top-1 class as the predicted class for each image.

Additionally, we evaluate our approach across various tasks to demonstrate its
advantages. These tasks include image-text retrieval on the MS COCO [33] and
Flickr30K [34] datasets, as well as linear probing and fine-tuning on
ImageNet-1K [5]. This comprehensive evaluation highlights the versatility and
effectiveness of GLIP in addressing the various challenges in the field.

Table 2: Zero-shot accuracy on ImageNet-1K classification. We pre-trained the models for 30 epochs on the CC12M [6] dataset by different image patch mask ratios with ViT-B/16 as the image encoder. Then, we fine-tuned them by an additional epoch without image masking.  Method | Masking | Inference | After tuning
---|---|---|---
Masking | Unmasking
CLIP | ✗ | ✗ | 35.5 | ✗
FLIP | 50% | 32.1 | 34.0 | 34.2
GLIP | 33.2 | 35.1 | 35.4
FLIP | 75% | 26.3 | 29.4 | 30.0
GLIP | 28.8 | 32.1 | 32.2
FLIP | 90% | 16.6 | 21.1 | 22.0
GLIP | 20.7 | 16.5 | 25.8

ImageNet zero-shot transfer. We initially compare our model against the CLIP
baseline on the ImageNet-1K dataset for zero-shot transfer tasks. As shown in
Table 2, our model, with a 50% masking rate and inference unmasking, achieved
a 35.1% accuracy, slightly trailing CLIP by 0.4%. However, after additional
tuning without image unmasking, our model performance is very close to CLIP.
In comparison, FLIP’s performance remains 1.3% below CLIP’s, even after
unmasking tuning. Subsequently, we evaluated GLIP against FLIP, also as shown
in Table 2. Here, GLIP surpassed FLIP by 1.1% in accuracy, employing a 50%
masking rate and inference unmasking. After fine-tuning, our model maintained
a 1.2% lead over FLIP. Further, as we increase the masking ratio, GLIP’s
advantage over FLIP extends from 1.2% (with a 50% masking ratio and tuning) to
3.8% (with a 90% masking ratio and tuning), showcasing our approach’s efficacy
across different masking ratios.

At a 90% masking rate, GLIP exceeds FLIP’s performance by 4.1% with inference
masking. However, performance drops substantially from 20.7% to 16.5% with
inference unmasking, suggesting a reduction in the model’s generalizability
when masking is removed. Nevertheless, small steps of fine-tuning can
substantially improve performance, from 16.5% to 25.8%, indicating that GLIP
is better at learning the relationship between images and text compared to
FLIP. Impressively, GLIP also achieved a zero-shot classification accuracy of
25.8% on ImageNet-1K, while requiring only 10% of the image encoder computing
resources normally required by an image encoder. Large masking ratios allow us
to use larger batch sizes to pre-train the model. We anticipate that GLIP will
effectively extend the advantages observed on smaller datasets to larger ones,
such as LAION-400M [19] and LAION-2B datasets [18]. This is because larger
mask ratios allow more samples to be seen in the same training time, a factor
that has been proven to enhance performance in the CLIP [1] and OpenCLIP [29].

Table 3: Linear probing and fine-tuning accuracy on ImageNet-1K classification. The performance of FLIP and GLIP that pre-trained on CC12M [6].  Dataset | Method | 0-shot | Linear | Finetuned
---|---|---|---|---
CC12M | CLIP | 35.5 | 59.87 | 71.80
FLIP | 34.2 | 59.13 | 71.75
GLIP | 35.4 | 59.97 | 71.80

ImageNet linear probing. Linear probing is to fine-tune a linear classifier on
top of a pre-trained model while keeping the model’s weights fixed. The second
column of Table 3 shows the performance outcomes of various methods pre-
trained on different datasets. In this evaluation, GLIP surpasses the
performance of FLIP, achieving an improvement of 0.84%. Additionally, GLIP
marginally outperforms CLIP, with a 0.1% higher accuracy.

ImageNet fine-tuning. For ImageNet fine-tuning, the process includes training
linear classifiers on the ImageNet dataset without freezing the pre-trained
model’s weights. In this case, CLIP and GLIP perform equally well on this
task, slightly better than FLIP, with a 0.05% lead over FLIP.

Table 4: Zero-shot accuracy on more classification datasets. FLIP and GLIP are pre-trained with 50% ratio image patch masking and fine-tuned an additional epoch without image masking.  Category | Dataset | CLIP | Before tuning | After tuning
---|---|---|---|---
FLIP | GLIP | FLIP | GLIP
Object | Food101 | 40.46 | 39.02 | 40.04 | 39.68 | 40.42
CIFAR10 | 66.22 | 59.41 | 51.63 | 56.19 | 49.24
CIFAR100 | 28.83 | 26.10 | 26.13 | 24.59 | 24.86
CUB | 7.28 | 9.42 | 9.23 | 9.79 | 9.98
Cars | 13.70 | 10.24 | 13.02 | 10.24 | 12.78
Aircraft | 2.38 | 2.59 | 3.11 | 2.47 | 3.05
DTD | 18.56 | 16.76 | 18.14 | 17.02 | 18.30
Oxford Pets | 51.67 | 50.06 | 54.30 | 50.33 | 54.95
Caltech101 | 71.06 | 66.82 | 69.72 | 65.99 | 70.16
Flowers102 | 1.37 | 2.48 | 3.47 | 1.60 | 3.51
MNIST | 14.46 | 9.80 | 9.36 | 9.80 | 9.74
STL10 | 91.00 | 87.25 | 88.75 | 87.70 | 87.64
GTSRB | 10.10 | 13.24 | 7.95 | 12.75 | 9.07
| ImageNet-1K | 35.51 | 33.95 | 35.12 | 34.22 | 35.35
Others | SUN397 | 49.77 | 47.69 | 48.10 | 47.35 | 47.94
EuroSAT | 23.92 | 12.48 | 20.20 | 12.18 | 19.38
RESISC45 | 34.84 | 33.65 | 32.59 | 33.84 | 33.10
Country211 | 4.36 | 4.18 | 3.97 | 4.29 | 4.03
PCam | 50.62 | 50.21 | 57.22 | 52.41 | 55.94
KITTI | 25.67 | 37.30 | 34.22 | 37.90 | 30.95
UCF101 | 40.10 | 37.67 | 38.09 | 37.48 | 38.33
Kinetics700 | 24.07 | 23.15 | 23.70 | 23.15 | 23.51
CLEVR | 18.48 | 20.58 | 21.39 | 18.77 | 21.48
HatefulMemes | 54.36 | 51.64 | 50.59 | 51.01 | 50.76
| SST2 | 50.14 | 50.41 | 49.48 | 50.58 | 49.37

Zero-shot classification on more datasets. We evaluate our method across a
variety of datasets to demonstrate its superiority. These are the same
datasets that were used in the original CLIP [1] and FLIP [2] papers. In order
to investigate whether GLIP has a dependence on the characteristics of data,
we divide the datasets into two groups: “Object” and “Other”. Images of
“Object” datasets depict an object and are framed so that the object is in the
middle of the image nearly without exception. Images of the “Other” datasets
include scenes, activities, and datasets that otherwise cannot be considered
“Object” datasets. Note that the “Other” datasets might also have a center
focus, meaning that the main subject matter occurs towards the center of the
image, although this subject material might not be an object (e.g., it might
be a set of objects in a scene). According to our scan of the content of these
datasets, the two datasets that are neither photographer framed nor cropped to
center the main subject content and clearly depart from the assumption of
center focus are EuroSat (satellite images) [35] and PCAM (medical images)
[36, 37].

In Table 4, we see that GLIP outperforms FLIP in in the majority of “Object”
datasets both before and after fine-tuning. Fine-tuning in general provides a
small but noticeable advantage in some but not all cases. Recall that the
number of patches used by GLIP is the same as FLIP, and the only difference is
that GLIP prefers to retain patches closer to the center during masking.

Moving to the “Other” datasets, we might expect that GLIP is less helpful,
since these images are less likely to have their main subject materially
located at the center of the image. We see that GLIP still delivers an
improvement over FLIP in relatively many cases. Surprisingly, this improvement
appears to be greatest for EuroSAT and PCam, the two datasets that are clearly
not center focused. These results are interesting because they demonstrate
that the potential of GLIP is not restricted to image data with specific
characteristics. We return to comment on the cases of EuroSAT and PCam in the
outlook.

In sum, before fine-tuning, The overall average absolute performance of GLIP
on the 26 downstream datasets is improved by 0.54% compared to FLIP. Following
fine-tuning, GLIP continues to hold its advantage across these datasets,
underscoring its robustness and efficiency in handling a variety of datasets.

It is worth noting that both FLIP and GLIP do not perform as well as CLIP on
low-resolution datasets, such as MNIST, CIFAR-10, CIFAR-100, STL10 [38, 39,
40]. The original FLIP paper [2] does not observe this issue. For our smaller
training set, it possibly could have been addressed by data augmentation, a
point we leave for future work. It is also important to note that in contrast
to the original FLIP paper [2], our FLIP model (50% sampling ratio) does not
consistently outperform CLIP. This suggests that very large batch sizes are
critical for optimal performance of the sampling approaches and deserve
further investigation.

Zero-shot robustness evaluation In Table 5, we evaluate robustness, following
the methodologies of Table 16 in [1]. GLIP surpasses FLIP on 4 out of 6
datasets by a margin of 1.15% and achieves comparable performance to CLIP
which is pre-trained on the entire images. This comparison highlights GLIP’s
effectiveness in enhancing robustness across diverse datasets.

Table 5: Zero-shot robustness evaluation, comparison the zero-shot accuracy performance of CLIP, FLIP, and GLIP on various datasets.  Dataset | CLIP | Before tuning | After tuning
---|---|---|---
FLIP | GLIP | FLIP | GLIP
ImageNet-A | 8.47 | 7.03 | 6.95 | 7.47 | 7.45
ImageNet-O | 37.45 | 39.80 | 38.75 | 39.85 | 38.95
ImageNet-R | 46.01 | 39.59 | 43.06 | 40.24 | 43.47
ImageNet Sketch | 23.57 | 19.97 | 22.03 | 20.24 | 22.20
ImageNetV2 | 30.07 | 29.09 | 30.12 | 29.24 | 30.47
ObjectNet | 21.85 | 18.38 | 19.32 | 19.56 | 20.98
Average | 27.90 | 25.64 | 26.71 | 26.10 | 27.25
Table 6: Zero-shot Image-Text Retrieval, we evaluated CLIP, FLIP, and GLIP
image-text retrieval performance on COCO and Flickr30k datasets. FLIP and CLIP
are pre-trained with 50% ratio image patch masking and fine-tuned an
additional epoch.

Model | Text Retrieval | Image Retrieval
---|---|---
Flickr30k | COCO | Flickr30k | COCO
R@1 | R@5 | R@10 | R@1 | R@5 | R@10 | R@1 | R@5 | R@10 | R@1 | R@5 | R@10
CLIP | 55.40 | 81.70 | 88.60 | 34.42 | 61.62 | 72.16 | 41.52 | 70.76 | 79.84 | 23.65 | 47.64 | 59.13
FLIP | 53.90 | 79.70 | 87.50 | 31.82 | 59.12 | 70.42 | 39.80 | 67.58 | 76.66 | 22.42 | 45.62 | 57.40
GLIP | 53.70 | 80.80 | 86.60 | 32.74 | 59.18 | 71.30 | 41.92 | 67.66 | 76.42 | 22.53 | 46.61 | 58.52

Image-Text retrieval Table 6 presents the performance of image-text retrieval
on the COCO [33] and Flickr30K [34] datasets. GLIP outperforms FLIP on both
datasets. However, the performance of both GLIP and FLIP falls short of that
achieved by CLIP, which was pre-trained on the CC12M dataset without any
masking. This contrasts with previous reports suggesting that FLIP surpasses
CLIP in zero-shot image-text retrieval tasks [2]. With these models being pre-
trained on a 400M dataset, our method may still hold an advantage when applied
to larger datasets. Notably, GLIP demonstrates a substantial advantage over
FLIP in image-text retrieval tasks.

Inverse Gaussian Masking We carry out an experiment in which we invert the
Gaussian mask used by GLIP in order to confirm the importance of the center
patches of the image. We “inverse GLIP” a standard task, i.e., ImageNet-1K
zero-shot classification. As expected, this strategy performs poorly (as shown
in row 2 Table 7), GLIP, which employs center masking, outperforms inverse
Gaussian masking by 4.5%. Additionally, inverse Gaussian masking falls 3.3%
short of FLIP, which utilizes random masking. Because ImageNet-1K is
photographer framed, we attribute this difference in performance to the
periphery of an image being more likely to contain background information
rather than the main subject of the image. The contrast between the
performance of inverse-GLIP and FLIP suggests that the strength of FLIP
actually derives from those random patches that it samples that happen to be
near the center of the image.

Table 7: Zero-shot classification performance on ImageNet-1K when we pre-train the model on CC12M using the random, inverse Gaussian and Gaussian masking strategies to mask 50% image patches. The image encoder is ViT-B/16. We set $\sigma$ to 0.2. Method | 0-shot
---|---
Before tuning | After tuning
random masking | 34.0 | 34.2
Inverse Gaussian Masking | 31.1 | 30.9
Gaussian masking | 35.1 | 35.4
Figure 4: Zero-shot classification performance on ImageNet-1K with different
values of $\sigma$ (cf. Formula 3). The models, using ViT-B/16 as the image
encoder, were pre-trained on the CC3M dataset for 30 epochs and fine-tuned for
one more epoch without masking.

Different $\sigma$ For our experiments, the value of $\sigma$ was initially
set by visualizations such as those in Figure 1 and with limited exploratory
experiments. Here, we investigate the sensitivity of the model to $\sigma$ We
again study a standard task, i.e., ImageNet-1K zero-shot, and train models
with different $\sigma$ values on the CC3M dataset. Results are presented in
Figure 4. From 0.1 to 0.2, the performance of the model improves by 0.6%, and
the model achieves optimal performance at $\sigma=0.20$, which is the value
that we had chosen via visualisation. An increase in $\sigma$ from 0.2 results
in decreased performance. However, the model still surpasses FLIP, which
utilizes a random masking strategy, for $\sigma$ values up to 0.8 after fine-
tuning. Thus, it is essential to balance the model’s focus on both the image
center and its edges, ensuring comprehensive attention across the entire
image. Further, from this experiment, we conclude that GLIP has a certain
robustness to the choice of $\sigma$, since a $\sigma$ somewhat larger than
the optimal size will decrease performance only mildly and still allow GLIP to
outperform FLIP.

##  5 Conclusion and Outlook

In this paper, we have introduced GLIP, a novel image masking strategy that
improves the pre-training efficiency of Vision-Language Models. Our method
outperforms the FLIP random image masking strategy by increasing the retention
of image patches at the center of images. We achieved better performance than
FLIP in various zero-shot tasks such as classification and image text
retrieval. Unlike A-CLIP, our approach simplifies the process by eliminating
the need for additional inference to compute the attention scores of image
patches. Across a wide range of tasks and datasets, GLIP demonstrates superior
performance compared to its FLIP counterparts, particularly with the use of a
large masking ratio during pre-training. We also demonstrated that the
performance of GLIP is similar to that of CLIP when we pre-train the model on
the CC12M dataset.

Moving forward, it will be interesting to study the improvement offered by
GLIP on very large-scale datasets. In the original work on FLIP [2],
experiments were reported on training sets with 400 million samples and 2
billion samples. Because FLIP is simply sampling patches, there are no
inherent constraints that prevent it from scaling to larger data. We point out
that GLIP, for the same reason, also lacks any inherent constraints. For this
reason, we are confident that GLIP will also deliver performance improvement
on datasets much larger than the set of 10 million training examples we used
here. Further, because GLIP is effective with a large masking ratio, it can
process more samples than FLIP given the same training time. For this reason,
we expect that GLIP will continue to show stronger competitiveness as datasets
grow larger.

An interesting aspect of GLIP is its applicability across a wide range of
datasets, as demonstrated by our experiments. Intuitively, it might be
expected that GLIP performs better on datasets with center focus, i.e., they
contain images that were taken by a photographer who was actively framing the
images so that the main subject material would be close to the center of the
image, and not at the edge. However, we have seen that GLIP shows a strong
advantage over FLIP on data where there is apparently no explicit center
focus, specifically, for the EuroSAT dataset [35], which contains remote
sensing images, and PCam [36, 37], which contains medical images of tumors. It
remains an open question as to why GLIP performs so well on datasets where the
main information does not have a strong tendency to be located towards the
middle of the image. A possible explanation is that patches at the center of
the image are simply more helpful to the vision encoder because they are
surrounded by full context. A patch located at the periphery of an image does
not enjoy a full context because the image ends at the image edge. Moving
forward, we will continue to explore the nature of the reasons for which the
centered masking of GLIP delivers an improvement over the random masking of
FLIP.

## References

  * [1] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever.  Learning transferable visual models from natural language supervision.  In International Conference on Machine Learning, 2021.
  * [2] Yanghao Li, Haoqi Fan, Ronghang Hu, Christoph Feichtenhofer, and Kaiming He.  Scaling language-image pre-training via masking.  In IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023.
  * [3] Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick.  Masked autoencoders are scalable vision learners.  In IEEE/CVF conference on computer vision and pattern recognition, pages 16000–16009, 2022.
  * [4] Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick.  Momentum contrast for unsupervised visual representation learning.  In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020.
  * [5] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei.  Imagenet: A large-scale hierarchical image database.  In IEEE Conference on Computer Vision and Pattern Recognition, 2009.
  * [6] Soravit Changpinyo, Piyush Sharma, Nan Ding, and Radu Soricut.  Conceptual 12M: Pushing web-scale image-text pre-training to recognize long-tail visual concepts.  In IEEE/CVF Computer Vision and Pattern Recognition Conference, 2021.
  * [7] Yifan Yang, Weiquan Huang, Yixuan Wei, Houwen Peng, Xinyang Jiang, Huiqiang Jiang, Fangyun Wei, Yin Wang, Han Hu, Lili Qiu, et al.  Attentive mask clip.  In IEEE/CVF International Conference on Computer Vision, pages 2771–2781, 2023.
  * [8] Chao Jia, Yinfei Yang, Ye Xia, Yi-Ting Chen, Zarana Parekh, Hieu Pham, Quoc Le, Yun-Hsuan Sung, Zhen Li, and Tom Duerig.  Scaling up visual and vision-language representation learning with noisy text supervision.  In International Conference on Machine Learning, 2021.
  * [9] Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio.  Neural machine translation by jointly learning to align and translate.  In International Conference on Learning Representations, 2015, 2015.
  * [10] Jan K Chorowski, Dzmitry Bahdanau, Dmitriy Serdyuk, Kyunghyun Cho, and Yoshua Bengio.  Attention-based models for speech recognition.  In Advances in Neural Information Processing Systems, volume 28. Curran Associates, Inc., 2015.
  * [11] Jiasen Lu, Jianwei Yang, Dhruv Batra, and Devi Parikh.  Hierarchical question-image co-attention for visual question answering.  Advances in neural information processing systems, 29, 2016.
  * [12] Volodymyr Mnih, Nicolas Heess, Alex Graves, et al.  Recurrent models of visual attention.  Advances in neural information processing systems, 27, 2014.
  * [13] Zhaoyang Niu, Guoqiang Zhong, and Hui Yu.  A review on the attention mechanism of deep learning.  Neurocomputing, 452:48–62, 2021.
  * [14] Ekin Dogus Cubuk, Barret Zoph, Jon Shlens, and Quoc Le.  Randaugment: Practical automated data augmentation with a reduced search space.  In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin, editors, Advances in Neural Information Processing Systems, volume 33, pages 18613–18624, 2020.
  * [15] Terrance DeVries and Graham W Taylor.  Dataset augmentation in feature space.  arXiv preprint arXiv:1702.05538, 2017.
  * [16] Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton.  Imagenet classification with deep convolutional neural networks.  Advances in neural information processing systems, 25, 2012.
  * [17] Connor Shorten and Taghi M Khoshgoftaar.  A survey on image data augmentation for deep learning.  Journal of big data, 6(1):1–48, 2019.
  * [18] Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade W Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, Patrick Schramowski, Srivatsa R Kundurthy, Katherine Crowson, Ludwig Schmidt, Robert Kaczmarczyk, and Jenia Jitsev.  LAION-5B: An open large-scale dataset for training next generation image-text models.  In Conference on Neural Information Processing Systems, Datasets and Benchmarks Track, 2022.
  * [19] Christoph Schuhmann, Richard Vencu, Romain Beaumont, Robert Kaczmarczyk, Clayton Mullis, Aarush Katta, Theo Coombes, Jenia Jitsev, and Aran Komatsuzaki.  LAION-400M: Open dataset of CLIP-Filtered 400 million image-text pairs, 2021.
  * [20] Piyush Sharma, Nan Ding, Sebastian Goodman, and Radu Soricut.  Conceptual Captions: a cleaned, hypernymed, image alt-text dataset for automatic image captioning.  In Annual Meeting of the Association for Computational Linguistics, 2018.
  * [21] Xiaoyi Dong, Jianmin Bao, Yinglin Zheng, Ting Zhang, Dongdong Chen, Hao Yang, Ming Zeng, Weiming Zhang, Lu Yuan, Dong Chen, Fang Wen, and Nenghai Yu.  MaskCLIP: masked self-distillation advances contrastive language-image pretraining.  In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10995–11005, 2023.
  * [22] Mahmoud Assran, Mathilde Caron, Ishan Misra, Piotr Bojanowski, Florian Bordes, Pascal Vincent, Armand Joulin, Mike Rabbat, and Nicolas Ballas.  Masked siamese networks for label-efficient learning.  In European Conference on Computer Vision, pages 456–473, 2022.
  * [23] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby.  An image is worth 16x16 words: Transformers for image recognition at scale.  In International Conference on Learning Representations, 2021.
  * [24] Rudolf Arnheim.  Art and visual perception: A psychology of the creative eye.  Univ of California Press, 1954.
  * [25] Roy H. Quan.  Photography and the creation of meaning.  Art Education, 32(2):4–9, 1979.
  * [26] Benjamin W Tatler.  The central fixation bias in scene viewing: Selecting an optimal viewing position independently of motor biases and image feature distributions.  Journal of vision, 7(14):4–4, 2007.
  * [27] Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton.  A simple framework for contrastive learning of visual representations.  In International Conference on Machine Learning, 2020.
  * [28] Aaron van den Oord, Yazhe Li, and Oriol Vinyals.  Representation learning with contrastive predictive coding, 2019.
  * [29] Gabriel Ilharco, Mitchell Wortsman, Ross Wightman, Cade Gordon, Nicholas Carlini, Rohan Taori, Achal Dave, Vaishaal Shankar, Hongseok Namkoong, John Miller, Hannaneh Hajishirzi, Ali Farhadi, and Ludwig Schmidt.  OpenCLIP, 2021.
  * [30] Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi.  BLIP: bootstrapping language-image pre-training for unified vision-language understanding and generation.  In ICML, 2022.
  * [31] Norman Mu, Alexander Kirillov, David A. Wagner, and Saining Xie.  SLIP: Self-supervision Meets Language-Image Pre-training.  In European conference on computer vision, 2022.
  * [32] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin.  Attention is all you need.  In Advances in Neural Information Processing Systems, 2017.
  * [33] Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick.  Microsoft COCO: Common objects in context.  In European Conference on Computer Vision, pages 740–755, 2014.
  * [34] Peter Young, Alice Lai, Micah Hodosh, and Julia Hockenmaier.  From image descriptions to visual denotations: New similarity metrics for semantic inference over event descriptions.  Transactions of the Association for Computational Linguistics, pages 67–78, 2014.
  * [35] Patrick Helber, Benjamin Bischke, Andreas Dengel, and Damian Borth.  Eurosat: A novel dataset and deep learning benchmark for land use and land cover classification.  IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 12(7):2217–2226, 2019.
  * [36] Babak Ehteshami Bejnordi, Mitko Veta, Paul Johannes van Diest, Bram van Ginneken, Nico Karssemeijer, Geert Litjens, Jeroen A. W. M. van der Laak, , and the CAMELYON16 Consortium.  Diagnostic Assessment of Deep Learning Algorithms for Detection of Lymph Node Metastases in Women With Breast Cancer.  JAMA, 318(22):2199–2210, 2017.
  * [37] Bastiaan S. Veeling, Jasper Linmans, Jim Winkens, Taco Cohen, and Max Welling.  Rotation equivariant cnns for digital pathology.  In Medical Image Computing and Computer Assisted Intervention, 2018.
  * [38] Adam Coates, Andrew Ng, and Honglak Lee.  An analysis of single-layer networks in unsupervised feature learning.  In International Conference on Artificial Intelligence and Statistics, 2011.
  * [39] Alex Krizhevsky, Geoffrey Hinton, et al.  Learning multiple layers of features from tiny images.  2009\.
  * [40] Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner.  Gradient-based learning applied to document recognition.  Proceedings of the IEEE, 86(11):2278–2324, 1998.


# Quantifying the COVID19 infection risk due to droplet/aerosol inhalation

Rahul Bale<EMAIL_ADDRESS>Akiyoshi Iida  Masashi Yamakawa  ChungGang Li
Makoto Tsubokura  RIKEN Center for Computational Science, Kobe 6500047, Japan
Department of Computational Science, Kobe University, Kobe, Japan.  Department
of Mechanical Engineering, Toyohashi Institute of Technology, Toyohashi, Japan
Department of Mechanical Engineering, Kyoto Institute of Technology, Kyoto,
Japan

###### Abstract

The dose-response model has been widely used for quantifying the risk of
infection of airborne diseases like COVID-19. The model has been used in the
room-average analysis of infection risk and analysis using passive scalars as
a proxy for aerosol transport. However, it has not been employed for risk
estimation in numerical simulations of droplet dispersion. In this work, we
develop a framework for the evaluation of the probability of infection in
droplet dispersion simulations using the dose-response model. We introduce a
version of the model that can incorporate the higher transmissibility of
variant strains of SARS-CoV2 and the effect of vaccination in evaluating the
probability of infection. Numerical simulations of droplet dispersion during
speech are carried out to investigate the infection risk over space and time
using the model. The advantage of droplet dispersion simulations for risk
evaluation is demonstrated through the analysis of the effect of humidity on
infection risk.

###### keywords:

COVID-19, Droplet dispersion, airborne disease transmission, dose response
model

##  1 Introduction

Dispersal of infectious pathogens such as viruses through airborne sputum
droplets can lead to the rapid transmission of diseases in the general
population posing a great risk to public health. There are several minor and
major infectious diseases that are transmitted through virus-carrying sputum
droplets. A benign example of airborne disease is the common cold, while more
severe examples include H1N1 influenza, severe acute respiratory syndrome
(SARS), middle east respiratory syndrome (MERS), and coronavirus disease 2019
(COVID-19)[1, 2]. Since its outbreak in 2019, COVID-19 has transformed into
the most destructive pandemic in over a century. The evidence on COVID-19 so
far suggests that the possible modes of transmission of SARS-CoV-2 include
respiratory droplets and aerosols, direct person-to-person contact and contact
with surfaces (fomite mode of transmission). Direct person-to-person contact
transmission and fomite transmission can be controlled by appropriate hygiene
practices. However, controlling airborne transmission is far more challenging.
Therefore, airborne transmission is likely the primary reason for turning
COVID-19 into a global pandemic [3, 4]. The viruses carrying sputum droplets
are generated not only during violent expiratory events like coughing and
sneezing, but they are also generated during routine respiratory activities
like speaking, singing, and breathing. This, coupled with the fact that SARS-
CoV2 is transmitted during presymptomatic and asymptomatic phases of
COVID-19[5], increases the infectiousness and the rapid spread of SARS-CoV2.

As part of developing effective strategies to mitigate the transmission of
COVID-19 or any airborne disease in future, there is a need for quantification
of the risk of infection under various social conditions. The well-mixed room
approximation is one of the approaches that has been used by researchers for
the estimation of infection of risk. By treating the concentration of carbon
dioxide in exhaled air as a proxy for virion concentration in the ambient
environment, the Wells-Riley model has been used for risk estimation in an
indoor environment based on room-scale averages[6, 7]. Recent works on this
quantification of COVID-19 have also used a similar approach and applied it to
indoor spaces such as bathrooms, laboratories and offices wherein the viral
shedding of occupants was taken the source of virions in room-average analyses
[8, 9, 10]. These approaches provide a measure of the average infection risk
for the whole environment under consideration rather than the local risk of
infection for each occupant. Yang et al. developed a framework for risk
estimation between individuals during casual conversation. The method involves
the estimation of viral concentration as a function of space and time by using
passive scalars as a proxy for aerosols ejected during speech. The risk of
infection is then estimated by evaluating the viral particles that are likely
to be within the breathing zone of a susceptible individual. A similar method
has been adopted by Singhal et al. for risk estimation during face-to-face
conversations[11]. These methods assume that the aerosols are well-mixed with
exhaled air close to the source of aerosol ejection. However, the well-mixed
aerosol assumption is reasonable for high aerosol concentration but not when
the aerosol concentration is low. Moreover, the approach cannot account for
the possible local variations in aerosol concentration due to changes in
environmental factors like temperature and humidity. The aerosolization of
small and medium-sized droplets is influenced by the humidity and temperature
of the ambient environment. Direct measurement of aerosols and droplets that
are likely to be within the breathing zone of a susceptible individual can
provide a better estimate of infection risk. The focus of the present work is
the estimation of infection risk by direct measurement of droplets and
aerosols in the breathing zone of susceptible individuals using numerical
simulation of droplet dispersion. To that end, we extend the dose-response
model for droplet dispersion simulations for expiratory events like coughing,
sneezing, speaking, etc. Using numerical simulations of droplet dispersion,
infection risk during face-to-face conversation is investigated in this work.
Furthermore, we also investigate the effect of humidity on the infection
probability during casual conversations.

##  2 Estimation of Infection Probability

The risk of infection due to inhalation of viruses is commonly quantified
using the dose-response model[12, 13, 14]. The model assumes that the number
of viral particles needed, on average, to infect an individual is $N_{0}$.
Assuming that the infection is a Poisson process, the probability of infection
can be written as

$P=1-e^{(-\frac{N}{N_{0}})},$ (1)

where $N$ is the total number of virions inhaled. In order to compute the
probability of infection $P$, both $N$ and $N_{0}$ have to be estimated. A
range of values have been reported in the literature for $N_{0}$. Prentiss et
al. have estimated a range of 322-2012 based on superspreading events at the
early stage of COVID-19 pandemic [15], other similar studies provide estimates
of the value that vary between 100 and 1000 [10, 9]. In this study, we choose
a value of 900 which lies within the range of values reported in the
literature.

The number of virions inhaled depends on the total duration of exposure, $T$,
the amount of air inhaled by a person breathing at the rate of B, and the
local concentration of virions, $C(t)$, in the breathing zone of the person.
Therefore, the relationship of $N$, with $T,B\;\&\;C$ can be expressed as

$N=B\int_{0}^{T}C(t)dt.$ (2)

While breathing rate is relatively constant for a given physical activity, the
local concentration is transient and can vary with time. Depending on the type
of physical activity, the breathing rate can vary from moderate values ($0.45$
m3/hr) to values as high as $3.3$ m3/hr [16, 17, 18]. The breathing rate for
sedentary activities like quite breathing, speaking at normal voice and
singing are in a similar range: $0.54\pm 0.21$ m3/hr, $0.54\pm 0.21$ m3/hr and
$0.61\pm 0.4$ m3/hr, respectively[16]. For moderate physical activities like
cycling and climbing stairs the breathing rate varies between $1.3-1.5$ m3/hr,
and for heaving activities like climbing with load, cross country skiing, the
range is $2.5-3.3$ m3/hr [17]. For the present study, we choose a value of
$B=0.5$ m3/hr which assumes that the subject at risk of infection is not
involved in any strenuous activity. With the value of $B$ known, the
estimation of $N$ depends on the estimation of $C(t)$. There have been two
main approaches of estimating $C(t)$ in literature. The first approach is
based on a well-mixed room averaged approximation[15, 10]. In this approach,
the virions emitted by an infected person are assumed to be well-mixed due to
air circulation and mixing within the domain under consideration. The change
in virion concentration in the room is tracked over time, but the
concentration is assumed to be uniform throughout the domain under
investigation. An alternate approach proposed by Yang et al.[19] involves
estimation of $C(t)$ over time and space. In this method, assuming that the
aerosol-laden air is well-mixed at the point of ejection during expiratory
events such as speaking, singing, coughing, etc., a passive scalar is used to
model the transport of aerosols in direct numerical simulation or large eddy
simulations. Taking passive scalar concentration as a proxy for virion
concentration, an expression of $C(t)$ is obtained.

Figure 1: A schematic of droplet dispersion (A) during face to face
conversation between two subjects, (B) between a group of people seated at a
table. The region highlighted by a blue box around the mouth and nose of a
susceptible subject, termed the breathing zone, is used to track droplets that
are likely to be inhaled.

In contrast to the two methods available in the literature, in this work, we
develop a new model to estimate virion inhalation as a function of space and
time in droplet dispersion simulations. As opposed to the well-mixed
approximation of aerosol-laden air in the passive scalar approach, in
simulations modeling direct droplet dispersion, the dynamics of
aerosol/droplet transport and evaporation are considered. In contrast to
aerosol transport, the dynamics of transport and mixing of droplets can be
different due to the larger inertia of droplets. Aerosolization of small and
medium droplets, depending on the ambient temperature and humidity, can
significantly alter local aerosol and virion concentration. Therefore, the
results of droplet dispersion simulations can be qualitatively different from
those of the passive scalar approach.

For the evaluation of infection probability in droplet dispersion simulations,
we estimate $N$ by tracking the number of droplets in a spatial region defined
such that the air within this region is likely to be inhaled by a susceptible
subject. Henceforth, we shall refer to this region as the breathing zone. A
schematic of the breathing zone is shown in Fig. 1. For simplicity, we have
chosen a small rectangular region around the mouth and nose to represent the
breathing zone. The shape of the region could in principle be arbitrary.
Example of how the breathing zone is defined in two scenarios is shown in Fig.
1. The first scenario is a face-to-face conversion between two persons. Second
is a conversion between a group of people seated at a table. In all such
situations, one or more infected subjects are modeled as the source of
droplets/aerosols while the remaining subjects are treated as being
susceptible to infection. A separate breathing zone is defined for each
subject who might be susceptible to inhaling the virus-laden droplets and
getting infected. A separate infection probability of each subject would be
evaluated by tracking the droplets in the respective breathing zones. We
choose a $10\times 10\times 15cm$ sized rectangular region to model the
breathing zone for the present study. The longer side of the box is along the
nose-ground direction. With regards to the placement of the breathing zone,
the centre of one of the edges of the top face of the box is placed at the
bottom edge of the nose and an adjacent face is placed in contact with the
outermost surface of the mouth of a subject. Lastly, the breathing zone
placement is such that its entire volume is outside the subject’s interior.

The number of virions the subject is likely to inhale depends on the local
concentration of virions in the breathing zone. The local concentration can be
written as

$C(t)=\frac{n(t)}{\upsilon_{\text{B}}},$ (3)

where $n(t)$ is the instantaneous count of virions at the time instant t in
the breathing zone and $\upsilon_{\text{B}}$ is the volume of the breathing
zone. The instantaneous virion count $n(t)$ is the product of the total
ejection volume of droplets and aerosols, $\upsilon_{d}^{0}(t)$, in the
breathing zone and the viral load or viral density $\lambda_{v}$ (copies/m3).
The superscript 0 in $\upsilon_{d}^{0}(t)$ is used to imply that the volume of
the droplets in question is the volume at the time of ejection from the mouth
of the infected subject. Even as a droplet evaporates, its instantaneous
volume, $\upsilon_{d}$, decreases but its injection volume remains unchanged.
Therefore, the total virion count of a given droplet remain constant as a
droplet evaporates and aerosolizes. The average viral load of SARS-CoV2 in the
sputum is $7\times 10^{6}$ copies/ml and the maximum reported value is
$2\times 10^{9}$ copies/ml[20]. For the analysis in the present work we choose
the value $7\times 10^{6}$ copies/ml.

The local virion concentration can now be expressed as a function of viral
load and droplet volume

$C(t)=\frac{\upsilon_{d}^{0}(t)\lambda_{v}}{\upsilon_{\text{B}}}.$ (4)

Substituting the above expression into Eq. 2 we obtain the following
expression for $N$

$N=\frac{B\lambda_{v}}{\upsilon_{\text{B}}}\int_{0}^{T}\upsilon_{d}^{0}(t)dt.$
(5)

In activities like speaking and singing over prolonged periods of time, the
rate of droplet generation and dispersion are expected to reach a quasi-
steady-state. The injection droplet volume that enters the breathing zone
would reach a steady state value $\overline{\upsilon}_{d}^{0}$. Therefore,
under steady-state conditions the above equation can be simplified to

$N=\frac{B\lambda_{v}\overline{\upsilon}_{d}^{0}T}{\upsilon_{\text{B}}}.$ (6)

This equation is applicable to quasi-steady processes like singing and
speaking and it is not applicable to transient situations like sneezing and
coughing. We must resort to Eq. 5 for transient cases.

Risk after vaccination and due to variant strains: Most of the major vaccines
for COVID-19 have reported high efficacy in preventing infection and very high
efficacy in preventing severe disease and hospitalization. For example,
vaccines by Pfizer, Moderna and AstraZeneca have been reported to have vaccine
efficacy of $95\%$[21], $94.1\%$[22] and $81.5\%$[23], respectively. From the
viewpoint of evaluating the infection probability of a vaccinated subject, the
effect of a vaccine may be interpreted as an increase in the minimum number of
virions needed to infect a person. This assumption implies that for exposure
to small doses of virions, the infection probability will be very low or
negligible. However, even an infected person, if exposed to very high doses of
virions, may be at risk of infection.

As SARS-CoV2 has spread through populations, it has mutated into several
variants[24]. It has been reported that the transmissibility of some of the
variants of SARS-CoV2 is higher than that of the original strain [25, 26]. For
example, the B.1.1.7 strain (alpha variant) has been reported to be $29\%$
more transmissible than the original strain[25] and the B.1.617.2 strain
(delta variant) has been estimated to be 43 to 90$\%$ more transmissible than
B.1.1.7 strain[26]. Within the framework of the infection risk model
considered in this work, the higher transmissibility of a given variant could
be interpreted as due to two reasons. First, the viral load of the variant
strains could be higher. Second, the minimum virion dose needed for infection
$N_{0}$ for a variant could be lower. The higher transmissibility could be due
to one or a combination of these two factors. Under this assumption, the
effect of vaccines and variant strains on the probability of infection can be
incorporated into the risk model given by Eq. 1 as follows.

$P=1-e^{(-\alpha\frac{N}{N_{0}})},$ (7)

where $\alpha$ is factor that accounts for higher transmissibility of variant
strains, lower risk of infection for vaccinated individuals, and with
$\alpha=1$ the above equation falls back to the original form in Eq. 1. For
the case where a person is vaccinated with a vaccine of efficacy $\eta_{vc}$,
it can be shown that $\alpha=1-\eta_{vc}$. The details of derivation of
$\alpha$ can be found in the Appendix section. The effect of vaccination may
be interpreted as an increase in the minimum number of virions $N_{0}$ needed
to infect a vaccinated person. If the efficacy of a vaccine is $100\%$ then
the $P=0$. But, when vaccine efficacy is 0, Eq. 7 returns to the original form
of $P$ in Eq. 1. For a variant strain with a higher transmissibility factor
$\tau$, we can write $\alpha=\tau$. For the alpha, strain the transmissibility
factor is $\tau=1.29$[25], and for the delta strain it is $\tau=2.45$[26]
(assuming 90$\%$ higher transmissibility compared to alpha variant). For the
original strain the transmissibility factor is $\tau=1$, in which case Eq.7
return to the original form in Eq.1.

##  3 Methods

###  3.1 Governing Equations

The flow solver used in the present work for carrying out the numerical
simulation of droplet dispersion is made of an Eulerian reference frame for
solving the fluid flow and species transport equations and a Lagrangian frame
for solving the droplet dynamics model. The equations of motion of mass,
momentum, energy and species transport can be expressed in compact notation as

$\frac{\partial\rm{U}}{\partial t}+\nabla\cdot\mathbf{F}=\mathbf{S}.$ (8)

Here, $\rm{U}$, $\mathbf{F}$ and $\mathbf{F}$ represent the primitive flow
variables, the combined convective and diffusive terms, and the source terms,
respectively [27]. The primitive variables vector and the flux vector are
expanded below.

$\mathbf{U}=\left(\begin{array}[]{c}\rho\\\ \rho u_{1}\\\ \rho u_{2}\\\ \rho
u_{3}\\\ \rho e\\\ \rho Y_{k}\end{array}\right),\quad
F_{i}=\left(\begin{array}[]{c}\rho u_{i}\\\ \rho u_{i}u_{1}+P\delta_{i1}-\mu
A_{i1}\\\ \rho u_{i}u_{2}+P\delta_{i2}-\mu A_{i2}\\\ \rho
u_{i}u_{3}+P\delta_{i3}-\mu A_{i3}\\\ \rho(\rho e+P)u_{i}-\mu
A_{ij}u_{j}+q_{i}\\\ \rho
u_{i}Y_{k}-\rho\hat{u}_{i}^{k}Y_{k}\end{array}\right)$ (9)

where the density and viscosity are represented by $\rho$ and $\mu$,
respectively. $\mathbf{u}$, $e$ and $P$ are the velocity, the total specific
energy and the pressure, respectively. The components of the velocity along
the principle directions $1,2,3$ are given by $(u_{1},u_{2},u_{3})$. The vapor
phase of water from the liquid sputum is modeled as passive scalar species
along with O2 and N2. The mass fraction of the species indexed $k$ is
represented by $Y_{k}$ and $\hat{u}^{k}_{i}$ is corresponding diffusion
velocity of the $k^{th}$ species. $\mathbf{q}=-\lambda\nabla T,$ is the heat
flux where $T\&\lambda$ represent temperature and thermal diffusivity,
respectively. The density and pressure are constrained together by the state
equation $P=\rho RT$, in which $R$ is the gas constant and $T$ is the
temperature. The total specific energy is given by

$e=\frac{P}{\gamma-1}+\frac{1}{2}u_{i}u_{i},$ (10)

where $\gamma$ is the ratio of the gas specific heat capacities. The diffusion
velocity $\hat{\bm{\mathrm{u}}}^{k}Y_{k}$ of the $k^{\textrm{th}}$ species is
defined in terms of the species diffusivity $D_{k}$, the relationship is given
by

$\hat{\bm{\mathrm{u}}}^{k}Y_{k}=D_{k}\nabla Y_{k}.$ (11)

The contribution to the source comes from the bouyancy term and the weak two-
way coupling between the droplet-model and flow equations. The source term
vector is given by

$\mathbf{S}=\begin{pmatrix}{0}\\\ {(\rho-\rho_{0})g_{1}}\\\
{(\rho-\rho_{0})g_{2}}\\\ {(\rho-\rho_{0})g_{3}}\\\
{(\rho-\rho_{0})g_{i}u_{i}}\\\ {S_{\rho Y_{k}}}\end{pmatrix}.$ (12)

$\rho$ and $\rho_{0}$ are the local and far field ambient density,
respectively and $\mathbf{g}$ is the acceleration due to gravity (eg.
$\mathbf{g}=(0,0,-9.81)m/s^{2}$) Of the species source terms $S_{\rho Y_{k}}$,
the non-droplet vapor species are zero.

###  3.2 Droplet Model

The widely used single droplet model is adopted in this work for modeling the
sputum droplet dynamics. The droplets are modeled as discrete Lagrangian
entities which are coupled with the Eulerian fluid flow equations for a weak-
two-way coupling. The droplet transport and evaporation are influenced by the
conditions of the ambient air, but the flow field is not affected by the
droplets except the species of the vapour phase of the liquid droplet. The
transport of the droplets is modeled by

$\displaystyle\frac{d\mathbf{x}_{d}}{dt}$ $\displaystyle=\mathbf{u}_{d},$ (13)
$\displaystyle\frac{d\mathbf{u}_{d}}{dt}$
$\displaystyle=\frac{3C_{D}}{4d_{d}}\frac{\rho}{\rho_{d}}\left(\mathbf{u}-\mathbf{u}_{d}\right)\left|\mathbf{u}-\mathbf{u}_{d}\right|+\mathbf{g},$

where $\mathbf{x}_{d}$ and $\mathbf{u}_{d}$ are the position and velocity of
an individual droplet, respectively. $d_{d}$ and $\rho_{d}$ are the droplet
diameter and liquid density of the droplet, respectively. $C_{D}$ is the drag
coefficient expressed a function of the droplet Reynolds number $Re_{d}$. The
droplet evaporation, influenced by the ambient air’s velocity, humidity and
temperature, is

$\displaystyle\frac{dT_{d}}{dt}$
$\displaystyle=\frac{Nu}{3Pr}\frac{c_{p}}{c_{l}}\frac{f_{1}}{\tau_{d}}\left(T-T_{d}\right)+\frac{1}{m_{d}}\left(\frac{dm_{d}}{dt}\right)\frac{L_{V}}{c_{p,d}}$
(14) $\displaystyle\frac{dm_{d}}{dt}$
$\displaystyle=-\frac{m_{d}}{\tau_{d}}\left(\frac{Sh}{3Sc}\right)\ln\left(1+B_{M}\right)$

The temperature $T_{d}$ is updated by tracking the convective heat transfer
with the ambient air and the evaporative heat loss. The mass rate of change of
the droplets is influenced by the local vapour fraction and air velocity which
are encapsulated in mass transfer number $B_{m}$ and the Sherwood number $Sh$.
Here, $m_{d}$ and $L_{V}$ are the mass of the droplet and the latent heat of
evaporation at the droplet temperature, respectively. $c_{p}$ and $c_{l}$ are
the specific heat at a constant pressure of the ambient air and the specific
heat capacity of the liquid droplet, and $\tau_{d}$ is the response time of
the droplet. Further details of the various terms involved in the droplet
model can be found in the work of Bale et al[28, 29].

###  3.3 Solver framework and simulation environment

A multi-physics solver known as CUBE[30, 31] has been used for all the
numerical simulations presented in this work. CUBE is a finite volume solver
based on a hierarchical meshing framework known as the building cube method
(BCM)[32]. The meshing framework allows local mesh refinement enabling the
high resolution in regions of interest while limiting the overall cell count.
The supercomputer Fugaku has been used for carrying out the numerical
simulation presented in this work. Fugaku comprises 158,976 nodes. Each node
is equipped with a Fujitsu A64FX processor, which consists of 48 compute cores
and 4 additional cores, and a memory of 32 GiB. The nodes are interconnected
with 28Gbps, 2 lanes, 10 port TofuD interconnect.

###  3.4 Droplet modeling parameters

Numerical simulation of droplet dispersion requires three main input
parameters - a) the distribution of the droplet diameter and a count of the
droplet number ejected from the mouth, b) the flow profile of the expiratory
event in consideration such as speaking or coughing, c) the average area of
the mouth opening. Data on the droplet size distribution and the number of
droplets for speech as well as cough have been widely reported in
literature[33, 34, 35, 36]. The distribution and the droplet number reported
for speaking in these studies varies significantly. The droplet concentration
($\\#$/L), a proxy for droplet number, for speaking reported by Duiguid,
Loudon and Roberts, and Chao et al. are 3.72, 223.25 and 150.8[37, 33, 34],
respectively. The difference in the reported droplet size distribution is also
as disparate as the data on droplet number. The diameter corresponding to the
highest droplet count reported by Loudon and Roberts was 6 $\mu$m, this value
in the study of Duguid and Chao et al. is about 12 $\mu$m, on the other hand,
Xie et al.[36] report a value as high as 50 $\mu$m. As there is so much
variation in the number and distribution of droplet size in this work we adopt
a combination of distribution Duguid[37] and Xie et al.[36]. The distribution
of droplet diameter adopted in this work is shown in Fig. 2.

For the flow generated from the mouth during speech, we adopt a sinusoidal
model. The flowrate generated when counting from 1 to 10 is modeled as
$\dot{q}=A_{i}\sin^{2}(\pi t/T_{i})$, where $A_{i}$ is the amplitude and
$T_{i}$ is the period of the $i^{th}$ utterance. After the word ’five’, the
flow direction is reversed to model inhalation balancing the volume of air
exhaled during speech counting from ’one’ to ’five’,. A similar inhalation is
modeled after the word ’ten’. The velocity of the flow over an area of 6 cm2
over one cycle of counting from 1 to 10 including the two inhalations
presented in Fig. 2. The period and amplitude of each utterance and the two
inhalation phases of the speech flow can be deduced from the figure. The final
parameter necessary to close the issue related to the boundary condition of
droplet ejection is the area of the mouth opening. To the best of the authors’
knowledge, the information of the area of the mouth opening during speech is
not available in the literature. An average mouth opening size of 4 cm2 was
reported by Gupta et al. for the situation of cough. Assuming that the mouth
opening during speech is on average larger than the opening during cough, we
choose a circular surface that is 6 cm2 in the area to model the mouth opening
during speech. The droplets are injected at the circular mouth model into the
domain at time instants that match the peak of the velocity of each utterance.

Figure 2: (a) the distribution of droplet diameter used to model the droplet
ejection during speech. (b) The velocity profile of speaking flow over one
cycle of speech which involves counting from 1 to 10 with two inhalation
phases that balance of the volume of air exhaled.

###  3.5 Simulation setup

The simulation setup involves a human model standing in an upright position
similar to the infected subject shown in Fig.3. The numerical mesh employed in
this work is also shown in the figure. A mesh spacing of $4$ mm is allocated
to the region immediately in front of the mouth, which is the source of
speaking flow and droplets, approximately up to a distance of 1 m. The
numerical dissipation of the Roe-scheme[38] used for the convective fluxes in
our solver enable us to carryout implicit large eddy simulations (ILES)[39].
The human model is modeled with the immersed boundary condition[40] to impose
no-slip and isothermal boundary conditions. The temperature of the human body
surface is set to 300K to include the effects of buoyancy-driven flow by the
human model, although the effect is not expected to be significant. The outer
boundaries of the computational domain are treated with the slip boundary
condition. The initial conditions for the simulation were set to the STP
conditions. The temperature, pressure and relative humidity were set to 297 K,
101.3×103 Pa and 50$\%$, respectively. The circular mouth geometry for
modeling the speaking flow is placed 1 cm in front of the mouth of the human
model. The flow generated by the speaking model is imposed on the circular
mouth geometry. The relative humidity of the flow emanating from the mouth
geometry is set to 90 $\%$. The droplets generated from speaking are injected
into the computational domain at random locations on the circular mouth
geometry. The initial velocity of the droplets is set to 0, the droplets are
to be driven by the flow from the time of injection into the domain. The
initial temperature of the droplets is chosen to be 308 K matching the
temperature of the interior of the human mouth and the density of the droplet
is set to 1000 kg/s

Figure 3: Blocks of the numerical mesh employed in the present study.
Subdivision of the blocks into $16$ equal parts along each direction produces
the cells. Figure 4: Comparison of droplet diameter as it evaporates with
experimental data of Ranz and Marshall[41, 42].

Figure 5: Dispersion of droplets during continuous speech at (a) $t=12$ s (b)
$t=24$ s

##  4 Results

###  4.1 Validation

A numerical simulation of the evaporation of a single isolated droplet was
carried out to validate the droplet model. The experiment of Ranz and
Marshall[41, 42], in which the evaporation dynamics of a motionless droplet
was carried, is used to validate our numerical simulation. The numerical setup
mimicked the experimental setup wherein a motionless droplet of initial
diameter $d_{d}=1050\;\mu$m is placed in an environment where the relative
humidity and the temperature of the surrounding air were $RH=0\%$ and $T=298$
K, respectively. The initial temperature of the droplet was $T_{d}=282$ K.
After the exposure of the droplet to the surrounding environment, the
evolution of the droplet’s diameter as it evaporates is tracked and compared
the experimental data of Ranz and Marshall. The comparison is plotted in Fig.
4 where it can be seen that there is excellent agreement between the
simulation results and the experimental data.

Figure 6: (a)The probability of infection due to exposure for 15 mins plotted
against distance during face-to-face conversation. (b) Evolution of $P$ over
time at distance of 0.5, 1.0, 2.0 m from the infection source.

###  4.2 Infection risk during speech

The infection risk model presented in this work is adopted to investigate the
probability of infection of a susceptible subject who is in a face-to-face
conversation with an infected individual. For this we carried out the
numerical simulation of dispersion of droplets ejected by an infected subject
in a standing pose assumed to be continuously speaking for the duration of the
simulation. In the simulation, we model only the geometry of the infected
subject, however, the geometry of the susceptible subject is not modeled. The
risk of infection of a virtual subject, whose dimensions are assumed to be
identical to that of the infected subject, is evaluated at varying distances
from the infected subject and also a function of time. The numerical
simulation is carried out in a computational domain measuring $32\times
32\times 16$ m3 along x, y and z-axis, respectively. The human geometry is
placed with the base of its feet at the centre of the computational domain
along x and y-axis and at the bottom along the z-axis. The details of the
boundary and initial conditions of the setup can be found in the Methods
section. The details of the speaking model employed in this work are provided
in the section Droplet modeling parameters. One cycle of the speaking model
involves counting from 1 to 10 with two inhalation phases after the words ’5’
and ’10’, respectively, for mass balance (see Fig. 2). The speaking is modeled
for the duration of the simulation by indefinitely looping the cycle of the
speaking model.

A visualization of instantaneous states of the dispersion of droplets at two
time instants is presented in Fig. 5. The size of the droplets is indicated by
the coloring scheme of the droplets. The largest droplets are colored red and
the smallest blue. It is evident that many of the droplets larger than 20
$\mu$m quickly settle on the ground under the influence of gravity. As the
initial condition for droplet velocity is 0, gravity contribution dominates
the velocity of the larger droplets. Therefore, the horizontal distance
traversed by larger droplets is not significant when compared to smaller
droplets. The influence of gravity on droplets smaller than 10$\mu$ m is
negligible because of aerosolization due to very short evaporation timescales
and consequently the flow-induced drag forces dominate the small droplet and
aerosol velocity. As a consequence, the smaller droplets and a small number of
medium-sized droplets remain airborne and they are carried by flow generated
by the speech. The dispersion of the aerosolized droplets in the horizontal
direction over two instants of time is shown in Fig. 5.

We next move on the investigation of infection probability of a virtual
subject placed at different distances in front of the infected subject. The
probability of infection at different distances in front of the infected
subject for an exposure duration of $T=15$ min is plotted in Fig. 6a. As for
the infectious dose $N_{0}$, we have chosen a value of 900 which lies with the
ranges values reported in the literature[15, 10, 9]. The variation of
infection probability over distance exhibits a decaying profile. At distances
less than 0.5m the probability of infection is greater than $70\%$, which
rapidly decays to less than $20\%$ as the distance is increased to 2 m. It can
be noted from Fig. 5 that the droplet concentration rapidly decays due to its
dispersion in the vertical (z-axis) and in-plane direction (x-axis) as the
droplets are advected away from the infection source thereby lowering the
infection probability with distance. The shaded region in the figure depicts
the change in the infection risk if the infectious dose is changed from 300 to
2000. The infection risk at a given distance is lower for larger values of
$N_{0}$ and vice-versa. It is interesting to note that the shaded region
narrows as the distance from the infected person increases from 0.25 to 2 m.
As the distance from the infection source increases, due to the dispersion of
droplets, the virion concentration in the inhalation zone decreases which in
turn decreases the number of inhaled virions. When the inhaled virion count is
small enough, the magnitude of the infectious dose $N_{0}$ becomes less
important resulting in the narrowing of the shaded region.

The variation of infection probability overtime at distances $D=(0.5,1.0,2.0)$
m is plotted in Fig. 6b. For large virion concentrations at closer distances
like $D=0.5$m, the $P$ rapidly increases and saturates to the maximum value
(1). On the other hand, at a further distance, due to lower virion
concentration, the rate of change of $P$ is more gradual. At any given instant
of time, say 10, 20, 30 min, etc., the plot provides the relative risk of
maintaining different distances from a likely infected person. Focusing on the
horizontal grid line corresponding to $P=0.2$., it can be seen that the
exposure time required for 20$\%$ probability of infection is approximately 3,
10 and 21 min for distances of 0.5, 1 and 2 m, respectively.

Figure 7: (a) comparison of infection probability over time of different
strains of SARS-CoV2. (b) Comparison of infection probability of vaccinated
and unvaccinated cases.

###  4.3 Effect of variants and vaccines

The probability of infection can change significantly due to variant strains
of SARS-CoV-2 and vaccination. To account for such factors a generalized form
of the equation for infection probability was introduced through Eq. 7. By
altering the parameter $\alpha$, the equation can be adapted for variant
strains of higher transmissibility factor $\tau$, and vaccinated individuals.
The transmissibility of B.1.1.7 strain (alpha variant) and the B.1.617.2
strain (delta variant) is $29\%$ and $145\%$ higher than the original strain,
i.e. $\tau=1.29$ and $\tau=2.45$ for the alpha and the delta variant,
respectively. Evidently, the value $\tau=1$ corresponds to the original
strain. A comparison of how the $P$, evaluated at a distance of 1 m, increases
with time for the original strain, the alpha and the delta variants is plotted
in Fig. 7a. For the duration of exposure of less than 20 min, we find that
there is a significant difference between the risk of infection of the
original strain and the delta strain. This difference gradually reduces over
time as P saturates to its maximum value. The probability of infection for the
delta variant is approximately 2 times greater than the original strain.
However, the difference between the infection probability of the alpha and the
original strain is not very significant.

Through the risk evaluation model presented in this work, it is also possible
to incorporate the effect of vaccination on the probability of infection. As
discussed in the previous section, the generalized form of Eq. 7 can be
applied to vaccinated cases by using the expression $\alpha=1-\eta_{vc}$. For
unvaccinated cases, we can set $\eta_{vc}=0$. To evaluate how the infection
risk changes due to vaccination, we choose two values for $\eta_{vc}$, $80\%$
and $95\%$, which approximately correspond to the efficacies of the
AstraZeneca and Pfizer vaccines, respectively. The comparison of $P$ evaluated
at a distance of 1m from the infected person for the different values of
$\eta_{vc}$ is presented in Fig. 7b. The infection probability remains below
$20\%$ for exposure periods of up to 40 mins for $\eta_{vc}=0.85$ and it
remains well below $10\%$ for exposure period plotted in the figure.

Figure 8: A snapshot of droplet dispersion at $t=24$ s for (a) $RH=50\%$ (b)
$RH=100\%$.

###  4.4 Role of humidity

Direct simulation of droplet dispersion, as opposed to simulations of scalar
transport as a proxy for aerosols, enables the investigation of the effect of
environmental factors like temperature and humidity on droplet aerosolization
and dispersion, and consequently on the risk of infection. We extend the
numerical simulation of droplet dispersion during speech presented in the
previous section to study the effect of humidity on infection risk. The
relative humidity of the ambient environment could significantly affect the
evaporation of medium and large droplets. This leads to the prevention or
aerosolization of medium droplets that have a higher virion count compared to
that of smaller droplets. As a consequence, the concentration of virions could
be significantly altered by humidity, thereby influencing the probability of
infection. In order to investigate role of humidity on infection risk, we
carried out a numerical simulation of droplet dispersion during speech under
three different humidity conditions $10\%,50\%$ and $100\%$. The numerical
setup and the boundary conditions are identical to those of the simulation in
the previous section. The only parameter varied is the relative humidity
($RH$). Three separate simulations were carried out in which the relative
humidity was set to $10\%,50\%$ and $100\%$.

In Fig. 8, a snapshot of the droplet dispersion at $t=24$ s for $RH=50\%$ and
$RH=100\%$ is presented. For the $RH=100\%$ case, lack of evaporation prevents
the aerosolization of medium-sized droplets. Droplets smaller than $5\;\mu$m
remain airborne for prolonged periods. The velocity of some of the droplets
between the sizes of 5 and $50\;\mu$, whose timing in ejection matches the
peak velocity of the speaking model, is initially dominated by the fluid
velocity. However, as the flow velocity decreases away from the mouth due to
dissipation, the influence of gravity dominates the velocity of these
droplets. This results in the droplets settling to the ground under the
influence of gravity. In contrast, at lower $RH$ values, the medium droplets
in question can get partially or completely aerosolized and remain airborne.
As a result, the concentration of aerosols and consequently the virions
depends directly on the relative humidity of the ambient environment. This
effect is quantified through the evaluation of the infection probability which
directly depends on the local virion concentration. The variation of infection
probability over distance from the infection source of the different humidity
cases are compared in Fig. 9a. As the local droplet concentration is strongly
influenced by the evaporation of medium droplets with higher virion count, the
number of virions likely to be inhaled decreases with increasing humidity. The
consequence of which can be seen in Fig. 9a and Fig. 9b. The trend of $P$
decreasing with distance is consistent across all the humidity cases. However,
for a given distance from the infection source, the probability of infection
is lower for higher humidity. Fig. 9b plots the evolution of $P$ over time
evaluated at a distance of 1 m from the infection source for the different
humidity cases. From these plots, it could be inferred that for a given
temperature higher humidity lowers the risk of infection. However, the
magnitude of reduction in the probability of infection could be strongly
influenced by the temperature of the ambient environment.

Figure 9: (a) Variation of infection probability over (a) distance from the
infection source, and (b) time of exposure under different humidity conditions

##  5 Summary and Discussion

In this work, we have developed a framework for quantifying the risk of
infection due to airborne diseases like COVID-19 using the dose-response model
in droplet-dispersion simulations. We have detailed in this work a methodology
for estimating the inhalation dose, by measuring the volume of
droplets/aerosols (proxies for virion count) in the breathing zone, directly
from droplet dispersion simulations. We have adopted this framework to
estimate the risk of infection from a person ejecting droplets while speaking.
For this, we have carried out a numerical simulation of droplet dispersion
from a single person standing and speaking continuously in an isolated
environment. The infection probability was found to decrease with distance
from the infected person. The magnitude of infection probability is strongly
influenced by the minimum infection dose $N_{0}$ which can be seen in Fig 6a.
If the minimum infection dose is assumed to be small then the infection risk
remains relatively large even at distances as far as 2m from the infected
person. On the other hand, if the infection dose is large then the infection
probability is small even at a distance of 1 m. Furthermore, as the
simulations have been carried out in a quiescent or poorly ventilated
environment, the infection probability is likely to be high. This prediction
can drastically change depending on the ambient environment’s flow conditions.
Therefore, we would not like to make any specific recommendations on social
distancing based on these results. The main purpose of this work is to
demonstrate the applicability of the framework of risk estimation using the
dose-response model in droplet dispersion simulations.

A generalized form of the dose-response model that can incorporate the effect
of increased transmissibility of various strains of COVID-19 and the effect of
vaccination has been presented in this work. A comparison of the infection
risk due to variant strains of higher transmissibility was with the standard
strain was presented. The results show that for exposure duration the
infection risk of variant strain can be significantly higher than the standard
strain. The difference in infection between strains decreases as exposure
duration increases. Similarly, we also presented a comparison of the infection
risk for a vaccinated person with that for an unvaccinated person. The
probability of infection for vaccinated persons is very low for short and
medium exposure duration. However, the infection risk can increase
significantly provided the exposure duration is very long.

One of the main advantages of droplet dispersion simulations is their ability
to investigate the effect of environmental factors, keeping all other
variables fixed, such as temperature and humidity on the infection risk. To
demonstrate this point, an investigation of the effect of humidity of the
ambient environment on the infection risk was carried out. This is an aspect
that cannot be analyzed using well-mixed room average analysis or passive
scale based aerosol transport models. The results of our analysis show that
the infection risk is strongly influenced by humidity due to its effect on the
evaporation of medium and small droplets. The infection risk at a given
distance from the infected person has an inverse dependence on humidity.
Lowering the ambient humidity increases the risk and vice-versa. This
relationship between infection risk and humidity depends, to some extent, on
the droplet diameter distribution adopted in the simulation. For example, a
hypothetical droplet diameter profile that includes only aerosols may not
result in a strong dependence of infection risk on humidity as the results of
this study indicate.

## Acknowledgements

This work was supported by JST CREST, Grant Number JPMJCR20H7, Japan, and
through the HPCI System Research Project (Project ID: hp210086)

## Author contributions statement

RB, AI, MT developed the dose-response model. RB, MT, MY developed the
speaking model. RB, CGL, MT developed flow solver. RB carried out the
simulations. RB, MT analyzed the data. RB wrote the manuscript. All authors
reviewed the manuscript and provided inputs to the final edit.

## References

  * [1] J. Wei, Y. Li, Airborne spread of infectious agents in the indoor environment, American journal of infection control 44 (9) (2016) S102–S108.
  * [2] J. Gralton, E. Tovey, M.-L. McLaws, W. D. Rawlinson, The role of particle size in aerosolised pathogen transmission: a review, Journal of Infection 62 (1) (2011) 1–13.
  * [3] L. Morawska, J. Cao, Airborne transmission of sars-cov-2: The world should face the reality, Environment international 139 (2020) 105730.
  * [4] S. Asadi, N. Bouvier, A. S. Wexler, W. D. Ristenpart, The coronavirus pandemic and aerosols: Does covid-19 transmit via expiratory particles?, Aerosol Science and Technology (2020) 635–638.
  * [5] L. Ferretti, C. Wymant, M. Kendall, L. Zhao, A. Nurtay, L. Abeler-Dörner, M. Parker, D. Bonsall, C. Fraser, Quantifying sars-cov-2 transmission suggests epidemic control with digital contact tracing, Science 368 (6491) (2020).
  * [6] S. Rudnick, D. Milton, Risk of indoor airborne infection transmission estimated from carbon dioxide concentration., Indoor air 13 (3) (2003) 237–245.
  * [7] C. M. Issarow, N. Mulder, R. Wood, Modelling the risk of airborne infectious disease using exhaled air, Journal of theoretical biology 372 (2015) 100–106.
  * [8] S. Smith, A. Somsen, C. van Rijn, S. Kooij, L. van der Hoek, R. Bem, D. Bonn, Probability of aerosol transmission of sars-cov-2 (2020).  doi:10.1101/2020.07.16.20155572.
URL https://europepmc.org/article/PPR/PPR189143

  * [9] B. L. Augenbraun, Z. D. Lasner, D. Mitra, S. Prabhu, S. Raval, H. Sawaoka, J. M. Doyle, Assessment and mitigation of aerosol airborne sars-cov-2 transmission in laboratory and office environments, Journal of Occupational and Environmental Hygiene 17 (10) (2020) 447–456.
  * [10] J. M. Kolinski, T. M. Schneider, Superspreading events suggest aerosol transmission of sars-cov-2 by accumulation in enclosed spaces, Physical Review E 103 (3) (2021) 033109.
  * [11] R. Singhal, S. Ravichandran, R. Govindarajan, S. S. Diwan, Virus transmission by aerosol transport during short conversations, arXiv preprint arXiv:2103.16415 (2021).
  * [12] W. F. Wells, et al., Airborne contagion and air hygiene. an ecological study of droplet infections., Airborne Contagion and Air Hygiene. An Ecological Study of Droplet Infections. (1955).
  * [13] T. Watanabe, T. A. Bartrand, M. H. Weir, T. Omura, C. N. Haas, Development of a dose-response model for sars coronavirus, Risk Analysis: An International Journal 30 (7) (2010) 1129–1138.
  * [14] T. Watanabe, T. A. Bartrand, T. Omura, C. N. Haas, Dose-response assessment for influenza a virus based on data sets of infection with its live attenuated reassortants, Risk Analysis: An International Journal 32 (3) (2012) 555–565.
  * [15] M. G. Prentiss, A. Chu, K. K. Berggren, Superspreading events without superspreaders: Using high attack rate events to estimate n o for airborne transmission of covid-19, MedRxiv (2020).
  * [16] B. Binazzi, B. Lanini, R. Bianchi, I. Romagnoli, M. Nerini, F. Gigliotti, R. Duranti, J. Milic-Emili, G. Scano, Breathing pattern and kinematics in normal subjects during speech, singing and loud whispering, Acta physiologica 186 (3) (2006) 233–246.
  * [17] Exposure factors handbook, Vol. 90, Office of Health and Environmental Assessment, US Environmental Protection Agency, 1989.
  * [18] D. Shimer, P. Jenkins, S. Hui, W. Adams, 132 measurement of breathing rate and volume in routinely performed daily activities, Epidemiology 6 (2) (1995) S30.
  * [19] F. Yang, A. A. Pahlavan, S. Mendez, M. Abkarian, H. A. Stone, Towards improved social distancing guidelines: Space and time dependence of virus transmission from speech-driven aerosol transport between two individuals, Physical Review Fluids 5 (12) (2020) 122501.
  * [20] R. Wölfel, V. M. Corman, W. Guggemos, M. Seilmaier, S. Zange, M. A. Müller, D. Niemeyer, T. C. Jones, P. Vollmar, C. Rothe, et al., Virological assessment of hospitalized patients with covid-2019, Nature 581 (7809) (2020) 465–469.
  * [21] F. P. Polack, S. J. Thomas, N. Kitchin, J. Absalon, A. Gurtman, S. Lockhart, J. L. Perez, G. P. Marc, E. D. Moreira, C. Zerbini, et al., Safety and efficacy of the bnt162b2 mrna covid-19 vaccine, New England Journal of Medicine (2020).
  * [22] L. R. Baden, H. M. El Sahly, B. Essink, K. Kotloff, S. Frey, R. Novak, D. Diemert, S. A. Spector, N. Rouphael, C. B. Creech, et al., Efficacy and safety of the mrna-1273 sars-cov-2 vaccine, New England Journal of Medicine 384 (5) (2021) 403–416.
  * [23] K. R. Emary, T. Golubchik, P. K. Aley, C. V. Ariani, B. Angus, S. Bibi, B. Blane, D. Bonsall, P. Cicconi, S. Charlton, et al., Efficacy of chadox1 ncov-19 (azd1222) vaccine against sars-cov-2 variant of concern 202012/01 (b. 1.1. 7): an exploratory analysis of a randomised controlled trial, The Lancet 397 (10282) (2021) 1351–1362.
  * [24] Sars-cov-2 variant classifications and definition, Center for Disease Control and Prevention (2021).
URL https://www.cdc.gov/coronavirus/2019-ncov/variants/variant-info.html

  * [25] F. Campbell, B. Archer, H. Laurenson-Schafer, Y. Jinnai, F. Konings, N. Batra, B. Pavlin, K. Vandemaele, M. D. Van Kerkhove, T. Jombart, et al., Increased transmissibility and global spread of sars-cov-2 variants of concern as at june 2021, Eurosurveillance 26 (24) (2021) 2100509.
  * [26] N. G. Davies, S. Abbott, R. C. Barnard, C. I. Jarvis, A. J. Kucharski, J. D. Munday, C. A. Pearson, T. W. Russell, D. C. Tully, A. D. Washburne, et al., Estimated transmissibility and impact of sars-cov-2 lineage b. 1.1. 7 in england, Science 372 (6538) (2021).
  * [27] T. Poinsot, D. Veynante, Theoretical and numerical combustion, RT Edwards, Inc., 2005.
  * [28] R. Bale, W. H. Wang, C.-G. Li, K. Onishi, K. Uchida, H. Fujimoto, R. Kurose, M. Tsubokura, A scalable framework for numerical simulation of combustion in internal combustion engines, in: Proceedings of the Platform for Advanced Scientific Computing Conference, 2020, pp. 1–10.
  * [29] R. Bale, C.-G. Li, M. Yamakawa, A. Iida, R. Kurose, M. Tsubokura, Simulation of droplet dispersion in covid-19 type pandemics on fugaku, in: Proceedings of the Platform for Advanced Scientific Computing Conference, 2021, pp. 1–11.
  * [30] N. Jansson, R. Bale, K. Onishi, M. Tsubokura, Cube: A scalable framework for large-scale industrial simulations, The International Journal of High Performance Computing Applications 1094342018816377.
  * [31] K. Nishiguchi, R. Bale, S. Okazawa, M. Tsubokura, Full eulerian deformable solid-fluid interaction scheme based on building-cube method for large-scale parallel computing, International Journal for Numerical Methods in Engineering 117 (2) (2019) 221–248.
  * [32] K. Nakahashi, Building-Cube Method for Flow Problems with Broadband Characteristic Length, in: Computational Fluid Dynamics 2002, Springer Berlin Heidelberg, 2003, pp. 77–81.
  * [33] R. G. Loudon, R. M. Roberts, Droplet expulsion from the respiratory tract, American Review of Respiratory Disease 95 (3) (1967) 435–442.
  * [34] C. Y. H. Chao, M. P. Wan, L. Morawska, G. R. Johnson, Z. Ristovski, M. Hargreaves, K. Mengersen, S. Corbett, Y. Li, X. Xie, et al., Characterization of expiration air jets and droplet size distributions immediately at the mouth opening, Journal of Aerosol Science 40 (2) (2009) 122–133.
  * [35] S. Asadi, A. S. Wexler, C. D. Cappa, S. Barreda, N. M. Bouvier, W. D. Ristenpart, Aerosol emission and superemission during human speech increase with voice loudness, Scientific reports 9 (1) (2019) 1–10.
  * [36] X. Xie, Y. Li, H. Sun, L. Liu, Exhaled droplets due to talking and coughing, Journal of the Royal Society Interface 6 (suppl_6) (2009) S703–S714.
  * [37] J. Duguid, The size and the duration of air-carriage of respiratory droplets and droplet-nuclei, Epidemiology & Infection 44 (6) (1946) 471–479.
  * [38] P. L. Roe, Approximate riemann solvers, parameter vectors, and difference schemes, Journal of computational physics 43 (2) (1981) 357–372.
  * [39] F. F. Grinstein, L. G. Margolin, W. J. Rider, Implicit large eddy simulation, Vol. 10, Cambridge university press Cambridge, 2007.
  * [40] C.-G. Li, M. Tsubokura, R. Bale, Framework for simulation of natural convection in practical applications, International Communications in Heat and Mass Transfer 75 (2016) 52–58.
  * [41] W. Ranz, W. R. Marshall, Evaporation from drops i, Chem. eng. prog 48 (3) (1952) 141–146.
  * [42] W. Ranz, W. R. Marshall, Evaporation from drops ii, Chem. eng. prog 48 (3) (1952) 173–180.

## Appendix

### Infection probability post vaccination

The probability of infection can be written as

$P=1-e^{(-\frac{N(t)}{N_{0}})}=1-\exp^{(-\frac{N_{c}}{N_{0}}t)}$ (15)

where $N_{c}$ is the virion dose per unit time under steady state conditions
of exposure such as speaking, singing etc. The exact form of $N_{c}$ is not
pertinent to the present derivation, it can deduced from Eq. 6. The
probability of no infection can be defined as $\hat{P}=1-P$. The net
probability of no infection is given by

$\displaystyle\hat{P}_{net}$
$\displaystyle=\int_{0}^{\infty}\exp^{(-\frac{N_{c}}{N_{0}}t)}dt$
$\displaystyle\hat{P}_{net}$ $\displaystyle=\frac{N_{0}}{N_{c}}$ (16)

Assuming that the vaccination leads to reduction in infection probability due
to increase in $N_{0}$, the net probability of no infection for a vaccinated
individual can be expressed as

$\hat{P}^{vc}_{net}=\frac{N_{0}^{vc}}{N_{c}}$ (17)

Let $\eta_{vc}$ be the efficacy of a given vaccine. The probability of no
infection with vaccination decreases by a factor of
$\hat{P}^{vc}_{net}\eta_{vc}$ if the vaccination is not administered.
Therefore, we can express a relationship between $\hat{P}_{net}$ and
$\hat{P}^{vc}_{net}$ as follows

$\hat{P}_{net}=\hat{P}^{vc}_{net}-\hat{P}^{vc}_{net}\eta_{vc}.$ (18)

Rearranging the terms we get

$\hat{P}^{vc}_{net}=\frac{\hat{P}_{net}}{1-\eta_{vc}}.$ (19)

Assuming an exposure to the same average virion dose per unit time, we can
obtain an expression for $N_{0}^{vc}$ from the above two equation as

$N_{0}^{vc}=\frac{N_{0}}{1-\eta_{vc}}$ (20)

The interpretation of the above two equations is straightforward. If the
vaccine efficacy the $100\%$ then the minimum number of virions needed for
infection will be $\infty$. Consequently, $\hat{P}=1$ at all time instants $t$
and $P=0$ and the net probability of no infection will be $\infty$. On the
other hand, if the vaccine efficacy is 0 then the ${N}^{vc}_{0}={N}_{0}$.
Therefore, probability of infection remains unchanged.

The general form of the probability of infection under vaccinated and
unvaccinated situations can be expressed as

$P=1-e^{(-\alpha\frac{N}{N_{0}})},$ (21)

where $\alpha=1$ for no vaccination cases and $\alpha=1-\eta_{vc}$ for
vaccinated cases.

The net probability of safety decreases for the variants strains due to higher
transmissibility $\tau$ compared to the original strain. With
$\hat{P}^{vr}_{net}$ representing the probability of no infection, its
relationship with $\hat{P}_{net}$ can be written as

$\hat{P}^{vr}_{net}=\frac{\hat{P}_{net}}{\tau}$ (22)

Along the same lines of derivation of $\alpha$ for the vaccination case, it
can be shown that $\alpha=\tau$.