# Addressing Source Scale Bias via Image Warping for Domain Adaptation

Shen Zheng Anurag Ghosh11footnotemark: 1 Srinivasa G. Narasimhan
Robotics Institute, Carnegie Mellon University
{shenzhen, anuraggh<EMAIL_ADDRESS>
http://instancewarp.github.io Equal contribution

###### Abstract

In visual recognition, scale bias is a key challenge due to the imbalance of
object and image size distribution inherent in real scene datasets.
Conventional solutions involve injecting scale invariance priors, oversampling
the dataset at different scales during training, or adjusting scale at
inference. While these strategies mitigate scale bias to some extent, their
ability to adapt across diverse datasets is limited. Besides, they increase
computational load during training and latency during inference. In this work,
we use adaptive attentional processing – oversampling salient object regions
by warping images in-place during training. Discovering that shifting the
source scale distribution improves backbone features, we developed a instance-
level warping guidance aimed at object region sampling to mitigate source
scale bias in domain adaptation. Our approach improves adaptation across
geographies, lighting and weather conditions, is agnostic to the task, domain
adaptation algorithm, saliency guidance, and underlying model architecture.
Highlights include +6.1 mAP50 for BDD100K Clear $\rightarrow$ DENSE Foggy,
+3.7 mAP50 for BDD100K Day $\rightarrow$ Night, +3.0 mAP50 for BDD100K Clear
$\rightarrow$ Rainy, and +6.3 mIoU for Cityscapes $\rightarrow$ ACDC. Our
approach adds minimal memory during training and has no additional latency at
inference time. Please see Appendix for more results and analysis.

Figure 1: Our method improves domain adaptive object detection across a
extensive range of lighting and weather scenarios in various target datasets
(or subsets) using supervision from the BDD100K source subset containing day
images in clear weather.

##  1 Introduction

Figure 2:  Image Warping to Address Source Domain Scale Bias. Bounding boxes
mark small, medium, and large objects in the image. Driving scenes contain
many objects that appear small in the image, some objects that have medium
image sizes, and few objects that appear large. Our approach shifts this
unbalanced scale distribution in datasets by oversampling medium and large-
sized objects to address the scale bias.

Visual detection and segmentation of things (i.e. object instances) and stuff,
across different scales is a fundamental challenge in computer vision [49, 35,
19, 45]. Natural scene datasets used for training different tasks often
exhibit bias of imaged objects towards certain scales (or sizes). For
instance, in autonomous driving situations, camera perspective and field of
view warrant that many more distant objects that appear small in the image are
captured compared to nearby objects that appear large, even if the real object
3D scale distribution is near-uniform. This results in an inherent imbalance
or bias in the distribution of sizes of object projections in the image,
leading to sub-optimal recognition models.

Common approaches to address scale bias are: (a) selecting an image subset to
create a uniform scale distribution but this is suboptimal as only a fraction
of the available data is used; (b) oversampling via data augmentation (e.g.,
multi-scale training [35] and multi-scale inference [10, 34, 13]), while
helpful, also introduces significant computational and storage overhead; (c)
introducing scale-invariance priors (e.g., feature pyramids [21, 23, 22]), but
this only partially alleviates scale bias, as both things and stuff vary
widely in scale and image locations.

We propose a new method to address scale bias, orthogonal to previous
techniques, using adaptive attentional processing. We warp images _in-place_
while training models to shift the unbalanced scale distribution of objects
(see Figure 2). While non-uniform warping techniques, i.e. allocating more
pixels using a predefined guidance, have been applied to improve image
classification [25], detection [36, 9, 37, 7] and segmentation [37] in
supervised settings, they do not principally examine and address the scale
bias for domain adaptation.

We propose Instance-level Saliency Guidance which analytically shifts the
object scale distribution to remove bias or match target distribution by
leveraging ground truth labels for maximum performance gain. Because the
warping is in-place, we directly re-sample objects (e.g. expand smaller ones)
and squeeze the large empty regions in the scene (e.g. road), allowing for a
better use of the image pixel budget for balanced distribution training. We
show that this warping improves backbone features of our model crucial for
domain adaptation. To make sure our outputs are in the original unwarped
configuration, we unwarp the features, as in LZU [37], and train without
warping the object labels. During inference, no warping is performed and the
method requires no additional computational latency. Our contributions are
highlighted as follows:

  * •

We are the first work to show that scale bias in real-to-real domain
adaptation can be effectively addressed through image warping, and that
shifting source scale distribution significantly improves backbone features.

  * •

Our approach is agnostic to task (detection or segmentation), adaptation
strategy, model architecture and warping guidance used for sampling.

  * •

Our approach improves domain adaptation across various lighting (+3.7 mAP50
for BDD100K Day $\rightarrow$ Night detection), weather conditions (+6.1 mAP50
for BDD100K Clear $\rightarrow$ DENSE Foggy detection), and geographies (+6.3
mIoU for Cityscapes $\rightarrow$ ACDC semantic segmentation). See Figure 1.

  * •

Our approach is efficient: we introduce train-time image warping and feature
unwarping for domain adaptation without need for test-time warping. This
results in minimal training overhead and no additional inference latency.

##  2 Related Work

Figure 3: Warping for Domain Adaptation. Consider the Unsupervised Domain
Adaptation setting (like [12, 17]), which adapt from source domain to target
domain in 2 stages – a supervised pre-training on the source domain and an
unsupervised adaptation stage from source to target. Components of our method
are marked in cyan. For a source domain image, we generate a saliency guidance
based on the dataset’s scale distribution and source bounding boxes (Section
3.2) to decrease scale bias and unwarp the features before computing the
supervised loss in the unwarped label space. Guidance could instead be
computed via Static Prior [36, 37] or Geometric Prior [9], but our Instance-
Level Guidance performs better and with no test-time latency (Sections 5.1 and
5.2). Unsupervised losses computed on the target image of the Domain
Adaptation Algorithm (say from [12, 17, 13, 14, 18, 6, 48]) remains the same.

Alleviating Scale Bias. Recognition across different scales is a historically
important problem. Priors that encourage scale invariance like feature (or
anchor) pyramids are widely used in pre-deep-learning approaches [23, 5] and
recently [27, 21, 22]. While helpful, these priors do not solve scale bias
completely, as an object can be bigger or smaller than pre-defined scales.
Common augmentation techniques such as multi-scale training [27, 35, 17, 13]
can considerably increase training time. Similarly, test time scaling [10, 34]
and multi-scale or multi-crop predictions [26, 13] improve accuracy at the
cost of latency and memory [15]. Another line of works select “optimal” scale
at inference [3, 8], but don’t improve the learned model itself. Compared to
earlier work, our method explicitly addresses scale bias requiring (a) little
additional memory during training since we warp images in-place; and (b) no
additional inference latency as we do not warp during testing. Crucially, our
approach is orthogonal to other scale bias reduction techniques, and can be
used in a plug-and-play manner.

Guided Image Warping. Non-uniform Spatial Transformations [42] have been used
to correct image distortions arising from camera acquisition. Recently,
spatial transformation methods [25, 16] for learned models have advocated
introducing image distortions instead. Learning to Zoom [25] showed over-
sampling task-specific salient regions is useful as visual recognition
requires combining information from different spatial resolutions. Building on
this paradigm, warping for other tasks like detection [36, 37] and
segmentation [37] were explored. Fovea [36], and Two Plane Prior [9] designed
“salient” priors to guide warping, introducing static & temporal, and
geometric saliency guidance respectively.

Our approach stands outs in the following ways: (a) Unlike previous image
warping efforts focusing on supervised settings, our innovative training
strategy and saliency guidance tailor image warping for unsupervised domain
adaptation to directly address scale bias. (b) Unlike former image warping
methods that indiscriminately process objects and background elements, we
introduces an analytical instance-level saliency guidance that targets regions
containing objects.

Image Transformations for Domain Adaptation. Image Transformations like
flipping, cropping, blurring, color jittering, grayscaling, contrast
adjustment are widely used as perturbations for domain adaptation [39, 45, 46,
24, 31, 48, 38, 6, 41, 43, 18, 12, 13, 11, 14, 17] to increase source domain
diversity and decrease domain gap to the target domain. For example, 2PCNet
[17] introduces a domain-specific Night Augmentation (NightAug) pipeline that
attempts to make day images appear closer to night images. To the best of our
knowledge, in-place image warping as a domain adaptive perturbation has not
been explored in prior works. Our approach is agnostic to the specific domain
adaptation training algorithm, and can be easily incorporated with any other
augmentation strategy.

Figure 4:  Image Warping with Different Priors. In-place warping follows a
zero-sum pixel constraint: enlarging one region necessitates shrinking
another. Bounding boxes mark small, medium, and large objects in the image.
Our method specifically targets object instances. In comparison, the Static
Prior fails when instance distribution deviates from the dataset average,
whereas the Geometric Prior fails for small objects unaligned with the
dominant vanishing point.

##  3 Method

We first describe how we warp images assuming a spatial guidance function
while remaining task agnostic. Then, we design our warping guidance that can
explicitly address scale bias (as shown in Figure 1). Finally, we detail how
to train models with warped images that adapt better to new domains. The
entire strategy is illustrated in Figure 3.

###  3.1 Image Warping and Feature Unwarping

We employ the saliency guided sampling mechanism from [25]. However, other
methods may also be used (like thin plate spline transformations [16, 7]).

Image Warping: We consider an inverse transformation $\mathcal{T}^{-1}_{S}$
parameterized by saliency $S$ such that,

$I^{\prime}(\mathbf{u})=W_{\mathcal{T}}(I)=I(\mathcal{T}_{S}^{-1}(\mathbf{u}))$
(1)

a resampled $I^{\prime}$ for every location $\mathbf{u}$ is obtained by using
warp $W_{\mathcal{T}}$ given input image $I$ and saliency guidance $S$, which
is a 2D map of the output image size. The rationale and choice of $S$ is
described in Section 3.2.

Feature Unwarping: If we operate in the space of warped images, our outputs
would also be warped. However, we do not wish to warp our labels to match the
input warped image, hence, we perform feature unwarping of spatial backbone
features and train the model without warping our labels or unwarping our
predictions, like [36, 9]. The benefit of this strategy is that it renders our
warping mechanism task agnostic. We adopt the strategy from LZU [37], which
approximates $\mathcal{T}$ with $\widetilde{\mathcal{T}}$, a piecewise tiling
of invertible bilinear maps, and then perform feature unwarping via,

${\mathcal{T}}^{-1}(\mathbf{u})\approx\widetilde{\mathcal{T}}^{-1}(\mathbf{u})=\begin{cases}\widetilde{\mathcal{T}}_{ij}^{-1}(\mathbf{u})&\text{
if
}\mathbf{u}\in\operatorname{Range}\left(\widetilde{\mathcal{T}}_{ij}\right)\\\
0&\text{ else }\end{cases}$ (2)

We restrict the space of allowable warps for $\mathcal{T}$ such that axis
alignment is preserved. Entire rows or columns are “stretched” or “compressed”
in this case, which has been observed to perform better [36, 9, 37]:

$\mathcal{T}_{S,u}^{-1}(u)=\frac{\int_{u^{\prime}}S_{u}(u^{\prime})k(u^{\prime},u)u^{\prime}}{\int_{u^{\prime}}S_{u}(u^{\prime})k(u,u^{\prime})}$
(3)

$\mathcal{T}_{S,v}^{-1}(v)=\frac{\int_{v^{\prime}}S_{v}(v^{\prime})k(v^{\prime},v)v^{\prime}}{\int_{v^{\prime}}S_{v}(v^{\prime})k(v,v^{\prime})}$
(4)

where $k$ is a Gaussian kernel and $\mathbf{u}=(u,v)$ denote the two image
axes in the pixel space. $S_{u}$ and $S_{v}$ are marginalized versions of $S$
along $u$ and $v$ respectively.

###  3.2 Designing Saliency Guidance

Warping guidance decides which image regions are warped (e.g., scaled) more
than others. Since images are warped in-place, if some regions are expanded,
others must be reduced in size (Figure 4). For the warping mechanism
considered [25, 36, 37], this requires designing a saliency function $S$. We
investigate three saliency functions, based on two approximate guidances
proposed in previous work [36, 37, 9] and one that we propose in this work.

Approximate Saliency Guidance: The Geometric Prior [9] assumes that a dominant
vanishing point exists in the distant scene, and designs a saliency map to
exploit this property and oversample objects that appear small. But this prior
tries to warp the entire scene that is farther away and the objects themselves
may not be oversampled sufficiently to overcome size bias (Figure 4).

The Static Prior [36, 37] assumes that training set statistics will remain
valid at test time, which is often not the case in domain adaptation due to
domain shift and label unavailability. This incorrect assumption leads to the
creation of a saliency map targeting average object locations, which may not
accurately represent specific target images where objects deviate from these
averages (Figure 4). Moreover, warping during the test phase, after
adaptation, increases inference latency and is less effective due to the
unknown scale distribution of the target domain (See Section 5.4 for
ablation). Essentially, these saliency approaches do not utilize actual object
labels and fail to concentrate on oversampling the objects, a strategy that
would significantly benefit the shifting of the object scale distribution.

Instance-level Saliency Guidance. We propose Instance-level Saliency Guidance
which explicitly shifts the object scale distribution by leveraging ground
truth labels (Figure 4). During training, source image $x_{a}$ has
corresponding labels $y_{a}$ available. For simplicity, we assume the labels
are bounding boxes.

To shift the object scale distribution, we oversample image regions
corresponding to every object instance. Following [1, 36], we use kernel
density estimation to model the bounding box saliency as a sum of Gaussians,

$S=\sum_{\left(c_{i},w_{i},h_{i}\right)\in
y_{a}}\mathcal{N}\left(c_{i},s\left[\begin{array}[]{cc}w_{i}&0\\\
0&h_{i}\end{array}\right]\right)$ (5)

where $c_{i}$, $w_{i}$ and $h_{i}$ represent the center, width, and height of
the bounding box, $\mathcal{N}(\mu,\Sigma)$ is the normal distribution and $s$
is the saliency scale or bandwidth.

We propose an expansion factor $f$ to decide the warping extent of all
objects,

$f=2^{\max\left(\left\lfloor\displaystyle\log_{2}\left(\sum_{\psi\in\Psi}\frac{\psi_{i}}{\psi_{i+1}}\right)\right\rfloor,0\right)}$
(6)

where $\Psi$ is the fraction of object size corresponding to an interval
between size thresholds $t_{i}$ and $t_{i+1}$ (See Figure 1). In our case, we
consider $\Psi=\\{\psi_{small},\psi_{medium}$, $\psi_{large}\\}$ to represent
the percentage of the small, medium, and large objects in the dataset, defined
by COCO [20] thresholds for object sizes in the image.

Saliency scale $s$ and expansion factor $f$ are inversely proportional, thus
$s=\frac{P}{f}$ where $P$ is a constant. We clip $S$ between of 0 and $U$,
since negative saliency values lead to holes, and an upper bound $U$ to
discourage extreme warping.

When segmentation masks are available but not bounding boxes, we convert them
to boxes, termed “from-seg” boxes. This conversion minimally impacts
performance, as detailed in our ablations in Section 5.4 and Supplementary.

Figure 5: Domain Adaptive Object Detection (BDD100K day $\rightarrow$ night).
Model trained with our method exhibits improved precision in distinguishing
between road elements, accurately identifying cars and correctly detecting
streetlights. Meanwhile, 2PCNet [17] appears to show a tendency to confuse
these classes.

###  3.3 Warping for Domain Adaptation

We consider unsupervised domain adaptation, wherein source domain
$\mathcal{X}_{a}$ is available with labels, i.e. dataset $X_{a}=\\{\
(x_{a},y_{a})\in\mathcal{X}_{a}\ \\}$ and target domain $\mathcal{X}_{b}$ only
consists of images, $X_{b}=\\{\ x_{b}\in\mathcal{X}_{b}\ \\}$.

Domain Adaption Framework: The standard framework comprises of two stages.
First, a supervised pre-training stage where a model
$\hat{y}_{a}=F_{\theta}(x_{a})$ is trained using supervised loss on source
domain, $\mathcal{L}_{sup}(y_{a},F_{\theta}(x_{a})).$ Second, in the
unsupervised adaptation stage the model is adapted to target domain
$\mathcal{X}_{b}$ using dataset $X_{b}$ through an additional unsupervised
loss. Generally, a student $F_{s,\phi}$ and teacher $F_{t,\kappa}$
(initialized using $F_{\theta}$) are trained jointly through self-training, by
employing psuedo labels [17, 12]. While training the student, a supervised
loss is used to preserve performance on the source domain $\mathcal{X}_{a}$ in
a supervised branch:

$\mathcal{L}_{sup}(y_{a},F_{s,\phi}(x_{a}))+\lambda\mathcal{L}_{unsup}(F_{t,\kappa}(x_{b}),F_{s,\phi}(x_{b}))$
(7)

The teacher is updated as the exponential moving average (EMA) of the student.

Warping and Unwarping in the Domain Adaptation Framework: The design decisions
that guide our choices are: (a) Maximize transferability of features from
supervised stage to unsupervised training stages. (b) Warping at test time
after adaptation is likely to be suboptimal as the target domain’s scale
distribution is unknown and can introduce latency. If the target distribution
is known, the Instance-level guidance can be shifted analytically to match
this distribution. Thus, our warping mechanism $W_{\mathcal{T}}$ operates only
where images from source domain are used, i.e. it is employed in the
supervised pre-training stage and the student model in the unsupervised stage
when source image is forwarded. We do experiment by warping target images too
while employing Static [36, 37] and Geometric prior [9] saliency guidances
(which do not use ground truth object labels to generate saliency), however,
that reduces performance (See Table 8).

Consider the recognition model $F$ as a combination of backbone $B$ and
prediction heads $H$, i.e. $F(x)=H(B(x))$. We warp the image and then unwarp
the feature maps from backbone $B$ respectively to compose,

$F^{\prime}(x_{a})=H(W_{\widetilde{\mathcal{T}}^{-1}}(B(W_{\mathcal{T}}(x_{a}))))$
(8)

following [37] for a source image $x_{a}$. Thus, losses are modified for the
supervised pre-training stage as,

$\mathcal{L}_{sup}(y_{a},F^{\prime}_{\theta}(x_{a}))$ (9)

and for the unsupervised training stage as,

$\mathcal{L}_{sup}(y_{a},F^{\prime}_{s,\phi}(x_{a}))+\lambda\mathcal{L}_{unsup}(F_{t,\kappa}(x_{b}),F_{s,\phi}(x_{b}))$
(10)

For our experiments, we use DAFormer [12] and 2PCNet [17] as our baseline
segmentation and detection domain adaption algorithms respectively. We do not
make additional assumptions about the domain adaptation setup, and thus our
method is agnostic to the task specification (segmentation or detection).
Other domain adaption algorithms [13, 14, 18, 6, 48] are also applicable.

Inference: We employ the teacher model $F_{t}$ during inference, and do not
require any image warping and feature unwarping at test time. This is in
contrast with image warping techniques proposed in the supervised setting [36,
9, 37], which incur overheads from test-time warping and assume additional
information (like vanishing points [9]) to generate saliency maps at test
time.

Supervised Setting: Our training method can also be applied in supervised
learning scenario by following the supervised pre-training steps to train a
model $F^{\prime}$. That is, omitting the unsupervised training stage
completely and only performing supervised pre-training stage – then removing
the warping and unwarping layers completely from the trained model. This is
unlike prior work [25, 36, 8, 37] which advocates warping at test time too.
Model performance improves significantly on target domain $\mathcal{X}_{b}$
without performing adaptation (See Table 5). For Source Domain Results, see
Supplementary Section C.

##  4 Experimental Methodology and Datasets

Figure 6: Domain Adaptive Object Detection (BDD100K Clear $\rightarrow$ DENSE
Foggy). Our method identifies streetlights and vehicles, which 2PCNet
mislabels as traffic signs. Our method ignores windows, which 2PCNet
misclassifies as pedestrians. Figure 7: Domain Adaptive Semantic Segmentation
(Cityscapes $\rightarrow$ ACDC). Our warping significantly improved
recognition of background elements like the sky, sidewalks, and buildings, as
well as foreground objects such as traffic lights and traffic signs,
indicating enhanced scene understanding ability.

We implement our source domain warping technique alongside a baseline domain
adaptation algorithm, ensuring fair comparisons by using the same model,
datasets, training schedules, hyperparameters, and random seeds.

Object Detection: Following the recent and popular domain adaptation strategy
2PCNet [17] (2023), we use Faster R-CNN [27] with ResNet-50 [10] adhering to
their training hyperparameters and protocols. While 2PCNet [17] solely focuses
on Day-to-Night adaptation, our method spans various adaptation scenarios. For
scenarios other than day-to-night adaptation, we exclude the NightAug
augmentation proposed in [17] for both the baseline and our method.

Semantic Segmentation: Aligning with the recent and popular domain adaptation
strategy DAFormer [12] (2022), we employ the same SegFormer [44] head and
MiT-B5 [44] backbone, and follow their training hyperparameters, protocols,
and seed for fair comparison. While DAFormer [12] shows results on Sim2Real
Gap which is not a focus of our work, results and discussion are available in
Supplementary Section A. Extensions of DAFormer, including HRDA [13] and MIC
[14], also benefits from our methodology, detailed in the Appendix B

###  4.1 Datasets

BDD100K [47] features 100,000 images with a resolution of 1280x720 for object
detection and segmentation, covering various weather conditions and times of
day. It includes 10 categories for annotation.

Cityscapes [4] provides 5,000 images of urban road scenes at a resolution of
2048x1024 with clear weather for semantic segmentation, with 19 categories.

DENSE [2] provides 12,997 images at a resolution of 1920x1024, capturing
diverse weather conditions such as heavy fog and heavy snow.

ACDC [32] is designed for adverse conditions (fog and snow), including 1,600
images at 2048x1024 for segmentation across 19 categories.

DarkZurich [30] is tailored for low-light conditions, offering 2,416 unlabeled
nighttime images and 151 labeled twilight images for segmentation, with a
resolution of 1920x1080, and a focus on urban settings.

Table 1: Adverse Weather: BDD100K Clear $\rightarrow$ DENSE Foggy. Our method
achieves notable improvements in +6.1 mAP, +6.1 mAP50, +9.6 mAP75 and +9.7
mAPl over 2PCNet. Instance Warping guidance is superior to the other priors.

Method | mAP | mAP50 | mAP75 | mAPs | mAPm | mAPl | person | car | truck | bike
---|---|---|---|---|---|---|---|---|---|---
2PCNet | 27.0 | 45.0 | 30.1 | 0.0 | 15.4 | 35.5 | 87.5 | 72.5 | 19.1 | 0.9
\+ Ours w/ Sta. Prior | 28.6 | 48.5 | 31.7 | 0.0 | 19.6 | 34.0 | 88.1 | 72.5 | 25.6 | 7.7
\+ Ours w/ Geo. Prior | 27.1 | 45.8 | 29.5 | 0.0 | 15.2 | 37.5 | 86.5 | 75.4 | 21.0 | 0.4
\+ Ours w/ Inst. Warp | 33.1 | 51.1 | 39.7 | 0.0 | 14.5 | 45.2 | 85.8 | 66.3 | 50.0 | 2.2

Table 2: Adverse Weather: BDD100K (Clear $\rightarrow$ Rainy) Object
Detection.  Tested on BDD100k Rainy Val. Our method improves 2PCNet [17]
(overall and per category) by +3.0 mAP50. Instance-level performs best on mAP
and mAP50 overall.

Method | mAP | mAP50 | mAP75 | mAPs | mAPm | mAPl | person | rider | car | truck | bus |  | motor
---
cycle
bike |  | traffic
---
light
| traffic
---
sign
2PCNet | 25.8 | 48.4 | 24.4 | 10.9 | 34.1 | 49.0 | 56.3 | 40.4 | 77.0 | 57.2 | 55.9 | 36.8 | 36.8 | 55.6 | 68.1
\+ Ours w/ Sta. Prior | 26.8 | 50.6 | 24.9 | 11.4 | 35.9 | 49.6 | 57.4 | 42.5 | 77.4 | 60.1 | 59.5 | 44.8 | 39.4 | 55.7 | 68.9
\+ Ours w/ Geo. Prior | 26.9 | 49.9 | 24.5 | 11.8 | 35.6 | 50.1 | 56.1 | 44.6 | 77.5 | 58.0 | 58.6 | 33.3 | 45.8 | 55.8 | 69.5
\+ Ours w/ Inst. Warp | 27.1 | 51.4 | 24.6 | 11.0 | 36.7 | 49.0 | 57.2 | 46.6 | 77.5 | 59.8 | 60.9 | 46.3 | 42.2 | 54.9 | 68.3

Table 3: Bad Lighting: BDD100K (Day $\rightarrow$ Night) Object Detection.
Tested on BDD100k Night Val. Our method improves state-of the-art 2PCNet [17]
(overall and per category) by +3.7 mAP50. Geometric and Instance-level priors
perform similarly overall with instance-level improving more categories.

Method | mAP | mAP50 | mAP75 | mAPs | mAPm | mAPl | person | rider | car | truck | bus |  | motor
---
cycle
bike |  | traffic
---
light
| traffic
---
sign
2PCNet | 23.5 | 46.4 | 21.1 | 9.5 | 26.0 | 43.3 | 54.4 | 30.8 | 73.1 | 53.8 | 55.2 | 37.5 | 44.5 | 49.4 | 65.2
\+ Ours w/ Sta. Prior | 25.3 | 49.4 | 22.4 | 10.3 | 27.4 | 45.1 | 58.2 | 39.3 | 75.0 | 53.6 | 54.5 | 41.4 | 49.6 | 52.9 | 69.6
\+ Ours w/ Geo. Prior | 25.6 | 49.7 | 22.9 | 10.0 | 27.7 | 45.7 | 58.8 | 41.8 | 75.0 | 55.8 | 55.2 | 40.8 | 49.3 | 50.5 | 69.4
\+ Ours w/ Inst. Warp | 25.6 | 50.1 | 22.6 | 9.8 | 27.6 | 46.6 | 58.9 | 42.7 | 75.1 | 55.0 | 55.5 | 42.6 | 50.1 | 51.9 | 69.0

Table 4: Changing Geography: BDD100K Clear $\rightarrow$ ACDC Object
Detection. Tested on ACDC Val. Our method improves 2PCNet [17] (overall and
per category) by +1.7 mAP50. Instance-level is superior to the other priors.

Method | mAP | mAP50 | mAP75 | mAPs | mAPm | mAPl | person | rider | car | truck | bus |  | motor
---
cycle
bike
2PCNet | 16.6 | 30.8 | 14.9 | 6.0 | 24.8 | 26.6 | 41.0 | 20.0 | 77.0 | 41.8 | 17.1 | 29.9 | 19.3
\+ Ours w/ Sta. Prior | 17.3 | 31.0 | 16.7 | 6.2 | 25.9 | 29.5 | 42.2 | 20.9 | 76.7 | 40.5 | 21.7 | 24.4 | 21.8
\+ Ours w/ Geo. Prior | 16.6 | 30.3 | 15.4 | 5.2 | 25.1 | 28.7 | 42.3 | 19.2 | 75.0 | 39.5 | 18.1 | 26.7 | 21.3
\+ Ours w/ Inst. Warp | 17.9 | 32.1 | 18.2 | 6.8 | 26.7 | 32.1 | 45.7 | 20.6 | 77.6 | 43.2 | 20.8 | 26.7 | 22.2

##  5 Experiments

Table 5: Cityscapes $\rightarrow$ ACDC Semantic Segmentation. Tested on ACDC
Val and Test-splits. Our method is superior, with or without Adaptation,
implying better generalization of backbone and improves DAFormer by +6.3 mIoU
on Test-split.

Method | mIoU | road |  | side
---
walk
building | wall | fence | pole |  | traffic
---
light
| traffic
---
sign
| vege-
---
tation
terrain | sky | person | rider | car | truck | bus | train |  | motor
---
cycle
bike
ACDC Val Split: After Supervised Pre-Training
DAFormer [12] | 55.8 | 77.6 | 57.4 | 77.9 | 39.4 | 30.4 | 48.3 | 54.5 | 46.4 | 70.0 | 37.9 | 72.2 | 52.6 | 32.0 | 81.4 | 72.7 | 73.7 | 70.4 | 37.6 | 27.3
\+ Ours w/ Sta. | 55.8 | 77.5 | 55.8 | 76.7 | 39.8 | 28.7 | 48.6 | 56.0 | 43.9 | 69.0 | 37.5 | 73.0 | 53.6 | 32.9 | 81.1 | 71.8 | 72.6 | 82.5 | 36.0 | 22.8
\+ Ours w/ Geo. | 55.7 | 80.4 | 57.3 | 78.3 | 37.7 | 28.4 | 48.2 | 54.4 | 44.8 | 70.3 | 37.4 | 75.1 | 52.8 | 30.5 | 80.7 | 75.6 | 72.8 | 65.8 | 38.9 | 29.8
\+ Ours w/ Inst. | 56.8 | 80.2 | 48.3 | 76.0 | 35.0 | 31.6 | 48.6 | 58.2 | 45.3 | 69.4 | 36.9 | 77.2 | 53.2 | 26.2 | 82.0 | 77.6 | 80.0 | 69.4 | 39.7 | 43.7
ACDC Val Split: After Unsupervised Domain Adaptation
DAFormer [12] | 57.6 | 72.7 | 57.5 | 80.1 | 42.5 | 38.0 | 50.9 | 45.1 | 50.0 | 71.1 | 38.5 | 67.0 | 56.0 | 29.9 | 81.8 | 76.6 | 78.9 | 79.9 | 40.8 | 36.7
\+ Ours w/ Sta. | 59.1 | 73.8 | 56.8 | 79.1 | 45.2 | 39.6 | 50.0 | 48.3 | 49.4 | 71.0 | 39.3 | 68.6 | 55.8 | 36.9 | 82.1 | 73.2 | 81.3 | 88.7 | 37.1 | 47.4
\+ Ours w/ Geo. | 58.4 | 76.4 | 57.6 | 79.9 | 42.2 | 39.0 | 51.7 | 46.4 | 49.5 | 72.0 | 39.0 | 71.1 | 56.5 | 30.4 | 82.7 | 79.3 | 79.1 | 69.8 | 41.5 | 44.7
\+ Ours w/ Inst. | 61.8 | 82.9 | 56.1 | 79.8 | 44.6 | 40.3 | 52.7 | 60.8 | 52.5 | 72.0 | 38.4 | 78.0 | 56.6 | 30.5 | 84.9 | 80.2 | 86.9 | 86.4 | 44.5 | 45.8
ACDC Test Split Results
ADVENT [40] | 32.7 | 72.9 | 14.3 | 40.5 | 16.6 | 21.2 | 9.3 | 17.4 | 21.2 | 63.8 | 23.8 | 18.3 | 32.6 | 19.5 | 69.5 | 36.2 | 34.5 | 46.2 | 26.9 | 36.1
MGCDA [30] | 48.7 | 73.4 | 28.7 | 69.9 | 19.3 | 26.3 | 36.8 | 53.0 | 53.3 | 75.4 | 32.0 | 84.6 | 51.0 | 26.1 | 77.6 | 43.2 | 45.9 | 53.9 | 32.7 | 41.5
DANNet [43] | 50.0 | 84.3 | 54.2 | 77.6 | 38.0 | 30.0 | 18.9 | 41.6 | 35.2 | 71.3 | 39.4 | 86.6 | 48.7 | 29.2 | 76.2 | 41.6 | 43.0 | 58.6 | 32.6 | 43.9
DAFormer [12] | 55.4 | 58.4 | 51.3 | 84.0 | 42.7 | 35.1 | 50.7 | 30.0 | 57.0 | 74.8 | 52.8 | 51.3 | 58.3 | 32.6 | 82.7 | 58.3 | 54.9 | 82.4 | 44.1 | 50.7
\+ Ours w/ Inst. | 61.7 | 83.0 | 53.2 | 85.5 | 47.4 | 38.3 | 46.0 | 51.4 | 57.8 | 73.9 | 56.2 | 82.1 | 61.3 | 32.3 | 85.5 | 69.0 | 68.9 | 82.5 | 46.7 | 52.0

Table 6: Cityscapes $\rightarrow$ DarkZurich Semantic Segmentation. Tested on
DarkZurich Val and Test. Our method is superior, with or without Adaptation,
implying better generalization of backbone and improves DAFormer by +3.3 mIoU
on Val-split.

Method | mIoU | road |  | side
---
walk
building | wall | fence | pole |  | traffic
---
light
| traffic
---
sign
| vege-
---
tation
terrain | sky | person | rider | car | truck | bus | train |  | motor
---
cycle
bike
DarkZurich Val Split: After Supervised Pre-Training
DAFormer | 30.0 | 90.7 | 52.7 | 45.5 | 18.4 | 39.0 | 38.4 | 27.5 | 18.3 | 59.0 | 26.5 | 11.1 | 20.4 | 21.1 | 58.6 | - | - | - | 22.1 | 20.0
\+ Ours w/ Sta. | 32.7 | 91.8 | 57.2 | 47.3 | 19.3 | 46.2 | 40.1 | 21.0 | 18.2 | 53.5 | 31.9 | 6.4 | 20.6 | 18.6 | 64.1 | - | - | - | 25.0 | 26.4
\+ Ours w/ Geo. | 30.2 | 89.9 | 50.5 | 45.9 | 20.5 | 36.8 | 39.1 | 21.0 | 19.8 | 54.7 | 28.4 | 7.9 | 19.0 | 31.1 | 58.4 | - | - | - | 18.1 | 33.4
\+ Ours w/ Inst. | 34.0 | 91.9 | 59.4 | 49.3 | 27.7 | 39.2 | 39.4 | 27.6 | 15.8 | 55.1 | 29.9 | 11.4 | 20.7 | 9.2 | 61.9 | - | - | - | 27.9 | 44.7
DarkZurich Val Split: After Unsupervised Domain Adaptation
DAFormer | 33.8 | 92.0 | 66.7 | 47.3 | 25.9 | 50.8 | 38.0 | 24.6 | 19.4 | 62.2 | 31.9 | 17.9 | 20.4 | 28.4 | 61.7 | - | - | - | 21.0 | 34.3
\+ Ours w/ Sta. | 33.8 | 93.3 | 68.5 | 50.0 | 28.4 | 54.3 | 41.7 | 19.9 | 22.7 | 54.0 | 41.5 | 14.1 | 22.9 | 7.2 | 62.6 | - | - | - | 24.6 | 36.8
\+ Ours w/ Geo. | 33.3 | 84.3 | 60.3 | 60.2 | 23.9 | 49.4 | 38.7 | 12.7 | 25.0 | 52.0 | 39.5 | 20.8 | 37.1 | 16.2 | 64.6 | - | - | - | 21.1 | 27.8
\+ Ours w/ Inst. | 37.1 | 88.7 | 70.9 | 60.1 | 42.1 | 49.6 | 39.8 | 47.9 | 21.5 | 49.8 | 35.7 | 25.4 | 23.8 | 25.9 | 66.8 | - | - | - | 24.3 | 32.4
DarkZurich Test Split Results
ADVENT [40] | 29.7 | 85.8 | 37.9 | 55.5 | 27.7 | 14.5 | 23.1 | 14.0 | 21.1 | 32.1 | 8.7 | 2.0 | 39.9 | 16.6 | 64.0 | 13.8 | 0.0 | 58.8 | 28.5 | 20.7
MGCDA [30] | 42.5 | 80.3 | 49.3 | 66.2 | 7.8 | 11.0 | 41.4 | 38.9 | 39.0 | 64.1 | 18.0 | 55.8 | 52.1 | 53.5 | 74.7 | 66.0 | 0.0 | 37.5 | 29.1 | 22.7
DANNet [43] | 44.3 | 90.0 | 54.0 | 74.8 | 41.0 | 21.1 | 25.0 | 26.8 | 30.2 | 72.0 | 26.2 | 84.0 | 47.0 | 33.9 | 68.2 | 19.0 | 0.3 | 66.4 | 38.3 | 23.6
DAFormer [12] | 53.8 | 93.5 | 65.5 | 73.3 | 39.4 | 19.2 | 53.3 | 44.1 | 44.0 | 59.5 | 34.5 | 66.6 | 53.4 | 52.7 | 82.1 | 52.7 | 9.5 | 89.3 | 50.5 | 38.5
\+ Ours w/ Inst. | 54.4 | 94.2 | 68.1 | 72.6 | 43.2 | 23.6 | 51.0 | 48.1 | 48.5 | 60.6 | 33.6 | 54.8 | 55.5 | 49.0 | 79.8 | 51.1 | 13.2 | 90.2 | 53.2 | 43.4

###  5.1 Domain Adaptive Object Detection

Tables 1, 2, 3 and 4 show improvements of our method over state-of-the-art
2PCNet [17] across lighting, weather and geographic adaptations respectively.
Moroever, our instance-level saliency guidance always improves over
approximate saliency guidance [36, 9]. Note that 2PCNet [17] focused solely on
day $\rightarrow$ night adaptation with it’s NightAug augmentation, which was
only used in the BDD100K day $\rightarrow$ night experiment (Table 3) and
omitted for others. Our method to address scale bias is more general in
comparison.

Figures 5 and 6 shows examples where 2PCNet [17] mistakes parts of the road
for cars and street lights for traffic lights. In contrast, our method
accurately detects these elements in difficult lighting and weather
conditions.

###  5.2 Domain Adaptive Semantic Segmentation

To demonstrate our method is task agnostic, we incorporate it within domain
adaptive semantic segmentation using the popular DAFormer [12] algorithm. The
resulting approach shows significant improvement on both Val and Test Splits
of the ACDC dataset (Table 5). Here, saliency guidance choice matters: our
instance-level image saliency guidance outperforms Static Prior [36] and
Geometric Prior [9]. Being agnostic to the DA algorithm, our method also
boosts DAFormer extensions like HRDA [13] and MIC [14] (See Supplementary
Section B). These extensions mitigate scale bias to some extent but add
significant training memory (+5GB) and inference latency (+800 ms on RTX 4090
GPU), whereas our approach requires negligible additional training memory
(+0.04 GB) and adds no additional inference latency.

Table 5 also shows our method’s superiority across scenarios, with or without
unsupervised domain adaptation. This suggests that the learned backhone
features due to our method generalize better to unseen domains. Additionally,
Figure 7 illustrates superior performance of our method in identifying both
background elements (e.g., sky, sidewalks, buildings) and foreground objects
(e.g., traffic lights and traffic signs).

Figure 8: Grad-Cam [33] Visualization of the Last Layer Feature of the Learnt
ResNet-50 Backbone. Learned features due to our method has a higher degree of
focus, implying better learned features and scene comprehension ability.

###  5.3 Visual Diagnosis of Learned Models

In Section 5.2, we noted that the learned backbone feature of our method
generalize better. To corroborate this claim, we employed GradCAM [33]
visualizations, showcasing ResNet features from detectors trained on BDD100K
[47] in Figure 8. The heatmap for models trained with 2PCNet without warping
reveals a dispersed focus across the image, indicating a lack of precision.
The choice of saliency guidance plays a critical role: warping guided by
Geometric Prior [9] results in less focus on the target (e.g., cars) and
unnecessary background emphasis. Conversely, our instance-level saliency
guidance achieves a focused heatmap on relevant objects, minimizing irrelevant
activations.

###  5.4 Ablation Study

Table 7: Ablations: (a) Studying the impact of hyperparamaters $P$ and $U$,
same parameters used for all other experiments. (b) Studying the impact of
balancing the scale distribution by undersampling the dataset.

(a) Saliency Product $P$ and Upperbound $U$. Trained on Cityscapes
$\rightarrow$ ACDC and tested on ACDC Val. While $P=2^{8}$ and $U=1.00$ are
optimal, our model requires minimal engineering due to the small mIoU range
(1.7%).

Parameters | aAcc | mIoU | mAcc
---|---|---|---
P=$2^{7}$, U=1.00 | 84.7 | 60.1 | 73.2
P=$2^{9}$, U=1.00 | 84.2 | 61.4 | 74.1
P=$2^{8}$, U=0.50 | 85.2 | 60.1 | 72.5
P=$2^{8}$, U=0.75 | 82.4 | 60.3 | 73.1
P=$2^{8}$, U=1.00 | 85.3 | 61.8 | 73.8

(b) Balanced Undersampling Strategy. Trained on Cityscapes and tested on
Cityscapes Val using Supervised strategy in Section 3.3. Undersampling images
to address scale bias hinders segmentation.

Sample | Percent | Method | aAcc | mIoU | mAcc
---|---|---|---|---|---
Uniform | 25% | SegFormer | 95.0 | 71.0 | 79.1
Ours | 95.1 | 71.9 | 80.0
50% | SegFormer | 95.3 | 73.6 | 81.4
Ours | 95.5 | 74.4 | 82.1
None | 100% | SegFormer | 95.6 | 75.3 | 82.9
Ours | 95.9 | 76.8 | 84.2

For all ablation studies, we utilize the Cityscape $\rightarrow$ ACDC domain
adaptive semantic segmentation task.

Saliency Product $P$ and Upperbound $U$. Varying $P$ and $U$ shows optimal
performance at $P=2^{8}$ and $U=1.0$, according to quantitative scores in
Table 7(a). Hence, $P=2^{8}$ and $U=1.0$ are adopted for all major
experiments.

Warping on Source vs. Target. Given the necessity of ground truth labels for
our instance-level saliency guidance, warping in the target domain for
unsupervised domain adaptation is infeasible. Nevertheless, as shown in Table
8 warping source domain images with our saliency guidance surpasses
performance using approximated saliency [36, 9] with both source and target
domain warping.

Warping with From-Seg vs. Groundtruth bboxes. For object detection, we utilize
ground truth bounding boxes for saliency guidance. However, semantic
segmentation tasks often lack this data, leading us to use bounding boxes
computed from segmentations, “from-seg” for short. Results indicate superior
performance with ground truth boxes, but “from-seg” boxes are a close second,
highlighting their effectiveness when ground truth is unavailable (Table 8).

Distribution Balancing by Undersampling. We analyzed undersampling strategies
that balance the percentages of small, medium, and large objects, aiming to
create a more uniform distribution that mitigates scale bias. As shown in
Table 7(b), our experiments applying 25% and 50% of undersampling to the
Cityscapes dataset indicates that models trained on these “uniform” subsets
perform worse than those trained on the complete dataset, highlighting the
challenges of addressing scale bias through simple subsampling.

Table 8: Ablation: Warping (source/target domain) and Bounding Box Type (from-
seg/ground truth). Trained on Cityscapes $\rightarrow$ ACDC. Tested on ACDC
Val. Using ground truth bounding boxes are better than using bounding boxes
derived from semantic segmentation. In case of Static prior and Geometric
prior, we can apply warping during target adaptation and testing as they do
not rely on ground truth. In case of Inst., target and test warping is
impossible as ground truth boxes are unavailable in the target domain, as per
the definition of unsupervised domain adaptation.

Method | Warp | Bboxes | aAcc | mIoU | mAcc
---|---|---|---|---|---
DAFormer | - | - | 81.9 | 57.6 | 70.6
\+ Ours w/ Sta. Prior | src & tgt & test | - | 82.5 | 55.3 | 68.3
src & tgt | - | 83.8 | 57.4 | 69.6
src | - | 82.3 | 59.1 | 73.3
\+ Ours w/ Geo. Prior | src & tgt & test | - | 82.2 | 57.7 | 71.6
src & tgt | - | 81.9 | 56.5 | 70.9
src | - | 83.2 | 58.4 | 70.9
\+ Ours w/ Inst. (tgt & test inapplicable) | src | from-seg | 84.0 | 60.1 | 73.5
src | gt | 85.3 | 61.8 | 73.8

##  6 Limitations and Conclusion

In this work, we address scale bias with attentional processing, using
saliency-guided image warping that integrates seamlessly into any unsupervised
domain adaptation method. Leveraging scale statistics from the source dataset
and ground truth labels, our approach effectively navigates challenges in
lighting, weather, and geographic differences. Extensive experiments
demonstrate the effectiveness of our warping method and instance-level
saliency guidance. Importantly, our approach adds no inference latency.
Depending on the available information (e.g., vanishing point, target
statistics), other priors could be used with our approach, showing improvement
over baselines, albeit to a lesser extent.

One limitation of our method arises in densely populated scenes (e.g., New
York’s Times Square), where the limited background space challenges the
expansion and contraction capabilities of our warping. Besides, our approach
is less effective on certain synthetic datasets, such as Synthia [29], which
do not exhibit realistic scale bias. However, our approach proves effective on
other synthetic datasets like GTA5 [28], a role-playing game dataset featuring
city driving components with realistic traffic and exhibiting realistic scale
biases.

Societal Impact. While our method demonstrates improved perception on publicly
available night and adverse weather driving datasets, extensive testing is
required before real-world deployment on self-driving vehicles.

Acknowledgment. This work was supported in parts by a sponsored research
contract from General Motors Research-Israel, NSF grant CNS-2038612, and DOT
RITA Mobility-21 Grant 69A3551747111.

## References

  * Bi et al. [2018] Hui Bi, Hui Tang, Guanyu Yang, Huazhong Shu, and Jean-Louis Dillenseger.  Accurate image segmentation using gaussian mixture model with saliency map.  _Pattern Analysis and Applications_ , 2018.
  * Bijelic et al. [2020] Mario Bijelic, Tobias Gruber, Fahim Mannan, Florian Kraus, Werner Ritter, Klaus Dietmayer, and Felix Heide.  Seeing through fog without seeing fog: Deep multimodal sensor fusion in unseen adverse weather.  In _CVPR_ , 2020.
  * Chin et al. [2019] Ting-Wu Chin, Ruizhou Ding, and Diana Marculescu.  Adascale: Towards real-time video object detection using adaptive scaling.  _MLSys_ , 2019.
  * Cordts et al. [2016] Marius Cordts, Mohamed Omran, Sebastian Ramos, Timo Rehfeld, Markus Enzweiler, Rodrigo Benenson, Uwe Franke, Stefan Roth, and Bernt Schiele.  The cityscapes dataset for semantic urban scene understanding.  In _CVPR_ , 2016.
  * Dalal and Triggs [2005] Navneet Dalal and Bill Triggs.  Histograms of oriented gradients for human detection.  In _CVPR_ , 2005.
  * Deng et al. [2021] Jinhong Deng, Wen Li, Yuhua Chen, and Lixin Duan.  Unbiased mean teacher for cross-domain object detection.  In _CVPR_ , 2021.
  * Ehteshami Bejnordi et al. [2022] Babak Ehteshami Bejnordi, Amirhossein Habibian, Fatih Porikli, and Amir Ghodrati.  Salisa: Saliency-based input sampling for efficient video object detection.  In _ECCV_ , 2022.
  * Ghosh et al. [2023a] Anurag Ghosh, Vaibhav Balloli, Akshay Nambi, Aditya Singh, and Tanuja Ganu.  Chanakya: Learning runtime decisions for adaptive real-time perception.  In _NeurIPS_ , 2023a.
  * Ghosh et al. [2023b] Anurag Ghosh, N Dinesh Reddy, Christoph Mertz, and Srinivasa G Narasimhan.  Learned two-plane perspective prior based image resampling for efficient object detection.  In _CVPR_ , 2023b.
  * He et al. [2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun.  Deep residual learning for image recognition.  In _CVPR_ , 2016.
  * He et al. [2022] Mengzhe He, Yali Wang, Jiaxi Wu, Yiru Wang, Hanqing Li, Bo Li, Weihao Gan, Wei Wu, and Yu Qiao.  Cross domain object detection by target-perceived dual branch distillation.  In _CVPR_ , 2022.
  * Hoyer et al. [2022a] Lukas Hoyer, Dengxin Dai, and Luc Van Gool.  Daformer: Improving network architectures and training strategies for domain-adaptive semantic segmentation.  In _CVPR_ , 2022a.
  * Hoyer et al. [2022b] Lukas Hoyer, Dengxin Dai, and Luc Van Gool.  HRDA: Context-aware high-resolution domain-adaptive semantic segmentation.  In _ECCV_ , 2022b.
  * Hoyer et al. [2023] Lukas Hoyer, Dengxin Dai, Haoran Wang, and Luc Van Gool.  MIC: Masked image consistency for context-enhanced domain adaptation.  In _CVPR_ , 2023.
  * Huang et al. [2017] Jonathan Huang, Vivek Rathod, Chen Sun, Menglong Zhu, Anoop Korattikara, Alireza Fathi, Ian Fischer, Zbigniew Wojna, Yang Song, Sergio Guadarrama, et al.  Speed/accuracy trade-offs for modern convolutional object detectors.  In _CVPR_ , 2017.
  * Jaderberg et al. [2015] Max Jaderberg, Karen Simonyan, Andrew Zisserman, et al.  Spatial transformer networks.  _NeurIPS_ , 2015.
  * Kennerley et al. [2023] Mikhail Kennerley, Jian-Gang Wang, Bharadwaj Veeravalli, and Robby T Tan.  2pcnet: Two-phase consistency training for day-to-night unsupervised domain adaptive object detection.  In _CVPR_ , 2023.
  * Li et al. [2022] Yu-Jhe Li, Xiaoliang Dai, Chih-Yao Ma, Yen-Cheng Liu, Kan Chen, Bichen Wu, Zijian He, Kris Kitani, and Peter Vajda.  Cross-domain adaptive teacher for object detection.  In _CVPR_ , 2022.
  * Lin et al. [2018] Di Lin, Yuanfeng Ji, Dani Lischinski, Daniel Cohen-Or, and Hui Huang.  Multi-scale context intertwining for semantic segmentation.  In _ECCV_ , 2018.
  * Lin et al. [2014] Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick.  Microsoft coco: Common objects in context.  In _ECCV_ , 2014.
  * Lin et al. [2017] Tsung-Yi Lin, Piotr Dollár, Ross Girshick, Kaiming He, Bharath Hariharan, and Serge Belongie.  Feature pyramid networks for object detection.  In _CVPR_ , 2017.
  * Liu et al. [2018] Shu Liu, Lu Qi, Haifang Qin, Jianping Shi, and Jiaya Jia.  Path aggregation network for instance segmentation.  In _CVPR_ , 2018.
  * Lowe [2004] David G Lowe.  Distinctive image features from scale-invariant keypoints.  _IJCV_ , 2004.
  * Mei et al. [2020] Ke Mei, Chuang Zhu, Jiaqi Zou, and Shanghang Zhang.  Instance adaptive self-training for unsupervised domain adaptation.  In _ECCV_ , 2020.
  * Recasens et al. [2018] Adria Recasens, Petr Kellnhofer, Simon Stent, Wojciech Matusik, and Antonio Torralba.  Learning to zoom: a saliency-based sampling layer for neural networks.  In _ECCV_ , 2018.
  * Redmon and Farhadi [2018] Joseph Redmon and Ali Farhadi.  Yolov3: An incremental improvement.  _arXiv preprint arXiv:1804.02767_ , 2018.
  * Ren et al. [2015] Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun.  Faster r-cnn: Towards real-time object detection with region proposal networks.  _NeurIPS_ , 2015.
  * Richter et al. [2016] Stephan R Richter, Vibhav Vineet, Stefan Roth, and Vladlen Koltun.  Playing for data: Ground truth from computer games.  In _ECCV_ , 2016.
  * Ros et al. [2016] German Ros, Laura Sellart, Joanna Materzynska, David Vazquez, and Antonio M Lopez.  The synthia dataset: A large collection of synthetic images for semantic segmentation of urban scenes.  In _CVPR_ , 2016.
  * Sakaridis et al. [2019] Christos Sakaridis, Dengxin Dai, and Luc Van Gool.  Guided curriculum model adaptation and uncertainty-aware evaluation for semantic nighttime image segmentation.  In _ICCV_ , 2019.
  * Sakaridis et al. [2020] Christos Sakaridis, Dengxin Dai, and Luc Van Gool.  Map-guided curriculum domain adaptation and uncertainty-aware evaluation for semantic nighttime image segmentation.  _TPAMI_ , 2020.
  * Sakaridis et al. [2021] Christos Sakaridis, Dengxin Dai, and Luc Van Gool.  Acdc: The adverse conditions dataset with correspondences for semantic driving scene understanding.  In _ICCV_ , 2021.
  * Selvaraju et al. [2017] Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra.  Grad-cam: Visual explanations from deep networks via gradient-based localization.  In _ICCV_ , 2017.
  * Shrivastava et al. [2016] Abhinav Shrivastava, Abhinav Gupta, and Ross Girshick.  Training region-based object detectors with online hard example mining.  In _CVPR_ , 2016.
  * Singh and Davis [2018] Bharat Singh and Larry S Davis.  An analysis of scale invariance in object detection snip.  In _CVPR_ , 2018.
  * Thavamani et al. [2021] Chittesh Thavamani, Mengtian Li, Nicolas Cebron, and Deva Ramanan.  Fovea: Foveated image magnification for autonomous navigation.  In _ICCV_ , 2021.
  * Thavamani et al. [2023] Chittesh Thavamani, Mengtian Li, Francesco Ferroni, and Deva Ramanan.  Learning to zoom and unzoom.  In _CVPR_ , 2023.
  * Tranheden et al. [2021] Wilhelm Tranheden, Viktor Olsson, Juliano Pinto, and Lennart Svensson.  Dacs: Domain adaptation via cross-domain mixed sampling.  In _WACV_ , 2021.
  * Tsai et al. [2018] Yi-Hsuan Tsai, Wei-Chih Hung, Samuel Schulter, Kihyuk Sohn, Ming-Hsuan Yang, and Manmohan Chandraker.  Learning to adapt structured output space for semantic segmentation.  In _CVPR_ , 2018.
  * Vu et al. [2019] Tuan-Hung Vu, Himalaya Jain, Maxime Bucher, Matthieu Cord, and Patrick Pérez.  Advent: Adversarial entropy minimization for domain adaptation in semantic segmentation.  In _CVPR_ , 2019.
  * Wang et al. [2021] Qin Wang, Dengxin Dai, Lukas Hoyer, Luc Van Gool, and Olga Fink.  Domain adaptive semantic segmentation with self-supervised depth estimation.  In _ICCV_ , 2021.
  * Wolberg [1990] George Wolberg.  _Digital image warping_.  IEEE Computer Society Press Los Alamitos, CA, 1990.
  * Wu et al. [2021] Xinyi Wu, Zhenyao Wu, Hao Guo, Lili Ju, and Song Wang.  Dannet: A one-stage domain adaptation network for unsupervised nighttime semantic segmentation.  In _CVPR_ , 2021.
  * Xie et al. [2021] Enze Xie, Wenhai Wang, Zhiding Yu, Anima Anandkumar, Jose M Alvarez, and Ping Luo.  Segformer: Simple and efficient design for semantic segmentation with transformers.  _NeurIPS_ , 2021.
  * Yang et al. [2018] Maoke Yang, Kun Yu, Chi Zhang, Zhiwei Li, and Kuiyuan Yang.  Denseaspp for semantic segmentation in street scenes.  In _CVPR_ , 2018.
  * Yang and Soatto [2020] Yanchao Yang and Stefano Soatto.  Fda: Fourier domain adaptation for semantic segmentation.  In _CVPR_ , 2020.
  * Yu et al. [2020] Fisher Yu, Haofeng Chen, Xin Wang, Wenqi Xian, Yingying Chen, Fangchen Liu, Vashisht Madhavan, and Trevor Darrell.  Bdd100k: A diverse driving dataset for heterogeneous multitask learning.  In _CVPR_ , 2020.
  * Zhang et al. [2021] Pan Zhang, Bo Zhang, Ting Zhang, Dong Chen, Yong Wang, and Fang Wen.  Prototypical pseudo label denoising and target structure learning for domain adaptive semantic segmentation.  In _CVPR_ , 2021.
  * Zhou et al. [2018] Peng Zhou, Bingbing Ni, Cong Geng, Jianguo Hu, and Yi Xu.  Scale-transferrable object detection.  In _CVPR_ , 2018.
  * Zou et al. [2018] Yang Zou, Zhiding Yu, BVK Kumar, and Jinsong Wang.  Unsupervised domain adaptation for semantic segmentation via class-balanced self-training.  In _ECCV_ , 2018.

##  Appendix A Synthetic to Real Domain Adaptation

Figure 9: Synthetic to Real Domain Adaptation. GTA [28] consist of realistic
scenes from vehicle’s perspective with realistic road traffic simulation, more
closely reflecting object scale biases present in real-world driving scenes.
In contrast, Synthia [29] consists of virtual world images that do not
represent the scale biases of natural driving scenes.

Our proposed method addresses source scale biases that are present in natural
scene datasets. Synthetic To Real (Sim2Real) Domain Gap is not a focus of our
work as a synthetic scene dataset may or may not exhibit the same bias.

However, we do explore if our method improves performance on the Sim2Real
Domain Gap by considering GTA [28] and Synthia [29] synthetic datasets. Table
9 and Table 10 show the results for GTA $\rightarrow$ Cityscapes and Synthia
$\rightarrow$ Cityscapes domain adaptation respectively.

Our method improves GTA $\rightarrow$ Cityscapes adaptation considerably,
while improvement on Synthia $\rightarrow$ Cityscapes adaptation is marginal,
which corresponds to our intuition regarding both the source datasets (See Fig
9).

Grand Theft Auto 5 (GTA) is a role playing game with a city driving component
along with realistic traffic, thus GTA images are close to real driving scenes
and hence exhibits similar scale biases. Synthia, on the other hand, is a
rendered dataset of a virtual city, and does not have realistic traffic
simulation of dynamic objects like cars and people. Morever, many images are
not captured from a vehicle’s perspective. Thus, images are not close to real
driving scenes and do not exhibit scale bias.

Table 9: Synthetic to Real Domain Adaptation: GTA $\rightarrow$ Cityscapes
Semantic Segmentation. Tested on Cityscapes Val. We were unable to reproduce
DAFormer’s [12] results averaged over 3 random seeds, hence we present results
with seed=0 for both DAFormer [12] and our method (DAFormer with Instance-
level Image Warping). We do not target adapation from synthetic datasets and
Sim2Real Domain Gap, as synthetic datasets may not have a scale bias. However,
GTA appears to have a scale bias like natural driving scenes. This is due to
realistic traffic simulation and all the captured imagery is from a vehicle’s
perspective (See Figure 9). Our method shows improved performance on many
individual categories.

Method | mIoU | road |  | side
---
walk
building | wall | fence | pole |  | traffic
---
light
| traffic
---
sign
| vege-
---
tation
terrain | sky | person | rider | car | truck | bus | train |  | motor
---
cycle
bike
CBST [50] | 45.9 | 91.8 | 53.5 | 80.5 | 32.7 | 21.0 | 34.0 | 28.9 | 20.4 | 83.9 | 34.2 | 80.9 | 53.1 | 24.0 | 82.7 | 30.3 | 35.9 | 16.0 | 25.9 | 42.8
DACS [38] | 52.1 | 89.9 | 39.7 | 87.9 | 30.7 | 39.5 | 38.5 | 46.4 | 52.8 | 88.0 | 44.0 | 88.8 | 67.2 | 35.8 | 84.5 | 45.7 | 50.2 | 0.0 | 27.3 | 34.0
CorDA [41] | 56.6 | 94.7 | 63.1 | 87.6 | 30.7 | 40.6 | 40.2 | 47.8 | 51.6 | 87.6 | 47.0 | 89.7 | 66.7 | 35.9 | 90.2 | 48.9 | 57.5 | 0.0 | 39.8 | 56.0
ProDA [48] | 57.5 | 87.8 | 56.0 | 79.7 | 46.3 | 44.8 | 45.6 | 53.5 | 53.5 | 88.6 | 45.2 | 82.1 | 70.7 | 39.2 | 88.8 | 45.5 | 59.4 | 1.0 | 48.9 | 56.4
DAFormer [12] | 68.3 | 95.7 | 70.2 | 89.4 | 53.5 | 48.1 | 49.6 | 55.8 | 59.4 | 89.9 | 47.9 | 92.5 | 72.2 | 44.7 | 92.3 | 74.5 | 78.2 | 65.1 | 55.9 | 61.8
DAFormer (seed=0) | 66.9 | 92.6 | 58.9 | 89.3 | 54.2 | 42.7 | 49.4 | 57.0 | 55.8 | 89.2 | 49.8 | 89.5 | 72.7 | 41.7 | 92.0 | 62.0 | 82.8 | 71.3 | 56.5 | 62.9
\+ Ours | 68.5 | 92.9 | 60.0 | 89.8 | 55.9 | 51.5 | 49.0 | 57.2 | 62.2 | 89.6 | 50.2 | 91.5 | 71.9 | 44.8 | 93.0 | 78.7 | 79.8 | 63.6 | 56.6 | 63.6

Table 10: Synthetic to Real Domain Adaptation: Synthia $\rightarrow$
Cityscapes Semantic Segmentation. Tested on Cityscapes Val. We were unable to
reproduce DAFormer’s [12] results averaged over 3 random seeds, hence we
present results with seed=0 for both DAFormer [12] and our method (DAFormer
with Instance-level Image Warping). We do not target adapation from synthetic
datasets and Sim2Real Domain Gap, as synthetic datasets may not have a scale
bias. Our improvement on Synthia $\rightarrow$ Cityscapes are marginal and we
believe this is because Synthia does not exhibit scale bias akin to natural
driving scenes (See Figure 9).

Method | mIoU | road |  | side
---
walk
building | wall | fence | pole |  | traffic
---
light
| traffic
---
sign
| vege-
---
tation
terrain | sky | person | rider | car | truck | bus | train |  | motor
---
cycle
bike
CBST [50] | 42.6 | 68.0 | 29.9 | 76.3 | 10.8 | 1.4 | 33.9 | 22.8 | 29.5 | 77.6 | - | 78.3 | 60.6 | 28.3 | 81.6 | - | 23.5 | - | 18.8 | 39.8
DACS [38] | 48.3 | 80.6 | 25.1 | 81.9 | 21.5 | 2.9 | 37.2 | 22.7 | 24.0 | 83.7 | - | 90.8 | 67.6 | 38.3 | 82.9 | - | 38.9 | - | 28.5 | 47.6
CorDA [41] | 55.0 | 93.3 | 61.6 | 85.3 | 19.6 | 5.1 | 37.8 | 36.6 | 42.8 | 84.9 | - | 90.4 | 69.7 | 41.8 | 85.6 | - | 38.4 | - | 32.6 | 53.9
ProDA [48] | 55.5 | 87.8 | 45.7 | 84.6 | 37.1 | 0.6 | 44.0 | 54.6 | 37.0 | 88.1 | - | 84.4 | 74.2 | 24.3 | 88.2 | - | 51.1 | - | 40.5 | 45.6
DAFormer [12] | 60.9 | 84.5 | 40.7 | 88.4 | 41.5 | 6.5 | 50.0 | 55.0 | 54.6 | 86.0 | - | 89.8 | 73.2 | 48.2 | 87.2 | - | 53.2 | - | 53.9 | 61.7
DAFormer (seed=0) | 59.2 | 87.8 | 47.6 | 87.6 | 43.7 | 5.8 | 49.0 | 48.0 | 53.1 | 82.3 | - | 71.8 | 71.7 | 46.0 | 87.5 | - | 49.3 | - | 52.7 | 63.4
\+ Ours | 59.4 | 92.7 | 57.7 | 87.1 | 44.8 | 7.1 | 46.2 | 46.9 | 53.6 | 80.6 | - | 68.2 | 72.5 | 42.5 | 90.6 | - | 41.6 | - | 55.2 | 63.3

##  Appendix B Comparison with other Domain Adaptation Strategies

Table 11: Comparison with other Domain Adaptation Strategies: Computational Overheads. We are using a single RTX 4090 GPU. The batch size is 1, and all the other training and inference configuration are the same their respective papers – DAFormer [12] and HRDA [13]. Instance-level image warping imposes minimal memory overheads during training. Moreover, as we do not warp during test time, our method does not have any inference latency overheads. Method | Training Memory | Inference Time
---|---|---
DAFormer | 16.41 GB | 195.0 ms
DAFormer + Ours | \+ 0.04 GB | \+ 0 ms
DAFormer + HRDA | \+ 5.11 GB | \+ 801.2 ms

Domain adaptation extensions have been proposed on the DAFormer [12] training
framework, such as HRDA [13] and MIC [14] which also show good performance.

Differences from DAFormer. HRDA [13] proposed a multi-scale high-resolution
crop training strategy along with sliding window inference. MIC [14] proposed
a masking consistency strategy for target domain images to improve spatial
context.

HRDA [13] departs from DAFormer [12] by increasing training and inference
image scales by a factor of 2 and performing sliding window inference (a test
time scale augmentation). Training and inference computational costs
significantly increase with HRDA’s strategy [13]. Correspondingly, our
inference costs are same as DAFormer [12] (as we do not warp during test-time)
while training time and memory consumption is marginally higher (See Table
11). Lastly, instance-level image warping is orthogonal to the aforementioned
training strategies and can be used simultaneously with other training
strategies.

Evaluation Methodology. Based on the above discussion, we apply HRDA’s and
MIC’s strategy on DAFormer, but for a fair comparison, we perform training and
inference following DAFormer’s [12] training/testing image scales and
evaluation paradigm. The alternative comparison would be to train and evaluate
on full-scale images for parity with HRDA, however, we do not have a GPU with
high memory availability (like NVIDIA Tesla A40 with 48 GB or Tesla A100 with
80 GB memory).

Results. Table 12 and 13 present results for Cityscapes $\rightarrow$
DarkZurich and Cityscapes $\rightarrow$ ACDC respectively. MIC (HRDA)
corresponds to MIC and HRDA training strategies added to DAFormer. We compare
with DAFormer and MIC (HRDA) with and without including our Instance-level
image warping approach. We observe smaller improvements in conjunction with
our method with MIC (HRDA) as HRDA uses multi-scale cropping strategies, which
also attempts to alleviate scale bias to some extent.

As our instance-level image warping approach is orthogonal to other training
strategies and can be used in plug-and-play manner, we studied the interaction
of our method with MIC [14] and HRDA [13] individually (See Table 14).

We observed that our method when employed in conjunction with MIC and HRDA
consistently improved performance, however, was not able to improve upon the
scenario where our method is applied to DAFormer directly. Moreover, HRDA
failed to improve upon DAFormer itself when trained on DAFormer’s training
scale (1024 $\times$ 512) instead of full scale (2048 $\times$ 1024).

We concluded that high resolution training is necessary for HRDA as posited
[13] and the strategy does not work at all image scales. Further, we observed
HRDA performs two kinds of cropping – a HR-detail crop to better segment small
object details and LR-context crop to adapt better to large objects. At
DAFormer’s training scales, the cropped size of the detail crop would be very
small and hurt performance. The advantage of our instance-level warping method
is that it increases focus on small object details, thus we replaced detail
crop with our instance level image warping. Removing detail crop from HRDA
improved performance over DAFormer itself, and in conjunction with our method
showed the best performance.

Table 12: Comparison with other Domain Adaptation Strategies: Cityscapes
$\rightarrow$ DarkZurich Semantic Segmentation. Tested on DarkZurich Val. Our
method improves IoU scores in conjunction with both DAFormer and MIC (HRDA)
strategies. We observe smaller improvements with MIC (HRDA) as HRDA uses
multi-scale cropping strategy, which alleviates scale bias to some extent.

Method | mIoU | road |  | side
---
walk
building | wall | fence | pole |  | traffic
---
light
| traffic
---
sign
| vege-
---
tation
terrain | sky | person | rider | car | truck | bus | train |  | motor
---
cycle
bike
DAFormer | 33.8 | 92.0 | 66.7 | 47.3 | 25.9 | 50.8 | 38.0 | 24.6 | 19.4 | 62.2 | 31.9 | 17.9 | 20.4 | 28.4 | 61.7 | - | - | - | 21.0 | 34.3
DAFormer + Ours | 37.1 | 88.7 | 70.9 | 60.1 | 42.1 | 49.6 | 39.8 | 47.9 | 21.5 | 49.8 | 35.7 | 25.4 | 23.8 | 25.9 | 66.8 | - | - | - | 24.3 | 32.4
MIC (HRDA) | 39.8 | 78.8 | 13.0 | 83.1 | 46.7 | 52.2 | 42.2 | 44.5 | 28.8 | 64.3 | 35.4 | 82.8 | 24.9 | 33.8 | 62.8 | - | - | - | 25.9 | 37.3
MIC (HRDA) + Ours | 40.1 | 75.8 | 5.8 | 83.3 | 55.7 | 54.8 | 44.3 | 30.5 | 38.0 | 66.3 | 42.7 | 82.4 | 39.4 | 15.1 | 69.3 | - | - | - | 25.1 | 32.9

Table 13: Comparison with other Domain Adaptation Strategies: Cityscapes
$\rightarrow$ ACDC Semantic Segmentation. Tested on ACDC Val. Our method
achieves a significant improvement in the IoU score in conjunction with both
DAFormer and MIC (HRDA) strategies. We observe smaller improvements with MIC
(HRDA) as HRDA uses multi-scale cropping strategy, which alleviates scale bias
to some extent.

Method | mIoU | road |  | side
---
walk
building | wall | fence | pole |  | traffic
---
light
| traffic
---
sign
| vege-
---
tation
terrain | sky | person | rider | car | truck | bus | train |  | motor
---
cycle
bike
DAFormer | 57.6 | 72.7 | 57.5 | 80.1 | 42.5 | 38.0 | 50.9 | 45.1 | 50.0 | 71.1 | 38.5 | 67.0 | 56.0 | 29.9 | 81.8 | 76.6 | 78.9 | 79.9 | 40.8 | 36.7
DAFormer + Ours | 61.8 | 82.9 | 56.1 | 79.8 | 44.6 | 40.3 | 52.7 | 60.8 | 52.5 | 72.0 | 38.4 | 78.0 | 56.6 | 30.5 | 84.9 | 80.2 | 86.9 | 86.4 | 44.5 | 45.8
MIC (HRDA) | 59.6 | 66.4 | 55.8 | 84.8 | 54.6 | 41.3 | 49.7 | 41.2 | 53.6 | 78.3 | 37.7 | 67.4 | 58.3 | 24.0 | 83.6 | 70.1 | 88.8 | 87.9 | 40.7 | 48.4
MIC (HRDA) + Ours | 61.6 | 80.6 | 38.6 | 83.7 | 50.5 | 41.3 | 50.5 | 56.7 | 49.6 | 75.2 | 39.4 | 83.9 | 58.9 | 32.0 | 86.0 | 75.4 | 91.8 | 88.0 | 40.2 | 48.5

Table 14: Comparison with other Domain Adaptation Strategies: Ablation on
Cityscapes $\rightarrow$ ACDC Semantic Segmentation. Tested on ACDC Val. We
show that our method improves performance in conjunction with base DAFormer
[12], MIC [14], HRDA [13] and HRDA without HR-detail crop (HRDA∗)
respectively. Our observations are – (a) Instance-level image warping improves
MIC, however, fails to improve beyond DAFormer with instance-level image
warping. – (b) HRDA performs worse than DAFormer at DAFormer’s training
scales, showing that full resolution training is needed for HRDA to perform
well, consistent with [13]. Our method only improves marginally as both
methods try to alleviate scale biases. – (c) As we train with DAFormer’s
training scales to maintain fairness, removing the HR-detail crop is
beneficial as the expected cropped size is small. Adding instance-level image
warping to focus on small object details instead of a detail crop, in
conjunction with HRDA’s context crop improves performance considerably.

Method | mIoU | road |  | side
---
walk
building | wall | fence | pole |  | traffic
---
light
| traffic
---
sign
| vege-
---
tation
terrain | sky | person | rider | car | truck | bus | train |  | motor
---
cycle
bike
DAFormer | 57.6 | 72.7 | 57.5 | 80.1 | 42.5 | 38.0 | 50.9 | 45.1 | 50.0 | 71.1 | 38.5 | 67.0 | 56.0 | 29.9 | 81.8 | 76.6 | 78.9 | 79.9 | 40.8 | 36.7
DAFormer + Ours | 61.8 | 82.9 | 56.1 | 79.8 | 44.6 | 40.3 | 52.7 | 60.8 | 52.5 | 72.0 | 38.4 | 78.0 | 56.6 | 30.5 | 84.9 | 80.2 | 86.9 | 86.4 | 44.5 | 45.8
MIC (DAFormer) | 58.8 | 61.1 | 59.5 | 73.2 | 47.4 | 45.2 | 51.4 | 44.7 | 48.2 | 78.4 | 38.1 | 51.4 | 60.4 | 41.3 | 84.3 | 78.5 | 84.3 | 78.9 | 43.9 | 46.5
MIC (DAFormer) + Ours | 60.6 | 72.8 | 62.9 | 73.4 | 45.1 | 36.5 | 52.9 | 49.0 | 49.9 | 76.9 | 39.8 | 65.5 | 60.6 | 40.4 | 85.2 | 80.9 | 90.5 | 87.0 | 41.0 | 41.3
HRDA (DAFormer) | 56.9 | 79.9 | 37.8 | 81.1 | 45.6 | 33.9 | 47.8 | 47.3 | 47.1 | 74.3 | 37.1 | 84.0 | 47.7 | 17.6 | 84.2 | 69.1 | 88.2 | 75.1 | 37.6 | 45.1
HRDA (DAFormer) + Ours | 57.7 | 85.6 | 48.2 | 71.7 | 41.6 | 39.4 | 50.8 | 17.7 | 47.8 | 75.2 | 37.9 | 81.4 | 56.3 | 25.0 | 82.3 | 73.4 | 88.8 | 82.3 | 45.3 | 46.7
HRDA∗ (DAFormer) | 58.3 | 68.5 | 59.4 | 82.8 | 50.4 | 40.8 | 50.3 | 42.4 | 44.4 | 77.6 | 38.0 | 69.4 | 55.7 | 27.2 | 83.2 | 77.4 | 78.2 | 79.0 | 34.9 | 48.4
HRDA∗ (DAFormer) + Ours | 62.1 | 89.7 | 61.1 | 83.9 | 43.4 | 39.5 | 52.7 | 43.1 | 45.0 | 75.6 | 38.7 | 86.1 | 55.0 | 28.0 | 84.9 | 81.1 | 88.5 | 86.0 | 44.9 | 53.4

Figure 10: Supervised Setting: Visual Comparison on Cityscapes Semantic
Segmentation. Our segmentation map more closely resembles the ground truth,
indicating a more accurate understanding of the urban scene. Notably, our
method effectively distinguishes categories such as sidewalks and roads,
despite the presence of occlusions (top row) and shadows (bottom row).

##  Appendix C Supervised Setting

In Section 3.2, we mentioned that source pre-training also improves
performance in the supervised setting on the source domain itself. We show
improvements in segmentation performance on Cityscapes, when applied to
SegFormer model (base architecture of the DAFormer [12] training strategy),
see Table 15. Please observe visual results of our method in Figure 10. We
also show improvements in detection performance on BDD100K (Day) and BDD100
(Clear) when applied to Faster R-CNN (base architecture of 2PCNet [17]
training strategy), see Table 16 and Table 17. BDD100K (Day) includes both
clear and bad weather images taken during the day, while BDD100K (Clear)
includes both day and night images taken in clear weather.

Table 15: Supervised Setting: Cityscapes Semantic Segmentation. Tested on
Cityscapes Val. Our instance-level image warping also improves segmentation on
source domain by +1.5 mIoU (along with target domain, See results on Table 4
and 5 in the main manuscript), implying that learned backbone features are
better.

Method | mIoU | road |  | side
---
walk
building | wall | fence | pole |  | traffic
---
light
| traffic
---
sign
| vege-
---
tation
terrain | sky | person | rider | car | truck | bus | train |  | motor
---
cycle
bike
SegFormer | 75.3 | 98.0 | 83.4 | 91.8 | 59.6 | 59.6 | 57.5 | 64.0 | 74.4 | 91.9 | 62.8 | 94.6 | 77.6 | 56.0 | 93.7 | 81.6 | 81.4 | 70.1 | 59.2 | 73.3
\+ Ours w/ Sta. Prior | 76.1 | 97.9 | 83.6 | 91.9 | 58.0 | 58.0 | 57.7 | 63.3 | 73.9 | 91.7 | 64.4 | 94.4 | 77.7 | 57.4 | 93.8 | 81.0 | 87.7 | 80.3 | 60.7 | 72.8
\+ Ours w/ Geo. Prior | 76.5 | 98.0 | 84.3 | 92.2 | 61.4 | 57.6 | 59.3 | 64.7 | 74.0 | 91.9 | 65.3 | 94.7 | 78.2 | 55.3 | 93.9 | 83.1 | 86.5 | 81.1 | 60.1 | 72.1
\+ Ours w/ Inst. | 76.8 | 98.1 | 84.8 | 92.2 | 59.9 | 58.3 | 59.6 | 65.1 | 75.4 | 92.3 | 66.2 | 94.8 | 78.2 | 55.3 | 94.2 | 82.0 | 85.7 | 81.3 | 61.8 | 74.4

Table 16: Supervised Setting: BDD100K (Day) Object Detection. Tested on
BDD100K Day Val. Day images here include both images captured in good and bad
weather. As shown by mAP50 (overall and per category), our method improves
detection performance in the source domain while our saliency guidance is
competitive with other saliency priors.

Method | mAP | mAP50 | mAP75 | mAPs | mAPm | mAPl | person | rider | car | truck | bus |  | motor
---
cycle
bike |  | traffic
---
light
| traffic
---
sign
FRCNN | 30.1 | 56.4 | 28.1 | 13.9 | 37.6 | 51.0 | 64.0 | 50.7 | 80.3 | 62.5 | 62.9 | 45.3 | 49.7 | 66.7 | 69.8
\+ Ours w/ Sta. Prior | 30.9 | 57.1 | 28.6 | 14.4 | 38.9 | 53.4 | 65.7 | 53.1 | 80.9 | 62.7 | 63.0 | 48.8 | 50.8 | 69.2 | 71.2
\+ Ours w/ Geo. Prior | 31.1 | 57.9 | 28.3 | 14.5 | 38.5 | 52.7 | 66.3 | 53.6 | 80.7 | 62.5 | 62.8 | 48.1 | 52.9 | 68.4 | 71.4
\+ Ours w/ Inst. | 30.7 | 57.2 | 27.9 | 14.5 | 38.4 | 52.8 | 66.4 | 53.3 | 80.8 | 62.4 | 63.7 | 47.7 | 51.7 | 68.6 | 71.1

Table 17: Supervised Setting: BDD100K (Clear) Object Detection. Tested on
BDD100K Clear Val. Clear images here include both day and night images. As
shown by mAP50 (overall and per category), our method improves detection
performance in the source domain while our saliency guidance is competitive
with other saliency priors.

Method | mAP | mAP50 | mAP75 | mAPs | mAPm | mAPl | person | rider | car | truck | bus |  | motor
---
cycle
bike |  | traffic
---
light
| traffic
---
sign
FRCNN | 25.4 | 49.6 | 22.5 | 12.2 | 30.3 | 44.2 | 59.3 | 38.8 | 76.5 | 53.2 | 54.7 | 43.1 | 45.6 | 56.4 | 68.7
\+ Ours w/ Sta. Prior | 26.0 | 50.2 | 22.9 | 11.9 | 31.2 | 44.5 | 59.2 | 38.8 | 76.7 | 54.5 | 55.7 | 45.5 | 46.0 | 56.6 | 69.2
\+ Ours w/ Geo. Prior | 25.9 | 50.3 | 22.8 | 12.0 | 31.0 | 44.8 | 59.4 | 41.1 | 76.7 | 53.7 | 56.1 | 42.7 | 46.9 | 56.7 | 69.3
\+ Ours w/ Inst. | 25.9 | 50.1 | 22.8 | 12.2 | 31.0 | 43.7 | 59.7 | 37.7 | 76.6 | 54.4 | 56.9 | 43.7 | 46.0 | 56.5 | 69.1

##  Appendix D Additional Domain Adaptations

Cityscapes $\rightarrow$ Foggy Cityscapes Table 18 shows the result for domain
adaptative object detection from Cityscapes $\rightarrow$ Foggy Cityscapes.

Table 18: Additional Domain Adaptations: Cityscapes $\rightarrow$ Foggy
Cityscapes.. Cityscapes and Foggy Cityscapes both feature identical
backgrounds, resulting in a shared scale bias. However, the synthetic fog
introduced in Foggy Cityscapes does not accurately mimic the properties of
real fog, thereby posing less of a challenge to segmentation algorithms. As a
result, the baseline model already attains high scores on the Foggy Cityscapes
dataset, making it difficult for our warping method to yield substantial
additional improvements.

Method | mIoU | road |  | side
---
walk
building | wall | fence | pole |  | traffic
---
light
| traffic
---
sign
| vege-
---
tation
terrain | sky | person | rider | car | truck | bus | train |  | motor
---
cycle
bike
DAFormer | 74.7 | 97.8 | 83.0 | 88.7 | 60.4 | 59.4 | 57.4 | 60.7 | 72.8 | 89.2 | 64.9 | 81.2 | 76.6 | 54.4 | 93.2 | 75.9 | 86.9 | 80.7 | 62.8 | 73.8
\+ Ours w/ Inst. | 75.5 | 98.0 | 84.2 | 89.7 | 59.7 | 61.9 | 57.7 | 60.4 | 73.3 | 89.4 | 62.2 | 82.4 | 76.7 | 56.3 | 92.8 | 82.2 | 88.0 | 82.0 | 63.5 | 73.9

##  Appendix E Additional Analysis

Pixel Accuracy Visualizations Figure 11 show per-pixel accuracy visualization.
For both Cityscapes [4] and ACDC [32], a noticeable predominance of red over
blue is observed, showing a clear advantage of our method upon DAFormer [12]
for semantic segmentation.

Figure 11: Visualization of the per-pixel accuracy difference between the
baseline and our method. Red indicates our method is better, whereas blue
means the baseline is better. The comparison for the ‘sky’ class has been
omitted, and the semantic segmentation maps are reshaped to 256 $\times$ 256
to improve the quality of the visualization. We can see that our method
outperforms the baseline across most pixels, with the exception of right-hand
side sidewalk pixels in the ACDC dataset (blue band) due to a significant
disparity in sidewalk width compared to the Cityscapes dataset.

BDD100K Clear $\rightarrow$ Rainy Object Detection. Qualitative comparisons
are shown in Figure 12. Our method demonstrates superior object detection
under rainy conditions by accurately identifying vehicles and minimizing false
positives, such as the misidentified pedestrians and cars evident in the
2PCNet [17] predictions.

Figure 12: Additional Analysis: BDD100K (clear $\rightarrow$ rainy) Object
detection. Our method demonstrates superior object detection under rainy
conditions by accurately identifying vehicles and minimizing false positives,
such as the misidentified pedestrians (persons) and cars evident in 2PCNet
[17] predictions.

Cityscapes $\rightarrow$ DarkZurich Semantic Segmentation. Qualitative
comparisons are shown in Figure 13. The proposed method exhibits a
segmentation output that is more closely aligned with groundtruth,
particularly while predicting road boundaries and consistently identification
of urban elements such as sidewalk, terrain, and traffic signs. This indicates
that our method has superior domain adaptation capabilities in challenging
low-light conditions.

Figure 13: Additional Analysis: Cityscapes $\rightarrow$ DarkZurich Semantic
Segmentation. The proposed method exhibits a segmentation output that is more
closely aligned with the groundtruth, particularly in the accurate predictions
of road boundaries and consistent identification of urban elements such as
sidewalk, terrain, and traffic signs. This indicates that our method has
superior domain adaptation capabilities in the challenging low-light
conditions.

##  Appendix F Additional Technical Details

Ground Truth Segmentation to Boxes: Instance-based image warping requires
bounding boxes, which are not provided in some semantic segmentation
benchmarks like GTA [28]. To circumvent this, we generate “from-seg” bounding
boxes from ground truth semantic segmentation maps. We do this by identifying
connected components that represent individual instances of foreground
categories — specifically, traffic lights, traffic signs, persons, riders,
cars, trucks, buses, trains, motorcycles, and bikes.

For each “from-seg” instance, we compute the bounding boxes by finding the
minimum enclosing axis-aligned rectangle. These “from-seg” bounding boxes are
then employed for instance-level image warping in a similar manner to ground
truth bounding boxes.