# Unsupervised Domain Adaptation
for Visual Navigation

Shangda Li∗, Devendra Singh Chaplot, Yao-Hung Hubert Tsai, Yue Wu,
Louis-Philippe Morency, Ruslan Salakhutdinov
Carnegie Mellon University
<EMAIL_ADDRESS><EMAIL_ADDRESS>equal contribution

###### Abstract

Advances in visual navigation methods have led to intelligent embodied
navigation agents capable of learning meaningful representations from raw RGB
images and perform a wide variety of tasks involving structural and semantic
reasoning. However, most learning-based navigation policies are trained and
tested in simulation environments. In order for these policies to be
practically useful, they need to be transferred to the real-world. In this
paper, we propose an unsupervised domain adaptation method for visual
navigation. Our method translates the images in the target domain to the
source domain such that the translation is consistent with the representations
learned by the navigation policy. The proposed method outperforms several
baselines across two different navigation tasks in simulation. We further show
that our method can be used to transfer the navigation policies learned in
simulation to the real world.

##  1 Introduction

In the past few years, a lot of progress has been made in learning to navigate
from first-person RGB images. Reinforcement learning have been applied to
train navigation policies to navigate to goals according to coordinates [18,
9, 44], images [56], object labels [18, 51], room labels [48, 49] and language
instructions [20, 11, 2, 15, 12, 43]. However, such navigation policies are
predominantly trained and tested in simulation environments. Our goal is to
have such navigation capabilities in the real-world. While some progress has
been made towards moving from game-like simulation environments to more
realistic simulation environments based on reconstructions [50, 6, 38] or 3D
modeling [26], there is still a significant gap between simulation
environments and real-world.

Training the above navigation policies in the real-world has not been possible
as current reinforcement learning methods typically require millions of
samples for training. Even if we parallelize the training across multiple
robots, it will still require multiple weeks on training with constant human
supervision due to safety concerns and battery limitations. This makes real-
world training practically infeasible and leaves us with the other option of
transferring models trained in simulation to the real-world, which highlights
the importance of domain adaptation methods.

Among domain adaptation techniques, unsupervised methods are favorable because
it is extremely expensive to collect parallel data for the purpose of visual
navigation. It essentially requires reconstructing real-world scenes in the
simulator separately for all possible scenarios one might deploy the
navigation model in such as different lighting conditions, time of day, indoor
vs outdoor, weather conditions, and so on. Reconstructing real-world scenes is
a tedious job requiring specialized cameras and significant human effort.
Unsupervised learning methods have the potential to overcome this difficulty
since they require only a few real-world images taken by regular cameras.

One possible solution involves using unsupervised image translation techniques
to translate visual perception from simulation to real-world and adapt the
navigation policy learned in simulation to the real-world. Although there
already exists a rich amount of prior work in unsupervised image translation
techniques that transfer images from one domain to another [54, 28, 22], prior
techniques are not well suited for navigation since the image translations are
agnostic of the navigation policy and instead focus on photo-realisticity and
clarity.

Figure 1: PBIT. The proposed policy-based image translation for unsupervised
visual navigation adaptation.

In this paper, we propose an unsupervised domain adaptation method for
transferring navigation policies from simulation to the real-world, by
unsupervised image translation subject to the constraint that the image
translation respects agent’s policy. In order to learn policy-based image
translation (PBIT) in an unsupervised fashion, we devise a disentanglement of
content and style in images such that the representations learnt by the
navigation policy are consistent for images with the same content with
different styles. See Figure 1 for the illustration of PBIT. Our experiments
show that the proposed method outperforms the baselines in transferring
navigation policies for different tasks between two simulation domains and
from simulation to the real-world.

##  2 Related Work

Simulation to real-world (Sim-to-Real) transfer of visual navigation policies
requires the adaptation for both visual perception and agent’s policy. Among
its wide range of relevant literature, we focus on discussing related work on
visual navigation, visual domain adaptation and policy transfer.

Visual Navigation. Visual Navigation has been widely studied in robotics for
over two decades. Bonin-Font et al. [4] provide an in-depth survey of visual
navigation methods in classical robotics. In the past few years, there has
been an increasing focus on learning-based methods for visual navigation. This
includes methods which tackle navigation tasks primarily requiring geometrical
scene understanding such as the pointgoal task [18, 1] where the relative
coordinate to the goal is given and the exploration task [13, 14, 9] where the
objective is to maximize the explored area. There has also been a lot of work
on navigation tasks involving more semantics such as image-goal [56, 10],
object-goal [51, 47, 29, 8, 7], high-level language goal [20, 11] and low-
level language instructions [2, 15].

While performance on semantic navigation tasks is still far from perfect even
in simulation, recent improvements in both visual simulation quality [33, 50,
6, 38] as well as algorithms [34, 9, 44] have led to impressive results on
geometric navigation tasks. However, most of the above works train navigation
policies using reinforcement or imitation learning in simulation and test on
different scenes in the same domain in the simulator. Some prior works which
tackle sim-to-real transfer for navigation policies directly transfer the
policy trained in simulation to the real-world without any domain adaptation
technique [18, 9]. We show that the proposed domain adaptation method can lead
to large improvements over direct policy transfer.

Visual Domain Adaptation. Simulation and real-world can be viewed as two
distinct visual domains, and adapting their visual perceptions can be regarded
as an image-to-image translation task. Thanks to the success of Generative
Adversarial Networks (GANs) [16] for matching cross-domain distribution, we
are able to adapt an image across domains without changing its context. For
example, pix2pix [23] changes only the style of an image (e.g., photograph
$\rightarrow$ portrait) while preserving its context (e.g., the same face of a
person). We note that, for Sim2Real navigation, some amount of context should
be preserved across domains, such as the obstacles and walls, to prevent
collisions.

If we have access to the paired cross-domain images, then pix2pix [23] and
BicycleGAN [55] serve as good candidates to model the context-preserving
adaptation. However, the paired data between simulation and real-world is
notoriously hard to collect [40] or even do not exist (e.g., we cannot always
build simulators for new environments). To tackle this challenge, numerous
visual domain adaptation approaches [39, 37, 54, 24, 52, 28] have been
proposed to relax the constraint of requiring paired data during training
time. Nevertheless, the above methods still assume one-to-one correspondence
across domains. As an example, these models can only generate the same target-
domain image given a source-domain image. We argue that it is more realistic
to assume a many-to-many mapping between simulation and real-world.

To learn multimodal mappings without paired data, prior works [22, 27]
disentangle the context and style of an image. Precisely, they assume the
context is shared across domains and the styles are specific to each domain.
Note that these models focus on realistic image generation, and hence it
remains unclear on how image translation benefits cross-domain visual
navigation. To further bridge the gap between navigation and image
translation, our key idea is to ensure the agent’s navigation policy be
consistent under domain translation. As a consequence, we propose to enforce
constraints such that the agent’s policy is only inferred from the shared
context across simulation and real-world.

Policy Transfer. Existing works on sim-to-sim or sim-to-real policy transfer
require domain knowledge specific to certain environments and tasks and extra
human supervision. [53] employs semantic loss and shift loss for consistent
generation to facilitate the domain translation. [32] build a new simulator
with diverse layouts populated with diverse furniture placements and defines
several auxiliary tasks that are specific to the Collision-Free Goal Reaching
task. [30] trains the domain translation module of its driving agent on human-
labeled segmentation dataset to generalize its learned policy from one domain
to the other. [5] utilizes customized simulator to improve a policy’s real-
world performance, given a policy already trained in the real world, which is
different from our setting where the policy is trained in the simulation and
then gets transferred to the real-world. [17] targets similar visual
navigation problems and defines auxiliary tasks for better sim-to-sim
transfer, which is complementary to our contribution. [3] relies on Supervised
Learning for Visitation Prediction which requires human-supervised
trajectories of both the real world and the simulation.

##  3 Methods

Denote the source domain as $(\mathcal{S}^{s},\mathcal{A},P^{s})$ and the
target domain as $(\mathcal{S}^{t},\mathcal{A},P^{t})$. $\mathcal{S}$ is the
state space, $\mathcal{A}$ is the set of actions, and
$P:\mathcal{S}\times\mathcal{A}\times\mathcal{S}\to\mathbb{R}$ is the
transition probability distribution. Note that we assume action spaces
$\mathcal{A}$ are shared across domains. Let
$\pi^{s}:\mathcal{S}^{s}\to\mathcal{A}$ be a navigation policy in the source
domain $s$ (the navigation policy is given). For our task setup, we have
access to some target-domain images $I^{t}\in\mathcal{S}^{t}$ during training,
but we cannot perform target-domain policy ($\pi^{t}$) training. Our objective
is to learn a many-to-many mapping $F:\mathcal{S}^{t}\to\mathcal{S}^{s}$ such
that the navigation policy under the source-to-target mapping,
$\pi^{t}(I^{t})=\pi^{s}(F(I^{t}))$, is effective in the target domain $t$.
Under Sim2Real setting, the source domain refers to simulator and the target
domain refers to real-world. Unless specified otherwise, we abbreviate
$\pi^{s}$ as $\pi$ for the rest of the paper.

Figure 2: Policy Decomposition. The Task-specific Navigation Policy ($\pi$)
can be sequentially decomposed into a Policy Feature Extractor ($F_{\pi}$) and
a Action Policy ($A_{\pi}$) such that $F_{\pi}$ extracts all task-specific
features (${\bf r}_{I}$) and throws away all domain-specific features from the
input image ($I$) and $A_{\pi}$ learns an action distribution function over
the task-specific features.

###  3.1 Policy Decomposition

As our objective is to transfer the task-specific navigation policy across
domains, we assume that the task itself is domain-invariant. As a consequence,
given a policy $\pi$ (for the navigation task in the source domain), we assume
that some intermediate task-specific representation inferred by the policy is
invariant from the source to the target domain. For example, a simple obstacle
avoidance navigation policy would extract task-specific and domain-invariant
features such as distance to obstacles at various angles and then learn a
policy over these features. Let $\pi$ be sequentially decomposed into a Policy
Feature Extractor ($F_{\pi}$) and a Action Policy ($A_{\pi}$). $F_{\pi}$
extracts all task-specific features (${\bf r}_{I}$ with $I$ indicating the
input image) and throws away all domain-specific features in the input image.
$A_{\pi}$ learns an action distribution function over the task-specific
features. We illustrate the policy decomposition in Figure 2.

###  3.2 Policy-based Consistency Loss

Recall that our objective is to learn an image translation model
$F:\mathcal{S}^{t}\to\mathcal{S}^{s}$, such that
$\pi^{t}(I^{t})=\pi^{s}(F(I^{t}))$. Based on the policy decomposition
described above, different translations of the same target-domain image would
have similar task-specific features. Precisely, if $I^{s}_{1}$ and $I^{s}_{2}$
are the translated images to the source domain from the same target-domain
image $I^{t}$, then $F_{\pi}(I^{s}_{1})\approx F_{\pi}(I^{s}_{2})$.

Figure 3: Policy-based Consistency Loss. Since the task is domain-invariant,
task-specific representations obtained from different domain-specific styles
but the same domain-invariant content should be similar.

To achieve the above policy consistency in an unsupervised fashion, we take
inspiration from style and content-based unsupervised methods designed for
image translation [22]. We assume that each image can be decomposed into a
domain-invariant content representation (${\bf c}$) and a domain-specific
style representation (${\bf s}$). Let $E_{I}^{s}$ be an Image Encoder for
domain $s$ which encodes an image ($I$) to domain-invariant content (${\bf
c}$) and domain-specific style (${\bf s}^{s}$): $E_{I}^{s}(I)=({\bf
c}_{I},{\bf s}^{s}_{I})$. On the contrary, let $D_{I}^{s}$ be an Image Decoder
which is the inverse of the Image Encoder: $D_{I}^{s}({\bf c}_{I},{\bf
s}^{s}_{I})=I$.

Since we assume the navigation task is domain-invariant, all the the task-
specific features are a subset of content representation ${\bf r}_{I}\in{\bf
c}_{I}$. Therefore, images generated from different styles but same content
should lead to the same task-specific features as shown in Figure 3. We
operationalize this idea using the following policy-based consistency loss:

$\small\mathcal{L}_{pol}=\mathbb{E}_{{\bf c}_{I^{t}}:({\bf c}_{I^{t}},\\_)\in
E_{I}^{t}(I^{t}),I^{t}\sim\mathcal{S}^{t},{\bf s}_{1}^{s}\sim p({\bf
s}^{s}),{\bf s}_{2}^{s}\sim p({\bf s}^{s})}[||F_{\pi}(D^{s}_{I}({\bf
c}_{I^{t}},{\bf s}_{1}^{s}))-F_{\pi}(D^{s}_{I}({\bf c}_{I^{t}},{\bf
s}_{2}^{s}))||_{1}]$ (1)

with ${\bf s}_{1}^{s}$ and ${\bf s}_{2}^{s}$ being two distinct styles sampled
from the the prior distribution $p({\bf s}^{s}):=\mathcal{N}({\bf 0},{\bf
I})$.

Note that in the above equation, $F_{\pi}(\cdot)$ is part of the given
navigation policy. We assume the navigation policy is trained before deciding
the target domain; hence, $F_{\pi}(\cdot)$ is fixed during the domain
adaptation phase. This ensures that the presented policy-based image
translation (PBIT) can be used for transferring a policy across domains
(potentially not anticipated during policy training) without re-training the
the navigation policy.

###  3.3 Reconstruction and Adversarial Loss

Using just policy-based consistency loss would make decoder $D_{I}$ ignore the
style and decode based only on the content. Inspired by prior work [54, 22],
to encourage the content to be domain-invariant and style representations to
be domain-specific, we adopt the following image and latent representation
reconstruction losses, and use $\mathcal{N}({\bf 0},{\bf I})$ for the prior
distributions of styles $p({\bf s}^{s})$ and $p({\bf s}^{t})$:

$\small\begin{split}\mathcal{L}_{im\\_rec}=&\mathbb{E}_{I^{t}\sim\mathcal{S}^{t}}[||D_{I}^{t}(E_{I}^{t}(I^{t}))-I^{t}||_{1}]+\mathbb{E}_{I^{s}\sim\mathcal{S}^{s}}[||D_{I}^{s}(E_{I}^{s}(I^{s}))-I^{s}||_{1}],\\\
\mathcal{L}_{lat\\_rec}=&\mathbb{E}_{{\bf c}_{I^{t}}:({\bf c}_{I^{t}},\\_)\in
E_{I}^{t}(I^{t}),I^{t}\sim\mathcal{S}^{t},{\bf s}^{s}\sim p({\bf
s}^{s})}[||E_{I}^{s}(D_{I}^{s}({\bf c}_{I^{t}},{\bf s}^{s}))-({\bf
c}_{I^{t}},{\bf s}^{s})||_{1}]\\\ &+\mathbb{E}_{{\bf c}_{I^{s}}:({\bf
c}_{I^{s}},\\_)\in E_{I}^{s}(I^{s}),I^{s}\sim\mathcal{S}^{s},{\bf s}^{t}\sim
p({\bf s}^{t})}[||E_{I}^{t}(D_{I}^{t}({\bf c}_{I^{s}},{\bf s}^{t}))-({\bf
c}_{I^{s}},{\bf s}^{t})||_{1}].\end{split}$ (2)

We also use adversarial losses to match the distribution of images to their
respective domains. Let ${\rm Dis}^{s}$ and ${\rm Dis}^{t}$ be the
discriminators for the source and target domains:

$\begin{split}\mathcal{L}_{adv}=&\mathbb{E}_{{\bf c}_{I^{t}}:({\bf
c}_{I^{t}},\\_)\in E_{I}^{t}(I^{t}),I^{t}\sim\mathcal{S}^{t},{\bf s}^{s}\sim
p({\bf s}^{s})}[\log{\rm Dis}^{s}(D_{I}^{s}({\bf c}_{I^{t}},{\bf s}^{s}))]\\\
&+\mathbb{E}_{{\bf c}_{I^{s}}:({\bf c}_{I^{s}},\\_)\in
E_{I}^{s}(I^{s}),I^{s}\sim\mathcal{S}^{s},{\bf s}^{t}\sim p({\bf
s}^{t})}[\log{\rm Dis}^{t}(D_{I}^{t}({\bf c}_{I^{s}},{\bf s}^{t}))]\\\
&+\mathbb{E}_{I^{s}\sim\mathcal{S}^{s}}[\log(1-{\rm
Dis}^{s}(I^{s}))]+\mathbb{E}_{I^{t}\sim\mathcal{S}^{t}}[\log(1-{\rm
Dis}^{t}(I^{t}))].\end{split}$ (3)

Putting everything together, our overall objective is

$\mathcal{L}_{full}:=\lambda_{pol}\mathcal{L}_{pol}+\lambda_{im\\_rec}\mathcal{L}_{im\\_rec}+\lambda_{lat\\_rec}\mathcal{L}_{lat\\_rec}+\lambda_{adv}\mathcal{L}_{adv},$
(4)

where $\lambda$s are hyper-parameters controlling the weight of each loss
during training. The cross-domain image translation model consists of
$D_{I}^{t},E_{I}^{t},D_{I}^{s}$ and $E_{I}^{s}$ as described above and the
optimization admits a mix-max objective:

$D_{I}^{t},E_{I}^{t},D_{I}^{s},E_{I}^{s}=\arg\min_{D_{I}^{t},E_{I}^{t},D_{I}^{s},E_{I}^{s}}\,\,\max_{{\rm
Dis}^{t},{\rm Dis}^{s}}\mathcal{L}_{full}.$ (5)

##  4 Experimental Setup

We conduct two sets of experiments to test the domain adaptation of navigation
policies in Sim-to-Sim and Sim-to-Real settings. In the Sim-to-Sim
experiments, we adapt navigation policies trained in the Gibson [50] domain to
the Replica [38] domain in the Habitat Simulator [33]. For Sim-to-Real, we
adapt policies trained in Gibson to real-world office scenes. For training the
domain adaptation models, we collect 7200 images randomly in 72 training
scenes in Gibson, 7200 images in 18 test scenes in Replica, and 1125 images in
the real-world.

We study domain adaptation for two navigation tasks, PointGoal and Exploration
for our experiments. The PointGoal task [2, 33] involves navigating to the
target location specified using point coordinates. Success Rate and Success
weighted by Path Length (SPL) [2] as evaluation metrics for PointGoal. An
episode is considered successful if the agent is within $0.2m$ radius of the
goal location at the end of the episode. The Exploration task [13, 9] involves
maximizing the coverage or the explored area within a fixed time budget of 500
steps. A traversable point is defined to be explored by the agent if it is in
the field-of-view of the agent and less than $3m$ away from the agent. We use
two evaluation metrics, the absolute coverage area in $m^{2}$ (Explored Area)
and the proportion of area explored in the scene (Explored Ratio).

For both the tasks, the agent has two sensors: RGB camera ($3\times 256\times
256$) and an odometry sensor ($x-y$ coordinates and orientation). For the
PointGoal task, the odometry sensor is used to compute the relative distance
and angle of the pointgoal at each time step. The action space consists of 3
actions: move_forward (0.25$m$), turn_left ($10\degree$), turn_right
($10\degree$).

###  4.1 Navigation Policy

The source domain navigation policy ($\pi_{s}$) is trained using reinforcement
learning. The reward for the PointGoal task is the decrease in geodesic
distance to the point goal, and for the Exploration task is the increase in
the explored area. The navigation policy is decomposed into a Policy Feature
Extractor ($F_{\pi}$) and an Action Policy ($A_{\pi}$) as shown in Fig 2.
$F_{\pi}$ is based on the 18-layer ResNet [19]. $F_{\pi}$ outputs
128-dimensional policy-related representations ${\bf r}_{I}$ given RGB images
of shape $3\times 256\times 256$. $A_{\pi}$ is based on a 2-layer GRU, which
takes ${\bf r}_{I}$ along with relative distance and angle of the point goal
(in the PointGoal task) or readings from the odometry sensor (in the
Exploration task) as input. We train a separate policy for each task in the
source Gibson domain using 72 scenes in the training set provided by [33]. The
policies are trained using PPO [36] for 30 million frames. In the PointGoal
task, the agent stops when the relative distance of the PointGoal is less than
$0.2m$, whereas, in the Exploration task, the agent explores for the maximum
episode length of 500 steps. More architecture and hyperparamater details for
navigation policy training are provided in supplementary.

###  4.2 Implementation Details

Given a navigation policy, we use the proposed method for domain adaptation
from Gibson to Replica and vice versa. The image encoders and decoders used in
PBIT consist of several convolutional layers and residual blocks. The exact
architecture is described in supplementary. Regarding hyperparameters, we use
$\lambda_{im\\_rec}=10,\lambda_{lat\\_rec}=1,\lambda_{adv}=1$ for all
experiments. We scale $\lambda_{pol}$ based on the mean of image features for
each task:
$\lambda_{pol}=\frac{1}{\mathbb{E}_{I^{s}\sim\mathcal{S}^{s}}[||E_{\pi}(I^{s})||_{1}]}$,
where $\mathbb{E}_{I^{s}\sim\mathcal{S}^{s}}[||E_{\pi}(I^{s})||_{1}]$ is
estimated using the Gibson images from the domain translation dataset. We use
the Adam optimizer [25] with $0.0001$ initial learning rate, $\beta_{1}=0.5$
and $\beta_{2}=0.999$. The learning rate is halved every 100K iterations. We
train the proposed model and all the baselines for 500K iterations.

###  4.3 Baselines

We compare the proposed PBIT model against the following baselines for domain
adaptation:
1) Direct Transfer: This is the most common method of transferring navigation
policies across domains, which involves directly testing the policy in the
target domain without any fine-tuning.
2) CycleGAN [54] is a competitive and popular unsupervised image translation
method. This method is designed for static image translation and is agnostic
to the navigation policy.
3) PBIT w.o. Policy Loss: This method is an ablation of the proposed method
without the policy-based consistency loss. This ablation is conceptually very
similar to prior works based on style and content disentanglement such as [22,
27]. We use the architecture and hyperparameters from the PBIT model for this
ablation to quantify the effect of the policy-based consistency loss on
performance.

##  5 Results

Table 1: Results. Comparisons between the proposed Policy-Based Image Translation (PBIT) and baselines on the PointGoal and Exploration tasks when transferred from the Gibson to the Replica domain. |  | PointGoal |  | Exploration
---|---|---|---|---
|  | SPL | Success Rate |  | Explored Ratio | Explored Area
Direct Transfer |  | 0.505 | 0.688 |  | 0.832 | 22.9
CycleGAN |  | 0.605 | 0.803 |  | 0.868 | 24.3
PBIT w.o. Policy Loss |  | 0.669 | 0.852 |  | 0.879 | 24.6
PBIT |  | 0.712 | 0.881 |  | 0.897 | 25.3

###  5.1 Gibson to Replica

For evaluation in Replica, we generate a separate test set of 900 episodes (50
episodes in each of the 18 scenes) for both the PointGoal and Exploration
tasks. The episodes are sampled using random starting locations and aggressive
rejection sampling of near-straight-line episodes as in prior work [33, 9].
The performances of our method and all the baselines for both the tasks are
presented in Table 1. PBIT outperforms all the baselines on both the tasks. It
improves the SPL from 0.605 to 0.712 for PointGoal and Explored Ratio from
0.868 to 0.897 for Exploration as compared to the CycleGAN baseline. There’s a
considerable difference between PBIT and the ablation, indicating the
importance of policy-based consistency loss. PBIT performed better than all
the baselines consistently across 3 different runs with a small standard
deviation of 0.007 SPL and 0.004 Explored Ratio. The Explored Ratios of all
the methods in Table 1 are high on an absolute level because the Replica
scenes are relatively small usually having one or two rooms. Just turning on
the spot leads to an explored ratio of 0.436.

In Figure 4, we visualize an example trajectory for the PointGoal task using
the proposed method PBIT (Fig 4 above) and the Direct Transfer baseline (Fig 4
below) for the same episode specification. The figure shows the images
observed by the agent in the target domain, the translated images, and a top-
down map (not visible to the agent) showing the point goal coordinates and the
agent’s path. The PBIT model successfully reaches the PointGoal within 46
steps while the Direct Transfer baseline is unable to reach the goal. The
image translations indicate that policy-relevant characteristics of the image
such as corners of obstacles, walls and free space are preserved during the
translation.

Figure 4: Trajectory Comparison between PBIT and the Direct Transfer Baseline
on the PointGoal Task in Replica.  The upper half of the figure is the
trajectory of PBIT: the agent successfully navigates from a corner of the
apartment to the fridge in 46 steps, by seeing the translated images by PBIT.
The agent takes almost the shortest path possible, as shown in the Top-Down
Map (not visible to the agent). The lower half of the figure is the trajectory
of the Direct Transfer baseline on the same test episode. The Direct Transfer
agent fails to navigate to the target location and gets lost, even after 460
steps.

###  5.2 Gibson to Real-world

For the real-world experiments, we conduct 18 trials each for the proposed
method and the Direct Transfer baseline for the PointGoal task. We transfer
the navigation policy to a LoCoBot [45] using the PyRobot API [31] for both
the methods. We conduct trials in 2 scenes, Seen and Unseen. 151 images among
the set of the 1125 real-world images used for training the PBIT model were
sampled in the Seen scene, whereas none of the images were sampled from the
Unseen scene. We terminate the episode if the robot collides with an obstacle
and count it as a failure. We allow a maximum episode length of 99 steps in
the real-world experiments.

Each trial specification and the corresponding results are presented in Table
2. PBIT achieves an absolute improvement of $55\%$ ($70\%$ vs $15\%$) in
success rate over the Direct Transfer baseline across all the trials. PBIT
also has a much lower collision rate as compared to the baseline.
Surprisingly, the PBIT model achieves 100% success rate in the Seen scene,
achieving an absolute improvement of $78.8\%$ ($100\%$ vs $22.2\%$). These
results indicate that the navigation policy can be reliably transferred to the
real-world using the proposed method given access to a few images in the real-
world scene. Even in the unseen scene, PBIT leads to a large absolute
improvement of $22.3\%$ ($55.6\%$ vs $33.3\%$) over the Direct Transfer
baseline.

In Figure 5, we show an example of a successful trajectory in the Unseen real-
world scene using PBIT. It shows some of the images seen by the agent during
the trajectory, the corresponding translations, and a third-person view of the
robot. The trajectory shows the PBIT is able to successfully navigate around
the blue chair obstacle to reach the pointgoal. The image translations shown
in the figure also indicate that the model generates good translations similar
to images in the Gibson domain. For example, the dark grey carpet floors in
the office space scenes in the real-world are successfully translated to brown
floors, representative of wooden floors of apartment scenes in the Gibson
domain. Similarly, wooden doors and staircases in the real-world are
translated to off-white walls which are common in Gibson scenes. At the same
time, navigation relevant details such as the boundary between the floor and
walls and other obstacles, are preserved during translation.

Figure 5: Sample Real World Trajectory on PointGoal Task. Figure contains raw inputs from Real World (row one), translated Gibson images by PBIT (row 2), and a third-person perspective from the back. The PBIT agent successfully reached it’s destination (a trash can) by avoiding an obstacle (a chair) in its way.  Table 2: Real-world results. Table comparing the performance of the Direct Transfer baseline and the proposed method PBIT across 18 trials in two scenes in the real-world. The training set for PBIT consists of some images sampled from Scene 1: Corridors (Seen), but no image from Scene 2: Public kitchen area (Unseen). The episode specification (starting relative distance and angle to the PointGoal) are shown in the left columns. Identical starting locations and episode specifications were used to evaluate both the methods. The performance of the baseline and PBIT are shown in the center and right, respectively. Success is abbreviated as Succ. Episode Specification |  | Baseline: Direct Transfer |  | PBIT
---|---|---|---|---
Ep No |  Dist ($m$) |  Angle ($\degree$) |  | Steps | Collision |  | Final
---
Dist
Succ. |  | Steps | Collision. |  | Final
---
Dist
Succ.
Scene 1 (Seen): Corridors
1 | 2.83 | 45.00 |  | 99 | 0 | 3.41 | 0 |  | 21 | 0 | 0.15 | 1
2 | 4.47 | 333.43 |  | 99 | 0 | 4.68 | 0 |  | 41 | 0 | 0.02 | 1
3 | 4.00 | 0.00 |  | 99 | 0 | 4.43 | 0 |  | 41 | 0 | 0.13 | 1
4 | 5.39 | 21.80 |  | 10 | 1 | 4.60 | 0 |  | 32 | 0 | 0.13 | 1
5 | 4.24 | 315.00 |  | 99 | 0 | 6.20 | 0 |  | 73 | 0 | 0.14 | 1
6 | 4.00 | 0.00 |  | 36 | 1 | 3.88 | 0 |  | 63 | 0 | 0.09 | 1
7 | 1.41 | 45.00 |  | 99 | 0 | 1.95 | 0 |  | 9 | 0 | 0.19 | 1
8 | 1.41 | 135.00 |  | 50 | 0 | 0.15 | 1 |  | 29 | 0 | 0.07 | 1
9 | 4.47 | 26.57 |  | 99 | 0 | 4.73 | 0 |  | 27 | 0 | 0.19 | 1
Scene Avg |  | 92 | 22.2% | 3.78 | 11.1% |  | 37.3 | 0% | 0.13 | 100%
Scene 2 (Unseen): Public kitchen area
1 | 2.00 | 0.00 |  | 32 | 1 | 0.50 | 0 |  | 36 | 0 | 0.18 | 1
2 | 2.00 | 0.00 |  | 35 | 1 | 0.56 | 0 |  | 10 | 0 | 0.04 | 1
3 | 2.24 | 333.43 |  | 31 | 0 | 0.02 | 1 |  | 16 | 0 | 0.07 | 1
4 | 2.24 | 153.43 |  | 38 | 0 | 0.08 | 1 |  | 43 | 0 | 0.09 | 1
5 | 4.12 | 345.96 |  | 80 | 1 | 4.10 | 0 |  | 44 | 1 | 1.96 | 0
6 | 4.47 | 26.57 |  | 99 | 0 | 4.01 | 0 |  | 99 | 0 | 2.85 | 0
7 | 4.47 | 26.57 |  | 99 | 0 | 4.49 | 0 |  | 70 | 0 | 0.14 | 1
8 | 5.39 | 21.80 |  | 99 | 0 | 5.50 | 0 |  | 99 | 0 | 2.73 | 0
9 | 2.83 | 45.00 |  | 99 | 0 | 3.57 | 0 |  | 99 | 0 | 2.84 | 0
Scene Avg |  | 77.5 | 33.3% | 2.54 | 22.2% |  | 59.0 | 11.1% | 1.21 | 55.6%
Overall |  | 84.8 | 27.7% | 3.16 | 16.7% |  | 48.2 | 5.6% | 0.67 | 77.8%

##  6 Conclusion

In this paper, we proposed a domain adaptation method for transferring
navigation policies from simulation to the real-world. Given a navigation
policy in the source domain, our method translates images from the target
domain to the source domain such that the translations are consistent with the
task-specific and domain-invariant representations learnt by the given policy.
Our experiments across two different tasks for domain transfer in simulation
show that the proposed method can improve the performance on the transferred
navigation policies over baselines. We also show strong performance of
navigation policies transferred from simulation to the real-world using our
method. In this paper, we considered navigation tasks involving mostly spatial
reasoning. In the future, the proposed method can be extended to navigation
tasks involving more geometric reasoning by incorporating semantic consistency
losses along with the policy consistency losses.

## References

  * [1] Peter Anderson, Angel Chang, Devendra Singh Chaplot, Alexey Dosovitskiy, Saurabh Gupta, Vladlen Koltun, Jana Kosecka, Jitendra Malik, Roozbeh Mottaghi, Manolis Savva, et al.  On evaluation of embodied navigation agents.  arXiv preprint arXiv:1807.06757, 2018.
  * [2] Peter Anderson, Qi Wu, Damien Teney, Jake Bruce, Mark Johnson, Niko Sünderhauf, Ian Reid, Stephen Gould, and Anton van den Hengel.  Vision-and-language navigation: Interpreting visually-grounded navigation instructions in real environments.  In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 3674–3683, 2018.
  * [3] Valts Blukis, Yannick Terme, Eyvind Niklasson, Ross A. Knepper, and Yoav Artzi.  Learning to map natural language instructions to physical quadcopter control using simulated flight.  In CoRL, 2019.
  * [4] Francisco Bonin-Font, Alberto Ortiz, and Gabriel Oliver.  Visual navigation for mobile robots: A survey.  Journal of intelligent and robotic systems, 53(3):263, 2008.
  * [5] Konstantinos Bousmalis, A. Irpan, Paul Wohlhart, Yunfei Bai, Matthew Kelcey, Mrinal Kalakrishnan, Laura Downs, J. Ibarz, P. Pastor, K. Konolige, S. Levine, and V. Vanhoucke.  Using simulation and domain adaptation to improve efficiency of deep robotic grasping.  2018 IEEE International Conference on Robotics and Automation (ICRA), pages 4243–4250, 2018.
  * [6] Angel Chang, Angela Dai, Thomas Funkhouser, Maciej Halber, Matthias Niessner, Manolis Savva, Shuran Song, Andy Zeng, and Yinda Zhang.  Matterport3d: Learning from rgb-d data in indoor environments.  International Conference on 3D Vision (3DV), 2017.
  * [7] Matthew Chang, Arjun Gupta, and Saurabh Gupta.  Semantic visual navigation by watching youtube videos.  arXiv preprint arXiv:2006.10034, 2020.
  * [8] Devendra Singh Chaplot, Dhiraj Gandhi, Abhinav Gupta, and Ruslan Salakhutdinov.  Object goal navigation using goal-oriented semantic exploration.  arXiv preprint arXiv:2007.00643, 2020.
  * [9] Devendra Singh Chaplot, Dhiraj Gandhi, Saurabh Gupta, Abhinav Gupta, and Ruslan Salakhutdinov.  Learning to explore using active neural slam.  In ICLR, 2020.
  * [10] Devendra Singh Chaplot, Ruslan Salakhutdinov, Abhinav Gupta, and Saurabh Gupta.  Neural topological slam for visual navigation.  In CVPR, 2020.
  * [11] Devendra Singh Chaplot, Kanthashree Mysore Sathyendra, Rama Kumar Pasumarthi, Dheeraj Rajagopal, and Ruslan Salakhutdinov.  Gated-attention architectures for task-oriented language grounding.  In Thirty-Second AAAI Conference on Artificial Intelligence, 2018\.
  * [12] Howard Chen, Alane Suhr, Dipendra Misra, Noah Snavely, and Yoav Artzi.  Touchdown: Natural language navigation and spatial reasoning in visual street environments.  In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 12538–12547, 2019.
  * [13] Tao Chen, Saurabh Gupta, and Abhinav Gupta.  Learning exploration policies for navigation.  arXiv preprint arXiv:1903.01959, 2019.
  * [14] Kuan Fang, Alexander Toshev, Li Fei-Fei, and Silvio Savarese.  Scene memory transformer for embodied agents in long-horizon tasks.  In CVPR, 2019.
  * [15] Daniel Fried, Ronghang Hu, Volkan Cirik, Anna Rohrbach, Jacob Andreas, Louis-Philippe Morency, Taylor Berg-Kirkpatrick, Kate Saenko, Dan Klein, and Trevor Darrell.  Speaker-follower models for vision-and-language navigation.  In Advances in Neural Information Processing Systems, pages 3314–3325, 2018.
  * [16] Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio.  Generative adversarial nets.  In Advances in neural information processing systems, pages 2672–2680, 2014.
  * [17] Daniel Gordon, Abhishek Kadian, D. Parikh, Judy Hoffman, and Dhruv Batra.  Splitnet: Sim2sim and task2task transfer for embodied visual navigation.  2019 IEEE/CVF International Conference on Computer Vision (ICCV), pages 1022–1031, 2019.
  * [18] Saurabh Gupta, James Davidson, Sergey Levine, Rahul Sukthankar, and Jitendra Malik.  Cognitive mapping and planning for visual navigation.  In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 2616–2625, 2017.
  * [19] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun.  Deep residual learning for image recognition.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016.
  * [20] Karl Moritz Hermann, Felix Hill, Simon Green, Fumin Wang, Ryan Faulkner, Hubert Soyer, David Szepesvari, Wojciech Marian Czarnecki, Max Jaderberg, Denis Teplyashin, et al.  Grounded language learning in a simulated 3d world.  arXiv preprint arXiv:1706.06551, 2017.
  * [21] Xun Huang and Serge Belongie.  Arbitrary style transfer in real-time with adaptive instance normalization.  In The IEEE International Conference on Computer Vision (ICCV), Oct 2017.
  * [22] Xun Huang, Ming-Yu Liu, Serge Belongie, and Jan Kautz.  Multimodal unsupervised image-to-image translation.  In Proceedings of the European Conference on Computer Vision (ECCV), pages 172–189, 2018.
  * [23] Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, and Alexei A Efros.  Image-to-image translation with conditional adversarial networks.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1125–1134, 2017.
  * [24] Taeksoo Kim, Moonsu Cha, Hyunsoo Kim, Jung Kwon Lee, and Jiwon Kim.  Learning to discover cross-domain relations with generative adversarial networks.  In Proceedings of the 34th International Conference on Machine Learning-Volume 70, pages 1857–1865. JMLR. org, 2017.
  * [25] Diederik P Kingma and Jimmy Ba.  Adam: A method for stochastic optimization.  arXiv preprint arXiv:1412.6980, 2014.
  * [26] Eric Kolve, Roozbeh Mottaghi, Winson Han, Eli VanderBilt, Luca Weihs, Alvaro Herrasti, Daniel Gordon, Yuke Zhu, Abhinav Gupta, and Ali Farhadi.  AI2-THOR: An Interactive 3D Environment for Visual AI.  arXiv, 2017.
  * [27] Hsin-Ying Lee, Hung-Yu Tseng, Jia-Bin Huang, Maneesh Singh, and Ming-Hsuan Yang.  Diverse image-to-image translation via disentangled representations.  In Proceedings of the European conference on computer vision (ECCV), pages 35–51, 2018.
  * [28] Ming-Yu Liu, Thomas Breuel, and Jan Kautz.  Unsupervised image-to-image translation networks.  In Advances in neural information processing systems, pages 700–708, 2017.
  * [29] Arsalan Mousavian, Alexander Toshev, Marek Fišer, Jana Košecká, Ayzaan Wahid, and James Davidson.  Visual representations for semantic target driven navigation.  In 2019 International Conference on Robotics and Automation (ICRA), pages 8846–8852. IEEE, 2019.
  * [30] Matthias Müller, A. Dosovitskiy, Bernard Ghanem, and V. Koltun.  Driving policy transfer via modularity and abstraction.  In CoRL, 2018.
  * [31] Adithyavairavan Murali, Tao Chen, Kalyan Vasudev Alwala, Dhiraj Gandhi, Lerrel Pinto, Saurabh Gupta, and Abhinav Gupta.  Pyrobot: An open-source robotics framework for research and benchmarking.  arXiv preprint arXiv:1906.08236, 2019.
  * [32] F. Sadeghi.  Divis: Domain invariant visual servoing for collision-free goal reaching.  ArXiv, abs/1902.05947, 2019.
  * [33] Manolis Savva, Abhishek Kadian, Oleksandr Maksymets, Yili Zhao, Erik Wijmans, Bhavana Jain, Julian Straub, Jia Liu, Vladlen Koltun, Jitendra Malik, et al.  Habitat: A platform for embodied ai research.  In ICCV, 2019.
  * [34] Alexander Sax, Jeffrey O Zhang, Bradley Emi, Amir Zamir, Silvio Savarese, Leonidas Guibas, and Jitendra Malik.  Learning to navigate using mid-level visual priors.  arXiv preprint arXiv:1912.11121, 2019.
  * [35] John Schulman, Philipp Moritz, Sergey Levine, Michael Jordan, and Pieter Abbeel.  High-dimensional continuous control using generalized advantage estimation.  arXiv preprint arXiv:1506.02438, 2015.
  * [36] John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov.  Proximal policy optimization algorithms.  arXiv preprint arXiv:1707.06347, 2017.
  * [37] Ashish Shrivastava, Tomas Pfister, Oncel Tuzel, Joshua Susskind, Wenda Wang, and Russell Webb.  Learning from simulated and unsupervised images through adversarial training.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2107–2116, 2017.
  * [38] Julian Straub, Thomas Whelan, Lingni Ma, Yufan Chen, Erik Wijmans, Simon Green, Jakob J. Engel, Raul Mur-Artal, Carl Ren, Shobhit Verma, Anton Clarkson, Mingfei Yan, Brian Budge, Yajie Yan, Xiaqing Pan, June Yon, Yuyang Zou, Kimberly Leon, Nigel Carter, Jesus Briales, Tyler Gillingham, Elias Mueggler, Luis Pesqueira, Manolis Savva, Dhruv Batra, Hauke M. Strasdat, Renzo De Nardi, Michael Goesele, Steven Lovegrove, and Richard Newcombe.  The Replica dataset: A digital replica of indoor spaces.  arXiv preprint arXiv:1906.05797, 2019.
  * [39] Yaniv Taigman, Adam Polyak, and Lior Wolf.  Unsupervised cross-domain image generation.  arXiv preprint arXiv:1611.02200, 2016.
  * [40] Eric Tzeng, Coline Devin, Judy Hoffman, Chelsea Finn, Pieter Abbeel, Sergey Levine, Kate Saenko, and Trevor Darrell.  Adapting deep visuomotor representations with weak pairwise constraints.  arXiv preprint arXiv:1511.07111, 2015.
  * [41] Dmitry Ulyanov, Andrea Vedaldi, and Victor Lempitsky.  Improved texture networks: Maximizing quality and diversity in feed-forward stylization and texture synthesis.  In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), July 2017.
  * [42] Ting-Chun Wang, Ming-Yu Liu, Jun-Yan Zhu, Andrew Tao, Jan Kautz, and Bryan Catanzaro.  High-resolution image synthesis and semantic manipulation with conditional gans.  In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2018.
  * [43] Xin Wang, Qiuyuan Huang, Asli Celikyilmaz, Jianfeng Gao, Dinghan Shen, Yuan-Fang Wang, William Yang Wang, and Lei Zhang.  Reinforced cross-modal matching and self-supervised imitation learning for vision-language navigation.  In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 6629–6638, 2019.
  * [44] Erik Wijmans, Abhishek Kadian, Ari Morcos, Stefan Lee, Irfan Essa, Devi Parikh, Manolis Savva, and Dhruv Batra.  Decentralized distributed ppo: Solving pointgoal navigation.  In ICLR, 2020.
  * [45] Christian Wögerer, Harald Bauer, Martijn Rooker, Gerhard Ebenhofer, Alberto Rovetta, Neil Robertson, and Andreas Pichler.  Locobot-low cost toolkit for building robot co-workers in assembly lines.  In International conference on intelligent robotics and applications, pages 449–459. Springer, 2012.
  * [46] Svante Wold, Kim Esbensen, and Paul Geladi.  Principal component analysis.  Chemometrics and intelligent laboratory systems, 2(1-3):37–52, 1987\.
  * [47] Mitchell Wortsman, Kiana Ehsani, Mohammad Rastegari, Ali Farhadi, and Roozbeh Mottaghi.  Learning to learn how to learn: Self-adaptive visual navigation using meta-learning.  In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 6750–6759, 2019.
  * [48] Yi Wu, Yuxin Wu, Georgia Gkioxari, and Yuandong Tian.  Building generalizable agents with a realistic and rich 3d environment.  arXiv preprint arXiv:1801.02209, 2018.
  * [49] Yi Wu, Yuxin Wu, Aviv Tamar, Stuart Russell, Georgia Gkioxari, and Yuandong Tian.  Bayesian relational memory for semantic visual navigation.  In Proceedings of the IEEE International Conference on Computer Vision, pages 2769–2779, 2019.
  * [50] Fei Xia, Amir R. Zamir, Zhi-Yang He, Alexander Sax, Jitendra Malik, and Silvio Savarese.  Gibson Env: real-world perception for embodied agents.  In Computer Vision and Pattern Recognition (CVPR), 2018 IEEE Conference on. IEEE, 2018.
  * [51] Wei Yang, Xiaolong Wang, Ali Farhadi, Abhinav Gupta, and Roozbeh Mottaghi.  Visual semantic navigation using scene priors.  arXiv preprint arXiv:1810.06543, 2018.
  * [52] Zili Yi, Hao Zhang, Ping Tan, and Minglun Gong.  Dualgan: Unsupervised dual learning for image-to-image translation.  In Proceedings of the IEEE international conference on computer vision, pages 2849–2857, 2017.
  * [53] J. Zhang, L. Tai, P. Yun, Y. Xiong, M. Liu, J. Boedecker, and W. Burgard.  Vr-goggles for robots: Real-to-sim domain adaptation for visual control.  IEEE Robotics and Automation Letters, 4(2):1148–1155, 2019.
  * [54] Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A Efros.  Unpaired image-to-image translation using cycle-consistent adversarial networks.  In Proceedings of the IEEE international conference on computer vision, pages 2223–2232, 2017.
  * [55] Jun-Yan Zhu, Richard Zhang, Deepak Pathak, Trevor Darrell, Alexei A Efros, Oliver Wang, and Eli Shechtman.  Toward multimodal image-to-image translation.  In Advances in neural information processing systems, pages 465–476, 2017.
  * [56] Yuke Zhu, Roozbeh Mottaghi, Eric Kolve, Joseph J Lim, Abhinav Gupta, Li Fei-Fei, and Ali Farhadi.  Target-driven visual navigation in indoor scenes using deep reinforcement learning.  In 2017 IEEE international conference on robotics and automation (ICRA), pages 3357–3364. IEEE, 2017.

##  Appendix A Visualization of Policy Representations

We analyze the policy representation before and after translation by reducing
the dimensionality of the policy representations using Principle Component
Analysis (PCA) [46]. In Figures 7 and 7, we visualize the policy
representations reduced to 2 dimensions using PCA in Replica and Real-World
respectively. Both figures show that PBIT brings the representations of target
domain Replica/Real-World images closer to the distribution of representations
of Gibson images.

Figure 6: We use PCA to visualize and compare the 128-dimensional task-
specific feature vectors produced by PointGoal agent trained in Gibson, when
given Gibson images, Replica images, or Replica images translated to Gibson by
PBIT as input. The figure shows the translated images by PBIT bridge the
policy domain gap between Gibson and Replica.

Figure 7: We use PCA to visualize and compare the 128-dimensional task-
specific feature vectors produced by PointGoal agent trained in Gibson, when
given Gibson images, Real-World images, or Real-World images translated to
Gibson by PBIT as input. The figure shows the translated images by PBIT bridge
the policy domain gap between Gibson and Real-World.

##  Appendix B Navigation Policy Training Details

Policy Architecture. The navigation policy is decomposed into a Policy Feature
Extractor ($F_{\pi}$) and a Action Policy ($A_{\pi}$) as shown in Fig 2.
$F_{\pi}$ is based on the 18-layer ResNet [19]. The architecture of $F_{\pi}$
is shown in Figure 8. $F_{\pi}$ outputs 128-dimensional policy-related
representations ${\bf r}_{I}$ given RGB images of shape $3\times 256\times
256$. $A_{\pi}$ is based on a 2-layer GRU, which takes ${\bf r}_{I}$ together
with readings from either the GPS+Compass sensor in PointGoal task or the base
odometry sensor in Exploration task.

Policy Training. The navigation policies are trained using PPO [36] with
Generalized Advantage Estimation [35]. The training dataset contains episodes
in 72 training scenes in Gibson as provided by [33]. We use 8 concurrent
workers and train for a total of 30 million frames. The time horizon for a PPO
update is 128 steps, number of epochs per PPO update is 2, and clipping
parameter is set to 0.2. We use discount factor 0.99, GAE parameter 0.95, and
the Adam optimizer [25] with learning rate $2.5\times 10^{-4}$.

##  Appendix C PBIT Model Architecture Details

We use several convolutional layers and residual blocks to construct the image
encoders $E_{I}^{s},E_{I}^{t}$ and image decoders $D_{I}^{s},D_{I}^{t}$. We
use Instance Normalization [41] in $E_{I}^{s},E_{I}^{t}$ and Adaptive Instance
Normalization [21] in $D_{I}^{s},D_{I}^{t}$. For the discriminators ${\rm
Dis}^{s},{\rm Dis}^{st}$, we adopt the multi-scale discriminator architecture
proposed by [42]. Detailed descriptions of the architecture are given in Table
3.

Figure 8: An illustration of the network architecture of the Policy Feature
Extractor. Table 3: Architecture specification of different parts of the
proposed PBIT model.

(a) Style Encoder $(256\times 256)$ RGB
---
Conv(32,(7x7),1)
Conv(64,(4x4),2)
Conv(128,(4x4),2)
Conv(128,(4x4),2)
Conv(128,(4x4),2)
global_avg_pool
dense(8)
$\rightarrow s\in\mathbb{R}^{8}$

(b) Content Encoder $(256\times 256)$ RGB
---
Conv(32,(7x7),1)
Conv(64,(4x4),2)
Conv(128,(4x4),2)
Residual(128)
Residual(128)
Residual(128)
Residual(128)
$\rightarrow c\in\mathbb{R}^{128\times 64\times 64}$

(c) Decoder $c$ and AdaIN$(s)$
---
Residual(128)
Residual(128)
Residual(128)
Residual(128)
UpSampling(2x2)
Conv(64,(5x5),1)
UpSampling(2x2)
Conv(32,(5x5),1)
UpSampling(2x2)
Conv(3,(7x7),1)
$(256\times 256)$ RGB

(d) Discriminator Multi-scale Input
---
Conv(64,(4x4),2)
Conv(128,(4x4),2)
Conv(256,(4x4),2)
Conv(512,(4x4),2)

##  Appendix D Additional Trajectory Visualizations

Figure 9: Additional Real World Trajectories. Figure showing raw inputs,
translated Gibson images by PBIT, and a third-person perspective for two
trajectories in the real-world for the PointGoal task..

Figure 10: Additional Replica Trajectories. Figure showing raw inputs,
translated Gibson images by PBIT, and the ground-truth top-down map (not
visible to the agent) for three trajectories in the Replica domain.