For each run of our experiments, we use either a single Nvidia GeForce RTX
2080 Ti or a single Nvidia RTX A-6000 GPU, together with Intel Xeon CPU, and
40GB of RAM. During training, the GPU memory usage is around 9GB. Depending on
which task, the total training iterations range between 10k to 25k, and the
total wall-clock training time range between 3.75 to 7.5 hours.

Learning Rate | $0.0003$
---|---
Buffer Size | $300$
Batch Size | $64$
Discount ($\gamma$) | $0.99$
Soft Update Coefficient | $0.005$
Table A.11: Hyperparameters of SAC for the RL-VMC baseline

Learning Rate | $0.0003$
---|---
Buffer Size | $300$
Batch Size | $64$
Discount ($\gamma$) | $0.99$
GAE Parameter $\gamma_{gae}$ | $0.99$
Clipping Parameter $\epsilon$ | $0.2$
Entropy Coeff $c_{1}$ | $0.0$
VF Coeff $c_{2}$ | $0.5$
Table A.12: Hyperparameters of PPO for the RL-Prim. and RL-Prim.Hist.
baselines

###  F.4 Action Primitives

BEHAVIOR-1K activities are long-horizon which require hundreds if not
thousands of environment steps (low-level robot control signals) to be
completed. A way to overcome this challenge (e.g., reinforcement learning) is
to modify the original action space with a set of action primitives, i.e.,
time-extended actions that correspond to multiple low-level commands and that
achieve some expected outcome, such as grasping an object or navigating to a
location. For our baselines, we defined six of these primitives, and
implemented them using a sampling-based motion planner.

All the primitives are compositional: collision-free paths of the entire robot
to navigate to a location, collision-free paths of the robot’s arm to reach a
6D_pose with the end-effector, or a predefined arm joint configuration,
trajectories where the end-effector follows a line in Cartesian space,
possibly colliding/interacting with the environment, and sequences of actions
to open/close the robot’s gripper while holding the position. All manipulation
primitives start with a trajectory to move the arm from a tucked configuration
(folded arm) to an untucked configuration to enable subsequent arm
interaction, and end with the inverse motion, from untucked to tucked
configuration, to allow subsequent collision-free navigation. The action
primitives take as parameter an object to apply the action on; we assume
access to the 3D position of the objects to obtain the necessary parameters to
query the motion planner. This procedure is replicated on the real robot,
where we combine detections from YOLO [70] with information from the depth map
of the RGB-D images to obtain the parameters (see Sec. G). For the navigation
actions, we assume a set of known relevant locations to navigate to (next to
the rectangles in Fig. A.9)

The primitives we used in our experiments can be seen in Fig. A.10 and
include:

  * a)

navigate: Collision-free trajectory of the entire robot to a location.

  * b)

pick: Composition of 1) a trajectory to a pre-grasp 6D_pose above the object
to pick, 2) a line trajectory in Cartesian space down towards the object,
interrupted when there is contact, 3) a closing action, 4) a first retracting
trajectory following a line upwards, and 5) a second retracting trajectory to
reach the untucked joint configuration. The primitive only executes if the
robot is not currently picking another object.

  * c)

place: Composition of 1) a trajectory to a pre-place 6D_pose above the object
to place the grasped object on, 2) an opening action, 3) a second retracting
trajectory to reach the untucked joint configuration. The primitive only
executes if the robot is currently holding an object.

  * d)

push: Composition of 1) a trajectory to a pre-push 6D_pose above the object to
push-open, 2) a line trajectory in Cartesian space down towards the object,
interrupted if there is contact, 3) a line trajectory in Cartesian space to
push-open the object, e.g., towards the robot, 4) a first retracting
trajectory following a line upwards, and 5) a second retracting trajectory to
reach the untucked joint configuration.

  * e)

dip: Composition of 1) a trajectory to a pre-dip 6D_pose above the object to
dip into, 2) a line trajectory in Cartesian space down towards the object to
dip, 3) a line trajectory in Cartesian space upwards, and 4) a retracting
trajectory to reach the initial joint configuration. The primitive only
executes if the robot is currently holding an object.

  * f)

wipe: Composition of 1) a trajectory to a pre-wipe 6D_pose above the object to
wipe, 2) a line trajectory in Cartesian space down towards the object to be
wiped, interrupted when there is contact, 3) a line trajectory in Cartesian
space horizontally to wipe left-right, 4) a line trajectory in Cartesian space
horizontally to wipe towards the robot, 4) a first retracting trajectory
following a line upwards, and 5) a second retracting trajectory to reach the
untucked joint configuration. The primitive only executes if the robot is
currently holding an object.

(a) navigate

(b) pick

(c) place

(d) push (open)

(e) dip

(f) wipe

Figure A.10: Overview of all the high-level action primitives with the pre-
conditions (left), post-conditions (right), and intermediate states (middle)
during execution.

###  F.5 Additional Metrics: success score Q

The success score Q [27] for each task is reported in Table A.13. We observe
the same trend in Q as in the success rate. This is another indication of the
effectiveness of our approach.

Method | Policy Features | Success score Q
---|---|---
Primitives | History | StoreDecoration | CollectTrash | CleanTable
RL-VMC |  |  | $0.0\pm 0.0$ | $0.0\pm 0.0$ | $0.0\pm 0.0$
RL-Prim. |  |  | $0.50\pm 0.05$ | $0.49\pm 0.04$ | $0.77\pm 0.08$
RL-Prim.Hist. |  |  | $0.59\pm 0.03$ | $0.68\pm 0.02$ | $0.88\pm 0.02$

Table A.13: Performance of three baseline methods, in terms of the success score Q. |
---|---
a) SAC Architecture | b) PPO Architecture
Figure A.11: Policy architecture of SAC (RL-VMC) and PPO (RL-Prim. and RL-
Prim.Hist.). The policy maps egocentric visual observations and proprioceptive
information into an action that controls the robot base, arm, and gripper. PPO
outputs a discrete high-level action primitive while SAC outputs a continuous
low-level joint control directly.

##  Appendix G Details of the Real-World Setup

##### Scene.

Our real-world experiments take place in a mockup studio-apartment in our lab
that includes a bedroom, living room, and dining area. This real-world scene
was modeled with a digital counterpart in simulation for BEHAVIOR-1K: the
mockup_apt scene. Fig. 5 depicts both real and digital twin side-by-side. The
digital model has been created by first, scanning the real-world apartment
using a phone and commercial software [83], then, replacing and projecting the
texture of walls, floors, and ceilings, and finally, replacing the existing
objects with 3D models from our BEHAVIOR-1K Dataset. This process should
minimize the effect caused by differences between the real world and the 3D
model. The simulated and the real robot use the same 2D map of the scene for
localization. In this way, 2D locations in the real world correspond to the
same 2D locations in simulation.

##### Robot platform.

We use in our experiments a Tiago++ model from PAL Robotics, with an
omnidirectional base, two 7-degrees-of-freedom arms with parallel-yaw
grippers, a 1-degree-of-freedom prismatic torso, two SICK LiDAR sensors (back
and front of the base), and an ASUS Xtion RGB-D camera mounted on the robot’s
head, which can be controlled in yaw and pitch. All sensors and actuators are
connected through the Robot Operating System, ROS [84]. The code runs on a
laptop with an Nvidia GTX 1070 that sends the commands to the onboard robot
computer to be executed.

##### Action primitives on the real robot.

We implemented a version of the action primitives navigate, pick and place on
the real robot similar to their implementation in OmniGibson. For navigation,
we use a 2D sampling-based motion planner on the 2D map we use for
localization and that has been created from the 3D model of the apartment. For
manipulation in the real world, we need to obtain the parameters necessary to
plan the manipulation primitives (pick and place) from the sensor signals. To
do that, we first obtain detections from a YOLO v3 [70] object detector on the
RGB images from the robot’s camera. If the robot attempts to execute an action
primitive on an object that has not been detected, we return failure (Object
Detection Failure in our analysis). If the object to manipulate has been
detected, we query the pixels in the depth map corresponding to the
21$\times$21 window at the center of the detection bounding box, and use this
information to compute the centroid of the corresponding 3D points. This
location will be used as an object location for grasping or placing. This
information is enough to create a sequence of paths using a sampling-based
motion planner (RRT [62]) similar to the ones used in simulation. The motion
planner uses a voxel representation of the scene obtained from the depth
sensor.

##### Experimental setup.

We randomized three parameters in our real robot experiments: the initial
location of the robot (chosen from the three possible activity locations), the
positions of the objects on the table, and the orientations of the objects
(upright or lying down). The experiment runs cover uniformly these parameters.
Additionally, for vision-based policies, we evaluated two lighting conditions:
full lighting (including the ceiling) and only lamps. Failures are defined as
follows: Motion planning failures occurred when the robot was unable to plan
an end effector trajectory to complete the action primitive. This includes
failures caused by navigation noise, where after navigating to the task
location, the robot was too far away from the target object to execute the
pick or place action. Grasping failures include errors that occurred during
the execution of the grasp such as pushing the object off the table while
attempting a grasp or losing the object because it slips out, as well as the
robot dropping the object after grasping it. Placing failures include the
robot dropping the object in the wrong location. Object detection failures
primarily resulted from our object detector, YOLO v3 [70], failing to detect
the object to interact with; a second, less common, object detection failure
corresponds to the depth camera returning invalid measurements for the
detected object. Finally, we consider policy failures when the policy selects
the same invalid action three times in a row, e.g., requesting to navigate to
the bin when it is already in front of the bin. The main policy error
corresponds to the policy repeatedly choosing the same action primitive. We
observed empirically that, even though the policy selection presents some
small randomness when the policy requests the same invalid action three times
in a row, most subsequent calls will be also the same invalid action and we
decide to terminate early.

###  G.1 Further Characterization of the Sim-Real Gap

We performed additional experiments to characterize the gap between simulation
and the real world. In our experiments, we moved the robot to different
locations in the mockup_apt scene and collected real sensor signals: RGB
images, depth maps, and LiDAR measurements. We then moved the robot in
simulation to the same locations and collected virtual sensor signals. Some of
the images are depicted in Fig. A.12. We observe that, thanks to our highly
realistic models, OmniGibson provides high-fidelity sensor signals that
approximate the real-world ones. However, some of the sources of noise in the
real sensors are not modeled currently in OmniGibson, contributing to a sim-
real gap, e.g., the poor dynamic range of the RGB camera, or the “shadow”
effects in depth maps due to the projected light mechanism. This analysis
indicates possible avenues to further close the sensor gap between simulation
and the real world, e.g., by including sensor noise models in OmniGibson or by
leveraging sim-to-real techniques (e.g., domain randomization, system
identification).

###  G.2 Additional Experiments

In addition to RL-Prim.Hist., we also evaluated RL-VMC and RL-Prim.Hist. in
the real world and observed a similar trend of performance in the real world
as in simulation: RL-VMC < RL-Prim. < RL-Prim.Hist.. RL-VMC still achieves
zero success because of sparse reward and exploration difficulty during
training. RL-Prim. has worse performance than RL-Prim.Hist.: on average, the
robot with RL-Prim.Hist. successfully places 0.64 cups/bottles, whereas the
one with RL-Prim. places 0. Qualitatively, RL-Prim. tends to get stuck in a
repetitive action loop because the agent is unaware of its action history.
This preliminary result offers us some confidence that OmniGibson can be a
reliable test bed for future sim-to-real robotics research.

Figure A.12: Comparison between real and simulated sensor signals. RGB, Depth
and LiDAR signals from the real-world sensors (first and third rows) and from
the simulated sensors (second and fourth rows) at two different locations of
the mockup_apt scene (location 1: first two rows, location 2: last two rows).
We observed a smaller sensor sim-real gap for LiDAR than for RGB and Depth:
RGB is heavily influenced by lighting conditions and camera settings while
Depth has difficulty in capturing reflective surfaces.

##  Appendix H Ethical Statement

BEHAVIOR-1K aims to drive embodied AI solutions that fulfill human need. A
primary ethical consideration of BEHAVIOR-1K is therefore the method used to
determine such need. To understand which activities would be useful for
humans, we survey 1,461 respondents on Amazon Mechanical Turk and collected 50
responses for each activity, aiming for a clear consensus signal. Though this
is a large group, the results are still biased by the fact that the
researchers, annotators, and data providers only represent a small population
relative to potential users of such technology. We plan to address these
biases by making BEHAVIOR-1K Dataset open-source and invite a wider community
to contribute to its knowledge base.

Furthermore, compared to the U.S. population, the demographic representation
in our survey skews white, male, non-disability status, mid-five figure
incomes, and 30-40 age range with more representation on the higher side than
the lower side of that range—closer to the demographics of Mechanical Turk
workers. This also biases the survey population toward certain responses,
creating potential ethical limitations.

Specifically this bias may affect the survey’s representation of people who
would be affected most by autonomous agents [85]. We therefore asked several
explicit questions such as “Do you do household work for a living?”, “If you
do household work for a living, would you benefit from assistance?”, “Do you
pay for someone else to do household work for you?”. Future work involves
using these questions to direct benchmark development.

language = C, basicstyle = , backgroundcolor = , stringstyle = , keywordstyle
= [2], keywordstyle = [3], keywordstyle = , otherkeywords = ?, morekeywords =
[2]ontop, inside, inroom, morekeywords = [3]define, problem, domain, objects,
init, goal, and, for_n_pairs, forall, :, exists, and,

Listing 1: BakingSugarCookies

⬇

(define (problem baking_sugar_cookies-0)

(:domain omnigibson)

(:objects

flour.n.01_1 - flour.n.01

granulated_sugar.n.01_1 - granulated_sugar.n.01

raw_egg.n.01_1 - raw_egg.n.01

vanilla.n.02_1 - vanilla.n.02

melted__butter.n.01_1 - melted__butter.n.01

sodium_carbonate.n.01_1 - sodium_carbonate.n.01

baking_powder.n.01_1 - baking_powder.n.01

electric_mixer.n.01_1 - electric_mixer.n.01

mixing_bowl.n.01_1 - mixing_bowl.n.01

sugar_cookie.n.01_1 sugar_cookie.n.01_2 sugar_cookie.n.01_3

sugar_cookie.n.01_4 sugar_cookie.n.01_5 sugar_cookie.n.01_6 -
sugar_cookie.n.01

oven.n.01_1 - oven.n.01

cookie_sheet.n.01_1 - cookie_sheet.n.01

flour__sack.n.01_1 - flour__sack.n.01

sugar__sack.n.01_1 - sugar__sack.n.01

sodium_carbonate__jar.n.01_1 - sodium_carbonate__jar.n.01

baking_powder__jar.n.01_1 - baking_powder__jar.n.01

mason_jar.n.01_1 - mason_jar.n.01

countertop.n.01_1 countertop.n.01_2 - countertop.n.01

vanilla__bottle.n.01_1 - vanilla__bottle.n.01

salt.n.02_1 - salt.n.02

salt__shaker.n.01_1 - salt__shaker.n.01

bowl.n.01_1 - bowl.n.01

electric_refrigerator.n.01_1 - electric_refrigerator.n.01

tablespoon.n.02_1 - tablespoon.n.02

floor.n.01_1 - floor.n.01

agent.n.01_1 - agent.n.01

)

(:init

(filled flour__sack.n.01_1 flour.n.01_1)

(ontop flour__sack.n.01_1 countertop.n.01_1)

(insource salt__shaker.n.01_1 salt.n.02_1)

(ontop salt__shaker.n.01_1 countertop.n.01_1)

(ontop tablespoon.n.02_1 countertop.n.01_1)

(filled sugar__sack.n.01_1 granulated_sugar.n.01_1)

(ontop sugar__sack.n.01_1 countertop.n.01_1)

(inside raw_egg.n.01_1 bowl.n.01_1)

(insource vanilla__bottle.n.01_1 vanilla.n.02_1)

(ontop vanilla__bottle.n.01_1 countertop.n.01_1)

(filled mason_jar.n.01_1 melted__butter.n.01_1)

(ontop mason_jar.n.01_1 countertop.n.01_1)

(filled sodium_carbonate__jar.n.01_1 sodium_carbonate.n.01_1)

(ontop sodium_carbonate__jar.n.01_1 countertop.n.01_2)

(filled baking_powder__jar.n.01_1 baking_powder.n.01_1)

(ontop baking_powder__jar.n.01_1 countertop.n.01_2)

(ontop electric_mixer.n.01_1 countertop.n.01_2)

(attached mixing_bowl.n.01_1 electric_mixer.n.01_1)

(inroom oven.n.01_1 kitchen)

(inroom countertop.n.01_1 kitchen)

(inroom countertop.n.01_2 kitchen)

(inside bowl.n.01_1 electric_refrigerator.n.01_1)

(inroom electric_refrigerator.n.01_1 kitchen)

(ontop cookie_sheet.n.01_1 countertop.n.01_2)

(inroom floor.n.01_1 kitchen)

(ontop agent.n.01_1 floor.n.01_1)

(future sugar_cookie.n.01_4)

(future sugar_cookie.n.01_1)

(future sugar_cookie.n.01_2)

(future sugar_cookie.n.01_6)

(future sugar_cookie.n.01_3)

(future sugar_cookie.n.01_5)

)

(:goal

(and

(real ?sugar_cookie.n.01_1)

(real ?sugar_cookie.n.01_2)

(real ?sugar_cookie.n.01_3)

(real ?sugar_cookie.n.01_4)

(real ?sugar_cookie.n.01_5)

(real ?sugar_cookie.n.01_6)

(forall

(?sugar_cookie.n.01 - sugar_cookie.n.01)

(and

(cooked ?sugar_cookie.n.01)

(ontop ?sugar_cookie.n.01 ?cookie_sheet.n.01_1)

)

)

)

)

)

Listing 2: CleanYourLaundryRoom

⬇

(define

(problem clean_your_laundry_room_1)

(:domain omnigibson)

(:objects

rag.n.01_1 - rag.n.01

dryer.n.01_1 - dryer.n.01

water.n.06_1 - water.n.06

vinegar.n.01_1 - vinegar.n.01

washer.n.03_1 - washer.n.03

dust.n.01_1 - dust.n.01

mold.n.05_1 - mold.n.05

floor.n.01_1 - floor.n.01

agent.n.01_1 - agent.n.01

)

(:init

(ontop rag.n.01_1 dryer.n.01_1)

(not

(covered water.n.06_1 rag.n.01_1)

)

(empty vinegar.n.01_1 washer.n.03_1)

(covered dust.n.01_1 dryer.n.01_1)

(covered mold.n.05_1 washer.n.03_1)

(onfloor agent.n.01_1 floor.n.01_1)

(filled water.n.06_1 bottle.n.01_1)

(onfloor bottle.n.01_1 floor.n.01_1)

(inroom washer.n.03_1 laundry_room)

(inroom floor.n.01_1 laundry_room)

)

(:goal

(and

(filled ?vinegar.n.01_1 ?washer.n.03_1)

(not

(covered ?dust.n.01_1 ?dryer.n.01_1)

)

(not

(covered ?mold.n.05_1 ?washer.n.03_1)

)

)

)

)

Listing 3: CleanTheBottomOfAnIron

⬇

(define (problem clean_the_bottom_of_an_iron-0)

(:domain omnigibson)

(:objects

tarnish.n.02_1 - tarnish.n.02

iron.n.04_1 - iron.n.04

ironing_board.n.01_1 - ironing_board.n.01

emery_paper.n.01_1 - emery_paper.n.01

countertop.n.01_1 - countertop.n.01

newspaper.n.03_1 - newspaper.n.03

floor.n.01_1 - floor.n.01

agent.n.01_1 - agent.n.01

)

(:init

(covered iron.n.04_1 tarnish.n.02_1)

(toggled_on iron.n.04_1)

(ontop iron.n.04_1 ironing_board.n.01_1)

(ontop ironing_board.n.01_1 floor.n.01_1)

(ontop emery_paper.n.01_1 countertop.n.01_1)

(ontop newspaper.n.03_1 ironing_board.n.01_1)

(inroom countertop.n.01_1 utility_room)

(inroom floor.n.01_1 utility_room)

(ontop agent.n.01_1 floor.n.01_1)

)

(:goal

(and

(not

(covered ?iron.n.04_1 ?tarnish.n.02_1)

)

)

)

)

Listing 4: PackingLunch

⬇

(define

(problem packing_lunches_1)

(:domain igibson)

(:objects

shelf.n.01_1 - shelf.n.01

water.n.06_1 - water.n.06

countertop.n.01_1 - countertop.n.01

apple.n.01_1 - apple.n.01

electric_refrigerator.n.01_1 \-

electric_refrigerator.n.01

hamburger.n.01_1 - hamburger.n.01

basket.n.01_1 - basket.n.01

)

(:init

(ontop water.n.06_1 countertop.n.01_1)

(inside apple.n.01_1

electric_refrigerator.n.01_1)

(inside hamburger.n.01_1

electric_refrigerator.n.01_1)

(ontop basket.n.01_1 countertop.n.01_1)

(inroom countertop.n.01_1 kitchen)

(inroom electric_refrigerator.n.01_1

kitchen)

(inroom shelf.n.01_1 kitchen)

)

(:goal

(and

(for_n_pairs

(1)

(?hamburger.n.01 - hamburger.n.01)

(?basket.n.01 - basket.n.01)

(inside ?hamburger.n.01 ?basket.n.01)

)

(for_n_pairs

(1)

(?basket.n.01 - basket.n.01)

(?water.n.06 - water.n.06)

(inside ?water.n.06 ?basket.n.01)

)

(for_n_pairs

(1)

(?basket.n.01 - basket.n.01)

(?apple.n.01 - apple.n.01)

(inside ?apple.n.01 ?basket.n.01)

)

(forall

(?basket.n.01 - basket.n.01)

(ontop ?basket.n.01 ?countertop.n.01_1)

)

)

)

)

Listing 5: ServingHorsDoeuvres

⬇

(define

(problem serving_hors_d_oeuvres_1)

(:domain igibson)

(:objects

tray.n.01_1 tray.n.01_2 - tray.n.01

countertop.n.01_1 - countertop.n.01

oven.n.01_1 - oven.n.01

sausage.n.01_1 sausage.n.01_2 - sausage.n.01

cherry.n.03_1 cherry.n.03_2 - cherry.n.03

electric_refrigerator.n.01_1 \-

electric_refrigerator.n.01

)

(:init

(ontop tray.n.01_1 countertop.n.01_1)

(ontop tray.n.01_2 countertop.n.01_1)

(inside sausage.n.01_1 oven.n.01_1)

(inside sausage.n.01_2 oven.n.01_1)

(inside cherry.n.03_1

electric_refrigerator.n.01_1)

(inside cherry.n.03_2

electric_refrigerator.n.01_1)

(inroom oven.n.01_1 kitchen)

(inroom electric_refrigerator.n.01_1

kitchen)

(inroom countertop.n.01_1 kitchen)

)

(:goal

(and

(exists

(?tray.n.01 - tray.n.01)

(and

(forall

(?sausage.n.01 - sausage.n.01)

(ontop ?sausage.n.01 ?tray.n.01)

)

(forall

(?cherry.n.03 - cherry.n.03)

(not

(ontop ?cherry.n.03 ?tray.n.01)

)

)

)

)

(exists

(?tray.n.01 - tray.n.01)

(and

(forall

(?cherry.n.03 - cherry.n.03)

(ontop ?cherry.n.03 ?tray.n.01)

)

(forall

(?sausage.n.01 - sausage.n.01)

(not

(ontop ?sausage.n.01 ?tray.n.01)

)

)

)

)

)

)

)