# Dark Matter Accounts for Perturbation in the GD-1 Stellar Stream

Carissma McGee  Howard University $|$ 2400 6th St NW, Washington, DC 20059
Center for Astrophysics $|$ Harvard & Smithsonian, 60 Garden Street,
Cambridge, MA 02138, USA  Ana Bonaca Center for Astrophysics $|$ Harvard &
Smithsonian, 60 Garden Street, Cambridge, MA 02138, USA

###### Abstract

The longest recognized stellar stream in the Milky Way Galaxy has an expanse
of over more than half the north sky. There was a physical disturbance within
the stream, 500 million years ago, which could have been the scar of a dark
matter collision. Due to its proximity to the galactic center, the GD-1
stellar stream can act as an antenna for gravitational perturbations. In 2018,
a significant gap in GD-1 was discovered due to perturbation. A stream gap
occurs when a massive object collides with the stellar stream. Based on the
chasm’s location and width, we can guess when and where the impact occurred.
Using globular cluster sky coordinates and simulated galactocentric
distributions, we calculated how close each globular cluster came to the GD-1
stream. This would be the first time a globular cluster has come close enough
to the GD-1 stream to impact another object. Clusters, on the other hand,
rarely approach GD-1, indicating that it was struck by something more exotic,
like a clump of dark matter, when discovered. A simulation and theoretical
model were created to better understand the GD-1 stellar stream behavior.
These details can be used to map the large-scale distribution of dark matter
in our galaxy as well as the small-scale structure of dark matter in the host
galaxies of the streams. Examination of stellar streams and detection of
subhalos will not only confirm the presence of dark matter but also reveal
information about its particle nature.

Globular Cluster, Dark Matter, Galactocentric Coordinates, Stream Impacts,
Subhalo

##  1 Introduction

Since the dawn of humanity, determining the number of stars visible in the
night sky and forecasting their behavior has been a subject of interest.
Timocharis of Alexandria and Hipparchus of Nicaea were among the first
astronomers to begin counting and cataloguing naked-eye stars, creating the
first magnitude-limited catalogues. Modern astronomers prefer catalogues with
varying maximum distance limits, as any sample with a fixed magnitude is
biased against intrinsically faint (and single) objects. We now know that they
account for a substantial portion of the objects in our Galaxy, despite the
fact that even the brightest of them are invisible to the naked eye.

Astronomers have been attempting to map our nearest neighbors since the first
measurements of stellar parallaxes. Gaia, the second astrometric space
mission, significantly improves the quality and quantity of sky parallax
measurements for about 1.5 billion objects Gaia Collaboration et al. (2016).
It enables the completion of volume-restricted samples while increasing the
distance limit. The Gaia Catalogue of Nearby Stars (GCNS), which is based on
Gaia Early Data Release 3 (Gaia EDR3, Gaia Collaboration 2021a) Kuzma et al.
(2021), extends the distance limit to 100 parsecs (Gaia Collaboration et al.,
2016). Gaia determined all of these extremely precise positions and
velocities, which we will use to map the stellar streams, identify
perturbations, and diagnose their causes. Gaia determined all of these
extremely precise positions and velocities, which we will use to map the
stellar streams, identify perturbations, and diagnose their causes.

The Milky Way Galaxy’s structure is fairly typical of a large spiral system
Chakrabarty et al. (2021). This structure is composed of a nucleus, a central
bulge, a disk, spiral arms, a spherical component, and a massive halo. Several
of these components are virtually indistinguishable. At the heart of the
Galaxy is an extraordinary object—a massive black hole encircled by a disk of
hot gas known as an accretion disk. Due to the Milky Way’s dense screen of
intervening dust, optical wavelengths cannot be used to observe the central
object or any of the material immediately surrounding it Bravo et al. (2021).
The nucleus is surrounded by an extended bulge of nearly spherical stars, the
majority of which are Population II stars, albeit with a high concentration of
heavy elements. Numerous globular clusters of similar stars are interspersed
among the stars, and both the stars and the clusters orbit the nucleus in
nearly radial orbits Vasiliev & Baumgardt (2021). Bulge stars can be seen
optically where they rise above the obscuring dust of the galactic plane. The
Milky Way halo contains a number of narrow, cool stellar streams that were
formed as a result of tidal stripping of low-mass star systems such as global
clusters Mandelker (2021). These streams are critical for testing dark matter
theories because they are used to map the Galaxy’s overall mass structure and
are highly sensitive to gravitational disturbance interactions with dark
matter Webb & Bovy (2021). As a collection of tidally stripped dwarf galaxies
and globular clusters, stellar streams provide excellent probes of the Milky
Way’s gravitational potential and can be used to precisely reconstruct the
galaxy’s past history.

Globular clusters are dense, symmetrical clusters of stars that primarily
orbit within the extended star halos that surround the majority of spiral
galaxies Bílek et al. (2021). Globular clusters are believed to contain some
of the galaxy’s oldest stars and formed early in the galaxy’s history.

Stellar streams are objects that enter the Milky Way and are unraveled by its
tidal forces. As a result, they are extremely useful for tracing the
hierarchical formation of the Milky Way galaxy’s structure Bravo et al.
(2021). GD-1 is an unusually diverse stellar stream, prompting speculation
about alternative processes that could account for various aspects of the
data. Stellar streams are collections of stars that arc around their host
galaxy in the form of elongated filaments Kohler (2021). These filaments are
thought to form when the tidal forces of the host galaxy disrupt a stream
progenitor, such as a globular cluster or a satellite dwarf galaxy.

Although the nature of dark matter is unknown, it appears to account for
roughly 82 percent of all matter in the universe. As a result, dark matter’s
gravitational interactions have shaped the evolution of the universe’s
structure Bonaca (2021). Each galaxy is surrounded by an enigmatic halo of
dark matter, which can be detected indirectly by observing its gravitational
effects. The invisibly large and spherical halo dwarfs the brilliant galaxy at
its center. Recent computer simulations have revealed a surprisingly clumpy
halo, with relatively dense concentrations of dark matter contained within
gravitationally bound’subhalos’ Mirabal & Bonaca (2021). The primary focus of
this investigation will be on these ”subhalos.”

There is a great deal of uncertainty about the origins of the Milky Way’s
stellar streams Mandelker (2021). Understanding the origins of these stellar
streams enables us to more precisely trace their paths, the duration of their
orbits, and any other gravitational interactions they may have encountered
over time. We will develop a simulation and theoretical model to reconstruct a
more accurate depiction of the GD-1 stellar stream behavior. These details are
useful not only for understanding galaxy evolution, but also for mapping the
big-picture distribution of dark matter in our galaxy and studying the small-
scale structure of dark matter in the host galaxies of the streams Webb & Bovy
(2021). Closer analysis of stellar streams along with detecting these subhalos
will not only be a strong indicator for the existence of dark matter but also
will give valuable information about its particle nature.

We are also interested to see if a dark-matter subhalo could have impacted the
GD-1 stream. This is of course hard to determine right now, because we have
not confirmed if they even exist in the time and space of our data set.
However, we have an idea where they could be. Simulated evidence of old galaxy
merging with the Milky Way closely fits the distribution of stars in the Milky
Way halo (Bonaca et al., 2021). This simulation also predicts where dark
matter from this galaxy should be, so we can take these predictions and test
if those subhalos could come close to GD-1 and other streams.

##  2 Data

###  2.1 GD-1 Stream

In the beginning, stellar streams are self-bound objects that fall into the
Milky Way and are unravelled by the tidal forces of the Milky Way’s potential
Mirabal & Bonaca (2021). As a result, they are extremely useful for tracing
the hierarchical formation of the Milky Way galaxy’s structure. GD-1 is a
stellar stream with a surprising number of characteristics, which has sparked
speculation about additional processes that could be used to explain various
aspects of the data.

By using uniform galactocentric coordinate structure throughout, I was able to
make my own stream impact visualization in Figure 1. Although quantitative
predictions for the kinematics of a spur formed by these processes have not
been made, significant velocity kicks are expected.

Figure 1: Arc of the GD-1 Stellar Stream via galactocentric coordinates.

We used the same Galactocentric coordinate system for the GD-1 stream,
globular clusters, and simulated dark matter.

This frame allowed for specifying the Sun-Galactic center distance, the height
of the Sun above the Galactic midplane, and the solar motion relative to the
Galactic center. However, as there is no modern standard definition of a
Galactocentric reference frame. Once the specific stream impact was located,
the next step was to analyze the distribution of Globular Clusters and check
whether they passed through that area.

###  2.2 Globular Cluster Distribution

$X$ | $Y$ | $Z$ | $V_{X}$ | $V_{Y}$ | $V_{Z}$
---|---|---|---|---|---
kpc | kpc | kpc | $\rm km\,s^{-1}$ | $\rm km\,s^{-1}$ | $\rm km\,s^{-1}$
-6.3 | -2.6 | -3.2 | -75.0 | 165.1 | 46.0
-8.2 | 0.0 | -8.9 | -7.0 | -52.0 | 51.3
-5.0 | -5.1 | -6.2 | -88.2 | -100.3 | -69.4
-22.2 | 4.7 | -26.2 | 227.7 | 54.2 | 8.2
-8.1 | -10.0 | -12.8 | 77.3 | 50.9 | 69.1
-14.9 | 8.0 | 3.7 | 65.3 | 197.7 | -19.4
-24.9 | -80.1 | -92.2 | 120.0 | -71.2 | 105.6
-61.5 | -41.8 | -59.3 | 92.0 | 16.9 | 133.6
-34.6 | 4.4 | -4.2 | 109.9 | -13.5 | 3.8
-12.4 | -8.9 | -6.9 | -83.4 | -72.8 | -81.1
-15.8 | -8.3 | -6.3 | -44.1 | -21.1 | 5.0
-12.4 | -9.5 | -2.9 | 94.9 | 19.5 | 77.7
-82.7 | -0.5 | 35.4 | 6.1 | 36.5 | -57.5
-14.0 | -38.7 | 4.8 | 114.9 | 211.2 | 187.1
-6.1 | -9.2 | -1.9 | 57.2 | 142.5 | 28.9
-5.2 | -7.1 | -2.6 | -224.9 | 106.5 | 105.0
-42.3 | -59.7 | 61.8 | 15.2 | 160.3 | 67.3
-7.5 | -4.8 | 0.8 | 258.1 | -205.4 | 150.6
-39.3 | -12.9 | 103.4 | 36.3 | -34.5 | 56.1
0.3 | -97.0 | 107.5 | 69.8 | 80.7 | 54.6
-9.3 | -4.1 | 18.8 | -39.8 | -31.3 | 128.3
-5.2 | -4.9 | -1.0 | -99.6 | 76.2 | 68.1
2.5 | -17.8 | 4.3 | -120.4 | 217.6 | 33.3
-4.0 | -7.2 | 6.1 | -167.9 | 282.2 | 16.7

Table 1: Distribution of globular clusters in the 6D phase space via Gaia Data
Release 2

Globular clusters orbit in the halo of our galaxy, centered on the Galaxy’s
center and expanding above and below the Galactic disk. Open star clusters
tend to orbit within the disk. We analyzed the distribution of the globular
clusters within 6D phase space utilizing both positions / velocity, and action
/ angle coordinates in combination with literature distance and line-of-view
velocity measures.

Globular star clusters contain hundreds of thousands of stars, and some – like
Omega Centauri –- contain millions of stars. The population of the center 10
kpc clusters is 50-80 km/s rotating and 100-120 km/s isotropic, while the
outer Galaxy shows that the cluster orbits are substantially anisotropical
Tadross & Hendy (2021). A proven combustion of clusters occurs in the outer
portion of the Galaxy with high radial motion. Finally, the complete cluster
system can explore a range of equilibrium distribution functions-based models
coupled with information on the potential of the Milky Way. Models with
circular velocity between 10 and 50 kpc stay in the 210-240 km/s range better
reflect the dynamics of clusters.

The Gaia Release 2 data shown in Table 1 calculates the average proper motions
of 150 Milky Way globular clusters (almost the whole known population), with a
typical uncertainty of 0.05mas/yr constrained primarily by systematic errors
Gaia Collaboration et al. (2016). The dataset for Table 1 comes from a
catalogue of mean proper motions of 150 Milky Way globular clusters, derived
from the Gaia data release 2 astrometry, in a paper by Vasiliev. The columns
are as follows: right ascension 2000, declination J2000, distance to the
cluster, line-of-sight velocity, and proper motion right ascension.

Figure 2: The distribution of globular clusters in the 6D phase space, using
both position/velocity and action/angle sky coordinates.

Figure 3: The distribution of globular clusters in the 6D phase space, using
both position/velocity and action/angle sky coordinates (cont.).

In Figure 2 and Figure 3, the sky coordinates of globular clusters maintained
the union of frame attributes for all built-in and user-defined coordinate
frames in the astropy. Individual frame classes hold only the required
attributes for that frame. The Sky Coordinates were analyzed on a three-tiered
system of objects: representations, frames, and a high-level class.
Representations classes are a particular way of storing a three-dimensional
data point (or points), such as Cartesian coordinates or spherical polar
coordinates. Frames are particular reference frames, which stored their data
in different representations, but have well- defined transformations between
each other. The Galaxy occupies a 3D region, hence this two-dimensional
diagram only displays a piece of the plane of the Milky Way from top to
bottom. Across various parts of the stream, clusters form along multiple
dipoles.

Figure 4: Globular clusters sky coordinates .

Figure 5: X, Y, and Z coordinates of Globular Cluster distribution

Figures 4 and 5, as well as variant observations of globular clusters in our
range, can be explained by differences in the evolutionary paths of stars with
similar compositions but different masses. The absolute magnitude at which the
brighter main-sequence stars depart from the main sequence (the turnoff point,
or ”knee”) indicates the cluster’s age, assuming that the majority of the
stars formed simultaneously. It is critical to note that globular clusters in
the Milky Way Galaxy are nearly as old as the universe, averaging
approximately 14 billion years and ranging between approximately 12 billion
and 16 billion years, although these ages are still being revised.

Figure 6: The Hamiltonian orbit calculated by first finding the gravitational
force and calculating how far it moves the object during a single time-step.

Figure 7: A color magnitude plot of exactly where lot exactly where the
globular clusters are at any given time.

To perform the query and to retrieve the data, we used the Vasiliev
distribution to generate an orbital model from the globular cluster date, then
we queried the Gaia science archive to download astrometric and kinematic data
(parallax, proper motion, radial velocity) for a sample of stars near the Sun.
Then we used data exclusively from data release 2 (DR2) from the Gaia mission
module in the astroquery package, astroquery.gaia. By providing an SQL query
to select our galactocentric coordinates and the sky position (ra, dec),
parallax, proper motion components (pmra, pmdec), radial velocity, and
magnitudes, we were able to visualize the orbit our of globular clusters in
focus in Figure 6 and 7.

###  2.3 Dark Matter Distribution

Due to their nearly universal ex situ origin, existing star streams are
important indications of galaxy mergers and dynamical friction inside the
Galactic halo. The stars revolving around the Milky Way’s halo shed light on
the distribution of dark matter in the galaxy, which had previously remained a
mystery to researchers.

Figure 8: Distribution Plot of Dark Matter Subhalo’s via the Galactocentric
Coordinate System.

Figure 8 represents the distribution of dark matter sub halos in the exact
same galactic coordinates as the orbit streams and globular clusters in the
previous section. Their phase-space grouping allows for the creation of a
precise global map of dark matter in the Milky Way, while their internal
structure may provide insight into the small-scale structure of dark matter in
their original home galaxies.

Table 2: Present-day positions and velocities of simulated dark-matter
particles in Galactocentric coordinates.

According to our investigation, the cosmologically motivated Dark Matter Halo
profile was accompanied by reasonable baryonic component models (to account
for the Milky Way’s stellar bulge and disk) as well as reasonable dark matter
component models (to represent the cosmos’ dark matter halo). The flattening
of the dark matter halo, as well as the circular velocity at the Solar radius,
were used as proxy measurements for the Galaxy’s total inner mass. Using the
Galactic potential model, we varied these two parameters in order to estimate
the GD-1 orbits. With this information, we determined that the GD-1 orbits
must have a velocity of 244+/-4 kilometers per second at the solar radius and
that the dark halo density must be in the range of 0.82+0.250.13 light years
at the solar radius (a mildly oblate halo) 2021ApJ...909L..26B. According to
one model, the Milky Way’s mass is 2.5 billion solar masses and is located
within 20 kpc of the Galactic Center. This provides the strongest support for
that model. Recent studies have revealed that there is a greater degree of
agreement among these values in the galactic halo, but there is still
significant disagreement over the shape of the halo, which ranges from mildly
oblate to mildly prolate in shape Bonaca (2021).

Figure 9: The Horbit calculated by first finding the gravitational force and
calculating how far it moves the Dark Matter Subhalo during a single time-
step.

The radial distribution of the subhalo population, shown within the orbits of
in Fig. 9, have a slope equal to or shallower than the slope of the density
profile for each resolution at all radii. At very small radii, subhaloes are
highly deficient, possibly as a result, at least in part, of artificial
disruption by the tidal forces from the central mass concentration. When a
subhalo migrates inwards via dynamical friction to a radius where its central
density is lower than the local density of the host halo, it will likely be
disrupted. Numerical disruption enhanced by the strong tides near host centres
could affect subhaloes. The flattened slope of the radial subhalo distribution
of GRP1 subhaloes relative to the mass profile interior to roughly 0.2rvir for
our highest resolution and interior to 0.3rvir for our lowest resolution
implies that disruption and/or stripping of subhaloes is important in the halo
central region. The increase in radius of the break in subhalo slope with
decreasing resolution suggests that numerical disruption, if present, is worse
for lower resolutions. However, we caution that the location of the break is
not well defined because of Poisson uncertainties. Given the uncertainties, it
is not possible to reliably separate spurious from real disruption that may be
present in our simulations at small radii. Thus, we have no evidence that the
central substructure number density has converged with resolution. Increasing
the mass resolution by a factor of 8 results in roughly a factor of
$\sim$2–2.5 more subhaloes at a given radius beyond roughly 0.3rvir, though
there is substantial noise in this estimate.

##  3 Orbital Methods

The Milky Way galaxy contains over 60 long and thin stellar streams Gull et
al. (2021). Due to their small size, these are expected to be tidally
dissolved globular clusters, although low-mass dwarf galaxy progenitors are
also permitted. Only a few streams are connected directly to a surviving
globular cluster. The vast majority of streams have no apparent progenitor
within them, and this section discusses how such streams formed.

In this section, we combine orbits for all of the Milky Way satellites (dwarf
galaxies and globular clusters) in an attempt to link them to observed stellar
streams. While we have shown that Milky Way streams and globular clusters are
not associated with the plane of satellites as systems, it can still be
interesting to associate individual VPOS members in order to try to establish
a common origin (e.g. group infall). Associating streams with known objects,
independent of the VPOS, can point to the stream’s progenitor and trace the
process of hierarchical formation.

In figure 6, we used the Galactocentric coordinates of the first globular
cluster. This provided the data necessary to calculate the orbit and determine
if the globular cluster was spending the same amount of time everywhere around
the orbit. The Hamiltonian object we created before has a function associated
with it that know to use the gravitational potential we specified already
Piatti (2021). By passing the initial position w0, we can view the size of the
time-step (in our case 1 Myr), and the number of time-steps (in our case 500).
The orbit is calculated by first finding the gravitational force and
calculating how far it moves the object during a single time-step, then
finding the gravitational force at the new position and repeating.

The full modeling of a satellite unraveling into a stellar stream is difficult
because the results are dependent on the shape of the Galactic potential, the
progenitor’s internal and orbital kinematics, and the effects of dynamical
friction and time-dependent substructures like the Galactic bar and the LMC
Piatti (2021). We integrate orbits in static potentials that do not include
the effects of Galactic substructures or the LMC for simplicity and because we
have limited information about each stream.

The initial positions for the globular cluster orbits are the same as in
Section 1.2. Prior simulations of these initial positions, sampling over
uncertainties in each distribution distance and proper motion were ran. We can
get a sense of which pairings are possible given the measurement uncertainties
by counting the fraction of orbits for each Milky Way satellite–stream pairing
that are ‘associated’.

##  4 Results

Figure 10: Combined Diagram of the Stellar Stream, Globular Cluster, and Dark
Matter Subhalo Distribution

As a culmination of the stellar stream, globular cluster, and dark matter data
from Gaia, we have demonstrated that the distributions of the Milky Way and
stellar stream normals (Section 2.1) as well as the orbital poles of globular
clusters (Section 2.2) are not significantly clustered in the direction of the
VPOS normal. It may be tempting to use these results to completely reject the
concept of a plane of satellites. We strongly advise against this, however.
Taking the observed plane of satellite galaxies at face value, we must
consider why other accreted matter – young halo globular clusters and stellar
streams – does not exhibit the same behavior. Figure 10, reveals the expanse
of the Milky Way examined in this project along with the data for each gala
distribution. This plot is imperative to viewing distances from the stellar
stream and determining if Dark Matter had an impact on the streams orbit, as
well as ruling out globular clusters as a possibility.

Figure 11: Combined Diagram of the Stellar Stream, Globular Cluster, and Dark
Matter Subhalo Distribution (Closer to the Stream).

The orbits of previous integrations can be used to show that the stream
encounter could not have been caused by any known globular cluster or dwarf
galaxy, as they have been orbiting for the past 2 billion years. Also, it is
assumed that the molecular cloud in the Milky Way disk was also mass, size,
and impact parameter arguments that the stream encounter could not have been
caused by. This hypothesized collision with a dark matter substructure,
similar to those predicted to exist in galactic halos in LCDM cosmology, is
the most plausible explanation for the gap-and-spur structure. While the GD-1
perturber density is higher than expected, the expected density of LCDM
subhalos in this mass range and area of the Milky Way is lower.

According to the results of orbit integrations performed back in time, the
stream encounter could not have been caused by any known globular cluster or
dwarf galaxy. In our opinion, the most likely explanation for our plot
distribution is that the galaxy GD-1 was previously involved in an encounter
with a clump of dark matter, such as those expected to exist in the galactic
halo. If we can pinpoint the location of the perturber at this time, it will
open the door to new research directions, including the search for additional
observational evidence that the perturber is, in fact, dark matter. Other
stars or gas clouds that are jostled around by the dark matter’s gravity, or
even gamma-rays associated with dark matter annihilations, which occur when
two dark matter particles slam into and destroy each other, releasing a flash
of energy, could be evidence of the existence of dark matter.

Figure 12: Combined Diagram of the Stellar Stream, Globular Cluster, and Dark
Matter Subhalo Distribution: Full Arc

Extragalactic stellar streams may shed new light on low-mass galaxies Gull et
al. (2021). The progenitors of these streams are globular clusters that were
previously a part of the globular cluster system of their host galaxies.
Perhaps this recently dissolved population. On the other hand, assuming that
streams formed recently implies that their host galaxies have larger pre-
infall halo masses, many of which already have a sizable population of
globular clusters. This is especially true for Sagittarius’s accretion, as its
impact on the Milky Way is highly dependent on its mass.

As illustrated in Figure 11 and 12, there are no globular clusters near the
stream impact, which I obtained directly from Gaia Data. Our findings have
broad implications for the use of stellar streams as dark matter tracers. To
mention a few: (1) combining shorter streams into a single, longer structure
increases their sensitivity to the gravitational potential’s global
properties; (2) high clustering of many stellar streams in phase space can be
used to constrain the gravitational potential directly; (3) if the retrograde
streams represent debris from a single merger event, they depict dynamical
friction in action, whose magnitude would be much greater than what is
visualized.

We conclude by discussing the most urgently needed new stream observations. To
extend this study to the entire population of streams in the Milky Way, high-
quality proper motions of streams are required, many of which are remote and
faint. Globular Clusters in the Milky Way. Our initial goal was to get
familiar with globular clusters in the Milky Way. By visualizing globular
clusters as point-mass objects, we were able to develop an improved
understanding of possible perturbers.

##  5 Discussion and Conclusions

GD-1 is located between 8 and 10 kpc from the Sun, according to an analysis of
the distances to various parts of the stream. There is only a shallow gradient
across 45 degrees of the sky, according to the analysis Benson (2021). It is
possible to see distinct density variations in the stream from a matched
density map, including deviations from a single orbital track and preliminary
evidence for stream fanning. Aside from that, we notice a distinct under-
density at $\phi_{1}=-40$ deg in the center of the stream track, which is
surrounded on either side by dense stream segments. It is clear that
interactions with the Milky Way disk or other dark matter sub-halos have
caused the GD-1 stream to be disrupted, as we have discovered.

Similarly, using our compilation of stream anchor points in Table 1, we
discovered that stellar stream normals are not clustered around the normal,
and that these findings are unaffected by geometric effects due to our
observers’ position.

We were successfully able to plot the sky positions of globular clusters
(i.e., R.A. and Dec). We found asymmetric distribution, and when studying
objects in the Milky Way, it was beneficial to convert the equatorial sky
coordinates into galactic sky coordi-nates, in whichl= 0, b= 0 points towards
the Galactic center. Using astropy coordinates, we converted the equatorial
coordinates of the Milky Way globular clusters into galactic coordi-nates and
plot them. We discovered multiple possible instances of Milky Way globular
clusters being associated with stellar streams using our orbit modeling
procedure. According to our orbit modeling, it is quite possible that the
globular clusters in the combined graphics in the previous section had a
significant effect on the stream.

Finally, we reiterated our caution against using our findings to invalidate
the concept of the plane of satellites. The satellite galaxies of the Milky
Way have a highly clustered distribution of orbital poles within the spread
figures of globular clusters, which contradicts expectations.

Our findings, however, refute the notion that the Milky Way’s satellite plane
is a part of a composed of satellite galaxies, globular clusters, and stellar
streams. They demonstrate that dark matter subhalos are the most likely
explanation for the GD-1 stream impact. Our hope is that these findings will
elicit interesting discussions about the stability of planar configurations
and will serve as a springboard for future research on satellite planes in the
context of hierarchical structure formation.

##  6 Future Applications

Known as dark matter, it is an intangible form of matter that accounts for the
vast majority of the universe’s mass and is responsible for the formation of
galaxies. While astronomers are unable to directly observe dark matter, they
can indirectly detect its presence by observing how the gravity of stars and
galaxies is affected by the matter’s gravitational pull 2021CmPhy...4..123J.
It can be difficult, if not impossible, to detect the tiniest dark matter
formations by looking for embedded stars because of the sparse star population
in the region. Even in the absence of stars, the search for dark matter
concentrations has proven to be problematic. Prior to Hubble’s discovery, the
Hubble research team used a technique that did not require them to look for
the gravitational influence of stars as tracers of dark matter, which saved
them time and money. In particular, the team concentrated on eight extremely
bright and distant cosmic ”streetlights,” which they designated as quasars
(regions around active black holes that emit enormous amounts of light).
Astronomers measured the amount of distortion caused by the gravity of a
massive foreground galaxy acting as a magnifying lens on the light emitted by
oxygen and neon gas orbiting each quasar’s black hole, according to the
findings.

To associate a satellite with a stream, we can use gala to integrate the
stream’s orbit backwards in time. It is possible to examine whether the orbit
of each object and stream pairing passes through the on-sky position of the
stream’s anchor points at approximately the correct heliocentric distance. An
orbit is considered an association if it passes through both stream anchor
points within the corresponding 1 angular and distance uncertainties. The next
step after this duration of research is analyzing the Milky Way globular
clusters and stellar streams for dynamical alignment with the Milky Way’s
plane of satellites.

Space telescope SPHEREX, which will be launched by NASA in 2024 and will be
the first of its kind, may provide the lens needed to advance our
understanding of Milky Way stream impact behavior. SPHERE is expected to
launch no earlier than June 2024 and no later than April 2025 Cigarran Diaz &
Mukherjee (2021). During its two-year mission, it will map the entire sky four
times, resulting in a massive database of stars, galaxies, nebulas (space
clouds of gas and dust), and a plethora of other celestial objects that will
be available to researchers. With a total of 102 near-infrared colors
observed, SPHERE will be the first NASA mission to create a full-sky near-
infrared spectroscopy map, a first for the agency Cigarran Diaz & Mukherjee
(2021).

Large and medium-sized galaxies have been found to contain concentrations of
dark matter, but until now, much smaller dark matter clumps have not been
discovered in their immediate vicinity. Without any observational evidence for
such small-scale clumps, some researchers have developed alternative theories,
such as warm dark matter, to account for their observations of them
Chakrabarty et al. (2021). According to this hypothesis, dark matter particles
are extremely fast, zipping along at a rate that prevents them from merging
and forming smaller concentrations of dark matter. It has been discovered that
dark matter is colder than previously thought at smaller scales. There have
been other observational tests of dark matter theories carried out in the
past, but ours provides the strongest evidence to date for the presence of
small clumps of cold dark matter in the universe. By combining the most recent
theoretical predictions, statistical tools, and new observations, we can
produce a more robust and up-to-date result than was previously possible
through the theoretical simulation of this research.

##  7 Acknowledgments

I would like to acknowledge the support of Dr. Ana Bonaca who provided
continued guidance, support, and a motivating spirit to explore every facet of
the perturbation to analyze probable causes. I would like to acknowledge Rohan
P. Naidu for providing data to help me better understand and curate my
combined models.

The SAO REU program is funded in part by the National Science Foundation REU
and Department of Defense ASSURE programs under NSF Grant no. AST1852268 and
2050813, and by the Smithsonian Institution.

## References

  * Benson (2021) Benson, A. J. 2021, MNRAS, 505, 2159, doi: 10.1093/mnras/stab1413
  * Bílek et al. (2021) Bílek, M., Zhao, H., Famaey, B., et al. 2021, arXiv e-prints, arXiv:2107.05667.  https://arxiv.org/abs/2107.05667
  * Bonaca (2021) Bonaca, A. 2021, in AAS/Division of Dynamical Astronomy Meeting, Vol. 53, AAS/Division of Dynamical Astronomy Meeting, 401.06
  * Bonaca et al. (2021) Bonaca, A., Naidu, R. P., Conroy, C., et al. 2021, ApJ, 909, L26, doi: 10.3847/2041-8213/abeaa9
  * Bravo et al. (2021) Bravo, M., Gawiser, E., Padilla, N. D., et al. 2021, arXiv e-prints, arXiv:2106.08818.  https://arxiv.org/abs/2106.08818
  * Chakrabarty et al. (2021) Chakrabarty, S. S., Han, Y., Gonzalez, A. H., & Sikivie, P. 2021, Physics of the Dark Universe, 33, 100838, doi: 10.1016/j.dark.2021.100838
  * Cigarran Diaz & Mukherjee (2021) Cigarran Diaz, C., & Mukherjee, S. 2021, arXiv e-prints, arXiv:2107.12787.  https://arxiv.org/abs/2107.12787
  * Gaia Collaboration et al. (2016) Gaia Collaboration, Prusti, T., de Bruijne, J. H. J., et al. 2016, A&A, 595, A1, doi: 10.1051/0004-6361/201629272
  * Gull et al. (2021) Gull, M., Frebel, A., Hinojosa, K., et al. 2021, ApJ, 912, 52, doi: 10.3847/1538-4357/abea1a
  * Kohler (2021) Kohler, S. 2021, Seeking the Origins of Galactic Stellar Streams, AAS Nova Highlights
  * Kuzma et al. (2021) Kuzma, P. B., Ferguson, A. M. N., & Peñarrubia, J. 2021, arXiv e-prints, arXiv:2108.02531.  https://arxiv.org/abs/2108.02531
  * Mandelker (2021) Mandelker, N. 2021, in Galaxy Cluster Formation II (GCF 2021) - Virtual Workshop, 63, doi: 10.5281/zenodo.5028479
  * Mirabal & Bonaca (2021) Mirabal, N., & Bonaca, A. 2021, arXiv e-prints, arXiv:2105.12131.  https://arxiv.org/abs/2105.12131
  * Piatti (2021) Piatti, A. E. 2021, MNRAS, 505, 3033, doi: 10.1093/mnras/stab1532
  * Tadross & Hendy (2021) Tadross, A. L., & Hendy, y. h. 2021, arXiv e-prints, arXiv:2104.11778.  https://arxiv.org/abs/2104.11778
  * Vasiliev & Baumgardt (2021) Vasiliev, E., & Baumgardt, H. 2021, MNRAS, 505, 5978, doi: 10.1093/mnras/stab1475
  * Webb & Bovy (2021) Webb, J. J., & Bovy, J. 2021, arXiv e-prints, arXiv:2108.02217.  https://arxiv.org/abs/2108.02217


# Task Planning for Object Rearrangement in Multi-room Environments

Karan Mirakhor, Sourav Ghosh, Dipanjan Das, Brojeshwar Bhowmick

###### Abstract

Object rearrangement in a multi-room setup should produce a reasonable plan
that reduces the agent’s overall travel and the number of steps. Recent state-
of-the-art methods fail to produce such plans because they rely on explicit
exploration for discovering unseen objects due to partial observability and a
heuristic planner to sequence the actions for rearrangement. This paper
proposes a novel hierarchical task planner to efficiently plan a sequence of
actions to discover unseen objects and rearrange misplaced objects within an
untidy house to achieve a desired tidy state. The proposed method introduces
several novel techniques, including (i) a method for discovering unseen
objects using commonsense knowledge from large language models, (ii) a
collision resolution and buffer prediction method based on Cross-Entropy
Method to handle blocked goal and swap cases, (iii) a directed spatial graph-
based state space for scalability, and (iv) deep reinforcement learning (RL)
for producing an efficient planner. The planner interleaves the discovery of
unseen objects and rearrangement to minimize the number of steps taken and
overall traversal of the agent. The paper also presents new metrics and a
benchmark dataset called MoPOR to evaluate the effectiveness of the
rearrangement planning in a multi-room setting. The experimental results
demonstrate that the proposed method effectively addresses the multi-room
rearrangement problem.

## Introduction

Organizing an untidy household according to user preferences is a challenging
task that encompasses multiple aspects, including perception, planning,
navigation, and manipulation (Batra et al. 2020). When an agent engages in
multi-room object rearrangement, it must rely on sensor data and prior
knowledge to devise a comprehensive plan that involves sequencing the object
movement to achieve the desired tidy state. The specifications for this goal
state can be defined using various modalities such as geometry, images,
language, etc. (Batra et al. 2020).

The existing research on object rearrangement has predominantly concentrated
on single-room setups, prioritizing perception and commonsense reasoning
aspects. However, these studies often assume that navigation and manipulation
abilities are already in place, and they do not emphasize the importance of
efficient planning in the rearrangement process. Methods such as (Kant et al.
2022; Sarch et al. 2022) employ image or language-based commonsense reasoning
to identify misplaced objects within the agent’s egoview and then utilize sub-
optimal heuristic planners for rearrangement. However, their reasoning based
anomaly detectors fail to address cases where the goal position of an object
is blocked or requires swapping with another misplaced object. Whereas methods
such as (Weihs et al. 2021; Gadre et al. 2022; Trabucco et al. 2022) focus on
egocentric perception and employ image, graph, or semantic map-based scene
representations to identify misplaced objects. They use greedy planners to
sequence actions for rearrangement according to explicit user specifications.
(Sarch et al. 2022) also performs user-specific room rearrangement by
utilizing semantic relations to identify misplaced objects within the agent’s
egoview through exploration, followed by rearrangement using a heuristic
planner. Most existing methods (Kant et al. 2022; Sarch et al. 2022; Trabucco
et al. 2022; Gadre et al. 2022) explicitly explore the room to locate objects
that are initially outside the agent’s egoview, compensating for the partial
information provided by the egocentric view. As shown in Fig 1, recent methods
(Sarch et al. 2022; Trabucco et al. 2022; Ghosh et al. 2022) suffer from high
traversal costs due to their sub-optimal planner, which becomes increasingly
problematic as the size of the rearrangement space grows in multi-room
settings.

Efficient planning is crucial for improving the effectiveness of rearrangement
tasks by optimizing the sequence of actions and minimizing the time and effort
required by the robot to achieve the desired goal state. In the context of
rearrangement task planning, (Ghosh et al. 2022) proposes a method that
assumes complete room visibility from a bird’s eye perspective. Their approach
aims to overcome planning challenges, including the combinatorial expansion of
rearrangement sequencing and swap case resolution, without requiring an
explicit buffer. However, their Euclidean distance-based reward does not
minimize overall agent traversal during planning, as shown in Fig 1. Their
state representation lacks scalability to large numbers of objects.
Additionally, their parameter network suffers from two main issues: (i)
predicting the goal location of non-blocked misplaced objects, which is
already known from the goal state in rearrangement, leading to performance
degradation. (ii) predicting the buffer location for swap cases without
considering the object’s geometry and the available free space, resulting in
poor generalization.

Figure 1: The graph shows the agent traversal for existing methods (Sarch et
al. 2022; Trabucco et al. 2022; Ghosh et al. 2022) v/s Ours with increasing
rearrangement area highlighting need for efficient planning. The error bars
show the standard deviation in the average traversal.

Furthermore, their reliance on ground truth object positions in both the
current and goal states is impractical in real-life scenarios. In contrast,
our focus is on a novel, practical and broader aspect of the object
rearrangement problem in the multi-room setting, under partial observability
using the egocentric camera view of the agent. Our main emphasis is on
efficient task planning, which is necessary for effective rearrangement as
illustrated in Fig 1. Task planning for efficient multi-room object
rearrangement under partial observability as depicted in Fig 2, presents
several significant challenges such as (i) uncertainty regarding the location
of unseen objects due to partial observability, (ii) scalability to a large
number of objects, (iii) combinatorial expansion of search space for the
sequencing due to simultaneous object discovery (for unseen objects) and
rearrangement, (iv) minimizing the overall traversal by the agent during
simultaneous object discovery and rearrangement. (v) resolving blocked goal
and swap cases (object collision) without explicit buffer.

In this paper, we present a novel hierarchical method for task planning that
aims to overcome the challenges mentioned earlier. Initially, our agent
utilizes egocentric perception to explore the house once and capture the
semantic and geometric configuration (Batra et al. 2020) of objects and
receptacles using any RGBD sensor (Patra et al. 2012) or SfM (Bhowmick et al.
2017), thus obtaining the goal state. The agent can also explore based on user
commands (Sriram et al. 2019). Following this, the objects within the rooms
are shuffled to make an untidy current state. Our hierarchical method then
divides the task planning problem into three components - the discovery of
unseen objects, collision resolution, and planning - to minimize the agent’s
overall traversal while simultaneously conducting object search, collision
resolution, and rearrangement. First, we propose a novel commonsense
knowledge-based Unseen Object Discovery Method using large language models
(LLMs) (Liu et al. 2019; Kant et al. 2022), that leverages the object-room-
receptacle semantics to predict the most probable room-receptacle for an
unseen object. Second, we propose a novel Cross-Entropy Method (CEM) based
collision resolution to produce buffer spaces for swap cases considering the
objects’ geometry and the size of receptacle-free spaces. Third, we use a Deep
RL-based planner to produce an action sequence for simultaneous object search
and rearrangement. To this extent, we define the Deep RL state space with a
novel Directed-Graph-based representation that combines the geometric position
of objects in the current state, the goal state along with the agent’s initial
position. The proposed representation effectively encodes the scene geometry,
facilitating rearrangement planning and enabling scalability of the Deep RL
state space to accommodate a large number of objects and scene invariant. By
combining all the previously mentioned components in a thoughtful manner, we
successfully address the combinatorial optimization problem in rearrangement.

The major contributions of this paper are :

  1. 1.

To the best of our knowledge, this is the first end-to-end method to address
the task planning problem for multi-room rearrangement from egocentric view
under partial observability, using a user-defined goal state.

  2. 2.

A novel Unseen Object Discovery Method that leverages object-room-receptacle
semantics using LLM to predict the most probable room-receptacle for an unseen
object.

  3. 3.

Introducing Cross-Entropy Method based Collision Resolution to find buffer
spaces for swap cases considering the objects’ geometry and the size of free
spaces.

  4. 4.

A new scalable and scene invariant Directed State Graph containing the
geometric information about the agent and objects in the current and goal
state as the Deep RL state.

  5. 5.

Use of Deep RL based planner to overcome combinatorial expansion in
rearrangement sequencing and, to optimize the overall traversal and the number
of steps taken.

  6. 6.

we devise a set of Evaluation criteria to gauge the efficacy of our method in
terms of the number of steps taken and the overall agent traversal.

  7. 7.

To address the inadequacies in existing benchmarks (Weihs et al. 2021) for
evaluating multi-room task planning under partial observability, we introduce
the MoPOR - Benchmark Dataset.

Figure 2: (a) shows the top down view of our Rearrangement task and (b) is the
agent’s initial egocentric view in the untidy current state for the same
setup. The solid 3D bounding boxes indicate the desired goal state for all
objects, while the dashed ones show the initial positions of visible objects
in the untidy current state. The dotted 3D bounding boxes represent initial
positions of unseen objects in the untidy current state. The apple (yellow),
an unseen object is inside the kitchen-fridge, while the vase (blue), pillow
(pastel cyan) and sponge (magenta) is on the living-table, living-sofa and
bedroom-chair respectively. There are two scenarios: a blocked goal case with
the vase (blue) and pillow (pastel cyan) and a swap case between the bowl
(green) and kettle (red).

## Methodology

### Preliminaries

For our setup, we use the apartment scenes from ProcThor (Deitke et al. 2022).
To begin our multi-room rearrangement, we perform exploration (Sarch et al.
2022) once in the entire scene to capture the user-specified goal state. Using
the RGB-D image and egomotion information at each step from Ai2Thor (Kolve et
al. 2017), the agent generates a 2D occupancy map ($\mathcal{M}^{\text{2D}}$)
for navigation and a 3D map ($\mathcal{M}^{\text{3D}}$) to augment the
positions of objects and receptacles in 3D to a global reference frame. A
d-DETR (Zhu et al. 2021) detector is used on each RGB image to obtain 2D
bounding boxes and semantic labels for objects and receptacles. The
corresponding 3D centroids are obtained using depth images, camera intrinsics
and extrinsics. Using object segmentation (Rusu and Cousins 2011) on the point
cloud of $\mathcal{M}^{\text{3D}}$ the agent records the corners of the 3D
bounding boxes of each object in the goal state. At the end of the goal state
exploration, we generate an object list
$\textbf{O}=\\{[\textbf{L}_{i},\textbf{P}_{i},\textbf{B}_{i}],i=1,2,..,N\\}$
and a room-receptacle list
$\textbf{R}=\\{[\textbf{L}^{R}_{i},\textbf{P}^{R}_{i}],i=1,2,..,N_{R}\\}$.
Here, $N$, L, $\textbf{P},\,\textbf{B}$ are the total numbers of objects,
their semantic labels, 3D object centroids and the corners of their 3D
bounding boxes respectively. $N_{R}$, $\textbf{L}^{R}$ and $\textbf{P}^{R}$
are the total numbers of receptacles, their semantic labels including the room
name from Ai2Thor (Kolve et al. 2017), and the 3D receptacle centroids
respectively. After capturing the goal state, we randomly shuffle a few
objects to make the room untidy and fork the agent at a random location in the
room. In this untidy current state, the agent’s knowledge is limited to the
visible part of the scene in its egocentric view and thus only a set of
objects
$\textbf{O}^{V}=\\{[\textbf{L}^{V}_{i},\textbf{P}^{V}_{i}],i=1,2,..,N_{V}\\}$
are captured using egocentric perception. $N_{V}$, $\textbf{L}^{V}$ and
$\textbf{P}^{V}$ are the number of visible objects, their semantic labels and
3D object centroids respectively in the current state. Additionally, the agent
creates a 2D grid map of free receptacle spaces $\mathcal{M}^{\text{R}}$ in
the current state using the instance segmentation mask from Ai2Thor along with
the depth image and egomotion to aid in the collision resolution. Comparing O
with $\textbf{O}^{V}$ allows for determining only the semantics of unseen
objects in the current state
$\textbf{O}^{\overline{V}}=\\{[\textbf{L}^{\overline{V}}_{i}],i=1,2,..,N_{\overline{V}}\\}$,
where $N_{\overline{V}}$ is the number of unseen objects and
$\textbf{L}^{\overline{V}}$ their semantic labels.

Input: Agent’s egoview RGB-D & egomotion

Result: Actions $\mathcal{A}=\\{a_{1},..,a_{N},done\\}$

1 Create $O,R,\mathcal{M}^{\text{2D}},\mathcal{M}^{\text{3D}}$ from Goal
State;

2 Create $O^{V},O^{\overline{V}},\mathcal{M}^{\text{R}}$ from Current State;

3 while  _a is not done_ do

4 $P^{\overline{V}R}\leftarrow\textit{UODM (}O^{\overline{V}},R\text{)}$;

5 if  _Collision between $O^{V}_{i}$ and $O^{V}_{j}$ _ then

6 if  _Swap case_ then

7 $P^{B}_{i}\leftarrow\textit{buffer for }O^{V}_{i}\textit{near}P^{V}_{j}$;

8 $P^{B}_{j}\leftarrow\textit{buffer for }O^{V}_{j}\textit{near}P^{V}_{i}$;

9

10 else if  _$O^{V}_{i}$ blocks goal of $O^{V}_{j}$_ then

11 $P^{B}_{j}\leftarrow P^{V}_{j}$;

12

13 else if  _$O^{V}_{j}$ blocks goal of $O^{V}_{i}$_ then

14 $P^{B}_{i}\leftarrow P^{V}_{i}$;

15

16

17 $s\leftarrow\textit{DSG (}O,P^{B},O^{V},O^{\overline{V}}\cup
P^{\overline{V}R}\text{)}$ ;

18 $a=\underset{a\in\mathcal{A}}{\arg\max}\,Q_{\theta}(s,a)$;

19 if  _$a=={O^{V}_{i}}$_ then

20 Pick-Place object $O^{V}_{i}$;

21 else if  _$a=={O^{\overline{V}}_{i}}$_ then

22 if  _$O^{\overline{V}}_{i}$ is found during traversal_ then

23 go to 1;

24 else if  _$O^{\overline{V}}_{i}$ is found in $P^{\overline{V}R}_{i}$_ then

25 Pick-Place object $O^{\overline{V}}_{i}$;

26 else

27 Remove predicted receptacle from R;

28

29 Remove receptacle/s without any $O^{\overline{V}}$ from R;

30 Update ($O^{V},O^{\overline{V}},R,\mathcal{M}^{\text{R}})$;

31

Algorithm 1 Algorithm for Task planner

### Overview

Given O, R, $\textbf{O}^{V}$ and $\textbf{O}^{\overline{V}}$, the agent must
efficiently plan a sequence of actions
$\mathcal{A}=\\{a_{1},...,\text{done}\\}$ to discover
$\textbf{O}^{\overline{V}}$ and simultaneously rearrange the misplaced objects
in $\textbf{O}^{V}$ to their desired goal position in the current state. The
Unseen Object Discovery Method leverages the object-room-receptacle
relationship using $\textbf{O}^{\overline{V}}$ and R to predict the probable
location $\textbf{P}^{\overline{V}R}$ for $\textbf{O}^{\overline{V}}$. The
Collision Resolution and Buffer Management method, uses the locations and
object bounding boxes from O and $\textbf{O}^{V}$ along with
$\mathcal{M}^{\text{R}}$ to detect object collision and predict the resolved
goal location $\textbf{P}^{B}$ for the blocked goal and swap case objects.
With O, $\textbf{O}^{V}$, $\textbf{O}^{\overline{V}}$,
$\textbf{P}^{\overline{V}R}$, and $\textbf{P}^{B}$, the agent constructs a
compact representation of the state space using the Directed State Graph. The
Deep RL planner uses the state space to produce the most optimal action
$a\in\mathcal{A}$ to either pick-place $\textbf{O}^{V}$ or search
$\textbf{O}^{\overline{V}}$ with the objective of minimizing the overall
traversal and the number of steps.

### Unseen Object Discovery Method

In the context of multi-room rearrangement, the agent must identify unseen
objects within the untidy current state, whether in the same room or different
rooms, to plan actions effectively. For example, as shown in Fig 2, when the
agent is in the kitchen, it cannot see items like the apple in the fridge-
kitchen, or the vase (blue), pillow (pastel cyan), and sponge (magenta) in the
living room and bedroom. To address this, we propose the Unseen Object
Discovery Method (UODM), leveraging Large Language Models’ (LLMs) commonsense
knowledge to predict probable room-receptacles for unseen objects. However,
LLMs may not always provide human-commonsense compliant predictions for untidy
scenes, as detailed in Appendix111http://tinyurl.com/14043Appendix. Therefore,
we leverage the semantic relationship between $\textbf{O}^{\overline{V}}$ and
R by finetuning their output embeddings from LLM using the AMT dataset (Kant
et al. 2022).

$\displaystyle L_{CE}=-\frac{1}{N_{E}}\sum^{N_{E}}\sum_{i=1}^{2}Y_{i}\log
p_{i}$ (1)
$L_{MSE}=\frac{1}{N_{SR}}\sum^{N_{SR}}_{i=1}(\overline{\chi}_{i}-\chi_{i})^{2}\vspace{-0.75mm}$
(2)

Unlike previous methods such as (Sarch et al. 2022), which focus solely on
object-receptacle relationships, our method takes into account the object and
room-receptacle semantic relationship to handle multi-room setups. We generate
the RoBERTa embeddings $\textbf{E}^{\overline{V}R}$ for pairwise concatenated
labels of unseen objects
$\\{\textbf{L}^{\overline{V}}_{i}\\}_{i=1,2,..,N_{\overline{V}}}$ and room-
receptacles $\\{\textbf{L}^{R}_{i}\\}_{i=1,2,..,N_{R}}$. As each object is
paired with every receptacle, the total number of embeddings for all the
object-room-receptacle (ORR) pairs is $N_{E}=N_{\overline{V}}\times N_{R}$.

Figure 3: Overall hierarchical pipeline of our proposed method.

We devise a two-step approach to enhance the accuracy and reduce the search
space for finding the exact ORR from the $N_{E}$ pairs. First, we use an MLP-
based Filter Network (FN) to predict the probability $\\{p_{i}\\}_{i=1,2}$ for
ORR from $N_{E}$. We train this network using a Cross-Entropy Loss ($L_{CE})$
on the ground truth class labels $\\{Y_{i}\\}_{i=1,2}$ for each ORR in the
dataset, as shown in Eq 1. Here, the class labels $\\{i=1:\textit{Probable
Class},\,2:\textit{Implausible Class}\\}$ indicate the probability of finding
a misplaced object at a given room-receptacle. In the second step, we use a
regression-based Ranking Network (RN) to estimate the probability scores
$\\{\overline{\chi}_{i}\\}_{i=1,2,...N_{SR}}$ for embeddings of probable
class, with $N_{SR}$ representing the total number of such embeddings. We use
a fully connected MLP, and train this network using MSE Loss ($L_{MSE}$) as
shown in Eq 2, with respect to the ground truth probability scores
$\\{\chi_{i}\\}_{i=1,..N_{E}}$ from human annotations. The room-receptacles
with the highest scores from RN are selected as the probable room-receptacles
for the unseen objects $O^{\overline{V}}$. Finally, the predicted room-
receptacle’s position
$\\{\textbf{P}^{\overline{V}R}_{i}\\}_{i=1,..N_{\overline{V}}}$ from the goal
state is used as the location for $\textbf{O}^{\overline{V}}$ in the current
state, as receptacles are static in the scene.

To prevent fruitless searches, we implement simple strategies as shown in Line
1:1. Line 1 exemplifies a situation where the agent comes across a receptacle
on its path that does not contain any $O^{\overline{V}}$. As a result, the
agent excludes it from future search attempts.

### Collision Resolution and Buffer Management

To ensure the robustness of our task planning in collision scenarios, we need
to identify and resolve the blocked goal and swap cases by leveraging the
geometry of objects ($B$) and the size of free spaces in receptacle occupancy
map ($\mathcal{M}^{\text{R}}$). We identify the collision cases between
objects in $\textbf{O}^{V}$, using $B_{i},B_{j}\in B$ for each pair of objects
and $\mathcal{M}^{\text{R}}$. We project $B_{i},B_{j}$ onto
$\mathcal{M}^{\text{R}}$ and find the maximum size rectangle $B^{M}_{i}$ and
$B^{M}_{j}$ that encloses their projections respectively. Additionally, we
require the projected goal positions $p^{M}_{i},p^{M}_{j}$ of the two objects
in $\mathcal{M}^{\text{R}}$. To identify the type of collision case, we check
(i) $B^{M}_{i}$ $\cap$ {$B^{M}_{j}\rightarrow p^{M}_{j}\\}(B^{M}_{j}$ centred
at $p^{M}_{j})\neq\phi$ and (ii) $\\{B^{M}_{i}\rightarrow p^{M}_{i}\\}$ $\cap$
$B^{M}_{j}\neq\phi$.Three cases arise based on these conditions: (i) If none
of the conditions are true, there is no collision, (ii) If only one of the
conditions is true, it is a blocked goal case, and (iii) If both conditions
are true, it is a swap case.

To resolve swap cases, we find a buffer space in a time bound and effective
manner, using a stochastic optimization over $p_{k}\in\mathcal{M}^{\text{R}}$
that maximizes the objective function $\mathcal{F}(p_{k})$ in Eq 3222shows an
objective function to find a buffer for $B^{M}_{i}$ near
$\\{B^{M}_{j}\rightarrow p^{M}_{j}\\}$. To perform this optimization, we use
the Cross-Entropy Method (CEM) - a simple derivative-free optimization
algorithm, which is parallelizable and moderately robust to local optima
(Rubinstein and Kroese 2004). CEM samples a batch of $N_{f}$ points from the
free spaces in $\mathcal{M}^{\text{R}}$ at each iteration. Then, it fits a
Gaussian distribution to the best $M_{f}$ $\leq$ $N_{f}$ samples based on
$\mathcal{F}(p_{k})$. Further samples for the next batch of $N_{f}$ are taken
from this Gaussian.

$\mathcal{F}(p_{k})=\begin{dcases}e^{-\mathcal{E}(p_{k},p^{M}_{j})},&\text{if
}\\{B^{M}_{i}\rightarrow p_{k}\\}\cap\\{B^{M}_{j}\rightarrow
p^{M}_{j}\\}=\phi\\\ 0,&\text{otherwise}\end{dcases}$ (3)

Here, $\mathcal{E}(p_{k},p^{M}_{j})=\|(p_{k}-p^{M}_{j})\|_{2}$. The buffer
locations obtained from CEM are used to update $P^{B}$ for swap case objects
as shown in Line 1:1. Once a swap object is placed in its buffer space, it is
considered a blocked goal case. To resolve a blocked goal case, the goal
blocked object is made static temporarily, unless the goal blocking object
vacates its goal position. This is done by updating $P_{B}$, as in Line 1:1.

### Rearrangement planner

Rearrangement planning is a long-horizon problem that suffers from the
combinatorial expansion with increasing number of objects. To this end, we
employ data-driven Q-learning (Watkins and Dayan 1992) that leverages the
Bellmann principle (Bellman 1957) for achieving optimality in long horizon
problems. Moreover, to aid the planner we use the Directed spatial graph as
the state space which allows for scene invariance and scalability to a large
number of objects.

#### Directed State Graph

We present a directed spatial graph ($\mathcal{G}=\\{V,E\\}$) to create a
concise state space representation to aid the planner in efficient
rearrangement sequencing. The edges in this directed graph represent all the
feasible paths for rearrangement completion, disregarding the redundant
information present in an undirected graph, thereby improving the training
efficiency (refer Appendix2). The nodes $V$ (shown in Fig 3) contain (i) the
agent node with 3D position of the agent, (ii) the source nodes with current
object positions and labels from $\\{O^{V},\\{O^{\overline{V}}\cup
P^{\overline{V}R}\\}\\}$ and (iii) the goal nodes with the goal object
positions and labels from $O$. Unlike (Ghosh et al. 2022), we include the
agent node in the Deep RL state space, as it enables the planner to
effectively select the initial action based on the agent’s position with
respect to the objects’. The directed edges of the graph connect: (i) the
agent node to every source node, (ii) the source node of each object to its
respective goal node, and (iii) the goal node of each object to the current
node of every other object. The edge attributes
$E_{c}=\\{\mathcal{D}(p^{G}_{i},p^{G}_{j})_{i\neq j}\\}$ include the length of
the shortest collision free path $\mathcal{D}(p^{G}_{i},p^{G}_{j})_{i\neq j}$
from the node position $p^{G}_{i}$ to connected node position $p^{G}_{j}$.
$\mathcal{D}(p^{G}_{i},p^{G}_{j})_{i\neq j}$ is computed using BFS algorithm
between the 2D projections of $p^{G}_{i},p^{G}_{j}$ on
$\mathcal{M}^{\text{2D}}$. For unseen objects in the current state, the source
object nodes in $G$ are augmented with $P^{{\overline{V}}R}$ from UODM.
Similarly, for blocked goal and swap cases, the goal nodes of such objects are
updated with $P^{B}$ from Collision Resolution and Buffer Method. This graph
representation helps the Deep RL state space to understand the geometric
information of the rearrangement context. We use a Graph Convolution Network
(GCN) to generate meaningful graph embedding from $\mathcal{G}$, that enables
the Deep RL state space to remain scalable and scene invariant.

#### Deep RL based Planner

The task planner should efficiently plan a sequence of actions to
simultaneously (i) rearrange visible objects and (ii) search for unseen
objects at probable receptacles. This reduces the agent’s overall travel time
as (i) it eliminates the requirement of explicit exploration to find unseen
objects and (ii) the rearrangement of visible objects inevitably leads to the
discovery of some unseen objects. To accomplish these objectives, we utilize a
Conservative Q-Learning based on reinforcement learning, which is similar to
the approach presented in (Kumar et al. 2020). The state space for Deep RL is
defined as $s=Z_{p}$ and action $a\in\mathcal{A}$ denotes the selected object
in $\textbf{O}^{V}$ or $\textbf{O}^{\overline{V}}$. Our approach follows the
principles of a Markov Decision Process (MDP), where a reward $r(s,a)$ is
received at each time step $t$ for selecting $a$ from the policy
$\pi(a|s)=\underset{a\in\mathcal{A}}{\arg\max}\,Q_{\theta}(s,a)$, that moves
the agent from the current state $s$ to the next state $\bar{s}$. So according
to Bellman equation Eq 4, our objective is to minimize the temporal difference
error to get the desired action.

$\displaystyle L_{TD}=\underset{(s,a,\bar{s})\leftarrow
R_{B}}{\frac{1}{2}\,\mathbb{E}}[(r(s,a)+\gamma\,\underset{\bar{a}\in\mathcal{A}}{\max}\,Q_{\bar{\theta}}(\bar{s},\bar{a})-Q_{\theta}(s,a))^{2}]$
(4)

Here, ${\theta}$ and $\bar{\theta}$ are the parameters of the Q-Network and
target Q-Network respectively and $R_{B}$ is the replay buffer. The Q-Network
tends to overestimate the policy value, which is undesirable and affects the
sampling efficiency (refer Appendix2). To prevent this, we employ the
Conservative Q-Learning technique similar to that used in (Kumar et al. 2020).
This ensures that the expected value of a policy under the learned Q-function
provides a lower-bound estimate of its true value. Therefore, the combined
loss obtained on adding the conservative lower bound loss to Eq 4 is shown in
Eq 5

$\displaystyle L_{CQL}=\underset{\underset{a\sim\pi(a|s)}{s\leftarrow
R_{B}}}{\alpha\,(\mathbb{E}}\,[Q_{\theta}(s,a)]-\underset{s,a\leftarrow
R_{B}}{\mathbb{E}}\,[Q_{\theta}(s,a)])+L_{TD}$ (5)

Here $\alpha\geq 0$ is tradeoff factor between the conservative loss term and
the TD error. When it comes to Long Horizon planning, using the sparse reward
is not an efficient method for training Deep RL, as noted in (Gehring et al.
2021).

Therefore, we compute a hierarchical dense reward structure which has three
components. (i) Rearrange-able Object Reward : If the agent selects a
misplaced object (an object whose position in O and $\textbf{O}^{V}$ differs
by at least 10cm), it receives a reward equal to the negative path length
required for the agent to complete the pick-and-place action for that object.
(ii) Static Object Reward : Penalizes the agent for moving non-rearrangeable
or static objects, thus preventing redundant moves. This reward also prevents
the Deep RL planner from choosing the goal-blocked object which is temporarily
set as static by making its goal position same as its current position, unless
the goal-blocking object vacates its goal position. (iii) Completion Reward :
If the agent correctly rearranges only the misplaced objects, it gets a high
positive reward. (iv) Collision Resolution : It prioritizes the goal-occupied
objects (objects which occupy the goal location of other objects) to move
first instead of the goal-blocked objects. More details regarding the reward
structure and the choice of rewards, can be found in the supplementary
material.

We employ an off-policy technique to train our Conservative Q-Learning method
using a diverse set of rearrangement configurations, which is similar to the
approach proposed by (Kalashnikov et al. 2018). To balance exploration and
exploitation, we use the $\epsilon$ greedy method, as described in
(Kalashnikov et al. 2018). To ensure stable Off-policy training, we update
$\bar{\theta}$ weights using polyak averaging on $\theta$, which is similar to
(Bester, James, and Konidaris 2019).

## Experiment

In this section, we describe the dataset, metrics, and detailed results of our
proposed method and its modules, in addressing the multi-room rearrangement
problem.

### Unseen Object Discovery dataset

The AMT dataset (Kant et al. 2022) consists of 268 object categories present
in 12 distinct rooms and 32 receptacle types. Each object-room-receptacle
(ORR) pair is evaluated by 10 annotators who rank them into one of three
classes: correct (positively ranked), misplaced (negatively ranked), or
implausible (not ranked). By calculating the mean inverse of the ranks
assigned to each ORR, we obtain the ground-truth scores. For our specific
problem, our preference order for ORRs is as follows: misplaced class, correct
class, and lastly, the implausible class. Hence, we re-label the classes and
their scores as (i) misplaced and correct class $\rightarrow$ probable class,
while (ii) implausible class remains the same.

### Benchmark Dataset for Testing - MoPOR

The existing benchmark dataset, RoomR (Weihs et al. 2021), is utilized to
evaluate rearrangement policies across different room scenarios. However, it
has certain limitations. It restricts the number of objects to a maximum of 5
and does not allow object placement within another receptacle, nor does it
include blocked goal or swap cases. Consequently, it cannot adequately assess
planning aspects such as the number of steps, agent traversal, blocked goals,
or swap cases. To overcome these limitations, we introduce MoPOR, a new
benchmark dataset designed specifically for testing task planners in Ai2Thor.
MoPOR encompasses a diverse collection of single-room scenes from iThor and
multi-room scenes from ProcThor (Deitke et al. 2022) 333Every scene has 4
types of rooms - living room, bedroom, bathroom, kitchen. It supports up to
108 object and receptacle categories. This dataset enables a wide range of
rearrangement scenarios involving up to 40 objects. Furthermore, MoPOR
includes random partial observability cases, object placement within
receptacles in the current state, as well as blocked goal and swap cases.
Moreover, it’s worth noting that object placement configurations in MoPOR
impact the effectiveness of sub-optimal planning policies in terms of agent
traversal. Further details on the distribution of objects, rooms and
receptacles in Appendix.

### Training

The training details of UODM and DSG with Deep RL planner are available in the
Appendix2.

### Evaluation Criteria

Metrics in (Weihs et al. 2021) do not highlight the efficacy of a task planner
to judge efficient sequencing to reduce the number of steps taken or the agent
traversal during rearrangement. For a fair evaluation of our method, and
comparison against the existing methods and ablations, we define the following
criteria:

  * •

SRN : Success Ratio measures rearrangement episode success and efficiency by
combining the binary success rate (S) and the Number of steps ($N_{S}$) taken
by the agent to rearrange a set number of objects ($N$). A higher SRN
indicates a more efficient and successful rearrangement episode, as it implies
a lower $N_{S}$ for a given $N$. (SRN = $S\times N/N_{S}$)

  * •

EOD: Efficiency in unseen Object Discovery is the ratio of the number of
unseen objects initially ($N_{\overline{V}}$) with respect to the number of
attempts to search ($N_{S\overline{V}}$). A higher EOD shows a lower
$N_{S\overline{V}}$ for a given $N_{\overline{V}}$ indicating a more efficient
search to find unseen objects.(EOD = $N_{\overline{V}}/N_{D}$)

  * •

TTL: Total Traversal Length metric shows the total distance traversed by the
agent during the successful completion of a rearrangement episode. In an
identical test configuration, a lower TTL indicates a more efficient
rearrangement sequencing .

| Number
---
of Objects
| Visible
---
Objects
| Unseen
---
Objects
| Swap
---
Case
Ours-GT | Ours | Ours-RS | Ours-GE | Ours-HP
|  | P.O | F.O |  | SRN | EOD | TTL | SRN | EOD | TTL | SRN | EOD | TTL | SRN | EOD | TTL | SRN | EOD | TTL
$10$ | 6 | 4 | 0 | 2 | 0.59 | 0.55 | 37.03 | 0.41 | 0.53 | 37.82 | 0.19 | 0.20 | 60.72 | 0.15 | 0.13 | 70.09 | 0.41 | 0.53 | 45.82
| 6 | 0 | 4 | 2 | 0.55 | 0.51 | 38.25 | 0.39 | 0.48 | 40.27 | 0.13 | 0.15 | 65.47 | 0 | NC | NC | 0.39 | 0.48 | 50.27
$20$ | 12 | 8 | 0 | 4 | 0.61 | 0.60 | 66.94 | 0.43 | 0.56 | 69.29 | 0.22 | 0.24 | 89.45 | 0.19 | 0.17 | 101.68 | 0.43 | 0.56 | 82.42
| 12 | 0 | 8 | 4 | 0.58 | 0.55 | 68.95 | 0.40 | 0.53 | 72.38 | 0.17 | 0.18 | 98.37 | 0 | NC | NC | 0.40 | 0.53 | 89.67
$30$ | 18 | 12 | 0 | 6 | 0.64 | 0.67 | 90.25 | 0.45 | 0.62 | 94.76 | 0.27 | 0.28 | 120.45 | 0.24 | 0.19 | 131.92 | 0.45 | 0.62 | 109.42
| 18 | 0 | 12 | 6 | 0.61 | 0.61 | 95.71 | 0.43 | 0.58 | 98.58 | 0.23 | 0.22 | 132.16 | 0 | NC | NC | 0.43 | 0.58 | 118.72

Table 1: Results for Multi-room Rearrangement with comparison against
Baselines. Here, P.O. indicates partially occluded cases i.e. objects which
are outside the field of view presently and F.O. stands for fully occluded
cases i.e. objects placed inside closed receptacles. Ours-GE does not handle
F.O. cases, therefore its EOD is non-computable (NC) due to division by zero.
The table shows that Ours-GT and Ours outperform the other baselines in terms
of the evaluation criteria.

### Baselines

We ablate our method against ground-truth perception, various methods for
object search and different planners. To study the effect of erroneous
perception on our method, we use (i) Ours-GT with ground-truth perception - 3D
object detection and instance segmentation mask from Ai2Thor (Kolve et al.
2017). To understand the importance of UODM in our method, we replace it by a
random search policy in (ii) Ours-RS, which predicts probable receptacles for
unseen objects with uniform probability and a greedy exploration strategy
(Chaplot et al. 2020) in (iii) Ours-GE that optimizes for map coverage to
discover all the unseen objects. To gauge the efficacy of our planner we
replace the Deep RL planner in our method with a heuristic planner in (iv)
Ours-HP that greedily selects an action with the shortest agent traversal to
complete the object pick-place.

### Results

#### Ablations

The state-of-the-art methods dealing with user-defined goal state do not
demonstrate their results in multi-room rearrangement task. Due to this
limitation, we gauge the performance of our method in Tab 1, by comparing it
against the set of baselines in a multi-room setting on MoPOR - Benchmark
Datset. Tab 1 indicates that our method is scalable to a large number of
objects, with consistently increasing SRN values for swap cases and both
scenarios of partial observability (P.O. : objects outside the field of view,
F.O. : objects inside closed receptacles). In addition, the consistently high
SRN for a growing number of swap cases indicates that Ours and Ours-GT can
effectively handle swap cases, using the Cross-entropy based method for buffer
prediction. The gradual increase in SNR and EOD with the increase in number of
objects can be attributed to the fact that rearrangement of visible objects
and the search for some unseen objects, indirectly aids in finding other
unseen objects. As anticipated, Ours-GT results in significantly better SNR,
EOD and TTL, compared to Ours and all the baselines, because it uses ground-
truth object detection and labelling.

The variation in the results between Ours and the baselines primarily arises
from their approaches to handle partial observability cases, since Ours and
the baselines employ the same buffer prediction method for swap cases. In both
the cases of partial observability (P.O. $\&$ F.O.), Ours performs
significantly better than Ours-GE, Ours-RS and Ours-HP in terms SRN, EOD and
TTL. This is due to the efficacy of the Unseen Object Discovery Method (UODM)
and the efficient planning of Deep RL during simultaneous object search and
rearrangement. Ours-GE incurs a high traversal cost in terms of TTL because it
explicitly explores the entire apartment to find the PO objects. Moreover,
Ours-GE fails to address the FO cases (SRN = 0) because the greedy exploration
policy (Chaplot et al. 2020) in terms of map coverage does not include opening
and closing receptacles to find FO. Whereas, Ours-RS randomly visits
receptacles to discover PO or FO cases, which again increases TTL. In
contrast, our approach performs similarly in both cases of partial
observability because UODM comprehends a semantic relationship between an
object and any type of room-receptacle - rigid or articulated. To gauge the
performance of the exploration strategy for object search in terms of EOD, we
consider each newly generated location or a set of navigational steps from the
exploration policy as a search attempt. More number of attempts to search in
Ours-GE and Ours-RS lead to a lower EOD as well as SRN. We observe that Ours-
RS performs slightly better than Ours-GE in terms of EOD and TTL for PO,
because Ours-RS interleaves object search and rearrangement, rather than doing
an explicit exploration strategy for finding objects. Ours-HP444Ours-HP has
similar UODM and Collision Resolution and Buffer Management Method only shows
a higher TTL compared to Ours for both PO and FO cases in partial
observability due to the greedy planning method.

Please refer to the appendix2 to find results for the analysis of the choice
of hyper-parameters for each of our learning-based modules.

#### Comparison against state-of-the-art methods for Room-Rearrangement

The existing methods train and show results on a single-room rearrangement
task. Therefore, for a fair comparison, we compare our method against them in
a single-room setting on MoPOR - Benchmark Dataset and RoomR (Weihs et al.
2021). Please refer to the appendix2 to find the detailed comparison tables
and discussion.

### Qualitative Results

To show the qualitative results of our method in multi-room rearrangement, we
have created multiple test scenario videos to highlight the robustness of our
method. We also evaluate our method in a new environment- Habitat, as
highlighted in our supplementary video555http://tinyurl.com/SuppVideo. This
transfer does not require any additional training for our UODM or Deep RL
planner. This demonstrates how our method excels in seamless sim-to-sim
transfer, reinforcing its suitability for deployment in real-world scenarios.
Please refer the supplementary video6.

## Limitations

Our approach is not capable of identifying unseen objects that are occluded
due to clutter on receptacles (for e.g. a spoon may become occluded, if bread,
box, lettuce etc. is placed before it). Our method also assumes the
availability of perfect motion planning and manipulation capabilities.

## Conclusion

This paper introduces a novel task planner designed for tidying up an
apartment while dealing with partial observability, which can be adapted to
various situations and generates a sequence of actions that minimizes the
agent’s overall traversal and the number of steps taken during simultaneous
unseen object discovery and rearrangement. In future work, we plan to explore
the deployment of our method in real-world.

## References

  * Batra et al. (2020) Batra, D.; Chang, A. X.; Chernova, S.; Davison, A. J.; Deng, J.; Koltun, V.; Levine, S.; Malik, J.; Mordatch, I.; Mottaghi, R.; Savva, M.; and Su, H. 2020.  Rearrangement: A Challenge for Embodied AI.  _ArXiv_ , abs/2011.01975.
  * Bellman (1957) Bellman, R. 1957.  A Markovian Decision Process.  _Journal of Mathematics and Mechanics_ , 6(5): 679–684.
  * Bester, James, and Konidaris (2019) Bester, C. J.; James, S. D.; and Konidaris, G. D. 2019.  Multi-pass q-networks for deep reinforcement learning with parameterised action spaces.  _arXiv preprint arXiv:1905.04388_.
  * Bhowmick et al. (2017) Bhowmick, B.; Patra, S.; Chatterjee, A.; Madhav Govindu, V.; and Banerjee, S. 2017.  Divide and conquer: A hierarchical approach to large-scale structure-from-motion.  _Computer Vision and Image Understanding_ , 157: 190–205.
  * Chaplot et al. (2020) Chaplot, D.; Gandhi, D.; Gupta, S.; Gupta, A.; and Salakhutdinov, R. 2020.  Learning to Explore using Active Neural SLAM.  In _Eighth International Conference on Learning Representations, ICLR 2022_.
  * Deitke et al. (2022) Deitke, M.; VanderBilt, E.; Herrasti, A.; Weihs, L.; Salvador, J.; Ehsani, K.; Han, W.; Kolve, E.; Farhadi, A.; Kembhavi, A.; and Mottaghi, R. 2022.  ProcTHOR: Large-Scale Embodied AI Using Procedural Generation.  In _NeurIPS_.  Outstanding Paper Award.
  * Gadre et al. (2022) Gadre, S. Y.; Ehsani, K.; Song, S.; and Mottaghi, R. 2022.  Continuous Scene Representations for Embodied AI.  _2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_ , 14829–14839.
  * Gehring et al. (2021) Gehring, C.; Asai, M.; Chitnis, R.; Silver, T.; Kaelbling, L. P.; Sohrabi, S.; and Katz, M. 2021.  Reinforcement Learning for Classical Planning: Viewing Heuristics as Dense Reward Generators.  In _International Conference on Automated Planning and Scheduling_.
  * Ghosh et al. (2022) Ghosh, S.; Das, D.; Chakraborty, A.; Agarwal, M.; and Bhowmick, B. 2022.  Planning Large-scale Object Rearrangement Using Deep Reinforcement Learning.  In _2022 International Joint Conference on Neural Networks (IJCNN)_ , 1–8.
  * Kalashnikov et al. (2018) Kalashnikov, D.; Irpan, A.; Pastor, P.; Ibarz, J.; Herzog, A.; Jang, E.; Quillen, D.; Holly, E.; Kalakrishnan, M.; Vanhoucke, V.; and Levine, S. 2018.  Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulation.  In Billard, A.; Dragan, A.; Peters, J.; and Morimoto, J., eds., _Proceedings of The 2nd Conference on Robot Learning_ , volume 87 of _Proceedings of Machine Learning Research_ , 651–673. PMLR.
  * Kant et al. (2022) Kant, Y.; Ramachandran, A.; Yenamandra, S.; Gilitschenski, I.; Batra, D.; Szot, A.; and Agrawal, H. 2022.  Housekeep: Tidying Virtual Households using Commonsense Reasoning.  In _European Conference on Computer Vision_.
  * Kolve et al. (2017) Kolve, E.; Mottaghi, R.; Han, W.; VanderBilt, E.; Weihs, L.; Herrasti, A.; Gordon, D.; Zhu, Y.; Gupta, A.; and Farhadi, A. 2017.  AI2-THOR: An Interactive 3D Environment for Visual AI.  _arXiv_.
  * Kumar et al. (2020) Kumar, A.; Zhou, A.; Tucker, G.; and Levine, S. 2020.  Conservative Q-Learning for Offline Reinforcement Learning.  In Larochelle, H.; Ranzato, M.; Hadsell, R.; Balcan, M.; and Lin, H., eds., _Advances in Neural Information Processing Systems_ , volume 33, 1179–1191. Curran Associates, Inc.
  * Liu et al. (2019) Liu, Y.; Ott, M.; Goyal, N.; Du, J.; Joshi, M.; Chen, D.; Levy, O.; Lewis, M.; Zettlemoyer, L.; and Stoyanov, V. 2019.  RoBERTa: A Robustly Optimized BERT Pretraining Approach.  _CoRR_ , abs/1907.11692.
  * Patra et al. (2012) Patra, S.; Bhowmick, B.; Banerjee, S.; and Kalra, P. 2012.  High Resolution Point Cloud Generation from Kinect and HD Cameras using Graph Cut.  In _VISAPP 2012 - Proceedings of the International Conference on Computer Vision Theory and Applications, Volume 2, Rome, Italy, 24-26 February, 2012_ , 311–316. SciTePress.
  * Rubinstein and Kroese (2004) Rubinstein, R. Y.; and Kroese, D. P. 2004.  _The Cross Entropy Method: A Unified Approach To Combinatorial Optimization, Monte-Carlo Simulation (Information Science and Statistics)_.  Berlin, Heidelberg: Springer-Verlag.  ISBN 038721240X.
  * Rusu and Cousins (2011) Rusu, R. B.; and Cousins, S. 2011.  3D is here: Point Cloud Library (PCL).  In _IEEE International Conference on Robotics and Automation (ICRA)_. Shanghai, China: IEEE.
  * Sarch et al. (2022) Sarch, G.; Fang, Z.; Harley, A. W.; Schydlo, P.; Tarr, M. J.; Gupta, S.; and Fragkiadaki, K. 2022.  TIDEE: Tidying Up Novel Rooms using Visuo-Semantic Commonsense Priors.  In _European Conference on Computer Vision_.
  * Sriram et al. (2019) Sriram, N. N.; Maniar, T.; Kalyanasundaram, J.; Gandhi, V.; Bhowmick, B.; and Madhava Krishna, K. 2019.  Talk to the Vehicle: Language Conditioned Autonomous Navigation of Self Driving Cars.  In _2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)_ , 5284–5290.
  * Trabucco et al. (2022) Trabucco, B.; Sigurdsson, G. A.; Piramuthu, R.; Sukhatme, G. S.; and Salakhutdinov, R. 2022.  A Simple Approach for Visual Rearrangement: 3D Mapping and Semantic Search.  In _Workshop on Learning from Diverse, Offline Data_.
  * Watkins and Dayan (1992) Watkins, C. J.; and Dayan, P. 1992.  Q-learning.  _Machine learning_ , 8: 279–292.
  * Weihs et al. (2021) Weihs, L.; Deitke, M.; Kembhavi, A.; and Mottaghi, R. 2021.  Visual Room Rearrangement.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 5922–5931.
  * Zhu et al. (2021) Zhu, X.; Su, W.; Lu, L.; Li, B.; Wang, X.; and Dai, J. 2021.  Deformable DETR: Deformable Transformers for End-to-End Object Detection.  In _9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021_. OpenReview.net.