measurements when designing observations of the stellar wake.

In this work, we have demonstrated that there are marked differences in the
density structure and kinematics of the LMC’s DF wake in a CDM vs. an FDM
universe, but these differences may be challenging to distinguish in
observations of the stellar halo. Significantly, we have also illustrated that
the self-gravity of the DM wake plays a crucial role in strengthening and
extending the stellar halo’s response to the DM wake - providing a new avenue
to test for the existence of DM. Next-generation spectroscopic surveys like
DESI (Collaboration et al., 2016a, b), LSST/Vera Rubin Observatory (Ivezić,
2019), and the Nancy Grace Roman Space Telescope are poised to provide
precision radial velocity and distance measurements of increasing numbers of
stars in the stellar halo. These measurements will provide an unprecedented
window into the underlying DM distribution of our Local Group (Dey et al.,
2023), and potentially the nature of DM itself.

##  8 acknowledgments

H.R.F. would like to thank Peter Behroozi, Arjun Dey, and Dennis Zaritsky for
productive discussions that greatly improved this manuscript, Tomer Yavetz for
a particularly helpful discussion on the correspondence between CDM and FDM
halos, and the anonymous referee for insightful comments that improved the
clarity of the paper and presentation of our results.

This work is based upon High Performance Computing (HPC) resources supported
by the University of Arizona TRIF, UITS, and Research, Innovation, and Impact
(RII) and maintained by the UArizona Research Technologies department. H.R.F.
thanks Derrick Zwickl and Chris Reidy for their assistance with MPI
troubleshooting, which was made possible through University of Arizona
Research Technologies Collaborative Support program.

H.R.F. and G.B. are supported by NSF CAREER AST-1941096. This work was
performed under the auspices of the U.S. Department of Energy by Lawrence
Livermore National Laboratory under contract DE-AC52-07NA27344. E.C.C.
acknowledges support for this work provided by NASA through the NASA Hubble
Fellowship Program grant HST-HF2-51502.001-A awarded by the Space Telescope
Science Institute, which is operated by the Association of Universities for
Research in Astronomy, Inc., for NASA, under contract NAS5-26555. F.A.G.
acknowledges support from ANID FONDECYT Regular 1211370 and by the ANID BASAL
project FB210003. F.A.G. acknowledges funding from the Max Planck Society
through a “Partner Group” grant. C.F.P.L. acknowledges funding from the
European Research Council (ERC) under the European Union’s Horizon 2020
research and innovation programme (grant agreement No. 852839).

##  Appendix A Comparison of the windtunnel wake to a wake in a live halo

Figure 31: Comparison between the DM overdensity of GC19’s Simulation 3 and
our Fiducial CDM simulation with self-gravity. The right panel is a
reproduction of the center panel of Figure 2. On the left, we compare this to
the overdensity of MW DM particles in GC19’s Simulation 3 when the LMC is at a
Galactocentric distance of 70 kpc. We compute the density in the GC19
simulation within a 120-kpc-thick slab centered on the Galactic $y$-$z$ plane
such that the volume of the slab is identical to our Figure 2. We also draw a
contour around the region with an overdensity greater than 0.38 as in the
right panel. The size and strength of the wake agree between both panels,
showing that the wakes formed in our windtunnels are a good approximation of
the wake formed in a live-halo simulation.

In this Appendix, we compare our Fiducial CDM (with self-gravity) wake to our
reference simulation (simulation 3 from GC19). Our windtunnel simulations have
several important inherent differences from the full-interaction scenarios
presented in GC19: in our simulations, the LMC is represented by a fixed
potential instead of as a live halo, there are no Galactic tides from the MW,
the LMC “travels” on a straight line instead of a curved orbit, and the DM
wind parameters are time-independent instead of varying along the orbit as the
LMC plunges into the MW’s halo. Therefore, it is vital to compare the CDM wake
simulated in this paper to the wake in our reference GC19 Simulation 3 to
ensure that our windtunnel is a reasonable laboratory to study differences
between CDM and FDM and the impact of self-gravity on the wake.

Figure 31 compares the strength and size of the DM wake in GC19’s Simulation 3
(left) to our Fiducial CDM simulation with self-gravity (right). Similar to
Figure 2, we draw a contour enclosing the region with an overdensity greater
than half the maximum overdensity in the right panel. Both wakes are very
similar in strength and size, demonstrating that our windtunnel simulation
framework can well-reproduce the wake formed in a full-interaction simulation.

##  Appendix B Transformation From Windtunnel to Galactic Coordinates

To perform transformations between coordinate frames (windtunnel simulation
box, Galactocentric, and Galactic), we make use of Astropy version 4.2.1
(Astropy Collaboration et al. 2013; Astropy Collaboration et al. 2018; Astropy
Collaboration et al. 2022), and adopt the definitions and conventions of
Galactocentric and Galactic coordinates as given in this version of Astropy.
Our steps for transforming between simulation box coordinates and
Galactocentric coordinates are as follows:

  1. 1.

For reference, we use the LMC’s present-day location and velocity vector from
Kallivayalil et al. (2013). The LMC’s orbit, as usual, is taken from GC19’s
Simulation 3.

  2. 2.

We rotate the simulation box such that the LMC’s unit velocity vector
(-$\hat{y}$ in the windtunnel frame) points in the same direction as the LMC’s
velocity vector at $r_{MW}=70$ kpc from the reference simulation.

  3. 3.

Next, we translate the simulation box such that the center of the box (where
the LMC potential is located) matches the present-day location of the LMC.
This ensures that the LMC is as close as possible to the correct location on
the sky after the next step.

  4. 4.

A further translation matches the location of the straight windtunnel orbit
and the curved orbit from the reference simulation at a Galactocentric
distance of 70 kpc. Together with the rotation, this ensures the LMC’s path in
the windtunnel is tangent to the LMC’s orbit at 70 kpc, which is the location
our Fiducial wind parameters are drawn from.

  5. 5.

Finally, the particles are given a velocity boost to remove the bulk wind
velocity, i.e. to ensure the wind particles have no net motion in a
Galactocentric frame.

Figures 19, 20, 21, and 22 in $\S$ 5.1 are in Galactic coordinates, and we use
Astropy’s built-in functionality for transforming between Galactocentric and
Galactic coordinates. When plotting the velocities of stars in Galactic
coordinates, we also remove the Sun’s motion about the Galactic Center.

## References

  * Adhikari et al. (2022) Adhikari, S., Banerjee, A., Boddy, K. K., et al. 2022, Astrophysical Tests of Dark Matter Self-Interactions, Tech. rep., doi: 10.48550/arXiv.2207.10638
  * Amorisco & Loeb (2018) Amorisco, N. C., & Loeb, A. 2018, First constraints on Fuzzy Dark Matter from the dynamics of stellar streams in the Milky Way, Tech. Rep. arXiv:1808.00464, arXiv.  http://arxiv.org/abs/1808.00464
  * Armengaud et al. (2017) Armengaud, E., Palanque-Delabrouille, N., Yèche, C., Marsh, D. J. E., & Baur, J. 2017, Monthly Notices of the Royal Astronomical Society, 471, 4606, doi: 10.1093/mnras/stx1870
  * Astropy Collaboration et al. (2013) Astropy Collaboration, Robitaille, T. P., Tollerud, E. J., et al. 2013, A&A, 558, A33, doi: 10.1051/0004-6361/201322068
  * Astropy Collaboration et al. (2018) Astropy Collaboration, Price-Whelan, A. M., Sipőcz, B. M., et al. 2018, AJ, 156, 123, doi: 10.3847/1538-3881/aabc4f
  * Astropy Collaboration et al. (2022) Astropy Collaboration, Price-Whelan, A. M., Lim, P. L., et al. 2022, apj, 935, 167, doi: 10.3847/1538-4357/ac7c74
  * Banik et al. (2021) Banik, N., Bovy, J., Bertone, G., Erkal, D., & de Boer, T. J. L. 2021, J. Cosmol. Astropart. Phys., 2021, 043, doi: 10.1088/1475-7516/2021/10/043
  * Bar et al. (2019) Bar, N., Blum, K., Lacroix, T., & Panci, P. 2019, Journal of Cosmology and Astroparticle Physics, 2019, 045, doi: 10.1088/1475-7516/2019/07/045
  * Bar-Or et al. (2019) Bar-Or, B., Fouvry, J.-B., & Tremaine, S. 2019, The Astrophysical Journal, 871, 28, doi: 10.3847/1538-4357/aaf28c
  * Bar-Or et al. (2021) —. 2021, The Astrophysical Journal, 915, 27, doi: 10.3847/1538-4357/abfb66
  * Benito et al. (2020) Benito, M., Criado, J. C., Hütsi, G., Raidal, M., & Veermäe, H. 2020, Physical Review D, 101, 103023, doi: 10.1103/PhysRevD.101.103023
  * Besla et al. (2007) Besla, G., Kallivayalil, N., Hernquist, L., et al. 2007, The Astrophysical Journal, 668, 949, doi: 10.1086/521385
  * Besla et al. (2010) —. 2010, The Astrophysical Journal, 721, L97, doi: 10.1088/2041-8205/721/2/L97
  * Bird et al. (2021) Bird, S. A., Xue, X.-X., Liu, C., et al. 2021, The Astrophysical Journal, 919, 66, doi: 10.3847/1538-4357/abfa9e
  * Carlson et al. (1992) Carlson, E. D., Machacek, M. E., & Hall, L. J. 1992, The Astrophysical Journal, 398, 43, doi: 10.1086/171833
  * Chamberlain et al. (2022) Chamberlain, K., Price-Whelan, A. M., Besla, G., et al. 2022, Implications of the Milky Way travel velocity for dynamical mass estimates of the Local Group, Tech. rep.  https://ui.adsabs.harvard.edu/abs/2022arXiv220407173C
  * Chan et al. (2022) Chan, H. Y. J., Ferreira, E. G. M., May, S., Hayashi, K., & Chiba, M. 2022, Monthly Notices of the Royal Astronomical Society, 511, 943, doi: 10.1093/mnras/stac063
  * Chandrasekhar (1943) Chandrasekhar, S. 1943, The Astrophysical Journal, 97, 255, doi: 10.1086/144517
  * Church et al. (2019) Church, B. V., Mocz, P., & Ostriker, J. P. 2019, Monthly Notices of the Royal Astronomical Society, 485, 2861, doi: 10.1093/mnras/stz534
  * Cohen et al. (2017) Cohen, J. G., Sesar, B., Bahnolzer, S., et al. 2017, The Astrophysical Journal, 849, 150, doi: 10.3847/1538-4357/aa9120
  * Collaboration et al. (2016a) Collaboration, D., Aghamousa, A., Aguilar, J., et al. 2016a, The DESI Experiment Part I: Science,Targeting, and Survey Design, arXiv, doi: 10.48550/arXiv.1611.00036
  * Collaboration et al. (2016b) —. 2016b, The DESI Experiment Part II: Instrument Design, arXiv, doi: 10.48550/arXiv.1611.00037
  * Collette (2013) Collette, A. 2013, Python and HDF5 (O’Reilly)
  * Conroy et al. (2021) Conroy, C., Naidu, R. P., Garavito-Camargo, N., et al. 2021, Nature, 592, 534, doi: 10.1038/s41586-021-03385-7
  * Conroy et al. (2019) Conroy, C., Bonaca, A., Cargile, P., et al. 2019, The Astrophysical Journal, 883, 107, doi: 10.3847/1538-4357/ab38b8
  * Cooper et al. (2022) Cooper, A. P., Koposov, S. E., Prieto, C. A., et al. 2022, Overview of the DESI Milky Way Survey, arXiv, doi: 10.48550/arXiv.2208.08514
  * Correa (2021) Correa, C. A. 2021, Monthly Notices of the Royal Astronomical Society, 503, 920, doi: 10.1093/mnras/stab506
  * Correa Magnus & Vasiliev (2022) Correa Magnus, L., & Vasiliev, E. 2022, Monthly Notices of the Royal Astronomical Society, 511, 2610, doi: 10.1093/mnras/stab3726
  * Cunningham et al. (2020) Cunningham, E. C., Garavito-Camargo, N., Deason, A. J., et al. 2020, The Astrophysical Journal, 898, 4, doi: 10.3847/1538-4357/ab9b88
  * Dalal et al. (2021) Dalal, N., Bovy, J., Hui, L., & Li, X. 2021, J. Cosmol. Astropart. Phys., 2021, 076, doi: 10.1088/1475-7516/2021/03/076
  * Dalal & Kravtsov (2022) Dalal, N., & Kravtsov, A. 2022, Not so fuzzy: excluding FDM with sizes and stellar kinematics of ultra-faint dwarf galaxies, Tech. rep.  https://ui.adsabs.harvard.edu/abs/2022arXiv220305750D
  * Deason et al. (2012) Deason, A. J., Belokurov, V., Evans, N. W., et al. 2012, Monthly Notices of the Royal Astronomical Society, 425, 2840, doi: 10.1111/j.1365-2966.2012.21639.x
  * Dentler et al. (2022) Dentler, M., Marsh, D. J. E., Hložek, R., et al. 2022, Monthly Notices of the Royal Astronomical Society, 515, 5646, doi: 10.1093/mnras/stac1946
  * Desjacques & Nusser (2019) Desjacques, V., & Nusser, A. 2019, Monthly Notices of the Royal Astronomical Society, 488, 4497, doi: 10.1093/mnras/stz1978
  * Dey et al. (2023) Dey, A., Najita, J. R., Koposov, S. E., et al. 2023, The Astrophysical Journal, 944, 1, doi: 10.3847/1538-4357/aca5f8
  * Dome et al. (2022) Dome, T., Fialkov, A., Mocz, P., et al. 2022, On the Cosmic Web Elongation in Fuzzy Dark Matter Cosmologies, arXiv.  http://arxiv.org/abs/2208.03827
  * Erkal & Belokurov (2020) Erkal, D., & Belokurov, V. A. 2020, Monthly Notices of the Royal Astronomical Society, 495, 2554, doi: 10.1093/mnras/staa1238
  * Erkal et al. (2020) Erkal, D., Belokurov, V. A., & Parkin, D. L. 2020, Monthly Notices of the Royal Astronomical Society, 498, 5574, doi: 10.1093/mnras/staa2840
  * Erkal et al. (2018) Erkal, D., Li, T. S., Koposov, S. E., et al. 2018, Monthly Notices of the Royal Astronomical Society, 481, 3148, doi: 10.1093/mnras/sty2518
  * Erkal et al. (2019) Erkal, D., Belokurov, V., Laporte, C. F. P., et al. 2019, Monthly Notices of the Royal Astronomical Society, 487, 2685, doi: 10.1093/mnras/stz1371
  * Ferreira (2021) Ferreira, E. G. M. 2021, Astron Astrophys Rev, 29, 7, doi: 10.1007/s00159-021-00135-6
  * Furlanetto & Loeb (2002) Furlanetto, S. R., & Loeb, A. 2002, The Astrophysical Journal, 565, 854, doi: 10.1086/324693
  * Garavito-Camargo et al. (2019) Garavito-Camargo, N., Besla, G., Laporte, C. F. P., et al. 2019, ApJ, 884, 51, doi: 10.3847/1538-4357/ab32eb
  * Garavito-Camargo et al. (2021) —. 2021, The Astrophysical Journal, 919, 109, doi: 10.3847/1538-4357/ac0b44
  * Garrow et al. (2020) Garrow, T., Webb, J. J., & Bovy, J. 2020, Monthly Notices of the Royal Astronomical Society, 499, 804, doi: 10.1093/mnras/staa2773
  * González-Morales et al. (2017) González-Morales, A. X., Marsh, D. J. E., Peñarrubia, J., & Ureña-López, L. A. 2017, Monthly Notices of the Royal Astronomical Society, 472, 1346, doi: 10.1093/mnras/stx1941
  * Górski et al. (2005) Górski, K. M., Hivon, E., Banday, A. J., et al. 2005, ApJ, 622, 759, doi: 10.1086/427976
  * Grady et al. (2021) Grady, J., Belokurov, V., & Evans, N. W. 2021, The Astrophysical Journal, 909, 150, doi: 10.3847/1538-4357/abd4e4
  * Gómez et al. (2015) Gómez, F. A., Besla, G., Carpintero, D. D., et al. 2015, The Astrophysical Journal, 802, 128, doi: 10.1088/0004-637X/802/2/128
  * Harris et al. (2020) Harris, C. R., Millman, K. J., van der Walt, S. J., et al. 2020, Nature, 585, 357, doi: 10.1038/s41586-020-2649-2
  * Helmi (2020) Helmi, A. 2020, Annual Review of Astronomy and Astrophysics, 58, 205, doi: 10.1146/annurev-astro-032620-021917
  * Hernquist (1990) Hernquist, L. 1990, The Astrophysical Journal, 356, 359, doi: 10.1086/168845
  * Hložek et al. (2017) Hložek, R., Marsh, D. J., Grin, D., et al. 2017, Physical Review D, 95, 123511, doi: 10.1103/PhysRevD.95.123511
  * Hu et al. (2000) Hu, W., Barkana, R., & Gruzinov, A. 2000, Physical Review Letters, 85, 1158, doi: 10.1103/PhysRevLett.85.1158
  * Hui (2021) Hui, L. 2021, Annual Review of Astronomy and Astrophysics, Volume 59, pp. 247-289, 59, 247, doi: 10.1146/annurev-astro-120920-010024
  * Hui et al. (2017) Hui, L., Ostriker, J. P., Tremaine, S., & Witten, E. 2017, Phys. Rev. D, 95, 043541, doi: 10.1103/PhysRevD.95.043541
  * Hunter (2007) Hunter, J. D. 2007, Computing in Science & Engineering, 9, 90, doi: 10.1109/MCSE.2007.55
  * Iršič et al. (2017) Iršič, V., Viel, M., Haehnelt, M. G., Bolton, J. S., & Becker, G. D. 2017, Physical Review Letters, 119, 031302, doi: 10.1103/PhysRevLett.119.031302
  * Ivezić (2019) Ivezić, v. 2019, The Astrophysical Journal
  * Kallivayalil et al. (2013) Kallivayalil, N., van der Marel, R. P., Besla, G., Anderson, J., & Alcock, C. 2013, The Astrophysical Journal, 764, 161, doi: 10.1088/0004-637X/764/2/161
  * Katz et al. (2019) Katz, D., Sartoretti, P., Cropper, M., et al. 2019, Astronomy & Astrophysics, Volume 622, id.A205, <NUMPAGES>19</NUMPAGES> pp., 622, A205, doi: 10.1051/0004-6361/201833273
  * Khlopov et al. (1985) Khlopov, M. Y., Malomed, B. A., & Zeldovich, Y. B. 1985, Monthly Notices of the Royal Astronomical Society, 215, 575, doi: 10.1093/mnras/215.4.575
  * Kobayashi et al. (2017) Kobayashi, T., Murgia, R., De Simone, A., Iršič, V., & Viel, M. 2017, Physical Review D, 96, 123514, doi: 10.1103/PhysRevD.96.123514
  * Koposov et al. (2023) Koposov, S. E., Erkal, D., Li, T. S., et al. 2023, Monthly Notices of the Royal Astronomical Society, doi: 10.1093/mnras/stad551
  * Lancaster et al. (2020) Lancaster, L., Giovanetti, C., Mocz, P., et al. 2020, J. Cosmol. Astropart. Phys., 2020, 001, doi: 10.1088/1475-7516/2020/01/001
  * Laporte et al. (2018a) Laporte, C. F. P., Gómez, F. A., Besla, G., Johnston, K. V., & Garavito-Camargo, N. 2018a, Monthly Notices of the Royal Astronomical Society, 473, 1218, doi: 10.1093/mnras/stx2146
  * Laporte et al. (2018b) Laporte, C. F. P., Johnston, K. V., Gómez, F. A., Garavito-Camargo, N., & Besla, G. 2018b, Monthly Notices of the Royal Astronomical Society, 481, 286, doi: 10.1093/mnras/sty1574
  * Lilleengen et al. (2023) Lilleengen, S., Petersen, M. S., Erkal, D., et al. 2023, Monthly Notices of the Royal Astronomical Society, 518, 774, doi: 10.1093/mnras/stac3108
  * Madelung (1927) Madelung, E. 1927, Zeitschrift fur Physik, 40, 322, doi: 10.1007/BF01400372
  * Marsh & Pop (2015) Marsh, D. J. E., & Pop, A.-R. 2015, Monthly Notices of the Royal Astronomical Society, 451, 2479, doi: 10.1093/mnras/stv1050
  * May & Springel (2021) May, S., & Springel, V. 2021, Monthly Notices of the Royal Astronomical Society, 506, 2603, doi: 10.1093/mnras/stab1764
  * May & Springel (2022) —. 2022, The halo mass function and filaments in full cosmological simulations with fuzzy dark matter, arXiv.  http://arxiv.org/abs/2209.14886
  * Mocz et al. (2017) Mocz, P., Vogelsberger, M., Robles, V., et al. 2017, Monthly Notices of the Royal Astronomical Society, 471, 4559, doi: 10.1093/mnras/stx1887
  * Mocz et al. (2020) Mocz, P., Fialkov, A., Vogelsberger, M., et al. 2020, Monthly Notices of the Royal Astronomical Society, 494, 2027, doi: 10.1093/mnras/staa738
  * Navarro et al. (1997) Navarro, J. F., Frenk, C. S., & White, S. D. M. 1997, The Astrophysical Journal, 490, 493, doi: 10.1086/304888
  * Ostriker (1999) Ostriker, E. C. 1999, The Astrophysical Journal, 513, 252, doi: 10.1086/306858
  * Patel et al. (2017) Patel, E., Besla, G., & Sohn, S. T. 2017, Mon. Not. R. Astron. Soc., 464, 3825, doi: 10.1093/mnras/stw2616
  * Patel et al. (2020) Patel, E., Kallivayalil, N., Garavito-Camargo, N., et al. 2020, The Astrophysical Journal, 893, 121, doi: 10.3847/1538-4357/ab7b75
  * Peebles (2010) Peebles, P. J. E. 2010, Orbit of the Large Magellanic Cloud in a Dynamical Model for the Local Group, Tech. rep., doi: 10.48550/arXiv.1009.0496
  * Petersen & Peñarrubia (2020) Petersen, M. S., & Peñarrubia, J. 2020, Monthly Notices of the Royal Astronomical Society, 494, L11, doi: 10.1093/mnrasl/slaa029
  * Petersen & Peñarrubia (2021) —. 2021, Nature Astronomy, 5, 251, doi: 10.1038/s41550-020-01254-3
  * Peñarrubia et al. (2016) Peñarrubia, J., Gómez, F. A., Besla, G., Erkal, D., & Ma, Y.-Z. 2016, Monthly Notices of the Royal Astronomical Society, 456, L54, doi: 10.1093/mnrasl/slv160
  * Power et al. (2003) Power, C., Navarro, J. F., Jenkins, A., et al. 2003, Monthly Notices of the Royal Astronomical Society, 338, 14, doi: 10.1046/j.1365-8711.2003.05925.x
  * Rogers & Peiris (2021) Rogers, K. K., & Peiris, H. V. 2021, Physical Review Letters, 126, 071302, doi: 10.1103/PhysRevLett.126.071302
  * Rozier et al. (2022) Rozier, S., Famaey, B., Siebert, A., et al. 2022, Constraining the Milky Way halo kinematics via its Linear Response to the Large Magellanic Cloud, arXiv.  http://arxiv.org/abs/2201.05589
  * Safarzadeh & Spergel (2020) Safarzadeh, M., & Spergel, D. N. 2020, The Astrophysical Journal, 893, 21, doi: 10.3847/1538-4357/ab7db2
  * Salem et al. (2015) Salem, M., Besla, G., Bryan, G., et al. 2015, ApJ, 815, 77, doi: 10.1088/0004-637X/815/1/77
  * Scannapieco & Brüggen (2015) Scannapieco, E., & Brüggen, M. 2015, The Astrophysical Journal, 805, 158, doi: 10.1088/0004-637X/805/2/158
  * Schive et al. (2014) Schive, H.-Y., Chiueh, T., & Broadhurst, T. 2014, Nature Physics, 10, 496, doi: 10.1038/nphys2996
  * Schive et al. (2016) Schive, H.-Y., Chiueh, T., Broadhurst, T., & Huang, K.-W. 2016, The Astrophysical Journal, 818, 89, doi: 10.3847/0004-637X/818/1/89
  * Schneider & Robertson (2017) Schneider, E. E., & Robertson, B. E. 2017, The Astrophysical Journal, 834, 144, doi: 10.3847/1538-4357/834/2/144
  * Schutz (2020) Schutz, K. 2020, Phys. Rev. D, 101, 123026, doi: 10.1103/PhysRevD.101.123026
  * Schwabe et al. (2016) Schwabe, B., Niemeyer, J. C., & Engels, J. F. 2016, Phys. Rev. D, 94, 043513, doi: 10.1103/PhysRevD.94.043513
  * Seabroke et al. (2021) Seabroke, G. M., Fabricius, C., Teyssier, D., et al. 2021, Astronomy & Astrophysics, 653, A160, doi: 10.1051/0004-6361/202141008
  * Shipp et al. (2019) Shipp, N., Li, T. S., Pace, A. B., et al. 2019, The Astrophysical Journal, 885, 3, doi: 10.3847/1538-4357/ab44bf
  * Shipp et al. (2021) Shipp, N., Erkal, D., Drlica-Wagner, A., et al. 2021, The Astrophysical Journal, 923, 149, doi: 10.3847/1538-4357/ac2e93
  * Sparre et al. (2020) Sparre, M., Pfrommer, C., & Ehlert, K. 2020, Monthly Notices of the Royal Astronomical Society, 499, 4261, doi: 10.1093/mnras/staa3177
  * Sparre et al. (2019) Sparre, M., Pfrommer, C., & Vogelsberger, M. 2019, Monthly Notices of the Royal Astronomical Society, 482, 5401, doi: 10.1093/mnras/sty3063
  * Spergel & Steinhardt (2000) Spergel, D. N., & Steinhardt, P. J. 2000, Physical Review Letters, 84, 3760, doi: 10.1103/PhysRevLett.84.3760
  * Springel (2010) Springel, V. 2010, Monthly Notices of the Royal Astronomical Society, 401, 791, doi: 10.1111/j.1365-2966.2009.15715.x
  * Springel et al. (2021) Springel, V., Pakmor, R., Zier, O., & Reinecke, M. 2021, Monthly Notices of the Royal Astronomical Society, 506, 2871, doi: 10.1093/mnras/stab1855
  * Tamfal et al. (2021) Tamfal, T., Mayer, L., Quinn, T. R., et al. 2021, The Astrophysical Journal, 916, 55, doi: 10.3847/1538-4357/ac0627
  * Trelles et al. (2022) Trelles, A., Valenzuela, O., Roca-Fábrega, S., & Velázquez, H. 2022, Astronomy and Astrophysics, 668, A20, doi: 10.1051/0004-6361/202243671
  * Tulin & Yu (2018) Tulin, S., & Yu, H.-B. 2018, Physics Reports, 730, 1, doi: 10.1016/j.physrep.2017.11.004
  * van der Marel et al. (2012) van der Marel, R. P., Fardal, M., Besla, G., et al. 2012, ApJ, 753, 8, doi: 10.1088/0004-637X/753/1/8
  * Vasiliev (2023) Vasiliev, E. 2023, The effect of the LMC on the Milky Way system, doi: 10.3390/galaxies11020059
  * Vasiliev et al. (2021) Vasiliev, E., Belokurov, V., & Erkal, D. 2021, Monthly Notices of the Royal Astronomical Society, 501, 2279, doi: 10.1093/mnras/staa3673
  * Vera-Ciro & Helmi (2013) Vera-Ciro, C., & Helmi, A. 2013, The Astrophysical Journal, 773, L4, doi: 10.1088/2041-8205/773/1/L4
  * Virtanen et al. (2020) Virtanen, P., Gommers, R., Oliphant, T. E., et al. 2020, Nature Methods, 17, 261, doi: 10.1038/s41592-019-0686-2
  * Vitsos & Gourgouliatos (2022) Vitsos, A., & Gourgouliatos, K. N. 2022, Dynamical Friction due to fuzzy dark matter on satellites described by axisymmetric logarthmic potentials, Tech. rep.  https://ui.adsabs.harvard.edu/abs/2022arXiv221106752V
  * Vogelsberger et al. (2016) Vogelsberger, M., Zavala, J., Cyr-Racine, F.-Y., et al. 2016, Mon. Not. R. Astron. Soc., 460, 1399, doi: 10.1093/mnras/stw1076
  * Vogelsberger et al. (2012) Vogelsberger, M., Zavala, J., & Loeb, A. 2012, Monthly Notices of the Royal Astronomical Society, 423, 3740, doi: 10.1111/j.1365-2966.2012.21182.x
  * Walker & Peñarrubia (2011) Walker, M. G., & Peñarrubia, J. 2011, The Astrophysical Journal, 742, 20, doi: 10.1088/0004-637X/742/1/20
  * Weinberg (1998a) Weinberg, M. D. 1998a, Monthly Notices of the Royal Astronomical Society, 297, 101, doi: 10.1046/j.1365-8711.1998.01456.x
  * Weinberg (1998b) —. 1998b, Monthly Notices of the Royal Astronomical Society, 299, 499, doi: 10.1046/j.1365-8711.1998.01790.x
  * Widrow & Kaiser (1993) Widrow, L. M., & Kaiser, N. 1993, ApJ, 416, L71, doi: 10.1086/187073
  * Xue et al. (2015) Xue, X.-X., Rix, H.-W., Ma, Z., et al. 2015, The Astrophysical Journal, 809, 144, doi: 10.1088/0004-637X/809/2/144
  * Yavetz et al. (2022) Yavetz, T. D., Li, X., & Hui, L. 2022, Phys. Rev. D, 105, 023512, doi: 10.1103/PhysRevD.105.023512
  * Yoshida et al. (2000) Yoshida, N., Springel, V., White, S. D. M., & Tormen, G. 2000, The Astrophysical Journal, 544, L87, doi: 10.1086/317306
  * Zagorac et al. (2022) Zagorac, J. L., Sands, I., Padmanabhan, N., & Easther, R. 2022, Physical Review D, 105, 103506, doi: 10.1103/PhysRevD.105.103506
  * Zonca et al. (2019) Zonca, A., Singer, L., Lenz, D., et al. 2019, Journal of Open Source Software, 4, 1298, doi: 10.21105/joss.01298


# Planning Visual-Tactile Precision Grasps via Complementary Use of Vision and
Touch

Martin Matak1 and Tucker Hermans1,2 Manuscript received: September 2, 2022;
Revised November 18, 2022; Accepted December 14, 2022.This paper was
recommended for publication by Hong Liu upon evaluation of the Associate
Editor and Reviewers’ comments. This work was supported by supported by NSF
Award #1846341, by DARPA grant N66001-19-2-4035, and by a Sloan Research
Fellowship.1 Martin Matak and Tucker Hermans are with the School of Computing
and the Robotics Center, University of Utah, Salt Lake City, UT, USA.
<EMAIL_ADDRESS>NVIDIA; Seattle, WA, USADigital Object Identifier
(DOI): see top of this page.

###### Abstract

Reliably planning fingertip grasps for multi-fingered hands lies as a key
challenge for many tasks including tool use, insertion, and dexterous in-hand
manipulation. This task becomes even more difficult when the robot lacks an
accurate model of the object to be grasped. Tactile sensing offers a promising
approach to account for uncertainties in object shape. However, current
robotic hands tend to lack full tactile coverage. As such, a problem arises of
how to plan and execute grasps for multi-fingered hands such that contact is
made with the area covered by the tactile sensors. To address this issue, we
propose an approach to grasp planning that explicitly reasons about where the
fingertips should contact the estimated object surface while maximizing the
probability of grasp success. Key to our method’s success is the use of visual
surface estimation for initial planning to encode the contact constraint. The
robot then executes this plan using a tactile-feedback controller that enables
the robot to adapt to online estimates of the object’s surface to correct for
errors in the initial plan. Importantly, the robot never explicitly integrates
object pose or surface estimates between visual and tactile sensing, instead
it uses the two modalities in complementary ways. Vision guides the robots
motion prior to contact; touch updates the plan when contact occurs
differently than predicted from vision. We show that our method successfully
synthesises and executes precision grasps for previously unseen objects using
surface estimates from a single camera view. Further, our approach outperforms
a state of the art multi-fingered grasp planner, while also beating several
baselines we propose.

###### Index Terms:

Multifingered Hands; Grasping; Deep Learning in Grasping and Manipulation

##  I Introduction

In this letter we look into enabling a robot to pick up everyday objects that
the robot hasn’t encountered before. We focus on the task of grasping
previously unseen objects as a fundamental step toward successfully deploying
robots in household environments. In such environments, the robot should be
able not to only pick up objects, but also to use them for everyday tasks [1].
This requires the ability of fine in-hand manipulation of objects, which makes
multi-fingered hands a good candidate as a general end effector. While multi-
fingered hands seem promising for downstream tasks, using them to grasp
everyday objects and ensuring the resulting grasp makes contact only with the
fingertips remains a challenging problem [2, 3].

It’s obvious that contact with an object must be made in order to successfully
pick the object up, but it’s unclear how to explicitly enforce this constraint
when a robot doesn’t know the complete object shape. This is the core focus of
this paper—establishing an in contact, precision grasp with a previously
unseen object when given a single RGBD image from a fixed camera-view. To help
account for the lack of information a single camera view provides, we examine
how tactile sensing can improve the grasp control process. In particular we
examine how to do this when the robot only has tactile sensing on its distal
links (fingertips), a common setup on contemporary hands [4, 5, 6].

In general, tactile sensors improve grasping. Calandra and colleagues [7] show
that tactile feedback, in addition to vision-based input, provides more
information for computing a good grasp, while Murali et al. [8] present an
approach to grasping using tactile sensing without vision. However, it’s
unclear how to get to a grasp such that tactile feedback is registered, i.e.
the contact is made with the tactile sensors when planning with fully
actuated, dexterous hands. Having only limited tactile coverage on most
robotic hands doesn’t help either. One approach [2] is to explicitly model a
precision grasp, which would result in a grasp where contact with the object
is made with fingertips of the hand, exactly where the tactile sensors are.
After the hand is in contact with the object, Veiga et al. [9] show how to
stabilize the grasp using tactile feedback. Apart from the tactile-guided
approaches, there are many vision-based methods to tackle grasping [10].
Several of these use 3D reconstruction from partial views to aid the grasp
planning process [11, 12, 13] and others examine reconstructing object shape
from touch [14, 8, 15]. However, no work has shown the ability to plan, adapt,
and ensure precision grasps for multi-fingered hands through joint visual and
tactile sensing in the real world. Indeed, as discussed in Section II, most
multi-fingered grasping approaches use simple, heuristic controllers to move
from a grasp preshape to an in-contact grasp. Further, similar works evaluated
in simulation make unrealistic assumptions such as full tactile coverage of
the robot and the ability to repeatedly make contact with the object without
it moving.

To address this gap, we propose a learning-guided and geometry informed grasp
controller to autonomously generate precision grasps on previously unseen
objects. We formulate grasp synthesis as an optimization problem with a hard
constraint that contact must be made between the estimated object surface and
the robot’s tactile fingertips. The robot uses a neural network that encodes
the probability of grasp success as its objective function as in [16]. We
generate an initial estimate of the object surface from the partial-view point
cloud using the PointSDF [11] neural network.

While the estimated object surface and pose provide a model useful for
planning a grasp, we also know it is very likely inaccurate. To account for
this we leverage tactile feedback during the grasp control phase to adapt to
the uncertain object shape and ensure contact. Notably, we do not fuse the
visual and tactile surface estimates into a single model, instead we use them
to separately plan and ensure contact. As such we avoid the difficult problem
of registering the different measurements under extrinsic calibration error
and possible object motion. After closing the hand, the robot evaluates an in-
contact grasp classifier to decide whether to lift the object. Figure 1
visualizes all stages of our approach.

We experimentally demonstrate that our approach can reliably generate multi-
fingered precision grasps for novel objects without explicitly integrating
vision and touch. We run real world robot experiments that quantify that using
both vision and touch improves grasp performance over using vision alone. Our
results additionally characterize the benefits of our approach over prior
work. We successfully plan and execute precision grasps for various, novel
objects observed from a single camera view. Some examples of our grasps are
shown in Fig. 4. Experiments videos and source code can be found on our
project website: https://sites.google.com/view/precision-grasps.

##  II Related work

We divide our discussion of previous grasp planning work between geometry-
based and learning-based approaches before discussing grasping with tactile
sensing.

###  II-A Geometry-based Grasp Synthesis

Geometric-based grasping approaches leverage a model of the object geometry in
order to compute a grasp metric. One of the most popular approaches in this
category is Eigengrasp [17]. Eigengrasp computes a grasp pose and joint
configuration in a low-dimensional “eigengrasp” space and closes the fingers
at equal joint velocity rates until detecting contact. Assuming full knowledge
of object shape, Roa and Suarez [18] determine contact regions on the object
to make contact with, such that the resulting grasp is in force closure.
Rosales et al. [19] solve for hand configuration to be in contact with pre-
specified regions on the object. Zheng and Qian [20] quantify friction and
contact position uncertainty when computing force closure grasps.

Another geometry-based, uncertainty aware grasp synthesis approach comes from
[21], which fuses multiple camera view to infer a “pre-touch” configuration to
maximize grasp success. Hang et al. [22] synthesize a grasp that is in contact
with an object, and once the hand is in contact, use a learned probabilistic
controller to stabilize it. Siddiqui [23] use a Bayesian inspired method to
efficiently explore a variety of grasps. After moving to a predicted pose, the
hand is closed using a simple controller until contact is detected. Morales et
al. [24] build a database of possible power and precision grasps and retrieve
a grasp for a target object at run time. They obtain precision grasps using a
simple heuristic controller, but execute no grasps in the real world. Planning
to grasp an object with uncertain pose, but known geometry, has also been
examined formally as a belief-space planning problem [25], however, scaling
this to be efficient with uncertainty in shape, while likely useful, is an
open challenge. In the case of uncertain shape, such as when estimated from a
single camera view as is our focus, these geometry-based grasp metrics will
likely be inaccurate or suboptimal due to inaccuracies in estimated shape.

###  II-B Learning-based Grasp Synthesis

The research literature contains numerous data-driven grasp approaches [26,
27, 28, 29, 7, 30, 3]. We build on the approach from [16] to generate a grasp
preshape. The primary novelty of our work comes after the robot moves to this
preshape. We classify learning-based grasp synthesis approaches between those
that generate grasps in contact with the object and grasps that are close to,
but not in contact with the object. We found no work that combines the two.

In contact grasps: Varley et al. [27] train a CNN to predict heatmaps for
fingertip locations on the object as well as palm pose, which are used to
synthesize a grasp. While Veres et al. [31] generate contact points on an
object for a 3-fingered hand using generative models (in simulation), Shao et
al. [32] manage to synthesize grasps as contact points on an object for
various grippers and demonstrate their approach in a real-world setting.
Sundermeyer and colleagues [33] predict grasp locations such that one finger
of a parallel jaw gripper makes contact with the observed point cloud, this
allows for a clever reparameterization of the 6DOF prediction to 4DOF. However
the second finger location is determined based on a predicted gripper width,
so it is unclear how to scale this approach to fully actuated multi-fingered
hands. All approaches in this category predict grasps in contact with the
object and are therefore brittle to shape estimation and camera noise. To
resolve this, other approaches predict grasps near the object, but not in
contact.

Near contact grasps: Kappler et al. [26] predict grasp as a palm pose, Wu et
al. [34] use a policy gradient method to predict grasp pose and joint angles
from a depth image. Lundell and colleagues [35, 12] use a GAN to predict
grasps that are almost on the surface of the object, yet not in contact.
Similar to [3], Mousavian et al. [36] initialize and refine a grasp pose using
a learned neural net. Our own previous work [2, 16, 3, 11] synthesizes a
preshape similarly as in this letter, but doesn’t explicitly reason about
making contact either through vision or touch. All the approaches in this
category after moving the robot to the preshape use either a simple heuristic
controller to close the hand or don’t specify how exactly the hand is closed.
We identify this as a gap in the literature and explore it here. For a deeper
overview of dexterous grasping, we refer the reader to [37].

###  II-C Tactile Enabled Grasp Evaluation and Control

Calandra and colleagues [38] propose a grasping approach that maximizes grasp
success probability using a sampling based method. Their model predicts
success probability based on tactile readings, RGB image, and a regrasping
action for a two-fingered gripper. If the predicted grasp success probability
is above a threshold (90%), the robot attempts to lift the object. Hogan et
al. [5] follow a similar approach but use a tactile-only grasp quality metric.
For their resampling strategy they simulate various actions (movements of the
gripper) and pick the action with the highest probability of success, as
predicted by the tactile-only metric. Wu et al. [39] use a deep reinforcement
learning method to implement a closed-loop approach to grasping. They use
tactile sensors on the 3-fingered Barett Hand as part of the observations used
by the RL policy. Dang and Allen [40] regrasp by searching for the closest
stable grasp in the database of all the previous grasps performed by the
robot. After finding such a grasp, they adjust the hand. Similar to the rest
of the work in this section, the tactile readings are used for the grasp
quality metric, but also to find similar grasps in the database. Murali et al.
[8] use learned corrective actions that map tactile input in latent space to
gripper adjustment in task space. Dragiev et al. [14] develop a tactile-
informed regrasp control law. Farias et al. [41] estimate object shape from
vision, update the shape estimate using tactile feedback, and use the
resulting model to estimate force closure. Both papers [41, 14] only evaluate
in simulation and assume the robot can detect contact anywhere on the hand.

##  III Problem Description

The problem we wish to solve has two main phases. First, the robot plans a
precision grasp that maximizes the probability of grasp success under
uncertain object surface estimation. Then, the robot must execute the planned
grasp using tactile feedback to compensate for shape uncertainty.

Let $f(\bm{q},\bm{o})$ estimate the probability of grasp success for
configuration $\bm{q}$ for object $\bm{o}$. Let the robot have $n$ degrees of
freedom (DoF) in the arm and $m$ DoF in the hand. We are searching for the
robot configuration $\bm{q}\in\mathbb{R}^{n+m}$ such that every fingertip is
on the surface of the object $\bm{o}$ and that $f$ is maximized. The surface
of the object $\bm{o}$ is estimated from a partial pointcloud. Let
$\phi_{p}^{i}:\mathbb{R}^{4}\rightarrow\mathbb{R}^{3}$ denote the forward
kinematics function for the position of the central point on the surface of
the fingertip $i$. Input to $\phi_{p}^{i}$ is in $\mathbb{R}^{4}$ because
every finger on the hand we work with has 4 DoF. Let $\texttt{SDF}(p,\bm{o})$
be the signed distance function between a point $p\in\mathbb{R}^{3}$ and the
object $\bm{o}$, returning positive values for $p$ outside the object,
negative inside, and 0 on the surface.

We formalize the search for the grasp configuration $\bm{q}^{*}$ as

$\displaystyle\bm{q}^{*}=\operatorname*{arg\,max}_{\bm{q}}\quad$
$\displaystyle f(\bm{q},\bm{o})$ (1) s.t.
$\displaystyle\bm{q}_{min}\preceq\bm{q}\preceq\bm{q}_{max}$ (2)
$\displaystyle\texttt{self\\_coll}(\bm{q})=0$ (3)
$\displaystyle\texttt{SDF}(\phi_{p}^{i}(\bm{q}),\bm{o})=0\;\forall
i\in\\{1,\ldots,F\\}.$ (4)

Equation (2) constrains the solution within joint limits, while the function
$\texttt{self\\_coll}(\cdot)$ counts the number of hand links in self-
collision and $F$ denotes the number of fingers we wish to make contact with
the object. In our experiments $F=4$.

A solution to the optimization problem defines a grasp such that fingertips
are in contact with the object surface and not in self collision. However, the
estimated object shape is likely imperfect and hence there is inherently a
mismatch between estimated object surface and the real one. This defines the
second part of the problem - moving the robot to the planned grasp knowing a
contact with the object might happen before or after expected. Not accounting
for this may result in knocking the object over or stopping before contact is
made. Formally, given current configuration $\bm{q}[k]$, tactile readings
$\tau[k]$, and target configuration $\bm{q}_{F}$, the robot must execute a
control policy $\bm{u}=\pi(\bm{q}[k],\tau[k],\bm{q}_{F})$ to achieve a grasp
where its tactile fingertips contact the object.

##  IV Our Approach

Our approach consists of four steps: (1) planning and moving the robot arm and
hand to a grasp preshape configuration, (2) planning an in-contact grasp using
the estimated object shape, (3) moving the robot fingers to the planned in-
contact grasp configuration and (4) evaluating the robot’s in-contact
classifier to decide whether to lift the object or regrasp it. The first two
steps base decisions purely on point cloud input, whereas the third step uses
tactile feedback to adapt the plan online. The fourth step uses visual input
and the current robot configuration. We do not fuse visual and tactile sensing
to make a unified model of the object surface, instead we use the two
separately to satisfy the contact constraint during planning and control
respectively. Figure 1 visualizes these steps. We now describe the four
components in turn, followed by implementation details of our learned models.

###  IV-A Grasp Preshape Synthesis

As the first step in our approach we plan a grasp preshape
$\bm{q}_{0}=[\bm{q}_{A},\bm{q}_{H}]$ which maximizes the probability of grasp
success, but does not make contact with the object. The preshape contains both
the arm configuration $\bm{q}_{A}\in\mathbb{R}^{n}$ and the hand configuration
$\bm{q}_{H}\in\mathbb{R}^{m}$. We solve for the preshape using the approach
from [16] which maximizes the probability of grasp success using a learned
neural network as its objective. The objective contains two terms: a mixture
density network (MDN) as a prior, and a preshape classifier as a likelihood
component. The MDN additionally serves as a sampler to initialize the
optimization. Both neural networks take as input a voxelized representation of
the object from the observed single-view RGBD image. We give further details
about the networks and their training below. The preshape planner serves to
move the palm of the hand to a pose close to the object and the proximal
joints of each finger into a configuration amenable to making a high quality
precision grasp. As in [16] we optimize using bound constrained BFGS [42].
Figure 1 left shows an example grasp preshape.

###  IV-B In-Contact Grasp Synthesis and Evaluation

After moving to the preshape configuration $\bm{q}_{0}$, the robot plans to
make contact with the object. The robot uses $\bm{q}_{0}$ to initialize its
optimization for a hand configuration in contact with the object. This differs
from the previous step, where we optimize for arm and hand configuration. For
the rest of the approach, the arm configuration $\bm{q}_{A}$ remains fixed and
we only change the joints of the fingers $\bm{q}_{H}$.

We reformulate the in-contact grasp synthesis problem from Eqs. (1)-(4), by
relaxing the SDF constraint and using an estimate of the SDF,
$\widehat{\texttt{SDF}}{}$. We add an additional term to the objective to
encourage alignment between the fingertip surface normal and the approach
vector resulting in the following penalty-method objective,
$\tilde{f}(\bm{q},\bm{o},\bm{p})=$

$\sum_{i=1}^{F}||{\widehat{\texttt{SDF}}{}(\bm{p}_{i},\bm{o})}||^{2}+||\phi_{p}^{i}(\bm{q})-\bm{p}_{i}||^{2}+\alpha||1-\cos\theta_{i}||^{2}\vspace{-6pt}$
(5)

where $\theta_{i}$ is angle between the normal on $i$th fingertip center point
and the approach vector $\bm{p}_{i}-\phi_{p}^{i}(\bm{q})$. Here $\bm{p}_{i}$
serves as an auxiliary decision variable defining a point between the
fingertip and object as in [43]. We initialize it halfway between the $i$th
fingertip and estimated object surface. We found that including $\bm{p}_{i}$
was important for numerical stability. Finally, $\alpha$ weighs between
minimizing distance and aligning orientation of the fingertip. We describe
implementation details of $\widehat{\texttt{SDF}}(\cdot)$ later in this
section.

We solve the objective from Eq. (5) using a Gauss-Newton solver with bound
constrains to handle the joint limits with $\alpha=\frac{1}{j^{2}}$. Here $j$
defines the current solver iteration. We further implement a projection method
on our line search to handle self collisions of the hand. We show an example
of a solution in Fig. 1 center left. We can see that the planned configuration
makes contact with the object at the estimated object surface. However, the
estimated object surface is likely inaccurate due to error in object pose
estimation and shape reconstruction. In the next subsection, we describe how
we execute our planned grasp to handle this uncertainty.

###  IV-C Making Contact

To move to the target configuration, we compute time-optimal trajectories for
each finger [44] and then scale the trajectories to be of equal duration, so
that all fingers reach their target configurations at the same time. This
lowers the chance of moving the object when making contact. The robot
evaluates the constraint described in Eq. (4) online using tactile feedback
during trajectory execution. This allows the robot to detect contact before
the trajectory is completely executed. If the tactile classifier $t(\tau[k])$
detects contact at timestep $k$, the constraint is satisfied and the finger in
contact stops moving. If after $K$ steps a finger reaches the desired
configuration, but does not detect contact, the robot moves its finger toward
the object surface following the gradient of $\widehat{\texttt{SDF}}$ in
attempt to make contact. The finger stops following the
$\widehat{\texttt{SDF}}{}$ gradient if it hasn’t made contact after $\bar{k}$
additional control iterations as this implies the shape estimate gradient is
uninformative at this location. Formally, $\pi(\bm{q}[k],\tau[k])=$

$\displaystyle\begin{cases}0&\mbox{if}\,t(\tau[k])\geq p_{c}\vee
k>K+\bar{k}\\\ \Delta\bm{q}[k]&\mbox{if}\,t(\tau[k])<p_{c}\land k\leq K\\\
-\gamma\nabla_{\bm{q}}\widehat{\texttt{SDF}}(\phi_{p}(\bm{q}),\bm{o})&\mbox{if}\,t(\tau[k])<p_{c}\land
K<k\leq K+\bar{k}\end{cases}$ (6)

where $\Delta\bm{q}[k]$ is the initial plan of $K$ steps; $p_{c}$ is the
tactile pressure threshold of 365 Pa; $\gamma=1.0$; and $\bar{k}=1$.

The hand tracks the joint positions commanded by $\pi$ using an inverse-
dynamics torque controller. An example of a grasp after running our controller
can be seen in Fig. 1 center right. Details about the classifier $t(\cdot)$
are explained in the next subsection. After making contact, we increase grasp
force prior to lifting using a heuristic. If more force is not applied, the
object slips through while the arm is attempting to lift it. The heuristic
commands $0.7$ rad increase to all but the first joint on non-thumb fingers,
and $0.5$ rad increase to last two joints of the thumb. A promising direction
for future work is a more systematic reasoning about the applied force.

Despite making contact with the object surface, the resulting configuration
might not generate a successful grasp. To estimate whether the robot should
lift the object with the resulting configuration, we evaluate our in-contact
grasp classifier $h_{c}$ to estimate the probability of grasp success. When
evaluating $h_{c}$, we assume the object pose has not changed, despite
applying more force, and feed in the same visualize input as for the preshape
classifier. In-hand object pose tracking is a challenging problem and out of
the scope for this paper. Similar to previous works [7, 39], if the output of
the in-contact grasp classifier is below a threshold, the robot can reject the
grasp and attempt a regrasp.

###  IV-D Learned Model Architectures

We now describe the learned MDN $g(\cdot)$ and the preshape grasp classifier
$h_{p}(\cdot,\cdot)$, which are used to generate the preshape solution
$\bm{q}_{0}$. Then we describe the model used to estimate the initial object
shape $\widehat{\texttt{SDF}}(\cdot)$ and the tactile classifier $t(\cdot)$
used to detect contact with the object. Finally, we describe our in-contact
grasp classifier $h_{c}(\cdot,\cdot)$ which is used to decide whether to lift
the object or attempt a regrasp.

Following [16], we use a mixture density network (MDN) $g(\cdot)$ that takes
the voxelized object representation as an input and outputs a palm pose in
object frame, as well as the joint positions for the two joints closest to the
palm for each finger. We model the preshape grasp classifier
$h_{p}(\cdot,\cdot)$ as a neural network described in [16]. The classifier
receives as input the palm pose in object frame, the voxelized object
representation, the object dimensions and the first two joint values of every
finger. The classifier outputs estimated probability of grasp success. Further
details on the training procedure are in Section V. We model our in-contact
grasp classifier $h_{c}$ similarly to our preshape grasp classifier $h_{p}$
with one difference: $h_{c}$ receives all hand joints as input, while $h_{p}$
receives only two joints per finger.

To reconstruct object shape from a single view, we use the PointSDF model from
[11] as our implementation of $\widehat{\texttt{SDF}}$. The model takes as
input the object point cloud and a query point and outputs the signed distance
to the query point, scaled to between -1 and 1. For further details see [11].

We use the BioTac sensor installed on all fingertips for tactile feedback
[45]. We define a contact classifier $t(\cdot)$ that receives as input the
fluid pressure in the sensor ($P_{DC}$). Before the hand starts to close, the
robot tares the sensor, such that the current pressure value reads zero. When
closing the hand, if the pressure exceeds the preset threshold $p_{c}$ for 10
consecutive timestamps, the classifier $t(\cdot)$ reports contact. The sensor
is sampled at a frequency of 100 Hz.

##  V Experiments

We start this section by describing the data collection procedure used to
train the grasp classifier. We then explain our baselines for comparison
before discussing experiments and results for evaluating making contact and
grasp success. Our experiments show: (1) that our proposed approach
outperforms the state-of-the-art (SoTA) comparison as well as tactile enhanced
SoTA; (2) the benefits of having both preshape and in-hand classifiers; and
(3) the importance of estimating object geometry through the learned SDF.
Figure 4 shows example grasps generated by our approach.

###  V-A Simulation: Data Collection and Training

We conduct all training only in simulation using the four-fingered, $m=16$ DoF
Allegro hand mounted on a Kuka LBR4 $n=7$ DoF arm, which is the same as our
real world setup. We use a Microsoft Azure Kinect camera to generate the point
cloud of the object on the table. We collected simulated grasp data using our
robot hand-arm setup inside the Gazebo simulator with the DART physics engine.
We use the built-in Gazebo Kinect camera to generate point clouds simulating
the Microsoft Azure Kinect camera we use in real-world experiments.

We collected training data using a heuristic, geometry-based planner adapted
from [16]. However, our approach differs in how the robot closes its hand
after moving to the preshape. We formulate hand-closing problem as the
optimization problem defined in Eq. (5), where we set $\alpha=0.0$ to speed up
the optimization, and solve it using Gauss-Newton. After moving to the
computed joint position, we increase the stiffness of the grasp by $0.2$rad to
mimic a grasp stability controller. During data collection for training (and
only for training), we use the true mesh of the object and no tactile
feedback. To check for self-collisions, we use the GJK algorithm [46]. In
total, we collected 13,831 samples, out of which 3777 are successful grasps.

We pretrain a voxel autoencoder on a 3D reconstruction task [16] and extract
the voxel encoder from the network for use in the grasp classifiers $h_{p}$
and $h_{c}$. We freeze the extracted weights and train the rest of the
classifier using the Adam optimizer. We find that pretraining the autoencoder
was a necessary step to train the classifiers successfully. On the validation
set (10% of the collected dataset) $h_{p}$ and $h_{c}$ achieve 78% and 79%
accuracy respectively. This is comparable to numbers reported in [16].
Following [16], we fit a mixture density network (MDN) that takes the object
point cloud as input and outputs a grap preshape sample. The MDN uses the same
voxel encoder as the classifiers.

Figure 2: The objects used in experiments. Objects from left to right; back
row: “pitcher”, “soft scrub”, “pringles”, “wooden block”, “cheezeit”, “master
chef”, “domino”, “mustard”; front row: “bowl”, “red cup”, “yellow cup”,
“squishy brick”, “rubic cube”, “tuna can”, “jello red”, “spam”, “jello brown”
and “chain”.

###  V-B Baselines for Comparison

We implement five baseline comparisons to quantify the benefits of the
different components of our approach. Our first baseline, B1 uses the
heuristic, grasp controller from [2] to close the hand. This represents a SoTA
comparison to our complete system. Since B1 does not use tactile sensors, we
implement B2 that uses the same controller as B1, but stops when the tactile
classifier detects contact instead of the sensed joint velocities. This allows
us to understand to what extent any improvement comes from adding tactile
sensors to the grasp controller. Baseline B3 samples a preshape configuration
from an MDN trained on all positive samples from the training dataset, but
closes the hand using our visual-tactile controller. This examines whether our
controller removes the need for the initial preshape optimization and hence
the preshape classifier. Further, we look into the importance of object shape
completion by introducing B4 which reconstructs only the visible region of the
object using marching cubes [47] from the camera’s partial point cloud, which
we use with the trimesh library [48] to evaluate the SDF. This contrasts with
our approach which uses PointSDF to estimate the entire shape of the object.
We implement baseline B5 as an additional comparison for evaluating the
ability for the controllers to make fingertip contact with the object without
moving it. We simply execute our controller $\pi$ open-loop stopping after $K$
steps ignoring tactile feedback.

###  V-C Making Contact in the Real World

We first evaluate our proposed controller on generating grasps that make
fingertip contact with the object without moving it. We report the number of
fingertips in contact with the object after closing the hand, as well as
whether the object remained flat on the table, tilted, or was knocked over by
the robot. We compare our controller to B1 and B5 by generating and saving a
preshape for a given object using our grasp planner and then executing the
selected grasp controller. We manually label whether the object tilted or
fell. We run experiments on 5 different objects (pringles, domino, soft scrub,
mustard, and wooden block) in 5 different poses resulting in 75 hand closing
episodes in total.

Figure 3: Number of fingertips in contact with the object after closing.

Figure 3 visualizes our results. Our approach results in more fingertips in
contact with the object than using B1 or B5. The object tilted only 3 times
when using our controller, while B5 tilted the object 10 times, and B1 tilted
21 times. Further, we report that our method never knocked over the object,
while the vision-only B5 knocked over an object once, and B1 did so three
times. Our method sometimes failed in making all four fingers contact the
object. We believe three issues contribute to this: (1) the object simply
stands out of reach for some fingers for the generated preshape, (2) the
$\widehat{\texttt{SDF}}$ is not accurate enough to lead the controller in the
right direction, (3) the tactile contact classifier is sometimes inaccurate.
While imperfect, we show next that our approach provides advantages over the
comparison methods for our task of interest–grasping.

###  V-D Grasping in the Real World

We perform real-world experiments using 18 objects from the YCB [49] object
set. These objects span different textures, shapes, masses, and sizes. We show
the experimental setup and objects used in Fig. 2. All experimental objects
but the pringles can are unseen in training. For experiments, we place a
single target object on the table and run our pipeline. We chose a fixed test
location on the table for the target object and rotate the object to 3
different angles, keeping the poses constant across the methods and objects.
Given a resulting robot joint configuration from our grasp planner, we perform
motion planning using the RRTConnect planner from MoveIt! to obtain a
collision free trajectory to the preshape configuration. We provide the
bounding box of the object to the motion planner for collision checking. We
find that this conservative approach yields fewer collisions with the object
compared to providing the reconstructed mesh of the object to the motion
planner. After executing the motion plan, the robot closes its hand using the
grasp controller. Then, the robot attempts to lift the object approximately
15cm above the table. Only if the object is in the hand after executing the
lifting trajectory, we label the attempt as a successful grasp. If the robot
moved to the preshape, but failed to lift the object, we label the attempt as
unsuccessful. The robot attempts 270 grasps total, 54 for each of the 5
comparative methods.

Figure 4: Example precision grasps resulting from our controller.

####  V-D1 Quantitative results

Figure 5 presents the grasp success rates for the taller objects that comprise
the back row in Fig. 2. Overall success rates for B1-4 and our approach are
$65\%,28\%,46\%,75\%$, and $75\%$, respectively.

Figure 5: Grasp success rates for each tall object and overall.

Our approach outperforms most of the baselines and successfully picks up most
of the objects. However, B1, B4, and our approach all fail to find a plan to
pick up “cheezeit”. This arises from the initial grasp preshape optimization
always coming into contact with the object. This seems to be somewhat an issue
of random initialization as B2, which uses the same preshape generation was
successful once. However, both B2 and B3 had the overall lowest performance.
Only B4 has a success rate equal to our proposed approach. As such we see that
the proposed in-contact grasp planner and associated tactile feedback
controller combine to improve over the previous SoTA B1. We can attribute the
lack of improvement from using tactile feedback in B2 to the fact that the
heuristic grasp controller does not explicitly try to make contact with the
tactile fingertips. Thus B2 would often make contact with other parts of the
hand when closing pushing away or knocking over the object. In contrast, while
our in-contact planner would generate target configurations that penetrate the
object, the resulting grasps often succeeded as the robot detected contact
with the tactile sensors.

Recall that the only difference between our proposed approach and B4 is the
SDF estimate. To further investigate the difference in performance, we examine
the “low profile” objects shown in the front row of Fig. 2. We define “low
profile” objects to be objects shorter than $10$cm. The tallest low-profile
object is “jello brown” with a height of $89$mm. Our approach achieves a 40%
rate for low-profile objects, while B1-4 have respective success rates of
$20\%,9\%,10\%$ and $18\%$. This result demonstrates a substantial improvement
in using PointSDF for reasoning about small-object geometry compared with
running marching cubes in B4 for planning.

That said, across the 10 different low profile objects, we were able to
execute only 78 out of 150 attempts, because of the resulting preshape coming
into collision with the table. This arises from the lack of explicit reasoning
about collisions with the environment in our optimization. We point out that
this is one of the biggest shortcomings of our method and we believe this is a
promising avenue for future work.

Finally, we look into the performance of our in-contact grasp classifier
$h_{c}$. We analyze the performance of the classifier using only B3, B4 and
our method as those use our proposed grasp controller. We filter out the data
where the trajectory couldn’t be executed. We show the resulting precision-
recall curve for executed grasps in Fig. 6. This shows the classifier performs
well in the real world on previously unseen objects despite being trained only
in simulation. It also shows that the classifier can be used to further
increase grasp success rates by executing grasps only if the classifier
prediction is above a high threshold. To construct the precision-recall
curves, in our experiments we didn’t filter out any grasps based on the
estimated probability of success. Finally, we see that the $h_{c}$ obtains
both high precision and recall when coupled with our proposed approach.

Figure 6: Precision-recall curve for our in-contact grasp classifier, $h_{c}$.
We compare the performance of our approach, B3, and B4, the three approaches
that use our proposed tactile controller.

####  V-D2 Qualitative results

While our approach outperforms the baselines quantitatively, we think it
shines brightest when one looks at the generated grasps. In Fig. 4 we show
some grasps resulting from our controller, where the robot successfully grasps
objects such that the tactile sensors contact the object. Since B1 and B2
don’t explicitly reason about making contact with the fingertips, we found
that the resulting grasps tend to make contact with other parts of the hand
instead. This makes our approach suitable for downstream tasks that require
updating the object shape or pose estimate or requiring precision in-hand
manipulation.

##  VI Conclusion

We proposed an approach for a robot to plan and execute multi-fingered,
precision grasps for previously unseen objects. Key to our methods success is
the use of initial shape estimation from a single camera view as a constraint
in the grasp planner, coupled with the use of tactile sensing to adapt the
grasp online. Despite training only in simulation, our approach works well in
the real world and generates more precision grasps than the comparison
approach [16].

## References

  * [1] S. Cruciani, B. Sundaralingam, K. Hang, V. Kumar, T. Hermans, and D. Kragic, “Benchmarking In-Hand Manipulation,” _IEEE Robotics and Automation Letters_ , vol. 5, no. 2, 1 2020.
  * [2] Q. Lu and T. Hermans, “Modeling Grasp Type Improves Learning-Based Grasp Planning,” _IEEE Robotics and Automation Letters_ , 2019.
  * [3] Q. Lu, M. V. der Merwe, and T. Hermans, “Multi-Fingered Active Grasp Learning,” in _IEEE/RSJ Intl. Conf. on Intelligent Robots and Systems_ , 2020.
  * [4] A. Yamaguchi and C. G. Atkeson, “Combining Finger Vision and Optical Tactile Sensing: Reducing and Handling Errors While Cutting Vegetables,” in _IEEE/RAS Intl. Conf. on Humanoid Robots_ , 2016.
  * [5] F. R. Hogan, M. Bauza, O. Canal, E. Donlon, and A. Rodriguez, “Tactile regrasp: Grasp adjustments via simulated tactile transformations,” in _IEEE/RSJ Intl. Conf. on Intelligent Robots and Systems_ , 2018.
  * [6] B. Sundaralingam, A. S. Lambert, A. Handa, B. Boots, T. Hermans, S. Birchfield, N. Ratliff, and D. Fox, “Robust learning of tactile force estimation through robot interaction,” in _IEEE Intl. Conf. on Robotics and Automation_ , 2019\.
  * [7] R. Calandra, A. Owens, M. Upadhyaya, W. Yuan, J. Lin, E. H. Adelson, and S. Levine, “The feeling of success: Does touch sensing help predict grasp outcomes?” in _Conference on Robot Learning_ , 2017.
  * [8] A. Murali, Y. Li, D. Gandhi, and A. Gupta, “Learning to grasp without seeing,” in _Intl. Symposium on Experimental Robotics_ , 2018.
  * [9] F. Veiga, B. B. Edin, and J. Peters, “In-hand object stabilization by independent finger control,” _Sensors_ , vol. 20, no. 6, 2020.
  * [10] J. Bohg, A. Morales, T. Asfour, and D. Kragic, “Data-driven grasp synthesis—a survey,” _IEEE Trans. on Robotics_ , vol. 30, no. 2, 2014\.
  * [11] M. Van der Merwe, Q. Lu, B. Sundaralingam, M. Matak, and T. Hermans, “Learning Continuous 3D Reconstructions for Geometrically Aware Grasping,” in _IEEE Intl. Conf. on Robotics and Automation_ , 2020.
  * [12] J. Lundell, E. Corona, T. Nguyen Le, F. Verdoja, P. Weinzaepfel, G. Rogez, F. Moreno-Noguer, and V. Kyrki, “Multi-fingan: Generative coarse-to-fine sampling of multi-finger grasps,” in _IEEE Intl. Conf. on Robotics and Automation_ , 2021.
  * [13] Z. Jiang, Y. Zhu, M. Svetlik, K. Fang, and Y. Zhu, “Synergies between affordance and geometry: 6-dof grasp detection via implicit representations,” in _Robotics: Science and Systems_ , 2021.
  * [14] S. Dragiev, M. Toussaint, and M. Gienger, “Uncertainty aware grasping and tactile exploration,” in _IEEE Intl. Conf. on Robotics and Automation_ , 2013.
  * [15] Z. Yi, R. Calandra, F. Veiga, H. van Hoof, T. Hermans, Y. Zhang, and J. Peters, “Active Tactile Object Exploration with Gaussian Processes,” in _IEEE/RSJ Intl. Conf. on Intelligent Robots and Systems_ , 2016.
  * [16] Q. Lu, M. V. der Merwe, B. Sundaralingam, and T. Hermans, “Multi-fingered grasp planning via inference in deep neural networks,” _IEEE Robotics & Automation Magazine_, 2020.
  * [17] M. T. Ciocarlie and P. K. Allen, “Hand posture subspaces for dexterous robotic grasping,” _Intl. Journal of Robotics Research_ , vol. 28, no. 7, 2009.
  * [18] M. A. Roa and R. Suarez, “Computation of independent contact regions for grasping 3-d objects,” _IEEE Trans. on Robotics_ , vol. 25, no. 4, 2009\.
  * [19] C. Rosales, L. Ros, J. M. Porta, and R. Suárez, “Synthesizing grasp configurations with specified contact regions,” _Intl. Journal of Robotics Research_ , vol. 30, no. 4, 2011.
  * [20] Y. Zheng and W.-H. Qian, “Coping with the grasping uncertainties in force-closure analysis,” _Intl. Journal of Robotics Research_ , vol. 24, no. 4, 2005.
  * [21] D. Chen, V. Dietrich, Z. Liu, and G. von Wichert, “A probabilistic framework for uncertainty-aware high-accuracy precision grasping of unknown objects,” _Journal of Intelligent & Robotic Systems_, vol. 90, no. 1, 2018.
  * [22] K. Hang, M. Li, J. A. Stork, Y. Bekiroglu, F. T. Pokorny, A. Billard, and D. Kragic, “Hierarchical fingertip space: A unified framework for grasp planning and in-hand grasp adaptation,” _IEEE Trans. on Robotics_ , vol. 32, no. 4, 2016.
  * [23] M. S. Siddiqui, C. Coppola, G. Solak, and L. Jamone, “Grasp stability prediction for a dexterous robotic hand combining depth vision and haptic bayesian exploration,” _Frontiers in Robotics and AI_ , 2021.
  * [24] A. Morales, T. Asfour, P. Azad, S. Knoop, and R. Dillmann, “Integrated grasp planning and visual object localization for a humanoid robot with five-fingered hands,” in _IEEE/RSJ Intl. Conf. on Intelligent Robots and Systems_ , 2006.
  * [25] R. Platt, L. Kaelbling, T. Lozano-Perez, and R. Tedrake, “Simultaneous Localization and Grasping as a Belief Space Control Problem,” in _Intl. Symposium on Robotics Research_ , vol. 2, 2011.
  * [26] D. Kappler, J. Bohg, and S. Schaal, “Leveraging big data for grasp planning,” in _IEEE Intl. Conf. on Robotics and Automation_ , 2015.
  * [27] J. Varley, J. Weisz, J. Weiss, and P. Allen, “Generating multi-fingered robotic grasps via deep learning,” in _IEEE/RSJ Intl. Conf. on Intelligent Robots and Systems_ , 2015.
  * [28] A. Saxena, L. L. S. Wong, and A. Y. Ng, “Learning grasp strategies with partial shape information,” in _AAAI National Conf. on Artificial Intelligence_ , 2008.
  * [29] M. S. Kopicki, D. Belter, and J. L. Wyatt, “Learning better generative models for dexterous, single-view grasping of novel objects,” _Intl. Journal of Robotics Research_ , vol. 38, no. 10-11, 2019.
  * [30] H. Merzić, M. Bogdanović, D. Kappler, L. Righetti, and J. Bohg, “Leveraging contact forces for learning to grasp,” in _IEEE Intl. Conf. on Robotics and Automation_ , 2019.
  * [31] M. Veres, M. Moussa, and G. W. Taylor, “Modeling grasp motor imagery through deep conditional generative models,” _IEEE Robotics and Automation Letters_ , vol. 2, no. 2, 2017.
  * [32] L. Shao, F. Ferreira, M. Jorda, V. Nambiar, J. Luo, E. Solowjow, J. A. Ojea, O. Khatib, and J. Bohg, “Unigrasp: Learning a unified model to grasp with multifingered robotic hands,” _IEEE Robotics and Automation Letters_ , vol. 5, no. 2, 2020.
  * [33] M. Sundermeyer, A. Mousavian, R. Triebel, and D. Fox, “Contact-graspnet: Efficient 6-dof grasp generation in cluttered scenes,” in _IEEE Intl. Conf. on Robotics and Automation_ , 2021.
  * [34] B. Wu, I. Akinola, and P. K. Allen, “Pixel-attentive policy gradient for multi-fingered grasping in cluttered scenes,” in _IEEE/RSJ Intl. Conf. on Intelligent Robots and Systems_ , 2019.
  * [35] J. Lundell, F. Verdoja, and V. Kyrki, “Ddgc: Generative deep dexterous grasping in clutter,” _IEEE Robotics and Automation Letters_ , vol. 6, no. 4, 2021.
  * [36] A. Mousavian, C. Eppner, and D. Fox, “6-dof graspnet: Variational grasp generation for object manipulation,” in _Intl. Conf. on Computer Vision_ , 2019.
  * [37] H. Duan, P. Wang, Y. Huang, G. Xu, W. Wei, and X. Shen, “Robotics dexterous grasping: The methods based on point cloud and deep learning,” _Frontiers in Neurorobotics_ , vol. 15, 2021.
  * [38] R. Calandra, A. Owens, D. Jayaraman, J. Lin, W. Yuan, J. Malik, E. H. Adelson, and S. Levine, “More than a feeling: Learning to grasp and regrasp using vision and touch,” _IEEE Robotics and Automation Letters_ , vol. 3, no. 4, 2018.
  * [39] B. Wu, I. Akinola, J. Varley, and P. K. Allen, “MAT: Multi-Fingered Adaptive Tactile Grasping via Deep Reinforcement Learning,” in _Conference on Robot Learning_ , 2019.
  * [40] H. Dang and P. K. Allen, “Stable grasping under pose uncertainty using tactile feedback,” _Autonomous Robots_ , vol. 36, 2014.
  * [41] C. de Farias, N. Marturi, R. Stolkin, and Y. Bekiroglu, “Simultaneous tactile exploration and grasp refinement for unknown objects,” _IEEE Robotics and Automation Letters_ , vol. 6, no. 2, pp. 3349–3356, 2021.
  * [42] R. H. Byrd, P. Lu, J. Nocedal, and C. Zhu, “A limited memory algorithm for bound constrained optimization,” _SIAM Journal on Scientific Computing_ , vol. 16, no. 5, pp. 1190–1208, 1995.
  * [43] I. Mordatch, Z. Popović, and E. Todorov, “Contact-invariant optimization for hand manipulation,” in _SIGGRAPH_ , 2012.
  * [44] T. Kunz and M. Stilman, “Time-optimal trajectory generation for path following with bounded acceleration and velocity,” in _Robotics: Science and Systems_ , 2012.
  * [45] N. Wettels, J. A. Fishel, and G. E. Loeb, “Multimodal tactile sensor,” in _The Human Hand as an Inspiration for Robot Hand Development_. Springer, 2014, pp. 405–429.
  * [46] E. Gilbert, D. Johnson, and S. Keerthi, “A fast procedure for computing the distance between complex objects in three-dimensional space,” _IEEE Journal on Robotics and Automation_ , vol. 4, no. 2, 1988.
  * [47] W. E. Lorensen and H. E. Cline, “Marching cubes: A high resolution 3d surface construction algorithm,” _SIGGRAPH Comput. Graph._ , vol. 21, no. 4, pp. 163––169, Aug 1987.
  * [48] Dawson-Haggerty et al., “trimesh.” [Online]. Available: https://trimsh.org/
  * [49] B. Calli, A. Singh, J. Bruce, A. Walsman, K. Konolige, S. Srinivasa, P. Abbeel, and A. M. Dollar, “Yale-cmu-berkeley dataset for robotic manipulation research,” _Intl. Journal of Robotics Research_ , vol. 36, no. 3, pp. 261–268, 2017.