[table]capposition=top capbtabboxtable[][]

# LG-VQ: Language-Guided Codebook Learning

Guotao Liang1,2, Baoquan Zhang1, Yaowei Wang2, Xutao Li1, Yunming Ye1
Huaibin Wang1, Chuyao Luo1, Kola Ye3, linfeng Luo3
1Harbin Institute of Technology, Shenzhen
2Peng Cheng Laboratory, 3SiFar Company
{lianggt<EMAIL_ADDRESS>
{zhangbaoquan<EMAIL_ADDRESS>
{lixutao<EMAIL_ADDRESS>
{luochuyao.dalian, kolaygm<EMAIL_ADDRESS>
###### Abstract

Vector quantization (VQ) is a key technique in high-resolution and high-
fidelity image synthesis, which aims to learn a codebook to encode an image
with a sequence of discrete codes and then generate an image in an auto-
regression manner. Although existing methods have shown superior performance,
most methods prefer to learn a single-modal codebook (_e.g._ , image),
resulting in suboptimal performance when the codebook is applied to multi-
modal downstream tasks (_e.g._ , text-to-image, image captioning) due to the
existence of modal gaps. In this paper, we propose a novel language-guided
codebook learning framework, called LG-VQ, which aims to learn a codebook that
can be aligned with the text to improve the performance of multi-modal
downstream tasks. Specifically, we first introduce pre-trained text semantics
as prior knowledge, then design two novel alignment modules (_i.e._ , Semantic
Alignment Module, and Relationship Alignment Module) to transfer such prior
knowledge into codes for achieving codebook text alignment. In particular, our
LG-VQ method is model-agnostic, which can be easily integrated into existing
VQ models. Experimental results show that our method achieves superior
performance on reconstruction and various multi-modal downstream tasks.

##  1 Introduction

In recent years, with the growing development of various multi-modal task
scenarios rombach2022high ; Ren2023Crossing ; Blattmann2023Align , unified
modeling of visuals and language has sparked considerable interest. Vector
Quantization (VQ)-based image modeling technique, exemplified by VQ-VAE
van2017neural  and VQ-GAN esser2021taming , has emerged as a pivotal approach
in the realm of unified modeling. The VQ methodology van2017neural  typically
follows a two-stage generation paradigm. In the initial stage, a trainable
discrete codebook is employed to quantize continuous image features into a
discrete token sequence to finish the reconstruction task. Subsequently, the
codebook is utilized for various downstream tasks by generative models
van2016conditional ; rombach2022high .

Learning a robust codebook during the initial stage is crucial for optimizing
performance in downstream tasks. At present, lots of VQ methods have been
proposed to achieve robust code representation huang2023towards ; huang2023not
; Dong23PeCo ; gu2022rethinking . For instance, VQ-GAN esser2021taming
introduces an adversarial training loss to learn a perceptually rich codebook.
Some other works consider improving the codebook representation from the
perspective of addressing the problem of codebook collapse Zheng2023CVQ ;
Zhang2023Regularized .

Although existing methods have shown superior performance, most methods only
focus on learning a single-modal codebook contains more low-level information
(_e.g._ , image’s pixel, edge, and texture), resulting in suboptimal
performance when the codebook is applied to multi-modal downstream tasks
(_e.g._ , text-to-image rombach2022high , image captioning Ren2023Crossing ,
VQA lu2016vqa ). That is because the codebook lacks high-level semantics and
the existence of modal gaps.

To address the above issue, we propose a novel codebook learning method
(_i.e._ , multi-modal codebook learning), called Language-Guided VQ (LG-VQ).
The novelty lies in utilizing pre-trained text semantics as supervised
information to guide the codebook to learn abundant multi-modal knowledge.

Figure 1: To answer the question, one not only needs to identify “women” and
“racket” but also understand the semantic relationship between them
(“holding”).

Specifically, we first employ a cross-modal pre-trained model (_i.e._ , CLIP
Alec2021clip ) to encode text semantics. Then, we propose two novel semantic
supervision modules to transfer the text semantics into codebook, _i.e._ ,
Semantic Alignment Module, and Relationship Alignment Module. Within the
semantic alignment module, we enhance the consistency between the semantic
representations of the codebook and text through global semantic alignment and
masked text prediction. On the other hand, simply aligning the text and
codebook in the holistic semantic space cannot satisfy more complex reasoning
tasks like image captioning and VQA. Inspired by some VQA techniques
mikolov2013word2vec ; suhail2021scenegraph ; lu2016vqa , the semantic
relationships between words play a very important role in various tasks of
natural language processing (See Fig. 1). Based on this fact, we further
propose to transfer the semantic relationships between words into codes to
achieve better alignment between the codes and words. Such a text-aligned
codebook helps alleviate modal gaps and improve codebook performance on cross-
modal tasks.

The contributions of this work are summarized as follows:

  * •

We point out the limitations of existing methods in learning an expressive
codebook since they learn a single-modal codebook. We propose a novel multi-
modal codebook learning method, named LG-VQ, which can enable the codebook to
effectively retain fine-grained reconstruction information while aligning with
the text.

  * •

Resorting to pre-trained text semantics, we propose two novel semantic
supervision modules, _i.e._ , Semantic Alignment Module and Relationship
Alignment Module, effectively learn text-aligned codebook. The advantage of
such alignment modules is the abundant context and relationship semantics
contained in pre-trained text can be sufficiently leveraged for enhancing
multi-modal codebook learning.

  * •

We conduct comprehensive experiments on four public datasets, which shows that
our LG-VQ method outperforms various state-of-the-art models on reconstruction
and various cross-modal tasks (_e.g._ , text-to-image, image captioning, VQA).

##  2 Related Works

###  2.1 Vector Quantization for Image Generation

Vector quantization (VQ) is designed to learn a codebook, which aims to encode
continuous image features into a discrete sequence. Then, the learned codebook
can be utilized for various downstream tasks. Oord et al. van2017neural  first
propose a novel VQ method called VQ-VAE. This method innovatively replaces the
prior distribution of Variational Autoencoder (VAE) with a discrete
deterministic distribution (_i.e._ , a codebook). To further improve the
performance of VQ, various models are proposed to learn a more expressive
codebook esser2021taming ; yu2021vitvq ; chang2022maskgit ;
lee2022autoregressive ; Dong23PeCo ; huang2023towards ; huang2023not ;
Lin23Catch . For example, VQ-GAN esser2021taming  addresses the issue of image
blur generated by VQ-VAE through the introduction of an adversarial training
loss. However, the above methods do not tackle the codebook collapse issue. To
address the issue, many novel methods are proposed from the perspective of
regularization RazaviOV2019Generating , codebook update Zheng2023CVQ .
Recently, inspired by the large language models (LLMs), instead of mapping
images to the visual code tokens, some works attempt to map the images to the
word tokens of LLMs by viewing images as “foreign languages” liu2023lqae ;
yu2023spae ; zhu2024beyond . However, because of the inherent differences
between vision and language, these works have difficulty assigning correct
semantic words to images.

Compared with the aforementioned methods, our approach focuses more on multi-
modal alignment in feature space (_i.e._ , learning a text-aligned codebook).
We use pre-trained text semantics to supervise the codebook learning. The
advantage is that the rich semantic information from the text can be fully
exploited for more robust codebook learning so that the codebook can not only
retain more reconstruction information but also be able to understand and
match text. More importantly, our method is model-agnostic, which can be
easily integrated into existing VQ models.

###  2.2 Vision-Language Representation Learning

Vision-language Pre-training (VLP) aims to learn multi-modal representations
from large-scale image-text pairs that can improve vision-language downstream
tasks, for example, VQAantol2015vqa . Early methods such as LXMERT
tan2019lxmert , UNITER chen2020uniter  employ pre-trained object detectors to
extract image region features, and fuse image features with text by a cross-
modal encoder to achieve the vision-language representation learning. Although
these methods achieve superior performance on downstream tasks, they require
high-resolution input images and pre-trained object detectors. To remove the
object detectors, a large number of researchers focus on learning two separate
representations for image and text Alec2021clip ; jia2021scaling ; Li2021Align
. For instance, CLIP Alec2021clip  learns a robust representation for each
image and text using contrastive learning based on large-scale image-text pair
data.

In this paper, we propose to employ pre-trained text semantics as supervised
information to guide codebook learning. Its advantage is that abundant multi-
modal knowledge contained in text can be fully leveraged for robust codebook
learning. Additionally, we design a novel relationship alignment module to
inject semantic relationships between words into codes.

Figure 2: The overall architecture of the proposed LG-VQ method. The right
part of the figure is the basic VQ-VAE module, left is our language-guided
module, which consists of three losses: global semantic alignment
($\mathcal{L}_{gsa}$), masked text prediction ($\mathcal{L}_{mtp}$), and
relationship alignment supervision ($\mathcal{L}_{ras}$). Here, pre-trained
text information guides discrete code tokens of an image to learning rich
semantic knowledge based on three losses.

##  3 Methodology

###  3.1 Preliminaries: VQ-VAE

VQ-VAE van2017neural , as a pioneering work on the VQ research domain, aims to
learn a discrete codebook to encode images into discrete token sequences
through an Encoder-Decoder framework. As illustrated in Fig. 2 right, the VQ-
VAE consists of a visual encoder $E_{\theta_{e}}(\cdot)$ with parameter
$\theta_{e}$, a token decoder $D_{\theta_{d}}(\cdot)$ with parameter
$\theta_{d}$, a quantizer $Q(\cdot)$, and a codebook is defined as
$\mathcal{Z}=\\{e_{k}\\}_{k=1}^{K}$ that consists of learnable $K$ entries
$e_{k}\in\mathbb{R}^{d_{z}}$ with dimension $d_{z}$. Given an input image
$x\in\mathbb{R}^{H\times W\times C}$, where $H$, $W$, and $C$ represent the
height, width, and channel of the image respectively. The visual encoder
$E_{\theta_{e}}(\cdot)$ learns to convert the original image into grid
features
$\hat{Z}=E_{\theta_{e}}(x)\in\mathbb{R}^{\frac{H}{f}\times\frac{W}{f}\times
d_{z}}$ and $f$ is the down-sampling factor. The quantizer $Q(\cdot)$ looks up
the nearest neighbor in the codebook for each grid representation
$\hat{z}_{i}\in\mathbb{R}^{d_{z}}$ in $\hat{Z}_{i}$ using the following
equation:

$z_{i}=Q(\hat{z}_{i})=\mathop{argmin}\limits_{e_{k}\in\mathcal{Z}}\|\hat{z}_{i}-e_{k}\|.$
(1)

The token decoder $D_{\theta_{d}}(\cdot)$ is used to reconstruct the original
image by $\widetilde{x}=D_{\theta_{d}}(Z)$, where $Z$ is discrete code tokens
of whole image obtained by Eq. 1. During training, the visual encoder
$E_{\theta_{e}}(\cdot)$, codebook $\mathcal{Z}$, and token decoder
$D_{\theta_{d}}(\cdot)$ are jointly optimized by minimizing the following
objective:

$\mathcal{L}_{vq}=\|x-\widetilde{x}\|_{2}^{2}+\|sg[E_{\theta_{e}}(x)]-Z\|_{2}^{2}+\omega\|E_{\theta_{e}}(x)-sg[Z]\|_{2}^{2},$
(2)

where, the first term is reconstruction loss, which measures the difference
between the original image $x$ and the reconstructed image $\widetilde{x}$.
$sg[\cdot]$ represents the stop-gradient operator, and the second term is
codebook loss, which encourages the codebook to be close grid features. The
third term is the “commitment loss” van2017neural , where $\omega$ serves as a
hyper-parameter. However, existing VQ-based methods mainly focus on the
learning of single-modal codebook, thereby limiting their applicability to
multi-modal downstream tasks.

###  3.2 Proposed Method: LG-VQ

Existing works attempt to improve codebook reconstruction capabilities to
obtain better performance on downstream tasks. However, ignoring modal
differences results in suboptimal performance when the codebook is applied to
cross-modal tasks. To address this issue, we propose to utilize the pre-
trained text semantics as supervised information to learn a text-aligned
codebook. Its advantage is abundant semantic information from text can be
fully exploited for more robust codebook learning to improve the performance
of reconstruction and cross-modal tasks. The comprehensive architecture of the
proposed LG-VQ method is illustrated in Fig. 2 left. It consists of two
supervision modules: Semantic Alignment Module (_i.e._ , $\mathcal{L}_{gsa}$
and $\mathcal{L}_{mtp}$), and Relationship Alignment Module (_i.e._ ,
$\mathcal{L}_{ras}$). The first module encourages global semantic consistency
between the codebook and text. The second module aims to transfer the rich
semantic relationship between words into codes. Next, we introduce these two
modules in detail.

####  3.2.1 Semantic Alignment Module

Considering that paired image and text data have consistent semantic
information and the missing information of masked data can be completed from
the other modality, we propose global semantic alignment, which aims to
enhance the consistency of global semantics between text and visual codes, and
masked text prediction, which uses visual codes to restore the masked words.
Next, we discuss how to align text and codebook in the semantic space.
Text Information Encoder: Instead of jointly training text and codebook from
scratch, we employ a pre-trained cross-modal model CLIP Alec2021clip  to
encode text information. Its advantage is that such text information already
has good cross-modal semantic knowledge and is beneficial for codebook
learning. Specifically, for a given text description of an image
$t=\\{w_{SOT},w_{1},w_{2},\cdots,w_{n-2},w_{EOT}\\}$, where $w_{i}$ denotes
the $i$-th word, $w_{SOT}$ and $w_{EOT}$ represent the $[start]$ token and
$[end]$ token, respectively, and $n$ is text sequence length. We use the text
encoder of a pre-trained CLIP model to obtain whole sequence embedding
$T\in\mathbb{R}^{n\times d_{t}}$:

$T=\\{e_{SOT},e_{w_{1}},e_{w_{2}},\cdots,e_{w_{n}},e_{EOT}\\}=\mathrm{CLIP}(t).$
(3)

Similar to CLIP, we use the $e_{EOT}$ to represent the global context feature
of the sequence.
Global Semantic Alignment aims to align text and image visual codes in the
global semantic space. For getting the global representation of visual codes,
we employ a vision transformer (ViT) $f_{\theta_{vt}}$ dosovitskiy2020vit  to
encode the discrete codes of image. Specifically, given an image, we firstly
obtain the discrete codes of image $Z$ by Eq. 1. Then, we introduce a
learnable global token $[CLS]$ at the beginning to form a token sequence
$Z_{c}$, where global token $[CLS]$ is employed to capture the image’s global
context information. We feed the sequence into $f_{\theta_{vt}}$ to get a new
visual code representation, that is:

$Z_{vt}=\\{e_{CLS},e_{1},e_{2},\cdots,e_{\frac{H}{f}\times\frac{W}{f}}\\}=f_{\theta_{vt}}(Z_{c}).$
(4)

Finally, we employ InfoNCE Van2018infonce , which maximizes the similarity
between visual and text in the global representation, as our learning
objective, where $\mathcal{B}$ is the batch size, $s(\cdot,\cdot)$ is cosine
similarity:

$\mathcal{L}_{gsa}=-\sum_{i\in\mathcal{B}}\log\frac{\exp(s(e_{CLS}^{i},e_{EOT}^{i}))}{\sum_{j\in\mathcal{B}}\exp(s(e_{CLS}^{j},e_{EOT}^{j}))}.$
(5)

Masked Text Prediction: To further enhance the semantic alignment, we propose
to use discrete visual codes to reconstruct the masked words from a more fine-
grained perspective, refer to Fig. 2 left. Formally, for a given fixed-length
text sequence of $n-2$, we first randomly sample the masking ratio $r$ from a
truncated Gaussian distribution Li2023MAGE . Subsequently, we randomly mask
out $r\cdot(n-2)$ words and replace them with learnable $[mask_{i}]$ tokens
based on their positions $i$. Next, a self-attention module
Vaswani2017attention  is employed to learn adaptive masked word embeddings
based on unmasked words. The resulting adaptive masked sequence is denoted as
$T^{msk}=\\{e_{SOT},m_{1},e_{w_{2}},m_{3},\cdots,e_{EOT}\\}$, where $m_{i}$ is
the mask token embedding at the $i$-th position in the sequence. Following
this, a cross attention decoder $f_{\theta_{m}}(\cdot,\cdot)$ is employed to
predict the masked word tokens given the discrete visual codes $Z_{vt}$
obtained by Eq. 4. Finally, we add a cross-entropy loss $H(\cdot,\cdot)$
between the ground-truth word tokens and the output of the decoder. Let
$y_{msk}$ denote a one-hot vocabulary distribution where the ground-truth word
token has a probability of 1, $f_{\theta_{m}}(Z_{vt},T^{msk})$ denote the
predicted probability of model for masked word tokens. That is:

$\mathcal{L}_{mtp}=-\mathbb{E}_{(Z_{vt},T^{msk})\sim\mathcal{B}}H(y_{msk},f_{\theta_{m}}(Z_{vt},T^{msk})).$
(6)

####  3.2.2 Relationship Alignment Module

While the two aforementioned loss functions for achieving good alignment at
holistic semantic space have demonstrated initial promise, they cannot satisfy
more complex reasoning tasks like image captioning and VQA. Inspired by some
VQA techniques mikolov2013word2vec ; suhail2021scenegraph ; lu2016vqa ;
antol2015vqa , the semantic relationships between pre-trained words play a
very important role in complex text reasoning tasks. For instance, as shown in
Fig 1, to answer question (“What is this woman holding?”), one needs to fully
understand the visual objects “women”, “racket”, and semantic relationship
between them (“holding”). Based on the above fact, we propose to transfer the
semantic relationship between words into codes. Such semantic relationships
enable the model to better understand the image for addressing complex
reasoning tasks.

Figure 3: Illustration of relationship alignment module, we use $Z_{vt}$ to
align with two words, then inject the semantic relationship of two words into
$Z$ codes.

But unfortunately, there is an issue there is no alignment between words and
codes. Thanks for the above two losses that have provided semantic alignment
of text and visual codes. To achieve the above idea, as shown in Fig. 3, we
first use $Z_{vt}$ to align with words. Then, we inject semantic relationships
between words into the initial codebook $Z$, instead of the $Z_{vt}$. Its
advantage is it can prevent codes from collapsing into a single point for
learning more diverse representations by relationship limiting. Then, $Z_{vt}$
primarily serves the purpose of aligning words and codes, but it is a crucial
step for subsequent processes. Specifically, given any two words of a
sentence, we use pre-trained word embedding Alec2021clip  to encode words,
$e_{w_{i}}$ and $e_{w_{j}}$. We employ cosine similarity to find the index of
the code from $Z_{vt}$ that is most similar to the word. Then, one can get
code embedding from $Z$ based on the index:

$e_{z_{i}}=Z[\mathop{argmax}\limits_{e_{z}\in
Z_{vt[1:]}}s(e_{w_{i}},e_{z}),:],\qquad
e_{z_{j}}=Z[\mathop{argmax}\limits_{e_{z}\in
Z_{vt[1:]}}s(e_{w_{j}},e_{z}),:].$ (7)

Next, we consider cosine similarity as a measure of semantic relationships
between words and leverage it to establish corresponding relationships between
codes achieving semantic relationship transfer. Finally, we utilize the
following loss function as learning objective:

$\mathcal{L}_{ras}=\sum_{(w_{i},w_{j})\in
t}(s(e_{w_{i}},e_{w_{j}})-s(e_{z_{i}},e_{z_{j}}))^{2}.$ (8)

####  3.2.3 Training Objective

We use three hyperparameters (_i.e._ , $\alpha$, $\beta$, and $\gamma$) to
control three losses, respectively. Finally, the overall objective function
is:

$\mathcal{L}=\mathcal{L}_{vq}+\alpha\mathcal{L}_{gsa}+\beta\mathcal{L}_{mtp}+\gamma\mathcal{L}_{ras}.$
(9)

##  4 Experiments

###  4.1 Experimental Settings

Evaluation Metrics. As our method is model-agnostic, we choose recent models,
including VQ-VAE van2017neural , VQ-GAN esser2021taming , and CVQ Zheng2023CVQ
as our backbone network. Following existing works Zhang2023Regularized ;
Zheng2023CVQ , we evaluate the reconstruction image quality on two evaluation
metrics, _i.e._ , Fréchet Inception Distance (FID) heusel2017fid  which
evaluates the perceptual similarity of reconstructed images and original
images, and Peak Signal-to-noise Ratio (PSNR) fardo2016psnr  is employed to
measure the pixel-level similarity between the reconstructed and original
images.
Dataset. We evaluate our method on four public datasets, including TextCaps
sidorov2019textcaps , CelebA-HQ liu2015cele , CUB-200 wah2011cub , and MS-COCO
lin2014coco .
Implementation Details. Following VQ-GAN esser2021taming , all images are
reshaped 256 $\times$ 256 for reconstruction and generation. Down-sampling
factor $f$ is set to 16. The codebook size $K$ is 1024. The batch size is 8.
In our experiments, we maintain consistent parameter settings between our
method LG-VQ and the chosen backbone networks (_i.e._ , VQ-VAE van2017neural ,
VQ-GAN esser2021taming , and CVQ Zheng2023CVQ ) for a fair comparison. For
each image, we randomly select a text from multi-text for training.

Table 1:  Results of image reconstruction on TextCaps, CelebA-HQ, CUB-200, and MS-COCO. “VQ-VAE+LG” denotes considering our method LG-VQ based on VQ-VAE.  Models | TextCaps | CelebA-HQ | CUB-200 | MS-COCO
---|---|---|---|---
FID$\downarrow$ |  PSNR$\uparrow$ |  FID$\downarrow$ |  PSNR$\uparrow$ |  FID$\downarrow$ |  PSNR$\uparrow$ |  FID$\downarrow$ |  PSNR$\uparrow$
VQ-VAE | 82.31 | 21.96 | 41.45 | 25.57 | 54.92 | 24.38 | 86.21 | 23.55
VQ-VAE+LG | 81.93 | 21.95 | 40.53 | 25.04 | 36.55 | 25.60 | 79.54 | 23.40
VQ-GAN | 24.08 | 19.64 | 5.66 | 24.10 | 3.63 | 22.19 | 14.45 | 20.21
VQ-GAN+LG | 20.35 | 19.92 | 5.34 | 23.75 | 3.08 | 22.47 | 10.72 | 20.50
CVQ | 16.35 | 20.24 | 5.19 | 23.15 | 3.61 | 22.29 | 9.94 | 20.48
CVQ+LG | 15.51 | 20.21 | 4.90 | 24.48 | 3.33 | 22.47 | 9.69 | 20.71

###  4.2 Discussion of Results

Table 1 illustrates the image reconstruction performance of our model compared
to the backbone model on multiple datasets. It can be observed that our method
LG-VQ outperforms all compared methods on most evaluations, which suggests
that our method is extremely effective and has strong generality. Compared
with FID, our PSNR improvement is marginal, this is reasonable in the VQ
research domain, which widely exists in previous VQ methods Lin23Catch ;
Zhang2023Regularized ; Zheng2023CVQ . The main reason is that PSNR only
measures the pixel-level similarity of the images, while FID can effectively
measure the diversity and semantic similarity of image generation. Compared
with backbone models, the key difference lies in that our method introduces
well pre-trained text semantics, which is beneficial to learning a more
expressive codebook. This shows the effectiveness of our method. We also
provide a qualitative comparison of the image reconstruction performance of
different methods, please refer to Appendix A.2 Fig. 10.
Table 2: Ablation study of our three loss functions on TextCaps and CUB-200.
Setting TextCaps CUB-200 FID$\downarrow$ FID$\downarrow$ (i) Baseline(VQ-GAN)
24.08 3.63 (ii) \+ $\mathcal{L}_{gsa}$ 23.01 3.39 (iii) \+ $\mathcal{L}_{mtp}$
21.54 3.49 (iv) \+ $\mathcal{L}_{mtp}$ \+ $\mathcal{L}_{ras}$ 20.77 3.32 (v)
\+ $\mathcal{L}_{mtp}$ \+ $\mathcal{L}_{gsa}$ 20.46 3.34 (vi) \+
$\mathcal{L}_{mtp}$ \+ $\mathcal{L}_{gsa}$ \+ $\mathcal{L}_{ras}$ 20.35 3.08
Table 3: Results (Recall@1) of masked word prediction on CelebA-HQ and
CUB-200. “Mask-1” denotes that text is randomly masked one word.  Dataset
Recall@1 CelebA-HQ Mask-1 99.55 Mask-3 99.24 CUB-200 Mask-1 83.65 Mask-3 80.17

###  4.3 Ablation Study

Are our three loss functions both effective? In Table 4.2, we conduct an
ablation study to show the effectiveness of the proposed three loss functions.
Specifically, the VQ-GAN serves as the baseline model (_i.e._ , without
introducing any loss). We do not conduct a separate experiment on
$\mathcal{L}_{ras}$ because this module requires code and words to be well
aligned. Based on the results from (i) $\sim$ (vi), we draw several key
conclusions: Firstly, each loss function plays a crucial role in improving the
performance of image reconstruction. Secondly, the performance of (iii)
outperforms (ii) by a large margin on TextCaps. This is reasonable because
TextCaps’s texts are richer and more diverse than CUB-200, it can provide more
knowledge for more fine-grained alignment between codes and text, which is
useful for the learning of a more robust codebook. Thirdly, analyzing the
results of (iii) and (iv), injecting word-level semantic relationships into
codes is beneficial, which confirms our motivation. Furthermore, the
performance of (v) outperforms (i), which is reasonable because the abundant
semantic knowledge from pre-trained text can be fully exploited for learning
more robust codebook representation. This supports the motivation of learning
a multi-modal codebook (_i.e._ , aligned with text). Finally, comparing the
results of (vi) with (i)$\sim$(v), fully considering all losses achieves the
best performance, indicating the effectiveness of our method.
Can our global semantic supervision align vision and language? In Fig. 4, we
provide several image-to-text retrieval cases on CelebA-HQ and CUB-200
datasets based on VQ-GAN+LG. From the figure, it can be observed that our
method can accurately retrieve text very similar to the image content,
achieving the alignment of vision and language. For example, row 2 examples
show that our method can precisely understand some key attributes of images
(_e.g._ , “gray hair”, “necktie”, “big nose” and “chubby”) and retrieve
similar text. This suggests that the codes learned through our method have
obtained good alignment with the text, which verifies the effectiveness of our
method. Moreover, such alignment is beneficial for learning robust code
representations and enhancing performance in multi-modal downstream tasks.
Figure 4: Examples of the top-1 most similar text selected on image-to-text
retrieval task. Figure 5: Examples of the top-1 word predicted on masked word
prediction task.
Can our codebook accurately predict masked words? To answer this question, we
conduct a word prediction task on test data based on VQ-GAN+LG by randomly
masking one word or three words of text, as shown in Table 4.2. We use
Recall@1 as the evaluation metric Li2021Align . From the table, our method
demonstrates accurate predictions of masked words, confirming the
effectiveness of our approach. Fine-grained word prediction can help the
codebook better understand the text semantics, which is crucial for improving
the performance of the downstream task. Additionally, several examples in Fig.
5 demonstrate our method’s ability that accurately predict subject words
(_e.g._ , wings, eyes) and verbs (_e.g._ , has, is, and smiling), further
affirming its strong multi-modal understanding capabilities.

Table 4: Results of similarity evaluation between codes and words on CUB-200 all test data. Method | VQ-GAN | VQ-GAN+LG
---|---|---
MSE$\downarrow$ | 0.6374 | 0.0351

Can our codebook learn the word semantic relationships? In Fig. 6, we
visualize the cosine similarity between words and the cosine similarity
between codes aligned with the words for a certain sample based on VQ-GAN+LG.
From the figure, we can see our codes can learn consistent relationships with
word semantics compared with VQ-GAN. For example, the similarity “code 33” vs
“code 232” (0.46) resembles “wings” vs “chest” (0.49). In addition, we provide
a quantitative similarity evaluation between codes and words in Table 4. From
the results, we can find that our codes indeed achieve consistent semantic
relationships with words.
Is our method effectively learning more diverse code representation? Following
Zhang2023Regularized , we directly feed each codebook embedding $e_{k}$ (size:
$1\times 1\times 256$) into the decoder $D_{\theta_{d}}(\cdot)$ to generate
codebook image (size: $16\times 16\times 3$). Then, we concatenate all
codebook images to form a big image with $32\times 32$ patches. Finally, we
visualize the result of VQ-GAN and our LG-VQ on TextCaps and MS-COCO as shown
in Fig. 7. This visualization suggests that our method enables the model to
learn more diverse code representations and improve codebook usage.

(a) word similarity

(b) LG-VQ code similarity

(c) VQ-GAN code similarity

Figure 6: Visualization of words similarity and image codes similarity aligned
with the word. We extract some representative words from the text as a
demonstration.

(a) The usage of codebook on VQ-GAN is 18.62% and VQ-GAN+LG is 43.58% on
TextCaps

(b) The usage of codebook on VQ-GAN is 40.09% and VQ-GAN+LG is 97.89% on MS-
COCO

Figure 7: Visualization of the codebook of VQ-GAN and LG-VQ on TextCaps and
MS-COCO.

###  4.4 Application

####  4.4.1 Image Generation

Following esser2021taming ; rombach2022high ; gu2022vqdiffusion , we conduct
two image generation downstream tasks (_i.e._ , text-to-image and semantic
image synthesis) to fully validate the effectiveness of the learned codebook
on CelebA-HQ.
Quantitative Evaluation. In Table 4.4.2, we compare our LG-VQ with the
backbone methods on CelebA-HQ datasets for text-to-image and semantic image
synthesis tasks. The results yield several key observations. Firstly, our LG-
VQ method outperforms baseline methods in both tasks, affirming our method’s
effectiveness. Secondly, the performance improvement of our method on the
text-to-image task exceeds that on the semantic image synthesis task, which is
reasonable due to the incorporation of pre-trained text knowledge enabling a
comprehensive understanding of the text. This further confirms the advantage
of introducing pre-trained text.
Qualitative Evaluation. We provide some synthesis examples comparing the
results of our LG-VQ with baseline methods in Fig. 8, showing the performance
in the text-to-image task. From the figure, we can see our method not only
comprehensively understands the given text conditions but also excels in
generating realistic images compared with baseline methods. For instance, our
method can capture the “glasses”, “man”, “long black hair”, and “no beard” key
attributions.

####  4.4.2 Visual Text Reasoning

Follow mokady2021clipcap ; ding2021cogview , we use the learned codebook to
conduct two visual text reasoning tasks: 1) image captioning on CUB-200; and
2) visual question answering (VQA) on COCO-QA ren2015cocoQA . For the
experimental setting, please refer to Appendix A.1.
Image captioning. Following mokady2021clipcap , we conduct the image
captioning task on the CUB-200 dataset. We compare a recent work V2L Tokenizer
zhu2024beyond  accepted by CVPR2024, which maps images to the word tokens of
LLMs. We select VQ-GAN as our backbone network. The results are shown in Table
7. For the results, we can see that our LG-VQ method outperforms the
performance of VQ-GAN. This is reasonable because the pre-trained text
provides rich context and relationship semantics for codebook learning, which
verifies our motivation for learning a text-aligned codebook to improve the
performance of the codebook on cross-modal tasks. On the other hand, the V2L
Tokenizer cannot achieve very good performance because it is difficult to
assign correct semantic language tokens to images. Compared with the V2L
Tokenizer, our method utilizes pre-trained text semantics as supervised
information. Its advantage is can make the codebook learn semantic information
consistent with the text (_i.e._ , learning a text-aligned codebook). And, our
method is model-agnostic, which can be easily integrated into existing VQ
models.
Visual Question Answering. We select VQ-GAN as the baseline. We conduct the
VQA task on the COCO-QA ren2015cocoQA  using the codebook trained on the MS-
COCO dataset. The results are shown in Table 6. From the results, we can see
that our LG-VQ method significantly improves the performance of VQ-GAN on VQA
task (approximately 8.32%$\uparrow$ on Accuracy). That is reasonable due to we
introduce pre-trained text semantics to enable us to obtain a codebook aligned
with the text, which is helpful for comprehensively understanding the given
text question. This confirms our motivation and the effectiveness of our
method.

Figure 8: Text-to-image synthesis and semantic image synthesis on CelebA-HQ.
Text with background color emphasizes generated details

Table 5: Results (FID$\downarrow$) of text-to-image synthesis and semantic
image synthesis on CelebA-HQ. “w/o” means without, and “w” means with.

Setting | VQ-VAE | VQ-GAN | CVQ
---|---|---|---
w/o LG | w LG | w/o LG | w LG | w/o LG | w LG
text-to-image | 49.51 | 49.36 | 15.29 | 12.61 | 13.23 | 12.33
image synthesis | 48.72 | 48.23 | 11.53 | 11.46 | 11.04 | 11.03

Table 6: Results of (Accuracy and WUPS wu1994wups ) VQA on COCO-QA
ren2015cocoQA  dataset using MS-COCO’s codebook.

Setting | VQA
---|---
Accuracy$\uparrow$ |  WUPS$\uparrow$
VQ-GAN |  37.82 $\pm$ 0.97 |  82.06 $\pm$ 0.54
VQ-GAN+LG |  40.97 $\pm$ 0.13 |  83.56 $\pm$ 0.11

Table 7: Results of image captioning on CUB-200 datasets. Model | Image Captioning |
---|---|---
BLEU4$\uparrow$ |  ROUGE-L$\uparrow$ |  METEOR$\uparrow$ |  CIDEr-D$\uparrow$
VQ-GAN | 1.29 | 33.40 | 24.47 | 93.62
V2L Tokenizer (CVPR24) zhu2024beyond  | 1.59 | 30.65 | 25.76 | 104.14
VQ-GAN+LG | 1.69 | 34.73 | 25.78 | 102.77

##  5 Conclusions

In this paper, we propose a novel codebook learning method, named LG-VQ. LG-VQ
is a model-agnostic method and can easily be integrated into existing VQ
models. In particular, we propose to incorporate pre-trained text semantics
into the codebook by two novel supervision modules, _i.e._ , semantic and
relationship. Quantitative and qualitative experiments demonstrate the strong
generality of our method, showing its ability to improve the performance of
the codebook in cross-modal tasks.
Limitations. In our current paper, we suppose each word aligns with a code,
but it fails to capture some more complex relationships between words and
codes (_e.g._ , one code aligns with multiple words). In the future, we plan
to investigate the relationships between codes and words. Moreover, although
our results show that the performance of VQ in visual text reasoning tasks can
be significantly improved, its results are still far lower than the
performance of image captioning or VQA models.
Broader impact Our paper shows that learning a multi-modal codebook (_i.e._ ,
a text-aligned codebook) can not only significantly improve the performance of
reconstruction but also the performance of the codebook on cross-modal tasks.
The potential impact of our research lies in its influence on future studies,
specifically in the area of unified modeling of multi-modal understanding and
generation. For instance, our work can be extended to interact with LLMs to
improve multi-modal understanding and generation capabilities. In particular,
our model can be used to generate images or text. It may be exploited to
produce some erroneous and unethical information, which needs to be handled
carefully before employing our model in practical applications.

## References

  * [1] Stanislaw Antol and Aishwarya Agrawal.  Vqa: Visual question answering.  In ICCV, pages 2425–2433, 2015.
  * [2] Huiwen Chang and Han Zhang.  Maskgit: Masked generative image transformer.  In CVPR, pages 11315–11325, 2022.
  * [3] Yen-Chun Chen and Linjie Li.  Uniter: Universal image-text representation learning.  In ECCV, pages 104–120. Springer, 2020.
  * [4] Ming Ding and Zhuoyi Yang.  Cogview: Mastering text-to-image generation via transformers.  NeurIPS, 34:19822–19835, 2021.
  * [5] Xiaoyi Dong and Jianmin Bao.  Peco: Perceptual codebook for bert pre-training of vision transformers.  In AAAI, volume 37, pages 552–560, 2023.
  * [6] Alexey Dosovitskiy and Lucas Beyer.  An image is worth 16x16 words: Transformers for image recognition at scale.  ICLR, 2021.
  * [7] Patrick Esser, Robin Rombach, and Bjorn Ommer.  Taming transformers for high-resolution image synthesis.  In CVPR, pages 12873–12883, 2021.
  * [8] Fernando A Fardo and Victor H Conforto.  A formal evaluation of psnr as quality measurement parameter for image segmentation algorithms.  arXiv preprint arXiv:1605.07116, 2016.
  * [9] Shuyang Gu and Dong Chen.  Vector quantized diffusion model for text-to-image synthesis.  In CVPR, pages 10696–10706, 2022.
  * [10] Yuchao Gu and Xintao Wang.  Rethinking the objectives of vector-quantized tokenizers for image synthesis.  arXiv preprint arXiv:2212.03185, 2022.
  * [11] Martin Heusel and Hubert Ramsauer.  Gans trained by a two time-scale update rule converge to a local nash equilibrium.  NeurIPS, 30, 2017.
  * [12] Mengqi Huang and Zhendong Mao.  Not all image regions matter: Masked vector quantization for autoregressive image generation.  In CVPR, pages 2002–2011, 2023.
  * [13] Mengqi Huang and Zhendong Mao.  Towards accurate image coding: Improved autoregressive image generation with dynamic vector quantization.  In CVPR, pages 22596–22605, 2023.
  * [14] Chao Jia and Yinfei Yang.  Scaling up visual and vision-language representation learning with noisy text supervision.  In ICML, pages 4904–4916. PMLR, 2021.
  * [15] Doyup Lee and Chiheon Kim.  Autoregressive image generation using residual quantization.  In CVPR, pages 11523–11532, 2022.
  * [16] Junnan Li and Ramprasaath Selvaraju.  Align before fuse: Vision and language representation learning with momentum distillation.  volume 34, pages 9694–9705, 2021.
  * [17] Tianhong Li and Huiwen Chang.  Mage: Masked generative encoder to unify representation learning and image synthesis.  In CVPR, pages 2142–2152, 2023.
  * [18] Tsung-Yi Lin and Michael Maire.  Microsoft coco: Common objects in context.  In ECCV, pages 740–755. Springer, 2014.
  * [19] Xinmiao Lin and Yikang Li.  Catch missing details: Image reconstruction with frequency augmented variational autoencoder.  In CVPR, pages 1736–1745, 2023.
  * [20] Hao Liu and Wilson and Yan.  Language quantized autoencoders: Towards unsupervised text-image alignment.  arXiv preprint arXiv:2302.00902, 2023.
  * [21] Ziwei Liu and Ping Luo.  Deep learning face attributes in the wild.  In ICCV, pages 3730–3738, 2015.
  * [22] Jiasen Lu and Jianwei Yang.  Hierarchical question-image co-attention for visual question answering.  NeurIPS, 29, 2016.
  * [23] Tomas Mikolov and Kai Chen.  Efficient estimation of word representations in vector space.  arXiv preprint arXiv:1301.3781, 2013.
  * [24] Ron Mokady and Amir Hertz.  Clipcap: Clip prefix for image captioning.  arXiv preprint arXiv:2111.09734, 2021.
  * [25] Aaron van den Oord and Yazhe Li.  Representation learning with contrastive predictive coding.  arXiv preprint arXiv:1807.03748, 2018.
  * [26] Alec Radford and Jong Wook Kim.  Learning transferable visual models from natural language supervision.  In ICML, pages 8748–8763. PMLR, 2021.
  * [27] Ali Razavi and Aaron Van den Oord.  Generating diverse high-fidelity images with vq-vae-2.  NeurIPS, 32, 2019.
  * [28] Mengye Ren and Ryan Kiros.  Exploring models and data for image question answering.  NeurIPS, 28, 2015.
  * [29] Yuchen Ren and Zhendong Mao.  Crossing the gap: Domain generalization for image captioning.  In CVPR, pages 2871–2880, 2023.
  * [30] Robin Rombach and Andreas Blattmann.  High-resolution image synthesis with latent diffusion models.  In CVPR, pages 10684–10695, 2022.
  * [31] Robin Rombach and Andreas Blattmann.  High-resolution image synthesis with latent diffusion models.  In CVPR, pages 10684–10695, 2022.
  * [32] Oleksii Sidorov and Ronghang Hu.  Textcaps: a dataset for image captioningwith reading comprehension.  2020\.
  * [33] Mohammed Suhail and Abhay Mittal.  Energy-based learning for scene graph generation.  In CVPR, pages 13936–13945, 2021.
  * [34] Hao Tan and Mohit Bansal.  Lxmert: Learning cross-modality encoder representations from transformers.  arXiv preprint arXiv:1908.07490, 2019.
  * [35] Aaron Van den Oord and Nal Kalchbrenner.  Conditional image generation with pixelcnn decoders.  NeurIPS, 29, 2016.
  * [36] Aaron Van Den Oord, Oriol Vinyals, et al.  Neural discrete representation learning.  NeurIPS, 30, 2017.
  * [37] Ashish Vaswani and Noam Shazeer.  Attention is all you need.  NeurIPS, 30, 2017.
  * [38] Catherine Wah and Steve Branson.  The caltech-ucsd birds-200-2011 dataset.  2011\.
  * [39] Zhibiao Wu and Martha Palmer.  Verb semantics and lexical selection.  arXiv preprint cmp-lg/9406033, 1994.
  * [40] Jiahui Yu and Xin Li.  Vector-quantized image modeling with improved vqgan.  arXiv preprint arXiv:2110.04627, 2021.
  * [41] Lijun Yu and Yong Cheng.  Spae: Semantic pyramid autoencoder for multimodal generation with frozen llms.  arXiv preprint arXiv:2306.17842, 2023.
  * [42] Jiahui Zhang and Fangneng Zhan.  Regularized vector quantization for tokenized image synthesis.  In CVPR, pages 18467–18476, 2023.
  * [43] Chuanxia Zheng and Andrea Vedaldi.  Online clustered codebook.  In ICCV, pages 22798–22807, 2023.
  * [44] Lei Zhu and Fangyun Wei.  Beyond text: Frozen large language models in visual signal comprehension.  arXiv preprint arXiv:2403.07874, 2024.

##  Appendix A Appendix / supplemental material

###  A.1 Experiment Details

Semantic image synthesis: All models follow the default setting of VQ-GAN-
Transformer111https://github.com/CompVis/taming-transformers. Specifically,
the vocabulary size, embedding number, and input sequence length are 1024,
1024, and 512, respectively. The layers and heads of the transformer are both
16. The training epoch is 30 for all models. The experiments are conducted on
1 4090 GPU with a batch size of 15 and one day of training time.
Text-to-image generation: All models follow the default setting of VQ-
Diffusion222https://github.com/microsoft/VQ-Diffusion. Specifically, the
layers of the transformer are 19 with dimension of 1024. The diffusion step is
100. The training epoch is 90 for all models. The experiments are conducted on
1 4090 GPU with a batch size of 24 and two days of training time.
Image captioning: Inspired by ClipCap [24], we use the trained codes to
replace the ClipCap’s prefix embeddings. The model framework is shown in Fig.
9 (a). The training epoch is 100 for all models. The experiments are conducted
on 2 4090 GPUs with a batch size of 60 and one day of training time.
Visual question answering: The COCO-QA [28] dataset is automatically generated
from captions in the Microsoft COCO dataset [18]. There are 78,736 train
questions and 38,948 test questions in the dataset, These questions are based
on 8,000 and 4,000 images respectively. There are four types of questions
including object, number, color, and location. Each type takes 70%, 7%, 17%,
and 6% of the whole dataset, respectively. All answers in this data set are
single word. Following the image captioning task, we use the last hidden
embedding to do VQA, as shown in Fig. 9 (b). Following the [22], we report
classification accuracy and Wu-Palmer similarity (WUPS). The training epoch is
50 for all models. The experiments are conducted on 2 4090 GPUs with a batch
size of 60 and one day of training time.

(a) Image captioning

(b) VQA

Figure 9: The architecture of the visual text reasoning based on GPT.

###  A.2 More Examples and Qualitative Results

We provide more examples of image reconstruction in Fig. 10, image-to-text
retrieval in Fig. 11 and Fig. 12. We also provide more image synthesis results
in Fig. 13 for semantic image synthesis, and text-to-image synthesis in Fig.
14. We provide some examples of image captioning in Fig. 15 and VQA in Fig.
16. Moreover, we also apply our method of image completion on CelebA-HQ,
generating images based on masked images. The results are shown in Fig. 17.

Figure 10: Reconstruction from different models on four datasets. The red-
color boxes highlight reconstruction details. Figure 11: Examples of the top-5
most similar text selected on Textcaps based on VQ-GAN+LG. The bold text means
the same as the ground truth result. Figure 12: Examples of the top-5 most
similar text selected on CUB-200 based on VQ-GAN+LG. The bold text means the
same as the ground truth result. Figure 13: Semantic image synthesis on
CelebA-HQ. Figure 14: More text-to-image generation on CelebA-HQ. Figure 15:
Image Captioning on CUB-200 based on VQ-GAN and VQ-GAN+LG. Figure 16: VQA on
COCO-QA based on VQ-GAN and VQ-GAN+LG. Figure 17: Image completion on CelebA-
HQ.


# A Systematic Comparison of Simulation Software for Robotic Arm Manipulation
using ROS2

Florent P. Audonnet1, Andrew Hamilton2 and Gerardo Aragon-Camarasa1,2 This
research has been supported by EPSRC DTA No. 2605103 and NVIDIA Corporation
for the donation of the Titan Xp GPU.1 School of Computing Science, University
of Glasgow, G12 8QQ, Scotland, United Kingdom<EMAIL_ADDRESS>and<EMAIL_ADDRESS>National Manufacturing Institute in
Scotland, Scotland, United Kingdom<EMAIL_ADDRESS>
###### Abstract

Simulation software is a powerful tool for robotics research, allowing the
virtual representation of the real world. However with the rise of the Robot
Operating System (ROS), there are new simulation software packages that have
not been compared within the literature. This paper proposes a systematic
review of simulation software that are compatible with ROS version 2. The
focus is research in robotics arm manipulation as it represents the most often
used robotic application in industry and their future applicability to digital
twins. For this, we thus benchmark simulation software under similar
parameters, tasks and scenarios, and evaluate them in terms of their
capability for long-term operations, success at completing a task,
repeatability and resource usage. We find that there is no best simulation
software overall, but two simulation packages (Ignition and Webots) have
higher stability than other while, in terms of resources usage, PyBullet and
Coppeliasim consume less than their competitors.

##  I INTRODUCTION

With the advent of deep learning technologies, current research efforts have
been focused on teaching robots how to perform various tasks autonomously.
However, a data-driven approach is required to acquire and process the vast
amount of data to effectively teach a robot how to perform a task which is
unfeasible using a real robotic testbed. For this, robot simulation software
[1, 2, 3, 4, 5] have been used to overcome the shortcomings of data-hungry AI
approaches and to allow the developer to obtain a constant environment [6]. In
a simulated environment the world can be controlled, including aspects that
would be impractical in reality. There is also no risk of damaging the robot
or human operators, and simulations allow to control the time which increases
the speed of data collection.

Figure 1:  Overview of the Simulation Software and Their Capabilities

Simulations are the gateway for Digital Twins, a high-fidelity representation
of the physical world [7], and can allow manufacturing to increase production
and flexibility of supply chains. Therefore, digital twinning consists of
interconnecting a simulation software to a real autonomous robotic system in
order to reduce the implementation time of manufacturing process when changing
a production line. A recent example of a digital twin solution for a robotic
arm can be found in [8] where the authors used ROS (Robot Operating System)
[9] to achieve seamless operation between the real and digital world. However,
simulation software are not perfect because their physics engines are not an
accurate representation of the real world. Furthermore, simulations allow for
perfect data capture with no noise which has powered research in deep learning
approaches for robotics.

In this paper, we propose to carry out a systematic benchmark of current
simulation software (Figure 1) to investigate their performance and
suitability to perform different robotic manipulation tasks using the ROS2
(Robot Operating System version 2). ROS has become the de facto communication
platform for modern robotic systems. We choose ROS2 because it supports a wide
array of devices (e.g. micro-controllers) which enables the integration of
Internet of Things (IoT). The latter is a main requirement for developing a
working digital twin system. ROS2 can also be used to bridge the gap between
AI-enabled robots and real world robot control. We choose robotic arms in this
paper as they are prevalent in automated manufacturing operations.

Figure 2:  Simulation Tasks Progression over Time. Task 1 (top), is a Pick and
Place task where the goal is to stack 3 columns of 5 cubes. Task 2 (bottom) is
a Throw task where the goal is to collapse a pyramid of 6 cubes by throwing a
cube at it.

We consider 2 tasks for the robot arm to perform. The first task is about
picking and placing an object which is a common operation in industry. The
second task consists of throwing a cube into a pyramid. We chose this throwing
task as we aim to test the accuracy and repeatability of the simulation
software to decide its potential suitability for building digital twins.
Figure 2 shows an overview of the tasks. We record the resources usage of each
simulation considered in this paper while performing each task in both a
headless and a graphical version. Our contributions consist of proposing a
systematic comparison of state-of-the-art robotic arm simulation software
using ROS2, the state-of-the-art version of the robot operating system.
Furthermore, we develop an experimental methodology to evaluate robot
simulations software for long-term operations and their success at completing
a task. We also devised an experimental validation system to evaluate the
stability of robot simulation software and their capacity to repeat a given
task.

##  II BACKGROUND

Benchmarking robotic simulation software can trace back its origins to Oreback
and Christensen [10]. They were the first to propose a methodology to test
robotic simulations. Their approach consisted of summarising the capabilities
of 3 simulators, considering quantitative values such as supported OS or
programming languages and qualitative opinions such as the learning curve or
the difficulty of installation. They also recorded the amount of RAM used
while using the simulation software to control a real robot. Kramer and
Scheutz [11] extended [10] and developed a comprehensive testing suite for
open-source robot simulation software. They devised a set of criteria based on
the software-development process and created a feature score based on
different properties such as usability, supported features (path planning,
voice recognition, etc.) and faults handling. They performed a low-level
implementation of a task on a real robot and recorded the resource usage.
However, the task is scarcely described and was only repeated three times.

Before ROS [9], roboticists used the simulation software as a middleware to
send data and commands to the robot, e.g. [11]. Staranowicz and Mariottini
[12] provided the first comparison of simulation software that used ROS as the
communication framework to control a robot. They compared the properties of
three open source and commercial simulations. They then demonstrated the
capabilities of Gazebo [13], a popular simulation software, with a real robot,
effectively creating a Digital Twin. However they neither recorded the
resources usage nor did they try a different simulator for the real world
task. Their work was then extended by Nogueira [14] who compared 2 simulators
and their integration with ROS, the ease of building the world and the CPU
usage.

Pitonakova et al. [15] adopted the methodology in [14]. They compared three
simulators and then ran extensive tests to record each simulator performance
on tasks involving multiple robotic arms. They recorded memory, CPU usage, and
the real time factor, meaning the speed at which the simulation runs. This is
vital for Digital Twining. It is also vital for machine learning, as the
faster the simulation runs without compromising the physics, the faster the
training of a machine learning model would be. They performed each tests with
and without a Graphical User Interface (GUI) and then compared the impact of
rendering the simulation to a screen. Ayala et al. [16] and Korber et al. [17]
followed the idea of recording resources usage during the running of the
experiment. After recapitulating the properties of each simulator, they coded
tasks and recorded, memory and CPU usage. Korber et al. compared four
simulation software on robotic manipulation tasks while Ayala et al. only
compared 3 for humanoid robot scenarios.

In this paper, we initially consider eight robot simulation software but
narrow our benchmark to five that support for ROS2, including two simulation
software that have not been considered in the literature. We also propose to
implement a pick and place and a throwing tasks to investigate the advantages
and limitations for each simulation software, their performance and,
ultimately, their suitability for Digital Twins.

##  III MATERIALS and METHODS

To evaluate and compare robotic simulation software, we develop our
methodology and experiments guided by the following research questions:

  1. RQ1

How does simulation software compare in terms of supporting long-term
operations while still succeeding at completing a given task?

  2. RQ2

How repeatable is the simulation software under the same scene and task
constrains?

  3. RQ3

Which simulation software would be more suitable for machine learning research
in terms of resource usage and idle time?

###  III-A Simulation Software

The above research questions inform our choice of the simulation software
investigated in this paper as shown on Figure 1. Not all of the simulation
software have support for ROS2. For this paper, we have attempted to implement
our own ROS2 bridge but with limited success due to the rapid development
cycle of ROS2. For completeness, we describe our experience while implementing
the ROS2 bridge for the simulations we do not use in this paper. Unity’s
existing bridge is limited as it does not support asynchronous communications
which are the underlying communication paradigm in ROS2. Mujuco conflicts with
ROS2 because ROS2 multithreaded controller is incompatible with Mujuco single
threading nature. Finally, we had to drop Gazebo because development efforts
have turned to Ignition, and there is currently an implementation error in the
controller code, causing our robot to move
erratically111https://github.com/ros-simulation/gazebo_ros2_control/issues/73.

We also consider simulations that feature a headless mode. This is because, a
headless mode is critical in a machine learning context (ref. RQ3). Therefore,
we analyse the impact of the GUI in terms of the resource usage. The robot
simulation software examined in this paper are:

1) Ignition [1] is a set of open source software libraries which are arranged
as multiple modular plugins written in Ruby and C++. They have been developed
by Open Robotics since 2019. It has a similar communication principle to ROS2.
We chose this simulator as it is the successor of Gazebo.

2)Webots [2] has been developed since 1998 by Cyberbotics, a spin-off company
from EPFL (Swiss Federal Institute of Technology Lausanne). It supports a wide
range of sensors and robot controllers out of the box, as well as being well
documented and including several examples files. In Figure 1, it has partial
headless support because it only disables the simulation rendering. There is
still a GUI visible. We considered it as it is one of the oldest simulation
software still being actively developed.

3) Isaac Sim [3] is a recent, Linux only, simulation environment developed by
Nvidia which runs on the PhysX engine, and can be programmed in Python or C.
By default, it integrates machine learning capabilities, and has in-built
plugins to generate synthetic data for domain adaptation and transfer
learning. The latter is possible because of its ray tracing capabilities which
allow for a visual simulation as close to reality. While it can be run in
headless mode, this is not possible while using their ROS2 extension since
there is an issue with the ROS2 plugin not loading when launched from a python
script instead of from the terminal.

4) PyBullet [4] is a Python-based simulation environment based on the Bullet
physics engine which has been in development since 2016. It is popular for
machine learning research as it is lightweight and easy to use. For this
paper, we implemented a ROS2 compatible plugin since there is no official ROS2
support.

5) Coppeliasim [5], previously known as V-REP, is a partially closed source
simulation environment, developed since 2010. It can be programmed directly in
Lua, or using external controllers in 6 other languages. We decided to include
it in our research as it has been compared in previous simulation software
reviews, e.g. [14, 15, 18, 16, 19].

TABLE I: Sub-Task overview and Design rationale Name |  Design |  Features Tested |  Data Recorded and Rationale
---|---|---|---
Task 1-A: Pick and Place |  A robotic arm randomly takes 5 cm cubes from a table with 21 cubes arranged in a $7\times 3$ grid. The task aim is to stack them into 3 towers of 5 cubes as can be seen in Figure 2. We consider 3 stacks in order to leave cubes on the table and to allow for more diversity in each repetition. We set the limit to 5 stacked cubes due to the height of the table and the capabilities of the robot. |  Friction, Gravity, Inertia |  This experiment addresses RQ2 which analyses the numbers of cubes correctly placed. It will also test the stability and suitability of the simulation for long operations, as recorded by the number of cubes still in place at the end (ref. RQ1). The idea of stacking cubes to analyse performance is motivated from [17].
Task 1-B: Pick and Place Headless |  We use the same setup as Task 1-A but without a GUI. This was chosen as in a machine learning setting, experiments need to be restarted multiple times and often run on a server with no screen (ref. RQ3).
Task 2-A: Throwing |  A robotic arm will pick up a cube and throw it towards a pyramid of 6 cubes. The arm goes as far back as mechanically possible and perform a throwing motion towards the pyramid in front of it. Figure 2 shows the trajectory taken by the robot during this task. The cube is released at $50\%$ of the trajectory. The pyramid is placed such that a successful throw at full power will collapse it. |  Friction, Gravity, Inertia, Kinetic |  This task benchmarks the accuracy and repeatability of the simulation software and addresses RQ2. The latter is carried out by recording the number of cubes that are displaced from their original position. This idea has been inspired by a contributor to Ignition222https://github.com/AndrejOrsula/ign_moveit2 demonstrating how to interface ROS2 and Ignition.
Task 2-B: Throwing Headless |  We follow the same design as Task 2-A, except without a GUI (ref. RQ3).

###  III-B Data Capturing

For data capturing and recording, we adopt the metrics mentioned in Section
II, namely, the processor usage (CPU), the memory usage (RAM) and the
execution time (ref. RQ1 & 3). We also record task-specific data, such as the
number of cubes placed or moved (ref. RQ2 & 3). The execution time is not
mentioned in the literature but was added as a metric for machine learning, in
which the running time can have major impact, as training a model involves
several iterations of the simulation. A delay of tens of seconds for one
iteration can turn into hours of difference for long training sessions. In
order to accurately record each simulation, we start recording 5 seconds
before the simulation starts and ends the recording 60 second after the task
has ended. We record processes only related to the simulation while discarding
the rest, such OS-specific processes.

###  III-C Robotic Tasks

We consider 2 tasks, each divided into 2 sub-tasks, in order to evaluate each
simulator considered here. A sub-task is repeated 20 times with the aim to
reduce the variance during data recording and to obtain an accurate
statistical characterisation of a simulation. In practice, we found that more
than 20 repetitions does not result in a statistical significant difference.
The 2 tasks along with their rationale are summarised in Table I. The task
execution logic is the same for all simulations. We must note that we use the
default simulation parameters to setup these tasks. This is to remove bias
while implementing the tasks and avoid tuning simulator specific parameters in
order to obtain an objective evaluation for each simulation software.

###  III-D Robot Control

There are 3 methods to control a robot using ROS2, namely, the joint
controller, the joint trajectory follower and the general purpose ROS
controller. The joint controller sets the position of the joints to a given
joint angle using hardware specific interfaces of a robot. This is the
simplest method as it provides no feedback to the controller. The joint
trajectory follower uses a ROS action client/server combination in which the
client sends the joint position for a given trajectory as a list along with a
delay. Then, the server continuously sends the current value of the joints as
a feedback mechanism until the trajectory is completed. This method works well
in practice and we have implemented it for Coppeliasim, PyBullet and Isaac
Sim. For the Ignition and Webots, we use the general purpose ROS controller
(ros_control) [20], which is not implemented for the other simulations. It
provides a wrapper for the joint trajectory follower described above, but also
provides different methods of control such as a velocity, effort or gripper
controller.

##  IV EXPERIMENTS

###  IV-A Methodology

We use a docker container with Nvidia Ubuntu 20.04 cudagl image for all
simulators except for Isaac Sim that cannot access the base graphics driver
API when using docker. Isaac Sim is thus executed in the base system from
where we run all experiments. ROS2 Foxy has been installed, along with
simulator specific packages. Docker has been used to easily provide an image
with all the necessary packages installed without conflict between different
simulations. It also allows for reproducibility of these experiments by
providing the same setup every time. The base system runs an Intel I7 10700
with 32GB of RAM and an Nvidia GeForce RTX 3060 with Ubuntu 20.04. We used
psutil333https://pypi.org/project/psutil/ which is a python package that
records the CPU and RAM usage. Each process was monitored at 10 Hz to minimise
the resources impact. For each simulator we used the recommended time step,
and we have fixed all simulators to run in real time. We use the Franka Panda
robot, and its model and configuration files provided by MoveIt 2 [21]. The
project repository can be found at
https://github.com/09ubberboy90/ros2_sim_comp.

Figure 3: Implementation Diagram

###  IV-B Implementation

The implementation comprises 4 components as shown in Figure 3 and as noted
below.

  1. 1

Simulator: launches the simulation software and spawns the robot along with
the gripper and arm controllers.

  2. 2

Pathfinder: launches Rviz (a ROS data visualisation software) and MoveIt 2.

  3. 3

Controller: chooses which object to grab from the list of collision objects
advertised by MoveIt 2, shown in Figure 3 as blue arrows. Then, the pick and
place or throwing task is executed accordingly.

  4. 4

The Object-Handler: spawns the objects and publishes their position to the
planning scene of MoveIt 2 at 2 Hz. We choose 2 Hz because the scene’s rate of
change in real time does not deviate considerably. Higher rates consume more
CPU usage which impacts the performance of all simulations in this paper.

In our implementation, both the arm and the gripper are controlled using a
position controller. We must note that the gripper controller available in
ROS1 has not yet been ported to ROS2. The latter causes issues while grasping
objects in the simulations (except Webots) as the amount of force is constant
with no feedback. To mitigate this issue, we command the gripper to close its
gripper beyond the optimal closing distance to ensure that the object is
grasped correctly. Webots does not have this issue because it implements PID
controller by default for each simulated motor. These 4 components are
launched using a single ROS2 launch file with ad hoc delays to ensure
everything is started as Part 3 and 4 do not work unless part 2 is launched.
For each simulation software, we are using the default physics time step. The
physics engine is also the default, except for Coppeliasim, in which we use
the Newton physics engine because the other supported physics engines causes
the gripper to fail to grip the cube.

###  IV-C Task 1 Experiments

TABLE II: Task 1 results Name | Failure (%) | Cubes Placed | Cubes at the end
---|---|---|---
(%) | of the task (%)
Min | Mean | Max | Min | Mean | Max
Ignition | 0 | 60 | 91 | 100 | 47 | 82 | 100
Ignition GUI | 0 | 80 | 94 | 100 | 67 | 88 | 100
Isaac Sim GUI | 10 | 47 | 89 | 100 | 20 | 65 | 100
Pybullet | 0 | 7 | 18 | 47 | 7 | 15 | 47
Pybullet GUI | 0 | 7 | 11 | 40 | 7 | 11 | 40
Coppeliasim | 30 | 67 | 92 | 100 | 47 | 79 | 100
Coppeliasim GUI | 15 | 53 | 91 | 100 | 47 | 76 | 100
Webots | 5 | 53 | 88 | 100 | 47 | 79 | 93
Webots GUI | 5 | 73 | 91 | 100 | 40 | 80 | 100

Table II shows the result of task 1, which addresses RQ1 & RQ2. The reason the
task times out (failure in Table II) is because the ROS controller fails to
start, or in the case of Coppeliasim, the client refuses to connect to the
simulation server. The rest of the metrics only focus on the successful
attempts.

Ignition and PyBullet did not have timeouts; however, PyBullet performs
significantly worse at stacking 5 towers than the other simulators as $15\%$
of the cubes in average (i.e 3 cubes) were correctly positioned at the end of
the simulation, and, therefore, the robot does not encounter scenarios where
it collapses towers. Ignition and Webots are the best performing simulations
at executing the task of stacking cubes, and at keeping the cubes in place.
Coppeliasim and Isaac Sim, are carrying out the task well at placing the cube
in the right place but, tend to have situations where the robot collapses the
towers. Furthermore, while Coppeliasim achieves $92\%$ success of placing
cubes, we can observe that it often times out, and reduces its overall
success. We can also observe in Table II that there is no a statistical
significant difference between headless and GUI modes. These results suggest
that Ignition (headless and GUI) succeeds at completing the task more
frequently using the default parameters (ref. RQ1) and has less variation over
different attempts (ref. RQ2).

TABLE III: Task 1 Resources usage. Name | CPU (%) | RAM (MB)
---|---|---
Ignition | 202 ± 94 | 686 ± 313
Ignition GUI | 205 ± 51 | 775 ± 159
Isaac Sim GUI | 134 ± 25 | 10070 ± 1885
Pybullet | 117 ± 41 | 663 ± 224
Pybullet GUI | 140 ± 28 | 919 ± 168
Coppeliasim | 135 ± 42 | 860 ± 262
Coppeliasim GUI | 100 ± 43 | 850 ± 379
Webots | 144 ± 72 | 1191 ± 550
Webots GUI | 162 ± 61 | 1322 ± 410

Table III shows that PyBullet headless consumes fewer resources overall, while
Isaac Sim, is the most memory intensive simulation as it consumes 10 times
more RAM than the next simulator (Webots GUI). This is inline with the current
trend of Pybullet being used in machine learning research (ref. RQ3). It is
worth noting that Coppeliasim uses fewer resources with a GUI than headless.
We speculate that this is because it was initially designed as a GUI
application, with headless support only added at a later date, thus having
received less development focus.

Figure 4: Task 1: Mean task start and end time

Figure 4 shows the spread of the start time and end time for each simulation
(ref. RQ3). As mentioned in IV-B, Isaac Sim has to be started manually, thus
the time that takes the simulation to start is not captured in the plot.
Ignition takes the most time to load, because it requires an additional
movement to the start position that it has to do to start the effort
controller. Webots finishes the earliest with little variation. Combined with
its relatively high success rate from Table II, Webots appears to be ideal for
prototyping robotic tasks and for machine learning experiments due to its
relatively high success rate from Table II and finishing the task and
simulation early with low variation. PyBullet, on the other hand, takes the
most time, and combined with its high failure rate (with the default
parameters), it may not be suitable for machine learning as it would take more
time to run a single experiment. Similarly, further parameter tuning would be
required in order to obtain a stable simulation that succeeds at completing
the task.

###  IV-D Experiment 2

TABLE IV: Task 2 results Name | Failure (%) | Cubes Moved %)
---|---|---
Min | Mean | Max
Ignition | 10 | 0 | 0 | 0
Ignition GUI | 0 | 0 | 0 | 0
Isaac Sim GUI | 0 | 0 | 18 | 83
Pybullet | 0 | 0 | 0 | 0
Pybullet GUI | 0 | 0 | 0 | 0
Coppeliasim | 32 | 0 | 9 | 50
Coppeliasim GUI | 20 | 0 | 0 | 0
Webots | 5 | 0 | 11 | 50
Webots GUI | 15 | 0 | 21 | 50

As shown in Table IV, which focuses on RQ1 & 3, only Webots throws
consistently. Isaac Sim consistently manages to throw the cube but fails to
hit the pyramid as the motion behaviour is not consistent. We speculate that
this is because we did not tune the simulation parameters and used the default
values. Coppeliasim and PyBullet manages to hit the pyramid, but the behaviour
is rare as the few times the arm manages to successfully perform the throwing
motion, the throw is not always at the same distance nor perfectly aligned.
Coppeliasim has a high timing out rate (failure column in Table IV) due to the
reasons mentioned in Section IV-C. Finally, for Ignition, the success at
hitting the pyramid is zero. We observe that, in most cases, the cube falls in
transit, especially when the arm is as far back as possible and starts to move
at full speed for the throwing motion. At this point, the robot and the cube
have the highest moment of inertia, and if the friction between the cube and
the gripper is not enough, the cube falls. We must note that we fix the
friction parameter to explore the default capabilities for each simulator. We
also notice that there are instances when the robot manages to throw the cube
but does not hit the pyramid. This is because the gripper controller had a
delay in opening its gripper, changing the thrown cube landing spot.

TABLE V: Task 2 Resources usage Name | CPU (%) | RAM (MB)
---|---|---
Ignition | 152 ± 99 | 524 ± 340
Ignition GUI | 137 ± 103 | 574 ± 371
Isaac Sim GUI | 148 ± 48 | 11123 ± 1865
Pybullet | 139 ± 32 | 730 ± 174
Pybullet GUI | 136 ± 31 | 828 ± 205
Coppeliasim | 115 ± 37 | 742 ± 185
Coppeliasim GUI | 98 ± 28 | 838 ± 214
Webots | 141 ± 74 | 1275 ± 261
Webots GUI | 134 ± 72 | 1207 ± 198

Table V shows similar results to task 1. Coppeliasim uses the lowest amount of
CPU while Ignition uses the less memory. The CPU usage for all simulations
observes less variation. This could be due to the simplicity of the world and
the short time of execution. As mentioned in IV-C, Coppeliasim still uses
fewer resources with a GUI than headless. Figure 5 shows similar start and end
time for all simulations, observing lower variations compared to task 1. The
reason for this is because the relatively short time of execution and the low
amount of path planning that can fail and delay the execution. For this
scenario, considering only the time of execution will not have and impact on
the choice for a machine learning approach as the difference between execution
is minimal. If the resource usage is important, then Coppeliasim should be
considered for machine learning tasks. Otherwise, a more successful simulation
should be considered such as Webots.

Figure 5: Task 2: Mean task start and end time

##  V CONCLUSIONS & FUTURE WORK

In this paper, we have investigated current robot simulation software
performance and their suitability to perform two different robotic
manipulation tasks. We have also developed a methodology to systematically
benchmark robot simulations under similar parameters, tasks and scenarios.
Based on our experimental results, Webots appears to be the more suitable for
long-term operations while still succeeding at completing a given task (ref.
RQ1) and be able to replicate the same simulation conditions across attempts
(ref. RQ2). Webots would only be suitable for machine learning if the
execution time and resources are not a requirement while training machine
learning models (ref. RQ3). Ignition, while comparable to Webots, is more
suited to answer RQ1 & RQ3. RQ2 is only satisfied if the task is slow moving
and constant. We must note that Ignition is still in development and some of
the challenges we encountered while implementing both tasks and carrying out
our experiments may be mitigated in the future. Coppeliasim and PyBullet have
less impact in terms of resource usage and are the most suited to answer RQ3.
That is, Coppeliasim provides better stability for task success at the cost of
timing out more often. Finally, Isaac Sim only satisfies RQ1, as the simulated
scene was not repeatable across attempts.

From our review and experimental results, we found that current robot
simulation software could not be used to develop a digital twin. This is
because the simulators considered in this paper cannot maintain a repeatable
simulated scene over time. We hypothesise that a continuous feedback mechanism
is needed between the simulation and reality similar to [22] in order to
maintain an accurate representation of the real environment. While this paper
focused on benchmarking robot simulation software, future work consists of
optimising each simulator to minimise failure rates and maximise task
completion, and benchmark them accordingly. Additionally, the Unreal Engine
plugin for ROS2, has recently seen more development and could potentially
replace Unity in our original plan. We also aim to specifically benchmark each
simulation in a machine learning context such as in [23] with the view to
develop a digital twin that can take advantage of a simulated environment to
deploy AI solutions for autonomous robotic systems.

## References

  * [1] O. Robotics, “https://ignitionrobotics.org/home.” [Online]. Available: https://ignitionrobotics.org/home
  * [2] Webots, “http://www.cyberbotics.com,” open-source Mobile Robot Simulation Software. [Online]. Available: http://www.cyberbotics.com
  * [3] N. Corporation, “Nvidia isaac sim.” [Online]. Available: https://developer.nvidia.com/isaac-sim
  * [4] E. Coumans and Y. Bai, “Pybullet, a python module for physics simulation for games, robotics and machine learning,” http://pybullet.org, 2016–2021.
  * [5] E. Rohmer, S. P. N. Singh, and M. Freese, “V-rep: A versatile and scalable robot simulation framework,” in _2013 IEEE/RSJ International Conference on Intelligent Robots and Systems_ , 2013, pp. 1321–1326.
  * [6] J. Banks, “1999: INTRODUCTION TO SIMULATION,” p. 7.
  * [7] Y. Lu, C. Liu, K. I.-K. Wang, H. Huang, and X. Xu, “Digital Twin-driven smart manufacturing: Connotation, reference model, applications and research issues,” _Robotics and Computer-Integrated Manufacturing_ , vol. 61, p. 101837, Feb. 2020. [Online]. Available: https://linkinghub.elsevier.com/retrieve/pii/S0736584519302480
  * [8] P. Tavares, J. Silva, P. Costa, G. Veiga, and A. Moreira, “Flexible Work Cell Simulator Using Digital Twin Methodology for Highly Complex Systems in Industry 4.0,” Nov. 2018, pp. 541–552.
  * [9] Stanford Artificial Intelligence Laboratory et al., “Robotic operating system.” [Online]. Available: https://www.ros.org
  * [10] A. Orebäck and H. I. Christensen, “Evaluation of Architectures for Mobile Robotics,” _Autonomous Robots_ , vol. 14, no. 1, pp. 33–49, Jan. 2003. [Online]. Available: https://doi.org/10.1023/A:1020975419546
  * [11] J. Kramer and M. Scheutz, “Development environments for autonomous mobile robots: A survey,” _Autonomous Robots_ , vol. 22, no. 2, pp. 101–132, Jan. 2007. [Online]. Available: http://link.springer.com/10.1007/s10514-006-9013-8
  * [12] A. Staranowicz and G. L. Mariottini, “A survey and comparison of commercial and open-source robotic simulator software,” in _Proceedings of the 4th International Conference on PErvasive Technologies Related to Assistive Environments_ , ser. PETRA ’11. New York, NY, USA: Association for Computing Machinery, May 2011, pp. 1–8. [Online]. Available: https://doi.org/10.1145/2141622.2141689
  * [13] N. Koenig and A. Howard, “Design and use paradigms for Gazebo, an open-source multi-robot simulator,” in _2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE Cat. No.04CH37566)_ , vol. 3, Sep. 2004, pp. 2149–2154 vol.3.
  * [14] L. Nogueira, “Comparative Analysis Between Gazebo and V-REP Robotic Simulators,” Tech. Rep., Dec. 2014.
  * [15] L. Pitonakova, M. Giuliani, A. Pipe, and A. Winfield, “Feature and Performance Comparison of the V-REP, Gazebo and ARGoS Robot Simulators,” in _Towards Autonomous Robotic Systems_ , M. Giuliani, T. Assaf, and M. E. Giannaccini, Eds. Cham: Springer International Publishing, 2018, pp. 357–368.
  * [16] A. Ayala, F. Cruz, D. Campos, R. Rubio, B. Fernandes, and R. Dazeley, “A Comparison of Humanoid Robot Simulators: A Quantitative Approach,” _arXiv:2008.04627 [cs]_ , Aug. 2020, arXiv: 2008.04627. [Online]. Available: http://arxiv.org/abs/2008.04627
  * [17] M. Körber, J. Lange, S. Rediske, S. Steinmann, and R. Glück, “Comparing Popular Simulation Environments in the Scope of Robotics and Reinforcement Learning,” _arXiv:2103.04616 [cs]_ , Mar. 2021, arXiv: 2103.04616. [Online]. Available: http://arxiv.org/abs/2103.04616
  * [18] M. S. P. d. Melo, J. G. d. S. Neto, P. J. L. d. Silva, J. M. X. N. Teixeira, and V. Teichrieb, “Analysis and Comparison of Robotics 3D Simulators,” in _2019 21st Symposium on Virtual and Augmented Reality (SVR)_ , Oct. 2019, pp. 242–251.
  * [19] S. Ivaldi, V. Padois, and F. Nori, “Tools for dynamics simulation of robots: a survey based on user feedback,” _arXiv:1402.7050 [cs]_ , Feb. 2014, arXiv: 1402.7050. [Online]. Available: http://arxiv.org/abs/1402.7050
  * [20] S. Chitta, E. Marder-Eppstein, W. Meeussen, V. Pradeep, A. R. Tsouroukdissian, J. Bohren, D. Coleman, B. Magyar, G. Raiola, M. Lüdtke, and E. F. Perdomo, “ros_control: A generic and simple control framework for ROS,” _Journal of Open Source Software_ , vol. 2, no. 20, p. 456, Dec. 2017. [Online]. Available: https://joss.theoj.org/papers/10.21105/joss.00456
  * [21] T. Coleman David, “Reducing the Barrier to Entry of Complex Robotic Software: a MoveIt! Case Study,” 2014, publisher: Università degli studi di Bergamo. [Online]. Available: https://aisberg.unibg.it//handle/10446/87657
  * [22] P. Chang and T. Padif, “Sim2Real2Sim: Bridging the Gap Between Simulation and Real-World in Flexible Object Manipulation,” in _2020 Fourth IEEE International Conference on Robotic Computing (IRC)_ , Nov. 2020, pp. 56–62.
  * [23] N. Pitsillos, A. Pore, B. S. Jensen, and G. Aragon-Camarasa, “Intrinsic Robotic Introspection: Learning Internal States From Neuron Activations,” in _2021 IEEE International Conference on Development and Learning (ICDL)_. Beijing, China: IEEE, Aug. 2021, pp. 1–7. [Online]. Available: https://ieeexplore.ieee.org/document/9515672/