# Robo3D: Towards Robust and Reliable 3D Perception against Corruptions

Lingdong Kong1,2,3,∗ Youquan Liu1,4,∗ Xin Li1,5,∗ Runnan Chen1,6 Wenwei
Zhang1,7
Jiawei Ren7 Liang Pan7 Kai Chen1 Ziwei Liu${}^{7,\textrm{\Letter}}$
1Shanghai AI Laboratory 2National University of Singapore 3CNRS@CREATE
4Hochschule Bremerhaven
5East China Normal University 6The University of Hong Kong 7S-Lab, Nanyang
Technological University
<EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract

The robustness of 3D perception systems under natural corruptions from
environments and sensors is pivotal for safety-critical applications. Existing
large-scale 3D perception datasets often contain data that are meticulously
cleaned. Such configurations, however, cannot reflect the reliability of
perception models during the deployment stage. In this work, we present
Robo3D, the first comprehensive benchmark heading toward probing the
robustness of 3D detectors and segmentors under out-of-distribution scenarios
against natural corruptions that occur in real-world environments.
Specifically, we consider eight corruption types stemming from severe weather
conditions, external disturbances, and internal sensor failure. We uncover
that, although promising results have been progressively achieved on standard
benchmarks, state-of-the-art 3D perception models are at risk of being
vulnerable to corruptions. We draw key observations on the use of data
representations, augmentation schemes, and training strategies, that could
severely affect the model’s performance. To pursue better robustness, we
propose a density-insensitive training framework along with a simple flexible
voxelization strategy to enhance the model resiliency. We hope our benchmark
and approach could inspire future research in designing more robust and
reliable 3D perception models. Our robustness benchmark suite is publicly
available111https://github.com/ldkong1205/Robo3D..

Figure 1: Taxonomy of the Robo3D benchmark. We simulate eight corruption types
from three categories: 1) Severe weather conditions, such as fog, rain, and
snow; 2) External disturbances that are caused by motion blur or result in the
missing of LiDAR beams; and 3) Internal sensor failure, including LiDAR
crosstalk, possible incomplete echo, and cross-sensor scenarios. Each
corruption is further split into three levels (light, moderate, and heavy)
based on its severity. ††${(*)}$ The first three authors contributed equally
to this work.

##  1 Introduction

3D perception aims to detect and segment accurate position, orientation,
semantics, and temporary relation of the objects and backgrounds around the
ego-vehicle in the three-dimensional world [3, 21, 27]. With the emergence of
large-scale autonomous driving perception datasets, various approaches in the
fields of LiDAR semantic segmentation and 3D object detection advent each
year, with record-breaking performances on the mainstream benchmarks [20, 4,
8, 19, 64].

Despite the great success achieved on the “clean” evaluation sets, the model’s
robustness against out-of-distribution (OoD) scenarios remain obscure. Recent
attempts mainly focus on probing the OoD robustness from two aspects. The
first line focuses on the transfer of 3D perception models to unseen domains,
e.g., sim2real [75], day2night [28], and city2city [32] adaptations, to probe
the model’s generalizability. The second line aims to design adversarial
examples which can cause the model to make incorrect predictions while keeping
the attacked input close to its original format, i.e., to test the model’s
worst-case scenarios [53, 9, 68].

In this work, different from the above two directions, we aim at understanding
the cause of performance deterioration under real-world corruption and sensor
failure. Current 3D perception models learn point features from LiDAR sensors
or RGB-D cameras, where data corruptions are inevitable due to issues of data
collection, processing, weather conditions, and scene complexity [52]. While
recent works target creating corrupted point clouds from indoor scenes [30] or
object-centric CAD models [63, 92, 2], we simulate corruptions on large-scale
LiDAR point clouds from the complex outdoor driving scenes [20, 4, 8, 64].

As shown in Fig. 1, we consider three distinct corruption sources that are
with a high likelihood to occur in real-world deployment: 1) Severe weather
conditions (fog, rain, and snow) which cause back-scattering, attenuation, and
reflection of the laser pulses [23, 22, 62]; 2) External disturbances, e.g.,
bumpy surfaces, dust, insects, that often lead to nonnegligible motion blur
and LiDAR beam missing issues [48]; and 3) Internal sensor failure, such as
the incomplete echo or miss detection of instances with a dark color (e.g.,
black car) and crosstalk among multiple sensors, which likely deteriorates the
3D perception accuracy [86, 7]. Besides the environmental factors, it is also
important to understand the cross-sensor discrepancy to avoid sudden failure
caused by the sensor configuration change.

To properly fulfill such pursues, we simulate physically-principled
corruptions on the val sets of KITTI [20], SemanticKITTI [4], nuScenes [8],
and Waymo Oepn [64], as our corruption suite dubbed Robo3D. Analogous to the
popular 2D corruption benchmarks [25, 84, 43], we create three severity levels
for each corruption and design suitable metrics as the main indicator for
robustness comparisons. Finally, we conduct exhaustive experiments to
understand the pros and cons of different designs from existing models. We
observe that modern 3D perception models are at risk of being vulnerable even
though their performance on standard benchmarks is improving. Through fine-
grained analyses on a wide range of 3D perception datasets, we diagnose that:

  * •

Sensor setups have direct impacts on feature learning. 3D perception models
trained on data collected with different sensor configurations and protocols
often yield inconsistent resilience.

  * •

3D data representations often coupled with the model’s robustness. The voxel
and point-voxel fusion approaches exhibit clear superiority over the
projection-based methods, e.g., range view.

  * •

3D detectors and segmentors show distinct sensitivities to different
corruption types. A sophisticated combination of both tasks is a viable way to
achieve robust and reliable 3D perception.

  * •

Out-of-context augmentation (OCA) and flexible rasterization strategies can
improve model’s robustness. We thus propose a solution to enhance the
robustness of existing 3D perception models, which consists of a density-
insensitive training framework and a simple flexible voxelization strategy.

The key contributions of this work are summarized as:

  * •

We introduce Robo3D, the first systematically-designed robustness evaluation
suite for LiDAR-based 3D perception under corruptions and sensor failure.

  * •

We benchmark 34 perception models for LiDAR-based semantic segmentation and 3D
object detection tasks, on their robustness against corruptions.

  * •

Based on our observations, we draw in-depth discussions on the design receipt
and propose novel techniques for building more robust 3D perception models.

##  2 Related Work

LiDAR-based Semantic Segmentation. The design choice of 3D segmentors often
correlates with the LiDAR representations, which can be categorized into raw
point [67], range view [72, 44, 31], bird’s eye view [88], voxel [12], and
multi-view fusion [40, 77] methods. The projection-based approach rasterizes
irregular point clouds into 2D grids, which avoids the need for 3D operators
and is thus more hardware-friendly for deployment [13, 89, 11]. The voxel-
based methods which retain the 3D structure are achieving better performance
than other single modalities [93, 83]. Efficient operators like the sparse
convolution are widely adopted to ease the memory footprint [66, 65]. Most
recently, some works start to explore possible complementary between two views
[40, 78, 50] or even more views [77]. Although promising results have been
achieved, the robustness of 3D segmentors against corruptions remains obscure.
As we will discuss in the next sections, these methods have the tendency of
being less robust, mainly due to the lack of a comprehensive robustness
evaluation benchmark.

LiDAR-based 3D Object Detection. Sharing similar basics with LiDAR
segmentation, modern 3D object detectors also adopt various data
representations. Point-based methods [59, 61, 82, 81] implicitly capture local
structures and fine-grained patterns without any quantization to retain the
original point cloud geometry. Voxel-based methods [80, 90, 85, 60, 37, 39,
36, 41] transform irregular point clouds to compact grids while only those
non-empty voxels are stored and utilized for feature extraction through the
sparse convolution [80]. Recently, some works [42, 91] start to explore long-
range contextual dependencies among voxels with self-attention [69]. The
pillar-based methods [34, 55] better balance the accuracy and speed by
controlling the resolution in the vertical axis. The point-voxel fusion
methods [57, 56] can integrate the merits of both representations to learn
more discriminative features. The above methods, however, mainly focused on
obtaining better performance on clean point clouds, while paying much less
attention to the model’s robustness. As we will show in the following
sections, these models are prone to degradation under data corruptions and
sensor failure.

Common Corruptions. The corruption robustness often refers to the capability
of a conventionally trained model for maintaining satisfactory performance
under natural distribution shifts. ImageNet-C [25] is the pioneering work in
this line of research which benchmarks image classification models to common
corruptions and perturbations. Follow-up studies extend on a similar aspect to
other perception tasks, e.g., object detection [43], image segmentation [29],
navigation [10], video classification [84], and pose estimation [70]. The
importance of evaluating the model’s robustness against corruptions has been
constantly proven. Since we are targeting a different sensor, i.e., LiDAR,
most of the well-studied corruption types – such as those designed for camera
malfunctions – become unrealistic or unsuitable for such a data format. This
motivates us to explore new taxonomy for defining more proper corruption types
for the 3D perception tasks in autonomous driving scenarios.

3D Perception Robustness. Several recent attempts proposed to investigate the
vulnerability of point cloud classifiers and detectors in indoor scenes [30,
52, 63, 92, 2]. Recently, there are works started to explore the robustness of
3D object detectors under adversarial attacks [47, 68, 76]. In the context of
corruption robustness, we notice several concurrent works [86, 35, 1, 17, 79].
These works, however, all consider a single task alone and might be
constrained by either a limited number of corruption types or candidate
datasets. Our benchmark properly defines a more diverse range of corruption
types for the general 3D perception task and includes significantly more
models from both LiDAR-based semantic segmentation and 3D object detection
tasks.

##  3 The Robo3D Benchmark

Tailored for LiDAR-based 3D perception tasks, we summarize eight corruption
types commonly occurring in real-world deployment in our benchmark, as shown
in Fig. 1. This section elaborates on the detailed definition of each
corruption type (Sec. 3.1), configurations of different robustness simulation
sets (Sec. 3.2), and evaluation metrics for robustness measurements (Sec.
3.3).

###  3.1 Corruption Types

Given a point $\mathbf{p}\in\mathbb{R}^{4}$ in a LiDAR point cloud with
coordinates $(p^{x},p^{y},p^{z})$ and intensity $p^{i}$, our goal is to
simulate a corrupted point $\mathbf{\hat{p}}$ via a mapping
$\mathbf{\hat{p}}=\mathcal{C}(\mathbf{p})$, with rules constrained by physical
principles or engineering experiences. Due to space limits, We present more
detailed definitions and implementation procedures of our corruption
simulation algorithms in the Appendix.

1) Fog. The LiDAR sensor emits laser pulses for accurate range measurement.
Back-scattering and attenuation of LiDAR points tend to happen in foggy
weather since the water particles in the air will cause inevitable pulse
reflection [5, 6]. In our benchmark, we adopt the physically valid fog
simulation method [23] to create fog-corrupted data. For each $\mathbf{p}$, we
calculate its attenuated response $p^{i_{\text{hard}}}$ and the maximum fog
response $p^{i_{\text{soft}}}$ as follows:

$p^{i_{\text{hard}}}=p^{i}e^{-2\alpha\sqrt{(p^{x})^{2}+(p^{y})^{2}+(p^{z})^{2}}}~{},$
(1)
$p^{i_{\text{soft}}}=p^{i}\frac{(p^{x})^{2}+(p^{y})^{2}+(p^{z})^{2}}{\beta_{0}}\beta_{\text{bs}}\times{p}^{i}_{\text{tmp}}~{},$
(2)
$\mathbf{\hat{p}}=\mathcal{C}_{\text{fog}}(\mathbf{p})=\begin{cases}(\hat{p}^{x},\hat{p}^{y},\hat{p}^{z},p^{i_{\text{soft}}}),&\text{if~{}~{}$p^{i_{\text{soft}}}>p^{i_{\text{hard}}}$}~{},\\\
(p^{x},p^{y},p^{z},p^{i_{\text{hard}}}),&\text{else}~{}.\end{cases}$ (3)

where $\alpha$ is the attenuation coefficient, $\beta_{\text{bs}}$ denotes the
back-scattering coefficient, $\beta_{0}$ describes the differential
reflectivity of the target objects, and the ${p}^{i}_{\text{tmp}}$ symbol is
the received response for the soft target term.

2) Wet Ground. The emitted laser pulses will likely lose certain amounts of
energy when hitting wet surfaces, which causes significantly attenuated laser
echoes depending on the water height $d_{w}$ and mirror refraction rate [62].
We follow [22] to model the attenuation caused by ground wetness. A pre-
processing step is taken to estimate the ground plane with existing semantic
labels or RANSAC [18]. Next, a ground plane point of its measured intensity
$\hat{p}^{i}$ is obtained based on the modified reflectivity, and the point is
only kept if its intensity is greater than the noise floor $i_{n}$ via the
following mapping:

$\mathcal{C}_{\text{wet}}(\mathbf{p})=\begin{cases}(p^{x},p^{y},p^{z},\hat{p}^{i}),&\text{if~{}~{}~{}~{}$\hat{p}^{i}>i_{n}$
\text{\&~{}~{}$\mathbf{p}\in$ ground}}~{},\\\
\text{None},&\text{elif~{}$\hat{p}^{i}<i_{n}$ \text{\&~{}~{}$\mathbf{p}\in$
ground}}~{},\\\ (p^{x},p^{y},p^{z},p^{i}),&\text{elif~{}$\mathbf{p}\notin$
\text{ground}}~{}.\end{cases}$ (4)

3) Snow. For each laser beam in snowy weather, the set of particles in the air
will intersect with it and derive the angle of the beam cross-section that is
reflected by each particle, taking potential occlusions into account [54]. We
follow [22] to simulate snow-corrupted data
$\mathcal{C}_{\text{snow}}(\mathbf{p})$ which is similar to the fog
simulation. This physically-based method samples snow particles in the 2D
space and modify the measurement for each LiDAR beam in accordance with the
induced geometry, where the number of sampling snow particles is set according
to a given snowfall rate $r_{s}$.

4) Motion Blur. Since the LiDAR sensor is often mounted on the roof-top or
side of the vehicle, it inevitably suffers from the blur caused by vehicle
movement, especially on bumpy surfaces or during U-turning. To simulate blur-
corrupted data $\mathcal{C}_{\text{motion}}(\mathbf{p})$, we add a jittering
noise to each coordinate $(p^{x},p^{y},p^{z})$ with a translation value
sampled from the Gaussian distribution with standard deviation $\sigma_{t}$.
This simulation process is shown as follows:

$\mathcal{C}_{\text{motion}}(\mathbf{p})=(p^{x}+o_{1},p^{y}+o_{2},p^{z}+o_{3},p^{i})~{},$
(5)

where $o_{1},o_{2},o_{3}$ are the random offsets sampled from Gaussian
distribution $N\in\\{0,{\sigma_{t}}^{2}\\}$ and
$\\{o_{1},o_{2},o_{3}\\}\in\mathbb{R}^{1\times 1}$.

5) Beam Missing. The dust and insect tend to form agglomerates in front of the
LiDAR surface and will not likely disappear without human intervention, such
as drying and cleaning [48]. This type of occlusion causes zero readings on
masked areas and results in the loss of certain light impulses. To mimic such
a behavior, we randomly sample a total number of $m$ beams and drop points on
these beams from the original point cloud to generate
$\mathcal{C}_{\text{beam}}(\mathbf{p})$:

$\mathcal{C}_{\text{beam}}(\mathbf{p})=\begin{cases}(p^{x},p^{y},p^{z},p^{i}),&\text{if~{}~{}~{}~{}$\mathbf{p}\notin
m$ },\\\ \text{None},&\text{else}~{}.\end{cases}$ (6)

6) Crosstalk. Considering that the road is often shared by multiple vehicles,
the time-of-flight of light impulses from one sensor might interfere with
impulses from other sensors within a similar frequency range [7]. Such a
crosstalk phenomenon often creates noisy points within the mid-range areas in
between two (or multiple) sensors. To simulate this corruption
$\mathcal{C}_{\text{cross}}(\mathbf{p})$, we randomly sample a subset of
$k_{t}$ percent points from the original point cloud and add large jittering
noise with a translation value sampled from the Gaussian distribution with
standard deviation $\sigma_{c}$. This simulation process is shown as follows:

$\mathcal{C}_{\text{cross}}(\mathbf{p})=\begin{cases}(p^{x},p^{y},p^{z},p^{i}),&\text{if~{}~{}$\mathbf{p}\notin$
\text{set of \\{$k_{t}$\\} }},\\\
(p^{x},p^{y},p^{z},p^{i})+\xi_{c},&\text{else}~{},\end{cases}$ (7)

where $\xi_{c}$ is the random offset sampled from Gaussian distribution
$N\in\\{0,{\sigma_{c}}^{2}\\}$ and $\xi_{c}\in\mathbb{R}^{1\times 4}$.

7) Incomplete Echo. The near-infrared spectrum of the laser pulse emitted from
the LiDAR sensor is vulnerable to vehicles or other instances with dark colors
[86]. The LiDAR readings are thus incomplete in such scan echoes, resulting in
significant point miss detection. We simulate this corruption which denotes
$\mathcal{C}_{\text{echo}}(\mathbf{p})$ by randomly querying $k_{e}$ percent
points for vehicle, bicycle, and motorcycle classes, via either semantic masks
or 3D bounding boxes. Next, we drop the queried points from the original point
cloud, along with their point-level semantic labels. Note that we do not alter
the ground-truth bounding boxes since they should remain at their original
positions in the real world. The overall operation can be summarized as
follows:

$\mathcal{C}_{\text{echo}}(\mathbf{p})=\begin{cases}(p^{x},p^{y},p^{z},p^{i}),&\text{if~{}~{}$\mathbf{p}\notin$
\text{set of \\{$k_{e}$\\} }},\\\ \text{None},&\text{else}~{}.\end{cases}$ (8)

8) Cross-Sensor. Due to the large variety of LiDAR sensor configurations
(e.g., beam number, FOV, and sampling frequency), it is important to design
robust 3D perception models that are capable of maintaining satisfactory
performance under cross-device cases [82]. While previous works directly form
such settings with two different datasets, the domain idiosyncrasy in between
(e.g. different label mappings and data collection protocols) further hinders
the direct robustness comparison. In our benchmark, we follow [71] and
generate cross-sensor data $\mathcal{C}_{\text{sensor}}(\mathbf{p})$ by first
dropping points of certain beams from the point cloud and then sub-sample
$k_{c}$ percent points from each beam. This simulation process is shown as
follows:

$\mathcal{C}_{\text{sensor}}(\mathbf{p})=\begin{cases}\text{None},&\text{if~{}~{}$\text{$\mathbf{p}\in$
set of \\{$k_{c}$\\}}$ },\\\
(p^{x},p^{y},p^{z},p^{i}),&\text{else}~{}.\end{cases}$ (9)

###  3.2 Corruption Sets

Following the above taxonomy, we create new robustness evaluation sets upon
the val sets of existing large-scale 3D perception datasets [20, 4, 8, 19, 64]
to fulfill SemanticKITTI-C, KITTI-C, nuScenes-C, and WOD-C. They are
constructed with eight corruption types under three severity levels, resulting
in a total number of 97704, 90456, 144456, and 143424 annotated LiDAR point
clouds, respectively. Kindly refer to the Appendix for more details in terms
of these robustness evaluation collections.

(a) KITTI-C

(b) SemanticKITTI-C

(c) nuScenes-C (Det3D)

(d) nuScenes-C (Seg3D)

(e) WOD-C (Det3D)

(f) WOD-C (Seg3D)

Figure 2: Benchmarking results of 34 LiDAR-based detection and segmentation
models on the six robustness sets in Robo3D. Figures from top to bottom: the
task-specific accuracy (mAP, mIoU, NDS, mAPH) vs. [first row] mean corruption
error (mCE), [second row] mean resilience rate (mRR), and [third row]
sensitivity analysis among different corruption types.

###  3.3 Evaluation Metrics

Corruption Error (CE). We follow [25] and use the mean CE (mCE) as the primary
metric in comparing models’ robustness. To normalize the severity effects, we
choose CenterPoint [85] and MinkUNet [66] as the baseline models for the 3D
detectors and segmentors, respectively. The CE and mCE scores are calculated
as follows:

$\text{CE}_{i}=\frac{\sum^{3}_{l=1}(1-\text{Acc}_{i,l})}{\sum^{3}_{l=1}(1-\text{Acc}_{i,l}^{\text{baseline}})}~{},~{}~{}~{}\text{mCE}=\frac{1}{N}\sum^{N}_{i=1}\text{CE}_{i}~{},$
(10)

where $\text{Acc}_{i,l}$ denotes the task-specific accuracy scores, i.e., mIoU
for LiDAR semantic segmentation, and AP, NDS, or APH(L2) for 3D object
detection, on corruption type $i$ at severity level $l$. $N=8$ is the total
number of corruption types.

Resilience Rate (RR). We define mean RR (mRR) as the relative robustness
indicator for measuring how much accuracy can a model retain when evaluated on
the corruption sets. The RR and mRR scores are calculated as follows.

$\text{RR}_{i}=\frac{\sum^{3}_{l=1}\text{Acc}_{i,l}}{3\times\text{Acc}_{\text{clean}}}~{},~{}~{}~{}~{}~{}\text{mRR}=\frac{1}{N}\sum^{N}_{i=1}\text{RR}_{i}~{},$
(11)

where $\text{Acc}_{\text{clean}}$ denotes the task-specific accuracy score on
the “clean” evaluation set.

Figure 3: The robustness comparisons among different LiDAR representations
(modalities) on SemanticKITTI-C.

##  4 Experimental Analysis

###  4.1 Benchmark Configuration

3D Perception Models. We benchmark 34 LiDAR-based detection and segmentation
models and variants. Detectors: SECOND [80], PointPillars [34] PointRCNN [59],
Part-A2 [60], PV-RCNN [56], CenterPoint [85], and PV-RCNN++ [58]. Segmentors:
SqueezeSeg [72], SqueezeSegV2 [73], RangeNet++ [44], SalsaNext [13], FIDNet
[89], CENet [11], PolarNet [88], KPConv [67], PIDS [87], WaffleIron [49],
MinkUNet [12], Cylinder3D [93], SPVCNN [66], RPVNet [77], CPGNet [38], 2DPASS
[78], and GFNet [50]. We also include three recent 3D augmentation methods,
i.e., Mix3D [46], LaserMix [33], and PolarMix [74].

Evaluation Protocol. Most models benchmarked follow similar data augmentation,
pre-training, and validation configurations. We thus directly use public
checkpoints for evaluation whenever applicable, or re-train the model
following default settings. We notice that some models use extra tricks on
original validation sets, e.g., test-time augmentation, model ensemble, etc.
For such cases, we re-train their models with conventional settings and report
the reproduced results. This is to ensure that the robustness comparisons
across different models on our corruption sets are fair and convincing. Kindly
refer to the Appendix for more details on training and evaluation protocols
and to access the pre-trained model weights for reproduction purposes.

###  4.2 Benchmark Analysis

In this section, we draw the following key observations based on the
benchmarking results and analyze the potential causes behind them.

O-1: 3D Perception Robustness \- existing 3D detectors and segmentors are
vulnerable to real-world corruptions. As shown in Fig. 2, although the models’
corruption errors often correlate with the task-specific accuracy (first row),
their resilience scores are rather flattened or even descending towards
vulnerabilities (second row). The per-corruption errors shown in Tab. 1 to
Tab. 6 further verify such crux. Taking 3D segmentors as an example: although
the very recent state-of-the-art methods [78, 50, 49] have achieved promising
results on the standard benchmark, they are actually less robust than the
baseline, i.e., their mCE scores are higher than MinkUNet [12]. A similar
trend appears for the 3D detectors, e.g. Fig. 2(c), where models with higher
NDS are becoming less resilient. Due to the lack of a suitable robustness
evaluation benchmark, the existing 3D perception models tend to overfit the
“clean” data distributions rather than realistic scenarios.

O-2: Sensor Configurations \- models trained with LiDAR data from different
sources exhibit inconsistent sensitivities to each corruption type. As shown
in the third row of Fig. 2, the same corruption applied on different datasets
shows diverse behaviors on model’s robustness. Different data collection
protocols and sensor setups cause a direct impact on model representation
learning. For example, 3D detectors trained on 64-beam datasets (KITTI, WOD)
are less robust to motion blur and snow, compared to their counterparts
trained on the sparser dataset (nuScenes). We conjecture that the low-density
inputs have incorporated certain resilience for models against noises that
occur locally but might become fragile for scenarios that lose points in a
global manner, i.e., the cross-sensor corruption.

Table 1: The Corruption Error (CE) of 22 segmentors on SemanticKITTI-C. Bold:
Best in col. Underline: Second best in col. Dark: Best in row. Red: Worst in
row.

Method |  mCE $\downarrow$ |  Fog |  Wet |  Snow |  Move |  Beam |  Cross |  Echo |  Sensor
---|---|---|---|---|---|---|---|---|---
MinkU18 [12] |  $100.0$ |  $100.0$ |  $100.0$ |  $100.0$ |  $100.0$ |  $100.0$ |  $100.0$ |  $100.0$ |  $100.0$
SqSeg [72] |  $164.9$ |  $183.9$ |  $158.0$ |  $165.5$ |  Dark$122.4$ |  $171.7$ |  red!9$188.1$ |  $158.7$ |  $170.8$
SqSegV2 [73] |  $152.5$ |  $168.5$ |  $141.2$ |  $154.6$ |  Dark$115.2$ |  $155.2$ |  red!9$176.0$ |  $145.3$ |  $163.5$
RGNet21 [44] |  $136.3$ |  red!9$156.3$ |  $128.5$ |  $133.9$ |  Dark$102.6$ |  $141.6$ |  $148.9$ |  $128.3$ |  $150.6$
RGNet53 [44] |  $130.7$ |  $144.3$ |  $123.7$ |  $128.4$ |  Dark$104.2$ |  $135.5$ |  $129.4$ |  $125.8$ |  red!9$153.9$
SalsaNext [13] |  $116.1$ |  red!9$147.5$ |  $112.1$ |  $116.6$ |  Dark$77.6$ |  $115.3$ |  $143.5$ |  $114.0$ |  $102.5$
FIDNet [89] |  $113.8$ |  $127.7$ |  $105.1$ |  $107.7$ |  Dark$88.9$ |  $116.0$ |  $121.3$ |  $113.7$ |  red!9$130.0$
CENet [11] |  $103.4$ |  $129.8$ |  $92.7$ |  $99.2$ |  Dark$70.5$ |  $101.2$ |  red!9$131.1$ |  $102.3$ |  $100.4$
PolarNet [88] |  $118.6$ |  $138.8$ |  $107.1$ |  $108.3$ |  Dark$86.8$ |  $105.1$ |  $178.1$ |  $112.0$ |  $112.3$
KPConv [67] |  $99.5$ |  $103.2$ |  $91.9$ |  $98.1$ |  $110.7$ |  $97.6$ |  red!9$111.9$ |  $97.3$ |  Dark$85.4$
PIDS1.2× [87] |  $104.1$ |  red!9$118.1$ |  $98.9$ |  $109.5$ |  $114.8$ |  $103.2$ |  $103.9$ |  $97.0$ |  Dark$87.6$
PIDS2.0× [87] |  $101.2$ |  $110.6$ |  $95.7$ |  $104.6$ |  red!9$115.6$ |  $98.6$ |  $102.2$ |  $97.5$ |  Dark$84.8$
Waffle [49] |  $109.5$ |  $123.5$ |  $90.1$ |  $108.5$ |  $99.9$ |  $93.2$ |  red!9$186.1$ |  $\mathbf{91.0}$ |  Dark$84.1$
MinkU34 [12] |  $100.6$ |  $105.3$ |  $99.4$ |  red!9$106.7$ |  $98.7$ |  Dark$97.6$ |  $99.9$ |  $99.0$ |  $98.3$
Cy3D${}_{\text{SPC}}$ [93] |  $103.3$ |  red!9$142.5$ |  $92.5$ |  $113.6$ |  Dark$70.9$ |  $97.0$ |  $105.7$ |  $104.2$ |  $99.7$
Cy3D${}_{\text{TSC}}$ [93] |  $103.1$ |  red!9$142.5$ |  $101.3$ |  $116.9$ |  Dark$61.7$ |  $98.9$ |  $111.4$ |  $99.0$ |  $93.4$
SPV18 [66] |  $100.3$ |  red!9$101.3$ |  $100.0$ |  $104.0$ |  Dark$97.6$ |  $99.2$ |  $100.6$ |  $99.6$ |  $100.2$
SPV34 [66] |  $\mathbf{99.2}$ |  $\mathbf{98.5}$ |  $100.7$ |  red!9$102.0$ |  Dark$97.8$ |  $99.0$ |  $\mathbf{98.4}$ |  $98.8$ |  $98.1$
RPVNet [77] |  $111.7$ |  red!9$118.7$ |  $101.0$ |  $104.6$ |  Dark$78.6$ |  $106.4$ |  $185.7$ |  $99.2$ |  $99.8$
CPGNet [38] |  $107.3$ |  $141.0$ |  $92.6$ |  $104.3$ |  Dark$\mathbf{61.1}$ |  $\mathbf{90.9}$ |  red!9$195.6$ |  $95.0$ |  $\mathbf{78.2}$
2DPASS [78] |  $106.1$ |  $134.9$ |  $\mathbf{85.5}$ |  $110.2$ |  Dark$62.9$ |  $94.4$ |  red!9$171.7$ |  $96.9$ |  $92.7$
GFNet [50] |  $108.7$ |  $131.3$ |  $94.4$ |  $\mathbf{92.7}$ |  Dark$61.7$ |  $98.6$ |  red!9$198.9$ |  $98.2$ |  $93.6$

Table 2: The Corruption Error (CE) of 12 segmentors on nuScenes-C (Seg3D).
Bold: Best in col. Underline: Second best in col. Dark: Best in row. Red:
Worst in row.

Method |  mCE $\downarrow$ |  Fog |  Wet |  Snow |  Move |  Beam |  Cross |  Echo |  Sensor
---|---|---|---|---|---|---|---|---|---
MinkU18 [12] |  $100.0$ |  $100.0$ |  $100.0$ |  $100.0$ |  $100.0$ |  $100.0$ |  $100.0$ |  $100.0$ |  $100.0$
FIDNet [89] |  $122.4$ |  $75.9$ |  $122.6$ |  $68.8$ |  red!9$192.0$ |  $164.8$ |  Dark$58.0$ |  $141.7$ |  $155.6$
CENet [11] |  $112.8$ |  $71.2$ |  $115.5$ |  $64.3$ |  red!9$156.7$ |  $159.0$ |  Dark$53.3$ |  $129.1$ |  $153.4$
PolarNet [88] |  $115.1$ |  $90.1$ |  $115.3$ |  Dark$59.0$ |  red!9$208.2$ |  $121.1$ |  $80.7$ |  $128.2$ |  $118.2$
Waffle [49] |  $106.7$ |  $94.7$ |  $99.9$ |  Dark$84.5$ |  red!9$152.4$ |  $110.7$ |  $91.1$ |  $106.4$ |  $114.2$
MinkU34 [12] |  $96.4$ |  Dark$93.0$ |  $96.1$ |  red!9$104.8$ |  $\mathbf{93.1}$ |  $\mathbf{95.0}$ |  $96.3$ |  $\mathbf{96.9}$ |  $\mathbf{95.9}$
Cy3D${}_{\text{SPC}}$ [93] |  $111.8$ |  $86.6$ |  $104.7$ |  Dark$70.3$ |  red!9$217.5$ |  $113.0$ |  $75.7$ |  $109.2$ |  $117.8$
Cy3D${}_{\text{TSC}}$ [93] |  $105.6$ |  $83.2$ |  $111.1$ |  Dark$69.7$ |  red!9$165.3$ |  $114.0$ |  $74.4$ |  $110.7$ |  $116.2$
SPV18 [66] |  $106.7$ |  $88.4$ |  $105.6$ |  $98.8$ |  red!9$156.5$ |  $110.1$ |  Dark$86.0$ |  $104.3$ |  $103.6$
SPV34 [66] |  $97.5$ |  Dark$95.2$ |  red!9$99.5$ |  $97.3$ |  $95.3$ |  $98.7$ |  $97.9$ |  $\mathbf{96.9}$ |  $98.7$
2DPASS [78] |  $98.6$ |  $76.6$ |  $\mathbf{89.1}$ |  Dark$76.4$ |  red!9$142.7$ |  $102.2$ |  $89.4$ |  $101.8$ |  $110.4$
GFNet [50] |  $\mathbf{92.6}$ |  $\mathbf{65.6}$ |  $93.8$ |  $\mathbf{47.2}$ |  red!9$152.5$ |  $112.9$ |  Dark$\mathbf{45.3}$ |  $105.5$ |  $117.6$

O-3: Data Representations \- representing the LiDAR data as raw points, sparse
voxels, or the fusion of them tend to yield better robustness. It can be
easily seen from Fig. 3 that the corruption errors of projection-based
methods, i.e. range view and BEV, are much higher than other modalities, for
almost every corruption type in the benchmark. Such disadvantages also hold
for fusion-based models that use a 2D branch, e.g., RPVNet [77] and GFNet
[50]. In general, the point-based methods [67, 49, 87] are more robust to
situations where a significant amount of points are missing while suffering
from translation, jittering, and outliers. We conjecture that the sub-sampling
and local aggregation widely used in existing point-based architectures are
natural rescues for point drops and occlusions. Among all representations,
voxel/pillar and point-voxel fusion exhibit a clear superiority under various
corruption types, as verified in Tab. 1, Tab. 2, and Tab. 3, respectively. The
voxelization processes that quantize the irregular points are conducive to
mitigating the local variations and often yield a more steady representation
for feature learning.

Table 3: The Corruption Error (CE) of 5 segmentors on WOD-C (Seg3D). Bold:
Best in col. Underline: Second best in col. Dark: Best in row. Red: Worst in
row.

Method |  mCE $\downarrow$ |  Fog |  Wet |  Snow |  Move |  Beam |  Cross |  Echo |  Sensor
---|---|---|---|---|---|---|---|---|---
MinkU18 [12] |  $100.0$ |  $100.0$ |  $100.0$ |  $100.0$ |  $100.0$ |  $100.0$ |  $100.0$ |  $100.0$ |  $100.0$
MinkU34 [12] |  $\mathbf{96.2}$ |  $\mathbf{96.0}$ |  $94.9$ |  red!9$99.5$ |  $\mathbf{96.2}$ |  $\mathbf{95.4}$ |  $\mathbf{96.8}$ |  $\mathbf{96.8}$ |  Dark$\mathbf{94.1}$
Cy3D${}_{\text{TSC}}$ [93] |  $106.0$ |  red!9$111.8$ |  $104.1$ |  Dark$\mathbf{98.4}$ |  $110.3$ |  $105.8$ |  $106.9$ |  $108.2$ |  $102.6$
SPV18 [66] |  $103.6$ |  red!9$105.6$ |  $104.8$ |  Dark$99.2$ |  $105.4$ |  $104.8$ |  $99.7$ |  $104.3$ |  $104.9$
SPV34 [66] |  $98.7$ |  $99.7$ |  $\mathbf{96.4}$ |  $100.4$ |  $100.0$ |  $98.5$ |  red!9$101.9$ |  $97.9$ |  Dark$95.0$

Table 4: The Corruption Error (CE) of 7 detectors on KITTI-C. Bold: Best in
col. Underline: Second best in col. Dark: Best in row. Red: Worst in row.

Method |  mCE $\downarrow$ |  Fog |  Wet |  Snow |  Move |  Beam |  Cross |  Echo |  Sensor
---|---|---|---|---|---|---|---|---|---
CenterPP [85] |  $100.0$ |  $100.0$ |  $100.0$ |  $100.0$ |  $100.0$ |  $100.0$ |  $100.0$ |  $100.0$ |  $100.0$
SECOND [80] |  $95.9$ |  $99.7$ |  red!9$100.6$ |  Dark$87.6$ |  $97.6$ |  $91.5$ |  $96.5$ |  $99.2$ |  $94.8$
P-Pillars [34] |  $110.7$ |  $115.8$ |  $106.4$ |  red!9$124.9$ |  $101.6$ |  Dark$95.3$ |  $117.6$ |  $109.9$ |  $113.9$
P-RCNN [59] |  $91.9$ |  $93.2$ |  $90.1$ |  $96.8$ |  $93.1$ |  $86.1$ |  red!9$100.9$ |  $92.4$ |  Dark$\mathbf{82.5}$
PartA2-F [60] |  $\mathbf{82.2}$ |  red!9$\mathbf{89.4}$ |  $\mathbf{75.8}$ |  $\mathbf{81.3}$ |  $\mathbf{86.2}$ |  $\mathbf{80.9}$ |  Dark$\mathbf{71.8}$ |  $\mathbf{83.6}$ |  $88.9$
PartA2-A [60] |  $88.6$ |  $92.6$ |  Dark$83.2$ |  red!9$94.6$ |  $86.4$ |  $87.0$ |  Dark$83.2$ |  $89.3$ |  $92.7$
PVRCNN [56] |  $90.0$ |  red!9$95.2$ |  $86.6$ |  $93.1$ |  $87.5$ |  Dark$86.0$ |  $87.1$ |  $90.0$ |  $94.7$

(a) Voxel Size on SemanticKITTI-C (Seg3D)

(b) Augmentation on SemanticKITTI-C (Seg3D)

(c) Voxel Size on WOD-C (Det3D)

(d) Augmentation on WOD-C (Det3D)

Figure 4: Corruption sensitivity analysis on voxel size (a & c) and
augmentation (b & d) for the baseline LiDAR semantic segmentation and 3D
object detection models [12, 85]. Different corruptions exhibit variances
under certain configurations.

O-4: Task Particularity \- The 3D detectors and segmentors show different
sensitivities to corruption scenarios. The detection task only targets
classification and localization at the object level; corruptions that occur at
points inside an instance range have less impact on detecting the object.
However, the segmentation task is to identify the semantic meaning of each
point in the point cloud. Such a task discrepancy is affecting the model’s
robustness across different corruptions. From Fig. 2 we find that 3D detectors
tend to be more robust to point-level variations, such as motion blur and
crosstalk. These two corruptions likely yield noise offsets that are out of
the grid size; while these point translations could easily be misclassified by
the 3D segmentation models. On the contrary, the 3D segmentors are more steady
to environmental changes like fog, wet ground, and snow. From hindsight, we
believe that a sophisticated combination of the detection and segmentation
tasks would be a viable solution for building more robust and reliable 3D
perception systems against different corruptions.

Table 5: The Corruption Error (CE) of 5 detectors on nuScenes-C (Det3D). Bold:
Best in col. Underline: Second best in col. Dark: Best in row. Red: Worst in
row.

Method |  mCE $\downarrow$ |  Fog |  Wet |  Snow |  Move |  Beam |  Cross |  Echo |  Sensor
---|---|---|---|---|---|---|---|---|---
CenterPP [85] |  $100.0$ |  $100.0$ |  $100.0$ |  $100.0$ |  $100.0$ |  $100.0$ |  $100.0$ |  $100.0$ |  $100.0$
SECOND [80] |  $97.5$ |  $95.4$ |  $96.0$ |  $96.1$ |  $100.8$ |  $99.3$ |  Dark$92.2$ |  $97.6$ |  red!9$102.6$
P-Pillars [34] |  $102.9$ |  $102.9$ |  $104.6$ |  $102.5$ |  red!9$106.4$ |  $102.4$ |  Dark$100.9$ |  $102.4$ |  $\mathbf{101.1}$
CenterLR [85] |  $98.7$ |  $97.9$ |  $96.5$ |  $97.7$ |  $102.2$ |  $101.1$ |  Dark$95.5$ |  $\mathbf{95.6}$ |  red!9$103.5$
CenterHR [85] |  $\mathbf{95.8}$ |  $\mathbf{93.0}$ |  $\mathbf{92.0}$ |  $\mathbf{94.9}$ |  $\mathbf{97.6}$ |  $\mathbf{98.4}$ |  Dark$\mathbf{91.1}$ |  $96.2$ |  red!9$103.2$

Table 6: The Corruption Error (CE) of 5 detectors on WOD-C (Det3D). Bold: Best
in col. Underline: Second best in col. Dark: Best in row. Red: Worst in row.

Method |  mCE $\downarrow$ |  Fog |  Wet |  Snow |  Move |  Beam |  Cross |  Echo |  Sensor
---|---|---|---|---|---|---|---|---|---
CenterPP [85] |  $100.0$ |  $100.0$ |  $100.0$ |  $100.0$ |  $100.0$ |  $100.0$ |  $100.0$ |  $100.0$ |  $100.0$
SECOND [80] |  $121.4$ |  $117.9$ |  $126.5$ |  $127.5$ |  Dark$113.4$ |  $121.3$ |  red!9$127.8$ |  $123.7$ |  $113.5$
P-Pillars [34] |  $127.5$ |  $120.8$ |  $135.2$ |  $129.7$ |  $115.2$ |  $123.0$ |  red!9$151.7$ |  $131.6$ |  Dark$113.1$
PVRCNN [56] |  $104.9$ |  $110.1$ |  $104.2$ |  Dark$95.7$ |  $101.3$ |  red!9$110.7$ |  $101.8$ |  $106.0$ |  $109.4$
PV++ [58] |  $\mathbf{91.6}$ |  red!9$\mathbf{95.7}$ |  Dark$\mathbf{88.3}$ |  $\mathbf{90.1}$ |  $\mathbf{93.2}$ |  $\mathbf{92.5}$ |  $\mathbf{88.9}$ |  $\mathbf{90.8}$ |  $\mathbf{93.2}$

Figure 5: The proposed density-insensitive training framework. The “full” and
“partial” point clouds are fed into the teacher branch and student branch,
respectively, for feature learning, while the latter is generated by randomly
masking the original point cloud. To encourage cross-density consistency, we
calculate the completion and confirmation losses which measure the distances
of sub-sampled teacher’s prediction and interpolated student’s prediction
between the other branch’s outputs.

O-5: Augmentation & Regularization Effects \- The recent out-of-context
augmentation (OCA) techniques improve 3D robustness by large margins; the
flexible rasterization strategies help learn more robust features. The in-
context augmentations (ICAs), i.e., flip, scale, and rotation, are commonly
used in 3D detectors and segmentors. Although these techniques help boost
perception accuracy, they are less effective in improving robustness. Recent
works [46, 33, 74] proposed OCAs with the goal of further enhancing model
performance on the “clean” sets. We implement these augmentations on baseline
models and test their effectiveness on our robustness evaluation sets, as
shown in Fig. 4 (b) & (d). Since corrupted data often deviate from the
training distribution, the model will inevitably degrade under OoD scenarios.
OCAs that mix and swap regions without maintaining the consistency of scene
layouts are yielding much lower CE scores across all corruptions, except for
wet ground, where the loss of ground points restricts the effectiveness of
scene mixing. Another key factor that influences the robustness (especially
for voxel- and point-voxel fusion-based methods) is representation capacity,
i.e., voxel size. As shown in Fig. 4 (a) & (c), the 3D segmentors under
translations within small regions (motion blur) favor a larger voxel size to
suppress the global translations; conversely, they are more robust against
outliers (fog, snow, and crosstalk) given more fine-grained voxelizations to
eliminate the local variations. For 3D detectors, a consensus is formed toward
using a higher voxelization resolution, and improvements are constantly
achieved across all corruption types in the benchmark.

##  5 Boosting Corruption Robustness

Motivated by the above observations, we propose two novel techniques to
enhance the robustness against corruptions. We conduct experiments on the
SemanticKITTI-C dataset without loss of generality and include more results on
other datasets in the Appendix.

Flexible Voxelization. The widely used sparse convolution [65] requires the
formal transformation of the point coordinates
$\mathbf{p}_{k}=(p^{x}_{k},p^{y}_{k},p^{z}_{k})$ into a sparse voxel. This
process is often formulated as follows:

$\mathbf{v}_{k}=(v^{x}_{k},v^{y}_{k},v^{z}_{k})=\texttt{floor}((\frac{p^{x}_{k}}{l^{x}}),(\frac{p^{y}_{k}}{l^{y}}),(\frac{p^{z}_{k}}{l^{z}}))~{},~{}$
(12)

where $l^{x}$, $l^{y}$, and $l^{z}$ denote the voxel size along each axis and
are often set as fixed values. As discussed in Fig. 4 (a) & (c), the model
tends to show an erratic resilience under different corruptions, e.g., favor a
larger voxel size for motion blur while is more robust against fog, snow, and
crosstalk with a smaller voxel size. To pursue better generalizability among
all corruptions, we switch the naive constant into a dynamic alternative
$l_{\text{dv}}=(l^{x}\pm\text{dv}^{x},l^{y}\pm\text{dv}^{y},l^{z}\pm\text{dv}^{z})$,
where $\text{dv}^{x}$, $\text{dv}^{y}$, $\text{dv}^{z}$ are the offsets
sampled from the continuous uniform distribution with an interval $\gamma$.

Figure 6: Ablation study on the masking ratio $\beta$ for models: [top]
trained w/ OCA and [bottom] trained w/ ICA.

Density-Insensitive Training. The natural corruptions often cause severe
occlusion, attenuation, and reflection of light impulses, resulting in the
unavoidable loss of LiDAR points in certain regions around the ego-vehicle
[62]. For example, the wet ground absorbs energy and loses points on the
surfaces [22]; the potential incomplete echo and beam missing caused by
reflection or dust and insects occlusion may lead to serious object failure
[86]. The 3D perception models that suffer from such OoD scenarios bear the
risk of being involved in safety-critical issues. It is worth noting that such
degradation is not compensable via either adjusting the voxel size or applying
OCA (see Fig. 4). Inspired by recent masking-based representation methods [24,
16, 45, 26], we propose a robust finetuning framework (see Fig. 5) that tends
to be less sensitive to density variations. Specifically, we design a two-
branch structure – a teacher net $\mathcal{G}_{\theta}^{\text{tea}}$ and a
student net $\mathcal{G}_{\theta}^{\text{stu}}$ – that takes a pair of high-
and low-density point clouds ($x$ and $\tilde{x}$) as the input, where the
sparser one is generated by randomly masking the points from the original
point cloud with a ratio $\beta$. Note that here we use the random mask to
sub-sample the given point clouds rather than simulating a specific corruption
type defined in our benchmark, since the corruption “pattern” in the actual
scenario is often hard to predict. The loss functions of the $k$-th sample
from the “full” view and the “partial” view are calculated as follows:

$\mathcal{L}_{\text{full}}=\mathcal{L}_{\text{task}}(y_{k},\mathcal{G}_{\theta}^{\text{tea}}(x_{k}))~{},~{}~{}\mathcal{L}_{\text{part}}=\mathcal{L}_{\text{task}}(\tilde{y}_{k},\mathcal{G}_{\theta}^{\text{stu}}(\tilde{x}_{k})),$
(13)

where $y_{k}$ and $\tilde{y}_{k}$ are original and masked ground-truths,
respectively. $\mathcal{L}_{\text{task}}$ denotes the task-specific loss,
e.g., RPN loss for detection and cross-entropy loss for segmentation.

To encourage cross-consistency between the high- and low-density branches, we
calculate $\mathcal{L}_{\text{part2full}}$ and
$\mathcal{L}_{\text{full2part}}$, where the former is to mimic dense
representations from sparse inputs (completion) and the latter is to pursue
local agreements (confirmation). The completion loss is calculated as the
distance between the teacher net’s prediction of the “full” input and the
interpolated student net’s prediction of the “partial” input, which can be
calculated as follows:

$\mathcal{L}_{\text{part2full}}=||~{}\mathcal{G}_{\theta}^{\text{tea}}(x),~{}\texttt{interp}(\mathcal{G}_{\theta}^{\text{stu}}(\tilde{x}))~{}||^{2}_{2}~{}.~{}$
(14)

Similarly, the confirmation loss for pursuing local agreements can be
calculated as follows:

$\mathcal{L}_{\text{full2part}}=||~{}\texttt{subsample}(\mathcal{G}_{\theta}^{\text{tea}}(x)),~{}\mathcal{G}_{\theta}^{\text{stu}}(\tilde{x})~{}||^{2}_{2}~{}.~{}$
(15)

The final objective is to optimize the summation of the above loss functions,
i.e.,
$\mathcal{L}=\mathcal{L}_{\text{full}}+\mathcal{L}_{\text{part}}+\alpha_{1}\mathcal{L}_{\text{part2full}}+\alpha_{2}\mathcal{L}_{\text{full2part}}$,
where $\alpha_{1}$ and $\alpha_{2}$ are the weight coefficients.

Implementation Details. We ablate each component and show the results in Tab.
7. Specifically, $\gamma$ is set as $0.02$ in our experiments, along with a
mask ratio $\beta=0.4$ for models w/ ICA and $\beta=0.6$ for models w/ OCA. We
initialize both teacher and student networks with the same baseline model and
finetune our framework for $6$ epochs in total. The weight coefficients are
set as $50$ and $100$, respectively.

Experimental Analysis. Despite its simplicity, we found this framework is
conducive to mitigating robustness degradation from corruptions. The simple
modification on voxel partition can boost the corruption robustness by large
margins; it reduces $2.6\%$ mCE and $1.5\%$ mCE upon the two baselines,
respectively. Then, we incorporate the cross-consistency learning between
“full” and “partial” views. Among all variants, the one with both completion
($\mathcal{L}_{\text{part2full}}$) and confirmation
($\mathcal{L}_{\text{full2part}}$) objectives achieves the best possible
results in terms of mCE and mRR.

We also show an ablation study of the masking ratio $\beta$ in Fig. 6. We
observe that there is often a trade-off between the model’s robustness and the
proportion of information occlusion; a ratio between $0.3$ to $0.6$ tends to
yield lower mCE (better robustness). It is worth noting that both flexible
voxelization and density-insensitive training will slightly lower the task-
specific accuracy on the “clean” sets, as shown in the last column of Tab. 7.
We conjecture that such an out-of-context consistency regularization will
likely relieve the 3D perception model from overfitting the training
distribution and in return, become more robust against unseen scenarios from
the OoD distribution.

Table 7: Ablation study on: [left] in-context (ICA) and out-of-context (OCA)
augmentations; [middle] voxelization strategies; and [right] density-
insensitive training losses.

Method |  ICA |  OCA | Size | $\mathcal{L}_{\text{part2full}}$ | $\mathcal{L}_{\text{full2part}}$ |  mCE $\downarrow$ |  mRR $\uparrow$ |  Clean
---|---|---|---|---|---|---|---|---
red!9Base [12] |  red!9✓ |  red!9 |  red!9Fixed |  red!9 |  red!9 |  red!9$100.0$ |  red!9$81.9$ |  red!9$62.8$
Ours - (1) |  ✓ |  | Flexible |  |  |  $97.4$ |  $84.2$ |  $\mathbf{62.9}$
Ours - (2) |  ✓ |  | Flexible | ✓ |  |  $96.4$ |  $85.1$ |  $62.7$
Ours - (3) |  ✓ |  | Flexible | ✓ | ✓ |  $\mathbf{96.1}$ |  $\mathbf{85.6}$ |  $62.7$
red!9Base [12] |  red!9 |  red!9✓ |  red!9Fixed |  red!9 |  red!9 |  red!9$86.0$ |  red!9$84.7$ |  red!9$\mathbf{69.2}$
Ours - (4) |  |  ✓ | Flexible |  |  |  $84.5$ |  $86.8$ |  $68.2$
Ours - (5) |  |  ✓ | Flexible | ✓ |  |  $83.8$ |  $88.1$ |  $67.9$
Ours - (6) |  |  ✓ | Flexible | ✓ | ✓ |  $\mathbf{83.2}$ |  $\mathbf{89.7}$ |  $68.1$

##  6 Discussion and Conclusion

In this work, we established a comprehensive evaluation benchmark dubbed
Robo3D for probing and analyzing the robustness of LiDAR-based 3D perception
models. We defined eight distinct corruption types with three severity levels
on four large-scale autonomous driving datasets. We systematically benchmarked
and analyzed representative 3D detectors and segmentors to understand their
resilience under real-world corruptions and sensor failure. Several key
insights are drawn from aspects including sensor setups, data representations,
task particularity, and augmentation effects. To pursue better robustness, we
proposed a cross-density consistency training framework and a simple yet
effective flexible voxelization strategy. We hope this work could lay a solid
foundation for future research on building more robust and reliable 3D
perception models.

Potential Limitation. Although we benchmarked a wide range of corruptions that
occur in the real world, we do not consider cases that are coupled with
multiple corruptions at the same time. Besides, we do not include models that
take multi-modal inputs, which could form future directions.

Acknowledgements. This research is part of the programme DesCartes and is
supported by the National Research Foundation, Prime Minister’s Office,
Singapore under its Campus for Research Excellence and Technological
Enterprise (CREATE) programme. This study is supported by the Ministry of
Education, Singapore, under its MOE AcRF Tier 2 (MOE-T2EP20221-0012), NTU NAP,
and under the RIE2020 Industry Alignment Fund – Industry Collaboration
Projects (IAF-ICP) Funding Initiative, as well as cash and in-kind
contribution from the industry partner(s). This study is also supported by the
National Key R&D Program of China (No. 2022ZD0161600).

## Appendix

In this appendix, we supplement more materials to support the findings and
conclusions in the main body of this paper. Specifically, this appendix is
organized as follows.

  * •

Sec. 7 provides a comprehensive case study for analyzing each of the eight
corruption types defined in the Robo3D benchmark.

  * •

Sec. 8 elaborates on additional implementation details for the generation of
each corruption type.

  * •

Sec. 9 includes additional (complete) experimental results and discussions for
the 3D detectors and segmentors benchmarked in Robo3D.

  * •

Sec. 10 attaches qualitative results for the benchmarked methods under each
corruption type.

  * •

Sec. 11 acknowledges the public resources used during the course of this work.

##  7 Case Study: 3D Natural Corruption

The deployment environment of an autonomous driving system is diverse and
complicated; any disturbances that occur in the sensing, transmission, or
processing stages will cause severe corruptions. In this section, we provide
concrete examples of the formation and effect of the eight corruption types
defined in the main body of this paper, i.e., fog, wet ground, snow, motion
blur, beam missing, crosstalk, incomplete echo, and cross-sensor.

Similar to the main body, we denote a point in a LiDAR point cloud as
$\mathbf{p}\in\mathbb{R}^{4}$, which is defined by the point coordinates
$(p^{x},p^{y},p^{z})$ and point intensity $p^{i}$. We aim to simulate a
corrupted point $\mathbf{\hat{p}}$ via a mapping
$\mathbf{\hat{p}}=\mathcal{C}(\mathbf{p})$, with rules constrained by physical
principles or engineering experiences. The detailed case study for each
corruption type defined in the Robo3D benchmark is illustrated as follows.

###  7.1 Fog

The weather phenomena are inevitable in driving scenarios [51]. Among them,
foggy weather mainly causes back-scattering and attenuation of LiDAR pulse
transmissions and results in severe shifts of both range and intensity for the
points in a LiDAR point cloud, as shown in Fig. 7.

In this work, we follow Hahner et al. [23] to generate physically accurate
fog-corrupted data using “clean” datasets. This approach uses a standard
linear system [51] to model the light pulse transmission under foggy weather.
For each $\mathbf{p}$, we calculate its attenuated response
$p^{i_{\text{hard}}}$ and the maximum fog response $p^{i_{\text{soft}}}$ as
follows:

$p^{i_{\text{hard}}}=p^{i}e^{-2\alpha\sqrt{(p^{x})^{2}+(p^{y})^{2}+(p^{z})^{2}}},$
(16)
$p^{i_{\text{soft}}}=p^{i}\frac{(p^{x})^{2}+(p^{y})^{2}+(p^{z})^{2}}{\beta_{0}}\beta\times{p^{i}}_{tmp}$
(17)
$\mathbf{\hat{p}}=\mathcal{C}_{\text{fog}}(\mathbf{p})=\begin{cases}(\hat{p}^{x},\hat{p}^{y},\hat{p}^{z},p^{i_{\text{soft}}}),&\text{if~{}~{}$p^{i_{\text{soft}}}>p^{i_{\text{hard}}}$}\text{,}\\\
(p^{x},p^{y},p^{z},p^{i_{\text{hard}}}),&\text{else}.\end{cases}$ (18)

where $\alpha$ is the attenuation coefficient, $\beta$ denotes the back-
scattering coefficient, $\beta_{0}$ describes the differential reflectivity of
the target, and the ${p^{i}}_{tmp}$ is the received response for the soft
target term.

###  7.2 Wet Ground

As introduced in the main body, the emitted laser pulses from the LiDAR sensor
tend to lose certain amounts of energy when hitting wet surfaces, which will
cause significantly attenuated laser echoes depending on the water height
$d_{w}$ and mirror refraction rate [22], as shown in Fig. 8.

In this work, we follow [22] to model the attenuation caused by ground
wetness. A pre-processing step is taken to estimate the ground plane with
existing semantic labels or RANSAC [18]. Next, a ground plane point of its
measured intensity $\hat{p}^{i}$ is obtained based on the modified
reflectivity, and the point is only kept if its intensity is greater than the
noise floor $i_{n}$ via mapping:

$\mathcal{C}_{\text{wet}}(\mathbf{p})=\begin{cases}(p^{x},p^{y},p^{z},\hat{p}^{i}),&\text{if~{}~{}~{}~{}$\hat{p}^{i}>i_{n}$
\text{\&~{}~{}$\mathbf{p}\in$ ground}}~{},\\\
\text{None},&\text{elif~{}$\hat{p}^{i}<i_{n}$ \text{\&~{}~{}$\mathbf{p}\in$
ground}}~{},\\\ (p^{x},p^{y},p^{z},p^{i}),&\text{elif~{}$\mathbf{p}\notin$
\text{ground~{}. }}\end{cases}$ (19)

Figure 7: Examples of the data corruptions introduced by fog, where the range
(bottom left) and intensity (bottom right) distributions are shifted from the
uniform distribution of the ego-vehicle. Image credit: Hahner et al. [23].
Figure 8: An example of the geometrical optical model of the light pulse
reflection in the wet ground corruption. Depending on the water height and
mirror refraction rate, the pulses emitted by the LiDAR sensor will lose
certain amounts of energy when hitting wet surfaces. Image credit: Hahner et
al. [22]. Figure 9: Examples of the data corruptions introduced by snow. As
shown in the top-left, the particles brought by snowfall will likely cause
false predictions for the objects in the 3D scene. Image credit: Hahner et al.
[22]. Figure 10: Typical range measurement behaviors that will likely cause
beam missing. (a) Echoes return from the target (“clean” scenarios). (b)
Echoes return from a dusty cloud between the sensor and the target (partial
beam missing). (c) No echo returns from either the dusty cloud or the target
(complete beam missing). Image credit: Phillips et al. [48]. Figure 11: An
Illustration of potential crosstalk scenarios in a multi-LiDAR system. [Left]
The basic principle of range detection in the LiDAR sensing cycle. [Middle]
The direct crosstalk scenario in a dual-LiDAR system. [Right] The indirect
crosstalk scenario caused by reflection in a dual-LiDAR system. Image credit:
Diehm et al. [15].

###  7.3 Snow

Snow weather is another adverse weather condition that tends to happen in the
real-world environment. For each laser beam in snowy weather, the set of
particles in the air will intersect with it and derive the angle of the beam
cross-section that is reflected by each particle, taking potential occlusions
into account [54]. Some typical examples of the snow-corrupted data are shown
in Fig. 9.

In this work, we follow [22] to simulate these snow-corrupted data
$\mathcal{C}_{\text{snow}}(\mathbf{p})$, which is similar to the fog
simulation. This physically-based method samples snow particles in the 2D
space and modify the measurement for each LiDAR beam in accordance with the
induced geometry, where the number of sampling snow particles is set according
to a given snowfall rate $r_{s}$.

Figure 12: Examples of the effect brought by motion blur on the registration
of a square room. The blue trajectory denotes the globally consistent map of
the environment; the yellow points were acquired while the LiDAR sensor was
moving from pose $a$ to pose $b$, resulting in a heavily skewed point cloud.
Image credit: Deschênes et al. [14].

###  7.4 Motion Blur

As one of the common in-vehicle sensors, LiDAR is often mounted on the rooftop
or side of the vehicle and inevitably suffers from the blur caused by vehicle
movement, especially on bumpy surfaces or during U-turning. A typical example
of the effect brought by motion blur is shown in Fig. 12.

In this work, to simulate blur-corrupted data
$\mathcal{C}_{\text{motion}}(\mathbf{p})$, we add a jittering noise to each
coordinate $(p^{x},p^{y},p^{z})$ with a translation value sampled from the
Gaussian distribution with standard deviation $\sigma_{t}$. The
$\mathcal{C}_{\text{motion}}(\mathbf{p})$ is shown as:

$\mathcal{C}_{\text{motion}}(\mathbf{p})=(p^{x}+o_{1},p^{y}+o_{2},p^{z}+o_{3},p^{i})~{},$
(20)

where $o_{1},o_{2},o_{3}$ are the random offsets sampled from Gaussian
distribution $N\in\\{0,{\sigma_{t}}^{2}\\}$ and
$\\{o_{1},o_{2},o_{3}\\}\in\mathbb{R}^{1\times 1}$.

###  7.5 Beam Missing

As shown in Fig. 10, the dust and insect tend to form agglomerates in front of
the LiDAR surface and will not likely disappear without human intervention,
such as drying and cleaning [48]. This type of occlusion causes zero readings
on masked areas and results in the loss of certain light impulses.

In this work, to mimic such a behavior, we randomly sample a total number of
$m$ beams and drop points on these beams from the original point cloud to
generate $\mathcal{C}_{\text{beam}}(\mathbf{p})$:

$\mathcal{C}_{\text{beam}}(\mathbf{p})=\begin{cases}(p^{x},p^{y},p^{z},p^{i}),&\text{if~{}~{}~{}~{}$\mathbf{p}\notin
m$ },\\\ \text{None},&\text{else}~{}.\end{cases}$ (21)

###  7.6 Crosstalk

Considering that the road is often shared by multiple vehicles (see Fig. 11),
the time-of-flight of light impulses from one sensor might interfere with
impulses from other sensors within a similar frequency range [7]. Such a
crosstalk phenomenon often creates noisy points within the mid-range areas in
between two (or multiple) sensors. Fig. 13 shows two real-world examples of
crosstalk-corrupted point clouds.

In this work, to simulate $\mathcal{C}_{\text{cross}}(\mathbf{p})$, we
randomly sample a subset of $k_{t}$ percent points from the original point
cloud and add large jittering noise with a translation value sampled from the
Gaussian distribution with standard deviation $\sigma_{c}$.

$\mathcal{C}_{\text{cross}}(\mathbf{p})=\begin{cases}(p^{x},p^{y},p^{z},p^{i}),&\text{if~{}~{}$\mathbf{p}\notin$
\text{set of \\{$k_{t}$\\} }},\\\
(p^{x},p^{y},p^{z},p^{i})+\xi_{c},&\text{else}~{},\end{cases}$ (22)

where $\xi_{c}$ is the random offset sampled from Gaussian distribution
$N\in\\{0,{\sigma_{c}}^{2}\\}$ and $\xi_{c}\in\mathbb{R}^{1\times 4}$.

Figure 13: Examples of the data corruptions introduced by crosstalk. The point
clouds are acquired by a Velodyne HDL-64 with interference from another sensor
of the same type in close vicinity. The crosstalk points are shown in blue.
Image credit: Diehm et al. [15].

###  7.7 Incomplete Echo

The near-infrared spectrum of the laser pulse emitted from the LiDAR sensor is
vulnerable to vehicles or other instances with dark colors [86]. The LiDAR
readings are thus incomplete in such scan echoes, resulting in significant
point miss detection (see Fig. 14 for a real-world example).

In this work, we simulate this corruption which denotes
$\mathcal{C}_{\text{echo}}(\mathbf{p})$ by randomly querying $k_{e}$ percent
points for vehicle, bicycle, and motorcycle classes, via either semantic masks
or 3D bounding boxes. Next, we drop the queried points from the original point
cloud, along with their point-level semantic labels. Note that we do not alter
the ground-truth bounding boxes since they should remain at their original
positions in the real world. This can be formed as:

$\mathcal{C}_{\text{echo}}(\mathbf{p})=\begin{cases}(p^{x},p^{y},p^{z},p^{i}),&\text{if~{}~{}$\mathbf{p}\notin$
\text{set of \\{$k_{e}$\\} }},\\\ \text{None},&\text{else}~{}.\end{cases}$
(23)

Figure 14: Examples of the data corruptions introduced by imcomplete echo. The
black car on the left has nearly zero pulse return, due to the destroyed echo
cycle. Image credit: Yu et al. [86].

###  7.8 Cross-Sensor

A typical cross-sensor example is shown in Fig. 15. Due to the large variety
of LiDAR sensor configurations (e.g., beam number, FOV, and sampling
frequency), it is important to design robust 3D perception models that are
capable of maintaining satisfactory performance under cross-device cases [82].
While previous works directly form such settings with two different datasets,
the domain idiosyncrasy in between (e.g., different label mappings and data
collection protocols) further hinders the direct robustness comparison.

In our benchmark, we follow [71] and generate cross-sensor data
$\mathcal{C}_{\text{sensor}}(\mathbf{p})$ by first dropping points of certain
beams from the point cloud and then sub-sample $k_{c}$ percent points from
each beam:

$\mathcal{C}_{\text{sensor}}(\mathbf{p})=\begin{cases}\text{None},&\text{if~{}~{}$\text{$\mathbf{p}\in$
set of \\{$k_{c}$\\}}$ },\\\
(p^{x},p^{y},p^{z},p^{i}),&\text{else}~{}.\end{cases}$ (24)

Figure 15: Examples of the data distribution discrepancy brought by cross-
sensor effect. (a) A typical point cloud acquired by a 64-beam LiDAR sensor.
(b) A simulated point cloud from 64 beams to 32 beams. (c) A typical point
cloud acquired by a 32-beam LiDAR sensor. Image credit: Wei et al. [71].

##  8 Additional Implementation Detail

In this section, we provide additional implementation details to enable the
reproduction of the corruption generations in the Robo3D benchmark. Note that
our physically principled corruption creation procedures can also be used on
other LiDAR-based point cloud datasets with minimal modifications.

###  8.1 Fog Simulation

Following [23], we uniformly sample the attenuation coefficient $\alpha$ from
$[0,0.005,0.01,0.02,0.03,0.06]$. For the SemanticKITTI-C, KITTI-C, nuScenes-C,
and WOD-C datasets, we set the back-scattering coefficient $\beta$ to
$\\{0.008,0.05,0.2\\}$ to split severity levels into light, moderate, and
heavy levels. The semantic classes of fog are $21$, $41$, and $23$ for
SemanticKITTI-C, nuScenes-C, and WOD-C, respectively. And
$\mathbf{p}\text{~{}belongs to fog class}$ will be mapped to class 0 or 255
(i.e., the ignored label).

###  8.2 Wet Ground Simulation

We follow [22] and set the parameter of water height $d_{w}$ to
$\\{0.2~{}mm,1.0~{}mm,1.2~{}mm\\}$ for different severity levels of wet
ground. Note that the ground plane estimation method is different across four
benchmarks. We estimate the ground plane via RANDSAC [18] for the KITTI-C
since it only provides detection labels. For SemanticKITTI-C, we use semantic
classes of road, parking, sidewalk, and other ground to build the ground
plane. The driveable surface, other flat, and sidewalk classes are used to
construct the ground plane in nuScenes-C. For WOD-C, the ground plane is
estimated by curb, road, other ground, walkable, and sidewalk classes.

###  8.3 Snow Simulation

We use the method proposed in [22] to construct snow corruptions. The value of
snowfall rate parameter $r_{s}$ is set to $\\{0.5,1.0,2.5\\}$ to simulate
light, moderate, and heavy snowfall for the SemanticKITTI-C, KITTI-C,
nuScenes-C, and WOD-C datasets, and the ground plane estimation is the same as
the wet ground simulation. The semantic class of snow is $22$, $42$, and $24$
for the SemanticKITTI-C, nuScenes-C, and WOD-C datasets, respectively. And
$\mathbf{p}\text{~{}belongs to snow class}$ will also be mapped to class 0 or
255 (i.e., the ignored label).

###  8.4 Motion Blur Simulation

We add jittering noise from Gaussian distribution with standard deviation
$\sigma_{t}$ to simulate motion blur. The $\sigma_{t}$ is set to
$\\{0.20,0.25,0.30\\}$, $\\{0.04,0.08,0.10\\}$, $\\{0.20,0.30,0.40\\}$ and
$\\{0.06,0.10,0.13\\}$ for the SemanticKITTI-C, KITTI-C, nuScenes-C, and WOD-C
datasets, respectively.

###  8.5 Beam Missing Simulation

The value of parameter $m$ (number of beams to be dropped) is set to
$\\{48,32,16\\}$ for the benchmark of SemanticKITTI-C, KITTI-C and WOD-C,
respectively, while set as $\\{24,16,8\\}$ for the nuScenes-C dataset.

###  8.6 Crosstalk Simulation

We set the parameter of $k_{t}$ to $\\{0.006,0.008,0.01\\}$ for the
SemanticKITTI-C, KITTI-C, and WOD-C datasets, respectively, and
$\\{0.03,0.07,0.12\\}$ for nuScenes-C dataset. The semantic class of crosstalk
is assigned to $23$, $43$, and $25$ for SemanticKITTI-C, nuScenes-C, and WOD-C
datasets, respectively. Meanwhile, the $\mathbf{p}\text{~{}belongs to
crosstalk class}$ will also be mapped to class 0 or 255 (i.e., the ignored
label).

###  8.7 Incomplete Echo Simulation

For SemanticKITTI-C, the point labels of classes car, bicycle, motorcycle,
truck, other-vehicle are used as the semantic mask. For nuScenes-C, we include
bicycle, bus, car, construction vehicle, motorcycle, truck and trailer class
label to build semantic mask. For WOD-C, we adopt the point labels of classes
car, truck, bus, other-vehicle, bicycle, motorcycle as the semantic mask. For
KITTI-C, we use 3D bounding box labels to create the semantic mask. The value
of parameter $k_{e}$ is set to $\\{0.75,0.85,0.95\\}$ for the four corruption
sets during the incomplete echo simulation.

###  8.8 Cross-Sensor Simulation

The value of parameter $m$ is set to $\\{48,32,16\\}$ for the SemanticKITTI-C,
KITTI-C, and WOD-C datasets, respectively, and $\\{24,16,12\\}$ for the
nuScenes-C dataset. Based on [71], we then sub-sample 50$\%$ points from the
remaining point clouds with an equal interval.

Figure 16: The 3D object detection realization of the proposed density-
insensitive training framework. The “full” and “partial” point clouds are fed
into the teacher branch and student branch, respectively, for feature
learning, while the latter is generated by randomly masking the original point
cloud. To encourage cross-density consistency, we calculate the completion and
confirmation losses which measure the distances of the teacher’s prediction
(BEV feature map) and the student’s prediction (BEV feature map) between the
other branch’s outputs. Figure 17: The 3D semantic segmentation realization of
the proposed density-insensitive training framework. The “full” and “partial”
point clouds are fed into the teacher branch and student branch, respectively,
for feature learning, while the latter is generated by randomly masking the
original point cloud. To encourage cross-density consistency, we calculate the
completion and confirmation losses which measure the distances of the sub-
sampled teacher’s prediction and interpolated student’s prediction between the
other branch’s outputs.

##  9 Additional Experimental Result

In this section, we provide the complete experimental results for each of the
3D detectors and segmentors benchmarked in Robo3D.

###  9.1 SemanticKITTI-C

The complete results in terms of corruption error (CE), resilience rate (RR),
and task-specific accuracy (IoU) on the SemanticKITTI-C dataset are shown in
Tab. 8, Tab. 9, and Tab. 10, respectively.

###  9.2 KITTI-C

The complete results in terms of corruption error (CE), resilience rate (RR),
and task-specific accuracy (AP) on the KITTI-C dataset are shown in Tab. 11,
Tab. 12, and Tab. 13, respectively.

###  9.3 nuScenes-C (Seg3D)

The complete results in terms of corruption error (CE), resilience rate (RR),
and task-specific accuracy (IoU) on the nuScenes-C (Seg3D) dataset are shown
in Tab. 14, Tab. 15, and Tab. 16, respectively. In addition to the benchmark
results, we also show the voxel size analysis results of nuScenes-C (Seg3D) in
Fig. 18 (a).

###  9.4 nuScenes-C (Det3D)

The complete results in terms of corruption error (CE), resilience rate (RR),
and task-specific accuracy (NDS) on the nuScenes-C (Det3D) dataset are shown
in Tab. 17, Tab. 18, and Tab. 19, respectively.

###  9.5 WOD-C (Seg3D)

The complete results in terms of corruption error (CE), resilience rate (RR),
and task-specific accuracy (IoU) on the WOD-C (Seg3D) dataset are shown in
Tab. 20, Tab. 21, and Tab. 22, respectively. In addition to the benchmark
results, we also show the voxel size analysis results of WOD-C (Seg3D) in Fig.
18 (b).

###  9.6 WOD-C (Det3D)

The complete results in terms of corruption error (CE), resilience rate (RR),
and task-specific accuracy (APH) on the WOD-C (Det3D) dataset are shown in
Tab. 23, Tab. 24, and Tab. 25, respectively.

###  9.7 Density-Insensitive Training

As stated in the main body, the corruptions in the real-world environment
often cause severe occlusion, attenuation, and reflection of LiDAR impulses,
resulting in the unavoidable loss of points in certain regions around the ego-
vehicle. To better handle such scenarios, we design a density-insensitive
training framework, with realizations on both detection (see Fig. 16) and
segmentation (see Fig. 17).

Since detection and segmentation have different optimization objectives, we
design different loss computation strategies within these two frameworks.
Specifically, the completion and confirmation losses for the detection
framework are calculated at the BEV feature maps; while for the segmentation
framework, these two losses are computed at the logits level. Our experimental
results in Tab. 26 verify the effectiveness of this approach on both tasks.
Although we use a random masking strategy to avoid information leaks, we
observe overt improvements in a wide range of corruption types that contain
point loss scenarios, such as beam missing, incomplete echo, and cross-sensor.
We believe more sophisticated designs based on our framework could further
boost the corruption robustness of 3D perception models.

##  10 Qualitative Experiment

In this section, we provide extensive qualitative examples for illustrating
the proposed corruption types and for comparing representative models
benchmarked in Robo3D.

###  10.1 Corruption Types

We show visualizations of the eight corruption types under three severity
levels (light, moderate, and heavy) in Fig. 19 and Fig. 20.

###  10.2 Visual Comparisons

For 3D object detection, we attach the qualitative results of SECOND [80] and
CenterPoint [85] under each of the eight corruption types in the WOD-C (Det3D)
dataset. The results are shown in Fig. 21 and Fig. 22.

For 3D semantic segmentation, we attach qualitative results of six segmentors,
i.e., RangeNet++ [44], PolarNet [88], Cylinder3D [93], RPVNet [77], SPVCNN
[66], and WaffleIron [49], under each of the eight corruption types in the
SemanticKITTI-C dataset. The results are shown in Fig. 23, Fig. 24, Fig. 25,
and Fig. 26.

###  10.3 Video Demos

In addition to the figures shown in this file, we have included four video
demos on our project page. Each of these demos consists of hundred of frames
that provide a more comprehensive evaluation of our proposed benchmark.

##  11 Public Resources Used

In this section, we acknowledge the use of the following public resources,
during the course of this work:

  * •

SemanticKITTI222http://semantic-kitti.org. CC BY-NC-SA 4.0

  * •

SemanticKITTI-API333https://github.com/PRBonn/semantic-kitti-api. MIT License

  * •

nuScenes444https://www.nuscenes.org/nuscenes. CC BY-NC-SA 4.0

  * •

nuScenes-devkit555https://github.com/nutonomy/nuscenes-devkit. Apache License
2.0

  * •

Waymo Open Dataset666https://waymo.com/open. Waymo Dataset License

  * •

RangeNet++777https://github.com/PRBonn/lidar-bonnetal. MIT License

  * •

SalsaNext888https://github.com/TiagoCortinhal/SalsaNext. MIT License

  * •

FIDNet999https://github.com/placeforyiming/IROS21-FIDNet-SemanticKITTI.
Unknown

  * •

CENet101010https://github.com/huixiancheng/CENet. MIT License

  * •

KPConv-PyTorch111111https://github.com/HuguesTHOMAS/KPConv-PyTorch. MIT
License

  * •

PIDS121212https://github.com/lordzth666/WACV23_PIDS-Joint. MIT License

  * •

WaffleIron131313https://github.com/valeoai/WaffleIron. Apache License 2.0

  * •

PolarSeg141414https://github.com/edwardzhou130/PolarSeg. BSD 3-Clause License

  * •

MinkowskiEngine151515https://github.com/NVIDIA/MinkowskiEngine. MIT License

  * •

Cylinder3D161616https://github.com/xinge008/Cylinder3D. Apache License 2.0

  * •

PyTorch-Scatter171717https://github.com/rusty1s/pytorch_scatter. MIT License

  * •

SpConv181818https://github.com/traveller59/spconv. Apache License 2.0

  * •

TorchSparse191919https://github.com/mit-han-lab/torchsparse. MIT License

  * •

SPVCNN202020https://github.com/mit-han-lab/spvnas. MIT License

  * •

CPGNet212121https://github.com/huixiancheng/No-CPGNet. Unknown

  * •

2DPASS222222https://github.com/yanx27/2DPASS. MIT License

  * •

GFNet232323https://github.com/haibo-qiu/GFNet. Unknown

  * •

PointPillars242424https://github.com/zhulf0804/PointPillars. Unknown

  * •

second.pytorch252525https://github.com/traveller59/second.pytorch. MIT License

  * •

OpenPCDet262626https://github.com/open-mmlab/OpenPCDet. Apache License 2.0

  * •

PointRCNN272727https://github.com/sshaoshuai/PointRCNN. MIT License

  * •

PartA2-Net282828https://github.com/sshaoshuai/PartA2-Net. Apache License 2.0

  * •

PV-RCNN292929https://github.com/sshaoshuai/PV-RCNN. Unknown

  * •

CenterPoint303030https://github.com/tianweiy/CenterPoint. MIT License

  * •

lidar-camera-robust-benchmark313131https://github.com/kcyu2014/lidar-camera-
robust-benchmark. Unknown

  * •

LiDAR-fog-sim323232https://github.com/MartinHahner/LiDAR_fog_sim.
NonCommercial 4.0

  * •

LiDAR-snow-sim333333https://github.com/SysCV/LiDAR_snow_sim. NonCommercial 4.0

  * •

mmdetection3d343434https://github.com/open-mmlab/mmdetection3d. Apache License
2.0

  * •

LaserMix353535https://github.com/ldkong1205/LaserMix. CC BY-NC-SA 4.0

Table 8: [Complete Results] The Corruption Error (CE) of each method on
SemanticKITTI-C. Bold: Best in column. Underline: Second best in column. All
scores are given in percentage ($\%$). Dark: Best in row. Red: Worst in row.
Symbol † denotes the baseline model adopted in calculating the CE scores.

Method | mCE $\downarrow$ | Fog | Wet | Snow | Motion | Beam | Cross | Echo | Sensor | mIoU $\uparrow$
---|---|---|---|---|---|---|---|---|---|---
MinkUNet18† [12] | $100.00$ | $100.00$ | $100.00$ | $100.00$ | $100.00$ | $100.00$ | $100.00$ | $100.00$ | $100.00$ | $62.76$
SqueezeSeg [72] | $164.87$ | $183.89$ | $158.01$ | $165.45$ |  Dark$122.35$ | $171.68$ |  red!9$188.07$ | $158.74$ | $170.81$ | $31.61$
SqueezeSegV2 [73] | $152.45$ | $168.50$ | $141.23$ | $154.64$ |  Dark$115.16$ | $155.24$ |  red!9$176.00$ | $145.27$ | $163.52$ | $41.28$
RangeNet21 [44] | $136.33$ |  red!9$156.27$ | $128.49$ | $133.93$ |  Dark$102.62$ | $141.58$ | $148.87$ | $128.29$ | $150.58$ | $47.15$
RangeNet53 [44] | $130.66$ | $144.28$ | $123.73$ | $128.38$ |  Dark$104.20$ | $135.53$ | $129.43$ | $125.81$ |  red!9$153.88$ | $50.29$
SalsaNext [13] | $116.14$ |  red!9$147.54$ | $112.06$ | $116.55$ |  Dark$77.62$ | $115.32$ | $143.52$ | $114.04$ | $102.47$ | $55.80$
FIDNet [89] | $113.81$ | $127.67$ | $105.13$ | $107.71$ |  Dark$88.88$ | $116.03$ | $121.32$ | $113.74$ |  red!9$130.03$ | $58.80$
CENet [11] | $103.41$ | $129.84$ | $92.72$ | $99.23$ |  Dark$70.50$ | $101.24$ |  red!9$131.13$ | $102.26$ | $100.39$ | $62.55$
PolarNet [88] | $118.56$ | $138.82$ | $107.09$ | $108.26$ |  Dark$86.81$ | $105.08$ | $178.13$ | $112.00$ | $112.25$ | $58.17$
KPConv [67] | $99.54$ | $103.20$ | $91.94$ | $98.14$ | $110.76$ | $97.64$ |  red!9$111.91$ | $97.34$ |  Dark$85.43$ | $62.17$
PIDS1.2× [87] | $104.13$ |  red!9$118.06$ | $98.94$ | $109.46$ | $114.83$ | $103.18$ | $103.94$ | $96.97$ |  Dark$87.64$ | $63.25$
PIDS2.0× [87] | $101.20$ | $110.61$ | $95.70$ | $104.64$ |  red!9$115.55$ | $98.56$ | $102.23$ | $97.54$ |  Dark$84.76$ | $64.55$
WaffleIron [49] | $109.54$ | $123.45$ | $90.09$ | $108.52$ | $99.85$ | $93.22$ |  red!9$186.08$ | $\mathbf{90.96}$ |  Dark$84.11$ | $\mathbf{66.04}$
MinkUNet34 [12] | $100.61$ | $105.28$ | $99.39$ |  red!9$106.66$ | $98.69$ |  Dark$97.64$ | $99.90$ | $99.01$ | $98.33$ | $63.78$
Cylinder3D${}_{\text{SPC}}$ [93] | $103.25$ |  red!9$142.53$ | $92.48$ | $113.57$ |  Dark$70.89$ | $96.98$ | $105.66$ | $104.21$ | $99.68$ | $63.42$
Cylinder3D${}_{\text{TSC}}$ [93] | $103.13$ |  red!9$142.51$ | $101.28$ | $116.89$ |  Dark$61.66$ | $98.88$ | $111.40$ | $99.01$ | $93.38$ | $61.00$
SPVCNN18 [66] | $100.30$ |  red!9$101.25$ | $100.02$ | $103.98$ |  Dark$97.60$ | $99.20$ | $100.58$ | $99.63$ | $100.19$ | $62.47$
SPVCNN34 [66] | $\mathbf{99.16}$ | $\mathbf{98.50}$ | $100.67$ |  red!9$101.99$ |  Dark$97.81$ | $98.99$ | $\mathbf{98.42}$ | $98.82$ | $98.11$ | $63.22$
RPVNet [77] | $111.74$ |  red!9$118.65$ | $100.98$ | $104.60$ |  Dark$78.58$ | $106.43$ | $185.69$ | $99.21$ | $99.78$ | $63.75$
CPGNet [38] | $107.34$ | $140.97$ | $92.61$ | $104.32$ |  Dark$\mathbf{61.05}$ | $\mathbf{90.91}$ |  red!9$195.63$ | $94.97$ | $\mathbf{78.24}$ | $61.50$
2DPASS [78] | $106.14$ | $134.92$ | $\mathbf{85.46}$ | $110.17$ |  Dark$62.91$ | $94.37$ |  red!9$171.72$ | $96.91$ | $92.66$ | $64.61$
GFNet [50] | $108.68$ | $131.34$ | $94.39$ | $\mathbf{92.66}$ |  Dark$61.73$ | $98.56$ |  red!9$198.90$ | $98.24$ | $93.64$ | $63.00$

Table 9: [Complete Results] The Resilience Rate (RR) of each method on
SemanticKITTI-C. Bold: Best in column. Underline: Second best in column. All
scores are given in percentage ($\%$). Dark: Best in row. Red: Worst in row.

Method | mRR $\uparrow$ | Fog | Wet | Snow | Motion | Beam | Cross | Echo | Sensor | mIoU $\uparrow$
---|---|---|---|---|---|---|---|---|---|---
SqueezeSeg [72] | $66.81$ | $59.63$ | $86.37$ | $71.81$ | $56.72$ | $79.12$ | $68.49$ |  Dark$87.50$ |  red!9$24.83$ | $31.61$
SqueezeSegV2 [73] | $65.29$ | $62.11$ |  Dark$84.84$ | $67.22$ | $55.11$ | $77.98$ | $64.63$ | $81.88$ |  red!9$28.54$ | $41.28$
RangeNet21 [44] | $73.42$ | $65.83$ | $86.70$ | $79.38$ | $66.09$ | $80.93$ | $80.55$ |  Dark$88.10$ |  red!9$39.79$ | $47.15$
RangeNet53 [44] | $73.59$ | $72.24$ | $85.64$ | $79.58$ | $59.85$ | $81.13$ |  Dark$91.63$ | $84.85$ |  red!9$33.76$ | $50.29$
SalsaNext [13] | $80.51$ |  red!9$62.53$ | $86.81$ | $81.63$ | $85.90$ |  Dark$88.94$ | $72.06$ | $86.08$ | $80.14$ | $55.80$
FIDNet [89] | $76.99$ | $74.25$ |  Dark$87.81$ | $84.49$ | $68.67$ | $83.88$ | $84.12$ | $81.92$ |  red!9$50.77$ | $58.80$
CENet [11] | $81.29$ |  red!9$68.27$ |  Dark$91.67$ | $85.76$ | $84.27$ | $89.18$ | $72.53$ | $85.37$ | $73.29$ | $62.55$
PolarNet [88] | $74.98$ | $66.60$ | $87.21$ | $84.96$ | $71.81$ |  Dark$93.00$ |  red!9$44.34$ | $84.17$ | $67.80$ | $58.17$
KPConv [67] | $82.90$ | $87.60$ |  Dark$92.81$ | $87.10$ |  red!9$41.34$ | $92.25$ | $85.86$ | $89.50$ | $86.71$ | $62.17$
PIDS1.2× [87] | $77.94$ | $75.73$ | $86.13$ | $77.25$ |  red!9$36.32$ | $86.85$ |  Dark$89.64$ | $88.24$ | $83.35$ | $63.25$
PIDS2.0× [87] | $78.42$ | $79.30$ | $86.71$ | $79.18$ |  red!9$34.84$ | $88.23$ |  Dark$88.94$ | $86.06$ | $84.07$ | $64.55$
WaffleIron [49] | $72.18$ | $68.93$ | $88.66$ | $74.65$ | $50.00$ |  Dark$89.76$ |  red!9$34.04$ | $88.66$ | $82.71$ | $\mathbf{66.04}$
MinkUNet18 [12] | $81.90$ | $89.02$ | $86.03$ | $84.89$ |  red!9$52.45$ | $89.74$ |  Dark$92.96$ | $86.73$ | $73.37$ | $62.76$
MinkUNet34 [12] | $80.22$ | $83.94$ | $85.09$ | $78.66$ |  red!9$52.99$ | $89.92$ |  Dark$91.53$ | $86.05$ | $73.61$ | $63.78$
Cylinder3D${}_{\text{SPC}}$ [93] | $80.08$ |  red!9$58.50$ | $90.59$ | $74.01$ | $82.70$ |  Dark$90.89$ | $88.27$ | $82.80$ | $72.88$ | $63.42$
Cylinder3D${}_{\text{TSC}}$ [93] | $\mathbf{83.90}$ |  red!9$60.84$ | $87.54$ | $74.41$ |  Dark$\mathbf{96.13}$ | $93.13$ | $87.85$ | $89.97$ | $81.34$ | $61.00$
SPVCNN18 [66] | $82.15$ | $88.55$ | $86.41$ | $82.31$ |  red!9$55.27$ | $90.72$ |  Dark$93.00$ | $87.40$ | $73.56$ | $62.47$
SPVCNN34 [66] | $82.01$ | $\mathbf{89.42}$ | $84.91$ | $82.81$ |  red!9$54.40$ | $89.78$ |  Dark$\mathbf{93.32}$ | $86.95$ | $74.45$ | $63.22$
RPVNet [77] | $73.86$ | $74.73$ | $83.98$ | $80.20$ | $74.18$ | $83.94$ |  red!9$35.51$ |  Dark$85.95$ | $72.42$ | $63.75$
CPGNet [38] | $81.05$ | $61.45$ | $93.32$ | $83.35$ | $96.02$ |  Dark$\mathbf{98.03}$ |  red!9$30.08$ | $\mathbf{92.23}$ | $\mathbf{93.97}$ | $61.50$
2DPASS [78] | $77.50$ | $62.62$ |  Dark$\mathbf{93.92}$ | $75.11$ | $89.46$ | $90.98$ |  red!9$44.05$ | $86.43$ | $77.40$ | $64.61$
GFNet [50] | $77.92$ | $66.73$ | $89.79$ | $\mathbf{90.02}$ |  Dark$93.00$ | $90.40$ |  red!9$27.21$ | $87.67$ | $78.54$ | $63.00$

Table 10: [Complete Results] The Intersection-over-Union (IoU) of each method
on SemanticKITTI-C. Bold: Best in column. Underline: Second best in column.
All scores are given in percentage ($\%$). Dark: Best in row. Red: Worst in
row.

Method | mCE $\downarrow$ | mRR $\uparrow$ | mIoU $\uparrow$ | Fog | Wet | Snow | Motion | Beam | Cross | Echo | Sensor
---|---|---|---|---|---|---|---|---|---|---|---
SqueezeSeg [72] | $164.87$ | $66.81$ | $31.61$ | $18.85$ | $27.30$ | $22.70$ | $17.93$ | $25.01$ | $21.65$ |  Dark$27.66$ |  red!9$7.85$
SqueezeSegV2 [73] | $152.45$ | $65.29$ | $41.28$ | $25.64$ |  Dark$35.02$ | $27.75$ | $22.75$ | $32.19$ | $26.68$ | $33.80$ |  red!9$11.78$
RangeNet21 [44] | $136.33$ | $73.42$ | $47.15$ | $31.04$ | $40.88$ | $37.43$ | $31.16$ | $38.16$ | $37.98$ |  Dark$41.54$ |  red!9$18.76$
RangeNet53 [44] | $130.66$ | $73.59$ | $50.29$ | $36.33$ | $43.07$ | $40.02$ | $30.10$ | $40.80$ |  Dark$46.08$ | $42.67$ |  red!9$16.98$
SalsaNext [13] | $116.14$ | $80.51$ | $55.80$ |  red!9$34.89$ | $48.44$ | $45.55$ | $47.93$ |  Dark$49.63$ | $40.21$ | $48.03$ | $44.72$
FIDNet [89] | $113.81$ | $76.99$ | $58.80$ | $43.66$ |  Dark$51.63$ | $49.68$ | $40.38$ | $49.32$ | $49.46$ | $48.17$ |  red!9$29.85$
CENet [11] | $103.41$ | $81.29$ | $62.55$ |  red!9$42.70$ |  Dark$57.34$ | $53.64$ | $52.71$ | $55.78$ | $45.37$ | $53.40$ | $45.84$
PolarNet [88] | $118.56$ | $74.98$ | $58.17$ | $38.74$ | $50.73$ | $49.42$ | $41.77$ |  Dark$54.10$ |  red!9$25.79$ | $48.96$ | $39.44$
KPConv [67] | $99.54$ | $82.90$ | $62.17$ | $54.46$ |  Dark$57.70$ | $54.15$ |  red!9$25.70$ | $57.35$ | $53.38$ | $55.64$ | $53.91$
PIDS1.2× [87] | $104.13$ | $77.94$ | $63.25$ | $47.90$ | $54.48$ | $48.86$ |  red!9$22.97$ | $54.93$ |  Dark$56.70$ | $55.81$ | $52.72$
PIDS2.0× [87] | $101.20$ | $78.42$ | $64.55$ | $51.19$ | $55.97$ | $51.11$ |  red!9$22.49$ | $56.95$ |  Dark$57.41$ | $55.55$ | $54.27$
WaffleIron [49] | $109.54$ | $72.18$ | $\mathbf{66.04}$ | $45.52$ | $58.55$ | $49.30$ | $33.02$ |  Dark$59.28$ |  red!9$22.48$ | $\mathbf{58.55}$ | $54.62$
MinkUNet18 [12] | $100.00$ | $81.90$ | $62.76$ | $55.87$ | $53.99$ | $53.28$ |  red!9$32.92$ | $56.32$ |  Dark$58.34$ | $54.43$ | $46.05$
MinkUNet34 [12] | $100.61$ | $80.22$ | $63.78$ | $53.54$ | $54.27$ | $50.17$ |  red!9$33.80$ | $57.35$ |  Dark$58.38$ | $54.88$ | $46.95$
Cylinder3D${}_{\text{SPC}}$ [93] | $103.25$ | $80.08$ | $63.42$ |  red!9$37.10$ | $57.45$ | $46.94$ | $52.45$ |  Dark$57.64$ | $55.98$ | $52.51$ | $46.22$
Cylinder3D${}_{\text{TSC}}$ [93] | $103.13$ | $\mathbf{83.90}$ | $61.00$ |  red!9$37.11$ | $53.40$ | $45.39$ |  Dark$58.64$ | $56.81$ | $53.59$ | $54.88$ | $49.62$
SPVCNN18 [66] | $100.30$ | $82.15$ | $62.47$ | $55.32$ | $53.98$ | $51.42$ |  red!9$34.53$ | $56.67$ |  Dark$58.10$ | $54.60$ | $45.95$
SPVCNN34 [66] | $\mathbf{99.16}$ | $82.01$ | $63.22$ | $\mathbf{56.53}$ | $53.68$ | $52.35$ |  red!9$34.39$ | $56.76$ |  Dark$\mathbf{59.00}$ | $54.97$ | $47.07$
RPVNet [77] | $111.74$ | $73.86$ | $63.75$ | $47.64$ | $53.54$ | $51.13$ | $47.29$ | $53.51$ |  red!9$22.64$ |  Dark$54.79$ | $46.17$
CPGNet [38] | $107.34$ | $81.05$ | $61.50$ | $37.79$ | $57.39$ | $51.26$ | $\mathbf{59.05}$ |  Dark$\mathbf{60.29}$ |  red!9$18.50$ | $56.72$ | $\mathbf{57.79}$
2DPASS [78] | $106.14$ | $77.50$ | $64.61$ | $40.46$ |  Dark$\mathbf{60.68}$ | $48.53$ | $57.80$ | $58.78$ |  red!9$28.46$ | $55.84$ | $50.01$
GFNet [50] | $108.68$ | $77.92$ | $63.00$ | $42.04$ | $56.57$ | $\mathbf{56.71}$ |  Dark$58.59$ | $56.95$ |  red!9$17.14$ | $55.23$ | $49.48$

Table 11: [Complete Results] The Corruption Error (CE) of each method on
KITTI-C. Bold: Best in column. Underline: Second best in column. All scores
are given in percentage ($\%$). Dark: Best in row. Red: Worst in row. Symbol †
denotes the baseline model adopted in calculating the CE scores.

Method | mCE $\downarrow$ | Fog | Wet | Snow | Motion | Beam | Cross | Echo | Sensor | mAP $\uparrow$
---|---|---|---|---|---|---|---|---|---|---
CenterPoint† [85] | $100.00$ | $100.00$ | $100.00$ | $100.00$ | $100.00$ | $100.00$ | $100.00$ | $100.00$ | $100.00$ | $68.70$
SECOND [80] | $95.93$ | $99.70$ |  red!9$100.64$ |  Dark$87.64$ | $97.60$ | $91.50$ | $96.50$ | $99.15$ | $94.75$ | $68.49$
PointPillars [34] | $110.67$ | $115.78$ | $106.39$ |  red!9$124.86$ | $101.63$ |  Dark$95.29$ | $117.62$ | $109.88$ | $113.88$ | $66.70$
PointRCNN [59] | $91.88$ | $93.16$ | $90.06$ | $96.81$ | $93.12$ | $86.11$ |  red!9$100.88$ | $92.41$ |  Dark$\mathbf{82.49}$ | $70.26$
Part-A2${}_{\text{Free}}$ [60] | $\mathbf{82.22}$ |  red!9$\mathbf{89.42}$ | $\mathbf{75.78}$ | $\mathbf{81.32}$ | $\mathbf{86.15}$ | $\mathbf{80.89}$ |  Dark$\mathbf{71.79}$ | $\mathbf{83.55}$ | $88.88$ | $\mathbf{76.28}$
Part-A2${}_{\text{Anchor}}$ [60] | $88.62$ | $92.56$ | $83.19$ |  red!9$94.63$ | $86.36$ | $87.03$ |  Dark$83.18$ | $89.32$ | $92.66$ | $73.98$
PV-RCNN [56] | $90.04$ | $95.18$ | $86.64$ |  red!9$93.08$ | $87.51$ |  Dark$86.03$ | $87.09$ | $90.02$ | $94.73$ | $72.36$

Table 12: [Complete Results] The Resilience Rate (RR) of each method on
KITTI-C. Bold: Best in column. Underline: Second best in column. All scores
are given in percentage ($\%$). Dark: Best in row. Red: Worst in row.

Method | mRR $\uparrow$ | Fog | Wet | Snow | Motion | Beam | Cross | Echo | Sensor | mAP $\uparrow$
---|---|---|---|---|---|---|---|---|---|---
PointPillars [34] | $74.94$ | $68.52$ |  Dark$100.01$ |  red!9$53.63$ | $70.60$ | $78.32$ | $89.97$ | $82.22$ | $56.22$ | $66.70$
SECOND [80] | $82.94$ | $77.73$ |  Dark$100.03$ | $\mathbf{80.19}$ | $71.82$ | $79.05$ | $98.10$ | $86.51$ |  red!9$70.08$ | $68.49$
PointRCNN [59] | $\mathbf{83.46}$ | $\mathbf{80.15}$ |  Dark$\mathbf{102.22}$ |  red!9$71.45$ | $73.33$ | $\mathbf{80.90}$ | $93.51$ | $\mathbf{88.27}$ | $\mathbf{77.90}$ | $70.26$
Part-A2${}_{\text{Free}}$ [60] | $81.87$ | $76.11$ |  Dark$100.01$ | $76.26$ | $72.30$ | $77.95$ | $\mathbf{99.10}$ | $86.08$ |  red!9$67.15$ | $\mathbf{76.28}$
Part-A2${}_{\text{Anchor}}$ [60] | $80.67$ | $76.49$ |  Dark$99.99$ | $69.37$ | $74.40$ | $76.21$ | $96.95$ | $85.55$ |  red!9$66.44$ | $73.98$
PV-RCNN [56] | $81.73$ | $76.51$ |  Dark$100.73$ | $72.03$ | $\mathbf{75.23}$ | $78.61$ | $97.28$ | $87.06$ |  red!9$66.35$ | $72.36$
CenterPoint [85] | $79.73$ | $77.29$ |  Dark$100.01$ | $70.68$ | $69.78$ | $72.61$ | $96.07$ | $85.74$ |  red!9$65.68$ | $68.70$

Table 13: [Complete Results] The Average Precision (AP) of each method on
KITTI-C. Bold: Best in column. Underline: Second best in column. All scores
are given in percentage ($\%$). Dark: Best in row. Red: Worst in row.

Method | mCE $\downarrow$ | mRR $\uparrow$ | mAP $\uparrow$ | Fog | Wet | Snow | Motion | Beam | Cross | Echo | Sensor
---|---|---|---|---|---|---|---|---|---|---|---
PointPillars [34] | $110.67$ | $74.94$ | $66.70$ | $45.70$ |  Dark$66.71$ |  red!9$35.77$ | $47.09$ | $52.24$ | $60.01$ | $54.84$ | $37.50$
SECOND [80] | $95.93$ | $82.94$ | $68.49$ | $53.24$ |  Dark$68.51$ | $54.92$ | $49.19$ | $54.14$ | $67.19$ | $59.25$ |  red!9$48.00$
PointRCNN [59] | $91.88$ | $\mathbf{83.46}$ | $70.26$ | $56.31$ |  Dark$71.82$ |  red!9$50.20$ | $51.52$ | $56.84$ | $65.70$ | $62.02$ | $\mathbf{54.73}$
Part-A2${}_{\text{Free}}$ [60] | $\mathbf{82.22}$ | $81.87$ | $\mathbf{76.28}$ | $\mathbf{58.06}$ |  Dark$\mathbf{76.29}$ | $\mathbf{58.17}$ | $\mathbf{55.15}$ | $\mathbf{59.46}$ | $\mathbf{75.59}$ | $\mathbf{65.66}$ |  red!9$51.22$
Part-A2${}_{\text{Anchor}}$ [60] | $88.62$ | $80.67$ | $73.98$ | $56.59$ |  Dark$73.97$ | $51.32$ | $55.04$ | $56.38$ | $71.72$ | $63.29$ |  red!9$49.15$
PV-RCNN [56] | $90.04$ | $81.73$ | $72.36$ | $55.36$ |  Dark$72.89$ | $52.12$ | $54.44$ | $56.88$ | $70.39$ | $63.00$ |  red!9$48.01$
CenterPoint [85] | $100.00$ | $79.73$ | $68.70$ | $53.10$ |  Dark$68.71$ | $48.56$ | $47.94$ | $49.88$ | $66.00$ | $58.90$ |  red!9$45.12$

Table 14: [Complete Results] The Corruption Error (CE) of each method on
nuScenes-C (Seg3D). Bold: Best in column. Underline: Second best in column.
All scores are given in percentage ($\%$). Dark: Best in row. Red: Worst in
row.

Method | mCE $\downarrow$ | Fog | Wet | Snow | Motion | Beam | Cross | Echo | Sensor | mIoU $\uparrow$
---|---|---|---|---|---|---|---|---|---|---
MinkUNet18† [12] | $100.00$ | $100.00$ | $100.00$ | $100.00$ | $100.00$ | $100.00$ | $100.00$ | $100.00$ | $100.00$ | $75.76$
FIDNet [89] | $122.42$ | $75.93$ | $122.58$ | $68.78$ |  red!9$192.03$ | $164.84$ |  Dark$57.95$ | $141.66$ | $155.56$ | $71.38$
CENet [11] | $112.79$ | $71.16$ | $115.48$ | $64.31$ |  red!9$156.67$ | $159.03$ |  Dark$53.27$ | $129.08$ | $153.35$ | $73.28$
PolarNet [88] | $115.09$ | $90.10$ | $115.33$ |  Dark$58.98$ |  red!9$208.19$ | $121.07$ | $80.67$ | $128.17$ | $118.23$ | $71.37$
WaffleIron [49] | $106.73$ | $94.76$ | $99.92$ |  Dark$84.51$ |  red!9$152.35$ | $110.65$ | $91.09$ | $106.41$ | $114.15$ | $76.07$
MinkUNet34 [12] | $96.37$ |  Dark$92.95$ | $96.09$ |  red!9$104.78$ | $\mathbf{93.05}$ | $\mathbf{95.04}$ | $96.27$ | $\mathbf{96.88}$ | $\mathbf{95.90}$ | $76.90$
Cylinder3D${}_{\text{SPC}}$ [93] | $111.84$ | $86.60$ | $104.68$ |  Dark$70.29$ |  red!9$217.47$ | $113.00$ | $75.67$ | $109.21$ | $117.78$ | $76.15$
Cylinder3D${}_{\text{TSC}}$ [93] | $105.56$ | $83.22$ | $111.08$ |  Dark$69.74$ |  red!9$165.28$ | $113.95$ | $74.42$ | $110.67$ | $116.15$ | $73.54$
SPVCNN18 [66] | $106.65$ | $88.42$ | $105.56$ | $98.78$ |  red!9$156.48$ | $110.11$ |  Dark$86.04$ | $104.26$ | $103.55$ | $74.40$
SPVCNN34 [66] | $97.45$ |  Dark$95.21$ |  red!9$99.50$ | $97.32$ | $95.34$ | $98.73$ | $97.92$ | $\mathbf{96.88}$ | $98.74$ | $76.57$
2DPASS [78] | $98.56$ | $76.57$ | $\mathbf{89.08}$ |  Dark$76.35$ |  red!9$142.65$ | $102.23$ | $89.39$ | $101.77$ | $110.44$ | $\mathbf{77.92}$
GFNet [50] | $\mathbf{92.55}$ | $\mathbf{65.60}$ | $93.83$ | $\mathbf{47.23}$ |  red!9$152.46$ | $112.94$ |  Dark$\mathbf{45.25}$ | $105.45$ | $117.64$ | $76.79$

Table 15: [Complete Results] The Resilience Rate (RR) of each method on
nuScenes-C (Seg3D). Bold: Best in column. Underline: Second best in column.
All scores are given in percentage ($\%$). Dark: Best in row. Red: Worst in
row.

Method | mRR $\uparrow$ | Fog | Wet | Snow | Motion | Beam | Cross | Echo | Sensor | mIoU $\uparrow$
---|---|---|---|---|---|---|---|---|---|---
FIDNet [89] | $73.33$ | $90.78$ |  Dark$95.29$ | $82.61$ | $68.51$ | $67.44$ | $80.48$ | $68.31$ |  red!9$33.20$ | $71.38$
CENet [11] | $76.04$ | $\mathbf{91.44}$ |  Dark$95.35$ | $84.12$ | $79.57$ | $68.19$ | $83.09$ | $72.75$ |  red!9$33.82$ | $73.28$
PolarNet [88] | $76.34$ | $81.59$ |  Dark$97.95$ | $90.82$ | $62.49$ | $86.75$ |  red!9$57.12$ | $75.16$ | $58.86$ | $71.37$
WaffleIron [49] | $72.78$ | $73.71$ |  Dark$97.19$ | $65.19$ | $78.16$ | $85.70$ |  red!9$43.54$ | $80.86$ | $57.85$ | $76.07$
MinkUNet18 [12] | $74.44$ | $70.80$ |  Dark$97.56$ | $53.26$ | $96.87$ | $90.47$ |  red!9$35.08$ | $84.25$ | $67.25$ | $75.76$
MinkUNet34 [12] | $75.08$ | $74.01$ | $97.44$ | $48.76$ |  Dark$\mathbf{97.84}$ | $\mathbf{91.16}$ |  red!9$38.13$ | $84.47$ | $\mathbf{68.87}$ | $76.90$
Cylinder3D${}_{\text{SPC}}$ [93] | $72.94$ | $78.59$ |  Dark$95.46$ | $76.26$ |  red!9$55.33$ | $84.64$ | $58.36$ | $79.45$ | $55.46$ | $76.15$
Cylinder3D${}_{\text{TSC}}$ [93] | $78.08$ | $83.52$ |  Dark$96.57$ | $79.41$ | $76.18$ | $87.23$ | $61.68$ | $81.55$ |  red!9$58.51$ | $73.54$
SPVCNN18 [66] | $74.70$ | $79.31$ |  Dark$97.39$ | $55.22$ | $78.44$ | $87.85$ |  red!9$49.50$ | $83.72$ | $66.14$ | $74.40$
SPVCNN34 [66] | $75.10$ | $72.95$ | $96.70$ | $54.79$ |  Dark$97.47$ | $90.04$ |  red!9$36.71$ | $\mathbf{84.84}$ | $67.35$ | $76.57$
2DPASS [78] | $75.24$ | $82.78$ |  Dark$\mathbf{98.51}$ | $69.89$ | $79.62$ | $87.06$ |  red!9$44.11$ | $81.10$ | $58.82$ | $\mathbf{77.92}$
GFNet [50] | $\mathbf{83.31}$ | $90.62$ |  Dark$98.35$ | $\mathbf{93.54}$ | $77.39$ | $83.96$ | $\mathbf{86.96}$ | $80.56$ |  red!9$55.09$ | $76.79$

Table 16: [Complete Results] The Intersection-over-Union (IoU) of each method
on nuScenes-C (Seg3D). Bold: Best in column. Underline: Second best in column.
All scores are given in percentage ($\%$). Dark: Best in row. Red: Worst in
row.

Method | mCE $\downarrow$ | mRR $\uparrow$ | mIoU $\uparrow$ | Fog | Wet | Snow | Motion | Beam | Cross | Echo | Sensor
---|---|---|---|---|---|---|---|---|---|---|---
FIDNet [89] | $122.42$ | $73.33$ | $71.38$ | $64.80$ |  Dark$68.02$ | $58.97$ | $48.90$ | $48.14$ | $57.45$ | $48.76$ |  red!9$23.70$
CENet [11] | $112.79$ | $76.04$ | $73.28$ | $67.01$ |  Dark$69.87$ | $61.64$ | $58.31$ | $49.97$ | $60.89$ | $53.31$ |  red!9$24.78$
PolarNet [88] | $115.09$ | $76.34$ | $71.37$ | $58.23$ |  Dark$69.91$ | $64.82$ | $44.60$ | $61.91$ |  red!9$40.77$ | $53.64$ | $42.01$
WaffleIron [49] | $106.73$ | $72.78$ | $76.07$ | $56.07$ |  Dark$73.93$ | $49.59$ | $59.46$ | $65.19$ |  red!9$33.12$ | $61.51$ | $44.01$
MinkUNet18 [12] | $100.00$ | $74.44$ | $75.76$ | $53.64$ |  Dark$73.91$ | $40.35$ | $73.39$ | $68.54$ |  red!9$26.58$ | $63.83$ | $50.95$
MinkUNet34 [12] | $96.37$ | $75.08$ | $76.90$ | $56.91$ | $74.93$ | $37.50$ |  Dark$\mathbf{75.24}$ | $\mathbf{70.10}$ |  red!9$29.32$ | $\mathbf{64.96}$ | $\mathbf{52.96}$
Cylinder3D${}_{\text{SPC}}$ [93] | $111.84$ | $72.94$ | $76.15$ | $59.85$ |  Dark$72.69$ | $58.07$ |  red!9$42.13$ | $64.45$ | $44.44$ | $60.50$ | $42.23$
Cylinder3D${}_{\text{TSC}}$ [93] | $105.56$ | $78.08$ | $73.54$ | $61.42$ |  Dark$71.02$ | $58.40$ | $56.02$ | $64.15$ | $45.36$ | $59.97$ |  red!9$43.03$
SPVCNN18 [66] | $106.65$ | $74.70$ | $74.40$ | $59.01$ |  Dark$72.46$ | $41.08$ | $58.36$ | $65.36$ |  red!9$36.83$ | $62.29$ | $49.21$
SPVCNN34 [66] | $97.45$ | $75.10$ | $76.57$ | $55.86$ | $74.04$ | $41.95$ |  Dark$74.63$ | $68.94$ |  red!9$28.11$ | $\mathbf{64.96}$ | $51.57$
2DPASS [78] | $98.56$ | $75.24$ | $\mathbf{77.92}$ | $64.50$ |  Dark$\mathbf{76.76}$ | $54.46$ | $62.04$ | $67.84$ |  red!9$34.37$ | $63.19$ | $45.83$
GFNet [50] | $\mathbf{92.55}$ | $\mathbf{83.31}$ | $76.79$ | $\mathbf{69.59}$ |  Dark$75.52$ | $\mathbf{71.83}$ | $59.43$ | $64.47$ | $\mathbf{66.78}$ | $61.86$ |  red!9$42.30$

Table 17: [Complete Results] The Corruption Error (CE) of each method on
nuScenes-C (Det3D). Bold: Best in column. Underline: Second best in column.
All scores are given in percentage ($\%$). Dark: Best in row. Red: Worst in
row. Symbol † denotes the baseline model adopted in calculating the CE scores.

Method | mCE $\downarrow$ | Fog | Wet | Snow | Motion | Beam | Cross | Echo | Sensor | NDS $\uparrow$
---|---|---|---|---|---|---|---|---|---|---
CenterPoint-PP† [85] | $100.00$ | $100.00$ | $100.00$ | $100.00$ | $100.00$ | $100.00$ | $100.00$ | $100.00$ | $100.00$ | $45.99$
SECOND-MH [80] | $97.50$ | $95.40$ | $96.01$ | $96.09$ | $100.81$ | $99.26$ |  Dark$92.16$ | $97.64$ |  red!12$102.64$ | $47.87$
PointPillars-MH [34] | $102.90$ | $102.85$ | $104.56$ | $102.53$ |  red!12$106.44$ | $102.39$ |  Dark$100.94$ | $102.42$ | $\mathbf{101.05}$ | $43.33$
CenterPoint-LR [85] | $98.74$ | $97.88$ | $96.46$ | $97.70$ | $102.15$ | $101.06$ |  Dark$95.54$ | $\mathbf{95.60}$ |  red!12$103.53$ | $49.72$
CenterPoint-HR [85] | $\mathbf{95.80}$ | $\mathbf{93.01}$ | $\mathbf{92.01}$ | $\mathbf{94.91}$ | $\mathbf{97.56}$ | $\mathbf{98.38}$ |  Dark$\mathbf{91.11}$ | $96.21$ |  red!12$103.23$ | $\mathbf{50.31}$

Table 18: [Complete Results] The Resilience Rate (RR) of each method on
nuScenes-C (Det3D). Bold: Best in column. Underline: Second best in column.
All scores are given in percentage ($\%$). Dark: Best in row. Red: Worst in
row.

Method | mRR $\uparrow$ | Fog | Wet | Snow | Motion | Beam | Cross | Echo | Sensor | NDS $\uparrow$
---|---|---|---|---|---|---|---|---|---|---
PointPillars-MH [34] | $\mathbf{77.24}$ | $76.53$ |  Dark$99.05$ | $68.06$ | $87.79$ | $\mathbf{77.57}$ | $79.88$ | $71.31$ |  red!9$\mathbf{57.70}$ | $43.33$
SECOND-MH [80] | $76.96$ | $\mathbf{79.38}$ |  Dark$\mathbf{99.42}$ | $\mathbf{70.86}$ | $86.32$ | $74.45$ | $\mathbf{84.19}$ | $71.28$ |  red!9$49.76$ | $47.87$
CenterPoint-PP [85] | $76.68$ | $76.13$ |  Dark$98.74$ | $67.91$ | $\mathbf{90.87}$ | $76.45$ | $76.58$ | $70.73$ |  red!9$56.06$ | $45.99$
CenterPoint-LR [85] | $72.49$ | $73.19$ |  Dark$95.21$ | $65.99$ | $81.54$ | $69.33$ | $76.65$ | $\mathbf{71.40}$ |  red!9$46.58$ | $49.72$
CenterPoint-HR [85] | $75.26$ | $78.61$ |  Dark$98.93$ | $69.03$ | $85.89$ | $71.97$ | $81.45$ | $69.75$ |  red!9$46.47$ | $\mathbf{50.31}$

Table 19: [Complete Results] The nuScenes Detection Score (NDS) of each method
on nuScenes-C (Det3D). Bold: Best in column. Underline: Second best in column.
All scores are given in percentage ($\%$). Dark: Best in row. Red: Worst in
row.

Method | mCE $\downarrow$ | mRR $\uparrow$ | NDS $\uparrow$ | Fog | Wet | Snow | Motion | Beam | Cross | Echo | Sensor
---|---|---|---|---|---|---|---|---|---|---|---
PointPillars-MH [34] | $102.90$ | $\mathbf{77.24}$ | $43.33$ | $33.16$ |  Dark$42.92$ | $29.49$ | $38.04$ | $33.61$ | $34.61$ | $30.90$ |  red!9$25.00$
SECOND-MH [80] | $97.50$ | $76.96$ | $47.87$ | $38.00$ |  Dark$47.59$ | $33.92$ | $41.32$ | $35.64$ | $40.30$ | $34.12$ |  red!9$23.82$
CenterPoint-PP [85] | $100.00$ | $76.68$ | $45.99$ | $35.01$ |  Dark$45.41$ | $31.23$ | $41.79$ | $35.16$ | $35.22$ | $32.53$ |  red!9$\mathbf{25.78}$
CenterPoint-LR [85] | $98.74$ | $72.49$ | $49.72$ | $36.39$ |  Dark$47.34$ | $32.81$ | $40.54$ | $34.47$ | $38.11$ | $\mathbf{35.50}$ |  red!9$23.16$
CenterPoint-HR [85] | $\mathbf{95.80}$ | $75.26$ | $\mathbf{50.31}$ | $\mathbf{39.55}$ |  Dark$\mathbf{49.77}$ | $\mathbf{34.73}$ | $\mathbf{43.21}$ | $\mathbf{36.21}$ | $\mathbf{40.98}$ | $35.09$ |  red!9$23.38$

Table 20: [Complete Results] The Corruption Error (CE) of each method on WOD-C
(Seg3D). Bold: Best in column. Underline: Second best in column. All scores
are given in percentage ($\%$). Dark: Best in row. Red: Worst in row.

Method | mCE $\downarrow$ | Fog | Wet | Snow | Motion | Beam | Cross | Echo | Sensor | mIoU $\uparrow$
---|---|---|---|---|---|---|---|---|---|---
MinkUNet18† [12] | $100.00$ | $100.00$ | $100.00$ | $100.00$ | $100.00$ | $100.00$ | $100.00$ | $100.00$ | $100.00$ | $69.06$
MinkUNet34 [12] | $\mathbf{96.21}$ | $\mathbf{96.00}$ | $94.90$ |  red!9$99.53$ | $\mathbf{96.20}$ | $\mathbf{95.43}$ | $\mathbf{96.79}$ | $\mathbf{96.75}$ |  Dark$\mathbf{94.08}$ | $\mathbf{70.15}$
Cylinder3D${}_{\text{TSC}}$ [93] | $106.02$ |  red!9$111.81$ | $104.08$ |  Dark$\mathbf{98.39}$ | $110.30$ | $105.77$ | $106.87$ | $108.24$ | $102.69$ | $65.93$
SPVCNN18 [66] | $103.60$ |  red!9$105.63$ | $104.79$ |  Dark$99.17$ | $105.41$ | $104.85$ | $99.74$ | $104.28$ | $104.91$ | $67.35$
SPVCNN34 [66] | $98.72$ | $99.67$ | $\mathbf{96.36}$ | $100.43$ | $100.00$ | $98.55$ |  red!9$101.93$ | $97.87$ |  Dark$94.97$ | $69.01$

Table 21: [Complete Results] The Resilience Rate (RR) of each method on WOC-C
(Seg3D). Bold: Best in column. Underline: Second best in column. All scores
are given in percentage ($\%$). Dark: Best in row. Red: Worst in row.

Method | mRR $\uparrow$ | Fog | Wet | Snow | Motion | Beam | Cross | Echo | Sensor | mIoU $\uparrow$
---|---|---|---|---|---|---|---|---|---|---
MinkUNet18 [12] | $91.22$ | $97.00$ | $88.31$ | $83.62$ |  Dark$99.80$ | $92.89$ | $94.66$ | $91.75$ |  red!9$81.73$ | $69.06$
MinkUNet34 [12] | $91.80$ | $\mathbf{97.38}$ | $89.78$ |  red!9$82.61$ |  Dark$\mathbf{99.93}$ | $93.78$ | $94.77$ | $92.02$ | $84.13$ | $\mathbf{70.15}$
Cylinder3D${}_{\text{TSC}}$ [93] | $\mathbf{92.39}$ | $95.69$ | $90.10$ | $\mathbf{88.62}$ |  Dark$99.68$ | $\mathbf{94.16}$ | $95.54$ | $91.52$ |  red!9$83.83$ | $65.93$
SPVCNN18 [66] | $91.60$ | $96.70$ | $87.78$ | $86.27$ |  Dark$99.84$ | $92.67$ | $\mathbf{97.19}$ | $91.74$ |  red!9$80.62$ | $67.35$
SPVCNN34 [66] | $92.04$ | $97.23$ | $\mathbf{90.44}$ |  red!9$83.42$ |  Dark$99.87$ | $93.71$ | $93.75$ | $\mathbf{92.94}$ | $\mathbf{84.96}$ | $69.01$

Table 22: [Complete Results] The Intersection-over-Union (IoU) of each method
on WOD-C (Seg3D). Bold: Best in column. Underline: Second best in column. All
scores are given in percentage ($\%$). Dark: Best in row. Red: Worst in row.

Method | mCE $\downarrow$ | mRR $\uparrow$ | mIoU $\uparrow$ | Fog | Wet | Snow | Motion | Beam | Cross | Echo | Sensor
---|---|---|---|---|---|---|---|---|---|---|---
MinkUNet18 [12] | $100.00$ | $91.22$ | $69.06$ | $66.99$ | $60.99$ | $57.75$ |  Dark$68.92$ | $64.15$ | $65.37$ | $63.36$ |  red!9$56.44$
MinkUNet34 [12] | $\mathbf{96.21}$ | $91.80$ | $\mathbf{70.15}$ | $\mathbf{68.31}$ | $62.98$ |  red!9$57.95$ |  Dark$\mathbf{70.10}$ | $\mathbf{65.79}$ | $\mathbf{66.48}$ | $\mathbf{64.55}$ | $\mathbf{59.02}$
Cylinder3D${}_{\text{TSC}}$ [93] | $106.02$ | $\mathbf{92.39}$ | $65.93$ | $63.09$ | $59.40$ | $58.43$ |  Dark$65.72$ | $62.08$ | $62.99$ | $60.34$ |  red!9$55.27$
SPVCNN18 [66] | $103.60$ | $91.60$ | $67.35$ | $65.13$ | $59.12$ | $58.10$ |  Dark$67.24$ | $62.41$ | $65.46$ | $61.79$ |  red!9$54.30$
SPVCNN34 [66] | $98.72$ | $92.04$ | $69.01$ | $67.10$ | $\mathbf{62.41}$ |  red!9$\mathbf{57.57}$ |  Dark$68.92$ | $64.67$ | $64.70$ | $64.14$ | $58.63$

Table 23: [Complete Results] The Corruption Error (CE) of each method on WOD-C
(Det3D). Bold: Best in column. Underline: Second best in column. All scores
are given in percentage ($\%$). Dark: Best in row. Red: Worst in row.

Method | mCE $\downarrow$ | Fog | Wet | Snow | Motion | Beam | Cross | Echo | Sensor | mAPH $\uparrow$
---|---|---|---|---|---|---|---|---|---|---
CenterPoint† [85] | $100.00$ | $100.00$ | $100.00$ | $100.00$ | $100.00$ | $100.00$ | $100.00$ | $100.00$ | $100.00$ | $63.59$
SECOND [80] | $121.43$ | $117.86$ | $126.51$ | $127.51$ |  Dark$113.37$ | $121.25$ |  red!9$127.82$ | $123.66$ | $113.48$ | $53.37$
PointPillars [34] | $127.53$ | $120.76$ | $135.23$ | $129.65$ | $115.23$ | $122.99$ |  red!9$151.71$ | $131.64$ |  Dark$113.05$ | $50.17$
PV-RCNN [56] | $104.90$ | $110.08$ | $104.22$ |  Dark$95.68$ | $101.33$ |  red!9$110.70$ | $101.84$ | $106.00$ | $109.37$ | $61.27$
PV-RCNN++ [58] | $\mathbf{91.60}$ | $\mathbf{95.71}$ |  Dark$\mathbf{88.32}$ |  red!9$\mathbf{90.05}$ | $\mathbf{93.24}$ | $\mathbf{92.50}$ | $\mathbf{88.94}$ | $\mathbf{90.81}$ | $\mathbf{93.23}$ | $\mathbf{67.45}$

Table 24: [Complete Results] The Resilience Rate (RR) of each method on WOD-C
(Det3D). Bold: Best in column. Underline: Second best in column. All scores
are given in percentage ($\%$). Dark: Best in row. Red: Worst in row.

Method | mRR $\uparrow$ | Fog | Wet | Snow | Motion | Beam | Cross | Echo | Sensor | mAPH $\uparrow$
---|---|---|---|---|---|---|---|---|---|---
PointPillars [34] | $81.23$ |  red!9$62.27$ |  Dark$99.16$ | $92.31$ | $69.62$ | $\mathbf{87.56}$ | $79.33$ | $86.53$ | $\mathbf{73.09}$ | $50.17$
SECOND [80] | $81.12$ |  red!9$61.63$ |  Dark$99.29$ | $88.44$ | $67.42$ | $83.79$ | $92.34$ | $87.76$ | $68.26$ | $53.37$
PV-RCNN [56] | $82.43$ |  red!9$60.91$ |  Dark$\mathbf{100.00}$ | $\mathbf{98.55}$ | $69.82$ | $80.84$ | $\mathbf{97.26}$ | $88.84$ | $63.21$ | $61.27$
CenterPoint [85] | $83.30$ |  red!9$\mathbf{67.72}$ |  Dark$98.82$ | $92.14$ | $68.45$ | $85.56$ | $94.86$ | $89.65$ | $69.16$ | $63.59$
PV-RCNN++ [58] | $\mathbf{84.14}$ |  red!9$67.46$ |  Dark$99.60$ | $92.97$ | $\mathbf{70.20}$ | $85.74$ | $95.94$ | $\mathbf{90.38}$ | $70.82$ | $\mathbf{67.45}$

Table 25: [Complete Results] The Average Precision (APH) of each method on
WOD-C (Det3D). Bold: Best in column. Underline: Second best in column. All
scores are given in percentage ($\%$). Dark: Best in row. Red: Worst in row.

Method | mCE $\downarrow$ | mRR $\uparrow$ | mAPH $\uparrow$ | Fog | Wet | Snow | Motion | Beam | Cross | Echo | Sensor
---|---|---|---|---|---|---|---|---|---|---|---
PointPillars [34] | $127.53$ | $81.23$ | $50.17$ |  red!9$31.24$ |  Dark$49.75$ | $46.07$ | $34.93$ | $43.93$ | $39.80$ | $43.41$ | $36.67$
SECOND [80] | $121.43$ | $81.12$ | $53.37$ |  red!9$32.89$ |  Dark$52.99$ | $47.20$ | $35.98$ | $44.72$ | $49.28$ | $46.84$ | $36.43$
PV-RCNN [56] | $104.90$ | $82.43$ | $61.27$ |  red!9$37.32$ |  Dark$61.27$ | $60.38$ | $42.78$ | $49.53$ | $59.59$ | $54.43$ | $38.73$
CenterPoint [85] | $100.00$ | $83.30$ | $63.59$ |  red!9$43.06$ |  Dark$62.84$ | $58.59$ | $43.53$ | $54.41$ | $60.32$ | $57.01$ | $43.98$
PV-RCNN++ [58] | $\mathbf{91.60}$ | $\mathbf{84.14}$ | $\mathbf{67.45}$ |  red!9$\mathbf{45.50}$ |  Dark$\mathbf{67.18}$ | $\mathbf{62.71}$ | $\mathbf{47.35}$ | $\mathbf{57.83}$ | $\mathbf{64.71}$ | $\mathbf{60.96}$ | $\mathbf{47.77}$

Table 26: The Corruption Error (CE) comparisons between the proposed density-
insensitive training framework and the baseline models [12, 85], on
SemanticKITTI-C and WOD-C (Det3D), respectively. The task-specific accuracy is
mean Intersection-over-Union (mIoU) for 3D semantic segmentation and mean
Average Precision (mAPH) for 3D object detection.

Method | mCE $\downarrow$ | Fog | Wet | Snow | Motion | Beam | Cross | Echo | Sensor | Acc
---|---|---|---|---|---|---|---|---|---|---
MinkUNet18 [12], ICA | $100.00$ | $100.00$ | $100.00$ | $100.00$ | $100.00$ | $100.00$ | $100.00$ | $100.00$ | $100.00$ | $62.76$
Ours, ICA | $96.10$ | $104.44$ | $97.09$ | $106.31$ | $85.42$ | $97.34$ | $97.30$ | $95.33$ | $86.08$ | $62.70$
MinkUNet18 [12], OCA | $86.09$ | $87.67$ | $77.90$ | $82.62$ | $73.82$ | $87.66$ | $97.88$ | $95.99$ | $85.15$ | $69.21$
Ours, OCA | $83.23$ | $77.01$ | $101.23$ | $72.53$ | $75.73$ | $79.11$ | $97.78$ | $87.56$ | $66.85$ | $68.13$
CenterPoint [85] | $100.00$ | $100.00$ | $100.00$ | $100.00$ | $100.00$ | $100.00$ | $100.00$ | $100.00$ | $100.00$ | $63.59$
Ours | $99.05$ | $99.77$ | $99.52$ | $100.00$ | $100.00$ | $97.26$ | $99.14$ | $99.70$ | $97.02$ | $63.56$

(a) Voxel Size on nuScenes-C (Seg3D)

(b) Voxel Size on WOD-C (Seg3D)

Figure 18: Corruption sensitivity analysis of the voxel size for the baseline
LiDAR semantic segmentation model [12]. The experiments are conducted on: a)
the nuScenes-C (Seg3D) dataset; and b) the WOD-C (Seg3D) dataset. Different
corruptions exhibit variances under certain configurations.

Figure 19: Visual examples of each corruption type under three severity levels
in our SemanticKITTI-C dataset.

Figure 20: Visual examples of each corruption type under three severity levels
in our nuScenes-C dataset.

Figure 21: Qualitative results of SECOND [80] under each of the eight
corruptions in WOD-C (Det3D). The green boxes represent the groundtruth, while
the red boxes are the predictions. Best viewed in colors.

Figure 22: Qualitative results of CenterPoint [85] under each of the eight
corruptions in WOD-C (Det3D). The green boxes represent the groundtruth, while
the red boxes are the predictions. Best viewed in colors.

Figure 23: Qualitative comparisons (error maps) of three LiDAR segmentation
models (RPVNet [77], SPVCNN [66], WaffleIron [49]) under the fog, wet ground,
snow, and motion blur corruptions, in SemanticKITTI-C. To highlight the
differences, the correct / incorrect predictions are painted in gray / red,
respectively. Each scene is visualized from the LiDAR bird’s eye view and
covers a $50$m by $50$m region, centered around the ego-vehicle. Best viewed
in colors.

Figure 24: Qualitative comparisons (error maps) of three LiDAR segmentation
models (RPVNet [77], SPVCNN [66], WaffleIron [49]) under the beam missing,
crosstalk, incomplete echo, and cross-sensor corruptions, in SemanticKITTI-C.
To highlight the differences, the correct / incorrect predictions are painted
in gray / red, respectively. Each scene is visualized from the LiDAR bird’s
eye view and covers a $50$m by $50$m region, centered around the ego-vehicle.
Best viewed in colors.

Figure 25: Qualitative comparisons (error maps) of three LiDAR segmentation
models (RangeNet++ [44], PolarNet [88], Cylinder3D [93]) under the fog, wet
ground, snow, and motion blur corruptions, in SemanticKITTI-C. To highlight
the differences, the correct / incorrect predictions are painted in gray /
red, respectively. Each scene is visualized from the LiDAR bird’s eye view and
covers a $50$m by $50$m region, centered around the ego-vehicle. Best viewed
in colors.

Figure 26: Qualitative comparisons (error maps) of three LiDAR segmentation
models (RangeNet++ [44], PolarNet [88], Cylinder3D [93]) under the beam
missing, crosstalk, incomplete echo, and cross-sensor corruptions, in
SemanticKITTI-C. To highlight the differences, the correct / incorrect
predictions are painted in gray / red, respectively. Each scene is visualized
from the LiDAR bird’s eye view and covers a $50$m by $50$m region, centered
around the ego-vehicle. Best viewed in colors.

## References

  * [1] Fatima Albreiki, Sultan Abughazal, Jean Lahoud, Rao Anwer, Hisham Cholakkal, and Fahad Khan.  On the robustness of 3d object detectors.  arXiv preprint arXiv:2207.10205, 2022.
  * [2] Antonio Alliegro, Francesco Cappio Borlino, and Tatiana Tommasi.  Towards open set 3d learning: A benchmark on object point clouds.  arXiv preprint arXiv:2207.11554, 2022.
  * [3] Eduardo Arnold, Omar Y Al-Jarrah, Mehrdad Dianati, Saber Fallah, David Oxtoby, and Alex Mouzakitis.  A survey on 3d object detection methods for autonomous driving applications.  IEEE Transactions on Intelligent Transportation Systems, 20(10):3782–3795, 2019.
  * [4] Jens Behley, Martin Garbade, Andres Milioto, Jan Quenzel, Sven Behnke, Cyrill Stachniss, and Juergen Gall.  Semantickitti: A dataset for semantic scene understanding of lidar sequences.  In IEEE/CVF International Conference on Computer Vision, pages 9297–9307, 2019.
  * [5] Mario Bijelic, Tobias Gruber, Fahim Mannan, Florian Kraus, Werner Ritter, Klaus Dietmayer, and Felix Heide.  Seeing through fog without seeing fog: Deep multimodal sensor fusion in unseen adverse weather.  In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11682–11692, 2020.
  * [6] Mario Bijelic, Tobias Gruber, and Werner Ritter.  A benchmark for lidar sensors in fog: Is detection breaking down?  In IEEE Intelligent Vehicles Symposium, pages 760–767, 2018.
  * [7] Lara Brinon-Arranz, Tiana Rakotovao, Thierry Creuzet, Cem Karaoguz, and Oussama El-Hamzaoui.  A methodology for analyzing the impact of crosstalk on lidar measurements.  IEEE Sensors, 8:1–4, 2021.
  * [8] Holger Caesar, Varun Bankiti, Alex H Lang, Sourabh Vora, Venice Erin Liong, Qiang Xu, Anush Krishnan, Yu Pan, Giancarlo Baldan, and Oscar Beijbom.  nuscenes: A multimodal dataset for autonomous driving.  In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11621–11631, 2020.
  * [9] Yulong Cao, Ningfei Wang, Chaowei Xiao, Dawei Yang, Jin Fang, Ruigang Yang, Qi Alfred Chen, Mingyan Liu, and Bo Li.  Invisible for both camera and lidar: Security of multi-sensor fusion based perception in autonomous driving under physical-world attacks.  In IEEE Symposium on Security and Privacy, pages 176–194, 2021\.
  * [10] Prithvijit Chattopadhyay, Judy Hoffman, Roozbeh Mottaghi, and Aniruddha Kembhavi.  Robustnav: Towards benchmarking robustness in embodied navigation.  In IEEE/CVF International Conference on Computer Vision, pages 15691–15700, 2021.
  * [11] Huixian Cheng, Xianfeng Han, and Guoqiang Xiao.  Cenet: Toward concise and efficient lidar semantic segmentation for autonomous driving.  In IEEE International Conference on Multimedia and Expo, 2022.
  * [12] Christopher Choy, JunYoung Gwak, and Silvio Savarese.  4d spatio-temporal convnets: Minkowski convolutional neural networks.  In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3075–3084, 2019.
  * [13] Tiago Cortinhal, George Tzelepis, and Eren Erdal Aksoy.  Salsanext: Fast, uncertainty-aware semantic segmentation of lidar point clouds.  In International Symposium on Visual Computing, pages 207–222, 2020\.
  * [14] Simon-Pierre Deschênes, Dominic Baril, Vladimír Kubelka, Philippe Giguère, and François Pomerleau.  Lidar scan registration robust to extreme motions.  In IEEE Conference on Robots and Vision, pages 17–24, 2021.
  * [15] Axel L. Diehm, Marcus Hammer, Marcus Hebel, and Michael Arens.  Mitigation of crosstalk effects in multi-lidar configurations.  Electro-Optical Remote Sensing XII, 10796:13–24, 2018.
  * [16] Xiaoyi Dong, Yinglin Zheng, Jianmin Bao, Ting Zhang, Dongdong Chen, Hao Yang, Ming Zeng, Weiming Zhang, Lu Yuan, Dong Chen, Fang Wen, and Nenghai Yu.  Maskclip: Masked self-distillation advances contrastive language-image pretraining.  arXiv preprint arXiv:2208.12262, 2022.
  * [17] Yinpeng Dong, Caixin Kang, Jinlai Zhang, Zijian Zhu, Yikai Wang, Xiao Yang, Hang Su, Xingxing Wei, and Jun Zhu.  Benchmarking robustness of 3d object detection to common corruption.  In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1022–1032, 2023.
  * [18] Martin A Fischler and Robert C Bolles.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography.  Communications of the ACM, 24(6):381–395, 1981.
  * [19] Whye Kit Fong, Rohit Mohan, Juana Valeria Hurtado, Lubing Zhou, Holger Caesar, Oscar Beijbom, and Abhinav Valada.  Panoptic nuscenes: A large-scale benchmark for lidar panoptic segmentation and tracking.  IEEE Robotics and Automation Letters, 7:3795–3802, 2022.
  * [20] Andreas Geiger, Philip Lenz, and Raquel Urtasun.  Are we ready for autonomous driving? the kitti vision benchmark suite.  In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3354–3361, 2012.
  * [21] Yulan Guo, Hanyun Wang, Qingyong Hu, Hao Liu, Li Liu, and Mohammed Bennamoun.  Deep learning for 3d point clouds: A survey.  IEEE Transactions on Pattern Analysis and Machine Intelligence, 43(12):4338–4364, 2020.
  * [22] Martin Hahner, Christos Sakaridis, Mario Bijelic, Felix Heide, Fisher Yu, Dengxin Dai, and Luc Van Gool.  Lidar snowfall simulation for robust 3d object detection.  In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16364–16374, 2022.
  * [23] Martin Hahner, Christos Sakaridis, Dengxin Dai, and Luc Van Gool.  Fog simulation on real lidar point clouds for 3d object detection in adverse weather.  In IEEE/CVF International Conference on Computer Vision, pages 15283–15292, 2021.
  * [24] Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick.  Masked autoencoders are scalable vision learners.  In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16000–16009, 2021.
  * [25] Dan Hendrycks and Thomas Dietterich.  Benchmarking neural network robustness to common corruptions and perturbations.  In International Conference on Learning Representations, 2019.
  * [26] Georg Hess, Johan Jaxing, Elias Svensson, David Hagerman, Christoffer Petersson, and Lennart Svensson.  Masked autoencoders for self-supervised learning on automotive point clouds.  arXiv preprint arXiv:2207.00531, 2022.
  * [27] Keli Huang, Botian Shi, Xiang Li, Xin Li, Siyuan Huang, and Yikang Li.  Multi-modal sensor fusion for auto driving perception: A survey.  arXiv preprint arXiv:2202.02703, 2022.
  * [28] Maximilian Jaritz, Tuan-Hung Vu, Raoul de Charette, Emilie Wirbel, and Patrick Pérez.  xmuda: Cross-modal unsupervised domain adaptation for 3d semantic segmentation.  In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12605–12614, 2020.
  * [29] Christoph Kamann and Carsten Rother.  Benchmarking the robustness of semantic segmentation models.  In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8828–8838, 2020.
  * [30] Oğuzhan Fatih Kar, Teresa Yeo, Andrei Atanov, and Amir Zamir.  3d common corruptions and data augmentation.  In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8963–18974, 2022.
  * [31] Lingdong Kong, Youquan Liu, Runnan Chen, Yuexin Ma, Xinge Zhu, Yikang Li, Yuenan Hou, Yu Qiao, and Ziwei Liu.  Rethinking range view representation for lidar segmentation.  arXiv preprint arXiv:2303.05367, 2023.
  * [32] Lingdong Kong, Niamul Quader, and Venice Erin Liong.  Conda: Unsupervised domain adaptation for lidar segmentation via regularized domain concatenation.  In IEEE International Conference on Robotics and Automation, pages 9338–9345, 2023.
  * [33] Lingdong Kong, Jiawei Ren, Liang Pan, and Ziwei Liu.  Lasermix for semi-supervised lidar semantic segmentation.  In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 21705–21715, 2023.
  * [34] Alex H Lang, Sourabh Vora, Holger Caesar, Lubing Zhou, Jiong Yang, and Oscar Beijbom.  Pointpillars: Fast encoders for object detection from point clouds.  In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12697–12705, 2019.
  * [35] Shuangzhi Li, Zhijie Wang, Felix Juefei-Xu, Qing Guo, Xingyu Li, and Lei Ma.  Common corruption robustness of point cloud detectors: Benchmark and enhancement.  arXiv preprint arXiv:2210.05896, 2021.
  * [36] Xin Li, Tao Ma, Yuenan Hou, Botian Shi, Yucheng Yang, Youquan Liu, Xingjiao Wu, Qin Chen, Yikang Li, Yu Qiao, and Liang He.  Logonet: Towards accurate 3d object detection with local-to-global cross-modal fusion.  In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 17524–17534, 2023.
  * [37] Xin Li, Botian Shi, Yuenan Hou, Xingjiao Wu, Tianlong Ma, Yikang Li, and Liang He.  Homogeneous multi-modal feature fusion and interaction for 3d object detection.  In European Conference on Computer Vision, pages 691–707, 2022\.
  * [38] Xiaoyan Li, Gang Zhang, Hongyu Pan, and Zhenhua Wang.  Cpgnet: Cascade point-grid fusion network for real-time lidar semantic segmentation.  In IEEE International Conference on Robotics and Automation, pages 11117–11123, 2022.
  * [39] Zhichao Li, Feng Wang, and Naiyan Wang.  Lidar r-cnn: An efficient and universal 3d object detector.  In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7546–7555, 2021.
  * [40] Venice Erin Liong, Thi Ngoc Tho Nguyen, Sergi Widjaja, Dhananjai Sharma, and Zhuang Jie Chong.  Amvnet: Assertion-based multi-view fusion network for lidar semantic segmentation.  arXiv preprint arXiv:2012.04934, 2020.
  * [41] Tao Ma, Xuemeng Yang, Hongbin Zhou, Xin Li, Botian Shi, Junjie Liu, Yuchen Yang, Zhizheng Liu, Liang He, Yu Qiao, et al.  Detzero: Rethinking offboard 3d object detection with long-term sequential point clouds.  arXiv preprint arXiv:2306.06023, 2023.
  * [42] Jiageng Mao, Yujing Xue, Minzhe Niu, Haoyue Bai, Jiashi Feng, Xiaodan Liang, Hang Xu, and Chunjing Xu.  Voxel transformer for 3d object detection.  In IEEE/CVF International Conference on Computer Vision, pages 3164–3173, 2021.
  * [43] Claudio Michaelis, Benjamin Mitzkus, Robert Geirhos, Evgenia Rusak, Oliver Bringmann, Alexander S. Ecker, Matthias Bethge, and Wieland Brendel.  Benchmarking robustness in object detection: Autonomous driving when winter is coming.  arXiv preprint arXiv:1907.07484, 2019.
  * [44] Andres Milioto, Ignacio Vizzo, Jens Behley, and Cyrill Stachniss.  Rangenet++: Fast and accurate lidar semantic segmentation.  In IEEE/RSJ International Conference on Intelligent Robots and Systems, pages 4213–4220, 2019.
  * [45] Chen Min, Dawei Zhao, Liang Xiao, Yiming Nie, and Bin Dai.  Voxel-mae: Masked autoencoders for pre-training large-scale point clouds.  arXiv preprint arXiv:2206.09900, 2022.
  * [46] Alexey Nekrasov, Jonas Schult, Or Litany, Bastian Leibe, and Francis Engelmann.  Mix3d: Out-of-context data augmentation for 3d scenes.  In International Conference on 3D Vision, pages 116–125, 2021.
  * [47] Won Park, Nan Liu, Qi Alfred Chen, and Z. Morley Mao.  Sensor adversarial traits: Analyzing robustness of 3d object detection sensor fusion models.  In IEEE International Conference on Image Processing, pages 484–488, 2019.
  * [48] Tyson Govan Phillips, Nicky Guenther, and Peter Ross McAree.  When the dust settles: The four behaviors of lidar in the presence of fine airborne particulates.  Journal of Field Robotics, 34:985–1009, 2017.
  * [49] Gilles Puy, Alexandre Boulch, and Renaud Marlet.  Using a waffle iron for automotive point cloud semantic segmentation.  arXiv preprint arXiv:2301.10100, 2023.
  * [50] Haibo Qiu, Baosheng Yu, and Dacheng Tao.  Gfnet: Geometric flow network for 3d point cloud semantic segmentation.  Transactions on Machine Learning Research, 2022.
  * [51] Ralph H. Rasshofer, Martin Spies, and Hans Spies.  Influences of weather phenomena on automotive laser radar systems.  In Advances in Radio Science, pages 49–60, 2011.
  * [52] Jiawei Ren, Liang Pan, and Ziwei Liu.  Benchmarking and analyzing point cloud classification under corruptions.  In International Conference on Machine Learning, pages 18559–18575, 2022.
  * [53] Giulio Rossolini, Federico Nesti, Gianluca D’Amico, Saasha Nair, Alessandro Biondi, and Giorgio Buttazzo.  On the real-world adversarial robustness of real-time semantic segmentation models for autonomous driving.  arXiv preprint arXiv:2201.01850, 2022.
  * [54] Alvari Seppänen, Risto Ojala, and Kari Tammi.  Adverse weather denoising from adjacent point clouds.  IEEE Robotics and Automation Letters, 8:456–463, 2022.
  * [55] Guangsheng Shi, Ruifeng Li, and Chao Ma.  Pillarnet: High-performance pillar-based 3d object detection.  arXiv preprint arXiv:2205.07403, 2022.
  * [56] Shaoshuai Shi, Chaoxu Guo, Li Jiang, Zhe Wang, Jianping Shi, Xiaogang Wang, and Hongsheng Li.  Pv-rcnn: Point-voxel feature set abstraction for 3d object detection.  In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10529–10538, 2020.
  * [57] Shaoshuai Shi, Li Jiang, Jiajun Deng, Zhe Wang, Chaoxu Guo, Jianping Shi, Xiaogang Wang, and Hongsheng Li.  Pv-rcnn++: Point-voxel feature set abstraction with local vector representation for 3d object detection.  International Journal of Computer Vision, pages 1–21, 2022.
  * [58] Shaoshuai Shi, Li Jiang, Jiajun Deng, Zhe Wang, Chaoxu Guo, Jianping Shi, Xiaogang Wang, and Hongsheng Li.  Rcnn++: Point-voxel feature set abstraction with local vector representation for 3d object detection.  International Journal of Computer Vision, 2022.
  * [59] Shaoshuai Shi, Xiaogang Wang, and Hongsheng Li.  Pointrcnn: 3d object proposal generation and detection from point cloud.  In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 770–779, 2019.
  * [60] Shaoshuai Shi, Zhe Wang, Jianping Shi, Xiaogang Wang, and Hongsheng Li.  From points to parts: 3d object detection from point cloud with part-aware and part-aggregation network.  IEEE Transactions on Pattern Analysis and Machine Intelligence, 43(8):2647–2664, 2020.
  * [61] Weijing Shi and Raj Rajkumar.  Point-gnn: Graph neural network for 3d object detection in a point cloud.  In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1711–1719, 2020.
  * [62] Jungil Shin, Hyunsuk Park, and Taejung Kim.  Characteristics of laser backscattering intensity to detect frozen and wet surfaces on roads.  Journal of Sensors, 2019.
  * [63] Jiachen Sun, Qingzhao Zhang, Bhavya Kailkhura, Zhiding Yu, Chaowei Xiao, and Z. Morley Mao.  Benchmarking robustness of 3d point cloud recognition against common corruptions.  arXiv preprint arXiv:2201.12296, 2022.
  * [64] Pei Sun, Henrik Kretzschmar, Xerxes Dotiwalla, Aurelien Chouard, Vijaysai Patnaik, Paul Tsui, James Guo, Yin Zhou, Yuning Chai, and Benjamin Caine.  Scalability in perception for autonomous driving: Waymo open dataset.  In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2446–2454, 2020.
  * [65] Haotian Tang, Zhijian Liu, Xiuyu Li, Yujun Lin, and Song Han.  Torchsparse: Efficient point cloud inference engine.  In Conference on Machine Learning and Systems, 2022.
  * [66] Haotian Tang, Zhijian Liu, Shengyu Zhao, Yujun Lin, Ji Lin, Hanrui Wang, and Song Han.  Searching efficient 3d architectures with sparse point-voxel convolution.  In European Conference on Computer Vision, pages 685–702, 2020\.
  * [67] Hugues Thomas, Charles R Qi, Jean-Emmanuel Deschaud, Beatriz Marcotegui, François Goulette, and Leonidas J Guibas.  Kpconv: Flexible and deformable convolution for point clouds.  In IEEE/CVF International Conference on Computer Vision, pages 6411–6420, 2019.
  * [68] James Tu, Mengye Ren, Sivabalan Manivasagam, Ming Liang, Bin Yang, Richard Du, Frank Cheng, and Raquel Urtasun.  Physically realizable adversarial examples for lidar object detection.  In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13716–13725, 2020.
  * [69] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, and Illia Polosukhin.  Attention is all you need.  In Advances in Neural Information Processing Systems, volume 30, 2017.
  * [70] Jiahang Wang, Sheng Jin, Wentao Liu, Weizhong Liu, Chen Qian, and Ping Luo.  When human pose estimation meets robustness: Adversarial algorithms and benchmarks.  In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11855–11864, 2021.
  * [71] Yi Wei, Zibu Wei, Yongming Rao, Jiaxin Li, Jie Zhou, and Jiwen Lu.  Lidar distillation: bridging the beam-induced domain gap for 3d object detection.  In European Conference on Computer Vision, pages 179–195, 2022\.
  * [72] Bichen Wu, Alvin Wan, Xiangyu Yue, and Kurt Keutzer.  Squeezeseg: Convolutional neural nets with recurrent crf for real-time road-object segmentation from 3d lidar point cloud.  In IEEE International Conference on Robotics and Automation, pages 1887–1893, 2018.
  * [73] Bichen Wu, Xuanyu Zhou, Sicheng Zhao, Xiangyu Yue, and Kurt Keutzer.  Squeezesegv2: Improved model structure and unsupervised domain adaptation for road-object segmentation from a lidar point cloud.  In IEEE International Conference on Robotics and Automation, pages 4376–4382, 2019.
  * [74] Aoran Xiao, Jiaxing Huang, Dayan Guan, Kaiwen Cui, Shijian Lu, and Ling Shao.  Polarmix: A general data augmentation technique for lidar point clouds.  In Advances in Neural Information Processing Systems, 2022.
  * [75] Aoran Xiao, Jiaxing Huang, Dayan Guan, Fangneng Zhan, and Shijian Lu.  Transfer learning from synthetic to real lidar point cloud for semantic segmentation.  In AAAI Conference on Artificial Intelligence, pages 2795–2803, 2022.
  * [76] Shaoyuan Xie, Zichao Li, Zeyu Wang, and Cihang Xie.  On the adversarial robustness of camera-based 3d object detection.  arXiv preprint arXiv:2301.10766, 2023.
  * [77] Jianyun Xu, Ruixiang Zhang, Jian Dou, Yushi Zhu, Jie Sun, and Shiliang Pu.  Rpvnet: A deep and efficient range-point-voxel fusion network for lidar point cloud segmentation.  In IEEE/CVF International Conference on Computer Vision, pages 16024–16033, 2021.
  * [78] Xu Yan, Jiantao Gao, Chaoda Zheng, Chao Zheng, Ruimao Zhang, Shuguang Cui, and Zhen Li.  2dpass: 2d priors assisted semantic segmentation on lidar point clouds.  In European Conference on Computer Vision, 2022.
  * [79] Xu Yan, Chaoda Zheng, Zhen Li, Shuguang Cui, and Dengxin Dai.  Benchmarking the robustness of lidar semantic segmentation models.  arXiv preprint arXiv:2301.00970, 2023.
  * [80] Yan Yan, Yuxing Mao, and Bo Li.  Second: Sparsely embedded convolutional detection.  Sensors, 18(10):3337, 2018.
  * [81] Zetong Yang, Yanan Sun, Shu Liu, and Jiaya Jia.  3dssd: Point-based 3d single stage object detector.  In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11040–11048, 2020.
  * [82] Zetong Yang, Yanan Sun, Shu Liu, Xiaoyong Shen, and Jiaya Jia.  Std: Sparse-to-dense 3d object detector for point cloud.  In IEEE/CVF International Conference on Computer Vision, pages 1951–1960, 2019.
  * [83] Maosheng Ye, Rui Wan, Shuangjie Xu, Tongyi Cao, and Qifeng Chen.  Efficient point cloud segmentation with geometry-aware sparse networks.  In European Conference on Computer Vision, pages 196–212, 2022\.