# Understanding the Impact of Edge Cases from Occluded Pedestrians for ML
Systems

Jens Henriksson1, Christian Berger2, Stig Ursing1
1Semcon AB, Gothenburg, Sweden, Email: {jens.henriksson,
<EMAIL_ADDRESS>2University of Gothenburg and Chalmers Institute of
Technology, Sweden, Email<EMAIL_ADDRESS>
###### Abstract

Machine learning (ML)-enabled approaches are considered a substantial support
technique of detection and classification of obstacles of traffic participants
in self-driving vehicles. Major breakthroughs have been demonstrated the past
few years, even covering complete end-to-end data processing chain from
sensory inputs through perception and planning to vehicle control of
acceleration, breaking and steering. YOLO (you-only-look-once) is a state-of-
the-art perception neural network (NN) architecture providing object detection
and classification through bounding box estimations on camera images. As the
NN is trained on well annotated images, in this paper we study the variations
of confidence levels from the NN when tested on hand-crafted occlusion added
to a test set. We compare regular pedestrian detection to upper and lower body
detection. Our findings show that the two NN using only partial information
perform similarly well like the NN for the full body when the full body NN’s
performance is 0.75 or better. Furthermore and as expected, the network, which
is only trained on the lower half body is least prone to disturbances from
occlusions of the upper half and vice versa.

###### Index Terms:

deep neural networks, robustness, out-of-distribution, automotive perception

##  I Introduction

The potential of using neural networks to improve advanced functionality has
been seen in a multitude of demonstrations during the past decade. Functions
such as voice recognition, image classification, and object tracking are some
fields, where new improved versions are often presented. However, one inherent
challenge that comes with deep neural networks (DNNs) is how to properly test
and verify the model to allow it to operate in safety critical applications.
Verification of DNNs are not a new topic at all, and was discussed in early
2000 on several occasions e.g. [1] who discussed the emerging approaches to
verify and validate neural networks. The majority of applications using DNNs
back then was small densely connected networks, where formal verification was
still applicable [2, 3]. Fast forwarding 20 years has given us new network
architectures, where the amount of parameters have grown out of reach for
formal verification, new layer types including stochastic elements such as
drop-out layers and input formats such as images where with high
dimensionality.

The spark that lit the interest of convolutional neural networks (CNNs) was
AlexNet [4], one of the first DNNs that managed to win the ImageNet LSVRC-2010
classification challenge. The network architecture highlighted already then
the potential of sequencing convolutional layers for feature extraction.
Feature extraction was previously hand-made, thus each feature being well
motivated. However, the feature extraction inside of the network that was
self-taught through a mini batch gradient descent iterative process, was far
superior and created more abstract features than was ever created by hand-made
feature extractions. AlexNet paved the way for several new ground breaking
architectures that each year outperformed the prior year’s best model. You-
Only-Look-Once (YOLO) was one of the ground breaking object detection and
classification networks that emerged [5] and is still being used today. We are
using YOLO as model during the experiments in this paper as described in Sec.
III

Advanced functionality that is to be deployed in safety critical applications
need to go through rigorous testing. For automotive vehicles, this includes
following the ISO 26262 [6], a functional safety standard for the full
automotive safety life cycle of a function including in part design,
implementation, and verification. As performance progressed within DNN
research, drawbacks regarding safety became apparent as the existing standard
did not have sufficient requirements or methodology how to properly test a
data driven approach [7, 8]. Issues include the fact that ML is training
based, has a black-box nature as the network abstractions are harder to
interpret for humans, and can suffer from instability, i.e. small variations
in image scenery or quality can impact the output severely. To combat this,
suggestions involve putting requirements on the training, testing, and
robustness of the construction of deep learning models [9].

Recent years has brought academic publications [10, 11] as well as technical
reports to address deep learning in safety critical automotive applications
[12, 13] addressing the issues of verifying ML. ISO 24448 - Safety of the
Intended Functionality (SOTIF) aims at supporting the reduction of
unreasonable risk that occurs due to functional insufficiency or by
foreseeable misuse of the function. The standard describes all potential
events within four categories: Safe known sates, safe unknown states, unsafe
known states, and unsafe unknown states. These definitions suggested the
adoption of out-of-distribution (OOD) detection as a safety mechanism [14,
15]. For example, deep nets can incorporate a rejection option, rather than
always giving a prediction [16]. Thus, allowing for a specific risk level to
be met during test time and proceed with caution if rejected instances occur.

OOD detection typically revolves in constructing an anomaly metric and
utilizing it to determine if a input sample strays too far from the training
distribution, thus indicating that the prediction may be unreliable [17, 18].
Since SOTIF does not provide any clear-cut suggestions of what to test for,
these anomaly metrics act as a good initial discussion point. The metric is
essentially the opposite of model confidence and can be accessed for any kind
of network [19]. Hence, in this paper we combine a common source of errors,
_occlusions_ , with a study on confidence level of the models to analyze if a
sub-network archetype benefits the object detection confidence and hence, can
be used to increase a model’s robustness.

###  I-A Problem Domain and Motivation

To support the development of testing criteria, this paper implements initial
testing examples by constructing an automated data manipulating technique to
study how a model’s object detection and classification network performs. The
experiments include occluding images by covering the upper body of pedestrians
(faces and shoulders) or lower body (legs) as well as testing color
distortion. In addition, the procedure is accompanied by training a NN
specifically focusing on the upper and lower half for the pedestrian class of
the same model to see if in any scenario the model may benefit from a cascade
to detect pedestrians. The following research questions are investigated:

  * •

What is the benefit of splitting a NN model to separately predict for upper-
half or lower-half in comparison to full body predictions?

  * •

To what extent can a cascade of full, upper-half, or lower-half body networks
improve confidence levels?

###  I-B Contributions

The experiments found that splitting up a model into a upper and lower part
did not lower the mean average precision (mAP) substantially for cases when
the full-body network achieves good confidence already (See Section III-C for
description on metrics). In some scenarios, an upper body and lower body
prediction model can support the existing full body detector; most typical
cases included occlusions that covered the face of pedestrian, where a lower
body network could emphasize that legs were found, thus allowing a cascading
vote to still find a robust detection. Furthermore, all experiments
highlighted that importance of high quality imagery, as any form of occlusions
or color adjustments hampered the investigated models.

###  I-C Structure of the Article

The remainder of this paper is structured as follows: Sec. II reflects on
additional related work, Sec. III describes the methods and design choices
used for the experiments as well as describes the metrics used for evaluation,
and Sec. IV presents our findings, which are further analyzed in Sec. V.
Finally, we conclude the paper and outline possible future work in Sec. VI.

##  II Related Work

Several papers suggest to look at epistemic uncertainty of the model, i.e.,
the uncertainty within a given model based on limitations of knowledge and
data [20, 21]. They suggest that safety can be described as a cost
minimization approach of epistemic uncertainty, or minimization of both risk
and uncertainty of harmful events that may occur. The topic of minimization of
epistemic uncertainty is missing from existing guidelines and are critical
when considering safety [20].

Ribeiro et al. [22] discuss _trust_ , which is a fundamental part in any
safety critical application. Their work presents a novel approach: Local
interpretable model-agnostic explanations (LIME) that allow any classifier to
elaborate why any prediction has been made. The explanations are formed in a
non-redundant way that are a locally faithfully interpretable representation.
The method is applicable for a multitude of functional methods such as random
forests and neural networks.

Selective classification, also known as a reject option, is presented by
Geifman et al. [23]. The idea is to construct a selective classifier of a
trained model in conjunction with a user-defined accepted risk level. Their
paper showcases several datasets and highlights the coverage trade-off that
altering the accepted risk levels have. A model with a low accepted risk level
predicts with high-confidence at the cost of rarely given a prediction. Their
highlighted results show a confidence of 99.9%, which, however, only covers
60% of the test samples.

Figure 1: A pedestrian prediction example on a sample image from the KITTI
Vision Benchmark Suite [24] with original training labels (left), upper
modified labels (center), and lower modified labels (right).

The idea of utilizing information from the final layer of a neural network has
been investigated in [18, 19]. Hendrycks et al. presented initial results on
using the inverse of maximum activation in the softmax as a measure of out-of-
distribution likelihood [19]. Liang et al. expanded on this idea and combined
the gradient information inside the network to create a larger separation of
inlier and outlier samples [18]. Furthermore, our previous study focused on
the performance of outlier detection as the training proceeded for a given
model. The results showed that outlier detection correlated with training
performance, as long as overfitting was avoided [25].

First steps towards a generic framework for perception safety was taken by
Czarnecki and Salay [26]. Their contribution states a set of factors critical
to perception uncertainty and their impact, and initial suggestions on how to
eliminate or reduce these negative impacts. One of the issues discussed is
detection of out-of-distribution samples. Angus et al. adopts novel approaches
of out-of-distribution detection to apply on semantic segmentation images to
allow the detection to highlight in what region of an image the neural network
prediction is uncertain [27].

##  III Methodology

To reach our research goal, we are defining the research methodology as
follows: Firstly, we derive three subsets from the original KITTI Vision
Benchmark Suite [24] with bounding box annotations for the contained
vulnerable road users: (A) the unmodified dataset, (B) the dataset labelled
“upper”, where we cut the annotation bounding box around a pedestrian
vertically by half and only mark the upper half of a pedestrian, and (C) the
dataset labelled “lower”, where we only use the lower half following the same
principle.

Secondly, we train three neural networks (NN) on these three datasets
respectively to obtain NNs that focus on particular parts of a scene for our
subsequent occlusion experiments. The network architecture used here is a
small version of the original YOLO network. Thirdly, we derive an occlusion
dataset from the original dataset, where we create occlusions on all contained
pedestrians by inserting a perspectively correctly scaled overlay image (paper
box and a wall). From this occlusion dataset, we also create a gray-scale
variant to investigate the effect of color-vs.-non-color input images on the
performance of the trained NNs in case of occlusions.

We evaluate the performance of the three models using the mean average
performance (mAP) and model inherent confidence111Inherent confidence refers
to the anchor point confidences that are used within the YOLO model
architecture, further described in Sec. III-C. In the following, the training
stage, dataset manipulations, and evaluation metrics are described in detail.

Figure 2: A test example image modified with automated data manipulation
method with box occlusion (left) and wall occlusion (right).

###  III-A Training

The dataset used during training and evaluation is the KITTI Vision Benchmark
Suite [24] and in particular the left color images of the 2D object detection.
The dataset consists of 7,481 annotated images, which in these experiments are
originally split up into a training (70%), testing (10%), and validation (20%)
set. Within the dataset, 527 images contain only pedestrians. For our
experiments, we divide these images as 40%, 40%, 20% into training, testing,
and validation set accordingly. Thus, the testing set contains 211 images with
only pedestrians and an additional 537 random images from the dataset.

We are using the YOLO [28] small model (implemented by Ultralytics [29] ) for
our NN architecture. The architecture consists of three parts: A model
backbone consisting of feature extraction, mainly through convolutional
layers; a model neck which emphasizes informational flow from lower layers to
upper layers [30]; and a model head consisting of anchor points as described
in the third version of YOLO [28].

Three YOLO models are trained with varying pedestrian labels as outlined
before: A regular model with the unmodified original labels, and two, where
the pedestrian labels are cut in half and for the upper and lower body
predictions, respectively, as depicted in Fig. 1 for each model’s prediction
on a sample image. Each model is trained for 100 epochs reaching 0.921, 0.916,
and 0.916 for the regular, upper, and lower model, respectively (Surprisingly,
upper and lower received same performance). Please refer to Sec. III-C for a
description of the performance metric.

Figure 3: The inherent confidence levels (y-axis) from the three YOLO models,
for all frames that include pedestrians (x-axis). All data is sorted based on
the full body network prediction confidence (black dots) in ascending order.

###  III-B Data Preparation & Manipulation

For our experiments, the labeled pedestrians are modified to include occlusion
objects of two variants: A package box case representing a person carrying a
box and hence, occluding the person’s upper body (referred to as “box
occlusion” in Sec. IV), as well as brick wall case, representing a person
walking behind a wall covering its body from the waist downwards (referred as
“wall occlusion” during Sec. IV). The data manipulation is processed
automatically by inserting the occlusion object based on the given label
location. The manipulation, as well as the code of the experimental setup is
made available here222https://github.com/jenshenriksson/kitti-occlusion-study.
In addition, we test the box-occlusion variant on gray-scale images to analyze
whether there is an effect from using color information. Illustrating examples
of the occlusions are shown in Fig. 2.

###  III-C Evaluation and Metrics

We are using two metrics to evaluate the experiments: Mean average precision
(mAP) indicating the overall performance of the model, and inherent prediction
confidence, which is used as estimate of the confidence of a prediction.

  * •

Mean Average Precision (mAP): Refers to area under the precision recall-curve,
where precision and recall metrics are calculated based on intersect of
unions, i.e the ratio of overlapping area of prediction and label over the
sum. Varying the accepted threshold of true positives from 0 to 100% yields
the precision-recall curve.

  * •

Inherent confidence: The YOLO output layer output format provides a large set
of anchor point predictions, where each prediction contains a class prediction
vector (similar to a softmax vector), in conjunction with bounding box
predictions. This most probably prediction from the class prediction vector is
used as confidence metric.

We structure our experimental results by firstly presenting the results of the
regular model performance, compared to the individual lower half and upper
half NN, respectively. The results are compared threefold: Comparing the mAP
to indicate the overall performance of a given model during an experiment, the
network confidence given in a sorted graph to compare the confidence levels
in-between the regular, upper half, and lower half network; and a difference
plot, where the variation of confidence is shown to highlight for which
scenario the model has a low performance.

The experiments are then repeated for the following occlusion tests: (1) Box
occlusion test, (2) wall occlusion test, (3) gray-scale, and (4) gray-scale
box occlusion test. During the reporting of the resulting plots (Fig. 5-11),
the black dots refer to the original performance of the regular test set
without any modifications, i.e the full body performance in Fig. 3. This
allows us to compare how the performance looks like without any occurring
occlusions. The blue, red, and green markers represent the regular, upper
half, and lower half models, respectively, in every experiment. Furthermore,
for visualization purposes, the results are sorted based on the regular model
performance on the given experiment, thus the x-axis in Fig. 5-11 only
represents indexes for test images.

Figure 4: Inherent confidence levels (y-axis) from the three models (full
body, upper and lower model represented by blue, green, and red markers,
respectively) when applied on the experiment with box occlusions. The data is
sorted in ascending order based on the full body models test result (blue
markers). Black markers refer to the results from Fig. 3 for comparison with
the non-occlusion performance.

Figure 5: Confidence difference of the full body results on the original
dataset from Fig. 3, and the same network when operating on box occlusions.
High differences indicate that the network is challenged by the modifications.
One frame was detected with the modifications, but missed in the original
model. This sample can be seen in Fig. 12

In our experiments, we are looking exclusively at pedestrians for this study
due to multiple reasons: Firstly, substantially occluding vehicles like cars
or trucks and hence, changing their shape, are rather seldom in traffic and
not particularly changing the main appearance of a car’s or truck’s body shape
that was used for training a network; in contrast, a pedestrian, especially
when primarily trained in largely non-occluded scenarios, is very likely to
also appear differently in real traffic situations because of carrying items
like boxes, plants, backpacks, and similar. Secondly, to evaluate the
robustness of an ML-enabled software system in use to perceive a vehicle’s
surrounding needs to reliably detect the most vulnerable road users:
Pedestrians and bicyclists. Hence, we designed our study to focus first and
foremost on pedestrians.

##  IV Results

We trained all three YOLO models for 100 epochs on the KITTI dataset with the
adjusted labels for the upper half and lower half prediction models. The
training resulted in 0.921, 0.916, and 0.916 mAP for the regular, upper half,
and lower half NN, respectively, showing that the regular networks outperforms
the upper and lower body which received same mAP score. Solely looking at the
pedestrian class shows the regular network at 0.874 mAP, upper half at 0.852
mAP, and lower half at 0.845 mAP indicating that solely training on the legs
of a pedestrian results in only slightly inferior results compared to the two
other networks. Throughout the experiments, the mAP score is used as the
benchmark to estimate the performance of the networks, and the difference in
mAP between the experimental results and the results above highlights how
severely the experimental setup harms the original network.

Figure 6: Inherent confidence levels (y-axis) from the three models (full
body, upper half, and lower half models represented by blue, green, and red
markers respectively) when applied on the experiment with wall occluded
manipulated data. The results are sorted in ascending order based on the full
body model’s test results (blue markers). Black markers are for referring to
the results from Fig. 3 for comparison.

Figure 7: Confidence difference of the full body results on the original
dataset from Fig. 3, and the same network when operating on wall occluded
modifications. High differences indicate that the network struggles with the
modification.

In Fig. 3, the inherent pedestrian prediction confidence is presented for
every frame that contains pedestrians in the testing set. This allows us to
study if there are any frames that are better predicted with either of the
sub-networks compared to the regular network. The average confidence is 0.844,
0.803, and 0.801 for the regular, upper half, and lower half networks
respectively, thus highlighting that the mAP as well as confidence of the
predictions are superior in the fully labeled model.

For the remainder of the results section, each experiment’s inherent
confidence plot will include black markers indicating the performance of the
original full body network performance, as seen in Fig. 3.

###  IV-A Performance in the Box Occlusion Case

Box occlusions as described in Sec. III-B refer to adding occlusion to the
images covering the upper half of a pedestrian. The mean average precision of
the pedestrian class in these experiments are 0.791, 0.584, and 0.787 for the
regular, upper and lower network respectively. Each network has lost
performance, as was expected for the full and upper half network. The fact
that the lower half network also has also a slightly reduced performance is
noteworthy, as intuitively adjusting the image outside of the label should
have little impact on its prediction performance.

However, in Fig. 5 where the inherent confidence scores are presented (sorted
based on the regular model’s confidence of the pedestrian prediction), it can
be seen that some test frames receives better confidence on predictions from
the two sub-networks compared to the original network. Looking at the average
confidence per sample with the manipulated samples for the regular, upper
half, and lower half networks, the average is 0.704, 0.532, and 0.766
respectively. In this scenario the lower half network outperforms the full
body network in 60% of the test samples in regards to confidence scores. This
indicates that there might be potential in using a cascade of networks when
prediction confidence from the main network is sub-par. One frame was oddly
missed in the original model, yet detected in the occluded experiment; out of
curiosity, the detected pedestrian from that frame can be seen in Fig. 12.

Fig. 5 shows the difference of the full body network when operating on the
original and modified data. Studying these results shows that 8.3% of frames
have lost their prediction fully (difference margin $\>50\%$), and the
majority of samples (black colored markers, representing 79.9% of samples)
indicate that the performance is better on the original dataset. On average,
the confidence level decreased with 0.132 percentage units. Red markers in
Fig. 5 indicate that the model actually performs better on the modified data.
Some samples do receive better performance on the modified dataset, however we
believe these samples lay within the margin of error, as no prediction stands
out.

Figure 8: Inherent confidence levels (y-axis) from the three models (full
body, upper half, and lower half models represented by blue, green, and red
markers, respectively) when applied on gray-scale data. The results are sorted
in ascending order based on the full body model’s test result (blue markers).
Black markers refer to the results from Fig. 3 for comparison.

Figure 9: Confidence difference of the full body results on the original
dataset from Fig. 3, and the same network when operating on gray-scale images.
High differences indicate that a network struggles with the applied the
modifications.

###  IV-B Performance on the Wall Occlusion Case

As described in Sec. III-B, wall occlusions refer to occluding the bottom half
of pedestrians by inserting a brick wall across the lower body. The mAP
pedestrian performance on the wall occlusions are 0.817, 0.823, and 0.638 for
the regular, upper half, and lower half models. Intuitively, the lower model
performance is substantially challenged, while the upper and regular network
performs satisfactorily. Noteworthy is that the upper network outperforms the
regular network also marginally.

Studying Fig. 7, all models have reduced confidence compared to the original
full network. The average confidence in prediction performance has been
reduced to 0.81, 0.764, and 0.654 for the regular, upper half, and lower half
models, respectively, and intuitively, the lower half model suffers the most.
Like the box occlusion experiment, several samples where the regular network
contains low confidence, receive higher confidence predictions from the two
models trained with adjusted labels (19.7% of samples receive higher
confidence from the upper network). Thus, similar to the box occlusions,
additional information can be gathered from the additional sub-networks.

The confidence difference between the original results without occlusions, and
the wall occlusions can be seen in Fig. 7. Comparing to the box occlusions,
fewer samples are fully missed in this experiment (1.0% compared to 8.3% for
box occlusions), thus indicating that the upper body half is considerable more
important for pedestrian detection. The average confidence level decreased
with 0.039 percentage units for the experiment, thus impacting the overall
performance much less compared to the box occlusions experiment.

Figure 10: Inherent confidence levels (y-axis) from the three models (full
body, upper half, and lower half models represented by blue, green, and red
markers, respectively) when applied on gray-scale data with box occlusions.
The results are sorted in ascending order based on the full body model’s test
result (blue markers). Black markers refer to the results from Fig. 3 for
comparison.

Figure 11: Confidence difference of the full body results on the original
dataset from Fig. 3, and the same network when operating on gray-scale box
occlusions. High differences indicate that a network struggles with the
modification.

###  IV-C Performance for the Gray-Scale Case

The gray-scale experiments are constructed to investigate if color information
has an impact on the confidence levels in case of occlusions. The images are
constructed by converting back and forth to gray-scale, thus preserving all
three channels for the RGB-image for the model. This allows the original
network not requiring any modifications. Testing the mAP score of pedestrians
resulted in 0.814 mAP for the regular (0.874 mAP with RBG-images), 0.796 mAP
for the upper model (0.852 mAP with RGB-images), and 0.795 mAP lower network
(0.845 mAP with RGB-images), thus seeing a drop for every model.

The sorted inherent model confidence can be seen in Fig. 9 showing that the
overall confidence has been reduced. The average prediction confidence per
sample is 0.816, 0.77, and 0.785 mAP for the regular, upper half, and lower
half networks, respectively. Comparing to the initial results, all models are
challenged from removing RGB information.

The number of missed predictions, though, increased to 14.3%, highlighting the
importance of color information within the networks. In Fig. 9, the confidence
reduction can be seen compared to the original network performance. On
average, the confidence level has been reduced with 0.05 percentage units.

###  IV-D Performance of the Gray-Scale Box Occlusion Case

Our last experiment combines the gray-scale images and box occlusion. The data
manipulation was created by first adding the object for occlusions, followed
by converting back and forth to gray-scale to preserve all color channels to
not modify the NN architecture to handle reductions in color channels. Testing
results yield 0.784 mAP for the regular, 0.768 mAP for the upper half, and
0.796 mAP for the lower half body networks. Hence, the lower half body network
outperforms the regular network when tested with removing color information
and having box occlusions.

The pedestrian prediction confidence can be seen in Fig. 11, where the samples
are sorted based on the regular network’s performance on the manipulated data.
The lower half body network outperforms the other candidates with an average
confidence level of 0.779 units, whereas the upper half body receives only
0.403 and the regular network gets 0.698 units.

Lastly, the results are compared to the original results with the full body
network in Fig. 5. The plot shows that 26.8% of frames have lost their
predictions (confidence difference above $50\%$, or no predictions at all),
thus showing that the network is severely hampered by the two data
manipulation techniques in use. However, looking at the confidence levels in
Fig. 11, the lower half network outperforms the regular network on 57.3% of
the samples, indicating that such sub-networks can yield performance benefits,
particular in edge cases situations.

##  V Analysis and Discussion

Requirements and suitable testing techniques for deep neural networks have
been relevant discussion points throughout academia and industry in recent
years. Even though several standards have been published (e.g., ISO/PAS 24448
(Road vehicles – Safety of the intended functionality), ISO/TR 4804 (Road
vehicles – Safety and cyber-security for automated driving systems), and UL
4600 (Standard for Safety for the Evaluation of Autonomous Products)), neither
have presented clear guidelines on how to properly test deep networks before
deployment. The standards describe the issue from a safety point of view and
without a detailed description of the data science point of view, thus leaving
wide gaps for interpretation. The fundamental idea for experiments as
presented in this paper demonstrate potential safety properties to look and
test for with performing robustness analysis for NN.

A generic safety property discussed previously is the idea of uncertainty
scores333outlier, anomaly, uncertainty and similar notions have been discussed
in the context of acting as an indication if an output is trustworthy from a
model. from a network. Such uncertainty metrics may be achieved through a
diverse set of ideas, such as solely looking at the output layer [19] or
something more rigorous [18]. The experiments in this paper utilize the built-
in confidence values as a reverse variant of uncertainty to study how this
confidence varies in a controlled experiment. Utilizing two sub-networks for
prediction classes like pedestrians (e.g., an upper half body predictor and
lower half body predictor) can be considered as a prediction certainty score
since if largely differing results from the networks are present, such
predictions need to be considered as uncertain as demonstrated in our
experiments.

Figure 12: Can you spot the pedestrian? An odd test example, where the
prediction was missed in the original network, but correctly classified after
adding occlusion. The pedestrian is located close to the center of the image,
just right of the small red vehicle.

Our experiments show that occlusions impacting objects where NN have been
trained on reduce the performance considerably. While this effect was
foreseeable, we could also show that splitting the training on unmodified
input data helps to break down such performance drop during the prediction
phase. When limiting the effect of lower confidence scores, it is safe to say
our experiments demonstrate that semantically guiding the training (ie.,
lower-half vs. upper-half of a body) contributes to the _explainability_ of
the results and overall performance of a NN. Hence, our experiments encourage
the use of more fine-grained training in combination with fine-grained
prediction in practice to better interpret the results for a (possibly)
weighted decision making process in an online system of cascaded NN that focus
on different aspect of an object.

While the aforementioned aspect is clearly improving the decision making
process in practice as shown by our experiments as well as contributing to a
NN model’s explainability, when the various segments of a single object result
in different prediction results: While when (nearly) all these segments have a
pair wise low difference value, there may be a high confidence in the overall
prediction; however, when these segments result in deviating pairwise results,
this may be considered as an indicator for a possible (partial) occlusion of
that said object. However, running multiple NN on the same input image incurs
additional computational overhead that needs to be considered for practical
applications. Hence, a cascade of NNs in practice may not be activated by
default but only when the main NN (e.g. the one searching for full body
objects) results in lower confidence values.

###  V-A Threats to Validity

In this section, we are addressing some known threats to the validity of our
study. Firstly, we experimented with a subset of the KITTI dataset using 748
images as test set to find mAP, and 314 images within the test set that
contained pedestrians for the experimentation. Furthermore, we only trained
the networks on one particular class for the prediction (ie., pedestrians) to
exclude any particular side-effects from other classes. In practice, one would
either use a NN covering multiple classes simultaneously or by constructing a
dedicated cascade of NNs where each is focusing on particular categories.

Secondly, our study did focus on single images only not considering sequences
of images from a situation that may also have an effect on the performance of
a NN. Hence, as also indicated as possible future work, the effect from
sequences needs to be further studied when splitting a NN for logically or
semantically relevant sub-segments for robustness or explainability.

Finally, for the experiments with the gray-scale images, we did not train the
three models particularly on gray-scale input data. The reasoning behind that
decision was to analyze the potential effect from color information compared
to non-available color information for occlusion experiments.

##  VI Conclusion and Future Work

The benefit of splitting a NN becomes apparent when studying the the models
performance on edge cases. We show this through a set of experiments in which
the models inherent confidence is studied and shows that the full body NN
suffers more in comparison to an upper and lower body model. Furthermore, we
discuss a cascading structure where the pedestrian detector model can be
complemented by an upper and lower body model to improve model confidence in
edge cases.

Next steps for this kind of study suggests that the experimental setup is to
be tested on a larger dataset that incorporates more varying scenarios. In
addition, it is of interest to study how the model performance would change if
a dataset that includes images in a time-series, thus allowing the models to
use preceding frames when giving a prediction.

## Acknowledgments

This research has been supported by the Strategic vehicle research and
innovation (FFI) programme in Sweden, via the project SALIENCE4CAV (ref.
2020-02946) and supported SMILE III (ref. 2019-05871). The research has been
supported by the Wallenberg AI, Autonomous Systems and Software Program (WASP)
funded by Knut and Alice Wallenberg Foundation.

## References

  * [1] B. J. Taylor, M. A. Darrah, and C. D. Moats, “Verification and validation of neural networks: a sampling of research in progress,” in _Proceedings of SPIE 5103, Intelligent Computing: Theory and Applications_ , vol. 5103, 2003, pp. 8–16.
  * [2] J. Hull, D. Ward, and R. R. Zakrzewski, “Verification and validation of neural networks for safety-critical applications,” in _Proceedings of the 2002 American Control Conference_ , vol. 6, 2002, pp. 4789–4794.
  * [3] J. Schumann and S. Nelson, “Toward V&V of Neural Network Based Controllers,” in _Proceedings of the 1st Workshop on Self-healing Systems_ , 2002, pp. 67–72.
  * [4] A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classification with deep convolutional neural networks,” _Advances in neural information processing systems_ , vol. 25, pp. 1097–1105, 2012.
  * [5] J. Redmon, S. Divvala, R. Girshick, and A. Farhadi, “You only look once: Unified, real-time object detection,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2016, pp. 779–788.
  * [6] {International Organization for Standardization}, _ISO 26262-2:2018 Road vehicles - Functional safety_ , 2018.
  * [7] K. Heckemann, M. Gesell, T. Pfister, K. Berns, K. Schneider, and M. Trapp, “Safe Automotive Software,” in _Knowledge-Based and Intelligent Information and Engineering Systems_. Springer, Berlin, Heidelberg, 2011, pp. 167–176.
  * [8] R. Salay, R. Queiroz, and K. Czarnecki, “An Analysis of ISO 26262: Using Machine Learning Safely in Automotive Software,” _arXiv:1709.02435_ , 2017, arXiv: 1709.02435.
  * [9] J. Henriksson, M. Borg, and C. Englund, “Automotive safety and machine learning: Initial results from a study on how to adapt the ISO 26262 safety standard,” in _Proceedings of the 1st International Workshop on Software Engineering for AI in Autonomous Systems_. IEEE, 2018, pp. 47–49.
  * [10] X. Huang, M. Kwiatkowska, S. Wang, and M. Wu, “Safety Verification of Deep Neural Networks,” _arXiv:1610.06940_ , 2016.
  * [11] S. A. Seshia, D. Sadigh, and S. S. Sastry, “Towards Verified Artificial Intelligence,” _arXiv:1606.08514_ , 2016. [Online]. Available: http://arxiv.org/abs/1606.08514
  * [12] {International Organization for Standardization}, _ISO/PAS 21448:2019 Road vehicles – Safety of the intended functionality_ , 2019.
  * [13] ——, _ISO/TR 21448:2019 Road vehicles – Safety and cybersecurity for automated driving systems_ , 2020.
  * [14] M. Borg, C. Englund, and B. Duran, “Traceability and Deep Learning - Safety-critical Systems with Traces Ending in Deep Neural Networks,” in _Proceedings of the Grand Challenges of Traceability: The Next Ten Years_ , 2017, pp. 48–49. [Online]. Available: https://arxiv.org/abs/1710.03129
  * [15] M. Borg, C. Englund, K. Wnuk, B. Duran, C. Levandowski, S. Gao, Y. Tan, H. Kaijser, H. Lönn, and J. Törnqvist, “Safely entering the deep: A review of verification and validation for machine learning and a challenge elicitation in the automotive industry,” _Journal of Automotive Software Engineering (Accepted for publication)_ , 2019.
  * [16] X. Ma, K. Driggs-Campbell, and M. J. Kochenderfer, “Improved robustness and safety for autonomous vehicle control with adversarial reinforcement learning,” _2018 IEEE Intelligent Vehicles Symposium (IV)_ , Jun 2018. [Online]. Available: http://dx.doi.org/10.1109/ivs.2018.8500450
  * [17] J. Henriksson, C. Berger, M. Borg, L. Tornberg, C. Englund, S. R. Sathyamoorthy, and S. Ursing, “Towards structured evaluation of deep neural network supervisors,” in _To Appear in the Proceedings of the 1st IEEE International Conference on Artificial Intelligence Testing_ , 2019.
  * [18] S. Liang, Y. Li, and R. Srikant, “Enhancing the reliability of out-of-distribution image detection in neural networks,” _arXiv preprint arXiv:1706.02690_ , 2017.
  * [19] D. Hendrycks and K. Gimpel, “A baseline for detecting misclassified and out-of-distribution examples in neural networks,” _arXiv preprint arXiv:1610.02136_ , 2016.
  * [20] K. Varshney, “Engineering safety in machine learning,” in _Proceedings of the 2016 Information Theory and Appl. Workshop_ , Jan. 2016, pp. 1–5.
  * [21] A. Carvalho, S. Lefevre, G. Schildbach, J. Kong, and F. Borelli, “Automated driving: The role of forecasts and uncertainty - A control perspective,” _European Journal of Control_ , 2015.
  * [22] M. T. Ribeiro, S. Singh, and C. Guestrin, “”Why Should I Trust You?”: Explaining the Predictions of Any Classifier,” in _Proceedings of the 22Nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining_ , ser. KDD ’16. New York, NY, USA: ACM, 2016, pp. 1135–1144. [Online]. Available: http://doi.acm.org/10.1145/2939672.2939778
  * [23] Y. Geifman and R. El-Yaniv, “Selective classification for deep neural networks,” in _Advances in Neural Information Processing Systems 30_ , I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, Eds. Curran Associates, Inc., 2017, pp. 4878–4887. [Online]. Available: http://papers.nips.cc/paper/7073-selective-classification-for-deep-neural-networks.pdf
  * [24] A. Geiger, P. Lenz, and R. Urtasun, “Are we ready for autonomous driving? the kitti vision benchmark suite,” in _Conference on Computer Vision and Pattern Recognition (CVPR)_ , 2012.
  * [25] J. Henriksson, C. Berger, M. Borg, L. Tornberg, S. R. Sathyamoorthy, and C. Englund, “Performance analysis of out-of-distribution detection on trained neural networks,” _Information and Software Technology_ , vol. 130, p. 106409, 2021.
  * [26] K. Czarnecki and R. Salay, “Towards a framework to manage perceptual uncertainty for safe automated driving,” _Lecture Notes in Computer Science_ , p. 439–445, 2018. [Online]. Available: http://dx.doi.org/10.1007/978-3-319-99229-7_37
  * [27] M. Angus, K. Czarnecki, and R. Salay, “Efficacy of pixel-level ood detection for semantic segmentation,” _arXiv preprint arXiv:1911.02897_ , 2019.
  * [28] J. Redmon and A. Farhadi, “Yolov3: An incremental improvement,” _arXiv preprint arXiv:1804.02767_ , 2018.
  * [29] G. Jocher, A. Stoken, J. Borovec, NanoCode012, A. Chaurasia, TaoXie, L. Changyu, A. V, Laughing, tkianai, yxNONG, A. Hogan, lorenzomammana, AlexWang1900, J. Hajek, L. Diaconu, Marc, Y. Kwon, oleg, wanghaoyang0106, Y. Defretin, A. Lohia, ml5ah, B. Milanko, B. Fineran, D. Khromov, D. Yiwei, Doug, Durgesh, and F. Ingham, “ultralytics/yolov5: v5.0 - YOLOv5-P6 1280 models, AWS, Supervise.ly and YouTube integrations,” Apr. 2021. [Online]. Available: https://doi.org/10.5281/zenodo.4679653
  * [30] S. Liu, L. Qi, H. Qin, J. Shi, and J. Jia, “Path aggregation network for instance segmentation,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2018, pp. 8759–8768.


[a,1]Sandra Consuegra Rodríguez 11footnotetext: On behalf of the CMS
Collaboration

# CMS Tracker Alignment: Legacy results from LHC Run 2 and Run 3 prospects

###### Abstract

The inner tracking system of the CMS experiment, which comprises Silicon Pixel
and Silicon Strip detectors, is designed to provide a precise measurement of
the momentum of charged particles and to reconstruct the primary and secondary
vertices. The movements of the different substructures of the tracker
detectors driven by the operating conditions during data taking, require to
regularly update the detector geometry in order to accurately describe the
position, orientation, and curvature of the tracker modules.
The procedure in which new parameters of the tracker geometry are determined
is known as alignment of the tracker. The alignment procedure is performed
several times during data taking using reconstructed tracks from collisions
and cosmic rays data, and later on, further refined after the data taking
period is finished. The tracker alignment performance corresponding to the
ultimate accuracy of the alignment calibration for the legacy reprocessing of
the CMS Run 2 data is presented. The data-driven methods used to derive the
alignment parameters and the set of validations that monitor the performance
of physics observables after the alignment are reviewed. Finally, the
prospects for the alignment calibration during the upcoming run of the LHC,
where more challenging operation conditions are expected, will be addressed.

##  1 The CMS tracker

The CMS Tracker comprises both the silicon pixel and strip detectors within a
length of 5.8 m and a diameter of 2.5 m [1]. The silicon pixel detector is the
closest detector in proximity to the interaction point and consists of a
barrel region (BPIX) and two forward endcaps (FPIX). In the configuration
operating until the end of the 2016 data-taking period (Phase-0 pixel
detector), the BPIX consisted of three layers and the FPIX of two disks in
each of the two endcaps, for a total of 1440 modules. The upgraded pixel
detector in operation since 2017 (Phase-1 pixel detector) has an additional
barrel layer in BPIX and an additional disk on each of the two endcaps of the
FPIX, which results in a total of 1856 modules [2]. The silicon strip detector
consists of 15 148 modules and is composed of four subsystems: the Tracker
Inner Barrel and Disks (TIB and TID), the Tracker Outer Barrel (TOB), and the
Tracker EndCaps (TEC).
The determination of the trajectory of charged particles (tracks) from signals
(hits) in the tracker, referred to as tracking, is instrumental for the
process of event reconstruction in CMS, with a good tracking performance of
particular relevance for many physics analyses. The measurement of the
momentum of the tracks relies on an accurate determination of the track
curvature induced by the magnetic field, which in turn requires a carefully
calibrated detector. The resolution of the momentum measurement is affected by
multiple scattering and by the limited knowledge of the position of each of
the track hits. The latter has contributions from two main components, the
spatial resolution of the modules, known as intrinsic hit resolution
($\approx$ 10 to 30 μm), and the precision that comes from the limited
knowledge of the position and orientation of the modules, known as alignment
position errors. The reduction of this last component well below the intrinsic
hit resolution is needed in order to fully exploit the remarkable resolution
of the CMS silicon modules. Starting from a precision after installation of
$\mathscr{O}$(100 μm), this is accomplished through a track-based alignment.

##  2 Alignment of CMS tracker

The alignment of the CMS tracker using reconstructed charged particles, known
as track-based alignment, constitutes a major challenge due to the enormous
number of degrees of freedom involved. A measured hit position is assigned to
every hit registered in the detector and a set of tracks is formed from the
combination of these hits. To each of these tracks, a set of track parameters
(e.g., parameters related to the track curvature and deflection angles due to
multiple scattering) is associated. The fitted tracks depend on so-called
alignment or global parameters as well, with nine of these parameters needed
per sensor to set the coordinates of the sensor center, rotational angles, and
surface deformations [3]. The track-based alignment method follows a least-
square approach, minimizing the sum of squares of the normalized track-hit
residuals from a set of tracks. The track hit-residuals are obtained by
subtracting the projection of the track prediction which depends on the track
and alignment parameters from the measured hit position. The resulting system
of equations for the alignment parameters obtained from the minimization of
the track-hit residuals is solved through a global fit [4]. The tracks are
then re-fitted assuming the new geometry defined by the updated set of
alignment parameters and the measurement of the track momentum is also
updated. The validation of the new alignment conditions follows.

##  3 Tracker alignment strategy for data

The hierarchical structure of the CMS tracker allows aligning high-level
mechanical structures up to single modules. Each year, before the beginning of
the data taking for physics analysis, at least the high-level structures of
the tracker are aligned using the available cosmic ray data from the
commissioning of the detector. This initial alignment allows a first non-
refined determination of the alignment constants, which at this point can show
large movements with respect to the initially assumed geometry, due to
temperature and magnetic field changes, as well as the reinstallation of
detector components during the detector shutdown. These alignment constants
are then used online for the express processing of the data. An automated
alignment continuously monitors the high-level structure movements of the
pixel detector and automatically corrects the geometry if the alignment
corrections exceed certain thresholds. A track-based alignment at a higher
granularity level is also periodically run offline. A heterogeneous sample of
tracks crossing the detector at different angles and covering their full
active area to relate different detector components is used. The automated
alignment is then refined with periodic updates from these campaigns happening
in parallel. After the data-taking period is finished, the full statistics of
the dataset collected during the year is used to provide the alignment
conditions for the reprocessing of the data. The ultimate accuracy of the
alignment calibration is derived and used for the final or legacy reprocessing
of the data. For the 2016-2018 period, up to $\approx$ 700k parameters and 220
geometries for different intervals of validity were determined, covering the
significant changes of the alignment conditions over time.

##  4 Legacy results

The tracker geometry obtained from each alignment fit can be compared to the
starting geometry for the particular fit, to identify unusual movements or
systematic distortions artificially introduced by the $\chi^{2}$ minimization
of the track-hit residuals. Further validations are carried out, inspecting
observables that are particularly sensitive to the quality of the alignment.
The alignment constants for the modeling of the data in simulation were
derived separately for each data-taking year and tuned to emulate the effects
of the residual misalignment left in data after the legacy reprocessing. The
comparison of the performance of the different sets of alignment constants
used during data taking, for the end-of-year reconstruction, and the legacy
reprocessing is illustrated in this section.

###  4.1 Tracking performance

The distribution of the median of the track-hit residuals per module (DMRs)
constitutes a measure of the tracking performance. Each track is refitted
without the hit under consideration. Therefore, the computation of the
residuals is said to be unbiased. The distribution of the median should result
in a narrow and centered distribution around zero since the track-based
alignment performs the minimization of the sum of the track-hit residuals. The
width of the distribution constitutes a measure of the local precision of the
alignment calibration, while deviations of the mean from zero indicate
possible biases due to change of conditions in the detector. The DMRs are
monitored for all the tracker substructures, as shown for the barrel and
forward pixel detector in Figure 1, for the local x-coordinate of the modules.
A significant improvement for the legacy reprocessing over the alignment
during data taking and the end-of-year reconstruction is observed.

Figure 1: The distribution of median residuals is shown for the local-x
coordinate in the barrel pixel (left) and forward pixel (right). The alignment
constants used for the Run 2 legacy reprocessing (green) are compared with the
ones used during data taking (blue) and for the end-of-year reconstruction
(red). The quoted means $\mu$ and standard deviations $\sigma$ correspond to
the parameters of a Gaussian fit to the distributions [5].

###  4.2 Vertexing performance

The effect of the alignment calibration on the reconstruction of physics
objects is also studied. The distance between tracks and the vertex
reconstructed without the track under scrutiny (unbiased track-vertex
residuals) is studied to evaluate the performance of the alignment in the
pixel detector, searching for potential biases in the primary vertex
reconstruction. The mean value of the unbiased track-vertex residuals is shown
in Figure 2 for the transverse plane as a function of the processed
luminosity. The large alignment corrections that were needed at the beginning
of each year of data taking can be observed.

Figure 2:  The mean track-vertex impact parameter in the transverse plane
$d_{xy}$ is shown as a function of the processed luminosity. The vertical
black lines indicate the first processed run for each data-taking year, and
the vertical dotted lines indicate a change in the pixel tracker calibration
[6].

###  4.3 Monitoring of systematic distortions

Systematic distortions of the tracker geometry, known as weak modes, can cause
biases in the track reconstruction, affecting the performance of physics
observables. In an ideally aligned tracker, the reconstructed invariant mass
of resonances such as the Z $\rightarrow\mu\mu$ decay should minimally depend
on the azimuthal angle or the pseudorapidity of the muons. Therefore, the
reconstructed mass of the resonance serves as a good observable to inspect the
geometry and detect the presence of weak modes. The improvement for the legacy
reprocessing in the uniformity of the reconstructed mass of the Z
$\rightarrow\mu\mu$ decay is demonstrated in Figure 3. The overlap validation
constitutes another tool to monitor the alignment looking for systematic
distortions, in this case by using hits from tracks that have two hits in
separate modules within the same layer of the tracker. Due to the small
distance between the two hits, there is a small uncertainty in the propagation
of the track parameters, and the double difference of the track hit-residuals
is very sensitive to systematic distortions. The mean overlap residuals are
shown in Figure 4 as a function of the processed luminosity. The overall
reduction of the residual values can be observed for the legacy reprocessing.

Figure 3: Reconstructed Z $\rightarrow\mu\mu$ mass as a function of the
difference in pseudorapidity $\eta$ between positively and negatively charged
muons. The full sample of dimuon events in the 2016-2018 period was used [5].
Figure 4: Mean difference in $\phi$ residuals for modules overlapping in the
$\phi$ direction in the barrel pixel as a function of the processed luminosity
[5, 6].

##  5 Run 3 prospects

The integrated luminosity collected by the CMS experiment is expected to be
doubled during Run 3 with respect to Run 2. Stronger variations of the Lorentz
drift of charged carriers released by charged particles passing through the
silicon sensors are foreseen due to larger irradiation doses. The alignment
procedure is sensitive to Lorentz drift changes induced by accumulated
radiation after $\approx$ 1 fb-1, while the pixel local reconstruction
calibration which corrects for this effect is only performed after $\approx$
10 fb-1. If the alignment is performed at a high enough granularity, inward
and outward-pointing modules are free to move separately and the bias coming
from Lorentz angle miscalibration can be absorbed. Therefore, a finer
granularity for the automated alignment is expected to be deployed during Run
3. A finer granularity will be used at earlier stages with respect to Run 2
for the alignment run offline as well, to better cope with radiation effects.
In addition, more geometries will be derived to cover the significant changes
over time.

##  6 Summary

The performance of the tracker alignment corresponding to the final set of
alignment conditions used for the legacy reprocessing of the CMS Run 2 data
has been presented. The data-driven methods used to derive the alignment
parameters and the set of validations that monitor the tracking and vertexing
performance, as well as the presence of systematic distortions in the detector
geometry, were reviewed. Finally, the prospects for the alignment calibration
during Run 3, in particular, the challenges related to the radiation damage
and the rapidly changing conditions were discussed.

## References

  * [1] CMS Collaboration, _The CMS Experiment at the CERN LHC_ , 2008 JINST 3 S08004, doi:10.1088/1748-0221/3/08/S08004.
  * [2] CMS Collaboration, _CMS Technical Design Report for the Pixel Detector Upgrade_ , CMS-TDR-011, CERN-LHCC-2012-016 (2012), doi:10.2172/1151650.
  * [3] V. Blobel and C. Kleinwort, _A new method for the high-precision alignment of track detectors_ , Proceedings of Conference on Advanced Statistical Techniques in Particle Physics, Durham, UK, 2002, https://inspirehep.net/literature/589639.
  * [4] CMS Collaboration, _Alignment of the CMS tracker with LHC and cosmic ray data,_ 2014 JINST 9 P06009, doi:10.1088/1748-0221/9/06/P06009.
  * [5] CMS Collaboration, _CMS Tracker Performance results for full Run 2 Legacy reprocessing_ , CMS-DP-2020-012, 2020, https://cds.cern.ch/record/2713208.
  * [6] CMS Collaboration, _Additional Run 2 CMS Tracker Alignment Performance Results,_ CMS-DP-2020-038, 2020, https://cds.cern.ch/record/2727090.


morphological and mass models for these two objects would be too uncertain to
compute their angular momentum.

##  Appendix D Deriving the angular momentum

Our goal is to derive the angular momentum of the stars in the MAGIC survey.
We can split distribution of stars into two main components: the stellar disk
and the stellar bulge. Among the two, the disk is the simplest to derive its
angular momentum because of the assumptions that are usually made on its
geometry and on its kinematics. Besides, bulges are usually thought of as
stellar systems dynamically supported by random motions. In our case, because
we model the bulge component as a spherical system, this means that its
angular momentum should be nearly null. Thus, we will make the assumption that
the bulge component does not posses any angular momentum and we will focus on
deriving that of the disk component only.

###  D.1 General derivation

The angular momentum of a system is defined from its 3D mass distribution
$\rho(\bm{r})$ and 3D velocity field $\bm{V}(\bm{r})$ as

$\bm{J}=\int_{\mathcal{V}}d\tau\leavevmode\nobreak\
\rho(\@vec{r})\leavevmode\nobreak\ \bm{r}\times\@vec{V}(\@vec{r}),$ (10)

where $\mathcal{V}$ represents the volume over which the integral is computed.
In cylindrical coordinates, the cross product writes

$\@vec{r}\times\@vec{V}(\@vec{r})=RV_{\theta}\leavevmode\nobreak\
\@vec{\hat{z}}+(zV_{R}-RV_{z})\leavevmode\nobreak\
\@vec{\hat{\theta}}-zV_{\theta}\leavevmode\nobreak\ \@vec{\hat{R}},$ (11)

where $\@vec{V}(\@vec{r})=V_{R}\leavevmode\nobreak\
\@vec{\hat{R}}+V_{\theta}\leavevmode\nobreak\
\@vec{\hat{\theta}}+V_{z}\leavevmode\nobreak\ \@vec{\hat{z}}$191919In the
following, when the velocity field is fully described by $V_{\theta}$, we will
refer to it as the galaxy rotation curve.. Assuming we want to derive the
angular momentum of a disk-like distribution where only the tangential
component of the velocity field is non-zero and does not depend on $z$, that
is $V_{R}=V_{z}=0$ and $V_{\theta}=V_{\theta}(R,\theta)$, then the angular
momentum simplifies to

$\@vec{J}=\int_{\mathcal{V}}d\theta\leavevmode\nobreak\ dR\leavevmode\nobreak\
dz\leavevmode\nobreak\ \rho(\@vec{r})\leavevmode\nobreak\
\left[R^{2}V_{\theta}(R,\theta)\leavevmode\nobreak\
\@vec{\hat{z}}-RzV_{\theta}(R,\theta)\leavevmode\nobreak\
<EMAIL_ADDRESS>(12)

###  D.2 Angular momentum of thick disks

The angular momentum of a disk galaxy with a non-zero thickness profile $f(z)$
is similar to that of a razor-thin disk galaxy with the same surface mass
density $\Sigma_{M}(R)$ if the three following criteria are met:

  1. (i)

$\rho(\@vec{r})=\Sigma_{\rm{M}}(R,\theta)f(z)$,

  2. (ii)

$\int_{R}dz\leavevmode\nobreak\ f(z)=1$,

  3. (iii)

$f(z)=f(-z)$.

Criterion (i) means that it is possible to separate the surface mass density
profile from the thickness profile. For instance, this is the case for a
double exponential profile where $f(z)\leavevmode\nobreak\
=\leavevmode\nobreak\ \exp\left\\{-|z/h_{z}|\right\\}/h_{z}$. Among the three
conditions, criterion (ii) is always met since it is required in order to
recover the surface mass density when integrating the 3D mass distribution
along the vertical axis with respect to the disk plane, that is
$\Sigma_{\rm{M}}(R,\theta)=\int_{\mathbb{R}}dz\leavevmode\nobreak\
\rho(\@vec{r})$. Finally, criterion (iii) is usually assumed to simplify the
shape of the vertical profile. If these three conditions are met, one can see
that the rightmost integral in Eq. 12 vanishes when integrating along the
vertical axis, whereas only the $R$ and $\theta$ integrations remain in the
leftmost one. Thus, we get an angular momentum integral similar to that of a
razor-thin disk202020For a razor-thin disk we have $f(z)=\delta(z)$, with
$\delta$ a Dirac distribution, so criteria (i) to (iii) are also met.

$\@vec{J}=\int_{\mathcal{S}}d\theta\leavevmode\nobreak\ dR\leavevmode\nobreak\
R^{2}\leavevmode\nobreak\
\Sigma_{\rm{M}}(R,\theta)V_{\theta}(R,\theta)\leavevmode\nobreak\
\@vec{\hat{z}},$ (13)

where $\mathcal{S}$ represents the surface over which the $R$ and $\theta$
integrations are carried out. It is common practice to normalise the angular
momentum by the total stellar mass to define a specific angular momentum
$j=|\@vec{J}|/M_{\star}$ (e.g. Fall 1983; Romanowsky & Fall 2012; Bouché et
al. 2021) but it is also possible to normalise it by the mass of the disk
only. For instance, if we normalise the angular momentum measured over some
surface $\mathcal{S}$ by the total disk mass, we get

$j=\frac{\int_{\mathcal{S}}d\theta\leavevmode\nobreak\ dR\leavevmode\nobreak\
R^{2}\leavevmode\nobreak\
\Sigma_{\rm{M}}(R,\theta)V_{\theta}(R,\theta)}{\int_{[0,2\pi)\times\mathbb{R}_{+}}d\theta\leavevmode\nobreak\
dR\leavevmode\nobreak\ R\leavevmode\nobreak\ \Sigma_{\rm{M}}(R,\theta)},$ (14)

or in the case where there is no dependence on $\theta$ for both the surface
mass density and the velocity:

$j=\frac{\int_{R}dR\leavevmode\nobreak\ R^{2}\leavevmode\nobreak\
\Sigma_{\rm{M}}(R)V_{\theta}(R)}{\int_{\mathbb{R}_{+}}dR\leavevmode\nobreak\
R\leavevmode\nobreak\ \Sigma_{\rm{M}}(R)}.$ (15)

###  D.3 Angular momentum for special rotation curves

In most general terms, if one wants to compute the specific angular momentum
for any surface mass density and velocity profile, one needs to numerically
integrate Eq. 14 or 15 depending on whether there is a $\theta$ dependence or
not. However, there exist for a few specific velocity profiles simplified
formulae which can be computed either numerically or analytically in certain
cases. We present in the following simplified expressions for some rotation
curves, assuming no $\theta$ dependence on both the surface mass density and
the rotation curve.

####  D.3.1 Constant rotation curve

If one assumes a constant rotation curve $V_{\theta}(R)=V_{\theta}$, then the
angular momentum up to radius $r$ is given by

$j(r)=V_{\theta}\times
M_{2}\\{\Sigma_{\rm{M}}\\}_{0}^{r}/M_{1}\\{\Sigma_{\rm{M}}\\}_{0}^{\infty},$
(16)

where $M_{k}\\{\Sigma_{\rm{M}}\\}_{\alpha}^{\beta}$ is the $k^{\rm{th}}$
radial moment of the surface brightness distribution $\Sigma_{\rm{M}}$
integrated between $\alpha$ and $\beta$, that is

$M_{k}\\{\Sigma_{\rm{M}}\\}_{\alpha}^{\beta}=\int_{\alpha}^{\beta}dR\leavevmode\nobreak\
R^{k}\leavevmode\nobreak\ \Sigma_{\rm{M}}(R).$ (17)

####  D.3.2 Flat model

If one assumes the rotation curve to be a flat model as in Abril-Melgarejo et
al. (2021), that is described by the following formula:

$V_{\theta}(R)=V_{\rm{t}}\times\cases{R}/R_{\rm{t}},&\mbox{if }R<R_{\rm{t}}\\\
1,\mbox{otherwise}{}$ (18)

then the angular momentum is given by

$j(r)=\frac{V_{\rm{t}}}{R_{\rm{t}}M_{1}\\{\Sigma_{\rm{M}}\\}_{0}^{\infty}}\times\cases{M}_{3}\\{\Sigma_{\rm{M}}\\}_{0}^{R},&\mbox{if
}R<R_{\rm{t}}\\\
M_{3}\\{\Sigma_{\rm{M}}\\}_{0}^{R_{\rm{t}}}+R_{\rm{t}}M_{2}\\{\Sigma\\}_{R_{\rm{t}}}^{r},\mbox{otherwise}{}$
(19)

###  D.4 Angular momentum for a Sérsic profile

If the surface mass density can be further described by a Sérsic profile
(Sérsic 1963) with Sérsic index $n$, effective radius $R_{\textit{eff}}$ and
central surface mass density $\Sigma_{{\rm{M}},0}$, then we have

$\@vec{J}=\Sigma_{M,0}\int_{R}dR\leavevmode\nobreak\
R^{2}e^{-b_{n}(R/R_{\textit{eff}})^{1/n}}\int_{\theta}d\theta\leavevmode\nobreak\
V_{\theta}(R,\theta)\leavevmode\nobreak\ \@vec{\hat{z}},$ (20)

where $b_{n}$ is the solution of the equation
$\Gamma\left(2n\right)\leavevmode\nobreak\ =\leavevmode\nobreak\
2\gamma\left(2n,b_{n}\right)$ (Graham et al. 2005), with $\gamma$ and $\Gamma$
the lower incomplete and complete gamma functions, respectively. Without any
$\theta$ dependence, the specific angular momentum writes

$j=\frac{2\leavevmode\nobreak\
b_{n}^{2n}}{(2n)!}\int_{R}dR\leavevmode\nobreak\
(R/R_{\textit{eff}})^{2}\leavevmode\nobreak\ V_{\theta}(R)\leavevmode\nobreak\
e^{-b_{n}(R/R_{\textit{eff}})^{1/n}},$ (21)

where the normalisation factor comes from the analytical expression for the
total mass of a Sérsic profile (e.g. Mercier et al. 2022). In the case of a
flat rotation curve as in Eq. 18, it is possible to compute analytically Eq.
17 and therefore Eq. 19. Indeed, we have

$\begin{split}M_{k}\\{\Sigma_{\rm{M}}\\}_{\alpha}^{\beta}=\frac{n\leavevmode\nobreak\
\Sigma_{{\rm{M}},0}\leavevmode\nobreak\
R_{\textit{eff}}^{k+1}}{b_{n}^{n(k+1)}}\left[\gamma\left(n(k+1),b_{n}\left(\frac{\beta}{R_{\textit{eff}}}\right)^{1/n}\right)\right.\\\
\left.-\gamma\left(n(k+1),b_{n}\left(\frac{\alpha}{R_{\textit{eff}}}\right)^{1/n}\right)\right]\end{split}.$
(22)

###  D.5 Recovering the RF12 approximation

The Romanowsky & Fall (2012) approximation, hereafter referred as RF12,
assumes an exponential disk with a constant rotation curve and computes the
angular momentum at infinity. Thus, we can use Eq. 19 and 22 to check whether
we recover it. First, we can see that for a flat model and in the limit where
the rotation curve becomes constant, that is $R_{\rm{t}}\rightarrow 0$, Eq. 19
reduces to 16. Furthermore, evaluated at infinity and for an exponential disk
model (i.e. $n=1$) Eq. 22 simplifies to

$M_{k}\\{\Sigma_{\rm{M}}\\}_{0}^{\infty}=\Sigma_{\rm{M},0}R_{\rm{d}}^{k+1}\Gamma(k+1),$
(23)

where $\Gamma$ is the complete gamma function and
$R_{\rm{d}}=R_{\textit{eff},\rm{d}}/b_{1}$ is the disk scale length, with
$b_{1}\approx 1.6783$. Thus, given Eq. 16, we recover the Romanowsky & Fall
(2012) approximation

$j=2V_{\theta}R_{\rm{d}}.$ (24)

###  D.6 Angular momentum for an exponential disk in equilibrium

It can be interesting to derive the angular momentum for an exponential disk
model that is stable against its own gravity, that is without any DM halo or
bulge component. We consider the case where the normalisation is taken at the
same radius as the angular momentum. Given the expression of the integrated
mass and circular velocity for such a model (see Eqs. C.1, D.8, and D.9 of
Mercier et al. (2022), its integral writes

$j(R)=\frac{\sqrt{\pi GR_{\rm{d}}\Sigma_{\rm{M,d}}(0)}}{2\pi
R_{\rm{d}}^{3}\gamma(2,R/R_{\rm{d}})}\int_{0}^{R}dr\,r^{3}e^{-r/R_{\rm{d}}}f\left(\frac{r}{2R_{\rm{d}}}\right),$
(25)

where $G$ is the gravitational constant, $R_{\rm{d}}$ is the disk scale
length, $\Sigma_{\rm{M,d}}(0)$ is the central surface mass density of the
disk, $\gamma$ is the lower incomplete gamma function, and $f$ is a function
of Bessel functions defined in Appendix D of Mercier et al. (2022). If we
define $y=x/R_{\rm{d}}$, then it simplifies to

$j(R)=\frac{\sqrt{GM_{\rm{d}}(R)R_{\rm{d}}}}{2\sqrt{2}\pi\gamma^{2}(2,R/R_{\rm{d}})}\times\int_{0}^{R/R_{\rm{d}}}dy\,y^{3}e^{-y}f(y/2),$
(26)

where $M_{\rm{d}}(R)$ is the mass of the disk within the radius $R$. At
$R_{22}=2.2R_{\rm{d}}$, the radius of maximum velocity for the disk, both
$\gamma$ and the integral on the right-hand side become constants and Eq. 26
further reduces to

$j(R_{22})\,[\rm{kpc\,km\,s^{-1}}]\approx 3.373\times
10^{-4}\times\sqrt{M_{\rm{d}}(R_{22})R_{\rm{d}}},$ (27)

where $M_{\rm{d}}(R_{22})$ is in and $R_{\rm{d}}$ in kpc.

##  Appendix E Catalogue of the main sample

Table 4: Column description of the catalogue used in this analysis.

No. | Title | Description
---|---|---
1 | ID | MUSE galaxy ID in the form X-CGrY, where X refers to the galaxy identification
|  | number within the field targeting COSMOS group CGrY
2 | z | Spectroscopic systemic redshift derived from the kinematics modelling
3 | RA | J2000 Right Ascension of morphological centre in decimal degrees
4 | Dec | J2000 Declination of morphological centre in decimal degrees
5 | N | Number of galaxies in the host structure. Field galaxies have a value of -99.
6 | R/R200 | Ratio of the projected distance of a galaxy to its host group’s centre with the radius of the
|  | group $R_{200}$ (see Sect. 2.3). Field galaxies have a value of -99.
7 | dV/sigma | Ratio of the systemic velocity of a galaxy along the line-of-sight with its host group’s
|  | velocity dispersion (see Sect. 2.3). Field galaxies have a value of -99.
8 | Eta | Environmental tracer defined in Sect. 2.3, combining parameters 6 and 7. Field galaxies have
|  | a value of -99.
9 | log over-density | Logarithm of the over-density around the galaxies as measured by the VMC technique
|  | (see the end of Sect. 2.3)
10 | log j_HST_MM | Logarithm of the specific stellar angular momentum () derived from HST images
|  | using the mass model rotation curve
11 | log j_HST_flat | Logarithm of the specific stellar angular momentum () derived from HST images
|  | using the flat model rotation curve
12 | log j_exp_MM | Logarithm of the specific stellar angular momentum () derived by numerically
|  | integrating Eq. 6 assuming an exponential disk mass distribution and a mass
|  | model rotation curve
13 | log j_exp_flat | Logarithm of the specific stellar angular momentum () analytically derived
|  | assuming an exponential disk mass distribution and a flat model rotation curve (see
|  | Appendix D)
14 | Gas dispersion | Average velocity dispersion of the ionised gas () measured in the velocity dispersion map
|  | of the [O ii] doublet after correcting for the LSF and beam-smearing
15 | log M* | Logarithm of the stellar mass (M_⊙)

Notes: The sample used in this catalogue corresponds to the 182 galaxies
selected in terms of their surface, S/N, and stellar disk inclination (see
Sect. 2.4). Complementary physical, morphological, and dynamical information
can be found in the morpho-kinematics catalogue published in Mercier et al.
(2022, catalogues can be cross-matched using the ID field). All logarithmic
values are given as decimal logarithms.