# EdgeLoc: A Communication-Adaptive Parallel System for Real-Time Localization
in Infrastructure-Assisted Autonomous Driving

Boyi Liu, Jingwen Tong, Yufan Zhuang, Jiawei Shao, Jun Zhang The authors
information.

###### Abstract

This paper presents EdgeLoc, an infrastructure-assisted, real-time
localization system for autonomous driving that addresses the incompatibility
between traditional localization methods and deep learning approaches. The
system is built on top of the Robot Operating System (ROS) and combines the
real-time performance of traditional methods with the high accuracy of deep
learning approaches. The system leverages edge computing capabilities of
roadside units (RSUs) for precise localization to enhance on-vehicle
localization that is based on the real-time visual odometry. EdgeLoc is a
parallel processing system, utilizing a proposed uncertainty-aware pose fusion
solution. It achieves communication adaptivity through online learning and
addresses fluctuations via window-based detection. Moreover, it achieves
optimal latency and maximum improvement by utilizing auto-splitting vehicle-
infrastructure collaborative inference, as well as online distribution
learning for decision-making. Even with the most basic end-to-end deep neural
network for localization estimation, EdgeLoc realizes a 67.75% reduction in
the localization error for real-time local visual odometry, a 29.95% reduction
for non-real-time collaborative inference, and a 30.26% reduction compared to
Kalman filtering. Finally, accuracy-to-latency conversion was experimentally
validated, and an overall experiment was conducted on a practical cellular
network. The system is open sourced at
https://github.com/LoganCome/EdgeAssistedLocalization.

###### Index Terms:

Connected vehicles, localization, infrastructure assisted.

##  1 Introduction

Localization is pivotal for autonomous vehicles, laying the groundwork for
subsequent navigational tasks. Traditional systems, employing sensor fusion
via Kalman filters, combine data from Inertial Measurement Units (IMU), Global
Positioning System (GPS), and visual odometry (VO) for consistently accurate
positioning. Recently, deep neural networks (DNNs) have emerged, offering end-
to-end (E2E) vision-based localization solutions capable of directly computing
the vehicle’s absolute pose with remarkable precision. Nevertheless, their
applications within autonomous driving are hindered by concerns over
reliability, computational demands, and latency during inference.

As shown in Fig. 1, our paper bridges the gap between traditional and deep
learning localization methods by proposing the deployment of E2E localization
DNNs on roadside units (RSUs), leveraging their edge computing capabilities to
bolster autonomous driving localization. This edge-assisted innovative
approach, however, introduces a set of challenges:  1) ensuring real-time
operations, 2) establishing robust vehicle-to-infrastructure communications,
3) integrating RSU-provided localization with on-vehicle systems, and 4)
optimizing the entire framework for live use without sacrificing accuracy or
safety.

Figure 1: Challenges and system overview of deploying end-to-end localization
deep neural networks on roadside units for infrastructure-assisted autonomous
driving.

To address these challenges, we develop EdgeLoc, a communication-adaptive
parallel system for real-time localization in infrastructure-assisted
autonomous driving. Unlike traditional infrastructure-assisted methods
focusing on minimizing latency for real-time performance but suffering from
uncontrollable network conditions, we propose a novel ROS-based system
framework that incorporates two parallel threads: real-time visual odometry
and non-real-time DNN inference. By integrating the real-time performance of
traditional methods with the edge computing-enhanced high accuracy of deep
learning approaches, our system leverages precise localization from a roadside
unit as the infrastructure to improve localization based on the real-time on-
vehicle visual odometry. This provides a novel and practical solution for
utilizing DNN models in real-time localization for autonomous driving.

The implementation of EdgeLoc encompasses several key developments:

  * •

The design and execution of a parallel processing system harmonizes VO and DNN
inference, optimizing both for real-time performance and accuracy.

  * •

The development of the pose fusion technique circumvents the limitations of
Kalman filtering with DNNs, addressing the challenges posed by the black-box
nature of DNNs and the potential for outliers in pose outputs.

  * •

It is critical to enhance system robustness to withstand variations in
communication status, enabling a crucial factor for the unpredictability of
real-world vehicle-to-infrastructure (V2I) communication scenarios.

  * •

Ensuring that the framework not only improves on-vehicle localization accuracy
but also maximizes this enhancement under operational conditions.

In this work, we address the above key technologies and implement EdgeLoc. The
main contributions are as follows:

  1. 1.

We propose a ROS-based framework comprising RSU, VO, and decision modules to
enable parallel collaborative computation under an action mechanism.

  2. 2.

A novel method is proposed to estimate the uncertainty of poses from the RSU,
addressing the reliability concerns associated with the black-box nature of
deep learning models and potential outliers in pose outputs. Notably, the
uncertainty estimation approach recognizes that localization accuracy is
contingent on both RSU latency and DNN pose uncertainty estimation. Therefore,
the estimation method is uniquely correlated to latency, thereby transforming
the localization accuracy optimization into a single objective of inference
latency optimization.

  3. 3.

Through auto-splitting of vehicle-infrastructure collaborative inference, the
system achieves optimal latency, and the maximum localization improvement is
guaranteed via online learning for decision-making.

  4. 4.

The online learning module seamlessly handles communication state transitions
and incorporates a window-based out-of-distribution solution to fortify
robustness against network fluctuations.

The remainder of this paper is organized as follows: Section 2 presents the
related work in this field. Section 3 describes how EdgeLoc achieves real-time
performance and solves Challenge 1 shown in Fig. 1. Section 4 explains how
EdgeLoc performs localization fusion and solves Challenge 3 shown in Fig. 1.
Section 5 discusses how EdgeLoc optimizes communication decision-making and
solves Challenge 2 shown in Fig. 1. Section 7 concludes the paper. Appendix A
provides proof of the optimality of the proposed method, addressing challenge
4 shown in Fig. 1.

##  2 Related Works

###  2.1 Localization in Autonomous Driving

Localization is a crucial task in autonomous driving [1], ensuring that
vehicles ascertain their exact localization within their surroundings.
Existing methods can be categorized into sensor-based integral localization
methods, such as visual odometry and LiDAR-based techniques, and absolute
localization systems like GPS. Recently, DNN-based methods have emerged as a
promising approach for end-to-end localization in modern autonomous vehicles.

####  2.1.1 Sensor-based Integral Localization

Relative localization techniques employ sensors like IMUs, cameras, and LiDARs
for real-time position estimation. For example, IMUs measure acceleration and
angular velocity via accelerometers and gyroscopes, respectively, and
integrate these data to estimate the vehicle’s position, velocity, and
orientation over time. However, this integration process can lead to drift and
error accumulation due to noise and biases. VO [2] analyzes camera image
sequences to compute motion by extracting and matching visual features between
consecutive frames, and then integrates these estimates to track the vehicle’s
trajectory [3, 4]. Despite its higher accuracy, VO is prone to cumulative
errors in feature-scarce environments [2]. On the other hand, LiDAR generates
detailed 3D maps through laser scanning, producing point clouds that aid in
accurate trajectory mapping [5]. Although effective in diverse conditions,
LiDAR demands significant computational resources.

Given the limitations of individual sensors regarding error accumulation and
environmental dependency [6, 7], sensor fusion techniques such as Kalman
filters or particle filters are commonly employed. These algorithms integrate
data from multiple sensors like IMUs, VO, LiDAR, and radar, to produce more
reliable and accurate localization estimates, thereby overcoming the
shortcomings of single-sensor systems.

####  2.1.2 Absolute Localization

Absolute localization techniques determine geographic coordinates using
external data such as satellite systems or pre-mapped environments. Two
primary approaches are Global Navigation Satellite Systems (GNSS) and DNN-
based methods [8].

The most widely used GNSS is GPS, which calculates position through
trilateration using signals from multiple satellites [9]. In addition, the
robustness and accuracy of the GPS localization can be enhanced by integrating
GPS with inertial navigation systems (INS) [10]. The INS compensates for GPS
signal loss, while GPS bound the error accumulation in INS.

DNN-based end-to-end localization directly regresses camera pose from images.
PoseNet [8] pioneered this, inspiring further works to improve performance
through geometric loss functions [11], uncertainty modeling [12], and LSTM
units [13]. Enhancements in network architectures like hourglass networks [14]
and MapNet [15] have further improved localization performance.

Nonetheless, these absolute localization methods encounter significant
challenges. The GPS accuracy can be affected by factors such as signal
blockage, multipath effects, and atmospheric conditions, particularly in urban
environments with tall buildings or tunnels [16]. On the other hand, DNN-based
localization methods face challenges in terms of computational efficiency,
latency, and robustness to adverse weather conditions. Therefore, it is
necessary to develop multiple-sensor fusion or infrastructure-assisted
approaches to address these issues.

###  2.2 Infrastructure-assisted Autonomous Driving

Infrastructure-assisted autonomous driving represents a promising paradigm to
enhance the safety, efficiency, and reliability of autonomous vehicles.
Utilizing roadside infrastructure, such as sensors, communication devices, and
computing units, this approach not only provides additional contextual
information about the vehicle’s surroundings but also significantly improves
its localization precision [17]. By seamlessly integrating data from
infrastructure-based sensors with onboard systems, autonomous vehicles can
achieve more accurate positioning. This integration extends their sensing
range and enhances their perception capabilities [18]. The collaborative model
improves situational awareness [19, 20] and facilitates coordinated decision-
making between vehicles and infrastructure [21]. As a result, this leads to
safer, more efficient computing [22] and better road navigation [23, 24].

Recent research has explored the potential of infrastructure-assisted
autonomous driving in various contexts. For instance, Shi et al. [25]
developed VIPS, a real-time perception fusion system that achieves decimeter-
level accuracy and low latency, demonstrating the feasibility of integrating
infrastructure-based perception with vehicle-based systems. Liu et al. [21]
proposed a framework for infrastructure-assisted autonomous vehicles,
addressing challenges in bandwidth and latency variations. Their work
emphasizes the importance of optimizing communication protocols and
architectures. Moreover, Yang et al. [26] studied infrastructure-vehicle
collaborative autonomous driving, considering scenarios with a large number of
RSUs and vehicles, providing insights into the design and deployment of large-
scale V2X networks. Jia et al. [27] demonstrated the benefits of
infrastructure-assisted collaborative perception in automated valet parking,
improving safety and increasing the maximum safe cruising speed. Tsukada et
al. [28] introduced networked roadside perception units (RSPUs) for
infrastructure-based cooperative perception, demonstrating advanced
performance in delivering messages to autonomous vehicles within strict
latency constraints. In addition, Hirata et al. [29] proposed a cooperative
path-planning model for autonomous vehicles at intersections using future path
sharing based on information from RSUs, showing improved safety and
efficiency. These studies underscore the significance of real-time data
processing, communication, and cooperative planning in enhancing the
capabilities and performance of autonomous vehicles in complex environments.

Despite the advancements in infrastructure-assisted localization, there are
still limitations, such as high infrastructure costs, the need for optimal
configuration of V2I networks, and the lack of effective fusion between
infrastructure-based and vehicle-based techniques. To address these
limitations, the proposed EdgeLoc system aims to integrate infrastructure-
assisted and vehicle-based techniques using a parallel processing
architecture. By combining the benefits of high-precision absolute
localization from RSUs with real-time relative localization from onboard
visual odometry, EdgeLoc achieves centimeter-level accuracy at a lower cost
while exhibiting strong robustness and adaptability to dynamic environments.

##  3 Real-Time Parallel Architecture of EdgeLoc

Figure 2: System architecture of EdgeLoc: the core components, communication
setup, and collaborative operations between the autonomous vehicle and
roadside unit.

The real-time performance of the EdgeLoc system is achieved through a novel
system architecture and workflow optimization. Our system employs a parallel
processing architecture that is built on top of ROS to seamlessly integrate
real-time localization from on-vehicle sensors with precise localization
enhancements from roadside infrastructure. By leveraging ROS’s ability to
efficiently manage parallel processes and communication between distributed
nodes, our system achieves high performance and low latency. As illustrated by
the red arrows in Fig. 2, the autonomous vehicle (AV) performs real-time
localization based on its onboard sensors, while simultaneously interacting
with the RSU for collaborative inference to improve accuracy.

###  3.1 System Architecture and Core Modules

The architecture of the EdgeLoc system, depicted in Fig. 2, is designed to
augment real-time localization capabilities for AVs. This section presents the
key modules that ensure the system’s robust and adaptive performance, even in
dynamic network environments.

System overview: The operational paradigm of the EdgeLoc system is as follows:
The AV conducts real-time localization through its onboard sensors and the
preliminary layers of the DNN designed for localization. The AV Inference &
Request module autonomously partitions the DNN model at an optimal split point
determined by the UCB online learning module. The initial inference results
are transmitted from the AV to the RSU via the ROS Master communication
protocol. Subsequently, the RSU, equipped with sufficient computing resources,
processes the remaining layers of DNN inference and sends the localization
results back to the AV. The UCB module continuously learns and adjusts its
decision-making model for DNN partitioning in response to the prevailing
network conditions and observed latencies. A communication status change
detection submodule employs a sliding window technique to monitor latency
distributions and activates the UCB module to refine its decision model upon
detecting substantial shifts in network conditions. In scenarios where AVs and
RSUs operate across disparate network segments, Virtual Private Network (VPN)
and Network Address Translation (NAT) mechanisms are employed to facilitate
uninterrupted communication by converting private IP addresses into public IP
addresses.

The AV Inference & Request Module plays a pivotal role in the system by
dynamically partitioning the DNN localization model. This module intelligently
adapts the computational workload distribution between the AV and the RSU
based on the prevailing network conditions and available resources. By
transmitting early inference results to the RSU via the ROS Master
communication mechanism, EdgeLoc effectively minimizes latency while
maximizing localization accuracy.

The UCB Online Learning Module is built upon the principles of reinforcement
learning and multi-armed bandit algorithms. The UCB algorithm [30] is a well-
established approach for balancing exploration and exploitation in decision-
making processes. In the context of EdgeLoc, the UCB module learns a decision
model that maps the current network status to the optimal split point for the
DNN model. The algorithm maintains a confidence bound for each potential split
point and updates these bounds based on the observed latencies. By selecting
the split point with the highest UCB, the UCB module ensures that the system
explores new split points while exploiting the knowledge gained from previous
observations. The UCB algorithm has been theoretically proven to achieve
logarithmic regret, guaranteeing near-optimal performance over time.

The Communication Status Change Detection Module employs advanced statistical
techniques to detect significant changes in network conditions. The submodule
maintains Gaussian distributions of the observed latencies for each split
point and utilizes the Kullback-Leibler (KL) divergence to measure the
difference between the distributions. The KL divergence is a well-established
metric in information theory that quantifies the dissimilarity between two
probability distributions. By comparing the KL divergence between the current
and previous latency distributions, the submodule can detect significant
shifts in network conditions. To mitigate the impact of transient
fluctuations, the submodule employs a sliding window technique, requiring
consistent distribution differences over multiple consecutive frames before
triggering a change detection. This approach ensures that the system adapts to
genuine changes in network conditions while filtering out temporary
disturbances.

To address the temporal misalignment between the AV and RSU localization
results, EdgeLoc incorporates a Time Zone Localization Result Revision Module.
This module employs a two-stage correction process to align the localization
results obtained from the RSU with the current time step of the AV. The
mathematical formulas governing this correction process are elegantly
illustrated in the light red background of Fig. 2.

On the RSU side, the RSU Inference Module is responsible for completing the
remaining layers of the DNN inference upon receiving the early inference
results from the AV. This module leverages the computational resources of the
RSU to perform the final stage of the E2E localization, generating the DNN
based localization outputs that are subsequently transmitted back to the AVs.

In scenarios where the AVs and RSUs are not within the same network, EdgeLoc
seamlessly integrates network address translation mechanisms to facilitate
communication. By establishing a VPN, EdgeLoc enables seamless interaction
between devices in different subnets, assigning virtual IP addresses to allow
communication as if they were in the same local network. The system utilizes
an internet gateway and Elastic IP NAT to map the private IP addresses of the
AVs and RSUs to public IP addresses, enabling communication across different
network segments.

By leveraging this processing architecture and the collaborative inference
approach, EdgeLoc achieves real-time, high-accuracy localization for
autonomous vehicles. The system intelligently adapts to network conditions,
minimizes latency, and ensures robust performance even in the presence of
network fluctuations.

###  3.2 Parallelism of EdgeLoc

Figure 3: The diagram includes five modules (end-to-end localization modules
on the vehicle and RSU, sensor loop module, localization revision module, and
action server), vertical lines representing the lifecycle of each module,
arrows showing interactions and invocations between modules, and the parallel
execution of real-time localization and RSU collaboration threads.

EdgeLoc is designed with a focus on parallelism to achieve real-time
performance. The system architecture, as illustrated in the sequence diagram
(Fig. 3), demonstrates two parallel threads that work in tandem to provide
efficient and accurate localization results.

The first thread, based on the vehicle’s onboard sensors, operates in a
continuous loop to handle the real-time localization tasks. This thread is
responsible for processing the data from various sensors such as camera, IMU,
or LiDAR, to estimate the vehicle’s position and orientation. The localization
module in AV performs the necessary computations and updates the localization
results in real-time through integral calculation as described in Section
2.1.1. This thread ensures that the AV always has access to the most recent
localization information, enabling it to receive real-time localization
results.

The second thread, involving the communication between the AV and the RSU,
runs in parallel to the sensor-based localization thread. This thread is
triggered when the AV sends a request to the UCB module for determining the
optimal splitting point of the DNN model. Based on the decision from the UCB
module, the AV performs the initial inference up to the determined split point
and sends the intermediate results to the RSU. The RSU then continues the DNN
inference using its computational resources, and sends the DNN’s localization
result back to the AV. The AV’s localization module integrates this high-
precision information with its own sensor-based estimate to obtain a refined
and more accurate positioning result, benefiting from the collaborative
inference process without requiring powerful onboard computing hardware.

The RSU’s localization module leverages the DNN model to process the data
received from the AV and provide an DNN based localization result. The DNN
takes the AV’s sensor data as input and outputs a refined pose estimate by
extracting relevant features and learning the complex mapping between the
input data and the vehicle’s position. This approach allows the RSU to utilize
its computational resources to perform the computationally intensive DNN
inference and a more accurate localization result. By offloading some of the
DNN computation to the RSU, the AV can benefit from the improved accuracy
without the need for powerful onboard computing hardware, making the system
more cost-effective and efficient. This parallel thread enables the AV to
leverage the RSU’s processing capabilities and the DNN’s ability to learn from
data, ultimately enhancing the overall localization performance.

Once the RSU’s localization module completes its processing, it sends the
localization result back to the AV. The AV’s localization module receives this
result and integrates it with its own sensor-based localization estimate. This
fusion of the two localization results is performed using our proposed method
in Section 4, to obtain a refined and more precise localization estimate. The
updated localization information is then published to other modules in the AV,
such as the perception, planning, and control modules, to facilitate accurate
and safe autonomous driving.

The parallelism of EdgeLoc offers several advantages. First, it allows the AV
to maintain real-time localization capabilities even in challenging
environments where sensor-based localization alone may be insufficient.
Second, it enables the AV to leverage the computational resources and
additional information available at the RSU, reducing the burden on the AV’s
onboard systems. Third, the parallel architecture ensures that the
localization process remains resilient and fault-tolerant. If one thread
experiences issues or delays, the other thread can continue to provide
localization results, maintaining the overall system’s robustness.

Moreover, the parallel design of EdgeLoc facilitates scalability and
adaptability. As the autonomous driving infrastructure evolves and expands,
additional RSUs can be deployed to cover larger areas and provide localization
assistance to a growing fleet of AVs. The parallel architecture allows for
seamless integration of new RSUs and the efficient distribution of
localization tasks across multiple units.

In short, the parallelism of EdgeLoc is a key aspect of its design, enabling
real-time localization in infrastructure-assisted autonomous driving. The two
parallel threads, one based on the AV’s sensors and the other involving AV-RSU
communication, work together to provide accurate, reliable, and timely
localization results. This parallel architecture enhances the system’s
accuracy, robustness, scalability, and adaptability, making EdgeLoc a
promising solution for the localization challenges in autonomous driving.

##  4 Localization Fusion of EdgeLoc

In this section, we introduce the localization fusion algorithm implemented in
EdgeLoc. The unsuitability of the Kalman filter motivates us to propose a
method compatible with EdgeLoc, which transforms the localization accuracy
problem into a latency reduction problem.

###  4.1 Limitations of Kalman Filtering in End-to-End Localization

Kalman filtering is a widely used sensor data fusion technique in traditional
localization systems. However, significant challenges arise when applying it
to localization based on DNNs. The primary obstacle is the difficulty in
estimating the Kalman gain of DNNs, which is crucial for the proper
functioning of the Kalman filter.

In Kalman filtering [31], the estimation of the system state relies on prior
knowledge of the process noise covariance $Q$ and the measurement noise
covariance $R$. These covariances are used to determine the optimal Kalman
gain $K$, which balances the predicted state and the measured state. The
update equations for the Kalman filter can be rewritten as follows to match
the symbols in the subsequent localization equations:

$\displaystyle L_{r,k|k-1}$ $\displaystyle=AL_{r,k-1|k-1}+BL_{\beta,k},$ (1)
$\displaystyle P_{k|k-1}$ $\displaystyle=AP_{k-1|k-1}A^{T}+Q,$ $\displaystyle
K_{k}$ $\displaystyle=P_{k|k-1}(P_{k|k-1}+R)^{-1},$ $\displaystyle L_{r,k|k}$
$\displaystyle=L_{r,k|k-1}+K_{k}(L_{\alpha,k}-L_{r,k|k-1}),$ $\displaystyle
P_{k|k}$ $\displaystyle=(I-K_{k})P_{k|k-1}.$

In the above formulas, $L_{r}$ represents the fused localization result,
$L_{\alpha}$ represents the DNN-based localization output, $L_{\beta}$
represents the real-time localization result from on-vehicle sensors, $P$
represents the state covariance, $A$ and $B$ are system matrices, and $I$ is
the identity matrix, $k$ is the time step.

However, when dealing with DNN-based localization, the measurement noise
covariance $R$ cannot be reliably estimated. DNNs are often regarded as black-
box models, making it challenging to characterize their error distribution.
Unlike traditional sensors with well-defined noise models, DNN outputs may
exhibit complex and unpredictable error patterns. The lack of a reliable noise
model impedes the application of Kalman filtering to fuse DNN localization
results with other sensor data.

Let $e_{k}$ represent the measurement error in DNN localization:

$e_{k}=L_{\alpha,k}-L_{r,k|k-1}.$ (2)

For traditional sensors, the measurement error typically follows a zero-mean
Gaussian distribution, i.e., $e_{k}\sim N(0,R)$. However, for DNN
localization, due to its black-box nature, the measurement error distribution
is unknown and unpredictable. We assume the true error distribution is
$e_{k}\sim N(\mu_{k},\Sigma_{k})$, where $\mu_{k}$ and $\Sigma_{k}$ are the
unknown mean and variance of the DNN outputs.

Substituting equation 2 into the Kalman filtering update equation, we get:

$\displaystyle L_{r,k|k}$
$\displaystyle=L_{r,k|k-1}+K_{k}(L_{\alpha,k}-L_{r,k|k-1})$ (3)
$\displaystyle=L_{r,k|k-1}+K_{k}e_{k}$
$\displaystyle=L_{r,k|k-1}+K_{k}(\mu_{k}+\varepsilon_{k}),$

where $\varepsilon_{k}\sim N(0,\Sigma_{k})$. From equation 3, it is evident
that if the DNN localization exhibits an unknown bias $\mu_{k}\neq 0$, the
Kalman filter estimation will incorporate this bias term $K_{k}\mu_{k}$. This
means that the Kalman filter introduces anomalies and biases from DNN
localization into the final fused localization result. Additionally, since
$\Sigma_{k}$ is unknown, the calculation of the Kalman gain $K_{k}$ becomes
unreliable, further degrading the performance of fused localization.

Due to the recursive nature of the Kalman filter, any error in state
estimation not only affects the current state prediction and update but also
propagates through the entire system over time. If the system matrices $A$, or
the control inputs $B$ do not adequately compensate for these errors, or if
the error sources are persistent, this can lead to cumulative errors and
eventual system instability. Therefore, applying Kalman filtering to fusion
with DNN-based localization presents inherent challenges. In DNN localization,
the uncertainty, which Kalman filtering treats, cannot be effectively modeled
and managed by the filters, leading to reduced performance in fusion
localization. Consequently, it is necessary to propose a localization fusion
method adapted to EdgeLoc.

###  4.2 The Proposed Localization Fusion Method Adapted to EdgeLoc

To address the limitations of Kalman filtering and enable effective fusion of
DNN-based localization results, we propose a novel fusion method tailored to
the characteristics of DNN outputs. Our method leverages the temporal
consistency of localization results and introduces an uncertainty estimation
technique that adapts to the latency of DNN inference. The proposed fusion
method is as follows:

$\begin{cases}L_{r}(t)=uL_{\alpha}(t)+(1-u)L_{r}(t-\Delta t),\\\
\qquad\text{if }t=x\Delta t,x\in\mathbb{N}\\\
L_{r}(t+a)=L_{r}(t+a-1)+L_{\beta}(t+a)-L_{\beta}(t+a-1),\\\ \qquad\text{if
}t\neq x\Delta t,x\in\mathbb{N},a\in[1,\Delta t],\\\ \end{cases}$ (4)

where $L_{r}(t)$ represents the fused localization result at time $t$,
$L_{\alpha}(t)$ represents the DNN-based localization output at time $t$,
$L_{r}(t-\Delta t)$ represents the fused localization result at the previous
time step, and $L_{\beta}(t)$ represents the real-time localization result
from on-vehicle sensors at time $t$. Specifically, the fusion process operates
in two distinct cases:

  * •

When $t=n\Delta t$, where $n\in\mathbb{N}$, EdgeLoc fuses the DNN-based
localization output $L——\alpha(t)$ and the previous result $L_{r}(t-\Delta
t)$. The parameter $u$ is a weighted coefficient determined by the estimated
output uncertainty of the DNN.

  * •

When $t\neq n\Delta t$, the fused localization result is updated incrementally
using the difference between the real-time localization $L_{\beta}(t)$ and its
corresponding value at the fusion time step $L_{\beta}(t+i)$, where
$i\in[1,\Delta t]$.

One key component in our fusion method is the uncertainty estimation of the
DNN-based localization output. We propose a latency-aware uncertainty
estimation technique that adapts to the inference time of the DNN. The
uncertainty estimation $\Lambda$ and the weight $u$ are given by

$\Lambda=\frac{1}{1+e^{-k(\Delta t-\Delta t_{0})}},$ (5)

and

$u=f(\Delta t)=1-\Lambda,$ (6)

where $\Delta t$ represents the DNN inference latency, $\Delta t_{0}$ is a
reference latency value, and $k$ is a scaling factor. The function $f(\Delta
t)$ is a sigmoid function that maps the latency to an uncertainty value in the
range of 0 to 1. As the latency increases, the uncertainty of the DNN output
increases and the weight $u$ in (6) decreases.

###  4.3 Improving Localization by Transforming Accuracy Improvement to
Latency Reduction

By integrating latency-aware uncertainty estimation into the fusion process,
our proposed method effectively reformulates the problem of improving
localization accuracy as a latency reduction problem in DNN inference. Rather
than directly targeting accuracy improvement, our approach focuses on
minimizing the latency of DNN inference, which in turn leads to enhanced
overall localization accuracy.

The relationship between latency and accuracy can be derived from the fusion
equation:

$\begin{split}L_{r}(t)&=uL_{\alpha}(t)+(1-u)L_{r}(t-\Delta t)\\\ &=f(\Delta
t)L_{\alpha}(t)+(1-f(\Delta t))L_{r}(t-\Delta t)\\\ &=\frac{L_{r}(t-\Delta
t)}{1+e^{-k(\Delta t-\Delta t_{0})}}+\left(1-\frac{1}{1+e^{-k(\Delta t-\Delta
t_{0})}}\right)L_{\alpha}(t).\end{split}$ (7)

As the latency $\Delta t$ decreases, the weight assigned to the DNN output
$L\alpha(t)$ increases, leading to a higher contribution of the DNN result to
the fused localization. Conversely, as the latency increases, the weight of
the DNN output decreases, and the fused localization relies more on the
previous fused result and the real-time localization from on-vehicle sensors.

The latency-accuracy relationship can be further quantified by considering the
expected localization error:

$\displaystyle\mathbb{E}$ $\displaystyle[|L_{r}(t)-L_{gt}(t)|]$ (8)
$\displaystyle=\mathbb{E}\left[|uL_{\alpha}(t)+(1-u)L_{r}(t-\Delta
t)-L_{gt}(t)|\right]$ $\displaystyle\leq
u\mathbb{E}[|L_{\alpha}(t)-L_{gt}(t)|]$
$\displaystyle\quad+(1-u)\mathbb{E}[|L_{r}(t-\Delta t)-L_{gt}(t)|]$
$\displaystyle=f(\Delta t)\mathbb{E}[|L_{\alpha}(t)-L_{gt}(t)|]$
$\displaystyle\quad+(1-f(\Delta t))\mathbb{E}[|L_{r}(t-\Delta t)-L_{gt}(t)|],$

where $L_{gt}(t)$ is the ground truth localization at time slot $t$.

As the latency $\Delta t$ decreases, the weight $f(\Delta t)$ assigned to the
DNN error term increases, and the expected localization error becomes more
dependent on the accuracy of the DNN output. By minimizing the DNN inference
latency, we effectively reduce the expected localization error and improve the
overall accuracy of the fused localization result.

In summary, our proposed fusion method addresses the limitations of Kalman
filtering for DNN-based localization by introducing a latency-aware
uncertainty estimation technique. By transforming the problem of accuracy
improvement into a problem of latency reduction, we enable effective fusion of
DNN outputs with real-time localization results from on-vehicle sensors. The
fusion equation and the latency-accuracy relationship provide a mathematical
foundation for optimizing the localization performance in our system.

##  5 Communication Optimization of EdgeLoc

Building upon the real-time parallel architecture and localization fusion
techniques, we now present the communication optimization strategies employed
in EdgeLoc to ensure robust and efficient performance in dynamic network
environments. The communication between vehicles and RSUs is subject to time-
varying network conditions, which can significantly affect the latency of DNN
inference and, consequently, the localization accuracy. To address this
challenge, we formulate the problem of selecting the optimal DNN split point
as a multi-armed bandit problem, enabling online learning to adapt to network
dynamics.

###  5.1 Sliding Window UCB Algorithm for Split Point Selection

To address the exploration-exploitation trade-off in the multi-armed bandit
problem and efficiently learn the optimal split point, we employ the UCB
algorithm. UCB is a well-established algorithm that balances the exploration
of less-chosen split points with the exploitation of the currently best-
performing split point.

Consider a set of $K$ possible split points for the DNN model, denoted as
$\left\\{s_{1},s_{2},\ldots,s_{K}\right\\}$. Each split point represents a
different partition of the DNN computation between the vehicle and the RSU.
The objective is to select the split point that minimizes the expected
localization error while adapting to the time-varying network conditions. At
each time step $t$, the UCB algorithm selects the split point $I_{t}$ based on
the following rule:

$I_{t}=\underset{s_{i}}{\arg\max}\left(\bar{X}_{i}+\sqrt{\frac{16\hat{\sigma}_{i,n_{i}-1}^{2}\ln(t-1)}{n_{i}-1}}\right),$

where $\bar{X}_{i}$ is the empirical mean of the localization errors observed
for the split point $s_{i}$ up to time slot $t-1$. In addition, $n_{i}$ is the
number of times that the split point $s_{i}$ has been selected up to time
$t-1$. Term $\hat{\sigma}_{i,n_{i}-1}^{2}$ is the empirical variance of the
localization errors for split point $s_{i}$. The empirical mean $\bar{X}_{i}$
is calculated as:

$\bar{X}_{i}=\frac{1}{n_{i}}\sum_{o=1}^{t}X_{i,o}\cdot\textbf{1}_{I_{o}=i},$

where $X_{i,o}$ represents the real localization error at the $i$-th split
point and time slot $o$.

To adapt to the time-varying network conditions, we employ a sliding window
variant of the UCB algorithm. Instead of considering the entire history of
observations, the sliding window UCB algorithm only considers the most recent
$W$ observations for each split point. The empirical mean $\bar{X}_{i}$ is
calculated using the observations within the sliding window:

$\bar{X}_{i}=\frac{1}{n_{i}(W)}\sum_{o=t-W+1}^{t}X_{i,o}\cdot\textbf{1}_{I_{o}=i},$

where $n_{i}(W)$ is the number of times split point $s_{i}$ has been selected
until time slot $t-1$ within the sliding window $W$. In particular, the
sliding window UCB algorithm allows the system to quickly adapt to changes in
the network conditions by focusing on the most recent observations. The window
size $W$ can be tuned based on the expected rate of change in the network
dynamics. By combining the sliding window UCB algorithm with EdgeLoc’s
localization fusion method, the system adaptively selects the optimal split
point under time-varying network conditions, minimizing the localization
error. This approach ensures that the EdgeLoc system can effectively adapt to
dynamic network environments and maintain high localization accuracy.

###  5.2 Mitigating Network Fluctuations with Distribution Comparison

To further enhance the robustness of the EdgeLoc system against network
fluctuations, we introduce a distribution comparison mechanism in conjunction
with the sliding window UCB algorithm. The system maintains Gaussian
distributions of the localization errors for each split point within the
sliding window.

At each time step, the system compares the current distribution of
localization errors with the previous distribution for each split point. If a
significant change in the distribution is detected, it indicates a potential
change in the network conditions. However, to avoid triggering unnecessary
retraining of the UCB algorithm due to temporary fluctuations, the system
requires consistent distribution changes for a predefined number of
consecutive time steps.

The sliding window size $W$ and the number of consecutive time steps required
for triggering UCB retraining can be adjusted based on the specific
characteristics of the network environment. For example, setting the sliding
window size to 50 and requiring 3 consecutive distribution changes can provide
a balance between responsiveness and stability.

By incorporating the distribution comparison mechanism and the sliding window
approach, the EdgeLoc system can effectively mitigate the impact of network
fluctuations and maintain robust performance in dynamic network environments.

###  5.3 Seamless Integration with EdgeLoc

The UCB-based split point selection algorithm, along with the sliding window
and distribution comparison mechanisms, is seamlessly integrated into the
EdgeLoc system. The system continuously monitors the network conditions,
maintains the sliding window of localization errors for each split point, and
compares the distributions to detect changes in the network environment.

When a consistent change in the network conditions is detected, the system
triggers the retraining of the UCB algorithm to adapt to the new environment.
The UCB algorithm learns the optimal split point based on the updated network
conditions, ensuring optimal performance in terms of localization accuracy and
inference latency.

The integration of these online learning and adaptation mechanisms into
EdgeLoc enables it to dynamically adjust to varying network conditions,
maintaining high localization accuracy and efficient resource utilization in
infrastructure-assisted autonomous driving scenarios. By leveraging the
parallel architecture (Section 3) and the latency-accuracy relationship
established in the localization fusion method (Section 4), the communication
optimization strategies presented in this section complete the EdgeLoc system,
ensuring robust and adaptive performance in real-world deployments.

The mathematical proofs and detailed analysis of the adapted UCB algorithm for
the EdgeLoc system are provided in the Appendix A.

##  6 Experiments

In this section, we first introduce the experimental environment and settings.
First, we validate the efficacy of EdgeLoc in addressing the challenges of
DNN-based E2E localization and traditional sensor-based methods. Subsequently,
we present experimental data to illustrate the impact of latency on
localization performance, demonstrating the effectiveness of our proposed
localization fusion method in transforming the problem of accuracy improvement
into a latency reduction problem. Finally, we perform a comprehensive system-
level test in a 5G cellular network environment to assess the real-world
applicability of our proposed solution, especially for different network
conditions.

Dataset: Real-time localization data was from the RobotCar dataset [32],
specifically utilizing VO results derived from cameras. The recorded VO
datasets were streamed into the system via the ROS environment to emulate live
sensor data, replicating the sensor inputs that an autonomous vehicle would
provide for localization.

Evaluation Metrics: The primary metric for evaluating the efficacy of EdgeLoc
was localization accuracy, which served as the definitive measure of the
system’s performance. The system’s performance was distilled into this
singular metric by integrating uncertainty estimation and localization fusion
techniques into EdgeLoc.

###  6.1 Comparative Validation of EdgeLoc in Solving E2E and Sensor-Based
Localization Problems

Figure 4: Comparison of localization errors for different methods over time.
The methods Visual Odometry, DNN-based E2E Localization[8], and EdgeLoc are
respectively shown from the second to fourth images. The red curves represent
the ground truth track. The left first sub-figure is an example of input
image, and the right first sub-figure is the map and the real trace. Figure 5:
Localization error comparison among VO, DNN-based E2E localization, and
EdgeLoc. Figure 7: Impact of latency on localization performance in the
EdgeLoc system. Red points indicate ground truth data for localization, while
blue lines represent EdgeLoc’s results. Latency is represented by green bars
(”ts” here means time steps, which are used to characterize latency after
normalization), and error values (Measure with y-axis value) by orange bars.
The right-hand y-axis shows the scale for error values. Latency increases
progressively from left to right in the subplots. Numerical mapping changes
were applied for time step values.

To validate the effectiveness of the EdgeLoc system, we conducted experiments
comparing its performance against single-method localization approaches,
namely, VO and DNN based localization. Fig. 4 illustrates the localization
errors of these three methods over time. The results reveal that the VO method
suffers from cumulative errors, resulting in a gradual drift in localization
accuracy. However, it is important to note that VO does not produce any
outliers in its localization estimates. On the other hand, the E2E DNN
approach exhibits a high degree of randomness in its localization errors, as
shown in the figure. Although the majority of its localization estimates are
highly accurate, the E2E DNN method generates a significant number of
outliers, which can severely impact the overall localization performance. As
evident from the figure, EdgeLoc achieves consistent and accurate localization
results without suffering from cumulative errors or producing outliers.

Fig. 5 provides a quantitative comparison of the localization errors for each
method using box plots. The box plot corresponding to the VO method confirms
that while it has the highest median error, it does not generate any outliers.
In contrast, the box plot for the E2E DNN approach reveals the presence of
numerous outliers, despite having a lower median error compared to VO.
EdgeLoc, as shown in its corresponding box plot, not only achieves the lowest
median error but also successfully eliminates outliers, demonstrating its
superior performance in terms of both accuracy and reliability.

Based on the experimental results provided, we further validate the
superiority of EdgeLoc over the Kalman filtering approach for localization
fusion. Fig. LABEL:fig:klm visually compares the localization results obtained
using the Kalman filter and EdgeLoc. It is evident that the Kalman filter
fails to effectively fuse the DNN localization outputs with the real-time
sensor-based localization, resulting in significant deviations and
inconsistencies in the estimated trajectory. In contrast, EdgeLoc achieves a
much smoother and more accurate localization result by employing its
uncertainty-aware fusion method.

Quantitatively, EdgeLoc’s total localization error is 2676.35, which is 30.26%
lower than the Kalman filter’s total error of 3837.65. This substantial
reduction highlights the limitations of the Kalman filter in handling the
uncertainties and potential outliers associated with DNN localization outputs.
Furthermore, EdgeLoc’s total error is 29.62% lower than the DNN localization
error of 3802.57 and 67.75% lower than the VO error of 8298.58, demonstrating
its ability to effectively leverage the strengths of both DNN and sensor-based
localization methods while mitigating their individual drawbacks.

Figure 8: Box plots of localization errors for different latency ranges in the
EdgeLoc system.

EdgeLoc’s ability to map latency to localization accuracy is a key innovation
that transforms the problem of improving accuracy into a problem of reducing
latency. To showcase this relationship, we conducted experiments with varying
latency conditions and evaluated their impact on localization error.

Fig. 7 demonstrates that as latency increases, there is a corresponding rise
in error, thus validating the efficacy of the latency-to-accuracy module in
the EdgeLoc framework. Fig. 8 presents box plots of the localization errors
for different latency ranges, further reinforcing the relationship between
latency and accuracy. As the latency decreases, the box plots show a
consistent reduction in both the median error and the spread of errors. This
demonstrates that by minimizing latency, EdgeLoc effectively improves
localization accuracy and reduces the variability in the localization results.

These results collectively highlight the importance of latency in the EdgeLoc
system. By focusing on minimizing latency through optimized collaborative
inference and efficient communication, EdgeLoc achieves improvements in
localization accuracy.

###  6.2 Overall System Performance in a Cellular Network Environment with
Different Network Speed

To validate the EdgeLoc system’s ability to adapt to varying network
conditions and consistently select the optimal DNN split point in a real-world
scenario, we conducted a comprehensive experiment in a cellular network
environment. This testing is crucial as it demonstrates the system’s
capability to effectively learn and converge to the optimal split point under
realistic network conditions, where factors such as signal strength,
interference, and network load may significantly impact performance. In this
experiment, we focus on evaluating EdgeLoc’s performance under different
network speeds, simulating various real-world scenarios. By adjusting the
network bandwidth using Wondershaper, we can assess the system’s ability to
dynamically select the optimal split point and maintain high localization
accuracy across a range of network conditions.

This section describes the experimental setup, including the 5G cellular
network deployment, and discusses the results obtained, focusing on the
effectiveness of the UCB algorithm for selecting the optimal DNN split point.

####  6.2.1 Experimental Setup and 5G Cellular Network Deployment

Figure 9: Experimental setup for evaluating EdgeLoc’s localization performance
in a 5G cellular network environment.

Fig. 9 illustrates the experimental setup designed to evaluate EdgeLoc’s
performance in a 5G cellular network environment. The setup consists of three
main components: User Equipment (UE), Core Network, and gNodeB (gNB). The UE
represents the autonomous vehicle and is equipped with the necessary threads
to execute real-time localization tasks and communicate with the gNB. The gNB
acts as the base station for the cellular network, providing cellular coverage
and facilitating communication between the UE and edge server in the EdgeLoc
system.

The Core Network is responsible for managing communication between the UE and
the gNB, ensuring reliable and secure data transmission. In this experimental
setup, the Core Network machine also serves as the Edge Server, providing
additional computational resources for DNN inference in the EdgeLoc system.
The software system used in this experiment is OpenAirInterface [33], an open-
source platform for 5G experimentation and prototyping. OpenAirInterface
enables the deployment of a complete 5G network, including the UE, gNB, and
Core Network components. We employ Wondershaper [34], a network traffic
control tool, to regulate the network speed between the UE and the gNB.
Control the network speed uploaded to gNB on the UE side

####  6.2.2 Performance Evaluation and Results

We conduct experiments to assess the localization accuracy achieved by EdgeLoc
under different network conditions and evaluate its capability to dynamically
select the optimal DNN split point.

Fig. LABEL:fig:subfig1 illustrates the online learning process of the adapted
UCB algorithm in EdgeLoc. The algorithm dynamically selects different DNN
split points over time, and converges to the optimal split point after a
certain number of iterations (marked by the star in the figure). Fig.
LABEL:fig:subfig2 illustrates the localization errors over time for EdgeLoc
using the adapted UCB algorithm under three different network conditions. The
experimental log results demonstrate that despite our deliberate manipulation
of network speeds to create varying network environments, EdgeLoc consistently
maintains its ability to select the optimal split point across the varying
network scenario. By leveraging the adapted UCB algorithm to dynamically
select the optimal DNN split point, EdgeLoc effectively adapts to changes in
the environment and optimizes performance accordingly.

##  7 conclusions

This paper presents EdgeLoc, a communication-adaptive parallel system for
real-time localization in infrastructure-assisted autonomous driving. EdgeLoc
leverages precise localization from roadside infrastructure to enhance real-
time localization based on onboard sensor-based visual odometry. The system
employs a parallel processing architecture, combining the real-time
performance of traditional methods with the high accuracy of deep learning
approaches. EdgeLoc achieves communication adaptivity through online learning
and addresses network fluctuations via window-based anomaly detection.

By auto-splitting V2I collaborative inference, EdgeLoc achieves optimal
latency. Meanwhile, online distributed learning for decision-making ensures
maximum localization improvement. Experimental results demonstrate that even
with the most basic E2E DNN for localization estimation, our system realize a
67.75% reduction in localization error for real-time local visual odometry, a
29.95% reduction for non-real-time collaborative inference, and a 30.26%
reduction compared to Kalman filtering.

EdgeLoc is implemented using ROS and open-sourced for easy deployment. Future
research directions include testing in larger-scale and more complex
environments, as well as incorporating other types of sensors and localization
algorithms. In summary, EdgeLoc provides a universal edge computing assisted
solution for high-accuracy real-time localization in infrastructure-assisted
autonomous driving. By leveraging the edge computing resources at RSUs and
integrating them with on-vehicle systems, EdgeLoc pushes the frontier of
collaborative perception and decision-making in intelligent transportation
systems. The system demonstrates the advantages of combining traditional
methods with deep learning approaches, as well as the benefits of parallel
computing architecture and online learning techniques. The system is open-
sourced and thus has potential value in research on infrastructure-assisted
autonomous driving, connected vehicles, etc.

##  Appendix A Localization Error Analysis of ParaLoc

In the scenario of modeling errors in localization systems, this can be
characterized based on the $K$-armed bandit problem. Consider random variables
$E_{i,n}$ for $1\leq i\leq K$ and $n\geq 1$, where each $i$ represents a
specific action, namely a splitting point. The variable $E_{i,t}$,
representing the localization error, is independent and identically
distributed with respect to indices $i$ and $t$. Let $\mu_{i}$ denote the mean
value of the random variable $E_{i,n}$ at action $i$. The optimal action’s
mean value is denoted by $\mu^{\ast}=\max_{i}\mu_{i}$.

A policy in the context of localization error modeling can be considered as an
algorithm that selects the next splitting point to use, based on the sequence
of past selections and their associated localization errors. In this scenario,
the UCB1-normal policy, as described before, is a well-known algorithm for the
bandit problem, particularly suitable for processes with Gaussian-distributed
errors.

For convenience, we reconstruct the UCB index as [35]:

$\Phi_{i,n}=\bar{X}_{T_{i}(n)}+\sqrt{\frac{16\hat{\sigma}^{2}_{i,n}\ln(n-1)}{T_{i}(n)-1}},$
(9)

where

$\bar{X}_{T_{i}(n)}=\frac{1}{T_{i}(n)}\sum_{t=1}^{n}X_{i,t}~{}1_{I_{t}=i},$
(10)

and

$\hat{\sigma}^{2}_{i,n}=\frac{1}{T_{i}(n)}\sum_{t=1}^{n}X^{2}_{i,t}1_{I_{t}=i}-\bar{X}^{2}_{T_{i}(n)}.$
(11)

Term $T_{i}(n)=\sum_{t=1}^{n}1_{I_{t}=i}$ denotes the number of times the
player selected arm $i$ until time slot $n$. This approach aims to minimize
the localization error by dynamically choosing the splitting point with the
highest expected reduction in error, as quantified by the UCB index. The
localization system is as follows:

$\begin{cases}L_{r}(t)=uL_{\alpha}(t)+(1-u)L_{r}(t-\Delta t),\\\
\qquad\text{if }t=x\Delta t,x\in\mathbb{N},\\\
L_{r}(t+a)=L_{r}(t+a-1)+L_{\beta}(t+a)-L_{\beta}(t+a-1),\\\ \qquad\text{if
}t\neq x\Delta t,x\in\mathbb{N};a\in[1,\Delta t].\\\ \end{cases}$ (12)

Theorem: For the $K$-armed bandit problem with Gaussian reward process, the
localization upper bound after $n$ plays by running the UCB1-nromal policy is
${R}_{n}\leq f(L_{\alpha},L_{\beta})$. Proof: We will provide a proof by
combining the UCB algorithm and the mathematical model of the proposed
localization system. We firstly define:

$X_{i,t}=L(t)-L_{r}(i,t),$ (13)

where $X$ represents localization errors, and $i$ represents the $i^{th}$
action (split point). $X_{i,t}$ represents the practical localization error at
the $i^{th}$ split point that was selected at time step $t$. Based on (12), we
can get:

$\begin{split}&\sigma^{2}_{n}=\frac{1}{n}\sum_{t=1}^{n}(L(t)-L_{r}(t))^{2}\\\
&-\left(\frac{1}{t}\left(\int_{0}^{t}\left[L(t)-(u\times
L_{\alpha}(t)+(1-u)L_{r}(t))\right]dt\right.\right.\\\
&+\left.\left.\int_{0}^{t}\left[L(t)-L(t-1)\right]dt-\int_{0}^{t}\left[L(t)-L_{\beta}(t-1)\right]dt\right)\right)^{2},\end{split}$
(14)

where

$u=\int_{0}^{\Delta t}(L(t)-L(t-\Delta
t))-\left(L_{\beta}(t)-L_{\beta}(t-\Delta t)\right).$ (15)

At each time slot $t$, a system pulls an action $I_{t}$ according to the
UCB1-normal policy and receives a reward (localization error) $X_{I_{t},t}$
from a Gaussian distribution with unknown mean and variance. After $n$ rounds,
the accumulated regret is defined as

$R_{n}=\max_{i=1,\ldots,K}\sum_{t=1}^{n}X_{i,t}-\sum_{t=1}^{n}X_{I_{t},t}.$
(16)

The pseudo-regret is

$\begin{split}\bar{R}_{n}&=\max_{i=1,\ldots,K}\mathbb{E}\left[\sum_{t=1}^{n}X_{i,t}-\sum_{t=1}^{n}X_{I_{t},t}\right]=n\mu^{\ast}-\sum_{t=1}^{n}\mathbb{E}[\mu_{I_{t}}].\end{split}$
(17)

Let $\Delta_{i}=\mu^{\ast}-\mu$ be the suboptimality parameter of arm $i$.
Then, the pseudo-regret can be rewritten as

$\begin{split}\bar{R}_{n}&=\left(\sum_{i=1}^{K}\mathbb{E}[T_{i}(n)]\right)\mu^{\ast}-\mathbb{E}\left[\sum_{i=1}^{K}T_{i}(n)\mu_{i}\right],\\\
&=\sum_{i=1}^{K}\Delta_{i}\mathbb{E}[T_{i}(n)].\end{split}$ (18)

According to Eq. (18), we can bound the regret $\bar{R}_{n}$ by simply
bounding each $\mathbb{E}[T_{i}(n)]$. For each $t>1$, we bound the probability
of the indicator function that $I_{t}=i$. Specifically, let $L$ be an
arbitrary positive integer and

$c_{t,s}=\sqrt{\frac{16\hat{\sigma}^{2}_{i,s}\ln(t-1)}{s_{i}-1}}.$ (19)

Then, we have

$\begin{split}T_{i}(n)&=1+\sum_{t=K}^{n}\left\\{I_{t}=i\right\\},\\\ &\leq
L+\sum_{t=K}^{n}\left\\{I_{t}=i,T_{i}(t-1)\geq L\right\\},\\\ &\leq
L+\sum_{t=K}^{n}\left\\{\bar{X}_{T^{\ast}(t-1)}+c_{t-1,T^{\ast}(t-1)}\leq\bar{X}_{T_{i}(t-1)}\right.\\\
&\left.\qquad+c_{t-1,T_{i}(t-1)},T_{i}(t-1)\geq L\right\\},\\\ &\leq
L+\sum_{t=K}^{n}\left\\{\min_{0<s<t}\bar{X}^{\ast}_{s}+c_{t-1,s}\leq\max_{L<s_{i}<t}\bar{X}_{i,s_{i}}+c_{t-1,s_{i}}\right\\},\\\
&\leq
L+\sum_{t=1}^{\infty}\sum_{s=1}^{t-1}\sum_{s_{i}=L}^{t-1}\left\\{\bar{X}^{\ast}_{s}+c_{t,s}\leq\bar{X}_{i,s_{i}}+c_{t,s_{i}}\right\\}.\\\
\end{split}$ (20)

Note that $\bar{X}^{\ast}_{s}+c_{t,s}\leq\bar{X}_{i,s_{i}}+c_{t,s_{i}}$
indicates that at least one of the three events must hold: (i)
$\bar{X}^{\ast}_{s}\leq\mu^{\ast}-c_{t,s}$; (ii)
$\bar{X}_{i,s_{i}}\geq\mu_{i}+c_{t,s_{i}}$; (iii)
$\mu^{\ast}\leq\mu_{i}+2c_{t,s_{i}}$. Thus, Eq. (20) can be rewritten as

$\begin{split}T_{i}(n)\leq&L+\sum_{t=1}^{\infty}\sum_{s=1}^{t-1}\sum_{s_{i}=L}^{t-1}\left\\{\mathrm{Pr}\left\\{\bar{X}^{\ast}_{s}\leq\mu^{\ast}-c_{t,s}\right\\}\right.\\\
&\left.+\mathrm{Pr}\left\\{\bar{X}_{i,s_{i}}\geq\mu_{i}+c_{t,s_{i}}\right\\}+\mathrm{Pr}\left\\{\mu^{\ast}\leq\mu_{i}+2c_{t,s_{i}}\right\\}\right\\}.\\\
\end{split}$ (21)

Next, we bound the probabilities of events (i), (ii), and (iii). The random
variable
$(\bar{X}_{i,s_{i}}-\mu_{i})/\sqrt{\hat{\sigma}^{2}_{i,s_{i}}/(s_{i}-1)}$ has
a student distribution with $s_{i}-1$ degrees of freedom. Then, using the fact
that $\mathrm{Pr}\left\\{X\geq a\right\\}\leq e^{-a^{2}/4}$ for the student
distribution, we have

$\begin{split}\mathrm{Pr}\left\\{\bar{X}^{\ast}_{s}\leq\mu^{\ast}-c_{t,s}\right\\}&=\mathrm{Pr}\left\\{\frac{\mu^{\ast}-\bar{X}^{\ast}_{s}}{\sqrt{\hat{\sigma}^{2}_{\ast,s}/(s-1)}}\geq
4\sqrt{\ln t}\right\\},\\\ &\leq t^{-4},\end{split}$ (22)

and

$\begin{split}\mathrm{Pr}\left(\bar{X}_{i,s_{i}}\geq\mu_{i}+c_{t,s_{i}}\right)&=\mathrm{Pr}\left(\frac{\bar{X}_{i,s_{i}}-\mu_{i}}{\sqrt{\hat{\sigma}^{i,s_{i}}/(s_{i}-1)}}\geq
4\sqrt{\ln t}\right),\\\ &\leq t^{-4},\end{split}$ (23)

for all $s_{i}\geq 8\ln t$ and $a=4\sqrt{\ln t}$. For event (iii), the random
variable $s_{i}\hat{\sigma}^{2}_{t,s_{i}}/\sigma^{2}_{i}$ has the
$\chi^{2}$-distributed with $s_{i}$ degrees of freedom [Wilks1962]. Thus, by
using the fact that $\mathrm{Pr}\left\\{X\geq 4s\right\\}\leq e^{-(s+1)/2}$
with $s=s_{i}-1$, we obtain

$\begin{split}&\mathrm{Pr}\left\\{\mu^{\ast}\leq\mu_{i}+2c_{t,s_{i}}\right\\},\\\
&=\mathrm{Pr}\left\\{\frac{s_{i}\hat{\sigma}^{2}_{i,s_{i}}}{\sigma^{2}_{i}}>\frac{(s_{i}-1)\Delta^{2}_{i}s_{i}}{64\sigma^{2}_{i}\ln
t}\right\\},\\\
&\leq\mathrm{Pr}\left\\{\frac{s_{i}\hat{\sigma}^{2}_{i,s_{i}}}{\sigma^{2}_{i}}>4(s_{i}-1)\right\\},\\\
&\leq e^{-s_{i}/2}\leq t^{-4},\end{split}$ (24)

for $s_{i}\geq\max\\{256\sigma^{2}_{i}/\Delta^{2}_{i},8\\}\ln t$. Thus, we can
set $L=\max\\{256\sigma^{2}_{i}/\Delta^{2}_{i},8\\}\ln t$. By submitting Eqs.
(22), (23), and (24) into Eq. (21), which yields

$\begin{split}\mathbb{E}[T_{i}(n)]&\leq\frac{8\ln
n}{\Delta_{i}^{2}}+\sum_{t=1}^{\infty}\sum_{s=1}^{t-1}\sum_{s_{i}=L}^{t-1}3t^{-4}\\\
&\leq\frac{256\sigma^{2}_{i}\ln n}{\Delta_{i}^{2}}+8\ln
n+\frac{\pi^{4}}{30}.\\\ \end{split}$ (25)

Therefore, the regret upper bound can be computed as

$\begin{split}\bar{R}_{n}&=\sum_{i=1}^{K}\Delta_{i}\mathbb{E}[T_{i}(n)]\\\
&\leq 256\ln n\sum_{i,i\neq i^{\ast}}\frac{\sigma^{2}_{i}}{\Delta_{i}}+(8\ln
n+\frac{\pi^{4}}{30})\sum_{j=1}^{K}\Delta_{i}.\end{split}$ (26)

From the above formula, it can be seen that the regret upper bound is only
related to $\sigma$ and $\Delta$, where $\Delta$ is an inherent property.
Regarding $\sigma$, based on (14), we have

$\begin{split}\sigma^{2}_{n}=&\frac{1}{n}\sum_{t=1}^{n}(L(t)-Lr(t))^{2}\\\
&-\frac{1}{t}\left(\int_{0}^{t}\left[L(t)-(u\times
L_{\alpha}(t)+(1-u)Lr(t))\right]\right.\\\
&+\left.\int_{0}^{t}\left[L(t)-L(t-1)\right]-\left[L(t)-L_{\beta}(t-1)\right]\right)^{2}.\end{split}$
(27)

We see that $\sigma$ is only related to $L_{\alpha}$, $L_{\beta}$, and $u$.
According to (15), $u$ is solely dependent on $\Delta t$, which corresponds to
each split point and thus is an inherent property. Therefore, $\sigma$ is only
related to $L_{\alpha}$ and $L_{\beta}$. Consequently, we can conclude that
the upper bound of $\bar{R}_{n}$ is only related with $L_{\alpha}$ and
$L_{\beta}$.

To sum up, the regret upper bound provides a theoretical foundation for the
optimality of the ParaLoc system. By proving that the regret upper bound of
the UCB1-normal policy only depends on the end-to-end DNN localization results
($L_{\alpha}$) and the real-time sensor-based localization results
($L_{\beta}$), we demonstrate that the ParaLoc system fully utilizes the
information provided by these two variables. This suggests that, as
$L_{\alpha}$ and $L_{\beta}$ are intrinsic components, no additional
enhancements can be made to the theoretical worst-case performance.

## References

  * [1] A. Woo, B. Fidan, and W. W. Melek, “Localization for autonomous driving,” Handbook of position location: theory, practice, and advances, second edition, pp. 1051–1087, 2018.
  * [2] F. Fraundorfer and D. Scaramuzza, “Visual odometry: Part ii: Matching, robustness, optimization, and applications,” IEEE Robotics & Automation Magazine, vol. 19, no. 2, pp. 78–90, 2012.
  * [3] T. Qin, P. Li, and S. Shen, “Vins-mono: A robust and versatile monocular visual-inertial state estimator,” IEEE Transactions on Robotics, vol. 34, no. 4, pp. 1004–1020, 2018.
  * [4] H. Zhan, C. S. Weerasekera, J.-W. Bian, and I. Reid, “Visual odometry revisited: What should be learnt?,” in 2020 IEEE international conference on robotics and automation (ICRA), pp. 4203–4210, IEEE, 2020.
  * [5] T. Shan, B. Englot, D. Meyers, W. Wang, C. Ratti, and D. Rus, “Lio-sam: Tightly-coupled lidar inertial odometry via smoothing and mapping,” in 2020 IEEE/RSJ international conference on intelligent robots and systems (IROS), pp. 5135–5142, IEEE, 2020.
  * [6] L. R. Agostinho, N. M. Ricardo, M. I. Pereira, A. Hiolle, and A. M. Pinto, “A practical survey on visual odometry for autonomous driving in challenging scenarios and conditions,” IEEE Access, vol. 10, pp. 72182–72205, 2022.
  * [7] L. de Paula Veronese, F. Auat-Cheein, F. Mutz, T. Oliveira-Santos, J. E. Guivant, E. De Aguiar, C. Badue, and A. F. De Souza, “Evaluating the limits of a lidar for an autonomous driving localization,” IEEE Transactions on Intelligent Transportation Systems, vol. 22, no. 3, pp. 1449–1458, 2020.
  * [8] A. Kendall, M. Grimes, and R. Cipolla, “Posenet: A convolutional network for real-time 6-dof camera relocalization,” in Proceedings of the IEEE International Conference on Computer Vision (ICCV), pp. 2938–2946, 2015.
  * [9] Y. Zhao, J. Li, X. Hu, and H. Sun, “Gnss spoofing detection and mitigation based on maximum likelihood estimation,” IEEE Transactions on Aerospace and Electronic Systems, vol. 57, no. 3, pp. 1817–1831, 2021.
  * [10] C. Jiang, G. Yang, J. Wang, and X. Zhang, “An improved ins/gnss integrated navigation method based on adaptive robust kalman filter,” IEEE Transactions on Instrumentation and Measurement, vol. 71, pp. 1–12, 2022.
  * [11] A. Kendall and R. Cipolla, “Geometric loss functions for camera pose regression with deep learning,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5974–5983, 2017.
  * [12] A. Kendall and Y. Gal, “What uncertainties do we need in bayesian deep learning for computer vision?,” in Advances in Neural Information Processing Systems (NeurIPS), pp. 5574–5584, 2017.
  * [13] F. Walch, C. Hazirbas, L. Leal-Taixe, T. Sattler, S. Hilsenbeck, and D. Cremers, “Image-based localization using lstms for structured feature correlation,” in Proceedings of the IEEE International Conference on Computer Vision (ICCV), pp. 627–637, 2017.
  * [14] I. Melekhov, J. Ylioinas, J. Kannala, and E. Rahtu, “Image-based localization using hourglass networks,” in Proceedings of the IEEE International Conference on Computer Vision Workshops (ICCVW), pp. 870–877, 2017.
  * [15] S. Brahmbhatt, J. Gu, K. Kim, J. Hays, and J. Kautz, “Geometry-aware learning of maps for camera localization,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2616–2625, 2018.
  * [16] G. Lu, Y. Zhao, H. Sun, and J. Li, “Gnss multipath error modeling and mitigation in urban areas using 3d city models,” GPS Solutions, vol. 26, no. 1, pp. 1–14, 2022.
  * [17] C. Creß, Z. Bing, and A. C. Knoll, “Intelligent transportation systems using roadside infrastructure: A literature survey,” IEEE Transactions on Intelligent Transportation Systems, 2023.
  * [18] B. Liu, L. Wang, X. Chen, L. Huang, D. Han, and C.-Z. Xu, “Peer-assisted robotic learning: a data-driven collaborative learning approach for cloud robotic systems,” in 2021 IEEE international conference on robotics and automation (ICRA), pp. 4062–4070, IEEE, 2021.
  * [19] B. Liu, L. Wang, M. Liu, and C.-Z. Xu, “Federated imitation learning: A novel framework for cloud robotic systems with heterogeneous sensor data,” IEEE Robotics and Automation Letters, vol. 5, no. 2, pp. 3509–3516, 2019.
  * [20] Y. He, L. Ma, Z. Jiang, Y. Tang, and G. Xing, “Vi-eye: semantic-based 3d point cloud registration for infrastructure-assisted autonomous driving,” in Proceedings of the 27th Annual International Conference on Mobile Computing and Networking, pp. 573–586, 2021.
  * [21] G. Liu, S. Salehi, E. Bala, C.-C. Shen, and L. J. Cimini, “Communication-constrained routing and traffic control: A framework for infrastructure-assisted autonomous vehicles,” IEEE Transactions on Intelligent Transportation Systems, vol. 23, no. 12, pp. 23844–23857, 2022.
  * [22] B. Liu, L. Wang, and M. Liu, “Roboec2: A novel cloud robotic system with dynamic network offloading assisted by amazon ec2,” IEEE Transactions on Automation Science and Engineering, 2023.
  * [23] D. Callegaro, S. Baidya, and M. Levorato, “Dynamic distributed computing for infrastructure-assisted autonomous uavs,” in ICC 2020-2020 IEEE International Conference on Communications (ICC), pp. 1–6, IEEE, 2020.
  * [24] B. Liu, L. Wang, M. Liu, and C.-Z. Xu, “Lifelong federated reinforcement learning: a learning architecture for navigation in cloud robotic systems,” IEEE Robotics and Automation Letters, vol. 4, no. 4, pp. 4555–4562, 2019.
  * [25] S. Shi, J. Cui, Z. Jiang, Z. Yan, G. Xing, J. Niu, and Z. Ouyang, “Vips: Real-time perception fusion for infrastructure-assisted autonomous driving,” in Proceedings of the 28th Annual International Conference on Mobile Computing And Networking, pp. 133–146, 2022.
  * [26] S. Yang, H. H. Yin, R. W. Yeung, X. Xiong, Y. Huang, L. Ma, M. Li, and C. Tang, “On scalable network communication for infrastructure-vehicle collaborative autonomous driving,” IEEE Open Journal of Vehicular Technology, vol. 4, pp. 310–324, 2022.
  * [27] Y. Jia, J. Zhang, S. Lu, B. Fan, R. Mao, S. Zhou, and Z. Niu, “Infrastructure-assisted collaborative perception in automated valet parking: A safety perspective,” arXiv preprint arXiv:2403.15156, 2024.
  * [28] M. Tsukada, T. Oi, M. Kitazawa, and H. Esaki, “Networked roadside perception units for autonomous driving,” Sensors, vol. 20, no. 18, p. 5320, 2020.
  * [29] M. Hirata, M. Tsukada, K. Okumura, Y. Tamura, H. Ochiai, and X. Défago, “Roadside-assisted cooperative planning using future path sharing for autonomous driving,” in 2021 IEEE 94th Vehicular Technology Conference (VTC2021-Fall), pp. 1–7, IEEE, 2021.
  * [30] L. Zhou, “A survey on contextual multi-armed bandits,” arXiv preprint arXiv:1508.03326, 2015.
  * [31] D. Simon, “Kalman filtering,” Embedded systems programming, vol. 14, no. 6, pp. 72–79, 2001.
  * [32] W. Maddern, G. Pascoe, C. Linegar, and P. Newman, “1 year, 1000 km: The oxford robotcar dataset,” The International Journal of Robotics Research, vol. 36, no. 1, pp. 3–15, 2017.
  * [33] N. Nikaein, M. K. Marina, S. Manickam, A. Dawson, R. Knopp, and C. Bonnet, “Openairinterface: A flexible platform for 5g research,” ACM SIGCOMM Computer Communication Review, vol. 44, no. 5, pp. 33–38, 2014.
  * [34] magnific0, “wondershaper.” https://github.com/magnific0/wondershaper, 2024.
  * [35] P. Auer, N. Cesa-Bianchi, and P. Fischer, “Finite-time analysis of the multiarmed bandit problem,” Machine learning, vol. 47, no. 2, pp. 235–256, 2002.