# Detecting Socially Abnormal Highway Driving Behaviors via Recurrent Graph
Attention Networks

Yue Hu, Yuhang Zhang, Yanbing Wang, Daniel Work  Vanderbilt
UniversityNashvilleUS yue.hu,
<EMAIL_ADDRESS>
(2023)

###### Abstract.

With the rapid development of Internet of Things technologies, the next
generation traffic monitoring infrastructures are connected via the web, to
aid traffic data collection and intelligent traffic management. One of the
most important tasks in traffic is anomaly detection, since abnormal drivers
can reduce traffic efficiency and cause safety issues. This work focuses on
detecting abnormal driving behaviors from trajectories produced by highway
video surveillance systems. Most of the current abnormal driving behavior
detection methods focus on a limited category of abnormal behaviors that deal
with a single vehicle without considering vehicular interactions. In this
work, we consider the problem of detecting a variety of socially abnormal
driving behaviors, i.e., behaviors that do not conform to the behavior of
other nearby drivers. This task is complicated by the variety of vehicular
interactions and the spatial-temporal varying nature of highway traffic. To
solve this problem, we propose an autoencoder with a Recurrent Graph Attention
Network that can capture the highway driving behaviors contextualized on the
surrounding cars, and detect anomalies that deviate from learned patterns. Our
model is scalable to large freeways with thousands of cars. Experiments on
data generated from traffic simulation software show that our model is the
only one that can spot the exact vehicle conducting socially abnormal
behaviors, among the state-of-the-art anomaly detection models. We further
show the performance on real world HighD traffic dataset, where our model
detects vehicles that violate the local driving norms.

Intelligent transportation; Spatial-temporal learning; Graph neural networks;
Anomaly detection

††journalyear: 2023††copyright: acmlicensed††conference: Proceedings of the
ACM Web Conference 2023; May 1–5, 2023; Austin, TX, USA††booktitle:
Proceedings of the ACM Web Conference 2023 (WWW ’23), May 1–5, 2023, Austin,
TX, USA††price: 15.00††doi: 10.1145/3543507.3583452††isbn:
978-1-4503-9416-1/23/04††ccs: Information systems Spatial-temporal
systems††ccs: Information systems Data mining

##  1\. Introduction

###  1.1. Motivation and challenges

Nowadays, the development of Internet of Things (IoT) technologies has greatly
advanced intelligent monitoring and management of urban transportation
systems. Sensor networks including highway surveillance cameras and radar
detectors, combined with Web of Things (WoT) technologies, have enabled
transportation authorities to intelligently monitor traffic systems at scales
and resolutions previously out of reach (Trirat and Lee, 2021; Gloudemans et
al., 2023; Tang et al., 2019). One of the emerging tasks in intelligent
traffic management is anomaly detection, since abnormal drivers could have
adversarial impact on the smoothness of traffic stream, or even pose safety
concerns. Yet it is infeasible for human operators to manually inspect and
analyze all of this data, given the now massive amount of data these systems
can generate. Consequently, there is need to spot the anomalies from terabytes
of data and highlight the scenes that need further human inspection. In this
work, we tackle the task of detecting abnormal driving behaviors, from
trajectories produced by IoT highway video surveillance systems.

The existing approaches to vehicular anomaly detection mainly fall into two
categories. The first set of approaches (Bai et al., 2019; Chen et al., 2021;
Wu et al., 2021; Sultani et al., 2018; Li et al., 2020) focus on detecting
severe events that cause vehicles to stop, and turn the problem into detecting
stalled cars via computer vision from surveillance videos. The second set of
approaches (Matousek et al., 2019; Moukafih et al., 2019; Alkinani et al.,
2020; Gatteschi et al., 2021) focus on single-car abnormal driving behaviors,
such as speeding and abrupt braking. Methods ranging from thresholding (Fazeen
et al., 2012; Carlos et al., 2018) to machine learning (Matousek et al., 2018,
2019; Moukafih et al., 2019) are applied on data obtained from a single car,
e.g. from on-board sensors and GPS devices.

However, the above approaches only cover a subset of abnormal driving
behaviors and treat cars in isolation from each other. Vehicles constantly
interact with their surroundings, and traffic context is needed for anomaly
detection. For example, a car at a constant speed of 50 mph might be perfectly
normal, yet a car at 50 mph on the inner most lane of highway is blocking all
the other cars driving at 65 mph or above, and should be considered an anomaly
on highway. As another example, abrupt braking might be considered abnormal,
but if the vehicle is braking because its front car is stalled, then we should
detect the front car as abnormal, whereas the braking car is doing what is
expected. Our task is to detect such socially abnormal behaviors that do not
conform to the commonly accepted and observed social norms, by developing a
contextual understanding of vehicle interactions.

Moreover, most of the existing methods are rule-based (Fazeen et al., 2012;
Carlos et al., 2018) or supervised-learning method (Moukafih et al., 2019;
Gatteschi et al., 2021), and can only detect pre-defined types of anomalies
such as stalled and speeding cars. Yet vehicles can behave anomalously in
unexpected ways, and building a comprehensive set of rules that includes every
possible occasion could be hard. Our goal is to build a model that can
identify a variety of anomalies with unsupervised learning.

Graph neural networks (GNN) have seen rapid development in recent years (Kipf
and Welling, 2016; Veličković et al., 2017; Wang et al., 2022; Wang and Derr,
2021), showing great advantage in modeling the complex relationships in graph
data. By representing vehicles as nodes and their relationships as edges,
interactions with neighbors can be modeled via a GNN. Two challenges exist in
developing a GNN for detecting anomalous driving behaviors. First, given the
spatial-temporally varying nature of trajectories, we need to deal with
dynamic graphs. This is a nontrivial problem as compared to anomaly detection
on only a static graph, or only considering time-varying signals. Second, we
need to take stochasticity into consideration, which is intrinsic in driving
behaviors. Under a particular context, there could be a range of acceptable
behaviors that should all be considered normal. For example, a car can conduct
lane changing from time to time, or have some variation in speed. A
deterministic model that fails to capture such normal stochasticity would
mislabel every lane-changing car as an anomaly.

###  1.2. Our approach

To detect socially abnormal behaviors on large scale trajectory data while
addressing the aforementioned challenges, we develop a model for Detecting
Socially Abnormal Behaviors (DSAB) in highway driving via Recurrent Graph
Attention Autoencoder. Graph attention networks combined with recurrent neural
networks is used to capture the spatial-temporal pattern of vehicle
trajectories, while dynamically taking each vehicle’s neighbors into
consideration based on vehicle states. To facilitate scalability while
capturing anomalous driving behaviors occurring over longer periods of time,
we sample the trajectories over a relatively long-horizon and coarse-grid time
window. We further use a sparse graph where the vehicles are only connected to
close neighbors to reduce computation. An autoencoder structure is used for
anomaly detection, which can encode and decode normal data well. To address
the stochasticity in driving behaviors, we reconstruct the probabilistic
distribution of the trajectories in decoding process. Samples with small
reconstruction probabilities are marked as anomalies.

With these designs, our model can detect the socialy abnormal driving
behaviors, and is scalable to thousands of cars over 5 miles of highway. We
show the effectiveness of our model on both simulation and real-world data.
First, we generate large-scale trajectory data with ground-truth anomaly
labels, via a microscopic traffic simulator, to quantitatively evaluate the
performance. Compared with the existing state-of-the-art methods, our model is
the only one that can detect the exact vehicle with abnormal behavior, whereas
the rest models can only detect anomalous scenes We also apply our method on
the real-world highD trajectory dataset (Krajewski et al., 2018), where our
model detects vehicles that violate the local driving norms.

To summarize, the contribution of our works is as follows:

  * •

We propose a new problem of detecting the exact anomalous vehicles that
violate the social interaction norms in highway driving, and develop a model
to solve it, achieving state of the art performance.

  * •

We develop a DSAB model based on Recurrent Graph Attention Networks. It well
captures the spatial-temporal trajectory dynamics, while considering both the
vehicular interactions and the stocasticity in driving behaviors.

  * •

We conduct extensive experiments on both simulation and real-world data sets,
and show the ability of our model to scale to large highway monitoring systems
with thousands of vehicles, and detect a variety of abnormal behaviors.

##  2\. Related work

Trajectory Modeling. The majority of trajectory modeling work focuses on the
prediction of future trajectories for humans or vehicles, and in this line,
modeling the interaction between the agents is gaining interest. To aggregate
information across agents, a pooling mechanism is used in the Social LSTM
(Alahi et al., 2016) and the Social GAN (Gupta et al., 2018), while an
attention mechanism is used in SoPhie (Sadeghian et al., 2019), and scene
context fusion via convolutional neural network is used in Desire (Lee et al.,
2017). With recent development of graph convolutional networks, works
including (Mohamed et al., 2020; Vemula et al., 2018; Huang et al., 2019)
model the agents as nodes and their relationships as edges, and develop
spatio-temporal GNN to learn the dynamics.

Anomaly detection. Anomaly detection has been an important task in
transportation. Unexpected autonomous driving condition detection is studied
in (Stocco et al., 2022, 2020). Extreme event detection in urban traffic is
studied in (Hu and Work, 2020; Hu et al., 2022). Our work detects anomalies on
graphs, and a comprehensive survey on graph anomaly detection can be found in
(Ma et al., 2021). Common detection methods on graphs include autoencoder-
based methods (Ding et al., 2019; Bandyopadhyay et al., 2020), generative
adversarial learning (Ding et al., 2021), and contrastive learning (Xu et al.,
2022). However, most of the existing works are on static graphs, whereas we
need to deal with dynamic graphs, adding the temporal dimension. The existing
methods for dynamic graph anomaly detection have different problem settings
compared to ours. Specifically, NetWalk (Yu et al., 2018) and TADDY (Liu et
al., 2021) deal with unattributed graphs with no node or edge features;
AddGraph (Zheng et al., 2019) and StrGNN (Cai et al., 2021) detect anomalous
edges.

The most relevant work with ours is STGAE (Wiederer et al., 2022), where a
spatio-temporal graph autoencoder is combined with kernel density estimation
(KDE) to detect abnormal driving behaviors. Yet while STGAE works well in
experiments with only two vehicles, the time complexity of KDE is too high for
detection in large numbers of vehicles. Moreover, STGAE tackles a different
task of detecting the existence of anomalies among several vehicles over a
stretch of road over a specific time, which we term as abnormal scene
detection. Compared with our task of detecting the specific abnormal vehicles,
abnormal scene detection is an easier task, since an abnormal vehicle can have
subsequent influence on its neighbor vehicles. For example, a slow or stalled
car can cause its following cars to break abruptly or change lanes, which
could be detected as abnormal, but should have been considered normal when
looking for the root cause. We demonstrate in our experiments that while most
baseline methods can work well on abnormal scene detection, they achieve poor
performance on abnormal vehicle detection.

##  3\. Method

In this section, we formulate the problem mathematically, then describe the
proposed DSAB model for anomaly detection on highways. In an overview, we
first construct the vehicle trajectories as a spatial-temporal dynamic graph.
Then we build an autoencoder with an encoder to compress the vehicle time
series in low-dimensional vectors, and a decoder to reconstruct the
probabilistic distribution of original input trajectories. At test time, we
use the reconstruction probability as a measurement of anomalous behavior. The
overview of the model is shown in Fig 1.

We further make the following considerations to best accommodate large scale
anomaly detection on highway driving. Unlike most works (Lee et al., 2017;
Alahi et al., 2016; Mohamed et al., 2020) that works on short-term fine-
grained trajectories, e.g., 3-10 Hz over 1-5s, we choose coarser-grained
sampling with larger time window. In addition to reducing the amount of data
needed to be processed, it also captures the anomalous driving behaviors that
are expected to occur over longer periods of time. E.g., a vehicle that is
speeding or tailgating will likely persist for more than a few seconds. While
computationally advantageous, sampling at a more coarse timescale requires a
different model to be used for lane changing. Specifically, the continuous bi-
variate Gaussian distribution, which is commonly used in trajectory modeling
(Vemula et al., 2018; Mohamed et al., 2020; Alahi et al., 2016), is no longer
suitable to capture a discrete lane-changing motion. Instead, we model
longitudinal motions as Gaussian distribution, and lateral lane location as
categorical distribution.

###  3.1. Problem formulation

Our model input is the observations of a set of $N$ vehicles on a highway over
time window $\mathcal{T}$, where $N$ can vary for different time windows. For
each vehicle $i$ at time $t\in\mathcal{T}$, the observation
$\mathbf{o}_{t}^{i}=[x_{t}^{i},y_{t}^{i},l_{t}^{i},v_{t}^{i},a_{t}^{i}]$
includes longitudinal position $x_{t}^{i}$, lateral position $y_{t}^{i}$,
driving lane id $l_{t}^{i}$, longitudinal speed $v_{t}^{i}$, and longitudinal
acceleration $a_{t}^{i}$.

Given the vehicle observations $\mathbf{o}_{t}^{i}$, $\forall
t\in\mathcal{T}$, our major goal is to detect the vehicles that have abnormal
behavior during the time window $\mathcal{T}$. Additionally, we also report
the performance of detecting abnormal scenes \- dividing the entire highway
into short stretches of length $\delta_{s}$, a scene contains all cars on a
stretch $\mathcal{S}$ during $\mathcal{T}$, and is labeled as abnormal if it
contains any abnormal cars.

Figure 1. DSAB model overview. We construct a spatial-temporal dynamic graph
to represent vehicles. An encoder compresses the dynamic graph into a low-
dimensional encode vector, and a decoder reconstructs the input vehicle
states, based on Recurrent Graph Attention Network (RGAT). In the encoder,
$\mathbf{X}_{t},\mathcal{E}_{t}$ denotes the node feature matrix and edge set
respectively. In the decoder, $\hat{\mathbf{X}_{t}}$ is the node
reconstruction matrix, $\mathcal{E}_{\mathcal{T}}$ is the union of all input
edge sets.

This figure has two parts. The top part shows the framework for the encoder
where $\T+1$ RGAT is displayed with corresponding inputs and outputs.
Similarly, the bottom part shows the framework for the decoder where $\T+1$
RGAT is displayed with corresponding inputs and outputs.

###  3.2. Graph construction for vehicle trajectories

We first construct a dynamic graph for the vehicle trajectories during time
window $\mathcal{T}$, denoted as
$G(\mathcal{T})=\\{\left(\mathcal{V},\mathcal{E}_{1}\right),\left(\mathcal{V},\mathcal{E}_{2}\right),\ldots,$
$\left(\mathcal{V},\mathcal{E}_{T}\right)\\}$ for discrete time steps
$t\in\mathcal{T}=\\{1,2,\dots,T\\}$. The node set $\mathcal{V}$ includes all
the vehicles on highway during the time window $\mathcal{T}$. The number of
nodes $N=|\mathcal{V}|$ is fixed for the graph of a specific time window, but
can vary for graphs of different time windows111We explain in implementation
details in Section 4.3 how we deal with vehicles that are not present in the
highway during the entire time window. At each time step $t$, the vehicle
observations are summarized into a raw observation matrix
$\mathbf{O}_{t}\in\mathbb{R}^{N\times d}$. $\mathbf{O}_{t}$ includes the
observation
$\mathbf{o}_{t}^{i}=[x_{t}^{i},y_{t}^{i},l_{t}^{i},v_{t}^{i},a_{t}^{i}]$ for
all vehicles $V_{i}\in\mathcal{V}$. There is an edge
$e^{ij}_{t}\in\mathcal{E}_{t}$ if vehicle $i$ and $j$ are close at time $t$,
that is, the two vehicles are less than $\delta_{x}$ feet apart
longitudinally, and less than $\delta_{l}$ lanes apart laterally. This is
based on the consideration that in reality, the vehicles are most influenced
by their close neighborhood vehicles. Thus we limit the neighbor range to
reduce storage and computational requirements.

###  3.3. Encoder

In this subsection, we explain how we use Recurrent Graph Attention Network
(RGAT) to encode the spatial-temporal vehicle trajectories. We adopt an RNN
which has proven to work well on time series data. Furthermore, to consider
the dynamic influence of each vehicle’s neighbors, we integrate the graph
attention network with RNN.

Specifically, we adopt the GRU (Cho et al., 2014; Chung et al., 2014) variant
of RNN, which is capable of learning long range dependencies in time via a
gating mechanism. Furthermore, similar to the practice of (Li et al., 2018;
Seo et al., 2018), we substitute the matrix multiplications in the original
GRU with graph convolutions denoted as $\ast_{\mathcal{G}}$, to operate on the
input and hidden states and capture neighborhood interactions:

(1)
$\displaystyle\mathbf{z}_{t}=\sigma(\mathbf{W}_{xz}\ast_{\mathcal{G}}\mathbf{X}_{t}+\mathbf{W}_{hz}\ast_{\mathcal{G}}\mathbf{H}_{t-1}+\mathbf{b}_{z}),$
$\displaystyle\mathbf{r}_{t}=\sigma(\mathbf{W}_{xr}\ast_{\mathcal{G}}\mathbf{X}_{t}+\mathbf{W}_{hr}\ast_{\mathcal{G}}\mathbf{H}_{t-1}+\mathbf{b}_{r}),$
$\displaystyle\tilde{\mathbf{H}}_{t}=\text{tanh}(\mathbf{W}_{xh}\ast_{\mathcal{G}}\mathbf{X}_{t}+\mathbf{W}_{hh}\ast_{\mathcal{G}}(\mathbf{r}_{t}\odot\mathbf{H}_{t-1}+\mathbf{b}_{h})),$
$\displaystyle\mathbf{H}_{t}=\mathbf{z}_{t}\odot\mathbf{H}_{t-1}+(1-\mathbf{z}_{t})\odot\tilde{\mathbf{H}}_{t},$

where $\mathbf{H}_{t}\in\mathbb{R}^{N\times d_{h}}$ is the hidden state,
$\mathbf{z}_{t}\in\mathbb{R}^{N\times d_{h}}$ and
$\mathbf{r}_{t}\in\mathbb{R}^{N\times d_{h}}$ are the update gate and the
reset gate respectively, with hidden dimension $d_{h}$. The weights
$\mathbf{W}_{xz}$, $\mathbf{W}_{hz}$, $\mathbf{W}_{xr}$, $\mathbf{W}_{hr}$,
$\mathbf{W}_{xh}$, $\mathbf{W}_{hh}$ and biases $\mathbf{b}_{z}$,
$\mathbf{b}_{r}$, $\mathbf{b}_{h}$ are trainable parameters. $\sigma$ is the
sigmoid function, and $\odot$ is element-wise multiplication.
$\mathbf{X}_{t}\in\mathbb{R}^{N\times d_{f}}$ is the input at time step $t$,
derived from the raw observation input $\mathbf{O}_{t}$, and consists of
numerical observations of position, speed, and acceleration
$[x_{t},y_{t},v_{t},a_{t}]\in\mathbb{R}^{N\times 4}$ , concatenated with lane
embeddings $\mathbf{h}_{\text{lane}_{t}}\in\mathbb{R}^{N\times d_{l}}$ for
categorical observations of driving lane IDs $\mathbf{l}_{t}$. Thus, the input
feature dimension $d_{f}=4+d_{l}$. Each driving lane ID is mapped to a
corresponding embedding vector with dimension $d_{l}$. The entries of the lane
embedding vectors are initialized at random and learned during training.

In terms of the graph convolution operator $\ast_{\mathcal{G}}$, instead of
the Chebyshev spectral graph convolutional operator (Defferrard et al., 2016)
adopted by (Seo et al., 2018), which uses pre-defined edge weights, we adopt
graph convolutions based on graph attention mechanism (Brody et al., 2021;
Veličković et al., 2017), which determines the relevance of the vehicle’s
neighbors dynamically based on the vehicle states. Next we introduce the graph
attention based convolution $\mathbf{W}\ast_{\mathcal{G}}\mathbf{X}_{t}$
operated on the input observation matrix $\mathbf{X}_{t}$, whereas
$\mathbf{W}\ast_{\mathcal{G}}\mathbf{H}_{t}$ follows the same process on
hidden states $\mathbf{H}_{t}$ with corresponding weights. Denoting
$\mathbf{x}_{t}^{i}\in\mathbb{R}^{d_{f}}$ as the entry for vehicle $i$ in
$\mathbf{X}_{t}$, $\mathcal{N}_{t}^{i}$ as the set of neighbors of node $i$ at
$t$, $\mathbf{W}\ast_{\mathcal{G}}\mathbf{X}_{t}$ works as follows:

(2) $\mathbf{x}^{\prime
i}_{t}=\mathbf{W}\ast_{\mathcal{G}}\mathbf{X}_{t}\coloneqq\alpha_{i,i}\mathbf{W}\mathbf{x}_{t}^{i}+\sum_{j\in\mathcal{N}_{t}^{i}}\alpha_{i,j}\mathbf{W}\mathbf{x}_{t}^{j},$

where $\mathbf{x}^{\prime i}_{t}\in\mathbb{R}^{d_{h}}$ is the output node
embedding, $\mathbf{W}\in\mathbb{R}^{d_{h}\times d_{f}}$ is the weight matrix.
The term $\alpha_{i,j}$ is the attention score calculated as:

(3) $\displaystyle
e_{i,j}=\left(\mathbf{a}^{\top}\mathrm{LeakyReLU}\left([\mathbf{W}\mathbf{x}_{t}^{i}\,\|\,\mathbf{W}\mathbf{x}_{t}^{j}]\right)\right),$
$\displaystyle\alpha_{i,j}=\text{softmax}(e_{i,j})=\frac{\text{exp}(e_{i,j})}{\sum_{k\in\mathcal{N}_{t}^{i}}\text{exp}(e_{i,k})},$

where $\|$ is the concatenate operator, and $\mathbf{a}\in\mathbb{R}^{2d_{h}}$
is a trainable weight vector. In (2) and (3), first every node input feature
goes through a linear transformation parameterized by $\mathbf{W}$. Then, the
attention coefficients $e_{i,j}$ are calculated by concatenating the
transformed node features, followed by a non-linear activation (LeakyReLU),
and a linear transformation with parameter $\mathbf{a}$. Then, for each node,
the attention coefficients of all its neighbors are normalized by a softmax
operator to reach the attention scores $\alpha_{i,j}$. Finally, the output
node embedding is calculated as the linear combination of its neighbor
transformed feature vectors, weighted by the attention scores.

Compared with works (Mohamed et al., 2020; Wiederer et al., 2022) that pre-
define edge weights as a function of physical distances, the attention
mechanism we use has more expressive power, and can comprehensively determine
the neighbor relevance based on observation information. Furthermore, the real
influence of vehicles in front of and behind an ego car is asymmetric (e.g.,
you must slow down immediately for slow-moving cars in front of you, but not
for slow cars behind you), and the attention formulation in (3) can achieve
such asymmetry. In comparison, when using a function of distance as edge
weights (Mohamed et al., 2020; Wiederer et al., 2022), the same importance is
assigned to cars that are close to a vehicle, regardless if they are in front
or behind the ego vehicle.

Moreover, multi-head attention is used to attend to different aspects of the
neighborhood information, similar to (Vaswani et al., 2017; Veličković et al.,
2017). Specifically, $K$ independent heads is used following (2), and the
final results of the heads are averaged:

(4) $\mathbf{x}^{\prime
i}_{t}=\frac{1}{K}\sum_{k=1}^{K}\left(\alpha_{i,i}^{k}\mathbf{W}^{k}\mathbf{x}_{t}^{i}+\sum_{j\in\mathcal{N}_{t}^{i}}\alpha_{i,j}^{k}\mathbf{W}^{k}\mathbf{x}_{t}^{j}\right).$

The hidden states for all vehicles are updated at every time step by (1), (2)
and (3). The final hidden state $\mathbf{H}_{T}$ is used as the encoded vector
embedding.

###  3.4. Decoder

The decoder works in the same way as the encoder, using the RGAT structure. To
avoid the computation burden of re-calculating the edge set thresholded by the
reconstructed vehicle position at every step, we use the union of all edge
sets at encoding time steps,
$\mathcal{E_{T}}=\mathcal{E}_{1}\cup\mathcal{E}_{2}\cup\dots\cup\mathcal{E}_{T}$.
The edge set only limits the range of neighbors each node attends to, while
the importance of neighbors are calculated dynamically with attention
mechanism. The decoded hidden state
$\hat{\mathbf{H}}_{t}\in\mathbb{R}^{N\times d_{h}}$ further goes through a
fully connected layer to produce the output vehicle state reconstruction
$\hat{\mathbf{X}}_{t}$. The details of $\hat{\mathbf{X}}_{t}$ will be
explained in Section 3.5. Then, the output $\hat{\mathbf{X}}_{t}$ is used as
input for the next recurrent step. We decode the time series from time $T$
backwards, since the encoder vector is most relevant to the states at time
$T$, which is most recently encoded. The initial input into the decoder GAT-
RNN is the vehicle states at time $T$.

###  3.5. Loss function

For each vehicle $i$ at time $t$, we assume its longitudinal position
$x_{t}^{i}$, speed $v^{i}_{t}$, and acceleration $a^{i}_{t}$ each follows an
univariate Gaussian distribution. That is,
$x^{i}_{t}\sim\mathcal{N}\left(\mu_{x_{t}^{i}},\sigma_{x_{t}^{i}}\right)$,
$v^{i}_{t}\sim\mathcal{N}\left(\mu_{v_{t}^{i}},\sigma_{v_{t}^{i}}\right)$,
$a^{i}_{t}\sim\mathcal{N}\left(\mu_{a_{t}^{i}},\sigma_{a_{t}^{i}}\right)$.
Further, we assume the lateral lane position $l_{t}^{i}$ is a discrete choice
among $L$ lanes, with an underlying categorical distribution
$\left\\{p_{1_{t}^{i}},\dots,p_{L_{t}^{i}}\right\\}$. The Gaussian
distribution and categorical distribution parameters are estimated by decoder
output $\hat{\mathbf{X}}$. That is,
$\hat{\mathbf{x}}^{i}_{t}=\left[\hat{\mu}_{x_{t}^{i}},\hat{\sigma}_{x_{t}^{i}},\hat{\mu}_{v_{t}^{i}},\hat{\sigma}_{v_{t}^{i}},\hat{\mu}_{a_{t}^{i}},\hat{\sigma}_{a_{t}^{i}},\hat{p}_{1_{t}^{i}},\dots,\hat{p}_{L_{t}^{i}}\right]$
estimates the mean and variance of position, speed, and acceleration, as well
as the probability of being in each lane.

Denoting the estimated probability density function of position, speed, and
acceleration as
$q\left(x^{i}_{t}|\hat{\mu}_{x_{t}^{i}},\hat{\sigma}_{x_{t}^{i}}\right)$,
$q\left(v^{i}_{t}|\hat{\mu}_{v_{t}^{i}},\hat{\sigma}_{v_{t}^{i}}\right)$, and
$q\left(a^{i}_{t}|\hat{\mu}_{a_{t}^{i}},\hat{\sigma}_{a_{t}^{i}}\right)$
respectively, we aim to minimize the negative log-likelihoods as follows:

(5)
$\displaystyle\mathcal{L}_{x_{t}^{i}}=-\text{log}\left(q\left(x^{i}_{t}|\hat{\mu}_{x_{t}^{i}},\hat{\sigma}_{x_{t}^{i}}\right)\right),$
$\displaystyle\mathcal{L}_{v_{t}^{i}}=-\text{log}\left(q\left(v^{i}_{t}|\hat{\mu}_{v_{t}^{i}},\hat{\sigma}_{v_{t}^{i}}\right)\right),$
$\displaystyle\mathcal{L}_{a_{t}^{i}}=-\text{log}\left(q\left(a^{i}_{t}|\hat{\mu}_{a_{t}^{i}},\hat{\sigma}_{a_{t}^{i}}\right)\right).$

As for lane classification, we aim to minimize the cross entropy loss as
follows:

(6)
$\mathcal{L}_{l_{t}^{i}}=-\sum_{l=1}^{L}\mathds{1}_{l_{t}^{i}}\text{log}\left(\hat{p}_{l_{t}^{i}}\right),$

where $\mathds{1}_{l_{t}^{i}}=1$ if vehicle $i$ is in lane $l$ at time t and 0
otherwise. The final loss is a weighted sum of the negative log-likelihood
losses and the cross entropy loss across all agents and all times:

(7)
$\displaystyle\mathcal{L}^{i}_{t}=\lambda_{x}\mathcal{L}_{x_{t}^{i}}+\lambda_{v}\mathcal{L}_{v_{t}^{i}}+\lambda_{a}\mathcal{L}_{a_{t}^{i}}+\lambda_{l}\mathcal{L}_{l_{t}^{i}},$
$\displaystyle\mathcal{L}=\sum_{i=1}^{N}\sum_{t=1}^{T}\mathcal{L}^{i}_{t},$

The weights are set as $\lambda_{x}=1$ , $\lambda_{v}=1$ , $\lambda_{a}=2$ and
$\lambda_{l}=2$ empirically.

###  3.6. Anomaly detection

For abnormal vehicle detection, the anomaly score for vehicle $i$ over time
window $\mathcal{T}$ is calculated by averaging the loss over all time steps:

(8)
$\displaystyle\alpha^{i}_{\mathcal{T}}=\frac{1}{T}\sum_{t=1}^{T}\mathcal{L}_{t}^{i},$

where $T$ is the length of time window $\mathcal{T}$. For abnormal scene
detection, we aggregate the loss of all the vehicles that appear in the
stretch $\mathcal{S}$ during $\mathcal{T}$ as the stretch anomaly score:

(9)
$\displaystyle\alpha^{\mathcal{S}}_{\mathcal{T}}=\text{max}(\mathcal{L}_{t}^{i}),\forall(i,t)\text{
that }x^{i}_{t}\in\mathcal{S}.$

The maximum aggregation is used instead of the mean for abnormal scene
detection, so that the score is more sensitive to the existence of anomalies,
and not averaged out by normal vehicles. Appendix C conducts a detailed
comparison between maximizing and averaging.

##  4\. Experiments

In this section, we use two data sources to evaluate the performance of our
method. First, simulation data is used to quantitatively compare the
performance of our method with the baselines, where we have the ground truth
anomaly labels. Second, highD dataset (Krajewski et al., 2018) is used to
qualitatively show our method works on real-world trajectories. The code is
publicly available on GitHub222https://github.com/yuehu9/DSAB-Detecting-
Socially-Abnormal-Drving-Behaviors.

###  4.1. Datasets

The detailed information of the two datasets is as follows.

####  4.1.1. Simulation data

TransModeler is a microscopic traffic simulator that generates vehicle
trajectories mimicking human driving behavior and interactions. In this work,
we generate a set of recordings at 1Hz on a 5-mile stretch of a 4-lane
highway. The recordings have different traffic flows and vehicle type
distribution to include different traffic conditions and abnormal scenarios.
Specifically, we include the following scenarios:

  * •

Normal traffic. A standard car following model, Modified General Motors
(Ahmed, 1999) is used, which has been demonstrated to correlate well with
field traffic data. The desired speed of the vehicles follows a typical
distribution found in real-world traffic, with around 5% speeding vehicles and
5% slow vehicles. We include both free flow and congested conditions with
varying traffic demands.

  * •

Speeding. The abnormal speeding cars drive at least 15 mph faster than the
other vehicles when it is possible to do so.

  * •

Slow. The abnormal slow cars drive at least 15 mph slower than the other
vehicles when it is possible to do so.

  * •

Tailgating. The tailgating vehicle’s headway is less than 0.5s from the lead
car. We simulate tailgating cars via a Constant Time Gap car-following model,
where drivers can keep a constant desired headway from the leading vehicle
(Wang and Rajamani, 2004).

  * •

Stalled car. A vehicle randomly stops on the road for a period.

  * •

Comprehensive scenario. We include all the above abnormal scenarios, i.e.,
speeding, slow, tailgating and stalled vehicles are all present in a single
scenario, to comprehensively evaluate model detection performance.

We include both common anomalies (e.g., speeding, tailgating and stalled
vehicles (Matousek et al., 2019; Alkinani et al., 2020; Moukafih et al., 2019;
Bai et al., 2019)) as well as those that are not commonly studied (e.g., slow
driving). Slow driving can create moving bottlenecks with adverse impacts on
traffic, but are difficult to identify using existing single-vehicle
approaches, demonstrating the importance of developing an interaction-based
detector.

The detailed experimental settings and distributions can be found in the
Appendix. We adjust the percentages in each abnormal scenario, so that the
abnormal behaving cars consist of around 3%-5% of total cars, randomly
distributed on the road. The actual anomaly rate has some variation across
different recordings because of simulation randomness. A total of 180 min of
normal traffic is used for training. The recordings of speeding, slow,
tailgating, stalled car and comprehensive scenarios are used for testing, each
lasting 10 min. The training data has 7,886 cars in total, producing 920,311
trajectories when segmented into 15s windows at 1s stride. The testing sets
together have a total of 5,067 cars and 630,718 trajectories. Detailed
statistics can be found in the Appendix.

We note that our training dataset is not perfectly clean, but contains a small
portion of abnormal data (e.g., speeding and slow cars). This is intentionally
done to simulate the real-world situation where we may not have a clean
labeled training set that is known to be free of anomalous vehicles.
Experiments excluding training anomalies can be found in Appendix D, which
shows similar results.

####  4.1.2. Real-world data

The HighD dataset (Krajewski et al., 2018) contains high accuracy vehicle
trajectories extracted from video recordings captured via unmanned aerial
vehicles over various stretches of German highways, each stretching
approximately 1300 feet long. We select a total of approximately 350 minutes
of recordings on three-lane highways as our training data, which covers both
light and heavy traffic conditions, with traffic flow varying from 1200 to
3600 vehicles/lane/hour. Then, we test on an unseen 15-min recording with a
flow of 2300 vehicles/lane/hour. The training data has 42,106 cars in total,
producing 459,187 trajectories when segmented into 10s windows at 1s stride.
The testing sets together has 1,795 cars and 21,840 trajectories. For the
HighD data, we do not have the ground truth anomaly labels. Thus, we
qualitatively examine the top anomalies detected by our model.

###  4.2. Baselines and Metrics

####  4.2.1. Baselines

We compare our method with simple heuristic methods as well as state-of-the-
art methods for anomaly detection on trajectory data. The baselines include:
i) Linear temporal interpolation (LTI) implemented by (Wiederer et al., 2022)
that uses a linear interpolation between the first and last position of the
vehicle to reconstruct the trajectory; ii) Constant Velocity Model (CVM)
(Schöller et al., 2020) that assumes constant speed as recorded at the first
observation time step to reconstruct the trajectory; iii) Robust tensor
Recovery (RTR) (Hu and Work, 2020) model that captures spatial-temporal
correlations via low-rank tensor decomposition, and detects sparse outliers
that deviates from the normal patterns; iv) Seq2Seq model (Sutskever et al.,
2014) that encodes and decodes the time series with two LSTM networks, and
uses Mean Square Error (MSE) as the reconstruction loss; v) Spatio-temporal
graph auto-encoder (STGAE) (Wiederer et al., 2022) that uses convolutional
networks temporally and graph convolutional networks spatially, and uses bi-
variate Gaussian reconstruction error to build autoencoder to derive anomaly
score333We use the STGAE-biv variant from the paper, since the version with
KDE is computationally too expensive. With $n$ trajectories in the training
set, and $m$ trajectories in the testing set, the KDE complexity is $O(mn)$.
The KDE did not complete given two days of computation time.; vi) DSAB-biv,
which is the variant of our model, that uses the same RGAT structure, but uses
the bi-variate Gaussian loss as in (Wiederer et al., 2022; Mohamed et al.,
2020). Each of the baseline methods produces an anomaly score for the vehicles
in each time step within a time window, and the same process is used following
Eq (8) and (9) to calculate the vehicle and scene anomaly scores.

Out of all the baselines, Seq2Seq, RTR, CVM and LTI consider only each
vehicle’s own trajectory, and STGAE considers the relationships between
vehicles. The heuristic models CVM and LTI are able to detect non-free-flow
scenarios where the vehicle speed changes dramatically. The tensor PCA can
capture linear correlations among trajectories, and the neural-network methods
Seq2seq and STGAE can further capture non-linear patterns.

####  4.2.2. Metrics

We follow the practice used in the anomaly detection works (Liu et al., 2022;
Ding et al., 2021; Pei et al., 2022), and include the following standard
metrics: i) ROC-AUC score, which is widely used for anomaly detection. ii)
Average precision, which summarizes the precision-recall curve into a single
value. iii) Precision@k. In settings where one is only able to e.g., manually
verify a fixed number of anomalies, Precision@k measures the relevance of the
samples we check.

###  4.3. Implementation details

The model setting for DSAB is as follows. For simulated data: For graph
construction, the distance threshold $\delta_{x}$ is 0.1 miles, and the lane
threshold $\delta_{l}=1$, i.e., vehicles only attend to its own and immediate
neighboring lanes within 0.1 miles upstream and downstream. The window size
$T$ is 15s, and sampling interval is 1s. The model hidden size $d_{h}=5$, the
number of attention heads $K=3$. In training, the model is trained for 500
epochs, with a batch size of 64. The initial learning rate is 0.05, and
decreases by half every 50 epochs. Gradient clipping is used to avoid
exploding gradients, with max norm of 1. For abnormal scene detection, the
stretch length $\delta_{s}$ is set to 0.15 miles. For the highD data: The
window size $T$ is 10s. The distance threshold $\delta_{x}$ is 0.2 mile. The
batch size is set to 256. All other configurations are the same as the
simulation data above.

Our model requires a constant number of vehicles in a single time window. For
vehicles with incomplete trajectories, due to vehicles entering or leaving the
observed stretch, two approaches can be used: i) discard the vehicles with
incomplete trajectories; or ii) linearly extrapolate the trajectories assuming
constant velocity dynamics, and then mask the extrapolated part when
calculating the loss. The first approach is used for simulation data on long
stretches with a small portion of incomplete trajectories. the second approach
is used for real-world highD data with short stretches, and a large fraction
of incomplete trajectories.

Table 1. Abnormal vehicle detection performance on the test set of comprehensive scenario, where all abnormal behaviors exist on the highway. Our model is the only one that can identify anomalous cars in traffic.  | Pre@100 | Pre@200 | Pre@500 | Avg Pre | AUC
---|---|---|---|---|---
LTI | 0.21 | 0.205 | 0.210 | 0.093 | 0.623
CVM | 0.15 | 0.140 | 0.142 | 0.089 | 0.617
RTR | 0.16 | 0.190 | 0.228 | 0.109 | 0.598
Seq2seq | 0.56 | 0.435 | 0.310 | 0.132 | 0.770
STGAE | 0.18 | 0.180 | 0.178 | 0.093 | 0.641
DSAB-biv | 0.15 | 0.150 | 0.114 | 0.062 | 0.655
DSAB (Ours) | 0.82 | 0.775 | 0.726 | 0.381 | 0.900
Table 2. Abnormal scene detection performance. While identifying abnormal scene is an easier task than identifying specific abnormal cars, and most method can perform well, ours is still the best in performance, with an increase of 0.1 in ROC-AUC score and average precision. | Pre@100 | Pre@200 | Pre@500 | Avg Pre | AUC
---|---|---|---|---|---
LTI | 0.89 | 0.860 | 0.834 | 0.690 | 0.726
CVM | 0.67 | 0.625 | 0.640 | 0.662 | 0.714
RTR | 0.89 | 0.895 | 0.864 | 0.769 | 0.744
Seq2seq | 0.87 | 0.870 | 0.860 | 0.742 | 0.745
STGAE | 0.82 | 0.740 | 0.776 | 0.666 | 0.678
DSAB-biv | 0.73 | 0.665 | 0.610 | 0.470 | 0.551
DSAB (Ours) | 0.93 | 0.950 | 0.912 | 0.859 | 0.841

###  4.4. Results and analysis

In this section, we quantitatively compare our method with the baselines on
simulation data. We also conduct ablation studies to see the influence of each
model component on performance.

Table 3. Abnormal car detection performance on each individual anomaly scenario. Our method is the only one that can detect slow and stalled car in traffic, and is also the best at detecting speeding cars.  | slow | speeding | tailgating | stalled
---|---|---|---|---
| Pre@100 | Avg Pre | AUC | Pre@100 | Avg Pre | AUC | Pre@100 | Avg Pre | AUC | Pre@100 | Avg Pre | AUC
LTI | 0.00 | 0.023 | 0.443 | 0.82 | 0.512 | 0.877 | 0.02 | 0.161 | 0.835 | 0.00 | 0.030 | 0.010
CVM | 0.01 | 0.023 | 0.432 | 0.67 | 0.472 | 0.871 | 0.06 | 0.177 | 0.840 | 0.00 | 0.030 | 0.010
RTR | 0.00 | 0.030 | 0.466 | 0.61 | 0.365 | 0.655 | 0.08 | 0.102 | 0.400 | 0.00 | 0.046 | 0.410
Seq2seq | 0.02 | 0.046 | 0.668 | 0.85 | 0.549 | 0.923 | 0.18 | 0.091 | 0.709 | 0.03 | 0.056 | 0.676
STGAE | 0.01 | 0.048 | 0.573 | 0.83 | 0.335 | 0.712 | 0.01 | 0.055 | 0.491 | 0.00 | 0.025 | 0.454
DSAB-biv | 0.00 | 0.025 | 0.465 | 0.29 | 0.092 | 0.600 | 0.05 | 0.043 | 0.436 | 0.13 | 0.066 | 0.696
DSAB (Ours) | 0.97 | 0.246 | 0.841 | 1.00 | 0.907 | 0.995 | 0.33 | 0.148 | 0.747 | 0.98 | 0.148 | 0.762

(a) attention heads

A line chart of Pre100 vs the number of attention heads K, where K has four
discrete values of 1,2,3 and 4, and Pre100 mostly centers at 0.75 but gets a
value of 0.5 when there is only one attention head

(b) hidden dimension

A line chart of Pre100 vs hidden dimension, where hidden dimension has four
discrete values of 5,10,15,20, and Pre100 has corresponding values between
0.75 and 1.00.

(c) attention distance

A line chart of Pre100 vs attention distance, where attention distance has
five discrete values of 0.00,0.05,0.10,0.15,0.20 and Pre100 has corresponding
values between 0.5 and 0.75.

(d) sample time

A line chart of Pre100 vs sample time, where sample time has four discrete
values of 1,2,3,4, and Pre100 has corresponding values between 0.5 and 1.00.

(e) window size

A line chart of Pre100 vs window size, where window size is from 2s to 20s and
Pre100 has corresponding values between 0.75 and 1.00.

Figure 2. Influence of different parameters. The model benefits from having
more than one attention head, and is not sensitive to the number of heads and
hidden dimensions otherwise. On spatio-temporal graph construction, when
attention distance threshhold equals zero, a car does not attend to any
neighbor, and the performance significantly drops. Otherwise, the model is not
sensitive to the window size and sample frequency, allowing us flexibility to
choose parameters.

####  4.4.1. Model comparison

We first benchmark the methods on the test set of the comprehensive scenario,
where all abnormal behaviors exist on the highway. The result for abnormal
vehicle detection is shown in Table 1. We can see that our model is the only
one that can identify the specific car that behaves abnormally in the traffic.
The precision scores at different k vales are constantly above 0.7 in our
method. For the other methods, the precision score are around 0.2 most of the
time, and are never higher than 0.6. Our average precision is also around 0.3
higher than the second best method. The ROC-AUC score for our method is 0.9,
which is more than 0.1 higher than the second best method. This shows that
without consideration of neighboring vehicles, Seq2Seq, RTR, CVM and LTI are
unable to detect the social anomalies. Meanwhile, although STGAE and DSAB-biv
both model the neighboring vehicle interactions via graph neural networks,
they use bi-viariate Gaussian trajectory loss and cannot well capture the
discrete lane-changing behaviors, and have similarly poor results.

We also report the performance of abnormal scene detection for all method,
shown in Table 2. We can see that the performance of all methods have a
significant improvement. This is because detecting abnormal scenes is easier
task than detecting the specific abnormal vehicles. For example, a slow or
stalled car can cause its following cars to brake abruptly, which baseline
models like CVM are able to detect. Moreover, when aggregating a group of
vehicles over space and time, the abnormal rate is higher, and the methods
work much better on a balanced dataset. Nonetheless, our method is still the
best, with an increase of 0.1 in AUC score and average precision.

####  4.4.2. Performance on individual anomaly scenarios

Next, we investigate the model performance on each individual anomaly
scenario. The result is shown in Table 3. We can see that our method is the
only one that can detect slow and stalled cars in traffic, with Precision@100
score above 0.9, compared with scores constantly less than 0.1 for other
methods. While all methods perform relatively well at detecting speeding cars,
our method is the best, with an improvement of 0.1 in Precision@100, and a
ROC-AUC score near 1. Our method is relatively less effective in detecting
tailgating cars, yet the Precision@100 is still the highest among all methods,
and no method is constantly better than ours in every metric. We note that
baseline methods can detect tailgating cars, because they have different car-
following dynamics as described in section 4.1.

####  4.4.3. Ablation study

In this section, we examine the influence of different components of the
model, as well as model sensitivity to the configurations of the spatio-
temporal graph data.

First, we study the influence of the model parameters, with results shown in
Fig 2(a) and 2(b). The influence of number of attention heads is shown in Fig
2(a). We can see that the model benefits from having more than one attention
head, while being not sensitive when the number of heads is larger than one.
Meanwhile, Fig 2(b) indicates that the model is not sensitive to the hidden
dimension. Thus we choose 5 as hidden dimension for a smaller model size.

Next, we study the influence of spatio-temporal graph parameters, with results
shown in Fig 2(c), 2(e) and 2(d). Spatially, the threshold of attention
distance determines how far away in distance one vehicle attends to as its
neighboring car. A threshold of zero means a car do not attend to any
neighbor, and consequently the spatial graph is not used. We can see in Fig
2(c) that when the threshold equals zero, the performance significantly drops
by 0.3 in Precision@100. When threshold is larger than 0.05 miles, the
performance is not sensitive to the threshold value. The result indicates that
the neighboring vehicles within 0.05 miles around the ego vehicle is the most
important. Temporally, we study the influence of the time window size and the
sample frequency. We first fix the sampling interval at 1s and vary the window
size from 2s to 20s, then fix the window size at 15s and vary the sampling
interval from 1s to 4s. Fig 2(e) and Fig 2(d) show the results. We can see
that the model performs best when interval is less or equal to 3s, and is not
sensitive to the window size and sample frequency otherwise. This allows us
flexibility to choose a larger window size and coarser sampling frequency to
aid computational efficiency.

###  4.5. Qualitative Results

In this section, we qualitatively show the performance of our model on real-
world HighD data. The traffic in this data has unique characteristics, and our
model is able to learn the norms and capture the anomalies that deviates from
the norm.

Specifically, the speed limit for German highways is very loose, at least 75
mph, or no speed limits in some parts. And it is not uncommon to observe
speeds larger than 80 mph. Thus, speeding is not ranked among the largest
anomalies. On the other hand, the typical traffic speed varies significantly
by lane, as shown in Fig 3(a). From leftmost to rightmost lane, the average
speed are 76, 69 and 56 mph respectively, calculated from training data. That
is to say, the faster the car, the more to the left the car tends to be.
Accordingly, the vehicle with much higher speed than its corresponding lane is
detected as abnormal, even though the speed is absolutely normal when lanes
are not considered. We manually inspect the vehicles with top anomaly scores,
and found them to be abnormal either because of aggressive driving with
dramatic acceleration/deceleration, or because of speed range violation with
respect to the lane.

We show some of the top anomalies in Fig 3. Each line denotes the trajectory
of a single car, with a dot denoting the starting point and triangle the end
point. We add small perturbations laterally to aid visualization. The speed,
acceleration and vehicle anomaly score at the corresponding time are shown.
The reconstruction shows the trajectories with the largest probability. Fig
3(b) shows the vehicle with largest anomaly score. The abnormal vehicle 1 is
cutting in front of vehicle 2, while having dramatic deceleration, forcing the
other car to change lane as well. Fig 3(c) shows the vehicle with second
largest anomaly score. The abnormal vehicle 1 is driving too fast with respect
to the lane it is in – 20 mph larger than the typical speed in rightmost lane.
The reconstruction actually puts vehicle 1 in the middle lane which has larger
typical speed.

While we presented the top two anomalies as identified by DSAB, with more
examples in Appendix E, the result illustrates that the method is applicable
to real world datasets.

(a) Normal traffic condition. The typical speed varies by lane. From leftmost
to rightmost lane, the average speed are 76, 69 and 56 mph respectively,
calculated from training data

A rectangular figure containing vehicle trajectories with x-axis as distance
and y-axis as lane id. There are multiple lines on the figure, each of which
denotes the trajectory of a single vehicle, with a dot denoting the starting
point and triangle denoting the endpoint. The lines has different colors to
represent the vehicle speed, where the color bar is roughly from red to yellow
to green and represents 40 to 60 to 80. There are several green lines on lane
0, several light green lines on lane1 and two orange lines on lane 2.

(b) Vehicle with largest anomaly score. The abnormal vehicle 1 is cutting in
front of vehicle 2, while having dramatic deceleration, forcing vehicle 2 to
change lane as well.

Four rectangular figures containing vehicle trajectories with x-axis as
distance and y-axis as lane id. There are multiple lines on the figure, each
of which denotes the trajectory of a single vehicle, with a dot denoting the
starting point and a triangle denoting the endpoint. The lines has different
colors to represent different features. For the first top figure, the color
represents vehicle acceleration where the color bar is from red to yellow to
green and represents -2 to 0 to 2. There is one red line with starting point
at lane 0 but end point at lane 1. The rest lines have relatively more yellow
or more green colors. For the second top figure, the color represents vehicle
speed where the color bar is from red to yellow to green and represents 40 to
60 to 80. All lines have a similar pattern with the first top figure but with
different colors. For the third figure, the color represents the original
normalized anomaly score of each vehicle where the color bar is from blue to
white to red and represents 0 to 0.5 to 1. The red line mentioned in the first
top figure also has red color here while the rest lines are all blue. For the
fourth figure, the color represents the reconstructed normalized anomaly score
of each vehicle where the color bar is from blue to white to red and
represents 0 to 0.5 to 1. The red line mentioned in the first top figure also
has red color here while the rest lines are all blue.

(c) Vehicle with second largest anomaly score. The abnormal vehicle 1 is
driving too fast with respect to the lane it is in. The reconstruction
actually puts vehicle 1 in the middle lane which has larger typical speed.

Four rectangular figures containing vehicle trajectories with x-axis as
distance and y-axis as lane id. There are multiple lines on the figure, each
of which denotes the trajectory of a single vehicle, with a dot denoting the
starting point and triangle denoting the endpoint. The lines has different
colors to represent different features. For the first top figure, the color
represents vehicle acceleration where the color bar is from red to yellow to
green and represents -2 to 0 to 2. There is one green line with index of
vehicle 1 on lane 2. The rest lines also have green colors. For the second top
figure, the color represents vehicle speed where the color bar is from red to
yellow to green and represents 40 to 60 to 80. There is one green line with
index of vehicle 1 on lane 2. The rest lines also have green colors. For the
third figure, the color represents the original normalized anomaly score of
each vehicle where the color bar is from blue to white to red and represents 0
to 0.5 to 1. The line with index of vehicle 1 in the first top figure has red
color here while the rest lines are all blue. For the fourth figure, the color
represents the reconstructed normalized anomaly score of each vehicle where
the color bar is from blue to white to red and represents 0 to 0.5 to 1. The
line with index of vehicle 1 in the first top figure has red color here while
the rest lines are all blue.

Figure 3. Qualitative study of real-world HighD traffic data

##  5\. Conclusion

Advanced connectivity and sensing will continue to transform intelligent
traffic management. In this work, we tackle an important problem of abnormal
driving behavior detection, using trajectories produced by IoT highway video
surveillance systems. Specifically we detect the exact anomalous vehicles
considering the vehicular interaction with DSAB, an autoencoder based on
Recurrent Graph Attention Networks. The results demonstrate the method
captures the spatial-temporal trajectory dynamics, while considering both the
neighbor interactions and the stocasticity in driving behaviors. Extensive
experiments on both simulation and real-world data sets show the ability of
our model to scale to large highway monitoring systems with thousands of
vehicles, and detect a variety of abnormal behaviors. The performance on
identifying single vehicle anomalies is state of the art, indicating potential
to pinpoint specific problematic vehicles in a large traffic stream.

Acknowledgments This work is supported by the National Science Foundation
(NSF) under Grant No. CIS-2033580 and the USDOT Dwight D. Eisenhower
Fellowship program under Grant No. 693JJ322NF5201.

## References

  * (1)
  * Ahmed (1999) Kazi Iftekhar Ahmed. 1999\.  _Modeling drivers’ acceleration and lane changing behavior_.  Ph. D. Dissertation. Massachusetts Institute of Technology.
  * Alahi et al. (2016) Alexandre Alahi, Kratarth Goel, Vignesh Ramanathan, Alexandre Robicquet, Li Fei-Fei, and Silvio Savarese. 2016.  Social lstm: Human trajectory prediction in crowded spaces. In _Proceedings of the IEEE conference on computer vision and pattern recognition_. 961–971.
  * Alkinani et al. (2020) Monagi H Alkinani, Wazir Zada Khan, and Quratulain Arshad. 2020.  Detecting human driver inattentive and aggressive driving behavior using deep learning: Recent advances, requirements and open challenges.  _Ieee Access_ 8 (2020), 105008–105030.
  * Bai et al. (2019) Shuai Bai, Zhiqun He, Yu Lei, Wei Wu, Chengkai Zhu, Ming Sun, and Junjie Yan. 2019\.  Traffic anomaly detection via perspective map based on spatial-temporal information matrix.. In _CVPR Workshops_. 117–124.
  * Bandyopadhyay et al. (2020) Sambaran Bandyopadhyay, Saley Vishal Vivek, and MN Murty. 2020\.  Outlier resistant unsupervised deep architectures for attributed network embedding. In _Proceedings of the 13th international conference on web search and data mining_. 25–33.
  * Brody et al. (2021) Shaked Brody, Uri Alon, and Eran Yahav. 2021.  How attentive are graph attention networks?  _arXiv preprint arXiv:2105.14491_ (2021).
  * Cai et al. (2021) Lei Cai, Zhengzhang Chen, Chen Luo, Jiaping Gui, Jingchao Ni, Ding Li, and Haifeng Chen. 2021.  Structural temporal graph neural networks for anomaly detection in dynamic graphs. In _Proceedings of the 30th ACM international conference on Information & Knowledge Management_. 3747–3756.
  * Carlos et al. (2018) Manuel Ricardo Carlos, Mario Ezra Aragón, Luis C González, Hugo Jair Escalante, and Fernando Martínez. 2018.  Evaluation of detection approaches for road anomalies based on accelerometer readings—Addressing who’s who.  _IEEE Transactions on Intelligent Transportation Systems_ 19, 10 (2018), 3334–3343.
  * Chen et al. (2021) Jingyuan Chen, Guanchen Ding, Yuchen Yang, Wenwei Han, Kangmin Xu, Tianyi Gao, Zhe Zhang, Wanping Ouyang, Hao Cai, and Zhenzhong Chen. 2021\.  Dual-modality vehicle anomaly detection via bilateral trajectory tracing. In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_. 4016–4025.
  * Cho et al. (2014) Kyunghyun Cho, Bart van Merriënboer, Dzmitry Bahdanau, and Yoshua Bengio. 2014.  On the Properties of Neural Machine Translation: Encoder–Decoder Approaches.  _Syntax, Semantics and Structure in Statistical Translation_ (2014), 103.
  * Chung et al. (2014) Junyoung Chung, Caglar Gulcehre, Kyunghyun Cho, and Yoshua Bengio. 2014.  Empirical evaluation of gated recurrent neural networks on sequence modeling. In _NIPS 2014 Workshop on Deep Learning, December 2014_.
  * Defferrard et al. (2016) Michaël Defferrard, Xavier Bresson, and Pierre Vandergheynst. 2016.  Convolutional neural networks on graphs with fast localized spectral filtering.  _Advances in neural information processing systems_ 29 (2016).
  * Ding et al. (2021) Kaize Ding, Jundong Li, Nitin Agarwal, and Huan Liu. 2021\.  Inductive anomaly detection on attributed networks. In _Proceedings of the Twenty-Ninth International Conference on International Joint Conferences on Artificial Intelligence_. 1288–1294.
  * Ding et al. (2019) Kaize Ding, Jundong Li, Rohit Bhanushali, and Huan Liu. 2019\.  Deep anomaly detection on attributed networks. In _Proceedings of the 2019 SIAM International Conference on Data Mining_. SIAM, 594–602.
  * Fazeen et al. (2012) Mohamed Fazeen, Brandon Gozick, Ram Dantu, Moiz Bhukhiya, and Marta C González. 2012.  Safe driving using mobile phones.  _IEEE Transactions on Intelligent Transportation Systems_ 13, 3 (2012), 1462–1468.
  * Gatteschi et al. (2021) Valentina Gatteschi, Alberto Cannavò, Fabrizio Lamberti, Lia Morra, and Paolo Montuschi. 2021\.  Comparing Algorithms for Aggressive Driving Event Detection Based on Vehicle Motion Data.  _IEEE Transactions on Vehicular Technology_ 71, 1 (2021), 53–68.
  * Gloudemans et al. (2023) Derek Gloudemans, Yanbing Wang, Junyi Ji, Gergely Zachar, Will Barbour, and Daniel B. Work. 2023\.  I-24 MOTION: An instrument for freeway traffic science.  https://doi.org/10.48550/ARXIV.2301.11198
  * Gupta et al. (2018) Agrim Gupta, Justin Johnson, Li Fei-Fei, Silvio Savarese, and Alexandre Alahi. 2018.  Social gan: Socially acceptable trajectories with generative adversarial networks. In _Proceedings of the IEEE conference on computer vision and pattern recognition_. 2255–2264.
  * Hu et al. (2022) Yue Hu, Ao Qu, and Dan Work. 2022.  Detecting extreme traffic events via a context augmented graph autoencoder.  _ACM Transactions on Intelligent Systems and Technology (TIST)_ 13, 6 (2022), 1–23.
  * Hu and Work (2020) Yue Hu and Daniel B Work. 2020.  Robust Tensor Recovery with Fiber Outliers for Traffic Events.  _ACM Transactions on Knowledge Discovery from Data (TKDD)_ 15, 1 (2020), 1–27.
  * Huang et al. (2019) Yingfan Huang, Huikun Bi, Zhaoxin Li, Tianlu Mao, and Zhaoqi Wang. 2019.  Stgat: Modeling spatial-temporal interactions for human trajectory prediction. In _Proceedings of the IEEE/CVF International Conference on Computer Vision_. 6272–6281.
  * Jeni et al. (2013) László A Jeni, Jeffrey F Cohn, and Fernando De La Torre. 2013.  Facing imbalanced data–recommendations for the use of performance metrics. In _2013 Humaine association conference on affective computing and intelligent interaction_. IEEE, 245–251.
  * Kipf and Welling (2016) Thomas N Kipf and Max Welling. 2016.  Semi-supervised classification with graph convolutional networks.  _arXiv preprint arXiv:1609.02907_ (2016).
  * Krajewski et al. (2018) Robert Krajewski, Julian Bock, Laurent Kloeker, and Lutz Eckstein. 2018.  The highD Dataset: A Drone Dataset of Naturalistic Vehicle Trajectories on German Highways for Validation of Highly Automated Driving Systems. In _2018 21st International Conference on Intelligent Transportation Systems (ITSC)_. 2118–2125.  https://doi.org/10.1109/ITSC.2018.8569552
  * Lee et al. (2017) Namhoon Lee, Wongun Choi, Paul Vernaza, Christopher B Choy, Philip HS Torr, and Manmohan Chandraker. 2017.  Desire: Distant future prediction in dynamic scenes with interacting agents. In _Proceedings of the IEEE conference on computer vision and pattern recognition_. 336–345.
  * Li et al. (2020) Yingying Li, Jie Wu, Xue Bai, Xipeng Yang, Xiao Tan, Guanbin Li, Shilei Wen, Hongwu Zhang, and Errui Ding. 2020.  Multi-granularity tracking with modularlized components for unsupervised vehicles anomaly detection. In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops_. 586–587.
  * Li et al. (2018) Yaguang Li, Rose Yu, Cyrus Shahabi, and Yan Liu. 2018\.  Diffusion Convolutional Recurrent Neural Network: Data-Driven Traffic Forecasting. In _International Conference on Learning Representations_.
  * Liu et al. (2022) Kay Liu, Yingtong Dou, Yue Zhao, Xueying Ding, Xiyang Hu, Ruitong Zhang, Kaize Ding, Canyu Chen, Hao Peng, Kai Shu, et al. 2022\.  Benchmarking Node Outlier Detection on Graphs.  _arXiv preprint arXiv:2206.10071_ (2022).
  * Liu et al. (2021) Yixin Liu, Shirui Pan, Yu Guang Wang, Fei Xiong, Liang Wang, Qingfeng Chen, and Vincent CS Lee. 2021.  Anomaly detection in dynamic graphs via transformer.  _IEEE Transactions on Knowledge and Data Engineering_ (2021).
  * Ma et al. (2021) Xiaoxiao Ma, Jia Wu, Shan Xue, Jian Yang, Chuan Zhou, Quan Z Sheng, Hui Xiong, and Leman Akoglu. 2021\.  A comprehensive survey on graph anomaly detection with deep learning.  _IEEE Transactions on Knowledge and Data Engineering_ (2021).
  * Matousek et al. (2019) Matthias Matousek, EL-Zohairy Mohamed, Frank Kargl, Christoph Bösch, et al. 2019\.  Detecting anomalous driving behavior using neural networks. In _2019 IEEE Intelligent Vehicles Symposium (IV)_. IEEE, 2229–2235.
  * Matousek et al. (2018) Matthias Matousek, Mahmoud Yassin, Rens van der Heijden, Frank Kargl, et al. 2018\.  Robust detection of anomalous driving behavior. In _2018 IEEE 87th Vehicular Technology Conference (VTC Spring)_. IEEE, 1–5.
  * Mohamed et al. (2020) Abduallah Mohamed, Kun Qian, Mohamed Elhoseiny, and Christian Claudel. 2020.  Social-stgcnn: A social spatio-temporal graph convolutional neural network for human trajectory prediction. In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_. 14424–14432.
  * Moukafih et al. (2019) Youness Moukafih, Hakim Hafidi, and Mounir Ghogho. 2019\.  Aggressive driving detection using deep learning-based time series classification. In _2019 IEEE International Symposium on INnovations in Intelligent SysTems and Applications (INISTA)_. IEEE, 1–5.
  * Pei et al. (2022) Yulong Pei, Tianjin Huang, Werner van Ipenburg, and Mykola Pechenizkiy. 2022.  ResGCN: attention-based deep residual modeling for anomaly detection on attributed networks.  _Machine Learning_ 111, 2 (2022), 519–541.
  * Sadeghian et al. (2019) Amir Sadeghian, Vineet Kosaraju, Ali Sadeghian, Noriaki Hirose, Hamid Rezatofighi, and Silvio Savarese. 2019.  Sophie: An attentive gan for predicting paths compliant to social and physical constraints. In _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_. 1349–1358.
  * Schöller et al. (2020) Christoph Schöller, Vincent Aravantinos, Florian Lay, and Alois Knoll. 2020.  What the constant velocity model can teach us about pedestrian motion prediction.  _IEEE Robotics and Automation Letters_ 5, 2 (2020), 1696–1703.
  * Seo et al. (2018) Youngjoo Seo, Michaël Defferrard, Pierre Vandergheynst, and Xavier Bresson. 2018.  Structured sequence modeling with graph convolutional recurrent networks. In _International conference on neural information processing_. Springer, 362–373.
  * Stocco et al. (2022) Andrea Stocco, Paulo J Nunes, Marcelo D’Amorim, and Paolo Tonella. 2022.  Thirdeye: Attention maps for safe autonomous driving systems. In _37th IEEE/ACM International Conference on Automated Software Engineering_. 1–12.
  * Stocco et al. (2020) Andrea Stocco, Michael Weiss, Marco Calzana, and Paolo Tonella. 2020.  Misbehaviour prediction for autonomous driving systems. In _Proceedings of the ACM/IEEE 42nd international conference on software engineering_. 359–371.
  * Sultani et al. (2018) Waqas Sultani, Chen Chen, and Mubarak Shah. 2018.  Real-world anomaly detection in surveillance videos. In _Proceedings of the IEEE conference on computer vision and pattern recognition_. 6479–6488.
  * Sutskever et al. (2014) Ilya Sutskever, Oriol Vinyals, and Quoc V Le. 2014\.  Sequence to sequence learning with neural networks.  _Advances in neural information processing systems_ 27 (2014).
  * Tang et al. (2019) Xianfeng Tang, Boqing Gong, Yanwei Yu, Huaxiu Yao, Yandong Li, Haiyong Xie, and Xiaoyu Wang. 2019.  Joint modeling of dense and incomplete trajectories for citywide traffic volume inference. In _The World Wide Web Conference_. 1806–1817.
  * Trirat and Lee (2021) Patara Trirat and Jae-Gil Lee. 2021.  Df-tar: a deep fusion network for citywide traffic accident risk prediction with dangerous driving behavior. In _Proceedings of the Web Conference 2021_. 1146–1156.
  * Vaswani et al. (2017) Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017.  Attention is all you need.  _Advances in neural information processing systems_ 30 (2017).
  * Veličković et al. (2017) Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. 2017.  Graph attention networks.  _arXiv preprint arXiv:1710.10903_ (2017).
  * Vemula et al. (2018) Anirudh Vemula, Katharina Muelling, and Jean Oh. 2018.  Social attention: Modeling attention in human crowds. In _2018 IEEE international Conference on Robotics and Automation (ICRA)_. IEEE, 4601–4607.
  * Wang and Rajamani (2004) Junmin Wang and R. Rajamani. 2004.  Should adaptive cruise-control systems be designed to maintain a constant time gap between vehicles?  _IEEE Transactions on Vehicular Technology_ 53, 5 (2004), 1480–1490.  https://doi.org/10.1109/TVT.2004.832386
  * Wang and Derr (2021) Yu Wang and Tyler Derr. 2021.  Tree decomposed graph neural network. In _Proceedings of the 30th ACM International Conference on Information & Knowledge Management_. 2040–2049.
  * Wang et al. (2022) Yu Wang, Wei Jin, and Tyler Derr. 2022.  Graph neural networks: Self-supervised learning.  _Graph Neural Networks: Foundations, Frontiers, and Applications_ (2022), 391–420.
  * Wiederer et al. (2022) Julian Wiederer, Arij Bouazizi, Marco Troina, Ulrich Kressel, and Vasileios Belagiannis. 2022\.  Anomaly Detection in Multi-Agent Trajectories for Automated Driving. In _Conference on Robot Learning_. PMLR, 1223–1233.
  * Wu et al. (2021) Jie Wu, Xionghui Wang, Xuefeng Xiao, and Yitong Wang. 2021\.  Box-level tube tracking and refinement for vehicles anomaly detection. In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_. 4112–4118.
  * Xu et al. (2022) Zhiming Xu, Xiao Huang, Yue Zhao, Yushun Dong, and Jundong Li. 2022.  Contrastive Attributed Network Anomaly Detection with Data Augmentation. In _Pacific-Asia Conference on Knowledge Discovery and Data Mining_. Springer, 444–457.
  * Yu et al. (2018) Wenchao Yu, Wei Cheng, Charu C Aggarwal, Kai Zhang, Haifeng Chen, and Wei Wang. 2018\.  Netwalk: A flexible deep embedding approach for anomaly detection in dynamic networks. In _Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining_. 2672–2681.
  * Zheng et al. (2019) Li Zheng, Zhenpeng Li, Jian Li, Zhao Li, and Jun Gao. 2019.  AddGraph: Anomaly Detection in Dynamic Graph Using Attention-based Temporal GCN.. In _IJCAI_. 4419–4425.

##  Appendix A Data details

We describe the detailed settings for simulation data, then provide the data
statistics for both simulation and highD data.

  * •

Normal traffic. A standard car following model, Modified General Motors
(Ahmed, 1999) is used, which has been demonstrated to correlate well with
field traffic data. The desired speed of the vehicles follows a typical
distribution found in real-world traffic, with the majority between 65-80 mph,
only 5% above 85 mph and 5% below 60 mph. Recordings of varying traffic
demands from 500 to 1600 vehicles/lane/hour are included, covering both free
flow and congested conditions.

  * •

Speeding scenario. We set 70% of the vehicles drive at a desired speed of 65
mph, and 30% above 85 mph. The traffic demand is 500 veh/l/hr.

  * •

Slow scenario. We set 98% of the vehicles drive at a desired speed of 65 mph,
and 2% below 50 mph. The traffic demand is 500 veh/l/hr.

  * •

Tailgating scenario. We set 46.80% cars to follow a Constant Time Gap model
and only a proportion of them could be tailgaters according to traffic
conditions. The traffic demand is 500 veh/l/hr.

  * •

Stalled car scenario. We randomly select 15 cars, each stopping for 5 minutes.
The traffic demand is 1500 veh/l/hr.

  * •

Comprehensive scenario. We set 89% of the vehicles drive at a desired speed of
65 mph, 10% at 85 mph, and 1% at 50 mph. In addition, we set 46.80% cars
following Constant Time Gap model and only a proportion of them could be
tailgaters according to traffic condition. We set 2 cars to each stall for 3
min. The traffic demand is 1000 veh/l/hr.

Cars are labeled anomaly only when it is actually behaving anomalously (e.g.,
when a car with desired speed of 85 mph can only drive at 65 mph because of
traffic conditions, it is not an anomaly at the corresponding time).

Table 4 shows the statistics of car count, and the total trajectory count when
divided into 10-15s time windows with 1s stride. We note that in simulation
data, the traffic flow in stalled car and comprehensive scenarios are higher,
because we want to check if we can detect the source anomaly car even if the
stalled car causes upstream congestion.

Table 4. Dataset statistics |  | Car count | Trajectory count
---|---|---|---
Simulation | training | 7,886 | 920,311
comprehensive | 1,281 | 147,254
slow | 636 | 75,809
speeding | 583 | 65,685
tailgating | 645 | 78,600
stalled | 1,922 | 263,370
HighD | training | 42,106 | 459,187
Testing | 1,795 | 21,840

##  Appendix B Description of Metrics

In this section, we describe the evaluation metrics.

i) area under the receiver operating characteristic curve (ROC-AUC) score. The
ROC curve plots the true positive rate (TPR) against the false positive rate
(FPR), and the ROC-AUC score calculates the area under the ROC curve. An ROC-
AUC score of 0.5 means the model is not able to discriminate anomalies, and an
ROC-AUC score of 1 means perfect anomaly detection.

ii) Average precision, which summarizes the precision-recall curve into a
single value, and is calculated as the weighted mean of previsions achieved at
each threshold, the weight being the increase in recall from the previous
threshold.

iii) Precision@k, which calculates the percentage of true anomaly among the
top k samples scored by the models to be anomalies.

We note that for vehicle detection problem, since the anomaly rate is only
3%-5%, severe data imbalance issue exists. It is shown in (Jeni et al., 2013)
that in heavily imbalanced datasets, metrics like F1 score downgrades
exponentially with data skewness, thus we do not include F1 metrics.
Meanwhile, ROC-AUC score is less influenced by data imbalance, but with a
caveat that model with distinctive performance could have very similar ROC-AUC
score. Precision@k shows how well we rank anomalies over normal samples, and
works well on imbalanced datasets where our focus is on the relatively rarely
occurring anomalies.

##  Appendix C Aggregator

We briefly describe the choice of loss aggregation over time and vehicles in
section 3.6. Averaging the losses is more conservative than maximizing. For
vehicle detection, since many abnormal behaviors persist for relatively long
time periods over several seconds, we choose to average over time.
Maximization results in around 0.1 decrease in average precision. For scene
detection, we choose maximization to avoid abnormal vehicles being averaged
out by normal vehicles. Averaging results in around 0.06 decrease in average
precision.

##  Appendix D Robustness analysis

We briefly explore the model robustness to anomalies in training data. As
described in section 4.1, our training data contains a small portion of
anomalies, to simulate the real-world situation of not having a perfectly
clean normal dataset. To evaluate the influence of training anomalies, we
compare the performance with model trained on clean training data. Table 5
shows the result. We can see that the results are similar, with differences in
scores all less than 0.05. There is also no clear trend of one dataset better
than the other. The result shows the model is robust to small number of
anomalies in the training data.

Table 5. Influence of anomalies in the training data. The performance of two training sets are comparable, with differences in scores all less than 0.05. |  | Training
---
data
Pre@100 | Pre@200 | Avg Pre | AUC
vehicle detection | Clean | 0.79 | 0.815 | 0.336 | 0.888
Polluted | 0.82 | 0.775 | 0.381 | 0.900
scene detection | Clean | 0.97 | 0.965 | 0.852 | 0.834
Polluted | 0.93 | 0.950 | 0.859 | 0.841

##  Appendix E HighD anomaly description

In this section, we provide more examples of the qualitative results for top
ranked anomalies in the HighD real-world data. We note that during ranking,
there might be samples with overlapping time windows, and we eliminate the
repetitions for diversity. The results are shown in Fig 4 and Fig 5. Out of
the top anomalies, most are because of drastic deceleration, as well as a
significant difference in speed relative to the speed of the surrounding cars,
or to the typical speed of the corresponding lane it is in.

(a) Vehicle with 3rd largest anomaly score. The abnormal vehicle (marked
$\ast$) is having dramatic deceleration and drives significantly slower than
other cars in its lane.

Four rectangular figures containing vehicle trajectories with x-axis as
distance and y-axis as lane id. There are multiple lines on the figure, each
of which denotes the trajectory of a single vehicle, with a dot denoting the
starting point and a triangle denoting the endpoint. The lines have different
color bars to represent different features.

For the first top figure, the color represents vehicle acceleration where the
color bar is from red to yellow to green and represents -2 to 0 to 2. There is
one extremely red line with starting point at lane 2 and end point at lane 2.
This line is marked by . The rest lines have relatively more yellow or green
colors. For the second top figure, the color represents vehicle speed where
the color bar is from red to yellow to green and represents 40 to 60 to 80.
All lines have a similar pattern with the first top figure but with different
colors. For the third figure, the color represents each vehicle’s original
normalized anomaly score where the color bar is from blue to white to red and
represents 0 to 0.5 to 1. The red line mentioned in the first top figure also
has red color here while the rest lines are all blue. For the fourth figure,
the color represents each vehicle’s reconstructed normalized anomaly score
where the color bar is from blue to white to red and represents 0 to 0.5 to 1.
The red line mentioned in the first top figure also has red color here while
the rest lines are all blue.

(b) Vehicle with 4th largest anomaly score. The abnormal vehicle (marked
$\ast$) is having dramatic deceleration.

Four rectangular figures containing vehicle trajectories with x-axis as
distance and y-axis as lane id. There are multiple lines on the figure, each
of which denotes the trajectory of a single vehicle, with a dot denoting the
starting point and a triangle denoting the endpoint. The lines have different
color bars to represent different features.

For the first top figure, the color represents vehicle acceleration where the
color bar is from red to yellow to green and represents -2 to 0 to 2. There is
one extremely red line with starting point at lane 1 and end point at lane 1.
This line is marked by . The rest lines have relatively more yellow or green
colors. For the second top figure, the color represents vehicle speed where
the color bar is from red to yellow to green and represents 40 to 60 to 80.
All lines have a similar pattern to the first top figure but with different
colors. For the third figure, the color represents each vehicle’s original
normalized anomaly score where the color bar is from blue to white to red and
represents 0 to 0.5 to 1. The red line mentioned in the first top figure also
has red color here while the rest lines are all blue. For the fourth figure,
the color represents each vehicle’s reconstructed normalized anomaly score
where the color bar is from blue to white to red and represents 0 to 0.5 to 1.
The red line mentioned in the first top figure also has red color here while
the rest lines are all blue.

Figure 4. Additional qualitative study of real-world HighD traffic data
(Part1)

(a) Vehicle with 5th largest anomaly score. The abnormal vehicle (marked
$\ast$) is having dramatic deceleration.

Four rectangular figures containing vehicle trajectories with x-axis as
distance and y-axis as lane id. There are multiple lines on the figure, each
of which denotes the trajectory of a single vehicle, with a dot denoting the
starting point and a triangle denoting the endpoint. The lines has different
colors to represent different features.

For the first top figure, the color represents vehicle acceleration where the
color bar is from red to yellow to green and represents -2 to 0 to 2. There is
one extremely red line with starting point at lane 2 and end point at lane 2.
This line is marked by . The rest lines have relatively more yellow or green
colors. For the second top figure, the color represents vehicle speed where
the color bar is from red to yellow to green and represents 40 to 60 to 80.
All lines have a similar pattern with the first top figure but with different
colors. For the third figure, the color represents each vehicle’s original
normalized anomaly score where the color bar is from blue to white to red and
represents 0 to 0.5 to 1. The red line mentioned in the first top figure also
has red color here while the rest lines are all blue. For the fourth figure,
the color represents each vehicle’s reconstructed normalized anomaly score
where the color bar is from blue to white to red and represents 0 to 0.5 to 1.
The red line mentioned in the first top figure also has red color here while
the rest lines are all blue.

(b) Vehicle with 6th largest anomaly score. The abnormal vehicle (marked
$\ast$) is having dramatic deceleration, while also driving too slow with
respect to the lane it is in, therefore reconstructed to rightmost lane which
has smaller typical speed.

Four rectangular figures containing vehicle trajectories with x-axis as
distance and y-axis as lane id. There are multiple lines on the figure, each
of which denotes the trajectory of a single vehicle, with a dot denoting the
starting point and a triangle denoting the endpoint. The lines has different
colors to represent different features.

For the first top figure, the color represents vehicle acceleration where the
color bar is from red to yellow to green and represents -2 to 0 to 2. There is
one extremely red line with starting point at lane 2 and end point at lane 2.
This line is marked by . The rest lines have relatively more yellow or green
colors. For the second top figure, the color represents vehicle speed where
the color bar is from red to yellow to green and represents 40 to 60 to 80.
All lines have a similar pattern with the first top figure but with different
colors. For the third figure, the color represents each vehicle’s original
normalized anomaly score where the color bar is from blue to white to red and
represents 0 to 0.5 to 1. The red line mentioned in the first top figure also
has red color here while the rest lines are all blue. For the fourth figure,
the color represents each vehicle’s reconstructed normalized anomaly score
where the color bar is from blue to white to red and represents 0 to 0.5 to 1.
The red line mentioned in the first top figure also has red color here while
the rest lines are all blue.

Figure 5. Additional qualitative study of real-world HighD traffic data (Part
2)