# End-To-End Planning of Autonomous Driving in Industry and Academia:
2022-2023

Gongjin Lan∗ ID , , Qi Hao∗ ID Gongjin Lan and Qi Hao are with the Department
of Computer Science and Engineering, Southern University of Science and
Technology, Shenzhen, 518055, China (e-mail<EMAIL_ADDRESS>
###### Abstract

This paper aims to provide a quick review of the methods including the
technologies in detail that are currently reported in industry and academia.
Specifically, this paper reviews the end-to-end planning, including Tesla FSD
V12, Momenta 2023, Horizon Robotics 2023, Motional RoboTaxi 2022, Woven Planet
(Toyota): Urban Driver, and Nvidia. In addition, we review the state-of-the-
art academic studies that investigate end-to-end planning of autonomous
driving. This paper provides readers with a concise structure and fast
learning of state-of-the-art end-to-end planning for 2022-2023. This article
provides a meaningful overview as introductory material for beginners to
follow the state-of-the-art end-to-end planning of autonomous driving in
industry and academia, as well as supplementary material for advanced
researchers.

###### Index Terms:

Autonomous driving, End-to-End, Motion planning.

##  I Introduction

The World Health Organization claims that the lives of approximately 1.3
million people and an economic cost of $277$ billion are cut short due to road
traffic crashes every year 111https://www.who.int/news-room/fact-
sheets/detail/road-traffic-injuries. The National Highway Transportation
Safety Administration (NHTSA) found that the critical reason for 94% (±2.2%)
of these crashes was assigned to human error [1]. Autonomous vehicles have the
potential to significantly mitigate human errors by eliminating many of the
mistakes that human drivers routinely make [2, 3]. By eliminating human
errors, autonomous driving can improve driver safety, and reduce economic and
congestion problems. Autonomous driving refers to the ability of a vehicle to
operate itself without human intervention.

Autonomous driving is a technology that controls the vehicle to reach the
destination by recognizing the external environment without driver
intervention. Trajectory planning is an essential task of autonomous driving,
which serves as a target value to be tracked by a controller. Predicting the
behavior of road users is a challenging task for a long planning time horizon
[4]. The majority of trajectory planning algorithms are based on the
assumption that other road users behave independently of the generated
trajectory. The predicted trajectories are assumed to be fixed during the
planning process. This motivates interactive trajectory planning, where the
reaction of other road users is taken into account already during the planning
phase. Current challenges are the computational effort and the consideration
of uncertainties in complex driving scenarios. Furthermore, learning
approaches such as reinforcement learning and classification algorithms that
can adapt to the variability of road users are investigated.

This paper aims to provide a quick review of the methods including the
technologies in detail that are currently reported in industry and academia.
Specifically, this paper reviews the end-to-end planning, including Tesla FSD
V12, Momenta 2023, Horizon Robotics 2023, Motional RoboTaxi 2022, Woven Planet
(Toyota): Urban Driver, and Nvidia. In addition, we review the state-of-the-
art academic studies that investigate end-to-end planning of autonomous
driving. The end-to-end planning of autonomous driving in industry and
academia 2022-2023 is summarized in Table I
222https://opendrivelab.com/e2ead/cvpr23
333https://youtu.be/OKDRsVXv49A?si=Y7dYYFXLqPorcxQL
444https://www.youtube.com/watch?v=hFQLJIvdQNU
555https://youtu.be/ZwhXilQKULY?si=nQjgx96PCiXld-uJ
666https://planning.l5kit.org/ 777https://github.com/georgeliu233/OPGP
888https://github.com/autonomousvision/tuplan_garage
999https://youtu.be/OKDRsVXv49A?si=Y7dYYFXLqPorcxQL
101010https://github.com/autonomousvision/tuplan_garage. This paper provides
readers with a concise structure and fast learning of state-of-the-art end-to-
end planning for 2022-2023, which is a meaningful overview as introductory
material for beginners to follow the state-of-the-art end-to-end planning of
autonomous driving in industry and academia, as well as supplementary material
for advanced researchers.

| Solutions | Source code | Publication | Video | Remarks
---|---|---|---|---|---
Industry |  Tesla FSD V12 2023 2 | ✗ | ✗ | 3 |  CVPR2023 E2EAD workshop oral without publication
Momenta 2023 | ✗ | ✗ | 4 |  CVPR2023 AD workshop oral without publication
Horizon Robotics 2023 | ✗ | [5] | 5 |  Honorable Runner-up (2nd) at nuPlan Planning Challenge of CVPR2023 E2EAD workshop
Motional L4-RoboTaxi 2022 | ✗ | [6] | ✗ |  DriveIRL: (remarkable performance) Inverse Reinforcement Learning
Woven Planet (Toyota) 2022: Urban Driver | 6 | [7] | 6 | A well-known baseline
Academia | Occupancy Prediction Planning | 7 | [8] | ✗ |  Occupancy prediction by combining multiple prediction
UniAD 2023 | 8 | [9] | ✗ | Best paper of CVPR 2023
NTU planning | ✗ | [10] | ✗ |  arXiv paper from Nanyang Technological University
nuPlan Planning Challenge CVPR 2023 | 9 | [11] | 10 |  Outstanding champion at nuPlan Planning Challenge CVPR 2023
TABLE I: The summary of end-to-end planning of autonomous driving in industry
and academia 2022-2023.

##  II Industry

In this section, we introduce the current mainstream end-to-end planning of
autonomous driving in the industry 2022-2023, including Tesla, Momenta, and
Nvidia. These end-to-end planning generally have not been publicly introduced
in detail, we aim to summarize their solutions by combining the available
information including oral presentations, videos, and news.

###  II-A Tesla FSD V12 2023

Although Tesla has not published any official research documents on its motion
planning technology, the recent works were presented at workshops and
conferences such as CVPR 2023 Workshop on End-to-End autonomous driving
111111https://opendrivelab.com/e2ead/cvpr23. A presentation video about the is
available at the page 121212https://youtu.be/OKDRsVXv49A?si=Y7dYYFXLqPorcxQL
that is the latest report. While there is limited information about the whole
pipeline in detail, it can be speculated that Tesla uses end-to-end planning.
The available information shows that Tesla uses end-to-end occupancy networks
in BEV space for planning. The occupancy-based planning weakens the dependence
of autonomous driving on high-definition maps. In addition, it shows many
advantages such as better multiple-sensing fusion and better planning in BEV
space. However, Tesla’s planning method generally needs a lot of data. Figure
1 shows the architecture of Tesla’s end-to-end planning method in BEV space.
In summary, there is significantly limited reported information in detail on
Tesla’s end-to-end planning currently.

Figure 1: Tesla’s architecture of end-to-end planning in BEV space that was
presented in CVPR2023 Workshop.

###  II-B Momenta 2023

Momenta is a leading autonomous driving technology company. As the same with
Tesla’s case, there is a lack of public information about Momenta’s planning
method. There are no published official research documents on its motion
planning technology. Momenta presented their work at the CVPR 2023 Workshop on
Autonomous Driving with the title “How Data-Driven Flywheel Enables Scalable
Path to Full Autonomy" 131313https://cvpr2023.wad.vision/. From the
presentation video 141414https://www.youtube.com/watch?v=hFQLJIvdQNU, we
speculate that deep learning-based end-to-end planning is used to design the
trajectory. Unlike Tesla’s occupancy network, Momenta uses online high-
definition map construction to generate real-time maps for planning. Momenta
claimed that they will complete the end-to-end planning and fully end-to-end
autonomous driving by 2025. Figure 2 shows the architecture of Momentat’s end-
to-end planning which is the latest informative piece to date.

Figure 2: Momenta’s architecture of end-to-end planning in BEV space that was
presented in CVPR2023 Workshop.

###  II-C Horizon Robotics 2023

Horizon Robotics presented its planning method in the NuPlan Challenge 2023
[5]. Horizon Robotics adopts a novel spatial-temporal heatmap representation
for planning, along with a corresponding post-solver to ensure a final plan
that is both safe and comfortable. The experimental results demonstrate an
effective balance between the vehicle’s progress and safety, generating safe
and comfortable trajectories. In the NuPlan competition, Horizon Robotics
achieved the second-highest overall score, while obtaining the best scores in
the ego progress and comfort metrics [5]. Figure 3 shows the architecture
Horizon Robotics for end-to-end planning in BEV space. From the publicly
available information, we speculate that Horizon Robotics generates a rough
trajectory and heatmap followed by occupancy and potential field to generate a
final safe planning.

Figure 3: The architecture of horizon Robotics for end-to-end planning in BEV
space [5].

###  II-D Motional L4-RoboTaxi 2022

In 2022, Motional L4-RoboTaxi introduced a learning-based planner (DriveIRL)
to drive a car in dense, urban traffic by using Inverse Reinforcement Learning
(IRL) [6]. DriveIRL architecture where the learned scoring component is shown
in Figure 4. DriveIRL generates a diverse set of trajectory proposals followed
by filtering with a lightweight and interpretable safety filter. A learned
model to score trajectory that is then tracked by the low-level controller
vehicle. DriveIRL is a clear and interpretable method that can be merged with
rules to improve safety and with learning based on improving long-term
planning. To date, the details of DriveIRL are not publicly available but
Motional claimed that it will be made public to help further research. In
summary, the planning of Motional L4-RoboTaxi by using inverse reinforcement
learning is not a fully end-to-end method. There are many inverse
reinforcement learning planning, but DriveIRL is one classic of these methods.
Although DriveIRL is not well-known, it has been applied to the physical
autonomous vehicle with a remarkable performance that outperforms the
intelligent driver model. Currently, some companies also use the same planning
method for L4-RoboTaxi since the method works well with the rule-based method
for solid planning.

Figure 4: DriveIRL architecture where the learned scoring component is
indicated with a dotted boundary [6].

###  II-E Woven Planet (Toyota): Urban Driver

Urban Driver 2022 is a well-known planning method that yields good
generalization and performance for complex, urban driving, which is generally
used as a baseline for comparison. Urban Driver 2022 learns autonomous driving
policy planning in an urban setting by using closed-loop training, mid-level
representations with a data-driven simulator and a large corpus of real-world
demonstrations. Specifically, a closed-loop simulator and rich, mid-level
vectorized representations are developed to learn policy planning capable of
performing a variety of manoeuvres. A new differentiable simulator that
enables efficient closed-loop simulation of realistic driving experiences
based on past demonstrations, and quickly computes policy gradients by
backpropagation through time, allowing fast learning. Policy gradients are
used to learn the state and the mapping function of actions. The trajectory
can be generated with the mapping function followed by the guide of loss
functions for the convergence of a remarkable trajectory. Figure 5 shows the
closed-loop training for learning driving policy planning. However, Woven
Planet Urban Driver was proposed in 2022, which could be a well-known baseline
but not a state-of-the-art method.

Figure 5: The proposed closed-loop training method of Woven Planet (Toyota)
for learning driving policy planning: Urban Driver 2022 for end-to-end
planning [7].

###  II-F Nvidia

####  II-F1 Tree-structured Policy Planning 2023

NVidia proposes a Tree Policy Planning (TPP) that is compatible with state-of-
the-art deep learning prediction models to generate multistage motion planning
[12]. TPP convert the continuous planning problem into a tractable discrete
Markov Decision Process (MDP) through the construction of two tree structures:
an ego trajectory tree for ego trajectory options, and a scenario tree for
multi-modal ego-conditioned environment predictions. Tree-structured sampling
is used to generate conditional prediction layer-by-layer followed by rule-
based constraints for scoring and selecting the planning. Figure 6 shows the
structure of Nvidia tree-structured policy planning. The efficacy of TPP is
demonstrated in closed-loop simulations based on the real-world nuScenes
dataset. The closed-loop simulation result shows that TPP significantly
outperforms the non-policy benchmarks with low computing costs that can be run
in real-time. However, Tree-structured policy planning generally takes a lot
of computation costs when there are a large number of planning nodes.

Figure 6: The structure of Nvidia Tree-structured Policy Planning: the ego-
motion sampler generates the trajectory tree, which is fed to the prediction
model for ego-conditioned prediction to generate the scenario tree [12].

####  II-F2 Differentiable Tree Policy Planning 2023

NVidia and Nanyang Technological University developed Differentiable Tree
Policy Planning (DTPP) [13]. The details are reported in the paper:
Differentiable Joint Conditional Prediction and Cost Evaluation for Tree
Policy Planning in Autonomous Driving. A tree-structured policy planner and a
differentiable joint training framework are proposed for both ego-conditioned
prediction and cost models, resulting in a direct improvement of the final
planning performance. For conditional prediction, DTPP uses a query-centric
transformer model that performs efficient ego-conditioned motion prediction.
For planning cost, DTPP proposes a learnable context-aware cost function with
latent interaction features, facilitating differentiable joint learning. We
validate our proposed approach using the real-world nuPlan It is claimed that
DTPP matches state-of-the-art planning methods and outperforms other learning-
based methods in planning quality. In addition, two points are claimed in DTPP
by:

  1. 1.

Joint training delivers significantly better performance than separate
training of the two modules.

  2. 2.

Tree-structured policy planning outperforms the conventional single-stage
planning approach.

The decision-making framework and the framework with joint learnable
prediction and cost evaluation models of NVidia & NTU DTPP 2023 are shown in
Figure 7 and Figure 8 respectively.

Figure 7: The proposed decision-making framework of NVidia & NTU DTPP 2023. A
tree-structured planner generates a multi-stage trajectory tree to generate a
scenario tree by using the conditional prediction model. The optimal planning
trajectory is selected using the cost evaluation of both trees, and the
conditional prediction and cost evaluation models are jointly learnable. [13].
Figure 8: The DTPP framework with joint learnable prediction and cost
evaluation models. The framework encompasses iterative node pruning and
expansion, guided by ego-conditioned prediction outcomes and cost evaluations.
During training, the loss of cost evaluation can be backpropagated to the
prediction module, enabling differentiable and joint optimization of both
modules. [13].

Specifically, DTPP uses tree search to explore the feasible trajectory sets
where the explored trajectories as the inputs. Conditional prediction is used
to predict the trajectories of all vehicles followed by the scoring module for
evaluation that guides the exploration in the next. In this way, DTPP explores
the various trajectories with rich diversity and interaction with the on-road
vehicles. The method in detail mainly contains three modules:

  * •

Conditional prediction: The conditional prediction generates the predicted
trajectories of all vehicles with the inputs of the ego-vehicles and the other
vehicles.

  * •

Scoring module: Inverse reinforcement learning is used to learn the scoring of
the predicted trajectories.

  * •

Tree Policy Search: Tree policy search is used to explore various candidate
trajectories.

DTPP has many advantages as:

  * •

Conditional prediction provides the gaming effect for planning.

  * •

Derivable in DTPP that returns the gradient and allows prediction and IRL to
be trained together, which is also a necessary condition for end-to-end
autonomous driving.

  * •

Tree policy planning would provide certain interactive deduction capabilities.

  * •

DTPP uses one-time re-encoding and multiple lightweight decodes to effectively
reduce the calculation delay

  * •

DTPP shows state-of-the-art learning-based planning and remarkable closed-loop
planning.

In addition, DTPP has many limitations that need to be improved. Here are some
possible improvements.

  * •

Prediction models can be used to guide the rules for generating the candidate
trajectories.

  * •

Conditional joint prediction can be used to generate predictions interactively
for each trajectory which improves the performance by the gaming effects.

  * •

Conditional joint prediction can be merged with inverse reinforcement learning
for scoring the trajectories and selecting the best one.

##  III Academia

In this section, we review the state-of-the-art academic studies that
investigated end-to-end planning in autonomous driving in the years 2022-2023.

###  III-A Occupancy Prediction Planning

Liu et al. from Nanyang Technological University proposed occupancy prediction
planning in the publication of Occupancy Prediction-Guided Neural Planner for
Autonomous Driving [8] 151515https://github.com/georgeliu233/OPGP. This work
proposed a two-stage occupancy prediction-guided neural planner that refines
earning-based planning through prediction guidance in a joint manner. The
framework of the integrated occupancy prediction and planning learning in OPGP
is shown in Figure 9 Predicted occupancy and multi-modal planning outputs from
an integrated learning-based framework followed by refining the planning from
transformed prediction and planning via prediction-guided optimizations, as
shown in Figure 10. In the first stage, an integrated learning-based framework
with Transformer backbones is designed for comprehensive occupancy predictions
and multi-modal planning objectives. In the second stage, a transformed
occupancy-guided optimization, built upon a curvilinear frame, achieves direct
planning refinement through handcrafted cost function designs. The prediction
and planning performance are extensively validated for enhanced safety and
driving smoothness. The effectiveness of the transformed occupancy prediction
guidance revealed increased scalability in handling undetected and occluded
actors when compared to agent-wise methods.

Figure 9: The framework of the integrated occupancy prediction and planning
learning in OPGP [8]. Figure 10: The illustration of the two-stage occupancy
prediction-guided neural planner [8].

###  III-B UniAD 2023

The CVPR 2023 best paper proposed planning-oriented autonomous driving that
incorporates full-stack driving tasks in one network [9]. A planning-oriented
end-to-end pipeline, UniAD, is proposed for planning. All perception and
prediction modules are designed in a transformer decoder structure, with task
queries as interfaces connecting each node. The pipeline of UniAD is shown in
Figure 11. A query-based design is proposed to connect all nodes in UniAD,
benefiting from richer representations for agent interaction in the
environment. A simple attention-based planner is developed to predict future
waypoints of the ego-vehicle considering the knowledge extracted from
preceding nodes. UniAD is designed by referring to Tesla’s occupancy
prediction network but with semantic segmentation and online mapping. It
trains each module and leverages the benefits of joint optimization from
preceding nodes to final planning in the driving scene. However, UniAD was
validated in the open-loop evaluation and has not been demonstrated in a
closed-loop evaluation.

Figure 11: Pipeline of Unified Autonomous Driving (UniAD) [9].

###  III-C NTU planning

Huang et al. proposed an end-to-end planning method in a paper titled
conditional predictive behavior planning with inverse reinforcement learning
for human-like autonomous driving [10]. This work proposed a learning-based
predictive behavior planning framework that comprises three modules: 1) a
behavior generation module that produces a diverse set of candidate behaviors
in the form of trajectory proposals, 2) a conditional prediction module that
predicts future trajectories of other agents based on each proposal, and 3) a
scoring module that evaluates the candidate plans using maximum entropy
inverse reinforcement learning. This behavior planning framework is shown in
Figure 12. The behavior generation module produces a diverse set of trajectory
proposals, while the conditional motion prediction module forecasts other
agents’ future trajectories jointly conditioned on each candidate plan. The
scoring module evaluates the candidate plans using a cost function learned
with maximum entropy inverse reinforcement learning. The conditional
prediction module can predict multi-modal futures given a candidate plan and
provide reactive predictions to different plans. Specifically, the conditional
motion prediction module takes as input the vectorized map, agent tracks, and
the ego-vehicle plan to predict other agents’ future trajectories. The
structure of the conditional prediction network is shown in Figure 13 where
the ego-vehicle plan is fused into the network to realize that the predictions
of other agents are conditioned on the AV’s planned trajectory. The IRL
scoring module calculates the probability of each behavior based on the
features of joint trajectories of the AV and other agents. The scoring module
with the learned cost function selects plannings that are close to human-
driving ones. The conditional prediction model improves the prediction
accuracy and facilitates the downstream scoring module to better evaluate
candidate decisions, thereby delivering human-like behaviors. Learning the
cost function is crucial in correctly evaluating the candidate plans to align
with human values.

Figure 12: The proposed behavior planning framework [13]. Figure 13: The
structure of the conditional prediction network [13].

###  III-D nuPlan Planning Challenge CVPR 2023

In this subsection, we review the winning method at the nuPlan Planning
Challenge CVPR 2023 Workshop on End-to-End Autonomous Driving
161616https://opendrivelab.com/e2ead/cvpr23.

The release of nuPlan marks a new era in vehicle motion planning research,
offering the first large-scale real-world dataset and evaluation schemes
requiring both precise short-term planning and long-horizon ego-forecasting.
Outstanding champion is titled by parting with misconceptions about learning-
based vehicle motion planning [11]
171717https://www.youtube.com/watch?v=cKIofruqbIk
181818https://github.com/autonomousvision/tuplan_garage. Figure 14 shows the
architecture of the outstanding champion at nuPlan planning challenge CVPR
2023. Prevalent misconceptions are used to learning-based vehicle motion
planning. PDM-closed selects a centerline, forecasts the environment, and
creates varying trajectory proposals, which are simulated and scored for
trajectory selection. The PDM-Hybrid module predicts offsets using the PDM-
Closed centerline, trajectory, and ego history, correcting only long-term
waypoints and limiting the learned model’s influence in closed-loop
simulation. PDM-Hybrid builds upon IDM and combines IDM with a learned ego-
forecasting component. It is an extremely simple and efficient planner which
outperforms an extensive set of competitors, winning the nuPlan planning
challenge CVPR 2023.

Figure 14: Architecture of the outstanding champion at nuPlan Planning
Challenge CVPR 2023 [11].

##  IV Conclusions

This article provides a meaningful overview as introductory material for
beginners to follow the state-of-the-art end-to-end planning of autonomous
driving in industry and academia, as well as supplementary material for
advanced researchers. This paper aims to provide a quick review of the methods
including the technologies in detail that are currently reported in industry
and academia. Specifically, this paper reviews the end-to-end planning,
including Tesla FSD V12, Momenta 2023, Horizon Robotics 2023, Motional
RoboTaxi 2022, Woven Planet (Toyota): Urban Driver, and Nvidia. In addition,
we review the state-of-the-art academic studies that investigate end-to-end
planning of autonomous driving. This paper provides readers with a concise
structure and fast learning of state-of-the-art end-to-end planning for
2022-2023.

## References

  * [1] U. NHTSA, “Critical reasons for crashes investigated in the national motor vehicle crash causation survey,” _DOT HS 812 115. Washington, DC: National Highway Traffic Safety Administration (NHTSA), US Department of Transportation_ , 2015.
  * [2] J. M. Anderson, K. Nidhi, K. D. Stanley, P. Sorensen, C. Samaras, and O. A. Oluwatola, _Autonomous vehicle technology: A guide for policymakers_. Rand Corporation, 2014.
  * [3] D. J. Fagnant and K. Kockelman, “Preparing a nation for autonomous vehicles: opportunities, barriers and policy recommendations,” _Transportation Research Part A: Policy and Practice_ , vol. 77, pp. 167–181, 2015.
  * [4] S. Teng, X. Hu, P. Deng, B. Li, Y. Li, Y. Ai, D. Yang, L. Li, Z. Xuanyuan, F. Zhu _et al._ , “Motion planning for autonomous driving: The state of the art and future perspectives,” _IEEE Transactions on Intelligent Vehicles_ , 2023.
  * [5] Y. Hu, K. Li, P. Liang, J. Qian, Z. Yang, H. Zhang, W. Shao, Z. Ding, W. Xu, and Q. Liu, “Imitation with spatial-temporal heatmap: 2nd place solution for nuplan challenge,” _arXiv preprint arXiv:2306.15700_ , 2023.
  * [6] T. Phan-Minh, F. Howington, T.-S. Chu, S. U. Lee, M. S. Tomov, N. Li, C. Dicle, S. Findler, F. Suarez-Ruiz, R. Beaudoin _et al._ , “Driving in real life with inverse reinforcement learning,” _arXiv preprint arXiv:2206.03004_ , 2022.
  * [7] O. Scheel, L. Bergamini, M. Wolczyk, B. Osiński, and P. Ondruska, “Urban driver: Learning to drive from real-world demonstrations using policy gradients,” in _Conference on Robot Learning_. PMLR, 2022, pp. 718–728.
  * [8] H. Liu, Z. Huang, and C. Lv, “Occupancy prediction-guided neural planner for autonomous driving,” _arXiv preprint arXiv:2305.03303_ , 2023.
  * [9] Y. Hu, J. Yang, L. Chen, K. Li, C. Sima, X. Zhu, S. Chai, S. Du, T. Lin, W. Wang _et al._ , “Planning-oriented autonomous driving,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2023, pp. 17 853–17 862.
  * [10] Z. Huang, H. Liu, J. Wu, and C. Lv, “Conditional predictive behavior planning with inverse reinforcement learning for human-like autonomous driving,” _IEEE Transactions on Intelligent Transportation Systems_ , 2023.
  * [11] D. Dauner, M. Hallgarten, A. Geiger, and K. Chitta, “Parting with misconceptions about learning-based vehicle motion planning,” _arXiv preprint arXiv:2306.07962_ , 2023.
  * [12] Y. Chen, P. Karkus, B. Ivanovic, X. Weng, and M. Pavone, “Tree-structured policy planning with learned behavior models,” _arXiv preprint arXiv:2301.11902_ , 2023.
  * [13] Z. Huang, P. Karkus, B. Ivanovic, Y. Chen, M. Pavone, and C. Lv, “Dtpp: Differentiable joint conditional prediction and cost evaluation for tree policy planning in autonomous driving,” _arXiv preprint arXiv:2310.05885_ , 2023.


* Deeb-Swihart et al. [2022] Deeb-Swihart J, Endert A, Bruckman A.  Ethical tensions in applications of ai for addressing human trafficking: A human rights perspective.  _Proc ACM Hum-Comput Interact_ 2022;6(CSCW2).  URL: https://doi.org/10.1145/3555186. doi:10.1145/3555186.
  * Li et al. [2018a] Li L, Simek O, Lai A, Daggett M, Dagli CK, Jones C.  Detection and characterization of human trafficking networks using unsupervised scalable text template matching.  In: _2018 IEEE International Conference on Big Data (Big Data)_. 2018a:3111–3120.  doi:10.1109/BigData.2018.8622189.
  * Upadhayay et al. [2021] Upadhayay B, Lodhia Z, Behzadan V.  Combating human trafficking via automatic osint collection, validation and fusion.  In: _ICWSM Workshops_. 2021:URL: https://api.semanticscholar.org/CorpusID:237419932.
  * Tong et al. [2017] Tong E, Zadeh A, Jones C, Morency LP.  Combating human trafficking with multimodal deep models.  In: _Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_. Vancouver, Canada: Association for Computational Linguistics; 2017:1547–1556.  URL: https://aclanthology.org/P17-1142. doi:10.18653/v1/P17-1142.
  * Paul Rupa and Gangopadhyay [2020] Paul Rupa A, Gangopadhyay A.  Multi-modal deep learning based fusion approach to detect illicit retail networks from social media.  In: _2020 International Conference on Computational Science and Computational Intelligence (CSCI)_. 2020:238–243.  doi:10.1109/CSCI51800.2020.00047.
  * Mukherjee et al. [2021] Mukherjee S, Sederholm T, Roman AC, Sankar R, Caltagirone S, Ferres JL.  A machine learning pipeline for aiding school identification from child trafficking images.  In: _Proceedings of the Conference on Information Technology for Social Good_. GoodIT ’21; New York, NY, USA: Association for Computing Machinery.  ISBN 9781450384780; 2021:297–300.  URL: https://doi.org/10.1145/3462203.3475924. doi:10.1145/3462203.3475924.
  * Hemadri et al. [2022] Hemadri RV, Singh A, Singh A.  Children safety and rescue (censer) system for trafficked children from brothels in india.  _Proceedings of the AAAI Conference on Artificial Intelligence_ 2022;36(11):11917–11925.  URL: https://ojs.aaai.org/index.php/AAAI/article/view/21449. doi:10.1609/aaai.v36i11.21449.
  * Koudelová et al. [2014] Koudelová J, Dupej J, Jaroslav B, Sedlak P, Velemínská J.  Modelling of facial growth in czech children based on longitudinal data: Age progression from 12 to 15 years using 3d surface models.  _Forensic Science International_ 2014;248.  doi:10.1016/j.forsciint.2014.12.005.
  * Anda et al. [2019] Anda F, Lillis D, Kanta A, Becker BA, Bou-Harb E, Le-Khac NA, Scanlon M.  Improving borderline adulthood facial age estimation through ensemble learning.  In: _Proceedings of the 14th International Conference on Availability, Reliability and Security_. ACM; 2019:URL: https://doi.org/10.1145%2F3339252.3341491. doi:10.1145/3339252.3341491.
  * Swaminathan et al. [2022] Swaminathan A, Ramakrishnan B, M K, R S.  Prediction of cyber-attacks and criminality using machine learning algorithms.  In: _2022 International Conference on Innovation and Intelligence for Informatics, Computing, and Technologies (3ICT)_. 2022:547–552.  doi:10.1109/3ICT56508.2022.9990652.
  * Raets and Janssens [2021] Raets S, Janssens J.  Trafficking and technology: Exploring the role of digital communication technologies in the belgian human trafficking business.  _European Journal on Criminal Policy and Research_ 2021;27:1–24.  doi:10.1007/s10610-019-09429-z.
  * Sethi et al. [2013] Sethi RJ, Gil Y, Jo H, Philpot A.  Large-scale multimedia content analysis using scientific workflows.  In: _Proceedings of the 21st ACM International Conference on Multimedia_. MM ’13; New York, NY, USA: Association for Computing Machinery.  ISBN 9781450324045; 2013:813–822.  URL: https://doi.org/10.1145/2502081.2502082. doi:10.1145/2502081.2502082.
  * Hundman et al. [2018] Hundman K, Gowda T, Kejriwal M, Boecking B.  Always lurking: Understanding and mitigating bias in online human trafficking detection.  In: _Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society_. AIES ’18; New York, NY, USA: Association for Computing Machinery.  ISBN 9781450360128; 2018:137–143.  URL: https://doi.org/10.1145/3278721.3278782. doi:10.1145/3278721.3278782.
  * Stylianou et al. [2017] Stylianou A, Schreier J, Souvenir R, Pless R.  Traffickcam: Crowdsourced and computer vision based approaches to fighting sex trafficking.  In: _2017 IEEE Applied Imagery Pattern Recognition Workshop (AIPR)_. 2017:1–8.  doi:10.1109/AIPR.2017.8457947.
  * Piras and Giacinto [2017] Piras L, Giacinto G.  Information fusion in content based image retrieval: A comprehensive overview.  _Information Fusion_ 2017;37:50–60.  URL: https://www.sciencedirect.com/science/article/pii/S1566253517300076. doi:https://doi.org/10.1016/j.inffus.2017.01.003.
  * Horan and Saiedian [2021] Horan C, Saiedian H.  Cyber crime investigation: Landscape, challenges, and future research directions.  _Journal of Cybersecurity and Privacy_ 2021;1(4):580–596.  URL: https://www.mdpi.com/2624-800X/1/4/29. doi:10.3390/jcp1040029.
  * Karpagam et al. [2022] Karpagam M, Jeyavathana R, Chinnappan S, Kanimozhi K, Sambath M.  A novel face recognition model for fighting against human trafficking in surveillance videos and rescuing victims.  _Soft Computing_ 2022;27:1–16.  doi:10.1007/s00500-022-06931-1.
  * Zavrsnik [2020] Zavrsnik A.  Criminal justice, artificial intelligence systems, and human rights.  _ERA Forum_ 2020;20.  doi:10.1007/s12027-020-00602-0.
  * Iqbal et al. [2020] Iqbal F, Debbabi M, Fung BC.  Machine learning for authorship attribution and cyber forensics.  Springer; 2020.
  * Ivasic-Kos [2022] Ivasic-Kos M.  Application of digital images and corresponding image retrieval paradigm.  _ENTRENOVA - ENTerprise REsearch InNOVAtion_ 2022;8(1):350–363.  URL: https://hrcak.srce.hr/ojs/index.php/entrenova/article/view/23867. doi:10.54820/entrenova-2022-0030.
  * Matasci et al. [2021] Matasci G, Plante J, Kasa K, Mousavi P, Stewart A, Macdonald A, Webster A, Busler J.  Deep learning for vessel detection and identification from spaceborne optical imagery.  _ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences_ 2021;V-3-2021:303–310.  doi:10.5194/isprs-annals-V-3-2021-303-2021.
  * Jiang et al. [2019] Jiang S, Min W, Mei S.  Hierarchy-dependent cross-platform multi-view feature learning for venue category prediction.  _IEEE Transactions on Multimedia_ 2019;21(6):1609–1619.  doi:10.1109/TMM.2018.2876830.
  * Nilwong et al. [2019] Nilwong S, Hossain D, Kaneko Si, Capi G.  Deep learning-based landmark detection for mobile robot outdoor localization.  _Machines_ 2019;7(2).  URL: https://www.mdpi.com/2075-1702/7/2/25. doi:10.3390/machines7020025.
  * Ammatmanee and Gan [2021] Ammatmanee C, Gan L.  Transfer learning for hostel image classification.  _Data Technologies and Applications_ 2021;ahead-of-print.  doi:10.1108/DTA-02-2021-0042.
  * Zhu et al. [2022] Zhu P, Para WR, Frühstück A, Femiani J, Wonka P.  Large-scale architectural asset extraction from panoramic imagery.  _IEEE Transactions on Visualization and Computer Graphics_ 2022;28(2):1301–1316.  doi:10.1109/TVCG.2020.3010694.
  * Grantham et al. [2019] Grantham NS, Reich BJ, Laber EB, Pacifici K, Dunn RR, Fierer N, Gebert M, Allwood JS, Faith SA.  Global forensic geolocation with deep neural networks.  2019\.
  * Salem et al. [2020] Salem T, Workman S, Jacobs N.  Learning a dynamic map of visual appearance.  In: _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_. 2020:12435–12444.
  * Black et al. [2020] Black S, Keshavarz S, Souvenir R.  Evaluation of image inpainting for classification and retrieval.  In: _2020 IEEE Winter Conference on Applications of Computer Vision (WACV)_. 2020:1049–1058.  doi:10.1109/WACV45572.2020.9093362.
  * Smelyakov et al. [2018] Smelyakov K, Sandrkin D, Ruban I, Vitalii M, Romanenkov Y.  Search by image. new search engine service model.  In: _2018 International Scientific-Practical Conference Problems of Infocommunications. Science and Technology (PIC S &T)_. 2018:181–186.  doi:10.1109/INFOCOMMST.2018.8632117.
  * Makantasis et al. [2015] Makantasis K, Protopapadakis E, Doulamis A, Matsatsinis N.  Semi-supervised vision-based maritime surveillance system using fused visual attention maps.  _Multimedia Tools and Applications_ 2015;75.  doi:10.1007/s11042-015-2512-x.
  * Yadav et al. [2019] Yadav N, Goyal A, Meenpal T.  A Feature Averaging Method for Kinship Verification: Proceeding of CISC 2017.  ISBN 978-981-13-0616-7; 2019:381–391.  doi:10.1007/978-981-13-0617-4_38.
  * Wu et al. [2022] Wu Y, Zhang Z, Peng P, Zhao Y, Qin B.  Leveraging multi-modal interactions among the intermediate representations of deep transformers for emotion recognition.  In: _Proceedings of the 3rd International on Multimodal Sentiment Analysis Workshop and Challenge_. MuSe’ 22; New York, NY, USA: Association for Computing Machinery.  ISBN 9781450394840; 2022:101–109.  URL: https://doi.org/10.1145/3551876.3554813. doi:10.1145/3551876.3554813.
  * Liu et al. [2022] Liu F, Li Z, Yang W.  Age-invariant adversarial feature learning for kinship verification.  _Mathematics_ 2022;10.  doi:10.3390/math10030480.
  * Kraus et al. [2020] Kraus M, Pollok T, Miller M, Kilian T, Moritz T, Schweitzer D, Beyerer J, Keim D, Qu C, Jentner W.  Toward Mass Video Data Analysis: Interactive and Immersive 4D Scene Reconstruction.  _Sensors_ 2020;20(18):5426.  doi:10.3390/s20185426.
  * Xiao et al. [2023] Xiao J, Guo H, Zhou J, Zhao T, Yu Q, Chen Y, Wang Z.  Tiny object detection with context enhancement and feature purification.  _Expert Systems with Applications_ 2023;211:118665.  URL: https://www.sciencedirect.com/science/article/pii/S0957417422017031. doi:https://doi.org/10.1016/j.eswa.2022.118665.
  * Yuan [2022] Yuan Y.  Geolocation of images taken indoors using convolutional neural network.  Master’s thesis; Monash University, Melbourne, Australia; 2022\.
  * Karnik et al. [2022b] Karnik T, Shivakumara P, Chowdhury PN, Pal U, Lu T, Anuar NB.  A new deep model for family and non-family photo identification.  _Multimedia Tools and Applications_ 2022b;81(2):1765–1785.  URL: https://doi.org/10.1007/s11042-021-11631-3. doi:10.1007/s11042-021-11631-3.
  * Workman et al. [2022] Workman S, Rafique MU, Blanton H, Jacobs N.  Revisiting near/remote sensing with geospatial attention.  In: _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_. 2022:1778–1787.
  * Shao et al. [2019a] Shao J, Du B, Wu C, Zhang L.  Can we track targets from space? a hybrid kernel correlation filter tracker for satellite video.  _IEEE Transactions on Geoscience and Remote Sensing_ 2019a;57(11):8719–8731.  doi:10.1109/TGRS.2019.2922648.
  * Shao et al. [2021] Shao J, Du B, Wu C, Gong M, Liu T.  Hrsiam: High-resolution siamese network, towards space-borne satellite video tracking.  _IEEE Transactions on Image Processing_ 2021;30:3056–3068.  doi:10.1109/TIP.2020.3045634.
  * Aspandi et al. [2019] Aspandi D, Martinez O, Binefa X.  Heatmap-guided balanced deep convolution networks for family classification in the wild.  In: _2019 14th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2019)_. 2019:1–5.  doi:10.1109/FG.2019.8756557.
  * Deshmukh and Kokate [2022] Deshmukh T, Kokate D.  Human face aging based on deep learning: A survey.  In: _Proceedings of the 3rd International Conference on Contents, Computing & Communication (2022)_. 2022:.
  * Yang et al. [2020] Yang J, Huang Y, Pang J, Xie Z, Li W.  Image privacy protection by particle swarm optimization based pivot pixel modification.  In: Song W, Lee K, Yan Z, Zhang L, Chen H, eds. _Internet of Things - ICIOT 2020 - 5th International Conference, Held as Part of the Services Conference Federation, SCF 2020, Honolulu, HI, USA, September 18-20, 2020, Proceedings_ ; vol. 12405 of _Lecture Notes in Computer Science_. Springer; 2020:1–16.  URL: https://doi.org/10.1007/978-3-030-59615-6_1. doi:10.1007/978-3-030-59615-6\\_1.
  * Kato [1992] Kato T.  Database architecture for content-based image retrieval.  In: _Image Storage and Retrieval Systems_ ; vol. 1662. SPIE; 1992:112–123.
  * Murdock et al. [2016] Murdock V, Clarke CL, Kamps J, Karlgren J.  Second workshop on search and exploration of x-rated information (sexi’16): Wsdm workshop summary.  In: _Proceedings of the Ninth ACM International Conference on Web Search and Data Mining_. WSDM ’16; New York, NY, USA: Association for Computing Machinery.  ISBN 9781450337168; 2016:697–698.  URL: https://doi.org/10.1145/2835776.2855118. doi:10.1145/2835776.2855118.
  * Lallie and Chen [2013] Lallie HS, Chen TY.  Geotract: Issues in pattern matching geospatial data on digital storage systems.  In: _3rd International Conference on Cybercrime, Security and Digital Forensics, University of Cardiff, Cardiff, UK_. 2013:.
  * Li et al. [2018b] Li X, Qu X, Xu W, Wang S, Tong Y, Luo L.  Validating the contextual information of outdoor images for photo misuse detection.  2018b.
  * Kakar and Sudha [2012] Kakar P, Sudha N.  Authenticating image metadata elements using geolocation information and sun direction estimation.  In: _2012 IEEE International Conference on Multimedia and Expo_. 2012:236–241.  doi:10.1109/ICME.2012.82.
  * Worring et al. [2016] Worring M, Koelma D, Zahalka J.  Multimedia pivot tables for multimedia analytics on image collections.  _IEEE Transactions on Multimedia_ 2016;18(11):2217–2227.  URL: https://doi.org/10.1109/TMM.2016.2614380. doi:10.1109/TMM.2016.2614380.
  * Stylianou et al. [2015] Stylianou A, Norling-Ruggles A, Souvenir R, Pless R.  Indexing open imagery to create tools to fight sex trafficking.  In: _2015 IEEE Applied Imagery Pattern Recognition Workshop (AIPR)_. 2015:1–6.  doi:10.1109/AIPR.2015.7444535.
  * Choi et al. [2022] Choi J, Wong CW, Hajj-Ahmad A, Wu M, Ren Y.  Invisible geo-location signature in a single image.  _IEEE Transactions on Information Forensics and Security_ 2022;17:2598–2613.  doi:10.1109/TIFS.2022.3185775.
  * Zhang et al. [2018] Zhang X, Wang S, Li Z, Ma S.  Landmark image retrieval by jointing feature refinement and multimodal classifier learning.  _IEEE Transactions on Cybernetics_ 2018;48(6):1682–1695.  doi:10.1109/TCYB.2017.2712798.
  * Salem et al. [2022] Salem T, Hwang J, Padilha R.  Timestamp estimation from outdoor scenes.  In: _Annual ADFSL Conference on Digital Forensics, Security and Law_. 2022:.
  * Peersman et al. [2016] Peersman C, Schulze C, Rashid A, Brennan M, Fischer C.  icop: Live forensics to reveal previously unknown criminal media on p2p networks.  _Digital Investigation_ 2016;18:50–64.  URL: https://www.sciencedirect.com/science/article/pii/S1742287616300779. doi:https://doi.org/10.1016/j.diin.2016.07.002.
  * Padilha et al. [2022] Padilha R, Salem T, Workman S, Andaló FA, Rocha A, Jacobs N.  Content-aware detection of temporal metadata manipulation.  _IEEE Transactions on Information Forensics and Security_ 2022;17:1316–1327.  doi:10.1109/TIFS.2022.3159154.
  * Li et al. [2017] Li X, Xu W, Wang S, Qu X.  Are you lying: Validating the time-location of outdoor images.  In: Gollmann D, Miyaji A, Kikuchi H, eds. _Applied Cryptography and Network Security_. Cham: Springer International Publishing.  ISBN 978-3-319-61204-1; 2017:103–123.  doi:10.1007/978-3-319-61204-1_6.
  * Mattmann and Sharan [2017] Mattmann CA, Sharan M.  Scalable hadoop-based pooled time series of big video data from the deep web.  In: _Proceedings of the 2017 ACM on International Conference on Multimedia Retrieval_. ICMR ’17; New York, NY, USA: Association for Computing Machinery.  ISBN 9781450347013; 2017:117–120.  URL: https://doi.org/10.1145/3078971.3079019. doi:10.1145/3078971.3079019.
  * Bessinger and Jacobs [2019] Bessinger Z, Jacobs N.  A generative model of worldwide facial appearance.  In: _2019 IEEE Winter Conference on Applications of Computer Vision (WACV)_. IEEE; 2019:1569–1578.
  * Katsaounidou et al. [2020] Katsaounidou AN, Gardikiotis A, Tsipas N, Dimoulas CA.  News authentication and tampered images: evaluating the photo-truth impact through image verification algorithms.  _Heliyon_ 2020;6(12):e05808.  URL: https://www.sciencedirect.com/science/article/pii/S2405844020326517. doi:https://doi.org/10.1016/j.heliyon.2020.e05808.
  * Ma et al. [2018] Ma X, Zhao Y, Qian X, Tang YY.  Multi-source fusion based geo-tagging for web images.  _Multimedia Tools and Applications_ 2018;77:16399–16417.  doi:https://doi.org/10.1007/s11042-017-5211-y.
  * Mookdarsanit and Rattanasiriwongwut [2017] Mookdarsanit P, Rattanasiriwongwut M.  Location estimation of a photo: A geo-signature mapreduce workflow.  _Engineering Journal_ 2017;21:295–308.  doi:10.4186/EJ.2017.21.3.295.
  * Kuznetsov and Myasnikov [2015] Kuznetsov A, Myasnikov V.  New algorithms for satellite data verification with and without the use of the imaged area vector data.  In: _WSCG 2015 Conference on Computer Graphics, Visualization and Computer Vision_. Václav Skala-UNION Agency; 2015:.
  * Li et al. [2021] Li L, Han L, Ding M, Cao H, Hu H.  A deep learning semantic template matching framework for remote sensing image registration.  _ISPRS Journal of Photogrammetry and Remote Sensing_ 2021;181:205–217.  URL: https://www.sciencedirect.com/science/article/pii/S0924271621002446. doi:https://doi.org/10.1016/j.isprsjprs.2021.09.012.
  * Lefèvre et al. [2017] Lefèvre S, Tuia D, Wegner JD, Produit T, Nassar AS.  Toward seamless multiview scene analysis from satellite to street level.  _Proceedings of the IEEE_ 2017;105(10):1884–1899.  doi:10.1109/JPROC.2017.2684300.
  * Ilčev [2019] Ilčev S.  Global Satellite Meteorological Observation (GSMO) Applications: Volume 2.  2019\.  ISBN 978-3-319-67046-1.  doi:10.1007/978-3-319-67047-8.
  * Shao et al. [2019b] Shao J, Du B, Wu C, Zhang L.  Tracking objects from satellite videos: A velocity feature based correlation filter.  _IEEE Transactions on Geoscience and Remote Sensing_ 2019b;57(10):7860–7871.  doi:10.1109/TGRS.2019.2916953.
  * Roca et al. [2016] Roca D, Martínez-Sánchez J, Lagüela S, Arias P.  Novel aerial 3d mapping system based on uav platforms and 2d laser scanners.  _Journal of Sensors_ 2016;2016:1–8.  doi:10.1155/2016/4158370.
  * Sharifzadeh et al. [2014] Sharifzadeh HR, Zhang H, Kong AWK.  Vein pattern visualization through multiple mapping models and local parameter estimation for forensic investigation.  In: _2014 22nd International Conference on Pattern Recognition_. 2014:160–165.  doi:10.1109/ICPR.2014.37.
  * Zhang et al. [2017] Zhang C, Ré C, Cafarella M, De Sa C, Ratner A, Shin J, Wang F, Wu S.  Deepdive: Declarative knowledge base construction.  _Commun ACM_ 2017;60(5):93–102.  URL: https://doi.org/10.1145/3060586. doi:10.1145/3060586.
  * Al-Kuwari and Wolthusen [2010] Al-Kuwari S, Wolthusen SD.  Probabilistic vehicular trace reconstruction based on rf-visual data fusion.  In: De Decker B, Schaumüller-Bichl I, eds. _Communications and Multimedia Security_. Berlin, Heidelberg: Springer Berlin Heidelberg.  ISBN 978-3-642-13241-4; 2010:16–27.
  * Tassone et al. [2017] Tassone C, Martini B, Choo KK.  Chapter 11 - forensic visualization: Survey and future research directions.  In: Choo KKR, Dehghantanha A, eds. _Contemporary Digital Forensic Investigations of Cloud and Mobile Applications_. Syngress.  ISBN 978-0-12-805303-4; 2017:163–184.  URL: https://www.sciencedirect.com/science/article/pii/B9780128053034000113. doi:https://doi.org/10.1016/B978-0-12-805303-4.00011-3.
  * Anda et al. [2020] Anda F, Becker BA, Lillis D, Le-Khac NA, Scanlon M.  Assessing the influencing factors on the accuracy of underage facial age estimation.  In: _2020 International Conference on Cyber Security and Protection of Digital Services (Cyber Security)_. 2020:1–8.  doi:10.1109/CyberSecurity49315.2020.9138851.
  * Sukumar [2015a] Sukumar SR.  Open research challenges with big data — a data-scientist’s perspective.  In: _2015 IEEE International Conference on Big Data (Big Data)_. 2015a:1272–1278.  doi:10.1109/BigData.2015.7363882.
  * Kamath et al. [2021] Kamath R, Rolwes G, Black S, Stylianou A.  The 2021 Hotel-ID to Combat Human Trafficking Competition Dataset.  2021\.
  * Ramchandani et al. [2021] Ramchandani P, Bastani H, Wyatt E.  Unmasking human trafficking risk in commercial sex supply chains with machine learning.  _SSRN Electronic Journal_ 2021;doi:10.2139/ssrn.3866259.
  * Varastehpour et al. [2019a] Varastehpour S, Sharifzadeh H, Ardekani I, Francis X.  Vein pattern visualisation and feature extraction using sparse auto-encoder for forensic purposes.  In: _2019 16th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS)_. 2019a:1–8.  doi:10.1109/AVSS.2019.8909860.
  * Alruwaili [2021] Alruwaili FF.  Custodyblock: A distributed chain of custody evidence framework.  _Information_ 2021;12(2):88.  URL: http://dx.doi.org/10.3390/info12020088. doi:10.3390/info12020088.
  * Salamat et al. [2018] Salamat MAA, Reyes SSG, Delfin EB, Rabago LR, Valguna RHG, Boydon EC.  Citesranger: Illegal wildlife trade detection tool.  2018\.
  * Granizo et al. [2020] Granizo SL, Valdivieso Caraguay AL, Barona López LI, Hernández-Àlvarez M.  Detection of possible illicit messages using natural language processing and computer vision on twitter and linked websites.  _IEEE Access_ 2020;8:44534–44546.  doi:10.1109/ACCESS.2020.2976530.
  * Galbraith et al. [2020] Galbraith C, Smyth P, Stern HS.  Statistical methods for the forensic analysis of geolocated event data.  _Forensic Science International: Digital Investigation_ 2020;33:301009.  URL: https://www.sciencedirect.com/science/article/pii/S2666281720302584. doi:https://doi.org/10.1016/j.fsidi.2020.301009.
  * Casino et al. [2022] Casino F, Pina C, López-Aguilar P, Batista E, Solanas A, Patsakis C.  SoK: Cross-border criminal investigations and digital evidence.  _Journal of Cybersecurity_ 2022;8(1):tyac014.  URL: https://doi.org/10.1093/cybsec/tyac014. doi:10.1093/cybsec/tyac014.
  * Böhme et al. [2016] Böhme R, Kirchner M, Katzenbeisser S, Petitcolas F.  Media forensics.  In: _Information Hiding_. 9; Artech House; 2016:231–259.
  * Srivastava et al. [2017] Srivastava A, Jain P, Demetriou S, Cox LP, Kim KH.  Camforensics: Understanding visual privacy leaks in the wild.  In: _Proceedings of the 15th ACM Conference on Embedded Network Sensor Systems_. SenSys ’17; New York, NY, USA: Association for Computing Machinery.  ISBN 9781450354592; 2017:URL: https://doi.org/10.1145/3131672.3131683. doi:10.1145/3131672.3131683.
  * Varastehpour et al. [2019b] Varastehpour S, Sharifzadeh H, Ardekani I, Francis X.  Vein pattern visualisation and feature extraction using sparse auto-encoder for forensic purposes.  In: _2019 16th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS)_. 2019b:1–8.  doi:10.1109/AVSS.2019.8909860.
  * Keivanmarz et al. [2020] Keivanmarz A, Sharifzadeh H, Fleming R.  Vein pattern visualisation using conditional generative adversarial networks.  In: _2020 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC)_. 2020:1310–1316.
  * Chhoriya [2019] Chhoriya P.  Automated criminal identification system using face detection and recognition.  _International Research Journal of Engineering and Technology (IRJET)_ 2019;6(10):910–914.
  * Sukumar [2015b] Sukumar SR.  Open research challenges with big data — a data-scientist’s perspective.  In: _2015 IEEE International Conference on Big Data (Big Data)_. 2015b:1272–1278.  doi:10.1109/BigData.2015.7363882.
  * Vajiac et al. [2023] Vajiac C, Chau DH, Olligschlaeger A, Mackenzie R, Nair P, Lee MC, Li Y, Park N, Rabbany R, Faloutsos C.  Trafficvis: Visualizing organized activity and spatio-temporal patterns for detecting and labeling human trafficking.  _IEEE Transactions on Visualization and Computer Graphics_ 2023;29(1):53–62.  doi:10.1109/TVCG.2022.3209403.
  * Gowda et al. [2017] Gowda T, Hundman K, Mattmann CA.  An approach for automatic and large scale image forensics.  In: _Proceedings of the 2nd International Workshop on Multimedia Forensics and Security_. MFSec ’17; New York, NY, USA: Association for Computing Machinery.  ISBN 9781450350341; 2017:16–20.  URL: https://doi.org/10.1145/3078897.3080536. doi:10.1145/3078897.3080536.
  * Kejriwal and Szekely [2018] Kejriwal M, Szekely P.  Technology-assisted investigative search: A case study from an illicit domain.  In: _Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems_. CHI EA ’18; New York, NY, USA: Association for Computing Machinery.  ISBN 9781450356213; 2018:1–9.  URL: https://doi.org/10.1145/3170427.3174364. doi:10.1145/3170427.3174364.
  * Parolin et al. [2021] Parolin ES, Khan L, Osorio J, Brandt PT, D’Orazio V, Holmes J.  3m-transformers for event coding on organized crime domain.  In: _2021 IEEE 8th International Conference on Data Science and Advanced Analytics (DSAA)_. 2021:1–10.  doi:10.1109/DSAA53316.2021.9564232.
  * Robinson et al. [2021] Robinson JP, Qin C, Shao M, Turk MA, Chellappa R, Fu Y.  The 5th recognizing families in the wild data challenge: Predicting kinship from faces.  In: _2021 16th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2021)_. Los Alamitos, CA, USA: IEEE Computer Society; 2021:1–7.  URL: https://doi.ieeecomputersociety.org/10.1109/FG52635.2021.9666990. doi:10.1109/FG52635.2021.9666990.
  * Serraoui et al. [2022] Serraoui I, Laiadi O, Ouamane A, Dornaika F, Taleb-Ahmed A.  Knowledge-based tensor subspace analysis system for kinship verification.  _Neural Networks_ 2022;151:222–237.  URL: https://www.sciencedirect.com/science/article/pii/S0893608022000983. doi:https://doi.org/10.1016/j.neunet.2022.03.020.
  * Wang et al. [2019] Wang S, Ding Z, Fu Y.  Cross-generation kinship verification with sparse discriminative metric.  _IEEE Transactions on Pattern Analysis & Machine Intelligence_ 2019;41(11):2783–2790.  doi:10.1109/TPAMI.2018.2861871.
  * Goyal et al. [2022] Goyal A, Meenpal T, Mukherjee M.  Family classification and kinship verification from facial images in the wild.  _Machine Vision and Applications_ 2022;33.  doi:10.1007/s00138-022-01341-7.
  * Yan and Song [2021] Yan H, Song C.  Multi-scale deep relational reasoning for facial kinship verification.  _Pattern Recognition_ 2021;110:107541.  URL: https://www.sciencedirect.com/science/article/pii/S0031320320303447. doi:https://doi.org/10.1016/j.patcog.2020.107541.
  * Chen et al. [2021] Chen X, Zhu X, Zheng S, Zheng T, Zhang F.  Semi-coupled synthesis and analysis dictionary pair learning for kinship verification.  _IEEE Transactions on Circuits and Systems for Video Technology_ 2021;31(5):1939–1952.  doi:10.1109/TCSVT.2020.3017683.
  * Chandaliya and Nain [2019] Chandaliya PK, Nain N.  Conditional perceptual adversarial variational autoencoder for age progression and regression on child face.  In: _2019 International Conference on Biometrics (ICB)_. 2019:1–8.  doi:10.1109/ICB45273.2019.8987410.
  * Robinson et al. [2018a] Robinson J, Shao M, Wu Y, Liu H, Gillis T, Fu Y.  Visual kinship recognition of families in the wild.  _IEEE Transactions on Pattern Analysis and Machine Intelligence_ 2018a;PP:1–1.  doi:10.1109/TPAMI.2018.2826549.
  * Robinson et al. [2018b] Robinson JP, Shao M, Fu Y.  To recognize families in the wild: A machine vision tutorial.  In: _Proceedings of the 26th ACM International Conference on Multimedia_. MM ’18; New York, NY, USA: Association for Computing Machinery.  ISBN 9781450356657; 2018b:2096–2097.  URL: https://doi.org/10.1145/3240508.3241471. doi:10.1145/3240508.3241471.
  * Goyal and Meenpal [2021] Goyal A, Meenpal T.  Patch-based dual-tree complex wavelet transform for kinship recognition.  _IEEE Transactions on Image Processing_ 2021;30:191–206.  doi:10.1109/TIP.2020.3034027.
  * Abbas and Shoaib [2022] Abbas A, Shoaib M.  Kinship identification using age transformation and siamese network.  _PeerJ Computer Science_ 2022;8:e987.
  * Stylianou et al. [2019] Stylianou A, Xuan H, Shende M, Brandt J, Souvenir R, Pless R.  Hotels-50k: A global hotel recognition dataset.  _Proceedings of the AAAI Conference on Artificial Intelligence_ 2019;33:726–733.  doi:10.1609/aaai.v33i01.3301726.
  * Anda et al. [2018] Anda F, Lillis D, Le-Khac NA, Scanlon M.  Evaluating automated facial age estimation techniques for digital forensics.  In: _2018 IEEE Security and Privacy Workshops (SPW)_. 2018:129–139.  doi:10.1109/SPW.2018.00028.
  * Vyawahare et al. [2022] Vyawahare R, Khanuja DHK, Agarkar DAA.  Synthetic face aging using cyclegan.  _Available at SSRN 4172218_ 2022;.
  * Du et al. [2020] Du X, Hargreaves C, Sheppard J, Anda F, Sayakkara A, Le-Khac NA, Scanlon M.  Sok: exploring the state of the art and the future potential of artificial intelligence in digital forensic investigation.  In: _Proceedings of the 15th International Conference on Availability, Reliability and Security_. ARES ’20; New York, NY, USA: Association for Computing Machinery.  ISBN 9781450388337; 2020:URL: https://doi.org/10.1145/3407023.3407068. doi:10.1145/3407023.3407068.
  * Mohamadi et al. [2021] Mohamadi S, Nasrabadi NM, Doretto G, Adjeroh DA.  Human age estimation from gene expression data using artificial neural networks.  In: _2021 IEEE International Conference on Bioinformatics and Biomedicine (BIBM)_. Los Alamitos, CA, USA: IEEE Computer Society; 2021:3492–3497.  URL: https://doi.ieeecomputersociety.org/10.1109/BIBM52615.2021.9669893. doi:10.1109/BIBM52615.2021.9669893.
  * Chandaliya and Nain [2021] Chandaliya PK, Nain N.  Child face age progression and regression using self-attention multi-scale patch gan.  In: _2021 IEEE International Joint Conference on Biometrics (IJCB)_. 2021:1–8.  doi:10.1109/IJCB52358.2021.9484329.

##  Appendix A Classification of the Systematically Reviewed Literature
(Electronic Supplement)

Table 1: Listing and Classification of Included Literature Legend
---
| High Relevancy/Applicability
| Medium Relevancy/Applicability
[regular] | Low Relevancy/Applicability
- | No Relevancy/Applicability or None
| Retrieval
| Classification
| Combined
| Image
| Video
| Audio
| Text
| Multimodal
| Outdoor
| Indoor
Reference |  Computer Vision |  Computer Vision Geolocation Technique |  Multimedia Geolocation |  Multimedia Data Type |  Digital Investigation |  Law Enforcement |  Human Trafficking |  Hotel Room Identification |  Indoor/Outdoor
---|---|---|---|---|---|---|---|---|---
Glistrup et al. [3] |  |  |  [regular] |  - |  - |  - |  ✓ |  |
Sethi et al. [30] |  |  - |  [regular] |  |  [regular] |  [regular] |  |  - |  -
Sethi et al. [30] |  |  - |  [regular] |  |  [regular] |  [regular] |  |  - |  -
Glistrup et al. [3] |  |  |  |  |  |  |  [regular] |  |
Stylianou et al. [68] |  |  |  |  |  |  |  |  |
Stylianou et al. [32] |  |  |  |  |  |  |  |  |
Yuan [54] |  |  |  |  |  |  |  |  |
Aronson [5] |  |  - |  |  |  |  |  [regular] |  - |
Salem et al. [45] |  |  - |  |  |  |  |  [regular] |  - |
Ivanov et al. [9] |  |  - |  |  |  |  [regular] |  - |  - |
Bessinger and Jacobs [76] |  |  - |  |  |  |  [regular] |  - |  - |
Li et al. [65] |  [regular] |  - |  |  |  |  - |  - |  - |
Kakar and Sudha [66] |  [regular] |  - |  |  |  |  - |  - |  - |
Salem et al. [71] |  |  - |  |  |  |  [regular] |  - |  - |
Mookdarsanit and Rattanasiriwongwut [79] |  [regular] |  - |  |  |  |  [regular] |  - |  - |
Friedland et al. [8] |  |  |  |  |  |  |  - |  - |
Padilha et al. [73] |  |  - |  |  |  |  [regular] |  - |  - |
Ivasic-Kos [38] |  |  |  [regular] |  |  |  |  [regular] |  - |
Moreira et al. [10] |  [regular] |  - |  [regular] |  |  |  [regular] |  - |  - |
Zhang et al. [70] |  |  |  |  |  [regular] |  - |  - |  - |
Li et al. [74] |  [regular] |  - |  |  |  |  |  - |  - |
Vo et al. [6] |  |  |  |  |  |  |  [regular] |  [regular] |
Choi et al. [69] |  [regular] |  - |  |  |  |  [regular] |  - |  - |
Zhu et al. [43] |  |  |  |  |  |  [regular] |  - |  - |
Ma et al. [78] |  |  |  |  |  |  [regular] |  - |  - |
Piras and Giacinto [33] |  |  |  [regular] |  |  |  - |  - |  - |  -
Yokota et al. [4] |  |  |  |  |  |  |  - |  - |
Al-Kuwari and Wolthusen [88] |  [regular] |  - |  |  |  |  |  - |  - |
Jaeyoung Choi [7] |  |  |  |  |  |  |  - |  - |
Grantham et al. [44] |  |  |  |  |  |  |  [regular] |  - |  -
Peersman et al. [72] |  [regular] |  |  |  |  |  |  [regular] |  - |
Lallie and Chen [64] |  [regular] |  - |  |  |  |  |  |  - |
Lefèvre et al. [82] |  |  |  |  |  |  [regular] |  - |  - |
Katsaounidou et al. [77] |  [regular] |  - |  [regular] |  |  |  |  - |  - |  -
Roca et al. [85] |  - |  - |  [regular] |  |  [regular] |  [regular] |  - |  - |
Worring et al. [67] |  [regular] |  - |  |  |  [regular] |  [regular] |  - |  - |
Ilčev [83] |  |  - |  |  |  |  [regular] |  - |  - |
Li et al. [81] |  |  - |  [regular] |  |  [regular] |  - |  - |  - |
Jiang et al. [40] |  |  |  |  |  |  [regular] |  - |  - |
Zerdoumi et al. [12] |  |  - |  |  |  |  [regular] |  - |  - |
Keerthi Chandra et al. [13] |  |  |  [regular] |  |  |  |  - |  - |
Kraus et al. [52] |  |  - |  [regular] |  |  |  |  - |  - |
Nilwong et al. [41] |  |  |  |  |  |  - |  - |  - |
Workman et al. [56] |  |  - |  |  |  |  |  - |  - |
Kuznetsov and Myasnikov [80] |  |  - |  |  |  |  |  - |  - |
Smelyakov et al. [47] |  |  |  - |  |  - |  - |  - |  - |
Iqbal et al. [37] |  |  |  [regular] |  |  |  |  - |  - |  -
Makantasis et al. [48] |  |  |  |  |  |  |  - |  - |
Aspandi et al. [59] |  |  |  [regular] |  |  |  |  |  - |
Deshmukh and Kokate [60] |  |  - |  [regular] |  |  [regular] |  |  - |  - |  -
Xiao et al. [53] |  |  - |  [regular] |  |  |  |  |  - |
Wu et al. [50] |  |  - |  - |  |  [regular] |  - |  - |  - |  -
Ammatmanee and Gan [42] |  |  |  |  |  |  |  |  |
Shao et al. [58] |  [regular] |  - |  |  |  [regular] |  |  [regular] |  - |
Shao et al. [57] |  |  - |  |  |  |  |  [regular] |  - |
Liu et al. [51] |  |  - |  [regular] |  |  |  |  |  - |  -
Karnik et al. [55] |  |  - |  [regular] |  |  |  |  |  - |  -
Yang et al. [61] |  |  - |  [regular] |  |  |  [regular] |  [regular] |  - |  -
Dimas et al. [1] |  - |  - |  [regular] |  - |  |  |  |  [regular] |  -
Hundman et al. [31] |  |  |  [regular] |  |  |  |  |  - |
Hemadri et al. [25] |  |  |  |  |  |  |  |  - |  -
Raets and Janssens [29] |  [regular] |  - |  |  - |  |  |  |  - |  -
Anda et al. [27] |  |  - |  [regular] |  |  |  |  |  - |  -
Serraoui et al. [111] |  |  - |  [regular] |  |  |  |  |  - |  -
Wang et al. [112] |  |  - |  [regular] |  |  |  |  |  - |  -
Chandaliya and Nain [116] |  |  - |  [regular] |  |  |  |  |  - |  -
Upadhayay et al. [21] |  [regular] |  - |  [regular] |  |  |  |  |  - |  -
Tong et al. [22] |  |  |  [regular] |  |  |  |  |  - |  -
Goyal et al. [113] |  |  |  [regular] |  |  |  |  |  - |  -
Goh et al. [14] |  - |  - |  - |  |  |  |  |  - |  -
Mukherjee et al. [24] |  |  |  |  |  |  |  |  - |
Sierra-Rodríguez et al. [18] |  [regular] |  - |  - |  |  |  [regular] |  |  - |  -
Matasci et al. [39] |  |  |  |  |  |  |  |  - |
Abbas and Shoaib [120] |  |  |  [regular] |  |  |  |  |  - |  -
Karpagam et al. [35] |  |  |  |  |  |  |  |  - |  -
Karnik et al. [15] |  |  |  [regular] |  |  |  |  |  - |  -
Yan and Song [114] |  |  |  - |  |  |  |  |  - |  -
Kshetri [2] |  - |  - |  - |  - |  [regular] |  [regular] |  [regular] |  - |  -
Koudelová et al. [26] |  |  - |  |  |  |  |  |  - |  -
Paul Rupa and Gangopadhyay [23] |  |  |  [regular] |  |  |  |  |  - |  -
Sukumar [91] |  |  - |  |  |  |  |  |  - |
Goyal and Meenpal [119] |  |  - |  [regular] |  |  [regular] |  [regular] |  |  - |  -
Murdock et al. [63] |  [regular] |  - |  |  |  |  |  |  - |  -
Kamath et al. [92] |  [regular] |  |  |  |  |  [regular] |  |  |
Mattmann and Sharan [75] |  [regular] |  - |  [regular] |  |  |  |  |  - |
Chen et al. [115] |  |  - |  [regular] |  |  |  [regular] |  [regular] |  - |  -
Casino et al. [99] |  - |  - |  - |  - |  |  |  |  - |  -
Vajiac et al. [106] |  [regular] |  - |  |  |  |  |  |  - |
Ramchandani et al. [93] |  [regular] |  - |  |  |  |  |  |  - |  -
Robinson et al. [117] |  |  |  |  |  |  |  |  - |  -
Robinson et al. [118] |  |  |  |  |  |  |  |  - |  -
Shao et al. [84] |  |  - |  |  |  |  [regular] |  [regular] |  - |
Yadav et al. [49] |  |  |  [regular] |  |  |  [regular] |  [regular] |  - |  -
Dasgupta et al. [16] |  |  |  [regular] |  |  |  |  |  - |
Anda et al. [90] |  |  - |  [regular] |  |  |  |  |  - |  -
Chandaliya and Nain [126] |  |  - |  [regular] |  |  |  |  |  - |  -
Salamat et al. [96] |  |  |  [regular] |  |  |  |  |  - |
Vinavatani et al. [17] |  [regular] |  |  |  |  |  |  |  - |  -
Li et al. [20] |  |  - |  |  |  |  |  |  - |  -
Granizo et al. [97] |  |  - |  [regular] |  |  |  |  |  - |  -
Deeb-Swihart et al. [19] |  |  - |  - |  - |  |  |  |  - |  -
Zhang et al. [87] |  - |  - |  - |  - |  [regular] |  [regular] |  |  - |  -
Mohamadi et al. [125] |  |  |  - |  |  |  [regular] |  |  - |  -
Swaminathan et al. [28] |  |  |  - |  |  |  |  [regular] |  - |  -
Vyawahare et al. [123] |  |  - |  - |  |  [regular] |  [regular] |  [regular] |  - |  -
Varastehpour et al. [94] |  |  - |  - |  |  |  |  [regular] |  - |  -
Chhoriya [104] |  |  - |  [regular] |  |  |  [regular] |  [regular] |  - |  -
Sukumar [105] |  [regular] |  - |  - |  - |  |  [regular] |  [regular] |  - |  -
Kejriwal and Szekely [108] |  - |  - |  [regular] |  - |  |  |  |  - |  -
Parolin et al. [109] |  |  - |  [regular] |  - |  |  |  |  - |  -
Zavrsnik [36] |  |  - |  - |  - |  |  |  |  - |  -
Horan and Saiedian [34] |  [regular] |  - |  - |  - |  |  |  |  - |  -
Tassone et al. [89] |  - |  - |  - |  - |  |  - |  [regular] |  - |  -
Alruwaili [95] |  - |  - |  - |  - |  |  [regular] |  - |  - |  -
Böhme et al. [100] |  [regular] |  - |  [regular] |  |  |  [regular] |  [regular] |  - |  -
Galbraith et al. [98] |  [regular] |  - |  |  |  |  |  |  - |
Srivastava et al. [101] |  [regular] |  - |  [regular] |  |  |  [regular] |  - |  - |
Sharifzadeh et al. [86] |  |  - |  - |  |  |  |  [regular] |  - |  -
Varastehpour et al. [102] |  |  - |  - |  |  |  |  [regular] |  - |  -
Keivanmarz et al. [103] |  |  - |  - |  |  |  |  [regular] |  - |  -
Robinson et al. [110] |  |  - |  [regular] |  |  |  |  |  - |  -
Gowda et al. [107] |  |  |  [regular] |  |  |  |  |  - |
Anda et al. [122] |  [regular] |  - |  [regular] |  |  |  |  [regular] |  - |  -
Black et al. [46] |  |  - |  |  |  |  [regular] |  [regular] |  - |
Stylianou et al. [121] |  |  |  |  |  |  |  |  |