Cross-Camera View-Overlap Recognition

ECCV-22 submission ID
ECCV-22 submission ID
Anonymous ECCV submission
Paper ID

Cross-Camera View-Overlap Recognition

Alessio Xompero0000-0002-8227-8529 Andrea Cavallaro0000-0001-5086-7858

A. Xompero and A. Cavallaro

Centre for Intelligent Sensing, Queen Mary University of London, London, U.K.

We propose a decentralised view-overlap recognition framework that operates across freely moving cameras without the need of a reference 3D map. Each camera independently extracts, aggregates into a hierarchical structure, and shares feature-point descriptors over time. A view overlap is recognised by view-matching and geometric validation to discard wrongly matched views.
The proposed framework is generic and can be used with different descriptors. We conduct the experiments on publicly available sequences as well as new sequences we collected with hand-held cameras. We show that Oriented FAST and Rotated BRIEF (ORB) features with Bags of Binary Words within the proposed framework lead to higher precision and a higher or similar accuracy compared to  NetVLAD, RootSIFT, and SuperGlue.

§ INTRODUCTION

View-overlap recognition is the task of identifying the same scene captured by a camera over time [13, 14, 15, 24, 25, 33, 39] or by multiple cameras [7, 12, 28, 34, 41] moving in an unknown environment. The latter scenario includes immersive gaming with wearable or hand-held cameras, augmented reality, collaborative navigation, and faster  scene  reconstruction. Existing methods first identify view overlaps with features associated to the whole image (view features) and then with features around interest points (local features) [13, 15, 25, 35].
View features can be a direct transformation of an image or an aggregation of local features. For example, convolutional neural networks can transform an image into a view feature (e.g., NetVLAD [1] or DeepBit [21]) or the original image can be down-sampled and filtered to use as view feature [28].
Local features describe a small area surrounding localised interest points with a fixed-length descriptor, resulting from computed statistics (histogram-based features) [22], pixel-wise comparisons (binary features) [29, 5, 20], or the output of convolutional neural networks [9, 10]. When a 3D map of the scene is available (or can be reconstructed), recognising view overlaps   involves the projection of the 3D points of the map onto a camera view to search for the closest local features. Finally, triangulation and optimisation refine the 3D map and the poses of the cameras [6, 7, 12, 24, 28, 31, 32, 34, 35, 41].

The above-mentioned methods are centralised [12, 24, 34, 35, 41] (i.e. they process images or features in a single processing unit) and hence have a single point of failure. We are interested in decentralising view-overlap recognition  and enable direct interaction across the cameras. To  this end, we decouple feature extraction and cross-camera view-overlap recognition and define (and design) how cameras exchange features. Specifically, we decentralise an efficient two-stage approach that was originally devised for a single, moving camera [13] and design a framework to recognise view-overlaps across two hand-held or wearable cameras simultaneously moving in an unknown environment. Each camera independently extracts, at frame level, view and local features. A camera then recognises view overlaps by matching the features of a query view from another camera with the features of a previous views, and geometrically validates the matched view through the epipolar constraint.

Our main contributions are a decentralised framework for cross-camera view-overlap recognition that decouples the extraction of the view features from the view-overlap recognition in support of the other camera, and is generic for different view and local features; and a dataset of image sequence pairs collected with hand-held and wearable cameras in four different environments, along with annotations of camera poses and view overlaps[The new sequences and all annotations can be found at: <https://www.eecs.qmul.ac.uk/ ax300/xview>].

§ RELATED WORK

View-overlap recognition uses image-level features (or view features) and local features to represent each view. This process can be centralised or decentralised. A centralised approach reconstructs, in a single processing unit, the 3D map using images or features received from all the cameras [12, 28, 34, 41]. With a decentralised approach, cameras exchange local and view features [7, 8].
Each camera can also create its own map and the cross-camera view matching can rely on a coarse-to-fine strategy [7, 12, 34], which first searches and matches the most similar view-feature in a database of accumulated view-features over time compared to the query view-feature and then matches local features associated with the 3D points in the local map. The coarse-to-fine strategy is important as view features are more sensitive to geometric differences, and hence additional steps, such as local feature matching and  geometric verification [17], are often necessary to validate a matched view [7, 13, 15, 24, 33].

Local features can be aggregated into hierarchical structures, such as bag of visual words [13, 15, 25, 26, 36] or binary search tree [33], spatio-temporal representations [39, 40], compact vectors summarising the first order statistics [18], or Fisher vectors [27]. Hierarchical structures can be pre-trained [13, 25, 26, 36] or built on-the-fly [15, 33], and can be used to speed up the search and matching of query local features with only a subset of local features stored in the structure. For example, Bags of Binary Words (DBoW) [13] coupled with binary features (e.g., Oriented FAST and Rotated BRIEF – ORB [29])  has been often preferred for 3D reconstruction and autonomous navigation due to the computational efficiency in feature extraction, searching, and matching, as well as the low storage requirements [7, 24, 34]. However, as binary features are generally less robust to large geometric differences [1, 3, 21] than histogram-based features (e.g., SIFT[22], RootSIFT [2]), centralised and decentralised approaches focused on the reconstruction aspect of the problem and on scenarios with substantial view-overlaps.

Finally, no publicly available dataset exists with recordings from multiple cameras freely moving  in an environment with annotations of the camera poses and view overlaps. Because of the lack of publicly available datasets and their annotations, reproducibility and fair comparison of the previous methods are still an open challenge. Annotations are so far provided only for single camera sequences [13, 16, 33, 37, 39]. Zou and Tan [41] released only one scenario with four camera sequences, but without annotations. To overcome the lack of datasets or their annotations, some  methods are evaluated on data acquired continuously with a single camera and then split into multiple parts to simulate a multi-camera scenario [7, 8]. However, this results in highly overlapping portions, especially at the beginning and at the end of the sub-sequences. Alternatively, annotations of the camera poses and view overlaps can be determined through image-based 3D reconstruction methods [35], but with the assumption that there are no moving objects or people in the scene.

§ DECENTRALISING THE RECOGNITION OF VIEW OVERLAPS

In this section, we discuss our design choices (the distributed messaging, the features to extract and share, and the coarse-to-fine recognition strategy) to  decentralise DBoW [13], an approach for real-time view-overlap recognition for a single moving camera that is based on binary features, corresponding bag of visual words, and a coarse-to-fine matching strategy.  Fig. <ref> illustrates the proposed framework focusing on the processing and communication of a camera.

Block diagram of the proposed decentralised framework for cross-camera view-overlap recognition. For each camera, the framework decouples the extraction of view features $\boldsymbol{v}_t$ and local features $\mathcal{F}_t = \{ \mtrx{x}_{i,t}, \mtrx{d}_{i,t} \}_{i=1}^{F_t}$, for each frame $I_t$, from the recognition of view overlaps. While features are aggregated into a hierarchical structure over time (bag of binary words, shown as a red registry), each camera shares the set of local features $\mathcal{Q}_t$ at a pre-defined sharing rate $f$ and after an initialisation window of length $L$ (clock symbol). The camera receiving $\mathcal{Q}_t$ re-aggregates the received local features into the view feature $\boldsymbol{v}_q$ if their number is higher than $\mu$, and matches $\boldsymbol{v}_q$  with all the view features up to frame $t$. The local features of the matched view $m_t$, $\mathcal{F}_m$, whose score $s_{q,m}$ was the highest and higher than the threshold $\alpha$, are then matched with the query local features, obtaining $\mathcal{M}_{m,q}$. The matched view $m_t$ is validated through robust fitting of the epipolar geometry only if the number of inliers $|\hat{\mathcal{M}}_{m,q}| > \rho$. For keeping the notation simple, we use the camera indexes 1 and 2 only for denoting the shared query features and matched view.

§.§ Camera Messaging

We use the ZeroMQ[<http://zeromq.org/>] distributed messaging to decentralise the recognition across cameras, and multi-threading to decouple (i) the processing of the sequence and (ii) the view-overlap recognition in support of the other camera. These two operations are performed independently from each other within a single camera.

To account for reliability, we adopt the ZeroMQ request-reply strategy: after sending a message containing the query view features, the camera waits for a reply from the other camera before processing the new frame. The reply includes a status denoting if a view overlap has been recognised and the corresponding frame in the sequence of the other camera.
We assume that the cameras exchange query views on a communication channel that is always available (i.e. the cameras remain within communication range), reliable (i.e. all query views are received), and ideal (i.e. no communication delays).

§.§ Sharing Query Views

For each frame $t$, each camera[For simplicity of notation, we do not use a camera index.] independently extracts a feature representing the image (view feature), $\boldsymbol{v} \in \mathbb{R}^G$, and a set of local features, $\mathcal{F}_t = \{ (\mtrx{d}_{i,t}, \mtrx{x}_{i,t}) \}_{i=1}^{F_t}$, that include their  location $\mtrx{x}_{i,t} \in \mathbb{R}^2$ (or interest point) and the corresponding binary descriptor $\mtrx{d}_{i,t} \in \{0,1\}^D$. The camera aims at extracting a pre-defined maximum target number of features $F$, but the number of interest points localised in the image, $F_t$, could be lower than $F$ due to the content (e.g., presence of corners) and appearance of the scene. We use ORB [29] as local feature and the bag of visual words devised by DBoW for binary features as view feature [13].
Specifically, this view feature stores the binary features in a vocabulary tree that is trained offline, using hierarchical k-means. Therefore, we add the extracted view and local features for each new frame to the vocabulary for speeding up the indexing, searching, and matching.

While moving, cameras share the features of the view a time $t$, $\mathcal{Q}_t = \mathcal{F}_t$, to query previous views of the scene observed by the other camera. Note that a camera shares only the set of local features because the view feature can be directly recovered by the other camera as an aggregation of the local features using the bag of visual words with a pre-trained vocabulary.
On the contrary, if we choose a view feature that can be obtained  with a different approach, e.g. convolutional neural networks [1, 21], then a camera should also share the view feature as part of the query.

Moreover, each camera shares the query features with a pre-defined rate $f$ that can be slower than the camera acquisition rate $r$ to avoid exchanging consecutive redundant information. Alternatively, automatic frame selection can be devised [24] and features would be shared only at the selected frames.
As the camera receiving the query view features will match these features with the features extracted in its previous frames to identify a view overlap, each camera avoids to share query features during the first $L$ frames (initialisation window). This allows a camera to add view and local features to the vocabulary tree to enable cross-camera view-overlap recognition. The sharing of the query features occurs only when

\begin{equation}
max(t-L,-1) - \left\lfloor \frac{max(t-L,-1)}{r}f \right\rfloor \frac{r}{f} = 0,
\end{equation}

where $\left\lfloor \cdot \right\rfloor$ is the floor operation. As an example, let the acquisition rate be $r=30$ fps, the sharing rate be $f=6$ fps, and the number of binary features, whose dimensionality is $D=256$ bits, be at  maximum $F=1,000$. By including the interest point (32$\times$2 bits) associated with each binary feature, each camera exchanges at most 240 kilobyte per second (kB/s). If a view feature extracted with a convolutional neural network, for example NetVLAD [1] whose dimensionality is $G=131,072$ bits (4,096 elements), then each camera exchanges at maximum about 256 kB/s.

§.§ View-Overlap Recognition

To recognise a view overlap, the camera receiving  $\mathcal{Q}_t$ first searches and matches the query view feature $\boldsymbol{v}_q$ within its local vocabulary tree up to the current frame $t$ using the following score [13],

\begin{equation}
s(\boldsymbol{v}_q,\boldsymbol{v}_k) = 1- \frac{1}{2}\left| \frac{\boldsymbol{v}_q}{|\boldsymbol{v}_q |} - \frac{\boldsymbol{v}_k}{|\boldsymbol{v}_k |} \right|,
\end{equation}

where $k=1, \ldots, t$.
Following the approach of DBoW [13], the matching of the view features can identify up to $Nr$ candidates with respect to the query view feature, depending on the acquisition rate $r$ and a minimum score $\alpha$. Matched view features are thus grouped to prevent consecutive images competing with each other [13]. Each group contains a minimum number of matched view-features depending on the acquisition rate $r$. Matched view-features in the same group have a maximum difference (number of frames) of $\beta r$. The group and view with the highest score is selected for further validation.

After identifying the candidate view $m_t$, local features of both views are matched via nearest neighbour search. To avoid ambiguities, we discard matches based on a threshold $\gamma$ for the Hamming distance between the descriptors of two local features, $H(\mtrx{d}_{i}^q, \mtrx{d}_{j}^m)$, and a threshold $\delta$ for the distance ratio between the closest and the second closest neighbour (Lowe's ratio test [22])[The threshold for the Lowe's ratio test is usually set in the interval $[0.6,0.8]$. The threshold for the Hamming distance is usually chosen based on the typical separation of matching and non-matching feature distributions for binary features.]. The final set of matched local features is

\begin{equation}
\mathcal{M}_{q,m} = \left\lbrace (\mtrx{d}_{i}^q, \mtrx{x}_{i}^q, \mtrx{d}_{j}^m, \mtrx{x}_{j}^m) | H(\mtrx{d}_{i}^q, \mtrx{d}_{j}^m) < \gamma, \frac{H(\mtrx{d}_{i}^q, \mtrx{d}_{j}^m)}{H(\mtrx{d}_{i}^q, \mtrx{d}_{l}^m)} < \delta \right\rbrace,
\end{equation}

where $\mtrx{d}_{i}^q$ is the $i$-th query binary feature from $\mathcal{Q}_t$, and $\mtrx{d}_{j}^m$ $\mtrx{d}_{l}^m$ are the closest and second closest neighbours found in the candidate view $m_t$, with $j \neq l$.

Matched local features are then geometrically validated through the epipolar constraint by estimating a matrix that encodes the undergoing rigid transformation (fundamental matrix) [17]: $(\mtrx{x}_i^q) \mtrx{F} (\mtrx{x}_j^m)^\top = 0$. As a set of eight correspondences between the two views is necessary to estimate the fundamental matrix $\mtrx{F}$, we discard the candidate view $m_t$ if $|\mathcal{M}_{q,m}| < \mu$ ($\mu=8$), where $| \cdot |$ is the cardinality of a set. On the contrary, when $|\mathcal{M}_{q,m}| > \mu$, we use robust fitting model with Random Sample Consensus [11] to obtain the hypothesis (fundamental matrix) with the highest number of inliers – i.e., matched features satisfying the epipolar constraint with a maximum re-projection error, $\hat{\mathcal{M}}_{q,m,}$. To enable the searching of multiple solutions while sampling different correspondences, we require a minimum of $\rho$ correspondences that are also inliers for the estimated fundamental matrix, i.e., $|\hat{\mathcal{M}}_{q,m,}| > \rho$.

§ VALIDATION

§.§ Strategies under Comparison

We compare the proposed framework based on DBoW with five alternative choices for local features, view features, and  matching strategy. We refer to each version of the framework based on the main alternative component: D-DBoW, D-NetVLAD, D-DeepBit, D-RootSIFT, D-SuperPoint, and D-SuperGlue.

D-NetVLAD replaces the matching of view features based on bag of visual binary words with view features directly extracted with a convolutional neural network that in the last layer aggregates the first order statistics of mid-level convolutional features, i.e., the residuals in different parts of the descriptors space weighted by a soft assignment (NetVLAD) [1]. D-NetVLAD aims to reproduce the decentralised approach in [7] but without the pre-clustering assignment across multiple cameras (more than two).
D-DeepBit also uses a convolutional neural network to represent an image with a view feature consisting of binary values (DeepBit) [21]. The parameters of the model are learned in an unsupervised manner with a Siamese network to achieve invariance to different geometric transformations (e.g., rotation and scale), enforce minimal quantisation error between the real-value deep feature and the binary code to increase the descriptiveness (quantisation loss), and maximise  the information capacity (entropy) for each bin by evenly  distributing the binary code (even-distribution loss). Both variants of the framework preserves the sharing of ORB features for the coarse-to-fine strategy and the efficient matching of the query binary features through the reconstructed bag of visual words vector at the receiving camera, after identifying the most similar view feature (NetVLAD or DeepBit).
D-RootSIFT replaces the ORB features with transformed SIFT features via L1 normalisation, square-root, and L2 normalisation to exploit the equivalence between the Hellinger distance when comparing histograms and the Euclidean distance in the transformed feature space and hence reduce the sensitivity to smaller bin values (RootSIFT) [2].
D-SuperPoint also replaces the binary features with local features that are extracted with a two-branch convolutional neural network trained in a self-supervised manner to jointly localise and describe interest points directly on raw input images, assuming that the model is a homography (SuperPoint) [9].
D-SuperGlue is coupled with SuperPoint features and replaces the nearest neighbour matching by performing context aggregation, matching, and filtering in a single end-to-end architecture that consists of a Graph Neural Network and an Optimal Matching layer [30].
For fair comparison and analysis of the impact of the chosen local features, D-RootSIFT, D-SuperPoint, and D-SuperGlue use NetVLAD as view feature. D-RootSIFT and D-SuperPoint use the nearest neighbour strategy with Lowe's ratio test to match the local features.

As sampling strategy for robust fitting model of the fundamental matrix, we also compare the use of RANSAC [11]
against MAGSAC++ [4], a recent robust estimator that uses a quality scoring function while avoiding the inlier-outlier decision, and showed to be fast, robust, and more geometrically accurate. RANSAC is used within D-DBoW, D-NetVLAD and D-DeepBit and we therefore provide the corresponding alternatives with MAGSAC++, whereas D-RootSIFT, D-SuperPoint, and D-SuperGlue use directly MAGSAC++.

§.§ Implementation Details

The framework is implemented in C++ with OpenCV, DLib, and ZeroMQ libraries. For evaluation purposes and reproducibility, we implemented the framework in a synchronous way, and ZeroMQ request-reply messages are sent every frame to keep the cameras synchronised. Moreover, we allow the camera that finishes earlier to process its own sequence to remain active and recognise view overlaps until the other camera also terminates its own sequence.
For all variants of the framework, we target to extract a maximum of $F=1,000$ local features (i.e., either ORB, RootSIFT, or SuperPoint). When matching binary features, we set the maximum Hamming distance to $\gamma=50$ as threshold to validate a match (for $D=256$ [5, 24]), while the threshold for Lowe's ratio test (or nearest neighbour distance ratio) [22] is $\delta=0.6$.
For NetVLAD, we use the best model (VGG-16 + NetVLAD + whitening), trained on Pitts30k [38], as suggested by the authors [1], whereas we use DeepBit 32-bit model trained on CIFAR10 [19] for DeepBit [21]. Similarly to the decentralised approach in Cieslewski et al.'s work [7], we consider the first 128 components of the NetVLAD descriptors and we use Euclidean distance to match descriptor, while we use the Hamming distance for DeepBit 32-bit model. For both NetVLAD and DeepBit, we extract the view features offline for each image of the sequences, and we provide the features as input to the framework. For the parameters of DBoW, SuperPoint, and SuperGlue, we used the values provided in the original implementations. We thus set $N=50$, $\alpha=0.03$, $\beta=3$, and the number of levels for direct index retrieval in the vocabulary tree to 2 for DBoW [13].
D-DBOW, D-NetVLAD, and D-DeepBit uses the RANSAC implementation provided by DLib with a minimum number of inliers, $\rho=12$. For fairness in the evaluation, D-RootSIFT, D-SuperPoint, and D-Superglue use the matched views identified by D-NetVLAD to extract and match the local features for the validation of the view. Note that the models of SuperPoint and SuperGlue were implemented in Python with PyTorch, and hence D-RootSIFT, D-SuperPoint, and D-Superglue are not part of the implemented framework. For MAGSAC++, we use the implementation provided by OpenCV 4.5.5 with a minimum number of inliers, $\rho=15$. For both RANSAC and MAGSAC++, we set the maximum number of iterations to 500, the probability of success to 0.99, and the maximum re-projection error to 2 pixels.

§.§ Dataset and View-overlap Annotation

We use sequence pairs from four scenarios: two scenarios that we collected with both hand-held and chest-mounted cameras – gate and backyard of four sequences each – and two publicly available datasets – TUM-RGB-D SLAM [37] and  courtyard[<https://drone.sjtu.edu.cn/dpzou/dataset/CoSLAM/>] from CoSLAM [41] – for a total of $\sim$28,000 frames ($\sim$25 minutes). From TUM-RGB-D SLAM, we use the sequences fr1_desk, fr1_desk2, and fr1_room to form the office scenario. The courtyard scenario consists of four sequences. We sub-sampled courtyard from 50 to 25 fps and  backyard from 30 to 10 fps for annotation purposes.
Tab. <ref> summarises the scenarios.

Description of the scenarios with multiple hand-held moving cameras.

Scenario     # Seq.     4cNumber of frames/seq.      fps     1cResolution

office     3      573      612     1,352     –      30     640x480

courtyard     4     2,849     3,118     3,528     3,454     25     800x450

gate     4     330     450     480     375     30     1280x720

backyard     4     1,217     1,213     1,233     1,235     10     1280x720

8lKEY – # Seq.: number of sequences; fps: frame per second.

We automatically annotate correspondent views by exploiting camera poses and calibration parameters estimated with COLMAP [35], a Structure-from-Motion pipeline, for gate, courtyard, and backyard. These data are already available for office.
When the frustum between two views intersects under free space assumption [23], we compute the viewpoint difference as the angular distance between the camera optical axes, the Euclidean distance between the camera positions, and the visual overlap as the ratio between the area spanned by projected 3D points within the image boundaries and the image area. Fig. <ref> visualises the distribution of the annotated angular distances across sequence pairs, showing how scenarios widely varies in their viewpoint differences based on the multiple camera motions.
For office, we localise a set of interest points (e.g., SIFT [22]) in the first view and we back-project the points in 3D by exploiting the depth value at the corresponding location. For the other scenarios, we re-project the 3D points associated to the interest points of a frame in the first view onto the second view by using the estimated camera poses. When annotating correspondent views, we can define a threshold on the visual overlap, i.e., image pairs are a valid correspondence if their visual overlap is greater than the threshold.
Note that a large overlap does not imply that the viewpoint is very similar as the angular and/or Euclidean distances can be large. Therefore, we constrain valid correspondent views to have an angular distance less than 70$^\circ$. In this work, we set the threshold on the overlap to 50% and an analysis of its impact on the performance results will be subject of future work.
Moreover, we compute the total number of annotated views as the number of frames with at least one annotated view, i.e., a frame with more than one annotated view counts as one.

pie chart,
slice type=NFIdc1,
slice type=bin1dc2,
slice type=bin2dc3,
slice type=bin3dc4,
slice type=bin4dc5,
slice type=bin5dc6,
slice type=bin6dc7,
slice type=bin7dc8,
slice type=bin8dc9,
slice type=bin9dc10,
slice type=bin10dc11,
pie values/.style=font=,
courtyard42.23/NFI, 9.74/bin1, 9.74/bin2, 8.80/bin3, 7.97/bin4, 7.01/bin5, 3.62/bin6, 3.26/bin7, 2.77/bin8, 2.41/bin9, 2.47/bin10

pie chart,
slice type=NFIdc1,
slice type=bin1dc2,
slice type=bin2dc3,
slice type=bin3dc4,
slice type=bin4dc5,
slice type=bin5dc6,
slice type=bin6dc7,
slice type=bin7dc8,
slice type=bin8dc9,
slice type=bin9dc10,
slice type=bin10dc11,
pie values/.style=font=,
gate5.04/NFI, 34.16/bin1, 30.09/bin2, 29.84/bin3, 0.86/bin4

pie chart,
slice type=NFIdc1,
slice type=bin1dc2,
slice type=bin2dc3,
slice type=bin3dc4,
slice type=bin4dc5,
slice type=bin5dc6,
slice type=bin6dc7,
slice type=bin7dc8,
slice type=bin8dc9,
slice type=bin9dc10,
slice type=bin10dc11,
pie values/.style=font=,
backyard41.30/NFI, 5.79/bin1, 10.63/bin2, 9.22/bin3, 9.10/bin4, 6.37/bin5, 4.12/bin6, 3.41/bin7, 3.23/bin8, 3.31/bin9, 3.52/bin10

Distribution of the angular distances in the outdoor scenarios.

[dc1,fill=dc1] (0,0) rectangle (1.ex,1.ex); no frustum intersection,
[dc2,fill=dc2] (0,0) rectangle (1.ex,1.ex); 0–15$^\circ$,
[dc3,fill=dc3] (0,0) rectangle (1.ex,1.ex); 15–30$^\circ$,
[dc4,fill=dc4] (0,0) rectangle (1.ex,1.ex); 30–45$^\circ$,
[dc5,fill=dc5] (0,0) rectangle (1.ex,1.ex); 45–60$^\circ$,
[dc6,fill=dc6] (0,0) rectangle (1.ex,1.ex); 60–75$^\circ$,
[dc7,fill=dc7] (0,0) rectangle (1.ex,1.ex); 75–90$^\circ$,
[dc8,fill=dc8] (0,0) rectangle (1.ex,1.ex); 90–105$^\circ$,
[dc9,fill=dc9] (0,0) rectangle (1.ex,1.ex); 105–120$^\circ$,
[dc10,fill=dc10] (0,0) rectangle (1.ex,1.ex); 120–135$^\circ$,
[dc11,fill=dc11] (0,0) rectangle (1.ex,1.ex); 135–150$^\circ$,

§.§ Performance Measures

We compare the framework with the different strategies using precision, recall, and accuracy as performance measures for each sequence pair in the dataset. We first compute the number of true positives (TP), false positives (FP), false negatives (FN), and true negatives (TN). A true positive is a pair of views that is recognised as in correspondence (or overlap) and their annotation is also a valid correspondence. A false positive is a pair of views that is recognised as in correspondence but their annotation is not. A false negative is a (query) view with shared features from one camera that is not matched with any other view in the second camera (or vice versa) by a method, but a corresponding view is annotated. A true negative is a view from one camera that is not matched with any other view in the second camera (or vice versa), and there is not any view that is annotated as correspondence in the second camera.

Precision is the number of correctly detected views over the total number of matched views: $P = TP/(TP + FP)$. Recall is the number of correctly detected views in overlap over the total number of annotated view correspondences: $R = TP/(TP + FN)$. Accuracy is the number of correctly recognised views in overlap and correctly rejected views over the total number of annotated view correspondences, correctly rejected views, and wrongly recognised views in overlap,

\begin{equation}
A = \frac{TP + TN}{TP + FP + FN + TN}.
\label{eq:accuracy}
\end{equation}

Because of the pairwise approach, we sum TP, FP, TN, and FN, across the two cameras before computing P, R, and A. We will discuss the results using the performance measures as percentages with respect to each sequence pair.

§.§ Parameter Setting

We performed an analysis of the number of repetitions to perform due to the non-deterministic behaviour introduced by RANSAC, the length of the initialisation window $L$, the acquisition rate $r$ affecting the matching of view features, and the sharing rate $f$. We evaluated the accuracy of D-DBoW, D-NetVLAD, and D-DeepBit with 100 repetitions on two sequence pairs, gate 1$|$2 and office 1$|$2 ($X|Y$ denotes the indexes of the camera sequences in a scenario). We observed that the median accuracy was converging to a stable result within the first 30 repetitions due to the reproducibility of the same set of results for many repetitions. Therefore, the rest of the experiments are performed by repeating the framework on the dataset for 30 repetitions and reporting the median value. We also observed that the median accuracy is not significantly affected when varying the length of the initialisation window and the acquisition rate. We therefore set $L=30$ to allow the cameras to accumulate an initial set of view features, and $r$ to the corresponding frame rate of the scenario in Tab. <ref>. We set $f=6$ fps after observing that an automatic frame selection algorithm based of feature point tracking was re-initialising the feature trajectories every 5 frames, on average, for a handheld camera, while a person is walking.

SuperGlue provides pre-trained weights for both outdoor and indoor scenes. We use the outdoor pre-trained weights for the scenarios gate, courtyard, and backyard, and the indoor pre-trained weights for office.

y dir=reverse,

xlabel=Precision (%),
xmin=-5, xmax=105,
ymin=0, ymax=20,
xlabel near ticks,
ylabel near ticks,
yticklabels=G1$|$2,G1$|$3,G1$|$4,G2$|$3,G2$|$4,G3$|$4,O1$|$2,O1$|$3,O2$|$3,B1$|$4,B2$|$3,C1$|$2,C1$|$3,C1$|$4,C2$|$3,C2$|$4,C3$|$4, AVG,
label style=font=,
tick label style=font=,

+[only marks, color=white, mark=*, mark options=fill=white] table[x=P-G, y=Scenario]
ıin 0,...,
coordinate [pos=ı/] (aı)

+[only marks, color=black, mark=*, mark options=fill=black] table[x=P-GL, y=Scenario]
ıin 0,...,
coordinate [pos=ı/] (bı)

ıin 0,...,
(aı) – (bı);

y dir=reverse,

xlabel=Recall (%),
xmin=-5, xmax=105,
ymin=0, ymax=20,
xlabel near ticks,
ylabel near ticks,
label style=font=,
tick label style=font=,

+[only marks, color=white, mark=*, mark options=fill=white] table[x=R-G, y=Scenario]
ıin 0,...,
coordinate [pos=ı/] (aı)

+[only marks, color=black, mark=*, mark options=fill=black] table[x=R-GL, y=Scenario]
ıin 0,...,
coordinate [pos=ı/] (bı)

ıin 0,...,
(aı) – (bı);

y dir=reverse,

xlabel=Accuracy (%),
xmin=-5, xmax=105,
ymin=0, ymax=20,
xlabel near ticks,
ylabel near ticks,
label style=font=,
tick label style=font=,

+[only marks, color=white, mark=*, mark options=fill=white] table[x=A-G, y=Scenario]
ıin 0,...,
coordinate [pos=ı/] (aı)

+[only marks, color=black, mark=*, mark options=fill=black] table[x=A-GL, y=Scenario]
ıin 0,...,
coordinate [pos=ı/] (bı)

ıin 0,...,
(aı) – (bı);

Effect of matching both view and local features compared to matching only view features with D-DBoW. Positive change is shown with the black circles on the right of the line, whereas negative change with the black circles on the left. The length of the line denotes the magnitude of the change.
KEY – sx$|$y: image sequence pair where s is either G (gate), O (office), B (backyard), or C (courtyard), and x and y are the indexes of the first and second sequences in each scenario; AVG: average.

§.§ Discussion

Fig. <ref> shows the change in precision, recall, and accuracy from using only the first step (matching only view features) to using also the local feature matching and geometric validation in the 17 sequence pairs from the four scenarios in the dataset. The two-stage approach improves both precision and accuracy of, respectively, 14.35 percentage points (pp) and 8.43 pp, on average. This is observable in almost all sequence pairs as wrongly matched views are invalidated, while also increasing the recognition of views that are not in overlap (true negative in accuracy). However, the disadvantage of the two-stage approach is the large drop in recall (-26.37 pp, on average), as many views in overlap with respect to the query views were not identified.

Comparisons of precision (P), recall (R), and accuracy (A) using the framework with different strategies for each testing sequence pair.

Sequence     Pair     Q1     Q2     M
D-DW     D-NV     D-DB      D-RS     D-SP     D-SG

3*1$|$2     3*59     3*83     P     81.58     100.00     100.00     68.91     96.61     67.20

R     24.33     11.76     11.76     80.39     42.54     84.00

A     25.35     15.49     15.49     59.86     44.37     59.86

3*1$|$3     3*59     3*89     P     39.23     40.00     53.57     29.51     0.00     26.92

R     9.30     4.49     4.35     43.90     0.00     70.00

A     68.92     68.92     69.26     55.41     61.82     40.54

3*1$|$4     3*59     3*68     P     100.00     100.00     100.00     90.91     100.00     75.00

R     15.76     5.43     15.22     54.35     31.52     60.00

A     38.98     31.50     38.58     62.99     50.39     59.84

3*2$|$3     3*83     3*89     P     97.92     100.00     100.00     87.30     100.00     56.14

R     54.65     34.48     9.20     65.88     45.40     78.05

A     76.74     66.86     54.07     77.91     72.38     60.47

3*2$|$4     3*83     3*68      P     100.00     0.00     100.00     100.00     100.00     35.90

R     1.79     0.00     0.88     11.50     4.42     14.74

A     26.49     25.17     25.83     33.77     28.48     29.80

3*3$|$4     3*89     3*68      P     0.00     0.00     0.00     0.00     0.00     0.00

R     0.00     0.00     0.00     0.00     0.00     0.00

A     43.31     49.04     47.77     36.94     46.82     29.94

3*1$|$2     3*108     3*116     P     54.55     42.68     20.00     44.07     42.86     29.25

R     44.40     10.75     0.66     20.23     13.24     53.75

A     48.44     35.04     30.80     39.29     36.61     37.05

3*1$|$3     3*108     3*264     P     25.68     16.67     0.00     20.59     28.22     17.31

R     10.84     0.70     0.00     5.51     4.83     20.45

A     60.08     60.22     60.48     60.48     61.16     58.06

3*2$|$3     3*116     3*264     P     21.43     35.15     33.33     22.58     28.08     15.31

R     11.63     3.24     0.62     4.67     5.25     13.64

A     55.53     57.37     57.11     56.05     56.58     53.16

3*1$|$4     3*237     3*240     P     84.52     76.47     0.00     66.67     64.29     36.47

R     22.72     4.79     0.00     20.07     19.49     27.19

A     50.52     40.67     38.16     47.38     46.75     42.35

3*2$|$3     3*236     3*240     P     77.14     90.00     100.00     73.53     66.67     44.02

R     11.99     3.34     0.32     28.90     28.00     40.87

A     40.55     35.92     34.03     47.90     45.80     41.39

3*1$|$2     3*553     3*613     P     0.73     0.00     0.00     0.00     0.00     0.60

R     1.25     0.00     0.00     0.00     0.00     2.11

A     64.28     77.79     83.75     74.96     75.90     45.45

3*1$|$3     3*553     3*693     P     0.45     0.00     0.00     0.68     0.00     0.62

R     0.29     0.00     0.00     0.27     0.00     0.99

A     55.58     61.80     67.01     58.67     58.75     37.52

3*1$|$4     3*553     3*678     P     4.91     5.22     0.00     4.60     3.86     2.13

R     2.70     1.06     0.00     1.43     1.45     2.93

A     68.07     72.26     74.90     70.92     69.86     54.96

3*2$|$3      3*613     3*693     P     41.52     40.34     21.32     39.32     37.92     19.65

R     24.48     8.89     0.72     12.28     12.13     21.67

A     39.36     37.02     34.72     38.82     38.17     27.45

3*2$|$4     3*613     3*678     P     19.34     32.73     10.00     31.43     30.43     12.58

R     11.06     4.86     0.35     6.08     6.48     14.53

A     48.92     54.92     54.11     54.92     54.69     41.36

3*3$|$4     3*693     3*678     P     42.38     38.83     29.79     41.18     47.95     23.32

R     22.81     5.65     0.81     9.21     11.97     26.10

A     51.50     47.05     45.62     48.14     49.53     42.01

11l.99QX: number of queries from the first (second) camera; M: performance measure; D-DW: D-DBoW; D-NV: D-NetVLAD; D-DB: D-DeepBit; D-RS: D-RootSIFT; D-SP: D-SuperPoint; D-SG: D-SuperGlue. For each row, the best- and second best-performing results are highlighted in bold and italic, respectively. Best-performing results lower than 10 are not highlighted.

axis lines*=box,
bar width=3.5pt,

tick label style=font=,

xmin=0.5,  xmax=5.5,

ylabel=Precision (%),
label style=font=,

+[ybar, black, fill=c1, draw opacity=0.5, error bars/.cd, y dir=both, y explicit] table[x=Scenario, y=BW-m, y error =BW-s];
+[ybar, black, fill=c2, draw opacity=0.5, error bars/.cd, y dir=both, y explicit] table[x=Scenario, y=NV-m, y error =NV-s];
+[ybar, black, fill=c3, draw opacity=0.5, error bars/.cd, y dir=both, y explicit] table[x=Scenario, y=DB-m, y error =DB-s];
+[ybar, black, fill=c4, draw opacity=0.5, error bars/.cd, y dir=both, y explicit] table[x=Scenario, y=BWM-m, y error =BWM-s];
+[ybar, black, fill=c5, draw opacity=0.5, error bars/.cd, y dir=both, y explicit] table[x=Scenario, y=NVM-m, y error =NVM-s];
+[ybar, black, fill=c6, draw opacity=0.5, error bars/.cd, y dir=both, y explicit] table[x=Scenario, y=DBM-m, y error =DBM-s];
+[ybar, black, fill=c7, draw opacity=0.5, error bars/.cd, y dir=both, y explicit] table[x=Scenario, y=RS-m, y error =RS-s];
+[ybar, black, fill=c8, draw opacity=0.5, error bars/.cd, y dir=both, y explicit] table[x=Scenario, y=SP-m, y error =SP-s];
+[ybar, black, fill=c9, draw opacity=0.5, error bars/.cd, y dir=both, y explicit] table[x=Scenario, y=SG-m, y error =SG-s];

axis lines*=box,
bar width=3.5pt,

tick label style=font=,

xmin=0.5,  xmax=5.5,

ylabel=Recall (%),
label style=font=,

+[ybar, black, fill=c1, draw opacity=0.5, error bars/.cd, y dir=both, y explicit] table[x=Scenario, y=BW-m, y error =BW-s];
+[ybar, black, fill=c2, draw opacity=0.5, error bars/.cd, y dir=both, y explicit] table[x=Scenario, y=NV-m, y error =NV-s];
+[ybar, black, fill=c3, draw opacity=0.5, error bars/.cd, y dir=both, y explicit] table[x=Scenario, y=DB-m, y error =DB-s];
+[ybar, black, fill=c4, draw opacity=0.5, error bars/.cd, y dir=both, y explicit] table[x=Scenario, y=BWM-m, y error =BWM-s];
+[ybar, black, fill=c5, draw opacity=0.5, error bars/.cd, y dir=both, y explicit] table[x=Scenario, y=NVM-m, y error =NVM-s];
+[ybar, black, fill=c6, draw opacity=0.5, error bars/.cd, y dir=both, y explicit] table[x=Scenario, y=DBM-m, y error =DBM-s];
+[ybar, black, fill=c7, draw opacity=0.5, error bars/.cd, y dir=both, y explicit] table[x=Scenario, y=RS-m, y error =RS-s];
+[ybar, black, fill=c8, draw opacity=0.5, error bars/.cd, y dir=both, y explicit] table[x=Scenario, y=SP-m, y error =SP-s];
+[ybar, black, fill=c9, draw opacity=0.5, error bars/.cd, y dir=both, y explicit] table[x=Scenario, y=SG-m, y error =SG-s];

axis lines*=box,
bar width=3.5pt,
xticklabels=gate, office, backyard, courtyard, all scenarios,
tick label style=font=,

xmin=0.5,  xmax=5.5,

ylabel=Accuracy (%),
label style=font=,

+[ybar, black, fill=c1, draw opacity=0.5, error bars/.cd, y dir=both, y explicit] table[x=Scenario, y=BW-m, y error =BW-s];
+[ybar, black, fill=c2, draw opacity=0.5, error bars/.cd, y dir=both, y explicit] table[x=Scenario, y=NV-m, y error =NV-s];
+[ybar, black, fill=c3, draw opacity=0.5, error bars/.cd, y dir=both, y explicit] table[x=Scenario, y=DB-m, y error =DB-s];
+[ybar, black, fill=c4, draw opacity=0.5, error bars/.cd, y dir=both, y explicit] table[x=Scenario, y=BWM-m, y error =BWM-s];
+[ybar, black, fill=c5, draw opacity=0.5, error bars/.cd, y dir=both, y explicit] table[x=Scenario, y=NVM-m, y error =NVM-s];
+[ybar, black, fill=c6, draw opacity=0.5, error bars/.cd, y dir=both, y explicit] table[x=Scenario, y=DBM-m, y error =DBM-s];
+[ybar, black, fill=c7, draw opacity=0.5, error bars/.cd, y dir=both, y explicit] table[x=Scenario, y=RS-m, y error =RS-s];
+[ybar, black, fill=c8, draw opacity=0.5, error bars/.cd, y dir=both, y explicit] table[x=Scenario, y=SP-m, y error =SP-s];
+[ybar, black, fill=c9, draw opacity=0.5, error bars/.cd, y dir=both, y explicit] table[x=Scenario, y=SG-m, y error =SG-s];

Comparison of mean and standard deviation of precision, recall, and accuracy between the different alternatives of the framework for cross-camera view recognition on the four scenarios and across all scenarios.

[c1,fill=c1] (0,0) rectangle (1.ex,1.ex); D-DBoW,
[c2,fill=c2] (0,0) rectangle (1.ex,1.ex); D-NetVLAD,
[c3,fill=c3] (0,0) rectangle (1.ex,1.ex); D-DeepBit,
[c4,fill=c4] (0,0) rectangle (1.ex,1.ex); D-DBoW with MAGSAC++,
[c5,fill=c5] (0,0) rectangle (1.ex,1.ex); D-NetVLAD with MAGSAC++,
[c6,fill=c6] (0,0) rectangle (1.ex,1.ex); D-DeepBit with MAGSAC++,
[c7,fill=c7] (0,0) rectangle (1.ex,1.ex); D-RootSIFT,
[c8,fill=c8] (0,0) rectangle (1.ex,1.ex); D-SuperPoint,
[c9,fill=c9] (0,0) rectangle (1.ex,1.ex); D-SuperGlue.

Tab. <ref> and Fig. <ref> compare precision, recall, and accuracy of D-DBoW, D-NetVLAD, D-DeepBit, D-RootSIFT, D-SuperPoint, and D-SuperGlue. Overall, D-RootSIFT achieves the highest mean accuracy across all scenarios (54.38%), followed by D-SuperPoint (52.83%), and D-DBoW (50.74%). Despite the 4 pp difference with D-RootSIFT, D-DBoW can benefit the fast inference of the bag of binary words and binary features for applications that may require nearly real-time processing. All variants have large standard deviations (around 10-20% of the median accuracy), suggesting that different camera motions can affect the recognition of the view overlaps. Except for courtyard, the bag of visual binary words of D-DBoW helps to achieve a higher accuracy than using D-NetVLAD and D-DeepBit as view features, independently of using RANSAC or MAGASC++ as robust estimator for validating the matched views. Note that MAGSAC++ does not contribute to improve the median accuracy of the cross-camera view-overlap recognition but achieves lower accuracy than RANSAC across all scenarios, except in some sequence pairs (e.g., in gate and backyard).
Tab <ref> provides detailed results also in terms of precision and recall for each sequence pairs, including the number of query views shared by each camera. We can observe how different types of motion pairs can result in performance that can either accurately recognise the view overlaps and remove false positives or invalidating all query views despite being true correspondences (with the overlap threshold set at 50%). There are challenging cases where the variants of the framework achieve a very low precision or recall, e.g., gate 3$|$4, courtyard 1$|$2, courtyard 1$|$3, and courtyard 1$|$4, but the accuracy is about 50% or higher, showing that the frameworks can correctly recognise that there are no views in overlap for given query views. Overall, any of the variants of the framework recognises view overlaps that are not true correspondences and fails to identify many view overlaps in many sequences as we can see from median precision and recall that are lower than 50% for most of the sequence pairs except for some of the scenario gate.  Nevertheless, D-DBoW  is a competitive option for cross-camera view-overlap recognition with higher median precision on average when using RANSAC, and higher median recall on average when using MAGSAC++.

D-DBoW     D-RootSIFT     D-SuperGlue

Comparison of matched view features between D-DBoW, D-RootSIFT, and D-SuperGlue for selected query views from the sequence pairs C2$|$4, G1$|$3, B1$|$4, C2$|$3, O1$|$2, G2$|$3 (from top to bottom). Note that we show inliers (matched local features validated after RANSAC/MAGSAC++) as green lines and outliers as red circles. KEY – C: courtyard; B: backyard; G: gate; O: office.

Fig. <ref> shows the matched views and matched local features (inliers after RANSAC or MAGSAC++) between D-DBoW, D-RootSIFT, and D-SuperGlue for sampled query views. As both D-RootSIFT and D-SuperGlue are using NetVLAD as view feature, the matched view is the same for both strategies, whereas D-DBoW can matched a different past views across cameras. Despite identifying a matched view, the framework can fail to match local features when the viewpoint or distance between the query and matched views increases (e.g., $>30^\circ$), showing the drop in recall discussed earlier. For example, we can observe this situation for D-DBoW and D-RootSIFT in the first two rows and the fourth row.
On the contrary and as expected, the number of matched local features increases as the viewpoint is more similar between the query and the matched view (e.g., $<15^\circ$) if view overlaps are recognised. Moreover, the framework, independently of the strategy, tends to match local features in flat and localised areas in the outdoor scenes gate, courtyard, and backyard, and on the corner of different objects in the office scenario. However, if $|\mathcal{M}_{m,q}| < \rho$ or the epipolar constraint is not satisfied (e.g, all the local features are co-planar), then the matched view $m$ is not validated.
D-SuperGlue shows to match a higher number of local features compared to D-DBoW and D-RootSIFT, but it can also match wrong local features, as shown in courtyard (fourth row).

§ CONCLUSION

We presented a decentralised framework to recognise overlapping views across freely moving cameras. The framework is modular and can be combined with various view and local features, and does not require a 3D map or any prior information about the environment.
Through coarse-to-fine matching and  geometric validation, the framework  can recognise overlaps in past views of the other camera and  improve the average accuracy  by discarding wrongly matched views.
As future work, we will consider the scalability of the framework with more than two cameras and robustness with a lossy communication channel.

[1]
Arandjelović, R., Gronat, P., Torii, A., Pajdla, T., Sivic, J.:
NetVLAD: CNN architecture for weakly supervised place recognition. IEEE
Trans. Pattern Anal. Mach. Intell.  40(6),  1437–1451 (Jun 2018)

[2]
Arandjelović, R., Zisserman, A.: Three things everyone should know to
improve object retrieval. In: Proc. IEEE Conf. Comput. Vis. Pattern
Recognit. Providence, RI, USA (16–21 Jun 2012)

[3]
Balntas, V., Tang, L., Mikolajczyk, K.: Binary online learned descriptors. IEEE
Trans. Pattern Anal. Mach. Intell.  40(3),  555–567 (Mar 2018)

[4]
Barath, D., Noskova, J., Ivashechkin, M., Matas, J.: MAGSAC++, a fast,
reliable and accurate robust estimator. In: Proc. IEEE Conf. Comput. Vis.
Pattern Recognit. Virtual (16–18 Jun 2020)

[5]
Calonder, M., Lepetit, V., Strecha, C., Fua, P.: BRIEF: Binary robust
independent elementary features. In: Proc. Eur. Conf. Comput. Vis. Heraklion,
Crete, Greece (5–11 Sep 2010)

[6]
Camposeco, F., Cohen, A., Pollefeys, M., Sattler, T.: Hybrid scene compression
for visual localization. In: Proc. IEEE Conf. Comput. Vis. Pattern Recognit.
Long Beach, CA, USA (16–20 Jun 2019)

[7]
Cieslewski, T., Choudhary, S., Scaramuzza, D.: Data-efficient decentralized
visual SLAM. In: Proc. IEEE Int. Conf. Robotics Autom. Brisbane, Australia
(21–25 May 2018)

[8]
Cieslewski, T., Scaramuzza, D.: Efficient decentralized visual place
recognition using a distributed inverted index. IEEE Robotics Autom. Lett.
2(2),  640–647 (Apr 2017)

[9]
DeTone, D., Malisiewicz, T., Rabinovich, A.: SuperPoint: Self-supervised
interest point detection and description. In: Proc. IEEE Conf. Comput. Vis.
Pattern Recognit. Workshops. Salt Lake City, UT, USA (18–22 Jun 2018)

[10]
Dusmanu, M., Rocco, I., Pajdla, T., Pollefeys, M., Sivic, J., Torii, A.,
Sattler, T.: D2-Net: A trainable CNN for joint description and detection
of local features. In: Proc. IEEE Conf. Comput. Vis. Pattern Recognit. Long
Beach, CA, USA (16–20 Jun 2019)

[11]
Fischler, M.A., Bolles, R.C.: Random sample consensus: A paradigm for model
fitting with applications to image analysis and automated cartography.
Communications of the ACM  24(6),  381–395 (Jun 1981)

[12]
Forster, C., Lynen, S., Kneip, L., Scaramuzza, D.: Collaborative monocular
SLAM with multiple micro aerial vehicles. In: Proc. IEEE Int. Conf. Intell.
Robot Syst. Tokyo, Japan (3–7 Nov 2013)

[13]
Gálvez-López, D., Tardos, J.D.: Bags of binary words for fast place
recognition in image sequences. IEEE Trans. Robotics  28(5),
1188–1197 (2012)

[14]
Gao, X., Wang, R., Demmel, N., Cremers, D.: LDSO: Direct sparse odometry with
loop closure. In: Proc. IEEE Int. Conf. Intell. Robot Syst. Madrid, Spain
(1–5 Oct 2018)

[15]
Garcia-Fidalgo, E., Ortiz, A.: iBoW-LCD: An appearance-based loop-closure
detection approach using incremental bags of binary words. IEEE Robotics
Autom. Lett.  3(4),  3051–3057 (Oct 2018)

[16]
Geiger, A., Lenz, P., Urtasun, R.: Are we ready for autonomous driving? The
KITTI vision benchmark suite. In: Proc. IEEE Conf. Comput. Vis. Pattern
Recognit. Providence, RI, USA (16–21 Jun 2012)

[17]
Hartley, R., Zisserman, A.: Multiple View Geometry in Computer
Vision. Cambridge University Press, second edn. (2003)

[18]
Jegou, H., Perronnin, F., Douze, M., Sánchez, J., Perez, P., Schmid, C.:
Aggregating local image descriptors into compact codes. IEEE Trans. Pattern
Anal. Mach. Intell.  34(9),  1704–1716 (Sep 2012)

[19]
Krizhevsky, A.: Learning multiple layers of features from tiny images. Tech.
rep., University of Toronto (Apr 2009)

[20]
Leutenegger, S., Chli, M., Siegwart, R.Y.: BRISK: Binary robust invariant
scalable keypoints. In: Proc. IEEE Int. Conf. Comput. Vis. Barcelona, Spain
(6–13 Nov 2011)

[21]
Lin, K., Lu, J., Chen, C., Zhou, J., Sun, M.: Unsupervised deep learning of
compact binary descriptors. IEEE Trans. Pattern Anal. Mach. Intell.
41(6),  1501–1514 (Jun 2019)

[22]
Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int.
J. Comput. Vis.  60(2),  91–110 (Nov 2004)

[23]
Moulon, P., Monasse, P., Perrot, R., Marlet, R.: OpenMVG: Open multiple view
geometry. In: Int. Workshop Reproducible Research Pattern Recognit. Cancún,
Mexico (4 Dec 2016)

[24]
Mur-Artal, R., Montiel, J., Tardos, J.: ORB-SLAM: A versatile and accurate
monocular SLAM system. IEEE Trans. Robotics  31(5),  1147–1163
(Oct 2015)

[25]
Mur-Artal, R., Tardós, J.D.: Fast relocalisation and loop closing in
keyframe-based SLAM. In: Proc. IEEE Int. Conf. Robotics Autom. Hong Kong,
China (31 May5 Jun 2014)

[26]
Nister, D., Stewenius, H.: Scalable recognition with a vocabulary tree. In:
Proc. IEEE Conf. Comput. Vis. Pattern Recognit. New York, NY, USA (17–22 Jun

[27]
Perronnin, F., Liu, Y., Sánchez, J., Poirier, H.: Large-scale image retrieval
with compressed Fisher vectors. In: Proc. IEEE Conf. Comput. Vis. Pattern
Recognit. San Francisco, CA, USA (Jun 2010)

[28]
Riazuelo, L., Civera, J., Montiel, J.M.M.: C$^2$TAM: A cloud framework for
cooperative tracking and mapping. Robotics and Autonomous Systems
62(4),  401 – 413 (Apr 2014)

[29]
Rublee, E., Rabaud, V., Konolige, K., Bradski, G.: ORB: An efficient
alternative to SIFT or SURF. In: Proc. IEEE Int. Conf. Comput. Vis.
Barcelona, Spain (6–13 Nov 2011)

[30]
Sarlin, P., DeTone, D., Malisiewicz, T., Rabinovich, A.: SuperGlue: Learning
feature matching with graph neural networks. In: Proc. IEEE Conf. Comput.
Vis. Pattern Recognit. Virtual (16–18 Jun 2020)

[31]
Sarlin, P.E., Cadena, C., Siegwart, R., Dymczyk, M.: From coarse to fine:
Robust hierarchical localization at large scale. In: Proc. IEEE Conf. Comput.
Vis. Pattern Recognit. Long Beach, CA, USA (16–20 Jun 2019)

[32]
Sattler, T., Maddern, W., Toft, C., Torii, A., Hammarstrand, L., Stenborg, E.,
Safari, D., Okutomi, M., Pollefeys, M., Sivic, J., Kahl, F., Pajdla, T.:
Benchmarking 6DoF outdoor visual localization in changing conditions. In:
Proc. IEEE Conf. Comput. Vis. Pattern Recognit. Salt Lake City, UT, USA
(18–22 Jun 2018)

[33]
Schlegel, D., Grisetti, G.: HBST: A Hamming distance embedding binary
search tree for feature-based visual place recognition. IEEE Robotics Autom.
Lett.  3(4),  3741–3748 (Oct 2018)

[34]
Schmuck, P., Chli, M.: CCM-SLAM: Robust and efficient centralized
collaborative monocular simultaneous localization and mapping for robotic
teams. J. of Field Robotics  36(4),  763–781 (Jun 2019)

[35]
Schönberger, J.L., Frahm, J.: Structure-from-motion revisited. In: Proc.
IEEE Conf. Comput. Vis. Pattern Recognit. Las Vegas, NV, USA (27–30 Jun

[36]
Sivic, J., Zisserman, A.: Video Google: A text retrieval approach to object
matching in videos. In: Proc. IEEE Int. Conf. Comput. Vis. Nice, France
(11–17 Oct 2003)

[37]
Sturm, J., Engelhard, N., Endres, F., Burgard, W., Cremers, D.: A benchmark for
the evaluation of RGB-D SLAM systems. In: Proc. IEEE Int. Conf. Intell.
Robot Syst. Vilamoura-Algarve, Portugal (7–12 Oct 2012)

[38]
Torii, A., Sivic, J., Okutomi, M., Pajdla, T.: Visual place recognition
with repetitive structures. IEEE Trans. Pattern Anal. Mach. Intell.
37(11),  2346–2359 (Mar 2015)

[39]
Tsintotas, K.A., Bampis, L., Gasteratos, A.: Probabilistic
appearance-based place recognition through bag of tracked words. IEEE
Robotics Autom. Lett.  4(2),  1737–1744 (Apr 2019)

[40]
Xompero, A., Lanz, O., Cavallaro, A.: A spatio-temporal multi-scale binary
descriptor. IEEE Trans. Image Process.  29,  4362–4375 (Jan 2020)

[41]
Zou, D., Tan, P.: CoSLAM: Collaborative visual SLAM in dynamic
environments. IEEE Trans. Pattern Anal. Mach. Intell.  35(2),
354–366 (Feb 2013)