#  Enhancing Person Re-Identification via
Uncertainty Feature Fusion and Wise Distance Aggregation

Quang-Huy Che<EMAIL_ADDRESS>Le-Chuong Nguyen<EMAIL_ADDRESS>Vinh-
Tiep Nguyen<EMAIL_ADDRESS>Multimedia Communications Laboratory
University of Information Technology, Ho Chi Minh City, Vietnam  Vietnam
National University, Ho Chi Minh City, Vietnam.

###### Abstract

The quest for robust Person re-identification (Re-ID) systems capable of
accurately identifying subjects across diverse scenarios remains a formidable
challenge in surveillance and security applications. This study presents a
novel methodology that significantly enhances Person Re-Identification (Re-ID)
by integrating Uncertainty Feature Fusion (UFFM) with Wise Distance
Aggregation (WDA). Tested on benchmark datasets - Market-1501, DukeMTMC-ReID,
and MSMT17 - our approach demonstrates substantial improvements in Rank-1
accuracy and mean Average Precision (mAP). Specifically, UFFM capitalizes on
the power of feature synthesis from multiple images to overcome the
limitations imposed by the variability of subject appearances across different
views. WDA further refines the process by intelligently aggregating similarity
metrics, thereby enhancing the system’s ability to discern subtle but critical
differences between subjects. The empirical results affirm the superiority of
our method over existing approaches, achieving new performance benchmarks
across all evaluated datasets. Code is available on Github.

###### keywords:

Person re-identification, Uncertainty Feature Fusion, Wise Distance
Aggregation

††journal:

top=1.1in, bottom=1.5in, left=0.7in, right=0.7in

##  1 Introduction

The aim of person re-identification (Re-ID) is to accurately recognize
individuals across various non-overlapping camera views, a pivotal task in
large-scale surveillance systems. Person Re-ID is challenged by several
factors, including cluttered backgrounds, variable lighting, changes in
posture, and obstructions, which commonly hinder identification. Specifically,
an individual may be captured by multiple cameras within these systems,
appearing across different frames as they move. Traditional retrieval
solutions predominantly employ deep metric learning, wherein models are
rigorously trained to convert images into vector representations. This
training ensures that samples from the same category closely resemble each
other in the vector space.

Figure 1: t-SNE visualization of image features [1]. Seven individuals from
the DukeMCMT-ReID dataset are randomly selected for querying and are
represented by different colors. Within a cluster comprising the top 20 query
results, pentagons indicate the query images, dots of the same color as the
pentagons represent correct query results, while gray dots indicate incorrect
results.

In the querying phase, the distance between the query vector and all vectors
in the database is computed, allowing for the retrieval of the most similar
objects based on these distances. ReID has captured increasing attention
within the computer vision community, owing to its significance for video
surveillance and the diverse methodologies proposed in recent times. A
distinctive feature of ReID is the acquisition of images from multiple
cameras, which introduces significant intra-class variance and a high degree
of inter-class distance. Despite the considerable progress achieved through
deep learning and large-scale public datasets, ReID remains a challenging
task.

Figure 2: Examples of images of the same individual from different camera
angles in the Market-1501 and DukeMTMC-ReID datasets.

To enhance ReID efficacy, recent studies have optimized the learned feature
embedding space and corresponding distance metrics using two concurrent
methodologies: representation learning [2, 3, 4] and metric learning [5, 6,
7]. In essence, these methodologies generate a unique representation for each
image to facilitate distance computations. However, this approach is
inherently limited as the features may only encapsulate specifics of that
particular image. Consequently, in situations involving different poses,
camera angles, or occlusions, objects belonging to the same class may exhibit
low distance. The disparity in the display areas of images captured from
different camera views means that certain view-specific signals generated in
one image may not be present in another, despite both images depicting the
same subject. More specifically, a query image captured from a pedestrian’s
front might be assigned a high distance to the front view of a different
pedestrian wearing a similar color outfit, rather than to an image of the
queried individual taken from behind, possibly carrying a backpack. This
challenge is depicted in several examples in Figure 2. Therefore, the detailed
features extracted from images are not robust in ReID tasks involving subjects
captured from varying angles.

In ReID problems, the representation of an object through multiple images has
prompted researcher [7] to investigate the use of class-specific images to
identify a representative image. This strategy shifts emphasis from the
specific attributes of individual objects to more generalized features that
represent the object across various images. Despite the novelty of this
method, identifying a representative for images within the same class for
synthesis poses a significant challenge. The difficulty stems from collecting
images through different cameras, complicating the process of identifying
which images pertain to the same subject, thereby complicating the
determination of a class representative. This challenge emphasizes the need
for advanced methods capable of enhancing the effectiveness of techniques to
identify representative individuals for each class in Re-ID tasks. This, in
turn, facilitates the exploitation of specific features and general
representations of the subject based on images captured from different
cameras.

The utilization of images of the subject provides detailed features after
extraction by the ReID model, while generalized features from the
representative images of the same class convey traits from various
perspectives. Employing each type of feature presents advantages and
compromises when computing the distance to the feature of the query image.
This introduces a research avenue on how to combine distances to leverage
their benefits optimally. A straightforward integration of distances [8, 9,
10] can be achieved by directly summing the similarities or manually assigning
weights during the aggregation process. However, this simplistic approach does
not always yield effective results in complex scenarios. Strategically
selecting weights for aggregation to optimize efficiency theoretically can
harness the strengths of various methodologies.

In summary, the contributions of this paper are as follows:

  * 1.

We introduce the Uncertainty Feature Fusion Method (UFFM), a novel approach to
feature synthesis that does not require labeled data for objects within the
same class.

  * 2.

We propose the Wise Distance Aggregation method, capable of integrating
various distances, including those from detailed features and generalized
features, to capitalize on their strengths.

  * 3.

We demonstrate that our methods achieves state-of-the-art performance on
multiple person ReID datasets.

The rest of the article is presented as follows: We review related
methodologies in Section 2 to understand the advantages and limitations in
Person ReID tasks. Our approach includes two main contributions: the
Uncertainty Feature Fusion Method (UFFM) and Wise Distance Aggregation (WDA),
which are detailed in Section 3. In Section 4, we evaluate our proposals on
Person ReID datasets including Market-1501, DukeMTMC-ReID, and MSMT17. In the
final Section 5, we draw some conclusions and suggest directions for future
research.

Figure 3: The process of calculating the distance between a query object and
an object in the gallery. The query object is captured at Camera 5, while the
object in the gallery is captured at Camera 6. Both objects are extracted
through a common backbone to calculate the distance $\textbf{D}_{dt}$.
Subsequently, the feature of the object in the gallery is subjected to the
Uncertainty Feature Fusion Method to compute the Uncertainty-Refined Feature.
Using the newly calculated Uncertainty-Refined Feature and the Detailed
Feature of the query object, we calculate $\textbf{D}_{ag}$. By combining
$\textbf{D}_{dt}$, $\textbf{D}_{ag}$, and $\textbf{D}_{cce}$ information of
both objects, we apply the WDA method to aggregate distances.

##  2 Related Work

###  2.1 Person Re-Identification

In the rapidly advancing context of Convolutional Neural Networks (CNNs),
underscored by landmark achievements on ImageNet, ResNet-50 [11] has swiftly
emerged as the architecture of choice for research and applications in the
field of person re-identification (ReID), as widely recognized in numerous
studies [12, 13]. The advent of the CLIP-ReID model [14], leveraging the
ResNet-50 encoder from CLIP [15] instead of relying on pre-training on
ImageNet, has set new benchmarks for tackling ReID challenges through a
bifurcated approach aimed at enhancing the quality of visual representation.
In the pursuit of exploiting local feature utilization, the study on the PCB
model [2] offered profound insights into segmenting the feature map into
horizontal partitions to effectively capture local features. Concurrently,
initiatives such as SAN [3] and TBE-Net [4] have underscored the advantage of
amalgamating both local and global features to augment identification
performance. Despite the effective re-identification of individuals across
disparate cameras by previous models, challenges in image identification
amidst obstructions prompted the introduction of specialized modules aimed at
distinguishing between body parts and personal belongings, thereby augmenting
recognition capability in complex scenarios [16, 17]. Moreover, the deployment
of a teacher-student framework in select studies [18, 19, 20] as a pivotal
strategy has proven efficacious in refining deep learning processes, thereby
elevating the precision and efficiency of object re-identification models.
Additionally, the genesis of innovative loss functions [5][6] has played a
crucial role in facilitating the convergence of identical ID features,
enabling efficient adaptation across a spectrum of ReID models. Advanced data
sampling methods, such as OfM [21] and QAConv-GS [22], have markedly improved
the generalization capability and efficiency of the deep learning process,
optimizing training methodologies and elevating the accuracy of ReID models.

Existing methodologies primarily focus on enhancing the model’s feature
extraction capability for individual objects without harnessing the potential
of combining features from multiple objects to create a composite feature rich
in information. In this vein, research [7] proposed utilizing the class
centroid by averaging the features of all objects within a class to represent
the collective features of that class. Although this approach has proven
effective in terms of query time and accuracy, the task of identifying which
objects from different cameras belong to a single class poses a significant
challenge in re-identification scenarios involving images captured from
diverse cameras. Conversely, another body of research [23] suggested
generating multi-view features by integrating features from several fixed
cameras. This proposed method employs a Graph Neural Network (GNN) to identify
neighboring features without necessitating the labels of objects belonging to
the same class. However, while proposing a method for synthesizing multi-view
features to enhance accuracy, the algorithmic implementation inadvertently
incorporates information from the query object’s camera, contravening the
cross-view matching setting.

In this study, we propose a novel method to synthesize features without the
prerequisite of label information of objects during the synthesis process.
Moreover, our synthesis method can be seamlessly applied to pre-trained models
without necessitating re-training. This attribute renders our method highly
applicable in real-world scenarios, extending its utility across various
domains.

###  2.2 Similarity aggregation

Integrating diverse similarity measures enhances re-identification accuracy by
leveraging the strengths of individual similarity measures. In the study by
[10], the integration of similarity metrics from various image features for
content-based image retrieval was predicated on the premise that synthesizing
information from multiple sources could more accurately reflect the semantic
similarity between images. The research [24] proposed combining cosine
similarity and Earth Mover’s Distance (EMD) [8] for face recognition,
employing a two-stage process where initial cosine distance filters faces,
followed by EMD ranking on the top K filtered results. This method improved
recognition accuracy, particularly with non-uniform face distributions. VOC-
ReID [9] introduced orientation and camera-based adjustments to refine vehicle
ReID, utilizing manual weight adjustments through experimentation,
demonstrating efficacy despite potential optimization issues. However, a
limitation of these approaches is the manual selection of combination weights,
which cannot optimize combined performance optimally. In 2023, Q. Huy et al.
[25] proposed combining cosine similarity with the Centroid measure [7] in the
task of face recognition. Instead of manually selecting combination weights,
the study proposed using feature division by triplets and incorporating a
machine learning model to generate combination weights. While this approach
yielded some promising initial results, the analysis within the study remained
underexplored, leaving the applicability to the re-identification problem
unclear. In this study, we elucidate the potential of combining features in
the re-identification task, introducing a method we term Wise Similarity
Aggregation (WSA).

##  3 Proposed Method

The proposed components of our work, when integrated with the ReID model, are
depicted in Figure 3. My primary contributions include the Uncertainty Feature
Fusion Method (UFFM) and Wise Distance Aggregation (WDA), both of which are
involved only in the execution process of the ReID model without affecting its
training phase. The Uncertainty Feature Fusion Method will be discussed in
greater detail in Section 3.1, while Section 3.2 focuses on explaining the
Wise Distance Aggregation method.

###  3.1 Uncertainty Feature Fusion Method (UFFM)

Our methodology adeptly fuses diverse features by leveraging analogous
characteristics. Directly synthesizing features from the same category into a
novel composite feature presents challenges, as it requires navigating the
absence of additional labels for images sourced from multiple cameras.
Therefore, our strategy capitalizes on the distribution within the learned
embedding space to fuse features effectively. Under the assumption that the
model is adeptly trained, patterns with identical identifying features will be
clustered together, while those with disparate identifying features will be
distinctly segregated. This proposed is termed Uncertainty Feature Fusion
Method (UFFM), aligning with scientific discourse for enhanced clarity and
precision.

We utilize the feature extractor $E_{\Theta}$, trained on the dataset Dtrain,
to map an image $I$ to a vector $f\in\mathbb{R}^{d}$, $f$ = $E_{\Theta}(I)$.
Utilizing $f$, we identify the K-nearest neighbors, denoted as $F^{nn}$ =
$\\{f_{k}^{nn}\\}_{k=1}^{K}$, where each $f_{k}^{nn}$ $\in\mathbb{R}^{d}$.
Although the selection of $K$ is unsupervised, an optimally structured
embedding space will still exhibit neighboring instances within the same class
as $f$. In this case, incorrect results may introduce noise to the fusion
features, but correct results will enhance the information in the fusion
features, as shown in Figure 4. To fuse features from neighboring features, a
simple averaging approach is not always effective due to variability in
distance between $f$ and $f_{k}^{nn}$. Therefore, we advocate for a weighted
synthesis approach, where distance play a crucial role in determining the
contribution of each neighbor to the composite feature. Leveraging the list of
K-nearest neighbors $F^{nn}$, we generate a corresponding list of cosine
distances.

Figure 4: An example of finding K nearest neighbors based on features, with
K=7. The black box represents the query object, green boxes indicate correct
results, while red boxes denote incorrect results.

$\textbf{D}^{nn}=\\{d_{k}^{nn}\\}_{k=1}^{K},\quad
d^{nn}_{k}={dst}(f,f^{nn}_{k})$ (1)

Upon obtaining the distance list $\textbf{D}^{nn}$, weights are computed in
relation to each distance, thereby delineating the contribution level of
neighboring features into our feature synthesis process. The equation below
elucidates the method of merging features based on their assigned weights,
where Uncertainty-Refined Feature, $f^{URF}$ represents the fusion feature
based on $f$:

$f^{URF}=\sum_{k=1}^{K}w_{k}\boldsymbol{\cdot}f^{nn}_{k},\quad
w_{k}=\frac{d^{nn}_{i}}{\sum_{i=1}^{K}d^{nn}_{i}}$ (2)

This ensures that the contribution of each neighboring feature is
proportionate to its distance to $f$, facilitating a weighted synthesis that
captures the most relevant and significant features from the nearest
neighbors.

In the evaluation phase of most Re-Identification (ReID) problems, the cross-
view matching setting is commonly applied. Initially, we define $\mathcal{Q}$
= $\\{q_{i}\\}_{i=1}^{N_{q}}$ and $\mathcal{G}$ = $\\{g_{j}\\}_{j=1}^{N_{g}}$
as the query set and the gallery set, respectively. Traditionally, for a query
$q_{i}$, it is required to compute the distances with each $g_{j}$ $\in$
$\mathcal{G}$ to obtain a distance list $List$(qi,
$\mathcal{G}$)=$\\{\textbf{D}_{dt}(q_{i},g_{j})\\}_{j=1}^{N_{g}}$. Then,
$List$(qi, $\mathcal{G}$) is sorted, excluding objects with both the same ID
and camera from the result list. This implies that valid results are those
captured by a different camera than that of the query. To ensure the cross-
view matching setting in the feature synthesis process with our method, we
address this by selecting only the $K$ features captured by a different camera
than that of the feature under consideration, or Camera$(f^{nn}_{i})$ $\neq$
Camera$(f)$. This approach ensures that the fusion feature does not carry the
characteristics of the query, thus maintaining fairness when using the cross-
view matching setting in the evaluation process. After applying the
Uncertainty Feature Fusion Method (UFFM) method on each gj in the gallery set,
resulting in $List$(qi,
$\mathcal{G}$)=$\\{\textbf{D}_{ag}(q_{i},g_{j})\\}_{j=1}^{N_{g}}$.

###  3.2 Wise Distance Aggregation

Table 1: Comparison of our method with person ReID SOTA methods. The best and second best results are marked in bold and underline respectively. Methods | Market-1501 | DukeMTMC-ReID | MSMT17
---|---|---|---
Rank@1 | mAP | Rank@1 | mAP | Rank@1 | mAP
BoT [12] (WCVPR’19) | 94.5 | 85.9 | 86.4 | 76.4 | 74.1 | 50.2
CLIP-ReID Baseline [14](AAAI’22) | 94.7 | 88.1 | 88.6 | 79.3 | 82.1 | 60.7
ISP [16] (ICCV’20) | - | - | 88.7 | 78.9 | - | -
HG [18] (BMVC’21) | 95.6 | 86.1 | 87.1 | 77.5 | - | -
PAT [26](CVPR’21) | 95.4 | 88.0 | 88.8 | 78.2 | - | -
CAL [27](ICCV’21) | 94.5 | 87.0 | 87.2 | 76.4 | 79.5 | 56.2
OfM [21](AAAI’21) | 94.9 | 87.9 | 89.0 | 78.6 | - | -
MGN+CircleLoss [5](AAAI’21) | 96.1 | 87.4 | - | - | 76.9 | 52.1
DRL-Net [28] (TMM’22) | 94.7 | 86.9 | 88.1 | 76.6 | 78.4 | 55.3
QAConv-GS [22](CVPR’22) | 91.6 | 75.5 | - | - | 79.1 | 49.5
CLIP-ReID [14](AAAI’22) | 95.7 | 89.8 | 90.0 | 84.7 | 84.4 | 63.0
AGW [29](TPAMI’22) | 95.1 | 87.8 | 89.0 | 79.6 | 68.3 | 49.3
FastReID [13](ACM MM’23) | 95.4 | 88.2 | 89.6 | 79.8 | 83.3 | 59.9
BPBreID [17](WACV’23) | 95.1 | 87.0 | 89.6 | 78.3 | - | -
BoT + Our | 96.2 | 91.0 | 90.3 | 83.1 | 82.0 | 62.3
CLIP-ReID Baseline + Our | 96.1 | 92.0 | 91.3 | 85.0 | 83.8 | 67.6

In multi-view camera systems, feature usage has proven effective for
generalizing and consistently identifying object characteristics across varied
observational environments. However, this method’s significant limitation is
its potential to overlook complex image details, thereby impeding the accurate
and comprehensive identification of an object’s unique attributes. To address
this challenge, this study introduces an advanced technique aimed at
synthesizing and optimizing information from two types of distances: distance
to Detailed Feature ($\textbf{D}_{dt}$) and distance to Uncertainty-Refined
Feature ($\textbf{D}_{ag}$). Rather than manually combining features with
predetermined weights, we propose an intelligent method for aggregating
distances, utilizing the cosine distance metric, which ranges from -1 to 1.
The expected form of the aggregated distance is:

$\textbf{D}=\alpha\boldsymbol{\cdot}\textbf{D}_{dt}+\beta\boldsymbol{\cdot}\textbf{D}_{ag}$
(3)

We define Ddt and Dag as inputs for the Linear Regression model, with the
expected output being 1 if the ID$q_{i}$ matches ID${g_{j}}$ and -1 otherwise.
We generate data for the linear regression model by randomly selecting
triplets: anchor (xa), positive (xp), and negative (xn). For each pair (xa và
xp), we create a data point for the training set labeled as 1, and for (xa và
xn), labeled as -1. After training, the model take the form similar to
Equation 4.

$\textbf{D}=w_{2}\boldsymbol{\cdot}\textbf{D}_{dt}+w_{1}\boldsymbol{\cdot}\textbf{D}_{ag}+w_{0}$
(4)

where, w2 và w1 denote the weights assigned to two types of distances. This
approach facilitates the integration of the strengths associated with
different distances. The detailed procedure is elaborated in Algorithm 1. To
maintain independence during the evaluation phase, only the training dataset
$\textbf{D}_{train}$, which includes known IDs for images from distinct
cameras, is utilized in the data preparation process. Rather than employing
uncertain feature fusion method to generate the uncertainty-refined feature
$f^{URF}$ within the $\textit{D}_{train}$ dataset, which encompasses IDs for
subjects observed through various cameras. This research exploits the
available data to fuse multi-view features via a method referred to as Certain
Feature Fusion Method (CFFM), where Refined Feature, $f^{RF}$ is delineated in
Equation 5. With $F^{cnn}$ = $\\{f_{k}^{cnn}\\}_{k=1}^{K}$ being the set of
features that are most similar to $f$ and of the same class as $f$. This
ensures that the features within $F^{cnn}$ are of the same class as $f$.

$f^{RF}=\frac{1}{K}\sum_{k=1}^{K}f^{cnn}_{k}$ (5)

Furthermore, to enhance the distinctiveness of features derived from different
camera views, we adapt the Cross-Camera Encouragement (CCE) technique,
initially proposed by Lin et al. [30], modifying its application to better
suit our methodology. The revised CCE formula is integrated into our composite
distance calculation, thereby enriching the feature discrimination capability
essential for reliable object re-identification:

Input: $N\geq 0$ ;

// Number of iterations

//Initialization step

$i\leftarrow 1$;

$D_{\text{train}}\leftarrow\emptyset$;

//Loop to prepare data

for  _$i\leftarrow 1$ to $N$_ do

//Extract feature step

$x_{a},x_{p},x_{n}\leftarrow D_{\text{train}}$;

$f_{a},f_{p},f_{n}\leftarrow{E}_{\Theta}(x_{a},x_{p},x_{n})$;

//Prepare positive data

$f^{RF}_{p}\leftarrow\textnormal{{CFFM}}(f_{p})$;

$\textbf{D}_{dt}^{p}\leftarrow\textnormal{{dst}}(f_{a},f_{p})$;

$\textbf{D}_{ag}^{p}\leftarrow\textnormal{{dst}}(f_{a},f^{RF}_{p})$;

$\textbf{D}_{cce}^{p}\leftarrow\textnormal{{CCE}}(x_{a},x_{p})$;

//Prepare negative data

$f^{RF}_{n}\leftarrow\textnormal{{CFFM}}(f_{n})$;

$\textbf{D}_{dt}^{n}\leftarrow\textnormal{{dst}}(f_{a},f_{n})$;

$\textbf{D}_{ag}^{n}\leftarrow\textnormal{{dst}}(f_{a},f^{RF}_{n})$;

$\textbf{D}_{cce}^{n}\leftarrow\textnormal{{CCE}}(x_{a},x_{n})$;

//Put the data into the training set of the model

$D_{\text{train}}\leftarrow
D_{\text{train}}\cup\bigg{\\{}\Big{[}\textbf{D}_{dt}^{p},\textbf{D}_{ag}^{p},\textbf{D}_{cce}^{p}\Big{]},1\bigg{\\}}$;

$D_{\text{train}}\leftarrow
D_{\text{train}}\cup\bigg{\\{}\Big{[}\textbf{D}_{dt}^{n},\textbf{D}_{ag}^{n},\textbf{D}_{cce}^{n}\Big{]},-1\bigg{\\}}$;

Algorithm 1 Triplet Data Generation Algorithm

$\begin{split}\textbf{D}_{cce}&=CCE(x_{a},x_{b})=\begin{cases}1,&\text{$c_{a}=c_{b}$}\\\
0,&\text{$c_{a}\neq c_{b}$}\end{cases}\end{split}$ (6)

To align with our methodological approach more closely, we incorporate
$\textbf{D}_{cce}$ into the composite distance calculation, which can be
substituted in the detailed distance formula as seen in Equation 3:

D
$\displaystyle=\alpha\boldsymbol{\cdot}\textbf{D}_{dt}+\beta\boldsymbol{\cdot}\textbf{D}_{ag}+\gamma\boldsymbol{\cdot}\textbf{D}_{cce}$
(7)

At this juncture, the aggregate distance can be optimized through the
employment of a Linear Regression model, as delineated in Equation 4 with
($\alpha$, $\beta$, $\gamma$) are weight of model.

##  4 Experimental Results

Table 2: Evaluation of our proposed methods with BoT Baseline and CLIP-ReID Baseline Baseline | UFFM | WDA | Market-1501 | DukeMTMC-ReID | MSMT17
---|---|---|---|---|---
Rank@1 | mAP | Rank@1 | mAP | Rank@1 | mAP
BoT Baseline |  |  | 94.5 | 85.9 | 86.4 | 76.4 | 74.1 | 50.2
2-9 | ✔ |  |  | 96.1
---
$\uparrow$ 1.6
| 91.2
---
$\uparrow$ 5.3
| 90.2
---
$\uparrow$ 3.8
| 83.3
---
$\uparrow$ 6.9
| 81.5
---
$\uparrow$ 7.4
| 61.8
---
$\uparrow$ 11.6
2-9 | ✔ | ✔ |  | 96.2±0.05
---
$\uparrow$ 1.7
| 91.0±0.1
---
$\uparrow$ 5.1
| 90.3±0.09
---
$\uparrow$ 3.9
| 83.1±0.13
---
$\uparrow$ 6.7
| 82.0±0.24
---
$\uparrow$ 7.9
| 62.3±0.3
---
$\uparrow$ 12.1
CLIP-ReID Baseline |  |  | 94.7 | 88.1 | 88.6 | 79.3 | 82.1 | 60.7
2-9 | ✔ |  |  | 95.7
---
$\uparrow$ 1.0
| 92.0
---
$\uparrow$ 3.9
| 90.7
---
$\uparrow$ 2.1
| 84.8
---
$\uparrow$ 5.5
| 83.4
---
$\uparrow$ 1.3
| 67.7
---
$\uparrow$ 7.0
2-9 | ✔ | ✔ |  | 96.1±0.13
---
$\uparrow$ 1.4
| 92.0±0.05
---
$\uparrow$ 3.9
| 91.3±0.45
---
$\uparrow$ 2.7
| 85.0 ±0.18
---
$\uparrow$ 5.7
| 83.8±0.36
---
$\uparrow$ 1.7
| 67.6±0.55
---
$\uparrow$ 6.9

###  4.1 Datasets and Settings

####  4.1.1 Datasets

We conducted training and evaluation of the proposed model on three Person
ReID datasets: Market1501[31], DukeMTMC-ReID[32] and MSMT17[33]:

  * 1.

Market-1501[31]: The Market-1501 dataset was collected using six cameras
positioned in front of a supermarket at Tsinghua University. It comprises a
total of 32,668 images belonging to 1,501 individuals. The dataset is divided
into 12,936 training images with 751 individuals and 19,732 testing images
with 750 individuals. Among the test images, 3,368 are selected as the query
set.

  * 2.

DukeMTMC-ReID[32]: The DukeMTMC-ReID dataset, a subset of DukeMTMC, is
tailored for image-based person re-identification. Captured from high-
resolution videos by eight cameras, it’s a significant pedestrian image
dataset. Images are meticulously cropped using hand-drawn bounding boxes. It
comprises 16,522 training images covering 702 identities, 2,228 query images
of the other 702 identities and 17,661 gallery images.

  * 3.

MSMT17[33]: MSMT17 is a comprehensive multi-scene multi-time person re-
identification dataset. It includes 180 hours of video footage captured by 12
outdoor cameras and 3 indoor cameras across 12 different time slots. With
4,101 annotated identities and 126,441 bounding boxes, it offers extensive
data for re-identification research.

####  4.1.2 Implementation Details

In this study, we utilize a Baseline model derived from BoT [12] and CLIP-ReID
Baseline [14], employing a ResNet50 architecture as its backbone. Our
methodology leverages readily available pretrained models, eliminating the
necessity for retraining. We standardize the dimensions of all images to 256
$\times$ 128 for the inference process. For the UFFM approach, we select a
coefficient of K=4 for Market-1501 and DukeMTMC-ReID, and K=6 for MSMT17. For
the results obtained by integrating the WDA, we conducted the algorithm five
times to ensure fairness, due to the random nature of the Triplet Data
Generation Algorithm. The results presented are the mean of these five runs.
All experiments were conducted using the PyTorch framework on an NVIDIA
GeForce RTX 4090 GPU, equipped with 32GB of RAM, and powered by an Intel®
Core™ i9-10900X CPU.

###  4.2 Comparison with Person ReID State-of-the-Art Methods

In our study, we benchmark our method against other person ReID approaches, as
detailed in Table 1. This table provides an in-depth comparison of the
performance of various methods across three standard datasets in person re-
identification: Market-1501, DukeMTMC-ReID, and MSMT17. Our comparison
primarily focuses on methods that employ a ResNet50 backbone and resize images
to 256 $\times$ 128\. The evaluation metrics, Rank-1 accuracy (Rank@1) and
mean Average Precision (mAP), serve as crucial benchmarks for assessing the
effectiveness of these methods in person re-identification tasks.

Table 3: Ablation study of distance metrics using WDA on Person ReID datasets. | WDA | Market-1501 | DukeMTMC-ReID | MSMT17
---|---|---|---|---
Ddt | Dag | Dcce | Rank@1 | mAP | Rank@1 | mAP | Rank@1 | mAP
BoT Baseline | ✔ |  |  | 94.5 | 85.9 | 86.4 | 76.4 | 74.1 | 50.2
| ✔ |  | 96.1 | 91.2 | 90.2 | 83.3 | 81.5 | 61.8
✔ | ✔ |  | 96.0 | 90.9 | 90.0 | 82.9 | 80.8 | 61.0
✔ |  | ✔ | 95.6 | 87.3 | 88.8 | 78.7 | 77.5 | 53.6
✔ | ✔ | ✔ | 96.2 | 91.0 | 90.3 | 83.1 | 82.0 | 62.3
CLIP-ReID Baseline | ✔ |  |  | 94.7 | 88.1 | 88.6 | 79.3 | 82.1 | 60.7
| ✔ |  | 95.7 | 92.0 | 90.7 | 84.8 | 83.4 | 67.7
✔ | ✔ |  | 95.9 | 91.9 | 90.6 | 84.6 | 83.4 | 67.2
✔ |  | ✔ | 95.4 | 88.8 | 90.5 | 80.8 | 82.2 | 58.7
✔ | ✔ | ✔ | 96.1 | 92.0 | 91.3 | 85.0 | 83.8 | 67.6

As depicted in Table 1, the integration of our approach with the CLIP-ReID
Baseline not only achieves the highest performance metrics, recording an mAP
of 92.0% and Rank-1 accuracy of 96.1% on Market-1501, as well as an mAP of
85.0% and Rank-1 accuracy of 91.3% on DukeMTMC-ReID, but also showcases
remarkable results on MSMT17 with an mAP of 67.6% and Rank-1 accuracy of
83.8%. When applied in conjunction with BoT, another foundational model in the
person ReID domain, our method similarly demonstrates significant
improvements, achieving a Market-1501 mAP of 91.0% and Rank-1 accuracy of
96.2%, alongside an mAP of 83.1% and Rank-1 accuracy of 90.3% on DukeMTMC-
ReID. This underlines the substantial enhancement potential of our proposed
method across various foundational architectures.

Notably, when compared to competitive models, and other iterations of CLIP-
ReID, our method exhibits superior performance. For instance, while
MGN+CircleLoss showed strong performance with a Rank-1 accuracy of 96.1% on
Market-1501, our approach with the CLIP-ReID Baseline surpasses this with both
higher mAP and Rank-1 metrics. Similarly, FastReID, which is renowned for its
efficiency and effectiveness, achieved a commendable Rank-1 accuracy of 83.3%
on MSMT17, yet our method still outperforms it, particularly in terms of mAP.
Moreover, our approach exceeds the CLIP-ReID’s own previous benchmarks,
setting new standards for mAP and Rank-1 accuracy across the board. These
advancements over competitive models do not merely highlight the efficacy of
our proposed method but also emphasize its adaptability and scalability to
diverse and complex datasets.

###  4.3 Ablation study

####  4.3.1 Effectiveness of UFFM and WDA

Table 2 demonstrate the effectiveness of our method when combined with the BoT
Baseline [12] and the CLIP-ReID Baseline [14]. This table provides a detailed
look at the performance improvements of our proposed methods compared to the
baselines of BoT and CLIP-ReID across three prominent datasets in the field of
Re-identification: Market-1501, DukeMTMC-ReID, and MSMT17. The outcomes are
measured using two primary metrics: Rank@1 and mAP, both of which reflect the
accuracy and reliability of the model in re-identifying identities.

For the BoT Baseline [12] on the Market-1501 dataset, we observe an
improvement from 94.5% to 96.2% in Rank@1 and from 85.9% to 91.0% in mAP after
applying UFFM and WDA, indicating significant performance enhancements. On the
DukeMTMC-ReID, Rank@1 increased from 86.4% to 90.3%, and mAP from 76.4% to
83.1%. Meanwhile, MSMT17 saw a Rank@1 increase from 74.1% to 82.0% and mAP
from 50.2% to 62.3%, marking a considerable leap in performance. The CLIP-ReID
Baseline [14] also witnessed similar improvements. With Market-1501, Rank@1
went up from 94.7% to 96.1%, and mAP from 88.1% to 92.0%. DukeMTMC-ReID saw
Rank@1 increase from 88.6% to 91.3% and mAP from 79.3% to 85.0%. The MSMT17
dataset showed a Rank@1 increase from 82.1% to 83.8% and mAP from 60.7% to
67.6%.

Although in all cases the application of UFFM+WDA results in higher Rank@1
scores than applying UFFM alone, there are instances where the mAP decreases.
This suggests that while the WDA method can enhance accuracy, it may, in some
cases, reduce the model’s ability to comprehensively identify results compared
to using the UFFM method alone. Nevertheless, these improvements indicate that
combining UFFM and WDA not only enhances accuracy but also strengthens the
model’s generalization capabilities across large and complex datasets.
Notably, the significant increase in mAP on MSMT17, a dataset characterized by
its high difficulty and diversity, demonstrates that these methods are
particularly effective in improving recognition capabilities under challenging
conditions.

####  4.3.2 Ablation study of UFFM and CFFM

Table 4 presents the results of the Feature Fusion Methods in the Triplet Data
Generation Algorithm 1. The results indicate that both Rank@1 and mAP are
improved when using the CFFM method to fuse features in the Triplet Data
Generation Algorithm across both the Market-1501 and DukeMTMC-ReID datasets,
with a pretrained CLIP-ReID Baseline.

Table 4: Comparison of Accuracy Using UFFM and CFFM | Market-1501 | DukeMTMC-ReID
---|---|---
Rank@1 | mAP | Rank@1 | mAP
UFFM | 95.7 | 91.9 | 90.6 | 84.7
CFFM | 96.1 | 92.0 | 91.3 | 85.0

####  4.3.3 Effectiveness of different distances

Table 3 presents the performance of combining different types of distances
using the Wise Distance Aggregation (WDA) method. The results indicate that
combining just two types of distances-either (Ddt and Dag) or (Ddt and
Dcce)-sometimes yields lower performance than using Dag alone. However,
simultaneously integrating all three distances: Ddt, Dag, and Dcce
consistently results in superior outcomes compared to using single distances
or any pair of distances. This demonstrates that a judicious combination of
multiple distance metrics can enhance the effectiveness of Re-ID solutions.

Figure 5: Market1501

###  4.4 Parameters analysis

(a) Impact of Parameter $K$ and $n$ on mAP Performance

(b) Impact of Parameters $K$ and $n$ on Rank@1 Performance

Figure 6: Impact of Parameters $K$ and $n$ on Market-1501

(a) Market1501

(b) DukeMTMC-ReID

Figure 7:  The results of the CLIP-ReID Baseline model are on the left and
those of the CLIP-ReID Baseline combined with our method are on the right.
Each row displays query images and the top 10 retrieved library images. Green
and red boxes indicate true positives and false positives, respectively.

Our method primarily relies on two parameters: $K$ and $n$, where $K$ directly
determines the number of nearest neighbors, and $n$ represents the number of
iterations for computing weights in the WDA method. We present the impact of
varying these parameters on Market-1501 dataset. The $K$ coefficient is
adjusted from 1 to 10, while $n$ is set to range from 200 to 1000, with
incremental steps of 200 for $n$. The metrics in Figure 6 illustrate that
Rank@1 accuracy and mAP do not follow a linear trend with increasing values of
$K$ or $n$. Notably, with the $K$ coefficient, there is always a trade-off
between mAP and Rank@1 accuracy when varying $K$. We set $n$ at 400 for both
datasets when assessing accuracy changes in both UFFM and UFFM+WDA methods.
Our findings, as depicted in Figure 5, indicate that both methods exhibit a
trade-off between Rank@1 accuracy and mAP with different $K$ values, and the
UFFM+WDA method consistently outperforms UFFM across various $K$ coefficients.

###  4.5 Visualization of results

We present visual examples of the top 10 rankings as determined by the
baseline CLIP-ReID model and our proposed method in Figure 7 for the
Market-1501 and DukeMTMC-ReID datasets, respectively. The results demonstrate
that the baseline model struggles with query retrieval when subjects change
posture or are occluded. Notably, integrating our method significantly
enhances performance, effectively handling subjects in various postures within
the Market-1501 dataset (Figure 7(a)) and achieving robust results even in
scenarios where subjects are occluded in the DukeMTMC-ReID dataset (Figure
7(b)). These visualizations underscore the efficacy of our approach in Feature
Fusion for improved re-identification.

##  5 Conclusion

This study has introduced two advanced methodologies: Uncertainty Feature
Fusion Method (UFFM) and Wise Distance Aggregation (WDA), along with their
empirical evaluations on standard datasets such as Market-1501, DukeMTMC-ReID,
and MSMT17. These methodologies not only enhance Rank-1 accuracy and mean
Average Precision (mAP) but also effectively address challenges posed by image
transformations due to varying camera angles and occlusions. The integration
of UFFM and WDA provides a robust and versatile approach for person re-
identification, paving new paths for the development of surveillance systems,
particularly in contexts that demand high accuracy and reliability. We
anticipate that these findings will make a significant contribution to the
field of camera-based person identification and encourage further research in
optimizing and adapting these methods to complex real-world scenarios.

##  6 Acknowledgement

This research was supported by The VNUHCMUniversity of Information
Technology’s Scientific Research Support Fund.

## References

  * [1] L. van der Maaten, G. Hinton, Viualizing data using t-sne, Journal of Machine Learning Research 9 (2008) 2579–2605.
  * [2] Y. Sun, L. Zheng, Y. Yang, Q. Tian, S. Wang, Beyond part models: Person retrieval with refined part pooling (and a strong convolutional baseline), in: V. Ferrari, M. Hebert, C. Sminchisescu, Y. Weiss (Eds.), Computer Vision – ECCV 2018, Springer International Publishing, Cham, 2018, pp. 501–518.
  * [3] J. Qian, W. Jiang, H. Luo, H. Yu, Stripe-based and attribute-aware network: a two-branch deep model for vehicle re-identification, Measurement Science and Technology 31 (2019).
  * [4] W. Sun, G. Dai, X. Zhang, X. He, X. Chen, Tbe-net: A three-branch embedding network with part-aware ability and feature complementary learning for vehicle re-identification, IEEE Transactions on Intelligent Transportation Systems 23 (9) (2022) 14557–14569.  doi:10.1109/TITS.2021.3130403.
  * [5] Y. Sun, C. Cheng, Y. Zhang, C. Zhang, L. Zheng, Z. Wang, Y. Wei, Circle loss: A unified perspective of pair similarity optimization, in: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020, pp. 6397–6406.  doi:10.1109/CVPR42600.2020.00643.
  * [6] C. Yan, G. Pang, X. Bai, C. Liu, X. Ning, L. Gu, J. Zhou, Beyond triplet loss: Person re-identification with fine-grained difference-aware pairwise loss, IEEE Transactions on Multimedia 24 (2022) 1665–1677.  doi:10.1109/TMM.2021.3069562.
  * [7] M. Wieczorek, B. Rychalska, J. Dabrowski, On the unreasonable effectiveness of centroids in image retrieval (2021).  arXiv:2104.13643.
  * [8] The earth mover’s distance as a metric for image retrieval.
  * [9] X. Zhu, Z. Luo, P. Fu, X. Ji, Voc-reid: Vehicle re-identification based on vehicle-orientation-camera, in: Proc. CVPR Workshops, 2020.
  * [10] M. Arevalillo-Herráez, J. Domingo, F. J. Ferri, Combining similarity measures in content-based image retrieval, Pattern Recognition Letters 29 (16) (2008) 2174–2181.  doi:https://doi.org/10.1016/j.patrec.2008.08.003.
URL https://www.sciencedirect.com/science/article/pii/S0167865508002511

  * [11] K. He, X. Zhang, S. Ren, J. Sun, Deep residual learning for image recognition, 2015, pp. 770–778.
  * [12] H. Luo, Y. Gu, X. Liao, S. Lai, W. Jiang, Bag of tricks and a strong baseline for deep person re-identification, in: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 2019, pp. 1487–1495.  doi:10.1109/CVPRW.2019.00190.
  * [13] L. He, X. Liao, W. Liu, X. Liu, P. Cheng, T. Mei, Fastreid: A pytorch toolbox for general instance re-identification, in: Proceedings of the 31st ACM International Conference on Multimedia, MM ’23, Association for Computing Machinery, New York, NY, USA, 2023, p. 9664–9667.  doi:10.1145/3581783.3613460.
URL https://doi.org/10.1145/3581783.3613460

  * [14] S. Li, L. Sun, Q. Li, Clip-reid: Exploiting vision-language model for image re-identification without concrete text labels, in: AAAI Conference on Artificial Intelligence, 2022.
  * [15] A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, G. Krueger, I. Sutskever, Learning transferable visual models from natural language supervision, in: International Conference on Machine Learning, 2021.
  * [16] K. Zhu, H. Guo, Z. Liu, M. Tang, J. Wang, Identity-guided human semantic parsing for person re-identification, ECCV (2020).
  * [17] V. Somers, C. D. Vleeschouwer, A. Alahi, Body part-based representation learning for occluded person re-identification, 2022, pp. 1613–1623.
  * [18] M. Kiran, R. G. Praveen, L. T. Nguyen-Meidine, S. Belharbi, L.-A. Blais-Morin, E. Granger, Holistic guidance for occluded person re-identification, in: British Machine Vision Conference, 2021.
URL https://api.semanticscholar.org/CorpusID:233231173

  * [19] P. Khorramshahi, V. Shenoy, R. Chellappa, Robust and scalable vehicle re-identification via self-supervision, in: 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 2023, pp. 5295–5304.  doi:10.1109/CVPRW59228.2023.00558.
  * [20] X. Jin, C. Lan, W. Zeng, Z. Chen, Uncertainty-aware multi-shot knowledge distillation for image-based object re-identification, in: AAAI Conference on Artificial Intelligence, 2020.
  * [21] One for more: Selecting generalizable samples for generalizable reid model 35 (2021) 3324–3332.  doi:10.1609/aaai.v35i4.16444.
URL https://ojs.aaai.org/index.php/AAAI/article/view/16444

  * [22] S. Liao, L. Shao, Graph Sampling Based Deep Metric Learning for Generalizable Person Re-Identification (June 2022).
  * [23] Y. Xu, Z. Jiang, A. Men, H. Wang, H. Luo, Multi-view feature fusion for person re-identification, Knowledge-Based Systems 229 (2021) 107344.  doi:https://doi.org/10.1016/j.knosys.2021.107344.
URL https://www.sciencedirect.com/science/article/pii/S0950705121006067

  * [24] H. T. Phan, A. M. Nguyen, Deepface-emd: Re-ranking using patch-wise earth mover’s distance improves out-of-distribution face identification, 2021, pp. 20227–20237.
  * [25] Q.-H. Che, H.-T. Le, M.-D. Ngo, H.-L. Tran, D.-D. Phan, Intelligent attendance system: Combining fusion setting with robust similarity measure for face recognition, in: 2023 International Conference on Multimedia Analysis and Pattern Recognition (MAPR), 2023, pp. 1–6.  doi:10.1109/MAPR59823.2023.10288710.
  * [26] Y. Li, J. He, T. Zhang, X. Liu, Y. Zhang, F. Wu, Diverse part discovery: Occluded person re-identification with part-aware transformer, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2021) 2897–2906.
  * [27] Y. Rao, G. Chen, J. Lu, J. Zhou, Counterfactual attention learning for fine-grained visual categorization and re-identification, in: ICCV, 2021.
  * [28] M. Jia, X. Cheng, S. Lu, J. Zhang, Learning disentangled representation implicitly via transformer for occluded person re-identification, IEEE Transactions on Multimedia 25 (2023) 1294–1305.  doi:10.1109/TMM.2022.3141267.
  * [29] M. Ye, J. Shen, G. Lin, T. Xiang, L. Shao, S. H. Hoi, Deep learning for person re-identification: A survey and outlook, IEEE Transactions on Pattern Analysis and Machine Intelligence 44 (06) (2022) 2872–2893.  doi:10.1109/TPAMI.2021.3054775.
  * [30] Y. Lin, L. Xie, Y. Wu, C. C. Yan, Q. Tian, Unsupervised person re-identification via softened similarity learning, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2020) 3387–3396.
  * [31] L. Zheng, L. Shen, L. Tian, S. Wang, J. Wang, Q. Tian, Scalable person re-identification: A benchmark, in: Computer Vision, IEEE International Conference on, 2015.
  * [32] E. Ristani, F. Solera, R. Zou, R. Cucchiara, C. Tomasi, Performance measures and a data set for multi-target, multi-camera tracking, in: European Conference on Computer Vision workshop on Benchmarking Multi-Target Tracking, 2016.
  * [33] L. Wei, S. Zhang, W. Gao, Q. Tian, Person transfer gan to bridge domain gap for person re-identification, in: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018, pp. 79–88.  doi:10.1109/CVPR.2018.00016.