# Progressive End-to-End Object Detection in Crowded Scenes

Anlin Zheng1,3* Yuang Zhang2* Xiangyu Zhang1 Xiaojuan Qi3 Jian Sun1
1 MEGVII Technology 2Shanghai Jiao Tong University 3University of Hong Kong
<EMAIL_ADDRESS><EMAIL_ADDRESS>{zhenganlin, zhangxiangyu,
<EMAIL_ADDRESS>

###### Abstract

In this paper, we propose a new query-based detection framework for crowd
detection. Previous query-based detectors suffer from two drawbacks: first,
multiple predictions will be inferred for a single object, typically in
crowded scenes; second, the performance saturates as the depth of the decoding
stage increases. Benefiting from the nature of the one-to-one label assignment
rule, we propose a progressive predicting method to address the above issues.
Specifically, we first select accepted queries prone to generate true positive
predictions, then refine the rest noisy queries according to the previously
accepted predictions. Experiments show that our method can significantly boost
the performance of query-based detectors in crowded scenes. Equipped with our
approach, Sparse RCNN achieves 92.0% AP, 41.4% $\text{MR}^{-2}$ and 83.2% JI
on the challenging CrowdHuman [37] dataset, outperforming the box-based method
MIP [8] that specifies in handling crowded scenarios. Moreover, the proposed
method, robust to crowdedness, can still obtain consistent improvements on
moderately and slightly crowded datasets like CityPersons [49] and COCO [27].
Code will be made publicly available at https://github.com/megvii-
model/Iter-E2EDET.

††footnotetext: * Equal contribution.

##  1 Introduction

Crowded object detection is a practical yet challenging research field in
computer vision. Many research efforts have been made and achieved impressive
progress [30, 5, 6, 48, 51, 8, 35, 14, 24] in the last few decades. However,
most of them [30, 5, 6, 48, 51, 8, 35, 14] require hand-craft components, e.g.
anchor settings and post-processing, resulted in sub-optimal performance in
handling scenes.

Figure 1: 1a. Sparse RCNN [39] introduces false positives in crowded scenes.
1b. Our approach can remove those false positives and ensure each object can
be detected only once. Green boxes indicate _true positives_ while red ones
represent _false positives_.

.2.3.4.5.6.7.8.91.00.05.10.15.20.25.30scoreratio of ratio _TP_ _FP_
-.03.00.03absolute relative improvement (a)

$0$$100$$200$$300$$400$$500$$600$$700$$800$$900$$1{,}000$$0$$2$$4$$6$$\cdot
10^{4}$#false positives#trues positives _Sparse R-CNN_ _Ours_ (b)

Figure 2:  2(a). The bottom histogram describes the prediction distribution of
Sparse RCNN [39] under different confidence scores, while the top one reflects
the absolute improvements achieved by our approach compared with Sparse RCNN
[39]. 2(b). The FP-TP curve when computing Average Precision (AP).

Recently, Carion et al. [3] proposed a fully end-to-end object detection
framework DETR, which introduces learnable queries to represent objects and
achieves competitive performance without any post-processing. It can be
categorized as a _query-based_ approach to differentiate from the _box-based_
[26, 25, 50] and _point-based_ [41, 46] methods. Following DETR [3], Sparse
RCNN [39] ensures object queries interact with local feature of Region of
Interest (RoI), while deformable DETR [54] proposes attention modules that
only attend to a small set of key sampling points. They further improve the
detection accuracy and mitigate several issues occurred in DETR: slow
convergence and high computational overhead.

The above success inspires us to study _query-based_ object detection methods
in crowded scenes, aiming at designing a more sophisticated end-to-end
detection framework. Although these  _query-based_ approaches [18, 54] can
obtain significant results on the slightly crowded datasets like COCO [27],
our initial studies show they suffer from several unresolved challenges in
crowded scenes:(1). the _query-based_ detector tends to infer multiple
predictions for a single object, with _false positives_ introduced. Figure. 1a
shows a common failure case; (2). The performance of a _query-based_ detector
becomes saturated or even worse as the depth of decoding stage increases,
which is depicted in the Appendix.

#### Our motivations.

Further investigations on the _query-based_ method, Sparse RCNN [39], yield
the following intriguing findings in crowd scenes. As described in Figure.
2(a), a large percentage of target objects can be accurately predicted by
those predictions with high confidence scores (e.g. higher than a threshold of
0.7), while containing few false positives. These predictions are more likely
to be _true positives_ that can be taken as _accepted predictions_. While the
rest, where a considerable number of _true positives_ and _false positives_
exist, can be regarded as _noisy predictions_. Naturally, if an object is
detected by one accepted prediction, there is no need for noisy predictions to
detect it again. Hence, _Why not strengthen the discrimination of those noisy
predictions given the context of the accepted predictions_? To this end, the
noisy queries can ‘perceive’ whether their targets have been detected or not.
If so, their confidence scores will be reduced and then filtered out.

#### Our contributions

Motivated by this, we propose a progressive prediction method equipped with a
prediction selector, relation information extractor, query updater, and label
assignment to improve the performance of query-based object detectors in
handling crowded scenes.

First, we develop a prediction selector to select queries associated with high
confidence scores as _accepted queries_ , leaving the rest as _noisy queries_.
Then, to let the _noisy queries_ ‘perceive’ whether their targets have been
detected or not, we design a relation extractor for relation modeling between
_noisy queries_ and their accepted neighbors. Further, a query updater is
developed by performing a new local self-attention attending to spatially-
related neighbors only. Finally, a new one-to-one label assignment rule is
introduced to assign samples among the accepted and refined noisy queries step
by step. With the proposed method, the above problems can be well addressed:
(1). Each object can be detected only once, which greatly decreases the number
of false positives while increasing the number of true positives, as described
in Figure. 1b; (2). As depicted in Figure. 2(b), the performance is
consistently improved compared with its counterparts [39, 54] that have the
same depth of decoding stage.

Our method is generic and can be incorporated into multiple architectures [39,
54], and delivers significant performance improvements of query-based
detectors. Equipped with our approach, Sparse RCNN [39] with _ResNet-50_ [16]
backbone obtains 92.0% AP, 41.4% $\text{MR}^{-2}$ and 83.2% JI on the
challenging dataset _CrowdHuman_ [37], outperforming the box-based method MIP
[8]. Besides, deformable DETR [54], equipped with our approach, also achieves
92.1% AP and 84.0% JI. Moreover, our approach works reasonably well for less
crowded scenes, e.g. the Sparse RCNN with our approach can still obtain 1.0%
$\text{MR}^{-2}$ and 1.1% AP gains on moderately and slightly crowded datasets
_Citypersons_ [49] and _COCO_ [27], respectively.

##  2 Related works

#### End-to-end object detection.

RelationNet [20] is one of the pioneering works trying to predict results
directly, achieving promising performance compared to their counterparts on
several famous benchmarks. DETR [3] introduces learnable queries to represent
objects and perform single prediction for each instance directly.
Subsequently, deformable-DETR [54] limits the attention field of each query to
a local area around the reference points to accelerate the convergence and
improve detection performance. Meanwhile, Sparse R-CNN [39] utilizes a fixed
set of learnable queries to formulate objects instead of a number of proxy
representation, e.g. anchors. Analogous to deformable DETR, RoIAlign [15] is
applied to limit the attention field in a local region. Adaptive Clustering
Transformer [53] proposes to improve the attention distribution in DETR’s
encoder by LSH approximate clustering for convergence acceleration. UP-DETR
[9] designs a new self-supervised method to improve the convergence speed of
DETR, while TSP[40] analyzes the main factors contributing to slow convergence
in DETR. SMCA [13] explores a better information interaction mechanism to
further accelerate convergence and improve the performance of DETR.

#### Object detection in crowded scenes.

Research community has poured much interest in exploring occlusion problems on
pedestrian detection. Specific methods have been proposed to mitigate this
problem, including detecting by parts [30, 5, 6, 48, 51] and improving hand-
crafted rules in training target design. Recently, CNN-based methods have
dominated the crowded object detection and achieved considerable gains.
Several works propose new loss functions to address problems of crowded
detection [47, 51]. Besides, the effectiveness of NMS is based on the
assumption that multiple instances rarely occur at the same location in an
image, which is not true in crowded scenes. But designing duplicate removal
for crowded scenes is non-trivial. Soft-NMS [1] and Softer-NMS [17] replace
hard removal of nearby proposals with score decay. Several works propose to
use a neural network to simulate the function of NMS for duplicates removal
[18, 34]. Others explore NMS-aware training, including NMS with adaptive
threshold [19, 28], feature embedding [36] and multiple prediction with set
suppression [8, 21], to tackle problem of object detection in crowded scenes.

Recently, PEDR [24] proposes several techniques to improve the performance of
_query-based_ detectors in coping with crowded detection, which is orthogonal
to ours. Their techniques are also applicable to our work.

#### Relation modeling for object detection.

As discussed in [20], early works [10, 12, 43, 44, 32] use object relations as
a post-processing step. The detected objects are re-scored by considering
object relationships. For example, co-occurrence, which indicates how likely
two object classes can exist in the same image, is used by DPM [11] to refine
object scores. The subsequent approaches [33, 7] try more complex relation
models, by taking additional positions and size into account. These methods
achieve moderate success in the pre-deep learning era but do not prove the
effectiveness in CNNs. Several recent works perform spatial reasoning [23, 38,
4, 18] to model object relations. Among them, GossipNet [18] and RelationNet
[20] are the representative methods. Both share the same spirit of modeling
relations among boxes. However, the network of GossipNet [18] is complex
(depth>80) and its computation cost is demanding. Although it allows end-to-
end learning in principle, no experimental evidence approves. RelationNet [20]
utilized the self-attention for feature interaction and obtained a promising
improvement in general object detection. Nevertheless, it doesn’t show a
promising performance in dealing with crowd scenes [37].

Recent works related to ours are PS-RCNN [14] and IterDet [35]. They proposed
to detect objects according to the previous predicted ones. They need to mask
the feature [14] or produce a history map [35] to memorize the previous
detections, introducing noise while limiting performance improvement [14] or
incur heavy computation [35]. Even so, both of them need a post-processing
method to remove duplicates in every iteration.

Recent query-based object detectors [39, 54, 53, 9, 40, 13] utilized learnable
queries to represent objects, and take advantage of the self/cross-attention
to model the relations among queries, detecting objects in an end-to-end
manner. Our work inherits the methodology and boosts their performance in
heavily, moderately, and slightly crowded scenes.

##  3 Methodology

In this section, we first revisit the query-based object detector, e.g. Sparse
RCNN [39] briefly. Next, we illustrate our approach primarily deployed on
Sparse RCNN explicitly. Finally, the main differences of detector design will
be discussed as follows.

###  3.1 Query Based Object Detector

Our approach can be deployed on most _query-based_ object detectors [3, 39,
54]. To illustrate the proposed method, we choose Sparse RCNN [39] as our
default instantiation. Figure. 4(a) depicts its object detection pipeline,
which can also be formulated as:

$\begin{split}x_{t-1}&\leftarrow\mathcal{P}^{box}(x^{FPN},b_{t-1}),\\\
q^{\star}_{t-1}&\leftarrow{\rm MSA}_{t-1}(q_{t-1}),\\\ q_{t}&\leftarrow{\rm
DynConv}_{t-1}(q^{\star}_{t-1},x_{t-1}),\\\
b_{t}&\leftarrow\mathcal{B}_{t-1}(q_{t}),\end{split}$ (1)

where $q\in R^{N\times d}$ denotes the learnable object query. $N$ and $d$
denote the number and dimension of query $q$, respectively. At stage $t$, an
RoIAlign [15] $\mathcal{P}^{box}$ extracts RoI features from FPN features
$x^{FPN}$, under the guidance of bounding box $b_{t-1}$ predicted by the
previous stage. Meanwhile, a multi-head self-attention module ${\rm
MSA}_{t-1}$ is applied to the input query $q_{t-1}$ to get the transformed
query $q^{\star}_{t-1}$. Then, a dynamic convolution module ${\rm
DynConv}_{t-1}$ takes both $x_{t-1}$ and $q^{\star}_{t-1}$ as inputs and
performs dynamic convolution to generates $q_{t}$ for the next stage.
Simultaneously, $q_{t}$ is fed into the box prediction branch
$\mathcal{B}_{t-1}$ for current bounding box prediction $b_{t}$, which is the
input of the next stage $t$.

Figure 3: The diagram of the proposed progressive end-to-end object detection
framework. First, the _prediction selector_ select queries associated with
high confidence scores as _accepted queries_ , leaving the rest as _noisy
queries_. Then _Relation information extractor_ models the relations between
_noisy queries_ and their neighbors from accepted predictions. Next, the
queries are fed into the _queries updater_ to be further refined by performing
a new local self-attention.

###  3.2 Our Method

As illustrated in Figure. 3, the proposed progressive predicting method
consists of several components: prediction selector, relation information
extractor, query updater, and label assignment, which will be introduced in
detail next.

$\mathbf{q}_{t-1}$$\mathbf{b}_{t-1}$$\mathbf{b}_{t}$$\mathbf{q}_{t}$DyConvt-1$\mathbf{x}^{\text{FPN}}$$\mathcal{P}^{\text{box}}$MSAt-1
(a) Sparse R-CNN [39].

$\mathbf{q}_{t-1}$$\mathbf{b}_{t-1}$$\mathbf{b}_{t}$$\mathbf{q}_{t}$DyConvt-1$\mathbf{x}^{\text{FPN}}$$\mathcal{P}^{\text{box}}$QUt-1$\mathcal{S}$$\mathcal{R}$
(b) _Our approach_

Figure 4: The diagram of decoding stage. $\mathcal{P}$–RoIAlignPool [15],
DynConv – Dynamic Convolution, MSA – Multi-head Self-Attention, $\mathcal{S}$
– Prediction Selector, $\mathcal{R}$ – Relation Information Extractor, QU –
Query Updater.

#### Prediction selector.

For the findings described in Sec. 1, a prediction selector is developed to
select those queries prone to generating predictions with high confidence
scores as accepted queries, while leaving the rest as noisy ones that need to
be further refined. This procedure can be formulated in Equ.(2).

$\begin{split}\mathcal{D}^{h}_{t-1}&\leftarrow\\{{b_{i}|s_{i}\geq s\wedge
b_{i}\in\mathcal{D}_{t-1}}\\},\\\
\mathcal{D}^{l}_{t-1}&\leftarrow\mathcal{D}_{t-1}-\mathcal{D}^{h}_{t-1}.\\\
\end{split}$ (2)

where $t$ is the stage number. $\mathcal{D}_{t-1}$ denote the whole
predictions produced by the whole queries in the previous $t-1$ stage.
$\mathcal{D}^{h}_{t-1}$ and $\mathcal{D}^{l}_{t-1}$ indicate the accepted
predictions and noisy predictions generated from the accepted and noisy
queries, respectively. $b_{i}$ and $s_{i}$ denote the predicted box and its
confidence score, respectively. $s$ is the confidence score threshold.

#### Relation information extractor.

As mentioned in Sec. 1, a large percentage of target objects can be accurately
predicted by the accepted queries. Therefore, if an object is detected by one
accepted prediction, there is no need for noisy predictions to detect it
again. In order to equip these noisy queries with the capability of perceiving
whether their targets have been detected or not, we develop a relation
information extractor to model the relation between the noisy predictions and
their accepted neighbors.

The detailed design of the relation information extractor is illustrated in
Figure. 5, with the procedure formulated in Equ.(3) as well. For each noisy
prediction $b_{i}$, we first find their accepted neighbors
$\mathcal{N}(b_{i})$ in $\mathcal{D}^{h}_{t-1}$, constructing the spatially-
related pairs $(b_{i},\mathcal{N}(b_{i}))$. Then, the encoded pairs together
with the intersection-over-union (IoU) between them are fed to a compact
network to obtain the geometry relation features $\mathcal{H}(b_{i})$. Since
the number of accepted neighbors corresponding to each noisy prediction is
uncertain. An aggregation function is employed to reduce $\mathcal{H}(b_{i})$
to the same feature dimension, while maintaining the permutation-invariance
property. In our approach, we use max pooling by default. Besides, the pooled
geometry features, fused with the transformed query features, are further
activated by a non-linear function.

$\begin{split}\mathcal{N}(b_{i})&\leftarrow\\{b_{j}|\mathcal{O}(b_{i},b_{j})\geq\theta\\},b_{i}\in\mathcal{D}^{l}_{t-1},b_{j}\in\mathcal{D}^{h}_{t-1},\\\
\mathcal{H}(b_{i})&\leftarrow\mathcal{U}(\mathcal{E}(b_{i},\mathcal{N}(b_{i}))),\quad
b_{i}\in{D}^{l}_{t-1},\\\
\mathcal{R}(b_{i})&\leftarrow\mathcal{T}(\textit{MaxPool}(\mathcal{H}(b_{i}))+\mathcal{F}(q_{i})).\end{split}$
(3)

where $\mathcal{N}(\cdot)$ represents a function that finds neighbors for a
box $b_{i}$ based on the IoU $\mathcal{O}(\cdot,\cdot)$ with a threshold
$\theta$. Here, we use it to find the accepted neighbors in
$\mathcal{D}^{h}_{t-1}$ for the noisy predictions in $\mathcal{D}^{l}_{t-1}$.
$\mathcal{E}(\cdot,\cdot)$ refers to the sine and cosine spatial positional
encoding function which is the same as that [20, 45]. Also,
$\mathcal{U}(\cdot,\cdot)$ denotes a function used to generate geometry
relation features $\mathcal{H}(b_{i})$ from the encoded inputs. The noisy
query $q_{i}$ corresponds to noisy prediction $b_{i}$ in
$\mathcal{D}^{l}_{t-1}$, transformed by the function $\mathcal{F}(\cdot)$. The
pooled geometry features and transformed query features $\mathcal{F}(q_{i})$
are fused through element-wise summation, followed by a function $\mathcal{T}$
to produce the desired relation features $\mathcal{R}(b_{i})$.

As depicted in Figure. 5, $\mathcal{U}(\cdot,\cdot)$ consists of two
consecutive fc layers with ReLU [31] activation to increase the non-linearity.
Note that $\mathcal{F}(\cdot)$ and $\mathcal{U}(\cdot)$ share the same
architecture, but are weight-independent. Here, the gradients of $q_{i}$ are
stopped from back-propagating to the previous stages.

$\mathbf{q}_{i}$$\mathbf{b}_{i},$$\mathcal{N}(\mathbf{b}_{i})$$\mathcal{E}$linear+rlinearlinear+rlinearMaxPool$+$$\mathcal{T}$
Figure 5: Relation information extractor $\mathcal{R}$. $\mathcal{E}$ – sine
and cosine spatial positional encoding function [20, 45], linear – fc layer, r
– ReLU, $\mathcal{T}$– fc layer.

#### Query updater.

To further refine the features of noisy queries, a query updater is developed,
which is formulated in Equ.(4). Since the data distribution of
$\mathcal{D}^{l}_{t-1}$ and $\mathcal{D}^{h}_{t-1}$ is different from that of
$\mathcal{D}_{t-1}$, a new set of learnable queries is first introduced to
complement the relation features through element-wise summation. Then the set
of complemented noisy queries is taken as the input query $q_{t-1}$ to perform
a new local self-attention ${\rm LMSA}_{t-1}$ and the subsequent dynamic
convolution given in Equ.(1).

$\begin{split}q_{t-1}&\leftarrow\\{\widetilde{q_{i}}|\widetilde{q_{i}}=\mathcal{R}(b_{i})+e_{i}\\},\
b_{i}\in\mathcal{D}^{l}_{t-1},e_{i}\in E\\\
q^{\star}_{t-1}&\leftarrow\text{LMSA}_{t-1}(q_{t-1}).\end{split}$ (4)

Since object detection mainly focuses on the local region in an image. We
design a new local self-attention module ${\rm LMSA}_{t}$ to update the noisy
query $q_{t-1}$. It ensures each query only interacts with local neighbors
instead of the whole queries over the full image. The local self-attention
first finds those neighbors of each query based on the boxes’ IoUs whose
values are greater than 0. Then it performs the ‘qkv’ mechanism in the same
way as MSA. To this end, we perform self-attention locally instead of
globally.

Different from the neighbor finding rule in [24], we adopt the function
$\mathcal{N}(\cdot)$ to select neighbors from $\mathcal{D}_{t-1}$ that are
spatially related to $q_{t-1}$ in terms of IoU. Note that, the new local self-
attention ${\rm LMSA}_{t-1}$ is used to replace the ${\rm MSA}_{t-1}$ in
Equ.(1) for feature interaction.

#### Label assignment

Since accepted queries tend to generate true positive predictions, while the
noisy ones involve a considerable number of true positives and false
positives. Towards end-to-end object detection, we introduce a new one-to-one
label assignment rule to assign samples step by step. Specifically, we first
match the accepted predictions $\mathcal{D}^{h}_{t-1}$ with the ground truth
set of objects. Then remove those targets that have been matched, and mainly
consider the bipartite matching between noisy predictions
$\mathcal{D}^{l}_{t-1}$ and the remaining ground truth set of objects. This
matching process is described in Algorithm 1111The HungarianMatch operation in
Algorithm 1 is a combinatorial optimization approach that solves the
assignment problem, which is commonly used in [3, 39, 54, 46] for one-to-one
label assignment., where the matching cost computation is slightly different
from the original version [39]. A spatial prior is adopted to compute the
matching cost $\mathcal{C}$, that is, the center of bounding box $b_{t}$ needs
to fall in the corresponding target box. Except for it, the formulation of the
matching cost function is identical to the original work.

Algorithm 1 Label Assignment for $\mathcal{D}^{l}_{t}$.

1:$\mathcal{D}^{l}_{t}$, $\mathcal{D}^{h}_{t}$, $\mathcal{G}$;

2:$\mathcal{D}^{l}_{t}$: results of $\mathcal{D}^{l}_{t-1}$ in Equ.(2) from
stage $t$;

3:$\mathcal{D}^{h}_{t}$: results of $\mathcal{D}^{h}_{t-1}$ in Equ.(2) from
stage $t$;

4:$\mathcal{G}$: target boxes.

5:The matched predictions $\mathcal{M}^{l}_{D}$ and corresponding targets
$\mathcal{M}^{l}_{G}$ after assignment.

6:Compute matching costs $\mathcal{C}^{h}_{t}$ between $\mathcal{D}^{h}_{t}$
and $\mathcal{G}$;

7:$\mathcal{M}^{h}_{G},\mathcal{M}^{h}_{D}=\text{HungarianMatch}(\mathcal{D}^{h}_{t},\mathcal{G},\mathcal{C}^{h}_{t})$;

8:$\mathcal{G}^{l}_{t}=\mathcal{G}-\mathcal{M}^{h}_{G}$;

9:Compute matching costs $\mathcal{C}^{l}_{t}$ between $\mathcal{D}^{l}_{t}$
and $\mathcal{G}^{l}_{t}$;

10:$\mathcal{M}^{l}_{G},\mathcal{M}^{l}_{D}=\text{HungarianMatch}(\mathcal{D}^{l}_{t},\mathcal{G}^{l}_{t},\mathcal{C}^{l}_{t})$;

11:return $\mathcal{M}^{l}_{G},\mathcal{M}^{l}_{D}$;

###  3.3 Difference of Detector Design

Generally, our approach can be deployed on most query-based object detectors
[3, 39, 54]. To illustrate the proposed method, we choose Sparse RCNN [39] as
our default instantiation. It consists $t$ ($t=6$ by default) decoding stages,
each of which performs prediction according to Equ. (1). As described in
Figure. 3, we keep the first $t-1$ decoding stages unchanged and only equip
the last stage with the proposed method. Therefore, main differences lie in
the last decoding stage, which will be described in the following.

#### Architecture of the last stage.

As depicted in Figure. 4(b), the last decoding stage $t$ first employs a
prediction selector $\mathcal{S}$ to split queries into accepted queries and
noisy queries according to the confidence scores of their associated
predictions. Then they are input to the relation information extractor
$\mathcal{R}$ to extract the relation feature between the noisy predictions
and their accepted neighbors. Finally, queries are fed into the query updater
QU to be further refined for recognition and localization.

#### Box prediction

Like [39], a box regression branch is used for box prediction in the first
$t-1$ stages. Differently, for the box prediction in the last stage, we
directly use the identity mapping results from the previous $t-1$ stage both
in the training and testing phase. This is because, at the latter layers, the
predicted bounding boxes are less likely to fluctuate, which is observed in
[24]. Meanwhile, the recognition branch remains the same as that of previous
stages.

#### Training loss

We adopt the set prediction loss adopted in [39, 54] for training. For the
samples to train stage $t$, we remove those samples from accepted predictions
$\mathcal{D}^{h}_{t-1}$. to mitigate the class-imbalance issue, we follow the
negative sample filtering mechanism [52] to early reject those well-classified
negative queries whose confidence scores are lower than 0.05.

##  4 Experiments

In this section, we evaluate our approach on heavily, moderately, and slightly
crowded datasets [37, 49, 27] to demonstrate the generality of the proposed
method in diverse scenarios.

#### Datasets.

We adopt three datasets – _CrowdHuman_ [37], _CityPersons_ [49] and _COCO_
[27] – for comprehensive evaluations on heavily, moderately and slightly
crowded situations, respectively. Table 1 lists the “instance density” of each
dataset. Since our proposed approach mainly aims to improve crowded
detections, we perform most of the comparisons and ablation experiments on
_CrowdHuman_. The evaluation experiments on _Citypersons_ [49] and _COCO_ [27]
are also conducted to suggest the proposed method is robust to crowdedness.

Dataset | # objects/img | # overlaps/img
---|---|---
CrowdHuman [37] | 22.64 | 2.40
CityPersons [49] | 6.47 | 0.32
COCO∗ [27] | 9.34 | 0.015
Table 1: _Instance density_ of each dataset. The threshold for overlap
statistics is $\mathrm{IoU}>0.5$. *Averaged by the number of classes.

#### Evaluation metrics.

Following [8], we mainly take three criteria: AP, $\text{MR}^{-2}$ and JI as
evaluation metrics. Generally, a larger AP, larger JI and smaller
$\text{MR}^{-2}$ indicates a better performance.

#### Implementation details.

Unless otherwise specified, we take Sparse RCNN [39] as our default
instantiation, using standard ResNet-50 [16] pre-trained on ImageNet as
backbone. We train our model with the Adam optimizer with a momentum of 0.9
and weight decay of 0.0001. Models are trained for 50, 000 iterations. The
initial learning rate is 0.00005 and reduced by a factor of 0.1 at iteration
37,500. The last stage joints the optimization after 5,000 iterations of
training. $\lambda_{cls}=2,\lambda_{L1}=5,\lambda_{giou}=2$. The default
number of proposal boxes, proposal features, and stages are set to 500, 500,
and 6, respectively. Additionally, The dimension of intermediate features in
relationship extractor $\mathcal{R}$ is 256. The gradients are detached at
proposal boxes from the second stage to stabilize training. Besides, those
negative samples, whose intersection-over-area (IoA) between any ignore region
is higher than a threshold of $0.7$, are not involved in training. Further,
the hyper-parameters $s$ and $\theta$ are $0.7$ and $0.4$ by default in
different query-based detectors [39, 54].

###  4.1 Experiments on CrowdHuman

CrowdHuman [37] contains 15,000, 4,370 and 5,000 images for training,
validation and test, respectively. For a fair comparison, we re-implement most
of the involved models [25, 21, 26, 28, 50, 8, 42, 46, 3, 54, 39, 24, 18, 20].
Results are evaluated on the validation set, using the full-body annotations
in the dataset.

#### Main results.

We compare with mainstream object detectors, including box-based: one-stage
[26, 50] , two-stages [8, 25, 21, 28], and point-based [42, 46] as well as
query-based [3, 54, 39, 24].

As shown in Table. 2, our approach outperforms these well-established
detectors, achieving significant performance improvements over the box-based,
point-based, and query-based counterparts, illustrating the effectiveness of
our approach in handling crowded scenes. Specifically, our method achieves
1.8% AP and 0.9% JI gains over the state-of-the-art box-based approach MIP
[8], which specializes in coping with crowded scenes.

The query-based method Sparse RCNN [39], equipped with the proposed method and
500 queries, can achieve 92.0% AP, 41.4% $\text{MR}^{-2}$ and 83.2% JI on the
challenging CrowdHuman dataset [37], which is 1.3%, 3.3% and 1.8% better than
its counterpart – original Sparse RCNN [39]. When increasing the number of
queries to 750, our approach can still obtain a better performance of 92.5% AP
and 83.3% JI. This is because more queries can cover more patterns of objects
in the image, such as scale, size, position, and other characteristics.
Additionally, equipped with our approach, deformable DETR [54]222The detail
implementation of deformable DETR with the proposed schema is illustrated in
the Appendix. can also obtain 2.2% $\text{MR}^{-2}$ improvements over the
original deformable DETR [54]. Moreover, It also achieves 1.4% AP and 1.6% JI
gains over the box-based method MIP [8], demonstrating the effectiveness of
our approach.

Method | #Queries | AP | $\text{MR}^{-2}$ | JI
---|---|---|---|---
box-based |  |  |  |
RetinaNet [26] | - | 85.3 | 55.1 | 73.7
ATSS [50] | - | 87.0 | 51.1 | 75.9
ATSS [50]+MIP [8] | - | 88.7 | 51.6 | 77.0
FPN [25]+NMS | - | 85.8 | 42.9 | 79.8
FPN [25]+soft NMS | - | 88.2 | 42.9 | 79.8
FPN+MIP [8] | - | 90.7 | 41.4 | 82.4
${\text{FPN}}^{{\dagger}}\text{+NMS}$ | - | 84.9 | 46.3 | –
${\text{Adaptive NMS}}^{{\dagger}}$ [28] | - | 84.7 | 47.7 | –
${\text{PBN}^{{\dagger}}}$ [21] | - | 89.3 | 43.4 | –
point-based |  |  |  |
FCOS [42] | - | 86.8 | 54.0 | 75.7
FCOS [42]+MIP [8] | - | 87.3 | 51.2 | 77.3
POTO [46] | - | 89.1 | 47.8 | 79.3
query-based |  |  |  |
DETR [3] | 100 | 75.9 | 73.2 | 74.4
PEDR [24] | 1000 | 91.6 | 43.7 | 83.3
D-DETR [54] | 1000 | 91.5 | 43.7 | 83.1
S-RCNN [39] | 500 | 90.7 | 44.7 | 81.4
S-RCNN [39] | 750 | 91.3 | 44.8 | 81.3
S-RCNN+_Ours_ | 500 | 92.0 | 41.4 | 83.2
S-RCNN+_Ours_ | 750 | 92.5 | 41.6 | 83.3
D-DETR+_Ours_ | 1000 | 92.1 | 41.5 | 84.0
Table 2: Comparisons of different methods on _CrowdHuman_ validation set, +MIP
represents multiple instance prediction with set NMS as post-processing.
${{\dagger}}$ indicates the approach is implemented by PBM [21]. S-RCNN –
Sparse RCNN [39]. D-DETR – deformable DETR [54].

#### Ablation study of different modules.

To explore the effectiveness of the proposed modules in Sec. 3.2, we conduct
extensive ablation study of the relation information extractor $\mathcal{R}$,
local self-attention module LMSA and the newly initialized embedding _E_. All
experiments are conducted on Sparse RCNN [39] with 500 queries, _ResNet-50_
[16] backbone and evaluated on _CrowdHuman_ dataset. Table.
LABEL:tbl:module_analysis shows that the relation information extractor
$\mathcal{R}$ can obtain an improvement of 0.8% AP, 1.7% $\text{MR}^{-2}$ and
1.6% JI. It indicates its effectiveness in reducing false positives and
recalling false negatives. Moreover, when equipped with the new local self-
attention LMSA, the performance on three evaluation metrics is further
boosted, since the local self-attention can reduce duplicates effectively.
Further, the newly initialized embeddings, aiming to approximate the new data
distribution of noise predictions, can slightly improve $\text{MR}^{-2}$.

#### Ablation study of hyper-parameter s.

To analyze the effect of the confidence score threshold $s$, we first
formulate the relation between detection boxes and target boxes in an image as
a bipartite graph $\mathcal{G}=(\mathcal{V},\mathcal{E})$. It consists of a
set $\mathcal{V}=\mathcal{D}\bigcup\mathcal{G}$ and nodes $\mathcal{E}$.
$\mathcal{D}$ represents a set of predicted boxes whose scores are higher than
the pre-defined score threshold, while $\mathcal{G}$ denotes the target boxes.
An edge in $\mathcal{E}$ is defined as overlapping when the _IoU_ value,
between a box in $\mathcal{D}$ and the other one from $\mathcal{G}$, is higher
than 0.5 by default333Here, we follow the procedure to compute evaluation
metric JI.. Hence, the matching results can be acquired after applying the
Hungarian Algorithm. As shown in Figure. 2, as the confidence score increases,
the number of true positives shows a clean upward trend while the number of
false positives decreases rapidly. Also, Figure. 7(a) depicts the performance
our method can achieve under different values of $s$, where the performance
increases slightly as $s$ increases. Thus, if not specific, we set $s$ to 0.7
by default.

#### Ablation study of hyper-parameter ${\theta}$.

Here, we analyze the effect of the hyper-parameter intersection-over-union
threshold $\theta$. As discussed in Sec. 1, making sure a box candidate can
‘perceive’ its neighbors helps a noisy query decide to decrease its confidence
score or not, which is also the prerequisite for our method to work
effectively. Different settings of intersection-over-union (_IoU_) threshold
$\theta$ may affect the performance of the whole detector. We perform
experiments on the _CrowdHuman_ dataset [37] with $s$ frozen as $0.7$ while
changing the value of $\theta$ linearly. From Figure. 7(b), we found our
approach is robust to the change of _IoU_ threshold. This success may
attribute to the good approximating feature of the newly designed components.

#### Comparison with previous relation modeling works.

To differentiate the previous works and ours, we evaluate several
representative relation modeling methods: RelationNet [20], GossipNet [18],
IterDet [35]. RelationNet [20] utilized self-attention modules to model the
relations among different predictions. Meanwhile, GossipNet [18] uses several
hand-designed relation blocks to explore the relationships among the predicted
boxes, while IterDet [35] iteratively infers predictions based on a historical
map produced from the previous iteration. We re-implement its re-scoring
version for RelationNet [20]. For
GossipNet444GossipNet:https://github.com/hosang/gossipnet and
IterDet555IterDet:https://github.com/saic-vul/iterdet, we use their open-
source implementations for evaluation. All models use FPN [25] with
_ResNet-50_ [16] as backbone, following the same training setting in [18, 20,
25].

As shown in Table. LABEL:tbl:relation_modeling, our approach shows better
performance when compared with previous relation modeling works. Surprisingly,
both RelationNet [20] and GossipNet [18] suffer from a significant drop in AP
and $\text{MR}^{-2}$. It could attribute to the sub-optimal label assignment
rule. Since both of them choose the prediction with the highest confidence
score around one target as the correct box and take the rest as negatives. The
predicted coordinates are not involved in computing loss, which might lead to
the performance degradation in crowded scenes.

$\mathcal{R}$ | LMSA | _E_ | AP | $\text{MR}^{-2}$ | JI
---|---|---|---|---|---
|  |  | 90.7 | 44.7 | 81.4
✓ |  |  | 91.5 | 43.0 | 83.0
✓ | ✓ |  | 92.0 | 42.0 | 83.5
✓ | ✓ | ✓ | 92.0 | 41.4 | 83.2

(a)

Method | #Queries | AP | $\text{MR}^{-2}$ | JI
---|---|---|---|---
GossipNet [18] | - | 80.4 | 49.4 | 81.6
RelationNet [20] | - | 81.6 | 48.2 | 74.6
IterDet [35] | - | 88.0 | 47.5 | 78.0
D-DETR+_Ours_ | 500 | 91.2 | 42.6 | 84.0
S-RCNN+_Ours_ | 500 | 92.0 | 41.4 | 83.2
(b)

Method | #Queries | $\text{MR}^{-2}$ | AP
---|---|---|---
FPN+NMS |  | 9.8 | 94.7
FPN+Soft-NMS [1] | - | 9.9 | 94.9
MIP [8] |  | 8.8 | 95.8
D-DETR [54] | 500 | 9.4 | 96.6
S-RCNN [39] | 500 | 10.0 | 96.8
D-DETR+_Ours_ | 500 | 7.8 | 96.7
S-RCNN+_Ours_ | 500 | 7.8 | 97.6
(c)

Table 3:  LABEL:tbl:module_analysis. Ablation study of different modules
proposed in our approach, taking Sparse RCNN [39] with 500 queries as our
default instantiation. LABEL:tbl:relation_modeling. Comparisons of different
relation modeling appraoches. All the experiments are conducted on
_CrowdHuman_ [37] dataset. LABEL:tbl:citypersons_eval Performance comparisons
of different methods on  _CityPersons_ [49]. Both box-based [8, 25] and query-
based approaches [39, 54] are evaluated.

1.7800.9021.4671.1400.3520.2610.9950.995DuplicateLocalizationBackgroundMissing$0$$0.5$$1$$1.5$$2$$\cdot
10^{4}$Number of errors (#errors)Sparse R-CNNours Figure 6: Error analysis on
Sparse RCNN [39] and our approach, with _ResNet-50_ [16] as backbone. The bar
plots show different error types that contribute to the false positives.

.424.831.920.427.833.920.414.832.921.410.834.920.415.834.9190.50.60.70.80.90.00.20.40.60.81.0MR-2JIAP
(a) confidence score threshold $s$

.425.835.919.427.833.918.431.834.919.414.832.921.422.836.920.418.835.920.423.835.9200.10.20.30.40.50.60.70.00.20.40.60.81.0MR-2JIAP
(b) intersection over union threshold $\theta$

Figure 7: Performance of the proposed method with different configurations of
hyper-parameter s and ${\theta}$ on _CrowdHuman_ [37] dataset.

#### Analysis on false positives.

To understand the factors contributing to the performance improvement, we
conduct an error analysis on our method. We adopt the recently proposed TIDE
[2] to compare our approach with the counterpart Sparse RCNN [39]. We analyzed
the composite error at Recall=0.9 for all methods. As illustrated in Figure.
7, our method performs better at removing duplication, providing more accurate
localization, and reducing mistaken recognition. Since part of queries can
perceive whether their targets are detected or not through the relation
information extractor. Also, the local self-attention module ensures queries
only interact with their neighbors rather than the whole. To this end, the
duplicates could be eliminated efficiently. Besides, with identity mapping
plugged in the last regression branch for box prediction, the number of
training samples in the previous decoding stage increases, making the
optimization much easier. Additionally, benefiting from the new learnable
embeddings for data distribution approximation, the representation ability of
object queries are further enhanced.

###  4.2 Experiments on Citypersons

CityPersons [49] is one of the widely used benchmarks for pedestrian
detection. It contains 5, 000 images (2, 975 for training, 500 for validation,
and 1, 525 for testing, respectively). Each image has a size of 1024 $\times$
2048\. To improve the overall performance, we proposed to pre-train all models
on the _CrowdHuman_ dataset and fine-tune them on  _CityPersons_
(_reasonable_) training subset, then tested on the (_reasonable_) validation
subset. For those  _box-based_ methods, we train and evaluate them with the
image resolution enlarged by 1.3$\times$ compared to the original one for
better accuracy. The _query-based_ approaches are trained and evaluated at the
original image size with 500 queries. The other settings remain the same as
those of Sparse RCNN [39] and deformable DETR [54].

###  4.3 Experiments on COCO.

According to Table 1, the crowdness of _COCO_ [27] is very low, which is
beyond our design purpose. Nevertheless, we still conduct an experiment on
this dataset to verify: 1) whether our method generalizes well to multi-class
detection; 2) whether our approach can still handle slightly crowded
scenarios, especially with _isolated_ instances.

Following the common practice of Sparse RCNN [39] with 300 queries, we use a
subset of 5000 images in the original validation set (named _minival_) for
validation while using the remaining images in the training and validation set
for training. Except for the proposed modules and label assignment rule in the
last stage, other settings remain the same as the original methods [54, 39].
Table. 4 shows the performance comparisons with deformable DETR [54] and
Sparse RCNN [39]. Moderate improvements are obtained, e.g. 0.9% AP higher than
the deformable DETR [54] and 1.1% AP higher than the Sparse RCNN [39]. The
experimental results reflect the effectiveness of our progressive predicting
approach in slightly crowded scenarios, proving the proposed method can also
solve the performance saturation problem of _query-based_ detectors.

Method |  AP |  $\text{AP}_{50}$ |  $\text{AP}_{75}$ |  $\text{AP}_{S}$ |  $\text{AP}_{M}$ |  $\text{AP}_{L}$
---|---|---|---|---|---|---
S-RCNN [39] |  45.0 |  64.2 |  49.1 |  27.6 |  47.5 |  59.1
D-DETR [54] |  45.8 |  64.5 |  49.4 |  28.2 |  49.0 |  61.7
S-RCNN+_Ours_ |  46.1 |  65.3 |  50.6 |  29.2 |  48.7 |  59.9
D-DETR+_Ours_ |  46.7 |  65.3 |  50.3 |  28.6 |  49.8 |  61.7
Table 4: Performance comparisons of different methods on _COCO_ 2017 [27]
_minival_ set.

##  5 Conclusion

In this paper, we propose a progressive prediction method to boost the
performance of _query-based_ object detectors in handling crowded scenes.
Equipped with our approach, two representatives query-based methods, Sparse
RCNN [39] and deformable DETR [54] achieve consistent improvements over the
heavily, moderately, as well as slightly crowded datasets [37, 49, 27], which
suggests our approach is robust to crowdedness. Since Sparse RCNN [39] and
deformable DETR [54] require large computing resources, making it difficult
for our method to be deployed on devices with limited computing capacity. How
to develop a computation-efficacy end-to-end detector is still under
exploration. Besides, we found the decision boundary for the noisy queries is
unclear. We believe that the performance can be further improved if a better
feature engineering method or loss function is adopted. However, it is beyond
the purpose of this work.

## References

  * [1] Navaneeth Bodla, Bharat Singh, Rama Chellappa, and Larry S. Davis.  Soft-nms – improving object detection with one line of code.  arXiv preprint arXiv:1704.04503, 2017.
  * [2] Daniel Bolya, Sean Foley, James Hays, and Judy Hoffman.  Tide: A general toolbox for identifying object detection errors.  In ECCV, 2020.
  * [3] Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko.  End-to-end object detection with transformers.  In European Conference on Computer Vision, pages 213–229, 2020\.
  * [4] Xinlei Chen and Abhinav Gupta.  Spatial memory for context reasoning in object detection.  CoRR, abs/1704.04224, 2017.
  * [5] Cheng Chi, Shifeng Zhang, Junliang Xing, Zhen Lei, Stan Li, and Xudong Zou.  Pedhunter: Occlusion robust pedestrian detector in crowded scenes.  In AAAI 2020 : The Thirty-Fourth AAAI Conference on Artificial Intelligence, 2020.
  * [6] Cheng Chi, Shifeng Zhang, Junliang Xing, Zhen Lei, Stan Li, and Xudong Zou.  Relational learning for joint head and human detection.  In AAAI 2020 : The Thirty-Fourth AAAI Conference on Artificial Intelligence, 2020.
  * [7] Myung Jin Choi, Antonio Torralba, and Alan S. Willsky.  A tree-based context model for object recognition.  IEEE Transactions on Pattern Analysis and Machine Intelligence, 34(2):240–252, 2012.
  * [8] Xuangeng Chu, Anlin Zheng, Xiangyu Zhang, and Jian Sun.  Detection in crowded scenes: One proposal, multiple predictions.  pages 12214–12223, 2020.
  * [9] Zhigang Dai, Bolun Cai, Yugeng Lin, and Junying Chen.  Up-detr: Unsupervised pre-training for object detection with transformers.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 1601–1610, June 2021.
  * [10] Santosh K. Divvala, Derek Hoiem, James H. Hays, Alexei A. Efros, and Martial Hebert.  An empirical study of context in object detection.  In 2009 IEEE Conference on Computer Vision and Pattern Recognition, pages 1271–1278, 2009.
  * [11] Pedro F. Felzenszwalb, Ross B. Girshick, David McAllester, and Deva Ramanan.  Object detection with discriminatively trained part-based models.  IEEE Transactions on Pattern Analysis and Machine Intelligence, 32(9):1627–1645, 2010.
  * [12] Carolina Galleguillos, Andrew Rabinovich, and Serge Belongie.  Object categorization using co-occurrence, location and appearance.  In 2008 IEEE Conference on Computer Vision and Pattern Recognition, pages 1–8, 2008.
  * [13] Peng Gao, Minghang Zheng, Xiaogang Wang, Jifeng Dai, and Hongsheng Li.  Fast convergence of DETR with spatially modulated co-attention.  CoRR, abs/2101.07448, 2021.
  * [14] Zheng Ge, Zequn Jie, Xin Huang, Rong Xu, and Osamu Yoshie.  PS-RCNN: detecting secondary human instances in a crowd via primary object suppression.  CoRR, abs/2003.07080, 2020.
  * [15] Kaiming He, Georgia Gkioxari, Piotr Dollár, and Ross Girshick.  Mask r-cnn.  computer vision and pattern recognition, 2017.
  * [16] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun.  Deep residual learning for image recognition.  In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 770–778, 2016.
  * [17] Yihui He, Xiangyu Zhang, Marios Savvides, and Kris Kitani.  Softer-nms: Rethinking bounding box regression for accurate object detection.  2018\.
  * [18] J. Hosang, R. Benenson, and B. Schiele.  Learning non-maximum suppression.  In CVPR, 2017.
  * [19] Jan Hendrik Hosang, Rodrigo Benenson, and Bernt Schiele.  A convnet for non-maximum suppression.  In 38th German Conference on Pattern Recognition, pages 192–204, 2016.
  * [20] Han Hu, Jiayuan Gu, Zheng Zhang, Jifeng Dai, and Yichen Wei.  Relation networks for object detection.  In 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3588–3597, 2018.
  * [21] Xin Huang, Zheng Ge, Zequn Jie, and Osamu Yoshie.  Nms by representative region: Towards crowded pedestrian detection by proposal pairing.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10750–10759, 2020.
  * [22] Diederik P Kingma and Jimmy Ba.  Adam: A method for stochastic optimization.  arXiv preprint arXiv:1412.6980, 2014.
  * [23] Jianan Li, Yunchao Wei, Xiaodan Liang, Jian Dong, Tingfa Xu, Jiashi Feng, and Shuicheng Yan.  Attentive contexts for object detection.  IEEE Transactions on Multimedia, 19(5):944–954, 2017.
  * [24] Matthieu Lin, Chuming Li, Xingyuan Bu, Ming Sun, Chen Lin, Junjie Yan, Wanli Ouyang, and Zhidong Deng.  Detr for crowd pedestrian detection.  arXiv preprint arXiv:2012.06785, 2020.
  * [25] Tsung-Yi Lin, Piotr Dollar, Ross Girshick, Kaiming He, Bharath Hariharan, and Serge Belongie.  Feature pyramid networks for object detection.  In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 936–944, 2017.
  * [26] Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollar.  Focal loss for dense object detection.  IEEE Transactions on Pattern Analysis and Machine Intelligence, 42(2):318–327, 2020.
  * [27] Tsung-Yi Lin, Michael Maire, Serge J. Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C. Lawrence Zitnick.  Microsoft coco: Common objects in context.  In European Conference on Computer Vision, pages 740–755, 2014\.
  * [28] Songtao Liu, Di Huang, and Yunhong Wang.  Adaptive nms: Refining pedestrian detection in a crowd.  In 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 6459–6468, 2019.
  * [29] Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo.  Swin transformer: Hierarchical vision transformer using shifted windows.  arXiv preprint arXiv:2103.14030, 2021.
  * [30] Ruiqi Lu and Huimin Ma.  Semantic head enhanced pedestrian detection in a crowd.  arXiv preprint arXiv:1911.11985, 2019.
  * [31] Andrew L Maas, Awni Y Hannun, Andrew Y Ng, et al.  Rectifier nonlinearities improve neural network acoustic models.  In Proc. icml, volume 30, page 3. Citeseer, 2013.
  * [32] Roozbeh Mottaghi, Xianjie Chen, Xiaobai Liu, Nam-Gyu Cho, Seong-Whan Lee, Sanja Fidler, Raquel Urtasun, and Alan Yuille.  The role of context for object detection and semantic segmentation in the wild.  In 2014 IEEE Conference on Computer Vision and Pattern Recognition, pages 891–898, 2014.
  * [33] Roozbeh Mottaghi, Xianjie Chen, Xiaobai Liu, Nam-Gyu Cho, Seong-Whan Lee, Sanja Fidler, Raquel Urtasun, and Alan Yuille.  The role of context for object detection and semantic segmentation in the wild.  In 2014 IEEE Conference on Computer Vision and Pattern Recognition, pages 891–898, 2014.
  * [34] Lu Qi, Shu Liu, Jianping Shi, and Jiaya Jia.  Sequential context encoding for duplicate removal.  In Advances in Neural Information Processing Systems, volume 31, pages 2049–2058, 2018.
  * [35] Danila Rukhovich, Konstantin Sofiiuk, Danil Galeev, Olga Barinova, and Anton Konushin.  Iterdet: Iterative scheme for objectdetection in crowded environments.  CoRR, abs/2005.05708, 2020.
  * [36] Niels Ole Salscheider.  Featurenms: Non-maximum suppression by learning feature embeddings.  In 2020 25th International Conference on Pattern Recognition (ICPR), pages 7848–7854, 2021.
  * [37] Shuai Shao, Zijian Zhao, Boxun Li, Tete Xiao, Gang Yu, Xiangyu Zhang, and Jian Sun.  Crowdhuman: A benchmark for detecting human in a crowd.  arXiv preprint arXiv:1805.00123, 2018.
  * [38] Russell Stewart, Mykhaylo Andriluka, and Andrew Y. Ng.  End-to-end people detection in crowded scenes.  In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 2325–2333, 2016.
  * [39] Peize Sun, Rufeng Zhang, Yi Jiang, Tao Kong, Chenfeng Xu, Wei Zhan, Masayoshi Tomizuka, Lei Li, Zehuan Yuan, Changhu Wang, and Ping Luo.  Sparse r-cnn: End-to-end object detection with learnable proposals.  arXiv preprint arXiv:2011.12450, 2020.
  * [40] Zhiqing Sun, Shengcao Cao, Yiming Yang, and Kris Kitani.  Rethinking transformer-based set prediction for object detection.  CoRR, abs/2011.10881, 2020.
  * [41] Zhi Tian, Chunhua Shen, Hao Chen, and Tong He.  Fcos: Fully convolutional one-stage object detection.  In 2019 IEEE/CVF International Conference on Computer Vision (ICCV), pages 9626–9635, 2019.
  * [42] Zhi Tian, Chunhua Shen, Hao Chen, and Tong He.  FCOS: A simple and strong anchor-free object detector.  2021\.
  * [43] Torralba, Murphy, Freeman, and Rubin.  Context-based vision system for place and object recognition.  In Proceedings Ninth IEEE International Conference on Computer Vision, pages 273–280 vol.1, 2003.
  * [44] Zhuowen Tu.  Auto-context and its application to high-level vision tasks.  In 2008 IEEE Conference on Computer Vision and Pattern Recognition, pages 1–8, 2008.
  * [45] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Ł ukasz Kaiser, and Illia Polosukhin.  Attention is all you need.  In I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc., 2017.
  * [46] Jianfeng Wang, Lin Song, Zeming Li, Hongbin Sun, Jian Sun, and Nanning Zheng.  End-to-end object detection with fully convolutional network.  arXiv preprint arXiv:2012.03544, 2020.
  * [47] Xinlong Wang, Tete Xiao, Yuning Jiang, Shuai Shao, Jian Sun, and Chunhua Shen.  Repulsion loss: Detecting pedestrians in a crowd.  arXiv preprint arXiv:1711.07752, 2017.
  * [48] Kevin Zhang, Feng Xiong, Peize Sun, Li Hu, Boxun Li, and Gang Yu.  Double anchor r-cnn for human detection in a crowd.  arXiv preprint arXiv:1909.09998, 2019.
  * [49] Shanshan Zhang, Rodrigo Benenson, and Bernt Schiele.  Citypersons: A diverse dataset for pedestrian detection.  In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 4457–4465, 2017.
  * [50] Shifeng Zhang, Cheng Chi, Yongqiang Yao, Zhen Lei, and Stan Z. Li.  Bridging the gap between anchor-based and anchor-free detection via adaptive training sample selection.  arXiv preprint arXiv:1912.02424, 2019.
  * [51] Shifeng Zhang, Longyin Wen, Xiao Bian, Zhen Lei, and Stan Z. Li.  Occlusion-aware r-cnn: Detecting pedestrians in a crowd.  In Proceedings of the European Conference on Computer Vision (ECCV), pages 637–653, 2018.
  * [52] Shifeng Zhang, Longyin Wen, Xiao Bian, Zhen Lei, and Stan Z. Li.  Single-shot refinement neural network for object detection.  In CVPR, 2018.
  * [53] Minghang Zheng, Peng Gao, Xiaogang Wang, Hongsheng Li, and Hao Dong.  End-to-end object detection with adaptive clustering transformer.  CoRR, abs/2011.09315, 2020.
  * [54] Xizhou Zhu, Weijie Su, Lewei Lu, Bin Li, Xiaogang Wang, and Jifeng Dai.  Deformable detr: Deformable transformers for end-to-end object detection.  In ICLR 2021: The Ninth International Conference on Learning Representations, 2021.

##  Appendix A Implementation of Deformable DETR with progressive predicting
method.

We also deploy our progressive predicting approach on the deformable DETR [54]
to demonstrate the generality of our method. Similar to Sparse RCNN [39], the
decoder in deformable DETR consists of 6 stages, which is depicted in Figure.
8(a). As described in the paper, we integrate our designed components into the
last decoding stage. Figure. 8(b) also depicts its detail architecture. For
the hyper-parameters setting, i.e confidence score threshold $s$, are
identical to those adopted in Sparse RCNN [39].

We choose the deformable DETR with iterative bounding box refinement.
Following deformable DETR [54], we use _ResNet-50_ [16] as backbone. The whole
detector is trained with an Adam optimizer [22] and a weight decay of 0.0001.
The total training duration is 50 epochs on 8 GPUs with 1 image per GPU. The
initial learning rate is 0.0002 and dropped by a factor of 0.1 after 40
epochs. The parameters initialization in the newly added components and losses
weights are identical to the original work [54]. The default number of queries
and stages is 500 and 6, respectively. The hyper-parameters $s$ and $\theta$
are also $0.7$ and $0.4$, respectively. The gradients are detached at proposal
boxes from the second stage to stabilize training. We stop gradient back-
propagation from the last stage to the previous ones. Besides, those negative
samples that overlap with ignore region with an intersection-over-area(IoA)
greater than $0.7$ are not involved in training.

$\mathbf{q}_{t-1}$$\mathbf{b}_{t-1}$MCAt-1MSAt-1$\mathbf{b}_{t}$$\mathbf{q}_{t}$MCAtMSAt$\mathbf{q}_{t+1}$$\mathbf{b}_{t+1}$
(a) Decoder in deformable DETR [54]. MCA – multi-head cross-attention, MSA –
multi-head self-attention.

$\mathbf{q}_{t-1}$$\mathbf{b}_{t-1}$MCAt-1MSAt-1$\mathbf{b}_{t}$$\mathbf{q}_{t}$MCAtLMSAt$\mathbf{q}_{t+1}$$\mathbf{b}_{t+1}$$\mathcal{S}$$\mathcal{R}$
(b) Decoder in SR-Deformable DETR (Ours). $\mathcal{S}$ – Prediction Selector,
$\mathcal{R}$ – Relation information extractor, LMSA – local multi-head self-
attention.

Figure 8:  8(a) is the architecture of decoding stage in deformable DETR [54];
8(b) describes the decoding stage structure equipped with our designed
components for progressive predicting schema.

##  Appendix B Performance change of a query-based decoder when handling
crowded scenes.

The performance of a _query-based_ detector would not be improved but will
degrade as the depth of a decoder increases when handling crowded scenes.
Experiments are conducted on _CrowdHuman_ dataset, taking Sparse RCNN based on
_ResNet-50_ as base detector. It equips with 500 queries. We adjust the depth
of its decoder while keeping the others unchanged. As is described in Table.
5, the performance degrades as the depth of the decoder increases.

#Depth | #Queries | AP | $\text{MR}^{-2}$ | JI
---|---|---|---|---
6 |  | 90.7 | 44.7 | 81.4
7 |  | 90.6 | 45.7 | 81.0
8 | 500 | 90.4 | 45.9 | 80.3
9 |  | 90.7 | 44.4 | 80.9
10 |  | 90.2 | 46.6 | 80.0
Table 5: Experiment analysis as the depth of a decoder increases, which
performs on _CrowdHuman_ dataset.

##  Appendix C Performance of query detector with large model in crowded
scenes.

To explore the detection upper bound of a query-based detector in tackling
crowded scenes, we replace the _ResNet-50_ with a large backbone, Swin-Large
[29]. Experiments are conducted on _CrowdHuman_ [37] and _CityPersons_ [49]
datasets, with the same training strategy described in the paper. As depicted
in Table. 6, our method can significantly boost the performance of a query-
based detector, which achieves a state-of-the-arts results on both
_CrowdHuman_ and _CityPersons_ validating datasets.

Method |  Dataset |  #Queries |  AP |  $\text{MR}^{-2}$ |  JI
---|---|---|---|---|---
S-RCNN |  |  500 |  93.1 |  39.9 |  85.1
S-RCNN+_Ours_ |  _CHuman_ |  500 |  93.8 |  37.4 |  86.5
D-DETR |  |  1000 |  93.4 |  39.6 |  86.3
D-DETR+_Ours_ |  |  1000 |  94.1 |  37.7 |  87.1
S-RCNN |  |  500 |  98.3 |  5.9 |  93.7
D-DETR |  _CPersons_ |  500 |  96.4 |  8.4 |  92.0
S-RCNN+_Ours_ |  |  500 |  98.4 |  4.9 |  94.2
D-DETR+_Ours_ |  |  500 |  97.5 |  5.9 |  93.7
Table 6: Experiment on _CHuman_(_CrowdHuman_) and _CPersons_(_CityPersons_)
with Swin-L [29]. S-RCNN – Sparse RCNN [39], D-DETR – Deformable DETR [54]

RelationNet [20] |  IterDet [35] |  Sparse R-CNN [39] |  D-DETR [3] | Sparse RCNN+Ours | D-DETR+Ours
---|---|---|---|---|---
|  |  |  |  |
|  |  |  |  |
|  |  |  |  |
|  |  |  |  |
|  |  |  |  |
|  |  |  |  |
|  |  |  |  |
|  |  |  |  |
|  |  |  |  |
Figure 9: Results visualization of RelationNet [20], IterDet [35], Sparse RCNN
[39], deformable DETR [54] and our approach based on them [39, 54]. Blue boxes
are true positive detections, light yellow boxes are missed instances and
orange boxes are false positives. Green boxes represent progressively refined
detections in our method.