# Efficient Human Pose Estimation by Learning Deeply Aggregated
Representations

Zhengxiong Luo$1$$2$$4$$5$$6$, Zhicheng Wang$1$, Yuanhao Cai$1$$3$, Guanan
Wang$1$$6$
Yan Huang$4$$5$$6$, Liang Wang$4$$5$$6$, Erjin Zhou$1$, Tieniu Tan$4$$5$, Jian
Sun$1$
$1$ Megvii Inc $2$ University of Chinese Academy of Sciences (UCAS) $3$
Tsinghua Shenzhen International Graduate School
$4$ Center for Research on Intelligent Perception and Computing (CRIPAC) $5$
National Laboratory of Pattern Recognition (NLPR)
$6$ Institute of Automation, Chinese Academy of Sciences (CASIA)
{luozhengxiong, wangzhicheng<EMAIL_ADDRESS>
{yhuang, wangliang<EMAIL_ADDRESS>
###### Abstract

In this paper, we propose an efficient human pose estimation network (DANet)
by learning deeply aggregated representations. Most existing models explore
multi-scale information mainly from features with different spatial sizes.
Powerful multi-scale representations usually rely on the cascaded pyramid
framework. This framework largely boosts the performance but in the meanwhile
makes networks very deep and complex. Instead, we focus on exploiting multi-
scale information from layers with different receptive-field sizes and then
making full of use this information by improving the fusion method.
Specifically, we propose an orthogonal attention block (OAB) and a second-
order fusion unit (SFU). The OAB learns multi-scale information from different
layers and enhances them by encouraging them to be diverse. The SFU adaptively
selects and fuses diverse multi-scale information and suppress the redundant
ones. This could maximize the effective information in final fused
representations. With the help of OAB and SFU, our single pyramid network may
be able to generate deeply aggregated representations that contain even richer
multi-scale information and have a larger representing capacity than that of
cascaded networks. Thus, our networks could achieve comparable or even better
accuracy with much smaller model complexity. Specifically, our DANet-72
achieves $70.5$ in AP score on COCO test-dev set with only $1.0G$ FLOPs. Its
speed on a CPU platform achieves $58$ Persons-Per-Second (PPS). The source
code will be publicly available for further research.

##  I Introduction

Human pose estimation (HPE) aims to localize all skeletal keypoints of given
persons in a single RGB image. It has wide applications in human activity
recognition, human-computer interaction, animation, etc. To attain sub-pixel
accuracy, HPE needs both local spatially precise and global semantically
discriminant representations. Thus rich multi-scale information is quite
important for HPE. Towards this motivation, as shown in Figure 2 (a), most
existing models [1, 2, 3, 4, 5] use a cascaded pyramid framework to repeatedly
extract and fuse pyramid features, i.e. features with different spatial sizes.
This framework largely boosts the performance of HPE but makes these networks
very deep and complex.

As shown in Figure 2 (b), a single pyramid network is much simpler. However,
without repeatedly extracting and fusing pyramid features, it can only
generate limited multi-scale information from features of four different
spatial sizes, which affects the accuracy a lot. This motivates us to think:
if features of every single spatial size could independently learn rich multi-
scale information, and further this information can be fused more effectively,
even a single pyramid network may be able to achieve remarkable accuracy. In
this paper, we propose a novel network, namely deeply aggregated network
(DANet). It aggregates layers with different receptive-fields sizes, in which
way multi-scale information could be extracted within features of the same
spatial size. And the extracted information is further fused by a second-order
fusion, which can maximize the effective representations in the final fused
features.

Figure 1: AP score of different models on COCO validation set with respect to
(a) flops and (b) speed. Figure 2: Different framework for human pose pose
estimation. (a) Cascaded Pyramid Network, (b) Single Pyramid Network.

Specifically, DANet has two key components, which are orthogonal attention
block (OAB) and second-order fusion unit (SFU). The OAB aggregates different
layers via a densely connected architecture similar to [6] and further
enhances the multi-scale information from two views. A mask attention unit is
designed to explicitly encourage different layers to be diverse by masking the
representations of input features out from output features. Thus, the top
features include less repeated information from the bottom layers. A channel
attention unit is used to improve the nonlinearity of each layer and thus help
the output features to be more different from the input ones. The SFU
adaptively selects and fuses multi-scale information from multifarious
features. It highlights the diversities of features and suppresses their
similarities via adaptive weights, which are obtained by modeling dependencies
among different features. As a result, the effective information in the final
fused features can be maximized. With the help of OAB and SFU, our single
pyramid network may be able to generate deeply aggregated representations,
which even contain more powerful multi-scale information and have a larger
representing capacity than that of cascaded networks. Consequently, comparable
or even better accuracy can be achieved with much smaller model complexity.

Our contributions can be summarized as four points:

  * 1.

Towards efficient human pose estimation (HPE), we propose a deeply aggregated
network (DANet), which focuses on exploring the multi-scale information within
features of the same spatial sizes and improving the method that is used to
fuse multi-scale information. As a result, our single pyramid network can
generate deeply aggregated representations which can produce better accuracy
than cascaded networks but with much smaller model complexity.

  * 2.

We propose a novel orthogonal attention block (OAB), which explores and
enhances multi-scale information from layers with different receptive-field
sizes. It encourages different layers to be more diverse and enables layer-
wise aggregated features to have a higher capacity.

  * 3.

We propose a novel second-order fusion unit (SFU), which adaptively selects
and fuses multi-scale information from multifarious features by modeling their
mutual dependencies. In this way, it maximizes the effective information in
the final fused features.

  * 4.

Experiments on both COCO and MPII datasets show that DANets could achieve a
better balance between cost and effectiveness than existing models.
Specifically, our DANet-72 can achieve $70.5$ in AP score on COCO test-dev set
with only $1.0G$ FLOPs. Its speed on the CPU platform achieves about $58$
Persons-Per-Second (PPS).

##  II Related Work

###  II-A Pyramid Fusion

Pyramid fusion is popular among models for human pose estimation (HPE),
semantic segmentation [7] and object detection [8, 9, 10, 11, 12]. These tasks
all require representations rich in multi-scale information. For human pose
estimation and semantic segmentation, global information is needed for
semantic discrimination and local information is used to determine spatial
positions. While in object detection, scale-invariant representations are
critical for hunting objects of various sizes. There are usually two paths in
these models, i.e. the top-down path and the bottom-up path. In the top-down
path, a backbone such as ResNet [13] is usually used to extract the pyramid
features. The feature sizes go from $1/4$ to $1/32$ of original images.

And in the bottom-up path, features with different spatial sizes are usually
directly summed up to form the final representations. But simple summation
ignores the dependencies between these features and thus limits the
representing capacity of the final representations. In this paper, we improve
this method by proposing a second-order fusion unit (SFU), which adaptively
selects and fuse multi-scale information from multifarious features by
modeling their mutual dependencies. In this way, it maximizes the effective
information in the final fused features.

###  II-B Cascaded Pyramid Framework

For segmentation and detection, a single feature pyramid can usually satisfy
the requirements of both tasks, even though the multi-scale information in a
single pyramid is relatively limited because both tasks are indeed not so
sensitive to sub-pixel accuracy as HPE. Sub-pixel localization needs much
finer and richer multi-scale information. Consequently, HPE networks usually
use a cascaded pyramid framework, which repeatedly extracts and fuse the
feature pyramid and generates representations that can detect the sub-pixel
difference. Since the proposal of Stacked Hourglass [1], the cascaded pyramid
framework has nearly become the dominant method in HPE [2, 4, 14, 15].
Recently, HRNet proposed in [16] uses a slightly different architecture that
can maintain the high-resolution representations during the whole network. But
in fact, it also repeatedly exchanges information between different scales.
Single pyramid networks such as SimpleBaseline [17, 18, 19] can only produce
inferior results.

The cascaded pyramid framework largely boosts the performance of HPE networks,
but it makes these networks very deep and complex in the meanwhile. To produce
relatively accurate results with a single pyramid network, we pay our
attention to layer-wise aggregation and a more effective fusion method. With
the help of diverse layer-wise aggregation and higher-order fusion, the
proposed DANet can generate deeply aggregated representations that contain
even richer multi-scale information and have a larger representing capacity
than cascaded networks.

###  II-C Densely Connected Architecture

Densely connected architecture is firstly proposed in [6] for image
classification. It constantly concatenates newly generated features to
previous ones. This inner structure makes it naturally have the property of
multi-scale fusion [20]. Compared with the fusion between features with
different spatial sizes, this layer-wise fusion has much finer granularity. It
also largely saves the computation because of feature reuse [6]. A similar
idea can also be discovered in [21], which designs an extremely efficient
network on the benefit of feature reuse. The multi-scale fusion and feature
reuse in densely connected architecture make it especially suitable for
efficient human pose estimation. But as discussed in [22, 21], original
densely connected architecture usually suffers from redundant connections,
which will harm the overall efficiency. In this paper, the proposed orthogonal
attention block (OAB) uses a similar architecture, but it is enhanced by mask
and channel attention units. These two attention units can explicitly help
different layers learn more diverse representations, which largely improves
the representing capacity and reduces the proportion of redundant connections.

Figure 3: The overview of DANet. $*$ means online hard keypoints mining (OHKM)
[18]. OAB denotes Orthogonal Attention Block and SFU denotes Second-order
Attention Unit.

##  III Proposed Method

In this section, we firstly introduce the overall structure of our network and
then go to the details.

As we have discussed above, we adopt a single pyramid framework. We hope the
features of every single spatial size can independently learn rich multi-scale
information. We notice that layers within the same spatial size also have
different receptive-field sizes. Thus, aggregation among these layers may also
be able to exploit multi-scale information. Motivated by this, we propose an
orthogonal attention block (OAB). It performs layer-wise aggregation via a
densely connected architecture similar to [6]. However, we further enhance it
by mask and channel attention units, which can explicitly help different
layers to be more diverse and reduce the redundant connections. It will be
further discussed in Sec III-A.

As shown in Figure 3, in the top-down path, OAB extracts a feature pyramid,
which contains features with different spatial sizes. And within each spatial
size, there are also layers with different receptive-field sizes. These
hierarchical features largely enrich multi-scale information. In the bottom-up
path, the pyramid features are fused by the proposed second-order fusion unit
(SFU), which can further maximize the effective information in the final fused
representations. The details will be discussed in Sec III-B. At last, the
deeply aggregated representations are then inputted to a regressor to estimate
the heatmaps for all keypoints.

Figure 4:  Details of Orthogonal Attention Block (OAB) and the details of
Second-order Fusion Unit (SFU). $k\times k$ means a convolution block
composited in the order of $BN-ReLU-conv$. And ‘DW’ means depthwise
convolution. ‘GAP’ denotes Global Average Pooling.

###  III-A Orthogonal Attention Block

As we have discussed in Sec II-C, original densely connected architecture
suffers from redundant connections. It can perform layer-wise aggregation to
exploit multi-scale information from different layers. However, it does not
take full advantage of this architecture.

To further explore the benefits of densely connected architecture and
alleviate the problem of redundant connections, we hope different layers can
learn representations as diverse as possible. Because in this way, the layer-
wise aggregated features can contain more effective information and have
higher representing capacity. Towards this motivation, two methods are
proposed. The first one is explicitly forcing different layers to learn
different representations, which is realized via a mask attention unit (MAU).
And the second one is improving the nonlinearity of each layer. which is
realized via a channel attention unit (CAU).

Mask Attention. To explicitly encourage newly generated features to be
different from input ones, we propose a mask attention unit (MAU) to mask
representations of input features out from output features. Thus, the top
features will include less repeated information from bottom layers. As shown
in Figure 4 (a), we firstly extract an attention map , denoted as $M$, which
represents representations of input features. Then we multiply $(1-M)$ to the
output features to mask out previous representations. To mathematically
illustrate this unit, suppose $f^{d}_{p}$ denotes input features of the
$d^{th}$ layer, and $f^{d}_{n}$ denotes the newly generated features. Then in
formula,

$\displaystyle M=\psi^{d}(f^{d}_{p})$ $\displaystyle M=Sigmoid(Abs(M))$ (1)
$\displaystyle f^{d}_{n}=\phi^{d}(f^{d}_{p})\times(1-M)$ $\displaystyle
f^{d+1}_{p}=Cat(f^{d}_{p},f^{d}_{n}),$

where $\phi^{d}(\cdot)$ denotes the mapping function of $d^{th}$ layer, and
$\psi^{d}(\cdot)$ denotes the function used to generate $M$.
$Cat(\cdot,\cdot)$ denotes concatenation. Through $Abs(\cdot)$ and
$Sigmoid(\cdot)$, $M$ becomes an attention map which highlights pixels with
relatively large values. Then $(1-M)$ acts like a mask, which masks
representations of $f^{d}_{p}$ out from $f^{d}_{n}$, and encourages it to be
more diverse.

We use a large kernel size, i.e. $9\times 9$ in MAU to extract the attention
map. Kernels with larger sizes are more robust to local noises and can extract
more informative attention. To reduce the computation cost, we use a depthwise
convolution to make a trade-off. Similar settings can also be found in [23,
24].

Channel Attention. Motivated by the success of SE unit [25] for ResNet [13],
we also use a channel attention unit (CAU) to improve the nonlinearity of each
layer. As shown in Figure 4 (a), a channel attention unit is inserted before
the $1\times 1$ convolutional layer. By modeling the correlation between
different channels, this unit is can effectively improve the overall
nonlinearity of original convolutional layers, and help the output feature to
be more different from input features.

With both mask and channel attention units, OAB can be mathematically
expressed as

$\displaystyle M=MA(f^{d}_{p})$ $\displaystyle C=CA(f^{d}_{p})$ (2)
$\displaystyle f^{d}_{n}=\phi^{d}(f^{d}_{p}\times C)\times(1-M)$
$\displaystyle f^{d+1}_{p}=Cat(f^{d}_{p},f^{d}_{n}),$

where $CA(\cdot)$ denotes the function that generates channel weights $C$, and
$MA(\cdot)$ denotes the function that generates attention map $M$. $C$ helps
improve the nonlinearity of $\phi^{d}(\cdot)$, and $1-M$ helps mask
representations of $f^{d}_{p}$ out from $f^{d}_{n}$. With the help of MAU and
CAU, OAB can generate layers with diverse representation, which largely
enriches the multi-scale information in the single pyramid, and helps DANet
generate deeply aggregated representations.

TABLE I: Details of different DANets.

Layers | Output Size | DANet-72 | DANet-88 | DANet-98 | DANet-102
---|---|---|---|---|---
Convolution | $128\times 96$ |  $7\times 7\times 64$, stride 2
Convolution | $64\times 48$ |  $3\times 3\times 64$, stride 2
Orthogonal Block (1) | $64\times 48$ | $\left[\begin{aligned} &1\times 1\quad 32\\\ &3\times 3\quad 32\end{aligned}\right]\times 3$ | $\left[\begin{aligned} &1\times 1\quad 32\\\ &3\times 3\quad 32\end{aligned}\right]\times 4$ | $\left[\begin{aligned} &1\times 1\quad 32\\\ &3\times 3\quad 32\end{aligned}\right]\times 4$ | $\left[\begin{aligned} &1\times 1\quad 32\\\ &3\times 3\quad 32\end{aligned}\right]\times 6$
Transition Layer (1) | $64\times 48$ | $1\times 1\times 96$ | $1\times 1\times 128$ | $1\times 1\times 128$ | $1\times 1\times 128$
$32\times 24$ |  $3\times 3$ MaxPooling, stride 2
Orthogonal Block (2) | $32\times 24$ | $\left[\begin{aligned} &1\times 1\quad 32\\\ &3\times 3\quad 32\end{aligned}\right]\times 6$ | $\left[\begin{aligned} &1\times 1\quad 32\\\ &3\times 3\quad 32\end{aligned}\right]\times 5$ | $\left[\begin{aligned} &1\times 1\quad 32\\\ &3\times 3\quad 32\end{aligned}\right]\times 12$ | $\left[\begin{aligned} &1\times 1\quad 32\\\ &3\times 3\quad 32\end{aligned}\right]\times 12$
Transition Layer (2) | $32\times 24$ | $1\times 1\times 192$ | $1\times 1\times 192$ | $1\times 1\times 256$ | $1\times 1\times 256$
$16\times 12$ |  $3\times 3$ MaxPooling, stride 2
Orthogonal Block (3) | $16\times 12$ | $\left[\begin{aligned} &1\times 1\quad 32\\\ &3\times 3\quad 32\end{aligned}\right]\times 12$ | $\left[\begin{aligned} &1\times 1\quad 32\\\ &3\times 3\quad 32\end{aligned}\right]\times 18$ | $\left[\begin{aligned} &1\times 1\quad 32\\\ &3\times 3\quad 32\end{aligned}\right]\times 16$ | $\left[\begin{aligned} &1\times 1\quad 32\\\ &3\times 3\quad 32\end{aligned}\right]\times 16$
Transition Layer (3) | $16\times 12$ | $1\times 1\times 384$ | $1\times 1\times 512$ | $1\times 1\times 512$ | $1\times 1\times 512$
$8\times 6$ |  $3\times 3$ MaxPooling, stride 2
Orthogonal Block (4) | $8\times 6$ | $\left[\begin{aligned} &1\times 1\quad 32\\\ &3\times 3\quad 32\end{aligned}\right]\times 12$ | $\left[\begin{aligned} &1\times 1\quad 32\\\ &3\times 3\quad 32\end{aligned}\right]\times 14$ | $\left[\begin{aligned} &1\times 1\quad 32\\\ &3\times 3\quad 32\end{aligned}\right]\times 14$ | $\left[\begin{aligned} &1\times 1\quad 32\\\ &3\times 3\quad 32\end{aligned}\right]\times 14$
Transition Layer (4) | $8\times 6$ | $1\times 1\times 512$ | $1\times 1\times 640$ | $1\times 1\times 640$ | $1\times 1\times 640$

###  III-B Second-order Fusion

To better utilize the extracted features, we also improve the fusion method.
Most existing models fuse the pyramid features by simple summation or a set of
learned but still fixed weights. These fusion methods ignore the mutual
dependencies between the pyramid features. Thus we propose a second-order
fusion unit (SFU) aimed to utilize the second-order relationship. As shown in
Figure 4 (b), we firstly concatenate the two input features. Then they are fed
to a three-layer convolutional block to generate the weight map $\lambda$,
which represents the mutual dependencies between the original two features.
These two features are then weighted by $\lambda$ and $1-\lambda$. Since
$\lambda$ and $1-\lambda$ have contrary patterns, the weighted features will
be decoupled to more independent ones. In this way, we suppress their
similarities while highlighting their diversities. As a result, there will be
less redundant information but more effective information in the final fused
features.

Suppose $f_{td}$ denotes feature maps from the top-down path and $f_{bu}$
denotes feature maps from the bottom-up path. Then SFU can be expressed as

$\small\lambda=Sigmoid(\theta(Cat(f_{td},f_{bu})))\quad f=\lambda
f_{td}+(1-\lambda)f_{bu},$ (3)

where $\theta(\cdot)$ denotes the function that generates aggregating weights.
In practice, this function is a combination of pointwise convolution and
depthwise convolution. The kernel size of depthwise convolution is also set as
$9$.

###  III-C Network instantiation

We instantiate DANets by determining the number of layers in OAB, i.e. the
number of layers in the top-down path. The growth rates for all the densely
connected architectures are set as $32$. The output channels of the $1\times
1$ bottleneck are also set as $32$. We adjust the output channels and the
number of layers for each stage to form models with different model sizes.
DANet-$S$ denotes that there are $S$ layers in the top-down path. The details
for DANet-72, DANet-88, DANet-98 and DANet-102 are shown in Table I.

##  IV Experiments

###  IV-A Experiments on COCO datasets

Dataset. We use COCO train2017 [26] as our training dataset, which includes
57K images and 150K person instances. We use COCO va2017 (5K images) and COCO
test-dev set (20K images) to evaluate our models. The evaluation metric is
based on Object Keypoint Similarity (OKS).

Training. We randomly rotate, flip and scale human boxes to augment training
instances. The rotation range is $[-45^{\circ},+45^{\circ}]$, and the scaling
range is $[0.7,1.35]$. Following [27], half-body transform is also performed
for data augmentation. The human boxes will be firstly extended to $4:3$ in
terms of hight-width ratio, and then resized to $256\times 192$. We use Adam
[28] as our optimizer. The initial learning rate is set as $3\times 10^{-4}$
and it decays linearly to $0$ at the end of the training. The batch size is
$256$ and we train all models by $2.8\times 10^{7}$ iterations no RTX2080TI
GPUs. All models are trained on $8$ 2080Ti GPUs.

Testing. During testing, we use the same detecting results as [19, 16, 18],
i.e. $56.4$ in score of AP on COCO val2017 set. Following the strategy in [1],
images and their flipped versions are all evaluated. Their predicted heatmaps
are averaged and then blurred by a Gaussian filter. To obtain the final
location, Instead of directly extracting the location of the highest response
in heatmaps, we adjust it by a quarter offset in the direction of the second-
highest response.

TABLE II: Comparison of results on COCO validation set. ’*’ denotes
reimplementing results. We cut the depth of HRNet-W32 by half and set its base
channel as 16 to form HRNet-W16.

Method |  Backbone |  Pretrain |  Input Size |  Params (M) |  GFLOPs |  AP |  AP50 |  AP75 |  APM |  APL |  AR
---|---|---|---|---|---|---|---|---|---|---|---
8-stage Hourglass [1] | Hourglass | N | $256\times 192$ | $25.2$ | $26.2$ | $66.9$ | - | - | - | - | -
CPN [18] | ResNet50 | Y | $256\times 192$ | $27.0$ | $6.2$ | $68.6$ | - | - | - | - | -
SimpleBaseline [19] | ResNet50 | Y | $256\times 192$ | $34.0$ | $8.9$ | $70.4$ | $80.6$ | $78.3$ | $67.1$ | $77.2$ | $76.3$
HRNet-W32 [16] | HRNet-W32 | Y | $256\times 192$ | $28.5$ | $7.1$ | $\color[rgb]{1,0,0}\mathbf{74.4}$ | $\color[rgb]{0,0,1}\mathbf{90.5}$ | $\color[rgb]{1,0,0}\mathbf{81.9}$ | $\color[rgb]{1,0,0}\mathbf{70.8}$ | $\color[rgb]{1,0,0}\mathbf{81.0}$ | $\color[rgb]{1,0,0}\mathbf{79.8}$
MSPN [2] | ResNet50 | N | $256\times 192$ | $95.5$ | $4.7$ | $73.3$ | $90.45$ | $80.3$ | $69.2$ | $\color[rgb]{0,0,1}\mathbf{79.4}$ | $78.6$
HRNet-W16 | *HRNet-W16 | N | $256\times 192$ | $6.9$ | $1.9$ | $71.8$ | $89.5$ | $78.9$ | $68.6$ | $76.8$ | $77.2$
MSPN [2] | ResNet18 | N | $256\times 192$ | $11.7$ | $2.5$ | $70.8$ | $89.5$ | $77.3$ | $66.6$ | $76.9$ | $76.2$
MSPN [2] | ResNet34 | N | $256\times 192$ | $21.4$ | $4.2$ | $73.1$ | $90.5$ | $80.2$ | $69.1$ | $79.1$ | $78.5$
DANet-72 | DANet-72 | N | $256\times 192$ | $\color[rgb]{1,0,0}\mathbf{3.4}$ | $\color[rgb]{1,0,0}\mathbf{1.0}$ | $71.0$ | $89.6$ | $78.1$ | $67.6$ | $76.2$ | $76.6$
DANet-88 | DANet-88 | N | $256\times 192$ | $5.3$ | $1.4$ | $71.9$ | $90.0$ | $79.0$ | $68.2$ | $77.3$ | $77.5$
DANet-98 | DANet-98 | N | $256\times 192$ | $5.7$ | $1.6$ | $72.8$ | $90.3$ | $79.9$ | $69.2$ | $78.3$ | $78.4$
DANet-102 | DANet-102 | N | $256\times 192$ | $5.8$ | $1.8$ | $\color[rgb]{0,0,1}\mathbf{73.4}$ | $\color[rgb]{1,0,0}\mathbf{90.6}$ | $\color[rgb]{0,0,1}\mathbf{80.5}$ | $\color[rgb]{0,0,1}\mathbf{69.9}$ | $78.6$ | $\color[rgb]{0,0,1}\mathbf{78.8}$

TABLE III: Comparison of results on COCO test-dev set.

Method |  Backbone |  Input Size |  Params (M) |  GFLOPs |  AP |  AP50 |  AP75 |  APM |  APL |  AR
---|---|---|---|---|---|---|---|---|---|---
CPN [18] |  ResNet-Inception | $384\times 288$ | $-$ | $-$ | $72.1$ | $91.4$ | $80.0$ | $68.7$ | $77.2$ | $78.5$
SimpleBaseline [19] | ResNet50 | $256\times 192$ | $34.0$ | $8.9$ | $70.0$ | $90.9$ | $77.9$ | $66.8$ | $75.8$ | $75.6$
SimpleBaseline [19] | ResNet152 | $256\times 192$ | $68.6$ | $15.7$ | $71.6$ | $91.2$ | $80.1$ | $68.7$ | $77.2$ | $77.3$
HRNet-W32 [16] | HRNet-W32 | $384\times 288$ | $28.5$ | $16.0$ | $\color[rgb]{1,0,0}\mathbf{74.9}$ | $\color[rgb]{0,0,1}\mathbf{92.5}$ | $\color[rgb]{1,0,0}\mathbf{82.8}$ | $\color[rgb]{1,0,0}\mathbf{71.3}$ | $\color[rgb]{1,0,0}\mathbf{80.9}$ | $\color[rgb]{1,0,0}\mathbf{80.1}$
DANet-72 | DANet-72 | $256\times 192$ | $\color[rgb]{1,0,0}\mathbf{3.4}$ | $\color[rgb]{1,0,0}\mathbf{1.0}$ | $70.5$ | $91.8$ | $78.2$ | $67.5$ | $75.0$ | $76.7$
DANet-88 | DANet-88 | $256\times 192$ | $5.3$ | $1.4$ | $71.5$ | $91.9$ | $79.5$ | $68.5$ | $76.0$ | $77.6$
DANet-98 | DANet-98 | $256\times 192$ | $5.7$ | $1.6$ | $72.6$ | $92.5$ | $80.7$ | $69.4$ | $77.2$ | $78.5$
DANet-102 | DANet-102 | $256\times 192$ | $5.8$ | $1.8$ | $\color[rgb]{0,0,1}\mathbf{72.9}$ | $\color[rgb]{1,0,0}\mathbf{92.6}$ | $\color[rgb]{0,0,1}\mathbf{80.8}$ | $\color[rgb]{0,0,1}\mathbf{69.8}$ | $\color[rgb]{0,0,1}\mathbf{77.4}$ | $\color[rgb]{0,0,1}\mathbf{78.8}$

Comparison with Existing Methods. Intensive experiments are done on DANet-72,
DANet-88, DANet-98, and DANet-102. The details of DANets are shown in Table I.
Reference models mainly include the families of HRNet [16], SimpleBaseline
[19] and MSPN [2]. As these models are usually much larger than DANets, we
reimplement some of their smaller versions to make a more comprehensive
comparison. The results on COCO val2017 and COCO test-dev are shown in Table
II and Table III respectively. Our DANet-102 has only $1.8G$ FLOPs but
achieves comparable results with models over $4G$ FLOPs. DANet-72 has even
only $1.0G$ FLOPs, but still achieves $71.0$ AP on COCO val2017 and $70.5$ AP
on COCO test-dev. To better illustrate the efficiency of DANets, we plot the
AP with respect to the FLOPs of different models. As shown in Figure 1 (a),
DANets take up the left corner of the diagram, which means that it can always
achieve better accuracy but with smaller model complexity.

Ablation Study. We investigate each component, i.e. CAU, MAU, and SFU, in
DANet respectively. We remove all components in DANet102 as Baseline1, and
then gradually add each component back. Since extra components will introduce
additional flops and parameters, to make a more comprehensive comparison, we
increase the growth rate in Baseline1 from $32$ to $64$ as Baseline2. The
results on COCO val2017 dataset are shown in Table IV. CAU, MAU, and SFU can
improve the AP score by $1.5$, $1.9$, and $0.7$ over baseline1 respectively.
Although extra flops and parameters are introduced, they have better cost-
effectiveness than Baseline2.

TABLE IV: Abation of study of different components in DANet-102. Results are
reported on COCO2017 val dataset.

Components | CAU | MAU | SFU | GFLOPs | Params(M) | AP
---|---|---|---|---|---|---
Baseline1 |  |  |  | $1.3$ | $3.4$ | $70.2$
Baseline2 |  |  |  | $1.8$ | $4.5$ | $71.6$
CAU | ✓ |  |  | $1.3$ | $4.8$ | $71.7$
MAU |  | ✓ |  | $1.6$ | $4.2$ | $72.1$
SFU |  |  | ✓ | $1.5$ | $3.5$ | $70.9$
CAU + MAU | ✓ | ✓ |  | $1.6$ | $5.7$ | $72.6$
CAU + MAU + SFU | ✓ | ✓ | ✓ | $1.8$ | $5.8$ | $73.4$

Further Discussion about CAU. The idea channel attention unit (CAU) is firstly
proposed in [25]. And since its proposal, it has been in used various models.
Although they have a similar inner structure, i.e. global pooling followed by
two fully connected layers and activation function, the position that CAU is
inserted counts a lot. In our case, when MAU is introduced, there are several
variants of OAB. As shown in Figure 5, depending on the position of CAU, OAB
has two other variants. Comparative experiments are done between them. Results
are shown in Table V. As one can see, although different variants have the
same flops and number of parameters, OAB is the best choice.

Figure 5: Different variants of channel attention in Orthogonal Attention
Blolck. TABLE V: Ablation study about variants of OAB. Results are reported on
COCO2017 val dataset.

Architecture | GFLOPs | Params | AP
---|---|---|---
Variant 1 | $1.6$ | $5.7$ | $72.3$
Variant 2 | $1.6$ | $5.7$ | $72.3$
OAB | $1.6$ | $5.7$ | $\mathbf{72.6}$

Further Discussion about MAU. Based on densely connected architecture, we
further prove the superiority of mask attention unit (MAU). Generally
speaking, mask attention unit can be regarded as a kind of spatial attention
unit, but it is more elaborate designed. To validate the details in MAU, we
compare it with other kinds of variants. We leave out the CAU and SFU in
DANet-102 as the Baseline1. Since attention units will introduce extra
complexity, we modify the growth rate of densely connected architecture from
$32$ to $64$ to form Baseline2. The first variant is called xunit [24], which
is also used in DxNet [23]. We also modify
$f^{d}_{n}=\phi^{d}(f_{p})\times(1-M)$ in Equation 1 to
$f^{d}_{n}=\phi^{d}(f_{p})\times M$ and $f^{d}_{n}=\phi^{d}(f_{p})\times(1+M)$
to form MAU-Variant1 and MAU-Variant2 respectively.

As shown in Table VI, despite that MAU introduces extra model complexity over
Baseline1, it outperforms Baseline2 by a large margin with smaller model size
and complexity. And among all kinds of spatial attention units, MAU also has
the best performance. Variant1 and Variant2 have the same complexity as MAU
but have inferior performances. This comparison indicates that every detail in
MAU plays an important role.

TABLE VI: Ablation study of different spatial attentions. Results on COCO
validation set are reported.

Methods | GFLOPs | Params (M) | AP
---|---|---|---
Baseline1 | $1.3$ | $3.4$ | $70.2$
Baseline2 | $1.8$ | $4.5$ | $71.6$
xunit [23] | $1.4$ | $3.6$ | $71.6$
MAU-Variant1 | $1.6$ | $4.2$ | $71.8$
MAU-Variant2 | $1.6$ | $4.2$ | $71.9$
MAU | $1.6$ | $4.2$ | $\mathbf{72.1}$

Further Discussion about SFU. The proposed second-order fusion unit can
maximize the effective information in the final fused features. To prove that
it works with different kinds of input features, ablation studies are done on
different backbones. Three backbone are chosen, i.e. Res18 [13], DANet-102
without MAU and CAU, and DANet-102. We investigate their performances when the
pyramid features are fused by summation or SFU. As shown in Table VII, with
different backbones, SFU can always improve the performance by a large margin.
Especially, with the help of SFU, DANet-102 can learn further deeply
aggregated representations, and the performance is improved by $0.8$ in the
score of AP, which makes the extra $0.2G$ FLOPs worthwhile.

TABLE VII: Ablation study of second-order fusion. Results on COCO validation
set are reported.

Backbone | Fusion | GFLOPs | Params (M) | AP
---|---|---|---|---
Res18 | Sum | $2.5$ | $11.8$ | $70.8$
SFU | $2.7$ | $11.9$ | $\mathbf{71.4}$
DANet-102 (No MAU and CAU) | Sum | $1.3$ | $3.4$ | $70.2$
SFU | $1.5$ | $3.5$ | $\mathbf{70.9}$
DANet-102 | Sum | $1.6$ | $5.7$ | $72.6$
SFU | $1.8$ | $5.8$ | $\mathbf{73.4}$

TABLE VIII: Results on MPII test dataset.

Methods | GFLOPs | Params(M) | Head | Shoulder | Elbow | Wrist | Hip | Knee | Ankle | Mean
---|---|---|---|---|---|---|---|---|---|---
Newell et al. [1] | $55$ | $26$ | $98.2$ | $96.3$ | $91.2$ | $87.1$ | $90.1$ | $87.4$ | $83.6$ | $90.9$
Ning et al. [26] | $124$ | $74$ | $98.1$ | $96.3$ | $92.2$ | $87.8$ | $90.6$ | $87.6$ | $82.7$ | $91.2$
Peng et al. [29] | $55$ | $26$ | $98.1$ | $96.6$ | $92.5$ | $\color[rgb]{0,0,1}\mathbf{88.4}$ | $90.7$ | $87.7$ | $83.5$ | $91.5$
Chu et al. [30] | $128$ | $58$ | $98.5$ | $96.3$ | $91.9$ | $88.1$ | $90.6$ | $88.0$ | $\color[rgb]{0,0,1}\mathbf{85.0}$ | $91.5$
Yang et al. [3] | $46$ | $28$ | $\color[rgb]{1,0,0}\mathbf{98.5}$ | $\color[rgb]{1,0,0}\mathbf{96.7}$ | $\color[rgb]{1,0,0}\mathbf{92.5}$ | $\color[rgb]{1,0,0}\mathbf{88.7}$ | $\color[rgb]{1,0,0}\mathbf{91.7}$ | $\color[rgb]{1,0,0}\mathbf{88.6}$ | $\color[rgb]{1,0,0}\mathbf{86.0}$ | $\color[rgb]{1,0,0}\mathbf{92.0}$
Sekii et al. [31] | $6$ | $16$ | $-$ | $-$ | $-$ | $-$ | $-$ | $-$ | $-$ | $88.1$
FPD [4] | $9$ | $\color[rgb]{1,0,0}\mathbf{3}$ | $98.3$ | $96.4$ | $91.5$ | $87.4$ | $90.9$ | $87.1$ | $83.7$ | $91.1$
DANet-72 | $\color[rgb]{1,0,0}\mathbf{1.6}$ | $3.4$ | $97.9$ | $96.3$ | $90.9$ | $87.0$ | $90.7$ | $87.1$ | $82.3$ | $90.6$
DANet-88 | $2.8$ | $5.3$ | $98.0$ | $96.4$ | $91.5$ | $87.3$ | $90.8$ | $87.6$ | $82.9$ | $91.0$
DANet-98 | $2.2$ | $5.7$ | $98.1$ | $96.5$ | $92.1$ | $87.6$ | $90.8$ | $88.2$ | $83.5$ | $91.4$
DANet-102 | $2.4$ | $5.8$ | $98.1$ | $\color[rgb]{1,0,0}\mathbf{96.7}$ | $\color[rgb]{0,0,1}\mathbf{92.3}$ | $87.9$ | $\color[rgb]{0,0,1}\mathbf{91.1}$ | $\color[rgb]{0,0,1}\mathbf{88.4}$ | $84.0$ | $\color[rgb]{0,0,1}\mathbf{91.6}$

Densely Connected Structure. In fact, the densely connected structure also
plays an important role in our network. Its layer-wise aggregation scheme
enables the network to exploit multi-scale information from layers with
different receptive-field sizes. In this section, we experimentally prove that
densely connected structure is more suitable for HPE than commonly used ResNet
[13]. A classical single pyramid network without any extra technical skills is
selected as our baseline. We replace the ResNet [13] backbone with DenseNet
[6]. As shown in Table IX, DenseNet can always achieve better results than
ResNet, which indicates that the multi-scale information from layers with
different receptive-field sizes may help improve the performance of a single
pyramid network.

TABLE IX: The results of single pyramid framework with different backbones on
COCO2017 val dataset.

Backbone | GFLOPs | Params (M) | AP
---|---|---|---
Res18 | $2.5$ | $47.0$ | $70.8$
Dense98 | $2.4$ | $26.3$ | $\mathbf{72.7}$
Res34 | $4.2$ | $85.5$ | $73.1$
Dense121 | $3.0$ | $33.4$ | $\mathbf{73.3}$
Res50 | $4.7$ | $95.4$ | $73.3$
Dense169 | $3.6$ | $57.4$ | $\mathbf{73.6}$

Diverse Representations. We analyze the diversities and redundancies by
visualizing convolutional weights that connect different layers in Figure 6.
The idea is intuitive. Less diverse features tend to get smaller convolutional
weights, i.e. more redundant connections because connecting such features take
no information gain. As shown in Figure 6, most connections in the original
dense block have only negligible weights, while more connections in OAB have
weights of significant values. It indicates that OAB may produce more diverse
features.

Figure 6: Illustration of weight distribution among different layers. The
color of pixel ($s$, $l$) encodes the average L1-norm of weights connecting
layer $s$ to $l$. (a) All layers in Dense block distribute most weights to
only latest layers and result in redundant connections. (b) With mask and
channel attention units, the redundant connections are largely reduced.

###  IV-B Inference Speed

To directly illustrate the efficiency of DANet, the inference speed of
different models are evaluated on the same CPU platform with $8$ CPUs
(Intel(R) Xeon(R) Gold6013@2.1GHZ). The inference speed is indicated by the
number of persons that the model can process in one second, i.e. Persons-Per-
Second (PPS). The input sizes are $256\times 192$. For each model, we report
the average speed of processing $1\times 10^{3}$ persons. We systematically
examined nowadays most popular HPE networks, including the family of HRNet
[16] and SimpleBaseline [19], and 8-stage Hourglass [1]. The inference speed
with respect to the accuracy on COCO validation set is shown in Figure 1 (b).
Our DANet-102 achieves 73.4 AP with the speed being about $32$ PPS. DANet-72
has a fast speed being about $58$ PPS, and its AP score is $71.0$. Compared
with other models, our DANets make a better trade-off between accuracy and
inference speed.

###  IV-C Experiments on MPII

We also train and evaluate our models on MPII [32]. It contains 25K scene
images and 40K annotated persons. We use the standard train/val/test data
split (26K for training, 3K for validation, and 11K for testing) [33]. The
results are evaluated by Percentage of Corrected Keypoints (PCK), which
indicates the fraction of correct predictions within an error threshold $\tau$
($\tau=0.5$ for MPII, i.e. PCKh@0.5). The results are shown in Table VIII. The
computation cost of DANets is far smaller than that of other models, but the
performances of DANets are comparable with models over 40G FLOPs.

###  IV-D Experiments on ImageNet

To further prove the effectiveness of orthogonal attention block (OAB), we
also compare it with other densely connected architectures on ILSVRC 2012
classification dataset [34]. We replace the dense block in DenseNet169 by OAB,
denoted as OAB-DenseNet169, and compare it with DenseNet201 and DxNet169 [23].
Their results on test dataset are shown in Table X. The accuracy of OAB-
DenseNet169 is 1.3 higher than that of DenseNet169 and even slightly higher
than DenseNet201. These results suggest that OAB can largely boost the
performance of densely connected architecture, not only for human pose
estimation but also potentially for other tasks.

TABLE X: Results of on ImageNet2012 test dataset. Top-1 and top-5 accuracies
with single-crop testing are reported.

Model | GFLOPs | Params (M) | Top-1(%) | Top-5(%)
---|---|---|---|---
DenseNet169 [6] | $3.1$ | $13.5$ | $23.80$ | $6.85$
DenseNet201 [6] | $4.0$ | $19.1$ | $22.58$ | $6.34$
DxNet169 [23] | $3.3$ | $13.8$ | $23.90$ | $7.01$
OAB-DenseNet169 | $3.6$ | $18.0$ | $\mathbf{22.50}$ | $\mathbf{6.32}$

##  V Conclusion

Most existing HPE models use a cascaded framework to extract rich multi-scale
information. This framework makes these networks very deep and complex. In
this paper, we focus on exploiting multi-scale information from layers with
different receptive-field sizes and then making full use of this information
by improving the fusion method. The proposed OAB learns multi-scale
information from different layers and enhances them by encouraging them to be
diverse. The SFU adaptively selects and fuses diverse multi-scale features to
maximize the effective information in final fused representations. With the
help of OAB and SFU, our single pyramid network can generate deeply aggregated
representations that contain even richer multi-scale information and have a
larger representing capacity than that of cascaded networks. Specifically,
DANet-72 achieves $70.5$ AP on COCO test-dev set, with $1.0G$ FLOPs. The
inference speed on a CPU platform can reach about $58$ PPS.

## References

  * [1] A. Newell, K. Yang, and J. Deng, “Stacked hourglass networks for human pose estimation,” in _European conference on computer vision_. Springer, 2016, pp. 483–499.
  * [2] W. Li, Z. Wang, B. Yin, Q. Peng, Y. Du, T. Xiao, G. Yu, H. Lu, Y. Wei, and J. Sun, “Rethinking on multi-stage networks for human pose estimation,” _arXiv preprint arXiv:1901.00148_ , 2019.
  * [3] W. Yang, S. Li, W. Ouyang, H. Li, and X. Wang, “Learning feature pyramids for human pose estimation,” in _proceedings of the IEEE international conference on computer vision_ , 2017, pp. 1281–1290.
  * [4] F. Zhang, X. Zhu, and M. Ye, “Fast human pose estimation,” in _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , 2019, pp. 3517–3526.
  * [5] Y. Cai, Z. Wang, Z. Luo, B. Yin, A. Du, H. Wang, X. Zhang, X. Zhou, E. Zhou, and J. Sun, “Learning delicate local representations for multi-person pose estimation,” in _European Conference on Computer Vision_. Springer, 2020, pp. 455–472.
  * [6] G. Huang, Z. Liu, L. Van Der Maaten, and K. Q. Weinberger, “Densely connected convolutional networks,” in _Proceedings of the IEEE conference on computer vision and pattern recognitionz_ , 2017, pp. 4700–4708.
  * [7] S. Liu, L. Qi, H. Qin, J. Shi, and J. Jia, “Path aggregation network for instance segmentation,” in _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , 2018, pp. 8759–8768.
  * [8] T.-Y. Lin, P. Dollár, R. Girshick, K. He, B. Hariharan, and S. Belongie, “Feature pyramid networks for object detection,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2017, pp. 2117–2125.
  * [9] T. Kong, F. Sun, C. Tan, H. Liu, and W. Huang, “Deep feature pyramid reconfiguration for object detection,” in _Proceedings of the European Conference on Computer Vision (ECCV)_ , 2018, pp. 169–185.
  * [10] W. Liu, D. Anguelov, D. Erhan, C. Szegedy, S. Reed, C.-Y. Fu, and A. C. Berg, “Ssd: Single shot multibox detector,” in _European conference on computer vision_. Springer, 2016, pp. 21–37.
  * [11] G. Ghiasi, T.-Y. Lin, and Q. V. Le, “Nas-fpn: Learning scalable feature pyramid architecture for object detection,” in _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , 2019, pp. 7036–7045.
  * [12] T. Kong, F. Sun, C. Tan, H. Liu, and W. Huang, “Deep feature pyramid reconfiguration for object detection,” in _Proceedings of the European Conference on Computer Vision (ECCV)_ , 2018, pp. 169–185.
  * [13] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2016, pp. 770–778.
  * [14] K. Su, D. Yu, Z. Xu, X. Geng, and C. Wang, “Multi-person pose estimation with enhanced channel-wise and spatial information,” in _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , 2019, pp. 5674–5682.
  * [15] L. Ke, M.-C. Chang, H. Qi, and S. Lyu, “Multi-scale structure-aware network for human pose estimation,” in _Proceedings of the European Conference on Computer Vision (ECCV)_ , 2018, pp. 713–728.
  * [16] K. Sun, B. Xiao, D. Liu, and J. Wang, “Deep high-resolution representation learning for human pose estimation,” in _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , 2019, pp. 5693–5703.
  * [17] S.-E. Wei, V. Ramakrishna, T. Kanade, and Y. Sheikh, “Convolutional pose machines,” in _Proceedings of the IEEE conference on Computer Vision and Pattern Recognition_ , 2016, pp. 4724–4732.
  * [18] Y. Chen, Z. Wang, Y. Peng, Z. Zhang, G. Yu, and J. Sun, “Cascaded pyramid network for multi-person pose estimation,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2018, pp. 7103–7112.
  * [19] B. Xiao, H. Wu, and Y. Wei, “Simple baselines for human pose estimation and tracking,” in _Proceedings of the European conference on computer vision (ECCV)_ , 2018, pp. 466–481.
  * [20] G. Huang, D. Chen, T. Li, F. Wu, L. Van Der Maaten, and K. Q. Weinberger, “Multi-scale dense convolutional networks for efficient prediction,” _arXiv preprint arXiv:1703.09844_ , vol. 2, 2017.
  * [21] N. Ma, X. Zhang, H.-T. Zheng, and J. Sun, “Shufflenet v2: Practical guidelines for efficient cnn architecture design,” in _Proceedings of the European Conference on Computer Vision (ECCV)_ , 2018, pp. 116–131.
  * [22] G. Huang, S. Liu, L. Van der Maaten, and K. Q. Weinberger, “Condensenet: An efficient densenet using learned group convolutions,” in _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , 2018, pp. 2752–2761.
  * [23] I. Kligvasser and T. Michaeli, “Dense xunit networks,” _arXiv preprint arXiv:1811.11051_ , 2018.
  * [24] I. Kligvasser, T. Rott Shaham, and T. Michaeli, “xunit: Learning a spatial activation function for efficient image restoration,” in _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , 2018, pp. 2433–2442.
  * [25] J. Hu, L. Shen, and G. Sun, “Squeeze-and-excitation networks,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2018, pp. 7132–7141.
  * [26] T.-Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollár, and C. L. Zitnick, “Microsoft coco: Common objects in context,” in _European conference on computer vision_. Springer, 2014, pp. 740–755.
  * [27] Z. Wang, W. Li, B. Yin, Q. Peng, T. Xiao, Y. Du, Z. Li, X. Zhang, G. Yu, and J. Sun, “Mscoco keypoints challenge 2018,” in _Joint Recognition Challenge Workshop at ECCV 2018_ , vol. 5, 2018.
  * [28] D. P. Kingma and J. Ba, “Adam: a method for stochastic optimization. corr abs/1412.6980 (2014),” 2014.
  * [29] X. Peng, Z. Tang, F. Yang, R. S. Feris, and D. Metaxas, “Jointly optimize data augmentation and network training: Adversarial data augmentation in human pose estimation,” in _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , 2018, pp. 2226–2234.
  * [30] X. Chu, W. Yang, W. Ouyang, C. Ma, A. L. Yuille, and X. Wang, “Multi-context attention for human pose estimation,” in _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , 2017, pp. 1831–1840.
  * [31] T. Sekii, “Pose proposal networks,” in _Proceedings of the European Conference on Computer Vision (ECCV)_ , 2018, pp. 342–357.
  * [32] M. Andriluka, L. Pishchulin, P. Gehler, and B. Schiele, “2d human pose estimation: New benchmark and state of the art analysis,” in _Proceedings of the IEEE Conference on computer Vision and Pattern Recognition_ , 2014, pp. 3686–3693.
  * [33] J. J. Tompson, A. Jain, Y. LeCun, and C. Bregler, “Joint training of a convolutional network and a graphical model for human pose estimation,” in _Advances in neural information processing systems_ , 2014, pp. 1799–1807.
  * [34] J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, “Imagenet: A large-scale hierarchical image database,” in _2009 IEEE conference on computer vision and pattern recognition_. Ieee, 2009, pp. 248–255.