# Adaptive Poincaré Point to Set Distance for Few-Shot Classification

Rongkai Ma1, Pengfei Fang2,3, Tom Drummond4, Mehrtash Harandi1,3 Corresponding
author.

###### Abstract

Learning and generalizing from limited examples, _i.e_., few-shot learning, is
of core importance to many real-world vision applications. A principal way of
achieving few-shot learning is to realize an embedding where samples from
different classes are distinctive. Recent studies suggest that embedding via
hyperbolic geometry enjoys low distortion for hierarchical and structured
data, making it suitable for few-shot learning. In this paper, we propose to
learn a context-aware hyperbolic metric to characterize the distance between a
point and a set associated with a learned set to set distance. To this end, we
formulate the metric as a weighted sum on the tangent bundle of the hyperbolic
space and develop a mechanism to obtain the weights adaptively and based on
the constellation of the points. This not only makes the metric local but also
dependent on the task in hand, meaning that the metric will adapt depending on
the samples that it compares. We empirically show that such metric yields
robustness in the presence of outliers and achieves a tangible improvement
over baseline models. This includes the state-of-the-art results on five
popular few-shot classification benchmarks, namely _mini_ -ImageNet, _tiered_
-ImageNet, Caltech-UCSD Birds-200-2011 (CUB), CIFAR-FS, and FC100.

##  1 Introduction

In the modern context of machine learning, deep neural networks (DNNs) have
enjoyed enormous success by leveraging the rich availability of labeled data
for supervised training. Despite this, deep supervised learning is primarily
limited in terms of scaling towards unseen samples due to the high cost of
acquiring large amounts of labeled data. This is in clear contrast to how
humans learn, where in many cases, only a handful of training examples are
sufficient for generalizing towards unseen samples. Few-Shot Learning (FSL)
addresses this critical problem through the development of algorithms that can
learn using limited data (Finn, Abbeel, and Levine 2017; Nichol, Achiam, and
Schulman 2018; Snell, Swersky, and Zemel 2017; Sung et al. 2018; Vinyals et
al. 2016; Ye et al. 2020; Hong et al. 2021; Wang et al. 2020).

Figure 1: (a): For a query sample in “red” class, outliers (_i.e_., yellow and
white circles) drag the prototype (_i.e_., the black circle) far away from the
real cluster center in the embedding space such that the nearest neighbor
classifier mis-classifies the query point into “green” class. (b): Our method
computes an adaptive point to set distance on the manifold, which is more
robust to outliers than prototypes. Best viewed in color.

Performing FSL well is essential towards creating robust frameworks that can
learn with the efficiency of humans. In many cases, FSL methods deem to learn
an embedding space to distinguish samples from different classes. Therein, the
embedding space is a multidimensional Euclidean space and is realized via a
deep neural network.

Employing hyperbolic geometry to encode data has been shown rewarding, as the
volume of space expands exponentially (Ganea, Bécigneul, and Hofmann 2018;
Khrulkov et al. 2020). Recent works have shown that a hierarchical structure
exists within visual datasets and that the use of hyperbolic embeddings can
yield significant improvements over Euclidean embeddings (Khrulkov et al.
2020; Fang, Harandi, and Petersson 2021).

Most existing FSL solutions learn a metric through comparing the distance
between a query sample and the class prototypes, often modeled as the mean
embeddings of each class. However, this does not take the adverse effects of
outliers and noises into consideration (Sun et al. 2019). This severely limits
the representation power of embedding-based methods since the outliers may
drag the prototype away from the true center of the cluster (see Fig.1). For a
more robust approach, we require an adaptive metric, which can faithfully
capture the distribution per class, while being robust to outliers and other
nuances in data (Fig. 1).

With this in mind, we propose learning a context-aware hyperbolic metric that
characterizes the point to set (dis)similarities. This is achieved through
employing a Poincaré ball to model hyperbolic spaces and casting the
(dis)similarity as a weighted-sum between a query and a class that is learned
adaptively. In doing so, each sample (from the support and query sets) is
modeled by a set itself (_i.e_., a feature map). Therefore, we propose to make
use of pairwise distances between elements of two sets, along with a
refinement mechanism to disregard uninformative parts of the feature maps.
This leads to a flexible and robust framework for the FSL tasks. We summarize
our contributions as follows:

  * •

We propose a novel adaptive Poincaré point to set (APP2S) distance metric for
the FSL task.

  * •

We further design a mechanism to produce a weight, dependent on the
constellation of the point, for our APP2S metric.

  * •

We conduct extensive experiments across five FSL benchmarks to evaluate the
effectiveness of the proposed method.

  * •

We further study the robustness of our method, which shows our method is
robust against the outliers compared to competing baselines.

##  2 Preliminaries

In what follows, we use $\mathbb{R}^{n}$ and $\mathbb{R}^{m\times n}$ to
denote the $n$-dimensional Euclidean space and space of $m\times n$ real
matrices, respectively. The $n$-dimensional hyperbolic space is denoted by
$\mathbb{H}_{c}^{n}$. The
$\mathrm{arctanh}:(-1,1)\to\mathbb{R},\mathrm{arctanh}(x)=\frac{1}{2}\ln(\frac{1+x}{1-x}),|x|<1$
refers to the inverse hyperbolic tangent function. The vectors and matrices
(or 3-D tensors) are denoted by bold lower-case letters and bold upper-case
letters throughout the paper.

###  2.1 Riemannian Geometry

In this section, we will give a brief recap of Riemannian geometry. A
manifold, denoted by $\mathcal{M}$, is a curved surface, which locally
resembles the Euclidean space. The tangent space at ${\bm{x}}\in\mathcal{M}$
is denoted by $T_{{\bm{x}}}\mathcal{M}$. It contains all possible vectors
passing through point ${\bm{x}}$ tangentially. On the manifold, the shortest
path connecting two points is a geodesic, and its length is used to measure
the distances on the manifold.

###  2.2 Hyperbolic Space

Hyperbolic spaces are Riemannian manifolds with constant negative curvature
and can be studied using the Poincaré ball model (Ganea, Bécigneul, and
Hofmann 2018; Khrulkov et al. 2020). The Poincaré ball
($\mathbb{D}_{c}^{n},g^{c}$) is a smooth $n$-dimensional manifold identified
by satisfying
$\mathbb{D}^{n}_{c}=\\{{\bm{x}}\in\mathbb{R}^{n}:c\lVert{\bm{x}}\rVert<1,c\geqslant
0\\}$111In the supplementary material, we provide further details regarding
the Poincaré ball model and its properties., where $c$ is the absolute value
of the curvature for a Poincaré ball, while the real curvature value is $-c$.
The Riemannian metric $g^{c}$ at ${\bm{x}}$ is defined as
$g^{c}=\lambda{{}_{{\bm{x}}}^{c}}^{2}g^{E}$, where $g^{E}$ is the Euclidean
metric tensor and $\lambda_{{\bm{x}}}^{c}$ is the conformal factor, defined
as:

$\lambda_{{\bm{x}}}^{c}\coloneqq\frac{2}{1-c\lVert{\bm{x}}\rVert^{2}}.$ (1)

Since the hyperbolic space is a non-Euclidean space, the rudimentary
operations, such as vector addition, cannot be applied (as they are not
faithful to the geometry). The Möbius gyrovector space provides many standard
operations for hyperbolic spaces. Essential to our developments in this work
is the Möbius addition of two points ${\bm{x}},{\bm{y}}\in\mathbb{D}^{n}_{c}$,
which is calculated as:

${\bm{x}}\oplus_{c}{\bm{y}}=\frac{(1+2c\langle{\bm{x}},{\bm{y}}\rangle+c\|{\bm{y}}\|^{2}){\bm{x}}+(1-c\|{\bm{x}}\|^{2}){\bm{y}}}{1+2c\langle{\bm{x}},{\bm{y}}\rangle+c^{2}\|{\bm{x}}\|^{2}\|{\bm{y}}\|^{2}}.$
(2)

The geodesic distance between two points
${\bm{x}},{\bm{y}}\in\mathbb{D}^{n}_{c}$ can be obtained as:

$d_{c}({\bm{x}},{\bm{y}})=\frac{2}{\sqrt{c}}\text{arctanh}(\sqrt{c}\|-{\bm{x}}\oplus_{c}{\bm{y}}\|).$
(3)

Another essential operation used in our model is the hyperbolic averaging. The
counterpart of Euclidean averaging in hyperbolic space is the $E\emph{instein
mid-point}$ which has the most simple form in $K\emph{lein}$ coordinates
(another model of the hyperbolic space which is isometric to the Poincaré
ball). Thus, we transform the points from Poincaré (_i.e_.,
${\bm{x}}_{\mathbb{D}}$) ball model to Klein model (_i.e_.,
${\bm{x}}_{\mathbb{K}}$) using the transformation:

${\bm{x}}_{\mathbb{K}}=\frac{2{\bm{x}}_{\mathbb{D}}}{1+c\|{\bm{x}}_{\mathbb{D}}\|^{2}}.$
(4)

Then, the hyperbolic averaging in Klein model is obtained as:

$\text{HypAve}({\bm{x}}_{1},\ldots,{\bm{x}}_{N})=\sum_{i=1}^{N}\gamma_{i}{\bm{x}}_{i}/\sum_{i=1}^{N}\gamma_{i},$
(5)

where $\gamma_{i}=\frac{1}{\sqrt{1-c\|{\bm{x}}_{i}\|^{2}}}$ are the Lorentz
factors. Finally, we transform the coordinates back to Poincaré model using:

${\bm{x}}_{\mathbb{D}}=\frac{{\bm{x}}_{\mathbb{K}}}{1+\sqrt{1-c\|{\bm{x}}_{\mathbb{K}}\|^{2}}}.$
(6)

In our work, we make use of the tangent bundle of the $\mathbb{D}^{n}_{c}$.
The logarithm map defines a function from $\mathbb{D}^{n}_{c}\to
T_{{\bm{x}}}\mathbb{D}_{c}^{n}$, which projects a point in the Poincaré ball
onto the tangent space at ${\bm{x}}$, as:

${\bm{\pi}}^{c}_{\bm{x}}({\bm{y}})=\frac{2}{\sqrt{c}\lambda^{c}_{\bm{x}}}\text{arctanh}(\sqrt{c}\|-{\bm{x}}\oplus_{c}{\bm{y}}\|)\frac{-{\bm{x}}\oplus_{c}{\bm{y}}}{\|-{\bm{x}}\oplus_{c}{\bm{y}}\|}.$
(7)

###  2.3 Point to Set Distance

Let $\mathcal{S}=\\{{\bm{s}}_{1},\ldots,{\bm{s}}_{k}\\}$ be a set. The
distance from a point ${\bm{p}}$ to the set $\mathcal{S}$ can be defined in
various forms. The min and max distance from a point ${\bm{p}}$ to the set
$\mathcal{S}$ are two simple metrics, which can be defined as:

$d_{\mathrm{p2s}}^{\mathrm{l}}({\bm{p}};\mathcal{S})=\inf\\{d({\bm{p}},{\bm{s}}_{i})|{\bm{s}}_{i}\in\mathcal{S}\\},$
(8)
$d_{\mathrm{p2s}}^{\mathrm{h}}({\bm{p}};\mathcal{S})=\sup\\{d({\bm{p}},{\bm{s}}_{i})|{\bm{s}}_{i}\in\mathcal{S}\\},$
(9)

where $\inf$ and $\sup$ are the infimum and supremum functions, respectively.
Given their geometrical interpretation, $d_{\mathrm{p2s}}^{\mathrm{l}}$ and
$d_{\mathrm{p2s}}^{\mathrm{h}}$ define the lower and upper pairwise bounds,
and fail to encode structured information about the set. Therefore, we opt for
a weighted-sum formalism to measure the distance between a point and a set in
§ 3.3.

##  3 Method

This section will give an overview of the proposed method, followed by a
detailed description of each component in our model.

Figure 2: (a): The overall pipeline of our method. Given an episode, we use a
backbone network to extract and map the inputs to a hyperbolic space. We then
project the support samples onto the tangent plane of the query point and
employ a refinement function $f_{\omega}$ to obtain the class and episode-
aware signature of every class in the support set. This is followed by a
mapping $f_{\phi}$ that weighs the importance of each sample of the support
set w.r.t. their corresponding class. This enables us to calculate an adaptive
point to set distance towards inference. (b): The S2S distance module. We
calculate a pair-wise distance between each feature vector of the two feature
maps as the input of the distance network $f_{\zeta}$. Then the $f_{\zeta}$
outputs a S2S distance, which is further used to compute the adaptive P2S.

###  3.1 Problem Formulation

We follow the standard protocol to formulate few-shot learning (FSL) with
episodic training. An episode represents an $N$-way $K$-shot classification
problem (_i.e_., the training set, named support set, includes $N$ classes
where each class has $K$ examples). As the name implies, $K$ (_i.e_., the
number of examples per class) is small (_e.g_., $K=1$ or $5$). The goal of
learning is to realize a function $\mathcal{F}:\mathcal{X}\to\mathbb{R}^{n}$
to embed the support set to a latent and possibly lower-dimensional space,
such that query samples can be recognized easily using a nearest neighbour
classifier. To be specific, an episode or task $\mathcal{E}_{i}$ consists of a
query set $\mathcal{X}^{q}=\\{({\bm{X}}^{q}_{i},y^{q}_{i})|i=1,\ldots,N\\}$,
where ${\bm{X}}^{q}_{i}$ denotes a query example222Without losing generality,
we use one sample per class as a query for presenting our method. In practice,
each episode contains multiple samples for the query image per class. sampled
from class $y^{q}_{i}$, and a support set
$\mathcal{X}^{s}=\\{({\bm{X}}^{s}_{ij},y^{s}_{i})|i=1,\ldots,N,j=1,\ldots,K\\}$,
where ${\bm{X}}^{s}_{ij}$ denotes the $j$-th sample in the class $y^{s}_{i}$.
The embedding methods for FSL, our solution being one, often formulate
training as:

$\mathcal{F}^{*}:=\operatorname*{arg\,min}_{\mathcal{F}}\sum_{{\bm{X}}^{q}_{u}\in\mathcal{X}^{q}}\delta\big{(}\mathcal{F}({\bm{X}}^{q}_{u}),\mathcal{F}({\mathcal{X}}^{s}_{v})\big{)}~{}~{}\text{s.t.}~{}y^{q}_{u}=y^{s}_{v},$
(10)

where $\delta$ measures a form of distance between the query and the support
samples.

###  3.2 Model Overview

We begin by providing a sketch of our method (see the conceptual diagram in
Fig. 2 and Fig. 2). The feature extractor network, denoted by $\mathcal{F}$,
maps the input to a hyperbolic space in our work. We model every class in the
support set by its signature. The signature is both class and episodic-aware,
meaning that the signature will vary if the samples of a class or samples in
the episode vary. This will enable us to calculate an adaptive distance from
the query point to every support-class while being vigilant to the arrangement
and constellation of the support samples. We stress that our design is
different from many prior works where class-specific prototypes are learned
for FSL. For example, in (Khrulkov et al. 2020; Snell, Swersky, and Zemel
2017; Sung et al. 2018), the prototypes are class-specific but not necessarily
episodic-aware.

To obtain the signatures for each class in the support set, we project the
support samples onto the tangent space of the query point and feed the
resulting vectors to a signature generator $f_{\omega}$. The signature
generator realizes a permutation-invariant function and refines and summarizes
its inputs to one signature per class. We then leverage a relational network
$f_{\phi}$ to contrast samples of a class against their associated signature
and produce a relational score. To obtain the adaptive P2S distance, we first
compute a set to set (S2S) distance between the query feature map and each
support feature map using the distance module $f_{\zeta}$. Moreover, a
weighted-sum is calculated using the relational score acting as the weight on
the corresponding S2S distance, which serves as the P2S distance.

Given P2S distances, our network is optimized by minimizing the adaptive P2S
distance between the query and its corresponding set while ensuring that the
P2S distance to other classes (_i.e_., wrong classes) is maximized.

###  3.3 Adaptive Poincaré Point to Set Distance

In FSL, we are given a small support set of $K$ images,
$\mathcal{X}^{s}_{i}=\\{{\bm{X}}^{s}_{i1},\ldots,{\bm{X}}^{s}_{iK}\\}$ per
class $y^{s}_{i}$ to learn a classification model. We use a deep neural
network to first encode the input to a multi-channel feature map, as
$\mathcal{S}_{i}=\mathcal{F}(\mathcal{X}_{i}^{s})$, with
$\mathcal{S}_{i}=\\{{\bm{S}}_{i1},\ldots,{\bm{S}}_{iK}|{\bm{S}}_{ij}\in\mathbb{R}^{H\times
W\times C}\\}$, where $H$, $W$, and $C$ indicate the height, width, and
channel size of the instance feature map. Each feature map consists of a set
of patch descriptors (local features), which can be further represented as
${\bm{S}}_{ij}=\\{{\bm{s}}_{ij}^{1},\ldots,{\bm{s}}_{ij}^{HW}|{\bm{s}}_{ij}^{r}\in\mathbb{R}^{C}\\}$.

In our work, we train the network to embed the representation in the Poincaré
ball; thus, we need to impose a constraint on patch descriptors at each
spatial location ${\bm{s}}_{ij}^{r}$ as follows:

${\bm{s}}_{ij}^{r}=\left\\{\begin{matrix}{\bm{s}}_{ij}^{r}&~{}\text{if}~{}\|{\bm{s}}_{ij}^{r}\|\leq\mu\\\
\mu{\bm{s}}_{ij}^{r}/\|{\bm{s}}_{ij}^{r}\|&~{}~{}\text{if}~{}\|{\bm{s}}_{ij}^{r}\|>\mu,\end{matrix}\right.$
(11)

where $\mu$ is the norm upper bound of the vectors in the Poincaré ball. In
our model, we choose $\mu=(1-\epsilon)/c$, where $c$ is the curvature of the
Poincaré ball and $\epsilon$ is a small value that makes the system
numerically stable. The same operation applies to the query sample, thereby
obtaining an instance feature map for the query sample
${\bm{Q}}=\\{{\bm{q}}^{1},\ldots,{\bm{q}}^{HW}\\},{\bm{q}}^{r}\in\mathbb{D}^{C}_{c}$.
Then the P2S distance between the query sample ${\bm{Q}}$ and the support set
per class $\mathcal{S}_{i}$ can be calculated using Eq. (8) or Eq. (9).
However, those two metrics only determine the lower or upper bound of P2S
distance, thereby ignoring the structure and distribution of the set to a
great degree. To make better use of the distribution of samples in a set, we
propose the adaptive P2S distance metric as:

$d_{\mathrm{p2s}}^{\mathrm{adp}}({\bm{Q}};{\mathcal{S}}_{i})\coloneqq\frac{\sum\limits_{j=1}^{K}w_{ij}d({\bm{Q}},{{\bm{S}}}_{ij})}{\sum\limits_{j=1}^{K}w_{ij}},$
(12)

where $w_{ij}$ is the adaptive factor for $d({\bm{Q}},{{\bm{S}}}_{ij})$. We
refer to the distance in Eq. (12) as Adaptive Poincaré Point to Set (APP2S)
distance, hereafter.

In Eq. (12), we need to calculate the distance between two feature maps
(_i.e_., $d({\bm{Q}},{\bm{S}}_{ij})$). In doing so, we formulate a feature map
as a set (_i.e_., $\\{{\bm{q}}^{1},\ldots,{\bm{q}}^{HW}\\}$ and
$\\{{\bm{s}}_{ij}^{1},\ldots,{\bm{s}}_{ij}^{HW}\\}$), such that a set to set
(S2S) distance can be obtained. One-sided Hausdorff and two-sided Hausdorff
distances (Huttenlocher, Klanderman, and Rucklidge 1993) are two widely used
metrics to measure the distance between sets. However, these two metrics are
sensitive to outliers (Huttenlocher, Klanderman, and Rucklidge 1993; Ibanez et
al. 2008). To alleviate this issue, we propose to learn the S2S distance by a
network $f_{\zeta}$. We first calculate the pair-wise distance between two
sets as ${\bm{D}}({\bm{Q}},{\bm{S}}_{ij})\in\mathbb{R}^{HW\times HW}$, where
each element in ${\bm{D}}$ is obtained by
$d_{h,w}=d_{c}({\bm{q}}^{h},{\bm{s}}_{ij}^{w})$ using Eq. (3), where
$h=1,\ldots,HW$ and $w=1,\ldots,HW$. Then we use a neural network to further
learn the distance between two feature maps (see Fig. 2), which is given by:

$d_{\mathrm{s2s}}({\bm{Q}},{\bm{S}}_{ij}):=f_{\zeta}({\bm{D}}).$ (13)

Comparing to the Hausdorff distance (Conci and Kubrusly 2018) (see
supplementary material), our set to set distance is more flexible and learned
through the optimization process.

To further obtain the weights of APP2S (_i.e_., $w_{ij}$), we make use of the
tangent space of the query sample. We first compute a mean query vector
${\bm{\bar{q}}}$ over the spatial dimensions of the feature map ${\bm{Q}}$
using Eq. (4)- Eq. (6). Then, we project the samples in the support set to the
tangent space of the mean query vector (see Fig. 2), using the logarithm map
as:

$\tilde{\mathcal{S}}={\bm{\pi}}^{c}_{\bar{{\bm{q}}}}({\mathcal{S}}),$ (14)

where $\tilde{\mathcal{S}}$ indicates the projected support set on the tangent
space at $\bar{{\bm{q}}}$. For the $i$-th class, we can obtain a set of
feature maps:
$\tilde{\mathcal{S}}_{i}=\\{\tilde{{\bm{S}}}_{i1},\ldots,\tilde{{\bm{S}}}_{iK}\\}$333The
projected feature map is also composed by the vectors at each spatial location
$\tilde{{\bm{S}}}_{ij}=\\{\tilde{{\bm{s}}}_{ij}^{{1}},\ldots,\tilde{{\bm{s}}}_{ij}^{{HW}}\\}$.
To obtain a meaningful weight $w_{ij}$, we first propose a signature
generator, which jointly refines sample representations in the support set and
summarizes the set representation per class as the class signature. As shown
in Fig. 2, the signature generator receives the projected support set
$\tilde{\mathcal{S}}=\\{\tilde{\mathcal{S}}_{1},\ldots,\tilde{\mathcal{S}}_{N}\\}$
as input and refines them for the follow-up task (_i.e_., obtaining the
weights for the APP2S). We denote the output of the refinement module by
$\hat{\mathcal{S}}=f_{\omega}(\tilde{\mathcal{S}})$
($\hat{\mathcal{S}}=\\{\hat{\mathcal{S}}_{1},\ldots,\hat{\mathcal{S}}_{N}\\}$,
$\hat{\mathcal{S}}_{i}=\\{\hat{{\bm{S}}}_{i1},\ldots,\hat{{\bm{S}}}_{iK}\\}$).
One can understand the refinement function as learning the context of the
support set by seeing all the samples, thereby highlighting the discriminative
samples and restraining the non-informative samples such as outliers for all
the samples. Then the signature for each class is obtained by summarizing as:
$\bar{{\bm{S}}}_{i}={\sum_{j=1}^{K}\hat{{\bm{S}}}_{ij}}/{K}$.

###### Remark 1

Our proposed set-signature generator $f_{\omega}$ is similar to the set-to-set
function in FEAT (Ye et al. 2020), in the sense that both functions perform
self-attention over the input features. However, the fundamental difference is
that our module exploits the relation between the spatial feature descriptors
of all samples in a support set (_e.g_., $\tilde{{\bm{s}}}^{r}_{ij}$), instead
of prototypes as proposed in FEAT (Ye et al. 2020), which possibly gives the
model more flexibility to encode meaningful features.

Given sample features in a class
$\tilde{\mathcal{S}}_{i}=\\{\tilde{{\bm{S}}}_{i1},\ldots,\tilde{{\bm{S}}}_{iK}\\}$
and the corresponding class signature $\bar{{\bm{S}}}_{i}$, we use a relation
generator (_i.e_., $f_{\phi}$ in Fig. 2) to compare the relationship between
an individual feature map and the class signature. In doing so, we first
concatenate the individual feature maps and their class signature along the
channel dimension to obtain a hybrid representation, as:

${\bm{G}}_{ij}=\mathrm{CONCAT}(\tilde{{\bm{S}}}_{ij},\bar{{\bm{S}}}_{i}).$
(15)

Given the hybrid representation ${\bm{G}}_{ij}$, the relation generator
produces a relation score as: ${w}_{ij}=f_{\phi}({\bm{G}}_{ij})$. This score
will serve as the adaptive factor for the APP2S distance metric in Eq. (12).
Note that the hybrid representation for the whole support and a support-class
set are denoted by $\mathcal{G}=\\{\mathcal{G}_{1},\ldots,\mathcal{G}_{N}\\}$
and $\mathcal{G}_{i}=\\{{\bm{G}}_{i1},\ldots,{\bm{G}}_{iK}\\}$, respectively.
Algorithm 1 summarizes the process of training our APP2S metric for FSL.

###### Remark 2

The point to set distance defined by Eq. (12) is different from that in
MatchingNet (Vinyals et al. 2016). MatchingNet formulates all the samples in
the support set as a set. In contrast, we treat the samples in a class as the
set, which makes our adaptive point to set distance fully contextual aware of
the whole support set (by the set-signature) and encodes the distribution of
each class.

Figure 3: The information flow of the signature generator and the relation
generator. The signature generator $f_{\omega}$ receives the projected support
set $\\{\tilde{\mathcal{S}}_{1},\tilde{\mathcal{S}}_{2}\\}$ as a bag and
outputs the refined representation per sample such that each element (_i.e_.,
$\tilde{{\bm{S}}}_{ij}$) sees all the other elements of the support set. Then
the refined representations per class/set $\hat{\mathcal{S}}_{1}$ and
$\hat{\mathcal{S}}_{2}$ are averaged to obtain the signature per class/set
$\bar{{\bm{S}}}_{1}$ and $\bar{{\bm{S}}}_{2}$. Finally, we concatenate one
projected sample $\tilde{{\bm{S}}}_{1j}$ and $\tilde{{\bm{S}}}_{2j}$ with the
corresponding class signature $\bar{{\bm{S}}}_{1}$ and $\bar{{\bm{S}}}_{2}$,
repectively, and feed it into relation generator $f_{\phi}$ to produce the
adaptive factors $\omega_{1j}$ and $\omega_{2j}$. Algorithm 1 Train network
using adaptive Poincaré point to set distance

Input: An episodes $\mathcal{E}$, with their associated support set
$\mathcal{X}^{s}=\\{({\bm{X}}^{s}_{ij},y^{s}_{i})|i=1,\ldots,N,j=1,\ldots,K\\}$
and a query sample ${\bm{X}}^{q}$

Output: The optimal parameters for
$\mathcal{F}\text{,}~{}f_{\omega},f_{\zeta},~{}\text{and}~{}f_{\phi}$

1:Map $\mathcal{X}^{s}$ and ${\bm{X}}^{q}$ into Poincaré ball

2:Obtain the tangent support set $\tilde{\mathcal{S}}$ using Eq. (7)

3:$\hat{\mathcal{S}}=f_{\omega}(\tilde{\mathcal{S}})$ $\triangleright$ the
refined support set

4:for $i$ in $\\{1,...,N\\}$ do

5: $\bar{{\bm{S}}}_{i}={\sum_{j=1}^{K}\hat{{\bm{S}}}_{ij}}/{K}$
$\triangleright$ the set signature

6: ${\bm{G}}_{ij}=\mathrm{CONCAT}(\tilde{{\bm{S}}}_{ij},\bar{{\bm{S}}}_{i})$
$\triangleright$ the hybrid representation

7: $\omega_{ij}=f_{\phi}({\bm{G}}_{ij})$$\triangleright$ the weight

8: Compute point to set distance and set to set distance using Eq. (12) and
Eq. (13)

9:end for

10:Optimize the model using Eq. (10)

##  4 Related Work

In this section, we discuss the literature on few-shot learning and highlight
those that motivate this work. Generally, there are two main branches on the
few-shot learning literature, optimization-based and metric-based methods. The
optimization-based methods (Antoniou, Edwards, and Storkey 2019; Chen et al.
2019; Finn, Abbeel, and Levine 2017; Flennerhag et al. 2019; Franceschi et al.
2018; Nichol, Achiam, and Schulman 2018), such as MAML and Reptile (Finn,
Abbeel, and Levine 2017; Nichol, Achiam, and Schulman 2018), aim to learn a
set of initial model parameters that can adapt to new tasks quickly using
backpropagation in the episodic regime, without severe overfitting. However,
this group of methods usually adopt a bi-level optimization setting to
optimize the initial parameters, which is computationally expensive during
inference.

On the other hand, our proposed method is closer to metric-based methods
(Simon et al. 2020; Snell, Swersky, and Zemel 2017; Sung et al. 2018; Vinyals
et al. 2016; Ye et al. 2020; Zhang et al. 2020; Tang et al. 2020), which
target to realize an embedding: $\mathbb{R}^{M}\rightarrow\mathbb{R}^{D}$ to
represent images in semantic space equipped with an appropriate distance
metric such that different categories are distinctive. Matching Network
(Vinyals et al. 2016) determines the query labels by learning a sample-wise
distance along with a self-attention mechanism that produces a fully
contextualized embedding over samples. Prototypical Network (Snell, Swersky,
and Zemel 2017) takes a step further from a sample-wise to a class-wise
metric, where all the samples of a class are averaged into a prototype to
represent the class in the embedding space. Relation Network (Sung et al.
2018) and CTM (Li et al. 2019a) replace the hand-crafted metric with a network
to encode the non-linear relation between the class representations and the
query embedding. Ye _et al_. (Ye et al. 2020) propose adopting a transformer
to learn the task-specific features for few-shot learning. Zhang _et al_.
(Zhang et al. 2020) adopt the Earth Mover’s Distance as a metric to compute a
structural distance between representation to obtain the labels for the query
images. Simon _et al_. (Simon et al. 2020) propose to generate a dynamic
classifier via using subspace. Along this line of research, most of the
previous methods utilize the global feature vectors as representations.
However, several recent works have demonstrated that utilizing the local
feature maps can further boost performance. Therefore, we follow these works
(Doersch, Gupta, and Zisserman 2020; Zhang et al. 2020; Wertheimer, Tang, and
Hariharan 2021; Lifchitz et al. 2019; Li et al. 2019b) to develop our model.

However, the majority of the aforementioned metric-based works employ various
metrics within Euclidean space. Ganea _et al_. (Ganea, Bécigneul, and Hofmann
2018) have proved that embedding via hyperbolic geometry enjoys low distortion
for hierarchical and structured data (_e.g_., trees) and developed the
hyperbolic version of the feed-forward neural networks and recurrent neural
networks (RNN). Moreover, a recent work (Khrulkov et al. 2020) has shown that
the vision tasks can largely benefit from hyperbolic embeddings, which
inspires us to further develop algorithms with hyperbolic geometry.

##  5 Experiments

###  5.1 Datasets

In this section, we will empirically evaluate our approach across five
standard benchmarks, _i.e_., _mini_ -ImageNet (Ravi and Larochelle 2016),
_tiered_ -ImageNet (Ren et al. 2018), Caltech-UCSD Birds-200-2011 (CUB) (Wah
et al. 2011), CIFAR-FS (Bertinetto et al. 2018) and Fewshot-CIFAR100 (FC100)
(Oreshkin, López, and Lacoste 2018). Full details of the datasets and
implementation are described in the supplementary material. In the following,
we will briefly describe our results on each dataset.

###  5.2 Main Result

We evaluate our methods for 100 epochs, and in each epoch, we sample 100 tasks
(episodes) randomly from the test set, for both 5-way 1-shot and 5-way 5-shot
settings. Following the standard protocol (Simon et al. 2020), we report the
mean accuracy with 95$\%$ confidence interval.

Model | Backbone |  _mini_ -ImageNet |  _tiered_ -ImageNet
---|---|---|---
5-way 1-shot | 5-way 5-shot | 5-way 1-shot | 5-way 5-shot
ProtoNet (Snell, Swersky, and Zemel 2017) | ResNet-12 | $60.37\pm{0.83}$ | $78.02\pm{0.57}$ | $61.74\pm{0.77}$ | $80.00\pm{0.55}$
MatchingNet (Vinyals et al. 2016) | ResNet-12 | $63.08\pm{0.80}$ | $75.99\pm{0.60}$ | $68.50\pm{0.92}$ | $80.60\pm{0.71}$
MetaOptNet (Lee et al. 2019b) | ResNet-12 | $62.64\pm{0.61}$ | $78.63\pm{0.46}$ | $65.99\pm{0.72}$ | $81.56\pm{0.53}$
Ravichandran _et al_. (Ravichandran, Bhotika, and Soatto 2019) | ResNet-12 | $60.71$ | $77.64$ | $66.87$ | $82.64$
DeepEMD (Zhang et al. 2020) | ResNet-12 | $65.91\pm{0.82}$ | $82.41\pm{0.56}$ | $71.16\pm{0.87}$ | $86.03\pm{0.58}$
P-transfer (Shen et al. 2021) | ResNet-12 | $64.21\pm{0.77}$ | $80.38\pm{0.59}$ | - | -
GLoFA (Lu, Ye, and Zhan 2021) | ResNet-12 | $66.12\pm{0.42}$ | $81.37\pm{0.33}$ | $69.75\pm{0.33}$ | $83.58\pm{0.42}$
DMF (Xu et al. 2021) | ResNet-12 | $\bm{67.76\pm{0.46}}$ | $82.71\pm{0.31}$ | $71.89\pm{0.52}$ | $85.96\pm{0.35}$
Hyperbolic ProtoNet (Khrulkov et al. 2020) | ResNet-12 | $*60.65\pm{0.18}$ | $*76.13\pm{0.21}$ | $*67.38\pm{0.14}$ | $*79.11\pm{0.22}$
Ours (APP2S) | ResNet-12 | $66.25\pm{0.20}$ | $\bm{83.42\pm{0.15}}$ | $\bm{72.00}\pm{0.22}$ | $\bm{86.23\pm{0.15}}$
LwoF (Gidaris and Komodakis 2018) | WRN-28-10 | $60.06\pm{0.14}$ | $76.39\pm{0.11}$ | - | -
wDAE-GNN (Gidaris and Komodakis 2019) | WRN-28-10 | $61.07\pm{0.15}$ | $76.75\pm{0.11}$ | $68.18\pm{0.16}$ | $83.09\pm{0.12}$
LEO (Rusu et al. 2018) | WRN-28-10 | $61.76\pm{0.08}$ | $77.59\pm{0.12}$ | $66.33\pm{0.05}$ | $82.06\pm{0.08}$
Su _et al_. (Su, Maji, and Hariharan 2020) | ResNet-18 | - | $76.60\pm{0.70}$ | - | $78.90\pm{0.70}$
AFHN (Li et al. 2020) | ResNet-18 | $62.38\pm{0.72}$ | $78.16\pm{0.56}$ | - | -
Neg-Cosine (Liu et al. 2020) | ResNet-18 | $62.33\pm{0.82}$ | $80.94\pm{0.59}$ | - | -
Hyperbolic ProtoNet (Khrulkov et al. 2020) | ResNet-18 | $*57.05\pm{0.16}$ | $*74.20\pm{0.14}$ | $*66.20\pm{0.12}$ | $*76.50\pm{0.13}$
Ours (APP2S) | ResNet-18 | $64.82\pm{0.12}$ | $81.31\pm{0.22}$ | $70.83\pm{0.15}$ | $84.15\pm{0.29}$
|  |  |  |  |

Table 1: Few-shot classification accuracy and 95 % confidence interval on
_mini_ -ImageNet and _tiered_ -ImageNet with ResNet backbones. “*” notes the
result obtained by the self-implemented network.

Model | Backbone | CIFAR-FS | FC100
---|---|---|---
5-way 1-shot | 5-way 5-shot | 5-way 1-shot | 5-way 5-shot
TEAM (Qiao et al. 2019) | ResNet-12 | $70.40$ | $81.30$ | - | -
ProtoNet (Snell, Swersky, and Zemel 2017) | ResNet-12 | $72.20\pm{0.70}$ | $83.50\pm{0.50}$ | $37.50\pm{0.60}$ | $52.50\pm{0.60}$
TADAM (Oreshkin, López, and Lacoste 2018) | ResNet-12 | - | - | $40.10\pm{0.40}$ | $56.10\pm{0.40}$
DeepEMD (Zhang et al. 2020) | ResNet-12 | - | - | $46.47\pm{0.78}$ | $63.22\pm{71}$
Hyperbolic ProtoNet (Khrulkov et al. 2020) | ResNet-12 | *$70.27\pm{0.22}$ | *$80.98\pm{0.16}$ | *$36.04\pm{0.18}$ | *$51.60\pm{0.18}$
Ours (APP2S) | ResNet-12 | $\bm{73.12\pm{0.22}}$ | $\bm{85.69\pm{0.16}}$ | $\bm{47.64\pm{0.21}}$ | $\bm{63.56\pm{0.22}}$
|  |  |  |  |

Table 2: Few-shot classification accuracy and 95 % confidence interval on
CIFAR-FS and FC100 with ResNet-12 backbones. “*” notes the result obtained by
the self-implemented network.

Model | 5-way 1-shot | 5-way 5-shot
---|---|---
MAML (Finn, Abbeel, and Levine 2017) | $69.96\pm{1.01}$ | $82.70\pm{0.65}$
RelationNet (Sung et al. 2018) | $67.59\pm{1.02}$ | $82.75\pm{0.58}$
Chen _et al_. (Chen et al. 2019) | $67.02$ | $83.58$
MatchingNet (Vinyals et al. 2016) | $72.36\pm{0.90}$ | $83.64\pm{0.60}$
SimpleShot (Wang et al. 2019) | $70.28$ | $86.37$
ProtoNet (Snell, Swersky, and Zemel 2017) | $71.88\pm{0.91}$ | $87.42\pm{0.48}$
DeepEMD ♣ (Zhang et al. 2020) | $76.65\pm{0.83}$ | $88.69\pm{0.50}$
P-transfer ♣ (Shen et al. 2021) | $73.88\pm{0.87}$ | $87.81\pm{0.48}$
Hyperbolic ProtoNet (Khrulkov et al. 2020) | $*73.70\pm{0.22}$ | $*85.55\pm{0.13}$
Ours (APP2S) | $\bm{77.64\pm{0.19}}$ | $\bm{90.43\pm{0.18}}$
|  |

Table 3: Few-shot classification accuracy and 95 % confidence interval on CUB.
“*” notes the result obtained by the self-implemented network. “$\clubsuit$”
denotes the method using ResNet-12 as the backbone, otherwise ResNet-18.

_mini_ -ImageNet. As shown in Table 1, we evaluate our model using ResNet-12
and ResNet-18 as the backbones on _mini_ -ImageNet. Between them, ResNet-12
produces the best results. In addition, our model also outperforms recent
state-of-the-art models in most of the cases. Interestingly, our model further
outperforms hyperbolic ProtoNet by 7.77% and 7.11% for 5-way 1-shot and 5-way
5-shot with ResNet-18, respectively. With ResNet-12, we outperform the
hyperbolic ProtoNet by 5.60% and 7.29% for 5-way 1-shot and 5-way 5-shot,
respectively.

_tiered_ -ImageNet. We further evaluate our model on _tiered_ -ImageNet with
ResNet backbones. The results in Table 1 indicate that with ResNet-12, our
model outperforms the hyperbolic ProtoNet by 4.62% and 7.12% for 5-way 1-shot
and 5-way 5-shot, respectively, and achieves state-of-the-art results for
inductive few-shot learning.

CIFAR-FS and FC100. As the results in Table 2 suggested, our model also
achieves comparable performance with the relevant state-of-the-state methods
on this dataset, with ResNet-12 backbone, which vividly shows the superiority
of our method.

CUB. We use ResNet-18 as our backbone to evaluate our method on the CUB
dataset. Table 3 shows that our model improves the performance over baseline
by 3.94% and 4.88% for 5-way 1-shot and 5-way 5-shot settings, respectively.
Besides, our model achieves 77.64% and 90.43% for 5-way 1-shot and 5-way
5-shot settings on this dataset, which outperforms state-of-the-art models
(_i.e_., DeepEMD (Zhang et al. 2020) and P-transfer (Shen et al. 2021)) and
achieve competitive performance on this dataset.

###  5.3 Robustness to Outliers

To further validate the robustness of our method, we conduct experiments in
the presence of outliers in the form of mislabelled images. In the first
study, we add a various number of outliers (_e.g_., 1, 2, 3, 4), whose classes
are disjoint to the support-class, to each class of the support set. We
performed this study with ResNet-12 backbone on the 5-way 5-shot setting on
_tiered_ -ImageNet. Fig. 4 shows that the performances of hyperbolic ProtoNet
degrade remarkably. On the contrary, both our APP2S and Euclidean AP2S are
robust to outliers, which shows the superiority of our adaptive metric.
Comparing to Euclidean AP2S, APP2S is even more robust (see the slope of Fig.
4) and performs consistently even in the presence of 20 outliers. This
suggests that integrating our proposed adaptive metric and hyperbolic geometry
can further bring robustness to our framework. In the second study (shown in
Fig. 4), we conduct the same experiments on _mini_ -ImageNet. The results show
a similar trend as the previous one, which further proves the effectiveness of
our proposed method.

Figure 4: Robustness analysis. Horizontal axis: The number of outliers.
Vertical axis: Accuracy. (a): The performance vs. the number of outliers on
_tiered_ -ImageNet. (b): The performance vs. the number of outliers on _mini_
-ImageNet.

###  5.4 Ablation Study

We further conduct the ablation study to verify the effectiveness of each
component in our method on the _tiered_ -ImageNet dataset using the ResNet-12
backbone.

Experiments Set-Up. For setting (ii) in Table 4, we disable the relation
module $f_{\phi}$ and signature generator $f_{\omega}$. The P2S distance can
be obtained by Eq. (12) and Eq. (13) with equal weights (_i.e_., $1$).
Moreover, we enable the relation generator $f_{\phi}$ but not the signature
generator in setting (iii). We use the class prototype instead of the
signature for this experiment. We enable both $f_{\phi}$ and $f_{\omega}$ and
use the Euclidean distances for setting (iv). In the end, we enable the
Poincaré ball but disable the $f_{\zeta}$ for setting (v). In terms of
implementation of (v), the backbone is designed to output a feature vector
instead of a feature map, such that the P2S distance can be directly computed
by Eq. (3) and Eq. (12).

Effectiveness of Point to Set Distance. In this experiment, we first evaluate
the effectiveness of the P2S distance by comparing to its point to point (P2P)
distance counterpart (_i.e_., hyperbolic ProtoNet). From Table 4, we could
observe that the P2S distance can learn a more discriminative embedding space
than P2P distance (_i.e_., (i) vs. (ii)), and the adaptive P2S can further
bring performance gain to our application (_i.e_., (ii) vs. (iii)). This
observation shows the potential of our P2S distance setting in the FSL task.

Effectiveness of Signature Generator. We further evaluate another essential
component in our work, _i.e_., the signature generator, which refines the
entire support set and produces a signature per class. As shown in Table 4
(_i.e_., (iii) and (vi)), we could observe that our method benefits from the
signature generator, which shows that the signature of each class could help
to generate an informative weight for individual feature map within the same
class.

Effectiveness of Hyperbolic Geometry. We also implement our model in the
Euclidean space to verify the effectiveness of our method. The row (iv) and
(vi) in Table 4 vividly show that the representation in the Poincaré ball has
a richer embedding than that in Euclidean spaces.

Effectiveness of Set to Set Distance. The comparison between (v) and (vi)
shows that our set to set distance generator associated with the feature map
outputs richer information than using a feature vector to directly compute the
APP2S.

ID | Model | $\mathbb{{D}}^{n}_{c}$ | P2S | ${f}_{\phi}$ | ${f}_{\omega}$ | ${f}_{\zeta}$ |  _tiered_ -ImageNet
---|---|---|---|---|---|---|---
5-way 5-shot
(i) | Hyperbolic ProtoNet | ✓ |  |  |  |  | $79.11\pm{0.22}$
(ii) | Hyperbolic P2S w/o $f_{\phi}$ | ✓ | ✓ |  |  | ✓ | $83.14\pm{0.17}$
(iii) | Hyperbolic P2S w/ $f_{\phi}$ | ✓ | ✓ | ✓ |  | ✓ | $84.88\pm{0.17}$
(iv) | Euclidean AP2S |  | ✓ | ✓ | ✓ | ✓ | $81.96\pm{0.18}$
(v) | APP2S w/o $f_{\zeta}$ | ✓ | ✓ | ✓ | ✓ |  | $84.12\pm{0.13}$
(vi) | APP2S | ✓ | ✓ | ✓ | ✓ | ✓ | $\bm{86.23\pm{0.15}}$
|  |  |  |  |  |  |

Table 4: The ablation study of our model, we start from the hyperbolic
ProtoNet (Khrulkov et al. 2020) towards APP2S.

##  6 Conclusion

In this paper, we propose a novel adaptive Poincaré point to set (APP2S)
distance metric for the few-shot learning, which can adapt depending on the
samples at hands. Empirically, we showed that this approach is expressive with
both hyperbolic geometry and Euclidean counterpart. Our model improves the
performances over baseline models and achieves competing results on five
standard FSL benchmarks.

##  7 Supplementary Material

In this supplementary material, we provide an additional description of
operations in Poincaré Ball and details of the each public few-shot learning
benchmark we used. Furthermore, we conduct additional experiments, including
ablation studies on the effect of the curvature $c$, global feature vector
implementation and parameter and time complexity analysis to analyze the our
model. Finally, we provide the details of the implementation of our model and
extra visualizations and discussion of APP2S.

###  7.1 Hyperbolic Operations

Exponential Map. The exponential map defines a function from
$T_{{\bm{x}}}\mathbb{D}_{c}^{n}\to\mathbb{D}^{n}_{c}$, which maps Euclidean
vectors to the hyperbolic space. Formally, it is defined as:

${\bm{\Omega}}^{c}_{\bm{x}}({\bm{v}})={\bm{x}}\oplus_{c}(\text{tanh}(\sqrt{c}\frac{\lambda^{c}_{{\bm{x}}}\|{\bm{v}}\|}{2})\frac{{\bm{v}}}{\sqrt{c}\|{\bm{v}}\|}).$
(16)

The exponential map and logarithmic map (introduced in the main paper) have
simpler forms when ${\bm{x}}=0$:

${\bm{\Omega}}^{c}_{\bm{0}}({\bm{v}})=\text{tanh}(\sqrt{c}\|{\bm{v}}\|)\frac{{\bm{v}}}{\sqrt{c}\|{\bm{v}}\|},$
(17)

${\bm{\pi}}^{c}_{\bm{0}}({\bm{y}})=\text{arctanh}(\sqrt{c}\|{\bm{y}}\|)\frac{{\bm{y}}}{\sqrt{c}\|{\bm{y}}\|}.$
(18)

Parallel Transport. Parallel transport provides a way to move tangent vectors
along geodesics
$P_{{\bm{x}}\to{\bm{y}}}:T_{\bm{x}}\mathcal{M}\xrightarrow{}T_{\bm{y}}\mathcal{M}$
and defines a canonical way to connect tangent spaces. For further details of
hyperbolic space and geometry, please refer to the thesis (Ganea 2019).

###  7.2 Set to Set Distance

Set to set distance has been widely adopted in computer vision tasks (Fang et
al. 2021; Conci and Kubrusly 2018; Huttenlocher, Klanderman, and Rucklidge
1993). In this section, we discuss the well-known Hausdorff distance. There
are two variants of Hausdorff distance, including one-sided Hausdorff distance
and bidirectional Hausdorff distance. The one-sided Hausdorff distance between
set $A=\\{{\bm{a}}_{1},{\bm{a}}_{2},\ldots,{\bm{a}}_{n}\\}$ and set
$B=\\{{\bm{b}}_{1},{\bm{b}}_{2},\ldots,{\bm{b}}_{n}\\}$ can be defined as:

${\mathrm{d_{Hau}^{o}}}(A,B)=\max_{{\bm{a}}\in A}\min_{{\bm{b}}\in
B}\mathrm{d}({\bm{a}},{\bm{b}}),$ (19)

and the bidirectional Hausdorff distance can be defined as:

$\mathrm{d_{Hau}^{bi}}(A,B)=\max({\mathrm{d_{Hau}^{o}}}(A,B),{\mathrm{d_{Hau}^{o}}}(B,A)).$
(20)

###  7.3 Datasets

_mini_ -ImageNet. The _mini_ -ImageNet is a subset of ImageNet (Deng et al.
2009). The size of images in _mini_ -ImageNet is fixed to 84 $\times$ 84\. It
has 100 classes, with each having 600 samples. We adopt the standard setting
form (Ravi and Larochelle 2016) to split the dataset into 64, 16, and 20
classes for training, validation, and testing.

_tiered_ -ImageNet. Like _mini_ -ImageNet, _tiered_ -ImageNet (Ren et al.
2018) is also sampled from ImageNet, while it has more classes than the _mini_
-ImageNet. This dataset is split into 351 classes from 20 categories, 97
classes from 6 categories, and 160 classes from 8 different categories for
training, validation, and testing.

CUB. The CUB dataset (Wah et al. 2011) consists of 11,788 images from 200
different species of birds. Following the standard split (Liu et al. 2020),
the CUB dataset is divided into 100 species for training, 50 species for
validation, and another 50 species for testing.

CIFAR-FS and FC100. Both CIFAR-FS (Bertinetto et al. 2018) and FC100
(Oreshkin, López, and Lacoste 2018) are modified from the CIFAR-100 dataset
containing 100 classes, with 600 samples per class. The CIFAR-FS is split into
64, 16, and 20 classes for training, validation, and testing, respectively.
While the FC100 dataset is split into 60, 20, and 20 classes for training,
validation, and testing, respectively.

###  7.4 Additional Experiments

Conv-4 Backbone. We also employ the simple 4-convolutional network (Conv-4) to
evaluate our method on _mini-_ ImageNet comparing with some early works. The
Table 5 summarizes our results.

Model | 5-way 1-shot | 5-way 5-shot
---|---|---
MatchingNet (Vinyals et al. 2016) | $43.56\pm{0.84}$ | $55.31\pm{0.73}$
MAML (Finn, Abbeel, and Levine 2017) | $48.70\pm{1.84}$ | $63.11\pm{0.92}$
RelationNet (Sung et al. 2018) | $50.44\pm{0.82}$ | $65.32\pm{0.70}$
R2-D2 (Bertinetto et al. 2018) | $48.70\pm{0.60}$ | $65.50\pm{0.60}$
Reptile (Nichol, Achiam, and Schulman 2018) | $49.97\pm{0.32}$ | $65.99\pm{0.58}$
ProtoNet (Snell, Swersky, and Zemel 2017) | $49.42\pm{0.78}$ | $68.20\pm{0.66}$
Neg-Cosine (Liu et al. 2020) | $52.84\pm{0.76}$ | $70.41\pm{0.66}$
Hyperbolic ProtoNet (Khrulkov et al. 2020) | $54.43\pm{0.20}$ | $72.67\pm{0.15}$
Ours (APP2S) | $\bm{55.73\pm{0.20}}$ | $\bm{72.86\pm{0.22}}$
|  |

Table 5: Few-shot classification accuracy and 95 % confidence interval on
_mini_ -ImageNet with Conv-4 Backbone.

Comparison with DN4 and FEAT. The comparison of our APP2S, DN4, and FEAT on
_mini_ -ImageNet and _tiered_ -ImageNet with Conv-4 and ResNet-12 backbones is
summarized in Table 6.

Model | Backbone |  _mini_ -ImageNet |  _tiered_ -ImageNet
---|---|---|---
1-shot | 5-shot | 1-shot | 5-shot
DN4 (Li et al. 2019b) | Conv4 | $51.24$ | ${71.02}$ | - | -
FEAT (Ye et al. 2020) | Conv-4 | $55.15$ | ${71.13}$ | - | -
Ours | Conv-4 | $\bm{55.73}$ | $\bm{72.86}$ | - | -
FEAT | ResNet-12 | $\bm{66.78}$ | $82.05$ | $70.80$ | $84.79$
Ours | ResNet-12 | $66.25$ | $\bm{83.42}$ | $\bm{72.00}$ | $\bm{86.23}$
|  |  |  |  |

Table 6: The comparison of our model, DN4 and FEAT on the _mini_ -ImageNet and
_tiered_ -ImageNet with Conv-4 and ResNet-12 backbones.

The Curvature of Poincaré ball. The curvature of the Poincaré ball is an
important parameter, which determines the radius of the Poincaré ball. We
conduct experiments with different values of $c$ on _tiered_ -ImageNet. The
results are summarized into Table 7. As the results suggested, our model is
not very sensitive to $c$. However, with a larger $c$ value, the performance
is slightly better.

Model | $\bm{0.7}$ | $\bm{0.5}$ | $\bm{0.1}$ | $\bm{0.05}$ | $\bm{0.01}$ | $\bm{0.01}$
---|---|---|---|---|---|---
APP2S | $\bm{86.23}$ | $85.92$ | $85.21$ | $85.18$ | $84.43$ | $84.19$
|  |  |  |  |  |

Table 7: The influence from the curvature of Poincaré ball on the performance,
given _tiered_ -ImageNet and ResNet-12 backbone on 5-way 5-shot setting.

1-shot case. To fully leverage the capability of APP2S for 1-shot setting, we
require more than one sample in the set. Therefore, we followed the practice
in (Simon et al. 2020) to augment the support images by flipping. To have a
fair comparison, we also applied augmentation to our baseline model (_i.e_.,
hyperbolic ProtoNet (Khrulkov et al. 2020)) on both _mini_ -ImageNet and
_tiered_ -ImageNet, given ResNet-12 backbone. Table 8 shows that the image
augmentation does not boost the performance of the baseline model
significantly.

Model |  _mini_ -ImageNet |  _tiered_ -ImageNet
---|---|---
w/o Aug. | w/ Aug. | w/o Aug. | w/ Aug.
hyperbolic ProtoNet | $*60.65$ | $*59.97$ | $*67.38$ | $*67.84$
APP2S | - | $66.25$ | - | $72.00$
|  |  |  |

Table 8: The accuracy without and with augmentation for 1-shot setting. “*”
notes the results obtained by self-implemented network.

any-shot setting. We follow the any-way & any-shot setting introduced in (Lee
et al. 2019a) to further validate the efficacy of our algorithm. We use a
variant of our final model (APP2S without $f_{\zeta}$) to perform this
experiments due to less computation requirement on this experiment setting.
The results are shown in Table 9.

Model |  _tiered_ -ImageNet |  _mini_ -ImageNet
---|---|---
Conv-4 | ResNet-12 | Conv-4 | ResNet-12
ProtoNet | - | - | $51.02\pm{0.30}$ | -
L2G ProtoNet(Lee et al. 2019a) | - | - | $53.00\pm{0.28}$ | -
Ours | $\bm{66.23\pm{0.15}}$ | $\bm{75.52\pm{0.19}}$ | $\bm{58.62\pm{0.16}}$ | $\bm{70.01\pm{0.17}}$
|  |  |  |

Table 9: The results of our model under any-way & any-shot setting compared to
ProtoNet and L2G PrptoNet.

Using Global Feature Vectors. We performed extra experiments using global
feature vectors in our method. The table below shows that our method, even
with global feature vectors, outperforms the Hyperbolic ProtoNet
significantly, and the local features can further boost our methods. Note that
we use ResNet-18 backbone for CUB dataset and ResNet-12 for the rest.

Dataset | Hyperbolic ProtoNet | Ours w/o global feature | Ours w/ local feature
---|---|---|---
_mini_ -ImgeNet | $76.13\pm{0.21}$ | ${80.13\pm{0.11}}$ | ${83.42\pm{0.15}}$
_tiered_ -ImageNet | $79.11\pm{0.22}$ | ${84.12\pm{0.13}}$ | ${86.23\pm{0.15}}$
CUB | $85.55\pm{0.13}$ | ${90.24\pm{0.15}}$ | ${90.43\pm{0.18}}$
CIFAR-FS | ${80.98\pm{0.16}}$ | ${84.08\pm{0.16}}$ | ${85.69\pm{0.16}}$

Table 10: The results of our model using the global feature vectors compared
to Hyperbolic ProtoNet.

Parameter and time complexity analysis. Comparing to Hyperbolic ProtoNet, we
have $3$ extra modules, including $f_{\omega}$, $f_{\phi}$ and $f_{\zeta}$ to
realize the adaptive distances. We summarize the parameter numbers (PNs) and
FLOPs for each module and the backbone network. We can find that the PNs and
FLOPs of our module are acceptable as compared with the backbone network. We
also compare the time complexity to the SOTA method, _i.e_., DeepEMD, given
that both methods are using local feature maps. The FPS value of our method is
$83$, as compared to $0.6$ of DeepEMD under the 5-shot setting, clearly
showing that our method runs faster than DeepEMD. Note that both models are
tested on a single Nvidia Quadro-GV100 graphic card.

complexity metrics | $f_{\omega}$ | $f_{\phi}$ | $f_{\zeta}$ | $f_{\theta}$
---|---|---|---|---
PNs ($\times 10^{6}$) | $1.64$ | $0.74$ | $0.02$ | $\bm{12.42}$
FLOPs ($\times 10^{9}$) | $2.01$ | $0.17$ | $0.0008$ | $\bm{6.98}$

Table 11: The Parameter and time complexity analysis.

###  7.5 Implementation Details

Network and Optimizer. We mainly use ResNet (He et al. 2016), including
ResNet-12 and ResNet-18, as our backbones across all datasets. We also employ
the simple 4-convolutional network (Conv-4) to evaluate our method comparing
with some early works. The size of the input image is fixed to 84 $\times$
84\. We use Adam (Kingma and Ba 2014) and SGD (Ye et al. 2020) for Conv-4 and
ResNet backbones, respectively. In the SGD optimizer, we adopt the L2
regularizer with 0.0005 weight decay coefficient. In the ResNet-12 backbones,
we disable the average pooling and remove the last fully connected (FC) layer,
such that the networks generate the feature map with size of $640\times
5\times 5$. For ResNet-18, we set the average pooling layer to generate the
feature map with the size of $512\times 5\times 5$. Note that we set $c$ (the
curvature of the Poincaré ball) to 0.7 and 0.5 for 5-way 5-shot setting and
for 5-way-1-shot setting, respectively, with ResNet backbones, across all
datasets. While for Conv-4, we set $c$ to 0.4 for both 5-way 5-shot and 5way
1-shot settings across all datasets.

Training. Following the excellent practice in state-of-the-art methods (Ye et
al. 2020; Zhang et al. 2020; Simon et al. 2020), network training has two
stages, _i.e_., pre-training stage and meta-learning stage. In the pre-
training process, the backbone network followed by a FC layer is trained on
all training classes with the standard classification task. The network with
the highest validation accuracy is selected as the pre-trained backbone for
the next training stage. In the meta-learning stage, we also follow the
standard training protocol, where the network is trained for 200 epochs, and
each epoch samples 100 tasks randomly. In order to create the set for 5 way
1-shot setting, we follow the previous practice in (Simon et al. 2020), which
augments the image per class by horizontal flipping.

Signature Generator. For the signature generator, we choose Transformer
encoder as the set refinement function $f_{\omega}$ as it performs
contextualization over the whole support set with permutation invariant
property. Note that the Transformer is implemented with single-head self-
attention because more heads do not boost the performance but require more
computational power for our model by experiments. Moreover, We follow the
implementation of (Carion et al. 2020) to provide spatial encoding along with
flattened feature map into the transformer.

Relation Generator. We implemented the relation generator using a simple two-
layer CNN followed by a flatten operation in the end. In the first layer, the
linear transformation is followed by the batch normalization and activation.
The second layer uses the sigmoid function to bound the output. Finally, a
softmax layer is implemented to convert the output into a probability
distribution. The structure of the relation generator can be summarized into
Table 12.

layer name | output size | operation parameter
---|---|---
conv1 | $3\times 3$ |  $3\times 3$, 64, stride 1
batch norm | $3\times 3$ | 64
relu | $3\times 3$ | -
dropout | $3\times 3$ | $p=0.5$
conv2 | $1\times 1$ |  $3\times 3$, 1, stride 1
batch norm | $1\times 1$ | 1
sigmoid | $1\times 1$ | -
|  |

Table 12: The structure of the Relation Generator.

Set to Set Distance Generator. We simply implement a two layer MLP (_i.e_.,
$625\rightarrow 25\rightarrow 1$) as the set to set distance generator. The
structure is summarized into Table 13

layer name | output size | operation parameter
---|---|---
linear1 | $25$ | $625\rightarrow 25$
1D batch norm | $25$ | $25$
relu | $25$ | -
dropout | $25$ | $p=0.5$
linear2 | $1$ | $25\rightarrow 1$
1D batch norm | $1\times 1$ | 1
|  |

Table 13: The stucture of the Set to Set Distance Generator.

###  7.6 Extra Visualizations and Discussion

We also provide extra visualizations to show that the APP2S will adapt
depending on the constellation of the points in a set. Fig. 5 shows that in
both cases 5 and 5, APP2S assigns larger weights (dark blue area) to the
points that are closer to the center of the cluster, while smaller weights
(light blue) to the outliers.

Figure 5: (a): The P2S distances based on the minimum and maximum distances
are sensitive to outliers and ignore the distribution of the points in the set
to a great degree. (b) and (c): Our proposed point to set distance is bounded
between the infimum and the supremum and, it is also non-linear due to the
weighted-sum. It covers the distribution of the individual sample in the set
and adapts based on the relationship between the sample distribution and
overall set distribution.

Our P2S. The existing P2S distance metrics (_i.e_., the min and max distances
discussed in Preliminary) only consider the lower bound and upper bound of P2S
distance, thereby ignoring the distribution of the samples of the set to a
great degree. Furthermore, such metrics are very sensitive to the outliers in
the set (see Fig. 5). Our proposed adaptive P2S distance is a more flexible
metric and able to adapt based on the distribution of the samples in the set.
See Fig. 5 and 5 for an example, the measurement from our proposed metric is
more flexible than the existing ones. Note that the weight (_i.e_., $w_{ij}$)
generated by our method is distance-dependent. This is due to the way we model
the problem using the tangent space of the hyperbolic space. To see this,
recall that the norm of projected sample vector in support-class, which is the
input of the relation generator, is indeed the geodesic distance between the
associated support vector and the query vector on the manifold (_i.e_.,
$\|{\tilde{{\bm{s}}}}_{ij}^{{r}}\|=d_{c}(\bar{{\bm{q}}},{{{\bm{s}}}}_{ij}^{{r}})$).

RelationNet. Our relation generator resembles the RelationNet. However,
instead of computing the relation score between the prototype and the query,
our relation generator computes the relation score between each support sample
and its corresponding class-signature, further used as the adaptive factors
for our point-to-set distance.

DN4. The distance in DN4 resembles the point to set distance in our work.
However and in contrast to DN4, our point to set distance is adaptive, while
that in DN4 is fixed weighted summation.

## References

  * Antoniou, Edwards, and Storkey (2019) Antoniou, A.; Edwards, H.; and Storkey, A. 2019.  How to train your MAML.  In _International Conference on Learning Representations_.
  * Bertinetto et al. (2018) Bertinetto, L.; Henriques, J. F.; Torr, P.; and Vedaldi, A. 2018.  Meta-learning with differentiable closed-form solvers.  In _International Conference on Learning Representations_.
  * Carion et al. (2020) Carion, N.; Massa, F.; Synnaeve, G.; Usunier, N.; Kirillov, A.; and Zagoruyko, S. 2020.  End-to-end object detection with transformers.  In _European Conference on Computer Vision_ , 213–229. Springer.
  * Chen et al. (2019) Chen, W.-Y.; Liu, Y.-C.; Kira, Z.; Wang, Y.-C. F.; and Huang, J.-B. 2019.  A Closer Look at Few-shot Classification.  In _International Conference on Learning Representations_.
  * Conci and Kubrusly (2018) Conci, A.; and Kubrusly, C. S. 2018.  Distance Between Sets - A survey.
  * Deng et al. (2009) Deng, J.; Dong, W.; Socher, R.; Li, L.-J.; Li, K.; and Fei-Fei, L. 2009.  Imagenet: A large-scale hierarchical image database.  In _IEEE Conference on Computer Vision and Pattern Recognition_.
  * Doersch, Gupta, and Zisserman (2020) Doersch, C.; Gupta, A.; and Zisserman, A. 2020.  Crosstransformers: spatially-aware few-shot transfer.  _arXiv preprint arXiv:2007.11498_.
  * Fang, Harandi, and Petersson (2021) Fang, P.; Harandi, M.; and Petersson, L. 2021.  Kernel Methods in Hyperbolic Spaces.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision_.
  * Fang et al. (2021) Fang, P.; Ji, P.; Petersson, L.; and Harandi, M. 2021.  Set Augmented Triplet Loss for Video Person Re-Identification.  In _Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision WACV_.
  * Finn, Abbeel, and Levine (2017) Finn, C.; Abbeel, P.; and Levine, S. 2017.  Model-agnostic meta-learning for fast adaptation of deep networks.  In _Proceedings of the 34th International Conference on Machine Learning-Volume 70_ , 1126–1135.
  * Flennerhag et al. (2019) Flennerhag, S.; Rusu, A. A.; Pascanu, R.; Visin, F.; Yin, H.; and Hadsell, R. 2019\.  Meta-Learning with Warped Gradient Descent.  In _International Conference on Learning Representations_.
  * Franceschi et al. (2018) Franceschi, L.; Frasconi, P.; Salzo, S.; Grazzi, R.; and Pontil, M. 2018.  Bilevel programming for hyperparameter optimization and meta-learning.  In _International Conference on Machine Learning_ , 1568–1577. PMLR.
  * Ganea, Bécigneul, and Hofmann (2018) Ganea, O.; Bécigneul, G.; and Hofmann, T. 2018.  Hyperbolic neural networks.  In _Advances in Neural Information Processing Systems_.
  * Ganea (2019) Ganea, O.-E. 2019.  _Non-Euclidean Neural Representation Learning of Words, Entities and Hierarchies_.  Ph.D. thesis, ETH Zurich.
  * Gidaris and Komodakis (2018) Gidaris, S.; and Komodakis, N. 2018.  Dynamic few-shot visual learning without forgetting.  In _IEEE Conference on Computer Vision and Pattern Recognition_.
  * Gidaris and Komodakis (2019) Gidaris, S.; and Komodakis, N. 2019.  Generating classification weights with gnn denoising autoencoders for few-shot learning.  In _IEEE Conference on Computer Vision and Pattern Recognition_.
  * He et al. (2016) He, K.; Zhang, X.; Ren, S.; and Sun, J. 2016.  Deep residual learning for image recognition.  In _IEEE conference on Computer Vision and Pattern Recognition_.
  * Hong et al. (2021) Hong, J.; Fang, P.; Li, W.; Zhang, T.; Simon, C.; Harandi, M.; and Petersson, L. 2021.  Reinforced Attention for Few-Shot Learning and Beyond.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_.
  * Huttenlocher, Klanderman, and Rucklidge (1993) Huttenlocher, D. P.; Klanderman, G. A.; and Rucklidge, W. J. 1993.  Comparing images using the Hausdorff distance.  _IEEE Transactions on Pattern Analysis and Machine Intelligence_.
  * Ibanez et al. (2008) Ibanez, L.; Audette, M.; Yeo, B.; Golland, P.; Tustison, N.; and Gee, J. 2008.  The Use of Robust Local Hausdorff Distances in Accuracy Assessment for Image Alignment of Brain MRI.  _Insight Journal_.
  * Khrulkov et al. (2020) Khrulkov, V.; Mirvakhabova, L.; Ustinova, E.; Oseledets, I.; and Lempitsky, V. 2020\.  Hyperbolic image embeddings.  In _IEEE/CVF Conference on Computer Vision and Pattern Recognition_.
  * Kingma and Ba (2014) Kingma, D. P.; and Ba, J. 2014.  Adam: A method for stochastic optimization.  _arXiv preprint arXiv:1412.6980_.
  * Lee et al. (2019a) Lee, H.; Na, D.; Lee, H. B.; and Hwang, S. J. 2019a.  Learning to Generalize to Unseen Tasks with Bilevel Optimization.  _arXiv preprint arXiv:1908.01457_.
  * Lee et al. (2019b) Lee, K.; Maji, S.; Ravichandran, A.; and Soatto, S. 2019b.  Meta-learning with differentiable convex optimization.  In _IEEE Conference on Computer Vision and Pattern Recognition_.
  * Li et al. (2019a) Li, H.; Eigen, D.; Dodge, S.; Zeiler, M.; and Wang, X. 2019a.  Finding task-relevant features for few-shot learning by category traversal.  In _IEEE/CVF Conference on Computer Vision and Pattern Recognition_.
  * Li et al. (2020) Li, K.; Zhang, Y.; Li, K.; and Fu, Y. 2020.  Adversarial Feature Hallucination Networks for Few-Shot Learning.  In _IEEE/CVF Conference on Computer Vision and Pattern Recognition_.
  * Li et al. (2019b) Li, W.; Wang, L.; Xu, J.; Huo, J.; Gao, Y.; and Luo, J. 2019b.  Revisiting local descriptor based image-to-class measure for few-shot learning.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 7260–7268.
  * Lifchitz et al. (2019) Lifchitz, Y.; Avrithis, Y.; Picard, S.; and Bursuc, A. 2019.  Dense classification and implanting for few-shot learning.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 9258–9267.
  * Liu et al. (2020) Liu, B.; Cao, Y.; Lin, Y.; Li, Q.; Zhang, Z.; Long, M.; and Hu, H. 2020.  Negative margin matters: Understanding margin in few-shot classification.  In _European Conference on Computer Vision_ , 438–455. Springer.
  * Lu, Ye, and Zhan (2021) Lu, S.; Ye, H.-J.; and Zhan, D.-C. 2021.  Tailoring Embedding Function to Heterogeneous Few-Shot Tasks by Global and Local Feature Adaptors.  In _Proceedings of the AAAI Conference on Artificial Intelligence_ , volume 35, 8776–8783.
  * Nichol, Achiam, and Schulman (2018) Nichol, A.; Achiam, J.; and Schulman, J. 2018.  On first-order meta-learning algorithms.  _arXiv preprint arXiv:1803.02999_.
  * Oreshkin, López, and Lacoste (2018) Oreshkin, B.; López, P. R.; and Lacoste, A. 2018.  Tadam: Task dependent adaptive metric for improved few-shot learning.  In _Advances in Neural Information Processing Systems_.
  * Qiao et al. (2019) Qiao, L.; Shi, Y.; Li, J.; Wang, Y.; Huang, T.; and Tian, Y. 2019.  Transductive episodic-wise adaptive metric for few-shot learning.  In _IEEE International Conference on Computer Vision_.
  * Ravi and Larochelle (2016) Ravi, S.; and Larochelle, H. 2016.  Optimization as a model for few-shot learning.  In _International Conference on Learning Representations_.
  * Ravichandran, Bhotika, and Soatto (2019) Ravichandran, A.; Bhotika, R.; and Soatto, S. 2019.  Few-shot learning with embedded class models and shot-free meta training.  In _IEEE International Conference on Computer Vision_.
  * Ren et al. (2018) Ren, M.; Triantafillou, E.; Ravi, S.; Snell, J.; Swersky, K.; Tenenbaum, J. B.; Larochelle, H.; and Zemel, R. S. 2018.  Meta-Learning for Semi-Supervised Few-Shot Classification.  In _International Conference on Learning Representations_.
  * Rusu et al. (2018) Rusu, A. A.; Rao, D.; Sygnowski, J.; Vinyals, O.; Pascanu, R.; Osindero, S.; and Hadsell, R. 2018.  Meta-Learning with Latent Embedding Optimization.  In _International Conference on Learning Representations_.
  * Shen et al. (2021) Shen, Z.; Liu, Z.; Qin, J.; Savvides, M.; and Cheng, K.-T. 2021.  Partial Is Better Than All: Revisiting Fine-tuning Strategy for Few-shot Learning.  In _Proceedings of the AAAI Conference on Artificial Intelligence_ , volume 35, 9594–9602.
  * Simon et al. (2020) Simon, C.; Koniusz, P.; Nock, R.; and Harandi, M. 2020.  Adaptive Subspaces for Few-Shot Learning.  In _IEEE/CVF Conference on Computer Vision and Pattern Recognition_.
  * Snell, Swersky, and Zemel (2017) Snell, J.; Swersky, K.; and Zemel, R. 2017.  Prototypical networks for few-shot learning.  In _Advances in Neural Information Processing Systems_.
  * Su, Maji, and Hariharan (2020) Su, J.-C.; Maji, S.; and Hariharan, B. 2020.  When does self-supervision improve few-shot learning?  In _European Conference on Computer Vision_ , 645–666. Springer.
  * Sun et al. (2019) Sun, S.; Sun, Q.; Zhou, K.; and Lv, T. 2019.  Hierarchical attention prototypical networks for few-shot text classification.  In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing_ , 476–485.
  * Sung et al. (2018) Sung, F.; Yang, Y.; Zhang, L.; Xiang, T.; Torr, P. H.; and Hospedales, T. M. 2018\.  Learning to compare: Relation network for few-shot learning.  In _IEEE Conference on Computer Vision and Pattern Recognition_.
  * Tang et al. (2020) Tang, H.; Li, Z.; Peng, Z.; and Tang, J. 2020.  BlockMix: meta regularization and self-calibrated inference for metric-based meta-learning.  In _Proceedings of the 28th ACM International Conference on Multimedia_ , 610–618.
  * Vinyals et al. (2016) Vinyals, O.; Blundell, C.; Lillicrap, T.; Wierstra, D.; et al. 2016.  Matching networks for one shot learning.  In _Advances in Neural Information Processing Systems_.
  * Wah et al. (2011) Wah, C.; Branson, S.; Welinder, P.; Perona, P.; and Belongie, S. 2011.  The caltech-ucsd birds-200-2011 dataset.
  * Wang et al. (2019) Wang, Y.; Chao, W.-L.; Weinberger, K. Q.; and van der Maaten, L. 2019.  Simpleshot: Revisiting nearest-neighbor classification for few-shot learning.  _arXiv preprint arXiv:1911.04623_.
  * Wang et al. (2020) Wang, Y.; Xu, C.; Liu, C.; Zhang, L.; and Fu, Y. 2020.  Instance credibility inference for few-shot learning.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 12836–12845.
  * Wertheimer, Tang, and Hariharan (2021) Wertheimer, D.; Tang, L.; and Hariharan, B. 2021.  Few-Shot Classification With Feature Map Reconstruction Networks.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 8012–8021.
  * Xu et al. (2021) Xu, C.; Fu, Y.; Liu, C.; Wang, C.; Li, J.; Huang, F.; Zhang, L.; and Xue, X. 2021\.  Learning Dynamic Alignment via Meta-filter for Few-shot Learning.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 5182–5191.
  * Ye et al. (2020) Ye, H.-J.; Hu, H.; Zhan, D.-C.; and Sha, F. 2020.  Few-shot learning via embedding adaptation with set-to-set functions.  In _IEEE/CVF Conference on Computer Vision and Pattern Recognition_.
  * Zhang et al. (2020) Zhang, C.; Cai, Y.; Lin, G.; and Shen, C. 2020.  DeepEMD: Few-Shot Image Classification with Differentiable Earth Mover’s Distance and Structured Classifiers.  In _IEEE/CVF Conference on Computer Vision and Pattern Recognition_.