# Diversifying Deep Ensembles: A Saliency Map Approach for Enhanced OOD
Detection, Calibration, and Accuracy

Stanislav Dereka
Tinkoff
<EMAIL_ADDRESS>
Ivan Karpukhin
Tinkoff
<EMAIL_ADDRESS>
Sergey Kolesnikov
Tinkoff
<EMAIL_ADDRESS>
###### Abstract

Deep ensembles achieved state-of-the-art results in classification and out-of-
distribution (OOD) detection; however, their effectiveness remains limited due
to the homogeneity of learned patterns within the ensemble. To overcome this
challenge, our study introduces a novel approach that promotes diversity among
ensemble members by leveraging saliency maps. By incorporating saliency map
diversification, our method outperforms conventional ensemble techniques in
multiple classification and OOD detection tasks, while also improving
calibration. Experiments on well-established OpenOOD benchmarks highlight the
potential of our method in practical applications.

##  1 Introduction

In recent years, deep neural networks (DNNs) achieved state-of-the-art results
in many computer vision tasks, including detection [1], classification [2],
image retrieval [3], and face recognition [4]. In image classification, DNNs
even surpassed human quality according to popular benchmarks [5]. Unlike
humans, DNNs performance largely degrades under domain shifts [6]. Therefore
measuring confidence, and detecting out-of-domain (OOD) data becomes a crucial
part of risk-controlled recognition.

Confidence estimation and OOD detection were extensively studied in the
context of deep learning. Some works aimed at calibration improvements in
softmax classification [7]. Another group of works studied Bayesian training
[8] . However, it was shown, that ensembles of DNNs [9] achieve state-of-the-
art in both confidence estimation and OOD detection [10]. The quality of the
ensemble was further improved by diversification of models predictions [11,
12, 13] or by applying different training strategies to the models [14]. At
the same time, previous works don’t study the relation between input features,
used by the models, and ensemble quality.

Figure 1: Saliency diversification: in contrast to Deep Ensembles, the models
within the proposed SDDE ensemble use different features for prediction,
leading to improved generalization and confidence estimation.

In this work, we propose a new ensemble training algorithm, namely Saliency
Diversified Deep Ensemble (SDDE)111The code is attached as supplementary
material and will be released with a camera-ready version of the paper., that
forces models to use independent input features for prediction. We do this by
estimating regions of interest, i.e. saliency maps, during training and
implementing a loss function for their diversification. We also show, that the
commonly used average aggregation of predicted probabilities loses important
information about OOD samples, and argue that confidence must be estimated
using logits before the softmax layer. By implementing both improvements, we
establish a new state-of-the-art in OpenOOD benchmark [10] in terms of test
set accuracy, confidence estimation, and OOD detection quality in most cases.

The claims of the current work are summarized as follows:

  1. 1.

We propose a diversification loss that increases diversity among ensemble
models using saliency maps.

  2. 2.

We study aggregation methods and show that logit aggregation improves OOD
detection accuracy compared to the traditional mean probability approach.

  3. 3.

We establish a new state-of-the-art in most classification, confidence
estimation, and OOD detection benchmarks from OpenOOD.

##  2 Preliminaries

In this section, we will give a brief description of the model and saliency
maps estimation approaches, used in the proposed SDDE method. Suppose
$x\in\mathbb{R}^{CHW}$ is an input image with height $H$, width $W$, and $C$
channels. Lets consider a classification model $f(x)=h(g(x))$, which consists
of Convolutional Neural Network (CNN)
$g:\mathbb{R}^{CHW}\rightarrow\mathbb{R}^{C^{\prime}H^{\prime}W^{\prime}}$ and
classifier
$h:\mathbb{R}^{C^{\prime}H^{\prime}W^{\prime}}\rightarrow\mathbb{R}^{L}$ with
$L$ equal to the number of classes. CNN produces $C^{\prime}$ feature maps
$M^{c},c=\overline{1,C^{\prime}}$ with spatial dimensions $H^{\prime}$ and
$W^{\prime}$. The output of the classifier is a vector of logits, that can be
mapped to class probabilities by a softmax layer.

The importance of image pixels for output prediction is usually evaluated by
computing saliency maps. A simple way to build a saliency map of the model is
to compute gradients of the output w.r.t. input features [15]:

$S_{Inp}(x)=\sum\limits_{i=1}^{L}\nabla_{x}f_{i}(x).$ (1)

It was shown in previous works, that input gradient produces noisy outputs
[15]. A more robust approach is to use Class Activation Maps (CAM) [16], or
their generalization, called GradCAM [17]. Both methods exploit the structure
of CNNs with bottleneck features having spatial directions. Unlike CAM,
GradCAM can be applied at any layer of the CNN, which is useful for small
input images. In GradCAM, the region of interest is estimated by analyzing
activations of the feature maps. The weight of each feature map channel is
computed as

$\alpha_{c}=\frac{1}{H^{\prime}W^{\prime}}\sum\limits_{i=1}^{H^{\prime}}\sum\limits_{j=1}^{W^{\prime}}\frac{\delta
f_{y}(x)}{\delta M^{c}_{i,j}},$ (2)

where $y\in\overline{1,L}$ is an image label. Having feature maps weights, the
saliency map is computed as

$S_{GradCAM}(x,y)=ReLU\left(\sum\limits_{c=1}^{C^{\prime}}\alpha_{c}M^{c}\right),$
(3)

where ReLU performs element-wise maximum between input and zero. The size of
CAM is equal to the size of feature maps. When visualized, CAMs are resized to
match the size of the input image.

Figure 2: Training pipeline: we compute saliency maps using the GradCAM method
for each model and apply a diversification loss. The diversification loss is
mixed with the cross-entropy loss for each model.

##  3 Method

The proposed SDDE method extends Deep Ensemble [9] by applying diversity loss
and by implementing a logits-based aggregation method for OOD detection.

###  3.1 Diversity Loss

In Deep Ensemble [9] models $f^{k},k\in\overline{1,N}$ are trained
independently. While there is a source of diversity in weight initialization,
the method doesn’t force different models to use different features. We thus
raise a question, how can we train the models, which rely on different input
features and implement different classification logic? We answer this question
by proposing a new loss function, which is motivated by previous research on
interpretability [17].

The idea behind SDDE is to make saliency maps [15] of the models as different
as possible, as shown in Figure 1. Suppose we have computed saliency map
$S^{k}(x,y)$ for each model. Models’ similarity can be measured as a mean
cosine similarity between their saliency maps. We thus propose the diversity
loss function, by computing a mean cosine similarity between the saliency maps
of different models:

$\mathcal{L}_{div}(x,y)=\frac{2}{N(N-1)}\sum\limits_{k_{1}>k_{2}}\left\langle
S^{k_{1}}(x,y),S^{k_{2}}(x,y)\right\rangle.$ (4)

In SDDE we use GradCAM [17], as it is more stable and requires less
computation than input gradient method[15]. As GradCAM is differentiable, the
diversity loss can be optimized via gradient descent along with cross-entropy
loss.

The diversity loss is mixed with cross-entropy during training in a single
optimization pipeline. The final loss has the following form:

$\mathcal{L}(x,y)=\lambda\mathcal{L}_{div}(x,y)+\frac{1}{N}\sum\limits_{k}\mathcal{L}^{k}_{CE}(x,y),$
(5)

where $\mathcal{L}^{k}_{CE}(x,y)$ is a standard cross-entropy loss for the
$k$-th model and $\lambda$ is a diversity loss weight.

###  3.2 Aggregation Methods for OOD detection

In real-life applications, input data can contain samples with classes unseen
during training or samples from different classification domains. Detecting
out-of-domain data is an important part of risk-controlled recognition. The
traditional approach to OOD detection is to apply a classification threshold
to the models’ confidence [10]. The confidence is usually estimated as a
maximum output probability. In the case of ensembles, the maximum is applied
to the average of probabilities, predicted by multiple models, but there are
other possible methods to use. We thus raise a question, which aggregation
leads to a higher OOD detection accuracy? In this work, we introduce and
compare multiple approaches.

Suppose there is an ensemble $f^{k}(x),k=\overline{1,N}$. Let’s denote softmax
outputs for each model as $p^{k}_{i}$ and average probabilities as $p_{i}$:

$p^{k}_{i}(x)=\frac{e^{f^{k}_{i}(x)}}{\sum\limits_{j}e^{f^{k}_{j}(x)}}=\mathrm{Softmax}_{i}(f^{k}),$
(6) $p_{i}(x)_{=}\frac{1}{N}\sum\limits_{k}{p^{k}_{i}(x)}.$ (7)

The rule of thumb in ensembles is to predict the confidence based on the
maximum output probability:

$U_{avg}(x)=\max\limits_{i}p_{i}(x).$ (8)

However, some works in offline reinforcement learning [18] suggest using
minimum aggregation, which penalizes rare data:

$U_{min}(x)=\min\limits_{k}\max\limits_{i}{p^{k}_{i}(x)}.$ (9)

The intuition behind the ensemble is that different models produce different
outputs when uncertainty is high. The straightforward consequence of this
intuition is that a high STD of model predictions indicates OOD data. We thus
introduce and evaluate STD aggregation:

$U_{STD}(x)=\sqrt{\mathrm{Var}_{k}\max\limits_{i}p^{k}_{i}(x)}.$ (10)

On the other hand, softmax activation loses some information about logits. For
example, an output of the softmax layer doesn’t depend on the input offset:

$\mathrm{Softmax}(x_{1}+\delta,x_{2}+\delta,\dots,x_{n}+\delta)=\mathrm{Softmax}(x_{1},x_{2},\dots,x_{n}),\forall\delta.$
(11)

Some previous works suggest that classification errors correspond to low logit
values [19]. We thus raise the question, of whether it is better to detect OOD
data using probabilities or logits. To answer this question, we introduce and
evaluate the logit-based variants of the aggregations above:

$U^{l}_{avg}(x)=\max\limits_{i}\frac{1}{N}\sum\limits_{k}f^{k}_{i}(x),$ (12)
$U^{l}_{min}(x)=\min\limits_{k}\max\limits_{i}{f^{k}_{i}(x)},$ (13)
$U^{l}_{STD}(x)=\sqrt{\mathrm{Var}_{k}\max\limits_{i}f^{k}_{i}(x)}.$ (14)

In the proposed SDDE method, we use $U^{l}_{avg}$ aggregation unless otherwise
specified. A detailed comparison is provided in the experiments section.

##  4 Related Work

###  4.1 Confidence Estimation

DNNs are prone to overfitting, which limits their ability to generalize and
predict confidence [7]. Multiple works attempted to fix this issue. A simple
approach to estimate models’ confidence is to take a probability of the
predicted class on the output of the softmax layer [20]. Several authors
proposed improvements to this method either for better confidence estimation
or higher OOD detection accuracy. Some works studied activation statistics
between layers to detect anomaly behavior [21]. Another approach is to use
Bayesian training, which usually leads to improved confidence prediction at
the cost of accuracy [8]. Other works attempted to use insights from classical
machine learning, like KNNs [22] and ensembles [9]. It was shown, that
ensembles reduce overfitting and produce confidence estimates, which
outperform most existing methods [10]. There are also multiple methods, which
make use of extra data, like OOD samples [23]. However, in many practical
applications, it is hard to predict the type of OOD data beforehand. In this
work, we consider methods, that use only in-domain data.

###  4.2 Ensembles Diversification

According to previous works, ensemble methods produce state-of-the-art results
in popular classification [9, 13] and OOD detection tasks [10]. It was shown,
that the quality of the ensemble largely depends on the diversity of
underlying models [14]. Some works improve diversity by implementing special
loss functions on predicted probabilities. The idea is to make predicted
distributions dissimilar for different models. The NCL loss [11] reduces
correlations between output probabilities, but can hurt the prediction of the
correct class. The ADP [12] solves this problem by diversifying only
probabilities of alternative classes. The DICE [13] loss reduces dependency
between bottleneck features of multiple models. While previous works
decorrelate either model predictions or bottleneck features, we directly
diversify input features, used by the models within an ensemble.

Ensemble diversification was also studied in offline Reinforcement Learning
(offline RL), where some methods use minimum aggregation to penalize unseen
states.[18]. However, the training scheme in offline RL can’t be directly
transferred to classification due to the differences in optimization
objectives. In this work, we for the first time study aggregation techniques
in the context of OOD detection for computer vision.

Table 1: Accuracy and calibration metrics. The best results for each dataset
and metric are bolded.

Dataset | Metric | DE | NCL | ADP | DICE | SDDE (Our)
---|---|---|---|---|---|---
MNIST | NLL ($10^{-2}$) $\downarrow$ | 3.81 $\pm$ 0.18 | 3.63 $\pm$ 0.16 | 3.52 $\pm$ 0.24 | 3.67 $\pm$ 0.22 | 3.68 $\pm$ 0.27
ECE ($10^{-2}$) $\downarrow$ | 0.21 $\pm$ 0.06 | 0.19 $\pm$ 0.07 | 0.31 $\pm$ 0.05 | 0.29 $\pm$ 0.07 | 0.28 $\pm$ 0.08
Brier score ($10^{-2}$) $\downarrow$ | 1.95 $\pm$ 0.09 | 1.89 $\pm$ 0.08 | 1.77 $\pm$ 0.12 | 1.84 $\pm$ 0.09 | 1.92 $\pm$ 0.15
Accuracy (%) $\uparrow$ | 98.72 $\pm$ 0.07 | 98.76 $\pm$ 0.03 | 98.85 $\pm$ 0.11 | 98.81 $\pm$ 0.09 | 98.73 $\pm$ 0.14
CIFAR10 | NLL ($10^{-2}$) $\downarrow$ | 13.04 $\pm$ 0.16 | 13.26 $\pm$ 0.14 | 13.55 $\pm$ 0.20 | 14.50 $\pm$ 0.26 | 12.77 $\pm$ 0.12
ECE ($10^{-2}$) $\downarrow$ | 0.84 $\pm$ 0.16 | 0.87 $\pm$ 0.13 | 1.00 $\pm$ 0.14 | 1.33 $\pm$ 0.06 | 0.82 $\pm$ 0.15
Brier score ($10^{-2}$) $\downarrow$ | 5.96 $\pm$ 0.08 | 6.03 $\pm$ 0.08 | 5.99 $\pm$ 0.07 | 6.33 $\pm$ 0.23 | 5.88 $\pm$ 0.07
Accuracy (%) $\uparrow$ | 96.07 $\pm$ 0.10 | 95.96 $\pm$ 0.08 | 96.07 $\pm$ 0.12 | 95.90 $\pm$ 0.16 | 96.08 $\pm$ 0.11
CIFAR100 | NLL ($10^{-2}$) $\downarrow$ | 72.14 $\pm$ 1.37 | 72.21 $\pm$ 0.48 | 75.43 $\pm$ 0.29 | 74.59 $\pm$ 0.54 | 69.29 $\pm$ 0.48
ECE ($10^{-2}$) $\downarrow$ | 3.78 $\pm$ 0.17 | 3.86 $\pm$ 0.23 | 3.73 $\pm$ 0.28 | 4.46 $\pm$ 0.36 | 3.45 $\pm$ 0.35
Brier score ($10^{-2}$) $\downarrow$ | 27.33 $\pm$ 0.76 | 27.34 $\pm$ 0.33 | 27.08 $\pm$ 0.17 | 27.12 $\pm$ 0.26 | 26.66 $\pm$ 0.33
Accuracy (%) $\uparrow$ | 80.95 $\pm$ 0.54 | 80.92 $\pm$ 0.20 | 81.18 $\pm$ 0.06 | 81.26 $\pm$ 0.25 | 81.27 $\pm$ 0.24
TIN | NLL ($10^{-2}$) $\downarrow$ | 1.21 $\pm$ 0.01 | 1.21 $\pm$ 0.01 | 1.27 $\pm$ 0.01 | 1.45 $\pm$ 0.01 | 1.20 $\pm$ 0.00
ECE ($10^{-2}$) $\downarrow$ | 4.86 $\pm$ 0.14 | 5.01 $\pm$ 0.41 | 5.80 $\pm$ 0.26 | 2.93 $\pm$ 0.17 | 4.99 $\pm$ 0.45
Brier score ($10^{-2}$) $\downarrow$ | 39.28 $\pm$ 0.00 | 39.32 $\pm$ 0.00 | 39.84 $\pm$ 0.00 | 45.93 $\pm$ 0.00 | 39.27 $\pm$ 0.00
Accuracy (%) $\uparrow$ | 71.38 $\pm$ 0.20 | 71.33 $\pm$ 0.13 | 71.23 $\pm$ 0.27 | 65.32 $\pm$ 0.20 | 71.41 $\pm$ 0.10

Table 2: OOD detection results. All methods are trained on the ID dataset and
tested on multiple OOD sources. Mean and STD AUROC values are reported. The
best results among different methods are bolded.

ID | Method | Near OOD | Far OOD | Total Near | Total Far
---|---|---|---|---|---
Fashion MNIST | NotMNIST | CIFAR10 | TIN | Texture | Places365
MNIST | DE [9] | 95.34 $\pm$ 0.53 | 89.56 $\pm$ 0.84 | 99.06 $\pm$ 0.15 | 98.93 $\pm$ 0.18 | 96.32 $\pm$ 0.79 | 99.06 $\pm$ 0.14 | 92.45 $\pm$ 0.60 | 98.34 $\pm$ 0.31
ADP [12] | 96.49 $\pm$ 0.67 | 90.98 $\pm$ 0.65 | 99.48 $\pm$ 0.18 | 99.38 $\pm$ 0.21 | 97.52 $\pm$ 0.77 | 99.44 $\pm$ 0.15 | 93.74 $\pm$ 0.48 | 98.96 $\pm$ 0.32
NCL [11] | 95.27 $\pm$ 0.42 | 89.19 $\pm$ 0.69 | 99.06 $\pm$ 0.11 | 98.97 $\pm$ 0.12 | 97.17 $\pm$ 0.45 | 99.03 $\pm$ 0.11 | 92.23 $\pm$ 0.53 | 98.56 $\pm$ 0.19
DICE [13] | 95.85 $\pm$ 0.22 | 89.88 $\pm$ 1.51 | 99.22 $\pm$ 0.19 | 99.10 $\pm$ 0.22 | 97.52 $\pm$ 1.05 | 99.20 $\pm$ 0.17 | 92.87 $\pm$ 0.67 | 98.76 $\pm$ 0.41
SDDE (Our) | 98.85 $\pm$ 0.38 | 94.27 $\pm$ 1.32 | 99.88 $\pm$ 0.03 | 99.84 $\pm$ 0.03 | 99.97 $\pm$ 0.01 | 99.82 $\pm$ 0.04 | 96.56 $\pm$ 0.67 | 99.88 $\pm$ 0.02
| CIFAR100 | TIN | MNIST | SVHN | Texture | Places365 |
CIFAR10 | DE [9] | 90.31 $\pm$ 0.23 | 91.84 $\pm$ 0.16 | 95.35 $\pm$ 0.53 | 95.05 $\pm$ 0.47 | 92.52 $\pm$ 0.30 | 91.29 $\pm$ 0.59 | 91.07 $\pm$ 0.17 | 93.55 $\pm$ 0.20
ADP [12] | 89.96 $\pm$ 0.25 | 91.62 $\pm$ 0.13 | 95.41 $\pm$ 0.24 | 94.66 $\pm$ 0.40 | 92.55 $\pm$ 0.12 | 91.38 $\pm$ 0.53 | 90.79 $\pm$ 0.17 | 93.50 $\pm$ 0.16
NCL [11] | 90.51 $\pm$ 0.12 | 91.93 $\pm$ 0.08 | 95.09 $\pm$ 0.29 | 94.81 $\pm$ 0.46 | 92.18 $\pm$ 0.34 | 92.00 $\pm$ 0.26 | 91.22 $\pm$ 0.09 | 93.52 $\pm$ 0.17
DiCE [13] | 89.09 $\pm$ 0.79 | 90.89 $\pm$ 0.48 | 94.46 $\pm$ 1.10 | 94.89 $\pm$ 0.49 | 92.36 $\pm$ 0.45 | 89.84 $\pm$ 1.14 | 89.99 $\pm$ 0.63 | 92.89 $\pm$ 0.47
SDDE (Our) | 91.43 $\pm$ 0.09 | 93.18 $\pm$ 0.14 | 97.11 $\pm$ 0.39 | 95.73 $\pm$ 0.40 | 93.53 $\pm$ 0.45 | 93.48 $\pm$ 0.11 | 92.31 $\pm$ 0.09 | 94.96 $\pm$ 0.15
| CIFAR10 | TIN | MNIST | SVHN | Texture | Places365 |
CIFAR100 | DE [9] | 80.83 $\pm$ 0.55 | 84.28 $\pm$ 0.52 | 80.24 $\pm$ 1.43 | 81.09 $\pm$ 1.27 | 80.16 $\pm$ 0.41 | 80.96 $\pm$ 0.26 | 82.55 $\pm$ 0.51 | 80.61 $\pm$ 0.51
ADP [12] | 81.12 $\pm$ 0.19 | 84.85 $\pm$ 0.22 | 79.54 $\pm$ 0.76 | 82.84 $\pm$ 2.16 | 81.80 $\pm$ 0.47 | 81.28 $\pm$ 0.11 | 82.98 $\pm$ 0.18 | 81.37 $\pm$ 0.54
NCL [11] | 81.11 $\pm$ 0.20 | 84.48 $\pm$ 0.19 | 79.66 $\pm$ 0.66 | 80.91 $\pm$ 1.99 | 80.30 $\pm$ 0.52 | 81.24 $\pm$ 0.17 | 82.79 $\pm$ 0.18 | 80.53 $\pm$ 0.66
DICE [13] | 81.42 $\pm$ 0.18 | 84.94 $\pm$ 0.21 | 83.37 $\pm$ 1.11 | 82.40 $\pm$ 1.96 | 81.43 $\pm$ 0.39 | 81.41 $\pm$ 0.27 | 83.18 $\pm$ 0.14 | 82.15 $\pm$ 0.85
SDDE (Our) | 81.97 $\pm$ 0.09 | 85.41 $\pm$ 0.14 | 81.86 $\pm$ 1.61 | 83.40 $\pm$ 0.92 | 82.67 $\pm$ 0.31 | 81.63 $\pm$ 0.16 | 83.69 $\pm$ 0.08 | 82.39 $\pm$ 0.50
| CIFAR10 | CIFAR100 | MNIST | SVHN | Texture | Places365 |
TIN | DE | 87.67 $\pm$ 0.89 | 84.69 $\pm$ 0.78 | 81.21 $\pm$ 4.36 | 84.59 $\pm$ 3.35 | 77.35 $\pm$ 0.32 | 82.78 $\pm$ 0.28 | 86.18 $\pm$ 0.82 | 81.48 $\pm$ 1.65
ADP | 87.21 $\pm$ 0.87 | 84.28 $\pm$ 0.66 | 81.01 $\pm$ 3.28 | 79.44 $\pm$ 3.73 | 76.69 $\pm$ 0.18 | 82.66 $\pm$ 0.26 | 85.75 $\pm$ 0.75 | 79.95 $\pm$ 1.70
NCL | 87.71 $\pm$ 0.90 | 85.08 $\pm$ 1.02 | 84.40 $\pm$ 2.09 | 84.63 $\pm$ 1.91 | 76.93 $\pm$ 0.56 | 82.93 $\pm$ 0.34 | 86.39 $\pm$ 0.96 | 82.22 $\pm$ 0.73
DICE | 84.35 $\pm$ 1.17 | 81.30 $\pm$ 0.60 | 90.00 $\pm$ 3.78 | 84.45 $\pm$ 1.55 | 74.65 $\pm$ 0.35 | 80.81 $\pm$ 0.07 | 82.83 $\pm$ 0.87 | 80.23 $\pm$ 0.88
SDDE (Our) | 89.16 $\pm$ 0.99 | 86.47 $\pm$ 0.70 | 82.49 $\pm$ 6.33 | 84.75 $\pm$ 2.72 | 79.80 $\pm$ 0.33 | 84.36 $\pm$ 0.28 | 87.81 $\pm$ 0.83 | 82.85 $\pm$ 2.24

Table 3: Comparison of different aggregation methods for OOD detection with
SDDE ensemble. The mean near and far OOD AUROC values (%) are reported.

Aggregation | MNIST | CIFAR10 | CIFAR100 | TIN
---|---|---|---|---
$U_{avg}(baseline)$ | 94.01 $\pm$ 0.65 / 99.18 $\pm$ 0.15 | 91.24 $\pm$ 0.12 / 93.73 $\pm$ 0.25 | 82.96 $\pm$ 0.14 / 80.84 $\pm$ 0.57 | 87.07 $\pm$ 0.76 / 81.54 $\pm$ 2.12
$U_{min}$ | 94.98 $\pm$ 0.68 / 99.56 $\pm$ 0.05 | 89.22 $\pm$ 3.77 / 91.49 $\pm$ 4.86 | 82.22 $\pm$ 0.37 / 80.59 $\pm$ 0.67 | 85.11 $\pm$ 0.73 / 81.60 $\pm$ 1.85
$U_{STD}$ | 93.42 $\pm$ 0.77 / 91.40 $\pm$ 1.00 | 87.19 $\pm$ 5.72 / 89.16 $\pm$ 6.86 | 71.76 $\pm$ 0.75 / 72.49 $\pm$ 0.56 | 57.66 $\pm$ 1.08 / 65.16 $\pm$ 1.64
$U^{l}_{avg}$ | 96.24 $\pm$ 0.45 / 99.70 $\pm$ 0.07 | 91.90 $\pm$ 0.13 / 94.55 $\pm$ 0.32 | 83.69 $\pm$ 0.08 / 82.39 $\pm$ 0.50 | 88.02 $\pm$ 0.74 / 83.11 $\pm$ 1.96
$U^{l}_{min}$ | 96.56 $\pm$ 0.67 / 99.88 $\pm$ 0.02 | 89.94 $\pm$ 4.10 / 92.21 $\pm$ 5.61 | 82.38 $\pm$ 0.35 / 81.51 $\pm$ 0.66 | 85.43 $\pm$ 0.74 / 82.41 $\pm$ 1.73
$U^{l}_{STD}$ | 50.06 $\pm$ 4.20 / 23.91 $\pm$ 2.93 | 78.49 $\pm$ 11.78 / 80.23 $\pm$ 13.04 | 52.78 $\pm$ 2.07 / 54.48 $\pm$ 1.93 | 32.74 $\pm$ 0.85 / 42.37 $\pm$ 2.22

##  5 Experiments

Figure 3: Pairwise cosine similarities distributions between Class Activation
Maps (CAMs) of ensemble models.

###  5.1 Experimental Setup

In our experiments, we follow the experimental setup and training procedure
from the OpenOOD benchmark [10]. We use ResNet18 [24] for CIFAR10, CIFAR100
[25], and TinyImageNet [26] and LeNet [27] for the MNIST dataset [27]. All
models are trained via SGD optimizer with a momentum of 0.9. The initial
learning rate is set to 1.0 and then reduced to $10^{-6}$ with Cosine
Annealing scheduler [28]. Training lasts 50 epochs on MNIST and 100 epochs on
CIFAR10, CIFAR100, and TinyImageNet. In contrast to OpenOOD, we perform a
multi-seed evaluation with 5 random seeds and report mean / STD values for
each experiment. We compare the proposed SDDE method with other ensembling and
diversification approaches, namely Deep Ensembles (DE) [9], Negative
Correlation Learning (NCL) [11], Adaptive Diversity Promoting (ADP) [12], and
DICE diversification loss [13]. In the default setup, we train ensembles of 5
models. In SDDE, we set the parameter $\lambda$ from the Equation 5 to 0.1 for
MNIST, and to 0.005 for CIFAR10/100 and TinyImageNet.

###  5.2 Ensemble Diversity

Figure 4: Class Activation Maps (CAMs) for SDDE and the baseline methods. SDDE
produces more diverse CAMs.

To demonstrate the effectiveness of the proposed diversification loss, we
evaluate saliency maps for all considered methods, as shown in Figure 4. The
ground truth label is not available during inference, so we compute CAMs for
the predicted class. It can be seen, that SDDE models use different input
regions for prediction. To highlight the difference between the methods, we
analyze pairwise cosine similarities between CAMs of different models in an
ensemble. The distributions of cosine similarities are presented in Figure 3.
It can be seen, that the baseline methods reduce similarity, compared to Deep
Ensemble, but SDDE achieves the lowest cosine similarities among all methods.
This effect persists even on OOD samples. In the following experiments, we
study the benefits of the proposed diversification.

###  5.3 Ensemble Accuracy and Calibration

Ensemble methods are expected to improve classification accuracy and
prediction calibration metrics. We compare these aspects of SDDE with other
ensembling methods by measuring test set classification accuracy, Negative
Log-Likelihood (NLL), Expected Calibration Error (ECE) [29], and Brier score
[30]. All metrics are computed after temperature tuning on the validation set
[31]. The results are presented in Table 1. It can be seen, that the SDDE
approach outperforms other methods on CIFAR10 and CIFAR100 in terms of both
accuracy and calibration. It also outperforms other methods on the TIN dataset
by accuracy, Brier Score, and test set NLL.

###  5.4 OOD Detection

We evaluate SDDE’s OOD detection ability using the OpenOOD benchmark. Since
SDDE doesn’t use external data for training, we make a comparison with
ensembles, that use only in-domain data. The results are presented in Table 2.
It can be seen, that SDDE achieves state-of-the-art results in almost all
cases, including total scores on near and far tests.

Figure 5: Classification accuracy and OOD detection quality depending on the
ensemble size.

In Section 3.2 we’ve covered ensemble prediction aggregation techniques, and
proposed multiple new methods. In this experiment, we aim to find the best-
performing aggregation in terms of near and far OOD detection quality. SDDE
results for multiple aggregation methods are presented in Table 3. It can be
seen, that $U^{l}_{min}$ achieves better quality on the MNIST dataset and
$U^{l}_{avg}$ is better in other cases. Based on these results, we choose
$U^{l}_{avg}$ as a default aggregation for SDDE.

###  5.5 Ensemble Size

The key parameter of the ensemble is the number of models. The large ensemble
usually provides a better confidence estimate, but the size negatively affects
the computation speed. The question rises, which number of models is the best?
To answer this question, we analyze the dependency of the quality on the
number of models. The results for classification and OOD detection quality are
presented in Figure 5. It can be seen, that SDDE effectively uses multiple
models leading to better results for all ensemble sizes starting from 3 in
CIFAR-10 and CIFAR-100 Near and Far setups.

##  6 Conclusion

In this work, we proposed SDDE, a novel ensembling method for classification
and OOD detection. SDDE forces the models within the ensemble to use different
input features for the prediction and increases ensemble diversity. According
to our experiments, SDDE outperforms popular ensembles on CIFAR10, CIFAR100,
and Tiny ImageNet datasets. At the same time, SDDE outperforms other methods
in OOD detection according to the OpenOOD benchmark and effectively scales to
a larger number of models. Improved confidence estimation and OOD detection
make SDDE a valuable tool for risk-controlled recognition.

## References

  * [1] Li Liu, Wanli Ouyang, Xiaogang Wang, Paul Fieguth, Jie Chen, Xinwang Liu, and Matti Pietikäinen,  “Deep learning for generic object detection: A survey,”  International journal of computer vision, vol. 128, pp. 261–318, 2020.
  * [2] Lucas Beyer, Olivier J Hénaff, Alexander Kolesnikov, Xiaohua Zhai, and Aäron van den Oord,  “Are we done with imagenet?,”  arXiv preprint arXiv:2006.07159, 2020.
  * [3] Kevin Musgrave, Serge Belongie, and Ser-Nam Lim,  “A metric learning reality check,”  in Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XXV 16. Springer, 2020, pp. 681–699.
  * [4] Jiankang Deng, Jia Guo, Niannan Xue, and Stefanos Zafeiriou,  “Arcface: Additive angular margin loss for deep face recognition,”  in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2019, pp. 4690–4699.
  * [5] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun,  “Delving deep into rectifiers: Surpassing human-level performance on imagenet classification,”  in Proceedings of the IEEE international conference on computer vision, 2015, pp. 1026–1034.
  * [6] Mei Wang and Weihong Deng,  “Deep visual domain adaptation: A survey,”  Neurocomputing, vol. 312, pp. 135–153, 2018.
  * [7] Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q Weinberger,  “On calibration of modern neural networks,”  in International conference on machine learning. PMLR, 2017, pp. 1321–1330.
  * [8] Ethan Goan and Clinton Fookes,  “Bayesian neural networks: An introduction and survey,”  Case Studies in Applied Bayesian Data Science: CIRM Jean-Morlet Chair, Fall 2018, pp. 45–87, 2020.
  * [9] Balaji Lakshminarayanan, Alexander Pritzel, and Charles Blundell,  “Simple and scalable predictive uncertainty estimation using deep ensembles,”  Advances in neural information processing systems, vol. 30, 2017\.
  * [10] Jingkang Yang, Pengyun Wang, Dejian Zou, Zitang Zhou, Kunyuan Ding, Wenxuan Peng, Haoqi Wang, Guangyao Chen, Bo Li, Yiyou Sun, et al.,  “Openood: Benchmarking generalized out-of-distribution detection,”  arXiv preprint arXiv:2210.07242, 2022.
  * [11] Changjian Shui, Azadeh Sadat Mozafari, Jonathan Marek, Ihsen Hedhli, and Christian Gagné,  “Diversity regularization in deep ensembles,”  arXiv preprint arXiv:1802.07881, 2018.
  * [12] Tianyu Pang, Kun Xu, Chao Du, Ning Chen, and Jun Zhu,  “Improving adversarial robustness via promoting ensemble diversity,”  in International Conference on Machine Learning. PMLR, 2019, pp. 4970–4979.
  * [13] Alexandre Ramé and Matthieu Cord,  “Dice: Diversity in deep ensembles via conditional redundancy adversarial estimation,”  in ICLR 2021-9th International Conference on Learning Representations, 2021.
  * [14] Asa Cooper Stickland and Iain Murray,  “Diverse ensembles improve calibration,”  arXiv preprint arXiv:2007.04206, 2020.
  * [15] Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman,  “Deep inside convolutional networks: Visualising image classification models and saliency maps,”  arXiv preprint arXiv:1312.6034, 2013.
  * [16] Bolei Zhou, Aditya Khosla, Agata Lapedriza, Aude Oliva, and Antonio Torralba,  “Learning deep features for discriminative localization,”  in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 2921–2929.
  * [17] Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra,  “Grad-cam: Visual explanations from deep networks via gradient-based localization,”  in Proceedings of the IEEE international conference on computer vision, 2017, pp. 618–626.
  * [18] Gaon An, Seungyong Moon, Jang-Hyun Kim, and Hyun Oh Song,  “Uncertainty-based offline reinforcement learning with diversified q-ensemble,”  Advances in neural information processing systems, vol. 34, pp. 7436–7447, 2021.
  * [19] Abdel Aziz Taha, Leonhard Hennig, and Petr Knoth,  “Confidence estimation of classification based on the distribution of the neural network output layer,”  arXiv preprint arXiv:2210.07745, 2022.
  * [20] Dan Hendrycks and Kevin Gimpel,  “A baseline for detecting misclassified and out-of-distribution examples in neural networks,”  arXiv preprint arXiv:1610.02136, 2016.
  * [21] Yiyou Sun, Chuan Guo, and Yixuan Li,  “React: Out-of-distribution detection with rectified activations,”  Advances in Neural Information Processing Systems, vol. 34, pp. 144–157, 2021.
  * [22] Yiyou Sun, Yifei Ming, Xiaojin Zhu, and Yixuan Li,  “Out-of-distribution detection with deep nearest neighbors,”  in International Conference on Machine Learning. PMLR, 2022, pp. 20827–20840.
  * [23] Jingkang Yang, Haoqi Wang, Litong Feng, Xiaopeng Yan, Huabin Zheng, Wayne Zhang, and Ziwei Liu,  “Semantically coherent out-of-distribution detection,”  in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021, pp. 8301–8309.
  * [24] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun,  “Deep residual learning for image recognition,”  in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 770–778.
  * [25] Alex Krizhevsky and Geoffrey Hinton,  “Learning multiple layers of features from tiny images,”  Tech. Rep. 0, University of Toronto, Toronto, Ontario, 2009.
  * [26] Ya Le and Xuan Yang,  “Tiny imagenet visual recognition challenge,”  CS 231N, vol. 7, no. 7, pp. 3, 2015.
  * [27] Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner,  “Gradient-based learning applied to document recognition,”  Proceedings of the IEEE, vol. 86, no. 11, pp. 2278–2324, 1998.
  * [28] Ilya Loshchilov and Frank Hutter,  “Sgdr: Stochastic gradient descent with warm restarts,”  arXiv preprint arXiv:1608.03983, 2016.
  * [29] Mahdi Pakdaman Naeini, Gregory Cooper, and Milos Hauskrecht,  “Obtaining well calibrated probabilities using bayesian binning,”  in Proceedings of the AAAI conference on artificial intelligence, 2015, vol. 29.
  * [30] Glenn W Brier et al.,  “Verification of forecasts expressed in terms of probability,”  Monthly weather review, vol. 78, no. 1, pp. 1–3, 1950.
  * [31] Arsenii Ashukha, Alexander Lyzhov, Dmitry Molchanov, and Dmitry Vetrov,  “Pitfalls of in-domain uncertainty estimation and ensembling in deep learning,”  arXiv preprint arXiv:2002.06470, 2020.