# Exploring the solution space of linear inverse problems with GAN latent
geometry

###### Abstract

Inverse problems consist in reconstructing signals from incomplete sets of
measurements and their performance is highly dependent on the quality of the
prior knowledge encoded via regularization. While traditional approaches focus
on obtaining a unique solution, an emerging trend considers exploring multiple
feasibile solutions. In this paper, we propose a method to generate multiple
reconstructions that fit both the measurements and a data-driven prior learned
by a generative adversarial network. In particular, we show that, starting
from an initial solution, it is possible to find directions in the latent
space of the generative model that are null to the forward operator, and thus
keep consistency with the measurements, while inducing significant perceptual
change. Our exploration approach allows to generate multiple solutions to the
inverse problem an order of magnitude faster than existing approaches; we show
results on image super-resolution and inpainting problems.

Index Terms—  Inverse problems, GANs

##  1 Introduction

Linear inverse problems are ubiquitous in the sciences as they are tasked with
reconstructing a signal of interest from a set of typically incomplete or
degraded measurements. In the imaging field alone [1], numerous problems of
interest such as deblurring, super-resolution, inpainting, compressed sensing,
and many more fit this framework. Due to the ill-posed nature of the problem,
one needs strong regularization to find reconstructions that fit the
measurements and the a priori knowledge of the signal properties. Traditional
approaches focused on hand-crafting regularizers to yield a unique solution by
casting reconstruction as a convex optimization problem [2]. However, one must
accept that the quality of this unique solution can only be as good as how
well the chosen regularizer function captures the signal properties. For this
reason, recently, data-driven methods based on neural networks [3, 4] started
learning priors directly from the complex distributions of the signals of
interest, resulting in improved reconstruction capabilities.

Nevertheless, even when using data-driven priors, we can hardly hope to
capture a perfect model of our signals of interest, which in turn affects how
faithful the reconstructed signal is to the true one that generated the
measurements. For this reason, a novel paradigm is emerging where multiple
feasible reconstructions are generated, in an effort to boost interpretability
of the inversion process and expose the biases of the models.

In this paper, we use generative adversarial networks (GANs) as priors
modeling the distribution of our signals of interest. We present a geometrical
perspective on the latent space of such models, which allows to explore the
solution space of a linear inverse problem. By exploration of the solution
space, we mean finding multiple reconstructions that are consistent with the
measurements but also consistent with the model of the data distribution. We
show that it is possible to modify an initial solution by moving towards
directions in the latent space that are “null” with respect to the
measurements operator (i.e., they do not significantly perturb the
measurements) while inducing semantic change. Our proposed technique, called
e-GLASS (exploring GAN LAtent Space Solutions), is general as it can be
applied to any linear inverse problem and is an order of magnitude faster than
state-of-the-art methods such as PULSE [5] which generate multiple solutions
by solving an optimization problem from different random initializations.

##  2 Background and related work

Let us start from a general linear forward model of the form:

$\displaystyle{\mathbf{y}}={\mathbf{A}}{\mathbf{x}}+{\mathbf{n}}$ (1)

where ${\mathbf{y}}\in{\mathbb{R}}^{m}$ is a noisy observation from an unknown
signal ${\mathbf{x}}\in{\mathbb{R}}^{n}$ with $m\leq n$, depending on the
specific problem; ${\mathbf{n}}$ is an additive noise and ${\mathbf{A}}$ is a
measurement matrix.

As an example, ${\mathbf{y}}$ can be a degraded image, e.g., with low
resolution or blurred, and we want to reconstruct the image ${\mathbf{x}}$
starting from the measurements ${\mathbf{y}}$. However, this problem is ill-
posed as there can be infinitely many solutions satisfying the measurements,
or even none due to noise. A large body of work has been devoted to the
development of priors to model ${\mathbf{x}}$ as accurately as possible to
regularize the problem towards admitting a unique solution. Such works frame
reconstruction as a Maximum a Posteriori (MAP) estimation problem, with the
unique solution obtained by solving

$\displaystyle\hat{{\mathbf{x}}}=\arg\min_{{\mathbf{x}}}\|{\mathbf{y}}-{\mathbf{A}}{\mathbf{x}}\|_{2}^{2}+\lambda
R({\mathbf{x}}),$

for some handcrafted regularizer $R$ encoding the prior. This line of works
typically defines priors such that the reconstruction problem is convex with a
unique global minimum. This means that a single solution to the problem can be
obtained, whose properties are strictly intertwined with the ability to craft
a suitable prior $R$.

New recent approaches involve generative models, such as GANs, to learn priors
in a data-driven fashion [6], [7]. A GAN learns a function $G$ that maps a
latent vector ${\mathbf{z}}$ into a sample from the data distribution. The
popular approach of GAN inversion solves inverse problems by seeking the
latent vector $\hat{{\mathbf{z}}}$ that best fits the measurements
${\mathbf{y}}$. This is done by minimizing the distance between ${\mathbf{y}}$
and the degraded version of the generated data $G(\hat{{\mathbf{z}}})$, under
the forward model ${\mathbf{A}}$:

$\displaystyle\hat{{\mathbf{z}}}=\arg\min_{{\mathbf{z}}}\|{\mathbf{y}}-{\mathbf{A}}G({\mathbf{z}})\|_{2}^{2},\quad\hat{{\mathbf{x}}}=G(\hat{{\mathbf{z}}}).$
(2)

Unlike convex optimization methods with handcrafted priors, GAN inversion is
non-convex due to the use of neural networks, thus admitting multiple local
minima.

While most works have used GAN inversion to generate a single solution to the
inverse problem, there has been recent growing interest in exploring the
solution space of inverse problems, i.e., finding multiple solutions, among
the infinitely many possible, that are consistent with the measurements and
some data prior. The seminal work on this topic is PULSE [5], which uses GAN
inversion to super-resolve low-resolution faces. PULSE generates multiple
plausible solutions by solving Eq. (2) via gradient descent, and starting from
different random guesses of ${\mathbf{z}}$. Due to the non-convex nature of
the optimization, different solutions may be reached when starting from
different initializations. The main drawback of PULSE lies in its complexity,
requiring to solve an optimization problem for each solution and the lack of
any guarantee that a different initialization will converge to a different
minimum. Other works [8] have sought to generate multiple solutions for the
super-resolution problem, but they lack generality and can only be applied to
very specific neural networks devised only for the super-resolution task.

Finally, we remark that there is extensive literature on GAN editability [9],
[10], [11], [12], seeking to manipulate the latent space of GANs to induce
semantically interesting transformations. However, such works are not in the
framework of solutions to inverse problems and are not concerned with fidelity
with measurements.

##  3 Proposed method

In this paper, we propose a method to explore multiple solutions of a linear
inverse problem, starting from a first solution $G({\mathbf{z}}_{0})$. The
method exploits geometrical properties of the GAN latent space $\mathcal{Z}$
to navigate in a neighborhood of ${\mathbf{z}}_{0}$ in such a way that the new
generated data preserve the condition in Eq. (1) (i.e., they are solutions to
the inverse problem) while manifesting novel semantic information with respect
to $G({\mathbf{z}}_{0})$.

The latent space $\mathcal{Z}$ can be seen as a Riemannian manifold, and a GAN
$G$ parametrizes a submanifold to the data space $\mathcal{X}$, and,
ultimately to the measurement space $\mathcal{Y}$ via the composition of
generator and forward model $\phi=G\circ{\mathbf{A}}$. Wang and Ponce [13]
argue that the geometry of $\mathcal{Z}$ in a neighborhood of
${\mathbf{z}}_{0}$ can be approximated by a positive semi-definite quadratic
form $H({\mathbf{z}}_{0})$:

$\displaystyle
d^{2}({\mathbf{z}}_{0},{\mathbf{z}})\approx\delta{\mathbf{z}}^{T}\frac{\partial^{2}d^{2}({\mathbf{z}}_{0},{\mathbf{z}})}{\partial{\mathbf{z}}^{2}}\biggr{\rvert}_{{\mathbf{z}}_{0}}\delta{\mathbf{z}},\quad{\mathbf{H}}({\mathbf{z}}_{0}):=\frac{\partial
d^{2}({\mathbf{z}}_{0},{\mathbf{z}})}{\partial{\mathbf{z}}^{2}}\biggr{\rvert}_{{\mathbf{z}}_{0}},$

dependent on a distance metric $d$ between latent vectors. While the authors
in [13] define $d$ between latents as the distance in the generated data space
$\mathcal{X}$, we also consider it in the measurement space $\mathcal{Y}$,
since we are interested in exploring how variations in the measurement space
affect the latent geometry. In particular, we define
$d_{\mathcal{Y}}({\mathbf{z}}_{1},{\mathbf{z}}_{2}):=\|\phi({\mathbf{z}}_{1})-\phi({\mathbf{z}}_{2})\|_{2}^{2}=\|{\mathbf{A}}G({\mathbf{z}}_{1})-{\mathbf{A}}G({\mathbf{z}}_{2})\|_{2}^{2}$
and induce the corresponding manifold described by Riemannian metric:

$\displaystyle{\mathbf{H}}_{\mathcal{Y}}({\mathbf{z}}_{0})=\frac{1}{2}\frac{\partial^{2}}{\partial{\mathbf{z}}^{2}}\|\phi({\mathbf{z}})-\phi({\mathbf{z}}_{0})\|_{2}^{2}|_{{\mathbf{z}}_{0}}={\mathbf{J}}_{\phi}^{T}({\mathbf{z}}_{0}){\mathbf{J}}_{\phi}({\mathbf{z}}_{0}),$

being
${\mathbf{J}}_{\phi}({\mathbf{z}}_{0})=\partial_{{\mathbf{z}}}\phi({\mathbf{z}})|_{{\mathbf{z}}_{0}}$
the Jacobian of $\phi=G\circ{\mathbf{A}}$ evaluated at point
${\mathbf{z}}_{0}$. Similarly, metric
${\mathbf{H}}_{\mathcal{X}}({\mathbf{z}}_{0})$ is induced by a suitable
distance in the data space. In this work, we will focus on images and,
consequently, we use the LPIPS distance (a perceptual metric defined from
features extracted by a pretrained network) [14] as
$d_{\mathcal{X}}({\mathbf{z}}_{1},{\mathbf{z}}_{2}):=\text{LPIPS}(G({\mathbf{z}}_{1}),G({\mathbf{z}}_{2}))$.
Backpropagation can be used to compute ${\mathbf{H}}_{\mathcal{Y}}$ and
${\mathbf{H}}_{\mathcal{X}}$.

Armed with this characterization of the geometry of the latent space, we seek
to generate a new latent vector corresponding to a solution as
${\mathbf{z}}_{1}={\mathbf{z}}_{0}+\eta{\mathbf{d}}$, i.e., by perturbing
${\mathbf{z}}_{0}$ along a direction ${\mathbf{d}}$ that maximizes perceptual
distance in the image space (large
$d_{\mathcal{X}}({\mathbf{z}}_{1},{\mathbf{z}}_{0})$) but minimizes distance
in the measurement space (small
$d_{\mathcal{Y}}({\mathbf{z}}_{1},{\mathbf{z}}_{0})$). In other words, we seek
to explore the subspace of $\mathcal{Z}$ around ${\mathbf{z}}_{0}$ that is
“null” with respect to the measurements operator but not so with respect to
perceptual distance.

One might wonder whether this is possible at all, and, in fact, the answer is
affirmative and relies on two main phenomena. The first was observed by Wang
and Ponce [13] and it is the anisotropy of the space described by
${\mathbf{H}}_{\mathcal{X}}({\mathbf{z}}_{0})$, i.e.,
${\mathbf{H}}_{\mathcal{X}}$ is described by a small number of principal
components, meaning that there is a large number of directions that have
little to no effect on perceptual quality and some significantly changing
it111[13] also note that the space is homogeneous, meaning that this property
is valid everywhere, regardless of the specific ${\mathbf{z}}_{0}$.. We
empirically observe the same regarding the geometry induced by the
measurements fidelity, i.e., ${\mathbf{H}}_{\mathcal{Y}}({\mathbf{z}}_{0})$.
The second phenomenon, which is at the basis of our work, is that the
directions from ${\mathbf{H}}_{\mathcal{X}}$ and ${\mathbf{H}}_{\mathcal{Y}}$
can be empirically decoupled. This means that it is indeed possible to find
directions that significantly affect perceptual distance while having little
to no impact on measurements, yielding novel solutions to the inverse problem.

Algorithm 1 e-GLASS: exploring GAN LAtent Space Solutions

${\mathbf{z}}_{0}$, $K$

New solution $\hat{{\mathbf{x}}}$

Compute Hessian
${\mathbf{H}}_{\mathcal{Y}}({\mathbf{z}}_{0})=\frac{1}{2}\frac{\partial^{2}}{\partial{\mathbf{z}}^{2}}\|{\mathbf{A}}G({\mathbf{z}})-{\mathbf{A}}G({\mathbf{z}}_{0})\|_{2}^{2}$

Compute Hessian
${\mathbf{H}}_{\mathcal{X}}({\mathbf{z}}_{0})=\frac{1}{2}\frac{\partial^{2}}{\partial{\mathbf{z}}^{2}}\text{LPIPS}(G({\mathbf{z}}),G({\mathbf{z}}_{0}))$

Compute eigenvectors
${\mathbf{H}}_{\mathcal{Y}}({\mathbf{z}}_{0})={\mathbf{U}}\Lambda{\mathbf{U}}^{T}$

Compute eigenvectors
${\mathbf{H}}_{\mathcal{X}}({\mathbf{z}}_{0})={\mathbf{V}}\Omega{\mathbf{V}}^{T}$

${\mathbf{d}}\leftarrow{\mathbf{v}}^{K}$

$\mathcal{J}=\\{$top-k eigenvectors of ${\mathbf{H}}_{\mathcal{Y}}\\}$

for $j\in\mathcal{J}$ do

${\mathbf{d}}\leftarrow{\mathbf{d}}-({\mathbf{d}}^{T}{\mathbf{u}}^{j}){\mathbf{u}}^{j}$

${\mathbf{d}}\leftarrow{\mathbf{d}}/\|{\mathbf{d}}\|$

end for

$\hat{{\mathbf{x}}}=G({\mathbf{z}}_{0}+\eta{\mathbf{d}})$

Algorithm 1 summarizes our proposed e-GLASS scheme to find such directions. We
first start by finding the latent code ${\mathbf{z}}_{0}$ corresponding to a
single solution by means of any state-of-the-art GAN inversion technique. Then
we compute the Hessians ${\mathbf{H}}_{\mathcal{Y}}({\mathbf{z}}_{0})$,
${\mathbf{H}}_{\mathcal{X}}({\mathbf{z}}_{0})$ and their eigenvectors:
${\mathbf{H}}_{\mathcal{Y}}({\mathbf{z}}_{0})={\mathbf{U}}\Lambda{\mathbf{U}}^{T}$,
${\mathbf{H}}_{\mathcal{X}}({\mathbf{z}}_{0})={\mathbf{V}}\Omega{\mathbf{V}}^{T}$.
We then need to measure the coupling between the two sets of eigenvectors via
the coupling matrix ${\mathbf{C}}={\mathbf{U}}^{T}{\mathbf{V}}$. It is
expected that the top eigenvectors in ${\mathbf{U}}$ are coupled with the top
eigenvectors in ${\mathbf{V}}$ as large perceptual distances typically also
correspond to large differences on the measurements. However, the bottom
eigenvectors are not correlated, indicating that the corresponding null spaces
do not necessarily intersect each other. The most interesting directions for
our problem are the eigenvectors that are among the top in ${\mathbf{V}}$ but
do not correlate with the top eigenvectors in ${\mathbf{U}}$. However,
directly choosing such direction is in general suboptimal, as it might still
increase the distance in the measurement space more than desired.

To solve this problem, we propose a geometrical method that removes the most
relevant correlations with the top eigenvectors of
${\mathbf{H}}_{\mathcal{Y}}$. This allows to obtain a new direction that is
hopefully still creating perceptually significant differences but projected as
much as possible onto the null space of ${\mathbf{H}}_{\mathcal{Y}}$ to
minimally change the measurements. To do this, we first choose
${\mathbf{d}}={\mathbf{v}}^{K}$ as the $K$-th top eigenvector to discard the
very top eigenvectors that are coupled with the top ones in ${\mathbf{U}}$.
Then, we project ${\mathbf{v}}^{K}$ onto the hyperplane orthogonal to the top
eigenvectors ${\mathbf{u}}^{j}$ with correlation larger than a threshold:

$\displaystyle{\mathbf{d}}\leftarrow{\mathbf{d}}-({\mathbf{d}}^{T}{\mathbf{u}}^{j}){\mathbf{u}}^{j},\quad{\mathbf{d}}\leftarrow{\mathbf{d}}/\|{\mathbf{d}}\|.$

This procedure is iterated until the resulting direction has no significant
correlation to the top eigenvectors of ${\mathbf{H}}_{\mathcal{Y}}$. This
leads to a projection of ${\mathbf{v}}^{K}$ onto the null space of
${\mathbf{H}}_{\mathcal{Y}}$. Multiple solutions to the inverse problem can be
explored by either changing the step $\eta$ along the direction, or trying a
new direction by computing ${\mathbf{d}}$ starting from ${\mathbf{v}}^{K-1}$,
${\mathbf{v}}^{K-2}$, …

##  4 Experimental Results

In this section, we experimentally evaluate the proposed method against state-
of-the-art techniques to explore multiple solutions. While the proposed method
is general and holds for different generative models and different inverse
problems, we focus on two notable inverse problems, i.e., image super-
resolution (SR) and inpainting (IP), presenting results for two different
generative models, i.e. BigGAN [15] and PGGAN [16]. For super-resolution, we
downscale the $256\times 256$ image to a $32\times 32$ image, while for
inpainting, we delete a semantically interesting area of a face from a full
image of size $1024\times 1024$.

We first present empirical evidence about our claims in Sec.3 that directions
that induce little change on measurements and significant perceptual change on
the reconstructions do exist. To show this we want to see that the chosen
direction correlates with the top eigenvectors of ${\mathbf{V}}$ while being
as orthogonal as possible to the top eigenvectors of ${\mathbf{U}}$. Fig. 1
shows in blue the correlation coefficient between the starting direction
${\mathbf{v}}^{K}$ and the eigenvectors in ${\mathbf{U}}$ and ${\mathbf{V}}$
and in red the same correlations but with respect to the final direction
${\mathbf{d}}$ provided by our method. It can be noticed that the final
direction has been successfully orthogonalized with respect to the directions
inducing significant variation on the measurements, while it retains good
correlation with directions inducing perceptual change.

(a) Correlation with ${\mathbf{U}}$.

(b) Correlation with ${\mathbf{V}}$.

Fig. 1: Correlation of initial direction ${\mathbf{v}}^{K}$ and final
direction ${\mathbf{d}}$ with eigenvectors of latent space metrics
${\mathbf{H}}_{\mathcal{Y}}$ and ${\mathbf{H}}_{\mathcal{X}}$. The final
direction is orthogonal to directions inducing large change in measurements,
but correlates with directions inducing significant perceptual change.

We now qualitatively and quantitatively examine the performance of the
proposed method for the chosen inverse problems in comparison with PULSE [5].
PULSE generates multiple solutions by solving the GAN inversion problem
multiple times from different random initialization, in the hope of converging
to a different local minimum. For our proposed method, we generate the initial
solution by means of the state-of-the-art GAN inversion technique proposed by
Abu Hussein et al. [11], where the inversion problem in Eq. (2) also optimizes
with respect to the GAN parameters to finetune them. Once the initial solution
is computed, we apply Algorithm 1 to find a new solution to the problem.

Fig. 2 shows a few results on the SR problem. The middle row shows what
reconstructions would be obtained if direction ${\mathbf{v}}^{K}$ were used
without our proposed algorithm. It can be noticed that there is significant
perceptual change but the $\ell_{2}$ norm with respect to the measurements is
poorly constrained, so that these reconstructions can be hardly called
feasible solutions. The last row shows the images generated by the direction
found by our method. We successfully constrain the $\ell_{2}$ norm with
respect to the measurements below the $10^{-2}$ threshold we consider
acceptable for feasibility. At the same time, perceptual variations are still
present in those regions where the highly downsampled nature of the
measurements leaves more freedom to fill in information, such as the color and
shape of the dog’s coat (from pale yellow to white, and the shape of the
ears). Finally, the top row shows some good solutions found by PULSE. Those
solutions are feasible according to our $\ell_{2}$ criterion but are less
perceptually convincing. Indeed while some solutions show different dogs,
these present some artifacts around the dog’s mouth or some blurring over the
whole dog’s coat. Instead, the two central solutions in Fig 2 (top row), show
unnatural dogs found by the PULSE algorithm that still satisfy the $\ell_{2}$
criterion.

Fig. 2: Top row: solutions found by PULSE (LR $\ell_{2}$ range: $[1.8\times
10^{-3},3\times 10^{-3}]$). Mid row: solutions found by using
${\mathbf{v}}^{8}$ and ${\mathbf{v}}^{1}2$ as directions (LR $\ell_{2}$ range:
$[2.4\times 10^{-2},4.5\times 10^{-2}]$). Bottom row: solutions found by
optimized ${\mathbf{d}}$ as direction (LR $\ell_{2}$ range: $[2.9\times
10^{-3},4.8\times 10^{-3}]$).

Fig. 3: Top row: solutions found by PULSE ($\ell_{2}$ range: $[1.3\times
10^{-4},1.5\times 10^{-4}]$). Mid row: solutions found by using
${\mathbf{v}}^{8}$ and ${\mathbf{v}}^{26}$ as directions ($\ell_{2}$ range:
$[3.9\times 10^{-3},4.7\times 10^{-3}]$). Bottom row: solutions found by
optimized ${\mathbf{d}}$ as direction ($\ell_{2}$ range: $[1.2\times
10^{-3},2\times 10^{-3}]$).

Fig. 3 shows the results for the IP problem. Even in this case, PULSE (top
row) found solutions with different perceptual changes, but still some of them
seem to introduce unnatural variations, like the one on the nose or some
distortion on the mouth. The second row shows images along two starting
eigenvectors of the Hessians without applying our method. Although notable
perceptual changes are visible, the $\ell_{2}$ distance on the measure is
outside of our constraint and indeed some differences on the eyes are created
with respect to the observations. Finally, the last row, exploits our
optimized direction showing how the $\ell_{2}$ distance on the measurements is
decreased while retaining good semantic changes in the masked area, such
variations in the lip thickness or the amount of beard.

We remark that PULSE may be able to find good solutions but it has two main
drawbacks that are solved by the proposed technique. First, it lacks any
explicit control on the measurements distance. Constraining the $\ell_{2}$
distance between the original measurements and the measurements of the
reconstruction to a feasibility threshold can only be done by enforcing a
stopping criterion on the inversion optimization problem. However, due to the
non-convex nature of the problem this often results in degenerate solutions
that no longer belong to the manifold of realistic images like some of the
ones previously shown.

Another advantage of the proposed method with respect to PULSE is the
computational complexity due to PULSE requiring to solve a full optimization
problem to generate a new solution. For our proposed method, this needs to be
only done once, coupled with the estimation of the Hessians, but then multiple
solutions can be generated almost instantaneously. Table 1 reports the time
required for the two methods to generate ten solutions. This time does not
account for bad solutions: indeed, discarding bad minima found by PULSE would
further increase its computational requirements.

Table 1: Computational time required to find 10 solutions, with our method and using different initializations (PULSE). | Inverse Problem | Model | Time (s) to 10 solutions
---|---|---|---
PULSE | SR | BigGAN | $3.5\times 10^{4}$
e-GLASS | SR | BigGAN | $\boldsymbol{3.7\times 10^{3}}$
PULSE | IP | PGGAN | $7.9\times 10^{3}$
e-GLASS | IP | PGGAN | $\boldsymbol{1.3\times 10^{3}}$

##  5 Conclusions

We proposed a novel framework to explore the solution space of linear inverse
problems. By exploiting the power of the geometry of the latent space around
an initial solution, we showed it is possible to search for novel solutions
that have semantic differences between each other. We confirmed the quality
and variety of the multiple solutions as well as the reduced complexity when
compared with state-of-the-art iterative optimization methods on the SR and IP
problems.

## References

  * [1] Mario Bertero, Patrizia Boccacci, and Christine De Mol,  Introduction to inverse problems in imaging,  CRC press, 2021.
  * [2] Julien Mairal, Francis Bach, Jean Ponce, and Guillermo Sapiro,  “Online dictionary learning for sparse coding,”  in Proceedings of the 26th annual international conference on machine learning, 2009, pp. 689–696.
  * [3] Alice Lucas, Michael Iliadis, Rafael Molina, and Aggelos K Katsaggelos,  “Using deep neural networks for inverse problems in imaging: beyond analytical methods,”  IEEE Signal Processing Magazine, vol. 35, no. 1, pp. 20–36, 2018\.
  * [4] Vishal Monga, Yuelong Li, and Yonina C Eldar,  “Algorithm unrolling: Interpretable, efficient deep learning for signal and image processing,”  IEEE Signal Processing Magazine, vol. 38, no. 2, pp. 18–44, 2021\.
  * [5] Sachit Menon, Alexandru Damian, Shijia Hu, Nikhil Ravi, and Cynthia Rudin,  “Pulse: Self-supervised photo upsampling via latent space exploration of generative models,”  in Proceedings of the ieee/cvf conference on computer vision and pattern recognition, 2020, pp. 2437–2445.
  * [6] Antonia Creswell and Anil Anthony Bharath,  “Inverting the generator of a generative adversarial network,”  IEEE transactions on neural networks and learning systems, vol. 30, no. 7, pp. 1967–1974, 2018.
  * [7] Weihao Xia, Yulun Zhang, Yujiu Yang, Jing-Hao Xue, Bolei Zhou, and Ming-Hsuan Yang,  “Gan inversion: A survey,”  arXiv preprint arXiv:2101.05278, 2021.
  * [8] Andreas Lugmayr, Martin Danelljan, and Radu Timofte,  “Ntire 2021 learning the super-resolution space challenge,”  in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021, pp. 596–612.
  * [9] Yujun Shen, Ceyuan Yang, Xiaoou Tang, and Bolei Zhou,  “Interfacegan: Interpreting the disentangled face representation learned by gans,”  IEEE Transactions on Pattern Analysis and Machine Intelligence, pp. 1–1, 2020.
  * [10] Xingang Pan, Xiaohang Zhan, Bo Dai, Dahua Lin, Chen Change Loy, and Ping Luo,  “Exploiting deep generative prior for versatile image restoration and manipulation,”  IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021\.
  * [11] Shady Abu Hussein, Tom Tirer, and Raja Giryes,  “Image-adaptive gan based reconstruction,”  in Proceedings of the AAAI Conference on Artificial Intelligence, 2020, vol. 34, pp. 3121–3129.
  * [12] Yujun Shen, Jinjin Gu, Xiaoou Tang, and Bolei Zhou,  “Interpreting the latent space of gans for semantic face editing,”  in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 9243–9252.
  * [13] Binxu Wang and Carlos R Ponce,  “A geometric analysis of deep generative image models and its applications,”  in International Conference on Learning Representations, 2021.
  * [14] Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang,  “The unreasonable effectiveness of deep features as a perceptual metric,”  in Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 586–595.
  * [15] Andrew Brock, Jeff Donahue, and Karen Simonyan,  “Large scale GAN training for high fidelity natural image synthesis,”  in International Conference on Learning Representations, 2019.
  * [16] Tero Karras, Timo Aila, Samuli Laine, and Jaakko Lehtinen,  “Progressive growing of gans for improved quality, stability, and variation,”  arXiv preprint arXiv:1710.10196, 2017.


11institutetext: Department of Electrical Engineering, Jadavpur University,
Kolkata, India 22institutetext: Theism Medical Diagnostics Centre, Kolkata,
India 33institutetext: Department of Computer Science & Engineering, Jadavpur
University, Kolkata, India

## Cervical Cytology Classification Using PCA & GWO Enhanced Deep Features
Selection

Hritam Basak  11 Rohit Kundu  11 Sukanta Chakraborty 22 Nibaran Das  33

Hritam Basak
Department of Electrical Engineering, Jadavpur University

188, Raja S.C. Mullick Road,

Jadavpur, Kolkata-700032, West Bengal, INDIA

Email<EMAIL_ADDRESS>
Rohit Kundu
Department of Electrical Engineering, Jadavpur University

188, Raja S.C. Mullick Road,

Jadavpur, Kolkata-700032, West Bengal, INDIA

Email<EMAIL_ADDRESS>
Sukanta Chakraborty
Theism Medical Diagnostics Centre

Dum Dum, Kolkata-700030, West Bengal, INDIA

Email<EMAIL_ADDRESS>
Nibaran Das*
Department of Computer Science & Engineering, Jadavpur University

188, Raja S.C. Mullick Road,

Jadavpur, Kolkata-700032, West Bengal, INDIA

Email<EMAIL_ADDRESS>
*Corresponding author: Nibaran Das

*Corresponding author email<EMAIL_ADDRESS>
*Corresponding author contact number: +913324572407

## Cervical Cytology Classification Using PCA & GWO Enhanced Deep Features
Selection

Hritam Basak  11 Rohit Kundu  11 Sukanta Chakraborty 22 Nibaran Das  33

###### Abstract

Cervical cancer is one of the most deadly and common diseases among women
worldwide. It is completely curable if diagnosed in an early stage, but the
tedious and costly detection procedure makes it unviable to conduct
population-wise screening. Thus, to augment the effort of the clinicians, in
this paper, we propose a fully automated framework that utilizes Deep Learning
and feature selection using evolutionary optimization for cytology image
classification. The proposed framework extracts Deep feature from several
Convolution Neural Network models and uses a two-step feature reduction
approach to to ensure reduction in computation cost and faster convergence.
The features extracted from the CNN models form a large feature space whose
dimensionality is reduced using Principal Component Analysis while preserving
99% of the variance. A non-redundant, optimal feature subset is selected from
this feature space using an evolutionary optimization algorithm, the Grey Wolf
Optimizer, thus improving the classification performance. Finally, the
selected feature subset is used to train an SVM classifier for generating the
final predictions. The proposed framework is evaluated on three publicly
available benchmark datasets: Mendeley Liquid Based Cytology (4-class)
dataset, Herlev Pap Smear (7-class) dataset, and the SIPaKMeD Pap Smear
(5-class) dataset achieving classification accuracies of 99.47%, 98.32% and
97.87% respectively, thus justifying the reliability of the approach. The
relevant codes for the proposed approach can be found in: GitHub

###### Keywords:

Deep Learning Cervical Cytology Evolutionary Optimization Principal Component
Analysis Grey Wolf Optimization

##  1 Introduction

Figure 1: Overall workflow of the proposed framework

Cancer is the second leading cause of death worldwide, causing around 9.6
million deaths every year and about 1/6th of the total deaths of the
population throughout the globe. Studies also suggest that the economic impact
of cancer is significant as most of these deaths have been reported from poor
and middle-income countries where the living index is low and healthcare
infrastructure is comparatively insufficient for diagnosis and potential
treatments. Among them, cervical cancer is the fourth most common cancer
worldwide, having around 570 thousands of reported cases every year and the
second most common cancer in women causing around 311 thousands of deaths per
year [17].

Pap smear test is currently the most reliable and renowned screening tool for
the detection of pre-cancerous cells and cervical lesions in women based on
microscopic studies of the cells. However, the process is time-consuming since
pathologists need a lot of time to classify each cell from a slide of over
10,000 cells. Thus, an automated detection tool needs to be developed to
classify pre-cancerous lesions for early detection and widespread screening.

With the advent of artificial intelligence and deep learning in the domain of
medical sciences and healthcare [4, 8, 6], it is more becoming to lie on the
results predicted by this decision-support system [7, 12] to undermine the
observer-bias issues. In this paper, we seek to develop an alternative
approach that utilizes the Deep learning-based feature extraction and
optimization algorithm that gives excellent multi-class classification
accuracy, performing robustly and outperforming several existing methods like
[18, 40, 11, 9, 31, 32, 15].

In this research, we extracted deep features [3, 5] from pre-trained CNN
models and concatenated the features to form a large feature space. This is
followed by the application of the Principal Component Analysis (PCA) method
for dimensionality reduction of the feature space, while keeping most of the
important features intact, for which we retained 99% of the variance of the
feature space. Some of the features extracted from the CNN classifiers might
be non-informative or might even lead to a higher misclassification rate. To
eliminate such redundant, misleading features, generally, evolutionary
optimization algorithms are preferred by researchers, but applying such
algorithms directly to CNN-extracted features leads to computational wastage
due to the very high dimensionality of the feature space. PCA reduces the
dimensionality of the feature space or number of data points by combining
highly correlated variables to form a smaller number of new variables, known
as ”principal components”. From PCA, we also get higher variations in the
data, even in a lower dimension. This new feature space of dimensionality
lower than the original space, when used as an input to a metaheuristic
optimization algorithm, significantly reduces the computation cost since a
lower population size is required for faster convergence. The number of
iterations required for reaching the global optima is also greatly reduced
(i.e., faster evolution).

Specifically, among different evolutionary optimization algorithms available,
we used the Grey Wolf Optimizer (GWO) [30] for optimal feature set selection.
The GWO is embedded with a Support Vector Machines (SVM) classifier [47] with
Radial Basis Function (RBF) kernel for fitness assignment and final
classification. This method significantly decreased the training time for the
classification task, while maintaining the competitive accuracy of
predictions. The overall workflow of the proposed framework is shown in Figure
1.

The contributions of this paper can be summarized as follows:

  1. 1.

In the current paper, we propose a framework for the optimal selection of deep
features extracted from Convolutional Neural Network (CNN) classifiers.

  2. 2.

The dimensionality of the feature set extracted from the CNNs is large and
thus Principal Component Analysis (PCA) is used to reduce the dimensionality
while retaining the highly discriminating features. The resulting feature
subset, when used as the input of GWO, reduces the computation and ensures
faster convergence.

  3. 3.

Optimal features are selected through the use of a nature-inspired
evolutionary optimization algorithm, the Grey Wolf Optimizer (GWO), for the
first time in the cervical cytology domain, which filters out only the non-
redundant features for making the final predictions.

  4. 4.

The proposed framework has been evaluated on three publicly available
datasets: the Herlev Pap Smear dataset [24], the SIPaKMeD Pap Smear dataset
[34] and the Mendeley Liquid Based Cytology dataset [23], achieving
classification accuracies of 98.32%, 97.87% and 99.47% respectively. The
proposed method outperformed traditional CNN based approaches and is
comparable to state-of-the-art methods.

The rest of the paper is organized as follows: Section 2 surveys some of the
recent developments in the automated detection of cervical cancer; Section 3
describes the proposed methodology in detail; Section 4 evaluates the
performance of the proposed framework on three publicly available benchmark
datasets and Section 5 concludes the paper.

##  2 Related Work

Previous studies show that feature extraction and selecting non-redundant
features is an important part of the classification process and it affects the
classification result significantly. Different methods have been explored over
the years, like traditionally handcrafted feature extraction and feature
selection [2], Simulated Annealing (SA)[37], Convolutional Neural Networks
[41], Fuzzy-C means [38], to name a few. Some have given very good results in
binary classification but not so much in multi-class classification, only a
few have successfully given good results even for multi-class classification
problem [9, 11, 27]. One of the major concerns is the availability of open-
access datasets and the volume of images available in them for each class,
which is a major setback for a lot of proposed methods in the literature.

Chankong et al. [11] used different classifiers for binary and multi-class
classification with the best accuracy obtained using ANN. William et al. [39]
used an enhanced Fuzzy C-means by extracting features from the cell images in
Herlev Dataset and got an accuracy of 98.88%. Byriel et al. [10] used the
ANFIS neuro-fuzzy classification technique which achieved an accuracy of 96%
on the 2-class problem but performed much worse on the 7-class problem. Zhang
et al. [45] used MLBC slides stained in H&E in two ways: once posing it as a
2-class dataset and again using it as a 7-class dataset. They used the Fuzzy
Clustering-means approach and got an accuracy of 96-97% in the 2-class problem
and 72-77% on the 7-class problem. Bora et al. [9] used Ensemble classifier on
the Herlev Dataset and got an accuracy of 96.5%. Marinakis et al. [26] in a
way similar to [45, 42] of using the Herlev Dataset as both a 2-class and a
7-class problem, used genetic algorithm combined with nearest neighbour
classifier and got the best result with 1 nearest neighbour classifier giving
an accuracy of 98.14% on the 2-class problem and 96.95% on the 7-class problem
both in 10-fold cross-validation. Zhang et al. [46] used a deep convolutional
neural network architecture thus removing the need for cell segmentation like
[22, 12] on the Herlev Dataset and achieved an accuracy of 98.3%.

However, the number of publicly available datasets related to the smear images
of cervical cytology is quite less and each of them contains only nearly a
thousand images or less. So it becomes quite difficult to design a classical
deep learning or machine learning model with that few images to classify
between these images with an improved accuracy than the preexisting methods.
However, transfer learning can be used for this purpose that can quite
significantly tackle this issue where we use a pre-trained model (for example
ResNet-50 trained with ImageNet [14]), fine-tune the model, and use that for
the classification purpose. Akter et al. [1] performed experimentations with
different machine learning classifiers performed detailed comparative analysis
on their performance. Data augmentation can be another solution where we can
virtually increase the dataset size by slight movement or rotation or some
other changes of the images. However, these methods cannot improve the results
significantly as they cannot add more features or information to the algorithm
to learn from. Therefore, as suggested by [33], we tried a new optimal feature
selection approach for improving the classification accuracy and robustness of
the task.

##  3 Materials and Method

The experiment consisted of the following steps: (1) Data acquisition
(collecting image datasets of Pap smear test results from different sources),
(2) Data Preprocessing (structuring the data incorrect formats and verifying
the datasets), (3) Feature extraction (Extracting the important features from
the datasets using different CNN models), (4a) Combining the features from
different CNNs (to increase the effectiveness of the features) (4b) Feature
reduction using Principal Component Analysis (PCA) method (to discard the
redundant features and to improve the classification time), (5) Fitting these
features to the classifier and (6) Analysis of the results. The whole task was
performed using a machine having NVIDIA Tesla K80 GPU with 12 GB of available
RAM size.

###  3.1 Datasets Used

(a) Herlev Dataset

(b) Mendeley Dataset

(c) SIPaKMeD Dataset

Figure 2: Examples of images from each class in the three publicly available
datasets

We use three publicly available cervical cytology datasets in this study for
evaluating the proposed classification framework:

  1. 1.

Herlev Pap Smear dataset by Jantzen et al. [24]

  2. 2.

Mendeley Liquid Based Cytology dataset by Hussain et al. [23]

  3. 3.

SIPaKMeD Pap Smear dataset by Plissiti et al. [34]

These datasets are described in brief in the following subsections.

Table 1: Distribution of images in the three publicly available datasets

Dataset | Class | Category | Cell type | Number of images
---|---|---|---|---
Herlev Pap Smear (Total: 917) | 1 | Normal | Intermediate Squamous Epithelial | 70
2 | Normal | Columnar Epithelial | 98
3 | Normal | Superficial Squamous Epithelial | 74
4 | Abnormal |  | Mild Squamous
---
non-keratinizing Dysplasia
182
5 | Abnormal |  | Squamous cell carcinoma
---
in-situ intermediate
150
6 | Abnormal |  | Moderate Squamous
---
non-keratinizing Dysplasia
146
7 | Abnormal |  | Severe Squamous
---
non-keratinizing Dysplasia
197
Mendeley LBC (Total: 963) | 1 | Normal | Negative for Intraepithelial Malignancy | 613
2 | Abnormal |  | Low grade Squamous
---
Intraepithelial Lesion (LSIL)
163
3 | Abnormal |  | High grade Squamous
---
Intraepithelial Lesion (HSIL)
113
4 | Abnormal | Squamous Cell Carcinoma (SCC) | 74
SIPaKMeD Pap Smear (Total: 4049) | 1 | Normal | Superficial-Intermediate | 831
2 | Normal | Parabasal | 787
3 | Abnormal | Koilocytotic | 825
4 | Abnormal | Dyskeratotic | 813
5 | Benign | Metaplastic | 793

####  3.1.1 Herlev Pap Smear Dataset

The Herlev Pap Smear dataset is a publicly available benchmark dataset
consisting of 917 single cell images distributed unevenly among 7 different
classes. The distribution of images in each class are tabulated in Table 1.

####  3.1.2 Mendeley Liquid Based Cytology Dataset

The Mendeley LBC dataset [23] developed at Obstetrics and Gynecology
department of Guwahati Medical College and Hospital, consists of 963 whole
slide images of cervical cytology distributed unevenly in four different
classes as shown in Table 1.

####  3.1.3 SIPaKMeD Pap Smear dataset

The SIPaKMeD Pap Smear dataset by Plissiti et al. [34] consists of 4049 images
of isolated cells (extracted from 966 whole slide images) categorized into
five different classes based on their cytomorphological features. The
distribution of images in the dataset is shown in Table 1.

###  3.2 Deep Features Extraction

Handcrafted or manual feature extraction using traditional Machine Learning
techniques has limitations both in terms of the number of features and their
correlations. Extracting features from a large dataset is a tedious task and
can incorporate human biases, affecting the quality of the features that can
eventually affect the classification task. Redundant features might be
extracted which might lead to higher rates of misclassification. So, in this
work, we extract deep features from CNN classifiers. Deep Learning models use
backpropagation to learn the important features themselves, and thus
eliminates the tedious process of using handcrafted features. For the present
study, we have used ResNet-50 [19], VGG-16 [35], Inception v3 [36] and
DenseNet-121 [21] for extraction of features from the penultimate layer of the
models.

While performing feature extraction from a CNN, we use the pre-trained model
and fine-tune the CNN using our data, letting each image propagate through the
layers in a forwarding direction, terminating at the pre-final layer, and
taking out the output of this layer as the feature vector. We use pre-trained
weights (Transfer Learning) in this study because the biomedical data is
scarce and insufficient for Deep Learning models to work efficiently if
trained from scratch. ImageNet [14] dataset consists of 14 billion images
divided into 1000 classes. We use the models pretrained on this dataset and
replace the final classification layer of size 1000 with a layer of size
equals to the number of classes in our dataset. A model pretrained on such a
large dataset already has learned important features from image data, and just
needs fine-tuning for less number of epochs to train the final classification
layer that we added.

####  3.2.1 VGG-16

The main characteristics of VGG nets [35] includes the use of $3\times 3$
convolution layers which gave a noticeable improvement in network performances
while making the network deep. $3\times 3$ receptive filters were used
throughout the entire net with strides of $1$. Local Response Normalization
(LRN) is not used in VGG Nets because memory consumption is more in such
cases. The small-sized convolution filters give VGG Nets a chance to have a
very large number of weight layers which in turn boosts performance. The input
has a shape of $224\times 224\times 3$. In the present work, we fine-tune the
VGG-16 model using our datasets, employing Stochastic Gradient Descent (SGD)
optimizer and Rectified Linear Unit (ReLU) activation function.

####  3.2.2 ResNet-50

The ResNet-50 architecture [19] consists of residual skip connections embedded
that make the training of the network easier. The gradient vanishing problem
is addressed at the same time due to the embedding of the skip connections,
which allows very deep networks to be accommodated for a controlled
computation cost. $224\times 224\times 3$ sized inputs are used in the
ResNet-50 model, with SGD optimizer and ReLU activation function.

####  3.2.3 Inception v3

The salient feature of the Inception v3 architecture [36], is the inception
blocks that use parallel convolutions followed by channel concatenation. This
leads to vivid features being extracted but with seemingly shallow networks.
Parallel convolutions also allow the overfitting problem to be addressed while
controlling the computational complexity. Inputs of shape $299\times 299\times
3$ are used with SGD optimizer and ReLU activation function for deep features
extraction.

####  3.2.4 DenseNet-121

The DenseNet model [21] was proposed to address the vanishing gradient descent
problem. The fundamental blocks in the DenseNet architecture are connected
densely to each other leading to low computational requirement since the
number of trainable parameters decreased heavily. The DenseNet architectures
add small sets of feature maps owing to their narrow architecture. We used the
DenseNet-121 variant using the SGD optimizer and ReLU activation function for
deep features extraction.

###  3.3 Principle Component Analysis

Principal Component Analysis (PCA) is a linear dimensionality reduction method
that transforms the higher dimensional data into a lower dimension by
maximizing the variance of the lower dimension. PCA was first introduced by
Wold et al. in 1987 [44], however, further development and implementation of
PCA in machine learning problems was done quite significantly in the later
period [45], [46]. The covariance matrix of the feature vector is computed
first and followed by the computation of eigenvectors of this matrix. The
eigenvectors that have the largest eigenvalues contribute to the formation of
new reduced dimensionality of the feature vector. Thus, instead of losing some
of the important features of the data, we kept the most important of the
features by preserving 99% of the variance. Before applying the PCA algorithm
for feature dimension reduction, we need to perform data preprocessing that is
required for the further steps. Depending upon the $n$-dimensional training
set $x^{(1)},x^{(2)},x^{(3)},\ldots x^{(n)}$, we need to perform mean
normalization or feature scaling similar to the supervised learning
algorithms. The mean of each feature is computed as in Equation 1

$\mu_{i}=\frac{1}{n}\sum_{j=1}^{n}x_{i}^{(j)}$ (1)

Now, we replace each of the $x_{i}$ value with the $x_{i}-\mu_{i}$ value so
that each of them has exactly zero mean value, however, if different features
have different mean values, we can scale them so that they belong in a
comparable range. In supervised learning, this scaling process of the $i^{th}$
element is defined by Equation 2, where, $s_{i}$ is the $|max-mean|$ value or
the static deviation of $i^{th}$ feature.

$x_{i}^{(j)}=\frac{x_{i}^{(j)}-\mu_{i}}{s_{i}}$ (2)

For reducing the dimension of the feature from $N$ to $m$ (where $m<N$), and
to define the surface in $N$-dimensional space onto which we project the data,
we need to find the mean square error of the projected data on the $m$
dimensional vector. The computational proof of the calculation of these m
vectors: $u^{(1)},u^{(2)},\ldots,u^{(m)}$ and the projected points:
$zu^{(1)},z^{(2)},\ldots,z^{(N)}$ on these vectors is complicated and beyond
the scope of this paper. The covariance matrix is computed as in Equation 3
where, the $x^{(j)}$ vector has $N\times 1$ dimension and $(x^{(j)})^{T}$ has
$1\times N$ dimension, thus making the covariance matrix of dimension of
$N\times N$. Next, we calculate the eigenvalues and eigenvectors of the
covariance matrix which represent the new magnitude of the feature vectors in
the transformed vector space and their corresponding directions respectively.
The eigenvalues quantify the variance of all the vectors as we are dealing
with the covariance matrix. If an eigenvector has high valued eigenvectors,
that means that it has high variance and contains various important
information about the dataset. On the other hand, eigenvectors will small
eigenvalues contain very small information about the dataset.

$covariance\;matrix=\frac{1}{N}\sum_{j=1}^{N}x^{(j)}\times(x^{(j)})^{T}$ (3)

Hence the $p^{th}$ complete principal component of a data vector $x^{(j)}$ in
the transformed coordinates can be allocated a score $t^{(p)}=x^{(j)}\times
w^{(p)}$ where $w^{(p)}$ is the $p^{th}$ eigenvector of the covariance matrix
of $x^{(j)}$. Therefore the full PCA decomposition of the vector $X$ can be
represented as $T=X\times W$, where $W$ is the eigenvector of the covariance
matrix. Now, we need to select $m$-number of eigenvalues from these $N$
eigenvectors by maximizing the variance of the preserved original data while
reducing the total square reconstruction error. Next, we calculate the
Cumulative Explained Variance (CEV) which is the sum of variances
(information) containing in the top $m$ principal components. Then we set a
threshold value above which, the eigenvalues will be considered as useful and
the rest will be discarded as unimportant features. For our experiment we have
set the threshold value to 99, meaning that we have kept 99% of the variance
of the data retained in the reduced feature vector. As different CNN extracts
features of different modalities, the number of selected features after PCA
and GWO are different based on the feature distribution in those feature sets.

The pseudo-code of dimensionality reduction using PCA is shown in Algorithm 1.

define_function: PCA

Input:

Feature set $X$ of dimension $d$

Compute Co-variance matrix $\Pi$

while (i$\leq$ d) do

while (j $\leq$ d) do

$\mu_{i}\longleftarrow$ sample mean of feature $i$

$\mu_{j}\longleftarrow$ sample mean of feature $j$

$\sigma_{ij}=\frac{1}{n}\sum\limits_{k=1}^{n}(x_{i}^{k}-\mu_{i})(x_{j}^{k}-\mu_{j})$

j=j+1

end while

$i=i+1$

end while

decompose $\pi$ into eigenvalues and eigenvectors

calculate cumulative explained variance (CEV)

if (CEV$\geq$ threshold) then

Construct projection matrix W

end if

transform input $X$ using $W$

obtain k-dimensional feature subspace $X^{\prime}$

return $X^{\prime}$

Algorithm 1 Pseudo-code for Principal Component Analysis

###  3.4 Grey Wolf Optimizer

Grey Wolf Optimization (GWO) [30] is a nature-inspired meta-heuristic
optimization algorithm that mimics the leadership hierarchy of Grey Wolf
($Canis$ $lupus$) and their hunting process for the optimization. Four types
of GWO agents are deployed for simulation of the optimization algorithm named
alpha, beta, delta, and omega. They mimic the three-step hunting methods of
the grey wolf: finding the prey, encircling them, and finally attacking them
for the sake of optimization.

The grey wolves follow the leadership of the alpha wolf, which is the topmost
category of their strict social hierarchy. The alpha wolf is not necessarily
the strongest and the fittest ones, but they can maintain the discipline of
the whole pack. The major decisions are taken by the alpha wolves but often
accompanied by the subordinates, the beta wolf. They are the next lower level
of wolves in their social hierarchy and convey the decisions o the alpha to
the lower levels of wolves. They are generally the fittest candidates for
alpha if the alpha becomes old or weak and plays a major role in maintaining
the pack as a whole. The next level of wolves is called delta and they play a
very major role in decision-making and other important activities of the pack.
The last and the least important category of the pack is named omega and they
often play the role of scapegoat in society. Thus the complete pack is formed
based on dominance hierarchy. The mathematical model for the steps of
optimization that mimics their hunting process is described below.

####  3.4.1 Social Hierarchy

Similar to the social hierarchy of grey wolf, the optimizer allocates the
three fittest solutions as alpha, beta, and delta and the rest of the search
agents are bound to arrange them and adjust accordingly as the parameters of
the alpha, beta, and delta wolves. These three wolves are followed by the
omega wolves.

####  3.4.2 Encircling the Prey (Optimal Solution)

To mathematically represent the encircling of prey, Equations 4, and 5 are
used where $t$ is the present iteration and $\vec{C}$ and $\vec{A}$ are the
coefficient vectors, $\vec{x_{p}}$ indicates the vector position of the prey
and $\vec{x}$ indicates the vector position of the grey wolf.

$\vec{D}=|\vec{C}\times\vec{x_{p}}(t)-\vec{x}(t)|$ (4)
$\vec{x}(t+1)=\vec{x_{p}}(t)-\vec{A}\times\vec{D}$ (5)

The expression for $\vec{A}$ and $\vec{C}$ areas in Equations 6 and 7
respectively, where $\vec{r_{1}}$ and $\vec{r_{2}}$ are the random valued
vectors between $0$ and $1$ inclusive and the value of $\vec{a}$ decreases
from $2$ to $0$ linearly with increase in iteration. The two random variables
$\vec{r_{1}}$ and $\vec{r_{2}}$ allow a grey wolf agent to reach any position
between the points. The agents of the grey wolf algorithm can position
themselves around the fittest solution by adjustment of the value of $\vec{C}$
and $\vec{A}$.

$\vec{A}=2\times\vec{a}\times\vec{r_{1}}-\vec{a}$ (6)
$\vec{C}=2\times\vec{r_{2}}$ (7)

The same is applicable for $n$-dimensional optimization where the grey wolf
agents move along the hyper-sphere or hyper-cubes around the fittest solution
obtained.

####  3.4.3 Reaching the Optimal Solution

In real life, grey wolves hunt for their prey being led by the alpha who is
accompanied by beta and delta wolves and the other wolves follow their
instructions. To simulate this hunting principle, we allow some random valued
agents and find their fitness and consider the three most accurate results as
alpha, beta, and delta as in abstract search space we have no idea about the
position of the agents and the prey. The rest of the agents including the
omega wolves are bound to change their positions and orientations according to
the three best wolf agents.

$D_{\alpha}=|\vec{C_{1}}\times\vec{x_{\alpha}}-\vec{x}|$ (8)
$D_{\beta}=|\vec{C_{2}}\times\vec{x_{\beta}}-\vec{x}|$ (9)
$D_{\delta}=|\vec{C_{3}}\times\vec{x_{\delta}}-\vec{x}|$ (10)
$\vec{x_{1}}=\vec{x_{\alpha}}-\vec{A_{1}}\times\vec{D_{\alpha}}$ (11)
$\vec{x_{2}}=\vec{x_{\beta}}-\vec{A_{2}}\times\vec{D_{\beta}}$ (12)
$\vec{x_{3}}=\vec{x_{\delta}}-\vec{A_{3}}\times\vec{D_{\delta}}$ (13)
$\vec{x}(t+1)=\frac{\vec{x_{1}}+\vec{x_{2}}+\vec{x_{3}}}{3}$ (14)

The search agents update their position by these equations, however, the final
position of the agents are not predefined, rather they are the random
positions according to the position of the alpha, beta, and delta agents and
within a certain circle which is determined by the position of the three best-
fit solutions.

####  3.4.4 Exploiting the Prey

Grey wolves encircle their prey until the prey stops movement and this
freezing the prey is known as exploiting. In the mathematical model, the value
of $\vec{a}$ is decreased with the agents approaching the prey, and hence the
value of $\vec{A}$ is modified further. The fluctuation of $\vec{A}$ is
stopped as the value of $\vec{A}$ changes from $\vec{-2a}$ to $\vec{+2a}$. The
value of $\vec{a}$ changes from $2$ to $0$ with the increase in iterations.
The search agents can take any position between their current position and the
position of the prey as the alpha, beta, and delta wolves approach the prey
for hunting.

####  3.4.5 Exploring for the prey

The grey wolf agents diverge in search of prey and they finally converge for
attacking the prey. In mathematical modelling, this phenomenon is regulated by
the value of $\vec{A}$; if the value of $\vec{A}$ is greater than $1$ or less
than $-1$, the grey wolf agents diverge from each other and find for some more
suitable prey. However, if $\vec{A}$ has a value between $-1$ and $+1$, the
agents converge towards the prey.

Figure 3: Flowchart showing the workflow of the Grey Wolf Optimization
algorithm used in the proposed framework.

The overall pseudo-code of the GWO algorithm is shown in Algorithm 2 and the
flowchart for the algorithm is shown in Figure 3.

define_function: GWO

Input:

Number of Search Agents: $n$

Maximum number of iterations: $MAX\\_ITER$

Initialize the GWO population $X_{i}\>\>\forall i=1,2,3,....,n$

Initialize a, A, and C // According to Equations 6,7

Calculate the fitness of each search agent

$X_{\alpha}$ = Alpha wolf (Best search agent)

$X_{\beta}$ = Beta wolf (Second best search agent)

$X_{\delta}$ = Delta wolf (Third best search agent)

while t $<$ $MAX\\_ITER$ do

for each search agent do

Update position of current search agent // According to Equation 5

end for

Update a, A, and C

Calculate the fitness of each search agent

Update $X_{\alpha},X_{\beta},X_{\delta}$ // According to Equations 11, 12 & 13

$t=t+1$

end while

return $X_{\alpha}$

Algorithm 2 Pseudo-code for the Grey Wolf Optimizer for feature selection.

###  3.5 Classification

After optimization, the final step is to fit the selected features to the
classifier for the classification task. Due to a large number of features in
some cases, we used incremental learning where a small batch of the dataset is
selected for training the classifier and the loop over all the dataset and
continue training until we reach convergence. This is fast and computationally
efficient. We used an SVM classifier with the ‘RBF’ kernel for the multi-class
classification task.

Input:

Raw RGB images: $I_{RGB}$

$f_{i}$=deep features extracted from $i^{th}$ CNN; $i={1,2,3,4}$

$f_{c}=concat(f_{1},f_{2},\ldots,f_{i})$;$i={1,2,3,4}$ // concatenation of
features

$F_{PCA}=PCA(f_{c})$ // Using algorithm 1

$F_{GWO}=GWO(F_{PCA})$ // Using Algorithm 2

train-test split$\to train_{F_{GWO}},test_{F_{GWO}}$

$clf=classifier.fit(train_{F_{GWO}})$ // Train SVM Classifier

$predicted=clf.predict(test_{F_{GWO}})$ // Make predictions on test set

Compare predictions and labels and evaluate performance

Algorithm 3 Pseudo-code for overall workflow.

####  3.5.1 Support Vector Machine

SVM [47] is a supervised learning model, which, in a set of training examples,
properly labelled with different classes, add new examples to each class
making a complete non-probabilistic binary classifier out of this SVM, and is
associated with some typical learning algorithms which analyse the data,
specifically used for regression and classification tasks. SVM model
representation of the training samples in the feature plane is such that a
separation between the examples belonging to different classes becomes so
prominent, that a curve can be fit in that space between two classes which
maintain maximum distances from every point of each class and SVM fits that
curve.

##  4 Results and Discussion

Table 2: Reduction in feature dimension and improvements in training time
after principal component analysis on the Herlev dataset

| Model used for
---
feature extraction
| No. of features
---
(before PCA)
| No. of features
---
(after PCA)
| Reduction in
---
feature dimension (%)
| Improvement in
---
average training time (%)
ResNet-50 [19] | 100353 | 383 | 99.62 | 88.425
VGG-16 [35] | 25088 | 364 | 98.55 | 85.215
DenseNet-121 [21] | 50177 | 330 | 99.34 | 81.449
Inception v3 [36] | 131073 | 325 | 99.75 | 84.228
ResNet-50+VGG-16 | 125441 | 456 | 99.63 | 80.221
DenseNet-121+Inception v3 | 181250 | 687 | 99.62 | 82.694
| ResNet-50+VGG-16+
---
DenseNet-121+Inception v3
306691 | 796 | 99.74 | 85.737

(a) Herlev Pap Smear dataset

(b) Mendeley LBC dataset

(c) SIPaKMeD Pap Smear dataset

Figure 4: ROC curves obtained by the proposed method for the three datasets:
(a) Herlev Pap Smear dataset (b) Mendeley LBC dataset and (c) SIPaKMeD Pap
Smear dataset.

After extracting the features from the dataset using the CNN architectures
said in Section 3, and the features were concatenated. We then use PCA (which
retained 99% of the variance of the data) for the reduction in the
dimensionality of the feature space and improvements in feature qualities
respectively. Table 2 shows the statistics of reduction in feature
dimensionality as well as the improvement of training time after this for the
Herlev dataset. Then, we used the GWO algorithm and finally split the dataset
and calculated the accuracy score for the training, validation, and testing
sets. The overall workflow is shown in the form of pseudo-code in Algorithm 3.
The results of our experiments are discussed in this section.

The metrics used for performance evaluation of the classification task for the
multi-class problem is calculated based on Equations 15, 16, 17, 18 which are
derived from a confusion matrix $M$.

$Accuracy=\frac{\sum_{i=1}^{N}M\textsubscript{ii}}{\sum_{i=1}^{N}\sum_{j=1}^{N}M\textsubscript{ij}}$
(15)
$Precision\textsubscript{i}=\frac{M\textsubscript{ii}}{\sum_{j=1}^{N}M\textsubscript{ji}}$
(16)
$Recall\textsubscript{i}=\frac{M\textsubscript{ii}}{\sum_{j=1}^{N}M\textsubscript{ij}}$
(17)
$F1-Score\textsubscript{i}=\frac{2}{\frac{1}{Precision\textsubscript{i}}+\frac{1}{Recall\textsubscript{i}}}$
(18)

To cross-validate the results of the classification task on different datasets
and different features, we performed an AUC-ROC test on different datasets.
The ROC (Receiver Operating Characteristics) curve is an important analyzing
tool for validating the clinical findings of our experiment. The different
line segments in the OVA (One Vs. All) ROC represent different classes stating
that how good the features and the classifier performance are for classifying
the different classes which can be broadly categorized in normal and infected
cases. It represents the graphical analysis of the TPR (True Positive Rate)
against the FPR (False Positive Rate) as the two operating characteristics
criterion of the classifier based on the features selected. A false-positive
result is a case when data of a healthy or uninfected class is predicted as an
unhealthy or infected case by a classifier and it’s a major drawback of the
classification task. This is reciprocated by the points lying far above the
diagonal line of the ROC curve suggesting that the TPR is significantly high
as compared to FPR. Another important feature for analyzing the classification
result is the AUC (Area Under Curve) of the ROC curve which was computed
considering the 97% of the confidence interval. The analysis using the AUC-ROC
curves for different datasets and different features are discussed further.

###  4.1 Results on Herlev Pap Smear Dataset

Figure 5: Results on the Herlev Pap Smear dataset

The results obtained on different experiments on the Herlev Pap Smear dataset
are shown in Figure 5. The best classification results observed this dataset
was achieved by merging the feature extracted from ResNet-50 and VGG-16
models, which gave the performance metrics as follows: Accuracy = 98.32%,
Precision = 98.66%, Recall = 97.65% and F1-score = 98.12%.

###  4.2 Results on the Mendeley LBC Dataset

Figure 6: Results on the Mendeley LBC dataset

The results obtained on different experiments on the Mendeley LBC dataset are
shown in Figure 6. The best results on this dataset are obtained by merging
features extracted from VGG-16, ResNet-50, Inception v3 and DenseNet-121:
Accuracy = 99.47%, Precision = 99.14%, Recall = 99.27% and F1-score = 99.20%.

###  4.3 Results on SIPaKMeD Pap Smear dataset

Figure 7: Results on the SIPaKMeD Pap Smear Dataset

The results obtained on different experiments on the SIPaKMeD Pap Smear
dataset are shown in Figure 7. The best results on the dataset are obtained by
merging features extracted from VGG-16 and ResNet-50: Accuracy = 97.87%,
Precision = 98.56%, Recall = 99.12% and F1-score = 98.89%.

Table 3: Accuracies and Losses on training, validation and testing sets using
both PCA and GWO on the three datasets dataset (all the accuracy measurements
are in % and measured after 30 epochs)

Dataset | Feature extractor model |  | Training
---
Accuracy
| Training
---
Loss
| Validation
---
Accuracy
| Validation
---
Loss
| Testing
---
Accuracy
| Testing
---
Loss
Herlev Pap Smear | ResNet-50 [19] | 97.77 | 0.026 | 96.33 | 0.032 | 96.55 | 0.028
VGG-16 [35] | 96.36 | 0.031 | 94.21 | 0.109 | 95.56 | 0.081
DenseNet-121 [21] | 97.89 | 0.02 | 97.01 | 0.024 | 96.61 | 0.024
Inception v3 [36] | 96.33 | 0.032 | 95.17 | 0.098 | 95.32 | 0.094
ResNet-50+VGG-16 | 98.77 | 0.011 | 98.00 | 0.019 | 98.32 | 0.016
DenseNet-121+Inception v3 | 97.91 | 0.026 | 96.01 | 0.031 | 97.62 | 0.027
| ResNet-50+VGG-16+
---
DenseNet-121+Inception v3
98.3 | 0.018 | 97.95 | 0.021 | 98.06 | 0.019
Mendeley LBC | ResNet-50 [19] | 96.88 | 0.066 | 96.11 | 0.071 | 96 | 0.079
VGG-16 [35] | 97.91 | 0.051 | 96.39 | 0.068 | 97.56 | 0.059
DenseNet-121 [21] | 97.16 | 0.061 | 96.15 | 0.07 | 96.32 | 0.071
Inception v3 [36] | 97.5 | 0.058 | 97.05 | 0.06 | 97.2 | 0.061
ResNet-50+VGG-16 | 99.04 | 0.039 | 97.96 | 0.054 | 98.64 | 0.054
DenseNet-121+Inception v3 | 98.06 | 0.044 | 96.49 | 0.067 | 97.02 | 0.064
| ResNet-50+VGG-16+
---
DenseNet-121+Inception v3
99.58 | 0.03 | 98.88 | 0.043 | 99.47 | 0.04
SIPaKMeD Pap Smear | ResNet-50 [19] | 96.85 | 0.028 | 96.77 | 0.049 | 96.03 | 0.048
VGG-16 [35] | 96.71 | 0.03 | 94.02 | 0.071 | 95.26 | 0.059
DenseNet-121 [21] | 96.31 | 0.035 | 96.04 | 0.058 | 96.12 | 0.046
Inception v3 [36] | 96.02 | 0.039 | 95.91 | 0.06 | 95.78 | 0.055
ResNet-50+VGG-16 | 98.48 | 0.014 | 97.55 | 0.041 | 97.87 | 0.034
DenseNet-121+Inception v3 | 97.32 | 0.02 | 95.39 | 0.066 | 96.33 | 0.044
| ResNet-50+VGG-16+
---
DenseNet-121+Inception v3
96.92 | 0.025 | 96.66 | 0.051 | 96.46 | 0.042

###  4.4 Comparison with Existing Literature

Table 4: Comparison (ACC, in %) with standard optimization algorithms: PSO =
Particle Swarm Optimization [25]; MVO = Mean Variance Optimization [16]; GWO =
Grey Wolf Optimizer [30]; MFO = Moth Flame Optimization [28]; WOA = Whale
Optimization Algorithm [29]; FFA = Firefly Algorithm [43]; BAT = Bat
Optimization Algorithm [44]; GA = Genetic Algorithm [13, 20]

Optimization | Mendeley LBC Dataset | Herlev Pap-smear Dataset | SIPaKMeD 5-class Dataset
---|---|---|---
Alogotithms | ACC | # of Features | ACC | # of Features | ACC | # of Features
PSO | 95.90 | 920 | 92.58 | 992 | 90.14 | 1014
MVO | 96.91 | 720 | 94.26 | 764 | 90.48 | 843
GWO | 92.14 | 810 | 92.40 | 807 | 89.98 | 791
MFO | 94.20 | 803 | 93.19 | 851 | 90.58 | 832
WOA | 95.08 | 843 | 92.36 | 847 | 90.58 | 802
FFA | 94.42 | 715 | 92.46 | 820 | 89.56 | 792
BAT | 95.64 | 857 | 94.58 | 762 | 90.21 | 749
GA | 98.23 | 724 | 95.26 | 784 | 95.43 | 796
PCA+GWO | 99.47 | 762 | 98.32 | 796 | 97.87 | 736

Several models have been proposed in the literature for cervical cell
classification as discussed in Section 2. Our proposed work and the results
achieved are therefore compared with some of these models that used the same
datasets to assess the reliability of the proposed framework and the results
are tabulated in Table 5. No papers as of yet have been published that use the
Mendeley LBC dataset, and thus we are unable to compare our method in that
dataset.

Table 5: Comparison of the proposed method with existing literature

Dataset | Method | Results
---|---|---
Herlev Pap Smear | Genctav et al. [18] |  | Precision: 88%±0.15
---
Recall: 93%±0.15
Bora et al. [9] | Accuracy: 96.51%
Win et al. [40] | Accuracy: 90.84%
Chankong et al. [11] | Accuracy: 93.78%
Proposed Method |  | Accuracy: 98.32%
---
Precision: 98.66%
Recall: 97.65%
F1-score: 98.12%
SIPaKMeD Pap Smear | Win et al. [40] | Accuracy: 94.09%
Plissiti et al. [34] |  | 1\. Deep Convolutional+SVM: 93.35%±0.62
---
2\. Deep Fully Connected+SVM: 94.44%±1.21
3\. CNN: 95.35%±0.42
Proposed Method |  | Accuracy: 97.87%
---
Precision: 98.56%
Recall: 99.12%
F1-score: 98.89%

###  4.5 McNemar’s Statistical Test

The McNemar’s statistical test has been performed in the present work, for the
statistical analysis of the proposed classification framework. For this, the
proposed model has been compared to the CNN models from which the features
were extracted and used for the final classification. The results are shown in
Table 6. To reject the null hypothesis that the two models are similar, the
$p-value$ from the McNemar’s test should remain below 5% (i.e., 0.05), and
from the table, it can be seen that for every comparison case, the
$p-value<0.05$. Thus, the null hypothesis can be rejected and it can be
concluded that the proposed model is dissimilar to any of the feature
extractor models and performs superior to them. Thus statistical analysis of
the proposed model justifies the reliability of the approach devised in this
research.

Table 6: Results obtained from McNemar’s statistical test. For all three datasets, the proposed framework is compared to the CNN models whose features have been used. The $p-value$ is less than 0.05 for every case and thus, the null hypothesis is rejected. McNemar’s Test | p-value
---|---
Performed with | Herlev Pap Smear | Mendeley LBC | SIPaKMeD Pap Smear
ResNet-50 | 0.0046 | 0.0012 | 0.0005
VGG-16 | 0.0001 | 0.0211 | 0.0007
DenseNet-121 | 0.0103 | 0.0089 | 0.0315
Inception v3 | 0.0007 | 0.0061 | 0.0100

##  5 Conclusions and Future Work

The need for automation in the cervical cancer detection domain arises due to
the high mortality rate throughout the close. Motivated by this cause, we
developed a fully automated detection framework that optimizes deep features
for classification. The two-level enhancement boosted the classification
performance while simultaneously reducing the training time significantly.
This research also explored the hybridization of multiple CNN-based deep
features to extract more discriminating information from the dataset.

An alternative way of feature selection is exalted in this research that uses
Principal Component Analysis (PCA) and Grey Wolf Optimization (GWO). The two-
level feature reduction approach introduced in this paper leverages the
advantages of both methods resulting in optimal feature set selection. The
proposed method achieves better results juxtaposed to end-to-end
classification with CNN models, while simultaneously reducing the computation
cost. Very high classification accuracy of 99.47%, 98.32%, and 97.87% on the
three publicly available benchmark datasets, namely Mendeley LBC, Herlev Pap
Smear and SIPaKMeD Pap Smear datasets respectively tantamount to state-of-the-
art methods.

However, there is scope for further improvement by utilizing different
classification models and using hybrid metaheuristic feature selection
algorithms. This paper craved a path for further research in this field as
well as multi-domain adaptation. The proposed pipeline can be used as a test-
bed for several classification problems, not only in biomedical applications
but in other computer vision problems as well. The feature selection can be
further addressed by developing an end-to-end multi-objective hybrid
optimization algorithm, that selects optimal feature set, where the objective
function aims to increase the classification performance by selecting the
least number of features, thereby reducing the computational cost
simultaneously.

## Acknowledgements

The work is supported by SERB (DST), Govt. of India (Ref. no.
EEQ/2018/000963).

## Conflict of interest

The authors declare that they have no conflict of interest.

## References

  * Akter et al. [2021] Akter L, Islam MM, Al-Rakhami MS, Haque MR, et al. (2021) Prediction of cervical cancer from behavior risk using machine learning techniques. SN Computer Science 2(3):1–10
  * AlMubarak et al. [2019] AlMubarak HA, Stanley J, Guo P, Long R, Antani S, Thoma G, Zuna R, Frazier S, Stoecker W (2019) A hybrid deep learning and handcrafted feature approach for cervical cancer digital histology image classification. International Journal of Healthcare Information Systems and Informatics (IJHISI) 14(2):66–87
  * Azaza et al. [2021] Azaza A, Abdellaoui M, Douik A (2021) Off-the-shelf deep features for saliency detection. SN Computer Science 2(2):1–10
  * Basak and Kundu [2020] Basak H, Kundu R (2020) Comparative study of maturation profiles of neural cells in different species with the help of computer vision and deep learning. In: International Symposium on Signal Processing and Intelligent Recognition Systems, Springer, pp 352–366
  * Basak and Rana [2021] Basak H, Rana A (2021) F-unet: A modified u-net architecture for segmentation of stroke lesion. In: Computer Vision and Image Processing, Springer Singapore, pp 32–43
  * Basak et al. [2020a] Basak H, Ghosal S, Sarkar M, Das M, Chattopadhyay S (2020a) Monocular depth estimation using encoder-decoder architecture and transfer learning from single rgb image. In: 2020 IEEE 7th Uttar Pradesh Section International Conference on Electrical, Electronics and Computer Engineering (UPCON), IEEE, pp 1–6
  * Basak et al. [2020b] Basak H, Kundu R, Agarwal A, Giri S (2020b) Single image super-resolution using residual channel attention network. In: 2020 IEEE 15th International Conference on Industrial and Information Systems (ICIIS), IEEE, pp 219–224
  * Basak et al. [2021] Basak H, Hussain R, Rana A (2021) Dfenet: A novel dimension fusion edge guided network for brain mri segmentation. arXiv preprint arXiv:210507962
  * Bora et al. [2017] Bora K, Chowdhury M, Mahanta LB, Kundu MK, Das AK (2017) Automated classification of pap smear images to detect cervical dysplasia. Computer methods and programs in biomedicine 138:31–47
  * Byriel [1999] Byriel J (1999) Neuro-fuzzy classification of cells in cervical smears. Master’s Thesis, Technical University of Denmark: Oersted-DTU, Automation
  * Chankong et al. [2014] Chankong T, Theera-Umpon N, Auephanwiriyakul S (2014) Automatic cervical cell segmentation and classification in pap smears. Computer methods and programs in biomedicine 113(2):539–556
  * Chattopadhyay and Basak [2020] Chattopadhyay S, Basak H (2020) Multi-scale attention u-net (msaunet): A modified u-net architecture for scene segmentation. arXiv preprint arXiv:200906911
  * De Jong [1975] De Jong KA (1975) Analysis of the behavior of a class of genetic adaptive systems. Tech. rep.
  * Deng et al. [2009] Deng J, Dong W, Socher R, Li LJ, Li K, Fei-Fei L (2009) Imagenet: A large-scale hierarchical image database. In: 2009 IEEE conference on computer vision and pattern recognition, Ieee, pp 248–255
  * Dey et al. [2020] Dey S, Das S, Ghosh S, Mitra S, Chakrabarty S, Das N (2020) SynCGAN: Using learnable class specific priors to generate synthetic data for improving classifier performance on cytological images. In: Communications in Computer and Information Science, Springer Singapore, pp 32–42, URL https://doi.org/10.1007/978-981-15-8697-2_3
  * Erlich et al. [2010] Erlich I, Venayagamoorthy GK, Worawat N (2010) A mean-variance optimization algorithm. In: IEEE Congress on Evolutionary Computation, IEEE, pp 1–6
  * Ferlay et al. [2019] Ferlay J, Colombet M, Soerjomataram I, Mathers C, Parkin D, Piñeros M, Znaor A, Bray F (2019) Estimating the global cancer incidence and mortality in 2018: Globocan sources and methods. International journal of cancer 144(8):1941–1953
  * GençTav et al. [2012] GençTav A, Aksoy S, ÖNder S (2012) Unsupervised segmentation and classification of cervical cell images. Pattern recognition 45(12):4151–4168
  * He et al. [2016] He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778
  * Holland et al. [1992] Holland JH, et al. (1992) Adaptation in natural and artificial systems: an introductory analysis with applications to biology, control, and artificial intelligence. MIT press
  * Huang et al. [2017] Huang G, Liu Z, Van Der Maaten L, Weinberger KQ (2017) Densely connected convolutional networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4700–4708
  * Huang et al. [2020] Huang J, Wang T, Zheng D, He Y (2020) Nucleus segmentation of cervical cytology images based on multi-scale fuzzy clustering algorithm. Bioengineered 11(1):484–501
  * Hussain et al. [2020] Hussain E, Mahanta LB, Borah H, Das CR (2020) Liquid based-cytology pap smear dataset for automated multi-class diagnosis of pre-cancerous and cervical cancer lesions. Data in Brief p 105589
  * Jantzen et al. [2005] Jantzen J, Norup J, Dounias G, Bjerregaard B (2005) Pap-smear benchmark data for pattern classification. Nature inspired Smart Information Systems (NiSIS 2005) pp 1–9
  * Kennedy and Eberhart [1995] Kennedy J, Eberhart R (1995) Particle swarm optimization. In: Proceedings of ICNN’95-International Conference on Neural Networks, IEEE, vol 4, pp 1942–1948
  * Marinakis et al. [2009] Marinakis Y, Dounias G, Jantzen J (2009) Pap smear diagnosis using a hybrid intelligent scheme focusing on genetic algorithm based feature selection and nearest neighbor classification. Computers in Biology and Medicine 39(1):69–78
  * Martínez-Más et al. [2020] Martínez-Más J, Bueno-Crespo A, Martínez-España R, Remezal-Solano M, Ortiz-González A, Ortiz-Reina S, Martínez-Cendán JP (2020) Classifying papanicolaou cervical smears through a cell merger approach by deep learning technique. Expert Systems with Applications 160:113707
  * Mirjalili [2015] Mirjalili S (2015) Moth-flame optimization algorithm: A novel nature-inspired heuristic paradigm. Knowledge-based systems 89:228–249
  * Mirjalili and Lewis [2016] Mirjalili S, Lewis A (2016) The whale optimization algorithm. Advances in engineering software 95:51–67
  * Mirjalili et al. [2014] Mirjalili S, Mirjalili SM, Lewis A (2014) Grey wolf optimizer. Advances in engineering software 69:46–61
  * Mitra et al. [2019] Mitra S, Dey S, Das N, Chakrabarty S, Nasipuri M, Naskar MK (2019) Identification of malignancy from cytological images based on superpixel and convolutional neural networks. In: Studies in Computational Intelligence, Springer Singapore, pp 103–122, URL https://doi.org/10.1007/978-981-13-7334-3_8
  * Mitra et al. [2020] Mitra S, Das N, Dey S, Chakrabarty S, Nasipuri M, Naskar MK (2020) Cytology image analysis techniques towards automation: Systematically revisited. arXiv preprint arXiv:200307529
  * Niedzielewski et al. [2020] Niedzielewski K, Marchwiany ME, Piliszek R, Michalewicz M, Rudnicki W (2020) Multidimensional feature selection and high performance parallex. SN Computer Science 1(1):1–7
  * Plissiti et al. [2018] Plissiti ME, Dimitrakopoulos P, Sfikas G, Nikou C, Krikoni O, Charchanti A (2018) Sipakmed: A new dataset for feature and image based classification of normal and pathological cervical cells in pap smear images. In: 2018 25th IEEE International Conference on Image Processing (ICIP), IEEE, pp 3144–3148
  * Simonyan and Zisserman [2014] Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:14091556
  * Szegedy et al. [2016] Szegedy C, Vanhoucke V, Ioffe S, Shlens J, Wojna Z (2016) Rethinking the inception architecture for computer vision. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2818–2826
  * Wang and Garibaldi [2005] Wang XY, Garibaldi JM (2005) Simulated annealing fuzzy clustering in cancer diagnosis. Informatica 29(1)
  * William et al. [2019a] William W, Ware A, Basaza-Ejiri AH, Obungoloch J (2019a) Cervical cancer classification from pap-smears using an enhanced fuzzy c-means algorithm. Informatics in Medicine Unlocked 14:23–33
  * William et al. [2019b] William W, Ware A, Basaza-Ejiri AH, Obungoloch J (2019b) A pap-smear analysis tool (pat) for detection of cervical cancer from pap-smear images. Biomedical engineering online 18(1):16
  * Win et al. [2020] Win KP, Kitjaidure Y, Hamamoto K, Myo Aung T (2020) Computer-assisted screening for cervical cancer using digital image processing of pap smear images. Applied Sciences 10(5):1800
  * Wu et al. [2018] Wu M, Yan C, Liu H, Liu Q, Yin Y (2018) Automatic classification of cervical cancer from cytological images by using convolutional neural network. Bioscience reports 38(6)
  * Xue et al. [2020] Xue D, Zhou X, Li C, Yao Y, Rahaman MM, Zhang J, Chen H, Zhang J, Qi S, Sun H (2020) An application of transfer learning and ensemble learning techniques for cervical histopathology image classification. IEEE Access 8:104603–104618
  * Yang [2009] Yang XS (2009) Firefly algorithms for multimodal optimization. In: International symposium on stochastic algorithms, Springer, pp 169–178
  * Yang and Gandomi [2012] Yang XS, Gandomi AH (2012) Bat algorithm: a novel approach for global engineering optimization. Engineering computations
  * Zhang et al. [2014] Zhang L, Kong H, Ting Chin C, Liu S, Fan X, Wang T, Chen S (2014) Automation-assisted cervical cancer screening in manual liquid-based cytology with hematoxylin and eosin staining. Cytometry Part A 85(3):214–230
  * Zhang et al. [2017] Zhang L, Lu L, Nogues I, Summers RM, Liu S, Yao J (2017) Deeppap: deep convolutional networks for cervical cell classification. IEEE journal of biomedical and health informatics 21(6):1633–1643
  * Zhang [2012] Zhang Y (2012) Support vector machine classification algorithm and its application. In: International Conference on Information Computing and Applications, Springer, pp 179–186