# Breast Histopathology Image Retrieval by Attention-based Adversarially
Regularized Variational Graph Autoencoder with Contrastive Learning-Based
Feature Extraction

Nematollah Saeidi<EMAIL_ADDRESS>Hossein Karshenas
<EMAIL_ADDRESS>Bijan Shoushtarian<EMAIL_ADDRESS>Sepideh
Hatamikia<EMAIL_ADDRESS>Ramona Woitek  ramona.woitek@dp-
uni.ac.at Amirreza Mahbod<EMAIL_ADDRESS>Artificial
Intelligence Department, Faculty of Computer Engineering, University of
Isfahan, Isfahan, Iran  Department of Computer Engineering Techniques, Mazaya
University College, Nasiriyah, Iraq  Research Center for Medical Image
Analysis and Artificial Intelligence, Department of Medicine, Danube Private
University, Krems an der Donau, Austria  Austrian Center for Medical
Innovation and Technology, Wiener Neustadt, Austria

###### Abstract

Breast cancer is a significant global health concern, particularly for women.
Early detection and appropriate treatment are crucial in mitigating its
impact, with histopathology examinations playing a vital role in swift
diagnosis. However, these examinations often require a substantial workforce
and experienced medical experts for proper recognition and cancer grading.
Automated image retrieval systems have the potential to assist pathologists in
identifying cancerous tissues, thereby accelerating the diagnostic process.
Nevertheless, due to considerable variability among the tissue and cell
patterns in histological images, proposing an accurate image retrieval model
is very challenging.

This work introduces a novel attention-based adversarially regularized
variational graph autoencoder model for breast histological image retrieval.
Additionally, we incorporated cluster-guided contrastive learning as the graph
feature extractor to boost the retrieval performance. We evaluated the
proposed model’s performance on two publicly available datasets of breast
cancer histological images and achieved superior or very competitive retrieval
performance, with average mAP scores of 96.5% for the BreakHis dataset and
94.7% for the BACH dataset, and mVP scores of 91.9% and 91.3%, respectively.

Our proposed retrieval model has the potential to be used in clinical settings
to enhance diagnostic performance and ultimately benefit patients.

###### keywords:

Computational pathology , Graph Neural Network , Graph Construction ,
Adversarial Training , Variational Graph Autoencoder , Attention Module ,
Image Retrieval

##  1 Introduction

Breast cancer is a significant health threat to women, leading to considerable
mortality worldwide [1]. It is the most commonly diagnosed cancer worldwide
and represents a quarter of all cancers diagnoses in women [2]. Female breast
cancer mortality peaked in 1989 and has since decreased by 42% by 2021,
translating to the avoidance of more than 490,000 deaths [3]. However, it is
still one of the main causes of cancer death among women in recent years. For
instance, according to 2018 statistics, breast cancer accounted for 15% of all
cancer-related deaths and witnessed a 24% surge in new cases worldwide [4].
While breast cancer carries a high mortality rate, early detection and
diagnosis followed by appropriate treatment can help mitigate its impact [5].
The urgency of early breast cancer detection has prompted innovative
solutions, such as computer-aided diagnosis systems based on histological
images [6]. This technology is a valuable tool to improve breast cancer
diagnosis and ultimately can save lives [7]. However, proper diagnosis based
on histological images often demands a substantial workforce and experienced
medical experts. To overcome this challenge, pathologists can benefit from
referring to archived databases for complex cases to find similar examples
with verified findings to minimize diagnostic errors.

Content-based image retrieval (CBIR) is an approach to identify visually
similar images in a database and can be used for various applications [8, 9].
For example, pathologists reviewing tissue sections for breast cancer
diagnosis may benefit from searching through labeled histopathological image
data for images resembling a case they are reviewing to improve their
diagnostic accuracy in distinguishing cancerous from non-cancerous tissue.
CBIR techniques have also been used for various non-medical applications such
as ”image search” for general and natural images, visual search for retail
products, and multiple tools for recognizing faces and artworks [10]. Prior
machine learning-based CBIR systems often employed models adapted to specific
applications, requiring the collection of labeled data for each particular
use, which presented a significant implementation challenge. Additionally, the
notion of ”similarity” in these approaches could have different
interpretations depending on the specific context [11]. For instance, two
histological images might be deemed similar because they originate from the
same organ, depict the same type of cancer, share similar staining, or exhibit
comparable histological characteristics.

Content-based medical image retrieval (CBMIR), a subset of CBIR, is
specifically adapted to medical images and has been used in the context of
radiology and histology in former studies [10, 11, 12]. CBMIR for histological
images, not only aids histopathology in grading new tissue samples but also
assists in analyzing patterns within similar tissues from previous patients
[12, 6]. CBMIR offers distinct advantages over classification tasks. For
instance, in classifying prostate cancer, a classification model provides
class labels for each input. In contrast, a CBMIR system produces the top-$k$
most similar images based on a query image, where $k$ represents the number of
desired results. Upon availability, it can also provide other relevant
information such as clinical meta-data or diagnosis reports [13]. Therefore,
CBMIR provides pathologists and practitioners with more comprehensive
information about diverse tissue patterns, enabling them to identify the most
similar patterns within different tissues and minimize intra-and inter-
observer variability [10, 14]. Another advantage of CBMIR is that it
inherently includes a clinician-in-the-loop, which is often required for
artificial intelligence (AI) applications in healthcare. Furthermore, a
pathologist using CBMIR in the diagnostic process is required to come to their
own diagnostic conclusion after incorporating additional information provided
by CBMIR systems rather than replacing the pathologist’s diagnosis with an AI-
based classifier. Unlike histological images with their reports written by
pathologists that are presented by CBMIR systems, a classifier is not
necessarily easily interpretable and - even if presented to a clinician as a
second opinion - may lead to automation bias [15, 16]. From a machine learning
perspective, CBMIR is typically performed using unsupervised techniques, while
classifiers are usually trained with supervised methods that require labeled
datasets.

Histopathology image retrieval poses a significant challenge due to the
substantial tissue and cell pattern variability [11]. Various semi- and fully-
automatic approaches have been proposed for histopathology image retrieval
[10]. Among these, graph neural networks (GNN) have gained considerable
attention in recent years. Considering the grid data structure in images and
the tremendous developments in convolutional neural networks (CNNs), many
graph convolutional neural network (GCNN)-based architectures have also been
extensively proposed for a wide range of computer vision tasks, including
image retrieval [17]. In GCNN, input images can be represented as the graph
nodes, and their features are usually extracted by pre-trained CNNs such as
Visual Geometry Group (VGG) [18] or EfficientNet [19] to construct the graph.
However, the exploited pre-trained models are typically trained on natural
images, and their extracted features may not be optimally suited for medical
data (e.g., histological images) [20, 21].

While GCNNs are usually trained in a supervised manner, some GCNN variants,
such as variational graph autoencoder (VGAE) [22], can be used for
unsupervised learning tasks. In VGAE, the encoder network maps input data to a
lower-dimensional latent space, and the decoder network maps latent space
vectors back to the original input data. Unlike conventional graph
autoencoders (GAE), VGAE uses a probabilistic encoding function that maps
input data to a probability distribution over the latent space. VGAE
performance can be further enhanced by incorporating other techniques, such as
adversarial training or adding an attention module to the model [22]. In
adversarially regularized variational graph autoencoder (ARVGA) [23], the
encoder is forced to generate embedding vectors closely resembling the data
from a prior distribution through adversarial training. Incorporating an
attention mechanism into CNNs [24] or GNNs [25] is another technique that has
been exploited. These techniques have also been jointly applied for specific
applications such as graph link prediction [26], graph clustering
visualization [26], and node classification in partially labeled graphs [27].

In this study, for the first time, we adapted and exploited attention-based
adversarially regularized variational graph autoencoders (A-ARVGAE) for the
histological breast image retrieval task. Moreover, unlike previous studies
that use conventional pre-trained CNNs for feature extraction to build the
graph, we used the recently developed clustering-guided contrastive learning
(CCL) feature extractor [28], which is explicitly designed for
histopathological images. We evaluated the performance of our proposed
approach on two binary and multi-class histopathological breast image
datasets, namely breast cancer histopathological database (BreakHis) [29] and
breast cancer histology images dataset (BACH [30]), and achieved superior or
very competitive performance compared to other state-of-the-art models. The
main contribution of this research can be summarized as following:

  * 1.

We adapted and exploited one of the state-of-the-art GNN models, which
incorporated both attention mechanism and adversarial training for breast
cancer histology image retrieval task.

  * 2.

Instead of using conventional pre-trained CNNs as the node feature extractor,
we made use of CLL, which is explicitly designed for histological images.

  * 3.

Through extensive experiments, we show the effectiveness of our approach for
both utilized GNN and feature extraction on two histopathological breast image
datasets, namely BreakHis and BACH and achieved superior or very competitive
performance compared to other approaches.

The rest of this paper is organized as follows: In Section 2, we present a
background on methods for medical image retrieval and graph representation
learning in histopathology. Section 3 describes our methodology, which
contains all steps for the proposed framework. Section 4 consists of the
experiments and evaluations with an analysis of the results. Finally, in
Section 5, we conclude the paper with suggestions for future works.

##  2 Related Works

This section carries out a literature review for medical image retrieval and
graph representation learning in histopathology, considering their relevance
to our work.

###  2.1 Medical Image Retrieval

Various methods have been proposed in the literature for medical image
retrieval. Text-based image retrieval systems were among the first attempts to
find similar images in a given database. However, these methods rely heavily
on expert annotation and proper keyword selection, making them generally
considered ineffective and time-consuming [31].

CBMIR models, on the other hand, rely on the content (features) of the images
instead of their given labels. Consequently, they can be used in an
unsupervised manner without the need for manual labeling. Conventional CBMIR
models consist of image preprocessing, hand-crafted or automatic feature
extraction, and feature similarity measurements using different distance-based
methods. To enhance interpretability, some approaches propose reducing the
disparity between the extracted features and the high-level semantic meanings
of the images by incorporating expert feedback or medical metadata into the
retrieval model [31, 32].

Commonly used preprocessing methods include noise reduction, contrast
enhancement, patch generation (e.g., from whole slide images), and
normalization (e.g., stain normalization in histological images) [33, 34].
However, many novel machine learning (ML)- and deep learning (DL)-based
feature extractors do not require any preprocessing steps. For feature
extraction, both hand-crafted feature extractors and DL-based automatic
feature extractors have been utilized. Hand-crafted features are employed to
depict patches’ morphological and textural characteristics. These features
include color histogram-based features, histogram of oriented gradients (HOG)
descriptor, local statistical features, global image structure tensor (GIST)
features, Gabor features, wavelet-based features, intensity-based features,
textural features, gray level co-occurrence matrix (GLCM) features, and scale-
invariant feature transform (SIFT) descriptors [31, 28]. These features
necessitate pre-defined transformation functions by experts. However, more
recently, the rapidly evolving field of DL has streamlined the automatic high-
level feature extraction process [11]. The ability of DL models to represent
complicated patterns has made them a common approach for many image processing
tasks.

Many pre-trained DL-based models, such as VGG19, ResNet, MobileNetV2,
Xception, EfficientNet, and DenseNet121 have been used as feature extractors
for tasks such as medical image classification or medical image retrieval [35,
36]. Hand-crafted or automatically extracted features of a query image are
then compared to the image features in a database to find similar images. To
do so, various similarity measurements such as distance-based retrieval (SDR),
customized query approach (CQA), or nearest-neighbor matching can be used
[37]. In the context of histopathology, Yottixel [38] and FISH [14] methods
select the most distinctive patches to represent whole slide images (WSIs)
through $K$-means clustering based on RGB histogram and spatial coordinate
features. Yottixel employs a pre-trained DenseNet121, trained on the ImageNet
dataset [39], to extract patch embeddings, which are then transformed into
binary barcodes. FISH trains a model based on an autoencoder on the cancer
genome atlas (TCGA) dataset to encode an integer index for each patch during
retrieval. However, it should be noted that using a pre-trained feature
extractor (trained with natural images) alone may lead to suboptimal
performance due to the domain shift between natural and histopathological
images. A recent cutting-edge method introduces a pre-trained model, CCL [28],
which is explicitly designed for histological images. CCL pretraining was
conducted with around 15 million image patches extracted from more than 32,000
WSIs.

The CCL learning method leverages large-scale unlabeled histopathological
image data, acquiring universal features that can be directly applied to
subsequent WSI retrieval tasks. In our study, we utilized CCL as a robust and
relevant feature extractor to build the graph.

###  2.2 Graph Representation Learning in Histopathology

While CNNs have demonstrated impressive performance in medical image analysis,
they often struggle to capture contextual information from neighboring areas,
especially in scenarios with a limited field of view, such as patch-based
analysis in histological images. In contrast to CNNs, GNNs excel at preserving
complex correlations among neighboring elements during the learning process.
GNNs have been successfully applied to various tasks, including tumor/object
localization, classification, survival prediction, and image retrieval [40].

In the field of computational pathology, several GNN models, such as cell
graphs, tissue graphs, cell-tissue graphs, and patch graphs, have been
proposed for tasks like node classification (e.g., nuclei classification) or
graph classification (e.g., histologic image patch categorization) [41, 42].
However, these applications have primarily been explored in a supervised
manner and at times need sophisticated preprocessing, such as using other
models like HoVerNet [43] or DDU-Net [44] to generate nuclei segmentation
masks as input for the graph-based model. Concerning clinical applications,
GNNs have found utility in different cancer types, including breast cancer,
colorectal cancer, and prostate cancer, among others [42].

To construct a graph, one needs to define nodes, node embeddings, and edges.
In the context of histological images, nodes could represent nuclei, image
patches, or tissue regions. Node embeddings are features of the nodes that can
be extracted either manually or automatically using pre-trained models or
other techniques. The edges depict connections between nodes, typically
determined by predefined criteria such as $k$-nearest neighbors or distances
between nodes. Once the graph is built, various GNN or GCN models can be
applied, depending on the specific applications.

A GCN comprises several key components: an aggregation layer to define how
features from neighbor nodes are aggregated, an activation layer to introduce
non-linearity, graph pooling to reduce graph size (if needed), and graph
convolutional operations to update node representations. It’s worth noting
that other variants, such as GraphSAGE [45], GAT [46], or ChebNet [47] have
been introduced in the literature as novel propagation rules to enhance
performance across different applications. The output of a GNN typically
depends on the application. For graph classification, a multilayer perceptron
(MLP) can be employed atop the graph. In the case of graph reconstruction for
unsupervised tasks, a linear projection decoder can be added to the model. As
highlighted in a recent review [40], most GNN-based works in computational
pathology are related to supervised classification tasks. Only a few studies
have delved into histological image retrieval [40, 28, 48, 13]. However, even
in these studies, the potential of attention mechanisms and adversarial
training have not been explored. In our research, for the first time, we
adapted and made use of both attention mechanism and adversarial training in
variational graph autoencoders for the histological image retrieval task.

##  3 Methodology

The general workflow of our proposed approach for histological image retrieval
is depicted in Figure 1. Subsequently, we discuss the details of the training
process, followed by the application of the model in the testing phase for a
given query image.

Figure 1: The general workflow of the proposed breast histological image
retrieval model involves distinct processes for training and testing.
A-ARVGAE: Attention-based adversarially regularized variational graph
autoencoders.

###  3.1 Dataset

The dataset refers to a collection of histological image patches used to train
and test the model. Each image has certain features and characteristics
related to a specific class or histopathological grade. We use two breast
histopathology image datasets in this study: BreakHis dataset [29] and BACH
dataset [30], which are subsequently detailed. Example images from the
datasets are shown in Figure 2.

  * 1.

BreakHis: This dataset comprises hematoxylin and eosin (H&E)-stained breast
histology microscopy images captured at various magnifications ($40\times$,
$100\times$, $200\times$, $400\times$) and is categorized into two primary
classes: benign and malignant classes. The dataset contains 7,909 images
including 2,480 benign and 5,429 malignant samples with a size of $700\times
460$ pixels in Portable Network Graphics (PNG) format.

  * 2.

BACH: This dataset comprises H&E-stained breast histology microscopy images
captured at $200\times$ magnification. It includes 400 images distributed
across four distinct classes: benign (100 images), in situ carcinoma (100
images), invasive carcinoma (100 images), and normal (100 images), each with a
resolution of $2048\times 1536$ pixels in Tagged Image File (TIF) format.

BreakHis(40$\times$) | BreakHis(100$\times$) | BreakHis(200$\times$) | BreakHis(400$\times$) | BACH(200$\times$)
---|---|---|---|---
|  |  |  |
Figure 2: Example images from the BreakHis (first four images) and the BACH
(last image) datasets.

###  3.2 Feature Extraction

The first step in our proposed retrieval model is the extraction of features
for all images in the dataset. Instead of using hand-crafted features or
conventional pre-trained CNNs as feature extractors, we utilize CCL as the
main feature extractor. Unlike other conventional pre-trained CNNs, CCL is
trained with histological images. We hypothesize that using such a pre-trained
feature extractor could lead to better descriptive features for the task of
histological image retrieval.

We also use a set of standard baseline models as feature extractors and
compare their retrieval performance with the CCL feature extractor. For this,
we employ well-known pre-trained models, namely VGG19 [18], MobileNetV2 [49],
DenseNet121 [50], Xception [51], InceptionResNetV2 [52], NASNetLarge [53], and
EfficientNetV2M [19]. The details of these baseline pre-trained models are
presented in Table 1. It is worth mentioning that all these pre-trained models
are initially trained on natural images using the ImageNet dataset [39], but
the CCL feature extractor used in this study is pre-trained with histological
image patches. The results of these comparisons are reported in Section 4
(Table 2 and Table 3).

Table 1: Details of baseline feature extractors. All these pre-trained models are initially trained on natural images using the ImageNet dataset [39]. Feature Extraction Model | Size (MB) | Parameters | Depth
---|---|---|---
VGG19 | 549 | 143.7 M | 19
MobileNetV2 | 14 | 3.5 M | 105
DenseNet121 | 33 | 8.1 M | 242
Xception | 88 | 22.9 M | 81
InceptionResNetV2 | 215 | 55.9 M | 449
NASNetLarge | 343 | 88.9 M | 533
EfficientNetV2M | 220 | 54.4 M | -

###  3.3 Graph Construction

GNNs are mostly applied on graph-based datasets. These datasets typically
include nodes, node features, edges, and adjacency matrices. In contrast, non-
structural datasets, such as those consisting of images, lack inherent graph
structures. This makes graph construction a crucial step for applying GNNs to
such data. For the retrieval task, graph construction methods often treat
subjects (e.g., images) as graph nodes ($V$), with the features of these
subjects serving as node features ($X$). Edges between nodes ($E$) can be
defined using distance metrics like cosine and Euclidean distances, indicating
the likelihood of interaction between entities [54, 55, 56, 57]. The topology
of these graphs is usually determined heuristically, employing strategies such
as a pre-defined proximity threshold, a nearest neighbor rule, or a
probabilistic model. To create edges between nodes, various algorithms such as
$k$ approximate nearest neighbor ($k$-ANN) can be used. In $k$-ANN, the
exactness of the neighbor relationship is relaxed to gain efficiency. This
approach is beneficial in high-dimensional feature spaces, where the curse of
dimensionality slows down exact nearest neighbor searches significantly. While
there are various implementations of ANN algorithms, the ANN benchmark [58] is
a widely-used library that encompasses most ANN-based methods. In our study,
we utilize the fast library for approximate nearest neighbors (FLANN) to
calculate distances between node embeddings, selecting the $k$ nearest nodes
to each entity as adjacent nodes. The computation of graph edges is conducted
as follows:

$e_{ij}=\begin{cases}1&\text{if }v_{j}\in k\text{-ANN}(v_{i})\\\
0&\text{otherwise}\end{cases}\quad(1)$

where $e_{ij}=1$ indicates the existence of an edge connection between the
$i$-th and $j$-th nodes in the graph, and $e_{ij}=0$ denotes no edge.
Utilizing the edges and nodes, the adjacency matrix ($A$) can be constructed
where the values of the matrix ($a_{ij}$) act as indicators, quantifying of
the connections between nodes $V_{i}$ and $V_{j}$.

It is worth mentioning that we use transductive learning, where the entire
dataset (training, validation, and test data) is observed to build the graph.
The summarized workflow to build the graph in our study is shown in Figure 3.

Figure 3: The framework of graph construction. CCL: Cluster guided contrastive
learning, $K$-ANN: $K$ approximate nearest neighbors, FLANN: fast library for
approximate nearest neighbors

###  3.4 A-ARVGAE

The overall architecture of the A-ARVGAE model is depicted in Figure 4. The
proposed architecture comprises four main components: the encoder (yellow
block), which also serves as the generator for adversarial training; the
embedding latent space (pink block); the decoder (blue block); and the
discriminator (green block), which is another essential component for
adversarial training.

The constructed graph, as described in the previous section, is fed into the
encoder of the A-ARVGAE model. The encoder includes a graph attention layer
that performs attention operations, batch normalization, leaky ReLU, and
dropout, followed by two graph convolutional layers. The attention-based
convolutional operation integrating attention mechanisms into convolutional
layers assigns varying weights to the neighbors, enabling the model to
concentrate on the most relevant parts of the graph when learning node
representations [46]. The attention mechanism aggregates information from
adjacent nodes with an adaptive weight allocation to neighboring nodes. As
indicated by prior studies, the use of a multi-head attention layer has been
shown to yield superior performance [26] and hence we also employ two-head
attention layer in our work.

Inspired by the VGAE [20], the encoder in our architecture does not encode an
input into a fixed latent representation. Instead, the outputs of the graph
convolutional layers are parameters (mean ($\mu$) and variance ($\sigma$))
that define a probability distribution over the possible latent
representations, denoted as $Z$, characterized by a normal distribution
$N(\mu,\sigma)$. Sampling from the latent space involves drawing samples from
the distribution $N(\mu,\sigma)$ to produce latent node embeddings. This
process employs the reparameterization trick, as introduced in [20], which
enables the backpropagation of gradients through the stochastic sampling
process for gradient descent optimization.

The decoder block employs a linear projection through a straightforward inner
product ($Z*Z^{T}$) followed by a sigmoid function, which results in the
reconstructed graph.

As the A-ARVGAE model incorporates adversarial training, it includes a
generator and a discriminator. Within this framework, the encoder functions
analogously to a generator in an adversarial network [23]. The generator tries
to deceive the discriminator by producing synthetic data, which, in this
context, refers to latent variables generated from the encoding of graph data.
On the other hand, the discriminator (green block) attempts to differentiate
whether samples are derived from real data or the generator. The discriminator
evaluates embeddings from the prior distribution (a Gaussian distribution) as
real and those from the latent variable $Z$ as synthetic or fake. The
discriminator block comprises a straightforward MLP with two linear layers
along with batch normalization, leaky reLU, and dropout to distinguish between
real (a prior Gaussian distribution) and fake (latent space $Z$) embeddings.

The complete model integrates three loss components: a regression-based
reconstruction loss (related to VGAE), a Kullback–Leibler divergence loss
(related to VGAE), and a binary cross-entropy loss (related to discriminator
in adversarial training). Further details regarding the model architecture and
cost functions can be found in the respective studies [23, 46].

We compare the results of our employed GNN model (A-ARVGAE) with other GNN-
based models, namely GAE, VGAE, and ARVGA, and also with another non-GNN-based
state-of-the-art approach, called RetCCL, represented in [28]. RetCCL is a
model that is recently developed for the histological image retrieval task,
which is based on CCL feature extraction and nearest neighbor search to find
similar images in a given dataset. The results of these comparisons are
reported in Section 4 (Table 4).

Figure 4: Architecture of attention-based adversarially regularized
variational graph autoencoder (A-ARVGAE). For interpretation of the references
to color in this figure legend, the reader is referred to the web version of
this article.

###  3.5 Trained Graph Encoder

After training, the trained A-ARVGAE encoder can transform any input image
into a vector in the embedding space.

###  3.6 Image Embedding Database

The embeddings generated by the trained A-ARVGAE encoder in the previous step
for each training image are stored in a database. This database is then
utilized in the testing phase to identify the top-$K$ similar images for a
given query image as explained in Section 3.7

###  3.7 Testing phase

As depicted in the lower part of Figure 1, for a query image, the model aims
to identify the top $K$ similar images in the database. Similar to the process
with training images, features are first extracted from the test image using
the CCL feature extractor. These features are then fed into the trained
A-ARVGAE model to obtain the embedding in a lower-dimensional space,
specifically from the encoder part of the model. After acquiring the embedding
of the query image, it is compared with the embeddings in the image database
to identify similar images. To find the most similar images, the ranking
algorithm evaluates and ranks the candidate images based on Euclidean
distance. These selected images are then arranged and ranked in ascending
order according to Euclidean distances. Once ranked, the top-$K$ similar
images are chosen as the final output of the retrieval model.

To evaluate the performance of the developed approach, we use two evaluation
metrics, namely, mean Average Precision (mAP) [59] and mean Majority Vote
(mMV) [14].

mAP is the average of the precision values computed at the ranks of relevant
images in the retrieved results. It is a widely used metric for evaluating the
effectiveness of a retrieval system. The formula for mAP is as follows:

$mAP(k)=\frac{1}{|Q|}\sum_{q=1}^{|Q|}\frac{1}{|R_{q}|}\sum_{k=1}^{|R_{q}|}\text{Precision}(k)\cdot\text{rel}(k)\quad(17)$

Where $|Q|$ is the number of queries, $R_{q}$ is the number of relevant images
for query image $q$, $\text{Precision}(k)$ is the precision at rank $k$ (the
ratio of relevant images retrieved in the top $k$ results to the total number
of retrieved images up to rank $k$), $\text{rel}(k)$ is an indicator function
that equals 1 if the image at rank $k$ is relevant and 0 otherwise. $|\cdot|$
denotes the length of the set.

mMV evaluates the frequency with which the primary diagnosis among the top-$k$
results aligns with the queried diagnosis as follows:

$mMV(k)=\frac{1}{|Q|}\sum_{q=1}^{|Q|}1\\{L_{q}\in
MV(\text{rel}(k))\\}\quad(18)$

Where $L_{q}$ represents the ground truth diagnosis of the image, and MV is
the predicted diagnosis of an image taken from the majority vote of top-$k$
samples. Similar to mAP, $\text{rel}(k)$ is an indicator function that equals
1 if the image at rank $k$ is relevant and 0 otherwise. Here, $|\cdot|$ also
denotes the length of the set.

##  4 Results & Discussion

In this section, we start with the implementation details, then delve into the
experimental setup, and finally present the obtained results along with a
detailed discussion.

###  4.1 Implementation Details

We used the PyTorch deep learning framework (version 2.2.0) and the PyTorch
Geometric library (version 2.4.0) to implement the models. Adam optimizer was
used to train the models with an initial learning rate (LR) of 0.0001. For
dropout layers, a probability of 0.2 was chosen in the model architecture. All
models were trained for 250 epochs, whereas in models with adversarial
components, the discriminator was trained for five iterations in each epoch.
To extract features from the training and test images, all images were down-
sampled to 256$\times$256 pixels to fit into the utilized GPU memory. For each
experiment, an identical training set (70%), validation set (10%), and test
set (20%) were used to ensure fair comparisons between different experiments.
The data splitting was done randomly by keeping the ratio between different
classes in the training, validation and test sets. In the process of graph
construction, the parameter $k$ was set to 25 for the BreakHis dataset and 15
for the BACH dataset. For the attention layer in the A-ARVGAE model, a 2-head
attention layer was used. All experiments were conducted on a single machine
using an NVIDIA GTX 1060 GPU and an Intel Core™ i7-9750H CPU.

###  4.2 Experiments

We performed two sets of experiments for each dataset in this study.

In the first set, we investigated the effect of using the CCL feature
extractor against other pre-trained models for the breast cancer histology
image retrieval task. The results of this experiment for the BreakHis and BACH
datasets are reported in Table 2 and Table 3, respectively. For a fair
comparison, we used the A-ARVGAE model in all experiments.

Table 2: Evaluation of the retrieval performance for the BreakHis dataset with different feature extractors. In all experiments attention-based adversarially regularized variational graph autoencoder is used as the retrieval model. Magnification | Feature Extraction | mAP(5) | mMV(5)
---|---|---|---
$40\times$ | VGG19 | 0.945 | 0.859
| MobileNetV2 | 0.957 | 0.894
| DenseNet121 | 0.941 | 0.864
| Xception | 0.899 | 0.743
| InceptionResNetV2 | 0.893 | 0.778
| NASNetLarge | 0.925 | 0.809
| EfficientNetV2M | 0.919 | 0.768
| CCL | 0.978 | 0.949
$100\times$ | VGG19 | 0.948 | 0.850
| MobileNetV2 | 0.947 | 0.884
| DenseNet121 | 0.953 | 0.879
| Xception | 0.903 | 0.739
| InceptionResNetV2 | 0.889 | 0.739
| NASNetLarge | 0.919 | 0.777
| EfficientNetV2M | 0.915 | 0.826
| CCL | 0.980 | 0.961
$200\times$ | VGG19 | 0.941 | 0.840
| MobileNetV2 | 0.955 | 0.870
| DenseNet121 | 0.946 | 0.860
| Xception | 0.904 | 0.796
| InceptionResNetV2 | 0.883 | 0.695
| NASNetLarge | 0.921 | 0.776
| EfficientNetV2M | 0.905 | 0.791
| CCL | 0.964 | 0.930
$400\times$ | VGG19 | 0.945 | 0.861
| MobileNetV2 | 0.941 | 0.850
| DenseNet121 | 0.955 | 0.895
| Xception | 0.911 | 0.784
| InceptionResNetV2 | 0.897 | 0.718
| NASNetLarge | 0.906 | 0.751
| EfficientNetV2M | 0.905 | 0.790
| CCL | 0.940 | 0.839
BreakHis Avg. | VGG19 | 0.944 | 0.852
| MobileNetV2 | 0.950 | 0.874
| Xception | 0.904 | 0.765
| InceptionResNetV2 | 0.890 | 0.727
| NASNetLarge | 0.917 | 0.778
| EfficientNetV2M | 0.911 | 0.791
| DenseNet121 | 0.948 | 0.874
| CCL | 0.965 | 0.919
Table 3: Evaluation of the retrieval performance for the BACH dataset with different feature extractors. In all experiments attention-based adversarially regularized variational graph autoencoder is used as the retrieval model. Magnification | Feature Extraction | mAP(5) | mMV(5)
---|---|---|---
$200\times$ | VGG19 | 0.899 | 0.787
| MobileNetV2 | 0.893 | 0.737
| DenseNet121 | 0.897 | 0.700
| Xception | 0.887 | 0.687
| InceptionResNetV2 | 0.872 | 0.675
| NASNetLarge | 0.892 | 0.800
| EfficientNetV2M | 0.882 | 0.762
| CCL | 0.947 | 0.913

As observed from Table 2 for the BreakHis dataset, the CCL feature extractor
outperforms all other pre-trained feature extractor models across all metrics,
except for 400$\times$ magnification, where DenseNet121 delivered better
retrieval performance. However, for the average performance across all
magnifications, CCL delivers the best results compared with all other pre-
trained models, including DenseNet121. The results in Table 3 also indicate
the superior performance of the CCL feature extractor compared with other pre-
trained models for the BACH dataset.

Overall, the results reported in Table 2 and Table 3 for the BreakHis and BACH
datasets show that the feature extractor trained on histological images
outperforms feature extractors trained with non-medical images.

In the second experiment, we compare the performance of the employed A-ARVGAE
model with other GNN-based and non-GNN-based models for both BreakHis and BACH
datasets. The results of these experiments are presented in Table 4.

Table 4: Comparison of different models (GNN-based models including GAE, VGAE, and ARVGA, non-GNN-based model, and ours) for the BreakHis and BACH datasets for the breast cancer histological image retrieval task. GAE: graph autoencoder, VGAE: variational graph autoencoder, ARVGA: adversarial regularized variational graph autoencoder, RetCCL: retrieval with cluster-guided contrastive learning Dataset | Method | mAP(5) | mMV(5)
---|---|---|---
BreakHis (40$\times$) | GAE | 0.901 | 0.773
| VGAE | 0.962 | 0.934
| ARVGA | 0.969 | 0.909
| RetCCL | 0.983 | 0.9789
| Ours | 0.978 | 0.9499
BreakHis (100$\times$) | GAE | 0.910 | 0.792
| VGAE | 0.966 | 0.903
| ARVGA | 0.949 | 0.909
| RetCCL | 0.970 | 0.951
| Ours | 0.980 | 0.961
BreakHis (200$\times$) | GAE | 0.926 | 0.825
| VGAE | 0.955 | 0.900
| ARVGA | 0.949 | 0.879
| RetCCL | 0.920 | 0.920
| Ours | 0.964 | 0.930
BreakHis (400$\times$) | GAE | 0.924 | 0.828
| VGAE | 0.923 | 0.806
| ARVGA | 0.932 | 0.834
| RetCCL | 0.897 | 0.889
| Ours | 0.940 | 0.839
BreakHis Avg. | GAE | 0.915 | 0.804
| VGAE | 0.951 | 0.885
| ARVGA | 0.949 | 0.882
| RetCCL | 0.942 | 0.934
| Ours | 0.965 | 0.919
BACH (200$\times$) | GAE | 0.919 | 0.740
| VGAE | 0.937 | 0.798
| ARVGA | 0.935 | 0.817
| RetCCL | 0.901 | 0.8942
| Ours | 0.947 | 0.913

As shown in Table 4, the A-ARVGAE model surpasses all GNN-based models i.e.,
GAE, VGAE, and ARVGA for both datasets. The superior performance of the
proposed model over ARVGA demonstrates the significance of the attention
component in the employed architecture. An attention component allows the
model to focus on specific parts of input data to calculate the model weights.

Compared with the RetCCL model, our approach delivers superior performance for
both evaluation indices on the BACH dataset. However, for the BreakHis
dataset, RetCLL shows slightly better performance for some magnifications.
Nevertheless, our approach still yields superior results in other
magnifications. The average results across all magnifications indicate the
superior performance of our approach compared with RetCCL, based on the mAP(5)
evaluation index (2.3% improved performance). However, for the MVP(5) score,
RetCCL exhibits a better average performance with a 1.5% margin.

Overall, while our approach has demonstrated superior or very competitive
performance in all experiments, it should be noted that GNN training typically
requires large datasets [60]. The utilized model in this study was trained on
the BreakHis and BACH datasets, which are relatively small.

For visual analysis, we randomly select five query images from the BreakHis
and BACH test sets (four from the BreakHis with different magnifications and
one from the BACH dataset) to visualize the model output (top five ranks).
Figure 5 depicts the query images and model output. As the results show, the
model is capable of providing relevant images (with identical classes as the
query images) for most of the samples.

| Query | Rank 1 | Rank 2 | Rank 3 | Rank 4 | Rank 5
---|---|---|---|---|---|---
BH (40$\times$) |  |  |  |  |  |
| M | M ✓ | M ✓ | M ✓ | M ✓ | M ✓
BH (100$\times$) |  |  |  |  |  |
| M | BN ✗ | M ✓ | M ✓ | M ✓ | M ✓
BH (200$\times$) |  |  |  |  |  |
| M | M ✓ | M ✓ | M ✓ | BN ✗ | BN ✗
BH (400$\times$) |  |  |  |  |  |
| M | M ✓ | M ✓ | M ✓ | M ✓ | M ✓
BACH (200$\times$) |  |  |  |  |  |
| N | N ✓ | N ✓ | N ✓ | N ✓ | N ✓
Figure 5: Five sample queries from the BreakHis and BACH datasets and their
top similar retrieved images. Successful and unsuccessful retrieval are
denoted by (✓) and (✗), respectively. M: Malignant class, BN: Benign class, N:
Normal class, BH: BreakHis

As suggested in [61], we also compare the embedding vectors of some example
image pairs with different degrees of similarity in Figure 6. As shown in the
figure, the embedding vectors for similar image pairs are very close to each
other. In contrast, the embedding vectors for dissimilar pairs are separable.

Figure 6: Comparison of embedding values for four image pairs. Similar pairs
refer to images within the same class, and dissimilar pairs refer to images in
different classes. The distances between the embedding of the two images in
each image pair are shown on the right side of the figure. For each image
pair, the left and right image embeddings are depicted in black and red,
respectively.

##  5 Conclusion & Future Works

Recently, there has been a growing trend in utilizing GNNs to address
challenges in analyzing histopathology images. This research introduces a new
GNN-based approach specifically designed for breast histopathology image
retrieval. We incorporated both attention mechanisms and adversarial training
in a variational graph auto-encoder model to enhance retrieval performance.
Moreover, instead of using conventional pre-trained models as the node feature
extractor, we employed the CCL feature extractor, specifically pre-trained on
histological images. Our results confirm that incorporating both attention
mechanisms and adversarial training consistently exhibits superior performance
compared to other GNN variants. Additionally, using the CCL feature extractor
further boosts performance in our experiments. While our approach delivers
promising results for breast histology image retrieval, certain aspects can be
considered for further investigation in future studies.

Transformers-based models have shown impressive performance in various
computer vision tasks. Some Transformer-based GNNs, such as graph transformer
networks [62], have been developed for tasks such as node classification. Such
models can be adapted and employed for the medical image retrieval task as
well.

While we use FLANN to build the graph in our model, other state-of-the-art ANN
methods and libraries, such as Faiss library [63] or HNSW method [9] can be
exploited to enhance performance. However, it should be noted that most of
these methods are slower in comparison to FLANN [37].

Another potential enhancement for the model is replacing the inner product
decoder with a non-linear neural network, a topic that can be addressed in
future studies.

Finally, using well-known boosting techniques, such as test-time augmentation
or ensembling, has been shown to enhance performance for tasks such as medical
image classification or segmentation [64, 65, 66]. Such techniques can also be
incorporated into the workflow, and their impact on the retrieval performance
can be investigated.

## Acknowledgments

This project is partially supported by the Ernst Mach Grant (reference number:
MPC-2023-00569).

## Conflict of Interest

The authors declare that they have no known competing financial interests or
personal relationships that could have appeared to influence the work reported
in this paper.

## Credit Authorship Contribution Statement

Nematollah Saeidi: Conceptualization, Methodology, Investigation, Software,
Writing – original draft. Hossein Karshenas: Supervision. Bijan Shoushtarian:
Supervision. Sepideh Hatamikia: Supervision. Ramona Woitek: Supervision.
Amirreza Mahbod: Conceptualization, Validation, Review & editing, Supervision.

## References

  * [1] R. Rashmi, K. Prasad, C. B. K. Udupa, Breast histopathological image analysis using image processing techniques for diagnostic puposes: A methodological review., Journal of medical systems 46 (1) (2021) 7.  doi:https://doi.org/10.1007/s10916-021-01786-9.
  * [2] M. Arnold, E. Morgan, H. Rumgay, A. Mafra, D. Singh, M. Laversanne, J. Vignat, J. R. Gralow, F. Cardoso, S. Siesling, I. Soerjomataram, Current and future burden of breast cancer: Global statistics for 2020 and 2040, The Breast 66 (2022) 15–23.  doi:https://doi.org/10.1016/j.breast.2022.08.010.
  * [3] D. S. Dizon, A. H. Kamal, Cancer statistics 2024: All hands on deck, CA: A Cancer Journal for Clinicians 74 (1) (2024) 8–9.  doi:https://doi.org/10.3322/caac.21824.
  * [4] A. E. Minarno, K. M. Ghufron, T. S. Sabrila, L. Husniah, F. D. S. Sumadi, CNN based autoencoder application in breast cancer image retrieval, in: International Seminar on Intelligent Technology and Its Applications, 2021, pp. 29–34.  doi:https://doi.org/10.1109/ISITIA52817.2021.9502205.
  * [5] H. Burstein, G. Curigliano, S. Loibl, P. Dubsky, M. Gnant, P. Poortmans, M. Colleoni, C. Denkert, M. Piccart-Gebhart, M. Regan, H.-J. Senn, E. Winer, B. Thurlimann, Estimating the benefits of therapy for early-stage breast cancer: the st. gallen international consensus guidelines for the primary therapy of early breast cancer 2019, Annals of Oncology 30 (10) (2019) 1541–1557.  doi:https://doi.org/10.1093/annonc/mdz235.
  * [6] Z. Tabatabaei, A. Colomer, K. Engan, J. Oliver, V. Naranjo, Residual block convolutional auto encoder in content-based medical image retrieval, in: IEEE 14th Image, Video, and Multidimensional Signal Processing Workshop, 2022, pp. 1–5.  doi:https://doi.org/10.1109/IVMSP54334.2022.9816325.
  * [7] S. Fuster, F. Khoraminia, U. Kiraz, N. Kanwal, V. Kvikstad, T. Eftestøl, T. C. Zuiverloon, E. A. Janssen, K. Engan, Invasive cancerous area detection in non-muscle invasive bladder cancer whole slide images, in: IEEE 14th Image, Video, and Multidimensional Signal Processing Workshop, 2022, pp. 1–5.  doi:https://doi.org/10.1109/IVMSP54334.2022.9816352.
  * [8] D. Agrawal, A. Agarwal, D. K. Sharma, Content-based image retrieval (cbir): A review, in: P. K. Singh, Y. Singh, J. K. Chhabra, Z. Illés, C. Verma (Eds.), Recent Innovations in Computing, Springer Singapore, Singapore, 2022, pp. 439–452.  doi:https://doi.org/10.1007/978-981-16-8892-8_33.
  * [9] Y. A. Malkov, D. A. Yashunin, Efficient and robust approximate nearest neighbor search using hierarchical navigable small world graphs, IEEE Transactions on Pattern Analysis and Machine Intelligence 42 (4) (2020) 824–836.  doi:https://doi.org/10.1109/TPAMI.2018.2889473.
  * [10] L. Z, Z. X, M. H, Z. S, Large-scale retrieval for medical image analytics: A comprehensive review, Medical image analysis 43 (2018) 66–84.  doi:https://doi.org/10.1016/J.MEDIA.2017.09.007.
  * [11] N. Hegde, J. D. Hipp, Y. Liu, M. Emmert-Buck, E. Reif, D. Smilkov, M. Terry, C. J. Cai, M. B. Amin, C. H. Mermel, P. Q. Nelson, L. H. Peng, G. S. Corrado, M. C. Stumpe, Similar image search for histopathology: SMILY, npj Digital Medicine 2 (1) (2019).  doi:https://doi.org/10.1038/s41746-019-0131-z.
  * [12] J. Silva-Rodríguez, A. Colomer, M. A. Sales, R. Molina, V. Naranjo, Going deeper through the Gleason scoring scale: An automatic end-to-end system for histology prostate grading and cribriform pattern detection, Computer Methods and Programs in Biomedicine 195 (2020).  doi:https://doi.org/10.1016/j.cmpb.2020.105637.
  * [13] Y. Zheng, B. Jiang, J. Shi, H. Zhang, F. Xie, Encoding histopathological wsis using gnn for scalable diagnostically relevant regions retrieval, in: D. Shen, T. Liu, T. M. Peters, L. H. Staib, C. Essert, S. Zhou, P.-T. Yap, A. Khan (Eds.), Medical Image Computing and Computer Assisted Intervention, Springer International Publishing, Cham, 2019, pp. 550–558.  doi:https://doi.org/10.1007/978-3-030-32239-7_61.
  * [14] C. Chen, M. Y. Lu, D. F. Williamson, T. Y. Chen, A. J. Schaumberg, F. Mahmood, Fast and scalable search of whole-slide images via self-supervised deep learning, Nature Biomedical Engineering 6 (12) (2022).  doi:https://doi.org/10.1038/s41551-022-00929-8.
  * [15] S. Tang, A. Modi, M. Sjoding, J. Wiens, Clinician-in-the-loop decision making: Reinforcement learning with near-optimal set-valued policies, in: International Conference on Machine Learning, PMLR, 2020, pp. 9387–9396.
  * [16] J. Herington, M. D. McCradden, K. Creel, R. Boellaard, E. C. Jones, A. K. Jha, A. Rahmim, P. J. Scott, J. J. Sunderland, R. L. Wahl, S. Zuehlsdorff, B. Saboury, Ethical considerations for artificial intelligence in medical imaging: Data collection, development, and evaluation, Journal of Nuclear Medicine 64 (12) (2023) 1848–1854.  doi:https://doi.org/10.2967/jnumed.123.266080.
  * [17] R. Yamashita, M. Nishio, R. K. G. Do, K. Togashi, Convolutional neural networks: an overview and application in radiology (2018).  doi:https://doi.org/10.1007/s13244-018-0639-9.
  * [18] K. Simonyan, A. Zisserman, Very deep convolutional networks for large-scale image recognition, Computational and Biological Learning Society, 2015, pp. 1–14.
  * [19] M. Tan, Q. Le, EfficientNetV2: Smaller models and faster training, in: M. Meila, T. Zhang (Eds.), Proceedings of the 38th International Conference on Machine Learning, Vol. 139 of Proceedings of Machine Learning Research, PMLR, 2021, pp. 10096–10106.
  * [20] T. N. Kipf, M. Welling, Variational graph auto-encoders, arXiv preprint arXiv:1611.07308 (2016).
  * [21] S. Denner, D. Zimmerer, D. Bounias, M. Bujotzek, S. Xiao, L. Kausch, P. Schader, T. Penzkofer, P. F. Jäger, K. Maier-Hein, Leveraging foundation models for content-based medical image retrieval in radiology, arXiv preprint arXiv:2403.06567 (2024).
  * [22] Z. Xiong, J. Cai, Multi-scale Graph Convolutional Networks with Self-Attention, arXiv preprint arXiv:2112.03262 (2021).
  * [23] S. Pan, R. Hu, G. Long, J. Jiang, L. Yao, C. Zhang, Adversarially regularized graph autoencoder for graph embedding, in: IJCAI International Joint Conference on Artificial Intelligence, Vol. 2018-July, 2018.  doi:https://doi.org/10.24963/ijcai.2018/362.
  * [24] M. Zheng, J. Xu, Y. Shen, C. Tian, J. Li, L. Fei, M. Zong, X. Liu, Attention-based CNNs for Image Classification: A Survey, in: Journal of Physics: Conference Series, Vol. 2171, 2022.  doi:https://doi.org/10.1088/1742-6596/2171/1/012068.
  * [25] H. Xia, S. Shao, C. Hu, R. Zhang, T. Qiu, F. Xiao, Robust clustering model based on attention mechanism and graph convolutional network, IEEE Transactions on Knowledge and Data Engineering 35 (5) (2023) 5203–5215.  doi:https://doi.org/10.1109/TKDE.2022.3150300.
  * [26] Z. Weng, W. Zhang, W. Dou, Adversarial Attention-Based Variational Graph Autoencoder, IEEE Access 8 (2020).  doi:https://doi.org/10.1109/ACCESS.2020.3018033.
  * [27] J. Xiao, Q. Dai, X. Xie, J. Lam, K. W. Kwok, Adversarially regularized graph attention networks for inductive learning on partially labeled graphs, Knowledge-Based Systems 268 (2023).  doi:https://doi.org/10.1016/j.knosys.2023.110456.
  * [28] X. Wang, Y. Du, S. Yang, J. Zhang, M. Wang, J. Zhang, W. Yang, J. Huang, X. Han, RetCCL: Clustering-guided contrastive learning for whole-slide image retrieval, Medical Image Analysis 83 (2023).  doi:https://doi.org/10.1016/j.media.2022.102645.
  * [29] F. A. Spanhol, L. S. Oliveira, C. Petitjean, L. Heutte, A Dataset for Breast Cancer Histopathological Image Classification, IEEE Transactions on Biomedical Engineering 63 (7) (2016).  doi:https://doi.org/10.1109/TBME.2015.2496264.
  * [30] G. Aresta, T. Araújo, S. Kwok, S. S. Chennamsetty, M. Safwan, V. Alex, B. Marami, M. Prastawa, M. Chan, M. Donovan, G. Fernandez, J. Zeineh, M. Kohl, C. Walz, F. Ludwig, S. Braunewell, M. Baust, Q. D. Vu, M. N. N. To, E. Kim, J. T. Kwak, S. Galal, V. Sanchez-Freire, N. Brancati, M. Frucci, D. Riccio, Y. Wang, L. Sun, K. Ma, J. Fang, I. Kone, L. Boulmane, A. Campilho, C. Eloy, A. Polónia, P. Aguiar, BACH: Grand challenge on breast cancer histology images, Medical Image Analysis 56 (2019).  doi:https://doi.org/10.1016/j.media.2019.05.010.
  * [31] R. Vishraj, S. Gupta, S. Singh, A comprehensive review of content-based image retrieval systems using deep learning and hand-crafted features in medical imaging: Research challenges and future directions, Computers and Electrical Engineering 104 (2022).  doi:https://doi.org/10.1016/j.compeleceng.2022.108450.
  * [32] S. Graham, F. Minhas, M. Bilal, M. Ali, Y. W. Tsang, M. Eastwood, N. Wahab, M. Jahanifar, E. Hero, K. Dodd, H. Sahota, S. Wu, W. Lu, A. Azam, K. Benes, M. Nimir, K. Hewitt, A. Bhalerao, A. Robinson, H. Eldaly, S. E. Raza, K. Gopalakrishnan, D. Snead, N. Rajpoot, Screening of normal endoscopic large bowel biopsies with interpretable graph learning: A retrospective study, Gut 72 (9) (2023).  doi:https://doi.org/10.1136/gutjnl-2023-329512.
  * [33] N. T. Singh, C. Kaur, A. Chaudhary, S. Goyal, Preprocessing of medical images using deep learning: A comprehensive review, in: International Conference on Augmented Intelligence and Sustainable Systems, 2023, pp. 521–527.  doi:https://doi.org/10.1109/ICAISS58487.2023.10250462.
  * [34] D. Murcia-Gómez, I. Rojas-Valenzuela, O. Valenzuela, Impact of image preprocessing methods and deep learning models for classifying histopathological breast cancer images, Applied Sciences 12 (22) (2022).  doi:https://doi.org/10.3390/app122211375.
  * [35] C. C. Ukwuoma, M. A. Hossain, J. K. Jackson, G. U. Nneji, H. N. Monday, Z. Qin, Multi-Classification of Breast Cancer Lesions in Histopathological Images Using DEEP_Pachi: Multiple Self-Attention Head, Diagnostics 12 (5) (2022).  doi:https://doi.org/10.3390/diagnostics12051152.
  * [36] A. Mahbod, G. Schaefer, R. Ecker, I. Ellinger, Pollen grain microscopic image classification using an ensemble of fine-tuned deep convolutional neural networks, in: International Conference on Pattern Recognition, Springer, 2021, pp. 344–356.  doi:https://doi.org/10.1007/978-3-030-68763-2_26.
  * [37] D. A. Suju, H. Jose, FLANN: Fast approximate nearest neighbour search algorithm for elucidating human-wildlife conflicts in forest areas, in: 2017 4th International Conference on Signal Processing, Communication and Networking, ICSCN 2017, 2017.  doi:https://doi.org/10.1109/ICSCN.2017.8085676.
  * [38] S. Kalra, H. R. Tizhoosh, C. Choi, S. Shah, P. Diamandis, C. J. Campbell, L. Pantanowitz, Yottixel – An Image Search Engine for Large Archives of Histopathology Whole Slide Images, Medical Image Analysis 65 (2020) 101757.  doi:https://doi.org/10.1016/J.MEDIA.2020.101757.
  * [39] J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, L. Fei-Fei, ImageNet: A large-scale hierarchical image database, in: IEEE Conference on Computer Vision and Pattern Recognition, 2009, pp. 248–255.  doi:https://doi.org/10.1109/CVPR.2009.5206848.
  * [40] D. Ahmedt-Aristizabal, M. A. Armin, S. Denman, C. Fookes, L. Petersson, Graph-based deep learning for medical diagnosis and analysis: Past, present and future, Sensors 21 (14) (2021).  doi:https://doi.org/10.3390/s21144758.
  * [41] D. Anand, S. Gadiya, A. Sethi, Histographs: graphs in histopathology, in: J. E. Tomaszewski, A. D. Ward (Eds.), Medical Imaging 2020: Digital Pathology, Vol. 11320, SPIE, 2020, p. 113200O.  doi:https://doi.org/10.1117/12.2550114.
  * [42] D. Ahmedt-Aristizabal, M. A. Armin, S. Denman, C. Fookes, L. Petersson, A survey on graph-based deep learning for computational histopathology, Computerized Medical Imaging and Graphics 95 (2022) 102027.  doi:https://doi.org/10.1016/j.compmedimag.2021.102027.
  * [43] S. Graham, Q. D. Vu, S. E. A. Raza, A. Azam, Y. W. Tsang, J. T. Kwak, N. Rajpoot, Hover-Net: Simultaneous segmentation and classification of nuclei in multi-tissue histology images, Medical Image Analysis 58 (2019) 101563\.  doi:https://doi.org/10.1016/j.media.2019.101563.
  * [44] A. Mahbod, G. Schaefer, G. Dorffner, S. Hatamikia, R. Ecker, I. Ellinger, A dual decoder u-net-based model for nuclei instance segmentation in hematoxylin and eosin-stained histological images, Frontiers in Medicine 9 (2022).  doi:https://doi.org/10.3389/fmed.2022.978146.
  * [45] W. Hamilton, Z. Ying, J. Leskovec, Inductive representation learning on large graphs, Advances in neural information processing systems 30 (2017).
  * [46] P. Veličković, G. Cucurull, A. Casanova, A. Romero, P. Lio, Y. Bengio, Graph attention networks, arXiv preprint arXiv:1710.10903 (2017).
  * [47] H. Sahbi, Learning laplacians in chebyshev graph convolutional networks, in: Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021, pp. 2064–2075.
  * [48] Y. Zheng, Z. Jiang, H. Zhang, F. Xie, Y. Ma, H. Shi, Y. Zhao, Histopathological Whole Slide Image Analysis Using Context-Based CBIR, IEEE Transactions on Medical Imaging 37 (7) (2018).  doi:https://doi.org/10.1109/TMI.2018.2796130.
  * [49] M. Sandler, A. Howard, M. Zhu, A. Zhmoginov, L.-C. Chen, MobileNetV2: Inverted residuals and linear bottlenecks, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018.  doi:https://doi.org/10.1109/CVPR.2018.00474.
  * [50] G. Huang, Z. Liu, L. Van Der Maaten, K. Q. Weinberger, Densely connected convolutional networks, in: IEEE Conference on Computer Vision and Pattern Recognition, 2017, pp. 2261–2269.  doi:https://doi.org/10.1109/CVPR.2017.243.
  * [51] F. Chollet, Xception: Deep learning with depthwise separable convolutions, in: IEEE Conference on Computer Vision and Pattern Recognition, 2017, pp. 1800–1807.  doi:https://doi.org/10.1109/CVPR.2017.195.
  * [52] C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens, Z. Wojna, Rethinking the inception architecture for computer vision, in: IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp. 2818–2826.  doi:https://doi.org/10.1109/CVPR.2016.308.
  * [53] B. Zoph, V. Vasudevan, J. Shlens, Q. V. Le, Learning transferable architectures for scalable image recognition, in: IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018, pp. 8697–8710.  doi:https://doi.org/10.1109/CVPR.2018.00907.
  * [54] M. Adnan, S. Kalra, H. R. Tizhoosh, Representation Learning of Histopathology Images using Graph Neural Networks, IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops 2020-June (2020) 4254–4261.  arXiv:2004.07399, doi:https://doi.org/10.1109/CVPRW50498.2020.00502.
  * [55] M. Lin, K. Wen, X. Zhu, H. Zhao, X. Sun, Graph Autoencoder with Preserving Node Attribute Similarity, Entropy 25 (4) (2023).  doi:https://doi.org/10.3390/e25040567.
  * [56] Y. Zheng, Z. Jiang, F. Xie, J. Shi, H. Zhang, J. Huai, M. Cao, X. Yang, Diagnostic Regions Attention Network (DRA-Net) for Histopathology WSI Recommendation and Retrieval, IEEE Transactions on Medical Imaging 40 (3) (2021).  doi:https://doi.org/10.1109/TMI.2020.3046636.
  * [57] Z. Gao, Z. Lu, J. Wang, S. Ying, J. Shi, A Convolutional Neural Network and Graph Convolutional Network Based Framework for Classification of Breast Histopathological Images, IEEE Journal of Biomedical and Health Informatics 26 (7) (2022).  doi:https://doi.org/10.1109/JBHI.2022.3153671.
  * [58] M. Aumüller, E. Bernhardsson, A. Faithfull, Ann-benchmarks: A benchmarking tool for approximate nearest neighbor algorithms, in: Similarity Search and Applications: 10th International Conference, SISAP 2017, Munich, Germany, October 4-6, 2017, Proceedings 10, Springer, 2017, pp. 34–49.
  * [59] Y. Zheng, Z. Jiang, J. Shi, F. Xie, H. Zhang, W. Luo, D. Hu, S. Sun, Z. Jiang, C. Xue, Encoding histopathology whole slide images with location-aware graphs for diagnostically relevant regions retrieval, Medical Image Analysis 76 (2022).  doi:https://doi.org/10.1016/j.media.2021.102308.
  * [60] Z. Wu, S. Pan, F. Chen, G. Long, C. Zhang, P. S. Yu, A comprehensive survey on graph neural networks, IEEE Transactions on Neural Networks and Learning Systems 32 (1) (2021).  doi:https://doi.org/10.1109/tnnls.2020.2978386.
  * [61] E. Parcham, M. Ilbeygi, M. Amini, CBCapsNet: A novel writer-independent offline signature verification model using a cnn-based architecture and capsule neural networks, Expert Systems with Applications 185 (2021) 115649.  doi:https://doi.org/10.1016/j.eswa.2021.115649.
  * [62] S. Yun, M. Jeong, S. Yoo, S. Lee, S. S. Yi, R. Kim, J. Kang, H. J. Kim, Graph transformer networks: Learning meta-path graphs to improve gnns, Neural Networks 153 (2022) 104–119.  doi:https://doi.org/10.1016/j.neunet.2022.05.026.
  * [63] J. Johnson, M. Douze, H. Jégou, Billion-scale similarity search with GPUs, IEEE Transactions on Big Data 7 (3) (2021) 535–547.  doi:https://doi.org/10.1109/TBDATA.2019.2921572.
  * [64] N. Moshkov, B. Mathe, A. Kertesz-Farkas, R. Hollandi, P. Horvath, Test-time augmentation for deep learning-based cell segmentation on microscopy images, Scientific Reports 10 (1) (2020) 1–7.  doi:https://doi.org/10.1038/s41598-020-61808-3.
  * [65] A. Mahbod, G. Dorffner, I. Ellinger, R. Woitek, S. Hatamikia, Improving generalization capability of deep learning-based nuclei instance segmentation by non-deterministic train time and deterministic test time stain normalization, Computational and Structural Biotechnology Journal 23 (2024) 669–678.  doi:https://doi.org/10.1016/j.csbj.2023.12.042.
  * [66] B. Bancher, A. Mahbod, I. Ellinger, R. Ecker, G. Dorffner, Improving mask r-cnn for nuclei instance segmentation in hematoxylin & eosin-stained histological images, in: MICCAI Workshop on Computational Pathology, Vol. 156, 2021, pp. 20–35.


Due to the length of the paper, it is a pending task to investigate the
version of Theorem 3.25 for two-step, three-step, and $n$-step iterated Ore
extensions. Also, the question on cv-polynomials and homomorphisms by
considering the nilpotent elements of the coefficient ring $R$ of the iterated
Ore extension, such as in the works of Rimmer, Ferrero and Kishimoto, and
Kikumasa, it is also of our interest. On the other hand, as a possible work
concerning morphisms between noncommutative rings related with Ore extensions,
we consider Double Ore extensions introduced and characterized by Zhang and
Zhang [46, 47]. Our interest is due to Carvalho et al. [6] who described those
double Ore extensions that can be presented as two-step iterated Ore
extensions. In this way, it is natural to investigate the notions of
homomorphism and cv-polynomial in the setting of these double extensions
compare them with the results appearing in [6].

## References

  * [1] J. Alev and F. Dumas, Invariants du corps de Weyl sous l’action de groupes finis, Comm. Algebra 25(5) (1997) 1655–1672.
  * [2] E. P. Armendariz, H. K. Koo and J. K. Park, Isomorphic Ore extensions, Comm. Algebra 15(12) (1987) 2633–2652.
  * [3] J. W. Brewer and E. A. Rutter, Isomorphic polynomial rings, Arch. Math. (Basel) 23(1) (1972) 484–488.
  * [4] K. Brown and K. R. Goodearl, Lectures on Algebraic Quantum Groups, Advanced Courses in Mathematics – CRM Barcelona (Birkäuser, Basel 2002).
  * [5] J. Bueso, J. Gómez-Torrecillas and A. Verschoren, Algorithmic Methods in Non-Commutative Algebra. Applications to Quantum Groups, Mathematical Modelling: Theory and Applications (Springer Dordrecht, 2003).
  * [6] B. Carvalho, S. A. Lopes and J. Matczuk, Double Ore extensions versus iterated Ore extensions, Comm. Algebra 39(8) (2011) 2838–2848.
  * [7] C. L. Chuang, T. K. Lee, C. K. Liu and Y. T. Tsai, Higher derivations of Ore extensions. Israel J. Math. 175(1) (2010) 157–178.
  * [8] C. L. Chuang and Y. T. Tsai, On the structure of semi-invariant polynomials in Ore extensions, J. Algebra 322(7) (2009) 2464–2491.
  * [9] C. L. Chuang and Y. T. Tsai, Higher derivations of Ore extensions by $q$-skew derivations, J. Pure Appl. Algebra 214(10) (2010) 1778–1786.
  * [10] C. L. Chuang, Automorphisms of Ore extensions, Israel J. Math. 197(1) (2013) 437–452.
  * [11] J. H. Chun, Isomorphic Ore Extensions of Monomorphism Type, Commun. Korean Math. Soc. 8(3) (1993) 361–371.
  * [12] P. M. Cohn, Skew Fields: Theory of General Division Rings, Encyclopedia of Mathematics and its Applications, Series Number 57 (Cambridge University Press 1995).
  * [13] D. B. Coleman and E. E. Enochs, Isomorphic polynomial rings, Proc. Amer. Math. Soc. 27(2) (1971) 247–252.
  * [14] W. Fajardo, C. Gallego, O. Lezama, A. Reyes, H. Suárez and H. Venegas, Skew PBW Extensions. Ring and Module-theoretic Properties, Matrix and Gröbner Methods, and Applications, Algebra and Applications (Springer, Cham 2020).
  * [15] M. Ferrero and K. Kishimoto, On automorphisms of skew polynomial rings of derivation type, Math. J. Okayama Univ. 22 (1980) 21–26.
  * [16] J. Gaddis, Isomorphisms of Some Quantum Spaces, in Ring Theory and Its Applications, Ring Theory Session in Honor of T. Y. Lam on his 70th Birthday 31st Ohio State-Denison Mathematics Conference May 25-27, 2012, The Ohio State University, Columbus, OH, eds. D. Van Huynh, S. K. Jain, S. R. López-Permouth S. Tariq Rizvi and C. S. Roman (Contemporary Mathematics 609, American Mathematical Society 2014 pp. 107–116).
  * [17] R. Gilmer, JR, $R$-automorphisms of $R[x]$, Proc. London Math. Soc. s3-18(2) (1968) 328–336.
  * [18] K. R. Goodearl and E. S. Letzter, Prime ideals in Skew and $q$-Skew Polynomial rings, Memoirs of the American Mathematical Society, Vol. 109, 521 (American Mathematical Society, 1994).
  * [19] K. R. Goodearl and R. B. Warfield Jr., An Introduction to Noncommutative Noetherian Rings (Cambridge University Press, 2004).
  * [20] H. Haynal, PI degree parity in $q$-skew polynomial rings, J. Algebra 319(10) (2008) 4199–4221.
  * [21] D. Jordan, The Graded Algebra Generated by Two Eulerian Derivatives, Algebr. Represent. Theory 4(3) (2001) 249–275.
  * [22] I. Kikumasa, Automorphisms of a Certain Skew Polynomial Ring of Derivation Type, Canad. J. Math. 42(6) (1990) 949–958.
  * [23] T. Y. Lam and A. Leroy, Vandermonde and Wronskian matrices over division rings, J. Algebra 119(2) (1988) 308–336.
  * [24] T. Y. Lam and A. Leroy, Homomorphisms Between Ore Extensions, In Azumaya Algebras, Actions and Modules, volume 124 of Proceedings of a Conference in Honor of Goro Azumaya’s Seventieth Birthday, May 23–27, 1990, eds. D. Haile and J. Osterburg, editors (Contemporary Mathematics, American Mathematical Society 1992 pp. 83–110).
  * [25] T. Y. Lam and A. Leroy, Hilbert 90 theorems over division rings, Trans. Amer. Math. Soc. 345(2) (1994) 595–622.
  * [26] T. Y. Lam, K. H. Leung, A. Leroy and J. Matczuk, Invariant and semi-invariant polynomials in skew polynomial rings, in Ring Theory, Israel Mathematical Conference Proceedings, ed. L. Rowen (Weismann Science 1, Israel, 1989)
  * [27] A. Leroy, $S$-derivations algébriques sur les anneaux premiers (in French), in Ring Theory, Antwerpen, 1985 (Lecture Notes in Math. 1197, Springer-Verlag, Berlin 1986 pp. 114–120).
  * [28] A. Leroy and J. Matczuk, Quelques remarques a propos des s-derivations, Comm. Algebra 13(6) (1985) 1229–1244.
  * [29] A. Leroy. Pseudo linear transformations and evaluation in Ore extensions, Bull. Belg. Math. Soc. 2(3) (1995) 321–347.
  * [30] A. Leroy, Noncommutative polynomial maps, J. Algebra Appl. 11(04) (2012) 1250076.
  * [31] T. Mansour and M. Schork, Commutation Relations, Normal ordering, and Stirling Numbers (Chapman and Hall/CRC: New York, 2015).
  * [32] U. Martínez-Peñas, Skew and linearized Reed-Solomon codes and maximum sum rank distance codes over any division ring, J. Algebra 504 (2018) 587–612.
  * [33] U. Martínez-Peñas and F. R. Kschischang, Evaluation and interpolation over multivariate skew polynomial rings, J. Algebra 525 (2019) 111–139.
  * [34] J. C. McConnell and J. C. Robson, Noncommutative Noetherian Rings, Graduate Studies in Mathematics (American Mathematical Society, 2001).
  * [35] A. Niño, M. C. Ramírez, and A. Reyes, Associated prime ideals over skew PBW extensions, Comm. Algebra 48(12) (2020) 5038–5055.
  * [36] A. Niño and A. Reyes, On centralizers and pseudo-multidegree functions for noncommutative rings having PBW bases, J. Algebra Appl. (2023) https://doi.org/10.1142/S0219498825501099
  * [37] O. Ore, Linear Equations in Non-commutative Fields, Ann. of Math. (2). 32(3) (1931) 463–477.
  * [38] O. Ore, Theory of Non-Commutative Polynomials, Ann. of Math. (2). 34(3) (1933) 480–508.
  * [39] Parmenter, M. Automorphisms of Twisted Polynomial Rings.
  * [40] A. Reyes and C. Sarmiento, On the differential smoothness of 3-dimensional skew polynomial algebras and diffusion algebras, Internat. J. Algebra Comput. 32(3) (2022) 529–559.
  * [41] A. Reyes and H. Suárez, Skew Poincaré-Birkhoff-Witt extensions over weak compatible rings, J. Algebra Appl. 19(12) (2020) 2050225.
  * [42] M. Rimmer, Isomorphisms between skew polynomial rings, J. Austral. Math. Soc. 25(3) (1978) 314–321.
  * [43] E. N. Shirikov, Two-generated graded algebras, Algebra Discrete Math. 4(3) (2005) 60–84.
  * [44] E. N. Shirikov, The Jordan plane over a field of positive characteristic, Math. Notes 82(2) (2007) 238–256.
  * [45] E. N. Shirikov, Jordanian plane, J. Math. Sci. (N.Y.) 154(2) (2008) 270–278.
  * [46] J. J. Zhang and J. Zhang, Double Ore extensions, J. Pure Appl. Algebra 212(12) (2008) 2668–2690.
  * [47] J. J. Zhang and J. Zhang J, Double extension regular algebras of type (14641), J. Algebra 322(2) (2009) 373–409.