# Biologically Inspired Deep Residual Networks for Computer Vision
Applications ††thanks: Citation: Authors. Title. Pages…. DOI:000000/11111.

Prathibha Varghese
Research Scholar,
Electronics & Communication Department,
Noorul Islam Centre for Higher Education,
Tamil Nadu, India
<EMAIL_ADDRESS>
&Dr. G. Arockia Selva Saroja
Associate Professor,
Department Electronics & Communication Engineering,
Noorul Islam Centre for Higher Education,
Tamil Nadu, India
<EMAIL_ADDRESS>

###### Abstract

Deep neural network has been ensured as a key technology in the field of many
challenging and vigorously researched computer vision tasks. Furthermore,
classical ResNet is thought to be a state-of-the-art convolutional neural
network (CNN) and was observed to capture features which can have good
generalization ability. In this work, we propose a biologically inspired deep
residual neural network where the hexagonal convolutions are introduced along
the skip connections. The performance of different ResNet variants using
square and hexagonal convolution are evaluated with the competitive training
strategy mentioned by [1]. We show that the proposed approach advances the
baseline image classification accuracy of vanilla ResNet architectures on
CIFAR-10 and the same was observed over multiple subsets of the ImageNet 2012
dataset. We observed an average improvement by 1.35% and 0.48% on baseline
top-1 accuracies for ImageNet 2012 and CIFAR-10, respectively. The proposed
biologically inspired deep residual networks were observed to have improved
generalized performance and this could be a potential research direction to
improve the discriminative ability of state-of-the-art image classification
networks.

_K_ eywords INDEX TERMS : ResNet $\cdot$ Deep learning $\cdot$ Hexagonal pixel
Square pixel $\cdot$ Convolutional neural networks

##  1 Introduction

Deep convolutional neural networks have made significant progress in the field
of computer vision [2, 3, 4]. The non-linearities in high dimensional image
data can be better captured with increased depth of networks. However, network
performance deteriorates with increased depth of networks due to the following
reasons:

  * •

Vanishing gradient: Using chain rule in backpropagation, gradients (partial
derivatives) are calculated. During the training process, depending upon the
activation function, gradients decrease exponentially ie., the gradient of the
network become smaller and smaller [5, 6].

  * •

Harder optimization: Adding more layers to the neural networks increases more
parameters, thereby leading to increased training errors [7].

The residual network (ResNet) [1] architecture explicitly addresses the
problem of vanishing gradients by including multiple skip connections along
with the main convolution layers. These connections are contrary to the
regular convolution layers and allow for easy backpropagation of the gradients
without any decay. Also, due to the inherent advantages of skip-connections
and its generalisation ability, ResNet is widely used as backbone feature
extractor in multiple computer vision applications [8, 9].

|
---|---
(a) | (b)
Figure 1: Natural hexagonal structures [10]. (a) Football, and (b) Honeycomb.

Majority of the neural network computations are performed on Euclidean data.
However, there has been a growing interest in geometry perception of
relational data and usage of those insights to learn from good non-Euclidean
representations. As shown in Fig. 1, hexagonal patterns are very commonly
observed in nature. Along with football and honeycomb structures, there are
few others such as bees compound eye, bubble raft, pencil base etc. Most
importantly, structure of human fovea also resembles hexagonal tessellation.
Fig. 2 shows the photoreceptor mosaic in the fovea captured by Nomarski
differential interference contrast microscopy (NDIC) at different levels of
inner segments throughout the retina [11]. Fig 2 (a) shows the profiles of
rods while the Fig. 2 (b) shows the profiles of foveal cones with small
intervening profiles of rods which are three times smaller than cones at
retinal locations.

On the back of the eye, the retina is a thin membrane (tissue) that aims to
capture light data and transforms them as neural signals which is further sent
to the brain for visual recognition. As shown in Fig. 2, the Fovea is a small
retinal area. It’s structural architecture is of a highly densely conical
shape and placed in a hexagonal position which captures sharper vision than
the square grating layout [12, 13]. Adding to that [14] has demonstrated that
there is an increase of 13.4$\%$ in transformation efficiency for the
hexagonal pixels when compared to pixels arranged in square tessellation. This
implies that hexagonal sampling requires 13.4$\%$ fewer pixels than when
compared to square tessellation to capture same amount of data. This results
in fewer storage requirements and less computational time. [12] has shown that
the hexagonal tessellation captures more information with identical number of
sampling points when compared to the data captured on square tessellation.
Apart from being close to human vision, as illustrated in [14, 15, 12, 13],
the structural arrangement of hexagonal lattice provides added advantage over
the square lattice counterpart. The authors in [12] and [13] show that there
is an improved precision on the detection of circular and near circular edges
of hexagonal images.

Any attempts in building the imaging systems whose sensing mechanism and data
processing abilities mimic the way human visual system captures and processes
the data will aid in building better systems close to human level performance.
However, due to the ease of practical realisation of mathematical operations
defined on Euclidean space, data captured on square tessellations are more
popular. Hex-mov predictions [16], imaging atmopsheric Cherenkov telescope
(IACT) [17], and ice cube data analysis [18] are some of the works which deal
with the data captured by sensors arranged on hexagonal lattice.

|
---|---
(a) | (b)
Figure 2: Optical section arrangement of the foveal cone mosaic [11]. (a)
Fovea central and (b) Fovea slope.

In this paper, we propose a biologically inspired hybrid architecture Hex-
ResNet to improve the generalisation ability of deep residual networks through
hexagonal convolutional filters. We show that the baseline performance of
ResNet architecture can be increased by incorporating minimal hexagonal
filters along a few of the skip connections. The key highlight of our approach
is to effectively combine the advantages offered by each of the individual
(square and hexagonal) tessellations. Unlike [19] which operate directly on a
hexagonal lattice, we use the implementation of [20] which implements
efficient routines for hexagonal convolution by using an equivalent sum of two
square convolutions. This way, we obtain the gain in performance by using the
efficient computational support defined on square tessellations. Our proposed
architecture has achieved improved validation and testing accuracy as compared
with the several existing ResNet models in terms of Top-1 and Top-5 accuracy.
We also show that this enhanced performance was obtained without significant
increase in computational overhead. We summarise our contributions and
findings as follows:

  * •

We modified the vanilla ResNet by incorporating hex convolutions along a few
of the skip connections and found an improvement in the baseline image
classification accuracy.

  * •

We validated the efficiency of the newly proposed Hex-ResNet architecture, by
performing extensive experiments on CIFAR-10 and ImageNet 2012 benchmark
datasets.

  * •

We show that incorporating hex convolutions along the skip connection paths,
consistently improves the baseline image classification accuracy across
multiple ResNet configurations.

The remainder of this paper is organized as follows: a review of the related
works is given in section 2 which covers all the related works to our
approach. Section 3 initially presents the preliminaries of classical ResNet
architectures, skip connections followed by the details of our proposed Hex-
ResNet architecture. In Section 4, we give the details of experimental results
and training procedures used on our proposed architecture using CIFAR-10 and
ImageNet datasets. We finally conclude with our observations in section 6.

##  2 RELATED WORKS

###  2.1 Residual Networks

The ability of deep residual networks to overcome the vanishing gradient
problem through skip connections made it an ideal candidate as a backbone
feature extractor for various computer vision tasks [1, 9, 21, 22]. [23]
proposed an improved ResNet via adjustable shortcut connections, and designed
a convex k strategy using different region parameters changing rules. [24]
shared competitive training settings and pre-trained models for several ResNet
configurations in the Timm open-source library. Pre-activation ResNets were
introduced in the study [25] by rearranging the elements in the building block
to improve the signal propagation path. All the popular works which used
ResNet as a backbone feature extractor operated on square lattice.

###  2.2 Hexagonal Processing

Data stored on hexagonal lattice was shown to have significant advantage over
the square lattice [26, 27, 28]. However, most of the existing display devices
capture data using pixels arranged in square tessellation due to the ease of
realising many mathematical operations defined on Euclidean geometry. This
current generation devices which capture data on square lattice will not have
the ability to take advantage of multiple benefits offered by hexagonal
lattice such as equidistance, higher angular resolution, smaller quantization
error, greater circular symmetry, and efficient sampling scheme [29, 30, 31].
Also, as shown in Fig. 2, the imaging sensors in human visual system are in
the form of hexagonal tessellation. Hence, for any intelligent visual system
to have human level performance they need to replicate the way data is
captured and processed in human visual system.

There has been a growing interest in building biologically inspired models in
the recent past. B-spline models have been successfully used for tasks such as
interpolation and sampling on square lattices. [32] has introduced hex-splines
for hexagonal sampled data. [33] has reported an improved edge detection
performance for curved objects sampled on hexagonal lattice. [34] has shown an
improvised PET image reconstruction on hexagonal lattice using a filtered back
projection and sparse dictionary based denoising approach. The authors in [35,
36] proposed an improvised image enhancement technique for hexagonal resampled
ultrasound images. While many existing approaches convert the data defined on
square lattice to hexagonal tessellation, there are a few approaches [37, 38,
39] which constructed imaging and display devices which directly captures data
on a hexagonal lattice.

###  2.3 Hexagonal Convolutional Neural Networks

[40] has proposed a deep learning based approach which performs both forward
and backward directly on hexagonal data as input and thus overcoming the
limitation of ZeroOut which tend to add significant computational overhead.
[19] has extended the theory of group equivariant convolutional neural
networks to hexagonal lattice so as to exploit the six-fold symmetries in $p6$
and $p6m$ groups. There are few other works [41, 42] which utilise the square
convolutional routines for hexagonal lattices. Unlike the previous techniques,
the proposed approach is a hybrid technique which use the square equivalent of
hexagonal filters along the skip connections of deep residual networks and
show an improvised performance over the classical ResNet architectures. Our
approach is an attempt to exploit the advantages offered by both square as
well as hexagonal tessellations. Also, we make use of existing square
convolutional routines offered by popular deep learning frameworks like
PyTorch to perform equivalent hexagonal convolutions.

##  3 Proposed approach

In this section, we will initially describe the preliminaries followed by the
details of proposed approach.

###  3.1 Classical Residual Network Architecture

The architecture of various configurations of classical ResNet [1] is
explained in Table 1. Initially, all the images were convolved to 64 layers
using a 7$\times$7 convolution with stride 2. Instead of applying shortcut
connections to a pair of 2$\times$2 filters, they were applied to the pair of
3$\times$3 filters. For relatively small networks like 18 and 34 layers non
bottlenecked blocks were used which consisted of 4 convolution blocks each
having two 3$\times$3 convolutions. For ResNet-18, these convolutions were
repeated twice for each filter map {64, 128, 256, 512} and for 34-layer ResNet
they were repeated in the order {3, 4, 6, 3}. To make ResNet more economical
on large datasets such as ImageNet, a deep bottleneck architecture was used
with 50 and 101 layers. Bottleneck architecture contains a stack of 1$\times$1
conv, 3$\times$3 Conv and a 1$\times$1 Conv repeated. In the last 1$\times$1
convolutions, the output channels are increased four times. For ResNet 50,
these blocks are repeated {3, 4, 6, 3} times and for ResNet 101 these blocks
are repeated as {3, 4, 23, 3}. Option B shortcut is used for increasing
dimensions. After the blocks, an average pool, fully connected layer and,
softmax were also applied.

Table 1: ResNet architectures for ImageNet/CIFAR-10 dataset.

###  3.2 Skip Connections

The residual shortcuts introduced in [1] are meant for the easy back
propagation of gradients and hence should not introduce any significant
computational overload. Based on the input and output dimensions of
image/features, the residual shortcuts are classified into two categories:

|
---|---
(a) | (b)
Figure 3: Fundamental block of ResNet architecture with skip connections. (a)
Identity shortcut [1] and (b) Projection shortcut [43].

  * •

Identity shortcut: Identity shortcut mapping can be used, when the input and
output dimensions match, it just requires simple mathematical additions for
tensors and can be utilised directly. It is really hard to find a direct
mapping from input to output using a standard neural network block. So,
instead of that, the difference between the input and output vectors is
calculated using the residual block. As in [44], identity shortcut is used
along with convolution, batch normalisation, max pooling, and activation
function as shown in Fig. 3 (a). and, fed as input to the add operator. Thus,
having a direct mapping plus the input passed to the activation function.

  * •

Projection shortcut: The projection shortcuts (as shown in Fig. 3 (b)) can be
used to capture the information which has not be derived or extracted along
the main information path in the residual ResNet block. This include $1\times
1$ convolution, batch normalization, and ReLU units. Linear projection
shortcut are also preferred when the input and output dimensions of residual
ResNet block are non-identical. The output of this projection shortcut is
combined with the output of fundamental block through the addition operator as
shown in Fig. 3 (b).

In the proposed approach, we will replace the square convolutions with
hexagonal convolutions along the projection shortcuts of residual ResNet
blocks Fig. 3. However, we leave the rest of identity shortcuts without any
modifications. The proposed approach to modify a few of the projection
shortcuts will not add any significant computational overhead and thus
avoiding the vanishing gradient problem. Through our experimental results, we
will show that our approach can aid in extracting better representations over
the classical ResNet architectures. In the next subsection, we will describe
the details of hexagonal convolutions.

###  3.3 Hexagonal Convolution Operations

[20] proposed HexagDLy, a pytorch package which performs hexgonal convolutions
and pooling by blending multiple rectangle-shaped filters with different
dilations and sizes. They define the size of kernel based on the number of
levels of neighbouring elements. These hexagonal convolutions achieve higher
efficiency than ZeroOut method [40] due to its divide-and-conquer approach.

|
---|---
(a) | (b)
Figure 4: Size of hexagonal kernels. (a) Size 1 and (b) Size 2.

Increasing the number of learnable parameters along the skip connection of
residual network architecture will again lead to vanishing gradient problem as
mentioned in Sec. 1 and hence, we will limit to use size one hexagonal kernels
in our approach. Figs. 4 (a) and (b) shows sample hexgonal kernels of size one
and two, respectively. Following [20], we will use the scheme so as to avoid
any significant overhead involved in the interpolation computations while
converting from square to hexagonal tessellations or vice-versa.

Figure 5: Split of size one hex kernel into equivalent rectangular kernels.
Figure 6: Implementation of hexagonal convolution [20]

Fig. 5 shows the an example of hexagonal convolution using a hexagonal kernel
of size one. Let $\mathbf{H}$ be the image data defined on hexagonal
tessellation and $\mathbf{S}$ be its equivalent image defined on square
tessellations. We denote the hexagonal kernels by $\mathbf{K}^{l}$, where $l$
indicates its size. For the ease of mathematical demonstration, we have used
the kernel weights as unity. But in the final implementation, we use the
trainable kernel weights. As shown in Fig. 6, we derive equivalent rectangular
kernels ($\mathbf{K}^{1}_{r1}\in\mathbb{R}^{2\times 3}$ and
$\mathbf{K}^{1}_{r2}\in\mathbb{R}^{3\times 1}$) corresponding to
$\mathbf{K}^{1}$. Note that a hexagonal kernel of size one will have two
equivalent rectangular kernels. Similarly, a hexagonal kernel of size $l$ will
have $l+1$ equivalent rectangular kernels. The convolutions with the kernels
$\mathbf{K}^{1}_{r1}$ and $\mathbf{K}^{1}_{r2}$ can be now easily done with
efficient routines available in PyTorch. However, in order to obtain hexagonal
convolutions through rectangular kernels, we need to appropriately pad
$\mathbf{S}$ in three different ways as shown in Fig. 6. Let $\mathbf{S}_{1}$,
$\mathbf{S}_{2}$, $\mathbf{S}_{3}$ be the three different padded versions of
$\mathbf{S}$. Mathematically, the convolution of rectangular kernels
$\mathbf{K}^{1}_{r1}$ and $\mathbf{K}^{1}_{r2}$ with $\mathbf{S}_{1}$,
$\mathbf{S}_{2}$, $\mathbf{S}_{3}$ can be represented as

$\displaystyle\mathbf{P}_{1}$
$\displaystyle=\mathbf{S}_{1}\ast_{(1,2)}\mathbf{K}^{1}_{r1}$ (1)
$\displaystyle\mathbf{P}_{2}$
$\displaystyle=\mathbf{S}_{2}\ast_{(1,2)}\mathbf{K}^{1}_{r1}$ (2)
$\displaystyle\mathbf{P}_{3}$
$\displaystyle=\mathbf{S}_{3}\ast_{(1,1)}\mathbf{K}^{1}_{r2}$ (3)

where $\mathbf{P}_{1}$, $\mathbf{P}_{2}$, $\mathbf{P}_{3}$ denote the result
of convolution with the kernels $\mathbf{K}^{1}_{r1}$ and
$\mathbf{K}^{1}_{r2}$. The operator $\ast_{(x,y)}$ denote the convolution with
stride of $x$ and $y$ units along the rows and columns, respectively. The next
step is to merge $\mathbf{P}_{1}$ and $\mathbf{P}_{2}$ by picking the
alternate columns as shown in Fig. 6. Mathematically we denote the merge
operation as follows

$\mathbf{P}_{12}=MERGE\left(\mathbf{P}_{1},\mathbf{P_{2}}\right).$ (4)

The square equivalent of hexagonal convolution is obtained by one final
addition operation as:

$\mathbf{Q}=\mathbf{P}_{12}\oplus\mathbf{P}_{3}$ (5)

where $\oplus$ denotes the element-wise addition operation. The output
$\mathbf{Q}$ is rearranged to hexagonal lattice if needed for further
processing as shown in Fig. 6.

###  3.4 Hex-ResNet

Figure 7: Fundamental block of Hex-ResNet.

In this section, we describe the details of proposed Hex-ResNet architecture.
Our model is a hybrid approach where we combine the symmetries offered by both
the square and hexagonal tessellations. The residual block of the proposed
architecture is shown in Fig. 7 where we incorporate hexagonal convolutions
along with batch normalisation (BN) (used for training convergence) and
rectified linear unit introduces the non-linearity into the network. We employ
regular $3\times 3$ convolution in the normal path with padding and stride set
to one. In the projection shortcut path, 2D Hex-convolution with kernel size
set to one is used. In order to match the input and output vectors of this
residual block, a variable stride is used with padding set to zero. Each
residual block can be mathematically modelled as Eq. 6.

$y^{\left[l+1\right]}=\begin{cases}ReLU(F(y^{\left[l\right]},\left\\{P_{i}^{\left[l+1\right]}\right\\})+y^{\left[l+1\right]}),\\\
\left(if\>size\left(F(y^{\left[l\right]},\left\\{P_{i}^{\left[l\right]}\right\\})\right)=size(y^{\left[l\right]})\right);\\\
ReLU(F(y^{\left[l\right]},\left\\{P_{i}^{\left[l+1\right]}\right\\})+P_{p}^{\left[l\right]}y^{\left[l\right]}),\\\
\left(if\>size\left(F(y^{\left[l\right]},\left\\{P_{i}^{\left[l\right]}\right\\})\right)\neq
size(y^{\left[l\right]})\right);\end{cases}$ (6)

where $y^{\left[l\right]}$ and $y^{\left[l+1\right]}$ are the input and output
vectors of the $l^{th}$ Residual block.
F($y^{\left[l\right]},\\{P_{i}^{\left[l\right]}\\}$) is a learnable residual
mapping function that can have several layers. $P_{p}^{\left[l\right]}$ is a
learnable linear projection matrix that allows mapping the size of
$y^{\left[l\right]}$ to the output size of F, which exists only in case of not
corresponding dimensions for performing the element-wise addition between F
and $y^{\left[l\right]}$. In our approach, since we combine the hexagonal
(along projection shortcut) and square convolutions (main path of residual
block) we use $\mathbf{Q}$ as it is for further processing without mapping it
to hexagonal lattice. Note that we limit to use size one hexagonal kernel so
as to avoid vanishing gradient problem. Hexagonal convolution is performed
using the procedure described in Fig. 8. The proposed Hex-ResNet framework
with a modified projection shortcut is shown in Fig. 8 where the total number
of weighted layers are 34. It is modelled by stacking together many residual
blocks. The final layer includes a global average pooling layer and a M-fully
connected layer with softmax activation function, where M denotes the number
of classes in the dataset. The same is followed for realising other ResNet
configurations with multiple layers as well. Note that we replace only the
convolutions along the projection shortcuts but we retain the rest of skip
connections without any modifications. The reason behind retaining the rest of
skip connections without any modification is to avoid the vanishing gradient
problem.

Figure 8: Hex-Residual Network Architecture for ImageNet and CIFAR-10.
(Readers are requested to zoom in to view the details.)

##  4 Experimental Results

In this section, we describe the details of our experimental results on
CIFAR10. Also, we demonstrate the effectiveness of our approach on a subset of
ImageNet. Following [1], we use Top-1 and Top-5 accuracy as the performance
metric.

###  4.1 Software

We use [20], a PyTorch based library which has routines for computing
hexagonal convolutions. The combination of precise padding and striding scheme
is used to preserve the hexagonal structure of the input image at the output
side.

###  4.2 Image Input Format

The hexagonal grid can be understood as two stacked rectangular grid
structures on a distinct square array. By shifting rows and columns half the
distance between consecutive nearby pixels, proper horizontal and vertical
alignment is achieved, resulting in a square grid array with rows and columns
aligned as shown in Fig. 6.

###  4.3 Hardware

Google colab pro+ is used which is a web application platform. This has many
built-in package that enables us to train our model on high-end GPUs. The GPU
device that was allocated during my experiments was mostly Tesla
V100-SXM2-16GB with compute capability 7.0" and A100-SXM4-40GB with compute
capability 8.0".

###  4.4 Datasets

####  4.4.1 CIFAR10

The CIFAR-10 [45] is one of the benchmark datasets used for image
classification. This dataset has 60000 images sorted into 10 different groups
and is used to test the proposed architecture. The entire dataset is divided
into training set with 50,000 images and a test set with 10,000 images. All
the training images were padded with 4 pixels on each side and a $32\times 32$
crop was randomly sampled from the padded image or its horizontal flip. No
augmentations were applied to the test dataset. The augmented train dataset
was then split into train and validation containing 45,000 and 5000 images
respectively. Fig. 9 shows the sample images of the training and test images
of CIFAR-10 dataset from different classes. Cross entropy loss was the
criterion with stochastic gradient descent as optimiser (learning rate=0.1,
momentum=0.9, weight decay=0.001). The model was then trained for 182 epochs
with the learning rate divided by 10 at 32k, 48k, and 64k iterations.

|
---|---
(a) | (b)
Figure 9: Sample images from CIFAR10: a) Training dataset and b) Testing
dataset.

####  4.4.2 ImageNet:

We trained our Hex-ResNet architecture on ImageNet 2012 which contains more
than 1.28 million images distributed among 1000 classes. Due to the constraint
in the available resources, we have limited our experiments to randomly
sampled 10 classes from available 1000 classes. We repeat our experiments on
ImageNet on randomly sampled five different subsets, where each has 10
different classes. Fig. 10 shows the sample of images from ImageNet dataset.
Each subset was further randomly split in 80:20 ratio into a train and
validation sets, respectively. Each training subset contains about 4270 images
and validation contains about 1000 images. We sample the same classes from
test set as was used in training and validation sets. Random horizontal flip
was used as the data augmentation technique and the images are resized to
$224\times 224$ resolution. We use the batch of 32 images while training the
model. The optimiser used is Stochastic Gradient Descent.

|
---|---
(a) | (b)
Figure 10: Sample images from ImageNet 2012: a) Training dataset and b)
Testing dataset.

###  4.5 Quantitative Analysis

In this section, we present the quantitative analysis of the proposed approach
on CIFAR-10 and ImageNet datasets. Table 2 shows the Top-1 and Top-5 accuracy
and error percentage for CIFAR-10 for various ResNet configurations. It can be
seen that the Hex-ResNet 20, 32, 44, and 56 has slightly greater validation
accuracy and less Top-1 error compared to the ResNet counterparts with same
number of layers, respectively. Also, Hex-ResNet brings down the top-1 error
by 0.9% when compared to ResNet model variant 20 and by 0.2% in higher ResNet
model 56. Figs. 11 (a)-(d) depicts the validation loss vs epochs for both the
base ResNet and Hex-ResNet configurations. Note that Hex-ResNet converges
faster as well as has lower validation loss when compared to their ResNet
counterparts. This clearly indicates the feature representation obtained
through the proposed architecture has improved generalisation ability when
compared to their respective ResNet configurations.

Table 2: Error rates and accuracy percentage on CIFAR-10.

Model Parameters Validation Acc Top1 acc % Top 1 error % Top 5 acc % Top 5
error % ResNet - 20 272, 474 91\. 64% 91\. 44% 8\. 56% 99\. 63% 0\. 37% ResNet
- 20(H) 287, 130 92\. 12% 92\. 36% 7\. 64% 99\. 67% 0\. 32% ResNet - 32 466,
906 92\. 55% 92\. 03% 7\. 96% 99\. 74% 0\. 26% ResNet - 32(H) 481, 114 92\.
55% 92\. 25% 7\. 75% 99\. 73% 0\. 27% ResNet - 44 661, 338 91\. 83% 92\. 14%
7\. 85% 99\. 76% 0\. 23% ResNet - 44(H) 675, 098 92\. 92% 92\. 44% 7\. 56%
99\. 76% 0\. 23% ResNet - 56 855, 770 92\. 97% 92\. 16% 7\. 84% 99\. 69% 0\.
31% ResNet - 56(H) 869, 082 93\. 14% 92\. 35% 7\. 65% 99\. 73% 0\. 27%

---
(a)
(b)
(c)
(d)
Figure 11: Validation Loss variation with respect to epochs for CIFAR-10
dataset. ResNet vs HexResNet: (a) 20 layers, (b) 32 layers, (c) 44 layers, and
(d) 56 layers.
---
(a)
(b)
(c)
(d)
Figure 12: Validation Loss variation with respect to epochs for ImageNet
dataset. ResNet vs HexResNet: (a) 20 layers, (b) 32 layers, (c) 44 layers, and
(d) 56 layers. Table 3: Error rates and accuracy percentage on ImageNet
dataset.

Model Parameters Validation Acc Top1 acc % Top 1 error% Top 5 acc % Top 5
error % ResNet - 20 272, 474 82\. 84% 9\. 40% 90\. 60% 52\. 80% 47\. 19%
ResNet - 20(H) 287, 130 83\. 61% 10\. 60% 89\. 40% 52\. 20% 47\. 80% ResNet -
32 466, 906 78\. 64% 10\. 20% 89\. 80% 49\. 60% 50\. 54% ResNet - 32(H) 481,
114 84\. 49% 11\. 40% 88\. 60% 51\. 60% 48\. 40% ResNet - 44 661, 338 82\. 56%
10\. 80% 89\. 20% 49\. 00% 51\. 00% ResNet - 44(H) 675, 098 80\. 60% 12\. 80%
87\. 20% 50\. 60% 49\. 40% ResNet - 56 855, 770 80\. 60% 8\. 60% 91\. 40% 49\.
20% 50\. 80% ResNet - 56(H) 869, 082 83\. 50% 10\. 60% 89\. 40% 52\. 20% 47\.
80%

We present the quantitative analysis correspond to ImageNet dataset in Table.
3. The results in Table 3 show that proposed Hex-ResNet configurations has
lesser validation error compared to their respective ResNet models. Also, Hex-
ResNet 34 is better than ResNet 20 layer with significantly lower training
error. This shows that the vanishing gradient problem is more well addressed
and acquired high accuracy percentage with the proposed Hex-ResNet
architecture. Secondly, compared to the classical ResNet counterparts, the
Hex-ResNet 20, 32, 44, and 56 layer reduces the Top-1 error leading to reduced
training error. This is one of the significant comparisons that depicts the
effectiveness of hex residual learning on extremely deeper systems. Thirdly,
Hex-ResNet bring down the top-1 error by 1.2% in lower ResNet models (20,32)
and 2% in higher ResNet models (44,56). This comparison clearly shows the
benefit of hex residual learning, especially on deeper network systems.
Lastly, Figs. 12 (a)-(d) shows the variation of validation loss with respect
to epochs for various ResNet configurations. It can be seen that the Hex-
ResNet architecture converges faster as compared to the classical ResNet.
Hence, Hex-ResNet has faster and more accurate convergence at the initial
stages itself. A slight exception could be found with the Hex-ResNet 56 which
performed slightly worse than the classical ResNet in the case of validation
error. The probable reason behind this outlier behavior is due to the fact
that the HexResNet56 is overfitting the ImageNet data. Also, the stability of
validation loss variation is much better than its corresponding ResNet
counterparts. This indicates that our approach is less prone to noisy data
than when compared to ResNet defined on pure square tessellations. Most
importantly, it can be seen from both the Tables. 2 and 3 that the number of
additional parameters due to hexagonal convolutions is minimal when compared
to the number of parameters in their respective ResNet counterparts.

##  5 CONCLUSION

In this research work, we aimed at a new novel architecture for ResNet which
replaces the existing square convolution by hexagonal convolution in skip
connection for challenging computer vision tasks. We have integrated the
hexagonal convolution and the existing training procedures under distinct
resource limitations. We have reported consistent improvement on baseline
architecture. From the experimental results, we could observe that the top-5
accuracy of Hex-ResNet is outperforming classical ResNet indicating its
generalization capabilities. It also shows that Hex-ResNet can learn lot more
details even with a smaller dataset, providing higher accuracy, less error
with classical ResNet in a similar dataset. For instance, on the deep Hex-
ResNet variants, we present improvements in Top-1 error and Top -5 accuracy,
ranging from 2% to 1.2% and 0.48% to 1.35%. These improvements are achieved
without increasing the model complexity. Future works can be extended to the
following two tasks: 1) to different computer vision applications with
improved training procedures. 2) to different deep neural networks to be
trained via Hex-convolutions.

## References

  * [1] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun.  Deep residual learning for image recognition.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016.
  * [2] Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton.  Imagenet classification with deep convolutional neural networks.  Advances in neural information processing systems, 25:1097–1105, 2012.
  * [3] Matthew D Zeiler and Rob Fergus.  Visualizing and understanding convolutional networks.  In European conference on computer vision, pages 818–833. Springer, 2014.
  * [4] Md Zahangir Alom, Tarek M Taha, Christopher Yakopcic, Stefan Westberg, Paheding Sidiki, Mst Shamima Nasrin, Brian C Van Essen, Abdul A S Awwal, and Vijayan K Asari.  The history began from alexie: A comprehensive survey on deep learning approaches.  arXiv preprint arXiv:1803.01164, 2018.
  * [5] Yoshua Bengio, Patrice Simard, and Paolo Frasconi.  Learning longterm dependencies with gradient descent is difficult.  IEEE transactions on neural networks, 5(2):157–166, 1994.
  * [6] Xavier Glorot and Yoshua Bengio.  Understanding the difficulty of training deep feedforward neural networks.  In Proceedings of the thirteenth international conference on artificial intelligence and statistics, pages 249–256. JMLR Workshop and Conference Proceedings, 2010.
  * [7] Mohammad Sadegh Ebrahimi and Hossein Karkeh Abadi.  Study of residual networks for image recognition.  In Intelligent Computing, pages 754–763. Springer, 2021.
  * [8] Zifeng Wu, Chunhua Shen, and Anton Van Den Hengel.  Wider or deeper: Revisiting the resnet model for visual recognition.  Pattern Recognition, 90:119–133, 2019.
  * [9] Muhammad Farooq and Abdul Hafeez.  Covid-resnet: A deep learning framework for screening of covid19 from radiographs.  arXiv preprint arXiv:2003.14395, 2020.
  * [10] www.pexels.com.
  * [11] Christine A Curcio, Kenneth R Sloan, Robert E Kalina, and Anita E Hendrickson.  Human photoreceptor topography.  Journal of comparative neurology, 292(4):497–523, 1990.
  * [12] Lee Middleton and Jayanthi Sivaswamy.  Hexagonal image processing: A practical approach.  Springer Science & Business Media, 2006.
  * [13] Phil Sheridan, Tom Hintz, and David Alexander.  Pseudo-invariant image transformations on a hexagonal lattice.  Image and Vision Computing, 18(11):907–917, 2000.
  * [14] Russell M Mersereau.  The processing of hexagonally sampled two-dimensional signals.  Proceedings of the IEEE, 67(6):930–949, 1979.
  * [15] K Mostafa, JY Chiang, and I Her.  Edge-detection method using binary morphology on hexagonal images.  The Imaging Science Journal, 63(3):168–173, 2015.
  * [16] Kenny Young, Gautham Vasan, and Ryan Hayward.  Neurohex: A deep q-learning hex agent.  In Computer Games, pages 3–18. Springer, 2016.
  * [17] Idan Shilon, Manuel Kraus, Matthias Büchele, Kathrin Egberts, Tobias Fischer, Tim Lukas Holch, Thomas Lohse, Ullrich Schwanke, Constantin Steppa, and Stefan Funk.  Application of deep learning methods to analysis of imaging atmospheric cherenkov telescopes data.  Astroparticle Physics, 105:44–53, 2019.
  * [18] Mirco Huennefeld.  Deep learning in physics exemplified by the reconstruction of muon-neutrino events in icecube.  Verhandlungen der Deutschen Physikalischen Gesellschaft, 2017.
  * [19] Emiel Hoogeboom, Jorn WT Peters, Taco S Cohen, and Max Welling.  Hexaconv.  arXiv preprint arXiv:1803.02108, 2018.
  * [20] Constantin Steppa and Tim L Holch.  Hexagdly—processing hexagonally sampled data with cnns in pytorch.  SoftwareX, 9:193–198, 2019.
  * [21] Heechul Jung, Min-Kook Choi, Jihun Jung, Jin-Hee Lee, Soon Kwon, and Woo Young Jung.  Resnet-based vehicle classification and localization in traffic surveillance systems.  In Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pages 61–67, 2017.
  * [22] Tongyan Gong and Huiqian Niu.  An implementation of resnet on the classification of rgb-d images.  In International Symposium on Benchmarking, Measuring and Optimization, pages 149–155. Springer, 2019.
  * [23] Baoqi Li and Yuyao He.  An improved resnet based on the adjustable shortcut connections.  IEEE Access, 6:18967–18974, 2018.
  * [24] Ross Wightman, Hugo Touvron, and Hervé Jégou.  Resnet strikes back: An improved training procedure in timm.  arXiv preprint arXiv:2110.00476, 2021.
  * [25] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun.  Identity mappings in deep residual networks.  In European conference on computer vision, pages 630–645. Springer, 2016.
  * [26] Daniel P Petersen and David Middleton.  Sampling and reconstruction of wave-number-limited functions in n-dimensional euclidean spaces.  Information and control, 5(4):279–323, 1962.
  * [27] Russell M Mersereau.  The processing of hexagonally sampled two-dimensional signals.  Proceedings of the IEEE, 67(6):930–949, 1979.
  * [28] Marcel JE Golay.  Hexagonal parallel pattern transformations.  IEEE Transactions on computers, 100(8):733–740, 1969.
  * [29] Mohammad Bagher Nourian and MR Aahmadzadeh.  Image de-noising with virtual hexagonal image structure.  In 2013 First Iranian Conference on Pattern Recognition and Image Analysis (PRIA), pages 1–5. IEEE, 2013.
  * [30] Sonya Coleman, Bryan Scotney, and Bryan Gardiner.  Tri-directional gradient operators for hexagonal image processing.  Journal of Visual Communication and Image Representation, 38:614–626, 2016.
  * [31] Illa Singh and Ashish Oberoi.  Comparison between square pixel structure and hexagonal pixel structure in digital image processing.  Int. J. Comput. Sci. Trends Technol, 3:176–181, 2015.
  * [32] Dimitri Van De Ville, Thierry Blu, Michael Unser, Wilfried Philips, Ignace Lemahieu, and Rik Van de Walle.  Hex-splines: A novel spline family for hexagonal lattices.  IEEE Transactions on Image Processing, 13(6):758–772, 2004.
  * [33] Lee Middleton and Jayanthi Sivaswamy.  Edge detection in a hexagonal-image processing framework.  Image and Vision computing, 19(14):1071–1081, 2001.
  * [34] Syed Tabish Abbas and Jayanthi Sivaswamy.  Pet image reconstruction and denoising on hexagonal lattices.  In 2015 IEEE International Conference on Image Processing (ICIP), pages 3481–3485. IEEE, 2015.
  * [35] Sonia H Contreras-Ortiz and Martin D Fox.  Hexagonal filters for ultrasound images.  Journal of Electronic Imaging, 23(4):043022, 2014.
  * [36] Sonia H Contreras Ortiz, Tsuicheng Chiu, and Martin D Fox.  Hexagonal adaptive filtering on compound ultrasound images.  In 2011 Annual International Conference of the IEEE Engineering in Medicine and Biology Society, pages 4856–4859. IEEE, 2011.
  * [37] Do-Hyeong Kim, Munkh-Uchral Erdenebat, Ki-Chul Kwon, Ji-Seong Jeong, Jae-Won Lee, Kyung-Ah Kim, Nam Kim, and Kwan-Hee Yoo.  Real-time 3d display system based on computer-generated integral imaging technique using enhanced ispp for hexagonal lens array.  Applied Optics, 52(34):8411–8418, 2013.
  * [38] Amir Hassanfiroozi, Yi-Pai Huang, Bahram Javidi, and Han-Ping D Shieh.  Hexagonal liquid crystal lens array for 3d endoscopy.  Optics express, 23(2):971–981, 2015.
  * [39] Ayatollah Karimzadeh.  Analysis of the depth of field in hexagonal array integral imaging systems based on modulation transfer function and strehl ratio.  Applied Optics, 55(11):3045–3050, 2016.
  * [40] Yunxiang Zhao, Qiuhong Ke, Flip Korn, Jianzhong Qi, and Rui Zhang.  Hexcnn: A framework for native hexagonal convolutional neural networks.  In 2020 IEEE International Conference on Data Mining (ICDM), pages 1424–1429. IEEE, 2020.
  * [41] Tobias Schlosser, Michael Friedrich, and Danny Kowerko.  Hexagonal image processing in the context of machine learning: Conception of a biologically inspired hexagonal deep learning framework.  In 2019 18th IEEE International Conference on Machine Learning and Applications (ICMLA), pages 1866–1873. IEEE, 2019.
  * [42] Junren Luo, Wanpeng Zhang, Jiongming Su, and Fengtao Xiang.  Hexagonal convolutional neural networks for hexagonal grids.  IEEE Access, 7:142738–142749, 2019.
  * [43] Dongwei Chen, Fei Hu, Guokui Nian, and Tiantian Yang.  Deep residual learning for nonlinear regression.  Entropy, 22(2):193, 2020.
  * [44] Baoqi Li and Yuyao He.  An improved resnet based on the adjustable shortcut connections.  IEEE Access, 6:18967–18974, 2018.
  * [45] Alex Krizhevsky and Geoffrey Hinton.  Learning multiple layers of features from tiny images.  IEEE Transactions on Power Electronics, 28(11):5049–5062, 2009\.