# Fingervein Verification using Convolutional Multi-Head Attention Network

Raghavendra Ramachandra
Norwegian University of Science and Technology (NTNU)
Norway
Email<EMAIL_ADDRESS>Sushma Venkatesh
AiBA AS
Norway
Email<EMAIL_ADDRESS>
###### Abstract

Biometric verification systems are deployed in various security-based access-
control applications that require user-friendly and reliable person
verification. Among the different biometric characteristics, fingervein
biometrics have been extensively studied owing to their reliable verification
performance. Furthermore, fingervein patterns reside inside the skin and are
not visible outside; therefore, they possess inherent resistance to
presentation attacks and degradation due to external factors. In this paper,
we introduce a novel fingervein verification technique using a convolutional
multihead attention network called VeinAtnNet. The proposed VeinAtnNet is
designed to achieve light weight with a smaller number of learnable parameters
while extracting discriminant information from both normal and enhanced
fingervein images. The proposed VeinAtnNet was trained on the newly
constructed fingervein dataset with 300 unique fingervein patterns that were
captured in multiple sessions to obtain 92 samples per unique fingervein.
Extensive experiments were performed on the newly collected dataset FV-300 and
the publicly available FV-USM and FV-PolyU fingervein dataset. The performance
of the proposed method was compared with five state-of-the-art fingervein
verification systems, indicating the efficacy of the proposed VeinAtnNet.

##  1 Introduction

Biometric verification systems have enabled magnitude of access control
applications including border control, smartphone access, banking, and finance
applications. Fingervein biometric characteristics are widely deployed in
various applications, particularly in banking sector. Fingervein biometrics
represent the vein structure underneath the skin of the finger, which can be
captured using near-infrared sensing. The blood flow in the fingervein absorbs
near-infrared light and appears dark compared to the neighborhood region,
indicating the visibility of the fingervein (refer Figure 1). The fingervein
structure has been shown to be unique [1, 34, 28] between fingers of same data
subject and between the data subjects. Compared to other biometric
characteristics, fingervein biometrics are known for their accuracy and
usefulness, and are less vulnerable to distortion. Furthermore, fingervein
biometrics provide a natural way of protecting biometric features, as they
reside inside the skin and thus more challenging to spoof.

Figure 1: Example fingervein images with and without image enhancement for the
same identity collected in first (top row) and second session (bottom row).

Fingervein biometrics have been widely studied in the literature, resulting in
various fingervein biometric verification algorithms [33, 9]. Early works are
based on extracting fingervein patterns such that the vein region is labeled
as one and the background is labeled as zero. Techniques such as Maximum
Curvature Points (MCP) [22], Repeated Line Tracking (RLT) [21], Wide Line
Detectors (WLD) [10], Mean Curvature (MC) [37] and Randon transform [26] have
been developed for reliable fingervein recognition. As these techniques can
extract the structure of the fingervein pattern, the use of a simple
comparator based on template matching using correction can achieve reliable
performance. However, these features are sensitive to a small degree of
fingervein rotation, noise, and reflection properties of the skin and NIR
illuminator.

Table 1: State-of-the-art fingervein verification using deep learning
techniques

[HTML]CBCEFB Authors | Year | Deep Learning Technique
---|---|---
Huafeng Qin et al., [24] | 2015 | Serial CNN architecture with 3 convolution layers and 2 fully connected layer.
Itqan et al., [11] | 2016 | Serial CNN architecture with 3 convolution layers and 1 fully connected layer.
Syafeeza Radzi et al., [27] | 2016 | Serial CNN architecture with 2 convolution layers.
Huafeng Qin et al., [25] | 2017 | Serial CNN architecture with 4 convolution layers. Path based training of CNN.
Cihui Xie et al., [41] | 2019 | Siamese network with 5 conventional layers and triplet loss function.
| Jong Min Song et al.,[36]
---
2019 |  | Serial CNN architecture with 8 convolution layers. Composite fingervein image is
---
generated by converting the 1-channel input image to 3-channel input image.
Rig Das et al., [5] | 2018 | Serial CNN architecture with 5 convolution layers.
| Hyung Gil Hong et al.,[7]
---
2017 | Serial CNN architecture with 12 convolution layers and 3 fully connected layers.
Su Tnag et al., [39] | 2019 | Siamese network with residual CNN architecture.
Borui Hou et al., [8] | 2019 | Convolutional autoencoder.
Junying Zeng et al., [43] | 2020 | Deformable convolution with U-NET type architecture.
Ridvan Salih Kuzu et al., [14] | 2020 |  | Serial CNN architecture with 6 convolution layers and 2 fully connected layers
---
with LSTM for classification.
Hengyi Ren et al., [31] | 2021 |  | Feature extraction using ResNet with squeeze and excitation
---
on the encrypted fingervein images.
R$\iota$dvan Salih Kuzu et al., [13] | 2021 |  | Custom DenseNet 161 with additive angular penalty and
---
large margin cosine penalty loss function.
Weili Yang et al., [42] | 2022 | Multi-view fingervein with individual CNNs and view pooling.
Huafeng Qin et al., [42] | 2022 | U-Net based architecture with attention module.
Tingting Chai et al., [4] | 2022 | Serial CNN architecture with 5 convolution layers and one fully connected layer.
Ismail et al., [3] | 2022 | Serial CNN architecture with 3 convolution layers and two fully connected layer.
Weiye Liu et al., [17] | 2023 | Residual Attention block with inception architecture.
Zhongxia Zhang et al., [44] | 2023 | Light weight CNN with spatial and channel attention module.
Chunxin Fang et al., [6] | 2023 | Light weight Siamese network with attention module.
Bin Wa et al., [20] | 2023 |  | Serial CNN architecture with 3 convolution layers and
---
bilinear pooling with multiple attention module.
[HTML]EFEFEF This work | 2024 |  | Serial CNN architecture with 3 convolution layers and
---
multi-head attention module connected in parallel
with normal and enhanced fingervein.

The global feature representation of fingervein patterns such as Local Binary
Patterns (LBP) [15], Gabor filters [19], Local Directional Code [35], Wavelet
Transform [23], Histogram of Gradients (HoG) [19] and pyramid image features
[18] are also developed for the fingervein verification. These features are
often used with Support Vector Machines (SVM) or Euclidean distances as
comparators. As these techniques are based on global features, they are highly
sensitive to variations in finger rotation and illumination.

The representation of a fingervein image to binary codes was developed to
improve template security, together with reliable verification. Binary coding
techniques include Discriminative Binary Codes [16], binary hash codes [38],
DoG code [28], ordinal code [28], contour Code [28] and competitive codes
[28]. Because these techniques can generate binary codes for the finger vein,
the Hamming distance is used as the comparator. Binary coding techniques
exhibit good verification accuracy; however, these features are sensitive to
variations in rotation and illumination.

Figure 2: Block diagram of the proposed method for fingervein verification

Deep-learning-based fingervein verification has been extensively studied in
the literature. Table 1 summarizes the deep-learning-based techniques proposed
for fingervein recognition. Early works are based on the serial convolution
architecture, which is inspired from existing CNN architectures that are
evaluated on the ImageNet dataset. Both shallow serial CNN networks with two
convolution layers and a deep CNN network with 12 convolutional layers have
been studied in the literature. However, the quantitative results indicate
that lightweight serial networks with a smaller number of convolution layers
exhibit better performance than deep serial networks. The possible degraded
performance of deep serial networks can be attributed to limited data
availability. The use of a pre-trained CNN for feature extraction has also
been explored in the literature, together with fine-tuning and augmented pre-
trained CNN networks. The quantitative results reported indicate a performance
similar to that of end-to-end trained deep CNN networks. The Siamese network
for fingervein verification was studied using different CNN configurations and
U-NET-based architectures. The quantitative performance is similar to that of
the serial CNN architecture. Recently, attention modules with lightweight
(three to four convolution layers) serial networks have been widely explored.
Different types of attention modules, including spatial, channel, and multi-
attention modules, were introduced. The quantitative performance of the
attention networks are comparable to that of other deep learning-based
techniques implemented for fingervein verification.

Even though the deep learning techniques are widely studied for the reliable
fingervein verification, the existing deep learning techniques indicates the
following drawbacks: (a) limited data: Existing techniques are evaluated on
the small-scale datasets that has 6-12 samples per data subject. This limits
the effectiveness of deep learning and leads to an over fitting. (b) Lack of a
consistent evaluation protocol: Even though most of the existing works have
used public datasets, the evaluation protocols are not consistent across
existing studies. This results in a limited comparison of existing techniques
for finger vein verification. In this study, we address the above-mentioned
limitation by introducing a new large-scale dataset with 75 data subjects,
resulting in 300 unique identities (as we collected four fingers per data
subject). For each unique fingervein, we collected 92 samples in multiple
sessions, varying from 1-4 days duration. Furthermore, we propose a novel
lightweight CNN architecture based on a convolutional multi-head attention
module. The main contributions of this study are as follows:

  * •

A novel fingervein verification technique based on a convolutional multi-head
attention network (VeinAtnNet) is proposed.

  * •

Introduced a new fingervein dataset with 300 unique identities captured from
75 data subjects, resulting in $300\times 92=27600$ fingervein images. The
dataset is available publicly for research purpose.

  * •

Extensive experiments were performed on both the newly introduced dataset and
the publicly available FV-USM and FV-PolyU datasets. The performance of the
proposed method was compared with that of five state-of-the-art fingervein
verification methods.

The rest of the paper is organised as follows: Section 2 discuss the proposed
method for the fingervein verification, Section 3 presents the quantitative
results of the proposed method with the state-of-the-art techniques and
Section 4 draws the conclusion.

##  2 Proposed Method

Figure 2 shows a block diagram of the proposed VeinAtnNet architecture for
reliable fingervein verification. The novelty of the proposed approach is that
it leverages the convolutional Multi-Head Attention (MHA) framework to achieve
accurate and reliable fingervein verification. The utility of MHA, together
with convolutional features, leads to a discriminant feature representation
that can contribute to the robust performance of the fingervein verification.

The proposed VeinAtnNet is a lightweight architecture with three Consecutive
Convolution Layers (CCL) and a Multihead Self-Attention (MSA) mechanism.
VeinAtnNet is connected independently with normal and enhanced fingervein
images whose comparison scores from the softmax layer are fused to make the
final verification decision. Given the captured fingervein image,
preprocessing is performed using Contrast Limited Adaptive Histogram
Equalization (CLAHE) [29] to enhance the fingervein pattern. In this work, we
employed the Contrast Limited Adaptive Histogram (CLAHE) as the fingervein
enhancement method by considering (a) the high quality of the fingervein
enhancement achieved when compared to other enhancement techniques, as
discussed in [9]. (b) Widely employed enhancement techniques in fingervein
literature that have reported high verification accuracy. Both the normal
(without enhancement) and enhanced fingervein images were resized to
$224\times 224\times 3$ pixels. The CCL performs the initial feature learning
of the fingervein images, which is further processed to obtain a rich feature
representation using MSA. Given the fingervein image $F_{v}^{R\times C\times
D}$, the final output features of MSA can be represented as follows:

$F_{MSA}=MSA(F_{CCL});whereF_{CCL}=CCL(F_{v});$ (1)

Where, $F_{MSA}$ denote the output features from MSA, $F_{CCL}$ denotes the
output features of CCL block. The $F_{MSA}$ is then used with softmax
classifier to make the final decision. In the following, we discuss the
building blocks of the proposed VeinAtnNet.

###  2.1 Consecutive Convolution Layers (CCL)

The CCL has three convolution modules that are serially connected. Each
convolution module has four different convolution layers (conv): a group
normalization (norm), an activation function layer (ReLu), and a pooling layer
(maxpool). Three convolution layers were used to extract the global features
from the fingervein images. The conv-1 layer has a filter size of $7\times 7$
the conv-2 layer has a $5\times 5$, conv-3 has a $3\times 3$ filter, and the
number of filters in all three conv layers is set to 32. The gradual decrease
in filter size ensures fine grinding (from global to local) of the fingervein
features. The convolution features were normalized using group normalization,
which reduced the sensitivity of the network for initialization. In
particular, we employed group normalization because it outperforms batch
normalization with a small size. The normalized features are then fed to the
activation unit (ReLU), which can introduce sparsity and improve the network
training speed. Finally, a pooling operation was performed to achieve a
compact feature representation. In this study, we employed max pooling, which
can capture texture information suitable for fingervein verification. The
output after three convolution modules is then passed through the group-
average pooling layer to obtain a compact representation of the features.
Finally, the features were flattened before being fed into the MSA module.

###  2.2 Multihead Self-Attention (MSA)

The features from the CCL module are then fed to the MSA module to further
refine the features $F_{CCL}$ to extract discriminant features suitable for
fingervein verification. In this study, we employed multihead attention [40]
with four different heads and 64 channels for keys and queries. Basically, MSA
runs the attention mechanism across all heads multiple times in parallel. The
independent attention outputs are then concatenated and transformed linearly.
MSA can be represented as follows [40]:

$Mu-Head(Q.K,V)=[H_{1},H_{2},H_{3},H_{4}]W$ (2)

where $W$ is the learnable parameter and $Q$, $K$ and $V$ represent the
queries, keys, and values, respectively. In this study, we employed scaled
dot-product attention across heads using $Q$, $K$ and $V$ as follows:

$Attention(Q.K,V)=softmax(\frac{QK^{T}}{sqrt(d_{k})})V$ (3)

The outcome of the MSA module was passed through the layer normalization layer
to generalize the final features. Finally, the normalized features are passed
through the fully connected and softmax layers to obtain the comparison score.

###  2.3 Score Level Fusion

The proposed VeinAtnNet was employed independently on normal and enhanced
finger vein images. Thus, given the test fingervein image, the proposed method
provides two comparison scores corresponding to normal and enhanced
fingerveins. We combined these two comparison scores using the sum rule to
make the final verification decision. Let the comparison score from the normal
fingervein image be $C_{n}$ and enhanced fingervein image be $C_{e}$, then
final verification score is computed as $V_{s}=(C_{n}+C_{e})$.

###  2.4 Implementation Details

The proposed network is based on Adaptive Moment Estimation (ADAM)
optimization to calculate loss. In this work, we employ the cross-entropy
loss, which can be defined as
$-\frac{1}{N}\sum_{n=1}^{N}\sum_{i=1}^{K}(T_{ni}\log(Y_{ni}))+(1-T_{ni})\log(1-Y_{ni})$,
where $N$ and $K$ denote the number of samples and classes, respectively,
$T_{ni}$ is the corresponding target value to $Y_{ni}$. During training, the
learning rate was set to 0.0001, the mini-batch size was set to 16, and the
number of epochs was set to 150. Furthermore, we performed data augmentation,
which included image reflection, translation, rotation, reflection, scaling,
and random noise with three different variances. This resulted in nine
different images for every image used in training the proposed method.
Finally, the proposed method is lightweight with only 58.2 K learnable
parameters. While the existing SOTA employed in this work namely; Bin Wa et
al., [20] has approximately 17.8M and Ismail et al., [3] has approximately
467.1K learnable parameters respectively.

##  3 Experiments and Results

In this section, we discuss the quantitative results of the proposed and
existing fingervein verification algorithms. The quantitative performance is
presented using the False Match Rate (FMR) and False Non-Match Rate (FNMR),
together with the Equal Error Rate (EER) value computed at FMR = FNMR. The
performance of the proposed method was compared with recently proposed
fingervein recognition algorithms based on multiple attentions [20] and deep
fusion [3] by considering their verification performance. Furthermore, we
compared the performance of the proposed method with well-established
fingervein verification techniques, such as MCP [22], RLT[21] and WLD [10]. In
the following section, we describe the newly collected fingervein dataset,
followed by the quantitative results.

###  3.1 FV-300 Fingervein dataset

In this study, we introduced a new fingervein dataset comprising 300 unique
fingerveins corresponding to 75 unique data subjects. The fingervein images
were collected using a custom camera system desired using a monochrome CMOS
camera with a resolution of $744\times 480$ pixels with two lighting sources
to illuminate the finger from both the back and side. The design aspects of
the finger vein capture device were inspired by [30]. The data collection was
carried out under indoor conditions, and for every data subject, two fingers
(index and middle) were captured from both the left and right hands, resulting
in four unique fingers. For each data subject, we captured 92 fingervein
images corresponding to individual fingers in multiple sessions. The duration
between sessions varies from to 1-4 days. The FV-300 dataset contained 75 data
subjects $\times$ 4 fingers $\times$ 92 = 27600 fingervein samples. Figure 3
shows an example of the fingervein images from the FV-300 dataset.

Figure 3: Example fingervein images from FV-300 dataset representing same
identity captured in three different sessions. Table 2: Quantitative
Performance of the proposed and state-of-the-art fingervein verification
methods

|  |  | TAR = (100-FNMR%) @ FMR =
---|---|---|---
Data set | Algorithms | EER(%) | 1% | 0.1% | 0.01%
[HTML]EFEFEF | MCP [22] | 4.74 | 88.93 | 64.41 | 44.35
[HTML]EFEFEF | RLT [21] | 31.30 | 19.35 | 9.52 | 4.68
[HTML]EFEFEF | WLD [10] | 13.55 | 77.47 | 74.11 | 73.15
[HTML]EFEFEF | Ismail et al., [3] | 9.14 | 72.92 | 40.95 | 7.34
[HTML]EFEFEF | Bin Wa et al., [20] | 19.94 | 21.25 | 5.31 | 0.95
[HTML]EFEFEFFV300 | Proposed Method | 0.54 | 99.73 | 99.13 | 90.36
[HTML]ECF4FF | MCP [22] | 17.74 | 46.95 | 27.25 | 18.36
[HTML]ECF4FF | RLT [21] | 29.63 | 32.29 | 29.70 | 29.16
[HTML]ECF4FF | WLD [10] | 18.17 | 54.17 | 34.89 | 16.73
[HTML]ECF4FF | Ismail et al., [3] | 41.78 | 4.38 | 1.16 | 0.34
[HTML]ECF4FF | Bin Wa et al., [20] | 37.53 | 5.25 | 1.65 | 0.35
[HTML]ECF4FFFV-USM | Proposed Method | 15.35 | 40.45 | 14.25 | 5.11
[HTML]FFCCC9 | MCP [22] | 14.25 | 52.29 | 34.40 | 20.64
[HTML]FFCCC9 | RLT [21] | 33.48 | 19.26 | 8.25 | 4.28
[HTML]FFCCC9 | WLD [10] | 16.53 | 65.29 | 48.16 | 38.53
[HTML]FFCCC9 | Ismail et al., [3] | 42.12 | 5.96 | 2.29 | 1.37
[HTML]FFCCC9 | Bin Wa et al., [20] | 40.90 | 3.21 | 0.45 | 0.45
[HTML]FFCCC9PolyU | Proposed Method | 5.52 | 74.77 | 30.27 | 19.26

(a)

(b)

(c)

Figure 4: DET Curves showing the verification performance of the proposed and
state-of-the-art fingervein verification methods

###  3.2 Experimental protocol

To effectively benchmark the performance of the proposed method, we used three
fingervein datasets: FV-300, FV-USM [2] and FV-PolyU [12]. To evaluate the
performance of the fingervein algorithm on FV-300 dataset, the fingervein
samples corresponding to each finger were divided into three independent sets
such that the training set had 70 images, the validation set had 12 images,
and the testing set had 10 images. This resulted in 300 $\times$ 10 = 3000
genuine and 300 $\times$ 299 $\times$ 10 = 897000 impostor scores,
respectively.

The verification performance of the FV-USM [2] dataset was evaluated by
training the fingervein verification algorithms on FV-300 dataset and fine-
tuning the trained networks on the FV-USM dataset. The FV-USM [2] dataset
comprised 492 unique fingervein identities captured in two sessions with six
samples each. Thus, the proposed method (and the existing methods employed in
this work that includes multiple attention [20] and deep fusion [3]) are
trained on the FV-300 dataset and fine-tuned using the first session data
(from FV-USM dataset) that has 6 samples per subject. Testing was performed
using the second-session data (from FV-USM dataset) with six samples per
subject. However, the conventional fingervein state-of-the-art techniques (MCP
[22], RLT [21] and WLD [10]) employed in this study do not require a training
set for learning. Therefore, we used the first-session data from the FV-USM
dataset as enrolment, and the second session data were used for testing. This
resulted in 492 $\times$ 6 = 2952 genuine and 492 $\times$ 491 $\times$ 6 =
1449432 impostor scores.

The verification performance of the fingervein algorithms (deep learning based
on the proposed method) on the FV-PolyU dataset was performed using a
procedure similar to that discussed for the FV-USM dataset. The fingervein
algorithms trained on the FV-300 dataset were fine-tuned using the FV-PolyU
dataset. The FV-PolyU dataset [12] employed in this work comprises 156 unique
identities, from which the finger vein index and middle fingers are captured
in two sessions with six samples each. Thus, the FV-PolyU dataset has 312
unique identities, and data from the first session are used to fine-tune both
the proposed and SOTA deep learning methods, which include multiple attention
[20] and deep fusion [3]) that are trained on the FV-300 dataset. Testing was
performed on the second session data, which resulted in 312 $\times$ 6 = 1872
genuine and 312 $\times$ 311 $\times$ 6 = 582192 impostor scores.

Table 3: Verification performance of the proposed method with normal and
enhanced fingervein data

[HTML]ECF4FF [HTML]ECF4FF |  [HTML]ECF4FF |  [HTML]ECF4FF |  [HTML]ECF4FFTAR = (100-FNMR%) @ FMR =
---|---|---|---
[HTML]ECF4FF [HTML]ECF4FFData Type | [HTML]ECF4FFAlgorithms | [HTML]ECF4FFEER(%) |  [HTML]ECF4FF1% |  [HTML]ECF4FF0.1% | 0.01%
Normal Fingervein | Proposed Method | 1.85 | 97.89 | 83.55 | 54.18
Enhanced Fingervein | Proposed Method | 1.13 | 98.87 | 90.53 | 60.76

###  3.3 Results and discussion

Table 2 shows the quantitative performance of the proposed and existing
fingervein verification techniques on both FV-300, FV-PolyU and FV-USM
datasets, and Figure 4 shows the DET curves. Existing methods were trained
using enhanced fingervein images to optimise the best performance. Based on
the results, the following are important observations:

  * •

Training and testing on the same dataset will indicate the improved
verification results of the deep learning based techniques. Therefore, the
performance of the deep learning techniques indicated an improved performance
on FV-300 compared to FV-USM and FV-PolyU dataset.

  * •

Traditional fingervein techniques (MCP [22], RLT [21] and WLD [10]) that are
based on template matching using correlation indicates the superior
performance on the FV-300 dataset compared to FV-USM and FV-PolyU dataset.
However, the performance of RLT [21] and WLD [10] do not indicate a
significant difference in the verification performance between three different
fingervein datasets employed in this work.

  * •

Among three traditional fingervein techniques employed in this work, the MCP
[22] indicated the best performance on both datasets. Furthermore, MCP [22]
demonstrated improved performance compared to the state-of-the-art deep
learning methods employed in this study.

  * •

The proposed method has indicated an outstanding verification performance with
EER = 0.54% and TAR = 90.36% @ FMR = 0.01% on FV-300 dataset. The proposed
method also indicated the best performance with an EER of 15.35% on the FV-USM
dataset. Similar performance is also noted on the FV-PolyU dataset with an EER
= 5.52% . However, the verification performance degraded at a lower FMR on
both FV-USM and FV-PolyU dataset.

  * •

Based on the results, it is worth nothing that, the deep learning techniques
depends on the training data and indicate the limitation to generalize on the
another dataset due to the limited number of samples available for fine-
tuning. However, compared with existing deep learning methods, the proposed
VeinAtnNet exhibits superior verification performance on three fingervein
datasets employed in this work.

###  3.4 Ablation Study of the proposed method

Table 4: Ablation study of the proposed method on FV-300 dataset

Consecutive Convolution Layers (CCL) |  Multi-head Self-Attention (MSA) | Proposed method
---|---|---
Conv-1 | Conv-2 | Conv-3 | EER (%)
✓ | X | X | ✓ | 8.29
✓ | ✓ | X | ✓ | 2.38
✓ | ✓ | ✓ | ✓ | 0.54

Figure 5: DET Curves indicating the performance of the proposed method with
different cases of ablation study

In this section, we present an ablation study of the proposed method by using
the FV-300 dataset. We considered three different cases in which Case-1
represent the performance with Conv-1 and MSA together. Case-2 shows the
performance of Conv-1, Conv-2, and MSA while Case-3 indicates the performance
of the proposed method with Conv-1, Conv-2, Conv-3, and MSA. Table 4 and
Figure 5 show the performance of the proposed method for different ablation
studies. The addition of convolutional layers with MSA can improve the overall
performance of the proposed VeinAtnNet for reliable fingervein verification.

Figure 6: EER of the proposed method with different number of convolution
layers.

We further investigated the role of adding additional convolution layers with
MSA to improve the verification accuracy. To this extent, we start computing
the verification accuracy starting with one Conv layer and increasing it to
five consecutive Conv layers with MSA. Figure 6 shows the verification
performance with EER for different depths of convolution layers. It should be
noted that the use of three consecutive layers with MSA can achieve the best
performance and further increase the depth by adding convolution layers. This
further justifies the choices made in designing the proposed method that has
indicated the best generalized verification performance compared to the five
different SOTA.

Figure 7: Illustration of LIME based explainability on the proposed method
based on the (a) normal and (b) enhanced fingervein images.

###  3.5 Interpretation of the proposed method

To interpret the decision achieved by the proposed method, we employed Local
interpretable model-agnostic explanations (LIME) [32] to explain the
perdition’s on the probe fingervein images. Because the proposed method is
based on both normal and enhanced fingervein images, we present the
qualitative and quantitative results for both fingervein image types. Table 3
indicates the quantitative performance of the proposed method with normal and
enhanced image alone. The obtained results indicated a similar verification
performance with EER and higher FMR values. However, with lower FMR values,
the proposed method exhibited better performance with enhanced fingervein
samples. Thus, the availability of the enhanced fingervein pattern indicates
more discriminant information to improve verification accuracy at low FMR
values.

Figure 7 shows the qualitative results of the LIME method for visualizing
important regions in the fingervein image, which has contributed to successful
verification. The LIME explanation is shown on the fingervein images from the
FV-300 dataset for successful verification prediction at FAR = 0.01%. As shown
in Figure 7, the proposed method utilises more image regions with normal
fingervein images compared with the enhanced fingervein to make the decision.
However, with enhanced fingervein images, the decision is based on a smaller
number of regions associated with vein pattern and particularly on the
minutiae points of the fingervein. These observations justify the improved
performance of the proposed method with enhanced fingervein images compared
with normal fingervein images.

##  4 Conclusion

Fingervein biometrics are widely employed in various secure access control
applications. In this study, we proposed a novel method based on a
convolutional multihead attention module for reliable fingervein verification.
The proposed VeinAtnNet is based on three consecutive convolution layers and
multihead attention with four heads and 64 channels connected in parallel to
the normal and enhanced fingervein samples. Finally, the decision is made
using the score-level fusion of the normal and enhanced fingerveins. Extensive
experiments were performed on both publicly and newly collected finger vein
datasets. The quantitative performance of the proposed method was benchmarked
using five state-of-the-art fingervein verification methods. The obtained
results indicate the superior performance of the proposed method on both
publicly available and newly collected fingervein datasets.

## References

  * [1] Finger vein authentication.  In White Paper, 2006.
  * [2] Mohd Shahrimie Mohd Asaari, Shahrel A Suandi, and Bakhtiar Affendi Rosdi.  Fusion of band limited phase only correlation and width centroid contour distance for finger based biometrics.  Expert Systems with Applications, 41(7):3367–3382, 2014.
  * [3] Ismail Boucherit, Mohamed Ould Zmirli, Hamza Hentabli, and Bakhtiar Affendi Rosdi.  Finger vein identification using deeply-fused convolutional neural network.  Journal of King Saud University-Computer and Information Sciences, 34(3):646–656, 2022.
  * [4] Tingting Chai, Jiahui Li, Shitala Prasad, Qi Lu, and Zhaoxin Zhang.  Shape-driven lightweight cnn for finger-vein biometrics.  Journal of Information Security and Applications, 67:103211, 2022\.
  * [5] Rig Das, Emanuela Piciucco, Emanuele Maiorana, and Patrizio Campisi.  Convolutional neural network for finger-vein-based biometric identification.  IEEE Transactions on Information Forensics and Security, 14(2):360–373, 2018.
  * [6] Chunxin Fang, Hui Ma, and Jianian Li.  A finger vein authentication method based on the lightweight siamese network with the self-attention mechanism.  Infrared Physics & Technology, 128:104483, 2023.
  * [7] Hyung Gil Hong, Min Beom Lee, and Kang Ryoung Park.  Convolutional neural network-based finger-vein recognition using nir image sensors.  Sensors, 17(6):1297, 2017.
  * [8] Borui Hou and Ruqiang Yan.  Convolutional autoencoder model for finger-vein verification.  IEEE Transactions on Instrumentation and Measurement, 69(5):2067–2074, 2019.
  * [9] Borui Hou, Huijie Zhang, and Ruqiang Yan.  Finger-vein biometric recognition: A review.  IEEE Transactions on Instrumentation and Measurement, 71:1–26, 2022\.
  * [10] Beining Huang, Yanggang Dai, Rongfeng Li, Darun Tang, and Wenxin Li.  Finger-vein authentication based on wide line detector and pattern normalization.  In Pattern Recognition (ICPR), 2010 20th International Conference on, pages 1269–1272, Aug 2010.
  * [11] KS Itqan, AR Syafeeza, FG Gong, N Mustafa, YC Wong, and MM Ibrahim.  User identification system based on finger-vein patterns using convolutional neural network.  ARPN Journal of Engineering and Applied Sciences, 11(5):3316–3319, 2016.
  * [12] Ajay Kumar and Yingbo Zhou.  Human identification using finger images.  IEEE Transactions on Image Processing, 21(4):2228–2244, 2012.
  * [13] R$\iota$dvan Salih Kuzu, Emanuele Maiorana, and Patrizio Campisi.  Loss functions for cnn-based biometric vein recognition.  In 2020 28th European Signal Processing Conference (EUSIPCO), pages 750–754. IEEE, 2021.
  * [14] Ridvan Salih Kuzu, Emanuela Piciucco, Emanuele Maiorana, and Patrizio Campisi.  On-the-fly finger-vein-based biometric recognition using deep neural networks.  IEEE Transactions on information Forensics and Security, 15:2641–2654, 2020.
  * [15] Eui Chul Lee, Hyunwoo Jung, and Daeyeoul Kim.  New finger biometric method using near infrared imaging.  Sensors, 11(3):2319–2333, 2011.
  * [16] Haiying Liu, Lu Yang, Gongping Yang, and Yilong Yin.  Discriminative binary descriptor for finger vein recognition.  IEEE Access, 6:5795–5804, 2017.
  * [17] Weiye Liu, Huimin Lu, Yifan Wang, Yupeng Li, Zhenshen Qu, and Yang Li.  Mmran: A novel model for finger vein recognition based on a residual attention mechanism: Mmran: A novel finger vein recognition model.  Applied Intelligence, 53(3):3273–3290, 2023.
  * [18] Yu Lu, Sook Yoon, Shiqian Wu, and Dong Sun Park.  Pyramid histogram of double competitive pattern for finger vein recognition.  IEEE Access, 6:56445–56456, 2018.
  * [19] Y. Lu, S. Yoon, S. J. Xie, J. Yang, Z. Wang, and D. S. Park.  Finger vein recognition using histogram of competitive gabor responses.  In 22nd International Conference on Pattern Recognition, pages 1758–1763, Aug 2014.
  * [20] Bin Ma, Kaixuan Wang, and Yueli Hu.  Finger vein recognition based on bilinear fusion of multiscale features.  Scientific Reports, 13(1):249, 2023.
  * [21] Naoto Miura, Akio Nagasaka, and Takafumi Miyatake.  Feature extraction of finger-vein patterns based on repeated line tracking and its application to personal identification.  Machine Vision and Applications, 15(4):194–203, 2004.
  * [22] Naoto Miura, Akio Nagasaka, and Takafumi Miyatake.  Extraction of finger-vein patterns using maximum curvature points in image profiles.  IEICE TRANSACTIONS on Information and Systems, 90(8):1185–1194, 2007.
  * [23] Kang Ryoung Park.  Finger vein recognition by combining global and local features based on svm.  Computing and Informatics, 30(2):295–309, 2011.
  * [24] Huafeng Qin and Mounîm A El-Yacoubi.  Finger-vein quality assessment by representation learning from binary images.  In Neural Information Processing: 22nd International Conference, ICONIP 2015, Istanbul, Turkey, November 9-12, 2015, Proceedings, Part I 22, pages 421–431. Springer, 2015.
  * [25] Huafeng Qin and Mounim A El-Yacoubi.  Deep representation-based feature extraction and recovering for finger-vein verification.  IEEE Transactions on Information Forensics and Security, 12(8):1816–1829, 2017.
  * [26] Huafeng Qin, Xiping He, Xingyan Yao, and Hongbing Li.  Finger-vein verification based on the curvature in radon space.  Expert Systems with Applications, 82:151–161, 2017.
  * [27] Syafeeza Ahmad Radzi, Mohamed Khalil Hani, and Rabia Bakhteri.  Finger-vein biometric identification using convolutional neural network.  Turkish Journal of Electrical Engineering and Computer Sciences, 24(3):1863–1878, 2016.
  * [28] R. Raghavendra and C. Busch.  Exploring dorsal finger vein pattern for robust person recognition.  In International Conference on Biometrics (ICB), pages 341–348, 2015.
  * [29] R. Raghavendra, Kiran B raja, Jayachander Surbiryala, and Christoph Busch.  A low-cost multimodal biometric sensor to capture finger vein and fingerprint.  In International Joint Conference on Biometrics (IJCB), pages 1–7, Sep 2014.
  * [30] Raghavendra Ramachandra, Kiran Bylappa Raja, Sushma Krupa Venkatesh, and Christoph Busch.  Design and development of low-cost sensor to capture ventral and dorsal finger vein for biometric authentication.  IEEE Sensors Journal, 19(15):6102–6111, 2019.
  * [31] Hengyi Ren, Lijuan Sun, Jian Guo, Chong Han, and Fan Wu.  Finger vein recognition system with template protection based on convolutional neural network.  Knowledge-Based Systems, 227:107159, 2021.
  * [32] Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin.  ” why should i trust you?” explaining the predictions of any classifier.  In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pages 1135–1144, 2016.
  * [33] Kashif Shaheed, Hangang Liu, Gongping Yang, Imran Qureshi, Jie Gou, and Yilong Yin.  A systematic review of finger vein recognition techniques.  Information, 9(9), 2018.
  * [34] Mohamed Shahin, Ahmed Badawi, and Mohamed Kamel.  Biometric authentication using fast correlation of near infrared hand vein patterns.  International Journal of Biological and Medical Sciences, 2(3):141–148, 2007.
  * [35] Pragati Sikarwar et al.  Finger vein recognition using local directional pattern.  In 2016 International Conference on Inventive Computation Technologies (ICICT), volume 1, pages 1–5. IEEE, 2016.
  * [36] Jong Min Song, Wan Kim, and Kang Ryoung Park.  Finger-vein recognition based on deep densenet using composite image.  Ieee Access, 7:66845–66863, 2019.
  * [37] Wonseok Song, Taejeong Kim, Hee Chan Kim, Joon Hwan Choi, Hyoun-Joong Kong, and Seung-Rae Lee.  A finger-vein verification system using mean curvature.  Pattern Recognition Letters, 32(11):1541–1547, 2011.
  * [38] Kun Su, Gongping Yang, Lu Yang, Dunfeng Li, Peng Su, and Yilong Yin.  Learning binary hash codes for finger vein image retrieval.  Pattern Recognition Letters, 117:74–82, 2019.
  * [39] Su Tang, Shan Zhou, Wenxiong Kang, Qiuxia Wu, and Feiqi Deng.  Finger vein verification using a siamese cnn.  IET biometrics, 8(5):306–315, 2019.
  * [40] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin.  Attention is all you need.  Advances in neural information processing systems, 30, 2017.
  * [41] Cihui Xie and Ajay Kumar.  Finger vein identification using convolutional neural network and supervised discrete hashing.  Pattern Recognition Letters, 119:148–156, 2019.
  * [42] Weili Yang, Junduan Huang, Zhuoming Chen, Junhong Zhao, and Wenxiong Kang.  Multi-view finger vein recognition using attention-based mvcnn.  In Biometric Recognition: 16th Chinese Conference, CCBR 2022, Beijing, China, November 11–13, 2022, Proceedings, pages 82–91. Springer, 2022\.
  * [43] Junying Zeng, Fan Wang, Jianxiang Deng, Chuanbo Qin, Yikui Zhai, Junying Gan, and Vincenzo Piuri.  Finger vein verification algorithm based on fully convolutional neural network and conditional random field.  IEEE access, 8:65402–65419, 2020.
  * [44] Zhongxia Zhang and Mingwen Wang.  Finger vein recognition based on lightweight convolutional attention model.  IET Image Processing, 2023.


# Quantum Computation and Arrows of Time

Nathan Argaman<EMAIL_ADDRESS>Department of Physics, Nuclear Research
Center – Negev, P.O. Box 9001, Be’er Sheva 84190, Israel

###### Abstract

Quantum physics is surprising in many ways. One surprise is the threat to
locality implied by Bell’s Theorem. Another surprise is the capacity of
quantum computation, which poses a threat to the complexity-theoretic Church-
Turing thesis. In both cases, the surprise may be due to taking for granted a
certain strict arrow-of-time assumption, whose applicability may be limited to
the classical domain. This possibility has been noted repeatedly in the
context of Bell’s Theorem. The argument concerning quantum computation is
described here. Further development of models which violate this strong arrow-
of-time assumption, replacing it by a weaker arrow, is called for.

##  .1 Introduction

Physics faces unresolved difficulties with arrows of time. This has been
evident at least since the discussions of Boltzmann’s H-Theorem and
Loschmidt’s paradox in the late 19th century. Although progress has been made
in connecting different arrows of time to the low-entropy big-bang origin of
the universe, the resulting understanding is still incomplete [see, _e.g._ ,
Schulman [1997]]. Nevertheless, “the” arrow of time is often taken for
granted, and is familiar from the “Newtonian schema” of kinematics plus
dynamics Wharton [2015]: it is often assumed that a physical system can always
be described as having a “state” (kinematics) which “evolves” (dynamics) from
the past to the future.

There are also some well-known exceptions—not all physics models conform to
the rules of this schema. For example, in order to find the “state” of a
system at a certain time according to the stationary-action principle, one
must specify inputs—the values of the position coordinates—at both its past
and future boundaries. This demonstrates the “Lagrangian schema,” which
requires an all-at-once or block-universe approach. By looking beyond the
standard schema, one is freed from the limitations of conventional thinking,
and is open to novel possibilities. Seeking such freedom is especially
relevant when an impasse is encountered; this article sets forth the claim
that the surprising power of quantum computing [_i.e._ , its tension with the
strong form of the Church-Turing Thesis Arora and Barak [2009]] is just the
type of “paradox” which calls for abandoning the standard arrow of time.

There already exist several lines of evidence that quantum physics is at issue
with the standard arrow of time [see also Dirac [1938], Wheeler and Feynman
[1945, 1949] in the classical context]. Early examples include discussions of
the Einstein-Podolsky-Rosen (EPR) “paradox” Costa de Beauregard [1953] and
delayed-choice experiments Bohr [1935], Wheeler [1978]; recent examples
include argumentation from time symmetry Leifer and Pusey [2017] and the
Pusey-Barrett-Rudolph (PBR) theorem Pusey et al. [2012] [the latter enquires
whether the quantum state is ontic or epistemic, _i.e._ , whether it describes
reality or merely the information one has regarding a system; see, _e.g._ ,
Leifer [2014]]. The strongest argument involves Bell’s Theorem [see, _e.g._ ,
Price [1997]].

As typically understood, Bell’s theorem proves that there is no hope for
reformulating Quantum Mechanics (QM) in terms of “hidden variables,” or
parameters directly describing events in spacetime. But this relies on
accepting the standard arrow-of-time rule, which is taken for granted by the
“Local Causality” (a.k.a. “Einstein Locality”) assumption of Bell’s Theorem.
Within such an approach, QM typically describes the “state” of a many-particle
system as a ray in an abstract and exponentially large Hilbert space, with
typical applications involving superpositions and complex probability
amplitudes.

Considering an alternative schema opens up the possibility of describing
quantum entanglement in terms of spacetime-based parameters with standard
probability rules Argaman [2010], Almada et al. [2016]. The apparently
nonlocal connection between distant regions $a$ and $b$ is achieved through
intermediate “hidden” parameters $\lambda$, situated far enough in the past
that they could reasonably serve as a classical common cause for events in $a$
and $b$. In defiance of the standard arrow-of-time rule, $\lambda$ is taken to
depend on the inputs in $a$ and $b$, which are thus indirectly connected,
through the overlap of their past lightcones. The fact that the “hidden”
$\lambda$ (a microscopic parameter) may depend on future inputs need not lead
to violations of Signal Causality, just like the collapse of the wavefunction
at $b$ due to a measurement at $a$ does not lead to violations of Signal
Locality in the standard discussion of Bell correlations. For this reason,
this type of “retrocausality” cannot lead to paradoxical causal loops. [Any
attempt to “measure” $\lambda$ so that its value will be correlated with that
of a macroscopic pointer would result in loss of the entanglement, as in a
“which path” detection in the context of two-slit interference; see, _e.g._ ,
Buks et al. [1998].]

So far, progress in developing a full reformulation of QM along these lines
has been slow [see Wharton and Argaman [2019] for a recent review]. If too
much freedom is allowed, one might obtain models with backward-in-time
signaling, and it has been argued that preventing this requires fine tuning
Wood and Spekkens [2015]. Although counterarguments are available Almada et
al. [2016], it seems that a physical principle, perhaps associated with the
entropic arrow of time, is needed. Such a principle could limit the excess
freedom resulting from removal of the standard arrow-of-time condition, and
lead to results which would systematically conform to the Signal-Causality
arrow-of-time rule.

A closely-related issue has to do with the degree of correlations allowed in
classical, quantum, or general non-local theories. In proving Bell’s Theorem,
one typically derives the CHSH inequality — the fact that in any locally
causal mathematical model, a certain combination of correlators cannot possess
a value larger than $2$ Clauser et al. [1969], Clauser and Horne [1974]. In
general, one can generate models where this combination achieves values up to
$4$ Popescu and Rohrlich [1994], but in QM its value is limited to $2\sqrt{2}$
Cirel’son [1980]. Again, it appears that a physical principle is involved in
limiting the exaggerated freedom of generic models. In fact, research in this
context has already made significant strides, involving several suggested
principles Linden et al. [2007], Pawłowski et al. [2009], Navascués and
Wunderlich [2010].

In the present work, it is suggested that the algorithmic complexity
achievable with quantum computation similarly provides motivation for
rejecting the standard arrow of time.111Other approaches connecting the flow
of time with quantum computation can be found in Aaronson [2005] and
Castagnoli et al. [2019]. Furthermore, here too it appears that a physical
principle remains to be identified, one that would limit the freedom obtained
with such a rejection. The argument is based on the distinction between a
Directed Acyclic Graph (DAG) and a non-directed graph.

Describing natural laws in spacetime in terms of mathematical parameters, and
discretizing spacetime into $N$ distinct events, leads to a DAG if the strong
arrow of time is maintained. Assuming that the laws are local, and that the
past is fixed and the future is not yet relevant, the mathematical rules for
each event are greatly simplified, and the number of steps in a simulation of
the physics is just the number of events, $N$. But it is not clear to begin
with that the arrow of time must be imposed [see, _e.g._ , Wharton [2015]]. In
particular, if there are stochastic rules that determine only how the
probabilities for each event depend on events in its vicinity (in both space
and time), without imposed arrows, finding the overall distribution for $N$
events may be a much more complicated computational task, due to the
requirement that all $N$ events “simultaneously” conform to the physical laws.
As an example, consider the task of finding the ground state of a three- (or
higher-)dimensional spin glass, which is known to be an NP-complete problem
Bachas [1984], Barahona [1982].

It is thus seen that if one assumes that Nature supplies us with finite
“machines” which operate according to local rules subject to the strong arrow
of time, all that can be achieved algorithmically is similar to a standard
algorithm with $N$ steps, taking $N$ to appropriately represent the finiteness
and the resolution pertaining to these “machines.” However, if the “machines”
provided by nature are not subject to an arrow-of-time rule, the possibility
that they might be capable of performing exponentially harder tasks appears to
be open. Our best understanding of quantum computing does not lead us to
expect natural “machines” to be able to solve NP-complete tasks. The
complexity class associated with quantum “machines” is the BQP class, which is
(to the best of our knowledge222Strictly speaking, that BQP is weaker than NP
but stronger than P is not a proven fact, but a conjecture which is assumed
here. ) much weaker Nielsen and Chuang [2000]. Thus, again, a physical
principle which is weaker than the standard arrow of time is required, one
that would limit the achievable complexity class from NP-complete to BQP (not
to P).

In the next two sections we will go through the above argument in a little
more detail, describing the connection between events in spacetime and
algorithmic steps in the presence (Section B) and in the absence (Section C)
of a strong arrow of time. A discussion section and conclusions follow.

##  .2 The strict arrow of time motivates the strong form of the Church-Turing
Thesis

Mathematical models of classical physics employ local variables or parameters
with a clear association of a place and a time for each parameter. A typical
example is provided by the values of the classical electric field,
$\bm{E}(\bm{x},t)$. In order to connect this with algorithmic complexity, it
is appropriate to discretize spacetime, taking a finite number $N$ of events,
$(\bm{x}_{n},t_{n})$, distributed reasonably uniformly, to provide a
sufficiently detailed representation of a finite region of (Minkowski)
spacetime, to some desired accuracy.

Within a kinematics plus dynamics schema, the state of the modelled system at
time $t$ would be represented in this picture by the events $m$ with times
$t_{m}$ between $t-\Delta t$ and $t$ for an appropriate small $\Delta t$, and
the values of the model parameters $\mu_{m}$ associated with these events. The
model obeys Local Causality if the dynamics specify a rule (which may be
either deterministic or probabilistic) for obtaining the value of the
parameters at the $n$th event from the parameters in its recent past and its
close vicinity, with spacelike separations avoided, so that the relevant
events are in the past relative to $t_{n}$ in all frames. We will denote the
set of indices of these earlier and nearby events by $r(n)$.

If an external input, such as an external force, acts at the $n$th event, the
value of the parameters at that event will be affected, but the values at
earlier times will not. The parameters $\mu_{n}$ associated with the $n$th
event thus include inputs $I_{n}$ and non-input parameters $Q_{n}$ (each of
these is in general a set of parameters, not limited to scalars). In the
deterministic case, the dynamical rule $F_{n}$ specifies the value of $Q_{n}$
as a function of $I_{n}$ and the earlier $\\{\mu_{m}\\}_{m\in r(n)}$ (the rule
$F_{n}$ depends also on the spacetime locations of $n$ and the $m$s, of
course). For example, a model discretizing Maxwell’s equations in this manner
would have $Q_{n}$ corresponding to the electromagnetic fields, and $I_{n}$
specifying the charge and current densities, the relevant inputs in this case.
(For stochastic models, $F_{n}$ determines the probability distribution of
$Q_{n}$.)

Assuming that the values of the parameters $\mu_{n}$ are appropriately
discretized as well, so that the modeling of each of the $N$ events is finite,
this description makes it obvious that the algorithmic complexity of a
simulation according to such a model is $O(N)$. Conversely, the modeled
physics cannot provide results which are not efficiently achievable by an
algorithm with $O(N)$ steps. That physical systems are (polynomially)
equivalent to algorithms in this sense is an expression of the strong form of
the Church-Turing thesis Arora and Barak [2009]. Barring problems with the
discretization scheme, classical physics indeed operates in this manner. That
quantum physics is different is discussed in the next sections.

Note that we have here considered only the number of steps in the algorithm,
$N$. It is of course possible for only $M$ out of the $N$ events to have
external inputs, such as initial conditions, with the other $N-M$ events
having no inputs (or having the corresponding $I_{n}$ set to zero or null in
some fashion). It is further possible to have the number of physical
parameters $N$ exponentially larger than the number of physical inputs $M$,
but this possibility is not of interest for the purposes of the present
discussion, which focuses on $N$ itself.

It is natural to take the $N$ spacetime events to be nodes of a graph, with
directed edges from the $m$s in $r(n)$ to $n$ itself, representing the
dynamical rules $F_{n}$. The resulting DAG (Directed Acyclic Graph) represents
the discretized mathematical model, as well as the algorithm which would carry
out a simulation according to the model.333This $O(N)$ graph is not to be
confused with the exponentially large configuration graph representing all the
possibilities for a model Arora and Barak [2009]. All the edges in the graph
are directed from the past to the future (that the graph is acyclic
corresponds to assuming a standard Minkowski geometry, with no closed time-
like curves).

Note how central the standard arrow-of-time assumption is to the logic leading
to the Church-Turing Thesis (it is not denied, of course, that there are
additional necessary assumptions, _e.g._ , regarding discretization—for the
present purposes it suffices that the standard arrow of time is one of the
necessary assumptions). It is the assumption that for each node $n$, the task
of obtaining the parameters $Q_{n}$ can be performed while taking all the
parameters from the past, _i.e._ , from $r(n)$, to be fixed, and ignoring all
the parameters relating to the future, which leads to the finiteness of this
task. Performing this for $N$ nodes is then necessarily an $O(N)$ task.

##  .3 Models with no arrow of time

Removal of the above arrow-of-time restriction would have dramatic
consequences for the algorithmic-complexity consideration. If a mathematical
model is associated not with a DAG but with an undirected graph of $N$ nodes,
its capacity for computation could be entirely different. In fact, for a
reasonable choice of the replacement for the rules $F_{n}$, the category of
computations which can be efficiently performed by a “machine” which would
efficiently generate solutions of the relevant model would be the NP class. As
already mentioned, this is known if the rules are replaced by those of a spin-
glass system Bachas [1984], Barahona [1982]. (Which rules should be expected
for future physics theories is of course completely open—for example, at the
dawn of QM, Heisenberg employed noncommuting-operator rules while Schroedinger
used differential equations.) A further simple example is described next.

The example involves a standard NP-complete problem, such as scheduling $M$
meetings within a finite given time $T$. The requirements concerning the
length of the meetings and the intended participants are to be specified by
inputs $I_{m}$, and the timing of each meeting by parameters of a different
type, $P_{m}$. As the problem is NP, it is known that $N$ steps are required
to verify that the meetings have no conflicts, with $N$ polynomial in $M$. It
is easy to construct a DAG with $N$ nodes representing the algorithm for
performing this verification process, beginning with the $I_{m}$s and $P_{m}$s
as inputs and resulting in an output $O$ which is true for a valid combination
of the timings $P_{m}$. The directions of the edges of the DAG lead from its
inputs $I_{m}$ and $P_{m}$ to its output $O$. Each of the $N$ steps is
associated with a rule $F_{n}$, consistent with the description of the
previous section.

Consider now removing the arrows from the graph. This could represent a model
where each rule $F_{n}$ is replaced by a weight $W_{n}$, which depends on the
same parameters involved in $F_{n}$. Thus, the rules of the model are local as
before, but the model dictates the overall behavior of the combination of
parameters $\\{Q_{n}\\}$, and cannot be easily separated into $N$ consecutive
steps. Combining all of the local rules involves multiplying the weights
$W_{n}$ for all $n$, and normalizing the weights to obtain a probability
distribution involves adding all the product weights, resulting in a
normalizing factor $Z=\sum_{\\{Q_{n}\\}}\left(\prod_{n}W_{n}\right)$. (In the
statistical mechanics context, $Z$ is called the partition function, and the
weights are given by an exponent involving the potential energy and the
temperature.)

Returning to the specific scheduling problem above, one can define each of the
weights $W_{n}$ as equal to unity for every combination which is consistent
with the rule $F_{n}$, and to zero otherwise. One may further set the inputs
$I_{m}$ for a specific scheduling task, and set the “output” $O$ to “true.” If
the time $T$ is not too short,444Dealing with shorter times, or with tasks for
which the structure of the graph and the $F_{n}$ rules depends not only on the
$I_{m}$s but also on the $P_{m}$s, requires more-complicated examples. every
valid schedule, _i.e._ , every valid combination of the $P_{m}$s, together
with the corresponding values of the other $Q_{n}$ parameters, would have a
weight of unity, and all other combinations would have a vanishing weight ($Z$
represents the number of valid schedules). The result of such a model would be
to generate at random one of the valid schedules. This too is, of course, an
NP-complete task.

The resulting pattern is similar to that described for the CHSH inequality in
the introduction. The standard arrow of time is a strong restriction that
would limit the capabilities of any model of a physical system to those of
standard algorithms, in accordance with the strong form of the Church-Turing
Thesis. Quantum systems are not as limited as that, but removing the arrow-of-
time restriction altogether would result in capabilities which are “too
powerful” according to reasonable expectations. A restriction is necessary,
but it needs to be less powerful in order to curtail the achievable complexity
class from NP to BQP, not to P.

##  .4 Discussion

The above argumentation may appear to be making a philosophical point, but the
intention is to apply this reasoning to guide developments in mathematical
physics. This follows Feynman [1965], who contended that nobody understands
QM. He demonstrated how predictive power can coexist with a lack of
understanding with the example of the astronomers of the Maya culture, who
possessed a mathematical procedure for predicting the appearances of the
moon—the timing of a new moon or of an eclipse—which did not involve any
conception of orbital paths. Feynman also suggested that developing
reformulations of existing theories can serve to improve our understanding,
even if no novel predictions are involved (examples include the development of
Lagrangean and Hamiltonian mechanics as alternative formulations of Newton’s
equations; for a long time, these only improved our understanding of classical
mechanics; much later, they also played essential roles in the development of
QM).

In this context, the upshot of the previous sections is that quantum
computation adds to our motivation to develop reformulations of QM which do
not conform to the standard arrow of time. But it is clear that some effective
arrow of time must be retained. Physical theories in general, and standard QM
in particular, conform to the Signal-Causality rule—they describe signaling to
the future, but not to the past (there are many aspects which are time
reversal-symmetric, but there is always something to break the symmetry, often
just a special treatment of initial conditions). Thus, the flow of accessible
information, relating to the inputs and the outputs of the theory, is always
from the past to the future.

In standard Schroedinger-picture QM, this past-to-future flow affects the
internal parameters of the theory as well—the quantum state or wavefunction is
taken to evolve from the past to the future (whether or not collapse is
allowed for). A reformulation breaking the standard-arrow-of-time rules would
involve some internal parameters which depend on other parameters in their
future (possibly a statistical dependence, _i.e._ , having a probability
distribution which depends on future parameters). In order for this future-
dependence to play an essential role, it must involve relationships which
cannot be simply inverted, such as a dependence on the externally-controlled
settings of future measurement devices. For this reason, the relevant arrow-
of-time condition involves future input parameters, and is called No Future-
Input Dependence in Wharton and Argaman [2019].

The situation concerning causality or the arrow of time in reformulations of
QM which would violate this condition is similar to that concerning locality
in standard QM, which violates locality in the sense of Bell’s Local-Causality
condition, but conforms to Signal Locality. Here the No Future-Input
Dependence condition would be violated for internal parameters, but the output
parameters would not have this characteristic—the Signal Causality condition
involving the outputs would be maintained.

As noted in the introduction, relaxing No Future-Input Dependence has dramatic
consequences for reformulations of QM. The generalization of Bell’s locality
condition to models with Future-Input Dependence is called Continuous Action,
and maintaining this locality condition has distinct advantages, in addition
to the necessary Signal Locality Wharton and Argaman [2019]. In fact, Bell’s
Local Causality condition can be seen to follow from requiring both Continuous
Action and No Future-Input Dependence (assuming Lorentz Covariance and the use
of standard mathematics and probability rules). Thus, if a reformulation of QM
with Continuous Action can indeed be found, it will accordingly be based on a
model with parameters with Future-Input Dependence.

It would be natural to view these parameters as providing a more-or-less
direct description of reality — ontic variables — with the standard “quantum
state” taken to merely represent the information available to an external
observer up to a time $t$. This is the psi-epistemic view of QM [see, _e.g._ ,
Caves et al. [2002]]. The arguments posited against this view in the past
would fail in the presence of Future-Input Dependence. The fact that this
state “evolves” with $t$ in an information-conserving manner (unitarity) would
be required by its role as representing unchanged information, as long as
indeed there is no update of the available information. Similarly, this “state
of knowledge” would have to suddenly change upon such an update, explaining
precisely why and how measurements cause “wavefunction collapse.”

This brief discussion only aims to indicate that the development of Future-
Input Dependent models with Continuous Action is feasible in principle. For
details, including concrete examples of toy models reproducing QM in the
specific context of Bell’s Theorem, see Wharton and Argaman [2019]. Developing
a full reformulation of QM along these lines appears to be challenging not
because of a necessity to deal with a particularly complicated situation, but
primarily because of the need to overcome the barrier associated with
conventional thinking concerning the arrow of time.

##  .5 Conclusion

When we use a mathematical model to describe the objective properties of a
physical system, we generally expect these properties to depend on the past of
the system, not on its future. This works well in the classical, macroscopic
domain, but the presence of quantum fluctuations and uncertainty appear to
undermine such thinking for quantum systems. The time-symmetry of microscopic
physical laws similarly speaks against such a distinction between the past and
the future.555Indeed, such time-symmetry arguments have been used to motivate
alternative time-symmetric interpretations of Quantum Mechanics, both the
transactional interpretation Cramer [1980] and the two-state-vector formalism
Aharonov and Vaidman [1991], but these approaches still employ the standard
quantum “state,” which for many-particle systems is exponentially complex and
cannot be represented in terms of local variables $\mu_{n}$. Allowing the
system’s “objective” microscopic parameters to depend on the specification of
the measurement to be made on the system at a later time, not only on the
earlier preparation, may resolve many a quantum mystery. As described above,
the “nonlocality” of Bell’s Theorem serves as the prime example — quantum
phenomena violate the relevant “no-action-at-a-distance” condition only when
this condition is formulated within models with such a strong past-future
distinction.

Generalizing the “no-action-at-a-distance” condition to models which are time-
reversal symmetric, or which possess a weaker arrow-of-time rule, removes the
restriction posed by Bell’s Theorem Wharton and Argaman [2019]. This could
serve to “explain” the power of quantum computation — if indeed microscopic
parameters are not subject to the rules of a DAG, the associated complexity
class need not be limited to P.

Once this point of view is accepted, one is faced with a sharply contrasting
problem. It is not that quantum computation is surprisingly powerful — it
becomes surprising that it is not even more powerful. A “physical principle”
must be imposed on the relevant family of models to limit the capacity from NP
to BQP. This is closely analogous to the search for a limiting physical
principle in the context of Tsirelson’s bound, which is related to Bell’s
Theorem and has been an active field in recent decades. Perhaps concepts from
quantum computation will provide additional clues or lead to new directions on
this adventure.

Acknowledgements: The author wishes to thank Scott Aaronson and the other
participants of the 6th FQXI conference (Castelvecchio Pascoli, Italy, July
2019) for thought-provoking discussions, and Oded Schwartz and Ken Wharton for
helpful comments on a draft of the manuscript.

## References

  * Aaronson [2005] S. Aaronson.  NP-complete problems and physical reality.  _ACM SIGACT News_ , 36(1):30–52, 2005.  doi: 10.1145/1052796.1052804.
  * Aharonov and Vaidman [1991] Y. Aharonov and L. Vaidman.  Complete description of a quantum system at a given time.  _Journal of Physics A: Mathematical and General_ , 24(10):2315, 1991.
  * Almada et al. [2016] D. Almada, K. Ch’ng, S. Kintner, B. Morrison, and K. Wharton.  Are retrocausal accounts of entanglement unnaturally fine-tuned?  _International Journal of Quantum Foundations_ , 2:1–14, 2016.
  * Argaman [2010] N. Argaman.  Bell’s theorem and the causal arrow of time.  _American Journal of Physics_ , 78:1007–1013, 2010.
  * Arora and Barak [2009] S. Arora and B. Barak.  _Computational Complexity: a modern approach_.  Cambridge University Press, 2009.
  * Bachas [1984] C. P. Bachas.  Computer-intractability of the frustration model of a spin glass.  _Journal of Physics A: Mathematical and General_ , 17(13):L709–L712, 1984.  doi: 10.1088/0305-4470/17/13/006.
  * Barahona [1982] F. Barahona.  On the computational complexity of Ising spin glass models.  _Journal of Physics A: Mathematical and General_ , 15(10):3241–3253, 1982.  doi: 10.1088/0305-4470/15/10/028.
  * Bohr [1935] N. Bohr.  Can quantum-mechanical description of physical reality be considered complete?  _Physical Review_ , 48:696–702, 1935.  doi: 10.1103/PhysRev.48.696.
  * Buks et al. [1998] E. Buks, R. Schuster, M. Heiblum, D. Mahalu, and V. Umansky.  Dephasing in electron interference by a ‘which-path’ detector.  _Nature_ , 391:871–874, 1998.  doi: 10.1038/36057.
  * Castagnoli et al. [2019] G. Castagnoli, E. Cohen, A. K. Ekert, and A. C. Elitzur.  A relational time-symmetric framework for analyzing the quantum computational speedup.  _Foundations of Physics_ , 49:1200–1230, 2019.
  * Caves et al. [2002] C. M. Caves, C. A. Fuchs, and R. Schack.  Quantum probabilities as Bayesian probabilities.  _Physical Review A_ , 65(2):022305, 2002.
  * Cirel’son [1980] B. S. Cirel’son.  Quantum generalizations of Bell’s inequality.  _Letters in Mathematical Physics_ , 4(2):93–100, 1980.  doi: 10.1007/BF00417500.
  * Clauser and Horne [1974] J. F. Clauser and M. A. Horne.  Experimental consequences of objective local theories.  _Physical Review D_ , 10(2):526, 1974.
  * Clauser et al. [1969] J. F. Clauser, M. A. Horne, A. Shimony, and R. A. Holt.  Proposed experiment to test local hidden-variable theories.  _Physical Review Letters_ , 23(15):880, 1969.
  * Costa de Beauregard [1953] O. Costa de Beauregard.  Une réponse à l’argument dirigé par Einstein, Podolsky et Rosen contre l’interprétation bohrienne des phénomènes quantiques.  _Comptes Rendus de l’Académie des Sciences_ , 236:1632–1634, 1953.
  * Cramer [1980] J. G. Cramer.  Generalized absorber theory and the Einstein-Podolsky-Rosen paradox.  _Physical Review D_ , 22(2):362, 1980.
  * Dirac [1938] P. A. M. Dirac.  Classical theory of radiating electrons.  _Proceedings of the Royal Society A_ , 167:148–169, 1938\.  doi: 10.1098/rspa.1938.0124.
  * Feynman [1965] R. P. Feynman.  _The Character of Physical Law_.  The MIT Press, 1965.
  * Leifer [2014] M. S. Leifer.  Is the quantum state real? An extended review of $\psi$-ontology theorems.  _Quanta_ , 3(1):67–155, 2014.
  * Leifer and Pusey [2017] M. S. Leifer and M. F. Pusey.  Is a time symmetric interpretation of quantum theory possible without retrocausality?  _Proceedings of the Royal Society A_ , 473(2202):20160607, 2017.
  * Linden et al. [2007] N. Linden, S. Popescu, A. J. Short, and A. Winter.  Quantum nonlocality and beyond: Limits from nonlocal computation.  _Physical Review Letters_ , 99:180502, 2007.  doi: 10.1103/PhysRevLett.99.180502.
  * Navascués and Wunderlich [2010] M. Navascués and H. Wunderlich.  A glance beyond the quantum model.  _Proceedings of the Royal Society A_ , 466:881–890, 2010\.  doi: 10.1098/rspa.2009.0453.
  * Nielsen and Chuang [2000] M. Nielsen and I. Chuang.  _Quantum Computation and Quantum Information_.  Cambridge University Press, 2000.  ISBN 0-521-63503-9.
  * Pawłowski et al. [2009] M. Pawłowski, T. Paterek, D. Kaszlikowski, V. Scarani, A. Winter, and M. Żukowski.  Information causality as a physical principle.  _Nature_ , 461(7267):1101, 2009.
  * Popescu and Rohrlich [1994] S. Popescu and D. Rohrlich.  Quantum nonlocality as an axiom.  _Foundations of Physics_ , 24(3):379–385, 1994\.  doi: 10.1007/BF02058098.
  * Price [1997] H. Price.  _Time’s arrow & Archimedes’ point: new directions for the physics of time_.  Oxford University Press, USA, 1997.
  * Pusey et al. [2012] M. F. Pusey, J. Barrett, and T. Rudolph.  On the reality of the quantum state.  _Nature Physics_ , 8(6):475, 2012.
  * Schulman [1997] L. S. Schulman.  _Time’s arrows and quantum measurement_.  Cambridge University Press, 1997.
  * Wharton [2015] K. Wharton.  The universe is not a computer.  In A. Aguirre, B. Foster, and Z. Merali, editors, _Questioning the foundations of physics_ , pages 177–189. Springer, 2015.
  * Wharton and Argaman [2019] K. Wharton and N. Argaman.  Bell’s theorem and spacetime-based reformulations of quantum mechanics.  _arXiv preprint arXiv:1906.04313_ , 2019.
  * Wheeler [1978] J. A. Wheeler.  The “past” and the “delayed-choice” double-slit experiment.  In A. Marlow, editor, _Mathematical Foundations of Quantum Theory_ , pages 9 – 48. Academic Press, 1978.  ISBN 978-0-12-473250-6.  doi: 10.1016/B978-0-12-473250-6.50006-6.
  * Wheeler and Feynman [1945] J. A. Wheeler and R. P. Feynman.  Interaction with the absorber as the mechanism of radiation.  _Reviews of Modern Physics_ , 17(2-3):157, 1945\.
  * Wheeler and Feynman [1949] J. A. Wheeler and R. P. Feynman.  Classical electrodynamics in terms of direct interparticle action.  _Reviews of modern physics_ , 21(3):425, 1949\.
  * Wood and Spekkens [2015] C. J. Wood and R. W. Spekkens.  The lesson of causal discovery algorithms for quantum correlations: Causal explanations of Bell-inequality violations require fine-tuning.  _New Journal of Physics_ , 17(3):033002, 2015\.