# Effective Goal-oriented 6G Communications: the Energy-aware Edge Inferencing
Case ††thanks: This work has been partly funded by the European Commission
through the H2020 project Hexa-X (Grant Agreement no. 101015956).

Mattia Merluzzi1, Miltiadis C. Filippou2, Leonardo Gomes Baltar2, Emilio
Calvanese Strinati1
1CEA-Leti, Université Grenoble Alpes, F-38000 Grenoble, France
2Intel Germany, Next Generation & Standards, 85579 Neubiberg, Germany
email:{mattia.merluzzi, emilio.calvanese-strinati}@cea.fr,{miltiadis.filippou,
<EMAIL_ADDRESS>
###### Abstract

Currently, the world experiences an unprecedentedly increasing generation of
application data, from sensor measurements to video streams, thanks to the
extreme connectivity capability provided by 5G networks. Going beyond 5G
technology, such data aim to be ingested by Artificial Intelligence (AI)
functions instantiated in the network to facilitate informed decisions,
essential for the operation of applications, such as automated driving and
factory automation. Nonetheless, while computing platforms hosting Machine
Learning (ML) models are ever powerful, their energy footprint is a key
impeding factor towards realizing a wireless network as a sustainable
intelligent platform. Focusing on a beyond 5G wireless network, overlaid by a
Multi-access Edge Computing (MEC) infrastructure with inferencing
capabilities, our paper tackles the problem of energy-aware dependable
inference by considering inference effectiveness as value of a goal that needs
to be accomplished by paying the minimum price in energy consumption. Both
MEC-assisted standalone and ensemble inference options are evaluated. It is
shown that, for some system scenarios, goal effectiveness above 84% is
achieved and sustained even by relaxing communication reliability requirements
by one decimal digit, while enjoying a device radio energy consumption
reduction of almost 23% at the same time. Also, ensemble inference is shown to
improve system-wide energy efficiency and even achieve higher goal
effectiveness, as compared to the standalone case for some system
parameterizations.

###### Index Terms:

Machine Learning, 6G, goal-oriented communications, sustainable operation

##  I introduction

Recently, research on the sixth generation of mobile communication systems
(6G) has kicked off all around the world [1, 2], aiming to identify new
technological enablers for wireless networking and service operation of
unprecedented capability. Different from previous mobile systems generations,
6G will embed human, physical and digital worlds into the same ecosystem and
infrastructure [1]. To this end, both human and machine type communications
will play a key role, paving the way to a holistic system in which different
in-network intelligent agents exchange and process massive amounts of data in
real-time to achieve common goals in several diverse scenarios (e.g.
industrial, automotive). Therefore, among the several pillars needed to enable
6G, it is worth mentioning: i) a massive deployment of distributed computing
and storage resources at the network edge; ii) a new communication
perspective, no longer driven by radio link performance, but rather by the
actual outcome of a communication strategy (effectiveness), per a predefined
goal.

The former is possible thanks to the Multi-access Edge Computing (MEC)
paradigm [3], which enables a powerful Information Technology (IT) environment
including a plethora of new services at network edge in short proximity to the
end user. The MEC paradigm poses several challenges in terms of a joint design
of communication and computing aiming to strike the best trade-offs in terms
of latency, energy, trustworthiness and, more in general, sustainability
targets. Focused on Artificial Intelligence/Machine Learning (AI/ML) related
workloads, an additional challenge is how to enhance learning/inference
reliability given a fixed pool of radio and processing resources as parts of
the network. Indeed, when considering connect-compute services tailored to
task offloading use cases (including, but not limited to learning/inference
tasks) [4], energy, end-to-end (E2E) latency, service dependability, etc. are
affected by the availability of both radio and computing resources, whose
scarcity may be severe in dense user deployments [5]. In this context, new
measures of reliability going beyond the classical communication concepts, are
ready to be explored, building on a paradigm shift from data-oriented to goal-
oriented communications [6, 7].

Goal-oriented communications go beyond the common paradigm of transmitting
data with a targeted communication reliability, as focus is rather on the
actual effect that the decoded information has on performing an action, a
concept also known as effectiveness [7]. While this is clear for human-to-
human communications, the application to connected intelligence is a subject
to be explored, paving the way to define new target performance indicators,
beyond the “legacy” ones defining wireless technologies up to and including
5G. Such new metrics are expected to bring up new trade-offs involving several
cross-layer parameters and performance.
Related work: Edge inference has recently gained considerable attention [6,
8]. In particular, data and model compression techniques have been shown to
enable energy-efficient and low latency edge inference, while guaranteeing
target performance in terms of accuracy [9, 10, 11], also in the case of
cooperative inference [12, 13]. Goal-oriented communications have been
proposed as a promising solution calling for the design of data encoding and
transmission strategies, such that a specific task is performed [6, 14]. In
[15], the authors propose a joint transmission and recognition scheme, showing
the effect of communication reliability on the accuracy, ignoring delay and
energy consumption, as well as cooperative inference. Finally, task
replication has been shown to improve reliability [16], also in the presence
of random edge server struggling or poor channel conditions [17].
Our contributions: Building on the aforementioned technological enablers, in
this paper we investigate the edge inference task from a goal-oriented
perspective. In particular, the goal embraces targeted inference
accuracy/reliability and E2E delay (goal value), subject to energy consumption
(goal cost), all affected by changing system parameters, such as wireless
channels, MEC Hosts’ (MEHs’) availability, and other cross-layer parameters.
Both standalone and ensemble (cooperative) inference settings are
investigated, to also show the impact of inference task replication. The main
contributions of this work are summarized as follows: i) we show how the goal-
oriented approach reduces device energy consumption requirements at target
effectiveness (with the latter involving delay and accuracy), even in the
presence of degraded uplink communication reliability regimes (Bit Error Rate
- BER); ii) we also show how a distributed deployment of MEHs enhances the
success rate of the goal (effectiveness), and even further reduces the goal
cost in some circumstances.
Organization of the paper: Section II presents the system scenario and network
deployment specifics and Section III defines an edge inference goal, with its
accompanied requirements and its incurred costs; Section IV elaborates upon
standalone and ensemble-based inference. Section V presents thoroughly the
performed numerical evaluations and observations. Finally, Section VI
concludes the paper.

##  II System model and setup

Figure 1: MEC-assisted inference scenario; use of helper MEH is optional to
enhance inference performance.

The system architecture, the main system entities, and their operations are
shown in Fig. 1. In particular, in our setting, $K$ end devices (e.g., sensors
or devices carrying them) of limited computing capability and power autonomy
upload, through the wireless connection to an Access Point (AP), input data to
nearby MEHs hosting intelligent agents, which are onboarded with inferencing
task-relevant pre-trained and pre-uploaded ML models. The intelligent agents,
in their turn, produce inferencing outputs (e.g., predictions or
recommendations) and provide them to the calling client application via
downlink radio transmission. The general assumed MEC infrastructure deployment
consists of multiple MEHs deployed across a network coverage area, where, each
MEH is either collocated to a radio AP, or located at aggregation points of
the access network, thus, being accessible by multiple APs. For brevity, we
focus on a part of a deployment consisting of two MEHs. The first one is a
primary MEH, which is collocated to the AP. Also, device requests can be
further possibly treated by another MEH, interfaced with the primary MEH via
wired (e.g., fiber) backhaul connection; this is termed after as a helper MEH.
The aim of a helper is to improve performance, e.g., in terms of goal
accomplishment, i.e., inference effectiveness, thanks to offering single point
of failure avoidance, as well as through inference diversity gains.

Let us now elaborate on the goal definition, its requirements and its incurred
costs for the focused system setup.

##  III Edge inference goal: Requirements & costs

A goal is defined as an information processing task accompanied by a set of
minimum requirements that must be fulfilled as part of task accomplishment. In
the case of edge inference, the goal is to provide inference results, within a
delay budget, overcoming a minimum tolerable level of inference accuracy. In
this work, the cost of the edge inference goal is split into an energy cost
for end devices to upload data via the uplink connection to an AP, and an
energy cost for the MEH(s) to execute the edge inference task. To this end, we
will present: i) the E2E delay model, comprising transmission signaling and
computing times; ii) the goal achievement and the corresponding outages
(affecting effectiveness); iii) the end devices and MEH energy consumption
models.

###  III-A End-to-end delay as a goal requirement

Edge inference tasks fall into the category of computation offloading
services, whose goal is to delegate the execution of computationally heavy
workloads from end devices (e.g., vehicles or robots carrying sensors) to
nearby accessible MEHs, and whose E2E delay generally entails three components
for an inference task: i) uplink communication delay to transmit input
inference data; ii) computation delay at the MEH to process input data; iii)
downlink communication delay to transmit inference outputs (e.g., labels) back
to the requesting device. In this paper, we neglect the last delay term, as in
several scenarios the result represents a significantly reduced amount of
bits, as compared to the input data size. Let us now elaborate on each delay
component.

####  III-A1 Uplink transmission delay

Let us suppose, without loss of generality, that device $k$ issues one
inference request at a given time instant, and that the pattern for inference
is encoded with $n_{k}$ bits. Thus, assuming that the transmission has to
occur with target BER denoted by $\textrm{BER}_{k}$, the uplink transmission
delay is given by:

$D_{k}^{u}=\frac{n_{k}}{B_{k}^{u}\log_{2}\left(1+\frac{\varphi(\textrm{BER})h_{k}^{u}p_{k}^{u}}{N_{0}B_{k}^{u}}\right)},$
(1)

where $B_{k}^{u}$ is the uplink bandwidth assigned to device $k$, $h_{k}^{u}$
is the channel power gain, $p_{k}^{u}$ is device $k$’s uplink transmit power,
and $N_{0}$ is the noise power spectral density at the receiver; whereas, the
BER margin can be approximated as
$\varphi(\textrm{BER}_{k})=\frac{-1.5}{\ln(5\textrm{BER}_{k})}$, or
$\varphi(\textrm{BER}_{k})=\frac{-1.5}{\ln(0.5\textrm{BER}_{k})}$ in case the
spectral efficiency falls below $4$111Tighter approximations can be used under
more specific assumptions on modulation and coding, which go beyond the scope
of this paper. [18]. This delay component is affected by both the device
transmit power and BER requirements, a fundamental aspect in defining the
difference between classical and goal-oriented communications.

####  III-A2 Remote computing delay for inferencing

We assume that each MEH $j$ (being a primary - $p$ or a helper - $h$) is
equipped with a CPU of maximum clock frequency $f_{j}^{\max}$. A portion of
$f_{j}^{\max}$ is assigned to each device submitting a request for inference
output. We denote by $\alpha_{k,j}\in[0,1]$ the percentage of CPU power
assigned to device $k$ at MEH $j$. Furthermore, we assume that this portion is
not always fully available, as MEH $j$ may dedicate part of it to higher
priority traffic upon request. We assume this priority traffic to be an
exogenous parameter that we cannot control, therefore, we denote by
$\beta_{k,j}\in[0,1]$ the percentage of available CPU for device $k$ at a
given time instant, also taking into account the concurrent arrival of higher
priority processing tasks. Therefore, the actual CPU cycle frequency allocated
to device $k$ by MEH $j$ is given by:

$f_{k,j}=\alpha_{k,j}\beta_{k,j}f_{j}^{\max}.$ (2)

Obviously, system performance is strongly affected by the statistics of
$\beta_{k,j}$. In the case MEH $h$ cooperates in the inference process, input
inference data must be transferred from MEH $p$ to MEH $h$, and results should
be then sent back to MEH $p$ to operate the necessary output aggregation. For
simplicity, we assume a high capacity backhaul with fixed Round Trip Time
(RTT) $D_{p,h}^{\textrm{bh}}$ between MEH $p$ and MEH $h$. Therefore, the
total inference computation delay is

$D_{k}^{c}=\max\left(\frac{J_{k,p}}{f_{k,p}},\frac{J_{k,h}}{f_{k,h}}+D_{p,h}^{\textrm{bh}}\right),$
(3)

where $J_{k,p}$ and $J_{k,h}$ denote the numbers of CPU cycles needed to run
the inference for device $k$ at MEH $p$ and $h$, respectively. Also, we
neglect the time needed by MEH $p$ to aggregate results, as we assume this to
be a simple operation, compared to the inference task.

Finally, the overall inference delay accounts for communication and
computation delays, thus reading as
$D_{k}^{\textrm{tot}}=D_{k}^{u}+D_{k}^{c}$. As a matter of goal
accomplishment, we refer as a delay outage, to the event
$\\{D_{k}^{\textrm{tot}}>D_{k}^{\max}\\}$, i.e., a delay outage occurs if the
total inference delay of a request exceeds a predefined threshold, required by
the device during a service agreement phase. However, this represents only the
first step of goal accomplishment, as from a goal perspective (correct in-time
result), an outage does not only depend on the delay of edge inference task
execution, but also on the inference effectiveness, as it follows in the next
section.

###  III-B Inference effectiveness and goal outage

The inference effectiveness can be directly represented by the accuracy of a
task, or by any other metric that can quantify the success of inference
operations. For example, for a classification task, it can be represented by
the correct classification rate, or by the entropy computed on a posteriori
probabilities at the output of a discriminative classifier. Instantaneously,
an inference outage occurs whenever a predefined measure does not meet a
target threshold, e.g., the mean squared error in a regression task is higher
than a value, or the predicted label in a classification task is incorrect.
Let us refer to this measure as inference value. Let us denote by
$\Theta_{k,j}\in\\{0,1\\}$ a binary variable that equals $1$ if and only if
the inference output related to a pattern generated by device $k$ and
processed by MEH $j$, is accurate. For example, for an image classification
task, $\Theta_{k,j}=1$ if an image generated by device $k$ is correctly
classified. Thus, we can refer to the event $\\{\Theta_{k,j}=0\\}$, as
inferencing outage. When exploiting ensemble inference, the inference
effectiveness can be improved by requesting the same inference task
computation to be addressed by both MEHs, with the aim of exploiting
computation and/or inference diversity. Therefore, we can write an aggregated
inference value as $\Theta_{k}=\displaystyle\Theta_{k,p}\cup\Theta_{k,h}$,
where the union operator represents an aggregation of inference values. Let us
note that this operation can be implemented in several ways, such as majority
voting and minimum entropy in classification tasks, or any other combining
strategy for different inference tasks. Finally, for a generic pattern
generated by device $k$, a goal outage event occurs if either a delay outage
or an inference outage occurs. Therefore, we can write the goal accomplishment
event as $\\{\\{\Theta_{k}=1\\}\;\cap\;\\{D_{k}^{\textrm{tot}}\leq
D_{k}^{\max}\\}\\}$.

###  III-C Energy consumption as the cost of a goal

In our scenario, we can identify two sources of energy consumption: i) the end
devices, consuming energy to transmit data; ii) the MEHs, consuming energy to
process data.

####  III-C1 Uplink transmission energy consumption

We assume that the device energy consumption is only due to the uplink
transmission. Therefore, recalling (1), the $k$-th device energy consumption
for uploading a given pattern for inference is $E_{k}^{u}=p_{k}^{u}D_{k}^{u}$,
which, similarly to the uplink transmission delay, is affected by both
transmit power and BER requirements.

####  III-C2 MEH computation energy consumption

The energy consumed by MEH $j$ to satisfy one inference request of device $k$
can be written as $E_{k,j}=\kappa_{j}f_{k,j}^{2}J_{k,j}$ [19], where
$\kappa_{j}$ is the effective switched capacitance of the processor. Thus, the
total energy consumption of both involved MEHs can be written as
$E_{\textrm{mec}}=\sum\nolimits_{k=1}^{K}\left(E_{k,p}+E_{k,h}\right)$, where
we recall that $K$ is the number of devices in the system.

###  III-D The trade-off between energy consumption & goal outage

The aim of this paper is to show the trade-off between average system energy
consumption and goal accomplishment rate, which we term after as goal
effectiveness. The former translates into the expectation of the energy terms
described in Section III-C, taken with respect to random parameters that
include: i) wireless fading channels ($h_{k}^{u},\forall k$); ii) MEH
availability ($\beta_{k,i},\forall k,i$). The latter, recalling the goal
accomplishment event presented in Section III-B, reads as:

$P_{k}^{g}=\mathbb{E}\left\\{\mathbf{1}\\{\Theta_{k,p}=1\\}\cdot\mathbf{1}\\{D_{k}^{\textrm{tot}}\leq
D_{k}^{\max}\\}\right\\},\;\;\forall k$ (4)

where $\mathbf{1}\\{\cdot\\}$ denotes the indicator function, i.e. the
argument of the expectation is a Bernoulli random variable, so that its
expectation is indeed the probability of the event. It should be noted that,
in this work, we do not perform any particular optimization of the introduced
parameters (e.g., power, computing resources, helper MEH selection, etc.), as
the aim is an exploration of the performance aimed at providing insights on
system deployment requirements, strategies, and optimization variables to be
controlled during system operations. A full optimization of the system is
beyond the scope of this paper and is left for future investigations. In the
following section, we detail the operation of standalone (i.e., single MEH)
and ensemble inference, from a goal effectiveness perspective.

##  IV Standalone versus ensemble inference

We assume that a central controller at network side, which is aware of the
goal requirements of a client application, collects information on channel
realizations, MEHs availability, and inter-MEH backhaul delay, aiming to
decide upon which MEH (MEH $p$, MEH $h$, or both) is selected to serve each
incoming inference request. The central controller also provides as its output
feedback (e.g., via a downlink control channel) on MEH resource availability,
that can be used by each requesting device to regulate its uplink transmit
power towards meeting the E2E delay requirement of the goal.
Standalone inference: Standalone inference takes place whenever MEH $p$ cannot
rely on the aid of a MEH $h$. In this case, only MEH $p$ can host devices’
requests, and a correct in-time inference is guaranteed only if MEH $p$ has
enough computing capacity to guarantee the delay constraint (given the current
connect-compute network conditions) and, at the same time, the inference is
performed with the targeted inference value. To save device energy, we assume
the device uplink transmit power to be chosen to exactly meet the delay
constraint, i.e., so as to meet a transmission delay
$D_{k}^{u}=\max\left(0,D_{k}^{\max}-\frac{J_{k,p}}{f_{k,p}}\right).$ If the
goal cannot be achieved with the device maximum power $p_{k}^{u,\max}$, we
assume that the device transmits at $p_{k}^{u,\max}$, thus getting the
inference results as soon as possible, i.e., with a best effort philosophy.
Ensemble inference: Ensemble inference takes place whenever both primary and
helper MEHs cooperate in processing inferencing requests, and it can end up in
the following cases, depending on the current connect-compute network
conditions: i) only the primary MEH performs inference; ii) only the helper
MEH performs inference; iii) both MEHs perform inference, and the result is
obtained from a combination of both results (cooperative inference). The
latter is obtained by a requesting device sending a pattern to the primary
MEH, which, in its turn, sends a copy of the pattern to the helper MEH, while,
in the meantime, starts processing the task; the device, in this case will
await for the response of both primary and helper. This operation is feasible
iff
$D_{k}^{u,\min}+\max\left(\frac{J_{k,p}}{f_{k,p}},\frac{J_{k,h}}{f_{k,h}}+D_{p,h}^{\textrm{bh}}\right)\leq
D_{k}^{\max}$ (cf. (3)), where $D_{k}^{u,\min}$ is the minimum communication
delay, obtained by plugging the maximum device transmit power $p_{k}^{u,\max}$
into (1). If this feasibility check returns a positive result, device $k$’s
request is treated by means of a cooperation, choosing its transmit power so
as to guarantee a transmission delay
$D_{k}^{u}=D_{k}^{\max}-\max\left(\frac{J_{k,p}}{f_{k,p}},\frac{J_{k,h}}{f_{k,h}}+D_{p,h}^{\textrm{bh}}\right)$.
On the other hand, if cooperation is not feasible under the current
conditions, standalone inference is performed at the fastest MEH, with
transmit power selected to guarantee the delay constraint, and with the best
effort strategy in case of infeasibility.

##  V Performance evaluation & discussion

(a) Goal effectiveness vs. device energy cons.

(b) Goal effectiveness vs. BER.

(c) Device energy cons. vs. BER.

(d) MEC system energy consumption.

(e) Goal effectiveness for $D_{p,h}^{\textrm{bh}}=0.25D_{1}^{\max}$.

(f) Goal effectiveness for $D_{p,h}^{\textrm{bh}}=0.75D_{1}^{\max}$.

Figure 2: Goal achievement and system costs with different connect-compute
network conditions.

For the sake of performance evaluation, we focus on an edge image
classification task. As indicator of inference effectiveness, we use the
correct classification over a test set.
Communication parameters: In our performance evaluation, we consider a single
device scenario, for the sake of simplicity in commenting the results.
However, a multi-device scenario would not strongly affect the take home
message of the simulations, which would only be influenced by the chosen task
scheduling strategies at both communication and computation tiers. Device
position is chosen uniformly randomly in a circle of radius $150$ meters
around a single AP, serving it at $3.5$ GHz, with bandwidth of $B=10$ MHz. We
generate the channels with path loss [20], and Rayleigh fading with zero mean
and unit variance. At the receiver, the noise power spectral density is
$N_{0}=-174$ dBm/Hz, while the maximum device transmit power is
$p_{1}^{u,\max}=20$ dBm.
Computation parameters: At the MEC deployment side, we consider the primary
MEH collocated with the AP, and one helper MEH, reachable within finite RTT
$D_{p,h}^{\textrm{bh}}$, to be specified for each simulation. Each MEH
operates at maximum CPU frequency of $f_{h}^{\max}=f_{p}^{\max}=4.5$ GHz.
Furthermore, we assume $\beta_{1,j},j=p,h$ (cf. (2)), to address device
requests, taking different values to compare performance in terms of MEHs
availability. Specifically, we assume $\beta_{1,p}$ and $\beta_{1,h}$ to be
uniformly distributed (in three different simulations) between $0$ and
$\beta_{1,j}^{\max}=[1,0.5,0.25],j=p,h$, where we also aim to capture the full
(deterministic) MEH availability benchmark. The delay threshold is set to
$D_{1}^{\max}=100$ ms.
Learning parameters: We evaluate the performance on the CIFAR-10 dataset [21],
which consists of 60000 32x32 pixel color images divided in 10 classes, with
6000 images per class. The training set is made of 50000 instances, which we
split into two equal sub-data sets to train the Convolutional Neural Networks
(CNNs) that are deployed at primary and helper MEHs, with the purpose of
learning diversity, so as to improve inference performance in the case of
cooperative inference. The test set is made of 10000 instances. We use a Deep
CNN trained via usual backpropagation with an $l_{2}$-penalized cross-entropy
loss, and an existing
architecture222https://appliedmachinelearning.blog/2018/03/24/achieving-90-accuracy-
in-object-recognition-task-on-cifar-10-dataset-with-keras-convolutional-
neural-networks/. Whenever exploited, the ensemble is built by summing the
scores (i.e., a posteriori probabilities) of the two classifiers, and choosing
the label with the highest aggregated score [16].
As already mentioned in Section III-D, results in terms of energy consumption
and goal effectiveness are obtained by averaging over realizations of wireless
channels and MEH’s availability. More specifically, we consider $10^{5}$
realizations.

###  V-A The trade-off between goal effectiveness and costs

As a first performance evaluation, in Fig. 2a, we show the goal effectiveness
(cf. (4)) as a function of the device transmit energy, obtained by varying the
BER requirements during transmission. Also, to have a complete view of system
performance, Figs. 2b and 2c show, respectively, the goal effectiveness and
the device energy consumption as functions of the same BER requirements used
to obtain Fig. 2a. In these figures, the different colors encode different
values of $\beta_{1,j}^{\max}$, $j=p,h$ (cf. (2)). At the same time, solid
lines represent standalone inference (i.e., only the primary MEH participates
in the inference process, without helper involvement), while dashed dotted
lines represent the ensemble inference case. Let us first focus on the solid
lines (i.e., standalone inference), to gain insights on goal effectiveness
with respect to BER requirements (i.e., classical communication reliability)
and device energy costs. Ensemble performance analysis will then come
naturally as an enhancement of the standalone case, to be described in what
follows below.
The effect of BER on goal effectiveness: First of all, we can notice how the
effectiveness increases, as the device transmit energy consumption increases,
as expected, with different performance depending on MEH’s availability
levels. This is due to the fact that the BER requirements are stricter (cf.
(1)). However, we can appreciate how, above a given threshold, BER
requirements do not strongly affect goal effectiveness, meaning that a
slightly increased BER leads to similar performance in terms of the
effectiveness, in all network conditions (obviously with different values
according to MEH availability). It is noteworthy that effectiveness is
invariant to communication reliability changes in the range
$\textrm{BER}=[10^{-4}$, $10^{-3}]$ (see Fig. 2b). This first observation
(obviously dependent on the specific application) suggests that classical
communication related reliability performance can be relaxed almost without
any impact on goal effectiveness, and, as a by-product of such relaxation, non
negligible gains in terms of device energy consumption are expected, as
visible from Fig. 2c, where the device energy consumption is plotted as a
function of the BER, for $\beta_{1,i}^{\max}=1,i=p,h$.
The effect of ensemble inference: Having appreciated that, for certain system
regimes, goal effectiveness may be kept almost constant by significantly
relaxing classical communication related reliability, we can now analyze the
beneficial effect of the helper MEH presence into the system. Indeed, from the
dashed dotted lines, we can notice how goal effectiveness experiences a great
improvement, if compared to the standalone inference case, on the basis of the
same device energy consumption cost (i.e., the same BER requirements).
Equivalently, the same effectiveness can be achieved at a lower goal cost, by
further relaxing the BER requirements (e.g., see the $80$ % effectiveness
case).
Of course, the lower energy consumption at the device side translates into a
higher energy consumption of the MEHs, as visible from Fig. 2d, where we show
the energy consumption of the MEC system for a single inference request, as a
function of $\beta_{1,i},i=p,h$, in case of standalone (first bar) and
ensemble inference (second bar).
By comparing standalone and ensemble performance for the same MEH
availability, we obtain an approximately doubled energy consumption at the MEC
infrastructure level for the ensemble case, as visible from Fig. 2d, due to
the obvious fact that, on average, the two MEHs address requests at the same
CPU speed. Nevertheless, we can notice an interesting outcome: by comparing
the ensemble inference at half maximum available MEH CPU (red dashed dotted
line), with the standalone inference at full maximum available CPU
availability (blue solid line), better performance in terms of effectiveness
against device energy consumption is guaranteed for the prior case, while also
achieving a lower energy consumption at the MEC network infrastructure level.
This result is due to the quadratic law of the dynamic CPU energy consumption
with respect to the CPU cycle frequency, and it shows that better performance
in terms of goal achievement can be obtained without paying any price (but
actually experiencing a gain), in terms of network energy consumption. In
particular, this behavior suggests that, in the case of ensemble inference, it
is convenient to have two available CPUs at half availability, rather than one
CPU at full availability (on average), for a twofold reason: i) the total MEC
energy consumption decreases; ii) the reliability improves, since single point
of failure issues are counteracted. Let us also notice that, for the
deterministic, the full CPU, and the half CPU availability cases, ensemble
inference outperforms the deterministic standalone method, due to the fact
that the two classifiers cooperate to improve their standalone capabilities.
Also, the ensemble inference at 25% CPU availability outperforms the
deterministic standalone classifier, with a greatly decreased energy
consumption (around $20$ times) at the MEC infrastructure level.
The effect of the backhaul delay: Of course, this remarkable superiority of
the ensemble setting is dictated by the zero delay backhaul assumption assumed
in this first simulation, which is more than ideal in practice. To this end,
we now show the effect of the backahul RTT on the performance enhancement of
the ensemble with respect to the standalone inference case. In Figs. 2e, 2f,
we plot the effectiveness as a function of the device energy consumption (i.e.
the same result as Fig. 2a), for two different backhaul RTT conditions, namely
$D_{p,h}^{\textrm{bh}}=0.25D_{1}^{\max}$ and
$D_{p,h}^{\textrm{bh}}=0.75D_{1}^{\max}$, respectively. Five MEC network
conditions are considered in each figure: i) the deterministic standalone ii)
the deterministic ensemble; iii) the ensemble with
$\beta_{1,i}^{\max}=1,i=p,h$; iv) the ensemble with
$\beta_{1,i}^{\max}=0.5,i=p,h$; v) the ensemble with
$\beta_{1,i}^{\max}=0.25,i=p,h$. From the plots, we can notice how the effect
of the backhaul RTT plays a key role in determining the beneficial effects of
the helper in the edge inference task, which decreases as the backhaul RTT
increases. Indeed, in Fig. 2f, the ensemble exhibits even degraded performance
compared with the standalone, although this is due to the fact that only the
deterministic standalone (i.e., its best case) is shown in the figure, which
translates into a higher network energy consumption (Fig. 2d).

##  VI Conclusions

In this paper, we presented the performance evaluation of an edge inference
service, following a goal-oriented approach, in which system performance is
not measured by classical radio reliability parameters (such as the BER), but
rather measured by the actual outcome of a communication strategy with respect
to the effectiveness of a properly defined goal. We described a MEC-enabled
wireless network in which multiple MEHs can cooperate to improve edge
inference performance, while trading off end device and MEC network energy as
costs of the goal. Through numerical simulations, we have shown how the
effectiveness can achieve good performance also in the presence of relaxed BER
requirements, suggesting that high communication reliability is not needed, as
far as inference performance reaches target levels. Furthermore, we have shown
how ensemble inference (implemented thanks to MEHs’ cooperation) can help
improving goal performance, depending on specific deployment characteristics,
such as the availability of MEHs and the backhaul delay interconnecting them.
We believe that such a goal-oriented performance evaluation can pave the way
for system deployment considerations involving communication and computing
perspectives. Also, thanks to the gained insights on system performance, we
hope that cross-layer optimization strategies can build on these results, to
further improve trade-offs involving energy consumption and effectiveness in
future wireless networks.

## References

  * [1] M. A. Uusitalo _et al._ , “6G vision, value, use cases and technologies from European 6G flagship project Hexa-X,” _IEEE Access_ , vol. 9, 2021.
  * [2] W. Saad, M. Bennis, and M. Chen, “A vision of 6G wireless systems: Applications, trends, technologies, and open research problems,” _IEEE Network_ , vol. 34, no. 3, pp. 134–142, 2020.
  * [3] S. Kekki, W. Featherstone, Y. Fang, P. Kuure, A. Li, A. Ranjan, D. Purkayastha, F. Jiangping, D. Frydman, G. Verin _et al._ , “ETSI White Paper: MEC in 5G networks,” _The European Telecommunications Standards Institute (ETSI), Tech. Rep. ETSI White Paper No. 28_ , 2018.
  * [4] M. Chen and Y. Hao, “Task offloading for mobile edge computing in software defined ultra-dense network,” _IEEE Journal on Selected Areas in Communications_ , vol. 36, no. 3, pp. 587–597, 2018.
  * [5] M. Emara, H. ElSawy, M. C. Filippou, and G. Bauch, “Spatiotemporal dependable task execution services in MEC-enabled wireless systems,” _IEEE Wireless Comm. Letters_ , vol. 10, no. 2, pp. 211–215, 2021.
  * [6] K. B. Letaief, Y. Shi, J. Lu, and J. Lu, “Edge Artificial Intelligence for 6G: Vision, Enabling Technologies, and Applications,” _IEEE Journal on Selected Areas in Communications_ , vol. 40, no. 1, pp. 5–36, 2022.
  * [7] E. Calvanese Strinati and S. Barbarossa, “6G networks: Beyond Shannon towards semantic and goal-oriented communications,” _Computer Networks_ , vol. 190, p. 107930, 2021.
  * [8] Z. Zhou _et al._ , “Edge Intelligence: Paving the Last Mile of Artificial Intelligence With Edge Computing,” _Proceedings of the IEEE_ , vol. 107, no. 8, pp. 1738–1762, 2019.
  * [9] X. Ran, H. Chen, X. Zhu, Z. Liu, and J. Chen, “DeepDecision: A Mobile Deep Learning Framework for Edge Video Analytics,” in _IEEE INFOCOM 2018 - IEEE Conference on Computer Communications_ , 2018, pp. 1421–1429.
  * [10] A. Galanopoulos, J. A. Ayala-Romero, D. J. Leith, and G. Iosifidis, “AutoML for Video Analytics with Edge Computing,” in _IEEE INFOCOM 2021 - IEEE Conference on Computer Comm._ , 2021, pp. 1–10.
  * [11] M. Merluzzi, P. Di Lorenzo, and S. Barbarossa, “Wireless Edge Machine Learning: Resource Allocation and Trade-Offs,” _IEEE Access_ , vol. 9, pp. 45 377–45 398, 2021.
  * [12] Y. Shi, K. Yang, T. Jiang, J. Zhang, and K. B. Letaief, “Communication-efficient edge AI: Algorithms and systems,” _IEEE Comm. Surveys Tutorials_ , vol. 22, no. 4, pp. 2167–2191, 2020.
  * [13] M. Merluzzi, A. Martino, F. Costanzo, P. Di Lorenzo, and S. Barbarossa, “Dynamic ensemble inference at the edge,” in _2021 IEEE Global Communications Conference (GLOBECOM)_ , 2021, pp. 1–6.
  * [14] J. Shao and J. Zhang, “Bottlenet++: An end-to-end approach for feature compression in device-edge co-inference systems,” in _2020 IEEE International Conference on Communications Workshops (ICC Workshops)_ , 2020, pp. 1–6.
  * [15] C.-H. Lee, J.-W. Lin, P.-H. Chen, and Y.-C. Chang, “Deep learning-constructed joint transmission-recognition for internet of things,” _IEEE Access_ , vol. 7, pp. 76 547–76 561, 2019.
  * [16] N. Shlezinger, E. Farhan, H. Morgenstern, and Y. C. Eldar, “Collaborative inference via ensembles on the edge,” in _ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_ , 2021, pp. 8478–8482.
  * [17] K. Li, M. Tao, and Z. Chen, “Exploiting computation replication for mobile edge computing: A fundamental computation-communication tradeoff study,” _IEEE Transactions on Wireless Communications_ , vol. 19, no. 7, pp. 4563–4578, 2020.
  * [18] A. Goldsmith and S.-G. Chua, “Variable-rate variable-power MQAM for fading channels,” _IEEE Transactions on Communications_ , vol. 45, no. 10, pp. 1218–1230, 1997.
  * [19] W. Yuan and K. Nahrstedt, “Energy-efficient CPU scheduling for multimedia applications,” _ACM Trans. Comput. Syst._ , vol. 24, no. 3, p. 292–331, aug 2006. [Online]. Available: https://doi.org/10.1145/1151690.1151693
  * [20] 3GPP, “Scenarios, frequencies and new field measurement results from two operational factory halls at 3.5 GHz for various antenna configurations,” _3GPP TSG RAN WG1 Meet. 95, Spokane,USA_ , 2018.
  * [21] A. Krizhevsky and G. Hinton, “Learning multiple layers of features from tiny images,” University of Toronto, Toronto, Ontario, Tech. Rep. 0, 2009.


# Investigating Randomly Generated Adjacency Matrices For Their Use In
Modeling Wireless Topologies

Gautam Bhanage and Sanjit Kaul
{gautamb<EMAIL_ADDRESS>
WINLAB, Rutgers University, North Brunswick, NJ 08902, USA

###### Abstract

Generation of realistic topologies plays an important role in determining the
accuracy and validity of simulation studies. This study presents a discussion
to justify why, and how often randomly generated adjacency matrices may not
not conform to wireless topologies in the physical world. Specifically, it
shows through analysis and random trials that, _more than $90\%$ of times, a
randomly generated adjacency matrix will not conform to a valid wireless
topology, when it has more than $3$ nodes_. By showing that node triplets in
the adjacency graph need to adhere to rules of a geometric vector space, the
study shows that the number of randomly chosen node triplets failing
consistency checks grow at the order of $O(base^{3})$, where $base$ is the
granularity of the distance metric. Further, the study models and presents a
probability estimate with which any randomly generated adjacency matrix would
fail realization. This information could be used to design simpler algorithms
for generating _k-connected_ wireless topologies.

##  I Introduction

Simulation studies can be easily setup for wired networks by generating a
random adjacency matrix for modeling a random topology. As long as finite non-
negative entries are chosen for the adjacency matrix, it could be used to
represent a valid wired topology. However, in this paper we discuss how, and
why this may not hold true in the case of wireless topologies.

Specifically, this study addresses the following questions:

  1. 1.

_Correctness:_ Are randomly generated topologies always valid, if not under
what conditions.

  2. 2.

_Frequency of Failure:_ What percentage of randomly generated matrices are
invalid?

  3. 3.

_Dominant Failure Factor:_ What feature of the matrix decides the probability
of the topology being invalid?

  4. 4.

_Implication:_ Using this understanding, we propose designing algorithms with
a relatively direct approach for generating k-connected graphs.

Rest of the paper is organized as follows. Section II shows an example where a
randomly generated adjacency matrix does not represent a wireless topology.
Section III describes the problem statement, and present our approach for
determination of valid matrices. Section IV presents a comparison of results
from random trials with an approximation generated by our probability
function. Finally, we present a brief conclusion.

##  II Discussion

Figure 1: Mapping problem definition as seen in $\Re^{1}$ space. If we have
node B connected to A and C, we cannot have another node D with connectivity
to A and C but not connected to B on the number line in $\Re^{1}$ space.

###  II-A Example Of An Invalid Wireless Topology

We first address the question of whether a randomly generated adjacency matrix
can result in a non-realizable wireless topology. Figure 1 shows the positions
of three previously mapped points A, B, and C in a one dimensional metric
space ($\Re^{1}$). For this problem, we consider that all nodes have similar
radio capabilities and can communicate with each other only if they are within
_unit_ distance of each other. As per this condition, we have node B connected
to nodes A and C.

Now consider a case where the adjacency matrix generating the topology in
Figure 1 has an additional entry for a fourth node D, which has links to A and
C but is not within coverage of node B. Such a wireless topology is physically
not possible in the one dimensional metric space ($\Re^{1}$). Note that this
problem cannot be solved by using a different channel, since all the nodes
will need to be on the same frequency to be connected111We refer to a
_connection_ between any nodes 1 and 2, as a general term to signify that 1
and 2 have a significant SNR to communicate with each other. This
_connectivity_ is at the layer-1 and is independent of any access control
mechanisms used at a higher layer in the network stack.. It is also important
to observe that this failure occurs even when we do not have any planarity
constraints like requiring non-intersecting graph edges. Non-uniform radio
coverage for nodes D and B also fails to solve the problem. This is because
both nodes D and B need to be on the line, and a non-uniform radio coverage in
either directions (left or right) will result in disconnection from nodes A
and/or C.

This problem can be extended to all higher dimensions in metric spaces
$\Re^{n}$, $n>0$, which could result in invalid physical topologies. The only
factor that varies across these dimensions is the nature of the wireless
coverage. In $\Re^{1}$, we consider a line of unit (manhattan) distance on
each side of the node, in $\Re^{2}$, the coverage can be assumed in the form
of a unit circle in the plane (euclidean distance), similarly, a unit sphere
in $\Re^{3}$ and so on.

###  II-B Problem Statement

Now that we have shown an example of an invalid wireless topology generation,
we will explicitly define the problem. Consider a network graph G which is
generated by a random adjacency matrix $A_{adj}[\texttt{ }]_{n\times n}$,
where $n$ denotes the number of nodes in the graph G. The individual entries
in $A_{adj}$ will denote the link conditions between corresponding wireless
nodes. In this study, given a specific $A_{adj}[\texttt{ }]$, we will define a
function _F_() to tell us whether the given adjacency matrix is capable of
realizing a valid wireless topology or not:

$F:G(A_{adj}[\texttt{ }]_{n\times n})\mapsto\\{Valid,Invalid\\}$ (1)

Once determined, _F_() can be used as a test for incrementally adding neighbor
nodes to an adjacency matrix for generating _k-connected_ graphs. We also
calculate the probability $(P_{F})$ with which _F_() will fail, which could be
used as a metric for determining the average number of trials that would be
required for valid wireless topology generation.

##  III Modeling

###  III-A Wireless Topologies $\&$ Vector Spaces

To determine _F_() defined above, we briefly discuss why the random adjacency
matrix used for Figure 1 fails. If we consider, the first three nodes A, B, C,
we observe that they satisfy the triangle inequality requirements in the
$\Re^{1}$ metric space. Let $\parallel.\parallel$ represent an arbitrary
distance norm. Triangle inequality requirement states that the sum of the
lengths of any two sides (say $\parallel x\parallel+\parallel y\parallel$) has
to be greater than the third side ($\parallel x+y\parallel$). While mapping
the fourth node D, with the requirement $A_{adj}(A,D)=1$, $A_{adj}(D,C)=1$,
and $A_{adj}(B,D)=0$, we observe that the triangle inequality fails for the
node sets $\\{B,D,A\\}$ and $\\{B,D,C\\}$. Thus using simple triangle
inequality as a test for the function $F$, i.e by determining if the generated
wireless topology fits in a geometric vector space, we can classify random
matrices as representing valid or invalid wireless topologies.

###  III-B Estimating Adjacency Matrix Failure Probability $(P_{F})$

A randomly generated adjacency matrix $A_{adj}$ for a wireless topology with
$n$ nodes will fail when any one combination of three links fails the triangle
inequality check. Hence, the probability of at least one failure is:
$P_{F}=1-P_{NF},$ where the $P_{NF}$ is the probability that no combination in
the adjacency graphs fails the triangle inequality check. Thus, $P_{F}$ can be
calculated as:

$P_{F}=1-(1-P_{\triangle})^{N_{pairs}},$ (2)

where $N_{pairs}$ denotes the number of combinations of nodes checked in a
randomly generated matrix, and $P_{\triangle}$ denotes the probability of
failure of the triangle inequality on any random adjacency triplet. We define
an _adjacency triplet_ as any single combination of three values
$A_{adj}(P,Q)$, $A_{adj}(Q,R)$ and $A_{adj}(P,R)$ that describe link
conditions between any three nodes P, Q and R. The $N_{pairs}$ are determined
by the number of non-diagonal entries in the adjacency matrix. Since the
adjacency matrix is representing a wireless topology, it should be symmetric
akin to a metric space distance matrix. Hence,
$N_{pairs}=(\frac{n^{2}-n}{2})\times(\frac{n^{2}-n}{2}-1)$.

Figure 2: Probability of failure of triangle inequality tests for unique
combinations and permutations of side triplets.

###  III-C Determining $P_{\triangle}$

To estimate $P_{\triangle}$, we use the complete set of adjacency triplets
($S_{3}$) described as:

$\texttt{ }S_{3}=\\{A_{adj}(P,Q),\texttt{ }A_{adj}(Q,R),\texttt{
}A_{adj}(P,R)\\},$ (3)

defined $\forall P,Q,R\in A_{adj}$. To determine $P_{\triangle}$ we can either
use combinations or permutations on $S_{3}$ to determine failure probability
of combinations. For all such possible permutations and combinations over
$S_{3}$, we determine $P_{\triangle}$ by calculating the fraction of adjacency
triplets that fail strict ($\leq$) and non-strict ($\leq$) triangle inequality
checks. While evaluating, we vary the _base_ , which denotes the maximum
number of discretized values that can be used to represent the link between
two points. E.g. when we chose the base as 1, the link represented in the
adjacency matrix can either take the values as 0 (off) or 1(on). If the base
is 2, the possible values are 0,1,2 and so on. Results for this model are as
described in Figure 2.

We observe that the fraction of permutations or combinations resulting in the
triangle inequality failure remain fairly constant, irrespective of the
increasing number base. Also, we note that the number of combinations being
evaluated are growing as $O(base^{3})$. Hence, we conclude that, the number of
unique combinations failing are also increasing at $O(base^{3})$ to keep the
ratio constant.

##  IV Monte Carlo Tests

In this section we estimate and compare the probability with which a randomly
generated adjacency matrix will fail when mapped as a wireless topology. We
compare our failure probability estimate ($P_{F}$) with the failure
probability obtained through randomized trials. In these comparisons, we use
the discreteness of link qualities (or the discreteness of distance) and the
size of wireless topologies as the two varying parameters.

###  IV-A Discreteness Of Link Connectivity Representation

Figure 3: Probability of failure of a randomly generated adjacency matrix in
representing a wireless topology as a function of the discreteness of distance
or connectivity.

In this test, as with the estimation of $P_{\triangle}$, we vary the maximum
number of discretized values (_base_) that can be used to represent the link
between two points. Results from our estimates and those from monte-carlo
tests are correspondingly marked as _Estimate:*_ and _MTC:*_ in the Figure 3.
The results show that our estimate of $P_{F}$ is able to closely match the
failure probability obtained from trials of $1000$ randomly generated
adjacency matrices for every distance _base_. For all topology sizes: $2,3,4$
(nodes each), we observe that the probability of the matrix failing to conform
to a wireless topology ($P_{F}$) can be high when the link connectivity is
coarsely described (E.g. on or off). This result is a direct consequence of:
$P_{F}\propto P_{\triangle}$. Hence, we observe that as $P_{\triangle}$
stabilizes for higher values of the distance _base_ , $P_{F}$ stabilizes too.

Figure 4: Comparison of estimated and observed failure probability as a
function of varying topology size.

###  IV-B Impact Of Varying Topology Size

The size of a topology can be changed by varying the number of nodes. Edges
are not explicitly used as a factor for changing topology size since the
number and type of edges are randomly decided. In this experiment, we vary the
size of the wireless topology from 1 to 10 nodes. For every topology, an edge
can have a value uniformly distributed among the number of discretized
distance values given by the _base_. We generate $1000$ random matrices for
each topology size.

As shown in the results in Figure 4 the estimated failure probability $P_{F}$
(denoted by _Estimate:*_) closely matches that obtained from random trials
(_MTC:*_). Further, we observe that failure probability quickly approaches 1.
This matches with our estimate since, $P_{F}\propto N_{pairs}$, and
$N_{pairs}$ increase at least as $O(n^{2})$. An important implication of this
result is that as the size of the wireless topology goes beyond $3$ nodes, it
is almost certain that a randomly generated adjacency matrix will not conform
to a wireless topology.

##  V Related Work

A class of studies has focussed on enumerating the characteristics of wired
networks [1, 2] that need to be taken into consideration while generating
topologies from random graphs. Consequently, a parallel area of research is
focussed on efficient generation [3] and improvement of the features of random
graphs to model real wired networks [4].

With concerns to wireless networks, [5] investigates the impact of spatial
distribution of nodes on the minimum node degree, and the k-connectivity in
random network graphs. We take a completely opposite view of the problem in
determining if a randomly generated adjacency could be used for faithfully
representing a realistic wireless topology.

##  VI Conclusions

This study describes an approach for determining if randomly generated
adjacency matrices can conform to wireless topologies in the physical world.
It is shown that these random matrices are prone to failure, specially for
topologies with more than 3 nodes. Using this information, an alternative
approach can be taken for random wireless topology creation. Instead of
designing simulation studies based on random placement of nodes and then
generating k-connected graphs, algorithms could be designed that would
iteratively add nodes to the adjacency graph based on k-connectivity
requirements as long as they do not violate constraints of the geometric
space.

## References

  * [1] M. B. Doar and A. Nexion, “A better model for generating test networks,” in _IEEE Global Telecommunications Conf. (Globecomm)_ , 1996, pp. 86–93.
  * [2] E. Zegura, K. Calvert, and S. Bhattacharjee, “How to model an internetwork,” in _In Proceedings of IEEE INFOCOM_ , 1996, pp. 594–602.
  * [3] J. Leskovec, D. Chakrabarti, J. Kleinberg, and C. Faloutsos, “Realistic, mathematically tractable graph generation and evolution, using kronecker multiplication,” _Lecture Notes in Computer Science: Knowledge Discovery In Databases_ , vol. 3721, pp. 133–145, 2005.
  * [4] A. Rodionov and H. Choo, “On generating random network structures: Connected graphs,” _Lecture Notes in Computer Science: Information Networking_ , vol. 3090, pp. 483–491, 2004.
  * [5] C. Bettstetter, “On the minimum node degree and connectivity of a wireless multihop network,” in _MobiHoc ’02: Proceedings of the 3rd ACM international symposium on Mobile ad hoc networking & computing_. New York, NY, USA: ACM, 2002, pp. 80–91.


11institutetext: Helmholz-Zentrum Dresden-Rossendorf, Institute of Radiation
Physics, 01328 Dresden, Germany 22institutetext: Technische Universität
Dresden, 01069 Dresden, Germany

# Development of a jet gas target system for the Felsenkeller underground
accelerator

Anup Yadav Konrad Schmidt Daniel Bemmerer 1122<EMAIL_ADDRESS>11
<EMAIL_ADDRESS>1122

###### Abstract

For direct cross section measurements in nuclear astrophysics, in addition to
suitable ion beams and detectors, also highly pure and stable targets are
needed. Here, using a gas jet as a target offers an attractive approach that
combines high stability even under significant beam load with excellent purity
and high localisation. Such a target is currently under construction at the
Felsenkeller underground ion accelerator lab for nuclear astrophysics in
Dresden, Germany. The target thickness will be measured by optical
interferometry, allowing an in-situ thickness determination including also
beam-induced effects. The contribution reports on the status of this new
system and outlines possible applications in nuclear astrophysics.

##  1 Introduction

The Felsenkeller underground laboratory is suitable to study, amongst others,
two important astrophysical reactions of interest: 3He($\alpha,\gamma$)7Be and
12C($\alpha,\gamma$)16O. The first reaction controls the hydrogen burning in
the Sun and also affects lithium production in Big Bang Nucleosynthesis (BBN).
In order to measure the astrophysical S-factor and angular distribution of
this reaction, we are planning to use a 3He gas jet target for comprehensive
data coverage of the entire BBN range.

The 12C($\alpha,\gamma$)16O reaction affects the abundance ratio of the two
elements carbon and oxygen. Due to the low cross-section in the Gamow window,
significant statistics for this reaction will be achieved with 12C beam
incident on a static windowless 4He gas target, and a 4$\pi$ detector.

To measure reactions of astrophysical interest targets are needed that are
locally well confined, highly dense, thermally stable, and chemically pure.
Measurements of the angular distribution require a highly localized target for
a high-resolution of the reaction energy and the emission angle. A solid
target often contains other elements through contaminations and requires
backing materials like tantalum for mechanical stability, which may increase
the beam induced background and energy straggling in case of implantation.
Solid targets can also degrade over time or exposure to a beam resulting in
issues like overheating, and blistering.

A wall-jet gas-target is an alternative solution, which can provide a high gas
density, high localization, chemically purity, and a thermal stability
KSchmidt ; 2 ; 4 ; 5 .

Figure 1: Astrophysical S-factor of the 3He($\alpha,\gamma$)7Be reaction as a
function of energy. The Felsenkeller gas-jet will help to perform
comprehensive measurements of the reaction covering the entire BBN range.

##  2 Experimental setup

###  2.1 Differential pumping system

Figure 2: Technical drawing of differential pumping stages. The beam enters at
the left and passes through apertures in between the pumping stages, the jet
chamber and the static on the most right where it is stopped in a beam
calorimeter. The pumping stages are evacuated by three 700 l/s and the jet
chamber by a 2300 l/s turbomolecular pump.

A recirculating gas-jet target (Fig. 2) creates a supersonic jet based on the
JENSA gas jet target KSchmidt . Here, the target density is 10 times less than
at JENSA due to the different physics case. The exhaust of the nozzle has a
cross-sectional area at the neck of less than 1 mm. About 12 mm below the
nozzle a cone-shaped gas catcher receives most of the gas jet. Catcher is
connected to a cone-shaped chamber, and this chamber is evacuated by a 2300
liter turbomolecular pump. The exhaust of all the pumps are connected to a
multi-stage root pump and this pump will then compress the gas back to the
initial inlet pressure. Before return back to the nozzle, the gas is also
passing a chemical getter that will clean it from residual contamination that
might have enter during the circulation.

Part of the gas jet diffusing in the jet chamber and return to the beam line.
This residual gas is evacuated by a 2300 liter turbomolecular pump on the top
of the jet chamber and three 700 liter turbomolecular pumps along the beam
line, all separated by copper apertures. These collimators reduce the pressure
down to $10^{-7}$ mbar, that is necessary for the ion accelerator. Further,
the apertures also reduce the beam size to a diameter of 5 mm or less, in
order to provide a well defined beam spot. Downstream of the jet, a extended
static gas chamber with a beam calorimeter (see the section 2.3) is installed
to determine the ion beam intensity.

In order to monitor the system, all the pumps, pressure gauges, and valves are
controlled by a LabVIEW system and parameters of the system are logged. The
gas system can be operated either in jet mode or in static mode.

###  2.2 Wall jet gas target

For measurements of low cross-section reactions, a high density of target
nucleui within the confined region is needed to maximize count rate. On the
other hand, a limited target thickness reduces reaction product energy loss
and straggling. The size of the jet should be comparable to the full beam spot
so the beam crosses the jet within the homogeneous center. The inlet pressure
should be sufficiently high to get a supersonic jet. The characteristics of
the gas wall jet that are aimed for, are summarized in Table 1.

The nozzle is placed vertically with the jet flowing downward with two inlet
connections. The shape of the nozzle is a converging-diverging, de Laval type
of nozzle [6]. The distance between the nozzle and catcher is around 12 mm as
suggested by the literature 4 ; 8  and by the beam size. This value can be
changed for future measurements.

Table 1: Aimed characteristics of the gas wall jet Parameter | Aim | Remark
---|---|---
Areal density |  $10^{18}$ atoms/$\mathrm{cm}^{2}$ | Single resonance measurement
Thickness | <1 mm | Reaction localization for angular distribution measurements
Width | 10 mm | Beam (5 mm diameter) crosses jet within homogeneous center
Inlet pressure |  $\sim$1 bar | Needed to get a supersonic jet
Mass flow | 1.4 bar l/s | Result of bottle neck cross section of $\sim$1 mm2

###  2.3 Static gas target with beam calorimeter

Downstream of the gas wall jet, a windowless, extended gas target of static
type (Fig. 3) is installed that contains the beam calorimeter. The concept is
based on the LUNA gas target[2]. In static mode, the gas enters the target
chamber from the right through the pipes shown in grey color. The gas flow is
regulated by two parallel metered valves (EVR 116) to keep a constant pressure
in the chamber. The expected target pressure is around 2.0 mbar.

A collimator between the jet target chamber and the entrance of extended
target limits the gas flow in the direction of the jet chamber. Any remaining
target gas is removed by the jet chamber pumps and three differential pumping
stages (see section 2.1).

It is not possible to use a Faraday cup to measure the beam intensity while
the target chamber is filled with gas because of secondary electrons,
therefore a different technique in form of a power compensation calorimeter is
used. The calorimeter is made from copper and consists of hot side with
heating resistor (12$\Omega$, 60W) inside the copper structure and cold side.
The hot and cold sides are always kept at a constant temperature with the help
of temperature regulated heating resistors at the hot side and circulating
low-temperature coolant from a high precision temperature stability chiller
(Julabo 600F) at the cold side. When the ion beam hits the calorimeter, the
resistor will reduce the heating to maintain a constant temperature measured
by a PT100 sensors. The reduction of the power consumption of the resistor, in
turn, enables the determination of the beam intensity at known beam energy.

Figure 3: Extended gas target chamber and a top view of beam calorimeter are
shown. Two grey pipes at the right side of the figure, to inlet the gas inside
the chamber. Figure 4: Extended target chamber with the side view of beam
calorimeter are shown. Two blue pipes at the right side of the figure are for
water cooling of calorimeter.

##  3 Measurement

###  3.1 Jet density measurement

For a continuous gas jet density measurement, a laser interferometry technique
3  is used, where the presence of gas induces an optical path length
difference between the signal arm (passing through the jet) and the reference
arm resulting in a phase shift on the interferogram. The optical path
difference depends on the density distribution of the gas jet and its
refractive index. The laser beam is expanded to illuminate the entire gas jet
with the help of lenses to see the shape of the jet.

With the use of laser interferometry, the relative density and the shape of a
jet are measured. The absolute density is determined by the energy loss
method, where the energy loss of alpha particles traversing the jet are
combined with tabulated experimental stopping powers at different points of
the jet. For this absolute thickness measurement, a silicon detector will be
mounted at the end of the beam line at one side of the jet and at the other
side a triple alpha source (239Pu, 241Am, 244Cm) will be mounted. In this way,
the energy of the alpha particles are measured with and without the jet and
the difference in energy gives the energy loss of the alpha particles in the
jet and hence the absolute density.

Figure 5: Technical drawing of the gas jet interferometry setup. the laser and
beam are showing in the red and blue colour respectively. The laser is passing
through the jet at 15∘ with respect to the beam axis inside a jet chamber that
is not shown here for the clarity.

###  3.2 Pressure profile measurement

Figure 6: Pressure profile measurements inside the chamber and beam line at
different needle valve opening connected to the nozzle.

For first jet tests, a needle valves is placed at the nozzle inlet to vary the
gas flow rate. To monitor the pressure at different regions, pressure gauges
are placed on all the pumping stages, the lower and upper cones of the
expansion chamber, the nozzle inlet, and at the exhaust of all pumps. The
pressure profile across the gas target setup at different needle valve opening
for a first jet test with a preliminary nozzle are shown in Fig. 5. The
pressure of the system without an inlet gas is in the order of $10^{-7}$ mbar
in the beam line and expansion chamber. The pressures at the second and third
pumping stages are not changing for different openings of needle valves.

At an opening of 30$\%$, the pressure in the jet chamber is higher than below
the catcher, indicating that there is more gas diffusing inside the chamber
than going directly in the catcher. This is caused by the too small opening of
the needle valve that is reducing the amount of gas to an inlet pressure of
$\sim$10 mbar. This low inlet pressure is not able to accelerate the gas to
supersonic velocity and hence not creating a jet. This behaviour changes at an
opening of 50$\%$ when the inlet pressure increases to $\sim$100 mbar and most
of the gas streams as a jet inside the gas catcher where about twice as much
gas flows compared to the gas chamber. At an opening of 100$\%$ (not yet
tested) the inlet pressure will reach 1 bar and create a supersonic jet into
the catcher, with less then 1$\%$ of the gas drifting into the jet chamber.

##  4 Outlook

The combined gas target has been built at the HZDR Rossendorf campus and will
be transferred to the Felsenkeller underground laboratory in 2023. First jet
tests have been performed to start a series of jet characterizations regarding
pressure, density, and thickness. These are continuously measured by an
interferometry setup that is in turn calibrated by an energy loss measurement
of alpha particles across the jet.

There is further development of enhanced nozzles. Computational Fluid Dynamic
(CFD) simulations in ANSYS Fluent are currently being carried out to optimize
the shape of those upcoming nozzles.

Finally, in-beam tests are foreseen, as soon as the target system is attached
to the beam line of the 5-MV Pelletron accelerator. This combination of a
wall-jet and a static gas target will be a unique in nuclear astrophysics and
might become a role model for future gas target developments.

##  5 Acknowledgement

Financial support by the European Union (ChETEC-INFRA, 101008324) is
gratefully acknowledged.

## References

  * (1) K. Schmidt, et al., Nucl. Instrum. Methods Phys. Res. A 911 (2018) 1-9.
  * (2) F. Ferraro, et al., Eur. Phys. J. A (2018) 54: 44\.
  * (3) J. P. Couperus, et al., Nucl. Instrum. Methods Phys. Res. A 830 (2016) 504-509.
  * (4) K. A. Chipps, et al., Nucl. Instrum. Methods Phys. Res. A 763 (2014) 533
  * (5) F. Favela, et al., Phys. Rev. ST Accel. Beams 18 (2015) 123502.
  * (6) G.P. de Laval, U.S. Patent 522066, 1894.
  * (7) A. Kontos, et al., Nucl. Instrum. Methods Phys. Res. A 664 (2012) 272.
  * (8) D. Rapagnani, et al., Nucl. Instrum. Methods Phys. Res. B 407 (2017) 217.
  * (9) Z. Meisel, et al., Nucl. Instrum. Methods Phys. Res. A 828 (2016) 8-14.


11institutetext: Research School of Computer Science,
Australian National University
11email<EMAIL_ADDRESS>
# Feature Selection on Thermal-stress Dataset

Xuyang Shen  Jo Plested  Tom Gedeon

###### Abstract

Physical symptoms caused by high stress commonly happen in our daily lives,
leading to the importance of stress recognition systems. This study aims to
improve stress classification by selecting appropriate features from Thermal-
stress data, ANUstressDB. We explored three different feature selection
techniques: correlation analysis, magnitude measure, and genetic algorithm.
Support Vector Machine (SVM) and Artificial Neural Network (ANN) models were
involved in measuring these three algorithms. Our result indicates that the
genetic algorithm combined with ANNs can improve the prediction accuracy by
19.1$\%$ compared to the baseline. Moreover, the magnitude measure performed
best among the three feature selection algorithms regarding the balance of
computation time and performance. These findings are likely to improve the
accuracy of current stress recognition systems.

###### Keywords:

Feature Selection Genetic Algorithms Artificial Neural Network

##  1 Introduction

Stress as an emotional language of humans represents the body’s reaction to
the environment. Light stress can be beneficial to our body [1], while high
stress may let us feel less energetic, give us headaches, and even cause major
illness. According to the latest report from the American Institute of Stress
[2], job stress affects 46$\%$ American adults and that it has escalated
gradually over the past decades. Correctly judging employees’ stress not only
helps leaders to adjust workload but also is favorable for the mental and
physical health of employees.

Traditional stress recognition and detection systems [3] require physiological
signals collected from special devices like blood pressure cuff, which is not
suitable to be utilized in normal circumstances. Therefore, Irani et al. [4]
proposed a real-time stress recognition system based on the physical
appearance of objects, which requires an RGB camera and a thermal camera. In
this paper, we focused on the same dataset as [4] and analyzed which feature
selection algorithms can sufficiently improve the stress classification for
Support Vector Machine(SVM) and Artificial Neural Network(ANN) respectively.

Reducing data sources from several physiological devices into two cameras
improves the feasibility of the real application’s recognition system but
requires stricter data pre-processing. Feature selection as one pre-processing
sub-task helps to filter out meaningless image information and hence improves
the model performance [5]. We first explore correlation analysis from
information theory that aims to measure the strength of the linear
relationship between features [6]. We also employ the magnitude measure that
is applied initially to prune neural networks [10]. It helps us to consider
the feature relationship from learned neurons through local search. The
genetic algorithm was introduced as the third method, which uses the
stochastic search to approach global optimization [7].

##  2 Method

###  2.1 Network Structure and Hyper-parameters

####  2.1.1 Support Vector Machine, SVM

SVM is a supervised machine learning model that performs both linear and non-
linear classification through the kernel trick [8, 9]. We explored it as a
baseline to compare with ANN as it is successfully applied in various
applications in computer vision [12]. The SVM utilized in this paper was
imported from Sklearn by using default hyper-parameters. To maintain the
fairness of each generation in genetic algorithms, we set a fixed value in the
random state.

Table 1: Hyper-Parameters of SVM Kernel | Degree | Gamma | Degree | Random State
---|---|---|---|---
rbf | 3 | auto | 3 | 22

####  2.1.2 Artificial Neural Network, ANN

We examined several architectures of fully connected neural networks (FCNN) to
recognize the stress. The first network topology is 10-10-6-2, being 10 input
neurons, 2 hidden layers, and 2 output neurons. However, it failed in
generalizing to data, as we shall show in Section 3. Since the inputs are the
features taken from raw pixels, a shallow neural network might struggle to
capture the complex relationships between the pixels. The second one is
10-18-16-8-2 (Fig. 1), which adds a few neurons in each hidden layer and one
extra hidden layer compared to the previous shallow FCNN.

One major issue of neural network design is how to optimize the network
architecture to achieve a balance between overfitting and generalization [13].
Considering that the shallow network cannot generalize well to thermal-stress
data, we decided to use the second network topology in later experiments (Fig.
1). The hyper-parameter of ANN was also adjusted to improve the efficiency of
training in genetic algorithms (Table 2). For instance, Table 3 indicates both
train-test separation and cross-validation can reflect similar test accuracy,
but the former highly reduces the computation time. Adam is selected as the
optimizer, which contributes to boosting the performance in the non-convex
problem in a shorter time [14].

Figure 1: Four-Layer neural network (ANN). Table 2: Hyper-Parameter of ANN Optimization | Learning Rate | Weight Decay | Epochs
---|---|---|---
Adam | 0.0008 | 0.0003 | 10000
Validation |  | Activation Function
---
of Hidden Layer
| Activation Function
---
of Output Layer
Random Seed
| Training: 0.7
---
Test: 0.3
ReLU | SoftMax | 7

###  2.2 Feature Selection Techniques

####  2.2.1 Correlation Analysis

Correlation indicates the dependence or relationship between two data
variables from a statistical perspective. The result of correlation is located
between -1 and 1, which represents the dependence between two variables from
negative correlation into positive correlation. We applied the correlation
analysis to perform feature selection by filtering highly correlated variables
from the statistical property of the dataset.

$corr(X,Y)=\frac{cov(X,Y)}{\sigma_{x}\
\sigma_{y}}=\frac{E\left[(X-\mu_{X})(Y-\mu_{Y})\right]}{\sigma_{x}\
\sigma_{y}}$ (1)

where $E$ means the expected values, and $cov$ refers to convariance.

####  2.2.2 Magnitude Measure

Gedeon [10] proposed the magnitude measure of input neuron to output neuron
based on the measure of input neuron to hidden layers [15]. It is a static
analysis applied for the input neuron in trained ANN. Compared with the
correlation analysis, the magnitude measure can analyze the weight between
layers, which indicates the impact of input neurons to others. As a result, we
can perform feature selection based on the ranking of input variables, giving
us higher quality results in the experiments.

$Q_{ik}=\sum_{r=1}^{nh}(P_{ir}\times P_{rk}),\ \text{where}\
P_{ir}=\frac{\left|W_{ij}\right|}{{\sum_{p=1}^{ni}}\left|W_{pk}\right|}\
P_{jk}=\frac{\left|W_{jk}\right|}{{\sum_{r=1}^{nh}}\left|W_{rk}\right|}$ (2)

where $P_{ij}$ refers the influence of input neuron $i$ to the hidden neuron
$j$, and $P_{jk}$ measures the effect of hidden neuron $j$ to the output
neuron $k$.

####  2.2.3 Genetic Algorithm

The magnitude measure has large limitations which make it not applicable to
SVM and highly non-deterministic in ANN as it performs the local search.
Therefore, we employed the genetic algorithm to analyze the feature selection
further.

Genetic algorithms, derived from natural selection [11], are one large class
of evolutionary algorithms, performing the stochastic search for an optimal
solution based on 5 components (described in Fig. 2). The fitness function is
the main module among these 5 components, responsible for evaluating each
chromosome (feature selection). Typically, hybrid values from train accuracy
and validation accuracy of models are selected as fitness scores. In contrast,
it is not suitable in our experiment since the model training on thermal-
stress dataset can easily overfit (Table 3). We only picked validation
accuracy; meanwhile, we also set a fixed seed for all random states in model
initialization and train-test dataset separation.

We also experimented with three different selection and crossover techniques,
as described below:

  * •

Use proportional selection to select one parent and randomly select the other
parent. The principle of this selection algorithm is that chromosomes with
large fitness values have a higher probability of passing their genes to
offspring.

$\varphi_{s}(x_{i}(t))=\frac{f_{r}(x_{i}(t))}{\sum_{l=1}^{ns}(f_{r}(x_{l}(t)))}$
(3)

where $n_{s}$ is the total number of individuals in the population;
$\varphi_{s}(x_{i})$ is the probability that $x_{i}$ will be selected;
$f_{r}(x_{i})$ is the scaled fitness value of $x_{i}$

  * •

Tournament selection for both parent: the best individual in the group of
$n_{ts}$ chromosomes will become the parents (set
$n_{ts}=0.6*\text{population}$). The principle of tournament selection is to
limit the chance of the best individual to dominate.

  * •

Use Hall of Fame selection to select one parent and use the proportional
selection to select the other parent; only the best individual of each
generation is chosen to be inserted into the hall of fame, which becomes a
parent pool for crossover operator.

Figure 2: Combining SVM and ANN with Genetic Algorithm to explore feature
selection

##  3 Dataset: Thermal-stress Dataset

The ANUStressDB [4] dataset was generated from an HCI experiment about stress
simulator, which involved 31 subjects. Their facial information was recorded
by a Microsoft webcam and FLIR camera at 30 frames per second at 640x480
pixels [4]. The raw data is then PCA processed to protect privacy.

The thermal-stress dataset contains 620 records with five features extracted
from RGB facial images and five features extracted from the thermal camera as
described above. Specifically, these ten features vectors are derived from the
result of principal component analysis (PCA) on the original raw data. It also
gives us a classification label of each record that is either stressful or
calm. The labels were manually constructed in the experiment environment
setting and validated by the questionnaire survey.

Although the dataset is well-balanced between two classes, it is still hard
for the model to make a correct prediction. From two visualizations (Fig. 3),
it indicates a high-class similarity between stressful and calm data. Most of
the data points from two different classes are overlapped. Most spatial
information is lost after the main feature extraction, which makes the
prediction task more difficult.

Figure 3: PCA and t-SNE dimensional reduction techniques are applied to
visualize the original ten-dimension data

The results of experiments also prove that the trained model fails in
prediction (Table 3). In the 10-18-16-8-2 model, high training accuracy
indicates the networks learn several features from training data but still
fail in the prediction.

Table 3: Training and Testing Accuracy of two models and validation methods (report the average result among 20 times running). 10-18-16-8-2: 10-input neurons; 18-hidden1 neurons; 16-hidden2 neurons; 8 hidden3 neurons, 2 output neurons. 10-10-6-2: 10-input neurons; 10-hidden1 neurons; 6-hidden2 neurons; 2 output neurons Model | Validation Method | Training Accuracy | Test Accuracy
---|---|---|---
10-18-16-8-2 | Training: 0.7 Test: 0.3 | 92.84% | 53.50%
10-18-16-8-2 | 5-Fold Cross-validation | 91.71% | 52.10%
10-10-6-2 | Training: 0.7 Test: 0.3 | 67.3% | 51.2%
10-10-6-2 | 5-Fold Cross-validation | 63.2% | 49.7%

##  4 Results and Discussion

###  4.1 Baseline of Feature Selection Experiment

To reflect the influence of different feature selection algorithms, we
computed the baseline of SVM and ANN model, which took all of ten features as
inputs (Table 4). The table’s data was the result of one run since all random
states were set in a fixed value.

Table 4: Baseline of SVM and ANN Model | Final Train Accuracy | Test Accuracy | Platform
---|---|---|---
SVM | 100% | 49% | MacOS
ANN | 95.33% | 46.47% | Linux

SVM with ten input features can reach 100$\%$ final training accuracy and
49$\%$ test accuracy. The evaluation accuracy of the ANN is 47$\%$ on Linux
platforms.

###  4.2 Correlation Analysis on Feature Selection

The absolute values of overall correlation among these ten features are all
lower than 0.1, and there are four attributes even lower than 0.005. To
quantify the performance of correlation analysis, we further explored the
influence of removing low correlated features to prediction accuracy (Fig. 4).
The result indicated that correlation analysis could not help gain better SVM
performance, and it contributes little to the ANN as well. Therefore, we
conclude that the correlation analysis cannot guide the feature selection in
positive ways.

Table 5: Correlation between different feature and labels, ranking from low correlation to high correlation Thermal_4 | Thermal_5 | RGB_4 | Thermal_1 | RGB_1 | RGB_2 | RGB_3 | Thermal_3 | Thermal_2 | RGB_5
---|---|---|---|---|---|---|---|---|---
-0.0015 | 0.0025 | 0.0039 | -0.0040 | 0.0054 | 0.0055 | -0.0093 | 0.010 | 0.015 | 0.023
Figure 4: Test accuracy of removing the lowest correlated feature to top 4
lowest correlated features T4: remove Thermal_4 feature; T4,T5: remove
Thermal_4 and Thermal_5 features, and etc. Besides, the training platform of
ANN is Linux.

###  4.3 Magnitude Measure on Feature Selection

Compared to the correlation analysis, the magnitude measure focuses on the
impact of dataset property on neural networks’ behavior. We calculated the
average magnitude value of input neurons behavior to the hidden neurons in 20
runs (Table 6) and applied the results to feature selection experiments (Fig.
5).

The result indicated that ”RGB_3”, ”Thermal_1”, ”RGB_1” negatively contribute
to the ANN’s training and prediction, which were shown to be less important
attributes in the magnitude measure. On the contrary, ”RGB_4” was incorrectly
marked as less important data to hidden neurons by the magnitude measure. It
was caused by the low stability of the magnitude measure. This method targets
the weight’s calculation of trained networks, which can be easily affected by
the initial weights assigned to each hidden neurons and final termination
judgment. Overall, the magnitude measure can provide valuable guidance to
feature selection for ANN.

Table 6: Magnitude of each attribute, ranking from low to high. To obtain a reasonable result, we set all random states in random values RGB_3 | Thermal_3 | RGB_4 | Thermal_1 | RGB_1 | Thermal_5 | RGB_2 | Thermal_4 | Thermal_2 | RGB_5
---|---|---|---|---|---|---|---|---|---
1.693 | 1.745 | 1.761 | 1.773 | 1.793 | 1.811 | 1.838 | 1.843 | 1.850 | 1.887
Figure 5: Test accuracy of removing least top1 – top 5 important variables.
The training platform of ANN is Linux.

###  4.4 Genetic Algorithm (GA) on Feature Selection

We found the magnitude measure to be sensitive to the neuron weights through
the previous experiments, which were updated through gradient descent in ANN’s
training. Backpropagation with gradient descent is a local search and is known
to have local-minima issue. Besides, the magnitude measure is also limited to
the learning model, which cannot be applied to SVM. Therefore, we also
explored a global search to feature selection, which is genetic algorithms
from evolutionary algorithms. Following the procedures introduced in section
2.2, we applied genetic algorithms to SVM and ANN, respectively, to further
analyze the feature selection. Evaluating each chromosome with the validation
accuracy of ANN requires 10 seconds, which means 60 population in one
generation needs 10 minutes. Hence, we employed two computer resources, which
causes a different baseline in Table 4.

The result in Table 7 indicated that GA applied with the ANN has a large
improvement to the evaluation accuracy, which achieved 19.1$\%$ at maximum. It
proved that stochastic global search with a high initial diversity has a
higher probability of finding the best optimization. Additionally, the
correlations between the attributes and the ground truths are low in the
thermal-stress dataset, exaggerating the difference between local search and
global search. The results of GA also showed that ”RGB_2”, ”RGB_4”,
”Thermal_2”, and ”Thermal_3” were four important features.

We also found that the population’s initial diversity has different influences
regarding different selection strategies of genetic algorithms. Specifically,
the proportional selection was more sensitive to the initial diversity than
then tournament selection. Hence, properly dominating superior genes is
beneficial to the stability of genetic algorithms.

Apart from applying genetic algorithms with ANN, we also explored the SVMs
(Table 8). Performing feature selection with genetic algorithms can improve
8$\%$ test accuracy of SVM at maximum. In conclusion, the genetic algorithm is
the best option for either ANN models or SVM models. Not only it has broad
applicability, but it also improves the model performance.

Table 7: GA algorithm applied with ANN. DNA order: [RGB_1, RGB_2, RGB_3, RGB_4, RGB_5, Thermal_1, Thermal_2, Thermal_3, Thermal_4, Thermal_5] 1 means this attribute is selected and vice versa GA | Proportional selection | Hall of Fame selection | Tournament selection
---|---|---|---
| Population
---
Generation
| 80
---
78
| 60
---
100
| 60
---
100
| 60
---
100
| 80
---
78
DNA | 0101001101 | 0110111011 | 0001111110 | 1101011110 | 1111110011
Test Accuracy | 0.56 | 0.51 | 0.55 | 0.54 | 0.53
Baseline | 0.47 | 0.47 | 0.47 | 0.47 | 0.47
Improvement | +0.09 | \+ 0.04 | +0.08 | \+ 0.07 | \+ 0.06
Table 8: GA algorithm applied with SVM. DNA order: [RGB_1, RGB_2, RGB_3, RGB_4, RGB_5, Thermal_1, Thermal_2, Thermal_3, Thermal_4, Thermal_5] 1 means this attribute is selected and vice versa GA | Proportional selection | Hall of Fame selection | Tournament selection
---|---|---|---
| Population
---
Generation
| 60
---
100
| 60
---
100
| 60
---
100
DNA | 0111010010 | 0111111011 | 0101100011
Test Accuracy | 0.52 | 0.53 | 0.50
Baseline | 0.49 | 0.49 | 0.49
Improvement | +0.03 | +0.04 | \+ 0.01

##  5 Conclusion and Future work

Feature selection as one of the essential parts of data pre-processing can
improve the prediction accuracy for both SVN and ANN. Different feature
selection algorithms may contribute differently to the improvement of model
performance. As we explored in the experiments, correlation analysis focuses
on the dataset’s statistical property to provide selection guidance, which
takes the least computation power but is sensitive to the dataset. Since the
correlation between attributes and ground truths is low in the thermal-stress
data, this algorithm cannot guide sufficient information for us to choose
features. The second algorithm we explored is the magnitude measure, which is
moderate among the three algorithms we discussed in section 3, with a balanced
ratio of performance and efficiency. However, the same as correlation
analysis, magnitude measure only provides feature selection guidelines to us,
and we have to decide which feature to keep or remove manually. Eventually,
genetic algorithms can select the most appropriate features for both SVM and
ANN but require huge computation power. For instance, 1,000 generations with
100 initial population of genetic algorithm with ANN need ten days to simulate
the evolution. We also discovered that the genetic algorithm with tournament
selection is less sensitive to population and generation, helping us to gain
an ideal performance in a limited time. We leave the exploration to find the
right balance between time and performance in feature selection by the genetic
algorithm as future work. Apart from different selection or crossover
configuration of genetic algorithm, we can also consider this from the
perspective of fitness functions.

### Acknowledgements

We thank Alasdair Tran for his invaluable feedback on the draft of this paper.

## References

  * [1] Stress Symptoms: Effects of Stress on the Body, https://www.webmd.com/balance/stress-management/stress-symptoms-effects_of-stress-on-the-body. Last accessed 21 May 2020
  * [2] Stress Effects - workplace stress, https://www.stress.org/workplace-stress. Last accessed 21 May 2020
  * [3] Chen, T., Yuen, P., Richardson, M., Liu, G., She, Z.: Detection of psychological stress using a hyperspectral imaging technique. IEEE Transactions on Affective Computing, 5(4), pp.391-405 (2014)
  * [4] Irani, R., Nasrollahi, K., Dhall, A., Moeslund, T.B., Gedeon, T.: Thermal super-pixels for bimodal stress recognition. In: 2016 Sixth International Conference on Image Processing Theory, Tools and Applications (IPTA), pp. 1-6 (2016)
  * [5] Peng, H., Long, F., Ding, C.: Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy. IEEE Transactions on pattern analysis and machine intelligence, 27(8), pp.1226-1238 (2005)
  * [6] James, G., Witten, D., Hastie, T., Tibshirani, R.: An introduction to statistical learning, Vol. 112, p. 18. New York: springer (2013) 10.1007/978-1-4614-7138-7
  * [7] Goldenberg, D.E.: Genetic algorithms in search, optimization and machine learning (1989)
  * [8] Cortes, C., Vapnik, V.: Support-vector networks. Machine learning,20(3), pp.273-297 (1995)
  * [9] Boser, B.E., Guyon, I.M., Vapnik, V.N.: July. A training algorithm for optimal margin classifiers. In: Proceedings of the fifth annual workshop on Computational learning theory, pp. 144-152 (1992)
  * [10] Gedeon, T.D.: Data mining of inputs: analysing magnitude and functional measures. International Journal of Neural Systems,8(02), pp.209-218 (1997)
  * [11] Mitchell, M.: An introduction to genetic algorithms. MIT press (1998)
  * [12] Taini, M., Zhao, G., Li, S.Z., Pietikainen, M.: Facial expression recognition from near-infrared video sequences. In: 19th International Conference on Pattern Recognition, pp. 1-4 (2008)
  * [13] Zhang, C., Bengio, S., Hardt, M., Recht, B., Vinyals, O.: Understanding deep learning requires rethinking generalization. arXiv preprint arXiv:1611.03530 (2016)
  * [14] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)
  * [15] Wong, P.M., Gedeon, T.D., Taggart, I.J.: An improved technique in porosity prediction: a neural network approach. IEEE Transactions on Geoscience and Remote Sensing, 33(4), pp.971-980 (1995)