# Stress Monitoring Using Low-Cost Electroencephalogram Devices: A Systematic
Literature Review

Gideon Vos  Maryam Ebrahimpour  Liza van Eijk  Zoltan Sarnyai  Mostafa Rahimi
Azghadi

###### Abstract

##### Introduction

Low-cost, consumer grade wearable health monitoring devices are increasingly
being used for mental health related studies including stress. While cortisol
response magnitude remains the gold standard indicator for stress assessment,
a growing number of studies have started to use low-cost EEG devices and
wrist-based wearable device monitors as primary recorders of biomarker data.
Low-cost EEG devices differ in technical sophistication, and more importantly
number of sensors available, and the location of these sensors according to
the 10-20 System of Electrode Placement, complicating the reproducibility of
reported study results.

##### Objective

The aim of this review is to provide an overview of the growing use of low-
cost EEG devices and machine learning methods for brain function assessment,
specifically stress. We also shed light on the machine learning techniques
commonly employed for stress research when using EEG brain function data, with
an emphasis on the reproducibility of the results reported.

##### Methods

This study reviewed published works contributing and/or using EEG devices for
detecting stress and their associated machine learning methods. The electronic
databases of Google Scholar, ScienceDirect, Nature and PubMed were searched
for relevant articles and a total of 60 articles were identified and included
in the final analysis. The reviewed works were synthesized into three
categories of low-cost EEG device usage for stress research, machine learning
techniques applied to EEG data for measuring stress, and future research
directions. For the machine learning studies reviewed, we provide an analysis
of their approach to results validation and reproducibility. The quality
assessment of the included studies was conducted in accordance with the IJMEDI
checklist.

##### Results

A number of studies were identified where low-cost EEG devices were utilized
to record brain function during phases of stress and relaxation. These studies
generally reported a high predictive accuracy rate, verified using a number of
different machine learning validation methods and statistical approaches. Of
these studies, only 50% reported health screening prior to experimentation and
a further 60% can be considered low-powered studies based on the small number
of test subjects used during experimentation. In addition, we discuss that
previous works focused on a single health condition (stress) with little
consideration for other potential underlying mental health conditions and how
it may affect the study results reported.

##### Conclusion

Low-cost consumer grade wearable devices including EEG and wrist-based
monitors are increasingly being used in stress-related studies.
Standardization of EEG signal processing and importance of sensor location
still requires further study, and research in this area will continue to
provide improvements as more studies become available.

###### keywords:

Stress , EEG , Machine Learning

###### PACS:

07.05.Mh , 87.85.fk

###### MSC:

68T01 , 92C99

††journal: International Journal of Medical Informatics

[inst1]organization=College of Science and Engineering, James Cook
University,addressline=James Cook Dr, city=Townsville, postcode=4811,
state=QLD, country=Australia

[inst2]organization=College of Public Health, Medical, and Vet Sciences, James
Cook University,addressline=James Cook Dr, city=Townsville, postcode=4811,
state=QLD, country=Australia

[inst3]organization=College of Health Care Sciences, James Cook
University,addressline=James Cook Dr, city=Townsville, postcode=4811,
state=QLD, country=Australia

This paper provides a review of the current state of utilizing ready to use,
wearable EEG devices and machine learning methods for stress measurement,
performed in accordance with the IJMEDI checklist.

Our review of EEG devices is provided by analyzing and synthesizing the
literature based on machine learning methods used, study quality and
reproducibility of results, with a further emphasis on the growing use of low-
cost EEG devices.

We show that the combination of low-cost EEG devices with machine learning
methods provide a viable alternative to the use of more intrusive medical
grade devices, with a growing number of studies incorporating the use of
wrist-based wearable health monitors for recording additional biomarkers.

##  1 Introduction

Stress can be defined as any type of change that causes physical, emotional,
or psychological strain. These changes in the environment can trigger a
cascade of biological responses in the brain and body [1], known as the stress
response, which helps the organism adapt to the dynamically changing external
and internal environment. This adaptation is achieved through the mobilization
of energy and its appropriate redistribution to organs that serve the
adaptational response. While the stress response is adaptive and beneficial in
the short term, long-term exposure to stress can have detrimental effects on
the body, including an increased risk of developing metabolic, cardiovascular,
and mental disorders, which can significantly compromise quality of life and
shorten life expectancy [2].

The rise in mental health issues [3] and the exacerbating effects of the
COVID-19 pandemic further highlights the importance of addressing stress and
its impact on mental health. Understanding the physiological and psychological
changes that occur in the body during stress response can inform the
development of effective interventions to mitigate the negative effects of
stress on mental health.

The stress response is initiated by the brain as the different sensory
cortical areas receive and process information from the changing environment
and distribute them to a variety of cortical areas to initiate the appropriate
response. Therefore, stress changes neuronal activity in the whole brain,
which can be detected by measuring the electrical activation of the cerebral
cortex by EEG [4].

EEG is a widely used technique to estimate changes in neurophysiological
activity associated with external stimuli and/or with the performance of
specific tasks [5]. This activity is measured via the electrical potential
generated by the asynchronous ﬁring of neurons [6] in the nervous system [7]
using electrodes placed on the scalp according to the 10-20 international
standard [8] (Figure 1). The 10-20 system was first presented at the 1957
Brussels IV International EEG Congress by Herbert Jasper, to standardize the
method of EEG placement. The numbers ‘10’ and ‘20’ refer to the distances
between adjacent electrodes, which are either 10% or 20% of the total distance
(front-back or right-left) of the skull.

Figure 1: The 10-20 international standard of EEG sensor placement, with
sensor placements supported by devices included in this review indicated in
blue..

The 10-20 system is based on the relationship between the location of an
electrode and the underlying area of the cerebral cortex (Figure 2). Each site
has a letter (to identify the lobe) and a number or another letter to identify
the hemisphere location. In 1985, an extension to the original 10-20 system
was proposed involving an increase in the number of electrodes from 21 to 74
[9].

EEG waveforms may be characterized based on their location, amplitude,
frequency, morphology, continuity (rhythmic, intermittent or continuous),
synchrony, symmetry, and reactivity. However, the most frequently used method
to classify EEG waveforms is by the frequency band, with the most commonly
studied waveforms being delta (0.5 to 4Hz), theta (4 to 7Hz), alpha (8 to
12Hz), sigma (12 to 16Hz), beta (13 to 30Hz) and gamma ($>$ 30Hz) [10]. Common
brain states typically associated with different EEG waveform frequencies
include concentration (gamma), anxiety dominant, active, external attention,
relaxed (beta), very relaxed, passive attention (alpha), deeply relaxed,
inward focused (theta) and sleep (delta) [11].

Figure 2: Anatomy of the human brain [12].

Prior studies [13, 14, 15] have utilized physiological measures including
heart rate (HR), electrodermal activity (EDA) and salivary cortisol to assess
the sympathetic nervous system and Hypothalamic-Pituitary-Adrenal (HPA) axis
response to stress. These measures can be influenced by multiple factors
including mental stress, and can further be affected by circadian rhythm and
physical activity. Functional neuroimaging techniques including EEG have also
previously been used to assess the brain’s response to stress [5, 16, 17] by
directly or indirectly measuring brain activity. EEG offers high temporal
resolution, at the cost of requiring trained neurophysiologists to aid in the
interpretation of results.

Although the assessment of cortisol response continues to be the primary
indicator for evaluating stress [18], an increasing number of studies are now
employing affordable, non-intrusive wearable health monitors and wireless EEG
devices as the main tools for recording biomarker data [19, 20, 21, 22] that
may correlate with stress. This is likely driven by the increasing
sophistication and miniaturization of device components at an ever reducing
cost. Figure 3 provides a visual timeline of the evolution of electronic
healthcare monitoring, illustrating the merging of traditional ECG with EEG,
resulting in more modular and mobile devices with a wide range of sensors,
albeit with fewer EEG channels.

Figure 3: Technological timeline of electronic healthcare monitoring.

Most wrist-based wearable health monitors previously utilized in stress
research record at least heart rate and electrodermal activity [23]. In
contrast, low-cost EEG devices differ in the number of available sensors and
sensor locations according to the 10-20 System of Electrode Placement [8],
complicating the reproducibility of reported study results.

A number of previous survey articles have studied the topics of stress
detection using EEG [24, 16, 17, 5]. Katmah _et al._ [16] reviewed existing
EEG signal analysis methods for assessing mental stress, and concluded that
variations in data analysis methods resulted in several contradictory results
with regards to frequency band importance. These variations could be due to
the study differences in experimental protocols, stressors employed,
preprocessing of signals and choice of machine learning algorithms utilized.
Newson _et al._ [24] reviewed 184 studies where EEG was used to study a
varying number of mental disorders, and found differences in frequency bands
in the resting state condition (eyes open and closed) across a spectrum of
psychiatric disorders. They further cautioned against the interpretation of
results that considered a single disorder in isolation when analyzing results.
While their study focussed on mental disorders, the question arises whether
similar differences may be relevant to stress-related EEG studies.

None of the prior reviews specifically addressed EEG hardware and more
specifically, the use of low-cost EEG devices for stress research. While
Giannakakis _et al._ [5] reviewed sensor placement and relative importance,
and Katmah _et al._ [16] addressed the use of machine learning algorithms and
signal processing techniques, neither highlighted sensor placement or sensor
limitations of low-cost EEG devices when used for stress research, and how
this could potentially affect machine learning model performance and the
reproducibility of the results reported in stress-related studies.

Towards addressing these questions, we first explore the current state of
stress detection and measurement using EEG, with a focus on low-cost EEG
devices. We further explore the available public datasets built using sensor
data recorded from low-cost EEG devices that could enable future stress
research. We review the machine learning approaches employed, and detection
accuracy scores attained using low-cost EEG devices. Finally, we discuss the
limitations of using such devices combined with machine learning techniques
for stress-related studies and suggest several future research directions.

##  2 Methods

###  2.1 Research questions

The main aim of this work is to provide an overview of the use of EEG devices
for stress detection and measurement when combined with machine learning
techniques, with an emphasis on low-cost wearable EEG devices retailing for
below USD1000.00. To assist in the assessment of the quality of the included
literature, the IJMEDI checklist was utilized. Thus, our research questions
can be formulated as follows:

  * 1.

RQ1: Which low-cost EEG devices are predominantly being used for stress
research, and what results are being reported?

  * 2.

RQ2: Which scalp sensor locations and frequency bands are predominantly used
for stress research, and are these physically supported by current low-cost
EEG devices?

  * 3.

RQ3: Which machine learning methods are being utilized in low-cost EEG studies
for stress research, and how are the results from these studies validated?

Answering these research questions will aid in gaining a better understanding
of the feasibility and potential of using lower-cost EEG devices for stress
monitoring, while providing guidance for the most appropriate machine learning
algorithms to employ for stress prediction.

###  2.2 Search strategy

We reviewed key published works (Figure 4) between 2013 and 2023 on acute
stress assessment using EEG signals, and additionally, recorded using low-cost
EEG devices. The electronic databases of Google Scholar, ScienceDirect, Nature
and PubMed were searched for relevant articles using the keywords _EEG AND
stress AND (wearable OR machine learning)_ , in title or abstract, and a total
of 700 papers were found. Duplicates were identified, and 8 were found and
removed, leaving the number of considered papers for the subsequent phases at
692. Titles and abstracts were scanned and irrelevant papers were excluded.

A small number of papers, in which the focus was stress in animals were
excluded. Studies where the main focus was mental health conditions other than
stress, including schizophrenia, bipolar disorder and major depression
disorder were also excluded. Studies utilizing medical-grade EEG devices were
not excluded at this stage. As a result, a total of 60 papers were chosen for
the systematic review process, to assist in answering the aforementioned
research questions. The chosen papers were then grouped by the high-level
topics of: _RQ1: Stress Assessment Using EEG_ , _RQ2: Low-Cost EEG Devices_ ,
_RQ3: Available Datasets for EEG-based Stress Measurement_ and _RQ3: Machine
Learning Techniques for EEG-based Stress Measurement_. Table
LABEL:tab:papersreviewed lists, in chronological order, the papers included in
this review. We further categorize these papers by study type, i.e. Original
Research or Review, and domain, i.e. stress or other mental states. A single
study [25] that pre-dates the search criteria (2012) was included in this
review due to its popular use as an open EEG dataset and high citation count.
To answer the research questions systematically, we have structured our paper
as shown in Fig. 5 to cover all the essential topics around utilizing low-cost
EEG devices for stress detection using machine learning.

Figure 4: Article screening process and the intermediate counts. Figure 5: Survey structure. Table 1: Studies included in this review. Paper |  Year | Title |  Study Type |  Domain
---|---|---|---|---
[25] |  2012 |  DEAP: A database for emotion analysis using physiological signals |  Research |  Emotion
[26] |  2014 |  Measuring neurophysiological signals in aircraft pilots and car drivers for the assessment of mental workload, fatigue and drowsiness |  Research |  Stress
[27] |  2015 |  EEG based stress monitoring |  Research |  Stress
[28] |  2015 |  Automated identification of abnormal adult EEGs |  Research |  Normal/Abnormal
[29] |  2015 |  Investigating critical frequency bands and channels for EEG-based emotion recognition with deep neural networks |  Research |  Emotion
[30] |  2015 |  Detection of stress/anxiety state from EEG features during video watching |  Research |  Stress
[31] |  2016 |  Stress assessment by prefrontal relative gamma |  Research |  Stress
[32] |  2018 |  Selection of neural oscillatory features for human stress classification with single channel EEG headset |  Research |  Stress
[33] |  2019 |  A mind-brain-body dataset of MRI, EEG, cognition, emotion, and peripheral physiology in young and old adults |  Research |  Emotion
[5] |  2019 |  Review on psychological stress detection using biosignals |  Review |  Stress
[34] |  2019 |  DASPS: A database for anxious states based on a psychological stimulation |  Research |  Anxiety
[24] |  2019 |  EEG frequency bands in psychiatric disorders: A review of resting state studies |  Review |  Various
[35] |  2019 |  A Novel wearable EEG and ECG recording system for stress assessment |  Research |  Stress
[36] |  2019 |  Classification of perceived human stress using physiological signals |  Research |  Stress
[19] |  2020 |  Stress monitoring using multimodal bio-sensing headset |  Research |  Stress
[20] |  2020 |  K-EmoCon, a multimodal sensor dataset for continuous emotion recognition in naturalistic conversations |  Research |  Emotion
[37] |  2020 |  EEG based classification of long-term stress using psychological labeling |  Research |  Stress
[38] |  2020 |  Stress recognition using Electroencephalogram (EEG) signal |  Research |  Stress
[39] |  2020 |  PASS: A multimodal database of physical activity and stress for mobile passive body/brain-computer interface research |  Research |  Stress
[40] |  2020 |  Multivariate correlation measures reveal structure and strength of brain-body physiological networks at rest and during mental stress |  Research |  Stress
[41] |  2020 |  Database for an emotion recognition system based on EEG signals and various computer games – GAMEEMO |  Research |  Emotion
[7] |  2020 |  On identification of driving-induced stress using electroencephalogram signals: A framework based on wearable safety-critical scheme and machine learning |  Research |  Stress
[42] |  2020 |  Detecting fatigue in car drivers and aircraft pilots by using non-invasive measures: The value of differentiation of sleepiness and mental fatigue |  Review |  Stress
[43] |  2020 |  Dynamic cognitive workload assessment for fighter pilots in simulated fighter aircraft environment using EEG |  Research |  Stress
[44] |  2020 |  A feasibility study of a complete low-cost consumer-grade brain-computer interface system |  Review |  Various
[45] |  2020 |  Emotion recognition using multi-modal data and machine learning techniques: A tutorial and review |  Review |  Emotion
[37] |  2020 |  EEG based classification of long-term stress using psychological labeling |  Research |  Stress
[38] |  2020 |  Stress recognition using electroencephalogram (EEG) signal |  Research |  Stress
[39] |  2020 |  PASS: A multimodal database of physical activity and stress for mobile passive body/ brain-computer interface research |  Research |  Stress
[40] |  2020 |  Multivariate correlation measures reveal structure and strength of brain-body physiological networks at rest and during mental stress |  Research |  Stress
[41] |  2020 |  Database for an emotion recognition system based on EEG signals and various computer games – GAMEEMO |  Research |  Emotion
[7] |  2020 |  On identification of driving-induced stress using electroencephalogram signals: A framework based on wearable safety-critical scheme and machine learning |  Research |  Stress
[42] |  2020 |  Detecting fatigue in car drivers and aircraft pilots by using non-invasive measures: The value of differentiation of sleepiness and mental fatigue |  Review |  Stress
[43] |  2020 |  Dynamic cognitive workload assessment for fighter pilots in simulated fighter aircraft environment using EEG |  Research |  Stress
[44] |  2020 |  A feasibility study of a complete low-cost consumer-grade brain-computer interface system |  Review |  Various
[45] |  2020 |  Emotion recognition using multi-modal data and machine learning techniques: A tutorial and review |  Review |  Emotion
[16] |  2021 |  A review on mental stress assessment methods using EEG signals |  Review |  Stress
[46] |  2021 |  Evaluating deep learning EEG-based mental stress classification in adolescents with autism for breathing entrainment BCI |  Research |  Stress
[47] |  2021 |  Performance evaluation of EEG based mental stress assessment approaches for wearable devices |  Research |  Stress
[48] |  2021 |  Enhancing EEG-based mental stress state recognition using an improved hybrid feature selection algorithm |  Research |  Stress
[49] |  2021 |  Human stress classification during public speaking using physiological signals |  Research |  Stress
[50] |  2021 |  Relationship between rework of engineering drawing tasks and stress level measured from physiological signals |  Research |  Stress
[51] |  2021 |  Monitoring army drivers’ workload during off-road missions: An experimental controlled field study |  Research |  Stress
[52] |  2021 |  A LSTM based deep learning network for recognizing emotions using wireless brainwave driven system |  Research |  Emotion
[53] |  2021 |  NeuroSense: Short-term emotion recognition and understanding based on spiking neural network modelling of spatio-temporal EEG patterns |  Research |  Emotion
[54] |  2021 |  A review on transfer learning in EEG signal analysis |  Review |  Various
[55] |  2021 |  Psychological and physiological effects of a green wall on occupants: A cross-over study in virtual reality |  Research |  Stress
[56] |  2021 |  Exploring the effects of EEG signals on collision cases happening in the process of young drivers’ braking |  Research |  Stress
[21] |  2022 |  Assessing electroencephalography as a stress indicator: A VR high-altitude scenario monitored through EEG and ECG |  Research |  Stress
[57] |  2022 |  A multi-modal open dataset for mental-disorder analysis |  Research |  Various
[58] |  2022 |  A multimodal perceived stress classification framework using wearable physiological sensors |  Research |  Stress
[59] |  2022 |  Evaluating the stressful commutes using physiological signals and machine learning |  Research |  Stress
[60] |  2022 |  Frontal lobe real-time EEG analysis using machine learning techniques for mental stress detection |  Research |  Stress
[61] |  2022 |  Stress classification using brain signals based on LSTM network |  Research |  Stress
[22] |  2022 |  Development of an EEG headband for stress measurement on driving simulators |  Research |  Stress
[62] |  2022 |  Genetic algorithm-based human mental stress detection and alerting in internet of things |  Research |  Stress
[63] |  2022 |  Real-time stress analysis affecting nurse during elective spinal surgery using a wearable device |  Research |  Stress
[64] |  2022 |  Emerging wearable biosensor technologies for stress monitoring and their real-world applications |  Review |  Stress
[65] |  2022 |  Wavelet based machine learning models for classification of human emotions using EEG signal |  Research |  Emotion
[66] |  2022 |  Fusion-based learning for stress recognition in smart home: An IoMT framework |  Research |  Stress
[17] |  2022 |  The neural correlates of psychosocial stress: A systematic review and meta-analysis of spectral analysis EEG studies |  Review |  Stress
[67] |  2023 |  Cheating detection in E-exams system using EEG signals |  Research |  Stress
[68] |  2023 |  Automated detection of Alzheimer’s disease and other neurophysiological applications based on EEG |  Research |  Various
[69] |  2023 |  Deep learning framework for classification of mental stress from multimodal datasets |  research |  stress
[70] |  2023 |  A deep learning approach for assessing stress levels in patients using electroencephalogram signals |  Research |  Stress
[71] |  2023 |  Brain activity monitoring for stress analysis through EEG dataset using machine learning |  Research |  Stress
[72] |  2023 |  Comparing online cognitive load on mobile versus PC-based devices |  Research |  Stress
[73] |  2023 |  An innovative random-forest-based model to assess the health impacts of regular commuting using non-invasive wearable sensors |  Research |  Stress
[74] |  2023 |  EEG emotion recognition using improved graph neural network with channel selection |  Research |  Emotion
[75] |  2023 |  Use of portable devices to measure brain and heart activity during relaxation and comparative conditions: Electroencephalogram, heart rate variability, and correlations with self-report psychological measures |  Research |  Stress

###  2.3 Assessment of the quality of the studies

Two reviewers (Vos and Rahimi Azghadi) used the IJMEDI checklist [76] to
independently evaluate the included studies’ quality. The IJMEDI checklist is
a quality assessment tool for medical artificial intelligence studies proposed
by the IJMEDI, which aims to distinguish high-quality machine learning studies
from simple medical data-mining studies. Six dimensions are included as 30
questions in the checklist: problem and data understanding, data preparation,
modeling, validation, and deployment. Each question can be answered as OK
(adequately addressed), mR (sufficient but improvable), and MR (inadequately
addressed). In high-priority items, OK, mR and MR were assigned scores of 2,
1, and 0, respectively, whereas, in low-priority items, the scores were
halved. The maximum possible score was 50 points, with study quality divided
into low (0–19.5), medium (20–34.5), and high (35–50).

##  3 Results

###  3.1 Stress Assessment Using EEG

The use of EEG in stress research relates to its purported sensitivity to
localized brain activity in regions involved in the generation of the stress
response, or activity associated with increased arousal or specific psycho-
emotional states, with the majority of studies supporting the view that in a
stressed state there is generally greater frontal right alpha activity in
relation to the left alpha activity [5]. Cortisol response magnitude remains
the gold standard indicator for stress assessment [18], as cortisol levels
quantify the endocrine response to stress [77]. Seo _et al._ [77] reported a
significant positive correlation between the cortisol level and relative high
beta power at both the anterior temporal lobes (Figure 2), and a tendency
toward a similar relationship at one of the occipital sites. Barzegar _et al._
[78] similarly found salivary cortisol levels increased significantly along
with the relative delta band frequency, while the beta bands and, in less
amount, the theta and gamma decreased, especially in the frontal region. These
effects on the cortisol stress hormone generally peak after about 20–30
min[79].

####  3.1.1 Sensor positions and recorded frequency bands

Table LABEL:tab:wavesreviewed provides an overview of the studies incorporated
in this review, explicitly detailing sensor positions and recorded frequency
bands. Among these, five [40, 7, 49, 67, 31] focused on alpha waves, ten [32,
37, 49, 50, 21, 58, 67, 73, 71, 60] investigated beta waves, two [39, 70]
explored delta waves, eight [32, 37, 39, 7, 21, 31, 71, 60] examined gamma
waves, and four [40, 7, 58, 60] delved into theta frequency. Importantly,
Newson _et al._ [24] in their review of EEG frequency bands in psychiatric
disorders noted that it is not always obvious whether changes in particular
frequency bands are specific to individual disorders, or whether overlap may
exist, limiting clinical diagnosis potential. This finding is of further
importance when machine learning classification models are utilized for
diagnosis or prediction when such models are trained on datasets containing
binary classes, for example, stressed/not-stressed.

####  3.1.2 Data recording considerations

Fifty of the 60 studies included in this review provided in-depth descriptive
detail of the data that were used during experimentation. Of these, 35 (70%)
did not provide access to their study data, complicating the ability to easily
reproduce study results. The remaining fifteen studies either published their
datasets or used datasets previously made public. Additionally, 44% of the
total 60 studies reviewed did not explicitly note health screening before
experimentation and data recording. A detailed health screening is of critical
importance in order to build an accurate baseline dataset for training machine
learning models, and as noted by Newson _et al._ [24], underlying or
undiagnosed mental health conditions can compromise model training if the data
from such subjects are erroneously introduced into the baseline dataset.

Table 2: Reported sensor placement with frequency band for stress studies included in this review. |  |  |  |  |  |  |
---|---|---|---|---|---|---|---
Paper |  Year |  Title |  Frontal |  Temporal |  Central |  Parietal |  Occipital
[31] |  2016 |  Stress assessment by prefrontal relative gamma |  Alpha, Beta, Theta |  Alpha |  Alpha, Gamma |  Alpha |
[32] |  2018 |  Selection of neural oscillatory features for human stress classification with single channel EEG headset |  Beta, Gamma |  |  |  |
[37] |  2020 |  EEG based classification of long-term stress using psychological labeling |  Beta, Gamma |  |  |  |
[39] |  2020 |  PASS: A multimodal database of physical activity and stress for mobile passive body/ brain-computer interface research |  |  Gamma, Delta |  |  Gamma, Delta |
[40] |  2020 |  Multivariate correlation measures reveal structure and strength of brain-body physiological networks at rest and during mental stress |  Theta, Alpha |  |  |  |
[7] |  2020 |  On identification of driving-induced stress using electroencephalogram signals: A framework based on wearable safety-critical scheme and machine learning |  Alpha |  Gamma |  |  Theta, Alpha |  Gamma
[49] |  2021 |  Human stress classification during public speaking using physiological signals |  Alpha, Beta |  |  |  |
[50] |  2021 |  Relationship between rework of engineering drawing tasks and stress level measured from physiological signals |  Beta |  Beta |  |  |
[55] |  2021 |  Psychological and physiological effects of a green wall on occupants: A cross-over study in virtual reality |  |  |  |  |
[21] |  2022 |  Assessing electroencephalography as a stress indicator: A VR high-altitude scenario monitored through EEG and ECG |  Beta |  Gamma, Beta |  |  Gamma, Beta |  Gamma, Beta
[58] |  2022 |  A Multimodal perceived stress classification framework using wearable physiological sensors |  |  Theta, Beta |  |  Theta, Beta |
[60] |  2022 |  Frontal lobe real-time EEG analysis using machine learning techniques for mental stress detection |  Theta, Beta, Gamma |  |  |  |
[67] |  2023 |  Cheating detection in E-exams system using EEG signals |  Alpha, Beta |  |  |  Alpha, Beta |
[73] |  2023 |  An innovative random-forest-based model to assess the health impacts of regular commuting using non-invasive wearable sensors |  Beta |  |  |  |
[70] |  2023 |  A deep learning approach for assessing stress levels in patients using electroencephalogram signals |  Delta |  Delta |  |  |
[71] |  2023 |  Brain activity monitoring for stress analysis through EEG dataset using machine learning |  Beta, Gamma |  |  |  |
|  |  |  |  |  |  |

###  3.2 Low-Cost EEG Devices in Stress Research

Consumer-oriented EEG devices such as the Emotiv EPOC [80] have been present
in the market since at least 2010 (Table 3). Most of these devices would be
considered low-cost in terms of total cost of ownership, operation and
interpretation, given that they are mostly re-usable with supporting software
and platforms to assist with interpretation of results. For this review, the
scope for low-cost device inclusion was limited to those retailing for less
than USD 1,000.00. Typical pricing for the devices included at the time of
writing ranged from USD249.99 for the InteraXon Muse 2 (5 sensors) [81], up to
USD849 for the Emotiv EPOC (14 sensors). This range was chosen to be
comparable to smartwatches with embedded health monitoring sensors that can be
used for stress measurement [23].

####  3.2.1 Dominant low-cost EEG devices

Of the studies included in this review using a low-cost EEG device, 23 used
either the Emotiv [27, 34, 37, 40, 72, 41, 7, 50, 55], InteraXon [58, 67, 61,
39, 49, 52] or NeuroSky [20, 38, 71, 32, 73, 75] range of devices [82] (Table
3). Two studies [46, 44] utilized the OpenBCI [83] series of devices
consisting of an 8-channel OpenBCI Cyton board [83], combined with a low-cost
skull cap and electrodes. A lower cost Ganglion board with 4 channels is also
available, retailing at USD499.00, compared to the Ganglion board which
retails for USD999.00. OpenBCI introduced their EEG Headband Kit in 2022 which
can be paired with either the Ganglion or Cyton board. The kit retails at
USD279.99.

####  3.2.2 Custom EEG devices

Three studies included in this review used custom designed EEG devices, mostly
built by hand. Lee _et al._ [19] designed a wearable in-and over-ear bio-
metric sensing device that measured EEG and electrocardiogram (ECG) signals
simultaneously. They further proposed a novel sensing electrode which is
highly conductive, dry and flexible, and designed with portability and comfort
in mind during long-term usage. Ahn _et al._ [35] designed a wearable, two-
channel EEG and single channel ECG device that was lightweight and exhibited
excellent noise management performance. Affanni _et al._ [22] designed a six-
channel EEG wearable headband that transmits data over WiFi to a laptop,
featuring a rechargeable battery with 10 hours of continuous transmission
life.

####  3.2.3 Additional EEG devices

Fifteen studies [25, 33, 21, 57, 68, 69, 60, 31, 48, 62, 43, 51, 53, 56, 65]
provided detail on the EEG hardware used for recording. Of these, the BioSemi
ActiveTwo [84] device was the most prominent and was used in 7 studies. The
remaining studies [28, 5, 24, 16, 59, 70, 30, 63, 64, 26, 42, 45, 54, 66, 17]
did not explicitly provide identifiable or named EEG hardware configuration.

Table 3: Low-cost EEG devices included in this review.

Device | Released | Sensors | Data Download?
---|---|---|---
Emotiv EPOC | 2010 | AF3, AF4, F3, F4, F7, F8, FC5, FC6, O1, O2, P7, P8, T7, T8 | Y
NeuroSky MindWave | 2011 | FP1 | Y
NeuroSky Mindwave Mobile | 2012 | FP1 | Y
Emotiv EPOC+ | 2013 | AF3, AF4, F3, F4, F7, F8, FC5, FC6, O1, O2, P7, P8, T7, T8 | Y
InteraXon Muse | 2014 | AF7, AF8, Fpz, TP9, TP10 | Y
Emotiv INSIGHT | 2015 | AF3, AF4, T7, T8, Pz | Y
InteraXon Muse 2 | 2018 | AF7, AF8, Fpz, TP9, TP10 | Y
NeuroSky MindWave Mobile II | 2018 | FP1 | Y
OpenBCI EEG Headband Kit | 2022 | Up to 8, User-configurable | Y

####  3.2.4 Sensor support and placement

Given the wide range of frequency bands reportedly used for stress research,
requiring multiple scalp sensors to cover multiple brain lobes (Table
LABEL:tab:wavesreviewed), it is important to consider sensor support and
placement for low-cost devices. As shown in Table 3, the Emotiv EPOC series of
devices provide the broadest range of sensors (14), while the OpenBCI provides
the most flexibility in sensor configuration due to its ability to pair with
any sensor configuration (up to 16 channels). The NeuroSky Mindwave Mobile 2
device provides a single sensor resting on the forehead above the eye (FP1
position) with a separate reference electrode that is clipped to the left ear.
Prior studies have reported the FP1 position to be involved in cognition,
working memory and perception [85].

In contrast, wrist and ring based wearable devices used for stress research
typically offer a smaller, but more specialized range of sensors including for
measuring heart or pulse rate, electrodermal activity, skin temperature and a
three-axis accelerometer. This leads to more restrictive but simpler device
placement and more standardization of sensor types across devices, resulting
in less pre-processing during data analysis. Additionally, due to the nature
of the recorded biomarkers, data produced by these sensors are simpler to
standardize across devices requiring less pre-processing for reproducing prior
study results.

###  3.3 Datasets for EEG-based Stress Measurement

Datasets that are publicly available (or available upon request) provide
researchers with pre-labeled EEG sensor data for experimentation and
reproducibility of prior results. Eight publicly available datasets were
included in this review (Table LABEL:tab:datasets), of which three (PASS,
BrainBodyStress, EDPMSC) [39, 40, 36] are specifically designed and labeled
for stress. Four are labeled for emotion (DEAP, SEED, K-EmoCon, GAMEEMO) [25,
29, 20, 41] and one dataset (DASPS) is labeled for anxiety [34]. Although
labeled for emotion, the DEAP, SEED, K-Emocon and GAMEEMO datasets could
potentially be utilized for stress research due to measuring states of arousal
and valence which are known characteristics of emotional stress [86]. Among
the datasets reviewed, the Emotiv EPOC EEG device was the most frequently
used. The remaining datasets use a variety of other EEG devices.

####  3.3.1 Dataset use and consideration

Some of the eight datasets included in this review are used more frequently.
For instance, six studies utilized the DEAP dataset [25, 69, 62, 53, 65, 74]
for experimentation, while only two used the SEED dataset [29, 74]. Parent _et
al._ [39] designed and utilized the PASS stress dataset. Similarly, Alakus _et
al._ [41] designed and utilized the GAMEEMO stress dataset, while Pernice _et
al._ [40] developed the BrainBodyStress dataset. Arsalan _et al._ [36]
collected and curated the EDPMSC dataset. The DEAP, DASPS, PASS, GAMEEMO,
BrainBodyStress and EDPMSC datasets specifically noted employing a health
screening protocol during dataset development.

####  3.3.2 Incorporation of additional biomarkers

Four datasets included additional biomarkers to enable extended
experimentation (Table LABEL:tab:datasets), including ECG, Galvanic Skin
Response (GSR) [25, 39, 40], environmental Temperature (TEMP) [39, 20], HR
[20], Blood Volume Pulse (BVP) [20, 40], Inter-Beat Interval (IBI) and EDA
[20]. For non-EEG biomarkers, the device most commonly used for data recording
was the Empatica E4 wrist-worn health monitor [87], commonly used in stress
research [13, 14, 23, 88].

####  3.3.3 Recording duration and stress labeling methods

The recording duration (Table LABEL:tab:datasets) of the datasets included in
this review varied from 3 minutes (DASPS dataset) to over 30 minutes (PASS and
BrainBodyStress datasets). Stress is not a binary condition, further
complicating clinical diagnosis when assessing EEG results. Researchers
typically employ several different known methods for labeling or assessing
biomarker data including the STROOP Color and Word Test [89, 90], utilized in
[27, 70], the Perceived Stress Scale [91], utilized in [32, 37, 58], and the
State-Trait Anxiety Inventory [92], utilized in [7, 61]. Alternatively,
researchers may label biomarker data according to relaxation periods and
periods where stress is induced, as utilized in [39, 40, 67, 75].

The DEAP dataset [25] recorded a 2-minute non-stressed baseline followed by a
1-minute display of a music video to assess arousal, valence, liking and
dominance. This was repeated for 20 trials. The SEED dataset [29] provided a
5-second hint prior to displaying a movie for 4 minutes, followed by a
45-second period to allow for self-assessment via a questionnaire. This was
followed by a 15-second rest period, repeated for a total of 15 trials. The
DASPS datasets [34] performed a total of 6 trials consisting of a 5-minute
baseline, 30-second stimuli, and a 4-minute post-stimuli period during which a
self-assessment questionnaire was completed.

The PASS dataset [39] simulated a stressful event through the use of two video
games, one being stressful while the other was considered non-stressful.
Additionally, this dataset implemented a 2-minute baseline followed by 5 to 15
minutes of game-play, with a 5-minute break between sessions. Two datasets
provided short, but well-balanced periods of baseline and stress conditions.
The BrainBodyStress dataset [40] used a 12-minute relaxation baseline,
followed by a 12-minute sustained attention task and a 7-minute mental
arithmetic task, while the EDPMSC dataset [36] used a 3-minute relaxation
baseline followed by 3 minutes of public speaking, and a 3-minute post-
activity phase.

####  3.3.4 Data formats and accessibility

The datasets included in this review are provided in a wide range of file
formats, including Comma-Separated Values (CSV), MATLAB [93], and the European
Data Format (EDF) [94]. Researchers looking to utilize these datasets in their
studies are therefore required to use statistical software capable of
supporting these formats or develop external conversion scripts. A number of
these datasets provided both raw and pre-processed filtered data. Providing
raw EEG data, as is the case with the DEAP, K-EmoCon, PASS and GAMEEMO
datasets (Table LABEL:tab:datasets), allows researchers to experiment with
various pre-processing and filtering techniques to match their experimental
requirements.

####  3.3.5 Data processing and artifact removal

Low-cost, consumer-oriented health tracking and monitoring devices frequently
provide a sophisticated software platform with built-in analytics to deliver
insights directly to users, often as a paid subscription model. While this is
of great benefit to consumers and other non-experts, for researchers this
likely implies less control over data processing and artifact removal. The
NeuroSky range of devices transmit processed data via a wireless Bluetooth
connection to a computer, and offer a wide range of plug-in modules to extend
the base software functionality of the device. NeuroSky devices apply a Fast
Fourier Transform (FFT) to the raw EEG prior to transmission. For the Emotiv
EPOC series devices, baseline removal is applied prior to finite impulse
response (FIR) filtering, channel rejection and independent component analysis
(ICA) for artifact removal. The InteraXon Muse devices can be configured to
provide raw, unprocessed data, while the OpenBCI devices provide raw EEG
signals by default.

Pernice _et al._ [40] performed artifact removal using a high-pass filter
(half power frequency of 1 Hz) and a low-pass filter (half power frequency of
20 Hz), followed by power spectral density (PSD) calculation for each EEG
signal using a 2-second sliding window with 50% overlap. Hou _et al._ [27],
Suryawanshi _et al._ [71] and AlShorman _et al._ [60] applied FFT to extract
the mean power spectrum for EEG frequency bands (delta, theta, alpha, beta and
gamma), while Majid _et al._ [58] applied FFT using a window size of 256 with
an overlap of 90%. Before applying the FFT, AlShorman _et al._ [60] filtered
the raw signal by using a band pass filter with a cut-off frequency of 0.1–30
Hz, while Suryawanshi _et al._ [71] applied a band-pass filter at 1Hz,
subsequently followed by a low-pass filter at 50Hz for artifact removal, a
standard practice in EEG signal processing [95].

While artifact removal and filtering techniques are common across all studies
reviewed, and are a normal part of EEG analysis, slight differences in
algorithmic application and sequence of implementation could potentially
hinder the reproducibility of prior reported results [16, 96, 24, 64]; more so
when the EEG signals are recorded using different devices. Reproducibility
would therefore require the exact step-by-step pre-processing techniques,
statistical software, libraries and algorithms on the raw EEG sensor data.

Numerous factors can affect EEG measurement [64] including sensor type,
placement, signal processing methods and artifact treatment and removal. In
this review, a large and varied number of methods were found to be utilized
showing a lack of standardization and no definitive approach to signal
processing employed across the studies included in this review. Katmah _et
al._ [16], Safayari _et al_ [96] and Newson _et al._ [24] noted critical
differences reported with respect to model accuracy and input features between
stress-related studies utilizing EEG and suggested the variation could be due
to lack of standardization during data pre-processing, feature extraction and
experimental setup and duration. Newson _et al._ [24] further noted
differences in band ranges that define each frequency range due to either
hardware configuration and limitations or pre-processing implementation during
data analysis, ultimately leading to differences in results interpretation.

Table 4: Datasets included in this review. Paper |  Year |  Dataset | Device(s) |  Biomarkers |  Subjects |  Gender |  Age (Mean) |  Health |  Condition |  Duration
---|---|---|---|---|---|---|---|---|---|---
[25] |  2012 |  DEAP |  Biosemi ActiveTwo |  EEG, GSR, ECG |  32 |  16 F, 16 M |  26.9 |  Y |  Emotion (SAM) |  15 min
[29] |  2015 |  SEED |  NeuroScan |  EEG |  15 |  8 F, 7 M |  |  |  Emotion |  5 min
[34] |  2019 |  DASPS |  Emotiv EPOC |  EEG |  23 |  13 F, 10 M |  30 |  Y |  Anxiety (HAM-A, SAM) |  3 min
[36] |  2019 |  EDPMSC |  InteraXon Muse |  EEG |  40 |  20 F, 20 M |  24.85 |  Y |  Stress |  6 min
[39] |  2020 |  PASS |  InteraXon Muse, BioHarness 3, E4 |  EEG, ECG, EDA, TEMP |  48 |  |  |  Y |  Stress (NASA-TLX, BORG) |  30 min
[20] |  2020 |  K-EmoCon |  NeuroSky MindWave, E4, Polar H7 |  EEG, HR, TEMP, IBI, BVP, EDA |  32 |  12 F, 20 M |  23.8 |  |  Emotion |  10 min
[41] |  2020 |  GAMEEMO |  Emotiv EPOC+ |  EEG |  28 |  |  23.5 |  Y |  Emotion (SAM) |  20 min
[40] |  2020 |  BrainBodyStress |  Emotiv EPOC+, Empatica E4 |  EEG, ECG, BVP |  18 |  5 F, 13 M |  25 |  Y |  Stress |  30 min

###  3.4 Machine Learning Techniques for EEG-based Stress Measurement

Of the studies included in this review, thirty applied machine learning
techniques to predict or measure acute stress response. Of these, thirteen
utilized a low-cost EEG device to record brain activity, as shown in Table
LABEL:tab:mlreviewed. The study by AlShorman _et al._ [60] where a 128-channel
EEG device was used, and Bhatnagar _et al._ [70] study did not report their
devices. Both of these studies reported the highest accuracy rates from the
thirty studies reviewed and were therefore used as a baseline for comparison
to low-cost EEG results reported.

####  3.4.1 Data modalities

Ten of the fifteen studies utilized only EEG data as input to their machine
learning algorithms. Five studies [39, 40, 49, 58, 73] implemented a multi-
modal approach incorporating additional biomarker data from ECG or other
health monitoring devices (EDA, GSR, TEMP, BP, PPG). Majid _et al._ [58] used
a signal fusion approach to combine raw EEG from a Muse device with GSR and
Photoplethysmography (PPG) signals from additional devices into a single
feature set. Additionally, three studies [58, 69, 19] employed fusion-based
architectures to combine the features from EEG with other health monitoring
devices.

####  3.4.2 Machine learning algorithms

The most widely applied machine learning algorithm across the studies reviewed
was the Support Vector Machine (SVM) (Table LABEL:tab:mlreviewed). SVM is a
linear model commonly used for classification and regression problems in
machine learning studies and can solve both linear and non-linear problems by
creating a line or a hyperplane to separate input data into separate classes.
When applied to stress research, these classes would typically be one of
stressed or non-stressed. A number of more advanced techniques were noted in
the studies reviewed, however, these were not applied to data recorded from
low-cost EEG devices. Bhatnagar _et al._ [70] utilized EEGNet, a compact
convolutional neural network for EEG-based brain–computer interfaces first
proposed by Lawhern _et al._ [97]. Also noted were the use of Spiking Neural
Networks (SNN) [53], graph models [74] and the use of genetic algorithms [62].

Table 5: Machine Learning studies included in this review. |  |  |  |  |  |  |
---|---|---|---|---|---|---|---
Paper |  Year |  Device(s) |  Biomarkers |  Algorithm(s) |  Accuracy |  Subjects |  Validation
[27] |  2015 |  Emotiv EPOC |  EEG |  SVM |  85.71 |  9 |  5-Fold
[32] |  2018 |  NeuroSky MindWave |  EEG |  SVM |  78.57 |  28 |  10-Fold
[7] |  2020 |  Emotiv EPOC+ |  EEG |  SVM, NN, RF |  97.95 (ENS) |  86 |  70/30
[39] |  2020 |  InteraXon Muse, BioHarness 3, Empatica E4 |  EEG, ECG, EDA, TEMP |  SVM |  74 |  48 |  5-Fold, LOSO
[40] |  2020 |  Emotiv EPOC+, Empatica E4 |  EEG, ECG, BVP |  LR |  |  18 |  Fisher F-test
[37] |  2020 |  Emotiv INSIGHT |  EEG |  SVM, NB, KNN, LR, MLP |  85.2 (SVM) |  33 |  10-Fold
[49] |  2021 |  InteraXon Muse |  EEG, GSR, PPG |  KNN, DT, RF, MLP, SVM |  96.25 (SVM) |  40 |  LOSO
[46] |  2021 |  OpenBCI |  EEG |  SVM, CNN, LTSM |  93.27 (LSTM) |  13 |  70/30
[60] |  2022 |  Geodesic |  EEG |  SVM, NB |  98 (SVM) |  14 |  3-Fold CV
[58] |  2022 |  InteraXon Muse Headband, Shimmer GSR |  EEG,GSR, PPG |  SVM, NB, MLP |  95 (MLP) |  40 |  LOSO
[61] |  2022 |  InteraXon Muse Headband |  EEG |  MLP, LSTM |  93.17 (LTSM) |  40 |  10-Fold
[70] |  2023 |  |  EEG |  EEGNet |  99.45 |  45 |  10-Fold
[67] |  2023 |  InteraXon Muse 2 |  EEG |  CNN |  97.37 |  15 |  70/30
[73] |  2023 |  NeuroSky Mindwave Mobile, MySignals |  EEG, BP |  RF, KNN, SVM, NB |  91 (RF) |  45 |  10-Fold
[71] |  2023 |  NeuroSky Mindwave Mobile |  EEG |  SVM, KNN, NN |  90 (SVM) |  |  70/30

As detailed in Table LABEL:tab:mlreviewed, nine studies tested several
different algorithms, with the best-performing algorithm shown in the
_Accuracy_ column. SVM outperformed other algorithms in all but four studies.
Of those, two [46, 61] utilized a dual-layer Long Short-Term Memory (LSTM)
network. The study by Sharif _et al._ [73] was the only one where a tree-based
algorithm provided the highest accuracy of any combination. While Halim _et
al._ [7] achieved their highest accuracy by combining the outputs of SVM,
Random Forest (RF) and a Neural Network (NN) through an ensembling (ENS)
approach, SVM outperformed both RF and NN before combining the predictions.

Eight of the fifteen studies [27, 32, 39, 37, 60, 61, 70, 73] validated their
results using a K-Fold cross-validation approach, while four [7, 46, 67, 71]
utilized a more traditional train-test split, where 70% of the data was used
for training, and the predicted results validated against the remaining 30%.

Of the fifteen studies listed in Table LABEL:tab:mlreviewed, seven reported
the relative importance of the frequency bands used as algorithm input
features. Three studies [7, 40, 70] reported the delta band to be the most
important input feature, while alpha was reported by two [40, 49], beta by two
[49, 73], gamma by two [7, 39] and theta by a single study [58]. Frontal and
temporal lobes were the most reported locations for sensor placement [39, 40,
70].

###  3.5 Study Quality

The _Supplementary File_ details the results of the IJMEDI quality assessment.
Table 6 summarizes the scores of each dimension and the total score of each
study. The average score of the included studies was 26 (range: 11–42.5). Most
of the studies were of medium quality, while two [39, 37] were of high
quality.

The majority of the studies lacked quality in the data preparation, validation
and deployment dimensions. Figure 6 shows the proportion of the different
answers in the high and low priority items. Importantly, high-priority items
were generally well addressed, with clear problem understanding and a very
strong focus on machine learning modeling techniques and implementation.

Figure 6:  Study quality assessment based on the IJMEDI checklist. Proportion
of the different answers in the high- and low-priority items. OK = adequately
addressed; mR = sufficient but improvable; MR = inadequately addressed.

Table 6: Quality assessment scores of the 14 ML-based studies according to the
IJMEDI checklist.

| Problem Understanding (10) | Data Understanding (6) | Data Preparation (8) | Modeling (6) | Validation (12) | Deployment (8) | Total (50)
---|---|---|---|---|---|---|---
[60] | 8 | 5 | 4 | 6 | 5 | 1 | 29
[49] | 7 | 5 | 2 | 6 | 6 | 3 | 29
[70] | 5 | 0 | 0 | 6 | 7 | 0.5 | 18.5
[7] | 8 | 4 | 2 | 6 | 7.5 | 2.5 | 30
[27] | 7 | 4 | 2 | 6 | 7 | 1 | 27
[58] | 8 | 2 | 2 | 6 | 7 | 0 | 25
[67] | 5 | 0 | 0 | 6 | 7 | 1 | 19
[39] | 9 | 4 | 8 | 6 | 9 | 6.5 | 42.5
[61] | 8 | 4 | 1 | 6 | 7 | 0 | 26
[32] | 8 | 2 | 2 | 6 | 6 | 1 | 25
[37] | 10 | 6 | 4 | 6 | 8 | 2 | 36
[73] | 7.5 | 2 | 4 | 6 | 6 | 0 | 25.5
[46] | 8 | 4 | 3 | 6 | 6 | 0 | 27
[71] | 2 | 0 | 1 | 6 | 2 | 0 | 11

##  4 Discussion

In order to build a robust machine learning model capable of accurately
measuring stress via low-cost EEG sensors, we consider three important
requirements. These include (i) EEG devices need to be technically capable of
reading signals that correlate with acute stress response; (ii) For
classifying stress through machine learning, high-quality training data needs
to be available to properly train a classification algorithm; (iii) The
resulting trained machine learning model should generalize by accurately
predicting for new, unseen data. The discussion of this review is therefore
focused on those three key requirements.

###  4.1 Requirement 1: Low-cost EEG Device Capabilities

Low-cost, wireless EEG devices are generally limited in terms of sensor
placement and count (Table 3). Hence, experiments utilizing these devices
would be naturally biased towards reporting positive outcomes concerning these
scalp locations. The majority of the studies included in this review focused
on the frontal, parietal and temporal regions (Figure 7), likely due to
limited device sensor support (Table 3) for central and occipital sites.
Within these studies, the beta, gamma and alpha frequency bands provided the
highest correlation with stress.

Figure 7: Frequency band utilization and sensor locations of the low-cost EEG
devices reviewed in this study.

####  4.1.1 Sensor support

Based on the findings of these aforementioned studies, these devices do appear
to offer sufficient sensor support for measuring stress. Seo _et al._ [77]
reports close relationships among EEG, ECG, and salivary cortisol indicative
of acute stress, with chronic stress particularly associated with high levels
of relative beta power, albeit at anterior temporal sites. Both the Emotiv and
InteraXon series of devices does have sensor support across frontal, parietal
and temporal areas, with the Emotiv range additionally supporting occipital
sites.

####  4.1.2 Ease of use

Of further importance to researchers when opting to use low-cost, wireless EEG
devices are ease of use [32, 7, 58, 67] and acquisition and operational costs
[61, 67]. A number of studies included in this review noted the mobility and
affordability of EEG headsets as a key requirement for their specific study
requirements [32, 7, 58, 67, 73]. Halim _et al._ [7] performed a study to
evaluate the feasibility of detecting stress while subjects were driving motor
vehicles, while Arsalan _et al._ [49] performed a study on measuring stress
during public speaking. A mobile EEG device was therefore a technical
necessity for both studies. However, very few studies have investigated stress
detection and measurement in the presence of varying levels of physical
activity [39].

####  4.1.3 Technical challenges

None of the studies included in this review reported any technical challenges,
device failure or significant signal quality issues that required additional
care or action during experimentation. While there are numerous benefits in
using these lower cost EEG devices, researchers need to be acutely aware of
any limitations when using such devices. For example, Pernice _et al._ [40]
noted in their study that EEG devices with dry electrode, scalp level
connectivity, such as those reviewed in this study, do not permit a perfectly
reliable interpretation of interacting brain areas, as they can be corrupted
by volume conduction effects or confounding factors. However, Hinrichs _et
al._ [98] compared dry and wet EEG systems and found slightly higher artefacts
for dry EEG systems while being more robust to 50Hz line noise, and concluded
that dry EEG systems comply with the needs of clinical applications.

###  4.2 Requirement 2: Machine Learning Input Data Quality

Robust machine learning models require high-quality training data, and a
number of datasets are publicly available containing EEG data for
experimentation and/or model validation (Table LABEL:tab:datasets). Three of
these datasets are specifically labeled for stress, of which the PASS dataset
contains the largest sample size and includes additional biomarkers (ECG, EDA,
TEMP) that have previously been shown to be useful features for building
machine learning models capable of detecting and measuring stress [77, 31,
23].

####  4.2.1 Data variability

A key challenge in utilizing these datasets for validation of trained models
would be adjustment for differences in device brand and sensor availability.
Variation in signal quality or sensor type could hinder a direct data
comparison when using both datasets as training and or validation pairs [64],
while differentiation in experimental recording protocol could introduce noise
into the data [64]. In their review, Katmah _et al._ [16] highlighted critical
differences between research findings and argued that variations in the data
analysis methods contributed to several contradictory results. Newson _et al._
[24] advise to refrain from making any clinical interpretations based on
current findings due to the significant lack of standardization in frequency
band definition.

Researchers considering the use of these datasets, whether for training
purposes or for the validation of novel models, will consequently need to
perform substantial pre-processing while accounting for potential device
limitations in order to ensure robust and accurate analyses.

####  4.2.2 Sample size

Of the datasets included in this review, most contained data for a small
sample size of test subjects, recorded using a number of different EEG
devices. Campos-Ugaz _et al._ [99] noted that almost all EEG studies have
small sample sizes, which considerably reduces the generalizability of their
findings. Sundaresan _et al._ [46] noted high inter- and intra-individual
variability of neuro-physiological signals to be a central challenge for EEG-
based stress studies, and further suggested using datasets containing a
sufficiently large number of test subjects.

However, simply increasing sample size may not address the issue of
generalization alone. Chaumon _et al._ [100] used Magnetoencephalography (MEG)
data from a large sample of subjects to examine the behavior of statistical
power measures and concluded that there is no unequivocal answer to the
question of an appropriate sample size in MEG or EEG experiments. Boudewyn _et
al_ [101] specifically aimed to determine the number of trials required to
achieve an Event-Related Potential (ERP) effect in EEG studies, and concluded
that several factors have an impact on statistical power in ERP studies,
including number of trials, sample size, and effect magnitude, as well as
interactions among these factors.

####  4.2.3 Data labeling

Finally, a critical aspect when performing supervised classification using
machine learning approaches is accurate data labeling. This refers to the
process of marking segments of data to correspond with the expected class, for
example stressed or non-stressed. The datasets included in this review all
included a baseline period labeled as non-stressed, with an experimental
period marked as stressed. The duration of both baseline and experimental
periods differed notably across the datasets, with the baseline period often
being short, resulting in a class imbalance that researchers will need to
adjust for when using these datasets for classification tasks.

####  4.2.4 Study quality

Having scored the machine learning studies included in this review using the
IJMEDI checklist, we found two [39, 37] of high quality, with nine studies
being of medium quality [60, 49, 7, 27, 58, 61, 73, 46, 32], and two studies
being of low quality [67, 71]. Most studies scored well in problem, validation
and modeling dimensions. Data preparation and deployment scores were notably
low, with little focus on the deployment of models in real-life scenarios,
including factors pertaining to sustainability, model bias and ethics. For
data preparation scoring, the IJMEDI checklist focuses on four key areas
including outlier detection and treatment, class balancing, missing-value
identification and management and data pre-processing, and lower scores
indicate that these items are not described or described in sufficient detail.
The highest overall quality score was attained by Parent _et al._ [39] (42.5),
with high scores across all dimensions. The study by Pernice _et al._ [40] was
excluded from the assessment due to it being primarily focused on multivariate
correlation analysis to investigate dynamic interactions between organ systems
in the human body during stress.

###  4.3 Requirement 3: Machine Learning Model Generalization

When training a machine learning model, researchers need to select the
appropriate learning method based on the desired outcome. This outcome could
be binary (stressed or non-stressed) or contiguous (level of stress being
experienced). All studies reviewed developed a binary model. In addition,
supervised and unsupervised learning [102] are two different types of machine
learning techniques which differ in the way the models are trained, and the
type of training data required. All fifteen studies included in this review
utilized supervised training and reported prediction for a binary class
(stressed/non-stressed). Figure 8 provides a summary of the mean predictive
accuracy reported across these studies, grouped by EEG sensor location and
frequency band used as primary algorithm input features.

####  4.3.1 Prediction accuracy and its significance

Ten of the fifteen studies (66.6%) reported predictive accuracy of more than
90% [7, 49, 46, 60, 58, 61, 70, 67, 73, 71], with the frontal and parietal
regions providing the most consistent results across the gamma, alpha and beta
frequency bands. However, Newson _et al._ [24] highlighted in their
examination of EEG frequency bands within psychiatric disorders that
distinguishing whether alterations in specific frequency bands are unique to
individual disorders or if there might be overlaps, poses challenges that
could restrict the potential for clinical diagnosis.

####  4.3.2 Health screening

Of the datasets included in this review, three [36, 29, 20] did not provide
extensive detail on health screening of subjects included in their datasets,
while a further three noted that subjects were healthy [25, 41, 40]. Two
datasets [34, 39] specifically noted that test subjects were health screened
specifically for neurological and psychological disorders. The lack of
consistent health screening and detailed information on the screening process
could potentially affect study reproducibility, and bring into question the
high accuracy results reported should undiagnosed mental health conditions be
present in the data.

Figure 8: Reported accuracy by frequency band utilization and sensor
locations.

####  4.3.3 Validation methods

A number of methods were employed for results validation in the studies
reviewed (Table LABEL:tab:mlreviewed), including self-scoring questionnaires
used as correlation measures against the predicted stress level reported by
the machine learning algorithm. Saeed _et al._ [37, 32] used the Perceived
Stress Score, while Halim _et al._ [7] reported using the Self-Assessment
Manikins, devised by Bradley _et al._ [103]. Phutela _et al._ [61] used the
State Trait Anxiety Inventory. Statistical methods were also reported, with
Parent _et al._ [39] validating their findings using a two-way ANOVA test,
while Pernice _et al._ [40] used the Fisher F-Test. Luck _et al._ [104]
however noted that statistically significant but unpredicted and unreplicable
effects in EEG experiments can easily be found due to the high random
variation in EEG data, making it hard for researchers to determine if a noted
effect is real, and that study replication and reproducibility should be key
to experiment validation. None of the fifteen studies reviewed validated their
findings on a new, unseen dataset.

####  4.3.4 Cross validation using other wearables

Three studies that incorporated signals from external devices to augment their
input features [49, 58, 73] noted a significant correlation between heart rate
and the predicted level of stress. Seo _et al._ [77] specifically reported a
correlation between HRV and high beta activity at anterior temporal sites,
opening the possibility of combining EEG with other wearable devices such as
the Empatica range to cross-validate stress measurement between these devices.
While a number of studies included in this review [75, 39, 40, 31] did measure
BVP, HR and HRV, none specifically cross-validated stress reported via EEG
with those reported via heart rate. This could potentially provide a way
forward for researchers by using additional wearable devices, such as the
Empatica [87] range, to record heart rate and other biomarkers that have been
shown to correlate with increased levels of stress [15], and using those as a
validation mechanism for stress-related EEG studies.

###  4.4 Summary

The significant observations from this review are:

  * 1.

Low-cost EEG devices are increasingly being used for mental health related
studies, including stress. However, sensor availability and placement differ
widely across devices, potentially limiting the reproducibility of results
reported from machine learning studies utilizing these devices.

  * 2.

A small number of datasets are publicly available for researchers to use in
stress related studies, including for results validation. However, most
studies included in this review did not validate their findings and results on
an independent, unseen dataset.

  * 3.

None of the studies included in this review sufficiently considered the impact
of underlying, undiagnosed mental health conditions on model predictive
accuracy. This is especially important when considering the small size of
study groups within the studies included in this review.

###  4.5 Challenges and future research directions

To achieve robust and reliable machine learning models suitable for real-world
monitoring of stress using low-cost EEG devices, future research should
address three formidable challenges:

  * 1.

The pre-processing of EEG data needs to be standardized to enable cross-device
studies with the same machine learning models. Future research can develop a
standard framework for dealing with data produced by low-cost EEG devices.

  * 2.

While several EEG datasets recorded using low-cost devices have been made
public, future work can provide larger datasets containing more significant
and high-quality baseline (non-stressed) periods along with stressed periods
to reduce data imbalance and assist researchers with model training and
validation.

  * 3.

A limited number of the studies reviewed utilized additional biomarkers
recorded with non-EEG wearable devices. The use of other wearable devices that
record biomarkers including heart rate and galvanic skin response could be
further explored in two different ways. One is as a mechanism for validating
the findings from stress-related studies, where EEG signals are the primary
data-providing mechanism. Another is in a multi-modal approach, where both EEG
and wearable biomarkers are used for stress prediction.

##  5 Conclusion

The main objective in algorithmic stress detection and measurement is to
develop a robust, highly accurate model that can provide reproducible results.
Low-cost EEG devices can potentially facilitate the development of machine
learning models that produce accurate stress measurements and are reproducible
across studies, irrespective of the study group or study setting. Nonetheless,
several important factors should be taken into account. The review presented
here synthesized the literature and discussed these essential factors in
previous studies. In particular, we reviewed and analyzed the devices
available, and their use in stress-related studies using machine learning
techniques, while including their advantages, limitations and experimental
quality. We also summarized our point of view on the challenges and
opportunities in this emerging domain. We believe this review will advance
knowledge in the general area of machine learning for stress detection using
low-cost EEG devices, helping the research efforts move one step closer to
realizing effective stress detection and management technology.

## References

  * [1] B. S. McEwen, Protective and damaging effects of stress mediators, New England Journal of Medicine 338 (3) (1998) 171–179.  doi:10.1056/nejm199801153380307.
  * [2] B. McEwen, Physiology and neurobiology of stress and adaptation: Central role of the brain, Physiological Reviews 87 (3) (2007) 873–904.  doi:10.1152/physrev.00041.2006.
  * [3] J. J. McGrath, A. Al-Hamzawi, J. Alonso, Y. Altwaijri, L. H. Andrade, E. J. Bromet, R. Bruffaerts, J. M. C. de Almeida, S. Chardoul, W. T. Chiu, L. Degenhardt, O. V. Demler, F. Ferry, O. Gureje, J. M. Haro, E. G. Karam, G. Karam, S. M. Khaled, V. Kovess-Masfety, M. Magno, M. E. Medina-Mora, J. Moskalewicz, F. Navarro-Mateu, D. Nishi, O. Plana-Ripoll, J. Posada-Villa, C. Rapsey, N. A. Sampson, J. C. Stagnaro, D. J. Stein, M. ten Have, Y. Torres, C. Vladescu, P. W. Woodruff, Z. Zarkov, R. C. Kessler, S. Aguilar-Gaxiola, A. Al-Hamzawi, J. Alonso, Y. A. Altwaijri, L. H. Andrade, L. Atwoli, C. Benjet, E. J. Bromet, R. Bruffaerts, B. Bunting, J. M. C. de Almeida, G. Cardoso, S. Chardoul, A. H. Cía, L. Degenhardt, G. D. Girolamo, O. Gureje, J. M. Haro, M. G. Harris, H. Hinkov, C. yi Hu, P. D. Jonge, A. N. Karam, E. G. Karam, G. Karam, A. E. Kazdin, N. Kawakami, R. C. Kessler, A. Kiejna, V. Kovess-Masfety, J. J. McGrath, M. E. Medina-Mora, J. Moskalewicz, F. Navarro-Mateu, D. Nishi, M. Piazza, J. Posada-Villa, K. M. Scott, J. C. Stagnaro, D. J. Stein, M. T. Have, Y. Torres, M. C. Viana, D. V. Vigo, C. Vladescu, D. R. Williams, P. Woodruff, B. Wojtyniak, M. Xavier, A. M. Zaslavsky, Age of onset and cumulative risk of mental disorders: a cross-national analysis of population surveys from 29 countries, The Lancet Psychiatry (jul 2023).  doi:10.1016/s2215-0366(23)00193-1.
  * [4] E. Hendry, B. McCallister, D. J. Elman, R. Freeman, D. Borsook, I. Elman, Validity of mental and physical stress models, Neuroscience and Biobehavioral Reviews 158 (2024) 105566.  doi:10.1016/j.neubiorev.2024.105566.
  * [5] G. Giannakakis, D. Grigoriadis, K. Giannakaki, O. Simantiraki, A. Roniotis, M. Tsiknakis, Review on psychological stress detection using biosignals, IEEE Transactions on Affective Computing PP (2019) 1–1.  doi:10.1109/TAFFC.2019.2927337.
  * [6] T. Zhang, M. R. Azghadi, C. Lammie, A. Amirsoleimani, R. Genov, Spike sorting algorithms and their efficient hardware implementation: a comprehensive survey, Journal of Neural Engineering 20 (2) (2023) 021001.  doi:10.1088/1741-2552/acc7cc.
URL https://dx.doi.org/10.1088/1741-2552/acc7cc

  * [7] Z. Halim, M. Rehan, On identification of driving-induced stress using electroencephalogram signals: A framework based on wearable safety-critical scheme and machine learning, Information Fusion 53 (2020) 66–79.  doi:10.1016/j.inffus.2019.06.006.
  * [8] H. Jasper, The ten-twenty electrode system of the international federation, Electroencephalography and Clinical Neurophysiology 10 (1958) 371–375.
  * [9] G. E. Chatrian, E. Lettich, P. L. Nelson, Modified nomenclature for the ”10%” electrode system, Journal of Clinical Neurophysiology: Official Publication of the American Electroencephalographic Society 5 (2) (1988) 183–186.
  * [10] C. S. Nayak, A. C. Anilkumar, EEG Normal Waveforms, in: StatPearls, StatPearls Publishing, Treasure Island (FL), 2023, p. 19.50.
URL http://www.ncbi.nlm.nih.gov/books/NBK539805/

  * [11] B. W. G. Priyanka A. Abhang, S. C. Mehrotra, Introduction to EEG- and Speech-Based Emotion Recognition, Elsevier, 2016.  doi:10.1016/c2015-0-01959-1.
  * [12] Physio-pedia, Brain gross anatomy, Brain - Anatomy (2 2024).
URL https://www.physio-pedia.com/File:Brain_gross_anatomy.png

  * [13] P. Siirtola, J. Röning, Comparison of regression and classification models for user-independent and personal stress detection, Sensors 20 (16) (2020) 4402.  doi:10.3390/s20164402.
  * [14] M. Gjoreski, M. Luštrek, M. Gams, H. Gjoreski, Monitoring stress with a wrist device using context, Journal of Biomedical Informatics 73 (2017) 159–170.  doi:10.1016/j.jbi.2017.08.006.
  * [15] C. Samson, A. Koh, Stress monitoring and recent advancements in wearable biosensors, Frontiers in Bioengineering and Biotechnology 8 (sep 2020).  doi:10.3389/fbioe.2020.01037.
  * [16] R. Katmah, F. Al-Shargie, U. Tariq, F. Babiloni, F. Al-Mughairbi, H. Al-Nashash, A review on mental stress assessment methods using EEG signals, Sensors 21 (15) (2021) 5043.  doi:10.3390/s21155043.
  * [17] G. Vanhollebeke, S. D. Smet, R. D. Raedt, C. Baeken, P. van Mierlo, M.-A. Vanderhasselt, The neural correlates of psychosocial stress: A systematic review and meta-analysis of spectral analysis EEG studies, Neurobiology of Stress 18 (2022) 100452.  doi:10.1016/j.ynstr.2022.100452.
  * [18] P. Boucher, P. Plusquellec, Acute stress assessment from excess cortisol secretion: Fundamentals and perspectives, Frontiers in Endocrinology 10 (nov 2019).  doi:10.3389/fendo.2019.00749.
  * [19] J. H. Lee, H. Gamper, I. Tashev, S. Dong, S. Ma, J. Remaley, J. D. Holbery, S. H. Yoon, Stress monitoring using multimodal bio-sensing headset, in: Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems, ACM, 2020, pp. 1–7.  doi:10.1145/3334480.3382891.
  * [20] C. Y. Park, N. Cha, S. Kang, A. Kim, A. H. Khandoker, L. Hadjileontiadis, A. Oh, Y. Jeong, U. Lee, K-EmoCon, a multimodal sensor dataset for continuous emotion recognition in naturalistic conversations, Scientific Data 7 (1) (sep 2020).  doi:10.1038/s41597-020-00630-y.
  * [21] V. Aspiotis, A. Miltiadous, K. Kalafatakis, K. D. Tzimourta, N. Giannakeas, M. G. Tsipouras, D. Peschos, E. Glavas, A. T. Tzallas, Assessing Electroencephalography as a Stress Indicator: A VR High-Altitude Scenario Monitored through EEG and ECG, Sensors 22 (15) (2022) 5792.  doi:10.3390/s22155792.
  * [22] A. Affanni, T. Aminosharieh Najafi, S. Guerci, Development of an EEG Headband for Stress Measurement on Driving Simulators, Sensors (Basel, Switzerland) 22 (5) (2022) 1785.  doi:10.3390/s22051785.
  * [23] G. Vos, K. Trinh, Z. Sarnyai, M. R. Azghadi, Generalizable machine learning for stress monitoring from wearable devices: A systematic literature review, International Journal of Medical Informatics 173 (2023) 105026.  doi:10.1016/j.ijmedinf.2023.105026.
  * [24] J. J. Newson, T. C. Thiagarajan, EEG frequency bands in psychiatric disorders: A review of resting state studies, Frontiers in Human Neuroscience 12 (jan 2019).  doi:10.3389/fnhum.2018.00521.
  * [25] C. M. S. Koelstra, J. L. M. Soleymani, A. Yazdani, Deap: A database for emotion analysis; using physiological signals, Affective Computing, IEEE Transactions on 3 (1) (2012) 18–31.
  * [26] G. Borghini, L. Astolfi, G. Vecchiato, D. Mattia, F. Babiloni, Measuring neurophysiological signals in aircraft pilots and car drivers for the assessment of mental workload, fatigue and drowsiness, Neuroscience & Biobehavioral Reviews 44 (2014) 58–75.  doi:10.1016/j.neubiorev.2012.10.003.
  * [27] X. Hou, Y. Liu, O. Sourina, Y. R. E. Tan, L. Wang, W. Mueller-Wittig, EEG based stress monitoring, in: 2015 IEEE International Conference on Systems, Man, and Cybernetics, IEEE, 2015, pp. 3110–3115.  doi:10.1109/smc.2015.540.
  * [28] S. López, G. Suarez, D. Jungreis, I. Obeid, J. Picone, Automated identification of abnormal adult EEGs, in: 2015 IEEE Signal Processing in Medicine and Biology Symposium (SPMB), 2015, pp. 1–5.  doi:10.1109/SPMB.2015.7405423.
  * [29] W.-L. Zheng, B.-L. Lu, Investigating critical frequency bands and channels for EEG-based emotion recognition with deep neural networks, IEEE Transactions on Autonomous Mental Development 7 (3) (2015) 162–175.  doi:10.1109/tamd.2015.2431497.
  * [30] G. Giannakakis, D. Grigoriadis, M. Tsiknakis, Detection of stress/anxiety state from EEG features during video watching, Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE Engineering in Medicine and Biology Society. Annual International Conference 2015 (2015) 6034–6037.  doi:10.1109/EMBC.2015.7319767.
  * [31] J. Minguillon, M. A. Lopez-Gordo, F. Pelayo, Stress Assessment by Prefrontal Relative Gamma, Frontiers in Computational Neuroscience 10 (2016) 101.  doi:10.3389/fncom.2016.00101.
  * [32] S. M. Umar Saeed, S. M. Anwar, M. Majid, M. Awais, M. Alnowami, Selection of Neural Oscillatory Features for Human Stress Classification with Single Channel EEG Headset, BioMed Research International 2018 (2018) 1049257\.  doi:10.1155/2018/1049257.
  * [33] A. Babayan, M. Erbey, D. Kumral, J. D. Reinelt, A. M. F. Reiter, J. Röbbig, H. L. Schaare, M. Uhlig, A. Anwander, P.-L. Bazin, A. Horstmann, L. Lampe, V. V. Nikulin, H. Okon-Singer, S. Preusser, A. Pampel, C. S. Rohr, J. Sacher, A. Thöne-Otto, S. Trapp, T. Nierhaus, D. Altmann, K. Arelin, M. Blöchl, E. Bongartz, P. Breig, E. Cesnaite, S. Chen, R. Cozatl, S. Czerwonatis, G. Dambrauskaite, M. Dreyer, J. Enders, M. Engelhardt, M. M. Fischer, N. Forschack, J. Golchert, L. Golz, C. A. Guran, S. Hedrich, N. Hentschel, D. I. Hoffmann, J. M. Huntenburg, R. Jost, A. Kosatschek, S. Kunzendorf, H. Lammers, M. E. Lauckner, K. Mahjoory, A. S. Kanaan, N. Mendes, R. Menger, E. Morino, K. Näthe, J. Neubauer, H. Noyan, S. Oligschläger, P. Panczyszyn-Trzewik, D. Poehlchen, N. Putzke, S. Roski, M.-C. Schaller, A. Schieferbein, B. Schlaak, R. Schmidt, K. J. Gorgolewski, H. M. Schmidt, A. Schrimpf, S. Stasch, M. Voss, A. Wiedemann, D. S. Margulies, M. Gaebler, A. Villringer, A mind-brain-body dataset of MRI, EEG, cognition, emotion, and peripheral physiology in young and old adults, Scientific Data 6 (1) (2019) 180308.  doi:10.1038/sdata.2018.308.
  * [34] A. Baghdadi, Y. Aribi, R. Fourati, N. Halouani, P. Siarry, A. M. Alimi, DASPS: A Database for Anxious States based on a Psychological Stimulation, Tech. rep., University of Sfax, Tunisia, arXiv:1901.02942 [cs] type: article (Aug. 2019).
  * [35] J. W. Ahn, Y. Ku, H. C. Kim, A Novel Wearable EEG and ECG Recording System for Stress Assessment, Sensors (Basel, Switzerland) 19 (9) (2019) 1991.  doi:10.3390/s19091991.
  * [36] A. Arsalan, M. Majid, A. R. Butt, S. M. Anwar, Classification of perceived mental stress using a commercially available EEG headband, IEEE Journal of Biomedical and Health Informatics 23 (6) (2019) 2257–2264.  doi:10.1109/jbhi.2019.2926407.
  * [37] S. M. U. Saeed, S. M. Anwar, H. Khalid, M. Majid, U. Bagci, EEG Based Classification of Long-Term Stress Using Psychological Labeling, Sensors (Basel, Switzerland) 20 (7) (2020) 1886.  doi:10.3390/s20071886.
  * [38] TuerxunWaili, Y. S. Alshebly, K. A. Sidek, M. G. M. Johar, Stress recognition using Electroencephalogram (EEG) signal, Journal of Physics: Conference Series 1502 (1) (2020) 012052.  doi:10.1088/1742-6596/1502/1/012052.
  * [39] M. Parent, I. Albuquerque, A. Tiwari, R. Cassani, J.-F. Gagnon, D. Lafond, S. Tremblay, T. H. Falk, PASS: A Multimodal Database of Physical Activity and Stress for Mobile Passive Body/ Brain-Computer Interface Research, Frontiers in Neuroscience 14 (2020) 542934.  doi:10.3389/fnins.2020.542934.
  * [40] R. Pernice, Y. Antonacci, M. Zanetti, A. Busacca, D. Marinazzo, L. Faes, G. Nollo, Multivariate Correlation Measures Reveal Structure and Strength of Brain-Body Physiological Networks at Rest and During Mental Stress, Frontiers in Neuroscience 14 (2020) 602584.  doi:10.3389/fnins.2020.602584.
  * [41] T. B. Alakus, M. Gonen, I. Turkoglu, Database for an emotion recognition system based on EEG signals and various computer games – GAMEEMO, Biomedical Signal Processing and Control 60 (2020) 101951.  doi:10.1016/j.bspc.2020.101951.
  * [42] X. Hu, G. Lodewijks, Detecting fatigue in car drivers and aircraft pilots by using non-invasive measures: The value of differentiation of sleepiness and mental fatigue, Journal of Safety Research 72 (2020) 173–187.  doi:10.1016/j.jsr.2019.12.015.
  * [43] M. K, P. S, A. K, R. D, V. Chinnadurai, V. S, R. K, S. Jayaraman, Dynamic cognitive workload assessment for fighter pilots in simulated fighter aircraft environment using EEG, Biomedical Signal Processing and Control 61 (2020) 102018.  doi:10.1016/j.bspc.2020.102018.
  * [44] V. Peterson, C. Galván, H. Hernández, R. Spies, A feasibility study of a complete low-cost consumer-grade brain-computer interface system, Heliyon 6 (3) (2020) e03425.  doi:10.1016/j.heliyon.2020.e03425.
  * [45] J. Zhang, Z. Yin, P. Chen, S. Nichele, Emotion recognition using multi-modal data and machine learning techniques: A tutorial and review, Information Fusion 59 (2020) 103–126.  doi:10.1016/j.inffus.2020.01.011.
  * [46] A. Sundaresan, B. Penchina, S. Cheong, V. Grace, A. Valero-Cabré, A. Martel, Evaluating deep learning EEG-based mental stress classification in adolescents with autism for breathing entrainment BCI, Brain Informatics 8 (1) (jul 2021).  doi:10.1186/s40708-021-00133-5.
  * [47] U. M. Al-Saggaf, S. F. Naqvi, M. Moinuddin, S. A. Alfakeh, S. S. A. Ali, Performance Evaluation of EEG Based Mental Stress Assessment Approaches for Wearable Devices, Frontiers in Neurorobotics 15 (2021) 819448\.  doi:10.3389/fnbot.2021.819448.
  * [48] A. Hag, D. Handayani, M. Altalhi, T. Pillai, T. Mantoro, M. H. Kit, F. Al-Shargie, Enhancing EEG-Based Mental Stress State Recognition Using an Improved Hybrid Feature Selection Algorithm, Sensors (Basel, Switzerland) 21 (24) (2021) 8370.  doi:10.3390/s21248370.
  * [49] A. Arsalan, M. Majid, Human stress classification during public speaking using physiological signals, Computers in Biology and Medicine 133 (2021) 104377.  doi:10.1016/j.compbiomed.2021.104377.
  * [50] J. Chae, S. Hwang, W. Seo, Y. Kang, Relationship between rework of engineering drawing tasks and stress level measured from physiological signals, Automation in Construction 124 (2021) 103560.  doi:10.1016/j.autcon.2021.103560.
  * [51] C. Diaz-Piedra, H. Rieiro, L. L. Di Stasi, Monitoring army drivers’ workload during off-road missions: An experimental controlled field study, Safety Science 134 (2021) 105092.  doi:10.1016/j.ssci.2020.105092.
  * [52] A. Sakalle, P. Tomar, H. Bhardwaj, D. Acharya, A. Bhardwaj, A LSTM based deep learning network for recognizing emotions using wireless brainwave driven system, Expert Systems with Applications 173 (2021) 114516.  doi:10.1016/j.eswa.2020.114516.
  * [53] C. Tan, M. Šarlija, N. Kasabov, NeuroSense: Short-term emotion recognition and understanding based on spiking neural network modelling of spatio-temporal EEG patterns, Neurocomputing 434 (2021) 137–148.  doi:10.1016/j.neucom.2020.12.098.
  * [54] Z. Wan, R. Yang, M. Huang, N. Zeng, X. Liu, A review on transfer learning in EEG signal analysis, Neurocomputing 421 (2021) 1–14.  doi:10.1016/j.neucom.2020.09.017.
  * [55] S. Yeom, H. Kim, T. Hong, Psychological and physiological effects of a green wall on occupants: A cross-over study in virtual reality, Building and Environment 204 (2021) 108134.  doi:10.1016/j.buildenv.2021.108134.
  * [56] X. Zhang, X. Yan, J. Stylli, M. L. Platt, Exploring the effects of EEG signals on collision cases happening in the process of young drivers’ braking, Transportation Research Part F: Traffic Psychology and Behaviour 80 (2021) 381–398.  doi:10.1016/j.trf.2021.05.010.
  * [57] H. Cai, Z. Yuan, Y. Gao, S. Sun, N. Li, F. Tian, H. Xiao, J. Li, Z. Yang, X. Li, Q. Zhao, Z. Liu, Z. Yao, M. Yang, H. Peng, J. Zhu, X. Zhang, G. Gao, F. Zheng, R. Li, Z. Guo, R. Ma, J. Yang, L. Zhang, X. Hu, Y. Li, B. Hu, A multi-modal open dataset for mental-disorder analysis, Scientific Data 9 (1) (2022) 178.  doi:10.1038/s41597-022-01211-x.
  * [58] M. Majid, A. Arsalan, S. M. Anwar, A Multimodal Perceived Stress Classification Framework using Wearable Physiological Sensors, Tech. rep., University of Engineering and Technology, Taxila, arXiv:2206.10846 [cs, eess] type: article (Jun. 2022).  doi:10.48550/arXiv.2206.10846.
  * [59] M. S. Sharif, M. R. T. Tamang, W. Elmedany, C. H. Fu, Evaluating the Stressful Commutes Using Physiological Signals and Machine Learning Techniques, in: 2022 International Conference on Innovation and Intelligence for Informatics, Computing, and Technologies (3ICT), 2022, pp. 175–180, iSSN: 2770-7466.  doi:10.1109/3ICT56508.2022.9990849.
  * [60] O. AlShorman, M. Masadeh, M. B. B. Heyat, F. Akhtar, H. Almahasneh, G. M. Ashraf, A. Alexiou, Frontal lobe real-time EEG analysis using machine learning techniques for mental stress detection, Journal of Integrative Neuroscience 21 (1) (2022) 020.  doi:10.31083/j.jin2101020.
  * [61] N. Phutela, D. Relan, G. Gabrani, P. Kumaraguru, M. Samuel, Stress classification using brain signals based on LSTM network, Computational Intelligence and Neuroscience 2022 (2022) 1–13.  doi:10.1155/2022/7607592.
  * [62] H. S. A. Hamatta, K. Banerjee, H. Anandaram, M. Shabbir Alam, C. A. Deva Durai, B. Parvathi Devi, H. Palivela, R. Rajagopal, A. Yeshitla, Genetic Algorithm-Based Human Mental Stress Detection and Alerting in Internet of Things, Computational Intelligence and Neuroscience 2022 (2022) 4086213.  doi:10.1155/2022/4086213.
  * [63] S. Sung, J.-W. Kwon, J.-E. Kim, Y.-J. Lee, S.-B. Lee, S.-K. Lee, S.-H. Moon, B. H. Lee, Real-Time Stress Analysis Affecting Nurse during Elective Spinal Surgery Using a Wearable Device, Brain Sciences 12 (7) (2022) 909.  doi:10.3390/brainsci12070909.
  * [64] J.-Y. Wu, C. T.-S. Ching, H.-M. D. Wang, L.-D. Liao, Emerging Wearable Biosensor Technologies for Stress Monitoring and Their Real-World Applications, Biosensors 12 (12) (2022) 1097.  doi:10.3390/bios12121097.
  * [65] S. Kumar G S, N. Sampathila, T. Tanmay, Wavelet based machine learning models for classification of human emotions using EEG signal, Measurement: Sensors 24 (2022) 100554.  doi:10.1016/j.measen.2022.100554.
  * [66] K. Rezaee, X. Yang, M. R. Khosravi, R. Zhang, W. Lin, G. Jeon, Fusion-based learning for stress recognition in smart home: An IoMT framework, Building and Environment 216 (2022) 108988.  doi:10.1016/j.buildenv.2022.108988.
  * [67] H. Mohammed, Q. Ali, Cheating Detection in E-exams System Using EEG Signals, International Conference on Scientific and Innovative Studies 1 (2023) 200–209.  doi:10.59287/icsis.601.
  * [68] E. Pérez Valero, Automated Detection of Alzheimer’s disease and other neurophysiological applications based on EEG, Universidad de Granada, 2023.
  * [69] Deep Learning Framework for Classification of Mental Stress from Multimodal Datasets - ProQuest (2023).
  * [70] S. Bhatnagar, S. Khandelwal, S. Jain, H. Vyawahare, A deep learning approach for assessing stress levels in patients using electroencephalogram signals, Decision Analytics Journal 7 (2023) 100211.  doi:10.1016/j.dajour.2023.100211.
  * [71] R. Suryawanshi, S. Vanjale, Brain Activity Monitoring for Stress Analysis through EEG Dataset using Machine Learning, International Journal of Intelligent Systems and Applications in Engineering 11 (1s) (2023) 236–240.
  * [72] C. L. Caldiroli, F. Gasparini, S. Corchs, A. Mangiatordi, R. Garbo, A. Antonietti, F. Mantovani, Comparing online cognitive load on mobile versus PC-based devices, Personal and Ubiquitous Computing 27 (2) (2023) 495–505.  doi:10.1007/s00779-022-01707-8.
  * [73] M. S. Sharif, M. Raj Theeng Tamang, C. H. Y. Fu, A. Baker, A. I. Alzahrani, N. Alalwan, An Innovative Random-Forest-Based Model to Assess the Health Impacts of Regular Commuting Using Non-Invasive Wearable Sensors, Sensors 23 (6) (2023) 3274.  doi:10.3390/s23063274.
  * [74] X. Lin, J. Chen, W. Ma, W. Tang, Y. Wang, EEG emotion recognition using improved graph neural network with channel selection, Computer Methods and Programs in Biomedicine 231 (2023) 107380.  doi:10.1016/j.cmpb.2023.107380.
  * [75] J. T. Teo, S. J. Johnstone, S. J. Thomas, Use of portable devices to measure brain and heart activity during relaxation and comparative conditions: Electroencephalogram, heart rate variability, and correlations with self-report psychological measures, International Journal of Psychophysiology 189 (2023) 1–10.  doi:10.1016/j.ijpsycho.2023.04.002.
  * [76] F. Cabitza, A. Campagner, The need to separate the wheat from the chaff in medical informatics, International Journal of Medical Informatics 153 (2021) 104510\.  doi:10.1016/j.ijmedinf.2021.104510.
  * [77] S.-H. Seo, J.-T. Lee, Stress and EEG, in: Convergence and Hybrid Information Technologies, InTech, 2010, p. 7.  doi:10.5772/9651.
  * [78] M. Barzegar, G. Jahromi, G. Meftahi, B. Hatef, The complexity of electroencephalographic signal decreases during the social stress, J Med Signals Sens 13 (1) (2023) 57–64.  doi:10.4103/jmss.jmss_131_21.
  * [79] J. H. D. M. Westerink, R. J. E. Rajae-Joordens, M. Ouwerkerk, M. van Dooren, S. Jelfs, A. J. M. Denissen, E. Penning de Vries, R. van Ee, Deriving a cortisol-related stress indicator from wearable skin conductance measurements: Quantitative model and experimental validation, Frontiers in Computer Science 2 (Sep. 2020).  doi:10.3389/fcomp.2020.00039.
  * [80] Emotiv, Emotiv (1 2022).
URL https://www.emotiv.com

  * [81] InteraXon, Muse (1 2022).
URL https://choosemuse.com

  * [82] NeuroSky, Neurosky (1 2022).
URL https://neurosky.com

  * [83] OpenBCI, Openbci (1 2022).
URL https://shop.openbci.com

  * [84] NEUROSPEC, Neurospec — research neurosciences (1 2022).
URL https://www.neurospec.com

  * [85] S. Bludau, S. Eickhoff, H. Mohlberg, S. Caspers, A. Laird, P. Fox, A. Schleicher, K. Zilles, K. Amunts, Cytoarchitecture, probability maps and functions of the human frontal pole, NeuroImage 93 (2013) 260–275.  doi:10.1016/j.neuroimage.2013.05.052.
  * [86] S.-Å. Christianson, Emotional stress and eyewitness memory: A critical review., Psychological Bulletin 112 (2) (1992) 284–309.  doi:10.1037/0033-2909.112.2.284.
  * [87] Empatica, Empatica — medical devices, ai and algorithms for remote patient monitoring (1 2022).
URL http://www.empatica.com

  * [88] A. A. T. Schuurmans, P. de Looff, K. S. Nijhof, C. Rosada, R. H. J. Scholte, A. Popma, R. Otten, Validity of the empatica e4 wristband to measure heart rate variability (hrv) parameters: a comparison to electrocardiography (ecg), Journal of Medical Systems 44 (11) (Sep. 2020).  doi:10.1007/s10916-020-01648-w.
  * [89] J. R. Stroop, Studies of interference in serial verbal reactions., Journal of Experimental Psychology 18 (6) (1935) 643–662.  doi:10.1037/h0054651.
  * [90] F. A. Hamid, M. N. M. Saad, A. S. Malik, Characterization stress reactions to stroop color-word test using spectral analysis, Materials Today: Proceedings 16 (2019) 1949–1958.  doi:10.1016/j.matpr.2019.06.073.
  * [91] S. Cohen, T. Kamarck, R. Mermelstein, Perceived stress scale (1983).  doi:10.1037/t02889-000.
  * [92] C. D. Spielberger, State-trait anxiety inventory for adults (1983).  doi:10.1037/t06496-000.
  * [93] MATLAB, Matlab (1 2022).
URL https://www.mathworks.com

  * [94] E. D. Format, European data format (1 2022).
URL https://www.edfplus.info

  * [95] N. Subramaniyam, Pitfalls of filtering the eeg signal - sapien labs: Neuroscience: Human brain diversity project, Sapien Labs (2023) 26.
  * [96] A. Safayari, H. Bolhasani, Depression diagnosis by deep learning using EEG signals: A systematic review, Medicine in Novel Technology and Devices 12 (2021) 100102.  doi:10.1016/j.medntd.2021.100102.
  * [97] V. J. Lawhern, A. J. Solon, N. R. Waytowich, S. M. Gordon, C. P. Hung, B. J. Lance, EEGNet: a compact convolutional neural network for EEG-based brain–computer interfaces, Journal of Neural Engineering 15 (5) (2018) 056013.  doi:10.1088/1741-2552/aace8c.
  * [98] H. Hinrichs, M. Scholz, A. K. Baum, J. W. Y. Kam, R. T. Knight, H.-J. Heinze, Comparison between a wireless dry electrode eeg system with a conventional wired wet electrode eeg system for clinical applications, Scientific Reports 10 (1) (Mar. 2020).  doi:10.1038/s41598-020-62154-0.
  * [99] W. A. Campos-Ugaz, J. P. P. Garay, O. Rivera-Lozada, M. A. A. Diaz, D. Fuster-Guillén, A. A. T. Arana, An overview of bipolar disorder diagnosis using machine learning approaches: Clinical opportunities and challenges, Iranian Journal of Psychiatry (apr 2023).  doi:10.18502/ijps.v18i2.12372.
  * [100] M. Chaumon, A. Puce, N. George, Statistical power: Implications for planning MEG studies, NeuroImage 233 (2021) 117894.  doi:10.1016/j.neuroimage.2021.117894.
  * [101] M. A. Boudewyn, S. J. Luck, J. L. Farrens, E. S. Kappenman, How many trials does it take to get a significant ERP effect? it depends, Psychophysiology 55 (6) (2017) e13049.  doi:10.1111/psyp.13049.
  * [102] D. Soni, Supervised vs. unsupervised learning, medium (2 2020).
  * [103] M. M. Bradley, P. J. Lang, Measuring emotion: The self-assessment manikin and the semantic differential, Journal of Behavior Therapy and Experimental Psychiatry 25 (1) (1994) 49–59.  doi:https://doi.org/10.1016/0005-7916(94)90063-9.
  * [104] S. J. Luck, N. Gaspelin, How to get statistically significant effects in any ERP experiment (and why you shouldn't), Psychophysiology 54 (1) (2016) 146–157.  doi:10.1111/psyp.12639.