:
ML4H Extended Abstract Arxiv Index 2021 2021 Machine Learning for Health
(ML4H) 2021

# Machine Learning for Real-Time, Automatic, and Early Diagnosis of
Parkinson’s Disease by Extracting Signs of Micrographia from Handwriting
Images

Riya Tyagi Authors contributed equally Tanish Tyagi 11footnotemark: 1 Ming
Wang  Lijun Zhang
<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract

Parkinson’s disease (PD) is debilitating, progressive, and clinically marked
by motor symptoms. As the second most common neurodegenerative disease in the
world, it affects over 10 million lives globally. Existing diagnoses methods
have limitations, such as the expense of visiting doctors and the challenge of
automated early detection, considering that behavioral differences in patients
and healthy individuals are often indistinguishable in the early stages.
However, micrographia, a handwriting disorder that leads to abnormally small
handwriting, tremors, dystonia, and slow movement in the hands and fingers, is
commonly observed in the early stages of PD. In this work, we apply machine
learning techniques to extract signs of micrographia from drawing samples
gathered from two open-source datasets and achieve a predictive accuracy of
94%. This work also sets the foundations for a publicly available and user-
friendly web portal that anyone with access to a pen, printer, and phone can
use for early PD detection.

###### keywords:

Parkinson’s Disease, Machine Learning, Micrographia, Early Detection,
Predictive accuracy

##  1 Introduction

Parkinson’s disease (PD) is a neurodegenerative, chronic, and progressive
nervous system disorder that affects movement. PD affects more than 10 million
people worldwide, and this number is expected to double by 2030. (Choi et al.,
2017) Symptoms include tremors, slowness of movement, stiffness, and changes
in writing skills. PD occurs when dopamine-producing nerve cells die, a
process that can take almost 10 years to reach its late and most severe
stages. A lack of dopamine results in further critical symptoms, such as
depression, anxiety, sleep disturbances, and dementia.

Parkinson’s disease cannot be treated, although specific drugs and medications
can assist with early-stage symptoms, potentially preventing the progression
to more severe stages. Early detection and prognosis of PD are crucial for
assisting patients to retain a good quality of life. However, diagnosing
Parkinson’s disease in its early stages is a very challenging task, and there
are currently no specific tests designed or approved to diagnose PD (United
States Government, 2021). Diagnosis is based on a thorough examination
performed by a trained medical official, including a neurological examination,
medical history evaluation, blood and laboratory tests, and brain scans. Even
with these procedures, PD is misdiagnosed up to 30% of the time because of the
many PD mimics, such as Essential Tremors and Drug-Induced Parkinson’s.
Additionally, up to 20% of PD patients are undiagnosed (Schrag et al., 2002).
Receiving a PD diagnosis is expensive, time-consuming, and non-accessible, as
reported by 21% of patients who had to visit their general provider thrice
before receiving a specialist referral for their condition. (Gavidia, 2020)

Tools that make the diagnosis process accurate, accessible, automatic, real-
time, early, and free of cost can have a tremendous impact on Parkinson’s
patients. In this study, we apply machine learning strategies to predict PD
based upon drawing samples that exhibit signs of micrographia.

##  2 Related Works

Prior works have incorporated voice recordings, electroencephalogram (EEG)
signals, and smart devices to create models that diagnose PD. Zhao et al.
(2014) hypothesized that patients with PD exhibit deficits in the production
of emotional speech. To test this conjecture, voice recordings from five
patients and seven healthy individuals were used to detect Parkinson’s
disease. Naive Bayes, Random Forests, and Support Vector Machines were applied
for classification, achieving 65.5% and 73.33% accuracies on classifying PD
and control. Oh et al. (2020) utilized EEG signals and a Convolutional Neural
Network (CNN) to detect PD by assessing whether the EEG signals depicted brain
abnormalities, achieving 88.25% accuracy. Drotár et al. (2014) employed the
use of a digitizing tablet to assess both in-air and on-surface kinematic
variables during handwriting of a sentence in 37 PD patients on medication and
38 age and gender-matched healthy controls. Using Support Vector Machines, an
accuracy of 85.61% was achieved.

These studies require patients to visit a neurologist to access a smart device
and undergo various scans like EEG and PET, making the process expensive,
time-consuming, and difficult to access. In this research, we aim to make the
PD diagnosis more accurate, efficient, and accessible by using images of
handwriting samples to automate the diagnosis process.

##  3 Dataset and Preprocessing

### Dataset

We formed our dataset by combining the New HandPD and Old HandPD dataset
(Pereira et al., 2016). It consisted of 158 individuals, split into 53 healthy
individuals and 105 PD patients. Each patient drew 8 images: 4 spirals and 4
meanders. With these images, we constructed 2 datasets, one with extracted
numerical features and the other with images. The process of assembling the
image dataset and its uses are discussed further in Appendix A.

tab:table1  Mean Age (SD) Left Handed Right Handed % Female Counts Healthy
44.11 (15.46) 7 46 54.7% 53 PD 58.75 (7.51) 5 69 20.3% 74

Table 1: Demographics of Data

### Preprocessing

We mimicked the feature extraction process used by (Pereira et al., 2016) to
extract 9 numerical features from the images. First, we extracted the
handwritten trace (HT) and exam trace (ET) were extracted from each image. The
HT is the outline the patient drew and the ET is the outline the patient tried
to replicate. Appendix A show images of the extracted HT and ET. Using the
traces, the below features were computed:

  1. F1:

The root mean square (RMS) of the differences between the HT and ET radii. The
radius of the HT or ET can be defined as the length of the straight line that
connects an arbitrary point to the center of the HT or ET.

$RMS=\sqrt{\frac{1}{N}*\sum_{\textsuperscript{i=1}}^{N}(r_{\textsuperscript{HT}}^{i}-r_{\textsuperscript{ET}}^{i})^{2}}$

$N$ is the number of sample points on the HT and ET, and
$r_{\textsuperscript{HT}}^{i}$ and $r_{\textsuperscript{ET}}^{i}$ denote the
HT and ET radii for the $i$th point.

  2. F2:

The maximum difference between the HT and ET radii.

$\Delta max=max(|r_{\textsuperscript{HT}}^{i}-r_{\textsuperscript{ET}}^{i}|)$

  3. F3:

The minimum difference between the HT and ET radii.

$\Delta min=min(|r_{\textsuperscript{HT}}^{i}-r_{\textsuperscript{ET}}^{i}|)$

  4. F4:

The standard deviation of the differences between the HT and ET radii.

$s=\sqrt{\frac{1}{N-1}\sum_{i=1}^{N}(r_{\textsuperscript{HT}}^{i}-r_{\textsuperscript{ET}}^{i})^{2}}$

  5. F5:

The Mean Relative Tremor (MRT) of a given individual’s HT. This is defined as
the mean difference between the radius of a given sample and its D left-
nearest neighbors. To find D, D = 1, 3, 5, 7, 10, 15, 20 were tested, with D =
10 maximizing the PD detection rate.

$MRT=\frac{1}{N-D}\sum_{\textsuperscript{i=D}}^{N}(|r_{\textsuperscript{HT}}^{\textsuperscript{i-D+1}}-r_{\textsuperscript{ET}}^{i}|)$

Figures 6-8 are computed utilizing the equation for relative tremor:
$|r_{\textsuperscript{ET}}^{i}-r_{\textsuperscript{HT}}^{\textsuperscript{i-D+1}}|$

  6. F6:

Maximum ET

  7. F7:

Minimum ET

  8. F8:

Standard Deviation of ET values

  9. F9:

The number of times the difference between HT and ET radii changes from
positive to negative.

The features were then normalized as follows:

$f[i]^{\prime}=\sum_{\textsuperscript{i=1}}^{9}\frac{f[i]-avg(i)}{std(i)}$

f[i]’ represents the normalized equivalent of feature f[i] for features 1-9.

Two other features we include in our predictions are a patient’s gender and
age, considering that Parkinson’s disease mainly impacts individuals above age
50 and is 1.5 times more common in males (Reekes et al., 2020).

Additionally, we noticed there was a large class imbalance in our dataset,
with 53 healthy and 105 PD patients. After performing experiments with our
machine learning models, we discovered that this imbalance led to overfitting
and undesirable performances, even after weighting the classes. To cope with
this, we truncated our original dataset by removing the 31 PD patients from
the New HandPD dataset, which greatly boosted our model’s ability to learn
useful patterns. Our final dataset of 127 patients, 53 healthy and 74 PD, was
randomly split between train (76.5%), validation (8.5%), and hold-out test
(15%) sets. Table LABEL:tab:table1 shows the demographics for our cohort of
individuals.

tab:table2  Model ACC AUC FP FN Sensitivity Specificity PPV NPV Threshold SVM
0.92 0.93 12 0 1.0 0.81 0.89 1.0 0.65 Logistic Regression 0.93 0.96 3 7 0.92
0.95 0.90 0.96 0.62

Table 2: Model Performance

fig:fig1

Figure 1: Regularized Logistic Regression (left) and Support Vector Machines
(right) ROC Curves

##  4 Methods

### (1) Support Vector Machines

In model 1, we utilized a Support Vector Machine (SVM) (Hearst et al., 1998)
to diagnose PD given the nine extracted numerical features, age, and gender.
We used 10-fold Cross-Validation (CV) (Stone, 1974) on the validation set to
maximize accuracy by choosing an optimal kernel (Radial Base, Linear, or
Polynomial), class weight (None or Balanced), probability threshold, and
lambda. After training and validating the model on the image level, we tested
it on the patient level, extracting all 8 images from 11 PD and 8 healthy
patients (15% test set) and selecting an optimal aggregation scheme for
patient-level predictions.

### (2) Regularized Logistic Regression

In model 2, we employed Regularized Logistic Regression (Tibshirani, 1996)
strategies, regressing the PD diagnosis label against the nine extracted
numerical features, age, and gender. With 10-fold CV, we iterated over
different lambda values, probability thresholds, and loss and solver functions
to maximize accuracy for our image-level predictions. We tested the model on
the patient level, extracting all 8 images from 11 PD and 8 healthy patients
(15% test set) and determining an optimal aggregation scheme for patient-level
predictions.

##  5 Results

To maximize the performance of both models, we utilized a probability
threshold. First, we trained, validated, and tested the models on the image
level. All performance metrics in Table LABEL:tab:table2 are based on image
level class assignments from the held-out test set.

However, our web portal will consider 6 - 8 different drawings per patient in
the diagnosis process. To mimic this functionality, we tested each model on
the patient level with an aggregation scheme. We chose an optimal scheme from
three options: applying a PD label to a patient if the model diagnosed 2/8
images as signs of PD, if the averaged probability score from all 8 images was
greater than 0.5, or if the model predicted more than half the images to
belong to a PD patient. The last scheme performed best because the model
mainly predicted only a few outlier images incorrectly (if any images were
predicted wrong) per patient.

Both models received the same patient-level accuracy: 0.9444. Additionally, we
compared their Receiver Operating Characteristic (ROC) curves (Figure
LABEL:fig:fig1) when tested on 10 random selections of 19 patients (11 PD and
8 Control). Both models performed similarly on each test set, although
Regularized Logistic Regression was more consistent. Optimal hyperparameters
for each model are listed below.

Support Vector Machines:

  * •

kernel = ‘rbf’, transforms radial data so it can be separated with a linear
separation boundary

  * •

class_weight = ‘balanced’, adds weight to each class depending on the class
balance

  * •

C = 100, lambda value

Regularized Logistic Regression:

  * •

penalty = ‘elasticnet’, a combination of L1 and L2 Regularization

  * •

l1_ratio = 0.75, uses 75% L1 Regularization, 25% L2 Regularization

  * •

solver = ‘saga’, only solver which supports ‘elasticnet’ penalty, the best-
performer

  * •

C = 0.1, lambda value

  * •

class_weight = ‘balanced’, adds weight to each class depending on the class
balance

  * •

fit_intercept = False, bias added to the decision function

##  6 Conclusion and Future Works

In this work, we applied classification algorithms to handwriting samples to
diagnose Parkinson’s disease in its early stages. Our dataset was relatively
small in size, which we will address in the future by gathering data through
our web portal. Our mockups and plans for the website are detailed in Appendix
A. Additionally, we devised a handwriting assessment based on precedents and
open-source datasets in this research area, which is available in Appendix B.
We strive to boost the accessibility of our handwriting assessment by online
deployment through a web portal. Ideally, a patient with a touch-screen device
will be able to take a quick, easy, online assessment. Once the web portal is
complete, we hope to perform clinical studies with our partners from Penn
State University. In these studies, we will evaluate the handwriting
assessment and develop a handwriting dataset that considers temporal stages of
Parkinson’s disease by analyzing handwriting and drawing samples from
different stages of PD.

The next iteration of our work with machine learning and deep learning models
will be to design a Convolutional Neural Network (CNN) for analysis of the
drawing images. We have already designed the feature extraction process,
pictured in Appendix C. The CNN will further improve prediction accuracy and
work toward automatic and real-time detection of Parkinson’s disease.

## References

  * Choi et al. (2017) Hongyoon Choi, Seunggyun Ha, Hyung Jun Im, Sun Ha Paek, and Dong Soo Lee.  Refining diagnosis of parkinson’s disease with deep learning-based interpretation of dopamine transporter imaging.  _NeuroImage: Clinical_ , 16:586–594, 2017.
  * Drotár et al. (2014) Peter Drotár, Jiří Mekyska, Irena Rektorová, Lucia Masarová, Zdenek Smékal, and Marcos Faundez-Zanuy.  Analysis of in-air movement in handwriting: A novel marker for parkinson’s disease.  _Computer methods and programs in biomedicine_ , 117(3):405–411, 2014.
  * Gavidia (2020) Matthew Gavidia.  Poll finds 1 in 4 people with parkinson disease misdiagnosed, Jul 2020\.  URL https://www.ajmc.com/view/poll-finds-1-in-4-people-with-parkinson-disease-misdiagnosed.
  * Hearst et al. (1998) M.A. Hearst, S.T. Dumais, E. Osuna, J. Platt, and B. Scholkopf.  Support vector machines.  _IEEE Intelligent Systems and their Applications_ , 13(4):18–28, 1998.  10.1109/5254.708428.
  * Oh et al. (2020) Shu Lih Oh, Yuki Hagiwara, U Raghavendra, Rajamanickam Yuvaraj, N Arunkumar, M Murugappan, and U Rajendra Acharya.  A deep learning approach for parkinson’s disease diagnosis from eeg signals.  _Neural Computing and Applications_ , 32(15):10927–10933, 2020.
  * Pereira et al. (2016) Clayton R Pereira, Danilo R Pereira, Francisco A Silva, João P Masieiro, Silke AT Weber, Christian Hook, and João P Papa.  A new computer vision-based approach to aid the diagnosis of parkinson’s disease.  _Computer Methods and Programs in Biomedicine_ , 136:79–88, 2016.
  * Reekes et al. (2020) Tyler Harrison Reekes, Christopher Ian Higginson, Christina Raye Ledbetter, Niroshan Sathivadivel, Richard Matthew Zweig, and Elizabeth Ann Disbrow.  Sex specific cognitive differences in parkinson disease.  _npj Parkinson’s Disease_ , 6(1):1–6, 2020.
  * Schrag et al. (2002) A Schrag, Y Ben-Shlomo, and N Quinn.  How valid is the clinical diagnosis of parkinson’s disease in the community?, Nov 2002.  URL https://jnnp.bmj.com/content/73/5/529.
  * Stone (1974) Mervyn Stone.  Cross-validatory choice and assessment of statistical predictions.  _Journal of the royal statistical society: Series B (Methodological)_ , 36(2):111–133, 1974.
  * Tibshirani (1996) Robert Tibshirani.  Regression shrinkage and selection via the lasso.  _Journal of the Royal Statistical Society: Series B (Methodological)_ , 58(1):267–288, 1996.
  * United States Government (2021) Health United States Government.  Parkinson’s disease: Hope through research, 2021.  URL https://www.ninds.nih.gov/Disorders/Patient-Caregiver-Education/Hope-Through-Research/Parkinsons-Disease-Hope-Through-Research#diagnosed.
  * Zhao et al. (2014) Shunan Zhao, Frank Rudzicz, Leonardo G Carvalho, César Márquez-Chin, and Steven Livingstone.  Automatic detection of expressed emotion in parkinson’s disease.  In _2014 IEEE international conference on acoustics, speech and signal processing (ICASSP)_ , pages 4813–4817. IEEE, 2014.

##  Appendix A Pictures and Use Cases of PD Diagnosis Web Portal Interface

There are two use cases of the web portal: submitting handwriting assessments
and viewing assessment history. The process to perform these actions is
detailed below.

Submitting Handwriting Assessments

  1. 1.

The user logs into their account on the portal (or creates an account if she
does not have one) and navigates to the “Submit A Test” page.

  2. 2.

The user prints out the handwriting assessment .pdf available on the page.

  3. 3.

The user fills out the exam, scans it, and uploads a photo to the page.

  4. 4.

After a few moments, the user receives a PD diagnosis, with their % chance of
having PD. The exam and its results are saved in the “Exam History” page.

Figure 2: ”Submit A Test” page UI

Viewing Assessment History

  1. 1.

The user logs into their account on the portal and navigates to the “Exam
History” page.

  2. 2.

The user can now view information about all their past exams, including the
date, a picture of their assessment, age and gender information submitted
alongside the assessment, and results.

  3. 3.

The user views their previous assessment submissions by clicking on the exam
icons available next to each exam number on the page.

Figure 3: ”Exam History” page UI

##  Appendix B Handwriting Assessment

Figure 4: Handwriting Assessment Developed for the Web Portal

##  Appendix C Image Feature Extraction Procedure

To develop useful image features to input into a Convolutional Neural Network
(CNN) model, we extracted an exam trace (ET) and a handwriting trace (HT) from
all our images and combined them. To do so, we followed the process below.

Exam Trace:

  1. 1.

Mean Blur (5 x 5 kernel)

  2. 2.

Binary Threshold

I represents each pixel (x, y) in an image, with three color channels: red,
green, and blue, written as I(r), I(g), and I(b)

$I((r,g,b))=\begin{cases}0&\text{if }I(r)<90\land I(g)<90\land I(b)<90\\\
1&\text{otherwise}\end{cases}$

  3. 3.

Dilate (4 x 4 kernel)

Figure 5: Exam Trace Extraction Process

Handwriting Trace:

  1. 1.

Median Blur (5 x 5 kernel)

  2. 2.

Binary Threshold

Consider I to represent a pixel in an image with three color channels: red,
green, and blue, written as I(r), I(g), and I(b)

$I((r,g,b))=\begin{cases}0&\text{if }I(r)<200\land I(g)<200\land I(b)<200\\\
1&\text{otherwise}\end{cases}$

  3. 3.

Difference Image (with extracted ET)

  4. 4.

Invert and Dilate Image

Figure 6: Handwriting Trace Extraction Process

Next, we combined the extracted exam trace and handwriting trace, producing an
image with minimal features that has the potential to form a strong CNN model.

Figure 7: Blending HT and ET to Form the Final Image