# Data-driven deep learning algorithms for time-varying infection rates of
COVID-19 and mitigation measures

K.D. Olumoyin<EMAIL_ADDRESS>A.Q.M. Khaliq  K.M. Furati  Department
of Mathematical Sciences, Middle Tennessee State University, Murfreesboro, TN
37132, USA  Department of Mathematics & Statistics, King Fahd University of
Petroleum and Minerals, Dhahran 31261, Saudi Arabia

###### Abstract

Epidemiology models with constant parameters may not capture the infection
patterns in the presence of pharmaceutical and non-pharmaceutical mitigation
measures during a pandemic, since infectiousness is a function of time. In
this paper, Epidemiology Informed Neural Network (EINN) algorithms are
introduced to discover time-varying infection rates for the COVID-19 pandemic.
Since there are asymptomatic infectives, mostly unreported, EINN learns the
probability that an infective individual is asymptomatic. Using cumulative and
daily reported cases of infectives, we simulate the impact of non-
pharmaceutical mitigation measures such as early detection of infectives,
contact tracing, and social distancing on the basic reproduction number. We
demonstrate the effectiveness of vaccination, a pharmaceutical mitigation
measure, together with non-pharmaceutical mitigation measures on the daily
reported infectives. The EINN algorithms discover time-varying infection and
recovery rates. The Mean Squared Error is used to demonstrate the accuracy of
the proposed EINN algorithms. Simulations are presented for Italy, South
Korea, United Kingdom, and the United States.

###### keywords:

data-driven algorithms , deep learning , Epidemiology Informed Neural Network
, time-varying , COVID-19 , asymptomatic infectives

††journal: Journal of elsevier

##  1 Introduction

COVID-19 data from different countries reflects various mitigation measures
[1, 2], such as social distancing, early detection of infectives, contact
tracing, and vaccination [3, 4, 5]. Time-varying infection rates have been
suggested to efficiently model the spread of an infectious disease such as
COVID-19. For example, nonlinear time-dependent infection rate functions were
introduced in [6, 7]. Given that it may not be possible to know the most
accurate form of a time-varying infection rate, we propose a data-driven
approach that learns time-varying infection and recovery rates.

A widely used threshold parameter for the spread or extinction of an
infectious disease in an epidemiology model is the basic reproduction number
[8]. It is defined as the average number of persons an infected person can
infect. When the basic reproduction number is less than one, the infectious
disease vanishes. In the SIR model [9], the basic reproduction number is
computed as the ratio of the infection rate to the recovery rate. In this
paper, we adopt the basic reproduction number for the asymptomatic-SIR model
introduced in [10]. We denote the basic reproduction number as BRN when the
model parameters are constant and when one or more of the model parameters are
time-varying, we also have a time-varying reproduction rate, which is denoted
as $R_{t}$. This time-varying reproduction rate, $R_{t}$, demonstrates the
spread pattern of COVID-19 throughout the duration of the pandemic.

There is an asymptomatic period for every infective individual, some papers
have reported a range of 7 to 14 days [11]. There are also asymptomatic
infectives that never show symptoms but are infectious [10]. Early study of
the spread of COVID-19 shows that some of the infectives are asymptomatic
infectives [12, 13] and they are mostly unreported in the publicly available
data [10]. An asymptomatic-SIR model is a modification to the traditional SIR
infectious disease model [9], that includes asymptomatic infectives and
asymptomatic recovered compartments, such was introduced in [10]. We introduce
modifications to the asymptomatic-SIR model by including a vaccination
compartment determined by a vaccination rate, time-varying infection and
recovery rates, and a probability term that determines the portion of the
total infectives that are asymptomatic infectives.

In March and April 2020, many countries instituted widespread lockdown [14]. A
model describing lockdown and lockdown relaxation was presented [15]. The
purpose of a lockdown is to limit human contacts and potentially lower
infection rates. Unfortunately, the impact of a comprehensive lockdown on the
economy is severe and thus other measures are more desirable.

Neural networks are universal approximators of continuous functions [16, 17].
In [18, 19], a feedforward neural network (FNN) is developed to learn
approximate solutions of differential equations by constraining the residual.
This FNN is called Physics Informed Neural Network (PINN). The authors in [20]
developed a deep learning solution of a COVID-19 model using an FNN. PINN has
been used successfully to solve ordinary differential equations [21] and
partial differential equations [18]. In [22], a PINN with a modified loss
function is used to learn the diffusivity in a nonlinear diffusion equation. A
Biologically Informed Neural Network (BINN) is proposed in [23] to learn the
nonlinear diffusion and reaction terms in a 1-D spatial dimensional reaction-
diffusion model. BINN is an extension to PINN for the discovery of underlying
dynamics of biological systems from sparse experimental data.

The Epidemiology Informed Neural Network (EINN) algorithms presented in this
paper are extensions to PINN for epidemiology models. The loss function is
extended to include time-varying rates using some known epidemiology facts
about the infectious disease. Interpolation is used to generate sufficient
training data. The proposed EINN algorithms are able to capture the dynamics
of the spread of the disease and the influence of the mitigation measure. The
asymptomatic infectives population is usually unreported in the publicly
available data [24]. EINN algorithms can learn the asymptomatic infectives
population by training on symptomatic infectives data that are available in
the reported public data.

The paper is organized as follows. In Section 2, we introduce and discuss the
asymptomatic-SIR model. The definition of the basic reproduction number (BRN)
and the time-varying reproduction rate $R_{t}$ are also presented. In Section
3, we describe the FNN. The architecture and loss function of EINN is
presented. In Section 4, the EINN algorithm for constant infection and
recovery rates is presented and applied to data from Italy, South Korea, and
the USA. In Section 5, using the EINN algorithm introduced in Section 4, we
simulate the impact of non-pharmaceutical mitigation measures including social
distancing, contact tracing, and early detection of infectives on the BRN and
the spread of COVID-19. The effectiveness of vaccination in reducing the daily
reported infective cases, a pharmaceutical mitigation measure, is discussed in
Section 6. In Section 7 and 8, EINN algorithms are presented for the discovery
of time-varying infection and recovery rates subject to non-pharmaceutical
mitigation measures. Finally, a summary of the results in this paper is
presented in Section 9.

##  2 Asymptomatic-SIR Model

The asymptomatic-SIR model introduced in [10] assumes that some of the
infectives are asymptomatic infectives. This group is infectious despite not
showing symptoms of COVID-19, probably do not get tested, and are usually
unreported in the various publicly available data. In Figure 1, the
interactions between the six different compartments of the asymptomatic-SIR
model are shown. The dynamics of these interactions can be represented as a
system of ordinary differential equations with time-varying parameters in eq.
(1).

Figure 1: Compartments in an Asymptomatic-SIR model with vaccination

$\begin{split}\frac{dS(t)}{dt}&=-\frac{1}{N}\beta(t)\Big{(}I(t)+J(t)\Big{)}S(t)-\kappa
S(t)\\\
\frac{dI(t)}{dt}&=\frac{1}{N}\beta(t)\xi\Big{(}I(t)+J(t)\Big{)}S(t)-\gamma(t)I(t)\\\
\frac{dJ(t)}{dt}&=\frac{1}{N}\beta(t)\Big{(}1-\xi\Big{)}\Big{(}I(t)+J(t)\Big{)}S(t)-\mu(t)J(t)\\\
\frac{dR(t)}{dt}&=\gamma(t)I(t)+\kappa\xi S(t)\\\
\frac{dU(t)}{dt}&=\mu(t)J(t)+\kappa(1-\xi)S(t).\end{split}$ (1)

The continuity equation is given by

$\begin{split}N=S+I+J+R+U.\end{split}$ (2)

The initial conditions are given by $S(t_{0})=S_{0}$, $I(t_{0})=I_{0}$,
$J(t_{0})=J_{0}$, $R(t_{0})=R_{0}$, and $U(t_{0})=U_{0}$. $t\geq t_{0}$
represent time in days and $t_{0}$ is the start date of the pandemic in the
model. $\beta(t)$, $\gamma(t)$, and $\mu(t)$ are time-varying parameters.
$\gamma(t)$ represents the time-varying recovery rate for the symptomatic
infectives. $N$ represents the total population eq. (2). It is assumed that
$N$ does not change throughout the pandemic and that infective individuals are
immediately infectious. We show the interactions between the compartments of
the model in Figure 1. The time-varying recovery rate of the asymptomatic
infectives is denoted by $\mu(t)$. $\beta(t)$ is the time-varying infection
rate, it usually depends on the infection vector. In the COVID-19 pandemic,
$\beta(t)$ depends also on contacts between individuals.

The asymptomatic-SIR model, eq. (1) considers six population compartments: the
Susceptible $S$, the symptomatic Infectives $I$ which correspond to the
reported infectives in the publicly available data, the asymptomatic
Infectives $J$ which correspond to the unreported infectives. The total
infectives are $I+J$. The rest of the compartments are the symptomatic
Recovered $R$, the asymptomatic Recovered $U$, and the vaccinated compartment
$\kappa S$, which is a loss from the susceptible group. $\kappa$ is the
average percentage of individuals that are vaccinated daily. $\xi(I+J)$
corresponds to the portion of the total infectives that are symptomatic and
reported. That is, $\xi$ represents the probability that an infective
individual is reported. On the other hand, $(1-\xi)(I+J)$ represents the
asymptomatic infectives. And so, $(1-\xi)$ is the probability that an
infective is an asymptomatic infective.

###### Definition 1

[10] The basic reproduction number (BRN) for the asymptomatic-SIR model eq.
(1) with constant parameters is given by

$BRN=\frac{\beta}{\xi\gamma+(1-\xi)\mu},\quad\xi\in(0,1).$ (3)

If $\xi=0$, $BRN=\beta/\mu$, when all the infective population are
asymptomatic.
If $\xi=1$, $BRN=\beta/\gamma$, when all the infective population are
symptomatic.
It is assumed that during the early period of the pandemic,
$\xi=\frac{I(0)}{I(0)+J(0)}$.

###### Definition 2

The time-varying reproduction rate $R_{t}$ for the asymptomatic-SIR model eq.
(1) with time-varying infection rate is given by

$R_{t}=\frac{\gamma\beta(t)}{\left[\gamma\left(\frac{I(t)}{I(t)+J(t)}\right)+\mu\left(1-\frac{I(t)}{I(t)+J(t)}\right)\right]^{2}}.$
(4)

Where $\gamma$ and $\mu$ may be constant or time-varying.

##  3 Neural network structure

###  3.1 Feedforward Neural Network (FNN)

We write an FNN as a function of $L$ layers, $t$ input and an output
$\mathcal{N}$

$\mathcal{N}(t;\theta)=\sigma(W_{L}\sigma(\ldots\sigma(W_{2}\sigma(W_{1}t+b_{1})+b_{2})\ldots)+b_{L}),$
(5)

where $\theta:=(W_{1},\ldots,W_{L},b_{1},\ldots,b_{L})$. $W_{k}$,
$k=1,\ldots,L$ is the set of the neural network weight matrices while $b_{k}$,
$k=1,\ldots,L$ is the set of the bias vectors. $\sigma$ is the activation
function. Given a collection of sample pairs $(t_{j},u_{j})$ $j=1,\dots M$,
where $u$ is some target function. The goal is to find $\theta^{*}$ by solving
the optimization problem

$\theta^{*}=\operatorname*{arg\,min}\limits_{\theta}\frac{1}{M}\sum_{j=1}^{M}||\mathcal{N}(t_{j};\theta)-u_{j}||_{2}^{2}.$
(6)

The function
$\frac{1}{M}\sum_{j=1}^{M}||\mathcal{N}(t_{j};\theta)-u_{j}||_{2}^{2}$ on the
right hand side of (6) is called the mean squared error (MSE) loss function. A
major task in training a network is to determine the suitable number of layers
and the number of neurons per layer needed, the choice of activation function,
and an appropriate optimizer for the loss function [25].

###  3.2 Epidemiology Informed Neural Network (EINN)

EINN is an FNN that includes the known epidemiology dynamics in its loss
function. EINN MSE loss function is a modification of the PINN MSE loss
function in order to learn time-varying model parameters. In this paper, EINN
is adapted for the asymptomatic-SIR model eq. (1), where the EINN MSE loss
function includes the known epidemiology dynamics such as a lockdown. We
observe that training data is not available for all the compartments in the
asymptomatic-SIR model, however, EINN is able to capture the epidemiology
interactions between the compartments because the epidemiology model residual
is included in the EINN MSE loss function. The EINN MSE loss function is given
by

$\begin{split}MSE&=\frac{1}{M}\sum_{j=1}^{M}||I(t_{j};\theta)-\tilde{I}(t_{j})||_{2}^{2}+\frac{1}{M}\sum_{j=1}^{M}||R(t_{j};\theta)-\tilde{R}(t_{j})||_{2}^{2}\\\
&+\frac{1}{M_{\beta}}\sum_{j=1}^{M_{\beta}}||\beta(t_{j};\theta)-\tilde{\beta}(t_{j})||_{2}^{2}\\\
&+\frac{1}{M_{\gamma}}\sum_{j=1}^{M_{\gamma}}||\gamma(t_{j};\theta)-\tilde{\gamma}(t_{j})||_{2}^{2}\\\
&+\frac{1}{M_{\kappa}}\sum_{j=1}^{M_{\kappa}}||\kappa
S(t_{j};\theta)-V(t_{j})||_{2}^{2}\\\
&+||J(0;\theta)-\tilde{J}(0)||_{2}^{2}+||U(0;\theta)-\tilde{U}(0)||_{2}^{2}\\\
&+\frac{1}{M}\sum_{i=1}^{6}\sum_{j=1}^{M}||L_{i}(t_{j};\theta)||_{2}^{2},\end{split}$
(7)

where the residual $L_{i}$, $i=1,\ldots 6$ is as follows

$\begin{split}L_{1}(t_{j};\theta)&=\frac{dS(t_{j};\theta)}{dt_{j}}+\frac{1}{N}\beta(t_{j};\theta)\Big{(}I(t_{j};\theta)+J(t_{j};\theta)\Big{)}S(t_{j};\theta)+\kappa
S(t_{j};\theta)\\\
L_{2}(t_{j};\theta)&=\frac{dI(t_{j};\theta)}{dt_{j}}-\frac{1}{N}\beta(t_{j};\theta)\xi\Big{(}I(t_{j};\theta)+J(t_{j};\theta)\Big{)}S(t_{j};\theta)+\gamma(t_{j};\theta)I(t_{j};\theta)\\\
L_{3}(t_{j};\theta)&=\frac{dJ(t_{j};\theta)}{dt_{j}}-\frac{1}{N}\beta(t_{j};\theta)\Big{(}1-\xi\Big{)}\Big{(}I(t_{j};\theta)+J(t_{j};\theta)\Big{)}S(t_{j};\theta)+\mu(t_{j};\theta)J(t_{j};\theta)\\\
L_{4}(t_{j};\theta)&=\frac{dR(t_{j};\theta)}{dt_{j}}-\gamma(t_{j};\theta)I(t_{j};\theta)-\kappa\xi
S(t_{j};\theta)\\\
L_{5}(t_{j};\theta)&=\frac{dU(t_{j};\theta)}{dt_{j}}-\mu(t_{j};\theta)J(t_{j};\theta)-\kappa(1-\xi)S(t_{j};\theta)\\\
L_{6}(t_{j};\theta)&=N-(S(t_{j};\theta)+I(t_{j};\theta)+J(t_{j};\theta)+R(t_{j};\theta)+U(t_{j};\theta)).\end{split}$
(8)

Figure 2: Schematic of the Epidemiology Informed Neural Network for nonlinear
time-varying infection and recovery rates

In Figure 2, EINN includes the time-varying infection and recovery rates as an
output of the neural network. $ICs$ represents the loss at $t=0$ in the neural
network output for the asymptomatic infectives $J(0;\theta)$ and the actual
asymptomatic infectives $J(0)$ determined according to the formula
$J(0)=(1-\xi)I(0)/\xi$. And the loss at $t=0$ in the neural network output for
the asymptomatic recovered $U(0;\theta)$ and the actual asymptomatic recovered
$U(0)$ determined according to the formula $U(0)=(1-\xi)R(0)/\xi$. $KPs$
represent the known dynamics in the infection rates pattern. $M$ is the number
of training points. M does not necessarily correspond to the number of
available data. $M$ is generated by fitting the data with an interpolation
function. For instance $\tilde{I}(t_{j})$, $j=1,\ldots,M$ is the training data
for the infectives after fitting with an interpolation function. $M_{\beta}$
and $M_{\gamma}$ are the number of training points used to enforce the known
dynamics of the infection rates pattern. On the other hand, $M_{\kappa}$ is
the number of vaccination days in the model, $M_{\kappa}$ is the number of
days $\kappa$ is not zero. $V(t_{j})$, $j=1,\ldots,M_{\kappa}$ is the daily
vaccination data for days $\kappa$ is not zero. The input to EINN is $t_{j}$,
$j=1,\ldots,M$. The epidemiology model is embedded in the residual eq. (8).
EINN solutions are not unique. In order to achieve good accuracy, we tune the
hyperparameters; such as the number of layers, number of training points, and
the learning rate. In all the simulations presented in this paper, we used 4
hidden layers, 64 neurons per layer, and the training loss was minimized in
40000 iterations. An interpolation function is used to generate 3000 training
points from the original data-set. The loss function is minimized by a
gradient-based optimizer such as the $L-BFGS-B$ [26].

##  4 EINN algorithm for constant infection rates

In this section, we present the EINN algorithm 1 for the asymptomatic-SIR
model with constant parameters. That is, in eq. (1), we set
$[\beta(t),\gamma(t),\mu(t)]$ to $[\beta,\gamma,\mu]$. The learned cumulative
infectives and the recovered solution is matched against the training data. In
this algorithm, the parameters represent average rates. We implement algorithm
1 using publicly available COVID-19 data [24].

1:Construct EINN specify the input: $t_{j}$, $j=1,\ldots,M$ Initialize EINN
parameter: $\theta$ Initialize the epidemiology and vaccination parameters:
$\lambda=[\beta,\gamma,\mu,\xi,\kappa]$

2:Specify the training set Training data: $\tilde{I}(t_{j})$,
$\tilde{R}(t_{j})$, $j=1,\ldots,M$ and $\tilde{V}(t_{j})$,
$j=1,\ldots,M_{\kappa}$. Asymptomatic population initialization:
$\tilde{J}(0)=(1-\xi)\tilde{I}(0)/\xi$ and
$\tilde{U}(0)=(1-\xi)\tilde{R}(0)/\xi$.

3:Train the neural network Specify an $MSE$ loss function:
$\begin{split}MSE&=\frac{1}{M}\sum_{j=1}^{M}||I(t_{j};\theta;\lambda)-\tilde{I}(t_{j})||_{2}^{2}+\frac{1}{M}\sum_{j=1}^{M}||R(t_{j};\theta;\lambda)-\tilde{R}(t_{j})||_{2}^{2}\\\
&+\frac{1}{M_{\kappa}}\sum_{j=1}^{M_{\kappa}}||\kappa
S(t_{j};\theta)-\tilde{V}(t_{j})||_{2}^{2}\\\
&+||J(0;\theta;\lambda)-\tilde{J}(0)||_{2}^{2}+||U(0;\theta;\lambda)-\tilde{U}(0)||_{2}^{2}\\\
&+\frac{1}{M}\sum_{i=1}^{6}\sum_{j=1}^{M}||L_{i}(t_{j};\theta;\lambda)||_{2}^{2}.\end{split}$
(9) Minimize the $MSE$ loss function eq. (7): compute
$\operatorname*{arg\,min}\limits_{\\{\theta;\lambda\\}}(MSE)$ using an
optimizer such as the $L-BFGS-B$.

4:return EINN solution $S(t_{j};\theta;\lambda)$, $I(t_{j};\theta;\lambda)$,
$J(t_{j};\theta;\lambda)$, $R(t_{j};\theta;\lambda)$,
$U(t_{j};\theta;\lambda)$, $j=1,\ldots,M$. parameters:
$\beta,\gamma,\mu,\xi,\kappa$.

Algorithm 1 EINN algorithm for Asymptomatic-SIR model with constant parameters

Using data from Italy, South Korea, and the United States starting from the
date of the first reported cases in the respective countries to the day before
vaccination data was reported. The cumulative infective and recovered
population data are observed to be non-exponential whenever a mitigation
measure such as a comprehensive lockdown is detected in the data. We take the
total population $N$ to be $60.36\times 10^{6}$, $51.64\times 10^{6}$, and
$328.2\times 10^{6}$ in Italy, South Korea and USA respectively. In Figure
3(a), Figure 4(a), and Figure 5(a), $M_{\kappa}$ is zero and so $\kappa=0$ for
all the period from the first reported cases to the day before vaccination
data is reported. In addition to learning the parameters, EINN learns $\xi$,
the probability that an infective is reported. Large $\xi$ indicates high
reported infectives.

As shown in Figures 3(a), 4(a), and 5(a) early in the pandemic, the cumulative
infective and recovered data closely resemble an exponential function. In
Figure 3, EINN algorithm 1 learns the constant model parameters as follows:
$\beta=0.03$, $\gamma=0.0121$, $\mu=0.0128$, $\xi=0.37$, BRN = $2.3922$. The
mean squared error (MSE) is $8.11\times 10^{-4}$. In Figure 4, EINN algorithm
1 learns the constant model parameters $\beta=0.0104$, $\gamma=0.0053$,
$\mu=0.0046$, $\xi=0.22$, and BRN = $2.1876$. The mean squared error (MSE) is
$1.3\times 10^{-5}$. In Figures 5, EINN algorithm 1 learns the constant model
parameters $\beta=0.0202$, $\gamma=0.0044$, $\mu=0.0089$ and $\xi=0.46$, BRN =
$2.9575$. The mean squared error (MSE) $=1.99\times 10^{-4}$. The Centers for
Disease Control and Prevention (CDC) estimate $1-\xi$ to be $0.40$ in the USA
[13]. A high population proportion of asymptomatic infectives was estimated in
[12] for China and Singapore.

(a) The learned symptomatic infectives and recovered population by the EINN
algorithm 1. An interpolation function is used to generate 3000 training
points from the cumulative Italy symptomatic infective and recovered data. The
learned plots are a near-perfect match of the original data. The relative
$L^{2}$-error of the symptomatic infectives and symptomatic recovered are
$5.12\times 10^{-3}$ and $8.9\times 10^{-3}$ respectively.

(b) EINN algorithm 1 learns the cumulative population of Italy that are
asymptomatic infectives and asymptomatic recovered from January 31 to December
11. This represents the infective population that is unreported in the
publicly available data [24].

(c) relative error of the learned symptomatic infectives and recovered
obtained by EINN, using the actual cumulative symptomatic infectives and
recovered data.

Figure 3: Simulation of Italy COVID-19 data

(a) The learned symptomatic infectives and recovered population by the EINN
algorithm 1. An interpolation function is used to generate 3000 training
points from the cumulative South Korea symptomatic infective and recovered
data. The learned plots are a near-perfect match of the original data. The
relative $L^{2}$-error of the symptomatic infectives and symptomatic recovered
are $7.73\times 10^{-2}$ and $7.01\times 10^{-2}$.

(b) EINN algorithm 1 learns the cumulative population of South Korea that are
asymptomatic infectives and asymptomatic recovered from January 22 to December
11. This represent the infective population that are unreported in the
publicly available data [24].

(c) relative error of the learned symptomatic infectives and recovered
obtained by EINN, using the actual cumulative symptomatic infectives and
recovered data.

Figure 4: Simulation of South Korea COVID-19 data

(a) The learned symptomatic infectives and recovered population were obtained
by the EINN algorithm 1. An interpolation function is used to generate 3000
training points from the cumulative USA symptomatic infective and recovered
data. The learned plots are a near-perfect match of the original data. The
relative $L^{2}$-error of the symptomatic infectives and symptomatic recovered
are $4.03\times 10^{-3}$ and $9.77\times 10^{-3}$.

(b) EINN algorithm 1 learns the cumulative population of USA that are
asymptomatic infectives and asymptomatic recovered from January 22 to December
11. This represent the infective population that are unreported in the
publicly available data [24].

(c) relative error of the learned symptomatic infectives and recovered
obtained by EINN, using the actual cumulative symptomatic infectives and
recovered data.

Figure 5: Simulation of USA COVID-19 data

The COVID-19 infectives population surge witnessed in March and April 2020
around the world forced many countries to institute strict lockdown measures.
This is largely successful in reducing the BRN in many countries,
unfortunately, it also resulted in economic hardship, such that we seek other
measures that also reduces the BRN to a number less than $1$. In recent
months, the measures that are promoted in most countries include contact
tracing, social distancing, and facial covering. The epidemiological meaning
of each of the model parameters in eq. (1) including $\xi$ are presented in
Sections 5.1, 5.3, and 5.2.

##  5 Non-pharmaceutical mitigation measures

The model parameters in an epidemiology model are influenced by mitigation
measures. For instance, social distancing correspond to reducing the infection
rate by reducing human contact. In this Section, we simulate different levels
of various non-pharmaceutical mitigation measure and we demonstrate its impact
on BRN and the spread of COVID-19.

###  5.1 Early detection of infectives

Early detection of infectives population leads to higher reported infectives.
This results in an early isolation of individuals who have had contact with
infective individuals. There are no reported data for the asymptomatic
infectives populations. Simulating with higher $\xi$ increases the symptomatic
infectives population. This corresponds to higher reported cases. Simulation
is presented for Italy, South Korea, and the U.S.A see tables 1 and 2.

$\xi$ |  $\beta$ |  $\gamma$ |  $\mu$ |  $\beta$ $\xi$ |  $\beta$ $(1-\xi)$ |  BRN
---|---|---|---|---|---|---
0.10 |  0.0081 |  0.0013 |  0.0081 |  0.00081 |  0.00729 |  1.0916
0.25 |  0.0099 |  0.0023 |  0.0161 |  0.00248 |  0.00743 |  0.7826
0.50 |  0.0140 |  0.0045 |  0.0496 |  0.00700 |  0.00700 |  0.5176
0.75 |  0.0109 |  0.0049 |  0.0144 |  0.00818 |  0.00273 |  1.4983
Table 1:  We set the $\xi$ parameters and use the EINN algorithm 1 to learn $\beta$, $\gamma$, and $\mu$ using cumulative Italy COVID-19 symptomatic infective and symptomatic recovered data from 01/31/2020 to 09/05/2020. $\xi$ |  $\beta$ |  $\gamma$ |  $\mu$ |  $\beta$ $\xi$ |  $\beta$ $(1-\xi)$ |  BRN
---|---|---|---|---|---|---
0.10 |  0.0090 |  0.0016 |  0.0044 |  0.0009 |  0.0081 |  2.1845
0.25 |  0.0150 |  0.0068 |  0.0095 |  0.0038 |  0.0113 |  1.6997
0.50 |  0.0153 |  0.0083 |  0.0121 |  0.0077 |  0.0077 |  1.5000
0.75 |  0.0116 |  0.0145 |  0.0033 |  0.0087 |  0.0029 |  0.9915
Table 2: Fixing $\xi$, we simulate predictions for $\beta$, $\gamma$, and
$\mu$ using EINN algorithm 1 and cumulative South Korea COVID-19 symptomatic
infective and symptomatic recovered data from 01/22/2020 to 09/05/2020.

Higher $\xi$ values in tables 1 and 2, increases the symptomatic infectives
population and reduces the asymptomatic population in general. This is
reflected in increase in the $\beta\xi$ column and decrease in the
$\beta(1-\xi)$ column. It is assumed that the total infective population is
unchanged as we raise $\xi$. This means that more people will be in
hospitalization/isolation. This translates to more recovery in the symptomatic
compartment. BRN decreases in general.

If the asymptomatic infectives population is much higher than the symptomatic
population, this results in higher BRN. There is a right balance between $I$
and $J$ population. We also see that the detection of early infectives alone
is not enough to mitigate an infectious disease like COVID-19. It should be
combined with other measures such as contact tracing of infectives.

###  5.2 Social distancing

It is widely understood that measures such as a lockdown, social distancing,
and widespread adoption of facial covering result in the mitigation of
COVID-19. Social distancing is often the most sought after measure at reducing
the BRN. The goal of social distancing is to reduce the average number of
human contacts. This is demonstrated by reducing $\beta$, the infection rate.
The impact of social distancing on the BRN is presented in the following
tables 3 and 4 .

$\beta$ |  $\gamma$ |  $\xi$ |  $\mu$ |  $\beta$ $\xi$ |  $\beta$ $(1-\xi)$ |  BRN
---|---|---|---|---|---|---
0.020 |  0.0071 |  0.6667 |  0.2234 |  0.0133 |  0.0067 |  0.2525
0.015 |  0.0042 |  0.4957 |  0.0653 |  0.0074 |  0.0076 |  0.4284
0.010 |  0.0048 |  0.6375 |  0.0060 |  0.0064 |  0.0036 |  1.9102
0.005 |  0.0031 |  0.7195 |  0.0037 |  0.0036 |  0.0014 |  1.5298
Table 3: Fixing $\beta$, we simulate predictions for $\gamma$, $\xi$, and $\mu$ using EINN algorithm 1 and cumulative Italy COVID-19 symptomatic infective and symptomatic recovered data 01/31/2020 to 09/05/2020. $\beta$ |  $\gamma$ |  $\xi$ |  $\mu$ |  $\beta$ $\xi$ |  $\beta$ $(1-\xi)$ |  BRN
---|---|---|---|---|---|---
0.020 |  0.0156 |  0.3359 |  0.0076 |  0.0067 |  0.0133 |  1.9442
0.015 |  0.0077 |  0.2839 |  0.0078 |  0.0043 |  0.0107 |  1.9301
0.010 |  0.0057 |  0.2454 |  0.0041 |  0.0025 |  0.0075 |  2.2259
0.005 |  0.0017 |  0.1369 |  0.0021 |  0.0007 |  0.0043 |  2.4447
Table 4: Fixing $\beta$, we simulate predictions for $\gamma$, $\xi$, and
$\mu$ using EINN algorithm 1 and cumulative South Korea COVID-19 symptomatic
infective and symptomatic recovered data 01/22/2020 to 09/05/2020.

Reducing $\beta$ in tables 3 and 4 correspond to a reduced symptomatic
infectives population $I$. However, the total infectives are unchanged as we
reduce $\beta$. There is an increase in asymptomatic infectives population
$J$. BRN does not decrease in general. Social distancing is effective when the
asymptomatic infective population $J$ diminishes. $\beta\xi$ and
$\beta(1-\xi)$ both decreases. Social distancing should be combined with
contact tracing and early detection of infectives population.

###  5.3 Contact tracing of infectives

Contact tracing is equivalent to increasing the symptomatic recovery and
asymptomatic recovery rates. However, since we do not have reported data for
the asymptomatic population, in this paper, we pursue contact tracing as an
increase in the symptomatic recovery rate. This is equivalent to reducing the
number of days an infective individual stays infective. In tables 5, 6, LABEL:
and 7, the impact of contact tracing is demonstrated by increasing the
symptomatic recovery rate.

$\gamma$ |  $\beta$ |  $\xi$ |  $\mu$ |  $\beta$ $\xi$ |  $\beta$ $(1-\xi)$ |  BRN
---|---|---|---|---|---|---
0.0005 |  0.0095 |  0.3345 |  0.0093 |  0.0032 |  0.0063 |  1.4946
0.0010 |  0.0093 |  0.4727 |  0.0116 |  0.0044 |  0.0049 |  1.4114
0.0050 |  0.0118 |  0.5633 |  0.0133 |  0.0066 |  0.0052 |  1.3682
0.0100 |  0.0104 |  0.7669 |  0.0036 |  0.0080 |  0.0024 |  1.2224
Table 5: Fixing $\gamma$, we simulate predictions for $\beta$, $\xi$, and $\mu$ using EINN algorithm 1 and cumulative Italy COVID-19 symptomatic infective and symptomatic recovered data 01/31/2020 to 09/05/2020. $\gamma$ |  $\beta$ |  $\xi$ |  $\mu$ |  $\beta$ $\xi$ |  $\beta$ $(1-\xi)$ |  BRN
---|---|---|---|---|---|---
0.0010 |  0.0124 |  0.1556 |  0.0071 |  0.0019 |  0.0105 |  2.0160
0.0050 |  0.0109 |  0.2582 |  0.0043 |  0.0028 |  0.0081 |  2.4326
0.0100 |  0.0147 |  0.2690 |  0.0086 |  0.0040 |  0.0107 |  1.6376
0.0150 |  0.0145 |  0.3679 |  0.0079 |  0.0053 |  0.0092 |  1.3794
Table 6: Fixing $\gamma$, we simulate predictions for $\beta$, $\xi$, and $\mu$ using EINN algorithm 1 and cumulative South Korea COVID-19 symptomatic infective and symptomatic recovered data 01/22/2020 to 09/05/2020. $\gamma$ |  $\beta$ |  $\xi$ |  $\mu$ |  $\beta$ $\xi$ |  $\beta$ $(1-\xi)$ |  BRN
---|---|---|---|---|---|---
0.0005 |  0.0198 |  0.6240 |  0.0566 |  0.0124 |  0.0074 |  0.9169
0.0010 |  0.0227 |  0.4152 |  0.0222 |  0.0094 |  0.0133 |  1.6943
0.0050 |  0.0228 |  0.5783 |  0.0261 |  0.0132 |  0.0096 |  1.6405
0.0100 |  0.0295 |  0.5253 |  0.0345 |  0.0155 |  0.0140 |  1.3638
Table 7: Fixing $\gamma$, we simulate predictions for $\beta$, $\xi$, and
$\mu$ using EINN algorithm 1 and cumulative USA COVID-19 symptomatic infective
and symptomatic recovered data 01/22/2020 to 09/05/2020.

The raising of $\gamma$ in tables 5, 6 and 7, increases the symptomatic
infectives population $I$ which is demonstrated in increased $\xi$ and
increased $\beta$. $\beta(1-\xi)$ decreases somewhat while $\beta\xi$
increases. This also results in a reduced BRN. Contact tracing is an efficient
mitigation measure in lowering the spread of COVID-19.

##  6 Pharmaceutical mitigation measure

The mitigation measures described in Section 5 are non-pharmaceutical
measures. In this Section, we discuss vaccination. In the fight against
COVID-19, countries such as USA and United Kingdom began to vaccinate in
December, 2020. A major goal of vaccination is to reduce the susceptible
population, that is people recover without getting infected. This constitute a
pharmaceutical mitigation measure. We considered the vaccination data for the
USA and United Kingdom, and simulate the effectiveness of vaccination on the
daily reported infectives. Algorithm 1 is implemented for the asymptomatic-SIR
model given as follows.

(a) The effectiveness of vaccination is demonstrated by learning the pre-
vaccination and post-vaccination epidemiology parameters using smooth daily
reported infectives data from the USA. The model is extrapolated for 3 cases.
The red curve is the case of no vaccination, here $\kappa=0$. For the blue
curve, we used the USA projection of $1000000$ daily vaccination. In the case
of the magenta curve, we learned $\kappa$ using the daily vaccination data.
The first reported case was $01/22/2020$, Vaccination data were first reported
$12/19/2020$. Our model suggests that daily reported infectives will be
asymptotically zero by May - June, 2021 if the current vaccination rate is
sustained together with the non-pharmaceutical mitigation measures.

(b) The effectiveness of vaccination is demonstrated by learning the pre-
vaccination and post-vaccination epidemiology parameters using smooth daily
reported infectives data from the United Kingdom. The model is extrapolated
for 2 cases. The red curve is the case of no vaccination, here $\kappa=0$. The
magenta curve, we learned $\kappa$ using the daily vaccination data. The first
reported case was $01/31/2020$, Vaccination data were first reported
$12/13/2020$. Our model suggests that daily reported infectives will be
asymptotically zero by April, 2021 if the current vaccination rate is
sustained together with the non-pharmaceutical mitigation measures.

Figure 6: Vaccination effectiveness for USA and United Kingdom

In Figure 6(a), using USA data, the mitigation effect of vaccination on the
daily infectives is demonstrated. Implementing algorithm 1, we obtained
$\kappa=0.00184$, which is slightly different from the projection of $\kappa$
= $0.00305$, corresponding to $1$ million people vaccination per day. In
Figure 6(b), using United Kingdom data, we simulate the impact of vaccination
on the daily reported infectives, using a smoothed daily vaccination data from
12/13/2020 to 02/05/2020 and smoothed daily reported infectives data. We
implement algorithm 1 and we obtained $\kappa=0.00305$. Our model predicts
that the daily infectives will be close to zero by April 2021. We demonstrate
the impact of increased social distancing together with the vaccination
effort. Social distancing corresponds to decreasing the infection rate
$\beta$. Increased social distancing reduces the daily reported infectives but
it extends the number of days daily infectives data is significant.

##  7 EINN algorithms for time-dependent infection rates

The model parameters in the previous section should be interpreted as average
values throughout the pandemic. We cannot assume that these parameters stay
constant because of the many government interventions and public perceptions
and responses. In this section, we compare a delayed-mitigation time-series
infection rate approach, a delayed-mitigation exponential time-dependent
infection rate approach [5, 11, 3] and a piecewise time-dependent infection
rate approach. In the case of the delayed-mitigation approach, Italy and the
USA instituted the first COVID-19 lockdown on March 9th and March 18th
respectively. That is 38 days after the first case in Italy and 57 days in the
USA. In 7.1 and 7.2, we simulated the time-varying infection rate assuming
there was no mitigation measure influencing the data pre-lockdown. We run
simulations for Italy, South Korea, and the United States from the date of the
first reported cases in the respective countries to the day before vaccination
data was reported, $\kappa=0$.

###### Remark 1

The time-dependent infection rate is non-constant in the presence of
mitigation measures in the cumulative infective data.

###  7.1 Delayed-mitigation time-series infection rate

Here, the infection rate $\beta(t)$ in eq. (1) is nonlinear and time-
dependent. The early COVID-19 lockdown that was instituted by several
countries was widespread in reach and effective in mitigating the spread of
the disease. The impact of lockdown was also immediately reflected in the
cumulative infective data. As a result, we consider the infection rate to be
constant pre-lockdown and a nonlinear time-series during and post-lockdown.
Accordingly we write $\beta(t)$

$\beta(t)=\begin{cases}\beta_{0}&0\leq t<K\\\ \beta_{0}q(t)&K<t,\end{cases}$
(10)

In eq. (10), $q(t)\in(0,1)$ during and post-lockdown for each value of $t$
[27]. $\beta_{0}$ correspond to the constant infection rate obtained from
early data using nonlinear regression. It is the infection rate before any
mitigation is enforced. $q(t)$ represents the presence of some mitigation in
the data. When $q(t)=1$, this corresponds to an absence of mitigation. Using
the reported cumulative data, the EINN algorithm 2 discovers the mitigation in
the cumulative symptomatic infectives and recovered data and its impact on the
infection rate. It is assumed that there is no mitigation very early in the
data, and so the infection and recovery rates are constants during the early
days of the pandemic.

1:Construct EINN specify the input: $t_{j}$, $j=1,\ldots,M$ Initialize EINN
parameter: $\theta$ Initialize the epidemiology and vaccination parameters:
$\lambda=[\gamma,\mu,\kappa]$

2:Construct neural network: $\beta$ specify the input: $t_{j}$,
$j=1,\ldots,M_{\beta}$ Initialize the neural network parameter: $\phi$

3:Specify $\beta_{0}$ obtained by nonlinear regression of early cumulative
infective population data

4:Specify EINN training set Training data: $\tilde{I}(t_{j})$ and
$\tilde{R}(t_{j})$, $j=1,\ldots,M$. Set $\xi$ according to the value obtained
from EINN algorithm 1. Asymptomatic population initialization:
$\tilde{J}(0)=(1-\xi)\tilde{I}(0)/\xi$ and
$\tilde{U}(0)=(1-\xi)\tilde{R}(0)/\xi$.

5:Train the neural networks Specify an $MSE$ loss function:
$\begin{split}MSE&=\frac{1}{M}\sum_{j=1}^{M}||I(t_{j};\theta;\lambda)-\tilde{I}(t_{j})||_{2}^{2}+\frac{1}{M}\sum_{j=1}^{M}||R(t_{j};\theta;\lambda)-\tilde{R}(t_{j})||_{2}^{2}\\\
&+\frac{1}{M_{\beta}}\sum_{j=1}^{M_{\beta}}||\beta(t_{j};\phi)-\beta_{0}||_{2}^{2}\\\
&+\frac{1}{M_{\kappa}}\sum_{j=1}^{M_{\kappa}}||\kappa
S(t_{j};\theta)-\tilde{V}(t_{j})||_{2}^{2}\\\
&+||J(0;\theta;\lambda)-\tilde{J}(0)||_{2}^{2}+||U(0;\theta;\lambda)-\tilde{U}(0)||_{2}^{2}\\\
&+\frac{1}{M}\sum_{i=1}^{6}\sum_{j=1}^{M}||L_{i}(t_{j};\theta;\phi;\lambda)||_{2}^{2}.\end{split}$
(11) Set $\beta(t_{j};\phi)$ in the residual
$\beta(t_{j};\phi)=\begin{cases}\beta_{0}&0\leq t_{j}\leq M_{\beta}\\\
\beta_{0}q(t_{j};\phi)&M_{\beta}<t_{j},\end{cases}$ (12) Minimize the $MSE$
loss function: compute
$\operatorname*{arg\,min}\limits_{\\{\theta;\phi;\lambda\\}}(MSE)$ using an
optimizer such as the $L-BFGS-B$.

6:return EINN solution $S(t_{j};\theta;\lambda)$, $I(t_{j};\theta;\lambda)$,
$J(t_{j};\theta;\lambda)$, $R(t_{j};\theta;\lambda)$,
$U(t_{j};\theta;\lambda)$, $j=1,\ldots,M$. epidemiology parameters: $\gamma$
and $\mu$ vaccination parameter: $\kappa$

7:return time-series epidemiology parameter: $\beta(t_{j};\phi)$,
$j=1,\ldots,M_{\beta}$.

Algorithm 2 EINN algorithm for Asymptomatic-SIR model with delayed-mitigation
time-series infection rate

In the EINN algorithm 2, $M_{\beta}$ correspond to the number of days number
of days mitigation is delayed in the data, which is equal to $K$ in eq (10).

(a) Using the infection rate eq. (10) in eq. (1), we fix $\xi=0.37$ and using
early data and nonlinear regression to obtain $\beta_{0}=0.22$. A time-
dependent infection rate $\beta$ is learned and plotted for cumulative Italy
COVID-19 data from January 31, 2020 to December 11, 2020. $N=60.36\times
10^{6}$. The first COVID-19 lockdown in Italy began March 9th. We set $K=40$
days. The time-varying basic reproduction rate $R_{t}$ is obtained using the
estimate eq. (4). The EINN algorithm 2 also learns $\gamma=0.0102$ and
$\mu=0.0091$. The MSE is $9.4\times 10^{-5}$. Post lockdown, other mitigation
measures including social distancing, widespread masking kept the infection
rate down. This is followed by a relaxation period that results in COVID-19
surge that was reported during the holiday.

(b) Using the infection rate eq. (10) in eq. (1), we fix $\xi=0.46$ and using
early data and nonlinear regression to obtain $\beta_{0}=0.279$. A time-
dependent infection rate $\beta$ is learned and plotted for cumulative U.S.A
COVID-19 data from January 22, 2020 to December 11, 2020. $N=328.2\times
10^{6}$. The Nationwide Lockdown that was put in place in the USA in 2020
began March 19th. We set $K=57$ days. The time-varying basic reproduction rate
$R_{t}$ is obtained using the estimate eq. (4). The EINN algorithm 2 learns
$\gamma=0.0014$ and $\mu=0.0261$. The MSE is $3.35\times 10^{-4}$. This
approach underestimate $R_{t}$ during the pre-lockdown period.

Figure 7: Delayed-mitigation time-series rates

In [11, 3], it was shown that during the early phase of the COVID-19 pandemic
when the cumulative infection population grew exponentially, the infection
rate was constant. This is demonstrated in Figures 7(a) and 7(b). This
coincide with the period before any mitigation measure. Incorporating measures
such as social distancing, lock-down and widespread adoption of facial
covering in an epidemiology model is complex. In [15] used a sigmoid function
to model a time dependent decrease in the transmission of COVID-19. In [11],
an exponentially decreasing function eq. (13) was used to represent the
infection rate $\beta(t)$ in eq. (1) to mimic the impact of lockdown. In
Section 7.2, we learn an exponentially decreasing infection rate assuming that
it takes the form of eq. (13). We demonstrate that this approach detects
various other mitigation measures post-lockdown.

###  7.2 Delayed-mitigation exponential time-dependent infection rate

The infection rate $\beta(t)$ in eq. (1) is defined as

$\beta(t)=\begin{cases}\beta_{0}&0\leq t\leq K\\\
\beta_{0}\exp{(-\eta(t-K))}&K<t.\end{cases}$ (13)

Where $K$ signifies the onset of government intervention including isolation,
quarantine and lock-down. $\eta$ is the rate at which human contact decreases.
We use EINN algorithm 3 to learn $\beta(t)$ assuming that it takes the form of
eq. (13) in eq. (1). This approach also learns $\eta$. We denote $K$ to be the
number of days between the date of the first reported case of COVID-19 and the
date lockdown was instituted.

1:Construct EINN specify the input: $t_{j}$, $j=1,\ldots,M$ Initialize EINN
parameter: $\theta$ Initialize the epidemiology and vaccination parameters:
$\lambda=[\gamma,\mu,\kappa]$

2:construct neural network: $\beta$ specify the input: $t_{j}$,
$j=1,\ldots,M_{\beta}$ Initialize the neural network parameter: $\phi$
Initialize the exponential decay parameter: $\eta$

3:Specify $\beta_{0}$ obtained by nonlinear regression of early cumulative
infective population data

4:Specify EINN training set Training data: $\tilde{I}(t_{j})$ and
$\tilde{R}(t_{j})$, $j=1,\ldots,M$. Set $\xi$ according to the value obtained
from EINN algorithm 1 Asymptomatic population initialization:
$\tilde{J}(0)=(1-\xi)\tilde{I}(0)/\xi$ and
$\tilde{U}(0)=(1-\xi)\tilde{R}(0)/\xi$.

5:Train the neural networks Specify an $MSE$ loss function:
$\begin{split}MSE&=\frac{1}{M}\sum_{j=1}^{M}||I(t_{j};\theta;\lambda)-\tilde{I}(t_{j})||_{2}^{2}+\frac{1}{M}\sum_{j=1}^{M}||R(t_{j};\theta;\lambda)-\tilde{R}(t_{j})||_{2}^{2}\\\
&+\frac{1}{M_{\beta}}\sum_{j=1}^{M_{\beta}}||\beta(t_{j};\phi;\eta)-\beta_{0}||_{2}^{2}\\\
&+\frac{1}{M_{\kappa}}\sum_{j=1}^{M_{\kappa}}||\kappa
S(t_{j};\theta)-\tilde{V}(t_{j})||_{2}^{2}\\\
&+||J(0;\theta;\lambda)-\tilde{J}(0)||_{2}^{2}+||U(0;\theta;\lambda)-\tilde{U}(0)||_{2}^{2}\\\
&+\frac{1}{M}\sum_{i=1}^{6}\sum_{j=1}^{M}||L_{i}(t_{j};\theta;\phi;\lambda;\eta)||_{2}^{2}.\end{split}$
(14) Set $\beta(t_{j};\phi;\eta)$ in the residual
$\beta(t_{j};\phi;\eta)=\begin{cases}\beta_{0}&0\leq t_{j}\leq M_{\beta}\\\
\beta_{0}\exp{(-\eta(t_{j}-M_{\beta}))}&M_{\beta}<t_{j},\end{cases}$ (15)
Minimize the $MSE$ loss function: compute
$\operatorname*{arg\,min}\limits_{\\{\theta;\phi;\lambda;\eta\\}}(MSE)$ using
an optimizer such as the $L-BFGS-B$.

6:return EINN solution $S(t_{j};\theta;\lambda)$, $I(t_{j};\theta;\lambda)$,
$J(t_{j};\theta;\lambda)$, $R(t_{j};\theta;\lambda)$,
$U(t_{j};\theta;\lambda)$, $j=1,\ldots,M$. epidemiology parameters: $\gamma$
and $\mu$ vaccination parameter: $\kappa$

7:return time-dependent epidemiology parameter: $\beta(t_{j};\phi;\eta)$,
$j=1,\ldots,M_{\beta}$. Rate of human contact decrease: $\eta$.

Algorithm 3 EINN algorithm for Asymptomatic-SIR model with delayed-mitigation
exponential time-dependent infection rate

In the EINN algorithm 3, $M_{\beta}$ corresponds to the number of days
mitigation is delayed in the data, which is equal to $K$ in eq (13).
$M_{\kappa}$ is the number of vaccination days.

(a) The delayed-mitigation exponential infection rate is learned using eq.
(13) in eq. (1). We set $K=40$ and we and fix $\xi=0.37$ in EINN algorithm 3.
We take $\beta_{0}=0.22$, obtained using early data and nonlinear regression.
EINN algorithm 3 learns $\eta=0.87$, the rate at which human contact
decreases. A learned delayed-mitigation exponential time-dependent infection
rate $\beta$ is plotted for cumulative Italy COVID-19 data from January 31,
2020 to December 11, 2020. $N=60.36\times 10^{6}$. The plotted time-varying
basic reproduction rate $R_{t}$ shows the impact of lockdown and the
mitigation measures post lockdown. The relaxation that followed resulted in
the COVID-19 surge and is detected in the learned $\beta$ and $R_{t}$. The
EINN algorithm 3 also learns $\gamma=0.0121$ and $\mu=0.0106$. The MSE is
$7.5\times 10^{-5}$.

(b) The delayed-mitigation exponential infection rate is learned using eq.
(13) in eq. (1). We set $K=57$ and we fix $\xi=0.46$ in EINN algorithm 3. We
take $\beta_{0}=0.279$, obtained using early data and nonlinear regression.
EINN algorithm 3 learns $\eta=0.60$, the rate at which human contact
decreases. A learned delayed-mitigation exponential time-dependent infection
rate $\beta$ is plotted for cumulative U.S.A COVID-19 data from January 22,
2020 to December 11, 2020. $N=328.2\times 10^{6}$. The plotted time-varying
basic reproduction rate $R_{t}$ is underestimated in the pre-lockdown stage.
The EINN algorithm 3 also learns $\gamma=0.001$ and $\mu=0.0224$. The MSE is
$3.88\times 10^{-4}$.

Figure 8: Delayed-mitigation exponential time-dependent rates

In Section 7.2, the delayed-mitigation exponential time-dependent infection
rate detects the impact of the year 2020 COVID-19 lockdown, as well as the
other mitigation measures post-lockdown using the parameter $\eta$. It is
however difficult to know if $\eta$ captures all the pattern in the time-
dependent infection rate as demonstrated in Figures 8(a) and 8(b), that is if
eq. (13) helps us to learn the most accurate form of $\beta$. For instance,
the time-varying basic reproduction rate $R_{t}$ is underestimated pre-
lockdown in the USA data and overestimated pre-lockdown in Italy data. $R_{t}$
is overestimated post-lockdown. One limitation in Section 7.1 and Section 7.2
is that the delayed-mitigation approach is not very useful when there is no
big impacting mitigation measure such as lockdown. For example, In South
Korea, mitigation measures such as social distancing, contact tracing, and
quarantine were used to effectively combat the spread of COVID-19 early on
without needing to lockdown entire cities [28].

###  7.3 Piecewise time-dependent infection rate

A piecewise time-dependent infection rate (16) is used to learn a time-
dependent infection rate $\beta$ in eq. (1). The piecewise-constant $\beta(t)$
is defined as follows,

$\beta(t)=\begin{cases}\beta_{0}&t\leq M_{1}\\\ \beta_{0}q_{1}&M_{1}<t\leq
M_{2}\\\ \beta_{0}q_{2}&M_{2}<t\leq M_{3}\\\ \beta_{0}q_{3}&M_{3}<t\leq
M_{4}\\\ &\vdots\\\ \beta_{0}q_{n}&M_{n}<t.\end{cases}$ (16)

The goal of the parameters $q_{1},\ldots,q_{n}$ in eq. (16) is to capture the
decrease/increase that occur in the infection rate $\beta(t)$. We choose
$M_{1},\ldots,M_{n}$ and we use EINN algorithm 4 to learn
$q_{1},\ldots,q_{n}$.

1:Construct EINN specify the input: $t_{j}$, $j=1,\ldots,M$ Initialize EINN
parameter: $\theta$ Initialize the epidemiology and vaccination parameter:
$\lambda=[\gamma,\mu,\kappa]$

2:construct neural network: $\beta$ specify the input: $t_{j}$,
$j=1,\ldots,M_{\beta}$ Initialize the neural network parameter: $\phi$
Initialize the exponential decay parameters: $\omega=[q_{1},\ldots,q_{n}]$

3:Specify $\beta_{0}$ obtained by nonlinear regression of early cumulative
infective population data

4:Specify EINN training set Training data: $\tilde{I}(t_{j})$ and
$\tilde{R}(t_{j})$, $j=1,\ldots,M$. Set $\xi$ according to the value obtained
from EINN algorithm 1 Asymptomatic population initialization:
$\tilde{J}(0)=(1-\xi)\tilde{I}(0)/\xi$ and
$\tilde{U}(0)=(1-\xi)\tilde{R}(0)/\xi$.

5:Train the neural networks Specify an $MSE$ loss function:
$\begin{split}MSE&=\frac{1}{M}\sum_{j=1}^{M}||I(t_{j};\theta;\lambda)-\tilde{I}(t_{j})||_{2}^{2}+\frac{1}{M}\sum_{j=1}^{M}||R(t_{j};\theta;\lambda)-\tilde{R}(t_{j})||_{2}^{2}\\\
&+||\beta(0;\phi;\omega)-\beta_{0}||_{2}^{2}\\\
&+\frac{1}{M_{\kappa}}\sum_{j=1}^{M_{\kappa}}||\kappa
S(t_{j};\theta)-\tilde{V}(t_{j})||_{2}^{2}\\\
&+||J(0;\theta;\lambda)-\tilde{J}(0)||_{2}^{2}+||U(0;\theta;\lambda)-\tilde{U}(0)||_{2}^{2}\\\
&+\frac{1}{M}\sum_{i=1}^{6}\sum_{j=1}^{M}||L_{i}(t_{j};\theta;\phi;\lambda;\omega)||_{2}^{2}.\end{split}$
(17) Set $\beta(t_{j};\phi;\omega)$ in the residual
$\beta(t_{j};\phi;\omega)=\begin{cases}\beta_{0}&0\leq t_{j}\leq M_{1}\\\
\beta_{0}q_{1}&M_{1}<t_{j}\leq M_{2}\\\ \beta_{0}q_{2}&M_{2}<t_{j}\leq
M_{3}\\\ \beta_{0}q_{3}&M_{3}<t_{j}\leq M_{4}\\\ &\vdots\\\
\beta_{0}q_{n}&M_{n}<t_{j}.\end{cases}$ (18) Minimize the $MSE$ loss function:
compute
$\operatorname*{arg\,min}\limits_{\\{\theta;\phi;\lambda;\omega\\}}(MSE)$
using an optimizer such as the $L-BFGS-B$.

6:return EINN solution $S(t_{j};\theta;\lambda)$, $I(t_{j};\theta;\lambda)$,
$J(t_{j};\theta;\lambda)$, $R(t_{j};\theta;\lambda)$,
$U(t_{j};\theta;\lambda)$, $j=1,\ldots,M$. epidemiology parameters: $\gamma$
and $\mu$ vaccination parameter: $\kappa$

7:return time-dependent epidemiology parameter: $\beta(t_{j};\phi;\omega)$,
$j=1,\ldots,M_{\beta}$. Rate of human contact decrease: $q_{1},\ldots,q_{n}$.

Algorithm 4 EINN algorithm for Asymptomatic-SIR model with piecewise time-
dependent infection rate

(a) Using the piecewise-constant infection rate eq. (16) to learn a time-
dependent infection rate $\beta(t)$ in eq. (1). We choose $M_{1},\dots,M_{n}$
and we use EINN algorithm 4 to learn $q_{1},\dots,q_{n}$. For the Italy
COVID-19 data, we fix $\xi=0.37$ and set $\beta_{0}=0.22$. Choosing $M_{1}=$
day $20$, $M_{2}=$ day $35$, and $M_{3}=$ day $100$, we learn $q_{1}=0.77$,
$q_{2}=0.60$, and $q_{3}=0.44$. A learned piecewise time-dependent infection
rate $\beta$ is plotted for cumulative Italy COVID-19 data from January 31,
2020 to December 11, 2020. $N=60.36\times 10^{6}$. It is not smooth because it
is learned using a discrete piecewise function. The plot of the time-varying
basic reproduction rate $R_{t}$ detects the lockdown, the mitigation measures
post-lockdown and the relaxation period that followed. The algorithm also
learns $\gamma=0.0112$ and $\mu=0.0075$. The MSE is $9.7\times 10^{-5}$.

(b) Using the piecewise-constant infection rate eq. (16) to learn a time-
dependent infection rate $\beta(t)$ in eq. (1). We choose $M_{1},\dots,M_{n}$
and we use EINN algorithm 4 to learn $q_{1},\dots,q_{n}$. For the Italy
COVID-19 data, we fix $\xi=0.22$ and set $\beta_{0}=0.379$. Choosing $M_{1}=$
day $20$, $M_{2}=$ day $35$, and $M_{3}=$ day $100$, we learn $q_{1}=0.83$,
$q_{2}=0.71$, and $q_{3}=0.38$. A learned piecewise time-dependent infection
rate $\beta$ is plotted for cumulative South Korea COVID-19 data from January
22, 2020 to December 11, 2020. $N=51.64\times 10^{6}$. It is not smooth
because it is learned using a discrete piecewise function. The plot of the
time-varying basic reproduction rate $R_{t}$ detects the impact of the various
mitigation measures. The algorithm also learns $\gamma=0.0173$ and
$\mu=0.0021$. The MSE is $9.0\times 10^{-6}$.

(c) Using the piecewise-constant infection rate eq. (16) to learn a time-
dependent infection rate $\beta(t)$ in eq. (1). We choose $M_{1},\dots,M_{n}$
and we use EINN algorithm 4 to learn $q_{1},\dots,q_{n}$. For the Italy
COVID-19 data, we fix $\xi=0.46$ and set $\beta_{0}=0.279$. Choosing $M_{1}=$
day $20$, $M_{2}=$ day $35$, and $M_{3}=$ day $100$, we learn $q_{1}=1.48$,
$q_{2}=1.12$, and $q_{3}=0.72$. A learned piecewise time-dependent infection
rate $\beta$ is plotted for cumulative USA COVID-19 data from January 22, 2020
to December 11, 2020. $N=328.2\times 10^{6}$. The plot of the time-varying
basic reproduction rate $R_{t}$ detects the lockdown, mitigation measures
post-lockdown and the relaxation period that followed. It is not smooth
because it is learned using a discrete piecewise function. The algorithm also
learns $\gamma=0.0035$ and $\mu=0.0088$. The MSE is $1.7\times 10^{-4}$.

Figure 9: piecewise time-dependent rates

The learned piecewise time-dependent infection rate obtained in Section 7.3
captures the mitigation measures that occur early in the COVID-19 data. It is
non-smooth as shown in Figures 9(a), 9(b), and 9(c) because we used a discrete
piecewise function in the training. However, the COVID-19 surge that was
reported in most publicly available data in tail end of 2020 are not detected
in the time-varying basic reproduction number $R_{t}$. We see that it is
difficult to know the most accurate form of a time-dependent infection rate
that captures all the pre-lockdown mitigation measures as well as the post-
lockdown ones. We seek a time-varying infection rate that also captures the
COVID-19 surge that is reflected in the cumulative infective population data
after low $R_{t}$ for most of summer 2020.

##  8 EINN algorithm for time-series infection and recovery rates

At this point, we are convinced that the infection rate $\beta(t)$ is
nonlinear time-varying. It is non-constant in the presence of mitigation
measures in the cumulative data, and it closely resembles an exponentially
decreasing function immediately after a wide reaching mitigation measure such
as lockdown. The impact of various combination of mitigation measures post-
lockdown including social distancing, contact tracing, early detection of
infectives and strict adoption of facial covering are detected in the
cumulative infective and recovered data. We show in this section, that we can
learn the infection rate $\beta(t)$ from data without any prior knowledge on
how the aforementioned mitigation measures influence the interactions between
the compartments of an epidemiological model such as in the asymptomatic-SIR.
This approach learns the infection rate $\beta(t)$ in eq. (1) as a nonlinear
time-series without knowing an explicit form for the infection rate. This is
called equation learning [23].

We showed earlier in section (4) that increasing $\gamma$, and $\mu$
correspond to promoting measures such as contact tracing. In this section, it
is shown that the symptomatic recovery rate $\gamma(t)$ and the asymptomatic
recovery rate $\mu(t)$ are also nonlinear time-varying. Observe that in
Section 7.1, we used the early infection rate $\tilde{\beta}$ in the loss
function eq. (7), this is an example of using data and some properties of the
epidemiology model in the loss function. In Section 7.2, we used an explicit
formula for $\beta(t)$ given by equation (13) in the loss function eq. (7),
this corresponds to using data and some known dynamics of the epidemiology
model in the loss function and similarly, Section 7.3 is also an example of
using data and some known dynamics of the epidemiology model in the loss
function eq. (7). In all of these approaches, we learn nonlinear time-
dependent infection rates. In this Section, we learn the infection and
recovery rates as nonlinear time-series without knowing their explicit form.
We already established that the infection and recovery rates are non-constant
in the presence of mitigation measures. Using early data and nonlinear
regression, $\tilde{\beta}$ and $\tilde{\gamma}$ are obtained. This approach
involves using a feed forward neural network to learn the time-series $S(t)$,
$I(t)$, $J(t)$, $R(t)$, and $U(t)$, which are the solutions to the
Asymptomatic-SIR model, and using a feed forward neural network to learn the
time-series $\beta(t)$, $\gamma(t)$, and $\mu(t)$. We run simulations for
Italy, South Korea, and the United States from the date of the first reported
cases in the respective countries to the day before vaccination data was
reported, $\kappa=0$. See Figures 10(a), 10(b), 11(a), 11(b), 12(a), and
12(b).

1:Construct EINN specify the input: $t_{j}$, $j=1,\ldots,M$ Initialize the
neural network parameter: $\theta$ Initialize the vaccination parameter:
$\kappa$ Specify $\beta_{0}$ obtained by nonlinear regression of early
cumulative infective population data Specify $\gamma_{0}$ obtained by
nonlinear regression of early cumulative recovered population data

2:Specify EINN training set Training data: $\tilde{I}(t_{j})$ and
$\tilde{R}(t_{j})$, $j=1,\ldots,M$. Set $\xi$ according to the value obtained
from EINN algorithm 1 Asymptomatic population initialization:
$\tilde{J}(0)=(1-\xi)\tilde{I}(0)/\xi$ and
$\tilde{U}(0)=(1-\xi)\tilde{R}(0)/\xi$.

3:Train the neural network Specify an $MSE$ loss function:
$\begin{split}MSE&=\frac{1}{M}\sum_{j=1}^{M}||I(t_{j};\theta;\kappa)-\tilde{I}(t_{j})||_{2}^{2}+\frac{1}{M}\sum_{j=1}^{M}||R(t_{j};\theta;\kappa)-\tilde{R}(t_{j})||_{2}^{2}\\\
&+||\beta(0;\theta;\kappa)-\beta_{0}||_{2}^{2}+||\gamma(0;\theta;\kappa)-\gamma_{0}||_{2}^{2}\\\
&+\frac{1}{M_{\kappa}}\sum_{j=1}^{M_{\kappa}}||\kappa
S(t_{j};\theta;\kappa)-\tilde{V}(t_{j})||_{2}^{2}\\\
&+||J(0;\theta)-\tilde{J}(0)||_{2}^{2}+||U(0;\theta;\kappa)-\tilde{U}(0)||_{2}^{2}\\\
&+\frac{1}{M}\sum_{i=1}^{6}\sum_{j=1}^{M}||L_{i}(t_{j};\theta;\kappa)||_{2}^{2}.\end{split}$
(19) Minimize the $MSE$ loss function: compute
$\operatorname*{arg\,min}\limits_{\theta}(MSE)$ using an optimizer such as the
$L-BFGS-B$.

4:return EINN solution $S(t_{j};\theta;\kappa)$, $I(t_{j};\theta;\kappa)$,
$J(t_{j};\theta\kappa)$, $R(t_{j};\theta;\kappa)$, $U(t_{j};\theta;\kappa)$,
$j=1,\ldots,M$. Time-series epidemiology parameters:
$\beta(t_{j};\theta;\kappa)$, $\gamma(t_{j};\theta;\kappa)$,
$\mu(t_{j};\theta)$, $j=1,\ldots,M$. vaccination parameter: $\kappa$.

Algorithm 5 EINN algorithm for Asymptomatic-SIR model with time-series
infection and recovery rates

In this section, we estimate the time-series basic reproduction rate $R_{t}$
by

$R_{t}=\frac{\gamma(t)\beta(t)}{\left[\gamma(t)\left(\frac{I(t)}{I(t)+J(t)}\right)+\mu(t)\left(1-\frac{I(t)}{I(t)+J(t)}\right)\right]^{2}}.$

(a) Using EINN algorithm 5 and fixing $\xi=0.37$, graph of the time-series
symptomatic recovery rate $\gamma(t)$ and time-series asymptomatic recovery
rate $\mu(t)$ for cumulative Italy COVID-19 data from January 31, 2020 to
December 11, 2020 are obtained by initializing to $\gamma(0)$ to
$\tilde{\gamma}=0.173$ obtained from nonlinear regression of early data.

(b) Time-series infection rate $\beta(t)$ is learned from data using EINN
algorithm 5 for cumulative Italy COVID-19 data from January 31, 2020 to
December 11, 2020. $\beta(0)$ is initialized to $\tilde{\beta}=0.22$, where
$\tilde{\beta}$ is obtained by nonlinear regression on early cumulative
infective data. The plot of the time-series basic reproduction rate $R_{t}$
detects the lockdown in March, the impact of the mitigation measures post-
lockdown and the COVID-19 surge following the end of the year holidays. The
MSE is $3.0\times 10^{-5}$.

Figure 10: Time-series infection and recovery rates for Italy data

(a) Using EINN algorithm 5 and fixing $\xi=0.22$, graph of the time-series
symptomatic recovery rate $\gamma(t)$ and time-series asymptomatic recovery
rate $\mu(t)$ for cumulative South Korea COVID-19 data from January 22, 2020
to December 11, 2020 are obtained by initializing to $\gamma(0)$ to
$\tilde{\gamma}=0.256$ obtained from nonlinear regression of early data.

(b) Time-series infection rate $\beta(t)$ is learned from data using EINN
algorithm 5 for cumulative South Korea COVID-19 data from January 22, 2020 to
December 11, 2020. $\beta(0)$ is initialized to $\tilde{\beta}=0.379$, where
$\tilde{\beta}$ is obtained by nonlinear regression on early cumulative
infective data. The plot of the time-series basic reproduction rate $R_{t}$
shows the early success in reducing the spread of COVID-19. However, it also
shows the COVID-19 surge that happened in South Korea in August, 2020. The MSE
is $1.2\times 10^{-5}$.

Figure 11: Time-series infection and recovery rates for South Korea data

(a) Using EINN algorithm 5 and fixing $\xi=0.46$, graph of the time-series
symptomatic recovery rate $\gamma(t)$ and time-series asymptomatic recovery
rate $\mu(t)$ for cumulative USA COVID-19 data from January 22, 2020 to
December 11, 2020 are obtained by initializing to $\gamma(0)$ to
$\tilde{\gamma}=0.217$ obtained from nonlinear regression of early data.

(b) Time-series infection rate $\beta(t)$ is learned from data using EINN
algorithm 5 for cumulative USA COVID-19 data from January 22, 2020 to December
11, 2020. $\beta(0)$ is initialized to $\tilde{\beta}=0.279$, where
$\tilde{\beta}$ is obtained by nonlinear regression on early cumulative
infective data. The plot of the time-series basic reproduction rate $R_{t}$
shows the impact of the lockdown on the spread of COVID-19, it also shows that
there is a surge in COVID-19. The MSE is $3.8\times 10^{-5}$.

Figure 12: Time-series infection and recovery rates for USA data

##  9 Conclusion

We have presented data-driven deep learning algorithms that discovers
infection and recovery rate patterns in an epidemiology model using cumulative
symptomatic infective and recovered data. These algorithms predict
asymptomatic infectives and asymptomatic recovered populations. The
asymptomatic population is usually unreported in the publicly available data.
We learn this population group from symptomatic population data. It is
demonstrated that time-dependent functions model the nonlinear infection
rates, however, it is difficult to know the most accurate form of these time-
dependent functions. The EINN algorithms presented, learns the nonlinear time-
varying infection and recovery rates without a pre-assumed pattern. This
approach is useful when the dynamics of an epidemiological model such as an
asymptomatic-SIR model is impacted by various mitigation measures. These
algorithms can be adapted to most epidemiology models. Non-pharmaceutical
mitigation measures such as early detection of symptomatic infectives
population, contact tracing, and social distancing are promoted by showing
their impact on the spread of COVID-19. The pharmaceutical mitigation measure
such as the effect of vaccination in reducing the daily reported infectives is
demonstrated. This study is useful in the event of a pandemic such as
COVID-19, where governmental interventions and public response and perceptions
interfere in the interaction of the compartments in an epidemiology model. In
future work, we will study Spatio-temporal differential equation models with
nonlocal diffusion term that arise naturally in migratory patterns of
infectives and non-infectives during a pandemic such as COVID-19.

## References

  * Lin et al. [2020] Q. Lin, S. Zhao, D. Gao, Y. Lou, S. Yang, S. Musa, M. Wang, Y. Cai, W. Wang, L. Yang, D. He,  A conceptual model for the coronavirus disease 2019 (COVID-19) outbreak in Wuhan, China with individual reaction and governmental action,  International journal of Infectious diseases 93 (2020) 211–216.
  * Tam et al. [2020] K. Tam, N. Walker, J. Moreno,  Effect of mitigation measures on the spreading of COVID-19 in hard-hit states in the U.S.,  PLoS Computational Biology 15 (2020) e0240877.
  * Liu et al. [2020] Z. Liu, P. Magal, O. Seydi, G. Webb,  Understanding unreported cases in the COVID-19 epidemic outbreak in Wuhan, China, and the importance of major public health interventions,  Biology 9 (2020) 50\.
  * Neves and Guerrero [2020] A. Neves, G. Guerrero,  Predicting the evolution of the COVID epidemic with the A-SIR model Lombardy, Italy and Sao paulo state Brazil,  Physica D 413 (2020) 132693.
  * Eikenberry et al. [2020] S. E. Eikenberry, M. Mancuso, E. Iboi, T. Phan, K. Eikenberry, Y. Kuang, E. Kostelich, A. B. Gummel,  To mask or not to mask: Modeling the potential for the face mask use by the general public to curtail the COVID-19 pandemic,  Infectious Disease Modelling 5 (2020) 293–308.
  * Jagan et al. [2020] M. Jagan, M. S. deJonge, O. Krylova, D. J. Earn,  Fast estimation of time-varying infectious disease transmission rates,  PLoS Computational Biology 16 (2020) e1008124.
  * Magri and Doan [2020] L. Magri, N. A. K. Doan,  First-principles machine learning modelling of COVID-19,  arXiv preprint (2020).
  * Magal et al. [2019] P. Magal, G. Webb, Y. Wu,  On the basic reproduction number of reaction-diffusion epidemic models,  SIAM Journal on Applied Mathematics 79 (2019) 284–304.
  * Kermack and McKendrick [1927] W. Kermack, A. McKendrick,  A contribution to the mathematical theory of epidemics,  Proceedings of the Royal Soceity 115A (1927) 700–721.
  * Gaeta [2021] G. Gaeta,  A simple SIR model with a large set of asymptomatic infectives,  Mathematics in Engineering 3 (2021) 1–39.
  * Liu et al. [2020] Z. Liu, P. Magal, O. Seydi, G. Webb,  Predicting the cumulative number of cases for the COVID-19 epidemic in China from early data,  Mathematical Biosciences and Engineering 17 (2020) 3040–3051.
  * He et al. [2020] X. He, E. H. Y. Lau, P. Wu, X. Deng, J. Wang, X. Hao, Y. C. Lau, J. Y. Wong, Y. Guan, X. Tan, X. Mo, Y. Chen, B. Liao, W. Chen, F. Hu, Q. Zhang, M. Zhong, Y. Wu, L. Zhao, F. Zhang, B. J. Cowling, F. Li, G. M. Leung,  Temporal dynamics in viral shedding and transmissibility of COVID-19,  Nature Medicine 26 (2020) 672–675.
  * Centers for Disease Control and Prevention (2 11) [CDC] Centers for Disease Control and Prevention (CDC), COVID-19 pandemic planning scenarios, https://www.cdc.gov/coronavirus/2019-ncov/hcp/planning-scenarios.html, Accessed: 2020-12-11.
  * Kaplan et al. [2 16] J. Kaplan, L. Frias, M. McFall-Johnsen, Our ongoing list of how countries are reopening, and which ones remain under lockdown, https://www.businessinsider.com/countries-on-lockdown-coronavirus-italy-2020-3, Accessed: 2020-12-16.
  * Tepekule et al. [2021] B. Tepekule, A. Hauser, V. Kachalov, S. Andressen, T. Scheier, P. Schreiber, et al,  Assessing the potential impact of transmission during prolonged viral shedding on the effect of lockdown relaxation on COVID-19,  PLoS Computational Biology 17 (2021) e1008609.
  * Cybenko [1989] G. Cybenko,  Approximation by superposition of a sigmoidal function,  Mathematics of Control, Signals and Systems 2 (1989) 303–314.
  * Hornik [1991] K. Hornik,  Approximation capabilities of multilayer feedforward networks,  Neural Networks 4 (1991) 251–257.
  * Raissi et al. [2019] M. Raissi, P. Perdikaris, G. E. Karniadakis,  Physics informed deep learning: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations,  Journal of Computational Physics 378 (2019) 686–707.
  * Raissi et al. [2020] M. Raissi, A. Yazdani, G. E. Karniadakis,  Hidden fluid mechanics: Learning velocity and pressure fields from flow visualizations,  Science 367 (2020) 1026–1030.
  * Liang et al. [2020] W. Liang, J. Yao, A. Chen, et al,  Early triage of critically ill COVID-19 patients using deep learning,  Nature Communications 11 (2020) 3543.
  * Yazdani et al. [2020] A. Yazdani, L. Lu, M. Raissi, G. E. Karniadakis,  Systems biology informed deep learning for inferring parameters and hidden dynamics,  PLoS Computational Biology 16 (2020) e1007575.
  * Tartakovsky et al. [2020] A. Tartakovsky, C. Marrero, P. Perdikaris, G. Tartakovsky, D. Barajas-Solano,  Physics-informed deep neural networks for learning parameters and constitutive relationships in subsurface flow problems,  Water Resources Research 56 (2020) e2019WR026731.
  * Lagergren et al. [2020] J. H. Lagergren, J. T. Nardini, R. E. Baker, M. J. Simpson, K. B. Flores,  Biologically-informed neural networks guide mechanistic modeling from sparse experimental data,  PLoS Computational Biology 16 (2020) e1008462.
  * Dong et al. [2020] E. Dong, H. Du, L. Gardner,  An interactive web-based dashboard to track COVID-19 in real time,  The Lancet Infectious Diseases 20 (2020) 533–534.
  * Goodfellow et al. [2016] I. Goodfellow, Y. Bengio, A. Courville, Deep Learning, MIT Press, Cambridge, 2016.
  * Byrd et al. [1995] R. H. Byrd, P. Lu, J. Nocedal, C. Zhu,  A limited memory algorithm for bound constrained optimization,  SIAM Journal on Scientific Computing 16 (1995) 1190–1208.
  * Ketcheson [2020] D. Ketcheson,  Modeling the spread of COVID-19,  SIAM NEWS 53 (May 2020).
  * Wikipedia [1 06] Wikipedia, Covid-19 pandemic in South Korea, https://en.wikipedia.org/wiki/COVID-19_pandemic_in_South_Korea, Accessed: 2021-01-06.


# Development of a Dual-Input Neural Model for Detecting AI-Generated Imagery

Jonathan Gallagher Computational Mathematics
University of Waterloo
Email<EMAIL_ADDRESS>William Pugsley Computational
Mathematics
University of Waterloo

###### Abstract

Over the past years, images generated by artificial intelligence have become
more prevalent and more realistic. Their advent raises ethical questions
relating to misinformation, artistic expression, and identity theft, among
others. The crux of many of these moral questions is the difficulty in
distinguishing between real and fake images. It is important to develop tools
that are able to detect AI-generated images, especially when these images are
too realistic-looking for the human eye to identify as fake. This paper
proposes a dual-branch neural network architecture that takes both images and
their Fourier frequency decomposition as inputs. We use standard CNN-based
methods for both branches as described in Stuchi et al. [7], followed by
fully-connected layers. Our proposed model achieves an accuracy of 94% on the
CIFAKE dataset, which significantly outperforms classic ML methods and CNNs,
achieving performance comparable to some state-of-the-art architectures, such
as ResNet.

##  I Introduction

With the advent of high-quality, publicly available artificial-intelligence
image generation tools such as DALL·E and Midjourney, billions of artificial
images have been created and shared online. Such fake images have only become
more realistic and consequently more difficult to identify as their popularity
has grown. The problem is only further complicated by the fact that algorithms
used in generating these images are themselves constantly and rapidly
evolving. Despite efforts to reliably identify artificially generated imagery,
it is often possible to circumvent technical solutions to this problem. For
example, AI tools intended for identifying these images may fail to generalize
when faced with images generated using different models than those used in the
creation of the training set [13]. Researchers at Mozilla found that most
current disclosure methods, such as labels and watermarks denoting an image as
AI-generated, “may not prevent or effectively address harm once it has
occurred” [9]. In light of this, it is important to develop new techniques
that assist in making the distinction between real and fake images. Neural
networks are an ideal candidate for this task because they can be trained to
classify images using information that is beyond the scope of human sensory
perception. Since image generating AI systems are designed to create outputs
as realistic as possible, it is this information that we may have to rely on.

Figure 1: Real and artificially-generated images from the CIFAKE dataset.

Within the existing literature, a wide array of neural architectures have been
used to detect AI-generated images [4, 6, 8]. These models range from simple
convolutional neural networks (CNN) to well-known image classification models
that have been specifically fine-tuned for the task, such as ResNet. Results
have been mixed, with even the best models failing to detect images generated
using a different method than their training data [13]. For example, ResNet-50
is a popular CNN-based architecture used in computer vision; when applied to
the task of detecting AI-generated images, it is capable of achieving an
accuracy of $99\%$ when tested on images created using the same generator as
the training samples [13]. However, it only has an accuracy of $58.6\%$ when
tested on a dataset including fake images from seven other generators.

In contrast to existing methods, which primarily make use of either the raw
pixel data or frequency spectrum data from a discrete Fourier transform (DFT)
as input to the network, we propose the implementation of a dual-input
architecture, integrating both the frequency content and the raw pixel data to
provide a unified representation of the underlying image structure. The
inclusion of frequency data can reveal features distinct from the raw data,
despite both originating from the same source [7]. This is due to the fact
that frequency analysis addresses patterns related to the rates of change and
periodicity within the data which might otherwise not be evident in the
spatial domain representation. In particular, images generated using GANs show
patterns that are visible to the human eye and which are generally not present
in normal images [12, 10, 3]. Fig. 2 displays a bird from the CIFAR-10 dataset
and its DFT; our model will train using both these images as inputs.

(a) A cool bird.

(b) The $\log$-magnitude of the DFT, centered around the largest magnitude.

Figure 2: An example image of a bird from CIFAR-10 and its corresponding DFT.
For demonstration purposes, we take the DFT of the gray-scaled image

##  II Model

The proposed neural network model consists of two distinct branches; the first
taking a colour image as input, and the second taking the discrete Fourier
transform of the same image as input. The layers in each branch do not
interact with nor affect parameter values held in each others layers. Branch
outputs are concatenated before being passed through further linear layers. An
overview of the architecture is pictured in Fig. 5.

As the DFT is a transformation of the original input, choosing to provide it
as additional input does not necessarily provide new information to the model.
However, the DFT does in fact present the information in a manner that may
allow features that would otherwise be less evident to be learned more easily.
In this sense, the proposed workflow can be likened to data augmentation.

###  II-A Frequency Domain Branch

We design the frequency branch following the methodology outlined by Stuchi et
al. [7]. The first step in the workflow involves the segmentation of image
into quadrants, which will be addressed as “blocks” throughout the remainder
of the text. If necessary, these blocks can be further segmented into sub-
blocks for an even more granular approach. The purpose of this segmentation is
to encourage the model to learn both local and global features within the
image. For example, a $32\times 32$ image can be split into four $16\times 16$
blocks, which can each be subsequently split into four $8\times 8$ sub-blocks,
with subsequent operations applied to each block independently. With the
previous example, we would have 21 sets of pixels to operate on (the entire
image, its four $16\times 16$ blocks, and the 16 $8\times 8$ sub-blocks). Fig.
3 provides a visual representation of the first stage of this process.

Figure 3: A $32\times 32$ image of a bird split into four $16\times 16$ sub-
blocks.

Prior to sectioning each image into quadrants, its content is parsed into one
of three colour channels (RGB) such that the two-dimensional DFT can be
applied to each channel separately. For a channel with size $A\times B$ and
pixel values $f_{lj}$, the corresponding entries of the DFT are

$F_{hk}=\sum_{l=0}^{A-1}\sum_{j=0}^{B-1}f_{lj}e^{-2\pi i(hl/A+kj/B)},\
\text{where}\\\ h=0,1,\ldots,A-1\ \text{and}\ k=0,1,\ldots,B-1.$ (1)

Since the outputs of DFTs are complex-valued, the outputs of the operation are
evaluated in terms of their complex magnitude, $M_{hk}$, for each frequency
value to get

$M_{hk}=\sqrt{(\text{Re}(F_{hk}))^{2}+(\text{Im}(F_{hk}))^{2}},$ (2)

where Re and Im denote the real and imaginary parts of a complex number,
respectively. This sacrifices some information but is necessary in order for
the network to learn using back-propagation. After determining the magnitude
we further take the natural logarithm of $M_{hk}$, as raw outputs may be
separated by several orders of magnitude. Next, we include a max pooling
layer, reducing the dimensionality of the problem space and speeding up the
training process. Continuing with the previous example, our 21 blocks have
been reduced to one $16\times 16$ block which was originally the entire image,
four $8\times 8$ blocks, and 16 $4\times 4$ blocks.

Figure 4: ReLU, LReLU, and PReLU with paramater $p=0.05$ activation functions.
Figure 5: Schematic of the branched network architecture depicting: a) the
frequency branch and b) the image branch, where layers which serve the same
purpose are color coded.

After processing and transforming the inputs, the data is passed through a
series of convolution/filtering layers. One convolution filter is used for
each block size. Continuing with our example, a filter’s parameters are
trained on the $16\times 16$ block, another on all four $8\times 8$ blocks,
and a third on the remaining $4\times 4$ blocks. Having taken the magnitude of
the frequency values, this step is no different from the convolution layers in
any other CNN. After which, data is passed through a rectified linear unit
(LReLU) activation function [2]. LReLU is similar to standard ReLU, except for
negative inputs where the function is linear with slope $0.01$. For input
current $x$, the activation is

$\text{LReLU}(x)=\begin{cases}x&\text{if }x\geq 0\\\ 0.01\cdot x&\text{if
}x<0\end{cases}.$ (3)

The function’s kink for non-zero values helps prevent dead neurons and overall
improves model training dynamics in deeper networks.

Finally, all blocks and sub-blocks are flattened and concatenated before
passing the results through a series of fully-connected layers. Within these
layers we opt to use the parametric rectified linear unit (PReLU) activation
functions [5]. PReLU is an adaptation of leaky ReLU, where the coefficient of
the negative portion of the function is a learnable parameter. This change
allows the activation function to adapt dynamically to the data during
training. For an input current $x$, the PReLU activation is

$\text{PReLU}(x)=\begin{cases}x&\text{if }x\geq 0\\\ p\cdot x&\text{if
}x<0\end{cases},$ (4)

where $p$ is a learnable parameter.

###  II-B Spatial Domain Branch

The spatial domain branch inherits a similar architecture from the frequency
branch. In a similar manner, the image is parsed into its RGB channels and
divided into sub-blocks before being fed into the learnable layers. We use the
same operations as described in the frequency layer, with the only exception
being the absence of the DFT. In contrast to the frequency layer, we have no
need to take the magnitude of the pixel values as they are not complex valued.
After splitting the image into its blocks and sub-blocks, we pass these
through our filtering layers followed by LReLU activation functions. The
convolutions are trained on blocks which are the same size as those present in
the frequency layer, however, their parameters are entirely distinct from the
filters in the other branch. After filtering, max-pooling is applied to down-
sample the feature map and reduce computational complexity. Similarly, the
spatial branch concludes with fully-connected layers and further PReLU
activation functions.

###  II-C Merged Layers

At their intersection, the outputs from both the frequency and image branches
are concatenated into a single vector, which is then passed through further
fully connected layers, again using PReLU activation. The final output layer
consists of a single neuron, representing the network’s estimate for the
probability that the input image is AI-generated. As such, the logistic
activation function is applied to the output such that it is constrained to
the range $[0,1]$.

The network is architected (padding, stride,filter size, etc.) such that the
outputs of both the spatial and frequency branches have identical
dimensionality. This way, each branch has an equal contribution to the merged
fully connected. If it is desired that one branch is weighed more heavily in
the model’s analysis its output can be made bigger, and vice versa.

Fig. 5 is a diagram of the complete model architecture. The fully connected
layers, filtering blocks and PReLU are highlighted as those with learnable
parameters.

(a) Accuracy of the model on the training/test set during the training loop.

(b) Binary cross-entropy loss of the model for the training/test set during
the training loop.

Figure 6: Performance metrics of the model during training.

##  III Experiment

The model’s performance was evaluated by training and testing using the CIFAKE
dataset, comparing its performance to the performance of other state-of-the-
art neural network architectures prevalent in the literature. The CIFAKE
dataset was intentionally selected because of its manageable size given
constraints on time and computational resources.

The following paragraph is a summary of the relevant properties of the CIFAKE
dataset, as found in the paper in which it was originally presented [1].
CIFAKE is composed of both the classic CIFAR-10 dataset, made up of real
images, and an equal number of artificial images generated using SDM, a latent
diffusion model. Each image is labelled as either real (class 0) or fake
(class 1). The dataset is composed of 100,000 training samples and 20,000 test
samples, each composed of equal parts authentic and generated images. To
provide certainty that the model is indeed learning to distinguish between
real and generated images, the fake images are generated to intentionally
resemble the ten classes in the original CIFAR-10 dataset. Otherwise, one
could argue that the model is learning to classify images as being an instance
of one of the ten classes from CIFAR-10 or an instance of whatever classes may
be present in the artificial dataset.

The models which were used as benchmarks are: a classic support vector machine
(SVM), a CNN, ResNet-50, VGGNet, and DenseNet [11]. For the last three models,
Wang et al. used weights which were pre-trained on the ImageNet dataset; these
weights were then fine-tuned using the CIFAKE images.

###  III-A Training

Since the ResNet, VGGNet, and DenseNet networks are trained on ImageNet data,
it is important that the images they take as inputs are processed in the same
way. While we do not use any pre-trained weights, we will follow the same data
pre-processing/augmentation procedures as in Wang et al. [11]. Our data should
be as similar to that used by our benchmarks as possible so that any
difference in performance can be attributed to the architecture. We
horizontally flip each image with 0.5 probability and normalize over each
colour channel using preset means $(0.485,\ 0.456,\ 0.406)$ and standard
deviations $(0.229,\ 0.224,\ 0.225)$.

Given the relatively low resolution of the CIFAKE images ($32\times 32$
pixels) we opted to split the image into one set of four $16\times 16$ sub-
blocks. Experimentation demonstrated that further segmentation resulted in
noise in the dataset and diminished performance. Testing further divisions
into $8\times 8$ and $4\times 4$ blocks reduced the model’s accuracy by more
than $0.20$, all other factors remaining equal.

The model was trained on the CIFAKE dataset for a total of $15$ epochs using
the Adam optimizer and batch sizes of $32$ images. Our learning rate is
scheduled such that it is held constant at $10^{-4}$ for the first ten epochs,
and reduced to $10^{-5}$ for the last five. As the network is intended to
perform binary classification it is suitable to implement binary cross-entropy
as our loss function.

A variety of regularization techniques were implemented in order to improve
the solution quality and prevent overfitting:

  * •

Dropout: Dropout was applied to each unit in the linear layers with
probability $0.5$. Dropout was not applied to the convolutional layers.

  * •

Weight Initialization: Uniform Glorot initialization was implemented for all
the linear layers’ weights and convolution layers’ filters. All biases were
initialized to zero.

  * •

Data Augmentation: As described previously, a random horizontal flip was
applied to the images before being normalized.

  * •

Scheduled Learning Rate: Learning rate was scheduled such that it decreases by
an order of magnitude every ten epochs to help the parameters stabilize around
an equilibrium.

##  IV Results

After training, the model achieved a test accuracy of $0.94$. Fig. 6 shows the
performance of the model throughout the training process. It is observed that
the training and test loss both decrease steadily during the first ten epochs,
after which the learning rate is reduced by an order of magnitude, as detailed
in section III-A. For remaining five epochs, the test loss stays constant
while the training loss slightly decreases. This is indicative of the model
converging upon a stable solution and not overfitting. Fig. 6(b) corroborates
this conclusion; the training and test accuracies steadily improve before the
test accuracy settles at an equilibrium solution while the training accuracy
grows slightly.

TABLE I: Results for SVM, CNN, ResNet, VGGNet, and DenseNet models from Wang et al. [11]. Model | Precision | Recall | F1-Score | Accuracy
---|---|---|---|---
SVM | 0.8020 | 0.8222 | 0.8120 | 0.8143
CNN | 0.8734 | 0.8574 | 0.8653 | 0.8640
ResNet | 0.9917 | 0.9066 | 0.9472 | 0.9495
VGGNet | 0.9657 | 0.9547 | 0.9602 | 0.9600
DenseNet | 0.9769 | 0.9779 | 0.9774 | 0.9774
Our Model | 0.9444 | 0.9321 | 0.9570 | 0.9451

A detailed account of the model’s performance in comparison to the benchmarks
can be found in TABLE I. We find that our model outperforms both the SVM and
CNN models. The original CIFAKE paper proposed another CNN-based model that
achieved a test accuracy of just under 0.93, which our model also outperforms
[1]. However, our model underperforms in all metrics when compared to VGGNet
and DenseNet. Our model is most similar to ResNet in performance; the
accuracies are within $0.01$ of each other and the F1-scores are even closer.
To understand this result we need to recognize that ResNet’s precision is
higher, but its recall is lower. Since the F1-score of a classifier is the
harmonic mean of its precision and recall, it is sensible that our values
average to similar results despite their discrepancies.

ResNet has high precision. Indeed, it is higher than any other model’s we
consider and higher than any value for recall. In the context of AI image
detection, this means that when ResNet classifies a picture as being AI-
generated, it is very likely to be correct. However, its relatively low recall
presents an issue. There will be proportionally more AI-generated images that
go undetected when we use ResNet over our model. Determining which of these
two models is more suitable will depend on the specific application and
problem context. For example, in some contexts it may be preferable to detect
many fake images even at the expense of increasing the rate of false
positives. The choice will require careful consideration as to whether or not
the harm from incorrectly classifying a real image as fake is less than the
harm from incorrectly classifying a fake image as real. This is a very
situational question and will depend for what purpose the user wishes to
distinguish between AI-generated and real images.

##  V Conclusion

This study introduced a novel dual-branch neural network architecture,
incorporating both image and discrete Fourier frequency decompositions to
detect AI-generated images with high reliability. In comparison to established
benchmarks, our model achieved an accuracy of $0.94$, outperforming
conventional machine learning and CNN-based approaches, and demonstrating
performance which is comparable to that of state of the art models such as
ResNet.

In its current state, the proposed architecture is not yet comparable to
VGGNet and DenseNet, falling short in all metrics. Despite this, a more
systematic approach to selecting and tuning hyper-parameters could potentially
result in further improve performance, provided a longer time horizon.

Future work should investigate the application of the model in a broader array
of contexts, such as how it performs on images which were generated using
methods that were not included in the training set. Doing so will allow for an
understanding of how generalizable our model is and where necessary
improvements can be made. Furthermore, the model would benefit from a more
sophisticated dataset, ideally one which more closely mimics the evolving
capabilities of state of the art image generation techniques. With CIFAKE, we
are limited to low resolution images of ten kinds of objects. Such a dataset
will ensure that we are assessing the models utility in real world scenarios,
where generative techniques are ever-evolving.

Going beyond the dataset, we can make changes to the architecture as well. Our
use of standard CNN tools is not the only approach. We can replace either of
the branches with residual neural networks, transformers, or many others. Even
had our model surpassed previous results, it would likely not remain in the
top spot for long. The use of Fourier decompositions is one more tool for
researchers and developers to consider. As the landscape of AI-generated
images continues to change, so to must the architectures and tools we use to
respond to it.

## References

  * [1] Jordan J. Bird and Ahmad Lotfi  “CIFAKE: Image Classification and Explainable Identification of AI-Generated Synthetic Images”, 2023  arXiv:2303.14126
  * [2] Shiv Ram Dubey, Satish Kumar Singh and Bidyut Baran Chaudhuri  “Activation functions in deep learning: A comprehensive survey and benchmark”  In _Neurocomputing_ 503, 2022, pp. 92–108  DOI: https://doi.org/10.1016/j.neucom.2022.06.111
  * [3] Diego Gragnaniello et al.  “Are GAN Generated Images Easy to Detect? A Critical Analysis of the State-of-the-Art”, 2021  arXiv:2104.02617
  * [4] Kaiming He, Xiangyu Zhang, Shaoqing Ren and Jian Sun  “Deep Residual Learning for Image Recognition”, 2015  arXiv:1512.03385
  * [5] Kaiming He, Xiangyu Zhang, Shaoqing Ren and Jian Sun  “Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification”, 2015  arXiv:1502.01852 [cs.CV]
  * [6] Ze Liu et al.  “Swin Transformer: Hierarchical Vision Transformer using Shifted Windows”, 2021  arXiv:2103.14030
  * [7] José Augusto Stuchi, Levy Boccato and Romis Attux  “Frequency Learning for Image Classification”, 2020  arXiv:2006.15476
  * [8] Hugo Touvron et al.  “Training Data-Efficient Image Transformers & Distillation through attention”, 2021  arXiv:2012.12877
  * [9] Ramak Molavi Vasse’i and Gabriel Udoh  “In Transparency We Trust?”  Mozilla, 2024  URL: https://foundation.mozilla.org/en/research/library/in-transparency-we-trust/research-report/#key-findings
  * [10] Sheng-Yu Wang et al.  “CNN-Generated Images Are Surprisingly Easy to Spot… For Now”, 2020  arXiv:1912.11035
  * [11] Yuyang Wang, Yizhi Hao and Amando Xu Cong  “Harnessing Machine Learning for Discerning AI-Generated Synthetic Images”, 2024  arXiv:2401.07358 [cs.CV]
  * [12] Xu Zhang, Svebor Karaman and Shih-Fu Chang  “Detecting and Simulating Artifacts in GAN Fake Images”, 2019  arXiv:1907.06515
  * [13] Mingjian Zhu et al.  “GenImage: A Million-Scale Benchmark for Detecting AI-Generated Image”, 2023  arXiv:2306.08571