# Synchrony/Asynchrony vs. Stationary/Mobile? The Latter is Superior…in
Theory.

Eli Gafni  UCLA<EMAIL_ADDRESS>and  Vasileios Zikas  Purdue University
<EMAIL_ADDRESS>
###### Abstract.

Like Asynchrony, _Mobility_ of faults precludes consensus. Yet, a model $M$ in
which Consensus is solvable, has an analogue relaxed model in which Consensus
is not solvable and for which we can ask, whether Consensus is solvable if the
system initially behaves like the relaxed analogue model, but eventually
morphs into $M$. We consider two relaxed analogues of $M$. The first is the
traditional Asynchronous model, and the second to be defined, the Mobile
analogue. While for some $M$ we show that Consensus is not solvable in the
Asynchronous analogue, it is solvable in all the Mobile analogues. Hence, from
this perspective Mobility is superior to Asynchrony.

The pie in the sky relationship we envision is: Consensus is solvable in $M$,
if and only if binary Commit-Adopt is solvable in the mobile analogue.

The “only if” is easy. Here we show case by case that the “if” holds for all
the common faults types.

###### Acknowledgements.

This version was submitted to PODC 2020.

##  1\. Introduction

###  1.1. Mobility and Message Adversary

The notion of _indulgence_ , a term usually frowned upon, has found merit in
distributed computing as an adjective for an algorithm that solves a task in
an environment which is initially asynchronous but eventually behaves
synchronously (PODC:Guerraoui00, ). An algorithm indulgences the asynchronous
period in the sense of preserving safety. It becomes live in the times of
synchrony. Obviously this notion can be extended to a task $T$, by saying that
$T$ is indulgent if there exist an indulgent algorithm to solve $T$. More
precisely, a task $T$ is indulgent if when $T$ is solvable in any model with
certain type and number of faults, it is also solvable eventually.

Thus, indulgence takes a task and all the models in which the task is
solvable. For each model we assume a relaxed model is defined. Then the task
is indulgent if it is solvable when the system starts in the respective
relaxed model, but eventually behaves as the (unrelaxed) model.

Thus indulgence has there parameters: The Task $T$, the fault type, and the
pairs of unrelaxed/relaxed models.

Here we consider the task to be Consensus, and the pairs to be
Synchronous/Asynchronous for the common fault types.

This paper’s motivation is the displeasing observation that for two fault
types consensus is not indulgent:

  1. (1)

Send-Omission Faults: Consensus in the synchronous case is achievable for
$t<n$, where $t$ is the number of faults and $n$ is the number of processors.
In contrast, it is a folklore that at the time of asynchrony no safety can be
maintained for $t\geq n/2$ as network partition occurs.

  2. (2)

Authenticated Byzantine: Consensus in the synchronous case is achievable for
$t<n/2$ (DolStr83, ; Fit02, ). In contrast, at the time of asynchrony no
safety can be maintained for $t\geq n/3$ (PODC:DLS84, ).

To avoid displeasure, we investigate replacing the Synchrony/Asynchrony pair
with the Stationary/Mobile pair. The system is synchronous, each processor
gets a signal of an end of a round after which no message is in transit to it.
The misbehavior is when the faults are mobile, and the desired behavior is
when the faults become stationary. If the number of fault is $t$, in the
stationary case only fixed $t$ processors can experience faults. In the mobile
case, each round $t$ different processors may exhibit faulty behaviour.

Usually, faults are attributed to processors. Here we restrict ourselves to an
adversary which attacks a processor by controlling their messages sending
interface. Thus a 1-omission resilient mobile system will be a _synchronus_
system where all send to all and in each round an adversary chooses a
processor and can drop any number of the messages the processor sends (cf.
(DBLP:journals/ipl/JayantiCT99, )). In the Byzantine case, the adversary can
not only drop messages but also tamper with them. And finally, we attend to
the Authenticated Byzantine, which posed the foremost technical difficulty of
defining the notion of mobility, and proving the desired result for it.

When defining authentication, one often thinks of its cryptographic
instantiation by means of digital signature. But authentication has an
English-Dictionary definition independent of signatures. It is about
verification that a claim holds. the verification of a claim that a painting
is by Picasso, or that a fossil is from the Paleolithic Period, or that a
Diary was written by Hitler, is called Authentication . Thus in this paper
Authenticated Byzantine is an abstract assumptions on which claims can be
verified (and consequently not forged) and which cannot. Obviously in our case
it will be about “if processor $p_{i}$ claimed it received a message $m$ at
round $j$ from $p_{k}$” can the receiver of such a claim can verify verify
whether the claim is true or not. Thus, it constraints the adversary, to make
only claims that cannot be verified as false. Consequently, forgetting the
means of verification, we consider authentication set of pairs $(p_{i},j)$
where $p_{i}$ is a processor and $j$ is round number. The messages sent by
$p_{i}$ at round $j$ can be forged while message generated by pairs not in the
set cannot be forged.

Our results in the Authenticated Byzantine is for this abstract constrains. To
our knowledge, Authenticated Byzantine was never defined for the mobile case
at this clean level of abstraction. A challenge this paper poses is to find an
implementation, e.g., by using cryptography under appropriate assumptions, of
the functionality of our abstract definition. Nevertheless, proof can still be
done on the abstract functionality level.

Our first encounter with idea of mobility was through the beautiful
observation of Santoro and Widmayer (STACS:SW89, ) that the FLP proof of
consensus impossibility translates verbatim to the mobile setting, and
afterwards for privacy-violating corruptions in the context of _proactive-
security_ (PODC:OstYun91, ).

It isn’t clear what Santoro and Widmayer had in mind as the cause of faults
when they talked about omission faults. Why did the omission faults occur?
There can be two views for that. One is that the processor misbehaved, the
other is that an adversary intercepted messages sent, and dropped them. Which
view one chooses, conceptually makes a big difference. In the Byzantine
failures case the generalization of the former is that a virus got control of
the processor. The generalization of the latter is that a deamon only tampered
with messages.

Theoretically speaking, the latter view of a deamon, rather than a virus—which
in fact gave rise to the notion of _message-adversary_ (MAd) by Afek and Gafni
—is more pleasing: In this approach a message adversary unified many seemingly
unrelated notions. For instance, MAd when generalized here to the Byzantine,
has processors being always innocent and good. Thus we avoid questions like
“are corrupt processors required to output and, if so, what should their
output be?” or specifying a problem in terms of “correct” and “incorrect”
which are notions associated with an execution, defeating the presentation of
task as solely a mathematical relation, independent of the environment in
which it is to be solved. The analogue of a function in centralized computing.

Alas, thinking of the adversary, especially, in the Byzantine case as being a
Message Adversary may not sit well with reality. This paper isn’t about
reality, it is about whether we can get the mathematics to be nicer.

This paper is in the mold of _set consensus_ (DBLP:conf/podc/Chaudhuri90, ).
There is no _killer application_ for set consensus. There might never be.
Nevertheless, the mathematics say that for distributed computing there is no
preference to consensus over set consensus. Similarly here. Yet, in fact, some
solutions for proactive security move in our theoretical direction of MAd
(PODC:OstYun91, ).

###  1.2. Consensus vs. Commit-Adopt

Commit-Adopt (CA) (DBLP:conf/podc/Gafni98, ) is a task, relaxing consensus. It
requires “liveness” only when all processors start with the same value.
Otherwise it should be safe, if one processor outputs, all adopt that output
value to be their new input value.

This paper brings to the fore the informal conjecture that Consensus is to
stationarity what CA is to mobility. One could have seen a glimpse of this in
the theoretically tainted pair Synchrony/Asynchrony. In the Byzantine case it
is Consensus vs. Realiable Broadcast (DBLP:journals/iandc/Bracha87, ). And
what is Reliable Broadcast if not shared-memory (appropriately defined) and
consequently CA (after some thought).The domain of models
Synchronous/Asynchronous for which the pair Consensus/Reliable Broadcast are
solvable, coincide i.e. $<n/3$.

There are two direction to the above conjecture. The easy direction is that
for any mobile setting in which binary CA is solvable, then consensus is
solvable in the stationary analogue. This follows from the construction in
(FOCS:BerGarPer89, ; FitMau98, ) of a consensus algorithm effectively made out
of CAs. The reverse direction is the heart of the conjecture. It for instance,
should imply: Take any Consensus algorithm for the stationary setting, if
executed in the analogue mobile setting and all processors start with the same
bit $b$, then they should all output $b$. We were not as yet successful in
proving this.

Rather, we went through case by case and showed the following: For all common
fault types, the number of faults Consensus can tolerate in the Stationary
case is the number of faults CA can tolerate in the analogue mobile case.

##  2\. Model

###  2.1. Consensus as a Task

A _task_ on $n$ processors is defined as a binary relation
$R\subseteq\mathcal{I}^{n}\times\mathcal{O}^{n}$ between $n$ tuples where
entries for the first tuple come from an _input space_ $\mathcal{I}$ and the
second tuple, the outputs come from an output space $\mathcal{O}$. The
interpretation of each $(x_{1},\ldots,x_{n})\in\mathcal{I}^{n}$ (resp.
$(y_{1},\ldots,y_{n})\in\mathcal{O}^{n}$) is that $x_{i}$ (resp. $y_{i}$) is
the input (resp. output) value of $p_{i}$.

For the consensus task, the corresponding relation, $R_{C}$, satisfies the
following, where every processor $p_{i}$ has input $v_{i}$:

  1. (1)

If a processor outputs $v$ then some processor must have input $v$

  2. (2)

If a processor outputs $v$ then no processor outputs $v^{\prime}\neq v$.

In this work we restrict our attention to binary consensus—i.e., the inputs
and output are bits.

Note that the above definition of consensus is different from the traditional
consensus definition in the Byzantine setting (PSL80, ; LSP82, ), which does
not consider the inputs and gives no guarantee on the outputs of
corrupted/Byzantine parties.

An interactive processor is a processor that in addition to local computation,
has the ability to communicate with other processors by sending them messages
over some network. A protocol among $n$ processors is a collection of $n$
interactive processor specifications.111We restrict our attention to non-
reactive tasks where processors receive only one input at the beginning and
produce a single local output (see below). This type of protocols is
sufficient for our results; however, one can extend this definition to
reactive tasks.

A model $M$ consists of two components, the system model and the communication
model. The system model specifies the types and capabilities of the
processors, along with the properties of the communication between them. The
failure (aka adversary) model specifies the types, e.g., fail-stop, omission,
byzantine, etc, and combination, e.g., up to $t$ faults, of different faults
that the processors might endure.

A task $T$ with corresponding relation $R_{T}$ is solvable in a model $M$ if
there exists a protocol in $M$ such that if processors start with some input
tuple, they all output from an output tuple relating, through $R_{T}$ , to the
input tuple.

###### Definition 0.

An $n$-processor task $T$ described by relation
$\mathcal{R}_{T}\subseteq\mathcal{I}^{n}\times\mathcal{O}^{n}$ is solvable in
a model $M$ if there exists a protocol $\Pi$ in $M$ such that
$\forall(x_{1},\ldots,x_{n})\in\mathcal{I}^{n}$, if every processor $p_{i}$
runs (his code of) $\Pi$ on input $x_{i}$, then every processor outputs
$y_{i}\in\mathcal{O}$ such that
$\left((x_{1},\ldots,x_{n}),(y_{1},\ldots,y_{n})\right)\in\mathcal{R}_{T}$.

##### Commit-Adopt

In addition to consensus we will use Commit-Adopt (CA) task
(DBLP:conf/podc/Gafni98, ). CA (also referred to as graded consensus
(TCC:ZikHauMau09, )) is similar in spirit to gradecast (STOC:FelMic88, ;
DBLP:conf/wdag/Ben-OrDH10, ; PODC:Aspnes10, ).

Its specification is as follows:

Every processor holds as input a value $v_{i}$. Every processor $p_{j}$
outputs a value which is either $commit(v_{j})$ or $adopt(v_{j})$ for some
value $v_{j}$, which equals the input of some processor.

  1. (1)

If all processors start with the same value $v_{i}=v$, then they all output
$commit(v)$,

  2. (2)

If a processor outputs $commit(v)$, then all processors output either
$commit(v)$, or $adopt(v)$.

In this work we focus our attention to binary CA where the input and output
values $v$ are bits.

In the remainder of this section, we describe the model(s) under
consideration. All models considered here share the same system-model
component, but are for different adversary models.

###  2.2. The System model

We assume $n$ interactive processors
$\mathcal{P}=\\{p_{1},p_{2},\ldots,p_{n}\\}$, also referred to as parties,
which can be interactive computing machines, e.g., Interactive Turing Machines
(ITM). As usual we will assume that the processors can perform polynomially
long (in $n$) computations (and can communicate polynomially long messages),
although our negative results even hold for unbounded processors. The
processors are connected by a complete reliable communication-network with a
dedicated channel between any two processors, a la LSP(LSP82, ; PSL80, ).
Here, reliable means that if $p_{j}$ receives a message from $p_{i}$ (on the
dedicated channel connecting the two processors) then $p_{j}$ knows that this
message was indeed sent by $p_{i}$, or by the adversary on behalf of
$p_{i}$—if $p_{i}$ was corrupted when the message was sent. We assume that the
protocol (and communication) is synchronous. In particular, the protocol
proceed in rounds, where in each round all processors send a (potentially
different) message to all other processors. All messages sent in any round
$\rho$ are delivered by the beginning of round $\rho+1$.

###  2.3. The (MAd-)Adversary Model

Recall that our definition of a protocol solving a task requires even
corrupted parties/processors to output a value and we give them the same
output guarantees (e.g., agreement in the context of consensus) as we give to
honest/uncorrupted parties. This makes our corruption model more suitable to
capture a mobile adversary who in the course of the protocol might corrupt
every processor.

To be able to achieve such stronger guarantees we consider adversaries that
operate at “the network interface” (e.g., communication tapes) of the parties,
rather than corrupting the parties’ internal state. This allows us to define
producing an output as writing it on a special write-only and append-only
output tape which is out of bounds for the adversary. Here is how our
adversary is defined.

We consider a central adversary who might affect messages sent by parties it
corrupts; we refer to such an adversary as a message adversary, in short MAd
(adversary). We note in passing that the notion of _corrupted processor_ at a
round is just for descriptive means of delineating the power of the MAd
Adversary. More concretely, a MAd adversary might intercept the outgoing
messages, rather than the internal state, of processors. In a nutshell, in
every round, each corrupted party prepares its messages for the current round,
according to the messages received from previous rounds, by following his
protocol instructions; depending on the privacy assumption on the underlying
communication model, a MAd adversary can tamper with theses messages.

To make the strongest possible statements, here we will consider the full
information model (DBLP:journals/jacm/FredericksonL87, ) whereby the adversary
gets to see all messages exchanged in the protocol. We will assume that,
subject to its constraints, the adversary is non-deterministic, and can
produce either garbage or the set of messages that will be most detrimental to
an operation of an algorithm. So rather than describing the adversary as an
algorithm operating in its history, we will describe simply which messages
cannot be non-deterministically produced given a corruption pattern. As an
example, unforgeability of signatures for keys inaccessible to the adversary,
e.g., keys of an uncorrupted party $p_{i}$, would correspond to restricting
the messages that the adversary might be able to inject to the protocol on
behalf of $p_{i}$ to those that are actually generated by $p_{i}$.

The adversary is described by means of when parties are corrupted—stationary,
mobile, or eventually stationary (mobile)—and by the corruption
types—omission, Byzantine, and Authenticated Byzantine, as discussed below.

##### The stationary MAd adversary

A stationary $t$-MAd Adversary is an adversary that can corrupt at most $t$
processor throughout an execution of a protocol.222This includes both static
and adaptive corruptions as defined in the cryptographic literature.

##### The mobile MAd adversary

The mobile $t$-MAd adversary is restricted to corrupting at most $t$
processors in a _round_. Thus over time all processor may experience message
tampering albeit in different round.

##### The eventually-stationary (mobile) MAd adversary

This is an adversary that for a finite (but unknown to the protocol) number
$\rho$ of rounds behaves as a mobile adversary, but from round $\rho+1$ on
becomes stationary. More concretely, an eventually-stationary $t$-MAd
adversary is an adversary that plays a mobile $t$-MAd adversary strategy for a
finite number of rounds and then chooses and from some point on confines its
corruption to a fixed set of at most $t$ processors.

####  2.3.1. MAd-Corruption Types considered in the paper

##### Send Omission failure

The adversary is restricted to just removing messages but it is not
constrained to remove all messages of a processor in rounds after it removed
some.

##### Byzantine failure

The adversary can tamper with messages replacing them with any of its own
choosing.

##### Authenticated Byzantine failure

Like Byzantine only that if at round $j$ processor $p_{i}$ was not corrupted,
then at subsequent rounds $k>j$ no processor can claim messages sent by
$p_{i}$ at round $j$ anything that has not really been sent, aside from just
pretending a message was not received. More formally, in the synchronous
authenticated Byzantine setting we will assume wlog that any message sent in a
protocol from $p_{i}$ in round $\rho$ has the formal $(p_{i},\rho,m)$ where
$m\in\\{0,1\\}^{*}$ is the contents of the message and $p_{i}$ and $\rho$ are
associated metadata. The authenticated byzantine adversary model mandates then
that if for any message $(p,\rho,m)$, where $p\in\mathcal{P}$, and for any
party $p^{\prime}\in\mathcal{P}$, $(p^{\prime},\rho^{\prime},m^{\prime})$ is
(an encoding of) a substring of $m$, where $\rho^{\prime}\leq\rho$, then
either $p^{\prime}$ was corrupted in round $\rho^{\prime}$ or he was
uncorrected and sent $(p^{\prime},\rho^{\prime},m^{\prime})$.

###### Remark 1 (Not giving up corrupted parties—even corrupted parties
produce outputs).

One might consider a natural MAd-analogue of standard Byzantine and omission
corruption to tamper with both incoming and outgoing communication. However,
defining things this way leads to complications with how broadcast and
consensus are defined. In particular, the traditional cryptographic
definitions give up corrupted parties, i.e., give no guarantees about the
output of corrupted parties. This means that not only these tasks cannot be
defined as simple functions taking only processor’s inputs into account but
have to also consider the adversary, but also, the definition might give up a
party that performs all its operations correctly, just because he is ”stained”
as being corrupted. Instead, here we we want to consider feasibility for the
task-based natural definition of consensus discussed above, where corrupted
processors are not discriminated. Clearly if the adversary can tamper or block
incoming communication, it is impossible to avoid corrupted parties from
outputting no output (e.g., $\perp$) or even a wrong output depending on the
setup and the adversary’s capabilities. For this reason, we restrict a MAd-
adversary to only tamper outgoing communication. Note that this adversary
might still inject messages as outgoing messages of corrupted parties.

##  3\. Cross-Model Reductions

Let $M$ be any of the above models in which in the stationary case Consensus
is solvable, and in the mobile case CA is solvable.

As a simple consequence of (PODC:Guerraoui00, ) we obtain an indulgent
protocol for consensus: Take any protocol $\Pi_{C}$ that solve consensus in
$M$ and any protocol $\Pi_{CA}$ that solves CA in the mobile analogue of $M$,
and run them alternately. A processor outputs when it commits in $\Pi_{CA}$.
Nevertheless, notice that processors in our models work forever (as usually
Consensus is to implement a ledger). Hence we have no notion of _halt_.

But indulgence is a motivating side show. We want to show that Consensus and
CA are twins, always solvable for the same $M$ with only stationery and
mobility, respectively.

One direction follows from the beginning of Section 4 below (Theorems 1 and
Corollary 2) and the fact that if a task is solvable in the mobile model it is
also solvable in the stationary.

Thus we get a theorem:

###### Theorem 1.

If CA is solvable in mobile model $M$, then Consensus is solvable in
stationary $M$ .

By applying (PODC:Guerraoui00, ) to the above theorem, we get the following
corollary.

###### Corollary 2.

If CA is solvable in mobile model $M$, then Consensus is indulgent in $M$.

We would like to have the theorem: If Consensus is solvable in $M$, then
binary CA is solvable in mobile $M$. We conjecture there is a way to prove
this generically, but for now we show it case by case.

##  4\. Commit-Adopt for a Stationary Adversary

As a warmup we start with stationary adversasry, i.e., the adversary corrupts
up to $t$ processors through the protocol and never changes his corruption.
Most of the results in this section can be easily obtained by existing
literature. Nonetheless, we include them here for completeness and to be able
to refer to them in the following section.

In order to establish the connection between consensus and commit-adopt we use
the following simple reduction from (Fit02, ; TCC:ZikHauMau09, ) (which in
terms relies on ideas from (FOCS:BerGarPer89, ; FitMau98, )). Let
$\mathtt{CA}$ be a protocol for commit-adopt in the stationary setting secure
against $t$ corruptions. Then the following simple phase king protocol, which
we refer to as $\mathtt{Consensus}_{Stat}$ allows us to construct consensus
out of $\mathtt{CA}$.

  * •

Let $x_{i}$ be the input of processor $p_{i}$. Every party sets
$temp_{i}:=x_{i}$

  * •

For $i=1,\ldots,n$

    1. (1)

The processors execute $\mathtt{CA}$ on inputs $temp_{1},\ldots,temp_{n}$;
Denote by $y_{i}$ the output of $p_{i}$ in $\mathtt{CA}$. By definition of
$\mathtt{CA}$, for each $p_{j}$, for some $b_{j}$ we have
$y_{j}\in\\{commit(b_{j}),adopt(b_{j})\\}$

    2. (2)

$p_{i}$ sends $b_{i}$ to every $p_{j}$ who denotes the received value as
$b_{i\rightarrow j}$.

    3. (3)

Each $p_{j}$ sets $temp_{j}:=\left\\{\begin{array}[]{l}b_{j}\text{ if
}y_{j}=commit(b_{j})\\\ b_{i\rightarrow j}\text{ otherwise}\end{array}\right.$

  * •

Every processor outputs $temp_{n}$

###### Theorem 1 ((Fit02, ; TCC:ZikHauMau09, )).

If $\mathtt{CA}$ solves Commit-Adopt in the stationary Byzantine MAd adversary
model then $\mathtt{Consensus}_{Stat}$ solves consensus in the same model.

It is straight-forward to verify that the above theorem applies verbatim to
send-omission corruption and the authenticated Byzantine setting. For
completeness we state this in the following corollary.

###### Corollary 2.

If $\mathtt{CA}$ solves Commit-Adopt in the stationary (send)-omission or
Authenticated Byzantine MAd adversary model then $\mathtt{Consensus}_{Stat}$
solves consensus in the corresponding model.

Furthermore, the following theorem follows trivially from the trivial
reduction of commit-adopt to consensus:

  1. (1)

The processors run consensus; denote by $y_{i}$ the output of processor
$p_{i}$

  2. (2)

Every $p_{i}$ outputs $commit(y_{i})$

###### Theorem 3.

If there exists a protocol for solving binary consensus in the stationary
(send)-omission, Byzantine, or Authenticated Byzantine MAd adversary model,
then there exists a protocol $\mathtt{CA}$ solving binary commit-adopt in the
corresponding model.

###  4.1. (Send)-Omission Corruption

A protocol for send-omission corruptions tolerating any number $t<n$ of
corrupted processors follows from (TCC:ZikHauMau09, ). This bound is trivially
tight. We note that the analogous positive result in our model—where every
party needs to output—follows directly from the corresponding bound for mobile
adversary (cf. Theorem 1). It following from Theorem 3 and Corollary 2 that
this bound is also tight for CA.

###### Theorem 4.

There exists a protocol for solving CA in the stationary (send)-omission t-MAd
adversary model if and only if $t<n.$

###  4.2. Byzantine

Lamport et al. (PSL80, ; LSP82, ) proved that byzantine consensus is possible
if and only if $t<n/3$ of the parties are corrupted. An efficient protocol for
this bound was later given by Berman et al. (FOCS:BerGarPer89, ). We will call
their protocol BGP-Consensus. Although their definition of consensus does not
give any guarantees on the output of corrupted processors, one can easily use
their protocol in a black-box way to add this guarantee, by adding and extra
round in which every party sends his output from BGP-Consensus to everyone,
and everyone outputs the value received by at least 2n/3 of the processors.
Since in BGP-Consensus all uncorrupted processors output the same value $v$,
all processors will receive $v$ for all of them and $v^{\prime}\neq v$ from at
most $t<n/3$ parties, so they will all output $v$. From the above and the
equivalence of consensus and commit-adopt in the byzantine model (Theorem 3
and Corollary 2) we get the following:

###### Theorem 5 ((FOCS:BerGarPer89, )).

There exists a protocol for solving CA in the stationary Byzantine $t$-MAd
adversary model if and only if $t<n/3.$

###  4.3. Authenticated Byzantine

In the Authenticated Byzantine setting a lower bound of $t<n/2$ corruptions
was proved by Fitzi (Fit02, , Proposition 3.1). A protocol for the
authenticated setting follows from the observation that, in the stationary
model, one can achieve our restrictions on the Authenticated Byzantine MAd
adversary by assuming (perfectly) unforgeable signatures, and having every
party digitally sign his messages using a standard existentially unforgeable
signatures scheme.333Indeed, our definition of Authenticated Byzantine is
equivalent to having an imaginary perfect signature scheme. Note that, in
reality, such signatures do not exists, hence in an actual realization, the
protocol will achieve consensus except with negligible probability.  Indeed,
under this implementation, the adversary will be unable to create any message
on behave of any uncorrupted processor.

The above observation implies that the folklore reduction of consensus to
broadcast for $t<n/2$ yields a consensus protocol in our model: Have every
party use Dolev-Strong broadcast (DolStr83, ) to broadcast his input to
everyone, and then take majority of the broadcasted values.

###### Theorem 6 ((Fit02, ; DolStr83, )).

There exists a protocol for solving binary CA in the stationary authenticated
Byzantine $t$-MAd adversary model if and only if $t<n/2.$

##  5\. Commit-Adopt for a Mobile Adversary

###  5.1. (Send)-Omission Corruption

We show that if in each round a MAd adversary can pick any $n-1$ processors
and drop any message of this processor it wishes, and then change to another
$n-1$ processors in the next round then nevertheless binary-CA is solvable.

The algorithm proceeds in two rounds:

  1. (1)

Every processor sends its input to everyone

  2. (2)

Every processor $p_{i}$: If there is a bit $b$ such that only $b$ was
received, then send $propose$-$commit(b)$ to everyone. Else send $no$-$commit$

  3. (3)

Every processor $p_{i}$: If for the bit $b$ only $propose$-$commit(b)$ was
received, then output $commit(b)$; else if for a unique $b$, $commit(b)$ was
received (by any party), output $adopt(b)$; otherwise output $adopt(0)$.

In each round there is at least one processor heard by all. Thus, after one
round where processors exchange inputs, a processor proposes commit $b$ in the
next round only if it heard just $b$. Since all heard the same bit from some
processor only a single bit can be proposed to be committed, in the next
round.

In the second round, a processor that only receives proposed commit $b$,
commits $b$, else if it receives proposed commit $b$, it adopts $b$. If a
processor committed then it received a proposed commit $b$ from the at least
one processor all hear from. thus, if one commits all receive proposed commit,
and thus will at least adopt.

###### Theorem 1.

There exists a protocol solving commit-adopt against an mobile MAD
$t$-adversary for any $t<n$.

###  5.2. Byzantine

It follows directly from Theorem 5 and the fact that a mobile adversary is at
least as strong as a stationary that at most $t<n/3$ Byzantine mobile
corruptions can be tolerated for commit-adopt. In the following we describe a
construction that meets this bound. Our commit-adopt protocol follows the
structure of the graded consensus construction from (Fit02, ; TCC:ZikHauMau09,
), where we use a weak consensus primitive.

#### Protocol
$\mathsf{WeakConsensus}(\mathcal{P},\vec{x}=(x_{1},\ldots,x_{n}))$

(1) Each $p_{i}\in\mathcal{P}$ sends $x_{i}$ to every $p_{j}$; $p_{j}$ denotes
the set of players who sent him $0$ (resp. $1$) as $P_{{j}}^{{}_{(0)}}$ (resp.
$P_{{j}}^{{}_{(1)}}$). (2) Each $p_{j}$ sets
$y_{j}:=\left\\{\begin{array}[]{cl}0&\text{ if
}|P_{{j}}^{{}_{(0)}}|>2n/3,\text{ else }\\\\[2.15277pt] 1&\text{ if
}|P_{{j}}^{{}_{(1)}}|>2n/3,\text{ else }\\\\[2.15277pt]
{\mbox{$\text{``{\sf\small n/v}''}$}}\end{array}\right.$
---

###### Theorem 2.

The protocol $\mathsf{WeakConsensus}$ satisfies the following properties
against an eventually-static mobile MAD $t$-adversary with $t<n/3$: (weak
consistency) There exists some $y\in\\{0,1\\}$ such that every
$p_{j}\in\mathcal{P}$ sets $y_{j}\in\\{y,{\mbox{$\text{``{\sf\small
n/v}''}$}}\\}$. (persistency) If every $p_{i}\in\mathcal{P}$ has the same
input $x$ then they all set $y_{j}:=y=x$. (termination) All parties set their
$y_{i}$ value after a single round

###### Proof.

(termination) Termination is trivial since all parties set the value of
$y_{i}$ at the end of their single round.

(weak consistency) Assume, that a player $p_{i}$ sets $y_{i}=0$. This means
that $y_{i}=0:|P_{{i}}^{{}_{(0)}}|>2n/3$. But since less than $1/3n$ of the
parties in $|P_{{i}}^{{}_{(0)}}|$ might be corrupted, this means that more
than 1/3 of the parties are uncorrupted during the protocol’s single round and
also send $0$ to every other $p_{j}$. Hence $|P_{{i}}^{{}_{(1)}}|\leq 2n/3$
which means that no $p_{j}$ will decide on $1$.

(persistency) If all non-actively corrupted players have input $0$ (the case
of pre-agreement on $1$ can be handled symmetrically) then every $p_{i}:\
\mathcal{P}$ receives $0$ from at least all those parties, i.e.,
$y_{i}=0:|P_{{i}}^{{}_{(0)}}|>2n/3$ and therefore outputs $0$. ∎

#### Protocol $\mathtt{CA}(\mathcal{P},\vec{x}=(x_{1},\ldots,x_{n}))$

(1) The players invoke $\mathsf{WeakConsensus}(\mathcal{P},\vec{x})$ and let
$y_{i}$ denote the value set by $p_{i}$ (note that
$y_{i}\in\\{0,1,{\mbox{$\text{``{\sf\small n/v}''}$}}\\}$). (2) Each
$p_{i}\in\mathcal{P}$ sends $y_{i}$ to every $p_{j}$. $p_{j}$ denotes the sets
of players who sent him $0,1$, and “n/v” as
$P_{{j}}^{{}_{(0)}},P_{{j}}^{{}_{(1)}}$, respectively (3) Each $p_{j}$ sets
$b_{j}:=\left\\{\begin{array}[]{ll}0&\text{ if
}|P_{{j}}^{{}_{(0)}}|\geq|P_{{j}}^{{}_{(1)}}|\\\ 1&\text{
otherwise}\end{array}\right.$
(4) Each $p_{j}$ output
$o_{j}:=\left\\{\begin{array}[]{ll}commit(b_{j})&\text{ if
}|P_{{j}}^{{}_{(z_{j})}}|>2n/3\\\ adopt(b_{j})&\text{
otherwise}\end{array}\right.$
---

###### Theorem 3.

The protocol $\mathtt{CA}$ solves commit-adopt against an mobile Byzantine
$t$-MAd adversary with $t<n/3$.

###### Proof.

We need to prove the following properties:

  * •

(Property 1) If for some $z\in\\{0,1\\}$ some processor $p_{i}\in\mathcal{P}$
outputs $commit(b)$ then every processor $p_{j}\in\mathcal{P}$ outputs
$o_{j}\in\\{commit(b),adopt(b)\\}$.

  * •

(Property 2) If every $p_{i}\in\mathcal{P}$ has the same input $x$ then they
output $commit(x)$.

The properties are proved in the following:

Property 1. Assume, that a player $p_{i}$ outputs $o_{j}=commit(0)$ (the case
of $o_{j}=commit(1)$ is handled symmetrically). This means that $p_{i}$
received $0$ in the second round from more than $2n/3$ parties, i.e.,
$|P_{{i}}^{{}_{(0)}}|>2n/3$. Since less than $n/3$ of these parties are
corrupted in Round $2$, this means that $p_{i}$ have received $0$ from more
than $n/3$ of the parties that were uncorrupted in Round 2, who therefore also
sent $0$ to every other $p_{j}$. Hence in Round 2 every $p_{j}$ has received
$0$ more than $n/3$ times, i.e., $|P_{{i}}^{{}_{(0)}}|>n/3$. Additionally,
since the message sent in Round 2 is the message that is set during Round 1
(i.e., the outcome of weak consensus) the output of parties from Round 1 must
have been in $\\{0,{\mbox{$\text{``{\sf\small n/v}''}$}}\\}$ and therefore no
party who is uncorrupted in Round 2 might sent $1$; hence, since there are at
most $n/3$ corrupted parties per round, $|P_{{j}}^{{}_{(1)}}|\leq
n/3<|P_{{j}}^{{}_{(0)}}|$ and every party sets $z_{j}=1$.

Property 2. If all non-actively corrupted players have input $0$ (the case of
pre-agreement on $1$ can be handled symmetrically) then by persistency of
$\mathsf{WeakConsensus}$ everyone sets $y_{i}=0$ in Round 1, hence in Round 2
every uncorrupted party sends $0$ and therefore every party $p_{i}$ received
$0$ from at least $2/3n$ times and therefore outputs $commit(0)$ ∎

###  5.3. Authenticated Byzantine

Again, it follows directly from Theorem 6 and the fact that a mobile adversary
is at least as strong as a stationary that at most $t<n/2$ Authenticated
Byzantine mobile corruptions can be tolerated for commit-adopt. In the
following we describe a construction that meets this bound.

We introduce a task that we call $t$-MAd-Authenticated-Byzantine-SM: Every
processor $p_{i}$ has an input value $x(i)$ from some domain $V$ and outputs a
vector $\vec{y}_{i}=(y(1)_{i},\ldots,y(n)_{i})$ such that each $y(j)_{i}\in
V\cup\\{\perp\\}$ and the following conditions hold:

  * •

There exists a set of indices $Ind\subseteq[n]$, with $|Ind|\geq n-t$ such
that for each $j\in Ind$, $y(j)_{i}=x(j)$ for all $i\in[n]$

  * •

if $p_{i}$ and $p_{j}$ output $y(\ell)_{i}\neq\perp$ and
$y(\ell)_{j}\neq\perp$, respectively, then $y(\ell)_{i}=y(\ell)_{j}$

#### Protocol $t-MAdABSM(\mathcal{P},\vec{x}=(x(1),\ldots,x(n)))$

(1) Every $p_{i}$ sends $x(i)$ to every $p_{j}$, who denotes the received
value as $v(i)_{j}$ ($v(i)_{j}$ is set to a default value, e.g., $0$, if no
value was received). Let $\vec{v}_{j}=(v(1)_{j},\ldots,v(n)_{j})$. (2) Every
$p_{i}$ sends $\vec{v}_{i}$ to every $p_{j}$. $p_{j}$ denotes the received
vector by $\vec{v}_{i\rightarrow j}=(v(1)_{{i\rightarrow
j}},\ldots,v(n)_{{i\rightarrow j}})$, where $\vec{v}_{i\rightarrow
j}:=\perp^{n}$ if $\vec{v}_{i\rightarrow j}\not\in V^{n}$ was received. (3)
Every $p_{j}$ and every $\ell\in[n]$: if for some $b\in V$ and some set
$I_{i}\subseteq[n]$ with $|I_{i}|\geq n-t$, $v(\ell)_{{i\rightarrow j}}=b$ for
all $i\in I_{i}$ and $v(\ell)_{{i\rightarrow j}}=\perp$ for all
$i\in[n]\setminus I_{i}$ then set $y(\ell)_{j}=b$ else set
$y(\ell)_{j}=\perp$. Output $\vec{y}=(y(1)_{j},\ldots,y(n)_{j})$.
---

###### Theorem 4.

Protocol $t-MAdABSM$ solves $t$-MAd-Authenticated-Byzantine-SM in the mobile
$t$-MAd Authenticated Byzantine adversary model for $t<n/2$.

#### Protocol $\tt CA(\mathcal{P},\vec{x}=(x(1),\ldots,x(n)))$

(1) Execute $t-MAdABSM(\mathcal{P},\vec{x})$; every processor $p_{j}$ denoted
its output as $\vec{z}_{j}$. (2) For each $p_{j}$: if there is a value
$v\neq\perp$ such that majority of the elements in $\vec{z}_{j}$ equal $v$,
then set $o(j):=propose.commit(v)$, else set $o(j):=propose.no.commit$. (3)
Execute $t-MAdABSM(\mathcal{P},\vec{o}=(o(1),\ldots,o(n)))$; every processor
$p_{j}$ denotes its output as $\vec{d}_{j}=(d(1)_{j},\ldots,d(n)_{j})$.  (4)
Every $p_{j}$: if for some bit $b$ the number of indices $\ell$ such that
$d(\ell)_{j}=commit(b)$ is more than $n/2$ then output $commit(b)$; else if
for some bit $b^{\prime}$: $|commit(b^{\prime})|>|commit(1-b^{\prime})|>0$
then $adopt(b^{\prime})$; else $adopt(x(j))$.
---

###### Theorem 5.

Protocol $\tt CA$ solves binary CA in the mobile $t$-MAd Authenticated
Byzantine adversary model for $t<n/2$.

###### Proof of Theorem 4 (sketch).

Consider processor $p_{i}$ which is not corrupted in line 1 (i.e. the first
round). All processor $p_{j}$ will have $v(i)_{j}=x(i)$; consequently since
minority is corrupted in the second round (line 2), then majority will send
$x(i)$ in the second round. On the other hand the processors corrupted in the
second round cannot forge $x(i)$, hence their value for $p_{i}$ will be $x(i)$
or $\bot$.

Now let $p_{i}$ be a processor corrupted in the first round. To output a value
different than $\bot$ (line 3) processor $p_{j}$ received same value majority
$v(i)_{k}$ from indices $k$ whose cardinality is at least a majority. At most
minority was corrupted out of this majority, thus this same value $v(i)_{k}$
is the only candidate value (not $\bot$ ) to be output by any processor for
the value $y(i)_{j}$ which is the same for all $j$.

∎

###### Proof of Theorem 5 (sketch).

First we show that if all start with the same value they all commit to this
value. By the definition of $t$-MAd-Authenticated-Byzantine-SM, all processors
will have a set $I$ with $|I|>n/2$ which return an input value which in this
case they are all the same. Consequently, all processors will propose to
commit this value $v$. Hence, by the property of $t$-MAd-Authenticated-
Byzantine-SM which is used the second time, the will all output $commit(v)$
for this value.

To complete the proof, we argue that if some processor $p_{i}$ outputs
$commit(v)$ then everyone outputs $adopt(v)$. Notice that only a single
$propose-commit(v)$ can be output in Line 2. This is because a propose commit
$v$ by $p_{i}$ requires that in a majority of indices in its output it has
value $v$. By the property of $t$-MAd-Authenticated-Byzantine-SM, no other
processor output conflicts with a different value on these majority indices,
then majority for another value for any other processor is impossible.

Assume now $p_{i}$ committed $v$. Then it returned from line 3 of its output
vector with majority $propose.commit(v)$. Let this majority value be $m$.
Since processors have no conflict on their output entries returning from
$t$-MAd-Authenticated-Byzantine-SM, it is easy to see that if another
processor $p_{j}$ has a $\bot$ in an entry in which $p_{i}$ has
$propose.commit(v)$, this is a result of $p_{j}$ having been corrupted in the
first round of the $t$-MAd-Authenticated-Byzantine-SM. Thus, for each index in
which $p_{i}$ has $propose.commit(v)$ for $p_{j}$ to miss it, the adversary
has to spend a corruption of the first round. Suppose $p_{j}$ has $q$ indices
in which $p_{i}$ output $propose.commit(v)$, and it output $\bot$ thus it has
another $t-q$ indices it can corrupt to have $propose.commit(\bar{v})$. Thus,
the number of $propose.commit(\bar{v})$ is $t-q$ which the number of
$propose.commit(v)$ it has is $m-q$. Since $m>t$ it will adopt $v$.

∎

##  6\. Conclusions

We have examined the Stationary/Mobile replacing Synchronous/Asynchronous and
shown that the former when considered in the MAd adversary model is indulgent
for common models unlike its counterpart.

We contend that there is a much richer clean distributed Theory when we
consider MAd adversary in the context of Stationary/Mobile as conjectured in
the paper. Showing that our case by case analysis was superfluous is a
beautiful challenge. Same goes for changing binary CA to multi-valued CA in
the mobile benign omission case.

Is beautiful theory with no current application worth developing? It is called
Basic Research (BR) and we still root for BR.

In (DBLP:journals/corr/DolevG16, ), Dolev and Gafni analyse mixtures of
Stationary and mobiles faults. It is interesting the re-examine in light of
this paper.

Finally, it will be nice to identify other natural “pairs” for which
indulgence can be defined.

## References

  * [1] James Aspnes.  A modular approach to shared-memory consensus, with applications to the probabilistic-write model.  In Andréa W. Richa and Rachid Guerraoui, editors, Proceedings of the 29th Annual ACM Symposium on Principles of Distributed Computing, PODC 2010, Zurich, Switzerland, July 25-28, 2010, pages 460–467. ACM, 2010.
  * [2] Michael Ben-Or, Danny Dolev, and Ezra N. Hoch.  Brief announcement: Simple gradecast based algorithms.  In Nancy A. Lynch and Alexander A. Shvartsman, editors, Distributed Computing, 24th International Symposium, DISC 2010, Cambridge, MA, USA, September 13-15, 2010. Proceedings, volume 6343 of Lecture Notes in Computer Science, pages 194–197. Springer, 2010.
  * [3] Piotr Berman, Juan A. Garay, and Kenneth J. Perry.  Towards optimal distributed consensus (extended abstract).  In 30th Annual Symposium on Foundations of Computer Science, Research Triangle Park, North Carolina, USA, 30 October - 1 November 1989, pages 410–415. IEEE Computer Society, 1989.
  * [4] Gabriel Bracha.  Asynchronous byzantine agreement protocols.  Inf. Comput., 75(2):130–143, 1987.
  * [5] Soma Chaudhuri.  Agreement is harder than consensus: Set consensus problems in totally asynchronous systems.  In Cynthia Dwork, editor, Proceedings of the Ninth Annual ACM Symposium on Principles of Distributed Computing, Quebec City, Quebec, Canada, August 22-24, 1990, pages 311–324. ACM, 1990.
  * [6] Danny Dolev and Eli Gafni.  Synchronous hybrid message-adversary.  CoRR, abs/1605.02279, 2016.
  * [7] Danny Dolev and H. Raymond Strong.  Authenticated algorithms for byzantine agreement.  SIAM J. Comput., 12(4):656–666, 1983.
  * [8] Cynthia Dwork, Nancy A. Lynch, and Larry J. Stockmeyer.  Consensus in the presence of partial synchrony (preliminary version).  In Tiko Kameda, Jayadev Misra, Joseph G. Peters, and Nicola Santoro, editors, Proceedings of the Third Annual ACM Symposium on Principles of Distributed Computing, Vancouver, B. C., Canada, August 27-29, 1984, pages 103–118. ACM, 1984.
  * [9] Paul Feldman and Silvio Micali.  Optimal algorithms for byzantine agreement.  In Janos Simon, editor, Proceedings of the 20th Annual ACM Symposium on Theory of Computing, May 2-4, 1988, Chicago, Illinois, USA, pages 148–161. ACM, 1988.
  * [10] Matthias Fitzi.  Generalized communication and security models in Byzantine agreement.  PhD thesis, ETH Zurich, 2002.
  * [11] Matthias Fitzi and Ueli M. Maurer.  Efficient byzantine agreement secure against general adversaries.  In Shay Kutten, editor, Distributed Computing, 12th International Symposium, DISC ’98, Andros, Greece, September 24-26, 1998, Proceedings, volume 1499 of Lecture Notes in Computer Science, pages 134–148. Springer, 1998.
  * [12] Greg N. Frederickson and Nancy A. Lynch.  Electing a leader in a synchronous ring.  J. ACM, 34(1):98–115, 1987.
  * [13] Eli Gafni.  Round-by-round fault detectors: Unifying synchrony and asynchrony (extended abstract).  In Brian A. Coan and Yehuda Afek, editors, Proceedings of the Seventeenth Annual ACM Symposium on Principles of Distributed Computing, PODC ’98, Puerto Vallarta, Mexico, June 28 - July 2, 1998, pages 143–152. ACM, 1998.
  * [14] Rachid Guerraoui.  Indulgent algorithms (preliminary version).  In Gil Neiger, editor, Proceedings of the Nineteenth Annual ACM Symposium on Principles of Distributed Computing, July 16-19, 2000, Portland, Oregon, USA, pages 289–297. ACM, 2000.
  * [15] Prasad Jayanti, Tushar Deepak Chandra, and Sam Toueg.  The cost of graceful degradation for omission failures.  Inf. Process. Lett., 71(3-4):167–172, 1999.
  * [16] Leslie Lamport, Robert E. Shostak, and Marshall C. Pease.  The byzantine generals problem.  ACM Trans. Program. Lang. Syst., 4(3):382–401, 1982.
  * [17] Rafail Ostrovsky and Moti Yung.  How to withstand mobile virus attacks (extended abstract).  In Luigi Logrippo, editor, Proceedings of the Tenth Annual ACM Symposium on Principles of Distributed Computing, Montreal, Quebec, Canada, August 19-21, 1991, pages 51–59. ACM, 1991.
  * [18] Marshall C. Pease, Robert E. Shostak, and Leslie Lamport.  Reaching agreement in the presence of faults.  J. ACM, 27(2):228–234, 1980.
  * [19] Nicola Santoro and Peter Widmayer.  Time is not a healer.  In Burkhard Monien and Robert Cori, editors, STACS 89, 6th Annual Symposium on Theoretical Aspects of Computer Science, Paderborn, FRG, February 16-18, 1989, Proceedings, volume 349 of Lecture Notes in Computer Science, pages 304–313. Springer, 1989.
  * [20] Vassilis Zikas, Sarah Hauser, and Ueli M. Maurer.  Realistic failures in secure multi-party computation.  In Omer Reingold, editor, Theory of Cryptography, 6th Theory of Cryptography Conference, TCC 2009, San Francisco, CA, USA, March 15-17, 2009\. Proceedings, volume 5444 of Lecture Notes in Computer Science, pages 274–293. Springer, 2009.


# A Stochastic Time Series Model for Predicting Financial Trends using NLP

Pratyush Muthukumar  Department of Computer Science, University of California,
Irvine  Jie Zhong  Department of Mathematics, California State University, Los
Angeles

###### Abstract

Stock price forecasting is a highly complex and vitally important field of
research. Recent advancements in deep neural network technology allow
researchers to develop highly accurate models to predict financial trends. We
propose a novel deep learning model called ST-GAN, or Stochastic Time-series
Generative Adversarial Network, that analyzes both financial news texts and
financial numerical data to predict stock trends. We utilize cutting-edge
technology like the Generative Adversarial Network (GAN) to learn the
correlations among textual and numerical data over time. We develop a new
method of training a time-series GAN directly using the learned
representations of Naive Bayes’ sentiment analysis on financial text data
alongside technical indicators from numerical data. Our experimental results
show significant improvement over various existing models and prior research
on deep neural networks for stock price forecasting.

##  1 Introduction

Stock market prediction has been a topic of interest among researchers,
corporations, and market enthusiasts for decades. While the lucrative and
prophetic nature of accurate stock market prediction is widely sought after,
it may be difficult to grasp the efficacy of financial forecasting given the
volatile nature of global stock markets. The latter sentiment is not new; in
Cowles 3rd, (1933), the US economist Alfred Cowles wrote that even the most
successful stock market forecasters did “little, if any, better than what
might be expected to result from pure chance”. However, with the rise of big
data systems and machine learning, we find that the perceived quality of
stochasticity in financial markets may actually contain a pattern embedded in
data that spans across financial sectors and fields of interest (Lobato and
Savin,, 1998).

Most machine learning approaches to predicting financial trends focus on an
analysis of numerical features and technical indicators of stocks. Various
machine learning models including Linear Regression, Logistic Regression,
Support Vector Machines (SVMs), Decision Trees, and Random Forests (RFs) have
been implemented to tackle this problem; see the survey paper (Kimoto et al.,,
1990) and references therein. More recently, deep learning models including
Artificial Neural Networks (ANNs), Recurrent Neural Networks (RNNs), and
Convolutional Neural Networks (CNNs) have also been used to predict stock
prices (Tsantekidis et al.,, 2017; Sokolov et al.,, 2020). The large majority
of these works utilize machine learning models trained on solely financial
numerical data. However, stock market prediction is an invaluable market
research strategy — stock prices are determined on the behavior of retail
investors who themselves base decisions on available news sources and
statistical measures. Thus, an entirely different category of feature
information can be learned through an analysis of textual data.

The impact of financial news and text data on the trends of the stock market
cannot be understated. The financial market is highly dependent on the
decisions retail investors take, which in turn, are dependent on the daily
news and information sources they read. Thus, by incorporating the information
of textual data to a financial forecasting model in addition to the technical
indicators available, a machine learning model can understand full-fledged
patterns of financial trends.

There has been previous work on utilizing Earnings Conference Calls (ECCs),
quarterly public briefings by a company’s upper management team, investment
team, and legal team. ECCs provide valuable insight into the financial health
of a company. Recent advancements in the field of Natural Language Processing
(NLP) allow state-of-the-art language models including Google BERT (Devlin et
al.,, 2019) and OpenAI’s GPT (Radford et al.,, 2019) series to be applied to
financial text data to learn word embeddings and representations which will
later be used to predict financial trends. These models rely on discerning the
sentiment of words or sentences of a financial textual article, which is
referred to as sentiment analysis in the field of NLP. Applying pretrained
language models to perform sentiment analysis is just one of the many machine
learning models that could be applied to solve this task; others include Naive
Bayes classification models, Convolutional Neural Networks, or Recurrent
Neural Networks; see the survey paper (Das et al.,, 2019) and references
therein.

Although we see great strides of innovation in machine learning models which
seek to predict financial trends through solely numerical features or solely
textual features as input, we seldom see a model that encompasses information
of both textual and numerical format as the basis for predicting the stock
market. Hence, we seek to utilize advanced machine learning models that
extract details from financial text sources and numerical indicators to
predict stock trends. We define our domain of interest to be the financial
market trends in the aerospace industry. The aerospace industry serves as the
perfect litmus test for the rest of the financial market, as the key public
and private corporations in this sector contributed to $151 billion in export
sales to the U.S. economy in 2018 alone. The aerospace industry is also
inherently global, and according to Dussauge and Garrette, (1995), forecasting
the outlook of this industry gives us a glimpse into the world economy as well
as the localized United States economy.

The challenge we faced when constructing such a hybrid model predominantly
stems from the question of effectively joining numerical and textual features
(or rather utilizing the output of a model trained on solely textual features
as an input for the one trained with numerical features). Additionally, it is
imperative that we construct a neural network architecture with the capability
to learn from data that may be sparse or less correlated, given the nature of
financial forecasting.

We propose a novel deep learning model in this paper which utilizes state-of-
the-art machine learning models including the Generative Adversarial Network
(GAN) to predict stock trends using financial text data and financial
numerical data. We invent a new method of directly using sentiment analysis
results on financial text data as an input to the time-series GAN. Our model
outperforms existing researched models in deep learning for financial
forecasting in various forecasting time horizons.

The remainder of the paper is structured as follows. Section 2 illustrates our
key contributions. Section 2.1 describes our model architecture. Section 3.1
and 3.2 outline our experimental testing methodologies. Section 3.3 describes
our model’s experimental results. Section 4 concludes our findings and
discusses our future work.

##  2 A Stochastic Time Series Model

Our proposed model provides the following technical contributions to the field
of financial forecasting with machine learning.

#### Robust Textual Understanding of Financial Data

We utilized state-of-the-art sentiment analysis techniques from the field of
NLP trained on continually updated data with high temporal resolution.
Moreover, we surpassed the standard depth of use for these sentiment analysis
techniques.

#### Advanced Time-Series Generative Prediction

We modified the increasingly popular Generative Adversarial Network (GAN) to
predict for data indexed over time through a generative process. Adel et al.,
(2018) has shown that generative models outperform traditional discriminative
models like Logistic Regression, Support Vector Regression, and Conditional
Random Fields in certain time-series tasks.

#### Novel Sequential Textual Embedding Technique

The novel aspect of our research is due to the hybridization of the modular
financial textual network and financial numerical network. To the best of our
knowledge, our technique is the first to directly include textual embeddings
to generate stock forecasts through our deep generative model. We invented a
new method to use the financial text sentiment analysis results as a latent
space input to the generator of the Generative Adversarial Network trained on
numerical features. Our results show a $32.2\%$ decrease in averaged NRMSE
error over multiple forecasting time horizons compared to the currently best
performing research utilizing other deep learning models for stock price
prediction.

###  2.1 Model Architecture

The structure of our two-stage model follows a sequential organization, where
the first unit of our model feeds as input to the second unit. The first
section of the model is a Naive Bayes’ classifier model to perform sentiment
analysis on our textual data inputs.

The goal of sentiment analysis is to systematically identify trends in textual
data, often word-by-word or sentence-by-sentence (Narayanan et al.,, 2009). We
performed both on a number of subsets of our textual data. The output of this
Naive Bayes’ classifier model is a set of sentiments, which is a sentiment
from the set {positive, neutral, negative}. It is evident why a classification
model works best for sentiment analysis, as the classifier predicts the
sentiment class for each sentence of a financial textual document.

The Naive Bayes’ algorithm is a fundamental generative model used widely in
machine learning. Tan et al., (2009) first proposed using Naive Bayes’
classification for sentiment analysis due to the relative independence of
words in a document. The goal of Naive Bayes sentiment analysis is to use
initial training sentiments of a set of individual words to predict the
sentiments of unknown words and thus the sentiment of each sentence in the
document. Specifically, given a set of word vectors
$X=\\{x_{1},x_{2},\dots,x_{k}\\}$, we predict $Y$, which is a label with $M$
possible classes. Models that seek to find
$\mathbb{P}(Y=y|X=\\{x_{1},x_{2},\dots,x_{k}\\})$ directly, are
discriminative. However, as a generative model, Naive Bayes’ classifier
reverses the conditioning,

$\mathbb{P}(Y|X)=\frac{\mathbb{P}(X|Y)\mathbb{P}(Y)}{\mathbb{P}(X)},$

by the classical Bayes formula.

Now, the most likely class of all possible classes $Y$ given the data $X$ is

$\displaystyle\arg\max_{y\in Y}\mathbb{P}(Y|X)$ $\displaystyle=\arg\max_{y\in
Y}\frac{\mathbb{P}(X|Y)\mathbb{P}(Y)}{\mathbb{P}(X)}$
$\displaystyle=\arg\max_{y\in Y}\mathbb{P}(X|Y)\mathbb{P}(Y)$
$\displaystyle=\arg\max_{y\in
Y}\mathbb{P}(x_{1},x_{2},\dots,x_{k}|Y)\mathbb{P}(Y).$

In practice, we find that calculating the probability
$\mathbb{P}(x_{1},x_{2},\dots,x_{k}|Y)$ is computationally infeasible, as we
would have required $M^{k}$ computations to find $\mathbb{P}(X|Y)$, where $k$
is the number of words in the document. Instead, we assume independence, i.e.,

$\mathbb{P}(x_{1},x_{2},\dots,x_{k}|Y)=\mathbb{P}(x_{1}|Y)\cdot\mathbb{P}(x_{2}|Y)\cdots\mathbb{P}(x_{k}|Y).$

Then, the most likely class becomes

$\arg\max_{y\in
Y}\mathbb{P}(x_{1},x_{2},\dots,x_{k}|Y)\mathbb{P}(Y)=\arg\max_{y\in
Y}\mathbb{P}(Y)\prod_{x_{k}\in X}\mathbb{P}(x_{k}|Y).$ (1)

The independence assumption here is plausible due to the nature of language:
individual words are not dependent on other words except for context and
grammar (Lewis,, 1998). Using independence, we perform the algorithm in linear
time: to compute the most likely class, we perform $M\cdot k$ computations. It
is important to note that the output of Naive Bayes’ sentiment analysis on a
document is an $n$-dimensional vector $v$, where each component
$v_{i}\in\\{-1,0,1\\}$ and $n$ denotes the number of sentences in the
document, assuming we are performing sentence-by-sentence Naive Bayes’
sentiment analysis. We feed this output vector $v$ as input to a second
section of our model in a novel procedure.

The second section in our model is a deep predictive model capable of
utilizing the numerical data and textual sentiment analysis embeddings to
identify patterns among the input data. To achieve this goal, we select the
Generative Adversarial Network (GAN) architecture. The GAN unit was developed
as a generative variant of the traditional Convolutional Neural Network used
for image-based deep learning (Goodfellow et al.,, 2014). GANs generate
samples similar to the training data through adversarial training.

The generative nature of the network allows GANs to be particularly effective
for financial data with multiple dimensions, low correlation, or sparsity
(Muthukumar and Zhong,, 2020). This is because GANs are inherently an
unsupervised learning model, since the predictive process of the network does
not require ground truth training data. Instead, the generator module of a GAN
uses randomly sampled input variables to create a prediction. In financial
forecasting, the generative model introduced in Krishnan et al., (2018)
provides benefits which allow for higher quality prediction on high
dimensional, sparse data.

The Generative Adversarial Network is comprised of two sub-networks: the
_generator_ and the _discriminator_. The generator is a neural network which
traditionally uses a fixed-length random vector as input to generate a sample.
The discriminator is another neural network which traditionally classifies
between training data and generated samples. It is important to note that the
generator is never exposed to the real world training data, as that is only
available to the discriminator. The generator and discriminator engage in a
two-player minimax game throughout the process of training, where the
discriminator seeks to minimize the discriminator loss or accurately classify
between a real and generated sample, while the generator seeks to maximize the
discriminator loss or “fool” the discriminator.

Let the generative network $G$ have a latent space input $x_{z}\sim p_{z},$
where $x_{z}$ acts as a random “seed” for the generator to sample. In a
traditional GAN, $x_{z}$ is a set of latent variables sampled from a prior
random noise distribution $p_{z}$ (Voynov and Babenko,, 2020). Let the
discriminative network $D$ have input either $x_{t}\sim p_{t}$ or $x_{g}\sim
p_{g}$ where $x_{t}$ and $x_{g}$ denote the ground truth training data and
generated samples respectively.

The loss function for the GAN, described in Goodfellow et al., (2014), is
derived from the binary cross entropy loss and is defined as

$E(G,D)=\frac{1}{2}(\mathbb{E}_{x\sim p_{t}}[1-D(x)]+\mathbb{E}_{x\sim
p_{g}}[D(x)]),$

where the goal of the algorithm is

$\max_{G}\bigg{(}\min_{D}\bigg{(}E(G,D)\bigg{)}\bigg{)}=\max_{G}\bigg{(}\min_{D}\bigg{(}\frac{1}{2}\big{(}\mathbb{E}_{x\sim
p_{t}}\big{[}1-D(x)\big{]}+\mathbb{E}_{x\sim
p_{g}}\big{[}D(x)\big{]}\big{)}\bigg{)}\bigg{)}.$

Intuitively, the discriminator seeks to minimize the likelihood of the
discriminator incorrectly classifying samples, while the generator seeks to
maximize this likelihood $E(G,D)$.

An oversight of the traditional GAN model is that the input of the generator
is sampled from a prior noise distribution without information from the
training dataset. Recently, Chen et al., (2016) proposed the model InfoGAN,
which alters the minimax loss function to include characteristics of the
random latent space input $x_{z}$. This allows the generator and discriminator
to iteratively improve the latent space variables, but the initialization of
the latent space variables remains as data points from a random sample.

We present a novel method of inputting relevant high-level embeddings as a
replacement to the random latent inputs $x_{z}$ used by the Generator model of
the GAN. The key idea of our work lies in the way in which the Naive Bayes’
sentiment analysis outputs are used as inputs for the Generative Adversarial
Network. Our architecture allows us to use high-level embeddings of financial
text data as an additional input when predicting for future stock trends on a
deep neural network with financial numerical data inputs. The Naive Bayes’
sentiment analysis on financial news texts produces an $n$-dimensional vector
$v$, where $n$ denotes the number of sentences in the text data document. We
use this vector $v$ as the latent variable “seed” $x_{z}$ in the GAN model,
which uses financial numerical data as the ground truth training data to
generate predictions. We build on the work from InfoGAN by not only using the
tweaked discriminator loss function that includes the latent variable $x_{z}$
as a tuneable parameter, but also directly initializing $x_{z}$ with the
$n$-dimensional vector $v$, instead of sampling from random noise distribution
$p_{z}$.

In the traditional GAN model supported in the PyTorch module, the default
value for the latent vector $x_{z}$ is a 100 dimensional vector created by
sampling 100 points from a standard normal distribution. For our model,
instead, we defined $x_{z}$ in our PyTorch GAN model using the Naive Bayes’
Sentiment Analysis vector output. To do so, we select the top 100 sentiment
analysis outputs that had the highest confidence probability. That is, the top
100 highest probabilities of the most likely class of all financial text
documents analyzed, defined as

$\arg\max_{y\in Y}\mathbb{P}(Y)\prod_{x_{k}\in X}\mathbb{P}(x_{k}|Y),$

are selected to form the 100 dimensional vector $x_{z}$. We then normalize the
resulting vector such that the mean of the points is zero and the standard
deviation is one. That is, for each $x_{z}^{(i)}\in x_{z},$

$x_{z}^{(i)}:=\frac{x_{z}^{(i)}-\mu_{x_{z}}}{\sigma_{x_{z}}},$

where $i=1,2,\dots,100$, $\mu_{x_{z}}$ is the mean of the elements in the
vector $x_{z}$, and $\sigma_{x_{z}}$ is the standard deviation of the elements
in $x_{z}$.

Thus, we have initialized our latent variables for the generator network of
the GAN model with the scaled vector of the top 100 strongest sentiments of
sentences in financial news documents for a stock. The GAN model continually
updates the initialized latent vector $x_{z}$ through the tweaked
discriminator network loss function

$E(G,D)=\frac{1}{2}(\mathbb{E}_{x\sim p_{t}}[1-D(x)]+\mathbb{E}_{x\sim
p_{z}}[D(x)]),$

as described in Chen et al., (2016).

In this way, we are able to ensure that meaningful representations from the
output of Naive Bayes’ sentiment analysis on financial texts are used as an
input to a Generative Adversarial Network trained on financial numerical data.
We also ensure that throughout the training process, the latent variable input
is fine-tuned for better prediction accuracies.

##  3 Experiments

###  3.1 Data Analysis

Figure 1: Visualization of Daily Historical Close Prices of 8 Aerospace Stocks

Our numerical dataset of financial indicators was sourced from the Yahoo
Finance historical stock price dataset. We select 8 stocks to apply in our
model, all of which represent a company in the aerospace industry. We focus on
the financial trends of the aerospace industry because this microcosm of the
financial sector accurately responds to the long-term and short-term effects
of current events globally (Bae and Duvall,, 1996). The eight stocks we select
are BA (Boeing), LMT (Lockheed Martin), AIR.PA (AirBus SE), RTX (Raytheon), GE
(General Electric), ERJ (Embraer SA), NOC (Northrup Grumman), and HON
(Honeywell). This portfolio of aerospace stocks is highly diverse and global,
with corporation headquarters in North America, South America, and Europe. We
use historical daily stock prices for all eight aerospace stocks from January
1, 2010 to March 6, 2020. We choose this timeframe as a decade worth of data
exhibits both short-term and long-term effects of current events on the
financial sector. The training data spans from January 1, 2010 to January 24,
2020. The testing data spans from January 25, 2020 to March 6, 2020. The stock
data includes the maximum price, minimum price, opening price, closing price,
adjusted closing price, and volume for each day. Figure 1 displays a
visualization of the daily closing prices in USD for the eight stocks
throughout the timeframe selected.

Figure 2: Non-aerospace Company Name Relative Sentiment Results for Context-
Based Sentiment Analysis on Boeing Financial News Documents

Our financial text dataset is collected from various financial news outlets.
We primarily use financial texts from Seeking Alpha, while also using texts
from Forbes, MarketWatch, and Twitter. We created a webscraper to collect and
download news articles where the headline contained at least one of the eight
names of the aerospace stocks we selected. The bulk of our financial texts are
in the form of Earnings Conference Calls (ECCs). The subject and keywords
spoken during these ECCs can have direct impacts on the future stock trends of
a company, so we analyze the sentiments of the ECCs using the Naive Bayes’
classifier.

For the financial text documents, in addition to applying the Naive Bayes’
classifier on each sentence to generate the latent vector input to the
generator network of the GAN model, we also perform other variants of
sentiment analysis, statistical calculations, and pre-processing techniques to
analyze the financial texts further. We invent and perform context-based
sentiment analysis on company names mentioned in the text documents that
aren’t the eight aerospace companies used we are predicting. Here, context-
based sentiment analysis is the technique in which we average the sentiment
values of words found around a specific keyword to determine its relative
sentiment.

Figure 3: Visualization of a Subset of the Financial Indicators Calculated
from the Raw Close Prices for each of the 8 Aerospace Stocks.

Figure 2 shows a visualization of the results from context-based sentiment
analysis on non-aerospace company names for Boeing financial text documents.
We can see that the relative sentiment values for companies like Korea
Electric Power Corp and Banco Santander SA are significantly higher than other
company names mentioned in financial news regarding Boeing stock. Intuitively,
these results show that when financial news articles include these company
names and Boeing stock together, they are mentioned in a positive view and
often lead to financial gain for Boeing.

We perform data pre-processing and compute various statistical features from
our raw numerical stock data in order to generate additional technical
indicators for our model. For example, we calculate the 7-day and 21-day
moving averages of all stocks for the timeframe we selected. We also calculate
the MACD or Moving Average Convergence Divergence by subtracting the 26-period
Exponential Moving Average (EMA) from the 12-period EMA. The general formula
for EMA is

$\text{EMA}=\text{Price}(t_{n})\times k+\text{EMA}(t_{n-1})\times(1-k),$

where $t_{n}$ and $t_{n-1}$ denotes the $n^{th}$ and ${n-1}^{th}$ timestep,
$k$ is defined as $2/(N+1)$ and $N$ denotes the number of timesteps in the EMA
period. We also calculate the upper and lower Bollinger bands by adding and
subtracting the 20-day rolling standard deviation from the 21-day moving
average, respectively. A visualization of the technical indicators for the
eight aerospace stocks over the dataset time period is shown in Figure 3.

We extract global and local trends in our stock prices using Fourier
transforms, which take a function and decompose it into various sine waves
that approximate the function. Fourier transforms are often used as a
financial forecasting tool, which convert time-series data to frequencies and
amplitudes of patterns. We decompose the historical close prices of the eight
stocks into various sine functions of various amplitudes and frames. The
Fourier transform of a function $g(t)$ is defined as

$G(\omega)=\int^{\infty}_{-\infty}g(t)e^{-i2\pi\omega t}dt.$

Figure 4 displays various Fourier transform sine wave function decompositions
for historical Boeing stock close prices.

Figure 4: Various Fourier Transform Sine Wave Decompositions for Historical
Daily Boeing Stock Close Price.

We also gathered additional numerical features by running an Auto Regressive
Integrated Moving Average (ARIMA) model to forecast future stock prices using
the historical stock price data. Although the forecasted stock prices were not
exceedingly accurate, we are able to include the auto-regressive parameters
for the best fit ARIMA model into our deep learning model. We find that an
ARIMA(5,1,0), or an ARIMA model with 5 autoregressive terms, first order
differencing, and no moving averages, performs the best with the given data.
We include information on the autocorrelation and partial autocorrelation
functions used to determine the optimal auto-regressive parameters into our
feature set as well.

###  3.2 Implementation

We create a Naive Bayes’ classifier to perform Naive Bayes’ sentiment analysis
on the vectorized representations of the financial text documents. We utilize
one-hot encoding to vectorize each sentence of the text dataset in order to
predict sentiments on a sentence-by-sentence level. The loss function we used
to optimize the most likely sentiment, or class, is defined in (1).

For the second component of our model, we used a Long-Short Term Memory (LSTM)
for the Generator network and a Convolutional Neural Network (CNN) for the
Discriminator network. The Generator architecture consists of an LSTM layer
with 128 input units for the 128 daily numerical features processed from
historical data for the eight aerospace stocks and a Dense layer with 1 output
to generate the stock price for a given time. The Discriminator architecture
consists of a 1-dimensional CNN with three Convolution layers and two Dense
layers with one output to generate the classification signal. The structure of
the GAN model is illustrated in Figure 5. The hyperparameter values for our
model after tuning is shown in Table 1.

Figure 5: Model Architecture of the LSTM Generator Network and CNN Discriminator Network of our GAN Model. Hyperparameter | Value
---|---
LSTM Weight Initializer | Xavier
LSTM Hidden Units | 500
LSTM Sequence Length (in days) | 30
LSTM Learning Rate | 0.01
LSTM Optimizer | Adam
LSTM Regularization | L1 (Lasso)
Conv1D Kernel Size | 5
Conv1D Stride | 2
Conv1D Padding | None
Conv1D Activation Function |  LeakyReLU ($\alpha=0.01$)
Dense Activation Function | ReLU
BatchNorm Momentum | 0.9
BatchNorm Epsilon | $1\times 10^{-5}$
Epochs | 500
Batch Size | 16
Table 1: Model Hyperparameter Values

###  3.3 Results

We seek to predict future stock prices for a variety of time frames using both
financial news texts and numerical features. We present results and error
margins for predicting future stock prices 1 day in the future, 15 days in the
future, and 30 days in the future. It is important to note that no additional
ground truth data is given within the 15 or 30 day prediction time frame, so
the model uses its own previous days’ predictions. By the nature of Long
Short-Term Memory (LSTM) models, our model provides predictions on a daily
time frequency, and this measure ensures that we are not performing data
leakage when predicting for 15 and 30 day forecast horizons. Our results show
that our model has the lowest average error and thus is the best performing
model among all baselines and existing deep learning models tested. To measure
the accuracy of our model, we used the Root Mean Square Error (RMSE) and
Normalized RMSE (NRMSE) error metrics. RMSE error is calculated as

$\text{RMSE}=\sqrt{\sum_{i=1}^{n}\frac{(\hat{y}_{i}-y_{i})^{2}}{n}},$

where $n$ is the number of observations, $\hat{y}$ is the predicted value, and
$y$ is the ground truth. NRMSE is calculated as

$\text{NRMSE}=\frac{\text{RMSE}}{\bar{y}},$

where $\bar{y}$ is the mean of the ground truth.

We included the NRMSE error metric in our error analysis because it allows us
to compare our model accuracies against other models of different scales and
prediction targets. Table 2 shows the RMSE and NRMSE results of our model on
various N-day forecast horizons where $N\in\\{1,15,30\\}.$

Metric | Model | N = 1 | N = 15 | N = 30
---|---|---|---|---
| ST-GAN | $\bf{0.16}$ | $\bf{2.39}$ | $\bf{4.37}$
| GAN | 0.74 | 11.74 | 20.41
| FC-LSTM | 0.41 | 6.13 | 13.24
RMSE | ARIMA(5,1,0) | 1.94 | 19.34 | 32.43
| Sentiment Analysis | 6.89 | 90.24 | 174.87
| GAN-FD | 0.28 | 3.41 | 8.25
| VolTAGE | 0.33 | 7.25 | 5.11
| DP-LSTM | 0.65 | 5.34 | 14.09
| ST-GAN | $\bf{0.00049}$ | $\bf{0.00751}$ | $\bf{0.01326}$
| GAN | 0.00229 | 0.03693 | 0.06193
| FC-LSTM | 0.00127 | 0.01928 | 0.04018
NRMSE | ARIMA(5,1,0) | 0.00600 | 0.06083 | 0.09841
| Sentiment Analysis | 0.02133 | 0.28383 | 0.53063
| GAN-FD | 0.00196 | 0.01114 | 0.02961
| VolTAGE | 0.00152 | 0.04241 | 0.01441
| DP-LSTM | 0.00162 | 0.00993 | 0.05114
Table 2: RMSE and NRMSE Error Values for $N$-day Forecast Horizons on Boeing
Stock

We show various baseline model error results against our model, _ST-GAN_ or
Stochastic Time-series Generative Adversarial Network. The baseline models
were all ran in the same experimental setting as described for our model. In
addition to the generic baselines (GAN, FC-LSTM, ARIMA, and Sentiment
Analysis), we also show results of running other existing models in stock
price forecasting in our experimental environment. We show RMSE and NRMSE
error values for the GAN-FD model proposed by Zhou et al., (2018) that uses a
GAN model with an LSTM Generator network and a CNN Discriminator network on
numerical stock price data. We also tested the VolTAGE model, proposed by
Sawhney et al., (2020), that uses a Graph Convolutional Network (GCN) model on
Earnings Conference Call data and historical stock prices in our experimental
setting. Finally, we tested the DP-LSTM model proposed by Li et al., (2019)
that uses a modified LSTM on time-series historical stock price and financial
news data.

Figure 7 shows a graph of the predicted stock prices for Boeing stock in a
30-day forecast horizon from January 25, 2020 to February 24, 2020. We show
the performance of the baseline models against our model, _ST-GAN_.

We also show the performance of our model against previous research models
only in Figure 7.

Figure 6: ST-GAN Performance Against All Experimental Models 30-day Forecast
Horizon Predicted Boeing Stock Price Graph (1/24/20 - 2/25/20)

Figure 7: ST-GAN Performance Against Previous Research Models 30-day Forecast
Horizon Predicted Boeing Stock Price Graph (1/24/20 - 2/25/20)

##  4 Conclusion

In this work, we have introduced a novel deep learning model that analyzes
financial news texts and financial numerical data to predict future aerospace
stock trends in the short and long term with unparalleled accuracies. We
proposed a novel architecture that applies Naive Bayes’ sentiment analysis to
financial news texts and uses the learned representations alongside financial
numerical data to train a Generative Adversarial Network (GAN) for time-series
prediction of stock prices. Our experimental results show that our model,
which utilizes cutting-edge technologies, may have a significant impact on the
practice of portfolio management.

Our error analysis using the RMSE and NRMSE error metrics shows significant
improvement over existing deep learning models in this field using similar
technologies. We believe our contributions can be applied to help researchers,
scholars, portfolio managers, investment officers, trustees, and consultants
make better decisions and predictions in the financial sector.

In the future, we expect to measure the isolated impact of our model’s robust
textual understanding from sentiment analysis on the overall prediction
accuracy. We also hope to increase the temporal frequency of our predictions
and expand our model prediction targets to a more diverse portfolio of stocks.

## References

  * Adel et al., (2018) Adel, T., Ghahramani, Z., and Weller, A. (2018).  Discovering interpretable representations for both deep generative and discriminative models.  In International Conference on Machine Learning, pages 50–59.
  * Bae and Duvall, (1996) Bae, S. C. and Duvall, G. J. (1996).  An empirical analysis of market and industry factors in stock returns of US aerospace industry.  Journal of Financial and Strategic Decisions, 9(2):85–95.
  * Chen et al., (2016) Chen, X., Duan, Y., Houthooft, R., Schulman, J., Sutskever, I., and Abbeel, P. (2016).  InfoGAN: Interpretable representation learning by information maximizing generative adversarial nets.  In Advances in neural information processing systems, pages 2172–2180.
  * Cowles 3rd, (1933) Cowles 3rd, A. (1933).  Can stock market forecasters forecast?  Econometrica: Journal of the Econometric Society, pages 309–324.
  * Das et al., (2019) Das, S. R., Kim, S., and Kothari, B. (2019).  Zero-Revelation RegTech: Detecting risk through linguistic analysis of corporate emails and news.  The Journal of Financial Data Science, 1(2):8–34.
  * Devlin et al., (2019) Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K. (2019).  BERT: Pre-training of deep bidirectional transformers for language understanding.  In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171–4186. Association for Computational Linguistics.
  * Dussauge and Garrette, (1995) Dussauge, P. and Garrette, B. (1995).  Determinants of success in international strategic alliances: Evidence from the global aerospace industry.  Journal of International Business Studies, 26(3):505–530.
  * Goodfellow et al., (2014) Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., and Bengio, Y. (2014).  Generative adversarial nets.  In Advances in neural information processing systems, pages 2672–2680.
  * Kimoto et al., (1990) Kimoto, T., Asakawa, K., Yoda, M., and Takeoka, M. (1990).  Stock market prediction system with modular neural networks.  In 1990 IJCNN international joint conference on neural networks, pages 1–6. IEEE.
  * Krishnan et al., (2018) Krishnan, R., Liang, D., and Hoffman, M. (2018).  On the challenges of learning with inference networks on sparse, high-dimensional data.  In International Conference on Artificial Intelligence and Statistics, pages 143–151. PMLR.
  * Lewis, (1998) Lewis, D. D. (1998).  Naive (bayes) at forty: The independence assumption in information retrieval.  In European conference on machine learning, pages 4–15. Springer.
  * Li et al., (2019) Li, X., Li, Y., Yang, H., Yang, L., and Liu, X.-Y. (2019).  DP-LSTM: Differential privacy-inspired lstm for stock prediction using financial news.  arXiv preprint arXiv:1912.10806.
  * Lobato and Savin, (1998) Lobato, I. N. and Savin, N. E. (1998).  Real and spurious long-memory properties of stock-market data.  Journal of Business & Economic Statistics, 16(3):261–268.
  * Muthukumar and Zhong, (2020) Muthukumar, P. and Zhong, J. (2020).  A stochastic time series model for predicting financial trends with NLP.  In Proceedings of the Southern California Machine Learning Symposium (SCMLS), page 68–71.
  * Narayanan et al., (2009) Narayanan, R., Liu, B., and Choudhary, A. (2009).  Sentiment analysis of conditional sentences.  In Proceedings of the 2009 conference on empirical methods in natural language processing, pages 180–189.
  * Radford et al., (2019) Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., and Sutskever, I. (2019).  Language models are unsupervised multitask learners.  OpenAI blog, 1(8):9.
  * Sawhney et al., (2020) Sawhney, R., Khanna, P., Aggarwal, A., Jain, T., Mathur, P., and Shah, R. (2020).  VolTAGE: Volatility forecasting via text-audio fusion with graph convolution networks for earnings calls.  In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 8001–8013.
  * Sokolov et al., (2020) Sokolov, A., Mostovoy, J., Parker, B., and Seco, L. (2020).  Neural embeddings of financial time-series data.  The Journal of Financial Data Science, 2(4):33–43.
  * Tan et al., (2009) Tan, S., Cheng, X., Wang, Y., and Xu, H. (2009).  Adapting naive bayes to domain adaptation for sentiment analysis.  In European Conference on Information Retrieval, pages 337–349. Springer.
  * Tsantekidis et al., (2017) Tsantekidis, A., Passalis, N., Tefas, A., Kanniainen, J., Gabbouj, M., and Iosifidis, A. (2017).  Forecasting stock prices from the limit order book using convolutional neural networks.  In 2017 IEEE 19th Conference on Business Informatics (CBI), volume 1, pages 7–12. IEEE.
  * Voynov and Babenko, (2020) Voynov, A. and Babenko, A. (2020).  Unsupervised discovery of interpretable directions in the GAN latent space.  arXiv preprint arXiv:2002.03754.
  * Zhou et al., (2018) Zhou, X., Pan, Z., Hu, G., Tang, S., and Zhao, C. (2018).  Stock market prediction on high-frequency data using generative adversarial nets.  Mathematical Problems in Engineering, 2018.