# Characterisations for split graphs and unbalanced split graphs111This
document is the results of the research project funded by the European Social
Fund (ESF).

Hany Ibrahim  University of applied science Mittweida, Germany

###### Abstract

We introduce a characterisation for split graphs by using edge contraction.
First, we use it to prove that any ($2K_{2}$, claw)-free graph with
$\alpha(G)\geq 3$ is a split graph. Next, we apply it to characterise any
pseudo-split graph. Finally, by using edge contraction again, we characterise
unbalanced split graphs, which we use to characterise Nordhaus–Gaddum graphs.

##  1 Introduction

Given graph $G$, we denote its vertex set by $V(G)$ ($V$ in short) if the
underlying graph is clear. Similarly, the edge set of $G$ is $E(G)$ ($E$ in
short). For vertex $v\in V$, the set of vertices adjacent to $v$ in $G$ is
denoted by $N(v)$. Further, for a set of vertices, $S\subseteq V$, the set of
vertices adjacent to $v$ in $S$ is denoted by $N_{S}(v)$ and defined as
$N_{S}(v)=N(v)\cap S$. The neighbour of $S$, denoted by $N(S)$, is
$\bigcup_{v\in S}N(v)\setminus S$. A vertex set, $S$, is called _dominating_
if $N(S)\cup S=V$. We denote a cycle graph, a complete graph, and an edgeless
graph by $C_{n}$, $K_{n}$, and $E_{n}$, respectively, where $n$ is the order
of the graph. Graph $G[S]$ has $S$ as the vertex set, and two vertices are
adjacent if and only if they are adjacent in $G$. $G[S]$ is called the
_vertex-induced subgraph_ of $G$. A _clique_ is a vertex set that induces a
complete subgraph in $G$. A set of mutually nonadjacent vertices is
_independent_. Furthermore, if two edges have no common vertex, they are
called _independent_. We denote a graph with four vertices and two independent
edges by $2K_{2}$.

Graph $G$ is called _$H$ -free_ if every vertex-induced (induced in short)
subgraph of $G$ is not isomorphic to graph $H$. Furthermore, $G$ is
_$(H_{1},H_{2},\dots,H_{k})$ -free_ if every vertex-induced subgraph of $G$ is
not isomorphic to any graph $H_{i}$, where $1\leq i\leq k$. For two disjoint
graphs, $G$ and $H$, the graph constructed from $G\cup H$ by adding edges from
any vertex in $G$ to any vertex in $H$ can be denoted by $G\vee H$.

By _identifying_ two adjacent vertices, $u$ and $v$, or _contracting_ the edge
between them, we obtain a graph constructed from $G$ by adding an edge between
$u$ and every neighbour of $v$. Then, $v$, along with all loops, is deleted,
followed by the deletion of all but one edge that forms multiple edges between
any two vertices. We denote the graph obtained by identifying $u$ and $v$ as
$G/uv$. If $e$ is the edge between $u$ and $v$, then we denote graph $G/uv$ by
$G/e$. In this study, we assume that any graph is a connected simple graph,
unless otherwise stated. For any other notion, we follow [2]. Long proofs are
divided into small claims in which only the ambiguous aspects are proven, for
example, Lemmas 7, Lemma 8 and Theorem 12.

The $KS$-partition of a graph is a partition of the vertex set, where $K$ is a
clique and $S$ is an independent set. Graph $G$ is called _split_ if it admits
a $KS$-partition. Split graphs were introduced in [8] and characterised as
follows:

######  Theorem 1.

[8] Graph $G$ is split if and only if $G$ is ($2K_{2}$, $C_{5}$,
$C_{4}$)-free.

In addition, split graphs were characterised in [8] as chordal graphs whose
complements are also chordal. Furthermore, these graphs were characterised by
their degree sequences in [10].

In Section 2, we present the characterisation of the split graphs. Then, in
Section 3, we use it to prove that any ($2K_{2}$, claw)-free graph with
$\alpha(G)\geq 3$ is a split graph. Finally, in Section 4, we characterise
unbalanced split graphs by using edge contraction, and we use this to
characterise Nordhaus–Gaddum graphs.

##  2 Split graphs characterisation

######  Proposition 2.

Let $G$ be a graph, $C\subseteq V(G)$ with $u\in C$ and $v\notin C$, where $u$
and $v$ are adjacent. If $N_{C}(v)\setminus\\{u\\}\subseteq N_{C}(u)$, then
$G[C]$ is isomorphic to $G/uv[C]$.

######  Proposition 3.

Let $G$ be a graph and $C\subseteq V(G)$. If $u$ and $v\notin C$, where $u$
and $v$ are adjacent, then $G[C]$ is isomorphic to $G/uv[C]$.

######  Proposition 4.

Let $G$ be a graph and $C\subseteq V(G)$. If $C$ is not dominant, then there
is an edge, $e\in E(G)$, such that $G[C]$ is isomorphic to $G/e[C]$.

###### Proof.

If $C$ is not dominant, then there is a vertex, $v\notin C$, that is not
adjacent to any vertex in $C$. Because $G$ is connected, there is a vertex,
$u\notin C$, such that $u$ is adjacent to $v$. Hence, according to Proposition
3, there exists an edge, $e\in E(G)$, such that $G[C]$ is isomorphic to
$G/e[C]$. ∎

######  Proposition 5.

If $G$ is a graph that is isomorphic to $C_{n}$ with $n\geq 4$, then $G/e$ is
isomorphic to $C_{n-1}$ for any edge, $e\in E$.

######  Proposition 6.

If $G$ is a graph that is isomorphic to $K_{n}$ with $n\geq 2$, then $G/e$ is
isomorphic to $K_{n-1}$ for any edge, $e\in E$.

Before we characterise the split graphs, we present a list of special graphs
in Figure 1. These graphs have an interesting property; although they are not
split graphs, we can construct one if we contract an edge in any of them. We
prove that these are the only graphs that possess such a property.

$H_{2}$$(W_{4})$$H_{3}$$(E_{2}\vee C_{4})$$\dots$$H_{1}^{l}$$(K_{2,l},l\geq
2)$$H_{4}$$(2K_{2})$$H_{5}$$(P_{5})$$H_{6}$$(hammer)$$H_{7}$$(butterfly)$
Figure 1:

######  Lemma 7.

Given graph $G$, if $G$ has an induced $C_{4}$, then either $G$ is isomorphic
to one of $\\{H_{1}^{l}:l\geq 2\\}$ , $H_{2}$, and $H_{3}$ or there is an
edge, $e\in E(G)$, such that $G/e$ has an induced $C_{4}$.

###### Proof.

Let $C=\\{p,q,r,s\\}\subseteq V(G)$ that induces $C_{4}$ with edges
$pq,qr,rs$, and $sp$.

Based on Proposition 3, we observe the following claim.

######  Claim 7.1.

If there are two adjacent vertices not in $C$, then there is an edge, $e\in
E(G)$, such that $G/e$ has an induced $C_{4}$.

By applying Proposition 4, the following claim is achieved.

######  Claim 7.2.

If $C$ is not dominant, then there is an edge, $e\in E(G)$, such that $G/e$
has an induced $C_{4}$.

By using Proposition 2, we obtain the following claim.

######  Claim 7.3.

If $v\notin C$ such that $|N_{C}(v)|=1$ or $v$ is adjacent to exactly two
adjacent vertices in $C$ or $|N_{C}(v)|=3$, then there is an edge, $e\in
E(G)$, such that $G/e$ has an induced $C_{4}$.

######  Claim 7.4.

If for every $v\notin C$, $v$ is adjacent to exactly the same two nonadjacent
vertices in $C$, then $G$ is isomorphic to $H_{1}^{l}$, where
$l=|V(G)\setminus C|+2$.

######  Claim 7.5.

If $|V(G)\setminus C|=1$, and the vertex not in $C$ is adjacent to all
vertices in $C$, then $G$ is isomorphic to $H_{2}$.

######  Claim 7.6.

If $|V(G)\setminus C|=2$, and each of the two vertices not in $C$ is adjacent
to all vertices in $C$, then $G$ is isomorphic to $H_{3}$.

######  Claim 7.7.

If $|V(G)\setminus C|=3$ and each of the three vertices in $V(G)\setminus C$
is adjacent to all vertices in $C$, then there is an edge, $e\in E(G)$, such
that $G/e$ has an induced $C_{4}$.

###### Proof.

If $\\{u,v,w\\}\subseteq V(G)\setminus C$ such that
$|N_{C}(u)|=|N_{C}(v)|=|N_{C}(w)|=4$, then $\\{p,r,v,w\\}$ induces $C_{4}$ in
$G/uq$. ∎

######  Claim 7.8.

If there are two vertices, $u$ and $v\notin C$, where
$|N_{C}(u)|=|N_{C}(v)|=2$ and $N_{C}(u)\cap N_{C}(v)=\\{\\}$, then there is an
edge, $e\in E(G)$, such that $G/e$ has an induced $C_{4}$.

###### Proof.

If $v$ is adjacent to exactly two adjacent vertices in $C$, then according to
Claim 7.3, there is an $e\in E(G)$ such that $G/e$ has an induced $C_{4}$.
Therefore, we assume that $G$ has no vertex that is adjacent to exactly two
adjacent vertices in $C$. Let $v\notin C$ be adjacent to exactly two
nonadjacent vertices in $C$, say $p,r$. If there is a vertex, $u\notin C$,
that is adjacent to two different nonadjacent vertices in $C$, say $q,s$, then
the vertex set, $\\{p,r,s,v\\}$, induces $C_{4}$ in $G/uq$. ∎

The following claim is proved in a similar way to the proof of Claim 7.8.

######  Claim 7.9.

If there are two vertices, $u$ and $v\notin C$, where $N_{C}(v)=2$ and
$N_{C}(u)=4$, then there is an edge, $e\in E(G)$, such that $G/e$ has an
induced $C_{4}$.

Based on Claims 7.1, $\dots$, 7.9, the proof is complete. ∎

######  Lemma 8.

Given graph $G$, if $G$ has an induced $2K_{2}$, then either $G$ is isomorphic
to one of $2K_{2}$, $P_{5}$, Hammer, Butterfly, and $C_{6}$ or there is an
edge, $e\in E(G)$, such that $G/e$ has an induced $2K_{2}$ or $C_{4}$.

###### Proof.

Let $C=\\{p,q,r,s\\}\subseteq V$, where $pq,rs\in E$, and $C$ induces $2K_{2}$
in $G$.

Based on Proposition 3, we observe the following claim.

######  Claim 8.1.

If there are two adjacent vertices not in $C$, then there is an edge, $e\in
E(G)$, such that $G/e$ has an induced $2K_{2}$.

By applying Proposition 4, the following claim is achieved.

######  Claim 8.2.

If $C$ is not dominant, then there is an edge, $e\in E(G)$, such that $G/e$
has an induced $2K_{2}$.

By using Proposition 2, we obtain the following claim.

######  Claim 8.3.

If $v\notin C$, such that $N_{C}(v)=1$ or $v$ is adjacent to exactly two
adjacent vertices in $C$, then there is an edge, $e\in E(G)$, such that $G/e$
has an induced $2K_{2}$.

######  Claim 8.4.

If $V=C$, then $G$ is isomorphic to $2K_{2}$.

######  Claim 8.5.

If $|V\setminus C|=1$ and the vertex in $V\setminus C$ is adjacent to exactly
two nonadjacent vertices in $C$, then $G$ is isomorphic to $P_{5}$.

######  Claim 8.6.

If $|V\setminus C|=1$ and the vertex not in $C$ is adjacent to exactly three
vertices in $C$, then $G$ is isomorphic to Hammer.

######  Claim 8.7.

If $|V\setminus C|=1$ and the vertex not in $C$ is adjacent to all vertices in
$C$, then $G$ is isomorphic to Butterfly.

######  Claim 8.8.

If there are two vertices, $u$ and $v\notin C$, such that
$|N_{C}(u)|=|N_{C}(v)|=2$, then $G$ is isomorphic to $C_{6}$ or there is an
edge, $e\in E(G)$, such that $G/e$ has an induced $2K_{2}$ or $C_{4}$.

###### Proof.

If $u$ or $v$ is adjacent to two adjacent vertices in $C$, then according to
Claim 8.3, there is an edge, $e\in E(G)$, such that $G/e$ has an induced
$2K_{2}$. Therefore, we assume that neither $u$ nor $v$ is adjacent to two
adjacent vertices in $C$. If $N_{C}(u)=N_{C}(v)$, say $N_{C}(u)=\\{p,r\\}$,
then vertex set $\\{p,r,u,v\\}$ induces $C_{4}$ in $G$. According to Lemma 7,
there is an edge, $e\in E(G)$, such that $G/e$ has an induced $C_{4}$. If
$|N_{C}(u)\cap N_{C}(v)|=1$, then there is a vertex set in $G$ that induces
$C_{5}$. Thus, according to Proposition 5, there is an edge, $e\in E(G)$, such
that $G/e$ has an induced $C_{4}$. If $N_{C}(u)\cap N_{C}(v)=\\{\\}$, say
$N_{C}(u)=\\{p,r\\}$ and $N_{C}(v)=\\{q,s\\}$, then vertex set
$\\{p,q,r,s,u,v\\}$ induces $C_{6}$ in $G$. If $w\notin C$, where $|N_{C}(w)|$
equals either $3$ or $4$, then $G$ has an induced $C_{4}$. Thus, $G$ is
isomorphic to $C_{6}$. ∎

######  Claim 8.9.

If there are two vertices, $u$ and $v\notin C$, such that
$|N_{C}(u)|=|N_{C}(v)|=3$, then there is an edge, $e\in E(G)$, such that $G/e$
has an induced $C_{4}$.

###### Proof.

For any two vertices, $u$ and $v\notin C$, such that
$|N_{C}(u)|=|N_{C}(v)|=3$, $G$ has an induced $C_{4}$ or $C_{5}$. Thus,
according to Lemma 7 and Proposition 5, there is an edge, $e\in E(G)$, such
that $G/e$ has an induced $C_{4}$. ∎

######  Claim 8.10.

If there are two vertices, $u$ and $v\notin C$, such that
$|N_{C}(u)|=|N_{C}(v)|=4$, then there is an edge, $e\in E(G)$, such that $G/e$
has an induced $C_{4}$.

###### Proof.

For any two vertices, $u$ and $v\notin C$, such that
$|N_{C}(u)|=|N_{C}(v)|=4$, $G$ has an induced $C_{4}$. Thus, according to
Lemma 7, there is an edge, $e\in E(G)$, such that $G/e$ has an induced
$C_{4}$. ∎

In a similar way to the proof of Claim 8.9, we can prove the following claim.

######  Claim 8.11.

If there are two vertices, $u$ and $v\notin C$, such that $|N_{C}(u)|=2$ and
$|N_{C}(v)|=3$, then there is an edge, $e\in E(G)$, such that $G/e$ has an
induced $C_{4}$.

In a similar way to the proof of Claim 8.10, we can prove the following two
claims.

######  Claim 8.12.

If there are two vertices, $u$ and $v\notin C$, such that $|N_{C}(u)|=2$ and
$|N_{C}(v)|=4$, then there is an edge, $e\in E(G)$, such that $G/e$ has an
induced $C_{4}$.

######  Claim 8.13.

If there are two vertices, $u$ and $v\notin C$, such that $|N_{C}(u)|=3$ and
$|N_{C}(v)|=4$, then there is an edge, $e\in E(G)$, such that $G/e$ has an
induced $C_{4}$.

Based on Claims 8.1, $\dots$, 8.13, the proof is complete. ∎

######  Theorem 9.

Given a connected graph, $G$, that is not isomorphic to any graph in Figure 1,
$G$ is split if and only if $G/e$ is split for any $e\in E(G)$.

###### Proof.

Let $G$ be a split graph with a $KS$-partition for $V$. An edge in $E(G)$ is
either connecting two vertices in $K$ or a vertex each in $K$ and $S$. Let
$u,v\in K$. According to Proposition 6, the order of the complete subgraph
induced by $K$ in $G$ decreases by $1$ in $G/uv$. Hence, we can partition
$V(G/uv)$ into $\\{K\setminus\\{v\\},S\\}$. Consequently, $G/uv$ is a split
graph. Let $e$ be an edge in $E(G)$ between a vertex in $K$ and a vertex in
$S$, say $v$. We can partition $V(G/e)$ into $\\{K,S\setminus\\{v\\}\\}$.
Thus, $G/e$ is a split graph.

Conversely, we prove that if $G$ is not split, then $G/e$ is not split for at
least an edge, $e\in E$. Consequently, and based on Theorem 1, we prove that
if $G$ is not split, then $G/e$ has an induced $2K_{2}$, $C_{5}$, or $C_{4}$.

According to Proposition 5, if $G$ has an induced $C_{5}$, then there is an
edge, $e\in E(G)$, such that $G/e$ is not split.

According to Lemma 7, if $G$ has an induced $C_{4}$, then either $G$ is
isomorphic to one of $\\{H_{1}^{l}$: $l\geq 2\\}$, $H_{2}$, and $H_{3}$ or
there is an edge, $e\in E(G)$, such that $G/e$ is not split.

According to Lemma 8, if $G$ has an induced $2K_{2}$, then either $G$ is
isomorphic to one of $2K_{2}$, $P_{5}$, Hammer, Butterfly or there is an edge,
$e\in E(G)$, such that $G/e$ is not split.

∎

Theorem 9 shows that we can partition the set of all graphs into the following
three parts:

  * 1.

The set of split graphs: the contraction of an edge in any graph in this set
constructs a split graph.

  * 2.

The set of graphs presented in Figure 1: these are non-split graphs in which
the contraction of an edge in any graph constructs a split graph.

  * 3.

The set of all non-split graph not presented in Figure 1: the contraction of
at least one edge in any graph in this set constructs a non-split graph.

This partition is presented in Figure 2.

Split graphs Graphs in Figure $1$ Non split graphs not in Figure $1$
$\forall_{e\in E}G/e$$\forall_{e\in E}G/e$$\exists_{e\in E}G/e$ Figure 2:

##  3 ($2K_{2}$, claw)-free graphs

A graph is called a _claw_ if it has four vertices and three edges with one
vertex adjacent to the other three. The _clique number_ of a graph is the
maximum cardinality of a clique in the graph. The _independent number_ of
graph $G$ is the maximum cardinality of an independent set in $G$ and is
denoted by $\alpha(G)$.

Moreover, graph $G$ is called _perfect_ if the clique number of $H$ equals its
chromatic number for any vertex-induced subgraph $H$ of $G$. A _hole_ is a
vertex-induced cycle in $G$ with a length of at least four. An _antihole_ is a
vertex-induced complement of a hole. In [5], it was shown that

######  Theorem 10.

A graph is perfect if and only if it contains neither an odd hole nor an odd
antihole.

In [3], it was proved that

######  Theorem 11.

If $G$ is connected ($2K_{2}$,claw)-free with $\alpha(G)\geq 3$, then $G$ is
perfect.

We use the characterisation in Theorem 9 to prove the following:

######  Theorem 12.

If $G$ is connected ($2K_{2}$,claw)-free with $\alpha(G)\geq 3$, then $G$ is
split.

###### Proof.

For the sake of contradiction, we assume that $G$ is a connected ($2K_{2}$,
claw)-free graph with $\alpha(G)\geq 3$, but it is not split. According to
Theorem 9, $G$ is either isomorphic to a graph in Figure 1: $H_{i}$, where
$1\leq i\leq 7$, or there is an edge, $e\in E$, where $G/e$ is not split.

######  Claim 12.1.

Graphs $H_{4},H_{5},H_{6}$, and $H_{7}$ are not $2K_{2}$-free.

######  Claim 12.2.

Graph $H_{1}^{l}$, where $l\geq 2$, is not $claw$-free if
$\alpha(H_{1}^{l})\geq 3$.

###### Proof.

The vertex set of $H_{1}^{l}$ is a union of set $C=\\{p,q,r,s\\}$ that induces
$C_{4}$ and $l-2$ vertices that are adjacent to exactly two nonadjacent
vertices in $C$, say $q,s$. If $\alpha(H_{1}^{l})\geq 3$, then $l\geq 3$. Let
$v\notin C$. Vertex set $\\{p,q,r,v\\}$ induces a claw. ∎

######  Claim 12.3.

Each graph $H_{2}$ and $H_{3}$ has no independent set with a cardinality
larger than $2$.

Based on Claims 12.1, 12.2, and 12.3, the following claim follows.

######  Claim 12.4.

If $G$ is a connected ($2K_{2}$,claw)-free graph with $\alpha(G)\geq 3$, then
$G$ is not isomorphic to any graph in Figure 1.

######  Claim 12.5.

If $G$ is $2K_{2}$-free, then $G/e$ is $2K_{2}$-free for any $e\in E(G)$.

######  Claim 12.6.

If $G/e$ has a vertex set, $C$, that induces $C_{4}$, then $G$ has an induced
$C_{4}$ or $C_{5}$. Further, if $G$ is $C_{5}$-free, then $G$ has either
vertex $v$ that is adjacent to exactly one vertex, two adjacent vertices, or
three vertices in $C$ or two adjacent vertices, $u$ and $v\in V\setminus C$.

######  Claim 12.7.

If $G$ is ($2K_{2}$, claw)-free with $\alpha(G)\geq 3$, then $G/e$ is
$C_{5}$-free for any $e\in E(G)$.

###### Proof.

For any edge, $e\in E(G)$, if $G/e$ has an induced $C_{5}$, then $G$ has an
induced $C_{5}$ or $C_{6}$. According to Theorem 11, $G$ does not have a
vertex set that induces $C_{5}$. Moreover, because $G$ is $2K_{2}$-free, $G$
has no vertex set that induces $C_{6}$. ∎

Let $C=\\{p,q,r,s\\}\subseteq V$ that induces $C_{4}$ in $G$ with edges
$pq,qr,rs$, and $sp$ .

######  Claim 12.8.

$C$ is dominating.

###### Proof.

For the sake of contradiction, we assume that $C$ does not dominate.
Therefore, there is a vertex, $v\notin C$, that is not adjacent to any vertex
in $C$. Because $G$ is connected, $v$ is adjacent to vertex $u\notin C$. If
$u$ is adjacent to no more than one vertex in $C$, say $p$, or exactly two
adjacent vertices in $C$, say $p,q$, then $\\{r,s,u,v\\}$ induces $2K_{2}$ in
$G$, and this is a contradiction. Otherwise, $u$ is adjacent to exactly either
two nonadjacent vertices in $C$, say $p,r$, or at least three vertices in $C$,
say $p,q,r$, and $\\{p,r,u,v\\}$ induces a claw in $G$, which is a
contradiction. ∎

######  Claim 12.9.

If $v\notin C$, such that $N_{C}(v)=1$ or $v$ is adjacent to exactly two
nonadjacent vertices in $C$, then $G$ is not claw-free.

###### Proof.

We assume that $v$ is adjacent to exactly either a vertex in $C$, say $p$, or
two nonadjacent vertices in $C$, say $p$ and $r$. Hence, $\\{p,q,s,v\\}$
induces a claw in $G$. ∎

######  Claim 12.10.

If $u$ and $v\notin C$, where $|N_{C}(u)|=|N_{C}(v)|=2$, then $u$ and $v$ are
adjacent.

###### Proof.

For the sake of contradiction, we assume that there are two vertices, $u$ and
$v\notin C$, where $|N_{C}(u)|=|N_{C}(v)|=2$ but $u$ and $v$ are not adjacent.
According to Claim 12.9, neither $u$ nor $v$ is adjacent to two nonadjacent
vertices in $C$. Thus, let $u$ be adjacent to $p$ and $q$. If $v$ is adjacent
to $p$ and $q$, then $\\{p,s,u,v\\}$ induces a claw in $G$, which is a
contradiction. Otherwise, $v$ is adjacent to $r$ or $s$; then, $\\{p,r,u,v\\}$
or $\\{q,s,u,v\\}$ induces $2K_{2}$ in $G$, which is a contradiction. ∎

######  Claim 12.11.

If $S$ is an independent set in $G$ with $|S|\geq 3$, then $S\cap C$ is empty.

###### Proof.

For the sake of contradiction, we assume that there is an independent set,
$S$, in $G$ with $|S|\geq 3$ and $S\cap C$ is not empty. If $|S\cap C|=2$,
then there is a vertex in $S$ that is adjacent to exactly one or two
nonadjacent vertices in $C$, which contradicts Claims 12.8 and 12.9.
Otherwise, $|S\cap C|=1$. Let $|S\cap C|=\\{p\\}$ and $u,v\in
S\setminus\\{p\\}$. According to Claims 12.8 and 12.10, at most one of $u$ and
$v$ is adjacent to exactly two adjacent vertices in $C$. If $u$ and $v$ are
adjacent to $q$ or $s$, then $\\{p,q,u,v\\}$ (or $\\{p,s,u,v\\}$) induces a
claw in $G$, which is a contradiction. ∎

######  Claim 12.12.

If $v\notin C$ and $|N_{C}(v)|=2$, then $v\notin S$, where $S$ is an
independent set in $G$ and $|S|\geq 3$.

###### Proof.

According to Claim 12.9, $v$ is not adjacent to two nonadjacent vertices in
$C$. Thus, for the sake of contradiction, we assume that there is a vertex,
$v\notin C$, that is adjacent to exactly two adjacent vertices in $C$, say $p$
and $q$, and $v\in S$, where $S$ is an independent set in $G$ with $|S|\geq
3$. According to Claim 12.11, $S$ does not have any vertex from $C$. Let $u$,
$w\in S\setminus\\{v\\}$. According to Claims 12.8 and 12.10 and because
$|N_{C}(v)|=2$, $N_{C}(u)$ and $N_{C}(w)$ are at least equal to 3. If $u$ and
$w$ are adjacent to $p$ (or $q$), then $\\{p,u,v,w\\}$ (or $\\{q,u,v,w\\}$)
induces a claw in $G$, which is a contradiction. Otherwise, w.l.o.g. $u$ is
adjacent to $p,r$, and $s$, and $w$ is adjacent to $q,r$, and $s$. Therefore,
$\\{p,r,v,w\\}$ induces $2K_{2}$ in $G$, which is a contradiction. ∎

######  Claim 12.13.

If $v\notin C$ and $|N_{C}(v)|=3$, then $v\notin S$, where $S$ is an
independent set in $G$ and $|S|\geq 3$.

###### Proof.

For the sake of contradiction, we assume that there is a vertex, $v\notin C$,
with $|N_{C}(v)|=3$ and $v\in S$, where $S$ is an independent set in $G$ and
$|S|\geq 3$. According to Claims 12.8, 12.11, and 12.12, if $u\in S$, then
$|N_{C}(u)|\geq 3$. Let $u,w\in S\setminus{v}$. According to the pigeonhole
principle, $|N_{C}(u)\cap N_{C}(v)\cap N_{C}(w)|\geq 1$. Let $p\in
N_{C}(u)\cap N_{C}(v)\cap N_{C}(w)$; then, $\\{p,u,v,w\\}$ induces a claw in
$G$, which is a contradiction. ∎

######  Claim 12.14.

If $v\notin C$ and $|N_{C}(v)|=4$, then $v\notin S$, where $S$ is an
independent set in $G$ and $|S|\geq 3$.

###### Proof.

For the sake of contradiction, we assume that there is a vertex, $v\notin C$,
with $|N_{C}(v)|=4$ and $v\in S$ where $S$ is an independent set in $G$ and
$|S|\geq 3$. According to Claims 12.8, 12.11, 12.12, and 12.13, if $u\in S$,
then $N_{C}(u)=4$. Then, any three vertices in $S$ and any vertex in $C$ form
a vertex set of cardinality $4$ that induces a claw in $G$, which is a
contradiction. ∎

Based on Claims 12.8, $\dots$, 12.14, we obtain the following claim.

######  Claim 12.15.

If $G$ is ($2K_{2}$,claw)-free with $\alpha(G)\geq 3$, then $G/e$ is
$C_{4}$-free for any $e\ inE(G)$.

Based on Claims 12.4, 12.5, 12.6, 12.7, and 12.15, the proof is complete. ∎

##  4 Unbalanced split graphs

A _star_ denoted by $S_{n}$ is a graph constructed by $E_{1}\vee E_{n}$, where
$n$ is a nonpositive integer. A split graph, $G$, is called _balanced split_
if $G$ has a $KS$-partition, where $|K|=\omega(G)$ and $|S|=\alpha(G)$, and
_unbalanced split_ , otherwise. Based on the work in [10], the following
theorem appears in [9] and its proof is presented in [4]:

######  Theorem 13 ([10],[9],[4]).

For any $KS$-partition of split graph $G$, exactly one of the following holds:
(i) $|K|=\omega(G)$ and $|S|=\alpha(G)$.
(ii) $|K|=\omega(G)-1$ and $|S|=\alpha(G)$.
(iii) $|K|=\omega(G)$ and $|S|=\alpha(G)-1$.
Moreover, in $(i)$, the $KS$-partition is unique.

######  Theorem 14.

Let $G$ be a split graph that is not isomorphic to any $S_{n}$ for $n\geq 2$.
Then, $G$ is an unbalanced split if and only if there is an edge, $e\in E(G)$,
such that $\omega(G/e)=\omega(G)-1$ and $G/e$ is an unbalanced split.

###### Proof.

Let $G$ be an unbalanced split graph, then $V(G)$ can be partitioned into a
$KS$-partition, where $|K|=\omega(G)-1$ and $|S|=\alpha(G)$. For any two
adjacent vertices, $u$ and $v\in K$, $V(G/uv)$ can be partitioned into
$K^{{}^{\prime}}S$-partitions, where $|K^{{}^{\prime}}|=\omega(G)-2$ and
$|S|=\alpha(G)$. Thus, $G/uv$ is an unbalanced split with
$\omega(G/uv)=\omega(G)-1$.

Conversely, we prove that if $G$ is a balanced split, then for any edge, $e$,
$G/e$ is a balanced split or $\omega(G/e)=\omega(G)$. Because $G$ is a
balanced split, then based on Theorem 13, $V(G)$ can be partitioned into the
unique $KS$-partition, where $|K|=\omega(G)$ and $|S|=\alpha(G)$. For any two
adjacent vertices, $u$ and $v\in K$, $V(G/uv)$ can be partitioned into
$K^{{}^{\prime}}S$-partitions, where$|K^{{}^{\prime}}|=\omega(G)-1$ and
$|S|=\alpha(G)$. Thus, either $G/uv$ is a balanced split with
$|K^{{}^{\prime}}|=\omega(G)-1$ or $G/uv$ is an unbalanced split with
$|K^{{}^{\prime}}|=\omega(G)$. For any two adjacent vertices, $u\in K$ and
$v\in S$, $V(G/uv)$ can be partitioned into $KS^{{}^{\prime}}$-partitions,
where $|K|=\omega(G)$ and $|S^{{}^{\prime}}|=\alpha(G)-1$. Thus,
$\omega(G/uv)=\omega(G)$. ∎

###  4.1 Pseudo-split graphs

A graph is called $G$ _pseudo-split_ if $G$ is ($2K_{2}$, $C_{4}$)-free. In
[1], the family of ($2K_{2}$, $C_{4}$)-free graphs was investigated and later
referred to as pseudo-split graphs in [11]. Different authors have
characterised pseudo-split graphs as follows.

######  Theorem 15 ([1] , [11]).

A graph is a pseudo-split if and only if its vertex set can be partitioned
into three sets, $A,B,$ and $C$, such that $A$ induces a clique, $B$ induces
an independent set, and $C$ induces $C_{5}$ or is empty, such that there are
all possible edges between $A$ and $C$, and there are no edges between $B$ and
$C$.

The following result is obtained from Theorems 9 and 15:

######  Theorem 16.

Given a connected graph, $G$, that is not isomorphic to any graph in Figure 1,
$G$ is pseudo-split if and only if

  * 1.

$G/e$ is split for any $e\in E(G)$ or

  * 2.

$V(G)$ can be partitioned into three sets, $A,B,$ and $C$, such that $A$
induces a clique, $B$ induces an independent set, and $C$ induces $C_{5}$,
where there are all possible edges between $A$ and $C$, and there are no edges
between $B$ and $C$.

###  4.2 Nordhaus–Gaddum graphs

A graph $G$ is called _Nordhaus–Gaddum_ (_$NG$_ in short) if
$\chi(G)+\chi(\bar{G})$ $=$ $|V(G)|$ $+$ $|V(\bar{G})|+1$. $NG$ graphs were
investigated and characterised in [7], [12], [6], and [4]. Based on the work
in [6], the authors in [4] proved the following characterisation for $NG$
graphs:

######  Theorem 17 ([4]).

Graph $G$ is an $NG$-graph if and only if $G$ is a pseudo-split but not a
balanced split.

In other words, the aforementioned theorem can be formulated as follows:

######  Theorem 18 ([4]).

Graph $G$ is an $NG$-graph if and only if

  * 1.

$G$ is unbalanced split or

  * 2.

$V(G)$ can be partitioned into three sets, $A,B,$ and $C$, such that $A$
induces a clique, $B$ induces an independent set, and $C$ induces $C_{5}$,
where there are all possible edges between $A$ and $C$, and there are no edges
between $B$ and $C$.

Based on Theorems 14 and 18, the following result is obtained:

######  Theorem 19.

Graph $G$ that is not isomorphic to any $S_{n}$ for $n\geq 2$ is an $NG$-graph
if and only if

  * 1.

there is an edge, $e\in E(G)$, such that $\omega(G/e)=\omega(G)-1$ and $G/e$
is unbalanced split, or

  * 2.

$V(G)$ can be partitioned into three sets, $A,B,$ and $C$, such that $A$
induces a clique, $B$ induces an independent set, and $C$ induces $C_{5}$,
where there are all possible edges between $A$ and $C$, and there are no edges
between $B$ and $C$.

##  5 Acknowledgments

The author would like to sincerely thank Peter Tittmann for his constructive
suggestions and discussions.

## References

  * Blázsik et al., [1993] Blázsik, Z., Hujter, M., Pluhár, A., and Tuza, Z. (1993).  Graphs with no induced $C_{4}$ and $2K_{2}$.  Discrete Mathematics, 115(1-3):51–55.
  * Bondy and Murty, [2000] Bondy, J. A. and Murty, U. S. (2000).  Graph theory.
  * Brause et al., [2019] Brause, C., Randerath, B., Schiermeyer, I., and Vumar, E. (2019).  On the chromatic number of $2{K}_{2}$-free graphs.  Discrete Applied Mathematics, 253:14–24.
  * Cheng et al., [2016] Cheng, C., Collins, K. L., and Trenk, A. N. (2016).  Split graphs and Nordhaus–Gaddum graphs.  Discrete Mathematics, 339(9):2345–2356.
  * Chudnovsky et al., [2006] Chudnovsky, M., Robertson, N., Seymour, P., and Thomas, R. (2006).  The strong perfect graph theorem.  Annals of mathematics, pages 51–229.
  * Collins and Trenk, [2013] Collins, K. L. and Trenk, A. (2013).  Nordhaus-Gaddum Theorem for the Distinguishing Chromatic Number.  The Electronic Journal of Combinatorics, 20(3):#P46.
  * Finck, [1968] Finck, H. (1968).  On the chromatic numbers of a graph and its complement.  Theory of graphs, pages 99–113.
  * Foldes and Hammer, [1977] Foldes, S. and Hammer, P. L. (1977).  Split graphs.  Proc. 8th southeast. Conf. on Combinatorics, graph theory, and computing; Baton Rouge 1977., pages 311–315.
  * Golumbic, [2004] Golumbic, M. C. (2004).  Algorithmic graph theory and perfect graphs.  Elsevier.
  * Hammer and Simeone, [1981] Hammer, P. L. and Simeone, B. (1981).  The splittance of a graph.  Combinatorica, 1(3):275–284.
  * Maffray and Preissmann, [1994] Maffray, F. and Preissmann, M. (1994).  Linear recognition of pseudo-split graphs.  Discrete Applied Mathematics, 52(3):307–312.
  * Starr and Turner III, [2008] Starr, C. L. and Turner III, G. E. (2008).  Complementary graphs and the chromatic number.  Missouri Journal of Mathematical Sciences, 20(1):19–26.


Further author information: (Send correspondence to G. B.)
G. B.: E-mail<EMAIL_ADDRESS>
# QSNET, a network of clocks for measuring the stability of fundamental
constants

G. Barontini  School of Physics and Astronomy, University of Birmingham,
Edgbaston, Birmingham B15 2TT, UK  V. Boyer  School of Physics and Astronomy,
University of Birmingham, Edgbaston, Birmingham B15 2TT, UK  X. Calmet
Department of Physics and Astronomy, University of Sussex, Brighton BN1 9QH,
UK  N. J. Fitch  Centre for Cold Matter, Blackett Laboratory, Imperial College
London, Prince Consort Road, London SW7 2AZ UK  E. M. Forgan  School of
Physics and Astronomy, University of Birmingham, Edgbaston, Birmingham B15
2TT, UK  R. M. Godun  National Physical Laboratory, Hampton Road, Teddington
TW11 0LW, UK  J. Goldwin  School of Physics and Astronomy, University of
Birmingham, Edgbaston, Birmingham B15 2TT, UK  V. Guarrera  School of Physics
and Astronomy, University of Birmingham, Edgbaston, Birmingham B15 2TT, UK  I.
R. Hill  National Physical Laboratory, Hampton Road, Teddington TW11 0LW, UK
M. Jeong  School of Physics and Astronomy, University of Birmingham,
Edgbaston, Birmingham B15 2TT, UK  M. Keller  Department of Physics and
Astronomy, University of Sussex, Brighton BN1 9QH, UK  F. Kuipers  Department
of Physics and Astronomy, University of Sussex, Brighton BN1 9QH, UK  H. S.
Margolis  National Physical Laboratory, Hampton Road, Teddington TW11 0LW, UK
P. Newman  School of Physics and Astronomy, University of Birmingham,
Edgbaston, Birmingham B15 2TT, UK  L. Prokhorov  School of Physics and
Astronomy, University of Birmingham, Edgbaston, Birmingham B15 2TT, UK  J.
Rodewald  Centre for Cold Matter, Blackett Laboratory, Imperial College
London, Prince Consort Road, London SW7 2AZ UK  B. E. Sauer  Centre for Cold
Matter, Blackett Laboratory, Imperial College London, Prince Consort Road,
London SW7 2AZ UK  M. Schioppo  National Physical Laboratory, Hampton Road,
Teddington TW11 0LW, UK  N. Sherrill  Department of Physics and Astronomy,
University of Sussex, Brighton BN1 9QH, UK  M. R. Tarbutt  Centre for Cold
Matter, Blackett Laboratory, Imperial College London, Prince Consort Road,
London SW7 2AZ UK  A. Vecchio  School of Physics and Astronomy, University of
Birmingham, Edgbaston, Birmingham B15 2TT, UK  S. Worm  School of Physics and
Astronomy, University of Birmingham, Edgbaston, Birmingham B15 2TT, UK

###### Abstract

The QSNET consortium is building a UK network of next-generation atomic and
molecular clocks that will achieve unprecedented sensitivity in testing
variations of the fine structure constant, $\alpha$, and the electron-to-
proton mass ratio, $\mu$. This in turn will provide more stringent constraints
on a wide range of fundamental and phenomenological theories beyond the
Standard Model and on dark matter models.

###### keywords:

variations of fundamental constants, atomic and molecular clocks, networks of
quantum sensors, dark matter, dark energy, physics beyond the Standard Model

##  1 Introduction

QSNET is aiming to create a world-leading programme to search for spatial and
temporal variations of fundamental constants using high-precision spectroscopy
in networked quantum clocks. The network will include existing Sr, Yb+ and Cs
atomic clocks at the National Physical Laboratory in London and several new
clocks that are currently being developed: a N${}_{2}^{+}$ molecular ion clock
at the University of Sussex, a CaF molecular optical lattice clock at Imperial
College London, and a Cf highly-charged ion clock at the University of
Birmingham. As the programme progresses, QSNET can be expanded and linked with
other clocks across the globe. An important objective of QSNET is to exploit
the networked approach, in which multiple quantum sensors will be linked,
having complementary sensitivities to changes of fundamental constants. In
particular, QSNET will achieve unprecedented accuracy in testing variations of
the fine structure constant, $\alpha$; and the electron-to-proton mass ratio,
$\mu$. Such variations could manifest as slow drifts, oscillations or
transient events, and all of them may be detectable by QSNET. This will allow
us either to discover new physics or to provide tighter constraints on e.g.
specific dark matter and dark energy models, soliton models and violations of
fundamental symmetries.

##  2 Theoretical background

Fundamental constants of nature are by definition assumed to be immutable in
time or space. However, since the late 1930’s, there have been speculations
[1, 2, 3, 4, 5], starting with Dirac’s large numbers hypothesis, that
fundamental constants could vary in time. The interest in such a time
evolution was revived some 20 years ago by astrophysical observations [6]
claiming a potential discovery that the fine structure constant was smaller in
the past at high redshift. However, this observation is not supported by other
groups see e.g. Chand et al. [7] and the situation remains open. A key issue
with astrophysical observation is that it is difficult to control all the
parameters. This is a very strong motivation to look for such effects using
earth based experiments. While astrophysical observations can probe effects of
time variation which accumulate over long periods of time i.e. some billions
of years (corresponding to redshifts of 3 to 5), measurements with clocks can
only be performed over periods of time corresponding to days or years.
However, clocks clearly surpass any astrophysical measurement of a potential
time evolution of fundamental constants in terms of precision. The unrivalled
precision of clocks provides us with an exceptional tool for measuring
variations of fundamental constants.

There are many theoretical motivations to consider a possible time evolution
of fundamental constants, see e.g. the review by Uzan [8]. Models can range
from those with extra-dimensions, to quintessence models and models of
extremely light dark matter. A time dependence of the coupling constants of
the Standard Model of particle physics can be parametrized by a scalar field
$\phi$ which couples to the electron $\psi_{e}$, light quarks ($u$, $d$ and
$s$-quarks) $\psi_{q}$, the photon $A_{\mu}$ or gluons $G^{a}_{\mu}$ according
to

$\displaystyle L$ $\displaystyle=$
$\displaystyle\kappa\phi\left(\frac{d^{(1)}_{e}}{4}F_{\mu\nu}F^{\mu\nu}-d^{(1)}_{m_{e}}m_{e}\bar{\psi}_{e}\psi_{e}\right)+\kappa\phi\left(\frac{d^{(1)}_{g}}{4}G_{\mu\nu}G^{\mu\nu}-d^{(1)}_{m_{q}}m_{q}\bar{\psi}_{q}\psi_{q}\right)$
(1)

with $\kappa=\sqrt{4\pi G_{N}}$,
$F_{\mu\nu}=\partial_{\mu}A_{\nu}-\partial_{\nu}A_{\mu}$ and
$G_{\mu\nu}=\partial_{\mu}G_{\nu}-\partial_{\nu}G_{\mu}-ig_{s}[G_{\mu},G_{\nu}]$
where $g_{s}$ is the QCD coupling constant and $G_{N}$ is the Newtonian
constant of gravitation. We could also add couplings to the field strength of
the neutrinos, heavier leptons and quarks, electroweak gauge bosons of the
standard model and to the Higgs bosons, but these particles usually do not
play an important role for very low energy tabletop experiments such as
clocks, so can be ignored at this stage. Note that these operators are
dimension 5 operators as they are suppressed by one power of the reduced
Planck scale $M_{P}=1/\sqrt{8\pi G_{N}}$.

For some applications, it might be necessary to consider scalar fields that
transform under some discrete, global or gauge symmetry in which case, the
simplest coupling to matter is given by dimension 6 operators

$L=\kappa^{2}\phi^{2}\left(\frac{d^{(2)}_{e}}{4}F_{\mu\nu}F^{\mu\nu}-d^{(2)}_{m_{e}}m_{e}\bar{\psi}_{e}\psi_{e}\right)+\kappa^{2}\phi^{2}\left(\frac{d^{(2)}_{g}}{4}G_{\mu\nu}G^{\mu\nu}-d^{(2)}_{m_{q}}m_{q}\bar{\psi}_{q}\psi_{q}\right).$
(2)

In other words, the interactions of the scalar field with stable matter are
suppressed by two powers of the reduced Planck scale. These are non-linear
couplings.

These simple Lagrangians can account for a variety of physical phenomena which
can be probed with the QSNET network, e.g. scalar dark matter models in which
case the magnitude of $\phi$ is related to the density of dark matter,
quintessence-like models [9], generic hidden sector scalar field [10], Kaluza-
Klein models [11], dilaton field models or Brans-Dicke fields, transient
phenomena, cosmic strings, and domain walls.

One expects on very general grounds that quantum gravity will generate an
interaction between any scalar field $\phi$ and regular matter with
$d_{j}^{(i)}\sim{\cal O}(1)$ whether such a coupling exists or not when
gravity decouples [12, 10, 13, 14, 15]. However, very light scalar fields
coupling linearly to regular matter (i.e. dimension five operators) are
essentially ruled out by the Eöt-Wash torsion pendulum experiment [16, 17, 18,
19] for $d_{j}^{(1)}\sim{\cal O}(1)$. Indeed, Eöt-Wash’s data implies that if
$d_{j}^{(1)}\sim 1$, the mass of the singlet scalar field must be heavier than
$10^{-2}$ eV, and thus to be relevant for clocks $d_{j}^{(1)}\ll 1$[12, 10,
13, 14, 15]. QSNET will therefore provide a very important test of quantum
gravity [12, 10, 13, 14, 15]. If a very light neutral scalar field with linear
coupling to regular matter was found with QSNET, we would learn that dimension
5 operators are not generated by quantum gravity. On the other hand, non-
linear couplings are far less constrained by current experiments and clocks
will be able to explore uncharted territory.

Besides quantum gravity, QSNET enables tests of grand unified theories [20,
21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31], as it allows us to measure at the
same time variations of $\alpha$ and $\mu$ using different clocks of the
network. In grand unified models, shifts in $\alpha$ and $\mu$ are related and
the functional dependence is very model dependent. QSNET could therefore
discriminate between models that predict a variation of the unified coupling
constant or a time-variation of the unification scale, or both. Finally, QSNET
can probe space-time symmetries such as Lorentz invariance, CPT, and also
probe models of space-time non-commutativity [32, 33]. QSNET thus enables
tests of cosmology, astrophysics and particle physics in a laboratory with
tabletop experiments.

##  3 The QSNET network

QSNET will include a range of clocks selected to maximize the sensitivities
$K_{\alpha}$ and $K_{\mu}$ to the variation of $\alpha$ and $\mu$. Here, we
summarize the clocks and give their sensitivity factors in atomic units.

  * •

A 133Cs microwave fountain clock. This clock is sensitive to changes in both
the fine structure constant ($K_{\alpha}=2.83$) and the electron-to-proton
mass ratio ($K_{\mu}=1$). There are several state-of-the-art Cs fountain
clocks around the world [34, 35, 36, 37, 38], reaching fractional frequency
uncertainties at the level of $1$–$2\times 10^{-16}$ limited by systematic
shifts.

  * •

A 87Sr optical clock. Clouds of about $10^{4}$ atoms can be trapped in an
optical lattice potential tuned close to a ‘magic wavelength’ at 813 nm, where
the differential polarisability between ground and excited states is zero. The
87Sr clock has very small sensitivity factors, $K_{\alpha}=+0.06$ and
$K_{\mu}=0$, useful when comparing against clocks with larger sensitivities.
The current state-of-the-art for the 87Sr optical lattice clock [39] has an
estimated fractional frequency uncertainty from systematic shifts of
$2.0\times 10^{-18}$.

  * •

A 171Yb+ optical clock. This clock features an octupole transition that is
very sensitive to variations of $\alpha$ with $K_{\alpha}=-5.95$. The current
state-of-the-art for the E3 transition [40] has an estimated fractional
frequency uncertainty from systematic shifts of $2.7\times 10^{-18}$.

  * •

A CaF molecular lattice clock. This clock will be based on the fundamental
vibrational transition in CaF, which has $K_{\mu}=0.5$. The main ideas for
such a clock were presented by Kajita [41]. The ultracold molecules will be
loaded into an optical dipole trap [42] or an optical lattice. Through a
careful choice of transition, Zeeman, dc Stark and ac Stark shifts all cancel
to high accuracy. In the optical lattice, the molecules are deep in the Lamb-
Dicke regime which eliminates first-order Doppler shifts. A 3D lattice also
eliminates collisional shifts. We estimate that systematic shifts can be
controlled at the level of $\simeq 8\times 10^{-18}$.

  * •

A molecular N${}^{+}_{2}$ clock. The vibrational clock transition has a
sensitivity of K${}_{\mu}=0.5$ and systematic shifts which are comparable with
the current best optical clocks [43] and facilitate frequency measurements at
an uncertainty below $10^{-18}$. The molecular ions need to be trapped
alongside atomic ions for sympathetic cooling and state detection. We evaluate
that the fractional frequency uncertainty from systematic shifts for
$N_{2}^{+}$ is $\simeq 4\times 10^{-18}$ under conditions that can be easily
reached in current experiments.

  * •

A Cf highly charged ion clock. Optical transition with values of
$|K_{\alpha}|\simeq 45$ are predicted to exist in the ionization states Cf15+
and Cf17+. Clocks based on highly-charged ions are expected to be less
sensitive to external perturbations, and with the information at hand, in
principle it would be possible to reach fractional frequency uncertainty on
the order of $10^{-19}$ for both ionization states [44]. Additionally, the
possibility of realizing a dual clock co-trapping Cf15+ and Cf17+ is
particularly appealing.

##  4 Summary

The QSNET project aims at realising a network of clocks with different
sensitivities to variations of $\alpha$ and $\mu$. Such a network will explore
large uncharted territories of the dark sector, and has the potential to
impose new constraints over many models and theories, widening our
understanding of the physics that governs the Universe. More specifically,
QSNET will be sensitive to slow drifts of $\alpha$ and $\mu$, with relevance
for, e.g., dark energy models and models that predict cosmological evolution
of fundamental constants, fast oscillations of $\alpha$ and $\mu$, that can be
linked to, e.g., virialised dark matter scalar fields, and transient events
due to kinks or topological defects in dark matter fields.

### Acknowledgments

This work was supported by STFC and EPSRC under grants ST/T00598X/1,
ST/T006048/1, ST/T006234/1, ST/T00603X/1,ST/T00102X/1,ST/S002227/1. We also
acknowledge the support of the UK government department for Business, Energy
and Industrial Strategy through the UK national quantum technologies
programme.

## References

  * [1] P. A. M. Dirac, Nature (London) 139, 323 (1937).
  * [2] P. A. M. Dirac, Proc. Roy. Soc. London A 165, 198 (1938).
  * [3] E. A. Milne, Relativity, Gravitation and World Structure, Clarendon Press, Oxford, (1935); Proc. Roy. Soc. A, 3, 242 (1937).
  * [4] P. Jordan, Naturwiss., 25, 513 (1937), Z. Physik, 113, 660 (1939).
  * [5] L. D. Landau, in W. Pauli, ed., Niels Bohr and the Development of Physics, McGraw-Hill, NY, p.52, (1955).
  * [6] J. K. Webb, et al., Phys. Rev. Lett. 87, 091301 (2001).
  * [7] H. Chand, R. Srianand, P. Petitjean and B. Aracil, Astron. Astrophys. 417, 853 (2004).
  * [8] J. P. Uzan, Living Rev. Rel. 14, 2 (2011).
  * [9] G. R. Dvali and M. Zaldarriaga, Phys. Rev. Lett. 88, 091303 (2002)
  * [10] X. Calmet, Phys. Lett. B 801, 135152 (2020)
  * [11] W. J. Marciano, Phys. Rev. Lett. 52, 489 (1984)
  * [12] X. Calmet, Eur. Phys. J. Plus 134, no.10, 503 (2019)
  * [13] X. Calmet and F. Kuipers, Eur. Phys. J. C 80, no.8, 781 (2020)
  * [14] X. Calmet and F. Kuipers, Phys. Lett. B 814 (2021), 136068
  * [15] X. Calmet and F. Kuipers, [arXiv:2107.13529 [hep-ph]].
  * [16] D. J. Kapner, et al., Phys. Rev. Lett. 98 (2007), 021101
  * [17] C. D. Hoyle, et al., Phys. Rev. D 70 (2004), 042004
  * [18] E. G. Adelberger, et al., Phys. Rev. Lett. 98, 131104 (2007)
  * [19] J. G. Lee, E. G. Adelberger, T. S. Cook, S. M. Fleischer and B. R. Heckel, Phys. Rev. Lett. 124, 101101 (2020)
  * [20] X. Calmet and H. Fritzsch, Eur. Phys. J. C 24 (2002), 639-642
  * [21] X. Calmet and H. Fritzsch, Phys. Lett. B 540 (2002), 173-178
  * [22] X. Calmet and H. Fritzsch, [arXiv:hep-ph/0211421 [hep-ph]].
  * [23] P. Langacker, G. Segre and M. J. Strassler, Phys. Lett. B 528 (2002), 121-128
  * [24] B. A. Campbell and K. A. Olive, Phys. Lett. B 345 (1995), 429-434
  * [25] K. A. Olive, M. Pospelov, Y. Z. Qian, A. Coc, M. Casse and E. Vangioni-Flam, Phys. Rev. D 66 (2002), 045022
  * [26] T. Dent and M. Fairbairn, Nucl. Phys. B 653 (2003), 256-278
  * [27] T. Dent, Nucl. Phys. B 677 (2004), 471-484
  * [28] S. J. Landau and H. Vucetich, Astrophys. J. 570 (2002), 463-469
  * [29] C. Wetterich, Phys. Lett. B 561 (2003), 10-16
  * [30] V. V. Flambaum and A. F. Tedesco, Phys. Rev. C 73 (2006), 055501
  * [31] X. Calmet and M. Keller, Mod. Phys. Lett. A 30 (2015) no.22, 1540028
  * [32] X. Calmet, Eur. Phys. J. C 41 (2005), 269-272
  * [33] C. E. Carlson, C. D. Carone and R. F. Lebed, Phys. Lett. B 518 (2001), 201-206
  * [34] S. Weyers, et al., Metrologia 6, 789 (2018)
  * [35] T. P. Heavner, et al., Metrologia 3, 174 (2014))
  * [36] J. Guéna, et al., IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control 59, 391 (2012)
  * [37] K. Szymaniec, et al., Journal of Physics: Conference Series 723, 012003 (2016)
  * [38] F. Levi, et al., Metrologia 3, 270 (2014)
  * [39] T. Bothwell, et al., Metrologia 6, 065004 (2019)
  * [40] C. Sanner, et al., Nature 7747, 204 (2019)
  * [41] M. Kajita, J. Phys. Soc. Jpn. 87, 104301 (2018)
  * [42] L. Anderegg, et al., Nat. Phys. 14, 890 (2018)
  * [43] M. Kajita, et al., Phys. Rev. A 89, 032509 (2014)
  * [44] S. G. Porsev, et al., Phys. Rev. A 102, 012802 (2020)


./Results

https://gitlab.mpi-magdeburg.mpg.de/goyalp/implicit_neuralodes

# Neural ODEs with Irregular and Noisy Data

Pawan Goyal<EMAIL_ADDRESS>Peter Benner  benner@mpi-
magdeburg.mpg.de

###### Abstract

Measurement noise is an integral part while collecting data of a physical
process. Thus, noise removal is necessary to draw conclusions from these data,
and it often becomes essential to construct dynamical models using these data.
We discuss a methodology to learn differential equation(s) using noisy and
irregular sampled measurements. In our methodology, the main innovation can be
seen in the integration of deep neural networks with the neural ordinary
differential equations (ODEs) approach. Precisely, we aim at learning a neural
network that provides (approximately) an implicit representation of the data
and an additional neural network that models the vector fields of the
dependent variables. We combine these two networks by constraining using
neural ODEs. The proposed framework to learn a model describing the vector
field is highly effective under noisy measurements. The approach can handle
scenarios where dependent variables are not available at the same temporal
grid. Moreover, a particular structure, e.g., second-order with respect to
time, can easily be incorporated. We demonstrate the effectiveness of the
proposed method for learning models using data obtained from various
differential equations and present a comparison with the neural ODE method
that does not make any special treatment to noise.

###### keywords:

Machine learning, deep neural networks, nonlinear differential equations,
neural ODEs, second-order systems, irregular data

  * •

This work blends neural networks with the neural ODE methods to learn
dynamical models from noisy measurements, as well as to obtain denoised data.

  * •

More precisely, two networks, one for an implicit representation of data and
the second one for describing dynamics, are combined by an integral form of
ODEs.

  * •

An extension to second-order ODEs is discussed.

  * •

The proposed methodology is capable of a situation when the dependent
variables are sampled irregularly. Moreover, they need not be measured on the
same irregular time grid.

##  1 Introduction

Uncovering dynamical models explaining physical phenomena and dynamic
behaviors has been active research for centuries 111For example, Isaac Newton
developed his fundamental laws based on measured data.. When a model
describing the underlying dynamics is available, it can be used for several
engineering studies such as process design, optimization, predictions, and
control. Conventional approaches based on physical laws and empirical
knowledge are often used to derive dynamical models. However, this is
impenetrable for many complex systems, e.g., understanding the Arctic ice pack
dynamics, sea ice, power grids, neuroscience, or finance, to only name a few
applications. Data-driven methods to discover models have enormous potential
to understand transient behaviors in the latter cases better. Furthermore,
data acquired using imaging devices or sensors are contaminated with
measurement noise. Therefore, systematic approaches that learn a dynamic model
with proper treatment of noise are required.

Towards this aim, the initial work [1] proposes a framework that explicitly
incorporates the noise into a numerical time-stepping method, namely a _Runge-
Kutta_ method. Though the approach has shown promising directions, its
scalability remains ambiguous as the approach explicitly needs noise estimates
and aims to decompose the signal explicitly into noise and ground truth.
Moreover, it requires that the Runge-Kutta method can give a reasonable
estimate at the next step. Additionally, irregular sampling (e.g., when
dependent variables are not even collected at the same time-grid) cannot be
applied, which can be highly relevant when information is gathered from
various sources, e.g., medical applications. This work discusses a deep
learning-based approach to learning a dynamic model by attenuating neural
networks with adaptive numerical integrations. It allows learning models to
represent the vector field accurately without estimating noise explicitly and
when dependent variables are arbitrarily irregularly sampled.

#### Our contributions:

Our work introduces a framework to learn dynamics models by innovatively
blending neural networks and numerical integration methods from noisy and
irregular measurements. Precisely, we aim at learning two networks; one that
approximately represents given measurement data implicitly, and the second one
approximates the vector field. We connect these two networks by enforcing an
integral form of ODEs as depicted in Figure 1. The appeal of the approach is
that we do not require an explicit estimate of noise to learn a model.
Furthermore, the proposed approach is applicable even if each dependent
variable is collected on a different time grid, which can be irregular.

Figure 1: The figure illustrates the framework for de-noise data and learning
a model describing underlying dynamics. For this, we determine an implicit
representation of data (approximately) by a network
$\mathcal{N}^{\textsf{I}}_{\theta}$ and another network for the vector field
$\mathcal{N}^{\textsf{Dyn}}_{\phi}$. These two networks are connected by
enforcing that the dynamics of the output of the implicit representation can
be given by $\mathcal{N}^{\textsf{Dyn}}_{\phi}$. Once the loss is minimized
(shown in (c)), we obtain an implicit network for de-noised data and a model
for the vector field $\mathcal{N}^{\textsf{Dyn}}_{\phi}$.

The remaining structure of the paper is as follows. In the next section, we
present a summary of relevant work. In Section 3, we present our deep
learning-based framework for learning dynamics from noisy measurements by
combining two networks. There, we also demonstrate the effectiveness of the
proposed methodology using various synthetic data with increasing noise
levels. Section 5 discusses the application of learning second-order dynamical
models. Moreover, in Section 6, we discuss how to handle irregular sampling of
measurements. We conclude the paper with a summary and future research
directions.

##  2 Relevant work

Data-driven methods to learn dynamic models have been studied for several
decades, see, e.g., [2, 3, 4]. Learning linear models from input-output data
goes back to Ho and Kalman [5]. There have been several algorithmic
developments for linear systems, for example, the eigensystem realization
algorithm [6, 7], and Kalman filter-based approaches [8, 9, 10]. Dynamic mode
decomposition has also emerged as a promising approach to construct models
from input-output data and has been widely applied in fluid dynamics
applications, see, e.g., [11, 12, 13]. Furthermore, there has been a series of
developments to learn nonlinear dynamic models. This includes, for example,
equations free modeling [14], nonlinear regression [15], dynamic modeling
[16], and automated inference of dynamics [17, 18, 19]. Utilizing symbolic
regression and an evolutionary algorithm [20, 21], learning compact nonlinear
models becomes possible. Moreover, leveraging sparsity (also known as sparse
regression), several approaches have been proposed [22, 23, 24, 25, 26, 27].
We also mention the work [28] that learns models using Gaussian process
regression. All these methods have particular approaches to handle noise in
the data. For example, sparse regression methods, e.g., [22, 23, 27] often
utilize smoothing methods before identifying models, and the work [28] handles
measurement noise as data represented like a Gaussian process.

Even though the aforementioned nonlinear modeling methods are appealing and
powerful in providing analytic expressions for models, they are often built
upon model hypotheses. For example, the success of sparse regression
techniques relies on the fact that the nonlinear basis functions, describing
the dynamics, lie in a candidate features library. For many complex dynamics,
such as the melting Arctic ice, the utilization of these methods is not
trivial. Thus, machine learning techniques, particularly deep learning-based
ones, have emerged as powerful methods capable of expressing any complex
function in a black-box manner given enough training data. Neural network-
based approaches in the context of dynamical systems have been discussed in
[29, 30, 31, 32] decades ago. A particular type of neural network, namely
recurrent neural networks, intrinsically models sequences and is often used
for forecasting [33, 34, 35, 36, 37] but does not explicitly learn the
corresponding vector field. Deep learning is also utilized to identify a
coordinate transformation so that the dynamics in the transformed coordinates
are almost linear or sparse in a high-dimensional feature basis, see, e.g.,
[38, 39, 40, 41]. Furthermore, we mention that classical numerical schemes are
incorporated with feed-forward neural networks to have discrete-time steppers
for predictions, see [31, 42, 43, 44]. The approaches in [31, 42] can be
interpreted as nonlinear autoregressive models [4]. A crucial feature of deep
learning-based approaches that integrates numerical integration schemes is
that vector fields are estimated using neural networks. Also, time-stepping is
done using a numerical integration scheme. Furthermore, in recent times,
_neural ordinary differential equations_ (Neural ODEs) in which neural
networks define the vector fields, have been proposed in [45], where it is
shown how to compute gradient with respect to network parameters efficiently
using adjoint sensitivities. As a result, one can utilize efficient black-box
numerical solvers to solve ODEs in a given time span using adaptive time-
stepping.

However, measurement data are often corrupted with noise, and these mentioned
approaches do not perform any specific noise treatment. The work in [1]
proposes a framework that explicitly incorporates the noise into a numerical
time-stepping method. Though the approach has shown a promising direction, its
scalability remains ambiguous. The approach explicitly needs noise estimates
by learning the decomposition of the signal into noise and ground truth. Also,
it relies on a Runge-Kutta scheme that can accurately estimate the variable at
the next step.

Furthermore, in scenarios where the data are collected on an irregular time
grid, the work [46] discussed a methodology by combining gated recurrent unit
(GRU) and neural ODEs. In the approach, an estimate for the initial condition
of (latent) ODEs is learned, and an ODE for the vector field is then
integrated using the estimated initial condition. However, long sequences are
quite challenging to estimate the initial condition given measurements future
in time. Although in [46], the measurements can be collected at an irregular
time grid, it still requires that all dependent variables are measured at the
same time grid. In the cases when each dependent variable is measured at a
different time-grid, then the approach [46] is not applicable.

##  3 Proposed Methodology—Implicit Networks Combined Neural ODEs

In this section, we discuss our framework to learn dynamical models using
noisy measurements without explicitly estimating noise. To achieve the goal,
we utilize the powerful approximation capabilities of deep neural networks and
their automatic differentiation feature with the neural ODEs approach [45],
which allows integrating a function, defining the vector field, with any
desired method and accuracy. For this, let us consider an autonomous nonlinear
differential equation:

$\dfrac{\texttt{{d}}}{\texttt{{d}}t}\mathbf{x}(t)=\mathbf{g}(\mathbf{x}(t)),\quad\mathbf{x}(0)=\mathbf{x}_{0},$
(1)

where $\mathbf{x}(t)\in\mathbb{R}^{n}$ denotes the solution at time $t$, and
the continuous function
$\mathbf{g}(\cdot):\mathbb{R}^{n}\rightarrow\mathbb{R}^{n}$ defines the vector
field. Furthermore, the solution $\mathbf{x}(t_{j})$ can be explicitly given
as:

$\mathbf{x}(t_{j})=\mathbf{x}(t_{i})+\int_{t_{i}}^{t_{j}}\mathbf{g}(\mathbf{x}(\tau))\texttt{{d}}\tau.$
(2)

Next, we further proceed to discuss our framework to learn dynamical models
from noisy measurements. The approach involves two networks. The first network
implicitly represents the variable as shown in Figure 1(b), and the second
network approximates the vector field, or the function $\mathbf{g}(\cdot)$.
These two networks are related by connecting the dependent variables at time
$t_{i}$ and $t_{j}$ as shown in (2). That is, the output of the implicit
network is not only in the vicinity of the measurement data but also its time-
evolution can be defined by $\mathbf{g}(\mathbf{x})$ or (2). To be
mathematically precise, let us denote noisy measurement data at time $t_{i}$
by $\mathbf{y}(t_{i})$. Furthermore, we consider a feed-forward neural
network, denoted by $\mathcal{N}^{\textsf{I}}_{\theta}$ parameterized by
$\theta$, that approximately yields an implicit representation of measurement
data, i.e.,

$\mathcal{N}^{\textsf{I}}_{\theta}(t_{i}):=\mathbf{x}(t_{i})\approx\mathbf{y}(t_{i}),$
(3)

where $i\in\\{1,\ldots,m\\}$ with $m$ being the total number of measurements.
Additionally, let us denote another neural network by
$\mathcal{N}^{\textsf{Dyn}}_{\phi}$ parameterized by $\phi$ that approximates
the vector field $\mathbf{g}(\cdot)$. We connect these two networks by
enforcing that the time-evolution of the output of the network
$\mathcal{N}^{\textsf{I}}_{\theta}$ can be described by
$\mathcal{N}^{\textsf{Dyn}}_{\phi}$, i.e.,

$\mathbf{x}(t_{i+1})\approx\mathbf{x}(t_{i})+\int_{t_{i}}^{t_{j}}\mathbf{g}(\mathbf{x}(\tau))d\tau,\quad\text{and}\quad\dfrac{\texttt{{d}}}{\texttt{{d}}t}\mathbf{x}(t_{i})\approx\mathcal{N}^{\textsf{Dyn}}_{\phi}\left(\mathbf{x}(t_{i})\right).$
(4)

As a result, our goal becomes to determine the network parameters
$\\{\theta,\phi\\}$ such that the following loss is minimized:

$\mathcal{L}=\lambda_{\texttt{MSE}}\cdot\mathcal{L}_{\texttt{MSE}}+\lambda_{\texttt{Integral}}\cdot\mathcal{L}_{\texttt{Integral}}+\lambda_{\texttt{Grad}}\cdot\mathcal{L}_{\texttt{Grad}},$
(5)

where

  * •

$\mathcal{L}_{\texttt{MSE}}$ denotes the mean square error of the output of
the network $\mathcal{N}^{\textsf{I}}_{\theta}$ and noisy measurements, i.e.,

$\left\|\mathcal{N}^{\textsf{I}}_{\theta}(t_{i})-\mathbf{y}(t_{i})\right\|^{2}_{F}.$
(6)

The loss enforces measurement data to in the vicinity of the output of the
implicit network, and $\lambda_{\texttt{MSE}}$ is its weighting parameter.

  * •

The term $\mathcal{L}_{\texttt{Integral}}$ links the two networks by comparing
the prediction, i.e.,

$\left\|\mathbf{x}(t_{j})-\mathbf{x}(t_{i})-\int_{t_{i}}^{t_{j}}\mathbf{g}(\mathbf{x}(\tau))d\tau\right\|^{2}_{F},$
(7)

and the parameter $\lambda_{\texttt{Integral}}$ defines its weight in the
total loss.

  * •

The vector field at the output of the implicit network can also be computed
directly using automatic differentiation, but it also can be computed using
the network $\mathcal{N}^{\textsf{Dyn}}_{\phi}$. The term
$\mathcal{L}_{\textsf{Grad}}$ penalizes its mismatch as follows:

$\left\|\mathcal{N}_{\phi}^{\textsf{Dyn}}(\mathbf{x}(t_{i}))-\dfrac{\texttt{{d}}}{\texttt{{d}}t}\mathbf{x}(t_{i})\right\|^{2}_{F},$
(8)

and $\lambda_{\texttt{Grad}}$ is its corresponding regularization parameter.

The total loss $\mathcal{L}$ can be minimized using a gradient-based optimizer
such as Adam [47]. Once the networks are trained and have found their
parameters that minimize the loss, we can generate the denoised variables
using the implicit network $\mathcal{N}_{\theta}^{\textsf{I}}$, and the vector
field by the network $\mathcal{N}_{\phi}^{\textsf{Dyn}}$. In the rest of the
paper, we denote the proposed methodology using Implicit–Neural ODEs (in sort
Imp-NODEs).

##  4 Numerical Experiments

We now present the performance of the approach discussed in Section 3 to de-
noise measurement data, as well as to learn a model for estimating the vector
field by means of an example. To that aim, we consider data obtained by
solving a differential equation that is then corrupted using white Gaussian
noise by varying the noise level. For a given percentage, we determine the
noise as follows:

$\nu\sim\mathcal{N}\left(0,\mu\right),\quad\text{with}\leavevmode\nobreak\
\leavevmode\nobreak\ \mu=\mathsf{Noise}\%.$

We have implemented our framework using the deep learning library PyTorch [48]
and have optimized both networks together using the Adam optimizer [47]. We
have used torchdiffeq [45], a Python package to integrate an ODE and to do
back-propagation to determine gradients, with the default settings.

Moreover, to obtain a good initial guess to employ the proposed Imp-NODEs, we
replace the integration term by its approximation using
$4^{\textsf{th}}$-order Runge-Kutta (RK4) method. It is computationally faster
because the RK4 method takes the fixed four calls of the function, defining
the vector field. We first train using this strategy for
$5\leavevmode\nobreak\ 000$ epochs, followed by training using an adaptive ODE
integration scheme within a given time span for $10\leavevmode\nobreak\ 000$
epochs for $\mu=\\{1\%,5\%\\}$, and for $1000$ epochs only for
$\mu=\\{10\%,\ldots,50\%\\}$ to do early stopping to avoid over-fitting. We
also make use of a learning scheduler, for which we reduced the learning rate
by one-tenth after each $4\leavevmode\nobreak\ 000$ epoches. The neural
network architecture design and hyperparameters are discussed in Appendix A,
and we have run all our experiments on a Nvidia® P100 GPU.

#### Cubic damped model:

We consider a damped cubic system, which is described by

$\displaystyle\dot{}\mathbf{x}(t)$
$\displaystyle=-0.1\mathbf{x}(t)^{3}+2.0\mathbf{y}(t)^{3},$ (9)
$\displaystyle\dot{}\mathbf{y}(t)$
$\displaystyle=-2.0\mathbf{x}(t)^{3}-0.1\mathbf{y}(t)^{3}.$

It has been one of the benchmark examples in discovering models using data,
see, e.g., [49, 27] but there, it is assumed that the dynamics can be given
sparsely in a high-dimensional feature dictionary. Here, we do not make any
such assumptions and instead learn the vector field using a neural network.
For this example, we take $2\leavevmode\nobreak\ 500$ data points in the time
interval $[0,25]$ by simulating the model using the initial condition $[2,0]$.
We add various noise levels in the clean data to have noisy measurements
synthetically. We construct neural networks for the implicit representation
and the vector field with the parameters given in Table 1.

We corrupt the data by adding mean-zero Gaussian white noise of variances
$\\{1\%,\ldots,50\%\\}$. We aim to obtain a denoised signal and a model,
defining its vector field. Before employing the method, we perform a pre-
processing step to noisy data using a low-pass filter to remove a large
portion of the high-frequency noise. We compare our methodology with the
neural ODE framework [45], which also focuses on learning a neural network,
defining the underlying vector field.

To train the implicit network, and the neural network for ODEs, we set
$\lambda_{\texttt{Integral}}=1.0$ and $\lambda_{\textsf{Grad}}=10^{-2}$ in the
loss function (5), and choose $\lambda_{\texttt{MSE}}=1.0$ for
$\mu=\\{1\%,5\%\\}$, and $\lambda_{\texttt{MSE}}=0.5$ for
$\mu=\\{10\%,20\%\\}$, and $\lambda_{\texttt{MSE}}=0.2$ for
$\\{30\%,40\%,50\%\\}$ to avoid over-fitting of noisy data using the implicit
network. Moreover, to integrate ODEs, we consider the time-span of
$\texttt{bs}\cdot\texttt{{dt}}$, where bs is treated as a batch-size and is
set to $4$ and $\texttt{{dt}}=10^{-2}$.

Having trained models, we plot the results in Figure 2, where the learned
vector fields from the proposed method (Imp-NODEs), and it is compared with
neural ODE [45] (Std-NODEs). Std-NODEs is also trained with the same
configurations and the same number of epochs as Imp-NODEs. It is clear from
the figures that Imp-NODEs is able to learn the underlying vector field
faithfully, whereas the Std-NODEs fails to identify the vector field
accurately. It is quite evident for higher levels of noise. Our approach
consists of an implicit network, aiming to generate denoised data whose
dynamics can be defined by a neural network. Thus, we plot the denoised data
obtained from the implicit network in Figure 3. It shows that using the
implicit network, we can obtain denoised data, close to the ground truth clean
data even for a high noise level, which, otherwise, is not possible by
employing solely Std-NODEs.

True vector field (VF) Imp-NODEsErrors b/w VFsStd-NODEs [45]Errors b/w VFs

Noise $1\%$

Noise $5\%$

Noise $10\%$

Noise $20\%$

Noise $30\%$

Noise $40\%$

Figure 2: Cubic2D example: A comparison of vector fields of the ground-truth
and learned models for various noise. The first column shows the ground truth
vector field; the second and fourth columns show the learned vector fields
from the Imp-NODEs and Std-NODEs, respectively; the third and fifth columns
present the errors between the learned and the ground truth vector fields.

Ground truthNoisy measurementsPre-processed De-noised from network

Noise $1\%$

Noise $5\%$

Noise $10\%$

Noise $20\%$

Noise $30\%$

Noise $40\%$

Figure 3: Cubic2D example: Denoised data from the trained implicit network for
various noise levels. The plots contain the ground truth, noisy measurements,
pre-processed data using a low-pass filter, and denoised data from the
implicit network.

We further compute the mean and median errors between the learned vector
fields and the ground truth for a quantitative comparison of these approaches
to learning vector fields. For batch size $\texttt{bs}=4$, we plot these
errors in Figure 4(a), showing the robustness of the proposed approach with
respect to the noise levels. Furthermore, we report the performance of both
approaches by changing the batch size, meaning by varying the time span for
integration (for batch size bs, the time span is
$\texttt{bs}\times\texttt{{dt}}$) for the corrupted data with $5\%$ noise. For
this study, we again plot the mean and medium errors between the learned and
ground truth vector fields in Figure 4(b). We notice that the errors are of
the same order with respect to the batch size for Imp-NODEs and perform better
than Std-NODEs in both mean and median errors for all batch sizes. We
highlight that although the errors are of the same order with respect to bs,
we notice a substantial increase in computational cost because of a large
integration time span.

Based on these studies, we empirically conclude that an implicit network looks
at generating data in the vicinity of the measurements, which can be
integrated by an ODE defined by a neural network.

(a) Mean and median error of the vector fields for Imp-NODEs and Std-NODEs for
various levels of noise.

(b) Mean and median error of the vector fields for Imp-NODEs and Std-NODEs for
different batch size.

Figure 4: Cubic2D example: Comparisons of the error between the learned and
ground truth vector fields for different noise levels and for different batch
sizes.

##  5 Second Order Neural ODEs for Noisy Measurements

Several dynamics of engineering process, particularly in electrical and
mechanical ones are of second-order, which can be given as follows:

$\ddot{\mathbf{x}}(t)=\mathbf{f}(\mathbf{x}(t),\dot{\mathbf{x}}(t)),$ (10)

where $\dot{\mathbf{x}}(t)$ and $\ddot{\mathbf{x}}(t)$ denote the first and
second derivatives of $\mathbf{x}(t)$ with respect to $t$, respectively. As
discussed in [50], it is advantageous to consider the companion first-order
system of (10) which is

$\begin{bmatrix}\ddot{\mathbf{x}}(t)\\\
\dot{\mathbf{x}}(t)\end{bmatrix}=\begin{bmatrix}\mathbf{f}(\mathbf{x}(t),\dot{\mathbf{x}}(t))\\\
\dot{\mathbf{x}}(t)\end{bmatrix},$ (11)

which inherently preserves the second-order behavior. The above system can be
seen as a first-order system with a constraint. The method proposed in the
previous section can be readily applied to learn second-order neural ODEs for
noisy measurements by incorporating implicit networks.

### Numerical example: Pendulum dynamics

To illustrate learning second-order dynamics, we consider a nonlinear pendulum
model described as

$\ddot{\mathbf{x}}(t)=-\sin(\mathbf{x}(t))-0.05\cdot\dot{\mathbf{x}}(t).$ (12)

We collect data using the initial condition
$[\dot{\mathbf{x}}(t),\mathbf{x}(t)]=[-0.5,2.0]$ at the time interval
$\texttt{{dt}}=0.25$s, which is then corrupted by adding Gaussian white noise
of $\mu=\\{5\%,20\%\\}$. We do not apply any pre-processing step for this
example since filtering with a large time-step is not trivial using a simple
filter such as a low-pass filter. Hence, we will also observe the filtering
capability of the implicit network with the raw data to obtain denoised data.
We employ the proposed scheme by combining an implicit network and neural ODEs
by imposing the second-order structure. We train networks with parameters
$\lambda_{\texttt{Integral}}=1.0$, $\lambda_{\texttt{Grad}}=10^{-2}$, and
$\lambda_{\texttt{MSE}}=1.0$ in (5) for $\mu=5\%$, and
$\lambda_{\texttt{MSE}}=0.1$ for $\mu=20\%$ to avoid over-fitting. We also use
an early-stopping for over-fitting, as discussed in the previous example. For
integration, we take the time-span of $2\cdot\texttt{{dt}}$.

We compare our results with neural ODEs for second-order systems, the approach
proposed in [50]. We plot the learned vector field from both methods in Figure
5, where we see better performance for the proposed method than the one
proposed in [50]. Particularly, it is quite apparent for a larger noise, see
Figure 5, where Std-NODEs for second-order systems fails to compare the vector
field (second row). Moreover, we also plot the denoised data, which is the
output of the trained implicit network in Figure 6, indicating recovery of the
data faithfully without using any prior pre-processing step as such.

True vector field (VF)Imp-NODEsError b/w VFsSO-Std-NODEs [50]Error b/w VFs

Noise $5\%$

Noise $20\%$

Figure 5: Pendulum example: A comparison of the learned vector fields for
second-order dynamical models using the proposed methodology and the one
proposed in [50] for noise $\\{5\%,20\%\\}$. The leftmost figure shows the
ground truth vector field; the second and fourth columns are obtained using
the proposed methodology and the one in [50], respectively; and the third and
fifth columns are the corresponding errors by comparing them with the ground
truth.

Noise $5\%$

Noise $20\%$

Figure 6: Pendulum example: The figures show the ground truth, noisy
measurements, and denoised data obtained from the implicit network.

##  6 Measurements at Irregular Sampling

Lastly, we illustrate the ready applicability of the proposed method when the
data are collected at an irregular time grid, especially when dependent
variables are not even measured in the same time frame. It is of particular
interest in medical applications, where often data comes at quite irregular
time intervals or when the sources of information are different. A similar
problem has been discussed in [46], but for this, an initial condition is
estimated using irregular trajectory points using ODE-RNN. It is then followed
by integrating neural ODEs. Assessing the initial condition from the given
irregular points is quite challenging, especially when the sequence is long,
and also integrating a long sequence imposes additional challenges. Moreover,
although the data can be collected at irregular intervals, all dependent
variables still need to be measured/estimated on the same time grid. On the
other hand, we can readily apply our proposed methods when all dependent
variables are collected on different time-grids because of learning an
implicit representation of the data.

We here present the framework for two-dimensional problems; however, it
readily extends arbitrary dimensional dynamics. Let us consider a dynamical
model as:

$\dot{\mathbf{x}}(t)=\mathbf{f}(\mathbf{x}(t)),$ (13)

where $\mathbf{x}=[\mathbf{x}_{1},\mathbf{x}_{2}]\in\mathbb{R}^{2}$. Next,
assume that the variable $\mathbf{x}_{1}$ is measured at the time-grid
$T_{1}=\\{t^{(1)}_{1},\ldots,t^{(1)}_{n}\\}$, whereas the variable
$\mathbf{x}_{2}$ is collected at the time-grid
$T_{2}=\\{t^{(2)}_{1},\ldots,t^{(2)}_{m}\\}$ with $T_{1}\neq T_{2}$. To learn
a model for the vector field representing the dynamics for $\mathbf{x}$ using
measurements at an irregular time-grid, we construct an implicit
representation for $\mathbf{x}$ so that both variables can estimated on the
same time-grid (let us denote it by $T=\\{t_{1},\ldots,t_{p}\\}$) but with a
constraint using measurements. Assume the implicit network and neural ODE
defining the vector field are denoted by $\mathcal{N}_{\theta}^{\textsf{I}}$
and $\mathcal{N}_{\phi}^{\textsf{Dyn}}$. To train the network, we define the
following loss function:

$\displaystyle\lambda_{\texttt{MSE}}\left(\left\|\left[\mathcal{N}_{\theta}^{\textsf{I}}(t^{(1)}_{i})\right]_{1}-\mathbf{x}_{1}(t^{(1)}_{i})\right\|+\left\|\left[\mathcal{N}_{\theta}^{\textsf{I}}(t^{(1)}_{i})\right]_{2}-\mathbf{x}_{1}(t^{(2)}_{i})\right\|\right)+\lambda_{\texttt{Grad}}\left\|\mathcal{N}_{\phi}^{\textsf{Dyn}}(\mathcal{N}_{\theta}^{\textsf{I}}(t_{i}))-\dfrac{d}{dt}\mathcal{N}_{\theta}^{\textsf{I}}(t_{i})\right\|$
$\displaystyle+\lambda_{\texttt{Integral}}\left\|\left(\mathcal{N}_{\theta}^{\textsf{I}}(t_{j})-\mathcal{N}_{\theta}^{\textsf{I}}(t_{i})\right)-\int_{t_{i}}^{t_{j}}\mathcal{N}_{\phi}^{\textsf{Dyn}}(\mathcal{N}_{\theta}^{\textsf{I}}(\tau))\texttt{{d}}\tau\right\|,$

where $\left[\cdot\right]_{k}$ denotes its $k$-element.

### Numerical example: Linear 2D

We illustrate the considered scenario using a linear 2D example, given by

$\displaystyle\dot{\mathbf{x}}(t)$
$\displaystyle=0.1\cdot\mathbf{x}(t)+2.0\cdot\mathbf{y}(t),$
$\displaystyle\dot{\mathbf{y}}(t)$
$\displaystyle=2.0\cdot\mathbf{x}(t)-0.1\cdot\mathbf{y}(t).$

We collect data using an initial condition $[\mathbf{x},\mathbf{y}]=[2,0]$
with a time interval $\texttt{{dt}}=0.2$ in the time interval $[0,20]$. We
randomly collect $60\%$ independently for the first and second dependent
variable, followed by corrupting them using Gaussian white noise of
$\mu=\\{10\%,20\%\\}$, as shown in the left-most column of Figure 7. Also, we
take the time-grid for prediction of the output of the implicit network as the
uniform grid with $\texttt{{dt}}=0.2$ for the time-interval $[0,20]$. For
learning models for the vector field, we set
$\lambda_{\texttt{Integral}}=1.0$, $\lambda_{\texttt{Grad}}=10^{-2}$, and
$\lambda_{\texttt{MSE}}=1.0$ for $\mu=10\%$ and $\lambda_{\texttt{MSE}}=0.1$
for $\mu=20\%$. We also do early-stopping for $\mu=20\%$. For time
integration, we consider the time span dt.

We show the estimates of the learned vector fields using the proposed
methodology and are compared with the ground truth in Figure 7, illustrating
faithful capturing the dynamics. Moreover, we can also recover the clean
signal faithfully without any prior information about the noise and any pre-
processing of the data.

True vector field (VF)Imp-NODEsError b/w VFs

Noise $10\%$

Noise $20\%$

Figure 7: Linear 2D example: The figures illustrate the ground truth and
learned vector field using Imp-NODEs. The left-most plots show the
irregularities in the sampled measurements, which clearly shows that the
$\mathbf{x}$ and $\mathbf{y}$ do not share the same time grid. The dotted
lines in the left-most figure show the ground truth full trajectories of
$\mathbf{x}$ and $\mathbf{y}$.

##  7 Discussion and Conclusion

This work has presented a new approach for learning dynamical models from
highly noisy time-series data and obtaining denoise data. Our framework blends
universal approximation capabilities of deep neural networks with neural ODEs.
The proposed scheme involves two networks to learn (approximately) an implicit
representation of the measurement data and of the vector field. These networks
are combined by enforcing that an ODE can explain the dynamics of the output
of the implicit network. We also discussed its extension to second-order
neural ODEs to learn second-order dynamical models using corrupted data.
Furthermore, we have presented that the proposed approach can readily handle
arbitrary sampled points in time. The dependent variables need not be
collected at the same time-grid. It is because we first construct an implicit
representation of the data that does not require data to be at a particular
grid.

In the future, we focus on utilizing the encoder-decoder framework combined
with an implicit network to learn latent ODEs to explain even richer dynamics
of the measurement data. Moreover, when the data are high-dimensional (e.g.,
coming from partial-differential equations), applying neural ODEs becomes
computationally intractable. However, it is known that the dynamics often lie
in a low-dimensional manifold. Therefore, in our future work, we aim to
utilize the concept of low-dimensional embedding to make learning
computationally more efficient for high-dimensional data. Furthermore, it
would be interesting to make use of expert knowledge and physical law to have
physics-obey neural ODEs so that the generalizability and extrapolation
capabilities of models can be further improved. Moreover, the performance of
the proposed approach depends on the hyper-parameters that weigh different
terms in the loss function. Therefore, we focus on developing an automatic
mechanism to determine these parameters.

## References

  * [1] S. H. Rudy, J. N. Kutz, and S. L. Brunton, “Deep learning of dynamics and signal-noise decomposition with time-stepping constraints,” J. Comput. Phys., vol. 396, pp. 483–506, 2019.
  * [2] J.-N. Juang, Applied System Identification.  Prentice-Hall, 1994.
  * [3] L. Ljung, System Identification – Theory for the User.  Upper Saddle River, NJ: Prentice-Hall, 2nd ed., 1999.
  * [4] S. A. Billings, Nonlinear System Identification: NARMAX Methods in the Time, Frequency, and Spatio-Temporal Domains.  John Wiley & Sons, 2013.
  * [5] B. Ho and R. E. Kálmán, “Effective construction of linear state-variable models from input/output functions,” Automatisierungstechnik, vol. 14, no. 1-12, pp. 545–548, 1966.
  * [6] J.-N. Juang and R. S. Pappa, “An eigensystem realization algorithm for modal parameter identification and model reduction,” J. Guidance, Control, and Dyn., vol. 8, no. 5, pp. 620–627, 1985.
  * [7] R. W. Longman and J.-N. Juang, “Recursive form of the eigensystem realization algorithm for system identification,” J. Guidance, Control, and Dyn., vol. 12, no. 5, pp. 647–652, 1989.
  * [8] J.-N. Juang, M. Phan, L. G. Horta, and R. W. Longman, “Identification of observer/Kalman filter Markov parameters-theory and experiments,” J. Guidance, Control, and Dyn., vol. 16, no. 2, pp. 320–329, 1993.
  * [9] M. Phan, L. G. Horta, J.-N. Juang, and R. W. Longman, “Linear system identification via an asymptotically stable observer,” J. Opt. Th. Appl., vol. 79, no. 1, pp. 59–86, 1993.
  * [10] M. Phan, J.-N. Juang, and R. W. Longman, “Identification of linear multivariable systems by identification of observers with assigned real eigenvalues,” J. Astronautical Sci., vol. 40, no. 2, 1992.
  * [11] R. E. Kalman, “A new approach to linear filtering and prediction problems,” J. Fluid Engrg., vol. 82, no. 1, 1960.
  * [12] P. J. Schmid, “Dynamic mode decomposition of numerical and experimental data,” J. Fluid Mech., vol. 656, pp. 5–28, 2010.
  * [13] J. H. Tu, C. W. Rowley, D. M. Luchtenburg, S. L. Brunton, and J. N. Kutz, “On dynamic mode decomposition: Theory and applications,” J. Comput. Dyn., vol. 1, no. 2, pp. 391–421, 2014.
  * [14] I. G. Kevrekidis, C. W. Gear, J. M. Hyman, P. G. Kevrekidis, O. Runborg, C. Theodoropoulos, et al., “Equation-free, coarse-grained multiscale computation: Enabling mocroscopic simulators to perform system-level analysis,” Comm. Math. Sci., vol. 1, no. 4, pp. 715–762, 2003.
  * [15] H. U. Voss, P. Kolodner, M. Abel, and J. Kurths, “Amplitude equations from spatiotemporal binary-fluid convection data,” Physical Rev. Lett., vol. 83, no. 17, p. 3422, 1999.
  * [16] H. Ye, R. J. Beamish, S. M. Glaser, S. C. Grant, C.-H. Hsieh, L. J. Richards, J. T. Schnute, and G. Sugihara, “Equation-free mechanistic ecosystem forecasting using empirical dynamic modeling,” Proc. Nat. Acad. Sci. U.S.A., vol. 112, no. 13, pp. E1569–E1576, 2015.
  * [17] M. D. Schmidt, R. R. Vallabhajosyula, J. W. Jenkins, J. E. Hood, A. S. Soni, J. P. Wikswo, and H. Lipson, “Automated refinement and inference of analytical models for metabolic networks,” Phy. Biology, vol. 8, no. 5, p. 055011, 2011.
  * [18] B. C. Daniels and I. Nemenman, “Automated adaptive inference of phenomenological dynamical models,” Nature Comm., vol. 6, no. 1, pp. 1–8, 2015.
  * [19] B. C. Daniels and I. Nemenman, “Efficient inference of parsimonious phenomenological models of cellular dynamics using S-systems and alternating regression,” PLoS One, vol. 10, no. 3, p. e0119821, 2015\.
  * [20] J. Bongard and H. Lipson, “Automated reverse engineering of nonlinear dynamical systems,” Proc. Nat. Acad. Sci. U.S.A., vol. 104, no. 24, pp. 9943–9948, 2007.
  * [21] M. Schmidt and H. Lipson, “Distilling free-form natural laws from experimental data,” Science, vol. 324, no. 5923, pp. 81–85, 2009.
  * [22] S. L. Brunton, J. L. Proctor, and J. N. Kutz, “Sparse identification of nonlinear dynamics with control (SINDYc),” IFAC-PapersOnLine, vol. 49, no. 18, pp. 710–715, 2016.
  * [23] N. M. Mangan, S. L. Brunton, J. L. Proctor, and J. N. Kutz, “Inferring biological networks by sparse identification of nonlinear dynamics,” IEEE Trans. Molecular, Biological and Multi-Scale Comm., vol. 2, no. 1, pp. 52–63, 2016.
  * [24] G. Tran and R. Ward, “Exact recovery of chaotic systems from highly corrupted data,” Multiscale Modeling & Simulation, vol. 15, no. 3, pp. 1108–1129, 2017.
  * [25] H. Schaeffer, G. Tran, R. Ward, and L. Zhang, “Extracting structured dynamical systems using sparse optimization with very few samples,” Multiscale Modeling & Simulation, vol. 18, no. 4, pp. 1435–1461, 2020.
  * [26] N. M. Mangan, J. N. Kutz, S. L. Brunton, and J. L. Proctor, “Model selection for dynamical systems via sparse regression and information criteria,” Proc. the Royal Society A: Mathematical, Physical and Engineering Sciences, vol. 473, no. 2204, p. 20170009, 2017.
  * [27] P. Goyal and P. Benner, “Discovery of nonlinear dynamical systems using a Runge-Kutta inspired dictionary-based sparse regression approach,” arXiv preprint arXiv:2105.04869, 2021.
  * [28] M. Raissi and G. E. Karniadakis, “Hidden physics models: Machine learning of nonlinear partial differential equations,” J. Comput. Phys., vol. 357, pp. 125–141, 2018.
  * [29] S. Chen, S. A. Billings, and P. Grant, “Non-linear system identification using neural networks,” Intern. J. Control, vol. 51, no. 6, pp. 1191–1214, 1990\.
  * [30] R. Rico-Martinez and I. G. Kevrekidis, “Continuous time modeling of nonlinear systems: A neural network-based approach,” in IEEE Intern. Conf. on Neural Networks, pp. 1522–1525, 1993.
  * [31] R. Gonzalez-Garcia, R. Rico-Martinez, and I. Kevrekidis, “Identification of distributed parameter systems: A neural net based approach,” Computers & Chemical Engrg., vol. 22, pp. S965–S968, 1998.
  * [32] M. Milano and P. Koumoutsakos, “Neural network modeling for near wall turbulent flow,” J. Comput. Phys., vol. 182, no. 1, pp. 1–26, 2002.
  * [33] Z. Lu, B. R. Hunt, and E. Ott, “Attractor reconstruction by machine learning,” Chaos: An Interdisciplinary Journal of Nonlinear Science, vol. 28, no. 6, p. 061104, 2018.
  * [34] S. Pan and K. Duraisamy, “Long-time predictive modeling of nonlinear dynamical systems using neural networks,” Complexity, vol. 2018, 2018.
  * [35] J. Pathak, Z. Lu, B. R. Hunt, M. Girvan, and E. Ott, “Using machine learning to replicate chaotic attractors and calculate Lyapunov exponents from data,” Chaos: An Interdisciplinary J. Nonlinear Sci., vol. 27, no. 12, p. 121102, 2017.
  * [36] J. Pathak, A. Wikner, R. Fussell, S. Chandra, B. R. Hunt, M. Girvan, and E. Ott, “Hybrid forecasting of chaotic processes: Using machine learning in conjunction with a knowledge-based model,” Chaos: An Interdisciplinary J. Nonlinear Sci., vol. 28, no. 4, p. 041101, 2018.
  * [37] P. R. Vlachas, W. Byeon, Z. Y. Wan, T. P. Sapsis, and P. Koumoutsakos, “Data-driven forecasting of high-dimensional chaotic systems with long short-term memory networks,” Proc. the Royal Society A: Mathematical, Physical and Engineering Sciences, vol. 474, no. 2213, p. 20170844, 2018.
  * [38] B. Lusch, J. N. Kutz, and S. L. Brunton, “Deep learning for universal linear embeddings of nonlinear dynamics,” Nature Comm., vol. 9, no. 1, pp. 1–10, 2018.
  * [39] N. Takeishi, Y. Kawahara, and T. Yairi, “Learning Koopman invariant subspaces for dynamic mode decomposition,” in Adances in Neural Information Processing Systems, vol. 31, pp. 1130–1140, 2017.
  * [40] E. Yeung, S. Kundu, and N. Hodas, “Learning deep neural network representations for Koopman operators of nonlinear dynamical systems,” in American Control Conference (ACC), pp. 4832–4839, IEEE, 2019.
  * [41] K. Champion, B. Lusch, J. N. Kutz, and S. L. Brunton, “Data-driven discovery of coordinates and governing equations,” Proc. Nat. Acad. Sci. U.S.A., vol. 116, no. 45, pp. 22445–22451, 2019.
  * [42] M. Raissi, P. Perdikaris, and G. E. Karniadakis, “Multistep neural networks for data-driven discovery of nonlinear dynamical systems,” arXiv preprint arXiv:1801.01236, 2018.
  * [43] M. Raissi, P. Perdikaris, and G. E. Karniadakis, “Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations,” J. Comput. Phys., vol. 378, pp. 686–707, 2019.
  * [44] M. Raissi, A. Yazdani, and G. E. Karniadakis, “Hidden fluid mechanics: Learning velocity and pressure fields from flow visualizations,” Science, vol. 367, no. 6481, pp. 1026–1030, 2020.
  * [45] R. T. Chen, Y. Rubanova, J. Bettencourt, and D. K. Duvenaud, “Neural ordinary differential equations,” in Advances Neural Inform. Processing Sys., vol. 32, pp. 6571–6583, 2018.
  * [46] Y. Rubanova, R. T. Q. Chen, and D. K. Duvenaud, “Latent ordinary differential equations for irregularly-sampled time series,” in Advances in Neural Information Processing Systems, vol. 32, 2019.
  * [47] D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” arXiv preprint arXiv:1412.6980, 2014.
  * [48] A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga, et al., “Pytorch: An imperative style, high-performance deep learning library,” arXiv preprint arXiv:1912.01703, 2019.
  * [49] S. L. Brunton, J. L. Proctor, and J. N. Kutz, “Discovering governing equations from data by sparse identification of nonlinear dynamical systems,” Proc. Nat. Acad. Sci. U.S.A., vol. 113, no. 15, pp. 3932–3937, 2016.
  * [50] A. Norcliffe, C. Bodnar, B. Day, N. Simidjievski, and P. Liò, “On second order behaviour in augmented neural ODEs,” in Advances in Neural Information Processing Systems, vol. 34, 2020.
  * [51] V. Sitzmann, J. Martel, A. Bergman, D. Lindell, and G. Wetzstein, “Implicit neural representations with periodic activation functions,” in Advances in Neural Information Processing Systems, vol. 33, 2020.
  * [52] D.-A. Clevert, T. Unterthiner, and S. Hochreiter, “Fast and accurate deep network learning by exponential linear units (ELUs),” arXiv preprint arXiv:1511.07289, 2015.

##  Appendix A Suitable Architectures and Chosen Hyper-parameter

Here, we briefly discuss neural network architectures suitable for our
proposed approach. We require two neural networks for our framework, one for
learning the implicit representation $\mathcal{N}_{\theta}^{\textsf{I}}$ and
the second one $\mathcal{N}_{\theta}^{\textsf{Dyn}}$ is to learn the vector
field. For implicit representation, we use a fully connected multi-layer
perceptron (MLP) as depicted in Figure 8(a) with periodic activation functions
(e.g., $\sin$) [51] which has shown its ability to capture finely detailed
features as well as the gradients of a function. To approximate the vector
field, we consider a simple residual-type network as illustrated in Figure
8(b) with _exponential linear unit_ (ELU) as an activation function [52]. We
choose ELU as the activation function since it is continuous and
differentiable and resembles a widely used activation function, namely
rectified linear unit (ReLU). Furthermore, to train implicit networks, we map
the input data to $[-1,1]$ as recommended in [51]. For the considered examples
in the paper, we report the architecture designs (e.g., numbers of neural and
layers) in Table 1.

Figure 8: The figure shows three potential simple architectures that can be used to learn either implicit representation or to approximate the underlying vector field. Diagram (a) is a simple multi-layer perceptron, and (b) is a residual-type network but fully connected.  Example | Networks | Neurons |  | Layers or
---
residual blocks
Learning rates
Cubic oscillator | For implicit representation | 20 | 4 | $10^{-3}$
For approximating vector field | 20 | 4 | $10^{-3}$
Pendulum | For implicit representation | 32 | 4 | $5\cdot 10^{-4}$
For approximating vector field | 20 | 4 | $10^{-3}$
Linear example | For implicit representation | 20 | 3 | $5\cdot 10^{-4}$
For approximating vector field | 20 | 2 | $10^{-3}$
Table 1: The table shows the information about network architectures and
learning rates.