# Towards Robust Data-Driven Automated Recovery of Symbolic Conservation Laws
from Limited Data

Tracey Oellerich(1), Maria Emelianenko(1)
(1)Department of Mathematical Sciences, George Mason University

## Abstract

Conservation laws are an inherent feature in many systems modeling real world
phenomena, in particular, those modeling biological and chemical systems. If
the form of the underlying dynamical system is known, linear algebra and
algebraic geometry methods can be used to identify the conservation laws. Our
work focuses on using data-driven methods to identify the conservation law(s)
in the absence of the knowledge of system dynamics. Building in part upon the
ideas proposed in [1], we develop a robust data-driven computational framework
that automates the process of identifying the number and type of the
conservation law(s) while keeping the amount of required data to a minimum. We
demonstrate that due to relative stability of singular vectors to noise we are
able to reconstruct correct conservation laws without the need for excessive
parameter tuning. While we focus primarily on biological examples, the
framework proposed herein is suitable for a variety of data science
applications and can be coupled with other machine learning approaches.

##  1 Introduction

Due to the importance of identifying conserved quantities in a variety of
practical applications, for decades this area has enjoyed much attention in
applied mathematical literature. Analytically, linear conservation laws can be
discovered from a given dynamical system by solving a linear system of
equations formed using a stoichiometric matrix [2, 3, 4]. Nonlinear relations
can be found using more sophisticated methodology as for instance discussed in
[5]. Methods in Lie group theory have also been heavily utilized for
determining conservation laws [6, 7, 8, 9, 10].

Approaching this problem computationally, we see there are multiple algorithms
designed to learn conservation laws given a dynamical system input. In [11],
manifold learning is applied to both learn conservation laws and identify
reduced state variables. AI-Poincaré 2.0[12] takes differential equation
inputs and uses trained neural nets and manifold learning to identify the
number of conservation laws as well as their symbolic representation.
Eliminating the need for a black-box methodology, the Sparse Invariant
Detector (SID)[13] uses linear algorithms to discover conservation laws.
However, in the age of data-driven modeling, we want to tackle a more
challenging question of recovering these relations without knowing the system
dynamics a priori.

Specialized approaches have been developed to specifically recover underlying
Hamiltonian functions[14, 15]. More general approaches, such as AI-Poincaré
[16] and ConservNet [17], use neural networks to recover conservation laws
from observation data. In [18], a deep neural net structure is paired with
Noether’s theorem to extract physical information about the underlying
physical system. Manifold learning with optimal transport [19] provides a
geometric approach shown to be robust to noise. In [20], machine learning is
used to both identify the conservation laws via first integrals as well as the
underlying dynamical system structure. Koopman theory coupled with data-driven
regression provides a symbolic representation of the conserved quantities [1].
Typically, these approaches can require anywhere from 100 points [20] to
upwards of $10^{3}$ points[12]. Higher-order systems may require additional
points for accurate recovery, although it is unclear what the relationship is
amongst order and necessary observations. Ability to handle noisy experimental
data also remains a outstanding challenge that needs to be addressed.

The SVD-based null space estimation method that we adopt in this paper is
relatively robust and less demanding in terms of data and underlying structure
requirements. In fact, as shown herein, in many cases only 20 points is
sufficient for accurate recovery of the underlying conserved structures even
in a high-noise environment. In addition to keeping the observable data size
requirements to a minimum, we also aim at learning the functional (symbolic)
form of the conservation law(s) which typically is not a direct outcome for
the algorithms mentioned above, with the notable exception of [1] which
utilizes the same sparse identification paradigm as used in this work. Some
well documented challenges associated with this approach include the need to
choose appropriate functional libraries and corresponding optimality
thresholds.

We develop a unified data-driven approach to automatically select the optimal
learning library and make an informed choice regarding the best polynomial
degree to be used. The main idea is to perform a sequence of data
transformations including SVD decomposition followed by echelon form reduction
and let numerical data drive the identification of the singular value gap and
inform consecutive selection of the optimization parameters. We demonstrate
the power of our method on several classes of problems including automatic
detection of multiple conservation laws and nonlinear conservation with
applications ranging from physics to biology.

To summarize, we propose a general methodology aimed at learning symbolic
forms of conservation law(s) that allows to: (1) limit the amount of data
needed for robust discovery of conservations in the presence of noise and
without the knowledge of the underlying dynamics, and (2) automate the process
of selecting optimal libraries and optimization thresholds based solely on the
estimated noise level and the singular values of the library matrix associated
with observation data.

We note that due to the relative ease of the SVD and LU-based implementation,
the proposed framework lends itself well to being integrated with other
learning approaches. For example, once the conservation laws have been
identified from the data, they can then be used to aid in dynamical system
recovery. Machine learning methods such as [21, 22, 23, 24] impose
conservation laws as constraints in the recovery process. On a similar note,
data-driven methods such as constrained sparse Galerkin [25], DMDc [26],
SINDy-c [27], and generalized Koopman Theory [28] allow for conservation laws
to be used as constraints in their optimization algorithms.

This work is organized as follows. In Section 2, we consider systems with
unknown structure which may or may not be conservative. Using predefined
candidate libraries provided by the user, we develop a general SVD-based to
infer conservation laws or show none exist for the given input. As this
approach is dependent on the robustness of the SVD to noise, in Section 4 we
derive error bounds for a class of library functions and perform numerical
validation experiments. To alleviate the burden of specific library choice on
the user, in Section 5 we propose an algorithm allowing users to input
multiple options and compare the results to make an informed decision. Using
benchmark examples defined in Section 3, in Section 6 we apply the algorithm
in both low data noisy environment. Concluding remarks can then be found in
Section 7.

##  2 Mathematical Notation and Methodology

For this work, we will consider systems modeled using differential equations.
Let
$\textbf{x}(t)=\textbf{x}=\begin{bmatrix}x_{1},x_{2},x_{3},...,x_{n}\end{bmatrix}\in\mathbb{R}^{n}$
be functions of time for each state variable $x_{i}=x_{i}(t)$. Consider the
differential equation system shown in Equation (2.1). Each state variable can
therefore be modeled by the differential equation,
$\dot{x}_{i}(t)=f_{i}(\textbf{x}(t))$.

$\begin{array}[]{c}\dot{\textbf{x}}(t)=\textbf{f}(\textbf{x})\\\ \end{array}$
(2.1)

The goal of this paper is to formulate a robust data-driven method for finding
all possible conservation laws and their respective functional forms for the
system(2.1) under consideration. We will seek conservation laws in the form
$g_{k}(\textbf{x})=C_{k},k=1,\ldots K$ and
$g_{k}:\mathbb{R}^{n}\to\mathbb{R}$. To simplify notation, we will use
$g(\textbf{x})$ to represent a single conservation law, using subscripts as
necessary.

The data-driven nature of this approach can be articulated as follows.
Following the formalism introduced in [1] we will seek an approximation for
$g(\textbf{x})$ in the form specified in (2.2).

$g(\textbf{x)}=C\approx\Theta(\textbf{x})\bm{\xi}$ (2.2)

where
$\Theta(\textbf{x})=[\bm{\theta}_{1}(\textbf{x}),\bm{\theta}_{2}(\textbf{x})\ldots,\bm{\theta}_{p}(\textbf{x})]^{T}\in\mathbb{R}^{1\times
p}$ and $\bm{\xi}=[\xi_{1},\ldots\xi_{p}]^{T}\in\mathbb{R}^{p}$.

In this setup, $\Theta(\textbf{x})$ is a symbolic library of candidate
functions, $\\{\bm{\theta}_{i}(\textbf{x})\\}_{i,\ldots p}$ which later on
will be evaluated at $N$ data points to construct the data matrix. The choice
of the library, in general left to the user, is motivated by the functional
forms one expects to see in a particular application. The corresponding
$\bm{\xi}\in\mathbb{R}^{p}$ is the vector of weights associated with the
library terms that we are trying to find. The $\Theta$-library can be
constructed in multiple ways as it is truly up to the user’s discretion which
terms to include. In this work, we will advocate for a robust library
selection approach to alleviate the burden of placing this decision on the
user.

######  Remark 1.

In our considerations, the term $\mathbf{{x}}^{k}$ denotes the multi-index
corresponding to all possible order $k$ monomials of $\mathbf{x}$. For
example, if $\mathbf{x}=[x_{1},x_{2}]$, then
$\textbf{x}^{3}=\begin{bmatrix}x_{1}^{3},&x_{1}^{2}x_{2},&x_{1}x_{2}^{2},&x_{2}^{3}\end{bmatrix}$.

An example of a candidate $\Theta$-library for a system with two state
variables, $\textbf{x}=[x_{1},x_{2}]$ is given by

$\displaystyle\bm{\Theta}(\textbf{x}(t))$
$\displaystyle=\begin{bmatrix}\textbf{x}&\textbf{x}^{2}&\sin(\textbf{x})&\ln(\textbf{x})\end{bmatrix}=\begin{bmatrix}x_{1}&x_{2}&x_{1}^{2}&x_{1}x_{2}&x_{2}^{2}&\sin(x_{1})&\sin(x_{2})&\ln(x_{1})&\ln(x_{2})\end{bmatrix}$

with $p=9$.

######  Remark 2.

Natural domain considerations must be applied when considering the choice of
the library functions. For instance, logarithmic functions should only be
included in the library if the data is strictly positive. For example, we
primarily consider biological networks in this work which should have strictly
positive values.

######  Remark 3.

In contrast with SINDy-based work [29], in our study we will omit the constant
1 term that is normally present and include the $\ln$ terms which have been
shown to appear in nonlinear conservation laws associated with chemical
reaction systems [30]. We will comment on the first note after the
construction of the algorithm.

By symbolically taking the derivative of (2.2) with respect to time we get

$0=\dfrac{d}{dt}C=\dfrac{d}{dt}g(\textbf{x})=\left(\frac{d\Theta}{dt}\right)\bm{\xi}=\Gamma(\textbf{x},\dot{\textbf{x}})\bm{\xi}$
(2.3)

where
$\Gamma(\textbf{x},\dot{\textbf{x}})=[\dot{\theta}_{1}(\textbf{x})\ldots\dot{\theta}_{p}(\textbf{x})]\in\mathbb{R}^{1\times
p}$ where $\theta_{i}$ corresponds to the i-th $\Theta$-library function. For
the sample $\Theta$-library library given, the associated $\Gamma$-library is

$\Gamma(\textbf{x},\dot{\textbf{x}})=\begin{bmatrix}\dot{x}_{1}&\dot{x}_{2}&2x_{1}\dot{x}_{1}&\dot{x}_{1}x_{2}+x_{1}\dot{x}_{2}&2x_{2}\dot{x}_{2}&\dot{x}_{1}\cos(x_{1})&\dot{x_{2}}\cos(x_{2})&\dfrac{\dot{x}_{1}}{x_{1}}&\dfrac{\dot{x}_{2}}{x_{2}}\end{bmatrix}.$
(2.4)

Let
$\textbf{X}=[\textbf{x}(t_{1}),\textbf{x}(t_{2}),...,\textbf{x}(t_{N})]^{T}\in\mathbb{R}^{N\times
n}$ be time series data for the system with time derivative data given by
$\dot{\textbf{X}}=[\dot{\textbf{x}}(t_{1}),\dot{\textbf{x}}(t_{2}),\ldots,\dot{\textbf{x}}(t_{N})]$.
If the derivative data is not available, then $\dot{\textbf{X}}$ can be
numerically approximated from $\textbf{X}(t)$ [31, 32]. For systems with an
abundance of noise, recent techniques in numerical differentiation can be used
[33, 34].

Adding the time series data to Equation (2.3), we have the over-determined
least squares problem

$\Gamma(\textbf{X},\dot{\textbf{X})}\bm{\xi}=0$ (2.5)

where $\Gamma\in\mathbf{R}^{N\times p}$ is the data matrix corresponding to
the symbolic derivative library $\Gamma(\textbf{x},\dot{\textbf{x}})$ defined
above.

We aim at finding a nontrivial minimal norm solution to the least squares
problem

$\xi=\operatorname*{argmin}_{\bm{\xi}^{\prime}}{||\Gamma(\textbf{X},\dot{\textbf{X}})\bm{\xi}^{\prime}||_{2}}$
(2.6)

which is equivalent to solving the null space problem for the matrix $\Gamma$.
This problem can be solved using singular value decomposition (SVD) of
$\Gamma=USV^{T}$, where $S\in\mathbb{R}^{N\times p}$ contains decreasing
singular values $\sigma_{i}\geq 0$ on the diagonal, $U\in\mathbb{R}^{N\times
N}$ and $V\in\mathbb{R}^{p\times p}$ are unitary matrices corresponding to the
left and right singular vectors, respectively [35, 36]. Alternatively, there
are more algorithmic approaches which can be employed to find the null space
(e.g. [1, 37, 38]), however here we explore an SVD approach as an alternative
that may pave the way towards a robust user-agnostic strategy. We aim at
minimizing complexity and hyperparameters, while designing a method able to
work with relatively small datasets.

Based on the SVD methodology, identifying the conservation law then translates
into finding all right singular vectors corresponding to zero or close to zero
singular values based on a certain cutoff. Figure 2.1 gives a schematic of the
process we intend to use for finding conservation law(s) for given data and
chosen $\Theta$-library. If the process fails to produce a $\sigma_{i}\approx
0$, then there are three possibilities: (1) the starting library does not
contain the appropriate terms, (2) inadequate data due to noise or amount, or
(3) the system does not contain a conservation law.

Main open questions in the outlined methodology that we aim to address below
are as follows. First, how does one appropriately choose a $\Theta$-library.
In some cases, for instance when dealing with biochemical reaction data, the
simplest form of the conservation law, that is a only first order polynomials,
could be the best fit. However, as we will see later, there are instances
where more complex functions are needed to obtain conservation. The second
question is identifying whether or not a system naturally contains multiple
conservation laws and weeding out spurious solutions. Finally, we are
interested in investigating the minimal amount of data necessary for
performing this analysis given an accuracy guarantee.

Figure 2.1: Flowchart detailing the process of identifying conservation laws
from data.

##  3 Benchmark Examples

In the following sections, we will be studying several networks which all
contain conserved quantities. These benchmark examples can be found in Table
3.1 and are referred to throughout the remainder of the work. Example 1, the
Volpert Network, contains second order reactions, but only a single linear
conservation law in all variables. Example 2 contains a mix of first and
second order reactions and has two governing conservation laws, neither
containing all state variables. Example 3 comes from chemistry and models the
chemical oxidation process. The conservation law in this system contains a
nonlinear logarithmic terms, making it harder to identify using traditional
methods. Finally, we examine a degrading network in Example 4 devoid of
conservation laws.

Table 3.1: Network examples used throughout this work. Data for these systems
is generated using the provided coefficients.

Example |  Variables |  Reactions |  Equations |  Coefficients Used
---|---|---|---|---
1:Volpert Network[39] |  $\begin{bmatrix}x_{1}\\\ x_{2}\\\ x_{3}\end{bmatrix}$ |  $\\!\begin{aligned} &X_{1}+X_{2}\xrightarrow{k_{1}}2X_{2}\\\ &X_{2}+X_{3}\xrightarrow{k_{2}}2X_{3}\\\ &X_{3}+X_{1}\xrightarrow{k_{3}}2X_{1}\end{aligned}$ |  $\\!\begin{aligned} &\dot{x}_{1}=k_{3}x_{1}x_{3}-k_{1}x_{1}x_{2}&\\\ &\dot{x}_{2}=k_{1}x_{1}x_{2}-k_{2}x_{2}x_{3}&\\\ &\dot{x}_{1}=k_{2}x_{2}x_{3}-k_{3}x_{1}x_{3}&\\\\[2.5pt] &x_{1}+x_{2}+x_{3}=C&\end{aligned}$ |  $\\!\begin{aligned} \\\ &=[1,1,1]\\\ \textbf{x}_{0}&=[1,2,0.5]\end{aligned}$
2:Two
Conservation Laws |  $\begin{bmatrix}x_{1}\\\ x_{2}\\\ x_{3}\\\ x_{4}\end{bmatrix}$ |  $\\!\begin{aligned} &X_{1}+X_{2}\xrightarrow{k_{1}}X_{3}\\\ &X_{3}\xrightarrow{k_{2}}X_{1}+X_{2}\\\ &X_{3}\xrightarrow{k_{3}}X_{1}+X_{4}\end{aligned}$ |  $\\!\begin{aligned} \dot{x}_{1}&=-k_{1}x_{1}x_{2}+k_{2}x_{3}+k_{3}x_{3}\\\ \dot{x}_{2}&=-k_{1}x_{1}x_{2}+k_{2}x_{3}\\\ \dot{x}_{3}&=k_{1}x_{1}x_{2}-k_{2}x_{3}-k_{3}x_{3}\\\ \dot{x}_{4}&=k_{3}x_{3}\\\\[2.5pt] x_{1}&+x_{3}=C_{1}\\\ x_{2}&+x_{3}+x_{4}=C_{2}\end{aligned}$ |  $\\!\begin{aligned} \\\ &=[1,1,1]\\\ \textbf{x}_{0}&=[1,2,0.5,0.3]\end{aligned}$
3:Chemical Oxidation Network[30] |  $\begin{bmatrix}x\\\ y\\\ S\end{bmatrix}$ |  $\\!\begin{aligned} &X+H_{2}O_{2}\xrightarrow{k_{I}}Y\\\ &Y+S\xrightarrow{k_{2}}X+P\\\ &Y\xrightarrow{k_{3}}\tilde{Y}\\\ \end{aligned}$ |  $\\!\begin{aligned} \dot{x}&=-k_{1}x+k_{2}yS\\\ \dot{y}&=k_{1}x-k_{2}yS-k_{3}y\\\ \dot{S}&=-k_{2}yS\\\\[2.5pt] x&+y-\dfrac{k_{3}}{k_{2}}\ln(S)=C\end{aligned}$ |  $\\!\begin{aligned} \\\ &=[2,0.4,1]\\\ \textbf{x}_{0}&=[0.75,0.5,2]\end{aligned}$
4:No
Conservation Laws |  $\begin{bmatrix}x_{1}\\\ x_{2}\end{bmatrix}$ |  $\\!\begin{aligned} &X_{1}\xrightarrow{k_{1}}0\\\ &X_{1}+X_{2}\xrightleftharpoons[k_{3}]{k_{2}}2X_{1}\end{aligned}$ |  $\\!\begin{aligned} &\dot{x}_{1}(t)=-k_{1}x_{1}+k_{2}x_{1}x_{2}\\\ &\hskip 40.00006pt-k_{3}x_{1}^{2}\\\ &\dot{x}_{2}(t)=-k_{2}x_{1}x_{2}+k_{3}x_{1}^{2}\end{aligned}$ |  $\\!\begin{aligned} \\\ &=[1,2,2]\\\ \textbf{x}_{0}&=[1,2]\end{aligned}$

##  4 Perturbation analysis

In order to develop robust and stable numerical algorithm we need to
understand the effect of noise on the solution produced by the SVD-based null-
space estimation. There is abundance of literature devoted to the subject of
studying perturbed rank-deficient systems (see e.g. [40, 35] and references
therein). Our goal in this work is to adapt existing theoretical results to
the specific case at hand related to the form of the library $\Gamma$ and
understand their implications for the sake of developing a robust numerical
framework.

The following classical result lays the foundation of our study.

######  Theorem 4.1 (Weyl [41, 42]).

For any additive perturbation $\tilde{A}=A+\mathcal{E}$, the following bound
on singular values $\sigma_{i}(A)$ is valid:

$\begin{array}[]{r@{}l}&\varepsilon_{\sigma_{i}}=|\tilde{\sigma}_{i}-\sigma_{i}|\leq\lVert\mathcal{E}\rVert_{2}\hskip
20.00003pt\forall i\end{array}$ (4.1)

Weyl’s theorem posits that the singular values of a matrix are perfectly
conditioned in a sense that no singular value can move more than the norm of
the additive perturbation. While powerful, this result is not practically
useful. However, we can go further due to the special structure afforded by
the $\Gamma$ matrix construction.

Let the exact data x be corrupted by noise $\varepsilon_{x}$, say
$\tilde{\textbf{x}}=\textbf{x}+\varepsilon_{x}$ with the corresponding
derivative approximation,
$\tilde{\dot{\textbf{x}}}=\dot{\textbf{x}}+\varepsilon_{\dot{x}}$. Numerical
differentiation is a complex problem in itself. Options include standard
finite difference approximation (poor choice for sparse data), iterative
finite differences, total variation regularization (TVR), Tikhonov
regularization, and the Savitzky-Golay filter. In our considerations, we will
primarily be utilizing Tikhonov regularization and the Savitzky-Golay filter
using the implementation provided in [43, 44, 45]. Resulting additive
perturbation of the library matrix is denoted as
$\tilde{\Gamma}=\Gamma+\mathcal{E}_{\Gamma}$.

First, we perform numerical experiments to understand the sensitivity of
singular values of $\Gamma$ to i.i.d. Gaussian additive noise. We use
synthetic data generated for each of the benchmark ODE systems using 200
points on the interval $[0,2]$, and choosing 100 interior points to alleviate
boundary effects in derivative approximation which is calculated using
Tikhonov regularization.

Figure 4.1 shows the relationship between the variance of the noise and the
corresponding numerically estimated error in singular values for each of the
given benchmark examples. It is clear that all errors in the singular values
fall directly below $\mathcal{E}_{\Gamma}$, as expected by Equation (4.1).
However, these results do not directly provide intuition on how this bound is
related to the observable quantities such as the noisy data error
$\varepsilon_{x}$ or noisy derivative approximation error
$\varepsilon_{\dot{x}}$. We also know that these dependencies will differ
depending on the choice of the libraries $\Theta$ and the functional forms
included therein. The following straightforward result provides this
connection for a class of polynomial library functions.

Example 1: Volpert Network
Example 2: Network with Two Conservation Laws
Example 3: Chemical Oxidation Network
Example 4: No Conservation Law

Figure 4.1: Log-Log plots displaying the effect of noise to the (Left)
singular values, $\sigma_{i}$, and (Right) the corresponding $\lVert\Gamma
v_{i}\rVert$. Each example is computed for the library corresponding to the
known conservation law, or the linear library in the example having no
conservation. Errors corresponding to
$\varepsilon_{\textbf{x}}$,$\varepsilon_{\dot{\textbf{x}}}$,
$\mathcal{E}_{\Gamma}$ will be shown for comparison.

######  Corollary 4.1 (Weyl-type theorem for $\Gamma$ matrix).

Let
$\textbf{X}=[\textbf{x}(t_{1}),\textbf{x}(t_{2}),...,\textbf{x}(t_{N})]^{T}\in\mathbb{R}^{N\times
n}$ be the time series data for the system with time derivative data given by
$\dot{\textbf{X}}=[\dot{\textbf{x}}(t_{1}),\dot{\textbf{x}}(t_{2}),\ldots,\dot{\textbf{x}}(t_{N})]$.
Consider the library matrix $\Theta(\textbf{X})\in\mathbb{R}^{N\times p}$ with
$\\{\theta_{i}(\textbf{X})\\}_{i=1,\ldots,p}$ and
$\theta_{i}\in\mathbb{P}^{k}$, where $\mathbb{P}^{k}$ denotes polynomials of
degree $k$, and the corresponding
$\Gamma(\textbf{X},\dot{\textbf{X}})=\dfrac{d}{dt}\Theta(\textbf{X})$. For
$\tilde{\Gamma}=\Gamma(\textbf{X}+\varepsilon_{\textbf{x}},\dot{\textbf{X}}+\varepsilon_{\dot{\textbf{x}}})=\Gamma+\mathcal{E}_{\Gamma}$,
then the following estimate holds

$\|\tilde{\sigma}_{i}-\sigma_{i}\|_{2}\leq\lVert\mathcal{E}_{\Gamma}\rVert_{2}\leq\sqrt{Np}\,\mathrm{max}(\|\varepsilon_{x}\|_{\infty},\|\varepsilon_{\dot{x}}\|_{\infty})$

###### Proof.

Notice that since $|\mathcal{E}_{{\Gamma}_{i,j}}|$ consists of
$\textbf{x},\dot{\textbf{x}}$ and combinations of thereof, we get
$|\mathcal{E}_{{\Gamma}_{i,j}}|\leq\mathrm{max}(\|\varepsilon_{x}\|_{\infty},\|\varepsilon_{\dot{x}}\|_{\infty})$
and the result then follows directly from Weyl’s Theorem and norm equivalence
since
$\lVert\mathcal{E}_{\Gamma}\rVert_{2}\leq\sqrt{Np}\lVert\mathcal{E}_{\Gamma}\rVert_{max}$.
∎

Note that the bound above is an overestimation of the error and is not
expected to be sharp. While it serves the purpose of elucidating the
approximate trend of the error, this result requires knowledge of the error in
$\dot{\textbf{x}}$, which is dependent on the numerical differentiation
method.

The following corollary gives a precise bound in the case of Tikhonov
regularization.

######  Corollary 4.2.

If Tikhonov regularization is used for derivative estimation and the interval
is rescaled to $[0,1]$, so that $t_{i}=ih$ with time spacing $h=1/N$, then by
the result in [46]

$\varepsilon_{\dot{x}}\leq
8k_{1}h^{2}+2k^{1/3}_{1}k_{2}^{2/3}||\varepsilon_{x}||^{2/3}_{\infty}$

where $k_{1}=64(1+2||f^{(3)}||_{2})$ and $k_{2}=\sqrt{4+2||f^{(3)}||^{2}_{2}}$
where $f(\mathrm{x})$ is the dynamics of the equation
$\dot{\mathrm{x}}=f(\mathrm{x})$ being approximated. It follows that

$\|\tilde{\sigma}_{i}-\sigma_{i}\|\leq\lVert\mathcal{E}_{\Gamma}\rVert_{2}\leq\sqrt{Np}\,(k_{1}h^{2}+2k_{2}||\varepsilon_{x}||^{2/3}_{\infty})$
(4.2)

Through these results, we see an interplay between the influence of
discretization error and noise. For very small values of added noise, the
error bound for $\Gamma$ is dominated by $\sqrt{Np}C_{1}h^{2}$, while as we
increase the error in x to values larger than $h^{2}$, we begin to see more of
the effect of noise on $\Gamma$. This trend is confirmed by our numerical
experiments in Figure 4.1, where the solid black line corresponds to the error
in $\Gamma$. We observe that the error remains constant until
$\|\varepsilon_{x}\|^{2/3}\approx h^{2}$, at which point the error begins to
increase. This will serve as a guiding principle for picking the number of
measurements needed for desired accuracy.

For recovering the conservation law, we are primarily interested in the right
singular vectors corresponding to smallest singular values. It’s important to
ensure that we can rely on reasonable perturbation bounds in this process.
Numerically, in Figure 4.1 we see stable behavior of the residual $\|\Gamma
v_{i}\|$ w.r.t. to noise. For the $v_{i}$ corresponding to the larger singular
values, there is very little change in $\|\Gamma v_{i}\|$, while for
$\sigma_{j}<\sigma_{\textrm{cutoff}}$, $\|\Gamma v_{i}\|$ follows the error in
$\sigma_{j}$, but is not necessarily bounded by $\mathcal{E}_{\Gamma}$.
Welin’s theorem is often used in this context to establish an upper bound on
the perturbation of singular subspaces [47]. Its drawback is that it combines
left and right singular vectors under one umbrella overestimates the error in
each of them in cases of “tall” or “long skinny” matrices. In our case, since
the calculation concerns right singular vectors, the following analytical
result proves more useful and provides insight into the numerical behavior.

######  Theorem 4.2 ([48]).

Let $\Gamma=U\Sigma V^{T}\in\mathbb{R}^{N\times p}$ is of rank $r$ and
$\tilde{\Gamma}=\tilde{U}\tilde{\Sigma}\tilde{V}^{T}=\Gamma+\mathcal{E}_{\Gamma}$
is its perturbation with i.i.d. zero-mean Gaussian noise $\mathcal{E}$ having
unit variance. More precisely, assume that for some $\tau>0$,
$\mathbb{E}\exp(t\mathcal{E}_{\Gamma})\leq\exp(\tau t)$ for all
$t\in\mathbb{R}$. Let singular values be $\sigma_{1}\geq\ldots\sigma_{r}\geq
0$ and define principle angles as

$\Theta(V,\tilde{V})=diag(\cos^{-1}(\sigma_{1}),\ldots,\cos^{-1}(\sigma_{r}))$

then for the distances $\|\sin(\Theta(V,\tilde{V}))\|$ and
$\|\sin(\Theta(U,\tilde{U}))\|$ between right and left singular subspaces
there exists $C>0$ that only depends on $\tau$ such that

$\mathbb{E}\|\sin(\Theta(V,\tilde{V})\|^{2}\leq\min\left(\frac{Cp(\sigma^{2}_{r}+N)}{\sigma^{4}_{r}},1\right)$
$\mathbb{E}\|\sin(\Theta(U,\tilde{U})\|^{2}\leq\min\left(\frac{CN(\sigma^{2}_{r}+p)}{\sigma^{4}_{r}},1\right)$

The authors of [48] also provide lower bounds for the subspace distances which
demonstrate sharpness of these perturbation results, with a noteworthy
difference between the bounds for left and right singular subspaces. These
bounds are non-trivial if there exists a constant $C_{gap}>0$ such that

$\sigma^{2}_{r}\geq C_{gap}(\sqrt{Np}+N)$ (4.3)

This conforms with the classical result about singular vector perturbation
behaving as $\|\mathcal{E}_{\Gamma}\|_{2}\sigma^{-1}_{r}$ [35]. However, the
explicit dependence on the matrix dimensions makes it more practically useful
in our context. In addition, spectral properties of $\Gamma$ play an important
role since for small $\sigma_{r}$ reliable recovery will become impossible.

In developing our computational framework, we will use the following guiding
principles based on this theoretical analysis:

  * •

The number of observations $N$ needs to significantly exceed the number of
library functions $p$ in $\Gamma$. If this is the case, Theorem 4.2 assures
that right singular vectors can be consistently recovered through solving the
SVD problem.

  * •

If some information about the noise and accuracy tolerance is available, $N$
and $p$ may be chosen to satisfy $\|\sin(\Theta(V,\tilde{V}))\|<tol$ using the
precise accuracy bound given in Theorem 4.2, with a caveat being the
difficulty in estimating the $C$ value.

  * •

For a given library and associated spectral properties of $\Gamma$, one may
calculate $\sigma_{p}$ value and ensure $N$ and $p$ satisfy the necessary
condition given in (4.3).

  * •

Condition (4.2) can be used to identify singular values associated with null
space of $\Gamma$.

  * •

One needs to be aware that there exists no stable algorithm for recovering
left singular vectors in the context of additive perturbation of this kind.

In summary, while SVD-based methodology imposes certain conditions to ensure
numerical accuracy and stability guarantees, in most cases they can be
achieved by adding a modest number of data points and reducing the list of
candidate functions. We are now ready to formulate the computational framework
allowing to automate this process in search for hidden conservation laws in a
given dataset.

##  5 Computational framework for learning conservation laws

Now we will develop an algorithm for identifying the optimal $\Theta$-library
from a set of possible libraries. The goal is to find a library that minimizes
(2.6) while not including extraneous terms in the conservation law. The found
conservation law will be compared to the known solution and the conserved
quantity in Equation (2.5) will be recorded.

Let us consider data given by $\textbf{X}\in\mathbb{R}^{N\times n}$ and a set
of distinct $\Theta$-libraries, $\Phi=\\{\Theta^{(1)},\ldots,\Theta^{(k)}\\}$
where $\Theta^{(i)}\in\mathbb{R}^{p_{i}}$. We wish to identify which
$\Theta^{(i)}$ is optimal, if such a $\Theta$ exists. For each $\Theta^{(i)}$,
we will run the analysis shown in Figure 2.1, paying special attention to all
the singular values in each case. Each $\Theta^{(i)}$-library will have a
corresponding $\Gamma^{(i)}$-library and it’s SVD,
$U^{(i)}S^{(i)}(V^{(i)})^{T}$.

Each $S^{(i)}$ contains the singular values
$\sigma^{(i)}_{1}\leq\ldots\leq\sigma^{(i)}_{p_{i}}$ on the diagonal. Let $j$
denote the index of the first singular value below a predefined cutoff such
that $\sigma_{j}^{(i)},\ldots,\sigma_{p_{i}}^{(i)}<\sigma_{\textrm{cutoff}}$.
Let

$\textrm{count}^{(i)}=\textrm{length}\begin{bmatrix}\sigma_{j}^{(i)}&\ldots&\sigma_{p_{i}}^{(i)}\end{bmatrix}=\begin{cases}0&\textrm{if
no }\sigma_{j}<\sigma_{\textrm{cutoff}}\\\
p_{i}-j+1&\textrm{otherwise}\end{cases}.$ (5.1)

If $\textrm{count}^{(i)}=0$, then the choice of library does not produce a
conservation law and no further analysis can be done for this choice of
$\Theta$-library. Furthermore, if $\textrm{count}^{(i)}>n$, we will also
discard these $\Theta^{(i)}$ as they have produced more conservation laws than
states in the system. This could indicate that the cutoff value is placed too
high or there is a library containing a subset of the functions which is more
optimal.

######  Remark 4.

The cutoff value will vary model to model. If prior knowledge about the noise
in the system is known, the cutoff may be estimated used the results in
Section 4. Based on Corollary 4.2, we suggest the following rule for choosing
the cutoff value:
$\sigma_{\textrm{cutoff}}=\sqrt{Np}\|\varepsilon_{x}\|_{\textrm{max}}^{2/3}$,
changing according to the library considered. This bound follows (4.2) while
still obeying Equation (4.1).

Of the remaining libraries, we will look closer at their corresponding
singular values. In practice, larger singular values,
$\begin{bmatrix}\sigma_{1}^{(i)}&\ldots&\sigma_{j-1}^{(i)}\end{bmatrix}$ are
often retained in order to produce matrix approximations for large matrices as
those are the largest contributors of information in the transformed system
[49]. In a similar sense, we are obtaining our null space approximations from
the smallest singular values,
$\begin{bmatrix}\sigma_{j}^{(i)}&\ldots&\sigma_{p_{i}}^{(i)}\end{bmatrix}$, as
they contribute the “least” amount of information. We are interested in the
gap between these two sets, defined by

$\delta^{(i)}=\sigma_{j-1}^{(i)}-\sigma_{j}^{(i)}$ (5.2)

Each $\delta^{(i)}$ measures the discrepancy between the singular values which
approximate the matrix and those which contribute to the null space. Optimal
libraries will have a large $\delta^{(i)}$, indicating that there is a clear
distinction between the two sets. Small $\delta^{(i)}$ indicate that
$\sigma_{j-1}^{(i)}$ could have potentially been included in the other set had
the threshold value allowed it. As we will see in the examples, most of these
significantly small $\delta^{(i)}$’s correspond to libraries which have a
large $p_{i}$.

######  Remark 5.

While not a sufficient criteria for dismissal, one can also check
$\Gamma^{(i)}v_{j}^{(i)}\leq\sigma_{\textrm{cutoff}}$ for all $j$. This will
reaffirm that the choice of library does indeed produce a conservation law for
the system. This is not always the best indicator of an optimal library as
small coefficients can be included in libraries as the algorithm attempts to
“tune” the result to the system.

Once the optimal library has been chosen, all that remains is to record the
corresponding right singular vector(s), $v_{j}$, for the $\sigma_{j}$’s. The
$v_{j}$ contain the coefficients for the terms in the chosen $\Theta$-library.
Flowchart 5.1 details this process and Algorithm 1 provides associated pseudo-
code.

Figure 5.1: Flowchart detailing the process for selecting the optimal
$\Theta$-library. For each potential library, singular values and vectors will
be recorded and count and $\delta$ are calculated. The optimal $\Theta$ will
correspond to the system with $\textrm{count}\in(0,n]$ and the largest
$\delta$. Once those have been selected, the corresponding singular vector(s)
can be defined as the coefficients for the functions in the chosen
$\Theta$-library.

######  Remark 6.

Recovered conservation laws can be simplified by finding the reduced row
echelon form (RREF), possibly using a permutation matrix to ensure the best
pivot points are chosen [35]. Choosing to write the recovered laws in this
form may provide insight into the underlying network structure as well as
eliminating any scaling.

Algorithm 1 Algorithm for finding optimal $\Theta$-library from a list of
candidate libraries.

$\textbf{X}\in\mathbb{R}^{N\times n}$ \- Data,
$d\textbf{X}\in\mathbb{R}^{N\times n}$ \- derivative data,
$\\{\Theta^{(i)}\\}_{i=1,\ldots,k}$ \- chosen libraries to test, cutoff value
- $\sigma_{\textrm{cutoff}}$

$\Theta^{(\textrm{opt})}$, $\bm{\Xi}$

$\Phi=\\{\Theta^{(1)}(\textbf{x}),\ldots,\Theta^{(k)}(\textbf{x})\\}$
$\triangleright$ Library of considered libraries

for $i\in\\{1,\ldots,k\\}$ do

$\Theta^{(i)}=\Phi_{i}$ $\triangleright$ Choose a new $\Theta$ library

$\Gamma^{(i)}=(D_{x}\Theta^{(i)})^{T}\dot{\textbf{x}}(t)=\begin{bmatrix}\nabla_{\textbf{x}}\theta_{1}^{(i)}(\textbf{X})\cdot
d\textbf{X}&\ldots&\nabla_{\textbf{x}}\theta_{p_{i}}^{(i)}(\textbf{X})\cdot
d\textbf{X}\end{bmatrix}$ $\triangleright$ Calculate $\Gamma$

$\Gamma^{(i)}=U^{(i)}S^{(i)}(V^{(i)})^{T}$

$S^{(i)}=\textrm{diag}(\sigma^{(i)}_{j})\hskip
10.00002pt\sigma_{1}\geq\ldots\geq\sigma_{p_{i}}$

$\textrm{ind}^{(i)}=\\{j|\sigma_{j}^{(i)}<\sigma_{\textrm{cutoff}}\\}$
$\triangleright$ Find indices where $\sigma\approx 0$

$count^{(i)}=\textrm{length}(\textrm{ind}^{(i)})$ $\triangleright$ Count the
number of nonzero entries

if $0<count^{(i)}<n$ then

$\delta^{(i)}=\\{\sigma^{(i)}_{j-1}-\sigma^{(i)}_{j}|j=\min(\textrm{ind}^{(i)})\\}$
$\triangleright$ Sigma-gap

else

go to next section$\triangleright$ Too many recovered conservation laws or
none.

end if

end for

opt $\leftarrow\operatorname*{argmax}\limits_{i}\delta^{(i)}$

Return: $\Theta^{(\textrm{opt})}$ &
$\bm{\Xi}=\\{V^{(opt)}_{j}|j\in\textrm{ind}^{(\textrm{opt}))}\\}$

Optional: $P\bm{\Xi}=LU\hskip 10.00002pt\rightarrow\hskip
10.00002pt\bm{\Xi}=\textrm{RREF}((P\bm{\Xi})^{T})P^{T}$

##  6 Numerical Results

Now we will implement Algorithm 1 for the examples in Table 3.1 for $N=20,100$
with noise variance $=0,1e-10,1e-5$. The candidate libraries will contain a
combination of: monomials up to order 3, trigonometric terms, and logarithmic
terms, with the largest library considered being
$\begin{bmatrix}\textbf{x}&\textbf{x}^{2}&\textbf{x}^{3}&\sin(\textbf{x})&\cos(\textbf{x})&\ln(\textbf{x})\end{bmatrix}$.
In the following figures, libraries will be represented by a triple,
$(a,b,c)$, corresponding to: polynomial terms up to order $a$, include
($b=1$)/exclude ($b=0$) trigonometric terms, and include ($c=1$)/exclude
($c=0$) logarithmic terms. We will only consider libraries in which $N\geq p$,
ensuring accurate null space recovery. A complete list of libraries considered
and their corresponding triple can be found in the Supplemental Notes.

Data is simulated from a known ODE system for $t=[0:dt:1]$, where
$dt=\frac{1}{N}$ and i.i.d. Gaussian noise with a given variance is added. The
corresponding derivative data will be generated using numerical
differentiation methods refined in [44]. Tikhonov regularization is used for
Examples 1, 2, and 4 whereas the Savitzky-Golay filter is used for Example 2.

######  Remark 7.

In our observations, the “ideal” number of points needed is dependent on the
libraries considered. According to Theorem 4.2, to ensure possibility of
proper recovery of singular subspaces, $\Gamma$ should necessarily be
rectangular with $N\geq p$. For methods to identify a sparse null space in the
under-determined case, see [50, 51]

For each, we graph all singular values, $\sigma_{i}$, for each candidate
library and the corresponding cutoff (black line). Both the number of
$\sigma_{i}<\sigma_{\textrm{cutoff}}$ as well as the distance between points
directly above and below $\sigma_{\textrm{cutoff}}$ will be used to determine
the optimal library. In each figure, we shade the (perturbed) singular
value(s) corresponding to the true conservation law. Ideally we wish to see
the algorithm selecting the libraries with shaded values and only shaded
values appearing below the cutoff. Once the optimal library is chosen, we can
consider the errors related to the recovered conservation law(s) follow and
overall method accuracy for 1000 runs. For more detailed analysis of the
overall process, refer to the tables in the Supplemental Notes.

All figures are generated using Matlab with formatting developed in [52].

###  6.1 Example 1

Figure 6.1 contains the analysis for Example 1 in Section 3. In all cases,
first order polynomial libraries produced the optimal conservation law. For
example, in the case of 20 points with a variance of 0, four libraries,
$(1,0,0),(1,0,1),(2,0,0),(1,1,1)$, have exactly one singular value below the
cutoff while the remaining libraries contain more than 3. Of the four
libraries meeting the counting criteria, by looking at $\delta$ as defined in
Equation (5.2), it is clear that the first library corresponding to linear
terms has the largest gap. This is shown numerically in the Supplemental
Notes.

Figure 6.1: Singular values corresponding to different library configurations
for Example 1 with $N$ points and given noise variance.

Table 6.1 contains error values for the data, $\Gamma$, the conserved
quantity, and the recovered law, in original and reduced form. As can be seen,
as we increase the noise added to the system, we are also increasing the
overall error in the conserved quantity and recovered vector, although it
remains close to the variance of the noise added. For example, in the first
setup with 20 points and no noise, the recovered conservation law is:

$0.5774\dot{x}_{1}+0.5774\dot{x}_{2}+0.5774\dot{x}_{3}=0\hskip
20.00003pt\Rightarrow\hskip 20.00003ptx_{1}+x_{2}+x_{3}=C$ (6.1)

In all cases, assuming appropriate cutoffs are defined, we were able to
achieve an accuracy of 100% when tested on 1000 randomly generated data sets.

N | $\|\varepsilon_{x}\|$ | $\lVert\varepsilon_{\dot{\textbf{x}}}\rVert$ | $\lVert\mathcal{E}_{\Gamma}\rVert$ | $\sum\limits_{j}\lVert\tilde{\Gamma}\xi_{\textrm{opt},j}\rVert$ | $\sum\limits_{j}\lVert\tilde{\Gamma}\xi_{\textrm{rref},j}\rVert$ | $\lVert\xi_{\textrm{exact}}-\xi_{\textrm{rref}}\rVert$ | Accuracy
---|---|---|---|---|---|---|---
20 points | 0 | 0.1836 | 0.1836 | 2.179e-14 | 3.746e-14 | 3.144e-15 | 100%
5.4849e-10 | 0.1836 | 0.1836 | 3.326e-8 | 5.761e-8 | 1.207e-9 | 100%
4.3606e-5 | 0.1834 | 0.1834 | 1.608e-3 | 2.785e-3 | 1.390e-4 | 100%
100 points | 0 | 0.05127 | 0.05127 | 3.538e-14 | 6.144e-14 | 1.351e-15 | 100%
4.7594e-10 | 0.05127 | 0.5127 | 2.021e-8 | 3.500e-8 | 9.294e-10 | 100%
4.579e-5 | 0.0513 | 0.0513 | 2.774e-3 | 4.805e-3 | 1.019e-4 | 100%
Table 6.1: Table of errors related to the found conservation law for Example
1.

###  6.2 Example 2

In Figure 6.2 we see the optimal library will again contain only first order
polynomials, however, we now have two $\sigma_{i}<\sigma_{\textrm{cutoff}}$,
indicating we have two recovered conservation laws. It is interesting to note
that two distinct conservation laws are recovered rather than one combined
law.

Figure 6.2: Singular values corresponding to different library configurations
for Example 2 with $N$ points and given noise variance.

Despite needing to recover two conservation laws, Table 6.2 contains a similar
trend occurring between $\varepsilon_{\textbf{x}}$ and the error incurred in
the recovered law and conserved quantity. As before, we recovered the correct
form of the conservation law 100% of the time for 1000 randomly generated test
sets.

N | $\|\varepsilon_{x}\|$ | $\lVert\varepsilon_{\dot{\textbf{x}}}\rVert$ | $\lVert\mathcal{E}_{\Gamma}\rVert$ | $\sum\limits_{j}\lVert\tilde{\Gamma}\xi_{\textrm{opt},j}\rVert$ | $\sum\limits_{j}\lVert\tilde{\Gamma}\xi_{\textrm{rref},j}\rVert$ | $\lVert\xi_{\textrm{exact}}-\xi_{\textrm{rref}}\rVert$ | Accuracy
---|---|---|---|---|---|---|---
20 points | 0 | 0.32781 | 0.32781 | 1.472e-14 | 2.322e-14 | 6.026e-15 | 100%
5.4588e-10 | 0.3278 | 0.3278 | 3.402e-8 | 5.453e-8 | 2.065e-9 | 100%
5.5253e-5 | 0.3280 | 0.3280 | 4.199e-3 | 7.029e-3 | 4.432e-4 | 100%
100 points | 0 | 0.1045 | 0.1045 | 3.034e-14 | 4.754e-14 | 5.989e-15 | 100%
6.522e-10 | 0.1045 | 0.1045 | 4.740e-8 | 9.034e-8 | 1.706e-9 | 100%
6.6525e-5 | 0.1049 | 0.1049 | 4.733e-3 | 7.136e-3 | 1.760e-4 | 100%
Table 6.2: Table of errors corresponding to the found conservation law for
Example 2.

###  6.3 Example 3

Our previous examples show we can recover both single and multiple
conservation laws, however, in each case, the recovered law was linear. In
Example 3, we will consider a system which contains a nonlinear conservation
law. In almost all the cases shown in Figure 6.3, libraries containing both
first order polynomials and logarithmic terms are optimal. The exception to
this is when $N=100$, and variance was $1e-5$. In this case, we recover the
correct form, however two laws are obtained. It is also worth note that in all
cases libraries containing only first order polynomials failed to have any
$\sigma_{i}<\sigma_{\textrm{cutoff}}$.

Figure 6.3: Singular values corresponding to different library configurations
for Example 3 with $N$ points and given noise variance.

For this system, noise from both the derivatives as well as the data play a
part in the noise in $\Gamma$. Regardless of the increase in sources of error
for $\Gamma$, Table 6.3 maintains the relationship between
$\mathcal{E}_{\Gamma}$ and the conserved quantity. Furthermore, we were able
to achieve the optimal library of the conservation law over 98% of the time
for 1000 randomly generated data sets. Further refinement of the derivative
calculations could see an increase in the overall method accuracy.

N | $\|\varepsilon_{x}\|$ | $\lVert\varepsilon_{\dot{\textbf{x}}}\rVert$ | $\lVert\mathcal{E}_{\Gamma}\rVert$ | $\sum\limits_{j}\lVert\tilde{\Gamma}\xi_{\textrm{opt},j}\rVert$ | $\sum\limits_{j}\lVert\tilde{\Gamma}\xi_{\textrm{rref},j}\rVert$ | $\lVert\xi_{\textrm{exact}}-\xi_{\textrm{rref}}\rVert$ | Accuracy
---|---|---|---|---|---|---|---
20 points | 0 | 1.1899e-4 | 2.811e-4 | 1.974e-6 | 2.269e-6 | 6.946e-4 | 100%
4.5634e-10 | 1.1899e-4 | 2.811e-4 | 1.973e-6 | 2.267e-6 | 6.950e-4 | 100%
3.310e-5 | 5.4656e-1 | 1.117 | 9.561e-4 | 1.026e-3 | 0.6206 | 98.6%
100 points | 0 | 4.4893e-8 | 1.674e-7 | 2.141e-8 | 2.460e-8 | 1.072e-6 | 100%
6.0004e-10 | 7.5211e-8 | 1.477e-7 | 1.594e-8 | 1.831e-8 | 6.129e-7 | 99.9%
6.3398e-5 | 2.0047e-3 | 5.553e-3 | 5.547e-3 | 9.219e-3 | NaN | 100%
Table 6.3: Table of errors corresponding to the found conservation law(s) for
Example 3. For the case of $N=100$ and
$\|\varepsilon_{\textbf{x}}\|=6.3398e-5$, we obtain two conservation laws
rather than one, therefore no direct comparison can be made between the exact
form and the recovered laws.

###  6.4 Example 4

Our final benchmark example contains no conservation law, testing if the
algorithm will output no optimal library. In Figure 6.4, we expect to see all
singular values above the designated cutoff, but this is not the case for
examples containing noise. While the algorithm may emit a conservation law, it
is important to examine the results and verify they correlate with the system.
In cases where the algorithm selected a library the corresponding $\bm{\xi}$
is both dense and associated with large libraries. To avoid this, either
decrease $\sigma_{\textrm{cutoff}}$ or exclude large libraries that can be
over-fitted.

Figure 6.4: Singular values corresponding to different library configurations
for Example 4 with $N$ points and given noise variance.

In Table 6.4, we have outlined the results of the algorithm when applied to
Example 4. In the case of noisy data, one needs to examine the output to
verify it is indeed a conservation law. In this case, all the “found
conservation laws” are dense and are a byproduct of the minimization process.

N | $\|\varepsilon_{x}\|$ | $\lVert\varepsilon_{\dot{\textbf{x}}}\rVert$ | Optimal Library | count | $\sum\limits_{j}\lVert\tilde{\Gamma}\xi_{\textrm{opt},j}\rVert$ | $\sum\limits_{j}\lVert\tilde{\Gamma}\xi_{\textrm{rref},j}\rVert$
---|---|---|---|---|---|---
20 points | 0 | 0.26317 | Not Found | NaN | NaN | NaN
4.5288e-10 | 0.26317 | (2, 1, 1) | 2 | 1.024e-5 | 1.270e-5
4.1076e-5 | 0.26315 | (2, 0, 1) | 2 | 6.205e-3 | 8.672e-3
100 points | 0 | 0.0761 | Not Found | NaN | NaN | NaN
3.6436e-10 | 0.07612 | (3, 0, 1) | 1 | 5.290e-6 | 6.484e-6
3.7872e-5 | 0.07625 | (2, 0, 0) | 1 | 0.01377 | 0.01718
Table 6.4: Table of errors corresponding to the found conservation law(s) for
Example 4.

###  6.5 Biological Application

While we have shown we can recover various forms of conservation, all
benchmark examples utilize mass-action kinetics. To show our approach is
independent of the system dynamics let us consider instead a system with
Michaelis–Menten kinetics, common in biological systems. We consider the
Mitogen Activated Protein Kinase (MAPK) Pathway presented in [53] containing 9
state variables (3 proteins each with 3 states of phosphorylation) and emits 3
conservation laws. The dynamical system representation and parameters used
will be included in the Supplemental Notes. Data is generated over
$t=[0:dt:1000]$ with $dt=\frac{1}{N}$. As before, i.i.d. Gaussian noise will
be added and derivatives are approximated with Tikhonov regularization.
Singular values for libraries considered are shown in Figure 6.5. In all
cases, the algorithm recovers 3 linear conservation laws.

Figure 6.5: Singular values corresponding to different library configurations
for the MAPK model [53] with $N$ points and given noise variance.

Table 6.5 provides error analysis for the recovered conservation laws. While
the overall error in $\Gamma$ has increased, most likely due to the change in
scale of the data, the results are consistent with the previous examples. In
all cases, we recovered the correct library 100% of the time.

N | $\|\varepsilon_{x}\|$ | $\lVert\varepsilon_{\dot{\textbf{x}}}\rVert$ | $\lVert\mathcal{E}_{\Gamma}\rVert$ | $\sum\limits_{j}\lVert\tilde{\Gamma}\xi_{\textrm{opt},j}\rVert$ | $\sum\limits_{j}\lVert\tilde{\Gamma}\xi_{\textrm{rref},j}\rVert$ | $\lVert\xi_{\textrm{exact}}-\xi_{\textrm{rref}}\rVert$ | Accuracy
---|---|---|---|---|---|---|---
20 points | 0 | 37.31 | 37.31 | 7.828e-14 | 1.383e-13 | 7.057e-13 | 100%
8.8961e-10 | 37.31 | 37.31 | 1.035e-10 | 1.962e-10 | 1.124e-9 | 100%
1.0553e-4 | 37.31 | 37.31 | 1.114e-5 | 1.944e-5 | 7.386e-5 | 100%
100 points | 0 | 4.052 | 4.052 | 5.838e-13 | 1.013e-12 | 4.588e-13 | 100%
1.4759e-9 | 4.052 | 4.052 | 2.255e-9 | 4.001e-9 | 9.180e-10 | 100%
1.2644e-4 | 4.052 | 4.052 | 1.915e-4 | 3.356e-4 | 9.087e-5 | 100%
Table 6.5: Table of errors corresponding to the found conservation laws for
the MAPK pathway in [53].

##  7 Discussion

In this work, we have presented an algorithm which can be used to infer
conservation laws from data, including the case of limited noisy data. We
tested our approach on several systems ranging in complexity and found being
useful for both linear and nonlinear dynamical systems, allowing us to
identify conservation laws regardless of the underlying structure. We have
shown we can recover both nonlinear conservation in Example 3 as well as
multiple conservation laws as demonstrated in Example 2 and the MAPK network.

Robustness of the proposed methodology is based on the provable stability of
the singular values and singular vectors to the level of noise present in the
data. While there are general results of this nature in the literature [35,
46], the exact error characterizations depend on the type of the matrix used.
Our numerical investigations confirm that low sensitivity is preserved in the
case of library matrix $\Gamma(\textbf{X},\dot{\textbf{X}})$ used in the
context of present application. In particular, the error
$\varepsilon_{\sigma_{i}}$ for the smallest singular values of $\Gamma$ is
tightly bounded above by derivative estimation error. Since this error may be
significant in some cases, the accuracy of the derivative approximation plays
an essential role in this discussion. A smoothing approach based on Tikhonov
regularization has shown potential to address this issue based on our
numerical experiments.

Overall, guided by our analytical and numerical observations, we developed the
following strategy for choosing an appropriate $\sigma_{\textrm{cutoff}}$. As
seen in Section 4, if the underlying system noise can be estimated a priori,
then $\sigma_{\textrm{cutoff}}$ will be proportional to the error, although it
may require tuning. Furthermore, provided an approximate error in the
derivative calculation, $\mathcal{E}_{\Gamma}$ can be estimated and
$\sigma_{\textrm{cutoff}}$ can be chosen such that it lies below
$\mathcal{E}_{\Gamma}$. If the error in the data is not accessible, then
possible solutions include pre-processing the data to remove noise, finding
more accurate derivative approximations, or methods of de-noising $\Gamma$
such as those presented in [54]. We can also consider optimizing
$\sigma_{\textrm{cutoff}}$ or finding alternative methods to remove it
entirely.

In addition, we see that while the use of RREF and LU decomposition allows for
significant improvement in our ability to find the reduced form of the
functional relationship, for some choices of the libraries it may result in
fragmentation of the recovered conservation laws.

While we restricted library options to 12 possible forms, different systems
may benefit from additional library options. In this work, each
$\Theta$-library is constructed by choosing classes of functions, such as x or
$\textbf{x}^{3}$ and iterates though subsets of the chosen classes, although
this limitation is not required for Algorithm 1 to function properly. For
systems containing multiple laws, not necessarily all of the same order,
allowing libraries to contain only a subset of proposed classes proves to be a
more effective strategy. For example, consider the 3-species Lotka-Volterra
model presented in [55] containing both a first order polynomial and a third
order polynomial conservation law..

$\displaystyle\dot{x}_{1}=x_{1}(x_{2}-x_{3})$
$\displaystyle\dot{x}_{2}=x_{2}(x_{3}-x_{1})$
$\displaystyle\dot{x}_{3}=x_{3}(x_{1}-x_{2})$ $\displaystyle
x_{1}+x_{2}+x_{3}=C_{1}$ $\displaystyle x_{1}x_{2}x_{3}=C_{2}$

If we consider only classes of functions in the library, the algorithm will
gravitate towards the library containing only first order polynomials. If
subsets of $\Theta=[\textbf{x},\textbf{x}^{2},\textbf{x}^{3}]$ are now
included, particularly the subsets of $[\textbf{x},\textbf{x}^{3}]$, we are
more likely to recover both laws. We do note that for
$\Theta=[x_{1},x_{2},x_{3},x_{1}x_{2}x_{3}]$, we recover both laws exactly,
however the corresponding $\delta$ is within machine epsilon of the $\delta$
for $\Theta=[x_{1},x_{2},x_{3}]$, indicating that it may be advantageous to
investigate several libraries with the largest $\delta$ to ensure all possible
conservation laws are recovered.

Building off these observations, constructing a global library of all possible
functions and iterating over the power set of this global library, excluding
the empty set and sets containing a single state variable, seems to be the
next logical step. While this approach will more accurately infer all possible
conservation laws, it will be significantly more computationally demanding for
if $\Theta_{\textrm{global}}^{T}\in\mathbb{R}^{k}$, $2^{k}-1-k$ sub-libraries
should be considered in the algorithm. Moreover, we observed for systems with
multiple conservation laws originating from different sub-libraries, $\delta$
can be significantly close. To alleviate this, we propose user supervision
might be needed if there is a close decision.

Future work will be focused on ways to derive minimal consistent symbolic
representations of the recovered conserved quantities and applying the
methodology to experimental data.

##  8 Acknowledgements

The authors gratefully acknowledge fruitful discussions with Mariaelena
Pieorbon. ME was supported in part by the Simons Foundation grant #854541. TO
was partially funded by the George Mason University Provost through their
Summer Research Fellowship and Dissertation Completion Grant.

## References

  * [1] E. Kaiser, J. N. Kutz, and S. L. Brunton, “Discovering conservation laws from data for control,” in 2018 IEEE Conference on Decision and Control (CDC), pp. 6415–6421, IEEE, 2018.
  * [2] M. Feinberg, Foundations of chemical reaction network theory.  Springer, 2019.
  * [3] T. Oellerich, M. Emelianenko, L. A. Liotta, and R. P. Araujo, “Biological networks with singular Jacobians: their origins and adaptation criteria,” bioRxiv 2021.03.01.433197, 2021.
  * [4] A. Dickenstein, “Biochemical reaction networks: An invitation for algebraic geometers,” in Mathematical congress of the Americas, vol. 656, pp. 65–83, Contemp. Math, 2016.
  * [5] A. Bressan, G.-Q. G. Chen, M. Lewicka, and D. Wang, Nonlinear conservation laws and applications.  Springer, 2011.
  * [6] R. O. Popovych and A. Bihlo, “Inverse problem on conservation laws,” Physica D: Nonlinear Phenomena, vol. 401, p. 132175, 2020.
  * [7] I. M. Anderson and J. Pohjanpelto, “Symmetries, conservation laws and variational principles for vector field theories,” in Mathematical Proceedings of the Cambridge Philosophical Society, vol. 120, pp. 369–384, Cambridge University Press, 1996.
  * [8] L. Peng, “Symmetries, conservation laws, and Noether’s theorem for differential-difference equations,” Studies in applied mathematics, vol. 139, no. 3, pp. 457–502, 2017.
  * [9] R. Khamitova, Symmetries and conservation laws.  PhD thesis, Växjö university press, 2009.
  * [10] P. J. Olver, Applications of Lie groups to differential equations, vol. 107.  Springer Science & Business Media, 1993.
  * [11] A. Holiday, M. Kooshkbaghi, J. M. Bello-Rivas, C. W. Gear, A. Zagaris, and I. G. Kevrekidis, “Manifold learning for parameter reduction,” Journal of computational physics, vol. 392, pp. 419–431, 2019.
  * [12] Z. Liu, V. Madhavan, and M. Tegmark, “Machine learning conservation laws from differential equations,” Physical Review E, vol. 106, no. 4, p. 045307, 2022.
  * [13] Z. Liu, P. O. Sturm, S. Bharadwaj, S. Silva, and M. Tegmark, “Discovering new interpretable conservation laws as sparse invariants,” 2023.
  * [14] S. Greydanus, M. Dzamba, and J. Yosinski, “Hamiltonian neural networks,” Advances in neural information processing systems, vol. 32, 2019.
  * [15] P. Toth, D. J. Rezende, A. Jaegle, S. Racanière, A. Botev, and I. Higgins, “Hamiltonian generative networks,” arXiv preprint arXiv:1909.13789, 2019\.
  * [16] Z. Liu and M. Tegmark, “Machine learning conservation laws from trajectories,” Physical Review Letters, vol. 126, no. 18, p. 180604, 2021\.
  * [17] S. Ha and H. Jeong, “Discovering conservation laws from trajectories via machine learning,” 2021.
  * [18] Y.-i. Mototake, “Interpretable conservation law estimation by deriving the symmetries of dynamics from trained deep neural networks,” Physical Review E, vol. 103, no. 3, p. 033303, 2021.
  * [19] P. Y. Lu, R. Dangovski, and M. Soljačić, “Discovering conservation laws using optimal transport and manifold learning,” Nature Communications, vol. 14, no. 1, p. 4744, 2023.
  * [20] S. Arora, A. Bihlo, R. Brecht, and P. Holba, “Model-free machine learning of conservation laws from data,” arXiv preprint arXiv:2301.07503, 2023.
  * [21] W. Zhang, S. Das, T.-W. Weng, A. Megretski, L. Daniel, and L. M. Nguyen, “Imposing conservation properties in deep dynamics modeling via contrastive learning,” 2022.
  * [22] T. Readshaw, W. P. Jones, and S. Rigopoulos, “On the incorporation of conservation laws in machine learning tabulation of kinetics for reacting flow simulation,” Physics of Fluids, vol. 35, no. 4, 2023.
  * [23] G.-Z. Wu, Y. Fang, Y.-Y. Wang, and C.-Q. Dai, “Modified physics-informed neural network method based on the conservation law constraint and its prediction of optical solitons,” 2021.
  * [24] K. Lee and K. T. Carlberg, “Deep conservation: A latent-dynamics model for exact satisfaction of physical conservation laws,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, pp. 277–285, 2021.
  * [25] J.-C. Loiseau and S. L. Brunton, “Constrained sparse Galerkin regression,” Journal of Fluid Mechanics, vol. 838, pp. 42–67, 2018.
  * [26] J. L. Proctor, S. L. Brunton, and J. N. Kutz, “Dynamic mode decomposition with control,” SIAM Journal on Applied Dynamical Systems, vol. 15, no. 1, pp. 142–161, 2016.
  * [27] S. L. Brunton, J. L. Proctor, and J. N. Kutz, “Sparse identification of nonlinear dynamics with control (SINDYc),” IFAC-PapersOnLine, vol. 49, no. 18, pp. 710–715, 2016.
  * [28] J. L. Proctor, S. L. Brunton, and J. N. Kutz, “Generalizing Koopmantheory to allow for inputs and control,” SIAM Journal on Applied Dynamical Systems, vol. 17, no. 1, pp. 909–930, 2018.
  * [29] S. L. Brunton, J. L. Proctor, and J. N. Kutz, “Discovering governing equations from data by sparse identification of nonlinear dynamical systems,” Proceedings of the national academy of sciences, vol. 113, no. 15, pp. 3932–3937, 2016.
  * [30] M. Emelianenko, D. Torrejon, M. A. DeNardo, A. K. Socolofsky, A. D. Ryabov, and T. J. Collins, “Estimation of rate constants in nonlinear reactions involving chemical inactivation of oxidation catalysts,” Journal of Mathematical Chemistry, vol. 52, pp. 1460–1476, 2014.
  * [31] A. Iserles, A first course in the numerical analysis of differential equations.  No. 44, Cambridge university press, 2009.
  * [32] J. Nocedal and S. J. Wright, Numerical Optimization.  Springer Series in Operations Research and Financial Engineering, Springer New York, NY, 2 ed., 2006.
  * [33] R. Chartrand, “Numerical differentiation of noisy, nonsmooth data,” International Scholarly Research Notices, vol. 2011, 2011.
  * [34] F. Van Breugel, J. N. Kutz, and B. W. Brunton, “Numerical differentiation of noisy data: A unifying multi-objective optimization framework,” IEEE Access, vol. 8, pp. 196865–196877, 2020.
  * [35] J. W. Demmel, Applied numerical linear algebra.  SIAM, 1997.
  * [36] G. H. Golub and C. F. van Loan, Matrix Computations.  JHU Press, 4 ed., 2013.
  * [37] Q. Qu, J. Sun, and J. Wright, “Finding a sparse vector in a subspace: Linear sparsity using alternating directions,” IEEE Transactions on Information Theory, vol. 62, no. 10, pp. 5855–5880, 2016.
  * [38] L.-A. Gottlieb and T. Neylon, “Matrix sparsification and the sparse null space problem,” in International Workshop on Randomization and Approximation Techniques in Computer Science, pp. 205–218, Springer, 2010.
  * [39] A. Mahdi, A. Ferragut, C. Valls, and C. Wiuf, “Conservation laws in biochemical reaction networks,” SIAM Journal on Applied Dynamical Systems, vol. 16, no. 4, pp. 2213–2232, 2017.
  * [40] M. Stewart, “Perturbation of the SVD in the presence of small singular values,” Linear algebra and its applications, vol. 419, no. 1, pp. 53–77, 2006.
  * [41] H. Weyl, “Das asymptotische verteilungsgestezder eigenwertlinearer partieller di erentialgleichungen mit einer anwendungaufder,” Theorieder Hohlraumstrahlung Mathematische Annalen, vol. 71, 1912.
  * [42] C. L. Lawson and R. J. Hanson, Solving Least Squares Problems.  SIAM, 1995.
  * [43] F. Van Breugel, Y. Liu, B. W. Brunton, and J. N. Kutz, “PyNumDiff: A python package for numerical differentiation of noisy time-series data,” Journal of Open Source Software, vol. 7, no. 71, p. 4078, 2022.
  * [44] J. Wagner, Regularised differentiation of measurement data in systems for healthcare-oriented monitoring of elderly persons.  PhD thesis, The Institute of Radioelectronics and Multimedia Technology, 2020.
  * [45] J. Wagner, “Regularised numerical differentiation,” 2023.
  * [46] S. Lu and Y. Wang, “First and second order numerical differentiation with Tikhonov regularization,” Frontiers of Mathematics in China, vol. 1, no. 3, pp. 354–367, 2006.
  * [47] P. Wedin, “Perturbation bounds in connection with singular value decomposition,” BIT, vol. 12, pp. 99–111, 1972.
  * [48] T. Cai and A. ZHang, “Rate-optimal perturbation bounds for singular subspaces with applications to high-dimensional statistics,” The Annals of Statistics, vol. 46(1), 2016.
  * [49] S. L. Brunton and J. N. Kutz, Data-driven science and engineering: Machine learning, dynamical systems, and control.  Cambridge University Press, 2019.
  * [50] T. F. Coleman and A. Pothen, “The null space problem i. complexity,” SIAM Journal on Algebraic Discrete Methods, vol. 7, no. 4, pp. 527–537, 1986\.
  * [51] T. F. Coleman and A. Pothen, “The null space problem ii. algorithms,” SIAM Journal on Algebraic Discrete Methods, vol. 8, no. 4, pp. 544–563, 1987\.
  * [52] F. G. Nievinski, “subtightplot,” 2024.
  * [53] B. N. Kholodenko, J. F. Hancock, and W. Kolch, “Signalling ballet in space and time,” Nature reviews Molecular cell biology, vol. 11, no. 6, pp. 414–426, 2010.
  * [54] B. P. Epps and E. M. Krivitzky, “Singular value decomposition of noisy data: noise filtering,” Experiments in Fluids, vol. 60, pp. 1–23, 2019.
  * [55] R. Schimming, “Conservation laws for Lotka–Volterra models,” Mathematical methods in the applied sciences, vol. 26, no. 17, pp. 1517–1528, 2003.

##  Appendix A Supplemental Notes

Table A.1 contains a list of all $\Theta$ libraries used for the analysis in
Section 6.

Table A.1: Table of $\Theta$ libraries considered for all graphs in Section 6. Triple | Library
---|---
(1, 0, 0) | $\Theta=[\textbf{X}]$
(2, 0, 0) | $\Theta=[\textbf{X},\textbf{X}^{2}]$
(3, 0, 0) | $\Theta=[\textbf{X},\textbf{X}^{2},\textbf{X}^{3}]$
(1, 1, 0) | $\Theta=[\textbf{X},\sin(\textbf{X}),\cos(\textbf{X})]$
(2, 1, 0) | $\Theta=[\textbf{X},\textbf{X}^{2},\sin(\textbf{X}),\cos(\textbf{X})]$
(3, 1, 0) | $\Theta=[\textbf{X},\textbf{X}^{2},\textbf{X}^{3}\sin(\textbf{X}),\cos(\textbf{X})]$
(1, 0, 1) | $\Theta=[\textbf{X},\ln(\textbf{X})]$
(2, 0, 1) | $\Theta=[\textbf{X},\textbf{X}^{2},\ln(\textbf{X})]$
(3, 0, 1) | $\Theta=[\textbf{X},\textbf{X}^{2},\textbf{X}^{3},\ln(\textbf{X})]$
(1, 1, 1) | $\Theta=[\textbf{X},\sin(\textbf{X}),\cos(\textbf{X}),\ln(\textbf{X})]$
(2, 1, 1) | $\Theta=[\textbf{X},\textbf{X}^{2},\sin(\textbf{X}),\cos(\textbf{X}),\ln(\textbf{X})]$
(3, 1, 1) | $\Theta=[\textbf{X},\textbf{X}^{2},\textbf{X}^{3}\sin(\textbf{X}),\cos(\textbf{X}),\ln(\textbf{X})]$

The following results correspond to the figures presented in Section 6. Table
A.2 details what values will be recorded.

Table A.2: Reference table Label | Interpretation
---|---
$\varepsilon_{\textbf{x}}$ | Error in the data - $\varepsilon_{\textbf{x}}=\lVert\textbf{x}-\tilde{\textbf{x}}\rVert$
Model | ordered triple corresponding to the form of the $\Theta$-library. See Table A.1 for reference.
len($\Theta$) | Length of $\Theta$
$\sigma_{\textrm{cutoff}}$ | Cutoff value used for library choice
$\delta$ |  $\sigma$\- difference as calculated in Equation (5.2)
count | count as calculated in Equation (5.1) taking into consideration the threshold value
$\sum_{j}\lVert\Gamma\xi_{cons,j}\rVert_{2}$ | Sum of 2-norms of found conservation(s)

###  A.1 Example 1

Below are the tables corresponding to the figures in Section 6.1.

$\varepsilon_{\textbf{x}}$ | Model | $\textrm{len}(\Theta)$ | $\sigma_{\textrm{cutoff}}$ | $\delta$ | count | $\sum_{j}\lVert\Gamma*\xi_{j}\rVert$
---|---|---|---|---|---|---
0 | (1, 0, 0) | 3.0 | 1.0e-10 | 4.0579 | 1.0 | 2.1789e-14
(2, 0, 0) | 9.0 | 1.0e-10 | 0.23527 | 4.0 | 8.1832e-14
(3, 0, 0) | 19.0 | 1.0e-10 | 0.0039731 | 10.0 | 1.7251e-13
(1, 0, 1) | 6.0 | 1.0e-10 | 0.084417 | 1.0 | 8.3465e-14
(2, 0, 1) | 12.0 | 1.0e-10 | 0.0021132 | 4.0 | 3.5442e-12
(1, 1, 0) | 9.0 | 1.0e-10 | 0.0039081 | 1.0 | 4.2259e-12
(2, 1, 0) | 15.0 | 1.0e-10 | 0.000077496 | 4.0 | 9.0724e-12
(1, 1, 1) | 12.0 | 1.0e-10 | 0.000023991 | 1.0 | 1.4756e-14
(2, 1, 1) | 18.0 | 1.0e-10 | 3.647e-7 | 4.0 | 4.4435e-14
Recovered Law: $0.57735x_{1}+0.57735x_{2}+0.57735x_{3}=C$
Reduced Law: $x_{1}+x_{2}+x_{3}=C$
5.4849e-10 | (1, 0, 0) | 3.0 | 5.1902e-6 | 4.0579 | 1.0 | 3.3263e-8
(2, 0, 0) | 9.0 | 8.9898e-6 | 0.23527 | 4.0 | 1.1613e-7
(3, 0, 0) | 19.0 | 0.000013062 | 0.0039729 | 10.0 | 2.2526e-7
(1, 0, 1) | 6.0 | 8.9898e-6 | 0.0039081 | 1.0 | 3.0055e-8
(2, 0, 1) | 12.0 | 0.00001038 | 0.0021131 | 4.0 | 1.0381e-7
(1, 1 , 0) | 9.0 | 8.9898e-6 | 0.0039081 | 1.0 | 3.0055e-8
(2, 1 , 0) | 15.0 | 0.000011606 | 0.00007741 | 4.0 | 9.1386e-8
(1, 1 , 1) | 12.0 | 0.00001038 | 0.000023963 | 1.0 | 2.6291e-8
(2, 1 , 1) | 18.0 | 0.000012713 | 0.000014258 | 6.0 | 8.735e-6
Recovered Law: $-0.57735x_{1}-0.57735x_{2}-0.57735x_{3}=C$
Reduced Law: $x_{1}+x_{2}+x_{3}=C$
4.3606e-05 | (1, 0, 0) | 3.0 | 0.0095964 | 4.0563 | 1.0 | 0.001608
(2, 0, 0) | 9.0 | 0.016621 | 0.22969 | 4.0 | 0.00534
(3, 0, 0) | 19.0 | 0.02415 | 0.035632 | 11.0 | 0.015822
(1, 0, 1) | 6.0 | 0.013571 | 0.082811 | 1.0 | 0.0015219
(2, 0, 1) | 12.0 | 0.019193 | 0.01559 | 5.0 | 0.0071915
(1, 1 , 0) | 9.0 | 0.016621 | 0.016893 | 2.0 | 0.005356
(2, 1 , 0) | 15.0 | 0.021458 | 0.027648 | 8.0 | 0.012539
(1, 1 , 1) | 12.0 | 0.019193 | 0.023277 | 5.0 | 0.0093485
(2, 1 , 1) | 18.0 | 0.023506 | 0.037703 | 11.0 | 0.017979
Recovered Law: $-0.57732x_{1}-0.57739x_{2}-0.57735x_{3}=C$
Reduced Law: $0.9999x_{1}+x_{2}+0.9999x_{3}=C$
Table A.3: Table of values corresponding to $N=20$ for Example 1 (the left graphs of Figure 6.1) $\varepsilon_{\textbf{x}}$ | Model | $\textrm{len}(\Theta)$ | $\sigma_{\textrm{cutoff}}$ | $\delta$ | count | $\sum_{j}\lVert\Gamma*\xi_{j}\rVert$
---|---|---|---|---|---|---
0.0 | (1, 0, 0) | 3.0 | 1.0e-10 | 8.8148 | 1.0 | 3.5381e-14
(2, 0, 0) | 9.0 | 1.0e-10 | 0.37496 | 4.0 | 1.4089e-13
(3, 0, 0) | 19.0 | 1.0e-10 | 0.0032626 | 10.0 | 2.9205e-13
(1, 0, 1) | 6.0 | 1.0e-10 | 0.023855 | 1.0 | 4.6508e-14
(2, 0, 1) | 12.0 | 1.0e-10 | 0.0033704 | 4.0 | 1.5425e-12
(3, 0, 1) | 22.0 | 1.0e-10 | 0.00006919 | 11.0 | 2.9114e-13
(1, 1, 0) | 9.0 | 1.0e-10 | 0.0063047 | 1.0 | 8.028e-12
(2, 1, 0) | 15.0 | 1.0e-10 | 0.000032995 | 4.0 | 1.1258e-13
(3, 1, 0) | 25.0 | 1.0e-10 | 6.6346e-7 | 10.0 | 2.6933e-13
(1, 1, 1) | 12.0 | 1.0e-10 | 5.1936e-6 | 1.0 | 2.3335e-14
(2, 1, 1) | 18.0 | 1.0e-10 | 5.581e-7 | 4.0 | 1.0469e-13
(3, 1, 1) | 28.0 | 1.0e-10 | 4.0946e-8 | 11.0 | 3.1179e-13
Recovered Law: $0.57735x_{1}+0.57735x_{2}+0.57735x_{3}=C$
Reduced Law: $x_{1}+x_{2}+x_{3}=C$
4.7594e-09 | (1, 0, 0) | 3.0 | 0.000010558 | 8.8148 | 1.0 | 2.0206e-8
(2, 0, 0) | 9.0 | 0.000018288 | 0.37496 | 4.0 | 6.7902e-8
(3, 0, 0) | 19.0 | 0.000026571 | 0.0032625 | 10.0 | 1.5507e-7
(1, 0, 1) | 6.0 | 0.000014932 | 0.023855 | 1.0 | 1.9563e-8
(2, 0, 1) | 12.0 | 0.000021117 | 0.0033704 | 4.0 | 6.5934e-8
(3, 0, 1) | 22.0 | 0.000028592 | 0.000069042 | 11.0 | 1.5934e-7
(1, 1, 0) | 9.0 | 0.000018288 | 0.0063047 | 1.0 | 1.8616e-8
(2, 1, 0) | 15.0 | 0.000023609 | 0.000032933 | 4.0 | 6.5453e-8
(3, 1, 0) | 25.0 | 0.000030479 | 0.000083452 | 13.0 | 6.8034e-6
(1, 1, 1) | 12.0 | 0.000021117 | 0.00026537 | 2.0 | 5.2107e-6
(2, 1, 1) | 18.0 | 0.000025863 | 0.000057904 | 6.0 | 2.2486e-6
(3, 1, 1) | 28.0 | 0.000032256 | 0.000081564 | 16.0 | 0.000020748
Recovered Law: $-0.57735x_{1}-0.57735x_{2}-0.57735x_{3}=C$
Reduced Law: $x_{1}+x_{2}+x_{3}=C$
4.5790e-05 | (1, 0, 0) | 3.0 | 0.022169 | 8.8123 | 1.0 | 0.0027741
(2, 0, 0) | 9.0 | 0.038397 | 0.36689 | 4.0 | 0.0094069
(3, 0, 0) | 19.0 | 0.05579 | 0.059744 | 12.0 | 0.076484
(1, 0, 1) | 6.0 | 0.031351 | 0.503 | 2.0 | 0.026322
(2, 0, 1) | 12.0 | 0.044337 | 0.37732 | 7.0 | 0.077914
(3, 0, 1) | 22.0 | 0.060033 | 0.056876 | 15.0 | 0.08438
(1, 1, 0) | 9.0 | 0.038397 | 0.03069 | 3.0 | 0.020124
(2, 1, 0) | 15.0 | 0.049571 | 0.063248 | 9.0 | 0.041129
(3, 1, 0) | 25.0 | 0.063995 | 0.061198 | 18.0 | 0.081371
(1, 1, 1) | 12.0 | 0.044337 | 0.032679 | 6.0 | 0.043595
(2, 1, 1) | 18.0 | 0.054302 | 0.062429 | 12.0 | 0.06318
(3, 1, 1) | 28.0 | 0.067726 | 0.058391 | 21.0 | 0.088529
Recovered Law: $0.57731x_{1}+0.57737x_{2}+0.57737x_{3}=C$
Reduced Law: $0.9999x_{1}+x_{2}+0.9999x_{3}=C$
Table A.4: Table of values corresponding to $N=100$ for Example 1 (the right
graphs of Figure 6.1)

###  A.2 Example 2

Below are the tables corresponding to the figures in Section 6.2.

$\varepsilon_{\textbf{x}}$ | Model | $\textrm{len}(\Theta)$ | $\sigma_{\textrm{cutoff}}$ | $\delta$ | count | $\sum_{j}\lVert\Gamma*\xi_{j}\rVert$
---|---|---|---|---|---|---
0 | (1, 0, 0) | 4.0 | 1.0e-10 | 1.8474 | 2.0 | 1.4716e-14
(2, 0, 0) | 14.0 | 1.0e-10 | 0.063937 | 9.0 | 4.1111e-14
(1, 0, 1) | 8.0 | 1.0e-10 | 0.0034213 | 2.0 | 1.3616e-12
(2, 0, 1) | 18.0 | 1.0e-10 | 0.00013746 | 9.0 | 6.3428e-12
(1, 1, 0) | 12.0 | 1.0e-10 | 0.00030492 | 4.0 | 1.7433e-12
(1, 1, 1) | 16.0 | 1.0e-10 | 3.8709e-7 | 4.0 | 1.0844e-11
Recovered Law: $\\!\begin{aligned}
&-0.31642x_{1}+0.63246x_{2}+0.31604x_{3}+0.63246x_{4}=C_{1}\\\
&0.70702x_{1}+0.00017x_{2}+0.70719x_{3}+0.00017x_{4}=C_{2}\end{aligned}$
Reduced Law: $\\!\begin{aligned} &x_{1}+x_{3}=C_{1}\\\
&x_{2}+x_{3}+x_{4}=C_{2}\end{aligned}$
5.4588e-10 | (1, 0, 0) | 4.0 | 5.9742e-6 | 1.8474 | 2.0 | 3.4017e-8
(2, 0, 0) | 14.0 | 0.000011177 | 0.063937 | 9.0 | 9.1189e-8
(1, 0, 1) | 8.0 | 8.4488e-6 | 0.0034213 | 2.0 | 3.0698e-8
(2, 0, 1) | 18.0 | 0.000012673 | 0.00013741 | 9.0 | 8.0334e-8
(1, 1, 0) | 12.0 | 0.000010348 | 0.0003049 | 4.0 | 3.4966e-8
(1, 1, 1) | 16.0 | 0.000011948 | 0.000017266 | 5.0 | 4.149e-7
Recovered Law: $\\!\begin{aligned}
&0.47612x_{1}-0.61411x_{2}-0.13799x_{3}-0.61411x_{4}=C_{1}\\\
&-0.61099x_{1}-0.15122x_{2}-0.76221x_{3}-0.15122x_{4}=C_{2}\end{aligned}$
Reduced Law: $\\!\begin{aligned} &x_{1}+x_{3}=C_{1}\\\
&x_{2}+x_{3}+x_{4}=C_{2}\end{aligned}$
5.5253e-05 | (1, 0, 0) | 4.0 | 0.012975 | 1.8447 | 2.0 | 0.0041992
(2, 0, 0) | 14.0 | 0.024275 | 0.056546 | 9.0 | 0.011785
(1, 0, 1) | 8.0 | 0.01835 | 0.11553 | 4.0 | 0.025573
(2, 0, 1) | 18.0 | 0.027525 | 0.10737 | 13.0 | 0.037722
(1, 1, 0) | 12.0 | 0.022474 | 0.020826 | 7.0 | 0.014176
(1, 1, 1) | 16.0 | 0.025951 | 0.070681 | 11.0 | 0.024704
Recovered Law: $\\!\begin{aligned}
&-0.05178x_{1}+0.5933x_{2}+0.5414x_{3}+0.59343x_{4}=C_{1}\\\
&-0.7728x_{1}+0.2189x_{2}-0.5538x_{3}+0.2189x_{4}=C_{2}\end{aligned}$
Reduced Law: $\\!\begin{aligned} &x_{1}+0.9998x_{3}-(3.7e-5)x_{2}=C_{1}\\\
&0.9998x_{2}+0.9996x_{3}+x_{4}=C_{2}\end{aligned}$
Table A.5: Table of values corresponding to $N=20$ for Example 2 (the left graphs of Figure 6.2) $\varepsilon_{\textbf{x}}$ | Model | $\textrm{len}(\Theta)$ | $\sigma_{\textrm{cutoff}}$ | $\delta$ | count | $\sum_{j}\lVert\Gamma*\xi_{j}\rVert$
---|---|---|---|---|---|---
0.0 | (1, 0, 0) | 4.0 | 1.0e-10 | 3.9512 | 2.0 | 3.0341e-14
(2, 0, 0) | 14.0 | 1.0e-10 | 0.025957 | 9.0 | 9.1177e-14
(3, 0, 0) | 34.0 | 1.0e-10 | 0.00029318 | 25.0 | 2.3278e-13
(1, 0, 1) | 8.0 | 1.0e-10 | 0.0036841 | 2.0 | 3.9258e-12
(2, 0, 1) | 18.0 | 1.0e-10 | 0.00013799 | 9.0 | 8.1403e-12
(3, 0, 1) | 38.0 | 1.0e-10 | 3.6522e-6 | 25.0 | 8.3401e-11
(1, 1, 0) | 12.0 | 1.0e-10 | 0.000094638 | 4.0 | 5.8228e-12
(2, 1, 0) | 22.0 | 1.0e-10 | 2.5575e-6 | 11.0 | 1.2546e-11
(3, 1, 0) | 42.0 | 1.0e-10 | 2.1804e-8 | 27.0 | 2.0866e-13
(1, 1, 1) | 16.0 | 1.0e-10 | 1.3843e-6 | 4.0 | 2.0776e-11
(2, 1, 1) | 26.0 | 1.0e-10 | 1.8045e-8 | 11.0 | 2.1566e-11
(3, 1, 1) | 46.0 | 1.0e-10 | 1.5864e-9 | 28.0 | 8.3203e-11
Recovered Law: $\\!\begin{aligned}
&0.3169x_{1}-0.6324x_{2}-0.3154x_{3}-0.6324x_{4}=C_{1}\\\
&-0.7067x_{1}-0.0006x_{2}-0.7074x_{3}-0.0006x_{4}=C_{2}\end{aligned}$
Reduced Law: $\\!\begin{aligned} &x_{1}+0.9999x_{3}=C_{1}\\\
&x_{2}+0.9999x_{3}+x_{4}=C_{2}\end{aligned}$
6.5220e-10 | (1, 0, 0) | 4.0 | 0.000015041 | 3.9512 | 2.0 | 4.7404e-8
(2, 0, 0) | 14.0 | 0.00002814 | 0.025957 | 9.0 | 1.336e-7
(3, 0, 0) | 34.0 | 0.000043852 | 0.00029305 | 25.0 | 2.5312e-7
(1, 0, 1) | 8.0 | 0.000021272 | 0.0036841 | 2.0 | 4.4098e-8
(2, 0, 1) | 18.0 | 0.000031907 | 0.00013793 | 9.0 | 1.1969e-7
(3, 0, 1) | 38.0 | 0.00004636 | 0.000074965 | 27.0 | 0.000010574
(1, 1, 0) | 12.0 | 0.000026052 | 0.000094605 | 4.0 | 5.228e-8
(2, 1, 0) | 22.0 | 0.000035275 | 0.00007154 | 13.0 | 0.000015945
(3, 1, 0) | 42.0 | 0.000048739 | 0.00018352 | 32.0 | 7.4805e-6
(1, 1, 1) | 16.0 | 0.000030083 | 0.000077422 | 6.0 | 0.000011873
(2, 1, 1) | 26.0 | 0.000038348 | 0.000074075 | 16.0 | 0.000021881
(3, 1, 1) | 46.0 | 0.000051007 | 0.000072377 | 35.0 | 0.000015447
Recovered Law: $\\!\begin{aligned}
&0.6582x_{1}-0.5237x_{2}+0.1345x_{3}-0.5237x_{4}=C_{1}\\\
&0.4082x_{1}+0.3545x_{2}+0.7628x_{3}+0.3545x_{4}=C_{2}\end{aligned}$
Reduced Law: $\\!\begin{aligned} &x_{1}+x_{3}x_{4}=C_{1}\\\
&x_{2}+x_{3}+x_{4}=C_{2}\end{aligned}$
6.6525e-05 | (1, 0, 0) | 4.0 | 0.032836 | 3.9484 | 2.0 | 0.0047325
(2, 0, 0) | 14.0 | 0.061431 | 0.26587 | 10.0 | 0.0397
(3, 0, 0) | 34.0 | 0.095733 | 0.096853 | 29.0 | 0.12524
(1, 0, 1) | 8.0 | 0.046437 | 0.17064 | 4.0 | 0.029846
(2, 0, 1) | 18.0 | 0.069656 | 0.32509 | 14.0 | 0.11871
(3, 0, 1) | 38.0 | 0.10121 | 0.10387 | 33.0 | 0.1401
(1, 1, 0) | 12.0 | 0.056874 | 0.13477 | 8.0 | 0.035431
(2, 1, 0) | 22.0 | 0.077008 | 0.30556 | 18.0 | 0.068139
(3, 1, 0) | 42.0 | 0.1064 | 0.09839 | 37.0 | 0.12802
(1, 1, 1) | 16.0 | 0.065672 | 0.22147 | 12.0 | 0.088162
(2, 1, 1) | 26.0 | 0.083716 | 0.35581 | 22.0 | 0.1323
(3, 1, 1) | 46.0 | 0.11135 | 0.10566 | 41.0 | 0.14254
Recovered Law: $\\!\begin{aligned}
&-0.1717x_{1}-0.5057x_{2}-0.6775x_{3}-0.5056x_{4}=C_{1}\\\
&0.7552x_{1}-0.3798x_{2}+0.3755x_{3}-0.3798x_{4}=C_{2}\end{aligned}$
Reduced Law: $\\!\begin{aligned} &x_{1}+x_{3}-(5.3e-5)x_{4}=C_{1}\\\
&x_{2}+1.0002x_{3}+0.9999x_{4}=C_{2}\end{aligned}$
Table A.6: Table of values corresponding to $N=100$ for Example 2 (the right
graphs of Figure 6.2)

###  A.3 Example 3

Below are the tables corresponding to the figures in Section 6.3.

$\varepsilon_{\textbf{x}}$ | Model | $\textrm{len}(\Theta)$ | $\sigma_{\textrm{cutoff}}$ | $\delta$ | count | $\sum_{j}\lVert\Gamma*\xi_{j}\rVert$
---|---|---|---|---|---|---
0 | (1, 0, 0) | 3.0 | 0.00001 | NaN | 0 | NaN
(2, 0, 0) | 9.0 | 0.00001 | 0.000072085 | 3.0 | 0.000011441
(3, 0, 0) | 19.0 | 0.00001 | 0.000045224 | 11.0 | 4.4834e-6
(1, 0, 1) | 6.0 | 0.00001 | 0.0027865 | 1.0 | 1.9743e-6
(2, 0, 1) | 12.0 | 0.00001 | 0.000027509 | 4.0 | 1.9893e-6
(1, 1, 0) | 9.0 | 0.00001 | 0.000017707 | 2.0 | 6.1443e-6
(2, 1, 0) | 15.0 | 0.00001 | 8.0627e-6 | 7.0 | 3.5526e-6
(1, 1, 1) | 12.0 | 0.00001 | 0.000014728 | 4.0 | 9.7154e-6
(2, 1, 1) | 18.0 | 0.00001 | 0.000010709 | 9.0 | 1.3221e-6
Recovered Law: $\\!\begin{aligned}
-0.34852x_{1}&-0.34831x_{2}+0.0003x_{3}+0.0001\ln(x_{1})\\\
&-0.00005\ln(x_{2})+0.87018\ln(x_{3})=C\end{aligned}$
Reduced Law: $\\!\begin{aligned}
x_{1}&+0.9994x_{2}-0.0008x_{3}-0.0003\ln(x_{1})\\\
&+0.0001\ln(x_{2})-2.4968\ln(x_{3})=C\end{aligned}$
4.5634e-10 | (1, 0, 0) | 3.0 | 0.00001 | NaN | 0 | NaN
(2, 0, 0) | 9.0 | 0.00001 | 0.000072089 | 3.0 | 0.000011438
(3, 0, 0) | 19.0 | 0.000011554 | 0.000045221 | 11.0 | 4.4825e-6
(1, 0, 1) | 6.0 | 0.00001 | 0.0027865 | 1.0 | 1.9731e-6
(2, 0, 1) | 12.0 | 0.00001 | 0.000027509 | 4.0 | 1.989e-6
(1, 1, 0) | 9.0 | 0.00001 | 0.000017708 | 2.0 | 6.1442e-6
(2, 1, 0) | 15.0 | 0.000010266 | 8.0607e-6 | 7.0 | 3.5526e-6
(1, 1, 1) | 12.0 | 0.00001 | 0.000014728 | 4.0 | 9.7154e-6
(2, 1, 1) | 18.0 | 0.000011246 | 0.000010709 | 9.0 | 1.3218e-6
Recovered Law: $\\!\begin{aligned}
-0.34852x_{1}&-0.34831x_{2}+0.0003x_{3}+0.0001\ln(x_{1})\\\
&-0.00005\ln(x_{2})+0.87018\ln(x_{3})=C\end{aligned}$
Reduced Law: $\\!\begin{aligned}
x_{1}&+0.9994x_{2}-0.0008x_{3}-0.0001\ln(x_{1})\\\
&+0.0001\ln(x_{2})-2.4968\ln(x_{3})=C\end{aligned}$
3.3100e-05 | (1, 0, 0) | 3.0 | 0.0079854 | NaN | 0 | NaN
(2, 0, 0) | 9.0 | 0.013831 | 0.014003 | 4.0 | 0.0044729
(3, 0, 0) | 19.0 | 0.020096 | 0.042745 | 14.0 | 0.014357
(1, 0, 1) | 6.0 | 0.011293 | 0.020824 | 1.0 | 0.00095611
(2, 0, 1) | 12.0 | 0.015971 | 0.058471 | 7.0 | 0.0069299
(1, 1, 0) | 9.0 | 0.013831 | 0.021211 | 5.0 | 0.015767
(2, 1, 0) | 15.0 | 0.017856 | 0.019962 | 10.0 | 0.0051929
(1, 1, 1) | 12.0 | 0.015971 | 0.035169 | 7.0 | 0.0047944
(2, 1, 1) | 18.0 | 0.01956 | 0.059301 | 13.0 | 0.0085682
Recovered Law: $\\!\begin{aligned}
-0.0378x_{1}&-0.0630x_{2}-0.0352x_{3}-0.0367\ln(x_{1})\\\
&-0.0228\ln(x_{2})+0.93182\ln(x_{3})=C\end{aligned}$
Reduced Law: $\\!\begin{aligned}
x_{1}&+1.665x_{2}+9.3197x_{3}+0.9798\ln(x_{1})+0.6024\ln(x_{2})\\\
&-2.4619\ln(x_{3})=C\end{aligned}$
Table A.7: Table of values corresponding to $N=20$ for Example 3 (the left graphs of Figure 6.3) $\varepsilon_{\textbf{x}}$ | Model | $\textrm{len}(\Theta)$ | $\sigma_{\textrm{cutoff}}$ | $\delta$ | count | $\sum_{j}\lVert\Gamma*\xi_{j}\rVert$
---|---|---|---|---|---|---
0.0 | (1, 0, 0) | 3.0 | 1.0e-7 | NaN | 0 | NaN
(2, 0, 0) | 9.0 | 1.0e-7 | 6.3238e-7 | 1.0 | 3.0538e-8
(3, 0, 0) | 19.0 | 1.0e-7 | 4.7212e-8 | 9.0 | 6.0347e-8
(1, 0, 1) | 6.0 | 1.0e-7 | 0.0055963 | 1.0 | 2.1413e-8
(2, 0, 1) | 12.0 | 1.0e-7 | 2.5271e-7 | 3.0 | 7.0863e-8
(3, 0, 1) | 22.0 | 1.0e-7 | 7.7252e-8 | 11.0 | 2.883e-8
(1, 1, 0) | 9.0 | 1.0e-7 | 9.3748e-7 | 1.0 | 1.9842e-8
(2, 1, 0) | 15.0 | 1.0e-7 | 5.2665e-7 | 6.0 | 8.9328e-8
(3, 1, 0) | 25.0 | 1.0e-7 | 9.3917e-8 | 14.0 | 7.7068e-9
(1, 1, 1) | 12.0 | 1.0e-7 | 1.2008e-6 | 3.0 | 7.9501e-8
(2, 1, 1) | 18.0 | 1.0e-7 | 2.2626e-7 | 8.0 | 8.9132e-8
(3, 1, 1) | 28.0 | 1.0e-7 | 7.5189e-8 | 17.0 | 5.3318e-8
Recovered Law: $-0.34815x_{1}-0.34816x_{2}+0.87038\ln(x_{3})=C$
Reduced Law: $x_{1}+x_{2}-2.5\ln(x_{3})=C$
6.0004e-10 | (1, 0, 0) | 3.0 | 0.000012322 | NaN | 0 | NaN
(2, 0, 0) | 9.0 | 0.000021342 | 0.00018738 | 3.0 | 4.358e-6
(3, 0, 0) | 19.0 | 0.00003101 | 0.00015757 | 12.0 | 7.9755e-6
(1, 0, 1) | 6.0 | 0.000017426 | 0.0055963 | 1.0 | 1.5939e-8
(2, 0, 1) | 12.0 | 0.000024644 | 0.00012653 | 5.0 | 7.567e-6
(3, 0, 1) | 22.0 | 0.000033368 | 0.00012176 | 14.0 | 7.1383e-6
(1, 1, 0) | 9.0 | 0.000021342 | 0.000033311 | 2.0 | 9.7926e-7
(2, 1, 0) | 15.0 | 0.000027553 | 0.000057519 | 8.0 | 6.3311e-6
(3, 1, 0) | 25.0 | 0.00003557 | 0.00016553 | 18.0 | 0.000010562
(1, 1, 1) | 12.0 | 0.000024644 | 0.000024303 | 4.0 | 1.3299e-6
(2, 1, 1) | 18.0 | 0.000030182 | 0.0001407 | 11.0 | 0.000032848
(3, 1, 1) | 28.0 | 0.000037644 | 0.00013138 | 20.0 | 7.676e-6
Recovered Law: $-0.34816x_{1}-0.34816x_{2}+0.87039\ln(x_{3})=C$
Reduced Law: $x_{1}+x_{2}-2.5\ln(x_{3})=C$
6.3398e-05 | (1, 0, 0) | 3.0 | 0.027539 | NaN | 0 | NaN
(2, 0, 0) | 9.0 | 0.047698 | 0.090692 | 5.0 | 0.0044301
(3, 0, 0) | 19.0 | 0.069304 | 0.25342 | 15.0 | 0.028176
(1, 0, 1) | 6.0 | 0.038946 | 0.083778 | 2.0 | 0.0055469
(2, 0, 1) | 12.0 | 0.055077 | 0.23125 | 8.0 | 0.020959
(3, 0, 1) | 22.0 | 0.074575 | 0.47261 | 18.0 | 0.048114
(1, 1, 0) | 9.0 | 0.047698 | 0.065736 | 5.0 | 0.0038466
(2, 1, 0) | 15.0 | 0.061578 | 0.12528 | 11.0 | 0.0071169
(3, 1, 0) | 25.0 | 0.079497 | 0.28407 | 21.0 | 0.02934
(1, 1, 1) | 12.0 | 0.055077 | 0.13098 | 8.0 | 0.018194
(2, 1, 1) | 18.0 | 0.067456 | 0.24255 | 14.0 | 0.025508
(3, 1, 1) | 28.0 | 0.084132 | 0.47579 | 24.0 | 0.050587
Recovered Law: $\\!\begin{aligned}
&-0.624x_{1}-0.277x_{2}+0.562x_{3}+0.187\ln(x_{1})\\\ &\hskip
8.00002pt-0.088\ln(x_{2})-0.415\ln(x_{3})=C_{1}\\\
&-0.379x_{1}-0.360x_{2}+0.026x_{3}+0.009\ln(x_{1})\\\ &\hskip
8.00002pt-0.005\ln(x_{2})+0.851\ln(x_{3})=C_{1}\end{aligned}$
Reduced Law: $\\!\begin{aligned} &x_{1}-1.63x_{3}-0.054\ln(x_{1})\\\ &\hskip
8.00002pt+0.253\ln(x_{2})+3.223\ln(x_{3})=C_{1}\\\
&x_{2}+1.642x_{3}+0.542\ln(x_{1})\\\ &\hskip
8.00002pt-0.252\ln(x_{2})-5.752\ln(x_{3})=C_{1}\end{aligned}$
Table A.8: Table of values corresponding to $N=100$ for Example 3 (the right
graphs of Figure 6.3)

###  A.4 Example 4

Below are the tables corresponding to the figures in Section 6.4.

$\varepsilon_{\textbf{x}}$ | Model | $\textrm{len}(\Theta)$ | $\sigma_{\textrm{cutoff}}$ | $\delta$ | count | $\sum_{j}\lVert\Gamma*\xi_{j}\rVert$
---|---|---|---|---|---|---
0 | (1, 0, 0) | 2.0 | 1e-10 | NaN | 0 | NaN
(2, 0, 0) | 5.0 | 1e-10 | NaN | 0 | NaN
(3, 0, 0) | 9.0 | 1e-10 | NaN | 0 | NaN
(1, 0, 1) | 4.0 | 1e-10 | NaN | 0 | NaN
(2, 0, 1) | 7.0 | 1e-10 | NaN | 0 | NaN
(3, 0, 1) | 11.0 | 1e-10 | NaN | 0 | NaN
(1, 1, 0) | 6.0 | 1e-10 | NaN | 0 | NaN
(2, 1, 0) | 9.0 | 1e-10 | NaN | 0 | NaN
(3, 1, 0) | 13.0 | 1e-10 | NaN | 0 | NaN
(1, 1, 1) | 8.0 | 1e-10 | NaN | 0 | NaN
(2, 1, 1) | 11.0 | 1e-10 | NaN | 0 | NaN
(3, 1, 1) | 15.0 | 1e-10 | NaN | 0 | NaN
Recovered Law: None |
4.2888e-10 | (1, 0, 0) | 2.0 | 3.5969e-6 | NaN | 0 | NaN
(2, 0, 0) | 5.0 | 5.6871e-6 | NaN | 0 | NaN
(3, 0, 0) | 9.0 | 7.6301e-6 | NaN | 0 | NaN
(1, 0, 1) | 4.0 | 5.0867e-6 | NaN | 0 | NaN
(2, 0, 1) | 7.0 | 6.7291e-6 | NaN | 0 | NaN
(3, 0, 1) | 11.0 | 8.4354e-6 | 0.000037867 | 1.0 | 4.5746e-6
(1, 1, 0) | 6.0 | 6.2299e-6 | NaN | 0 | NaN
(2, 1, 0) | 9.0 | 7.6301e-6 | NaN | 0 | NaN
(3, 1, 0) | 13.0 | 9.1702e-6 | 0.000010112 | 2.0 | 1.1658e-6
(1, 1, 1) | 8.0 | 7.1937e-6 | NaN | 0 | NaN
(2, 1, 1) | 11.0 | 8.4354e-6 | 0.000089774 | 2.0 | 0.000010243
(3, 1, 1) | 15.0 | 9.8504e-6 | 0.000040058 | 4.0 | 2.5907e-6
Recovered Law: $\\!\begin{aligned}
&-0.2423x_{1}+0.7977x_{2}-0.0143x_{1}^{2}-0.0055x_{1}x_{2}-0.1071x_{2}^{2}-0.0778\sin(x_{1})\\\
&-0.0386\sin(x_{2})-0.1794\cos(x_{1})+0.1763\cos(x_{2})+0.1742\ln(x_{1})-0.4382\ln(x_{2})=C_{1}\\\
&0.7738x_{1}+0.1106x_{2}-0.2916x_{1}^{2}-0.0009x_{1}x_{2}-0.0145x_{2}^{2}-0.5181\sin(x_{1})\\\
&-0.0040\sin(x_{2})-0.1684\cos(x_{1})+0.0244\cos(x_{2})-0.0504\ln(x_{1})-0.0624\ln(x_{2})=C_{2}\end{aligned}$
4.1076e-05 | (1, 0, 0) | 2.0 | 0.0075294 | NaN | 0 | NaN
(2, 0, 0) | 5.0 | 0.011905 | NaN | 0 | NaN
(3, 0, 0) | 9.0 | 0.015972 | 0.034424 | 3.0 | 0.0063841
(1, 0, 1) | 4.0 | 0.010648 | NaN | 0 | NaN
(2, 0, 1) | 7.0 | 0.014086 | 0.042318 | 2.0 | 0.0062048
(3, 0, 1) | 11.0 | 0.017658 | 0.032374 | 5.0 | 0.013121
(1, 1, 0) | 6.0 | 0.013041 | 0.019994 | 1.0 | 0.0025748
(2, 1, 0) | 9.0 | 0.015972 | 0.045666 | 4.0 | 0.013106
(3, 1, 0) | 13.0 | 0.019196 | 0.035502 | 7.0 | 0.0088183
(1, 1, 1) | 8.0 | 0.015059 | 0.02468 | 3.0 | 0.0078267
(2, 1, 1) | 11.0 | 0.017658 | 0.054078 | 6.0 | 0.018811
(3, 1, 1) | 15.0 | 0.02062 | 0.033739 | 9.0 | 0.014411
Recovered Law: $\\!\begin{aligned}
&0.6572x_{1}+0.5337x_{2}-0.1609x_{1}^{2}-0.0370x_{1}x_{2}\\\ &\hskip
8.00002pt-0.0706x_{2}^{2}-0.2599\ln(x_{1})-0.4283\ln(x_{2})=C_{1}\\\
&-0.5800x_{1}+0.5550x_{2}+0.1508x_{1}^{2}-0.0342x_{1}x_{2}\\\ &\hskip
8.00002pt-0.0735x_{2}^{2}+0.3492\ln(x_{1})-0.4520\ln(x_{2})=C_{2}\end{aligned}$
Table A.9: Table of values corresponding to $N=20$ for Example 4 (the left graphs of Figure 6.4) $\varepsilon_{\textbf{x}}$ | Model | $\textrm{len}(\Theta)$ | $\sigma_{\textrm{cutoff}}$ | $\delta$ | count | $\sum_{j}\lVert\Gamma*\xi_{j}\rVert$
---|---|---|---|---|---|---
0 | (1, 0, 0) | 2.0 | 1e-10 | NaN | 0 | NaN
(2, 0, 0) | 5.0 | 1e-10 | NaN | 0 | NaN
(3, 0, 0) | 9.0 | 1e-10 | NaN | 0 | NaN
(1, 0, 1) | 4.0 | 1e-10 | NaN | 0 | NaN
(2, 0, 1) | 7.0 | 1e-10 | NaN | 0 | NaN
(3, 0, 1) | 11.0 | 1e-10 | NaN | 0 | NaN
(1, 1, 0) | 6.0 | 1e-10 | NaN | 0 | NaN
(2, 1, 0) | 9.0 | 1e-10 | NaN | 0 | NaN
(3, 1, 0) | 13.0 | 1e-10 | NaN | 0 | NaN
(1, 1, 1) | 8.0 | 1e-10 | NaN | 0 | NaN
(2, 1, 1) | 11.0 | 1e-10 | NaN | 0 | NaN
(3, 1, 1) | 15.0 | 1e-10 | NaN | 0 | NaN
Recovered Law: None
3.6436e-10 | (1, 0, 0) | 2.0 | 7.2145e-6 | NaN | 0 | NaN
(2, 0, 0) | 5.0 | 0.000011407 | NaN | 0 | NaN
(3, 0, 0) | 9.0 | 0.000015304 | NaN | 0 | NaN
(1, 0, 1) | 4.0 | 0.000010203 | NaN | 0 | NaN
(2, 0, 1) | 7.0 | 6.7291e-6 | NaN | 0 | NaN
(3, 0, 1) | 11.0 | 0.000016919 | 0.000096483 | 1.0 | 5.2896e-6
(1, 1, 0) | 6.0 | 0.000012496 | NaN | 0 | NaN
(2, 1, 0) | 9.0 | 0.000015304 | NaN | 0 | NaN
(3, 1, 0) | 13.0 | 0.000018393 | 0.00004666 | 3.0 | 9.4759e-6
(1, 1, 1) | 8.0 | 0.000014429 | NaN | 0 | NaN
(2, 1, 1) | 11.0 | 0.000016919 | 0.000031836 | 2.0 | 0.000011024
(3, 1, 1) | 15.0 | 0.000019758 | 0.000139 | 5.0 | 0.000019939
Recovered Law: $\\!\begin{aligned}
&0.2608x_{1}-0.8158x_{2}-0.1429x_{1}^{2}+0.0951x_{1}x_{2}+0.2414x_{2}^{2}+0.0281x_{1}^{3}-0.0074x_{1}^{2}x_{2}\\\
&\hskip
9.00002pt-0.0235x-1x_{2}^{2}-0.0287x_{2}^{2}-0.1261\ln(x_{1})+0.4008\ln(x_{2})=C\end{aligned}$
3.7872e-05 | (1, 0, 0) | 2.0 | 0.015949 | NaN | 0 | NaN
(2, 0, 0) | 5.0 | 0.025217 | 0.24234 | 1.0 | 0.013766
(3, 0, 0) | 9.0 | 0.033832 | 0.02819 | 3.0 | 0.010148
(1, 0, 1) | 4.0 | 0.022555 | 0.17651 | 1.0 | 0.019882
(2, 0, 1) | 7.0 | 0.029837 | 0.46712 | 3.0 | 0.033594
(3, 0, 1) | 11.0 | 0.037403 | 0.019872 | 5.0 | 0.02294
(1, 1, 0) | 6.0 | 0.027624 | 0.23913 | 2.0 | 0.027542
(2, 1, 0) | 9.0 | 0.033832 | 0.41062 | 5.0 | 0.048114
(3, 1, 0) | 13.0 | 0.040661 | 0.031443 | 7.0 | 0.014386
(1, 1, 1) | 8.0 | 0.031897 | 0.024736 | 3.0 | 0.0092047
(2, 1, 1) | 11.0 | 0.037403 | 0.5477 | 7.0 | 0.062978
(3, 1, 1) | 15.0 | 0.043677 | 0.024525 | 9.0 | 0.024955
Recovered Law:
$-0.5016x_{1}+0.8013x_{2}+0.2218x_{1}^{2}-0.1194x_{1}x_{2}-0.2070x_{2}^{2}=C$
Table A.10: Table of values corresponding to $N=100$ for Example 4 (the right
graphs of Figure 6.4

###  A.5 MAPK Pathway

The dynamical system for the MAPK pathway as presented in [53] is below.

$\displaystyle Raf\rightleftarrows pRaf$ $\displaystyle MEK\rightleftarrows
pMEK$ $\displaystyle ERK\rightleftarrows pERK$ $\displaystyle
pRaf\rightleftarrows ppRaf$ $\displaystyle pMEK\rightleftarrows ppMEK$
$\displaystyle pERK\rightleftarrows ppERK$

$\displaystyle\dfrac{d[Raf]}{dt}$
$\displaystyle=\dfrac{V^{4}_{\textrm{max}}[pRaf]/K_{m_{4}}}{1+[ppRaf]/K_{m_{3}}+[pRaf]/K_{m_{4}}}-\dfrac{k_{1}^{\textrm{cat}}[RasGTP][Raf]/K_{m_{1}}}{1+[Raf]/K_{m_{1}}+[pRaf]/K_{m_{2}}}\cdot\dfrac{1+F[ppERK]/K_{f}}{1+[ppERK]/K_{f}}$
$\displaystyle\dfrac{d[pRaf]}{dt}$
$\displaystyle=\dfrac{k_{1}^{\textrm{cat}}[RasGTP][Raf]/K_{m_{1}}}{1+[Raf]/K_{m_{1}}+[pRaf]/K_{m_{2}}}\cdot\dfrac{1+F[ppERK]/K_{f}}{1+[ppERK]/K_{f}}+\dfrac{V^{3}_{\textrm{max}}[ppRaf]/K_{m_{3}}}{1+[ppRaf]/K_{m_{3}}+[pRaf]/K_{m_{4}}}$
$\displaystyle-\dfrac{k_{2}^{\textrm{cat}}[RasGTP][pRaf]/K_{m_{2}}}{1+[Raf]/K_{m_{1}}+[pRaf]/K_{m_{2}}}\cdot\dfrac{1+F[ppERK]/K_{f}}{1+[ppERK]/K_{f}}-\dfrac{V^{4}_{\textrm{max}}[pRaf]/K_{m_{4}}}{1+[ppRaf]/K_{m_{3}}+[pRaf]/K_{m_{4}}}$
$\displaystyle\dfrac{d[ppRaf]}{dt}$
$\displaystyle=\dfrac{k_{2}^{\textrm{cat}}[RasGTP][pRaf]/K_{m_{2}}}{1+[Raf]/K_{m_{1}}+[pRaf]/K_{m_{2}}}\cdot\dfrac{1+F[ppERK]/K_{f}}{1+[ppERK]/K_{f}}-\dfrac{V^{3}_{\textrm{max}}[ppRaf]/K_{m_{3}}}{1+[ppRaf]/K_{m_{3}}+[pRaf]/K_{m_{4}}}$
$\displaystyle\dfrac{d[MEK]}{dt}$
$\displaystyle=\dfrac{V^{8}_{\textrm{max}}[pMEK]/K_{m_{8}}}{1+[ppMEK]/K_{m_{7}}+[pMEK]/K_{m_{8}}}-\dfrac{k_{5}^{\textrm{cat}}[ppRaf][MEK]/K_{m_{5}}}{1+[MEK]/K_{m_{5}}+[pMEK]/K_{m_{6}}}$
$\displaystyle\dfrac{d[pMEK]}{dt}$
$\displaystyle=\dfrac{k_{5}^{\textrm{cat}}[ppRaf][MEK]/K_{m_{5}}}{1+[MEK]/K_{m_{5}}+[pMEK]/K_{m_{6}}}-\dfrac{k_{6}^{\textrm{cat}}[ppRaf][pMEK]/K_{m_{6}}}{1+[MEK]/K_{m_{5}}+[pMEK]/K_{m_{6}}}$
$\displaystyle+\dfrac{V^{7}_{\textrm{max}}[ppMEK]/K_{m_{7}}}{1+[ppMEK]/K_{m_{7}}+[pMEK]/K_{m_{8}}}-\dfrac{V^{8}_{\textrm{max}}[pMEK]/K_{m_{8}}}{1+[ppMEK]/K_{m_{7}}+[pMEK]/K_{m_{8}}}$
$\displaystyle\dfrac{d[ppMEK]}{dt}$
$\displaystyle=\dfrac{k_{6}^{\textrm{cat}}[ppRaf][pMEK]/K_{m_{6}}}{1+[MEK]/K_{m_{5}}+[pMEK]/K_{m_{6}}}-\dfrac{V^{7}_{\textrm{max}}[ppMEK]/K_{m_{7}}}{1+[ppMEK]/K_{m_{7}}+[pMEK]/K_{m_{8}}}$
$\displaystyle\dfrac{d[ERK]}{dt}$
$\displaystyle=\dfrac{V_{12}^{\textrm{max}}[pERK/K_{m_{12}}]}{1+[ppERK]/K_{m_{1}1}+[pERK]/K_{m_{12}}+[ERK]/K_{m_{13}}}-\dfrac{k_{9}^{\textrm{cat}}[ppMEK][ERK]/K_{m_{9}}}{1+[ERK]/K_{m_{9}}+[pERK]/K_{m_{10}}}$
$\displaystyle\dfrac{d[pERK]}{dt}$
$\displaystyle=\dfrac{k_{9}^{\textrm{cat}}[ppMEK][ERK]/K_{m_{9}}}{1+[ERK]/K_{m_{9}}+[pERK]/K_{m_{10}}}-\dfrac{k_{10}^{\textrm{cat}}[ppMEK][pERK]/K_{m_{10}}}{1+[ERK]/K_{m_{9}}+[pERK]/K_{m_{10}}}$
$\displaystyle+\dfrac{V_{11}^{\textrm{max}}[ppERK/K_{m_{11}}]}{1+[ppERK]/K_{m_{1}1}+[pERK]/K_{m_{12}}+[ERK]/K_{m_{13}}}-\dfrac{V_{12}^{\textrm{max}}[pERK/K_{m_{12}}]}{1+[ppERK]/K_{m_{1}1}+[pERK]/K_{m_{12}}+[ERK]/K_{m_{13}}}$
$\displaystyle\dfrac{d[ppERK]}{dt}$
$\displaystyle=\dfrac{k_{10}^{\textrm{cat}}[ppMEK][pERK]/K_{m_{10}}}{1+[ERK]/K_{m_{9}}+[pERK]/K_{m_{10}}}-\dfrac{V_{11}^{\textrm{max}}[ppERK/K_{m_{11}}]}{1+[ppERK]/K_{m_{1}1}+[pERK]/K_{m_{12}}+[ERK]/K_{m_{13}}}$

The following coefficients were used to generate data:

  * •

$V_{3}=2.5$, $V_{4}=3.75$, $V_{7}=3$, $V_{8}=3.75$, $V_{11}=3.75$, $V_{12}=5$

  * •

$K_{m_{1}}=100$, $K_{m_{2}}=200$, $K_{m_{3}}=50$, $K_{m_{4}}=100$,
$K_{m_{5}}=250$, $K_{m_{6}}=250$, $K_{m_{7}}=80$, $K_{m_{8}}=250$,
$K_{m_{9}}=250$, $K_{m_{10}}=120$, $K_{m_{11}}=20$, $K_{m_{12}}=300$

  * •

$k_{1}=1$, $k_{2}0.25$, $k_{5}=2.5$, $k_{6}=0.5$, $k_{9}=0.125$,
$k_{10}=0.125$

  * •

$[RasGTP]=I=10$

  * •

Negative feedback: $F=0.17$, $K_{f}=15$

  * •

Initial values: $[Raf]_{0}=298$, $[pRaf]_{0}=1$, $[ppRaf]_{0}=1$,
$[MEK]_{0}=298$, $[pMEK]_{0}=1$, $[ppMEK]_{0}=1$, $[ERK]_{0}=298$,
$[pERK]_{0}=1$, $[ppERK]_{0}=1$

Below are the tables corresponding to the figures in Section 6.5.

$\varepsilon_{\textbf{x}}$ | Model | $\textrm{len}(\Theta)$ | $\sigma_{\textrm{cutoff}}$ | $\delta$ | count | $\sum_{j}\lVert\Gamma*\xi_{j}\rVert$
---|---|---|---|---|---|---
0 | (1, 0, 0) | 9.0 | 1.0e-10 | 0.039315 | 3.0 | 7.8277e-14
(1, 0, 1) | 18.0 | 1.0e-10 | 0.0008595 | 3.0 | 1.1575e-13
Recovered Law: $\\!\begin{aligned}
&0.0730\sum_{i=1}^{3}x_{i}+0.0983\sum_{i=4}^{6}x_{i}+0.5642\sum_{i=7}^{9}x_{i}=C_{1}\\\
&-0.0357\sum_{i=1}^{3}x_{i}-0.5668\sum_{i=4}^{6}x_{i}+0.1034\sum_{i=7}^{9}x_{i}=C_{2}\\\
&0.5716\sum_{i=1}^{3}x_{i}-0.0480\sum_{i=4}^{6}x_{i}-0.0655\sum_{i=7}^{9}x_{i}=C_{3}\end{aligned}$
Reduced Law: $\\!\begin{aligned} &x_{1}+x_{2}+x_{3}=C_{1}\\\
&x_{4}+x_{5}+x_{6}=C_{2}\\\ &x_{7}+x_{8}+x_{9}=C_{3}\end{aligned}$
8.8961e-10 | (1, 0, 0) | 9.0 | 0.00001241 | 0.039315 | 3.0 | 1.0353e-10
(1, 0, 1) | 18.0 | 0.00001755 | 0.0008595 | 3.0 | 2.4288e-11
Recovered Law: $\\!\begin{aligned}
&0.2457\sum_{i=1}^{3}x_{i}+0.5222\sum_{i=4}^{6}x_{i}-0.0158\sum_{i=7}^{9}x_{i}=C_{1}\\\
&0.5017\sum_{i=1}^{3}x_{i}-0.2310\sum_{i=4}^{6}x_{i}+0.1679\sum_{i=7}^{9}x_{i}=C_{2}\\\
&-0.1455\sum_{i=1}^{3}x_{i}+0.0851\sum_{i=4}^{6}x_{i}+0.5521\sum_{i=7}^{9}x_{i}=C_{3}\end{aligned}$
Reduced Law: $\\!\begin{aligned} &x_{1}+x_{2}+x_{3}=C_{1}\\\
&x_{4}+x_{5}+x_{6}=C_{2}\\\ &x_{7}+x_{8}+x_{9}=C_{3}\end{aligned}$
1.0553e-04 | (1, 0, 0) | 9.0 | 0.029962 | 0.03931 | 3.0 | 0.000011135
(1, 0, 1) | 18.0 | 0.042372 | 0.016559 | 8.0 | 0.046897
Recovered Law: $\\!\begin{aligned}
&0.5632\sum_{i=1}^{3}x_{i}+0.0379\sum_{i=4}^{6}x_{i}+0.1209\sum_{i=7}^{9}x_{i}=C_{1}\\\
&-0.0237\sum_{i=1}^{3}x_{i}+0.5727\sum_{i=4}^{6}x_{i}-0.0689\sum_{i=7}^{9}x_{i}=C_{2}\\\
&0.1244\sum_{i=1}^{3}x_{i}-0.0622\sum_{i=4}^{6}x_{i}-0.5603\sum_{i=7}^{9}x_{i}=C_{3}\end{aligned}$
Reduced Law: $\\!\begin{aligned}
&x_{1}+x_{2}+x_{3}+10^{-5}\mathcal{O}(\textbf{x})=C_{1}\\\
&x_{4}+x_{5}+x_{6}+10^{-5}\mathcal{O}(\textbf{x})=C_{2}\\\
&x_{7}+x_{8}+x_{9}+10^{-5}\mathcal{O}(\textbf{x})=C_{3}\end{aligned}$
Table A.11: Table of values corresponding to $N=20$ for MAPK example (the left graphs of Figure 6.5) $\varepsilon_{\textbf{x}}$ | Model | $\textrm{len}(\Theta)$ | $\sigma_{\textrm{cutoff}}$ | $\delta$ | count | $\sum_{j}\lVert\Gamma*\xi_{j}\rVert$
---|---|---|---|---|---|---
0.0 | (1, 0, 0) | 9.0 | 1.0e-10 | 0.40612 | 3.0 | 5.8377e-13
(2, 0, 0) | 54.0 | 1.0e-10 | 9.8839e-11 | 26.0 | 1.4308e-10
(1, 0, 1) | 18.0 | 1.0e-10 | 0.0054736 | 3.0 | 4.5144e-12
(2, 0, 1) | 63.0 | 1.0e-10 | 9.7605e-11 | 26.0 | 1.3207e-10
(1, 1, 0) | 27.0 | 1.0e-10 | 0.20173 | 3.0 | 6.8272e-13
(2, 1, 0) | 72.0 | 1.0e-10 | 8.9034e-11 | 26.0 | 1.6291e-10
(1, 1, 1) | 36.0 | 1.0e-10 | 0.0011957 | 3.0 | 6.3639e-12
(2, 1, 1) | 81.0 | 1.0e-10 | 8.6342e-11 | 26.0 | 1.5094e-10
Recovered Law: $\\!\begin{aligned}
&-0.0120\sum_{i=1}^{3}x_{i}+0.0010\sum_{i=4}^{6}x_{i}+0.5772\sum_{i=7}^{9}x_{i}=C_{1}\\\
&0.1357\sum_{i=1}^{3}x_{i}-0.5611\sum_{i=4}^{6}x_{i}+0.0038\sum_{i=7}^{9}x_{i}=C_{2}\\\
&0.5610\sum_{i=1}^{3}x_{i}+0.1357\sum_{i=4}^{6}x_{i}+0.0114\sum_{i=7}^{9}x_{i}=C_{3}\end{aligned}$
Reduced Law: $\\!\begin{aligned} &x_{1}+x_{2}+x_{3}=C_{1}\\\
&x_{4}+x_{5}+x_{6}=C_{2}\\\ &x_{7}+x_{8}+x_{9}=C_{3}\end{aligned}$
1.4759e-09 | (1, 0, 0) | 9.0 | 0.000038889 | 0.40612 | 3.0 | 2.2547e-9
(2, 0, 0) | 54.0 | 0.000095258 | 0.11421 | 27.0 | 8.5191e-7
(1, 0, 1) | 18.0 | 0.000054997 | 0.0054736 | 3.0 | 2.0546e-9
(2, 0, 1) | 63.0 | 0.00010289 | 0.000073636 | 30.0 | 0.00005838
(1, 1, 0) | 27.0 | 0.000067358 | 0.20173 | 3.0 | 2.0246e-9
(2, 1, 0) | 72.0 | 0.00010999 | 0.042939 | 27.0 | 7.0647e-7
(1, 1, 1) | 36.0 | 0.000077778 | 0.0011957 | 3.0 | 1.5459e-9
(2, 1, 1) | 81.0 | 0.00011667 | 0.00011868 | 32.0 | 0.0002048
Recovered Law: $\\!\begin{aligned}
&0.4287\sum_{i=1}^{3}x_{i}+0.3669\sum_{i=4}^{6}x_{i}-0.1216\sum_{i=7}^{9}x_{i}=C_{1}\\\
&0.3191\sum_{i=1}^{3}x_{i}-0.4385\sum_{i=4}^{6}x_{i}-0.1975\sum_{i=7}^{9}x_{i}=C_{2}\\\
&0.2180\sum_{i=1}^{3}x_{i}-0.0794\sum_{i=4}^{6}x_{i}+0.5286\sum_{i=7}^{9}x_{i}=C_{3}\end{aligned}$
Reduced Law: $\\!\begin{aligned} &x_{1}+x_{2}+x_{3}=C_{1}\\\
&x_{4}+x_{5}+x_{6}=C_{2}\\\ &x_{7}+x_{8}+x_{9}=C_{3}\end{aligned}$
1.2644e-04 | (1, 0, 0) | 9.0 | 0.075573 | 0.40605 | 3.0 | 0.00019149
(2, 0, 0) | 54.0 | 0.18512 | 1.4879 | 29.0 | 0.37502
(1, 0, 1) | 18.0 | .10688 | 0.32461 | 11.0 | 0.26634
(2, 0, 1) | 63.0 | 0.19995 | 0.084683 | 37.0 | 0.30556
(1, 1, 0) | 27.0 | 0.1309 | 0.20166 | 3.0 | 0.00017206
(2, 1, 0) | 72.0 | 0.21375 | 0.13573 | 31.0 | 0.4558
(1, 1, 1) | 36.0 | 0.15115 | 0.1464 | 11.0 | 0.13342
(2, 1, 1) | 81.0 | 0.22672 | 0.13743 | 40.0 | 0.50621
Recovered Law: $\\!\begin{aligned}
&-0.1896\sum_{i=1}^{3}x_{i}+0.5162\sum_{i=4}^{6}x_{i}+0.1756\sum_{i=7}^{9}x_{i}=C_{1}\\\
&-0.5134\sum_{i=1}^{3}x_{i}-0.2316\sum_{i=4}^{6}x_{i}+0.1265\sum_{i=7}^{9}x_{i}=C_{2}\\\
&0.1836\sum_{i=1}^{3}x_{i}-0.1146\sum_{i=4}^{6}x_{i}+0.5352\sum_{i=7}^{9}x_{i}=C_{3}\end{aligned}$
Reduced Law: $\\!\begin{aligned}
&x_{1}+x_{2}+x_{3}+10^{-5}\mathcal{O}(\textbf{x})=C_{1}\\\
&x_{4}+x_{5}+x_{6}+10^{-5}\mathcal{O}(\textbf{x})=C_{2}\\\
&x_{7}+x_{8}+x_{9}+10^{-5}\mathcal{O}(\textbf{x})=C_{3}\end{aligned}$
Table A.12: Table of values corresponding to $N=100$ for MAPK example (the
right graphs of Figure 6.5)