# A stochastic gradient descent approach with partitioned-truncated singular
value decomposition for large-scale inverse problems of magnetic modulus data

Wenbin Li Corresponding Author: Wenbin LiSchool of Science, Harbin Institute
of Technology, Shenzhen, Shenzhen 518055, China. Email<EMAIL_ADDRESS>Kangzhi Wang School of Science, Harbin Institute of Technology, Shenzhen,
Shenzhen 518055, China. Email<EMAIL_ADDRESS>Tingting Fan School of
Science, Harbin Institute of Technology, Shenzhen, Shenzhen 518055, China.
Email<EMAIL_ADDRESS>
(October 10, 2021)

##  1 Abstract

We propose a stochastic gradient descent approach with partitioned-truncated
singular value decomposition for large-scale inverse problems of magnetic
modulus data. Motivated by a uniqueness theorem in gravity inverse problem and
realizing the similarity between gravity and magnetic inverse problems, we
propose to solve the level-set function modeling the volume susceptibility
distribution from the nonlinear magnetic modulus data. To deal with large-
scale data, we employ a mini-batch stochastic gradient descent approach with
random reshuffling when solving the optimization problem of the inverse
problem. We propose a stepsize rule for the stochastic gradient descent
according to the Courant-Friedrichs-Lewy condition of the evolution equation.
In addition, we develop a partitioned-truncated singular value decomposition
algorithm for the linear part of the inverse problem in the context of
stochastic gradient descent. Numerical examples illustrate the efficacy of the
proposed method, which turns out to have the capability of efficiently
processing large-scale measurement data for the magnetic inverse problem. A
possible generalization to the inverse problem of deep neural network is
discussed at the end.

##  2 Introduction

Solution of magnetic inverse problem aims to reconstruct susceptibility
distributions from magnetic measurements, which is an important task in
detections and surveys employing magnetic approaches. Applications include
exploration of geological resources [13], detection of unexploded ordnance
[15], surveillance for naval warfare [22], and others. A main difficulty of
solving the magnetic inverse problem comes from its ill-posedness, and there
is a strong non-uniqueness of solution for a given set of magnetic measurement
data. To deal with the non-uniqueness, level-set approaches have been proposed
for the inverse problems of magnetic intensity data [9, 12] and magnetic
gradient tensor data [10], where it assumes a given constant susceptibility
supported in an unknown domain and the level-set function is used to represent
topological shape of the unknown domain. In this work, we adopt the level-set
formulation and develop efficient computational method for massive datasets
which were infeasible to process due to the restriction of computational
resource. We will work on the large-scale inverse problem of nonlinear
magnetic modulus data, and we propose a mini-batch stochastic gradient descent
approach with partitioned-truncated singular value decomposition for the
solution of the level-set function modeling the volume susceptibility
distribution.

When meeting with large-scale dataset in realistic magnetic inverse problem,
we realize that the computational resource becomes inadequate as both the
memory access and the computational time are demanding. A direct and trade-off
strategy is to take a subsample of the measurement dataset, but the problem is
how to choose an appropriate subsample preserving the data feature under the
restriction of computational resource. As a result, we look for the employment
of full dataset with limited computational resource, and the stochastic
approach comes to our mind. The stochastic gradient methods have achieved
great success in large-scale machine learning where massive datasets are
involved in the training process; e.g., see [3] for a comprehensive review.
The stochastic approaches use part of the dataset in every computation, which
significantly saves memory usage and computational effort, whereas the full
dataset is employed during each epoch if the without-replacement sampling
strategy is adopted, or the full dataset is touched in expectation if the
with-replacement sampling strategy is used [19]. Moreover, it is reported that
with the help of noisy gradients, the stochastic approaches have the
capability of escaping saddle points [5] and local minima [7] for non-convex
optimization problems. Due to these advantages, we propose to introduce the
stochastic strategy into the solution of the magnetic inverse problem. We use
a mini-batch approach to reduce variances of stochastic gradients, and we
formulate the iteration process by a partial differential equation and propose
a stepsize rule according to its Courant-Friedrichs-Lewy condition. In
addition, A partitioned-truncated singular value decomposition algorithm is
developed to efficiently compute the linear part of the inverse problem in the
context of stochastic gradient approach. The details will be shown in the part
of main algorithm.

We consider the nonlinear modulus data in the magnetic inverse problem for the
following two reasons. Firstly, the modulus data are nonlinear stacking of
multiple components of the magnetic vector field, which are considered to be
less sensitive to systematic measurement noises than the directionally
dependent vector data. Secondly, a well-posedness theory of gravity inverse
problem suggests that the modulus of gravity field can uniquely determine
gravity source [6, 11], and since the magnetic inverse problem shares
similarity to the gravity inverse problem, we believe that the measurement of
magnetic modulus data is sufficient for the recovery of the susceptibility
distribution. More details will be discussed in the formulation of the inverse
problem of magnetic modulus data. In addition, we realize that the general
form of the nonlinear magnetic inverse problem is similar to the architecture
of neural network, and the proposed algorithm can be generalized to solve the
inverse problem of deep neural network, which will be discussed in the section
of conclusion and discussion.

##  3 Inverse problem of magnetic modulus data

Given an inducing magnetic field of strength $B^{0}$ and direction
$\hat{\mathbf{B}}^{0}$, the magnetic vector field resulting from a
distribution of magnetic susceptibility $\kappa$ is modeled by the equation

$\mathbf{B}(\mathbf{r})=\left(B_{1}(\mathbf{r}),\,B_{2}(\mathbf{r}),\,B_{3}(\mathbf{r})\right)^{T}=\frac{1}{4\pi}B^{0}\int_{\Omega}\mathbf{K(r,\tilde{r})}\,\kappa(\mathbf{\tilde{r}})\,\mathrm{d}\mathbf{\tilde{r}},$
(3.1)

with

$\mathbf{K(r,\tilde{r})}=\nabla_{\mathbf{r}}\left(\hat{\mathbf{B}}^{0}\cdot\nabla_{\mathbf{r}}\left(\frac{1}{|\mathbf{r}-\mathbf{\tilde{r}}|}\right)\right),$
(3.2)

where $\mathbf{r}$ and $\mathbf{\tilde{r}}$ denote 3D spatial coordinates:
$\mathbf{r}\in\Gamma$ (the data measurement boundary),
$\mathbf{\tilde{r}}\in\Omega$. The nonlinear magnetic modulus is given by,

$d(\mathbf{r})=\left(\sum_{s=1}^{3}B_{s}(\mathbf{r})^{2}\right)^{\frac{1}{2}}\,;$
(3.3)

since $d(\mathbf{r})$ is nonlinear stacking of multiple components of the
magnetic vector field $\mathbf{B}(\mathbf{r})$, it is presumably less
sensitive to systematic measurement noises than $\mathbf{B}(\mathbf{r})$ which
is directionally dependent. The nonlinear inverse problem reads as follows:
given the modulus data $d(\mathbf{r})$ on the measurement boundary $\Gamma$,
reconstruct the magnetic susceptibility parameter $\kappa(\mathbf{\tilde{r}})$
in the domain $\Omega$.

It is well known that the solution of magnetic inverse problem is non-unique,
and there exist infinitely many susceptibility distributions which can
reproduce a given set of magnetic data. To deal with the non-uniqueness, we
consider the volume susceptibility distribution in the following form,

$\kappa(\mathbf{\tilde{r}})=\kappa_{0}\chi_{D}(\mathbf{\tilde{r}})\,,$ (3.4)

where $\kappa_{0}$ is a constant that approximates the average value of the
susceptibility, and $\chi_{D}$ is the characteristic function of a domain $D$:
$\chi_{D}(\mathbf{\tilde{r}})=1$, $\mathbf{\tilde{r}}\in D$;
$\chi_{D}(\mathbf{\tilde{r}})=0$, $\mathbf{\tilde{r}}\notin D$. We assume that
the constant $\kappa_{0}$ is known as a priori information, and invert for the
domain $D$ to recover the shape of the magnetic source. The setup is motivated
by a well-posedness theory in gravity inverse problem [6, 11]. Let $U$ denote
the gravitational potential generated by a volume mass distribution
$\mu=\rho_{0}\chi_{D}$; if $\rho_{0}$ is a known constant and the domain $D$
satisfies some geometric constraints, the exterior gravity modulus $|\nabla
U|$ uniquely determines the shape of $D$. Realizing that the magnetic inverse
problem shares similarity to the gravity inverse problem, see, e.g. [10], we
consider the volume susceptibility distribution as shown in equation (3.4),
and expect that the domain $D$ can be well recovered from the magnetic modulus
data $d$ as the constant value $\kappa_{0}$ is given.

Similar to our previous works [9, 10, 12], we adopt a level-set formulation
for the volume susceptibility distribution,

$\kappa(\mathbf{\tilde{r}})=\kappa_{0}H(\phi(\mathbf{\tilde{r}}))\,,$ (3.5)

where $\phi(\mathbf{\tilde{r}})$ is the level-set function used to depict the
topological shape of $D$, and $H(\cdot)$ is the Heaviside function: $H(x)=1$,
$x\geq 0$; $H(x)=0$, $x<0$. In level-set computation,
$\phi(\mathbf{\tilde{r}})$ is maintained to be a continuous signed-distance
function to the boundary of $D$,

$\phi(\mathbf{\tilde{r}})=\left\\{\begin{array}[]{ccc}\mathrm{dist}(\mathbf{\tilde{r}},\partial
D)&,&\mathbf{\tilde{r}}\in D\\\ -\mathrm{dist}(\mathbf{\tilde{r}},\partial
D)&,&\mathbf{\tilde{r}}\in\bar{D}^{c}\end{array}\right.\,,$ (3.6)

where $\mathrm{dist}(\mathbf{\tilde{r}},\partial D)$ denotes the distance
between $\mathbf{\tilde{r}}$ and $\partial D$. As a result, equation (3.5) is
a natural expression for the volume susceptibility distribution $\kappa$ as
shown in equation (3.4), and the zero level-set
$\\{\mathbf{\tilde{r}}\mid\phi(\mathbf{\tilde{r}})=0\\}$ indicates the
location of the boundary $\partial D$. The inverse problem is then transformed
into the following form: given the magnetic modulus data $d(\mathbf{r})$ on
the measurement boundary $\Gamma$, recover the level-set function
$\phi(\mathbf{\tilde{r}})$ in $\Omega$ so that the magnetic susceptibility
$\kappa(\mathbf{\tilde{r}})$ is evaluated by equation (3.5).

##  4 Main algorithm

###  4.1 Data fitting and optimization

The inverse problem is solved by fitting the magnetic modulus data with an
optimal solution of the level-set function. Let $d^{*}(\mathbf{r})$ denote the
measured modulus data at $\mathbf{r}\in\Gamma$, and $d(\phi,\mathbf{r})$
denote the predicting data with the value of $\phi(\mathbf{\tilde{r}})$. The
usually used $l^{2}$ fitting function is defined as follows,

$E(\phi)=\frac{1}{|\Gamma|}\int_{\Gamma}\frac{\left(d(\phi,\mathbf{r})-d^{*}(\mathbf{r})\right)^{2}}{2}\,\mathrm{d}\mathbf{r}$
(4.1)

where $|\Gamma|$ denotes the area of the measurement surface $\Gamma$.
Equation (4.1) can be viewed as an expectation of the data misfit on $\Gamma$:

$\mathbb{E}(f(\phi,\mathbf{r}))=\int_{\Gamma}f(\phi,\mathbf{r})\,p(\mathbf{r})\,\mathrm{d}\mathbf{r}$
(4.2)

with

$p(\mathbf{r})=\frac{1}{|\Gamma|}\qquad\mathrm{and}\qquad
f(\phi,\mathbf{r})=\frac{1}{2}\left(d(\phi,\mathbf{r})-d^{*}(\mathbf{r})\right)^{2}\,.$
(4.3)

A regularization term is added to the data fitting function, so that
minimizing the total objective function leads to the admissible solution of
$\phi$. We propose to use the $H^{1}$ semi-norm of $\phi$ as the
regularization term:

$E_{r}(\phi)=\frac{1}{2}\int_{\Omega}|\nabla\phi(\mathbf{\tilde{r}})|^{2}\,\mathrm{d}\mathbf{\tilde{r}}\,.$
(4.4)

As discussed in [11], minimizing $E_{r}(\phi)$ has the effect of shrinking the
measure (area or length) of the interface characterized by the zero level-set
$\\{\mathbf{\tilde{r}}\mid\phi(\mathbf{\tilde{r}})=0\\}$, so that it avoids
forming sharp oscillations on the interface.

The level-set function $\phi(\mathbf{\tilde{r}}))$ is then recovered by
solving the optimization problem:

$\arg\min_{\phi}\,E_{t}(\phi):=\mathbb{E}(f(\phi,\mathbf{r}))+\alpha
E_{r}(\phi)\,,$ (4.5)

where $\alpha$ is a weighting parameter that controls the amount of
regularization applied.

###  4.2 Mini-batch stochastic gradient descent with random reshuffling

The gradient-descent based approach is favorable for solving the optimization
problem of equation (4.5) with large scale. The derivative of $E_{t}(\phi)$ is
given by the formula,

$\frac{\partial
E_{t}(\phi)}{\partial\phi}=\frac{\partial\mathbb{E}(f(\phi,\mathbf{r}))}{\partial\phi}+\alpha\frac{\partial
E_{r}(\phi)}{\partial\phi}=\frac{\partial\mathbb{E}(f(\phi,\mathbf{r}))}{\partial\phi}-\alpha\Delta\phi$
(4.6)

with

$\frac{\partial\mathbb{E}(f(\phi,\mathbf{r}))}{\partial\phi}=\int_{\Gamma}\frac{\partial
f(\phi,\mathbf{r})}{\partial\phi}\,p(\mathbf{r})\,\mathrm{d}\mathbf{r}\,;$
(4.7)

the level-set function $\phi$ is updated according to the negative gradient
direction,

$\frac{\partial\phi}{\partial t}=-\frac{\partial
E_{t}(\phi)}{\partial\phi}=-\frac{\partial\mathbb{E}(f(\phi,\mathbf{r}))}{\partial\phi}+\alpha\Delta\phi$
(4.8)

where $t$ denotes the pseudo time variable that enables the evolution of
$\phi$. As $f(\phi,\mathbf{r})$ is defined in the form of equation (4.3), we
have

$\frac{\partial
f(\phi,\mathbf{r})}{\partial\phi}=\left(d(\phi,\mathbf{r})-d^{*}(\mathbf{r})\right)\frac{\partial
d(\phi,\mathbf{r})}{\partial\phi}\,.$ (4.9)

Considering that $d(\phi,\mathbf{r})$ is related to $\phi$ through equations
(3.1)-(3.3) and equation (3.5), formally we can compute $\frac{\partial
d}{\partial\phi}$ by the chain rule,

$\frac{\partial
d(\phi,\mathbf{r})}{\partial\phi(\mathbf{\tilde{r}})}=\sum_{s=1}^{3}\frac{\partial
d}{\partial B_{s}}\frac{\partial
B_{s}}{\partial\kappa}\frac{\partial\kappa}{\partial\phi}=\frac{B^{0}\kappa_{0}}{4\pi}\delta(\phi(\mathbf{\tilde{r}}))\sum_{s=1}^{3}\frac{B_{s}(\mathbf{r})}{d(\mathbf{r})}K_{s}(\mathbf{r},\mathbf{\tilde{r}})\,.$
(4.10)

Here, $\delta(\phi)$ denotes the Dirac delta function arising from the
derivative of the Heaviside function $H(\phi)$, and
$K_{s}(\mathbf{r},\mathbf{\tilde{r}})$ denotes the $s$-th component of the
magnetic integral kernel $\mathbf{K(r,\tilde{r})}$ so that

$\big{(}K_{1}(\mathbf{r},\mathbf{\tilde{r}}),K_{2}(\mathbf{r},\mathbf{\tilde{r}}),K_{3}(\mathbf{r},\mathbf{\tilde{r}})\big{)}=\mathbf{K(r,\tilde{r})}\,.$
(4.11)

Substituting equation (4.10) into equation (4.9), we have

$\frac{\partial
f(\phi,\mathbf{r})}{\partial\phi}=\frac{B^{0}\kappa_{0}}{4\pi}\,\delta(\phi(\mathbf{\tilde{r}}))\sum_{s=1}^{3}\frac{B_{s}(\mathbf{r})}{d(\mathbf{r})}\left(d(\phi,\mathbf{r})-d^{*}(\mathbf{r})\right)K_{s}(\mathbf{r},\mathbf{\tilde{r}})\,.$
(4.12)

The derivative of the expected data misfit $\mathbb{E}(f(\phi,\mathbf{r}))$ is
then obtained by putting equation (4.12) into equation (4.7).

In physical surveys, the measurement data are sampled at discrete coordinates.
Denoting the set of measurement points by $\\{\mathbf{r}_{i}\mid
i=1,2,\cdots,M\\}$, the expected data misfit $\mathbb{E}(f(\phi,\mathbf{r}))$
reduces to the empirical data misfit:

$f^{M}(\phi)=\frac{1}{M}\sum_{i=1}^{M}f(\phi,\mathbf{r}_{i})\,.$ (4.13)

There will be difficulties in minimizing $f^{M}(\phi)$ directly when the size
of data becomes large. To illustrate it, we derive $\frac{\partial
f^{M}}{\partial\phi}$ from equation (4.13) and equation (4.12),

$\frac{\partial
f^{M}(\phi(\mathbf{\tilde{r}}))}{\partial\phi(\mathbf{\tilde{r}})}=\frac{B^{0}\kappa_{0}}{4\pi
M}\,\delta(\phi(\mathbf{\tilde{r}}))\sum_{s=1}^{3}\sum_{i=1}^{M}\frac{B_{s}(\mathbf{r}_{i})}{d(\mathbf{r}_{i})}\left(d(\phi,\mathbf{r}_{i})-d^{*}(\mathbf{r}_{i})\right)K_{s}(\mathbf{r}_{i},\mathbf{\tilde{r}})\,.$
(4.14)

Here, $\mathbf{\tilde{r}}$ denotes the grid points in the computational domain
$\Omega$. In a regular-size computation, the number of grid points is in the
order of $10^{4}$, e.g. 40 grids each direction in a 3D domain; if the number
of data measurements is also in this order, i.e. $M\sim 10^{4}$, the dimension
of the integral kernel $K_{s}(\mathbf{r}_{i},\mathbf{\tilde{r}})$ will be
grater than $10^{8}$. It becomes demanding in memory accessing and
computational time as the derivative is computed repeatedly in the gradient-
descent based iterative algorithm, and the computation will be infeasible if
the size of data and the number of grid points grow larger.

We propose to utilize a mini-batch stochastic gradient descent approach [3, 8]
to solve the optimization problem of equation (4.5) with large scale. Instead
of using all the measurement data, we employ a part of them to realize the
expected data misfit $\mathbb{E}(f(\phi,\mathbf{r}))$ and its derivative:

$\displaystyle f^{S_{k}}(\phi)$ $\displaystyle=$
$\displaystyle\frac{1}{|S_{k}|}\sum_{i\in S_{k}}f(\phi,\mathbf{r}_{i})\,,$
(4.15) $\displaystyle\frac{\partial f^{S_{k}}(\phi)}{\partial\phi}$
$\displaystyle=$ $\displaystyle\frac{1}{|S_{k}|}\sum_{i\in
S_{k}}\frac{\partial f(\phi,\mathbf{r}_{i})}{\partial\phi}\,,$ (4.16)

where $S_{k}\subset\\{1,2,\cdots,M\\}$ denotes a mini-batch of the index set,
and $|S_{k}|$ denotes the batch size, i.e. the number of elements in $S_{k}$.
Using $\frac{\partial f^{S_{k}}(\phi)}{\partial\phi}$ instead of
$\frac{\partial f^{M}(\phi)}{\partial\phi}$ to evaluate
$\frac{\partial\mathbb{E}(f(\phi,\mathbf{r}))}{\partial\phi}$ in equation
(4.8), the level-set function $\phi$ is then evolved according to the
following formula,

$\frac{\partial\phi}{\partial t}=-\frac{\partial
f^{S_{k}}(\phi)}{\partial\phi}+\alpha\Delta\phi\,.$ (4.17)

The sampling strategies of $S_{k}$ can be generally divided into two
categories: with-replacement sampling and without-replacement sampling [1, 2,
19]. In the with-replacement sampling, $S_{k}$ is uniformly drawn from the
data index set $\\{1,2,\cdots,M\\}$ for each iteration, so that
$\mathbb{E}\left(f^{S_{k}}(\phi)\right)=\mathbb{E}(f(\phi,\mathbf{r}))$ and
$\mathbb{E}\big{(}\frac{\partial
f^{S_{k}}(\phi)}{\partial\phi}\big{)}=\frac{\partial\mathbb{E}(f(\phi,\mathbf{r}))}{\partial\phi}$.
In this case, the evolution equation (4.17) can be seen as a noisy version of
the exact updating as shown in equation (4.8); for instance, if one sets
$|S_{k}|=1$, i.e. uniformly picking single sample from the data set for each
iteration, the algorithm reduces to the classical stochastic gradient descent
approach, which is also referred to as stochastic approximation in some
context [4, 16, 18]. The with-replacement sampling strategy is preferred by
theoretical analysis since it realizes the exact updating rule in expectation.
However, it suffers from practical drawbacks such as requiring truly random
data access and hence longer runtime [19]. In practice, it is common to use
the without-replacement sampling strategy, where the measurement data are
employed sequentially in some random or even deterministic order.

We propose to use the without-replacement sampling with random reshuffling in
our mini-batch stochastic gradient descent algorithm. The details are
summarized in Algorithm 1. In every epoch, the data set is randomly permuted
and then partitioned into mini-batches, and the iterative algorithm employs
the mini-batches successively in order to evaluate the stochastic gradient
descent. The shuffling strategy ensures that every sampling point is reached
during each epoch. The mini-batch strategy has the capability of reducing
variances when estimating the stochastic gradients [3, 8]. In step 7 of
Algorithm 1, the level-set function $\phi$ is reinitialized to maintain the
shape of the signed-distance function as shown in equation (3.6). This is a
standard operation in level-set computation, which aims to make $\phi$
sufficiently smooth without changing the location of the zero level set. For
the details, we refer readers to our previous works [11, 9] and textbooks on
level-set methods [17].

Algorithm 1. Mini-batch stochastic gradient descent algorithm with random
reshuffling for the inverse problem of magnetic modulus data.

1:Initialize the level-set function $\phi$ ; set iteration number
$\overline{k}=0$ .

2:for epoch $e=1,2,\cdots,N_{e}$ do

3: Sample a random permutation $\\{\sigma(1),\sigma(2),\cdots,\sigma(M)\\}$ of
the index set $\\{1,2,\cdots,M\\}$.

4: With a given batch size $b$, partition the shuffled index set into
$n=\frac{M}{b}$ subsets: $S_{k}=\\{\sigma((k-1)b+1),\cdots,\sigma(kb)\\}$,
$k=1,2,\cdots,n$.

5: for $k=1$ to $n$ do

6: Update $\phi$ according to equation (4.17): $\phi:=\phi+\Delta
t\big{(}\frac{\partial\phi}{\partial t}\big{)}$.

7: Reinitialize $\phi$ to maintain the signed distance property.

8: Iteration number $\overline{k}:=\overline{k}+1$.

9: end for

10:end for

###  4.3 Stepsize rule

The choice of stepsize is an important aspect in the stochastic gradient
descent algorithm [16, 20]. An inappropriate choice of stepsize can have a
disastrous effect on the performance of convergence. In this work, we propose
a stepsize rule according to the Courant-Friedrichs-Lewy (CFL) condition of
the evolution equation (4.17).

Similar to equation (4.14), we have

$\frac{\partial
f^{S_{k}}(\phi)}{\partial\phi}=\frac{B^{0}\kappa_{0}}{4\pi|S_{k}|}\,\delta(\phi(\mathbf{\tilde{r}}))\sum_{s=1}^{3}\sum_{i\in{S_{k}}}\frac{B_{s}(\mathbf{r}_{i})}{d(\mathbf{r}_{i})}\left(d(\phi,\mathbf{r}_{i})-d^{*}(\mathbf{r}_{i})\right)K_{s}(\mathbf{r}_{i},\mathbf{\tilde{r}})\,.$
(4.18)

We use a numerical version of the Dirac delta function [21],

$\delta(\phi)\doteq\delta_{\epsilon}(\phi)=\chi_{T_{\epsilon}}|\nabla\phi|\,,$
(4.19)

where $\chi_{T_{\epsilon}}$ denotes the characteristic function of
$T_{\epsilon}$, and
$T_{\epsilon}=\\{\tilde{\mathbf{r}}:|\phi(\tilde{\mathbf{r}})|<\epsilon\\}$ is
a neighborhood of the zero level set. Substituting (4.19) into (4.18), the
stochastic gradient can be evaluated in the following way,

$\frac{\partial f^{S_{k}}(\phi)}{\partial\phi}=V_{n}|\nabla\phi|$ (4.20)

with

$V_{n}:=\frac{B^{0}\kappa_{0}}{4\pi|S_{k}|}\,\chi_{T_{\epsilon}}\sum_{s=1}^{3}\sum_{i\in{S_{k}}}\frac{B_{s}(\mathbf{r}_{i})}{d(\mathbf{r}_{i})}\left(d(\phi,\mathbf{r}_{i})-d^{*}(\mathbf{r}_{i})\right)K_{s}(\mathbf{r}_{i},\mathbf{\tilde{r}})\,.$
(4.21)

The evolution equation (4.17) then reduces to

$\frac{\partial\phi}{\partial t}=-V_{n}|\nabla\phi|+\alpha\Delta\phi\,,$
(4.22)

which can be viewed as a Hamilton-Jacobi equation with an artificial viscosity
term. Discretizing $\frac{\partial\phi}{\partial t}$ using a direct forward
Euler scheme: $\frac{\partial\phi}{\partial
t}\approx\frac{\phi^{n+1}-\phi^{n}}{\Delta t}$, the CFL condition for
stability is

$\Delta t\left(\frac{\max|V_{n}|}{\min\\{\Delta x,\Delta y,\Delta
z\\}}+\frac{2\alpha}{\Delta x^{2}}+\frac{2\alpha}{\Delta
y^{2}}+\frac{2\alpha}{\Delta z^{2}}\right)<1\,.$ (4.23)

The stepsize of the mini-batch stochastic gradient descent algorithm is taken
as

$\Delta t=C\left(\frac{\max|V_{n}|}{\min\\{\Delta x,\Delta y,\Delta
z\\}}+\frac{2\alpha}{\Delta x^{2}}+\frac{2\alpha}{\Delta
y^{2}}+\frac{2\alpha}{\Delta z^{2}}\right)^{-1}\,,$ (4.24)

where $C$ denotes a constant with $C\in(0,1)$. In practice, as the
regularization parameter $\alpha$ can be very small,
$\frac{\alpha}{\min\\{\Delta x,\Delta y,\Delta z\\}}\ll\max|V_{n}|$, one can
drop the terms including $\alpha$ in equation (4.24), and the stepsize is
taken as

$\Delta t=C\left(\frac{\max|V_{n}|}{\min\\{\Delta x,\Delta y,\Delta
z\\}}\right)^{-1}\,.$ (4.25)

###  4.4 Partitioned-truncated SVD for matrix multiplications in the
stochastic gradient descent

Large-scale matrix multiplications arise from the evaluation of gradient
direction (as shown in equation (4.21)) and the discretization of the magnetic
integral (3.1). Considering formula (3.2) and formula (4.11), the magnetic
integral kernel has the following form,

$\big{(}K_{1}(\mathbf{r},\mathbf{\tilde{r}}),K_{2}(\mathbf{r},\mathbf{\tilde{r}}),K_{3}(\mathbf{r},\mathbf{\tilde{r}})\big{)}=\frac{1}{|\mathbf{r}-\mathbf{\tilde{r}}|^{3}}\left[\frac{3\left(\hat{\mathbf{B}}^{0}\cdot(\mathbf{r}-\mathbf{\tilde{r}})\right)(\mathbf{r}-\mathbf{\tilde{r}})}{|\mathbf{r}-\mathbf{\tilde{r}}|^{2}}-\hat{\mathbf{B}}^{0}\right]\,,$
(4.26)

which decays rapidly as the distance $|\mathbf{r}-\tilde{\mathbf{r}}|$ is
increasing. This decaying property is common to the integral kernel of
potential-field data, i.e. magnetic and gravity data. We have proposed a
partitioned-truncated singular value decomposition (SVD) algorithm to develop
low rank approximation to the decaying integral kernel in several different
setups and for various types of potential-field data [14, 9, 11]. Here, we
propose a variation of the partitioned-truncated SVD so that it is compatible
with the mini-batch stochastic gradient descent approach.

####  4.4.1 Matrix multiplications involving the decaying kernel

To evaluate the stochastic gradient as shown in equation (4.21), we consider
the following formulation,

$G^{S_{k}}_{s}(\mathbf{\tilde{r}}):=\chi_{T_{\epsilon}}\sum_{i\in{S_{k}}}\frac{B_{s}(\mathbf{r}_{i})}{d(\mathbf{r}_{i})}\left(d(\phi,\mathbf{r}_{i})-d^{*}(\mathbf{r}_{i})\right)K_{s}(\mathbf{r}_{i},\mathbf{\tilde{r}})\,,\qquad
S_{k}\subset\\{1,2,\cdots,M\\},\ s\in\\{1,2,3\\}\,.$ (4.27)

To evaluate $G^{S_{k}}_{s}$ we need to compute the magnetic components
$B_{s}(\mathbf{r}_{i})$: $i\in{S_{k}},\,s\in\\{1,2,3\\}$, and we extract the
following integral,

$I^{S_{k}}_{s}(\mathbf{r}_{i}):=\int_{\Omega}K_{s}(\mathbf{r}_{i},\mathbf{\tilde{r}})\,\kappa(\mathbf{\tilde{r}})\,\mathrm{d}\mathbf{\tilde{r}}\,,\qquad
i\in S_{k}\subset\\{1,2,\cdots,M\\},\ s\in\\{1,2,3\\}\,.$ (4.28)

Let the computational domain $\Omega\subset\mathbf{R}^{3}$ be a rectangular
domain, $\Omega=[0,X]\times[0,Y]\times[-Z,0]$, and let the measurement
boundary $\Gamma$ satisfies $\Gamma\subset\\{\mathbf{r}=(x,y,z)\mid z>0\\}$.
This setup corresponds to a common situation where the magnetic measurement is
above the survey domain. Suppose that the domain $\Omega$ is uniformly
discretized into $N=n_{x}n_{y}n_{z}$ grid points:
$\mathbf{\tilde{r}}_{1},\mathbf{\tilde{r}}_{2},\cdots,\mathbf{\tilde{r}}_{N}$,
and recall that there are $M$ measurement points on $\Gamma$ and the mini-
batch size $|S_{k}|=b$. After discretization, the evaluation of
$I^{S_{k}}_{s}(\mathbf{r}_{i})$ and $G^{S_{k}}_{s}(\mathbf{\tilde{r}})$
reduces to the evaluation of matrix multiplications,

$\displaystyle\mathbf{I}^{S_{k}}_{s}$ $\displaystyle=$
$\displaystyle\mathbf{K}^{S_{k}}_{s}\hskip 0.5pt\mathbf{\kappa}^{T}\,,$ (4.29)
$\displaystyle\mathbf{G}^{S_{k}}_{s}$ $\displaystyle=$
$\displaystyle\mathbf{d}^{S_{k}}_{s}\mathbf{K}^{S_{k}}_{s}\mathbf{D}\,,$
(4.30)

where

$\displaystyle\mathbf{K}^{S_{k}}_{s}$ $\displaystyle=$
$\displaystyle\left[K_{s}(\mathbf{r}_{i},\mathbf{\tilde{r}}_{j})\right]_{b\times
N}\,,\quad i\in S_{k},\ j=1,\cdots,N\,,$ $\displaystyle\mathbf{d}^{S_{k}}_{s}$
$\displaystyle=$
$\displaystyle\left[\frac{B_{s}(\mathbf{r}_{i})}{d(\mathbf{r}_{i})}\left(d(\phi,\mathbf{r}_{i})-d^{*}(\mathbf{r}_{i})\right)\right]_{1\times
b}\,,\quad i\in S_{k}\,,$ $\displaystyle\mathbf{\kappa}$ $\displaystyle=$
$\displaystyle\left[\kappa(\mathbf{\tilde{r}}_{1}),\cdots,\kappa(\mathbf{\tilde{r}}_{N})\right]\,,$
$\displaystyle\mathbf{D}$ $\displaystyle=$
$\displaystyle\mathrm{diag}\left\\{\chi_{T_{\epsilon}}(\mathbf{\tilde{r}}_{1}),\cdots,\chi_{T_{\epsilon}}(\mathbf{\tilde{r}}_{N})\right\\}\,.$

####  4.4.2 Preconditioning on the full kernel matrix

The multiplication matrix $\mathbf{K}^{S_{k}}_{s}$ appearing in (4.29) and
(4.30) is a sub-matrix of the full kernel matrix
$\mathbf{K}_{s}=\left[K_{s}(\mathbf{r}_{i},\mathbf{\tilde{r}}_{j})\right]_{M\times
N}\,,\ i=1,\cdots,M,\ j=1,\cdots,N$. Considering that the full kernel matrix
$\mathbf{K}_{s}$ is always constant once the measurement points
$\mathbf{r}_{i}$ and the grid points $\mathbf{\tilde{r}}_{j}$ are fixed, we
perform preconditioning including partitioning and truncating on
$\mathbf{K}_{s}$.

The component $K_{s}(\mathbf{r}_{i},\mathbf{\tilde{r}}_{j})$ decays rapidly as
$|\mathbf{r}_{i}-\mathbf{\tilde{r}}_{j}|$ increases. Since
$\mathbf{r}_{i}\in\Gamma\subset\\{\mathbf{r}=(x,y,z)\mid z>0\\}$, we order the
grid points
$\\{\mathbf{\tilde{r}}_{1},\mathbf{\tilde{r}}_{2},\cdots,\mathbf{\tilde{r}}_{N}\\}$
according to their z-coordinates so that
$\tilde{z}_{1}\leq\tilde{z}_{2}\leq\cdots\leq\tilde{z}_{N}$, where we denote
$\mathbf{\tilde{r}}_{j}=(\tilde{x}_{j},\tilde{y}_{j},\tilde{z}_{j})$. The set
of grid points in the computational domain is then partitioned into $n_{z}$
subsets,

$\\{\mathbf{\tilde{r}}_{1},\mathbf{\tilde{r}}_{2},\cdots,\mathbf{\tilde{r}}_{N}\\}=\bigcup_{h=1}^{n_{z}}\\{\tilde{\mathbf{r}}^{h}_{j}\mid
1\leq j\leq\tilde{M}:=n_{x}n_{y}\\}$ (4.31)

where $\tilde{\mathbf{r}}^{h}_{j}=\tilde{\mathbf{r}}_{j+(h-1)\tilde{M}}$ and
so $\tilde{z}_{j}^{h}=(h-1)\Delta z$. The full kernel matrix
$\mathbf{K}_{s}=\left[K_{s}(\mathbf{r}_{i},\mathbf{\tilde{r}}_{j})\right]_{M\times
N}$ is accordingly partitioned into $n_{z}$ sub-matrices along columns,

$\mathbf{K}_{s}=\left[\mathbf{K}_{s}^{1},\mathbf{K}_{s}^{2},\cdots,\mathbf{K}_{s}^{n_{z}}\right],\quad\mathrm{where}\quad\mathbf{K}_{s}^{h}=\left[K_{s}(\mathbf{r}_{i},\mathbf{\tilde{r}}_{j}^{h})\right]_{M\times\tilde{M}}\,,\
1\leq h\leq n_{z}\,.$ (4.32)

In (4.32), every sub-matrix $\mathbf{K}_{s}^{h}$ corresponds to the grid
points in the same depth. We perform the singular value decomposition for
$\mathbf{K}_{s}^{h}$,

$\mathbf{K}_{s}^{h}=\mathbf{U}_{s}^{h}\mathbf{S}_{s}^{h}(\mathbf{V}_{s}^{h})^{T},\quad
1\leq h\leq n_{z},\ s\in\\{1,2,3\\}\,,$ (4.33)

where $\mathbf{U}_{s}^{h}\in\mathbf{R}^{M\times
M},\,\mathbf{V}_{s}^{h}\in\mathbf{R}^{\tilde{M}\times\tilde{M}}$ are unitary
matrices, and

$\mathbf{S}_{s}^{h}=\mathrm{diag}\left\\{\sigma_{1}^{(s,h)},\sigma_{2}^{(s,h)},\cdots,\sigma_{\min\\{M,\tilde{M}\\}}^{(s,h)}\right\\}\in\mathbf{R}^{M\times\tilde{M}}$
(4.34)

with the singular values
$\sigma_{1}^{(s,h)}\geq\sigma_{2}^{(s,h)}\geq\cdots\geq\sigma_{\min\\{M,\tilde{M}\\}}^{(s,h)}\geq
0$. Then we truncate $\mathbf{S}_{s}^{h}$ by choosing a thresholding parameter
$\epsilon_{svd}$ and setting

$\tilde{\sigma}_{i}^{(s,h)}=\left\\{\begin{array}[]{lcr}\sigma_{i}^{(s,h)}&,&\sigma_{i}^{(s,h)}\geq\epsilon_{svd}\\\
0&,&\sigma_{i}^{(s,h)}<\epsilon_{svd}\end{array}\right.,\qquad 1\leq
i\leq\min\\{M,\tilde{M}\\}\,.$ (4.35)

Denoting $\tau_{sh}$ the number of non-zero $\tilde{\sigma}_{i}^{(s,h)}$, we
have the truncated singular value decomposition for $\mathbf{K}_{s}^{h}$ in
the following way,

$\mathbf{K}_{s}^{h}\approx\tilde{\mathbf{U}}_{s}^{h}\tilde{\mathbf{S}}_{s}^{h}(\tilde{\mathbf{V}}_{s}^{h})^{T},\quad
1\leq h\leq n_{z},\ s\in\\{1,2,3\\}\,,$ (4.36)

where
$\tilde{\mathbf{S}}_{s}^{h}=\mathrm{diag}\left\\{\tilde{\sigma}_{1}^{(s,h)},\tilde{\sigma}_{i}^{(s,h)},\cdots,\tilde{\sigma}_{\tau_{sh}}^{(s,h)}\right\\}\in\mathbf{R}^{\tau_{sh}\times\tau_{sh}}$,
$\tilde{\mathbf{U}}_{s}^{h}\in\mathbf{R}^{M\times\tau_{sh}}$ is composed of
the first $\tau_{sh}$ columns of $\mathbf{U}_{s}^{h}$, and
$\tilde{\mathbf{V}}_{s}^{h}\in\mathbf{R}^{\tilde{M}\times\tau_{sh}}$ is
composed of the first $\tau_{sh}$ columns of $\mathbf{V}_{s}^{h}$. The
truncating process reduces the dimension of the matrices.

Finally we compute

$\tilde{\mathbf{Q}}_{s}^{h}=\tilde{\mathbf{S}}_{s}^{h}(\tilde{\mathbf{V}}_{s}^{h})^{T}\,,$
(4.37)

so that

$\mathbf{K}_{s}^{h}\approx\tilde{\mathbf{U}}_{s}^{h}\tilde{\mathbf{Q}}_{s}^{h},\quad
1\leq h\leq n_{z},\ s\in\\{1,2,3\\}\,.$ (4.38)

The preconditioning is completed and the matrices
$\tilde{\mathbf{U}}_{s}^{h},\,\tilde{\mathbf{Q}}_{s}^{h}$ are stored for later
use.

####  4.4.3 Evaluation of the stochastic gradient

Equation (4.32) and equation (4.38) imply the partitioned-truncated SVD for
$\mathbf{K}^{S_{k}}_{s}$:

$\displaystyle\mathbf{K}_{s}^{S_{k}}$ $\displaystyle=$
$\displaystyle\left[\mathbf{K}_{s}^{1,S_{k}},\mathbf{K}_{s}^{2,S_{k}},\cdots,\mathbf{K}_{s}^{n_{z},S_{k}}\right]\,,$
(4.39) $\displaystyle\mathbf{K}_{s}^{h,S_{k}}$ $\displaystyle\approx$
$\displaystyle\tilde{\mathbf{U}}_{s}^{h,S_{k}}\tilde{\mathbf{Q}}_{s}^{h},\qquad
1\leq h\leq n_{z},\ s\in\\{1,2,3\\}\,,$ (4.40)

where $\tilde{\mathbf{U}}_{s}^{h,S_{k}}$ is constructed by taking the rows of
$\tilde{\mathbf{U}}_{s}^{h}$ with indices $i\in S_{k}$.

To evaluate the matrix multiplications (4.29) and (4.30), we perform
partitions on $\mathbf{\kappa}$ and $\mathbf{D}$ in the same way as the
partition of $\mathbf{K}_{s}^{S_{k}}$,

$\displaystyle\mathbf{\kappa}$ $\displaystyle=$
$\displaystyle[\mathbf{\kappa}_{1},\mathbf{\kappa}_{2},\cdots,\mathbf{\kappa}_{n_{z}}]\,,$
(4.41) $\displaystyle\mathbf{D}$ $\displaystyle=$
$\displaystyle\mathrm{diag}\left\\{\mathbf{D}_{1},\mathbf{D}_{2},\cdots,\mathbf{D}_{n_{z}}\right\\}\,.$
(4.42)

Finally, we propose the formulas of matrix multiplications for the stochastic
gradient:

$\displaystyle\mathbf{I}_{s}^{S_{k}}$ $\displaystyle=$
$\displaystyle\sum_{h=1}^{n_{z}}\mathbf{K}_{s}^{h,S_{k}}\left(\mathbf{\kappa}_{h}\right)^{T}\approx\sum_{h=1}^{n_{z}}\tilde{\mathbf{U}}_{s}^{h,S_{k}}\left(\tilde{\mathbf{Q}}_{s}^{h}\left(\mathbf{\kappa}_{h}\right)^{T}\right)\,,$
(4.43) $\displaystyle\mathbf{G}_{s}^{S_{k}}$ $\displaystyle=$
$\displaystyle\left[\mathbf{G}_{s}^{1,S_{k}},\mathbf{G}_{s}^{2,S_{k}},\cdots,\mathbf{G}_{s}^{n_{z},S_{k}}\right]\,,\quad\mathrm{with}\quad\mathbf{G}_{s}^{h,S_{k}}\approx\left(\mathbf{d}_{s}^{S_{k}}\tilde{\mathbf{U}}_{s}^{h,S_{k}}\right)\left(\tilde{\mathbf{Q}}_{s}^{h}\mathbf{D}_{h}\right)\,.$
(4.44)

Realizing that $\mathbf{D}_{h}$ is a diagonal matrix with either $0$ or $1$
entries, $\tilde{\mathbf{Q}}_{s}^{h}\mathbf{D}_{h}$ in (4.44) can be simply
obtained by taking the columns of $\tilde{\mathbf{Q}}_{s}^{h}$ with the same
indices of the non-zero diagonal entries in $\mathbf{D}_{h}$ and setting the
other columns to be $\mathbf{0}$.

##  5 Results

We include some test examples to illustrate the efficacy of the proposed
algorithm. The measurement magnetic modulus data are generated using equations
(3.1)-(3.3). We discretize the magnetic integral (3.1) directly and we are not
using the partitioned-truncated SVD when generating the measurement data.
$5\%$ Gaussian noises are added to each component $B_{s}$,

$B_{s}^{*}=B_{s}\left(1+\eta\cdot\mathcal{N}(0,1)\right)\,,$

where $\eta=5\%$ and $\mathcal{N}(0,1)$ denotes the Gaussian noises with $0$
mean and standard deviation $1$. Then the synthetic nonlinear modulus data are
generated according to equation (3.3),

$d^{*}(\mathbf{r})=\left(\sum_{s=1}^{3}B_{s}^{*}(\mathbf{r})^{2}\right)^{\frac{1}{2}}\,.$

In the test examples, the inducing magnetic field has the strength of
$B^{0}=5.95\times 10^{4}$ nT, and the direction $\hat{\mathbf{B}}^{0}$ is
prescribed by inclination $I^{0}$ and declination $D^{0}$ so that

$\hat{\mathbf{B}}^{0}=(\cos I^{0}\sin D^{0},\cos I^{0}\cos D^{0},-\sin
I^{0})\,,$

where we designate the direction of $x$-axis as east, the direction of
$y$-axis as north, and the direction of $z$-axis as negative depth. The
average susceptibility value in the level-set formulation (3.5) is given as
$\kappa_{0}=0.05$. In the inversion algorithm, we use the partitioned-
truncated SVD when evaluating the predicted data and the stochastic gradient,
where the thresholding parameter for truncating is taken as
$\epsilon_{svd}=10^{-5}$.

###  5.1 Examples in a shallow region

Magnetic susceptibilities are reconstructed in the computational domain
$\Omega=[0\ 1]\times[0\ 1]\times[-0.5\ 0]\,km$, and we discretize $\Omega$
uniformly into $41\times 41\times 21$ mesh grids. The measurement boundary is
taken as $\Gamma=[0\ 1]\times[0\ 1]\times\\{z=0.1\\}\,km$, along which 10,000
measurement points are randomly distributed; Figure 1 (a) shows the picture of
measurement points along $\Gamma$. In the inversion algorithm, we set the
mini-batch size to be $b=200$ for the stochastic gradient descent. The initial
guess of the level-set function $\phi$ is set as follows,

$\phi_{initial}=1-\sqrt{\frac{(x-0.5)^{2}}{0.35^{2}}+\frac{(y-0.5)^{2}}{0.35^{2}}+\frac{(z-0.25)^{2}}{0.15^{2}}}\,;$

the zero level-set of the initial structure is an ellipsoid as shown in Figure
1 (b).

Figure 1: Setup of examples in the shallow region. (a) Measurement points
along $\Gamma=[0\ 1]\times[0\ 1]\times\\{z=0.1\\}\,km$; (b) initial guess.

####  5.1.1 Example 1

As shown in Figure 2 (a), the true model of the volume susceptibility consists
of 3 magnetic sources, where the cube is centered at $(0.75,0.2,-0.3)$ with
side length $0.15$, and the sphere is centered at $(0.8,0.75,-0.2)$ with
radius $0.1$. We assume an inducing field with inclination and declination
$(I^{0},D^{0})=(75^{\circ},25^{\circ})$. Figure 2 (b) plots the magnetic
modulus data on the measurement surface. Figure 3 provides the inversion
results. Figure 3 (a) shows the recovered solution using the proposed
stochastic gradient descent algorithm with partitioned-truncated SVD; Figure 3
(b) plots the data discrepancy $d-d^{*}$; Figures 3 (c) and 3 (d) plot the
cross-sections along $x=0.25$ and $x=0.75$, respectively, where the dashed
line indicates the true model and the solid line indicates the recovered
solution. The solution adequately reproduces the magnetic modulus data,
although we employ the partitioned-truncated SVD to approximate matrix
multiplications and we never use all the 10,000 measurement data at once in
the inversion algorithm. The recovered susceptibility structure matches well
with the true model, where both the shape and location of the magnetic sources
are successfully recovered. In Figure 4, we provide more information to
illustrate the performance of the inversion algorithm. Figure 4 (a) presents
the evolution of the mini-batch misfit function $f^{S_{k}}$ as defined in
equation (4.15); Figure 4 (b) plots $l^{2}$-difference between the recovered
susceptibility $\kappa$ and the true susceptibility $\kappa^{*}$. The misfit
function $f^{S_{k}}$ is oscillating partly due to the nonlinearity of level-
set inversion and partly due to the noisy nature of stochastic gradient
descent. On the other hand, the performance of the $l^{2}$-difference
$\|\kappa-\kappa^{*}\|_{2}$ implies a smooth convergence.

Figure 2: Example 1. (a) True model; (b) magnetic modulus data with Gaussian
noises.

Figure 3: Example 1. Inversion results. (a) recovered solution; (b) data
discrepancy $d-d^{*}$; (c)-(d) cross-sections along $x=0.25$ and $x=0.75$,
respectively, where the dashed line indicates the true model and the solid
line indicates the recovered solution.

Figure 4: Example 1. Performance of the inversion algorithm. (a) Mini-batch
misfit function $f^{S_{k}}$; (b) $l^{2}$-difference between the recovered
susceptibility $\kappa$ and the true susceptibility $\kappa^{*}$:
$\|\kappa-\kappa^{*}\|_{2}$. The misfit function $f^{S_{k}}$ is oscillating
partly due to the nonlinearity of level-set inversion and partly due to the
noisy nature of stochastic gradient descent. On the other hand, the
performance of the $l^{2}$-difference $\|\kappa-\kappa^{*}\|_{2}$ implies a
smooth convergence.

####  5.1.2 Example 2

Figure 5 (a) shows the true model of the volume susceptibility, where two
dipping prisms are distributed in the computational domain. The inducing field
has the inclination and declination $(I^{0},D^{0})=(75^{\circ},25^{\circ})$.
Figure 5 (b) plots the magnetic modulus data on the measurement surface.
Figure 6 provides the inversion results, where Figure 6 (a) shows the
recovered solution, Figure 6 (b) plots the data discrepancy $d-d^{*}$, and
Figures 6 (c), 6 (d) plot the cross-sections along $x=0.25$ and $x=0.75$,
respectively. The magnetic modulus data are adequately reproduced. The
recovered solution correctly captures the shape and location of the magnetic
prisms, where the depth resolution looks perfect in the displayed cross-
sections. In Figure 7, we present the mini-batch misfit function $f^{S_{k}}$
and the $l^{2}$-difference $\|\kappa-\kappa^{*}\|_{2}$. The performance of
$\|\kappa-\kappa^{*}\|_{2}$ shows a smooth convergence although the mini-batch
misfit function $f^{S_{k}}$ is oscillating.

Figure 5: Example 2. (a) True model; (b) magnetic modulus data with Gaussian
noises.

Figure 6: Example 2. Inversion results. (a) recovered solution; (b) data
discrepancy $d-d^{*}$; (c)-(d) cross-sections along $x=0.25$ and $x=0.75$,
respectively, where the dashed line indicates the true model and the solid
line indicates the recovered solution.

Figure 7: Example 2. Performance of the inversion algorithm. (a) Mini-batch
misfit function $f^{S_{k}}$; (b) $\|\kappa-\kappa^{*}\|_{2}$.

####  5.1.3 Example 3

As shown in Figure 8 (a), we consider a susceptibility model including 4
distinct magnetic sources. The cube is centered at $(0.75,0.2,-0.2)$ with side
length $0.15$, the sphere is centered at $(0.8,0.75,-0.35)$ with radius $0.1$,
and the two dipping prisms with different depths are enclosed in the region
$\\{(x,y,z)\mid x\in(0.1,0.3)\\}$. We assume an inducing field with
inclination and declination $(I^{0},D^{0})=(75^{\circ},25^{\circ})$. Figure 8
(b) shows the magnetic modulus data on the measurement surface. The inversion
results are shown in Figure 9, where Figure 9 (a) shows the recovered
solution, Figure 9 (b) plots the data discrepancy, and Figures 9 (c), 9 (d)
plot the cross-sections along $x=0.25$ and $x=0.75$, respectively. In
addition, we present the mini-batch misfit function $f^{S_{k}}$ and the
$l^{2}$-difference $\|\kappa-\kappa^{*}\|_{2}$ in Figure 10. We conclude that
the solution successfully recovers the susceptibility model and adequately
reproduces the noisy magnetic modulus data. The performance of
$\|\kappa-\kappa^{*}\|_{2}$ shows a smooth convergence although the mini-batch
misfit function $f^{S_{k}}$ is oscillating.

Figure 8: Example 3. (a) True model; (b) magnetic modulus data with Gaussian
noises.

Figure 9: Example 3. Inversion results. (a) recovered solution; (b) data
discrepancy $d-d^{*}$; (c)-(d) cross-sections along $x=0.25$ and $x=0.75$,
respectively, where the dashed line indicates the true model and the solid
line indicates the recovered solution.

Figure 10: Example 3. Performance of the inversion algorithm. (a) Mini-batch
misfit function $f^{S_{k}}$; (b) $\|\kappa-\kappa^{*}\|_{2}$.

Figure 11: Setup of examples in a cubic domain. (a) Measurement points along
$\Gamma=[-1\ 2]\times[-1\ 2]\times\\{z=0.1\\}\,km$; (b) initial guess.

###  5.2 Examples in a cubic domain

In the following examples, we implement the inversion algorithm in a cubic
domain with larger depth. Depth resolution is generally a difficult task in
magnetic inverse problem. Since the measurement surface is above the
computational domain and the magnetic integral kernel decays rapidly as the
distance between source and measurement increases, the responses of deep
structures are likely to be suppressed by those of shallow structures and
noise contaminations. As a result, depth resolution is an important criterion
to evaluate the performance of a magnetic inversion algorithm. In the
following examples, we include susceptibility distributions with larger
variances in depth, which aims to test the capability of depth resolution for
the proposed inversion algorithm.

The computational domain is $\Omega=[0\ 1]\times[0\ 1]\times[-1\ 0]\,km$,
which is uniformly discretized into $41\times 41\times 41$ mesh grids. The
measurement boundary is taken as $\Gamma=[-1\ 2]\times[-1\
2]\times\\{z=0.1\\}\,km$, along which 10,000 measurement points are randomly
distributed; Figure 11 (a) plots the measurement points along $\Gamma$. Again,
we set the mini-batch size to be $b=200$ for the stochastic gradient descent.
The initial guess of the level-set function $\phi$ is taken as:

$\phi_{initial}=0.4-\sqrt{(x-0.5)^{2}+(y-0.5)^{2}+(z-0.5)^{2}}\,,$

and so the zero level-set of the initial structure is a sphere as shown in
Figure 11 (b).

####  5.2.1 Example 4

Figure 12 (a) shows the susceptibility model, where two magnetic sources are
located at different depths of the computational domain. The dipping prism is
located in the shallow region, and the sphere is centered at
$(0.75,0.75,-0.7)$ with radius $0.1$. We assume an inducing field with
inclination and declination $(I^{0},D^{0})=(90^{\circ},0^{\circ})$. Figure 12
(b) shows the magnetic modulus data on the measurement surface. This is a
typical example where the responses of deep structures are inapparent in the
data profile. Figure 13 provides the inversion results using the mini-batch
stochastic gradient descent algorithm with partitioned-truncated SVD. Figure
13 (a) shows the recovered solution; Figure 13 (b) plots the data discrepancy
$d-d^{*}$; Figure 13 (c) plots the cross-section of the solution along
$x=0.25$; Figure 13 (d) plots the cross-section along $x=0.75$. The
performances of the mini-batch misfit function $f^{S_{k}}$ and the
$l^{2}$-difference $\|\kappa-\kappa^{*}\|_{2}$ are presented in Figure 14 (a)
and Figure 14 (b), respectively.

The solution successfully recovers the susceptibility model and adequately
reproduces the magnetic modulus data. The depth resolution is amazing, given
that the susceptibility model has large variance in depth and the spherical
source in deep region has invisible response in the data profile. This test
example illustrates that the inversion algorithm does not loose depth
resolution in the solution, although the mini-batch stochastic gradient
descent only employs part of data at each iteration and the partitioned-
truncated SVD provides inaccurate approximation for matrix multiplications. In
fact, since the stochastic gradient descent has the capability of escaping
saddle points [5] and local minima [7], we believe that it can contribute to
improving depth resolution in the magnetic inverse problem.

Figure 12: Example 4. (a) True model; (b) magnetic modulus data with Gaussian
noises.

Figure 13: Example 4. Inversion results. (a) recovered solution; (b) data
discrepancy $d-d^{*}$; (c)-(d) cross-sections along $x=0.25$ and $x=0.75$,
respectively, where the dashed line indicates the true model and the solid
line indicates the recovered solution.

Figure 14: Example 4. Performance of the inversion algorithm. (a) Mini-batch
misfit function $f^{S_{k}}$; (b) $\|\kappa-\kappa^{*}\|_{2}$.

####  5.2.2 Example 5

In this example, we consider a susceptibility model including 3 magnetic
sources with different depths. The true model is shown in Figure 15 (a). The
measurement magnetic modulus data with Gaussian noises are shown in Figure 15
(b), where we assume an inducing field with inclination and declination
$(I^{0},D^{0})=(75^{\circ},25^{\circ})$. The inversion results are shown in
Figure 16, where Figure 16 (a) shows the recovered solution, Figure 16 (b)
plots the data discrepancy, and Figures 16 (c), 16 (d) plot two cross-sections
of the solution. The performances of the mini-batch misfit function
$f^{S_{k}}$ and the $l^{2}$-difference $\|\kappa-\kappa^{*}\|_{2}$ are
presented in Figure 17. We conclude that the inversion algorithm successfully
recovers the susceptibility model and adequately reproduces the measurement
data. The depth resolution is reasonable if not perfect, given that the
susceptibility model has distinct variance in depth, and the task of
recovering depth is generally difficult. With the techniques of stochastic
gradient descent and partitioned-truncated SVD, the proposed inversion
algorithm is able to deal with large-scale measurement data efficiently in the
magnetic inverse problem.

Figure 15: Example 5. (a) True model; (b) magnetic modulus data with Gaussian
noises.

Figure 16: Example 5. Inversion results. (a) recovered solution; (b) data
discrepancy $d-d^{*}$; (c)-(d) cross-sections along $x=0.20$ and $x=0.65$,
respectively, where the dashed line indicates the true model and the solid
line indicates the recovered solution.

Figure 17: Example 5. Performance of the inversion algorithm. (a) Mini-batch
misfit function $f^{S_{k}}$; (b) $\|\kappa-\kappa^{*}\|_{2}$.

##  6 Conclusion and discussion

We have proposed a mini-batch stochastic gradient descent approach with
partitioned-truncated SVD for large-scale inverse problems of magnetic modulus
data. We employ a level-set formulation for susceptibility model and recover
the volume susceptibility distribution from nonlinear magnetic modulus data by
solving the level-set function. To deal with massive amount of measurement
data, we propose a stochastic gradient descent approach to solve the level-set
optimization problem with large scale. When evaluating the stochastic
gradients, we consider a mini-batch strategy to reduce variances, and we use
the without-replacement sampling with random reshuffling to explore full data
set at every epoch. Realizing that the iteration process can be formulated as
a Hamilton-Jacobi equation, we propose a step-size rule for the stochastic
gradient descent according to the Courant-Friedrichs-Lewy condition of the
evolution PDE. To further improve the computational efficiency, we take
advantage of the decaying property of the magnetic integral kernel, and
propose a partitioned-truncated SVD for matrix multiplications in the context
of stochastic gradient descent.

Numerical examples are included to illustrate the efficacy of the proposed
method. The solutions successfully recover the susceptibility models and
adequately reproduce the magnetic modulus data. The inversion algorithm is
able to reconstruct susceptibility distributions with large variances in
depth. As an important criterion to evaluate the performance of the magnetic
inversion algorithm, the depth resolution is very good in all the test
examples. We believe that the stochastic gradient approach can help to improve
depth resolution due to its capability of escaping saddle points and local
minima. With the techniques of stochastic gradient descent and partitioned-
truncated SVD, the proposed method enables us to efficiently process large-
scale magnetic data which was infeasible due to the restriction of hardware
resource.

The inverse problem of magnetic modulus data can be formulated as the general
form

$d=\sigma_{2}\circ L\circ\sigma_{1}(\phi)\,:\quad\mathrm{reconstruct}\ \phi\
\mathrm{from}\ d\,,$ (6.1)

where $\sigma_{1}$ denotes the nonlinear level-set formulation as shown in
(3.5), $L$ denotes the linear integral as shown in (3.1), and $\sigma_{2}$
denotes the nonlinear stacking as shown in (3.3); after discretization, $L$ is
the linear operator of matrix multiplications. The forward operator in formula
(6.1) is similar to the architecture of deep neural network:

$\Psi=\sigma_{N}\circ L_{N}\circ\cdots\sigma_{i}\circ
L_{i}\circ\cdots\circ\sigma_{1}\circ L_{1}\,,$ (6.2)

where $\sigma_{i}$’s denote nonlinear activating functions, and $L_{i}$’s
denote linear operators including affine mappings, convolutions, etc. In this
work, we have proposed an efficient stochastic gradient descent approach with
partitioned-truncated SVD for the inverse problem of (6.1). The algorithm can
be generalized to solve the inverse problem of the deep neural network as
shown in (6.2), provided that the linear operators $L_{i}$ have some sort of
low-rank property. The inverse problem should be stated as follows: given a
well-trained deep neural network $\Psi=\sigma_{N}\circ
L_{N}\circ\cdots\circ\sigma_{1}\circ L_{1}$ and its output response
$d=\Psi(\phi)$, reconstruct the input signal $\phi$. We will do further
explorations on this subject.

## Acknowledgments

Wenbin Li is supported by NSFC (grant no. 41804096), Natural Science
Foundation of Guangdong Province (grant no. 2018A030313341), and Natural
Science Foundation of Shenzhen (grant no. JCYJ20190806144005645).

## References

  * [1] Léon Bottou.  Curiously fast convergence of some stochastic gradient descent algorithms.  In Proceedings of the symposium on learning and data science, Paris, volume 8, pages 2624–2633, 2009.
  * [2] Léon Bottou.  Stochastic gradient descent tricks.  In Neural networks: Tricks of the trade, pages 421–436. Springer, 2012.
  * [3] Léon Bottou, Frank E Curtis, and Jorge Nocedal.  Optimization methods for large-scale machine learning.  SIAM Review, 60(2):223–311, 2018.
  * [4] Michael C Fu.  Optimization for simulation: Theory vs. practice.  INFORMS Journal on Computing, 14(3):192–215, 2002.
  * [5] Rong Ge, Furong Huang, Chi Jin, and Yang Yuan.  Escaping from saddle points—online stochastic gradient for tensor decomposition.  In Conference on learning theory, pages 797–842. PMLR, 2015.
  * [6] Victor Isakov.  Inverse source problems.  American Mathematical Society, Providence, Rhode Island, 1990.
  * [7] Robert Kleinberg, Yuanzhi Li, and Yang Yuan.  An alternative view: When does SGD escape local minima?  In International Conference on Machine Learning, pages 2698–2707. PMLR, 2018.
  * [8] Mu Li, Tong Zhang, Yuqiang Chen, and Alexander J Smola.  Efficient mini-batch training for stochastic optimization.  In Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 661–670, 2014.
  * [9] Wenbin Li, Wangtao Lu, Jianliang Qian, and Yaoguo Li.  A multiple level-set method for 3D inversion of magnetic data.  Geophysics, 82(5):J61–J81, 2017.
  * [10] Wenbin Li and Jianliang Qian.  A level-set algorithm for the inverse problem of full magnetic gradient tensor data.  Applied Mathematics Letters, 107:106416, 2020.
  * [11] Wenbin Li and Jianliang Qian.  Simultaneously recovering both domain and varying density in inverse gravimetry by efficient level-set methods.  Inverse Problems and Imaging, 15(3):387–413, 2021.
  * [12] Wenbin Li, Jianliang Qian, and Yaoguo Li.  Joint inversion of surface and borehole magnetic data: A level-set approach.  Geophysics, 85(1):J15–J32, 2020.
  * [13] Y. Li and D. Oldenburg.  3-D inversion of magnetic data.  Geophysics, 61(2):394–408, 1996.
  * [14] W. Lu, S. Leung, and J. Qian.  An improved fast local level set method for three-dimensional inverse gravimetry.  Inverse Problems and Imaging, 9:479–509, 2015.
  * [15] Marc Munschy, Daniel Boulanger, Patrice Ulrich, and Mustapha Bouiflane.  Magnetic mapping for the detection and characterization of UXO: Use of multi-sensor fluxgate 3-axis magnetometers and methods of interpretation.  Journal of Applied Geophysics, 61(3-4):168–183, 2007.
  * [16] Arkadi Nemirovski, Anatoli Juditsky, Guanghui Lan, and Alexander Shapiro.  Robust stochastic approximation approach to stochastic programming.  SIAM Journal on Optimization, 19(4):1574–1609, 2009.
  * [17] Stanley Osher and Ronald Fedkiw.  Level set methods and dynamic implicit surfaces, volume 153.  Springer Science & Business Media, 2006.
  * [18] Herbert Robbins and Sutton Monro.  A stochastic approximation method.  The Annals of Mathematical Statistics, pages 400–407, 1951.
  * [19] Itay Safran and Ohad Shamir.  How good is SGD with random shuffling?  In Conference on Learning Theory, pages 3250–3284. PMLR, 2020.
  * [20] Conghui Tan, Shiqian Ma, Yu-Hong Dai, and Yuqiu Qian.  Barzilai-borwein step size for stochastic gradient descent.  In Proceedings of the 30th International Conference on Neural Information Processing Systems, pages 685–693, 2016.
  * [21] H. K. Zhao, T. Chan, B. Merriman, and S. Osher.  A variational level set approach to multiphase motion.  Journal of computational physics, 127(1):179–195, 1996.
  * [22] Michael S Zhdanov, Martin Čuma, Glenn A Wilson, and Louis Polomé.  3D magnetization vector inversion for SQUID-based full tensor magnetic gradiometry.  In SEG Technical Program Expanded Abstracts 2012, pages 1–5. Society of Exploration Geophysicists, 2012.


# Leveraging the Inductive Bias of Large Language Models for Abstract Textual
Reasoning

Christopher Michael Rytting
Department of Computer Science
Brigham Young University
Provo, UT 84602
<EMAIL_ADDRESS>
David Wingate
Department of Computer Science
Brigham Young University
Provo, UT 84602
<EMAIL_ADDRESS>
Use footnote for providing further information about author (webpage,
alternative address)— _not_ for acknowledging funding agencies.

###### Abstract

Large natural language models (such as GPT-3 or T5) demonstrate impressive
abilities across a range of general NLP tasks. Here, we show that the
knowledge embedded in such models provides a useful inductive bias, not just
on traditional NLP tasks, but also in the nontraditional task of training a
symbolic reasoning engine. We observe that these engines learn quickly and
generalize in a natural way that reflects human intuition. For example,
training such a system to model block-stacking might naturally generalize to
stacking other types of objects because of structure in the real world that
has been partially captured by the language describing it. We study several
abstract textual reasoning tasks, such as object manipulation and navigation,
and demonstrate multiple types of generalization to novel scenarios and the
symbols that comprise them. We also demonstrate the surprising utility of
compositional learning, where a learner dedicated to mastering a complicated
task gains an advantage by training on relevant simpler tasks instead of
jumping straight to the complicated task.

##  1 Introduction

Natural language processing (NLP) has seen major progress thanks to
probabilistic language models (LMs) like GPT-2 [1], BERT [2], and T5 [3].
These are pre-trained in a general, task-agnostic way on large corpora of
unstructured text and then fine-tuned on specific tasks. This method has
achieved state of the art performance on popular NLP tasks like question-
answering, textual entailment, text summarization, neural machine translation,
and more [4].

These models are not just impressive for the high scores they achieve on
quantitative language tasks; the text they generate often reflects patterns of
“real-world” structure, suggesting that embedded in the weights of the LM is
implicit knowledge of physics, object persistence, containment, spatial
relationships, causal mechanisms, material properties, and other common-sense
knowledge central to human reasoning and intuition. If that is so, then they
ought to provide a useful inductive bias in learning to perform symbolic
reasoning tasks that mirror real-world tasks.

In this paper, we attempt to leverage and characterize this inductive bias by
training reasoning engines that demonstrate some of the hallmarks of human
reasoning ability: learning (a) rules (b) that generalize well (c) from few
examples. Concretely, we fine-tune T5 on a suite of symbolic reasoning tasks
and study generalization along multiple different axes beyond the normal
test/train split: we examine cardinality generalization, object
generalization, part-of-speech generalization, and show (in the spirit of
curriculum learning) that LMs can leverage combinations of learned subskills
to master complicated composite tasks with both better sample efficiency and
higher terminal performance than learning directly on the complicated tasks.

We see our contributions as four-fold. First, we demonstrate a high level of
performance by connectionist models on tasks resembling symbolic classical AI
tasks, demonstrating some symbolic ability of such models in light of recent
calls to unify the principles of both symbolism and connectionism. Secondly,
we demonstrate the breakdown of reasoning ability by manufacturing our own
reasoning datasets that can be tweaked in systematic ways, instead of simply
split into training/validation/test sets. This means that we can assess our
models’ ability to both interpolate and extrapolate in systematic, symbolic,
and grammatical ways, and otherwise flex with changing distributions. Thirdly,
we demonstrate the ability of large LMs to reason compositionally, that is, to
learn two kinds of reasoning separately and then to combine those different
kinds of reasoning on a novel composite task, to which they are both relevant.
Lastly, we demonstrate the inductive bias we hypothesize is present in large
LMs and can be leveraged to assist in the formation of reasoning engines.

##  2 Related Work

Transformer-based LMs [5] are the dominant LM architecture, including the
original GPT models [6, 1], BERT [2, 3] and the recent 175 billion parameter
GPT-3 [4]. They are trained via generic maximum likelihood learning on vast,
unstructured text corpora, but often exhibit zero/few-shot learning abilities
on a variety of NLP tasks, having grasped key mechanics of natural language by
learning to simply predict a missing word; it is this inductive bias we hope
to harness.

These models implicitly house a rich world model with concepts and relations.
Petroni et al. query a language model (instead of a traditional, symbolic
knowledge base) for relational data explicitly expressed in natural langauge
[7]. Bosselut et al. expand this scope, attempting to generate explicit
commonsense knowledge graphs using pre-trained LMs [8], and Bouraoui et al.
perform relation extraction with BERT, which can be construed as finding and
specifying edges [9]. This work treats the formation of explicit knowledge
graphs using LMs, whereas ours treats the leveraging of implicit knowledge
graphs using LMs.

Our tasks are inspired by classic examples of good old-fashioned AI, including
Blocks World [10] and STRIPS planners [11], where a system’s state, composed
of symbols such as toy blocks, must be manipulated to achieve a desired end
given a rule set. In contrast to these original problems, we _learn the
symbolic rules_ governing a similar system from examples. Our work is
therefore similar in spirit to program induction, where a connectionist model
must learn a structured program from examples; examples include the Neural
Turing Machine [12] and the Differentiable Neural Computer [13]. Weber et al.
combine neural networks with logic programming to solve multi-hop reasoning
tasks in natural language [14]. We don’t insist on expressing explicit
representations of symbolic rules learned, instead reasoning about rules
learned by examining out-of-distribution performance.

Our tasks are similar to the bAbI dataset [15]. However, we focus on
systematic study of specific dimensions of generalization that is impossible
with bAbI. bAbI questions evaluate _whether_ certain skills are possessed; our
questions evaluate _to what extent_ these skills are possessed, how they
generalize, and when they break down. Finally, the idea of compositional
learning draws inspiration from multi-task learning, which was explored in
[16] and more recently in a deep learning approach [17]. Supplementary
Training on Intermediate Labeled-data Tasks (STILTs) was introduced in [18],
which uses two stages of pre-training, where the first stage is unsupervised
like with other pre-trained models, but where the second is on some data-rich
intermediate supervised task.

##  3 Data Generation, Evaluation, and Training Protocol

The central questions we ask in this paper revolve around whether large LMs
possess some world model or inductive bias that helps them to learn reasoning
rules from few examples. These rules should be used to successfully
extrapolate not just to novel instances, but to novel tasks. Upon learning to
track objects across containers, for example, a language model should be able
to generalize to new types of objects, marginally more complicated scenarios
than it has seen before, and leverage skills already learned to progress more
quickly than if it hadn’t.

We show how LMs can generalize in these ways and others by systematically
fine-tuning them on classes of scenarios where they can learn rules. Across
tasks, our reasoners deduce the final state of an environment given natural
language descriptions of an initial state and actions taken on it.

###  3.1 Task Types

Containers.  In the first task, which we call containers, we manipulate
objects in various containers and ask the reasoner to track the state of the
environment. The initial state of the environment is a random allotment of
n_objects objects into n_containers containers. The names of these objects and
containers are sampled uniformly and without replacement from lists of
candidates. Examples of such candidates are given in the appendix. Once
sampled and organized, the objects and containers are converted into a plain
English expression describing their organization. Then, there is a random
manipulation of this initial state, where an object is randomly taken from a
container and placed into another container. The task of the reasoning engine
is to describe the final state of the environment.

Table 1: Examples of scenarios for each task. Non-bold words indicate natural
language templates that are filled in with randomly sampled words.
Additionally, template phrases can repeat to describe a variable number of
containers, rooms, objects, or navigation steps.
Task |  Prefix |  Target
---|---|---
Container |  The bin contains a ball and a snake. The box contains a quilt. Took a quilt from the box and put it in the bin. |  The bin contains a ball, a quilt, and a snake. The box contains no objects.
Navigation Route |  The garden is to the west of the kitchen. The bedroom is to the south of the kitchen. To get from the kitchen to the garden, you must go |  to the west.
Navigation Result |  The garden is to the west of the kitchen. The bedroom is to the south of the kitchen. If you start in the kitchen and go to the west, you will end in the |  garden.
Hard Object / Composite Task |  The kitchen is to the north of the garage. The garden is to the west of the kitchen. The bedroom is to the south of the garage. There is a bin containing a book in the bedroom. There is a box in the garden containing no objects. Took a book from the bin in the bedroom. Went to the north twice, then went to the west. Placed it. |  The box in the garden contains a book. The bin in the bedroom contains no objects

For training, each scenario uses 2-8 objects and 2-3 containers, sampled
uniformly. We construct sentences by sampling without replacement from a
uniform distribution of container names and object names and filling a
template accordingly. Our training set of object and container names comes
from a proprietary linguistic dataset, which has data on commonness, part of
speech, etc. of each word. We divide this dataset into nouns and verbs and
subtract their intersection from both. We split the unique nouns into a train
set (n=36566) and a validation set (n=12189) and a container set (n=9), the
former two of which are to be used as object names and the latter of which is
to be used for container names during training. For each set (train and val),
we find the 2000 most common and join the dataset on concreteness ratings from
Brysbaert et al. [19] to take the 2000 most concrete nouns from each set. We
also generate a list of random strings by sampling uniformly from single digit
integers and lowercase English characters, ranging in length randomly from 5
to 10.

Navigation.  In both versions of navigation, a natural-language description of
a map of the environment is provided to the reasoning engine. It is generated
by sampling from a list of common locations including “kitchen", “garden", and
others. These locations are then composed into a grid where they are in north-
south-west-east orientation to each other. In the first task, called
_navigation route_ , the reasoner is additionally given a starting point and a
destination. The reasoner must provide a valid route from origin to
destination. In the second task, called _navigation result_ , we require that
the reasoner, given a starting point and a route, determine the location where
they would find themselves in the map. For training, the maximum number of
locations is 8 and the minimum is 3.

Composite task.  We are interested in assessing the ability of a reasoning
engine to perform a task which combines multiple elemental types of knowledge.
Specifically, can a learner master two skills separately, and then leverage
knowledge from both to perform the composite task better and more quickly than
it would have by learning directly on the composite task? This task, called
hard object, is intended to be a composition of navigation and containers. The
reasoner is given a verbal map of the world and an action taken. In this
action, an object is taken from its container, carried on a route indicated by
successive moves in cardinal directions, and placed. The reasoner must then
describe the new state of the containers.

###  3.2 Generalization Types

Given our overarching interest in large LMs’ ability to yield reasoners that
generalize well, we craft several types of experiments to probe the bounds of
different kinds of generalization, namely:

Cardinality generalization. This tests a model’s structural understanding of
the domain. If a reasoner is trained on scenarios with $k$ objects, steps of
navigations, rooms, etc. can it generalize to more than $k$ objects, steps of
navigations, rooms, etc.?

Object generalization. This is a semantic test of whether or not the reasoner
can leverage prior knowledge of English to generalize to new, never-before-
seen objects. For example, if several different container training scenarios
are composed of objects from different distributions of words (e.g. 2000
concrete nouns vs. 2000 most common nouns vs. 2000 randomly sampled nouns),
which distribution of training scenarios results in the best model
generalization to scenarios composed with new, previously unseen nouns?

Part-of-speech generalization. This is another semantic test based on the idea
that a reasoner that understands language will perform better on “right" words
than on “wrong" words. For example, on the container task, the model should
perform well when nouns are objects and poorly when verbs or random strings
are. Intuitively, a scenario such as “The bin contains a dethrone and a
transpose" makes less sense, and is statistically less likely, than a scenario
such as “The bin contains a ball and a snake"; the naturalness of the scenario
descriptions should positively correlate with generalization. We hypothesize
that natural nouns will work best, since nouns are often sensible things to
move from one container to the other, and that arbitrary strings and verbs
will both decrease performance.On the other hand, random strings have probably
never been encountered by the model, and it might learn to simply copy
whatever tokens are in specific places, treating strings as opaque IDs,
symbols without meaning. Verbs should actively confuse the model, since a
verb’s linguistic role is structurally and semantically different than a
noun’s; it is a symbol not without meaning, but with the wrong meaning.

Reasonable phrasing generalization. Finally, what if we replace the templates
themselves? Instead of replacing the objects in a scenario, we replace the
English scaffolding we insert them into by mapping deterministically from each
English word to a gibberish word composed of English morphemes but possessing
no meaning (See 4.5). A reasoner for which language is meaningful will have a
harder time adjusting to this task, while a reasoner for which language is
simply a sequence of meaningless strings will struggle no more with this task
than with its original English version.

###  3.3 Base Model and Training Details

On all experiments, we use T5’s 3 billion parameter architecture, either fine-
tuning a pretrained model or training from scratch. We do so on Nvidia Tesla
V100 32GB GPUs with a batch size of 1 and a learning rate of 0.003. We train a
different reasoning engine for all three elemental tasks for 1000 total steps.
After evaluating them on their respective tasks, we train them on the other
base task for 1000 steps and eventually on the composite task for 1000 steps
(3000 total steps).

###  3.4 Metrics

For each experiment, we gauge performance using three metrics. The first is
exact equality of true final state. This is obviously the highest standard, as
it mimics perfectly the reasoning we would expect a human to carry out after
having learned the rules and format of the dynamical system. The second metric
is substring equality. We want to see how many individual statements from the
predicted final state are contained in the true final state. Sometimes the
reasoning is flawed, but on target (e.g. if the reasoner predicts, after
moving a hammer from a box to a bin, that it is in both the box and the bin,
it should be given partial credit). The third metric is the standard BLEU
score, to see how similar the sentences are on subword levels.

Interpolation and Extrapolation. Throughout our experiments, we assess
different kinds of generalization. The first, which we term _interpolation_ ,
refers to testing a LM on new instances that are drawn from the same
distribution as the training set. For example, in the container task, we might
test on scenarios that use already-seen objects and containers, but arranged
in novel scenarios. The second, which we term _extrapolation_ , refers to
testing a LM on new instances that are drawn from a different distribution
than the training distribution. For example, in the container task, we might
test on scenarios that involve new, never-before-seen objects, containers, or
cardinalities.

Structural and Semantic Generalization. Finally, we distinguish between
_structural generalization_ , where we test things like cardinality
generalization, and _semantic generalization_ , where we explore new words, or
new types of words.

Figure 1: Model performance on 4 experiments over the course of fine-tuning.
Compare solid lines (off-the-shelf pre-trained models) and dotted lines
(tabula rasa models) of the same color.

##  4 Results

We now systematically test interpolation and extrapolation, assessing both
semantic and structural generalization. We begin with several baselines, and
then explore increasingly difficult tasks.

###  4.1 Comparison with Baseline Methods

Since our claims have to do with the inductive bias of large, pre-trained LMs,
we first establish a wide baseline gulf between pre-trained models and those
trained from scratch (tabula rasa). We train two such versions of T5-3B on the
same training set (1000 scenarios with maximums of 8 objects and 3 containers
and minimums of 2 objects and 2 containers) and then measure two kinds of both
interpolative and extrapolative performance: systematic, meaning the number of
objects and containers, and semantic, meaning the words used for objects in
reasoning scenarios (See 3.4 and 3.2 for more info). Thus, the two models’
performance is compared on 4 total experiments with 7200 examples each: Interp
\- same number of containers and objects and same words used for object names
(as training set); SemExtrap \- same number of containers and objects (as
training set), different words used for object names; SysExtrap \- minimums of
4 containers and 10 objects and maximums of 5 containers and 19 objects, same
words used for object names (as training set); SemSysExtrap \- minimums of 4
containers and 10 objects and maximums of 5 containers and 19 objects,
different words used for object names (than training set); The average BLEU
scores for each model checkpoint (taken every 100 steps during fine-
tuning/training) are displayed in Figure 1. These baselines make clear that
pre-trained language models that have been fine-tuned wildly outperform
language models trained from scratch.

###  4.2 Containers

We now begin our primary experiments, starting by testing cardinality
generalization. We train on scenarios with a maximum of 8 objects and 3
containers, and validate structural generalization on scenarios with up to 19
objects and 5 containers. The results are shown in Figure 2. The model is able
to correctly predict the exact string a majority of the time even well outside
the training domain. This kind of generalization could only be reached by
grasping, to some extent, rules like those used when enumerating long English
lists. This is one way in which the inductive bias of LMs aids in the
reasoner’s good performance.

We now test object generalization and part-of-speech generalization. Here, we
train models with scenarios generated by parameterizing our templates with
various sets of nouns, and then testing by parameterizing with a different set
of words. The results are shown in Table 2. Each row represents a different
training set; columns represent performance on different test sets. We tested
four different training sets: "all" represents nouns sampled uniformly from
our set of all 36,566 nouns. "2k common" represents the 2000 most common
nouns, "2k concrete" represents the 2000 most concrete nouns; "2k random"
represents 2000 randomly sampled nouns. There are 4 groups of columns,
representing different validation conditions. "Training words" represents a
validation condition where the reasoner was tested on new scenarios that used
the same set of nouns (eg, a different mix of containers and objects, or a
different number of them). "Validation words" represents a condition with new
scenarios using never-before-seen words of a specific type. "New POS"
represents words that are an entirely different part of speech, or random
words. The general conclusion is that, regardless of the metric used, it is
best to train on the largest non-specific set of nouns possible. Doing so
gives good generalization across a wide variety of alternative words,
including verbs and random strings.

Figure 2: The blue box encompasses the training domain. Each cell is the
average of performance over 100 generated instances for each combination of
_n_objects_ and _n_containers_. There is an obvious drop off higher in the
number of objects, but this is a truncation issue due to long sentences (note
that substring scores don’t degrade nearly as much).

Table 2: Detailed results on the container task, showing various aspects of interpolation and extrapolation, both to new scenarios, new types of words, and new surrounding texts. | training words | validation words | new pos
---|---|---|---
| all | concrete | common | 2000 | all | concrete | common | verbs | toverbs | random
bleu
all | 0.95 | 0.98 | 0.99 | 0.95 | 0.95 | 0.98 | 0.99 | 0.99 | 0.95 | 0.82
2k common | 0.93 | 0.96 | 0.95 | 0.93 | 0.93 | 0.96 | 0.96 | 0.99 | 0.95 | 0.81
2k concrete | 0.93 | 0.96 | 0.97 | 0.93 | 0.93 | 0.96 | 0.96 | 0.99 | 0.95 | 0.78
2k random | 0.91 | 0.94 | 0.95 | 0.91 | 0.91 | 0.94 | 0.95 | 0.99 | 0.95 | 0.81
exact
all | 0.79 | 0.88 | 0.91 | 0.79 | 0.78 | 0.87 | 0.90 | 0.86 | 0.66 | 0.44
2k common | 0.60 | 0.71 | 0.65 | 0.59 | 0.60 | 0.69 | 0.67 | 0.83 | 0.63 | 0.37
2k concrete | 0.60 | 0.68 | 0.75 | 0.59 | 0.59 | 0.69 | 0.71 | 0.86 | 0.60 | 0.31
2k random | 0.56 | 0.61 | 0.68 | 0.55 | 0.55 | 0.60 | 0.64 | 0.87 | 0.63 | 0.40
substring
all | 0.95 | 0.97 | 0.96 | 0.95 | 0.95 | 0.97 | 0.96 | 0.98 | 0.97 | 0.96
2k common | 0.82 | 0.90 | 0.85 | 0.82 | 0.83 | 0.88 | 0.87 | 0.97 | 0.95 | 0.93
2k concrete | 0.83 | 0.87 | 0.91 | 0.82 | 0.83 | 0.88 | 0.89 | 0.98 | 0.92 | 0.84
2k random | 0.80 | 0.83 | 0.86 | 0.80 | 0.80 | 0.83 | 0.85 | 0.98 | 0.94 | 0.91

Figure 3: Models trained on concrete nouns were most confused by verbs and
random strings.

We now explore the distributions of the training nouns. Many of the words
passed between containers were nouns that would not be passed in between
containers, like "year", and so we conducted an experiment where we trained on
the 2000 most concrete nouns, a random subsample of 200 of those nouns, and 20
of the most sensible nouns to be moved between containers, like "marble" or
"mouse". As can be seen in Figure 2, the model trained on sensible nouns gets
most confused at verbs and random strings, relative to its peers. This
suggests that the distribution of very concrete nouns is farther from the
distribution of verbs than the larger distribution of concrete nouns.

###  4.3 Navigation

We now turn our attention to the navigation and navigation route tasks. In
this section, we use exact string accuracy as our metric, as both substring
and BLEU scores ignore the (critical) sequential nature of the predictions.

####  4.3.1 Interpolation and Extrapolation

Our primary results are in Figure 4. For both the navigation route and
navigation result tasks, we trained on distributions of maps that contained
between 3 to 8 total rooms; this resulted in total path plans that were 1 to 5
steps long.

In the middle, we see results for the navigation result task (where the model
must predict the result of a specific sequence of actions). Here, we see
several interesting phenomena. Overall, the model does a good job of making
accurate predictions, with a 79% marginal accuracy over the training regime.
There are two situations where the model performs especially well: the first
column represents tasks where there is only a single step in the route; the
model shows a strong ability to extrapolate to any number of rooms. The second
case is the "diagonal" in the upper-right. This represents situations where
the map is a linear chain, with very few junctions in the map. In this case,
the model also shows a strong ability to generalize to new situations
involving a new number of rooms, and a new number of steps. It is unclear why
the model has such trouble with two-step planning problems, although we
hypothesize it may have something to do with the training distribution, as
discussed later.

Figure 4: Validation performance on the navigation route and navigation result
tasks. Entries represent exact string equality. Training regime is outlined in
blue box. See text for details.

In contrast, on the left of Figure 4, we see results for the navigation route
task. Here, we see similarly strong performance in the training regime, with a
82% marginal accuracy. However, extrapolating to new situations shows mixed
results: the model is easily able to extrapolate to any number of rooms, and
like the navigation result task, it performs especially well in the case of
single-step planning. However, it seems to be completely unable to generalize
to new numbers of steps needed in the planning process.

What accounts for the difference? The biggest difference between the
navigation route and navigation result tasks is the format of the output: in
the navigation result task, the model always outputs a single word (the room
name), regardless of the complexity of the map or the plan. In contrast, on
the navigation route task, longer plans require a longer output. The model
seems to struggle with having to generate sentences that are longer than any
it has seen before. While we have used exact string equality in all of the
tests reported here, we note that on longer routes, sometimes the model’s
output would be a substring of the correct output, as in the example below,
which is correct, but missing the final step:

Target: to the west, then to the west, then to the north, then to the north,
then to the north, then to the north
Prediction: to the west, then to the west, then to the north, then to the
north, then to the north

###  4.4 Compositional Reasoning on the Composite Task

Finally, we tested the _hard object_ task, where the reasoner must consider
objects, containers, and maps. To explore this, we test four conditions. We
train a learner exclusively on the task (HardObj). We also train a learner
first on the navigation task for 1000 steps, then on the container task for
1000 steps, then on the hard object task for 1000 steps (Nav-Cont-HardObj). We
also reverse the order of navigation and container pre-training (Cont-Nav-
HardObj), and we test a random mixture of pretraining, with 2000 steps of a
50/50 mixture of sentences drawn from both tasks (ContNav5050-HardObj). For
each regime, we trained 10 models and averaged their performance on a test set
of 5000 held-out examples.

The results in Figure 5 are surprising: the models that learned first on other
kinds of tasks learned the composite task more quickly and achieved better
ultimate performance. Like curriculum learning, this seems to indicate
subskills can persist and be ported to superskills for improved sample
efficiency. Instead of fine-tuning on a single hard task (and thereby gaining
proficiency in only that task), a better strategy may be to fine-tune on a
wide variety of elemental tasks, and focus more on combining them, thereby
potentially solving not just one, but a combinatorially large number of
complex tasks.

Figure 5: Confidence bands represent one standard deviation above and below
mean performance under each metric. The composite learners exhibit better
sample efficiency and a higher absolute performance ceiling than the learner
dedicated exclusively to the composite task. Figure 6: Extrapolation ability
as a function of training set distribution. See text for discussion.

####  4.4.1 Importance of Training Set Distribution

The T5 model seems unusually sensitive to the training set distribution.
Figure 6 explores this. In our first attempt at creating a training
distribution for the navigation tasks, we randomly created a map with a
uniformly selected number of rooms, and then randomly selected source and
destination rooms. This resulted in the performance shown in the right-hand
panels (C and D) of Figure 6. Panel (D) shows most of the training data is
concentrated in the upper-left corner, meaning short paths in small maps. As a
result (panel (C)), the model was almost completely unable to model length 4
or length 5 plans, because they rarely appeared in the training set.

An alternative is to sample a desired path length uniformly, and then generate
a map, source, and destination with that length. This results in the training
set distribution shown in panel (B), which concentrates many more examples on
longer paths in larger maps, but with fewer examples of two-step plans. The
resulting accuracy is shown in Panel (A). Here, we see that accuracy is
greatly improved for four- and five-step plans, although two-step accuracy
suffers. However, even the carefully constructed training distribution with
uniformly sampled path lengths does not induce generalization to paths longer
than 5 steps, as shown in Figure 4. Understanding this phenomenon is an
important direction for future research.

###  4.5 Does all that reading really help?

At the heart of our paper is the idea that natural language provides a useful
inductive bias. To explore this directly, we modify our object-container task
in the following way: We devise a one-to-one mapping from each word in our
sensible English templates to words in an invented gibberish. Since the
mapping is one-to-one, the English templates and the gibberish templates are
identically structured but with substituted words. For example, the word "The"
is mapped to the gibberish word "Xrq", the word "contains" to "sixnqkxb", etc.
Noun slots in both templates are still filled with the original English nouns
(i.e. not substituted with gibberish). Figure 7 shows the results: it is
harder for T5 to master the gibberish domain. If there were no inductive bias
of English-trained models aiding our learners on English tasks, then the
mastery of these two grammars would grow at the same pace, since they are just
as structured as one another and thus presumably as predictable and learnable
as one another. This is a bit of evidence in favor of the notion that the
inductive bias of large LMs does, in fact, aid in learning quickly and
generalize well in several different senses.

Figure 7: The advantage of English scenarios over equally consistent reasoning
scenarios in an invented language is evidence for the aid of language models’
inductive bias.

##  5 Conclusions and Future Work

The central goal of this paper has been to investigate if connectionist LMs
can learn something akin to symbolic rules that generalize in “natural" ways.
We have shown strong performance on a suite of reasoning tasks, including
tasks that we consider to be simple–an object manipulation task and a
navigation task–and more difficult–a composite task that combines the other
two. On the container task, we show near perfect performance on the training
domain and considerably far outside of it and an eventual marked drop-off in
performance. On the navigation task, we show differing patterns of
generalization on different subtasks, with slightly lower performance than on
the container task. We have also demonstrated some of the boundaries of
generalization by identifying several different kinds of generalization and
measuring them. We do not claim to have explored all the axes of
generalization possible in the study of these language models, but rather
shown how the generalization ability of LMs can be probed. But these results
are exciting because the models perform so well outside of the training domain
that it seems as if some general, symbolic rules have been learned.

Beyond generalizing just to new examples on learned tasks, we have
demonstrated that these models are aided in the performance of complex tasks
by learning first on elemental tasks. This hints at the prospect of learning
difficult tasks more quickly and better by learning more simple tasks first.

We have additionally argued that natural language provides a powerful
inductive bias for symbolic tasks by comparing on scenarios across different,
but equivalent grammars, one in natural language and one in an invented
language. Intuitively, learning the distribution of language that describes
the world helps a learner to understand the distribution of that world itself.
Thus, it is plausible that LMs might be effective ways to provide autonomous
agents with priors and world models.

Implicitly, we have demonstrated that connectionist architectures are capable
of strong performance on some classically symbolic tasks. This is done in
light of recent calls to incorporate guiding principles of symbolic AI into
connectionist models; it is exciting that language models seem to possess at
least some of the rule-learning and generalization that humans possess, as
opposed to the mere ability to recognize patterns and interpolate over a well-
explored training domain.

## References

  * Radford et al. [2019] Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever.  Language models are unsupervised multitask learners.  _OpenAI Blog_ , 1(8):9, 2019.
  * Devlin et al. [2018] Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova.  Bert: Pre-training of deep bidirectional transformers for language understanding.  _arXiv preprint arXiv:1810.04805_ , 2018.
  * Raffel et al. [2019] Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu.  Exploring the limits of transfer learning with a unified text-to-text transformer.  _arXiv preprint arXiv:1910.10683_ , 2019.
  * Brown et al. [2020] Tom B Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al.  Language models are few-shot learners.  _arXiv preprint arXiv:2005.14165_ , 2020.
  * Vaswani et al. [2017] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin.  Attention is all you need.  In _Advances in neural information processing systems_ , pages 5998–6008, 2017.
  * Radford et al. [2018] Alec Radford, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever.  Improving language understanding by generative pre-training.  _URL https://s3-us-west-2. amazonaws. com/openai-assets/researchcovers/languageunsupervised/language understanding paper. pdf_ , 2018.
  * Petroni et al. [2019] Fabio Petroni, Tim Rocktäschel, Patrick Lewis, Anton Bakhtin, Yuxiang Wu, Alexander H Miller, and Sebastian Riedel.  Language models as knowledge bases?  _arXiv preprint arXiv:1909.01066_ , 2019.
  * Bosselut et al. [2019] Antoine Bosselut, Hannah Rashkin, Maarten Sap, Chaitanya Malaviya, Asli Celikyilmaz, and Yejin Choi.  Comet: Commonsense transformers for automatic knowledge graph construction.  _arXiv preprint arXiv:1906.05317_ , 2019.
  * Bouraoui et al. [2020] Zied Bouraoui, Jose Camacho-Collados, and Steven Schockaert.  Inducing relational knowledge from bert.  In _Proceedings of the AAAI Conference on Artificial Intelligence_ , volume 34, pages 7456–7463, 2020.
  * Winograd [1971] Terry Winograd.  Procedures as a representation for data in a computer program for understanding natural language.  Technical report, Massachusetts Institute of Technology, 1971.
  * Fikes and Nilsson [1971] Richard E Fikes and Nils J Nilsson.  Strips: A new approach to the application of theorem proving to problem solving.  _Artificial intelligence_ , 2(3-4):189–208, 1971\.
  * Graves et al. [2014] Alex Graves, Greg Wayne, and Ivo Danihelka.  Neural turing machines, 2014.  URL http://arxiv.org/abs/1410.5401.
  * Graves et al. [2016] Alex Graves, Greg Wayne, Malcolm Reynolds, Tim Harley, Ivo Danihelka, Agnieszka Grabska-Barwińska, Sergio Gómez Colmenarejo, Edward Grefenstette, Tiago Ramalho, John Agapiou, Adrià Puigdomènech Badia, Karl Moritz Hermann, Yori Zwols, Georg Ostrovski, Adam Cain, Helen King, Christopher Summerfield, Phil Blunsom, Koray Kavukcuoglu, and Demis Hassabis.  Hybrid computing using a neural network with dynamic external memory.  _Nature_ , 538(7626):471–476, 2016.
  * Weber et al. [2019] Leon Weber, Pasquale Minervini, Jannes Münchmeyer, Ulf Leser, and Tim Rocktäschel.  Nlprolog: Reasoning with weak unification for question answering in natural language.  _arXiv preprint arXiv:1906.06187_ , 2019.
  * Weston et al. [2015] Jason Weston, Antoine Bordes, Sumit Chopra, Alexander M Rush, Bart van Merriënboer, Armand Joulin, and Tomas Mikolov.  Towards ai-complete question answering: A set of prerequisite toy tasks.  _arXiv preprint arXiv:1502.05698_ , 2015.
  * Caruana [1997] Rich Caruana.  Multitask learning.  _Machine learning_ , 28(1):41–75, 1997.
  * Liu et al. [2015] Xiaodong Liu, Jianfeng Gao, Xiaodong He, Li Deng, Kevin Duh, and Ye-Yi Wang.  Representation learning using multi-task deep neural networks for semantic classification and information retrieval.  2015\.
  * Phang et al. [2018] Jason Phang, Thibault Févry, and Samuel R Bowman.  Sentence encoders on stilts: Supplementary training on intermediate labeled-data tasks.  _arXiv preprint arXiv:1811.01088_ , 2018.
  * Brysbaert et al. [2014] Marc Brysbaert, Amy Beth Warriner, and Victor Kuperman.  Concreteness ratings for 40 thousand generally known english word lemmas.  _Behavior research methods_ , 46(3):904–911, 2014\.

## References

## Checklist

  1. 1.

For all authors…

    1. (a)

Do the main claims made in the abstract and introduction accurately reflect
the paper’s contributions and scope? [Yes] We demonstrate the claimed
inductive bias across a suite of experiments in Section 4.

    2. (b)

Did you describe the limitations of your work? [Yes] In Section 5, we make
explicit the obvious truth that we have not demonstrated a general ability of
LMs to learn anything, nor have we done all the probing that could be done of
LMs, lest a reader should conclude from the models’ impressive performance on
toy tasks that they could learn any reasoning task and perfrom this well.

    3. (c)

Did you discuss any potential negative societal impacts of your work? [No] The
risks associated with our paper are not unique to it, and are probably shared
by all of deep learning. They include environmental risks from intensive
computation required for training neural networks and the capacity for
eventually automating humans whose skills deep learning could learn to
automate. Our tasks are, at the end of the day, toy tasks, and our paper makes
no advance significant enough to threaten the livelihood of a human being
anytime soon or increase the environmental footprint of training neural
networks.

    4. (d)

Have you read the ethics review guidelines and ensured that your paper
conforms to them? [Yes] We have.

  2. 2.

If you are including theoretical results…

    1. (a)

Did you state the full set of assumptions of all theoretical results? [N/A]

    2. (b)

Did you include complete proofs of all theoretical results? [N/A]

  3. 3.

If you ran experiments…

    1. (a)

Did you include the code, data, and instructions needed to reproduce the main
experimental results (either in the supplemental material or as a URL)? [Yes]
We include the source code and appropriate raw data files in our supplemental
material submission, and we outline the data generation procedure, the
evaluation protocol, the training regime, and everything else necessary for
reproduction either in the main body of the paper or in the appendix. The only
exception to this was the proprietary dataset mentioned in Subsection 3.1.

    2. (b)

Did you specify all the training details (e.g., data splits, hyperparameters,
how they were chosen)? [Yes] See Subsection 3.3 and the appendix.

    3. (c)

Did you report error bars (e.g., with respect to the random seed after running
experiments multiple times)? [Yes] We did this in Figure 5, but we didn’t do
this in other figures where the difference in performance was more clear.

    4. (d)

Did you include the total amount of compute and the type of resources used
(e.g., type of GPUs, internal cluster, or cloud provider)? [Yes] See
Subsection 3.3

  4. 4.

If you are using existing assets (e.g., code, data, models) or
curating/releasing new assets…

    1. (a)

If your work uses existing assets, did you cite the creators? [Yes] We cite
the creators of T5 in Section 1 and the formulator of the concreteness ratings
in Subsection 3.1, but the proprietary dataset is not public, and so we didn’t
cite the creator. We created all other assets.

    2. (b)

Did you mention the license of the assets? [Yes] We mention the only license
we know of (that belonging to T5) in the appendix.

    3. (c)

Did you include any new assets either in the supplemental material or as a
URL? [Yes] We include source code to regenerate our datasets.

    4. (d)

Did you discuss whether and how consent was obtained from people whose data
you’re using/curating? [No] The pretrained models were generated on huge web
scrapes of material in the public domain, and it would be impossible to
acquire consent for the use of this data.

    5. (e)

Did you discuss whether the data you are using/curating contains personally
identifiable information or offensive content? [No] Language models are known
to have privacy leaks, and the only extent to which our methods might lead to
privacy leaks is shared by all research using/fine-tuning language models, so
we didn’t feel the need to document this risk.

  5. 5.

If you used crowdsourcing or conducted research with human subjects…

    1. (a)

Did you include the full text of instructions given to participants and
screenshots, if applicable? [N/A]

    2. (b)

Did you describe any potential participant risks, with links to Institutional
Review Board (IRB) approvals, if applicable? [N/A]

    3. (c)

Did you include the estimated hourly wage paid to participants and the total
amount spent on participant compensation? [N/A]