# Reconstruction of the Density Power Spectrum from Quasar Spectra using
Machine Learning

Maria Han Veiga1, Xi Meng2, Oleg Y. Gnedin2, Nickolay Y. Gnedin3, Xun Huan4
1 Michigan Institute for Data Science, University of Michigan, Ann Arbor, MI
48109, USA
2 Department of Astronomy, University of Michigan, Ann Arbor, MI 48109, USA
3 Fermi National Accelerator Laboratory, Batavia, IL 60510, USA
Kavli Institute for Cosmological Physics, The University of Chicago, Chicago,
IL 60637 USA
Department of Astronomy & Astrophysics, The University of Chicago, Chicago, IL
60637 USA
4 Department of Mechanical Engineering, University of Michigan, Ann Arbor, MI
48109, USA Contact e-mail<EMAIL_ADDRESS>
(Submitted )

###### Abstract

We describe a novel end-to-end approach using Machine Learning to reconstruct
the power spectrum of cosmological density perturbations at high redshift from
observed quasar spectra. State-of-the-art cosmological simulations of
structure formation are used to generate a large synthetic dataset of line-of-
sight absorption spectra paired with 1-dimensional fluid quantities along the
same line-of-sight, such as the total density of matter and the density of
neutral atomic hydrogen. With this dataset, we build a series of data-driven
models to predict the power spectrum of total matter density. We are able to
produce models which yield reconstruction to accuracy of about 1% for
wavelengths $k\leq 2\,h\,\mathrm{Mpc}^{-1}$, while the error increases at
larger $k$. We show the size of data sample required to reach a particular
error rate, giving a sense of how much data is necessary to reach a desired
accuracy. This work provides a foundation for developing methods to analyse
very large upcoming datasets with the next-generation observational
facilities.

###### keywords:

methods: data analysis — methods: numerical — quasars: absorption lines —
large-scale structure of Universe

††pubyear: 2021††pagerange: Reconstruction of the Density Power Spectrum from
Quasar Spectra using Machine Learning–A

##  1 Introduction

New major observational facilities in the coming decade – in particular the
James Webb Space Telescope and the next generation 30-meter class telescopes
such as the GMT, TMT, and E-ELT – will produce an explosion in the quantity
and quality of observational data for thousands of quasars. Current samples of
$\sim 100$ quasars already known at redshift $z>6$ (Wang et al., 2019) will
increase to several thousand. The largest progress will occur in quasar
absorption spectroscopy, which allows a unique and particularly powerful probe
of quasar environment. Development of advanced numerical methods is required
to take full advantage of these upcoming datasets.

Recently, machine learning (ML) methods have begun to be introduced in
astronomical applications. For example, unsupervised deep learning methods
have been used to identify and discover exoplanets (Shallue & Vanderburg,
2018; McCauliff et al., 2015), or classification models to generate accurate
galaxy morphology catalogues (Domínguez Sánchez et al., 2018), predict
structure formation (He et al., 2019). For an non-exhaustive review of ML
methods in astronomy please refer to (Ball & Brunner, 2010; Carleo et al.,
2019).

In this paper we show how ML methods can be used to relate the clustering of
transmission spikes in quasar absorption spectra to the clustering of matter
along the observer’s line-of-sight (LoS). To make this possible, we use data
from the current state-of-the-art numerical simulations of galaxy formation
(Gnedin, 2014). The outputs of these simulations allow us to produce a large
sample of synthetic quasar spectra, for which we know the corresponding true
properties of cosmic gas, responsible for each spectral feature in the
simulated spectra. The complexity in inferring gas properties from spectral
data lies in the multitude of physical processes taking place around quasars
and the non-trivial relation between the locations in physical space and
specific wavelengths of spectral features (the so-called “redshift space”). We
will leverage supervised learning to discover these identifying features and
predict matter properties from spectral data.

The absorption spectra of quasars outside of their host galaxies provide clues
on the interplay of cosmic gas flows, formation of stars inside nearby
galaxies, and the clustering of matter on cosmological scales. Previous
efforts to explore the relation between the quasar absorption spectra and
clustering of matter along the quasar line of sight focused primarily on
cross-correlating absorption spectra with galaxies as tracers of the full
matter distribution (Kakiichi et al., 2018; Becker et al., 2018; Meyer et al.,
2019; Garaldi et al., 2019).

Our primary goal is to go beyond a simple cross-correlation analysis and to
construct mappings from the synthetic quasar spectra (“input”) to properties
of matter along the observer’s LoS (let us call this “labels”). The space of
potential labels is large: thermodynamic and ionization properties of cosmic
gas at each location in space, its chemical composition, distances to nearest
galaxies of various masses and star formation rate, duty cycle of quasar
activity, dynamical state of the host galaxy, etc. The connections between any
of these labels and the input are currently unknown. We seek to build these
mappings using deep learning, in order to leverage the very large dataset of
quasar spectra and matter properties that can be produced from the galaxy
formation simulation.

In this paper we report the initial effort in this direction. We analyze one
simulation output at redshift $z\approx 6$ and post-process it to produce a
set of 100,000 synthetic absorption spectra. Each LoS samples a region of
about 200 comoving $h^{-1}$ Mpc in length. As a first step, we aim to
reconstruct the matter density power spectrum (PS) along an observed LoS in
the quasi-linear regime on scales 1-100 comoving Mpc.

The rest of the paper is organized as follows. We describe the data generation
procedure in section 2, followed by the problem formulation and ML set-up in
section 3. The results are presented in section 4 with discussion in section
5, and finally conclusions in section 6.

##  2 Synthetic spectra

The first step in our exploration is to create a large dataset of synthetic
quasar spectra with known underlying line-of-sight matter properties. To
produce such a dataset, we use state-of-the-art numerical simulations of
galaxy formation performed with the Adaptive Refinement Tree (ART) code
(Kravtsov, 1999; Kravtsov et al., 2002; Rudd et al., 2008). These simulations
rely on the Adaptive Mesh Refinement algorithm to boost spatial and temporal
resolution for both gas dynamics and gravity calculations. The ART code
includes many physical processes that are critically important for modeling
the formation of galaxies and quasars. The basic components are gravitational
dynamics of dark matter and stars, and fluid dynamics of cosmic gas. The ART
code computes the thermodynamics of cosmic gas in a manner that is fully self-
consistent, without the commonly made assumption of thermodynamic equilibrium.
It tracks the abundances of heavy elements released by stellar winds and
supernovae and models the non-equilibrium chemistry network of molecular
hydrogen, which is needed to identify star-forming regions within galaxies.
The ART code includes some of the most advanced modeling of stellar feedback:
momentum ejection due to radiation pressure and stellar winds of young stars,
ionizing radiation of massive stars, thermal energy of supernova explosions,
and their kinetic feedback deposited directly into gas cells and into driving
supersonic turbulence.

The ART code accounts for the radiative transfer of ionizing and ultraviolet
radiation, which is critically important for proper modeling of the
interstellar and intergalactic gas. The local radiation flux affects
ionization states of the chemical elements, heating of cold gas, formation and
dissociation of molecular clouds, and the radiation pressure on gas and dust.
Modeling of radiative transfer will allow us to follow the propagation of
ionizing radiation from the quasar – a critical advantage of our simulations
in modeling the structure of the proximity zones. The primary data source for
the proposed work are the numerical simulations with the ART code from the
Cosmic Reionization on Computers project (Gnedin, 2014).

In the present analysis we use the output of one simulation at redshift
$z\approx 6$. The size of the comoving simulation box is $80\,h^{-1}$ Mpc,
which is not long enough to match the range of observed spectra in the quasar
vicinity. Taking advantage of the periodic boundary conditions in the
simulation box, we wrapped a given line 2.5 times around the box, to obtain
full line length $L=200\,h^{-1}$ Mpc. Such wrapping has been shown to result
in negligible artifacts induced by periodic boundary conditions (Dall’Aglio &
Gnedin, 2010). We sample each line-of-sight with 10240 resolution elements and
the spatial comoving resolution $\Delta x\approx 0.0195\,h^{-1}$ Mpc. In
total, we generated 100,000 synthetic line-of-sight spectra.

In addition, for each LoS spectrum, we store the information on the total
density of matter $\rho(x)$, the density of neutral atomic hydrogen $\rho_{\rm
HI}(x)$, the gas temperature $T(x)$, and the gas velocity $u(x)$. Using these
quantities we can compute the optical depth at a wavelength $\lambda$ as an
integral along the line:

$\tau(\lambda)=\int\rho_{\rm
HI}\frac{\sigma_{0}\,c}{\sqrt{\pi}\,m_{H}\,b(T)}\exp{\left[-\frac{(u_{\lambda}-u)^{2}}{b(T)^{2}}\right]}\frac{dx}{1+z},$
(1)

where $\sigma_{0}=4.5\times 10^{-18}$ cm2 is the cross-section for resonant
Ly$\alpha$ absorption, $b(T)=(2kT/m_{H})^{1/2}$ is the Doppler parameter,
$m_{H}$ is the mass of hydrogen atom, and $c$ is the speed of light (Gnedin,
2016). The dependency on wavelength $\lambda$ enters through $u_{\lambda}$:

$\frac{u_{\lambda}(x)}{c}=\frac{\lambda}{\lambda_{\rm em}}-1-z(x)$

where $\lambda_{\rm em}$ is the rest-frame emission wavelength. In this work
we focus on the normalized absorption flux, $F=e^{-\tau}$, without explicitly
modeling the intrinsic quasar emission spectrum.

Given a data field along the line-of-sight (such as transmitted flux or the
density) we generate the 1-dimensional perturbation power spectrum along that
LoS. The 1D power spectrum $P_{k}^{1D}$ is related the commonly used 3D
version $P^{3D}_{k}$ via an integral over all higher wavenumbers (Lumsden et
al., 1989):

$P_{k}^{1D}=\int_{k}^{\infty}P^{3D}_{k^{\prime}}\frac{k^{\prime}}{2\pi}dk^{\prime}.$
(2)

It incorporates information about all small scales. The 1D PS of the
transmitted quasar flux was first analyzed using cosmological simulations by
Croft et al. (1998).

##  3 Methodology

###  3.1 Problem overview

We are interested in the following inverse problem: given an observed flux
spectrum $F(\lambda)$, estimate the fluid quantities that produced this flux.
As a first step, we seek to build a mapping from $F$ to the power spectrum of
matter density $\rho$. In the context of this mapping, flux is the _input_ ,
and density is the _output_. We approach this goal by training a deep neural
network (DNN) model in a supervised learning manner using the simulation
datasets.

###  3.2 Data transformation

Compact high-density regions (halos, filaments) may contribute
disproportionately to the 1D power spectrum. In Appendix A we show examples of
three lines of sight that result in very different shape and normalization of
$P_{k}^{1D}$ depending on whether they happen to pass close to a high enough
density region or not.

To reduce this variation among individual lines, and to attribute larger
importance toward regions of low density (and therefore higher transmission
flux), we study three transformations of the matter density along a given
line-of-sight. Given the overdensity $\delta(x)\equiv\rho(x)/\bar{\rho}-1$,
where $\bar{\rho}$ is the global average matter density of the universe at the
studied epoch, we consider the following transformations:

  1. 1.

Linear transformation: $t(\rho)=\delta$

  2. 2.

Logarithmic transformation: $t(\rho)=\ln{(1+\delta)}$

  3. 3.

Inverse transformation: $t(\rho)=1-(1+\delta)^{-1}$.

All three transformations converge to $t(\rho)\approx\delta$ when $\delta\ll
1$. Nevertheless, as shown in Appendix A, the logarithmic and inverse
transformations significantly reduce the variation among individual lines-of-
sight.

The performance of the three choices within the DNN model will be presented in
section 4. The transformed density field now takes the role of the output.

###  3.3 Data representation

We then represent the signals of $F$ and $t(\rho)$ via truncated Fourier
expansion:

$f(x)\approx\sum_{\alpha=0}^{N(k_{\rm
max})}\hat{f}_{\alpha}\,\exp(-i2\pi\alpha x),$ (3)

where $\hat{f}_{\alpha}\in\mathbb{R}$ denotes linear Fourier coefficients and
$N(k_{\rm max})$ the number of terms considered. We select $N(k_{\rm max})$
based on the maximum wavenumber $k_{\rm max}$ we are interested in studying,
through the following relation:

$N(k_{\rm max})=\frac{L}{2\pi}k_{\rm max},$ (4)

where $L=200\,h^{-1}$ Mpc is the largest length-scale from the simulation
domain.

Figure 1 shows that at wavenumbers $k\gtrsim 1\,h\,\mathrm{Mpc}^{-1}$, the
cosmic perturbations are already in a non-linear regime. The residual mean
square (rms) mass fluctuation on scale $k$,

$\Delta_{1}\equiv\frac{kP_{k}^{1D}}{\pi},$

reaches $\Delta_{1}(k,z)\approx 0.78$ for $k=1\,h\,\mathrm{Mpc}^{-1}$, and
$\Delta_{1}(k,z)\approx 4.2$ for $k=8\,h\,\mathrm{Mpc}^{-1}$. Thus this range
of wavenumbers includes a mildly non-linear regime. As $k_{\mathrm{max}}$ is
increased, we expect it will become harder to predict the PS from quasar
spectra because of potential contribution of local galactic sources to
ionization of hydrogen.

Figure 1: An illustration of the linear power spectrum of cosmological
perturbations in one and three dimensions at the epoch $z\approx 6$ studied
here. In both cases we show the dimensionless mass perturbation:
$k^{3}P_{k}/2\pi^{2}$ (black line) and $kP_{k}^{1D}/\pi$ (red line). For
comparison we also show the average 1D power spectrum along all lines of sight
in the analyzed simulation output (blue line). Circles mark the choices of the
limiting wavenumber $k_{\mathrm{max}}$, which corresponds to the transition to
non-linear regime.

Our simulation data has two components: the density field and the flux. The
density field has a length of $L$, sampled uniformly by $N=10240$ points. The
flux fields do not have the same length, so in order to keep their spectral
representation consistent, we truncate all fluxes to the smallest common
length. With this procedure, the flux is represented by $N_{F}=2800$ points.

After performing Fourier transform on the flux and density signals, we can
extract their respective power spectra for each data pair $j$:

$\displaystyle F_{j}$
$\displaystyle=\left(|\hat{F}_{0}|^{2},\dots,|\hat{F}_{N(k_{\rm
in})}|^{2}\right)^{T}$ (5) $\displaystyle t_{j}$
$\displaystyle=\left(|\hat{t}_{1}|^{2},\dots,|\hat{t}_{N(k_{\mathrm{max}})}|^{2}\right)^{T}.$
(6)

Here $k_{\rm in}$ and $k_{\mathrm{max}}$ denote the maximum wavenumber used to
express the input and output, respectively. In our analysis we adopt $k_{\rm
in}=8k_{\mathrm{max}}$111Except for
$k_{\mathrm{max}}=8\,h\,\mathrm{Mpc}^{-1}$, where we take $k_{\rm
in}=4k_{\mathrm{max}}$ due to the limited number of the input modes., chosen
to ensure that there is enough information in the input to inform the output.
Other choices for this ratio can also be adopted, or one can also keep the
input size $N(k_{\rm in})$ fixed throughout the experiments.

Note that we omit the $0$-th mode on purpose in Equation 6. The $0$-th mode
corresponds to the mean value of the density, and it cannot be recovered from
the observed flux as it is completely degenerate with the value of the
ionizing background. We study the power of the Fourier modes instead of the
Fourier coefficients themselves because the former leads to a translation-
invariant representation.

Analyzing the variable relations line-by-line would be noisy and difficult. We
thus consider taking small groupings of the lines in an effort to average out
this variability. In this setup, each group contains $N_{g}$ independent LoS
and we compute the average of their power spectra:

$\displaystyle F_{g}(k)=\frac{1}{N_{g}}\sum_{j=1}^{N_{g}}F_{j}(k),\qquad
t_{g}(k)=\frac{1}{N_{g}}\sum_{j=1}^{N_{g}}t_{j}(k).$ (7)

We will also vary the grouping size to study the extent of prediction
performance improvement as a result of the reduced statistical variation.

Figure 2: Schematic of the end-to-end inverse problem pipeline. On the left-
hand side, the raw signal and its representation (feature) are shown, which
will be the input for the neural network. In the middle, neural network
structure is shown, depending on the sizes of the input and output vectors. In
this diagram, we are setting $N_{\mathrm{in}}=8$ and $N_{\mathrm{out}}=2$,
whereas in reality $N_{\mathrm{in}}$ is typically 8 times $N_{\mathrm{out}}$,
and $N_{\mathrm{out}}$ depends on the maximum wavelength $k_{\mathrm{max}}$
that we are interested in studying (for example,
$k_{\mathrm{max}}=1\,h\,\mathrm{Mpc}^{-1}$ leads to $N_{\mathrm{out}}=32$).
The right-hand side shows the output of the neural network, as well as the
output generation from the raw data, which we use for model training.

###  3.4 Deep Neural Network model

Our inverse problem entails learning a mapping

$\mathcal{M}:\mathcal{F}\to\tau$ (8)

where $\mathcal{F}\subset\mathbb{R}^{N_{\mathrm{in}}}$ is the vector space
containing elements $F_{g}$, and $\tau\subset\mathbb{R}^{N_{\mathrm{out}}}$ is
the vector space containing elements $t_{g}$. Here $N_{\mathrm{in}}=N(k_{\rm
in})+1$, and $N_{\mathrm{out}}=N(k_{\mathrm{max}})$.

We approximate $\mathcal{M}$ with $\mathcal{M}_{\theta}$ using a densely-
connected DNN (i.e., multilayer perceptron) of the form:

$\mathcal{M}_{\theta}(F_{g}):=h_{N}(...(h_{0}(A_{0}F_{g}+b_{0}))),$ (9)

where $h_{i}$ are nonlinear activation functions, and
$\theta=\\{A_{i},b_{i}\\}$ collects all trainable parameters (weights and bias
parameters) of the DNN. Note that the size of the output (dimension of $\tau$)
will depend on the maximal wavenumber $k_{\mathrm{max}}$ we are interested in
predicting. Because of this we consider the following 5-layer architecture,
shown schematically in Figure 2:

  * •

An input layer of size $N_{\mathrm{in}}$ accepting vector $F_{g}$.

  * •

A first hidden layer also with $N_{\mathrm{in}}$ neurons.

  * •

A second hidden layer with ${(N_{\mathrm{in}}+N_{\mathrm{out}})/2}$ neurons.

  * •

A third hidden layer with $N_{\mathrm{out}}$ neurons.

  * •

An output layer producing a prediction vector of size $N_{\mathrm{out}}$.

This architecture was chosen after empirically observing that, when using too
few degrees of freedom, the model would under-predict the output variation,
and the performance of the network decreased with shallower networks.
Rectified linear units (ReLU)222This activation function is chosen because it
is simple and more computationally efficient when compared to hyperbolic
tangent or sigmoid. Although there are other suitable activation functions,
typically modifications to ReLU (e.g., leaky ReLU, parametric ReLU or
randomised leaky ReLU), the empirical improvement on the performance of these
is not significant (Xu et al., 2015). are employed for all activation
functions $h_{i}$ of hidden layers, and linear activation is used for the
output layer.

In addition, we normalise the inputs and outputs. For the inputs, we apply a
standard scaling relation on each feature (component-wise):

$F_{\mathrm{in},\alpha}=\frac{F_{\alpha}-\mu_{\alpha}}{\sigma_{\alpha}},\quad\alpha=1,\cdots,N_{\mathrm{in}},$
(10)

where $\mu_{\alpha}$ is the sample average and $\sigma_{\alpha}$ is the sample
standard deviation of the $\alpha$-th feature across the data points. We are
careful to compute these estimates only from the training data to avoid data
leakage.

For the outputs, we simply divide the data by a fixed normalization constant,
$P_{\mathrm{norm}}$:

$t_{\mathrm{out}}=\frac{t_{g}}{P_{\mathrm{norm}}}.$ (11)

This constant is derived from the data and corresponds to the average
magnitude of the power spectrum, up to wavenumber $4\,h\,\mathrm{Mpc}^{-1}$.
We define it up to a rescaling factor of order unity as

$P_{\mathrm{norm}}=c\bar{P},\quad\bar{P}=\frac{1}{N_{\mathrm{points}}\,N(4)}\sum_{k}^{N(4)}\sum_{l=1}^{N_{\mathrm{points}}}t_{g,l}(k).$
(12)

The value of $\bar{P}$ remains fixed for the entirety of all numerical
experiments. We will investigate the rescaling factor, expected to be $c\sim
1$, as a hyperparameter. This normalization reduces the output range to be of
order unity and is similar to standard scaling. We have tested that such a
normalisation is extremely important for the performance of the network, as it
keeps the input and output values within a small range. Without the
appropriate normalisation, the network often fails.

We designate $N_{\mathrm{los}}=10^{4}$ lines to be a training set, and
separate $10^{4}$ lines to be a testing set that will only be used for the
final model evaluation. Then $N_{\mathrm{points}}=N_{\mathrm{los}}/N_{g}$ is
the number of independent groupings used for training. The training set is
further partitioned to perform 5-fold cross-validation (see Figure 3) for
selecting hyperparameters summarized in Table 1. The total number of model
parameters is listed in Table 2.

Note that while the density transformation $t(\delta)$, grouping size $N_{g}$,
and rescaling factor $c$ are choices that we make to optimize a particular
model $\mathcal{M}:\mathcal{F}\to\tau$, the maximum wavenumber
$k_{\mathrm{max}}$ directly influences what the model predicts. In an ideal
situation, we would like to be able to produce a model that predicts values
accurately for a large $k_{\mathrm{max}}$ because this would allow studies of
a wide range of modes. However, as $k$ increases, non-linear evolution of the
density field may break the connection between total matter density and
properties of neutral hydrogen, and therefore, we expect that the model will
not be able to produce accurate results. This is confirmed in our numerical
experiments, where we observe that the model performance degrades as we
increase $k_{\mathrm{max}}$.

Figure 3: Representation of the dataset division: the training set is used for
training, with cross-validation to select the various hyperparameters
presented in Table 1; the testing set is held out completely, and only used
for the final model evaluation.

While we employ a standard mean squared error (MSE) as the training loss, we
will use a customized error metric for cross-validation and testing, described
in the next subsection. The training loss minimization is performed using the
ADAM algorithm (Kingma & Ba, 2015), with learning rate 0.001 and 1st and 2nd
moment exponential decay rates of $(\beta_{1},\beta_{2})=(0.9,0.999)$ coupled
with back-propagation and a batch size of 128 datapoints for each iteration.
An early stopping criteria with patience of 15 epochs is used, and we recover
the best weights attained during the training phase as the final model.
Lastly, we also vary the fraction of training data used to explore the
sensitivity with respect to training data size, and the amount of training
data needed to attain a certain accuracy for our chosen DNN architecture.

Table 1: Scientific model hyperparameters. In addition to standard DNN hyperparameters (e.g., network architecture, activation function, etc.) we are interested in finding the optimal configuration of our model with respect to how we represent the data.  Hyperparameter | Choices
---|---
Density transformation $t(\delta)$ |  $\delta$, $\ln(1+\delta)$, $1-(1+\delta)^{-1}$
Grouping size $N_{g}$ | 1, 2, 5, 10, 20, 50, 100
Rescaling factor $c$ in $P_{\rm norm}$ |  $\nicefrac{{1}}{{4}}$,$\nicefrac{{1}}{{2}}$, $1$, $2$
Max output $k_{\mathrm{max}}(\,h\,\mathrm{Mpc}^{-1})$ | 1, 2, 4, 8
Table 2: Number of free parameters (weights and biases) for the chosen model Equation 9. These scale with the input $N_{\mathrm{in}}$ and output size $N_{\mathrm{out}}$.  $N_{\mathrm{in}},N_{\mathrm{out}}(k_{\mathrm{max}})$ | Number of free parameters
---|---
256, 31 (1) | 107,999
512, 63 (2) | 432,063
1024, 127 (4) | 1,728,383
1024, 254 (8) | 1,931,905

###  3.5 Evaluating performance

We evaluate the predictive performance of the model for the matter density
power spectrum with the following error function:

$\mathcal{E}_{0}=\frac{1}{N_{\rm
out}}\sum_{k}^{k_{\mathrm{max}}}\frac{|\mathcal{P}_{k}^{\mathrm{pred}}-\mathcal{P}_{k}^{\mathrm{true}}|}{\mathcal{P}_{k}^{\mathrm{pred}}+\mathcal{P}_{k}^{\mathrm{true}}}$
(13)

where

$\mathcal{P}_{k}\equiv\frac{1}{N_{\mathrm{points}}}\sum_{l=1}^{N_{\mathrm{points}}}t_{\mathrm{out},l}(k)$
(14)

is the spectrum coefficient corresponding to wavenumber $k$ averaged over all
groupings in the test (or validation) set. $\mathcal{P}_{k}^{\mathrm{true}}$
is the signal calculated directly from the simulation. This function is the
symmetric mean absolute percentage error (SMAPE) between
$\mathcal{P}_{k}^{\mathrm{pred}}$ and $\mathcal{P}_{k}^{\mathrm{true}}$, which
measures the relative errors between these quantities. Note that the number of
terms in the sum will vary depending on $k_{\mathrm{max}}$.

We use this relative difference of the predicted and true power spectrum
because it is the most basic measure of model performance. Perturbation modes
in the quasi-linear regime, when $\Delta_{1}(k)\lesssim 1$, are independent of
each other. If the model is unable to recover the average power spectrum in
the quasi-linear regime, it would not be able to predict the power at
different $k$ for individual lines or groups of lines. Taking the relative
difference, normalized by
$\mathcal{P}_{k}^{\mathrm{pred}}+\mathcal{P}_{k}^{\mathrm{true}}$, allows us
to minimize the influence of a single mode with the largest value of
$\mathcal{P}_{k}$.

We note that we employ an evaluation loss (for validation/testing) in Equation
13 that differs from the training loss.

We select this evaluation loss to illustrate several ideas. First, one may
choose the loss functions to reflect specific desirable properties of the
model prediction. For example in this case, Equation 13 indicates the average
error (over all validation points), but one may also target the variance,
other moments, pointwise matching, etc. Second, penalizing the average is a
more lenient metric compared to the MSE loss, and we wish to begin by
evaluating the models with this easier-to-achieve statistic. At the same time,
retaining the MSE for training can help ensure the models to produce
physically realistic outputs, and prevent fortuitous cancellations of wildly
unrealistic predictions that may still yield good averages. In the future, we
will explore evaluation metrics for individual or small groups of LoS.
Overall, both the training and evaluation losses may be customized to better
align with the desired properties of the models. Typically, the training loss
for a gradient based optimisation must be differentiable, and the performance
metric should be easily interpretable.

##  4 Results

We begin by addressing the choice of model hyperparameters from Table 1. We
select these hyperparameters by performing 5-fold cross-validation as
illustrated in Figure 3 – that is, finding the best hyperparameter setting
offering the lowest average (over all folds) cross-validation loss based on
Equation 13. Approaching this multi-variate mixed-integer optimisation problem
directly is very expensive; instead, we investigate the effects of one
hyperparameter at a time while fixing all others, and progressively zoom in to
their well-performing values. Our final findings, as well as results
illustrating these choices, are summarised on Table 3.

Table 3: Optimal choices of hyperparameters based on numerical experiments.  Hyperparameter | Best Choices | Evidence
---|---|---
Rescaling factor $c$ in $P_{\mathrm{norm}}$ | 1 or $\nicefrac{{1}}{{2}}$ | Figure 4
Density transformation $t(\delta)$ | $\ln(1+\delta)$ | Figure 5
Grouping size $N_{g}$ | 5 or 10 |  Figure 4, Figure 5
Max output $k_{\mathrm{max}}(\,h\,\mathrm{Mpc}^{-1})$ | 1 or 2 |  Figure 4, Figure 5

Figure 4 shows how the performance of the model changes with the normalisation
factor $P_{\mathrm{norm}}$ and density transformation. The results are
computed from validation sets as the average over the 5 folds, with the error
bar denoting plus/minus one standard deviation. From this plot, we first note
that the logarithmic and inverse transformations perform very similarly, while
the original overdensity $\delta$ (“linear transformation”) performs
significantly worse. The normalisation factor $P_{\mathrm{norm}}$ does not
appear to influence the performance much for the logarithmic and inverse
transformation models, but it has a greater effect for the linear
transformation. This conclusion holds within the considered range of
$P_{\mathrm{norm}}$, and the error deteriorates noticeably as we move away
from this range. Selecting for the lowest error, we thus set the normalisation
to be $P_{\mathrm{norm}}=\bar{P}/2$ (i.e. $c=\nicefrac{{1}}{{2}}$) for the
remaining numerical experiments in this paper.

Table 4 lists the values of $\mathcal{E}_{0}$ for the three choices of density
transformation, and for grouping size $N_{g}$ from 1 to 100. The linear
transformation performs the worst across all $N_{g}$, while the logarithmic
and inverse transformations behave similarly; these trends are consistent with
Figure 4. The behavior can be further understood from Appendix A, where we
show that the linear transformation is most dominated by high density peaks,
and therefore, expect the model to perform worse. At last, we choose the
logarithmic transformation since it is more intuitive than the inverse
transformation.

Figure 4: Selection of normalisation constant $P_{\mathrm{norm}}$ for the
model output, defined by Equation 12, with different choices of the output
transformation $t(\delta)$. Using the original overdensity $\delta$ results in
a much reduced accuracy of the model inference, as discussed in detail in
Appendix. The logarithmic and inverse transformation give similar results. The
results are robust to the choice of values of $P_{\mathrm{norm}}$ in the
considered range.

In Figure 5, we compare different grouping size $N_{g}$ and maximum wavenumber
$k_{\mathrm{max}}$, while fixing output transformation and normalisation
factor to be our best choices: $\ln(1+\delta)$ and
$P_{\mathrm{norm}}=\bar{P}/2$, respectively. We attain the best performance
with $N_{g}=5$ or $10$, although the results remain similar for any grouping
size smaller than 50. The figure also shows how far into the small scales
(corresponding to larger $k$) the density field can be probed:
$k_{\mathrm{max}}=1,2\,h\,\mathrm{Mpc}^{-1}$ attains relatively low error,
while $k_{\mathrm{max}}=4,8\,h\,\mathrm{Mpc}^{-1}$ models perform visibly
worse. We thus take our default $k_{\mathrm{max}}$ to be
$2\,h\,\mathrm{Mpc}^{-1}$ in this study, although the choice of
$k_{\mathrm{max}}$ can also be influenced by the investigation context (e.g.,
if one is interested in probing a specific range of scales); see section 5 for
more discussion.

Table 4: Model performance factor $\mathcal{E}_{0}$ for different grouping size $N_{g}$ and output transformation $t(\delta)$, with fixed $k_{\mathrm{max}}=2\,h\,\mathrm{Mpc}^{-1}$ and $N_{\mathrm{los}}=8000$.  $N_{g}$ | $\delta$ | $\log(1+\delta)$ | $1-1/(1+\delta)$
---|---|---|---
1 | 0.118 | 0.018 | 0.020
2 | 0.122 | 0.019 | 0.016
5 | 0.169 | 0.016 | 0.013
10 | 0.196 | 0.014 | 0.014
20 | 0.188 | 0.014 | 0.015
50 | 0.056 | 0.020 | 0.022
100 | 0.065 | 0.028 | 0.024
Figure 5: Model performance for different grouping size $N_{g}$ and maximum
wavenumber $k_{\mathrm{max}}$. We show the best-case results with the largest
data size $N_{\mathrm{los}}=8000$ and optimized values of the density
transformation $t(\delta)$ and normalisation $P_{\mathrm{norm}}$. Limiting the
model to larger scales of $k_{\mathrm{max}}\leq 2\,h\,\mathrm{Mpc}^{-1}$
results in better model performance, while the dependence on the grouping size
is weak as long as $N_{g}<50$.

In Figure 6, we show the error on the reconstruction of the power spectrum as
a function of number of lines-of-sight considered in the training process
($N_{\mathrm{los}}$), for $k_{\mathrm{max}}=2\,h\,\mathrm{Mpc}^{-1}$ and three
choices of the grouping size $N_{g}$. As expected, the error decreases with
the number of lines-of-sight in the training set. However, we can observe that
after using a training dataset of a few thousand, the performance of the
models appears to converge to a constant value of the error.

This trend suggests that for this task and with our model architecture and
training selections, we do not require more than a few thousand data points to
obtain convergence of the model’s performance. If a larger dataset is
available from an actual observation, the model complexity could be increased
to match it. For a smaller dataset ($N_{\mathrm{los}}\lesssim 1000$), Figure 6
can be used to read off the expected value of error $\mathcal{E}_{0}$.

Figure 6: Accuracy of reconstruction of the logarithmic density transformation
as a function of training set size $N_{\mathrm{los}}$, for several choices of
grouping size $N_{g}$. We observe that increasing the size of training dataset
leads to an improved performance when the dataset sizes are small, however, as
the dataset size becomes larger, the performance gains are reduced. After
approximately 1000 LoS, the model’s performance increases only marginally.

Figure 7: An illustrative example of the effect of rescaling the output in a
model with $N_{\mathrm{los}}=8000$, $N_{g}=10$,
$k_{\mathrm{max}}=8\,h\,\mathrm{Mpc}^{-1}$, and $t(\delta)=\ln(1+\delta)$. Top
panel: One validation point. We can observe that the model predicts less
variance with $k$ on large scales and more variance on small scales, compared
to the true simulation. Two models with different $P_{\mathrm{norm}}$ produce
similar, but not identical power spectra. Bottom panel: Standard deviation of
the predicted power spectrum in the full validation dataset as a function of
$k$. The amount of variance increases with the normalisation factor
$P_{\mathrm{norm}}$.

From all these experiments, we conclude that the best choice of
hyperparameters for this model is the following: $t(\delta)=\ln(1+\delta)$,
$N_{g}=10$, $c=\nicefrac{{1}}{{2}}$ and
$k_{\mathrm{max}}=2\,h\,\mathrm{Mpc}^{-1}$. With this choice and using
$N_{\mathrm{los}}=3200$ to train our fiducial model, we take a new, completely
unseen test set to evaluate the performance of our model. We find
$\mathcal{E}_{0}=0.014$, very similar to the results shown in Table 4. This
demonstrates that we obtain equal performance on unseen data.

Since our performance metric $\mathcal{E}_{0}$ evaluates models on the
averaged value of power spectrum over many modes $k$, we can now study how
well the fiducial model works as a function of $k$. In the top panel of Figure
7 we show an example of reconstruction of the dimensionless mass fluctuation
$kP_{k}^{1D}/\pi$ for one randomly selected validation point. On large scales
($k\lesssim 1\,h\,\mathrm{Mpc}^{-1}$) the model predicts smoother function of
$k$, while on smaller scales it visibly predicts more variation with $k$. We
address this issue further in section 5.

The bottom panel of this figure shows that the amount of variance in predicted
power shows that no choice of the normalization factor allows the model to
match the amount of variance of the true simulated data at all $k$. We note
that this comparison of the variance should not be interpreted as direct
performance of the model, because the variance was not included in evaluation
metric. Instead, it is a demonstration of additional results predicted by the
model. In the future, if the pointwise variance is deemed an important
quantity for us to capture, it may be explicitly incorporated into the
training and evaluation loss functions.

Finally, in Figure 8, we show direct comparison between the predicted and true
power spectra, for several choices of $k_{\mathrm{max}}$. All other
hyperparameters are kept fixed at their optimal choices:
$t(\delta)=\ln(1+\delta)$, $c=\nicefrac{{1}}{{2}}$, and $N_{g}=10$. The ratio
of the predicted to true power spectrum is close to unity at all scales and
oscillates around a mean of $\sim 1$ as $k$ increases. On the logarithmic
x-axis scale many modes are compressed at high $k$. To display the mean trend
more clearly, we smooth the ratio with a moving window of 0.1 dex in
$\log{k}$, for small-scale modes with $k>0.2\,h\,\mathrm{Mpc}^{-1}$. The
larger-scale modes are displayed without smoothing.

We can observe that as $k_{\mathrm{max}}$ increases, the error on the
prediction also increases gradually. The median of all lines remains within 5%
of the true PS, but near $k\sim 8\,h\,\mathrm{Mpc}^{-1}$, individual lines
deviate (both, over and under) by up to 10% for the interquartile range and by
a larger amount for other lines. This is consistent with the behavior of the
performance metric $\mathcal{E}_{0}$ shown in Figure 5, which shows more
variance in the average predictions for higher values of $k$. Here we can
verify that the source of error is dominated by the larger wavenumbers. This
justifies our selection of only moderately non-linear scales,
$k_{\mathrm{max}}=2\,h\,\mathrm{Mpc}^{-1}$, as our optimal model.

Figure 8: Deviation of the model output from true values of PS, for models
with different choices of $k_{\mathrm{max}}$. On small scales
$k>0.2\,h\,\mathrm{Mpc}^{-1}$, the ratio is smoothed with a moving window in
$\log{k}$ with the size of approximately 0.1 dex. For these scales solid lines
show the median of the data points, while shaded regions show the 25%-75%
interval of the moving window range. On larger scales the solid lines show the
actual ratio.

##  5 Discussion

###  5.1 Present Model

Some of the parameters we consider, such as the grouping size $N_{g}$, density
transformation $t(\delta)$, or scaling factor $c$ are genuine hyper-
parameters, which can be fixed by optimizing the model performance. The two
key parameters – $k_{\mathrm{max}}$ and $N_{\mathrm{los}}$ – however, may
instead be determined by the scientific problem in question and the size of
observational sample. There are currently over 100 quasars known above $z=6$
(Bañados et al., 2016; Wang et al., 2019), but the largest progress in
increasing observational samples is expected after the next generation,
30-meter class telescopes become operational later in this decade. With the
approximately 10 times increase in collecting area over the current largest
existing telescopes, and the steep slope of the quasar luminosity function,
$N(>L)\propto L^{-1.7}$, the next generation of quasar survey will increase
observational samples by a factor of 30–50. This served as our motivation for
considering $N_{\mathrm{los}}$ in the range of a few thousand, even though we
can generate much larger simulation datasets. Even that amount of
observational data is probably a maximum limit, and therefore in future works,
it would be useful to consider models that could be trained with less data.

The choice of $k_{\mathrm{max}}$ is largely problem-dependent. For example,
for cosmological constraints $k_{\mathrm{max}}\sim 1\,h\,\mathrm{Mpc}^{-1}$
may suffice, as the matter power spectrum is currently calibrated to about 1%
precision to that scale and to 3% precision to $k_{\mathrm{max}}\sim
5\,h\,\mathrm{Mpc}^{-1}$ (Ho et al., 2021; Aricò et al., 2021, at $z=0$, but
the accuracy at higher redshifts is expected to be comparable or even better
due to weaker clustering on a given spatial scale). For other applications,
pushing deeper into the non-linear regime may be desirable. For example,
probing the gas power spectrum to $k_{\mathrm{max}}\sim
10\,h\,\mathrm{Mpc}^{-1}$ will be useful for comparing the clustering of the
cosmic gas in the general intergalactic medium with that in quasar proximity
zones (Chen & Gnedin, 2021), as a way of constraining quasar lifetimes and
observational biases.

For our particular problem in this paper, we had to make a number of choices
in setting up and training the neural networks. Main constraints were the
computing time required to train each neural network and a large physical
hyperparameter space to explore, in addition to the hyperparameters associated
with the neural network model (size, activation function, regularisation,
number of epochs, loss function). Such a large space of hyperparameters also
made it prohibitively expensive to use a standard hyperparameter optimizer
such as Hyperopt (Bergstra et al., 2021). Some of the lessons that we learned
in the process of working on this problem were: (i) the importance of scaling
the values of input and output (subsection 3.4); (ii) monitoring for model
over-fitting and stopping the training early to minimize its effects; and
(iii) keeping in mind that the number of gradient updates per epoch varies as
we vary the dataset size, which may require adjustment to the early-stopping
criteria.

One issue that became clear in our current results is the inability of the
model to recover the full variance of the dataset (Figure 7). Relative to the
true power spectrum, our best model predicts smoother function of $k$ on large
scales ($k\lesssim 1\,h\,\mathrm{Mpc}^{-1}$), while on smaller scales it
predicts visibly more variance with $k$. The former is expected – in the
linear regime (on large spatial scales) individual $k$ modes are independent
and hence uncorrelated. Since in the current implementation we use all the
modes as input, the neural net generically introduces correlations among them.
The model does not automatically remove such spurious correlations because the
training loss does not penalize for their presence. Alternatively, one can
imagine training a separate model for each $k$ mode, but such a setup would
not account for the real physical correlations between the modes in the mildly
non-linear regime that are included in our current model. The latter is
important and allows our best model to recover the power spectrum even for
$k\gg 1\,h\,\mathrm{Mpc}^{-1}$.

One may consider using data-driven models which can represent more complex
mappings, for example, through convolutional neural networks (CNN), to fix the
lack of variance on large scales. However, a convolution of neighboring data
points introduces a correlation between them. In addition, the modes on small
scales are indeed strongly correlated, but the mode correlation length is
strongly wavenumber dependent (becoming zero in the linear regime of large
scales), and such a strongly variable correlation between modes does not
easily decompose into a series of convolutions. Hence, it appears that the
only way to both explicitly eliminate correlations between $k$ modes in the
linear regime and to allow for such correlations in the mildly non-linear
regime is to include an appropriate penalty in the training loss. It is not
clear yet how to achieve this without prior knowledge of the signal to be
recovered.

One can also consider how observational effects such as readout noise or
instrumental resolution, absent in our synthetic input spectra, may affect the
power spectrum recovery. In this exploratory work we did not consider such
observational effects; since they are all important on small scales, we do not
expect them to affect sufficiently small wavenumbers. One effect that does
affect large scales is the quasar continuum placement. Modern, PCA-based
continuum placement models are highly accurate (Bosman et al., 2021), but not
so accurate as to make the quasar continuum errors negligible. The bias in the
continuum placement is, fortunately, estimated to be below the precision of
our power spectrum recovery procedure, and hence the continuum placement
errors can be averaged out to below that precision with a rather modest number
of LoS (a few hundred).

###  5.2 Future work

In this work we considered the task of predicting the average density power
spectrum from normalized quasar absorption spectra. As a next step, we are
interested in predicting individual LoS quantities from the observed spectra.
In this case, the model takes in a representation of one-dimensional quasar
spectra data (for example, through Fourier modes), and predicts a one-
dimensional representation of the density $\rho$. These pairwise quantities
can be considered a _time-series_ (with the variable of “time” in this case
being distance along the line), using one to predict the other. In this
problem, other network architectures, such as recurrent neural networks
(RNNs), might be more suitable to encode the variance associated with
individual LoS.

There are additional issues to consider for this problem. For example, the
quasar light is fully absorbed in regions that are dense enough, producing
saturated absorption, making no model able to accurately predict the density
in those regions. Thus, quasar spectra only contain information about regions
with sufficiently low densities. This limitation can be mitigated if our
ability to produce a trustworthy prediction is expressed through model
uncertainty, for example, using Bayesian neural networks (Sun et al., 2017).

With the degeneracy of high density regions in mind, we can consider a
classification problem instead, where we classify regions along the LoS simply
as high, medium or low density regions. This is equivalent to a binned
representation of the density field.

Other promising directions include probing even smaller scales
($k>10\,h\,\mathrm{Mpc}^{-1}$) to investigate quasar proximity zones, as well
as accounting for the effect of ionizing radiation escaping from massive
galaxies near a given LoS.

##  6 Conclusions

We generate a novel dataset of pairs of 1-dimensional density fields and the
correspondent absorption spectra from the CROC suite of galaxy formation
simulations. We build an end-to-end methodology to infer the matter density
power spectrum from the quasar spectra. We explore various ways to describe
the data and set-up the model in order to optimize the metric specified in
Equation 13. With the best set of hyperparameters, we are able to reconstruct
the power spectrum to an accuracy near 1% up to
$k_{\mathrm{max}}=2\,h\,\mathrm{Mpc}^{-1}$. As $k_{\mathrm{max}}$ increases,
the scatter produced by the model increases.

In future work, we would explore more challenging tasks, such as predicting
the 1-dimensional density field from observed spectra and including full
observed quasar spectrum. This work provides a foundation for developing
advanced analysis methods for upcoming observations with JWST and 30-meter
class ground-based facilities.

## Acknowledgments

We thank the Michigan Institute for Data Science for support in the form of
the Propelling Original Data Science (PODS) grant. OG and XM were supported in
part by the U.S. National Science Foundation through grant 1909063. MHV was
supported by MIDAS. This manuscript has been co-authored by Fermi Research
Alliance, LLC under Contract No. DE-AC02-07CH11359 with the U.S. Department of
Energy, Office of Science, Office of High Energy Physics. This work used
resources of the Argonne Leadership Computing Facility, which is a DOE Office
of Science User Facility supported under Contract DE-AC02-06CH11357. An award
of computer time was provided by the Innovative and Novel Computational Impact
on Theory and Experiment (INCITE) program. This research was also part of the
Blue Waters sustained-petascale computing project, which was supported by the
National Science Foundation (awards OCI-0725070 and ACI-1238993) and the state
of Illinois. Blue Waters was a joint effort of the University of Illinois at
Urbana-Champaign and its National Center for Supercomputing Applications.

## Data availability

The data underlying this article will be shared on reasonable request to the
corresponding author.

## References

  * Aricò et al. (2021) Aricò G., Angulo R. E., Zennaro M., 2021, arXiv e-prints, p. arXiv:2104.14568
  * Bañados et al. (2016) Bañados E., et al., 2016, ApJS, 227, 11
  * Ball & Brunner (2010) Ball N. M., Brunner R. J., 2010, International Journal of Modern Physics D, 19, 1049
  * Becker et al. (2018) Becker G. D., Davies F. B., Furlanetto S. R., Malkan M. A., Boera E., Douglass C., 2018, ApJ, 863, 92
  * Bergstra et al. (2021) Bergstra J., Yamins D., Cox D. D., 2021, Hyperopt: A Python Library for Optimizing the Hyperparameters of Machine Learning Algorithms
  * Bosman et al. (2021) Bosman S. E. I., Ďurovčíková D., Davies F. B., Eilers A.-C., 2021, MNRAS, 503, 2077
  * Carleo et al. (2019) Carleo G., Cirac I., Cranmer K., Daudet L., Schuld M., Tishby N., Vogt-Maranto L., Zdeborová L., 2019, Rev. Mod. Phys., 91, 045002
  * Chen & Gnedin (2021) Chen H., Gnedin N. Y., 2021, arXiv e-prints, p. arXiv:2101.11627
  * Croft et al. (1998) Croft R. A. C., Weinberg D. H., Katz N., Hernquist L., 1998, ApJ, 495, 44
  * Dall’Aglio & Gnedin (2010) Dall’Aglio A., Gnedin N. Y., 2010, ApJ, 722, 699
  * Domínguez Sánchez et al. (2018) Domínguez Sánchez H., Huertas-Company M., Bernardi M., Tuccillo D., Fischer J. L., 2018, Monthly Notices of the Royal Astronomical Society, 476, 3661
  * Garaldi et al. (2019) Garaldi E., Gnedin N. Y., Madau P., 2019, ApJ, 876, 31
  * Gnedin (2014) Gnedin N. Y., 2014, ApJ, 793, 29
  * Gnedin (2016) Gnedin N. Y., 2016, Saas-Fee Advanced Course, 43, 1
  * He et al. (2019) He S., Li Y., Feng Y., Ho S., Ravanbakhsh S., Chen W., Póczos B., 2019, Proceedings of the National Academy of Sciences, 116, 13825
  * Ho et al. (2021) Ho M.-F., Bird S., Shelton C. R., 2021, arXiv e-prints, p. arXiv:2105.01081
  * Kakiichi et al. (2018) Kakiichi K., et al., 2018, MNRAS, 479, 43
  * Kingma & Ba (2015) Kingma D. P., Ba J., 2015, in 3rd International Conference on Learning Representations, ICLR 2015. San Diego, CA
  * Kravtsov (1999) Kravtsov A. V., 1999, PhD thesis, NEW MEXICO STATE UNIVERSITY
  * Kravtsov et al. (2002) Kravtsov A. V., Klypin A., Hoffman Y., 2002, ApJ, 571, 563
  * Lumsden et al. (1989) Lumsden S. L., Heavens A. F., Peacock J. A., 1989, MNRAS, 238, 293
  * McCauliff et al. (2015) McCauliff S. D., et al., 2015, The Astrophysical Journal, 806, 6
  * Meyer et al. (2019) Meyer R. A., Bosman S. E. I., Kakiichi K., Ellis R. S., 2019, MNRAS, 483, 19
  * Rudd et al. (2008) Rudd D. H., Zentner A. R., Kravtsov A. V., 2008, ApJ, 672, 19
  * Shallue & Vanderburg (2018) Shallue C. J., Vanderburg A., 2018, The Astronomical Journal, 155, 94
  * Sun et al. (2017) Sun S., Chen C., Carin L., 2017, in Singh A., Zhu J., eds, Proceedings of Machine Learning Research Vol. 54, Proceedings of the 20th International Conference on Artificial Intelligence and Statistics. PMLR, Fort Lauderdale, FL, USA, pp 1283–1292, http://proceedings.mlr.press/v54/sun17b.html
  * Wang et al. (2019) Wang F., et al., 2019, ApJ, 884, 30
  * Xu et al. (2015) Xu B., Wang N., Chen T., Li M., 2015, Empirical Evaluation of Rectified Activations in Convolutional Network (arXiv:1505.00853)

##  Appendix A Data transformation

Figure 9: Top panel: Illustration of the matter overdensity along a short part
of 3 randomly chosen lines of sights. Line B has several large spikes, which
bias the 1D PS. Bottom three panels: The power spectrum of $\delta$,
$\ln{\delta}$, and inverse $1+\delta$ for each of the three lines. Because of
the large scatter of points, for clarity we do not connect them at high $k$.
Black lines show the corresponding average of all lines of sight in the
simulation.

The PS for individual LoS may differ significantly because of a few high-
density regions dominating the total. The transformations of density field
described in subsection 3.2 are designed to mitigate their effects.

Figure 9 illustrates how the logarithmic and inverse transformations can
significantly reduce the variation and range of individual lines. We use an
example of three lines chosen randomly from the training dataset. The top
panel shows the matter overdensity along a small part (5%) of the line length,
for clarity. Line B is significantly different from lines A and C in that it
displays a very large (and rare) peak of overdensity $\delta>10^{3}$. This
small-scale peak leads to a much higher normalization of the original 1D PS at
all wavenumbers (see second panel), relative to the average over all 100,000
lines, because of the integration over small scales (see Equation 2). On the
other hand, the PS of lines A and C happen to be systematically lower than the
average. Thus the PS of any of these lines are not representative of the
cosmic average.

In contrast, the logarithmic transformation brings the resulting PS of all
three lines close to each other and to the cosmic average (third panel of
Figure 9). The inverse transformation behaves similarly (bottom panel of
Figure 9). Therefore, either of these transformations allows us to use even a
relatively small number of LoS to obtain a representative measure of the
matter PS. For most of our results, we choose the logarithmic transformation,
as it has a more intuitive interpretation.


# Rings whose subrings are all Noetherian or Artinian

Nathan Blacher

###### Abstract

We study noncommutative rings whose proper subrings all satisfy the same chain
condition. We show that if every proper subring of a ring $R$ is right
Noetherian, then $R$ is either right Noetherian or the trivial extension of
$\mathbb{Z}$ by the Prüfer $p$-group for a prime $p$. We also prove that if
every proper subring of $R$ is right Artinian, then $R$ is either right
Artinian or $\mathbb{Z}$. For commutative rings, both results were proved by
Gilmer and Heinzer in 1992. Our result for right Artinian subrings only
generalises the absolute case of their commutative result. We generalise the
full result (when only certain subrings are right Artinian) in the context of
PI rings.

##  1 Introduction and Notation

Subrings naturally inherit some ring-theoretic properties, such as being a
domain, the ring’s characteristic and satisfying a given polynomial identity.
Chain conditions however can differ widely between a ring and its subrings.
For instance a field can contain a subring which is not Noetherian. In [1],
Gilmer and Heinzer characterised commutative Noetherian rings whose subrings
are all Noetherian. In [2], they showed that a commutative ring whose proper
subrings are all Noetherian is either Noetherian or isomorphic to the trivial
extension of $\mathbb{Z}$ by the Prüfer $p$-group for a prime $p$. In the same
paper they prove that a commutative ring whose proper subrings are all
Artinian is either Artinian or isomorphic to $\mathbb{Z}$. We generalise both
latter results to all rings, without the commutative hypothesis.

######  Theorem 1.

Let $R$ be a ring which is not right Noetherian. Suppose that every proper
subring of $R$ is right Noetherian. Then $R$ is the trivial extension of
$\mathbb{Z}$ by the Prüfer $p$-group $\mathbb{Z}\left(p^{\infty}\right)$ for
some prime $p$.

The proof of this theorem relies on the commutative version and follows the
strategy of [2], with the necessary changes to adapt it to the noncommutative
setting.

######  Theorem 2.

Let $R$ be a ring which is not right Artinian. Suppose that every proper
subring of $R$ is right Artinian. Then $R$ is isomorphic to $\mathbb{Z}$.

In the commutative case, the authors of [2] proved the stronger result that if
$C$ is a proper subring of $R$ such that every proper subring of $R$
containing $C$ is Artinian, then $R$ is Artinian. They did this by using
classical Krull dimension. In particular they showed that the dimension of a
commutative ring cannot strictly exceed that of all its subrings. This
approach does not work in the noncommutative setting as classical Krull
dimension is not as well behaved there. However in the context of rings with
polynomial identity, we recover the zero-dimensional case of this result, in
the form of Proposition 4.2. This allows us to recover the stronger result for
PI rings.

######  Theorem 3.

Let $R$ be a PI ring and $C$ a central proper subring of $R$. Suppose that
every proper subring of $R$ containing $C$ is right Artinian. Then $R$ is
right Artinian.

In [3], Gilmer and O’Malley considered (possibly noncommutative) rings without
1 whose subrings are all right Noetherian. They showed that such a ring is
either right Noetherian or the zero ring on a Prüfer $p$-group. Of course in
this context, the condition on subrings is much stronger than in ours, as for
instance every one-sided ideal is a subring in a ring without 1.

All rings considered in this paper are associative with $1$. Subrings of a
given ring $R$ must contain the identity element of $R$. We denote inclusion
by $\subset$ and strict inclusion by $\subsetneq$.

For a prime number $p$, the Prüfer $p$-group is $\mathbb{Z}[1/p]/\mathbb{Z}$
with usual addition, denoted by $\mathbb{Z}(p^{\infty})$. It is an infinite
group whose proper subgroups are all finite. More precisely, every proper
subgroup of $\mathbb{Z}(p^{\infty})$ is cyclic and generated by $1/p^{n}$ for
some $n\geq 0$.

For a commutative ring $C$ and a $C$-module $M$, the trivial extension of $C$
by $M$ is the ring whose additive group is the direct sum $C\oplus M$,
equipped with the multiplication
$(c_{1},m_{1})(c_{2},m_{2})=(c_{1}c_{2},c_{1}m_{2}+c_{2}m_{1})$. This
construction can be found in [4, Example 2.22(A)]. It is sometimes called
Nagata’s idealisation in the literature.

Following [5, Chapter 6], we call classical Krull dimension of a ring $R$ the
maximal length of a chain of prime ideals in $R$. We denote it by
$\operatorname{Cl.K.dim}.K.dim(R)$. For a right $R$-module $M$, the Krull
dimension of $M$ is the deviation of the lattice of submodules of $M$. We
denote it by $\operatorname{K}(M)$.

### Acknowledgements.

The author would like to thank Vladimir Bavula for suggesting to look at the
problems considered in this paper and for helpful comments and discussions.
The author is supported by EPSRC grant EP/T517835/1.

##  2 Rings whose proper Subrings are right Noetherian

The question we are considering was solved in the commutative case by Gilmer
and Heinzer with the following result.

######  Theorem 2.1.

[2, Corollary 4] Let $R$ be a commutative ring which is not Noetherian.
Suppose that every proper subring of $R$ is Noetherian. Then $R$ is the
trivial extension of $\mathbb{Z}$ by $\mathbb{Z}(p^{\infty})$ for some prime
$p$.

We generalise this result to noncommutative rings. Our proof relies on Theorem
2.1 and essentially follows the strategy of [2], though some changes are
needed to adapt it to the noncommutative world. We first need a result which
is essentially a reformulation of [3, Theorem 3.1].

######  Proposition 2.2.

[3, Theorem 3.1] Let $R$ be a ring and $I\lhd_{r}R$ a right ideal of $R$ that
is not finitely generated. If every proper submodule of $I_{R}$ is finitely
generated, then $I^{2}=0$.

###### Proof.

Take any element $x$ in $I$. If $xI=I$ then $I=xR$ is finitely generated,
contradicting our hypothesis. So the right ideal $xI$ is properly contained in
$I$. Thus $xI$ is finitely generated and there are $i_{1},\ldots,i_{n}$ in $I$
such that $xI=xi_{1}R+\ldots+xi_{n}R$. Taking any $r$ in $I$, we have
$xr=xi_{1}r_{1}+\ldots+xi_{n}r_{n}$ for some $r_{1},\ldots,r_{n}$ in $R$. Thus
$x(r-i_{1}r_{1}-\ldots-i_{n}r_{n})=0$ i.e. $r-i_{1}r_{1}-\ldots-i_{n}r_{n}$
belongs to the right annihilator of $x$ in $I$, which we call $A$. In
particular, $r$ belongs to $A+i_{1}R+\ldots+i_{n}R$, and since $r$ was
arbitrary in $I$, we have $I=A+i_{1}R+\ldots+i_{n}R$. Since $I_{R}$ is not
finitely generated, $A$ cannot be finitely generated either and it follows
that $A=I$, that is $xI=0$. As $x$ was arbitrary in $I$, this implies that
$I^{2}=0$. ∎

We start by studying pairs $C\subsetneq R$, where $C$ is a central proper
subring of a ring $R$, and every proper subring of $R$ containing $C$ is right
Noetherian. Theorem 1 will then follow by taking $C$ to be the minimal subring
of $R$. In the commutative case, Lemmas 2.3 and 2.4 and Propositions 2.6 and
2.7 are statements of [2, Theorem 3].

######  Lemma 2.3.

Let $R$ be a ring and $C$ a central proper subring of $R$. Suppose that every
proper subring of $R$ containing $C$ is right Noetherian. If a right ideal $I$
of $R$ is not finitely generated, then $I^{2}$ is contained in $(I\cap C)R$.
If moreover $I^{2}=0$ then $R$ is commutative.

###### Proof.

Since $C$ is Noetherian, its ideal $I\cap C$ is finitely generated, and thus
so is the ideal $I^{\prime}:=(I\cap C)R$ of $R$. As $I_{R}$ is not finitely
generated then neither is $I/I^{\prime}$ as a right $R$-module, nor as a right
ideal of $R/I^{\prime}$. Therefore the ring $R/I^{\prime}$ is not right
Noetherian, but each of its proper subrings containing
$(C+I^{\prime})/I^{\prime}$ is right Noetherian.

Now, since $I_{R}$ is not finitely generated, neither is $I$ as a right ideal
of $C+I$. Thus the ring $C+I$ is not right Noetherian and it follows that
$C+I=R$. Therefore as Abelian groups we have

$R/I^{\prime}=(C+I^{\prime})/I^{\prime}\oplus I/I^{\prime}.$

Moreover, for any right ideal $A$ of $R/I^{\prime}$ properly contained in
$I/I^{\prime}$, we have that $(C+I^{\prime})/I^{\prime}\oplus A$ is a proper
subring of $R/I^{\prime}$. Thus $(C+I^{\prime})/I^{\prime}\oplus A$ is right
Noetherian and it follows that $A$ is finitely generated as a right
$R/I^{\prime}$-module. Thus the right ideal $I/I^{\prime}$ satisfies the
hypothesis of Proposition 2.2 and we have $\left(I/I^{\prime}\right)^{2}=0$,
i.e. $I^{2}\subset I^{\prime}$ as required.

Now since $R=C+I$, its commutator satisfies $[R,R]=[C+I,C+I]=[I,I]\subset
I^{2}$ using that $C$ is central. Thus if $I^{2}=0$, then $R$ is commutative
as required. ∎

######  Lemma 2.4.

Let $R$ be a ring and $C$ a central proper subring of $R$. Suppose that every
proper subring of $R$ containing $C$ is right Noetherian. Then $R$ has ACC on
semiprime ideals. In particular, for any ideal $A\lhd R$, there are only
finitely many prime ideals minimal with respect to containing $A$.

###### Proof.

Let $I_{1}\subsetneq I_{2}\subsetneq\ldots$ be a chain of semiprime ideals and
let $I$ denote the union $\bigcup_{j\geq 1}I_{j}$. Since $C$ is Noetherian,
$(I\cap C)R$ is finitely generated as a right ideal of $R$ and thus there
exists a $k\geq 1$ such that $I_{k}$ contains $(I\cap C)R$. If the chain does
not terminate, then $I_{R}$ is not finitely generated and by Lemma 2.3,
$I^{2}$ is contained in $(I\cap C)R$. Thus $I^{2}$ is contained in $I_{k}$,
which is only possible when $I_{k}=I$, as $I_{k}\subset I$ and $I_{k}$ is
semiprime. For the last statement, see for instance [6, Proposition 4.4.13]. ∎

######  Proposition 2.5.

Let $R$ be a ring and $C$ a central proper subring of $R$. Suppose that every
proper subring of $R$ containing $C$ is right Noetherian. If $C$ is Artinian
then $R$ is right Noetherian.

###### Proof.

We assume that $R$ is not right Noetherian and we seek a contradiction. In the
commutative case, this result is a consequence of [2, Corollary 3] so it is
enough to show that $R$ is commutative. Let $I\lhd_{r}R$ be a right ideal of
$R$ that is not finitely generated. Take a strictly descending chain of right
ideals $I=I_{0}\supsetneq I_{1}\supsetneq\ldots$ such that no $I_{j}$ is
finitely generated. We aim to show the chain must terminate.

First, for any $j\geq 0$ such that $I_{j+1}$ exists, since the right ideal
$I_{j+1}$ is not finitely generated, the ring $C+I_{j+1}$ is not right
Noetherian and hence $C+I_{j+1}=R$. Thus we have $I_{j+1}\cap C\subsetneq
I_{j}\cap C$. Therefore the chain $I_{0}\supsetneq I_{1}\supsetneq\ldots$
yields a chain $I_{0}\cap C\supsetneq I_{1}\cap C\supsetneq\ldots$ of ideals
of $C$, which must terminate as $C$ is Artinian. So the original chain
terminates too and there is $k\geq 0$ such that the right ideal $I_{k}$ is not
finitely generated, but all its proper submodules are. By Proposition 2.2, it
follows that $(I_{k})^{2}=0$. Therefore $R$ is commutative by Lemma 2.3. ∎

######  Proposition 2.6.

Let $R$ be a ring and $C$ a central proper subring of $R$. Suppose that every
proper subring of $R$ containing $C$ is right Noetherian. If $R$ is prime then
it is right Noetherian.

###### Proof.

Suppose that there exists a right ideal $I\lhd_{r}R$ that is not finitely
generated. We seek a contradiction. By Lemma 2.3 we have $I^{2}\subset(I\cap
C)R$. In particular, if $I^{2}\neq 0$ then $I$ intersects $C$ nontrivially. On
the other hand if $I^{2}=0$, we know that $R$ is commutative. In any case we
can take a nonzero central element $x$ in $I$.

If the right ideal $xI$ is not finitely generated then $R=C+xI$. So for any
$y$ in $I$, we have $y=c+xz$ for some $c$ in $C$, $z$ in $I$. In particular
$c=y-xz$ belongs to $I$ and since $y$ was arbitrary in $I$ it follows that

$I\subset I\cap C+I^{2}\subset(I\cap C)R\subset I,\text{ so }I=(I\cap C)R.$

As $C$ is Noetherian, $(I\cap C)R=I$ is finitely generated, contradicting our
choice of $I$.

So $xI$ is finitely generated, and we write $xI=xi_{1}R+\ldots+xi_{n}R$ for
some $n\geq 1$ and $i_{1},\ldots,i_{n}$ in $I$. Then for any $r$ in $I$ we
have elements $r_{1},\ldots,r_{n}$ of $R$ such that
$xr=xi_{1}r_{1}+\ldots+xi_{n}r_{n}$. It follows that

$0=Rx(r-i_{1}r_{1}-\ldots-i_{n}r_{n})=xR(r-i_{1}r_{1}-\ldots-i_{n}r_{n})$

using that $x$ is central in $R$. Since $R$ is prime and $x$ nonzero, this
implies that $r=i_{1}r_{1}+\ldots+i_{n}r_{n}$. As $r$ was arbitrary in $I$, we
have $I=i_{1}R+\ldots+i_{n}R$, contradicting our choice of $I$. ∎

######  Proposition 2.7.

Let $R$ be a ring and $C$ a central proper subring of $R$. Suppose that every
proper subring of $R$ containing $C$ is right Noetherian. If $R$ is not right
Noetherian then the prime radical $N$ of $R$ is not Noetherian as a right
$R$-module and $R=C+N$. If moreover $C$ is reduced then $R$ is the trivial
extension of $C$ by $N$.

###### Proof.

Let $N$ be the prime radical of $R$, that is
$N=\bigcap_{P\in\operatorname{Spec}(R)}P$. First we assume that $N_{R}$ is
Noetherian and seek a contradiction. Since $N_{R}$ is Noetherian and $R$ is
not right Noetherian, it follows that the ring $R/N$ is also not right
Noetherian. Now, by Lemma 2.4, there are only finitely many minimal primes in
$R$, that is $N=P_{1}\cap\ldots\cap P_{m}$. This yields a natural inclusion of
$R$-modules $R/N\hookrightarrow\bigoplus_{i=1}^{m}R/P_{i}$, and since $R/N$ is
not right Noetherian, then there is at least one index $1\leq j\leq m$ such
that $R/P_{j}$ is not right Noetherian.

So $R/P_{j}$ is a prime ring which is not right Noetherian. Its subring
$(C+P_{j})/P_{j}$ is Noetherian and hence proper. Moreover, every proper
subring of $R/P_{j}$ containing $(C+P_{j})/P_{j}$ is right Noetherian. This is
impossible by Proposition 2.6, and it follows that $N_{R}$ is not right
Noetherian. In particular, the ring $C+N$ is not right Noetherian and
therefore $C+N=R$.

Now in the case where $C$ is reduced, we obtain $C\cap N=0$ as $N$ is nil (see
for instance [5, Theorem 0.2.6]). Thus $R=C\oplus N$ and it remains to show
that $N^{2}=0$. If $M$ is a proper submodule of $N_{R}$, then the proper
subring $C\oplus M$ of $R$ is right Noetherian, and $M_{R}$ is finitely
generated. Thus, by Proposition 2.2, $N^{2}=0$ as required. ∎

We can now prove Theorem 1, that is we generalise Theorem 2.1 to the
noncommutative setting.

###### Proof of Theorem 1.

Since $R$ is not right Noetherian, it must have characteristic zero by
Proposition 2.5, taking $C$ to be the minimal subring of $R$. But $\mathbb{Z}$
is reduced so $R$ is commutative by Proposition 2.7. We are done by Theorem
2.1. ∎

##  3 Rings whose proper Subrings are right Artinian

The question was also answered for the Artinian property by Gilmer and
Heinzer, with the following result.

######  Theorem 3.1.

[2, Theorem 2] Let $R$ be a commutative ring and $C$ a proper subring of $R$.
Suppose that every proper subring of $R$ containing $C$ is Artinian. Then $R$
is Artinian.

Taking $C$ to be the minimal subring of $R$ immediately yields the following
result.

######  Corollary 3.2.

[2, Corollary 2] Let $R$ be a commutative ring which is not Artinian. Suppose
that every proper subring of $R$ is Artinian. Then $R$ is isomorphic to
$\mathbb{Z}$.

In the noncommutative context, we start by noting that the Noetherian case
yields the following immediate consequence.

######  Lemma 3.3.

Suppose that every proper subring of a ring $R$ is right Artinian. Then $R$ is
right Noetherian.

###### Proof.

Every proper subring of $R$ is right Artinian, and hence right Noetherian. By
Theorem 1, $R$ is either right Noetherian or the trivial extension of
$\mathbb{Z}$ by $\mathbb{Z}\left(p^{\infty}\right)$ for some prime $p$. But
this trivial extension strictly contains $\mathbb{Z}$ as a subring, which is
not Artinian. It follows that $R$ is right Noetherian. ∎

We now prove Theorem 2, that is we generalise Corollary 3.2 to noncommutative
rings.

###### Proof of Theorem 2.

We assume that $R$ is not $\mathbb{Z}$ and seek a contradiction. If the
characteristic of $R$ is zero, then $R$ contains a copy of $\mathbb{Z}$ as a
proper subring, which contradicts our hypothesis. Therefore $R$ has finite
characteristic. By Lemma 3.3, $R$ is right Noetherian. Take an ideal $I$ of
$R$ maximal with respect to $R/I$ not being right Artinian. Then $R/I$
satisfies the hypothesis of the theorem. So without loss of generality we can
take $R$ to be $R/I$. Furthermore, every proper homomorphic image of $R/I$ is
right Artinian. We will show this yields a contradiction.

We first show that $R$ has prime characteristic. Write $p_{1}^{l_{1}}\cdots
p_{n}^{l_{n}}$ for the prime factorisation of the characteristic of $R$. We
have a ring isomorphism between the minimal subring
$\mathbb{Z}/(p_{1}^{l_{1}}\cdots p_{n}^{l_{n}})\mathbb{Z}$ and
$\Pi_{i=1}^{n}\mathbb{Z}/p_{i}^{l_{i}}\mathbb{Z}$. The preimage of
$(1,0,\ldots,0)$ under this isomorphism gives us a nonzero central idempotent
$e$ in $R$. Thus $R=Re\oplus R(1-e)$ as a direct sum of ideals. If $e\neq 1$,
then both $R/Re\cong R(1-e)$ and $R/R(1-e)\cong Re$ are proper homomorphic
images of $R$, and hence Artinian as right $R$-modules. In this case, $R_{R}$
is a direct sum of Artinian modules and is hence right Artinian as a ring,
contradicting our hypothesis. Therefore $e=1$, that is the characteristic of
$R$ is a prime power $p^{l}$.

Now consider the minimal subring $\mathbb{Z}/p^{l}\mathbb{Z}$ of $R$. If
$R=\mathbb{Z}/p^{l}\mathbb{Z}+pR$ then
$R=\mathbb{Z}/p^{l}\mathbb{Z}+p(\mathbb{Z}/p^{l}\mathbb{Z}+pR)=\mathbb{Z}/p^{l}\mathbb{Z}+p^{2}R$
and repeating the substitution yields
$R=\mathbb{Z}/p^{l}\mathbb{Z}+p^{l}R=\mathbb{Z}/p^{l}\mathbb{Z}$, which is
impossible as $R$ is not right Artinian. Thus $\mathbb{Z}/p^{l}\mathbb{Z}+pR$
is a proper subring of $R$. In particular it is right Artinian. Therefore the
right $(\mathbb{Z}/p^{l}\mathbb{Z}+pR)$-module $pR$ is Artinian, and it
follows that $pR$ is Artinian as a right $R$-module. The short exact sequence

$0\to pR\to R\to R/pR\to 0$

then implies that $R/pR$ is not right Artinian. This is only possible if
$R/pR$ is not a proper homomorphic image of $R$, i.e. if $pR=0$. Therefore the
characteristic of $R$ is $p$.

Now, $R$ is an $F_{p}=\mathbb{Z}/p\mathbb{Z}$-algebra which is not right
Artinian and such that every proper subring is right Artinian. Take an
infinite descending chain $R\supsetneq A_{1}\supsetneq A_{2}\supsetneq\ldots$
of right ideals of $R$. The right $R$-module $A_{2}$ is not Artinian, so the
ring $F_{p}+A_{2}$ is not right Artinian, hence $F_{p}+A_{2}=R$. Take $x$ in
$A_{1}\setminus A_{2}$. Then $x=u+a$ for some $a$ in $A_{2}$ and nonzero $u$
in $F_{p}$. But $u=x-a$ belongs to $A_{1}$, which is impossible as $F_{p}$ is
a field and $A_{1}$ is a proper ideal. This is the required contradiction, and
it follows that $R$ is isomorphic to $\mathbb{Z}$. ∎

The proof of [2, Theorem 2] uses classical Krull dimension. It is not as well
behaved in the general noncommutative setting, and our proof of Theorem 2
followed a different approach. In the next section, we recover some of the
results of [2] in the context of rings satisfying a polynomial identity (PI).

##  4 Classical Krull Dimension 0 in PI rings

######  Definition 4.1.

[6, Definition 1.7.27] Let $R$ be a ring, and $W$ a subring of $R$. Call $R$
an extension of $W$ if $R=W\operatorname{C}_{R}(W)$, where
$\operatorname{C}_{R}(W)$ is the centraliser of $W$ in $R$, i.e. elements of
$R$ that commute with $W$.

Of course, every ring is an extension of any of its central subrings. We will
use the following fact repeatedly.

######  Remark.

[6, Remark 1.9.5] Let $R$ be an extension of a subring $W$. If $P$ is a prime
ideal of $R$, then $W\cap P$ is a prime ideal of $W$.

The following is a generalisation to PI rings of the case $n=0$ of [2, Theorem
1].

######  Proposition 4.2.

Let $R$ be a PI ring that is an extension of a proper subring $W$. Suppose
that every proper subring of $R$ affine over $W$ has classical Krull dimension
zero. Then the classical Krull dimension of $R$ is also zero.

###### Proof.

Let us assume that there are prime ideals $P_{0}\subsetneq P_{1}$ in $R$. We
seek a contradiction. Since $R/P_{0}$ is a prime PI ring, the nontrivial ideal
$P_{1}/P_{0}$ contains a nonzero central element, $\bar{c}$ say (see for
instance [6, Theorem 1.6.27]). Now, $R$ is an extension of $W$ so $R/P_{0}$ is
an extension of $W^{\prime}:=(W+P_{0})/P_{0}$. Moreover since $\bar{c}$ is
central, $R/P_{0}$ also is an extension of $W^{\prime}[\bar{c}]$. The ring
$W^{\prime}[\bar{c}]$ is prime and its prime ideal $P_{1}^{\prime}:=P_{1}\cap
W^{\prime}[\bar{c}]$ is nonzero as it contains $\bar{c}\neq 0$. Thus the
classical Krull dimension of $W^{\prime}[\bar{c}]$ is at least one.

Taking $c$ in the preimage of $\bar{c}$ under the natural projection
$R\twoheadrightarrow R/P_{0}$, we have
$\operatorname{Cl.K.dim}.K.dimW[c]\geq\operatorname{Cl.K.dim}.K.dimW^{\prime}[\bar{c}]\geq
1$. It follows by hypothesis that $W[c]=R$. We can show similarly that
$W[c^{2}]=R$. Therefore $c$ is algebraic over $W$ and in particular $\bar{c}$
is algebraic over $W^{\prime}$. Let $l\geq 1$ be minimal such that there exist
$w_{0},\ldots,w_{l}\in W^{\prime}$ not all zero and satisfying
$\sum_{i=0}^{l}w_{i}\bar{c}^{i}=0$. Then
$-w_{0}=\bar{c}\left(\sum_{i=0}^{l-1}w_{i+1}\bar{c}^{i}\right)$ belongs to
$P_{1}/P_{0}$, as $\bar{c}$ does. Moreover, since $\bar{c}$ is a central
element in the prime ring $R/P_{0}$, it must be regular. Then, by minimality
of $l$, it follows that $w_{0}$ is not zero and thus the prime ideal
$W^{\prime}\cap(P_{1}/P_{0})$ is nontrivial. Therefore

$1\leq\operatorname{Cl.K.dim}.K.dimW^{\prime}=\operatorname{Cl.K.dim}.K.dimW/(W\cap
P_{0})\leq\operatorname{Cl.K.dim}.K.dimW=0$

which is absurd, showing that the classical Krull dimension of $R$ is zero. ∎

######  Proposition 4.3.

Let $R$ be a PI ring with classical Krull dimension zero. The following are
equivalent:

  1. (i)

$R$ is right Artinian;

  2. (ii)

the right Krull dimension of $R$ exists;

  3. (iii)

every ideal of $R$ is finitely generated as a right $R$-module.

###### Proof.

That (i) implies (ii) and (iii) is trivial. Now if $R$ has right Krull
dimension $\operatorname{K}(R_{R})$, then
$\operatorname{K}(R_{R})=\sup\\{\operatorname{K}(R/P):P\in\operatorname{Spec}(R)\\}$
(see for instance [5, Corollary 6.3.8]). This is zero as every prime ideal of
$R$ is maximal, and the simple PI rings $R/P$ are Artinian by Kaplansky’s
Theorem. Thus (ii) implies (i).

Finally, we prove that (iii) implies (i). We assume that $R$ is not right
Artinian, and seek a contradiction. Since $R$ has ACC on ideals, we can take
an ideal $P$ of $R$ maximal with respect to $R/P$ not being right Artinian.
Let $A,B$ be ideals strictly containing $P$. Since $A_{R}$ is finitely
generated, it follows that $A/AB$ is finitely generated as a right module over
$R/B$. As $R/B$ is right Artinian, this implies that $A/AB$ is Artinian as a
right $R$-module. Thus, the short exact sequence of right $R$-modules
$A/AB\hookrightarrow R/AB\twoheadrightarrow R/A$ implies that the ring $R/AB$
is right Artinian. Therefore $AB$ is not contained in $P$, and we have shown
that $P$ is prime. The classical Krull dimension of $R$ is zero so $P$ is
maximal, and $R/P$ is a simple PI ring, which is Artinian by Kaplansky’s
Theorem, contradicting our choice of $P$. ∎

Lastly, we prove Theorem 3, i.e. we generalise Theorem 3.1 to PI rings.

###### Proof of Theorem 3.

Every proper subring of $R$ containing $C$ is right Artinian and hence has
classical Krull dimension zero. Since $R$ is an extension of $C$, this implies
that the classical Krull dimension of $R$ is also zero by Proposition 4.2. By
Proposition 2.5, $R$ is right Noetherian. Thus $R$ satisfies condition (iii)
of Proposition 4.3 and it follows that $R$ is right Artinian. ∎

## References

  * [1] R. Gilmer and W. Heinzer.  Noetherian pairs and hereditarily noetherian rings.  Archiv der Mathematik, 41:131–138, 1983.
  * [2] R. Gilmer and W. Heinzer.  An application of jonsson modules to some questions concerning proper subrings.  Mathematica Scandinavica, 70:34–42, 1992.
  * [3] R. Gilmer and M. O’Malley.  Non-noetherian rings for which each proper subring is noetherian.  Mathematica Scandinavica, 31(1):118–122, 1972.
  * [4] T. Y. Lam.  Lectures on Modules and Rings.  Graduate Texts in Mathematics. Springer New York, NY, 1998.
  * [5] J. C. McConnell and J. C. Robson.  Noncommutative Noetherian rings.  Chichester : Wiley, 1987.
  * [6] L. H. Rowen.  Polynomial Identities in Ring Theory.  Academic Press, 1980.

School of Mathematics and Statistics
University of Sheffield
Hicks Building
Sheffield S3 7RH
United Kingdom
email<EMAIL_ADDRESS>