$\displaystyle-83160J[-2+\nu_{1},-1+\nu_{2}]+2772J[-2+\nu_{1},\nu_{2}]-51480J[-1+\nu_{1},-7+\nu_{2}]$
$\displaystyle+312312J[-1+\nu_{1},-6+\nu_{2}]-792792J[-1+\nu_{1},-5+\nu_{2}]+1081080J[-1+\nu_{1},-4+\nu_{2}]$
$\displaystyle-840840J[-1+\nu_{1},-3+\nu_{2}]+360360J[-1+\nu_{1},-2+\nu_{2}]-72072J[-1+\nu_{1},-1+\nu_{2}]$
$\displaystyle+3432J[-1+\nu_{1},\nu_{2}]+6435J[\nu_{1},-8+\nu_{2}]-51480J[\nu_{1},-7+\nu_{2}]$
$\displaystyle+180180J[\nu_{1},-6+\nu_{2}]-360360J[\nu_{1},-5+\nu_{2}]+450450J[\nu_{1},-4+\nu_{2}]$
$\displaystyle-360360J[\nu_{1},-3+\nu_{2}]+180180J[\nu_{1},-2+\nu_{2}]-51480J[\nu_{1},-1+\nu_{2}]$
$\displaystyle+6435J[\nu_{1},\nu_{2}]).$

##  Appendix B Calculation of 13-contributions in the Tau model

The 13-type loop correction due to contact operators from the exponential map
is given by

$\begin{split}&\delta
P_{13}=(b_{1}-b_{\eta}f\mu^{2})P_{\text{lin}}(k)\int_{\bf
q}P_{\text{lin}}(q)\left[(-\tau_{0})^{3}(b^{\tau}_{1}+f\mu^{2})Z^{2}_{1}({\bf
q})+4\tau_{0}^{2}Z_{2}({\bf k},{\bf q})Z_{1}({\bf q})\right]\,.\end{split}$
(B.1)

The first integral $\propto\tau_{0}^{3}$ can be easily evaluated:

$\begin{split}\int_{\bf q}P_{\text{lin}}(q)(b^{\tau}_{1}+f(\hat{\bf
z}\cdot\hat{\bf
q})^{2})^{2}=\left((b^{\tau}_{1})^{2}+\frac{2b^{\tau}_{1}f}{3}+\frac{f^{2}}{5}\right)\int_{0}^{\Lambda}dq~{}q^{2}P_{\text{lin}}(q)\,.\end{split}$
(B.2)

The rightmost term in Eq. (B.1) has the following full expression:

$\begin{split}\int_{\bf q}P_{\text{lin}}(q)&(b^{\tau}_{1}+f(\hat{\bf
z}\cdot\hat{\bf
q})^{2})\Bigg{[}\frac{b^{\tau}_{2}}{2}+b^{\tau}_{\mathcal{G}_{2}}\left(\frac{({\bf
q}\cdot{\bf k})^{2}}{k^{2}q^{2}}-1\right)+b^{\tau}_{1}F_{2}({\bf q},{\bf
k})\\\ &+f\frac{(\hat{\bf z}\cdot({\bf q}+{\bf k}))^{2}}{|{\bf q}+{\bf
k}|^{2}}G_{2}({\bf q},{\bf k})+\frac{f(\hat{\bf z}\cdot({\bf q}+{\bf
k}))}{2}\left(\frac{\mu}{k}(b^{\tau}_{1}+f(\hat{\bf z}\cdot\hat{\bf
q})^{2})+\frac{(\hat{\bf z}\cdot\hat{\bf
q})}{q}(b^{\tau}_{1}+f\mu^{2})\right)\Bigg{]}\end{split}$

Let us compute these terms one by one.

$\begin{split}\int_{\bf q}P_{\text{lin}}(q)&(b^{\tau}_{1}+f(\hat{\bf
z}\cdot\hat{\bf
q})^{2})\frac{b^{\tau}_{2}}{2}=\frac{b^{\tau}_{2}}{2}\left(b^{\tau}_{1}+\frac{f}{3}\right)\sigma^{2}_{\Lambda}\end{split}$
(B.3)

$\begin{split}\int_{\bf q}P_{\text{lin}}(q)&(b^{\tau}_{1}+f(\hat{\bf
z}\cdot\hat{\bf q})^{2})b^{\tau}_{\mathcal{G}_{2}}\left(\frac{({\bf
q}\cdot{\bf
k})^{2}}{k^{2}q^{2}}-1\right)=\left\\{b^{\tau}_{\mathcal{G}_{2}}\left(-b_{1}^{\tau}\frac{2}{3}-\frac{4f}{15}+\frac{2\mu^{2}f}{15}\right)\right\\}\sigma_{\Lambda}^{2}\end{split}$
(B.4) $\begin{split}b^{\tau}_{1}\int_{\bf
q}P_{\text{lin}}(q)&(b^{\tau}_{1}+f(\hat{\bf z}\cdot\hat{\bf
q})^{2})\Bigg{[}\frac{5}{7}+\frac{2}{7}(\hat{\bf k}\cdot\hat{\bf
q})^{2}\Bigg{]}=\left\\{(b^{\tau}_{1})^{2}\frac{17}{21}+\frac{9fb_{1}^{\tau}}{35}+\frac{4fb_{1}^{\tau}\mu^{2}}{105}\right\\}\sigma_{\Lambda}^{2}\end{split}$
(B.5) $\begin{split}&\int_{\bf q}P_{\text{lin}}(q)(b^{\tau}_{1}+f(\hat{\bf
z}\cdot\hat{\bf q})^{2})\frac{f(\hat{\bf z}\cdot({\bf q}+{\bf
k}))}{2}\left(\frac{\mu}{k}(b^{\tau}_{1}+f(\hat{\bf z}\cdot\hat{\bf
q})^{2})+\frac{(\hat{\bf z}\cdot\hat{\bf
q})}{q}(b^{\tau}_{1}+f\mu^{2})\right)\\\
&=\left\\{\mu^{2}\left(\frac{(b^{\tau}_{1})^{2}f}{2}+\frac{b^{\tau}_{1}f^{2}}{2}+\frac{f^{3}}{5}\right)+\frac{(b^{\tau}_{1})^{2}f}{6}+\frac{b^{\tau}_{1}f^{2}}{10}\right\\}\sigma_{\Lambda}^{2}\,.\end{split}$
(B.6)

So far our analysis has been exact. In order to compute the UV contribution
from the $G_{2}$ terms we need to use the high-$q$ expansion.:

$\begin{split}&\int_{\bf q}P_{\text{lin}}(q)(b^{\tau}_{1}+f(\hat{\bf
z}\cdot\hat{\bf q})^{2})f(\hat{\bf z}\cdot{\bf
q})^{2}\Bigg{[}\frac{3}{7}+\frac{4}{7}(\hat{\bf k}\cdot\hat{\bf
q})^{2}\Bigg{]}=\\\
&\left\\{\frac{1}{735}f(133b^{\tau}_{1}+75f)+\frac{8}{735}f\mu^{2}(7b^{\tau}_{1}+6f)\right\\}\sigma_{\Lambda}^{2}~{}\,.\end{split}$
(B.7)

Combinging these terms together we find

$\delta
P_{13}=(b_{1}-b_{\eta}f\mu^{2})P_{\text{lin}}(k)[A_{0}+\mu^{2}A_{2}]\sigma^{2}_{\Lambda}\,,$
(B.8)

where

$\begin{split}&A_{0}=-\frac{1}{735}\tau_{0}^{2}\Bigg{(}-2380(b^{\tau}_{1})^{2}-1470b^{\tau}_{1}b^{\tau}_{2}+1960b^{\tau}_{1}b_{\mathcal{G}_{2}}^{\tau}-1288b^{\tau}_{1}f-490b^{\tau}_{2}f+784b_{\mathcal{G}_{2}}^{\tau}f\\\
&-300f^{2}+147b^{\tau}_{1}f^{2}(2-\tau_{0})+490(b^{\tau}_{1})^{2}f(1-\tau_{0})+735(b^{\tau}_{1})^{3}\tau_{0}\Bigg{)}\,,\\\
&A_{2}=-\frac{1}{735}\Bigg{(}-336b^{\tau}_{1}f-392b^{\tau}_{\mathcal{G}_{2}}f+3f^{2}(-64+49f(-4+\tau_{0}))\\\
&+490b^{\tau}_{1}f^{2}(-3+\tau_{0})+735(b^{\tau}_{1})^{2}f(-2+\tau_{0})\Bigg{)}\tau_{0}^{2}\,.\end{split}$
(B.9)

##  Appendix C Fits to the simulations for z=3.2

Figure 6: Triangle plot and marginalized projections for bias parameters of the EFT model for the Lyman alpha flux power spectrum of the Sherwood simulations at $z=3.2$. We show results for four choices of $k_{\rm max}$: 3, 4, 5 and 6 $h\text{Mpc}^{-1}$ (gray, green, blue, red, respectively). Param | best-fit | mean$\pm\sigma$ | 95% lower | 95% upper
---|---|---|---|---
$b_{1}$ | $-0.304$ | $-0.3045_{-0.016}^{+0.015}$ | $-0.3358$ | $-0.2727$
$b_{\eta}$ | $0.2076$ | $0.203_{-0.034}^{+0.029}$ | $0.1388$ | $0.2688$
$b_{2}$ | $-0.5508$ | $-0.4961_{-0.34}^{+0.36}$ | $-1.209$ | $0.1886$
$b_{\mathcal{G}_{2}}$ | $-0.2443$ | $-0.2505_{-0.3}^{+0.33}$ | $-0.877$ | $0.3608$
$b_{\eta^{2}}$ | $1.093$ | $1.043_{-0.12}^{+0.16}$ | $0.7617$ | $1.344$
$b_{\delta\eta}$ | $-1.501$ | $-1.37_{-0.28}^{+0.26}$ | $-1.909$ | $-0.8206$
$b_{(KK)_{\parallel}}$ | $-0.6474$ | $-0.6203_{-0.75}^{+0.83}$ | $-2.196$ | $0.9142$
$b_{\Pi^{[2]}_{\parallel}}$ | $-0.9004$ | $-0.8074_{-0.23}^{+0.22}$ | $-1.251$ | $-0.3656$
$10^{3}c_{0}/[h^{-1}\text{Mpc}]^{2}$ | $7.31$ | $7.31_{-0.90}^{+0.90}$ | $5.53$ | $9.09$
$10^{3}c_{2}/[h^{-1}\text{Mpc}]^{2}$ | $4.24$ | $4.24_{-0.57}^{+0.57}$ | $3.10$ | $5.38$
$10^{3}c_{4}/[h^{-1}\text{Mpc}]^{2}$ | $-12.2$ | $-12.2_{-0.54}^{+0.54}$ | $-13.4$ | $-11.1$
$b_{\Pi^{[3]}_{\parallel}}$ | $2.41$ | $2.41_{-0.09}^{+0.09}$ | $2.22$ | $2.58$
$b_{\delta\Pi^{[2]}_{\parallel}}$ | $-5.03$ | $-5.03_{-0.1}^{+0.1}$ | $-5.23$ | $-4.83$
$b_{(K\Pi^{[2]})_{\parallel}}$ | $-2.56$ | $-2.56_{-0.14}^{+0.14}$ | $-2.84$ | $-2.29$
$b_{\eta\Pi^{[2]}_{\parallel}}$ | $0.69$ | $0.69_{-0.19}^{+0.19}$ | $0.31$ | $1.07$
Table 2: Same as Table 1, but for $z=3.2$ and $k_{\rm
max}=5~{}h\text{Mpc}^{-1}$ data.

In this Section we present results for the Sherwood simulation data analysis
at $z=3.2$. This redshift is somewhat higher than $z=2.8$ used in the main
text, so one may expect that the non-linear corrections would be suppressed
and the fit would be better down to smaller $k_{\rm max}$. This is indeed the
case. In Fig. 6 we show the results for four choices of $k_{\rm max}$: 3, 4, 5
and 6 $h\text{Mpc}^{-1}$. We see that the posteriors for $k_{\rm
max}=3,4,5~{}h\text{Mpc}^{-1}$ are fully consistent with each other. The
posteriors for $k_{\rm max}=3,4$ are highly non-Gaussian, and even feature a
multi-island structure for certain parameters. This is likely due to a large
number of parameters in the fit, which the data at a given $k_{\rm max}$
cannot constrain.

The $k_{\rm max}=5~{}h\text{Mpc}^{-1}$ posterior has significant overlap with
those of the $k_{\rm max}=3,4$ cases. In contrast, the $k_{\rm
max}=5~{}h\text{Mpc}^{-1}$ and $k_{\rm max}=6~{}h\text{Mpc}^{-1}$ posteriors
are in noticeable tension with each other in the $b_{\delta\eta}-b_{\eta}$
plane. This suggests that the $k_{\rm max}=6~{}h\text{Mpc}^{-1}$ results are
biased. Thus, we select $k_{\rm max}=5~{}h\text{Mpc}^{-1}$ as a baseline for
$z=3.2$. As anticipates, this is larger than our baseline choice $k_{\rm
max}=3~{}h\text{Mpc}^{-1}$ for $z=2.8$. The 1d marginalized posteriors for
this case are shown in Table 2.

## References

  * [1] L. Hernquist, N. Katz, D. H. Weinberg and J. Miralda-Escude, _The Lyman alpha forest in the cold dark matter model_ , _Astrophys. J. Lett._ 457 (1996) L51 [astro-ph/9509105].
  * [2] J. Miralda-Escude, R.-y. Cen, J. P. Ostriker and M. Rauch, _The Lyman alpha forest from gravitational collapse in the CDM + Lambda Model_ , _Astrophys. J._ 471 (1996) 582 [astro-ph/9511013].
  * [3] R. A. C. Croft, D. H. Weinberg, N. Katz and L. Hernquist, _Intergalactic Helium Absorption in Cold Dark Matter Models_ , _APJ_ 488 (1997) 532 [astro-ph/9611053].
  * [4] R. A. C. Croft, D. H. Weinberg, N. Katz and L. Hernquist, _Recovery of the power spectrum of mass fluctuations from observations of the Lyman alpha forest_ , _Astrophys. J._ 495 (1998) 44 [astro-ph/9708018].
  * [5] P. McDonald, J. Miralda-Escude, M. Rauch, W. L. W. Sargent, T. A. Barlow, R. Cen et al., _The Observed probability distribution function, power spectrum, and correlation function of the transmitted flux in the Lyman-alpha forest_ , _Astrophys. J._ 543 (2000) 1 [astro-ph/9911196].
  * [6] R. A. C. Croft, D. H. Weinberg, M. Bolte, S. Burles, L. Hernquist, N. Katz et al., _Towards a precise measurement of matter clustering: Lyman alpha forest data at redshifts 2-4_ , _Astrophys. J._ 581 (2002) 20 [astro-ph/0012324].
  * [7] M. Zaldarriaga, U. Seljak and L. Hui, _Correlations across scales in the Lyman alpha forest: Testing the gravitational instability paradigm_ , _Astrophys. J._ 551 (2001) 48 [astro-ph/0007101].
  * [8] M. Zaldarriaga, L. Hui and M. Tegmark, _Constraints from the Lyman alpha forest power spectrum_ , _Astrophys. J._ 557 (2001) 519 [astro-ph/0011559].
  * [9] M. Zaldarriaga, R. Scoccimarro and L. Hui, _Inferring the linear power spectrum from the Lyman-alpha forest_ , _Astrophys. J._ 590 (2003) 1 [astro-ph/0111230].
  * [10] SDSS collaboration, P. McDonald et al., _The Linear theory power spectrum from the Lyman-alpha forest in the Sloan Digital Sky Survey_ , _Astrophys. J._ 635 (2005) 761 [astro-ph/0407377].
  * [11] SDSS collaboration, P. McDonald et al., _The Lyman-alpha forest power spectrum from the Sloan Digital Sky Survey_ , _Astrophys. J. Suppl._ 163 (2006) 80 [astro-ph/0405013].
  * [12] U. Seljak, A. Slosar and P. McDonald, _Cosmological parameters from combining the Lyman-alpha forest with CMB, galaxy clustering and SN constraints_ , _JCAP_ 10 (2006) 014 [astro-ph/0604335].
  * [13] A. Slosar, A. Font-Ribera, M. M. Pieri, J. Rich, J.-M. Le Goff, É. Aubourg et al., _The Lyman- $\alpha$ forest in three dimensions: measurements of large scale flux correlations from BOSS 1st-year data_, _JCAP_ 2011 (2011) .
  * [14] V. Iršič et al., _The Lyman $\alpha$ forest power spectrum from the XQ-100 Legacy Survey_, _Mon. Not. Roy. Astron. Soc._ 466 (2017) 4332 [1702.01761].
  * [15] S. Chabanier et al., _The one-dimensional power spectrum from the SDSS DR14 Ly $\alpha$ forests_, _JCAP_ 07 (2019) 017 [1812.03554].
  * [16] S. Bird, M. Viel and M. G. Haehnelt, _Massive neutrinos and the non-linear matter power spectrum_ , _MNRAS_ 420 (2012) 2551 [1109.4416].
  * [17] N. Palanque-Delabrouille et al., _Constraint on neutrino masses from SDSS-III/BOSS Ly $\alpha$ forest and other cosmological probes_, _JCAP_ 02 (2015) 045 [1410.7244].
  * [18] C. Pedersen, A. Font-Ribera, T. D. Kitching, P. McDonald, S. Bird, A. Slosar et al., _Massive neutrinos and degeneracies in Lyman-alpha forest simulations_ , _JCAP_ 04 (2020) 025 [1911.09596].
  * [19] M. Garny, T. Konstandin, L. Sagunski and M. Viel, _Neutrino mass bounds from confronting an effective model with BOSS Lyman- $\alpha$ data_, _JCAP_ 03 (2021) 049 [2011.03050].
  * [20] N. Afshordi, P. McDonald and D. N. Spergel, _Primordial black holes as dark matter: The Power spectrum and evaporation of early structures_ , _Astrophys. J. Lett._ 594 (2003) L71 [astro-ph/0302035].
  * [21] C. Dvorkin, K. Blum and M. Kamionkowski, _Constraining Dark Matter-Baryon Scattering with Linear Cosmology_ , _Phys. Rev. D_ 89 (2014) 023519 [1311.2937].
  * [22] M. Garny, T. Konstandin, L. Sagunski and S. Tulin, _Lyman- $\alpha$ forest constraints on interacting dark sectors_, _JCAP_ 09 (2018) 011 [1805.12203].
  * [23] L. Fuß and M. Garny, _Decaying Dark Matter and Lyman- $\alpha$ forest constraints_, 2210.06117.
  * [24] S. Goldstein, J. C. Hill, V. Iršič and B. D. Sherwin, _Canonical Hubble-Tension-Resolving Early Dark Energy Cosmologies are Inconsistent with the Lyman- $\alpha$ Forest_, 2303.00746.
  * [25] M. Viel, J. Lesgourgues, M. G. Haehnelt, S. Matarrese and A. Riotto, _Constraining warm dark matter candidates including sterile neutrinos and light gravitinos with WMAP and the Lyman-alpha forest_ , _Phys. Rev. D_ 71 (2005) 063534 [astro-ph/0501562].
  * [26] U. Seljak, A. Makarov, P. McDonald and H. Trac, _Can sterile neutrinos be the dark matter?_ , _Phys. Rev. Lett._ 97 (2006) 191303 [astro-ph/0602430].
  * [27] A. Boyarsky, J. Lesgourgues, O. Ruchayskiy and M. Viel, _Lyman-alpha constraints on warm and on warm-plus-cold dark matter models_ , _JCAP_ 05 (2009) 012 [0812.0010].
  * [28] M. Viel, G. D. Becker, J. S. Bolton and M. G. Haehnelt, _Warm dark matter as a solution to the small scale crisis: New constraints from high redshift Lyman- $\alpha$ forest data_, _Phys. Rev. D_ 88 (2013) 043502 [1306.2314].
  * [29] J. Baur, N. Palanque-Delabrouille, C. Yèche, C. Magneville and M. Viel, _Lyman-alpha Forests cool Warm Dark Matter_ , _JCAP_ 08 (2016) 012 [1512.01981].
  * [30] A. Garzilli, A. Boyarsky and O. Ruchayskiy, _Cutoff in the Lyman \alpha forest power spectrum: warm IGM or warm dark matter?_ , _Phys. Lett. B_ 773 (2017) 258 [1510.07006].
  * [31] J. Baur, N. Palanque-Delabrouille, C. Yeche, A. Boyarsky, O. Ruchayskiy, E. Armengaud et al., _Constraints from Ly- $\alpha$ forests on non-thermal dark matter including resonantly-produced sterile neutrinos_, _JCAP_ 12 (2017) 013 [1706.03118].
  * [32] A. Boyarsky, M. Drewes, T. Lasserre, S. Mertens and O. Ruchayskiy, _Sterile neutrino Dark Matter_ , _Prog. Part. Nucl. Phys._ 104 (2019) 1 [1807.07938].
  * [33] N. Palanque-Delabrouille, C. Yèche, N. Schöneberg, J. Lesgourgues, M. Walther, S. Chabanier et al., _Hints, neutrino bounds and WDM constraints from SDSS DR14 Lyman- $\alpha$ and Planck full-survey data_, _JCAP_ 04 (2020) 038 [1911.09073].
  * [34] A. Garzilli, A. Magalich, O. Ruchayskiy and A. Boyarsky, _How to constrain warm dark matter with the Lyman- $\alpha$ forest_, _Mon. Not. Roy. Astron. Soc._ 502 (2021) 2356 [1912.09397].
  * [35] L. Hui, J. P. Ostriker, S. Tremaine and E. Witten, _Ultralight scalars as cosmological dark matter_ , _Phys. Rev. D_ 95 (2017) 043541 [1610.08297].
  * [36] V. Iršič, M. Viel, M. G. Haehnelt, J. S. Bolton and G. D. Becker, _First constraints on fuzzy dark matter from Lyman- $\alpha$ forest data and hydrodynamical simulations_, _Phys. Rev. Lett._ 119 (2017) 031302 [1703.04683].
  * [37] K. K. Rogers and H. V. Peiris, _Strong Bound on Canonical Ultralight Axion Dark Matter from the Lyman-Alpha Forest_ , _Phys. Rev. Lett._ 126 (2021) 071302 [2007.12705].
  * [38] A. Slosar et al., _Measurement of Baryon Acoustic Oscillations in the Lyman-alpha Forest Fluctuations in BOSS Data Release 9_ , _JCAP_ 04 (2013) 026 [1301.3459].
  * [39] BOSS collaboration, A. Font-Ribera et al., _Quasar-Lyman $\alpha$ Forest Cross-Correlation from BOSS DR11 : Baryon Acoustic Oscillations_, _JCAP_ 05 (2014) 027 [1311.1767].
  * [40] E. Aubourg et al., _Cosmological implications of baryon acoustic oscillation measurements_ , _Phys. Rev._ D92 (2015) 123516 [1411.1074].
  * [41] H. du Mas des Bourboux et al., _The Completed SDSS-IV Extended Baryon Oscillation Spectroscopic Survey: Baryon Acoustic Oscillations with Ly $\alpha$ Forests_, _Astrophys. J._ 901 (2020) 153 [2007.08995].
  * [42] A. Slosar et al., _The Lyman-alpha forest in three dimensions: measurements of large scale flux correlations from BOSS 1st-year data_ , _JCAP_ 09 (2011) 001 [1104.5244].
  * [43] A. Cuceu, A. Font-Ribera, B. Joachimi and S. Nadathur, _Cosmology beyond BAO from the 3D distribution of the Lyman- $\alpha$ forest_, _Mon. Not. Roy. Astron. Soc._ 506 (2021) 5439 [2103.14075].
  * [44] A. Cuceu, A. Font-Ribera, S. Nadathur, B. Joachimi and P. Martini, _Constraints on the Cosmic Expansion Rate at Redshift 2.3 from the Lyman- $\alpha$ Forest_, _Phys. Rev. Lett._ 130 (2023) 191003 [2209.13942].
  * [45] C. Gordon et al., _3D Correlations in the Lyman- $\alpha$ Forest from Early DESI Data_, 2308.10950.
  * [46] BOSS collaboration, R. A. C. Croft et al., _Large-scale clustering of Lyman $\alpha$ emission intensity from SDSS/BOSS_, _Mon. Not. Roy. Astron. Soc._ 457 (2016) 3541 [1504.04088].
  * [47] DESI collaboration, A. Aghamousa et al., _The DESI Experiment Part I: Science,Targeting, and Survey Design_ , 1611.00036.
  * [48] DESI collaboration, C. Ramirez-Perez et al., _The Lyman- $\alpha$ forest catalog from the Dark Energy Spectroscopic Instrument Early Data Release_, 2306.06312.
  * [49] A. Borde, N. Palanque-Delabrouille, G. Rossi, M. Viel, J. S. Bolton, C. Yèche et al., _New approach for precise computation of Lyman- $\alpha$ forest power spectrum with hydrodynamical simulations_, _JCAP_ 07 (2014) 005 [1401.6472].
  * [50] G. Rossi, N. Palanque-Delabrouille, A. Borde, M. Viel, C. Yeche, J. S. Bolton et al., _Suite of hydrodynamical simulations for the Lyman- $\alpha$ forest with massive neutrinos_, _Astron. Astrophys._ 567 (2014) A79 [1401.6464].
  * [51] S. Bird, M. Vogelsberger, M. Haehnelt, D. Sijacki, S. Genel, P. Torrey et al., _Damped Lyman- $\alpha$ absorbers as a probe of stellar feedback_, _Mon. Not. Roy. Astron. Soc._ 445 (2014) 2313 [1405.3994].
  * [52] J. S. Bolton, E. Puchwein, D. Sijacki, M. G. Haehnelt, T.-S. Kim, A. Meiksin et al., _The Sherwood simulation suite: overview and data comparisons with the Lyman $\alpha$ forest at redshifts 2 $\leq$ z $\leq$ 5_, _Mon. Not. Roy. Astron. Soc._ 464 (2017) 897 [1605.03462].
  * [53] A. Arinyo-i Prats, J. Miralda-Escudé, M. Viel and R. Cen, _The Non-Linear Power Spectrum of the Lyman Alpha Forest_ , _JCAP_ 12 (2015) 017 [1506.04519].
  * [54] L. Anderson, A. Pontzen, A. Font-Ribera, F. Villaescusa-Navarro, K. K. Rogers and S. Genel, _Cosmological Hydrodynamic Simulations with Suppressed Variance in the Ly $\alpha$ Forest Power Spectrum_, _Astrophys. J._ 871 (2019) 144 [1811.00043].
  * [55] A. Smith, R. Kannan, E. Garaldi, M. Vogelsberger, R. Pakmor, V. Springel et al., _The thesan project: Lyman- $\alpha$ emission and transmission during the Epoch of Reionization_, _Mon. Not. Roy. Astron. Soc._ 512 (2022) 3243 [2110.02966].
  * [56] M. T. Tillman et al., _An Exploration of AGN and Stellar Feedback Effects in the Intergalactic Medium via the Low Redshift Lyman- $\alpha$ Forest_, 2307.06360.
  * [57] B. Greig, E. Komatsu and J. S. B. Wyithe, _Cosmology from clustering of Lyman-alpha galaxies: breaking non-gravitational Lyman-alpha radiative transfer degeneracies using the bispectrum_ , _Mon. Not. Roy. Astron. Soc._ 431 (2013) 1777 [1212.0977].
  * [58] J. J. Givans and C. M. Hirata, _Redshift-space streaming velocity effects on the Lyman- $\alpha$ forest baryon acoustic oscillation scale_, _Phys. Rev. D_ 102 (2020) 023515 [2002.12296].
  * [59] S.-F. Chen, Z. Vlah and M. White, _The Ly $\alpha$ forest flux correlation function: a perturbation theory perspective_, _JCAP_ 05 (2021) 053 [2103.13498].
  * [60] J. J. Givans, A. Font-Ribera, A. Slosar, L. Seeyave, C. Pedersen, K. K. Rogers et al., _Non-linearities in the Lyman- $\alpha$ forest and in its cross-correlation with dark matter halos_, _JCAP_ 09 (2022) 070 [2205.00962].
  * [61] Planck collaboration, N. Aghanim et al., _Planck 2018 results. VI. Cosmological parameters_ , 1807.06209.
  * [62] R. Murgia, V. Iršič and M. Viel, _Novel constraints on noncold, nonthermal dark matter from Lyman- $\alpha$ forest data_, _Phys. Rev. D_ 98 (2018) 083540 [1806.08371].
  * [63] D. C. Hooper, N. Schöneberg, R. Murgia, M. Archidiacono, J. Lesgourgues and M. Viel, _One likelihood to bind them all: Lyman- $\alpha$ constraints on non-standard dark matter_, _JCAP_ 10 (2022) 032 [2206.08188].
  * [64] K. K. Rogers, H. V. Peiris, A. Pontzen, S. Bird, L. Verde and A. Font-Ribera, _Bayesian emulator optimisation for cosmology: application to the Lyman-alpha forest_ , _JCAP_ 02 (2019) 031 [1812.04631].
  * [65] S. Bird, K. K. Rogers, H. V. Peiris, L. Verde, A. Font-Ribera and A. Pontzen, _An Emulator for the Lyman-alpha Forest_ , _JCAP_ 02 (2019) 050 [1812.04654].
  * [66] C. Pedersen, A. Font-Ribera, K. K. Rogers, P. McDonald, H. V. Peiris, A. Pontzen et al., _An emulator for the Lyman- $\alpha$ forest in beyond-$\Lambda$CDM cosmologies_, _JCAP_ 05 (2021) 033 [2011.15127].
  * [67] M. M. Ivanov, M. Simonović and M. Zaldarriaga, _Cosmological Parameters from the BOSS Galaxy Power Spectrum_ , _JCAP_ 05 (2020) 042 [1909.05277].
  * [68] G. D’Amico, J. Gleyzes, N. Kokron, D. Markovic, L. Senatore, P. Zhang et al., _The Cosmological Analysis of the SDSS/BOSS data from the Effective Field Theory of Large-Scale Structure_ , 1909.05271.
  * [69] M. M. Ivanov, M. Simonović and M. Zaldarriaga, _Cosmological Parameters and Neutrino Masses from the Final Planck and Full-Shape BOSS Data_ , _Phys. Rev. D_ 101 (2020) 083504 [1912.08208].
  * [70] A. Chudaykin, K. Dolgikh and M. M. Ivanov, _Constraints on the curvature of the Universe and dynamical dark energy from the Full-shape and BAO data_ , _Phys. Rev. D_ 103 (2021) 023507 [2009.10106].
  * [71] M. M. Ivanov, E. McDonough, J. C. Hill, M. Simonović, M. W. Toomey, S. Alexander et al., _Constraining Early Dark Energy with Large-Scale Structure_ , _Phys. Rev. D_ 102 (2020) 103502 [2006.11235].
  * [72] O. H. E. Philcox and M. M. Ivanov, _BOSS DR12 full-shape cosmology: $\Lambda$CDM constraints from the large-scale galaxy power spectrum and bispectrum monopole_, _Phys. Rev. D_ 105 (2022) 043517 [2112.04515].
  * [73] M. M. Ivanov, _Cosmological constraints from the power spectrum of eBOSS emission line galaxies_ , 2106.12580.
  * [74] M. White et al., _Cosmological constraints from the tomographic cross-correlation of DESI Luminous Red Galaxies and Planck CMB lensing_ , _JCAP_ 02 (2022) 007 [2111.09898].
  * [75] S.-F. Chen, Z. Vlah and M. White, _A new analysis of galaxy 2-point functions in the BOSS survey, including full-shape information and post-reconstruction BAO_ , _JCAP_ 02 (2022) 008 [2110.05530].
  * [76] S.-F. Chen, M. White, J. DeRose and N. Kokron, _Cosmological analysis of three-dimensional BOSS galaxy clustering and Planck CMB lensing cross correlations via Lagrangian perturbation theory_ , _JCAP_ 07 (2022) 041 [2204.10392].
  * [77] A. Chudaykin and M. M. Ivanov, _Cosmological constraints from the power spectrum of eBOSS quasars_ , 2210.17044.
  * [78] M. M. Ivanov, O. H. E. Philcox, G. Cabass, T. Nishimichi, M. Simonović and M. Zaldarriaga, _Cosmology with the galaxy bispectrum multipoles: Optimal estimation and application to BOSS data_ , _Phys. Rev. D_ 107 (2023) 083515 [2302.04414].
  * [79] U. Seljak, _Bias, redshift space distortions and primordial nongaussianity of nonlinear transformations: application to Lyman alpha forest_ , _JCAP_ 03 (2012) 004 [1201.0594].
  * [80] A. M. Cieplak and A. Slosar, _Towards physics responsible for large-scale Lyman- $\alpha$ forest bias parameters_, _JCAP_ 03 (2016) 016 [1509.07875].
  * [81] D. Baumann, A. Nicolis, L. Senatore and M. Zaldarriaga, _Cosmological Non-Linearities as an Effective Fluid_ , _JCAP_ 1207 (2012) 051 [1004.2488].
  * [82] J. J. M. Carrasco, M. P. Hertzberg and L. Senatore, _The Effective Field Theory of Cosmological Large Scale Structures_ , _JHEP_ 09 (2012) 082 [1206.2926].
  * [83] V. Desjacques, D. Jeong and F. Schmidt, _Large-Scale Galaxy Bias_ , _Phys. Rept._ 733 (2018) 1 [1611.09787].
  * [84] G. Cabass, M. M. Ivanov, M. Lewandowski, M. Mirbabayi and M. Simonović, _Snowmass White Paper: Effective Field Theories in Cosmology_ , in _2022 Snowmass Summer Study_ , 3, 2022, 2203.08232.
  * [85] M. M. Ivanov, _Effective Field Theory for Large Scale Structure_ , 2212.08488.
  * [86] V. Desjacques, D. Jeong and F. Schmidt, _The Galaxy Power Spectrum and Bispectrum in Redshift Space_ , _JCAP_ 1812 (2018) 035 [1806.04015].
  * [87] C. M. Hirata, _Tidal alignments as a contaminant of redshift space distortions_ , _MNRAS_ 399 (2009) 1074 [0903.4929].
  * [88] F. Bernardeau, S. Colombi, E. Gaztanaga and R. Scoccimarro, _Large scale structure of the universe and cosmological perturbation theory_ , _Phys. Rept._ 367 (2002) 1 [astro-ph/0112551].
  * [89] Z. Zheng, R. Cen, H. Trac and J. Miralda-Escude, _Radiative Transfer Modeling of Lyman Alpha Emitters. II. New Effects in Galaxy Clustering_ , _Astrophys. J._ 726 (2010) 38 [1003.4990].
  * [90] S. Wyithe and M. Dijkstra, _Non-Gravitational Contributions to the Clustering of Ly-alpha Selected Galaxies: Implications for Cosmological Surveys_ , _Mon. Not. Roy. Astron. Soc._ 415 (2011) 3929 [1104.0712].
  * [91] D. Martens, C. M. Hirata, A. J. Ross and X. Fang, _A Radial Measurement of the Galaxy Tidal Alignment Magnitude with BOSS Data_ , _Mon. Not. Roy. Astron. Soc._ 478 (2018) 711 [1802.07708].
  * [92] A. Obuljen, W. J. Percival and N. Dalal, _Detection of anisotropic galaxy assembly bias in BOSS DR12_ , _JCAP_ 10 (2020) 058 [2004.07240].
  * [93] M. Simonovic, T. Baldauf, M. Zaldarriaga, J. J. Carrasco and J. A. Kollmeier, _Cosmological perturbation theory using the FFTLog: formalism and connection to QFT loop integrals_ , _JCAP_ 1804 (2018) 030 [1708.08130].
  * [94] A. Chudaykin, M. M. Ivanov, O. H. E. Philcox and M. Simonović, _Nonlinear perturbation theory extension of the Boltzmann code CLASS_ , _Phys. Rev. D_ 102 (2020) 063533 [2004.10607].
  * [95] M. Rauch, _The lyman alpha forest in the spectra of quasistellar objects_ , _Ann. Rev. Astron. Astrophys._ 36 (1998) 267 [astro-ph/9806286].
  * [96] A. Loeb and S. R. Furlanetto, _The First Galaxies in the Universe_. 2013\.
  * [97] D. Blas, M. M. Ivanov and S. Sibiryakov, _Testing Lorentz invariance of dark matter_ , _JCAP_ 10 (2012) 057 [1209.0464].
  * [98] B. Audren, D. Blas, M. M. Ivanov, J. Lesgourgues and S. Sibiryakov, _Cosmological constraints on deviations from Lorentz invariance in gravity and dark matter_ , _JCAP_ 03 (2015) 016 [1410.6514].
  * [99] J. E. Gunn and B. A. Peterson, _On the Density of Neutral Hydrogen in Intergalactic Space._ , _APJ_ 142 (1965) 1633.
  * [100] eBOSS collaboration, S. Alam et al., _Completed SDSS-IV extended Baryon Oscillation Spectroscopic Survey: Cosmological implications from two decades of spectroscopic surveys at the Apache Point Observatory_ , _Phys. Rev. D_ 103 (2021) 083533 [2007.08991].
  * [101] N. Y. Gnedin and L. Hui, _Probing the universe with the Lyman alpha forest: 1. Hydrodynamics of the low density IGM_ , _Mon. Not. Roy. Astron. Soc._ 296 (1998) 44 [astro-ph/9706219].
  * [102] L. Hui and N. Y. Gnedin, _Equation of state of the photoionized intergalactic medium_ , _Mon. Not. Roy. Astron. Soc._ 292 (1997) 27 [astro-ph/9612232].
  * [103] T. Theuns, J. Schaye and M. Haehnelt, _Broadening of QSO Lyman-alpha forest absorbers_ , _Mon. Not. Roy. Astron. Soc._ 315 (2000) 600 [astro-ph/9908288].
  * [104] A. Pontzen, _Scale-dependent bias in the baryonic-acoustic-oscillation-scale intergalactic neutral hydrogen_ , _Phys. Rev. D_ 89 (2014) 083010 [1402.0506].
  * [105] S. Gontcho A Gontcho, J. Miralda-Escudé and N. G. Busca, _On the effect of the ionizing background on the Ly $\alpha$ forest autocorrelation function_, _Mon. Not. Roy. Astron. Soc._ 442 (2014) 187 [1404.7425].
  * [106] A. D’Aloisio, M. McQuinn and H. Trac, _Large Opacity Variations in the High-redshift Ly $\alpha$ Forest: the Signature of Relic Temperature Fluctuations From Patchy Reionization_, _Astrophys. J. Lett._ 813 (2015) L38 [1509.02523].
  * [107] P. Montero-Camacho, C. M. Hirata, P. Martini and K. Honscheid, _Impact of inhomogeneous reionization on the Lyman- $\alpha$ forest_, _Mon. Not. Roy. Astron. Soc._ 487 (2019) 1047 [1902.02892].
  * [108] G. Cabass and F. Schmidt, _A new scale in the bias expansion_ , _JCAP_ 05 (2019) 031 [1812.02731].
  * [109] E. Pajer and M. Zaldarriaga, _On the Renormalization of the Effective Field Theory of Large Scale Structures_ , _JCAP_ 08 (2013) 037 [1301.7182].
  * [110] D. Blas, M. Garny and T. Konstandin, _Cosmological perturbation theory at three-loop order_ , _JCAP_ 1401 (2014) 010 [1309.3308].
  * [111] T. Konstandin, R. A. Porto and H. Rubira, _The effective field theory of large scale structure at three loops_ , _JCAP_ 11 (2019) 027 [1906.00997].
  * [112] S. Pueblas and R. Scoccimarro, _Generation of Vorticity and Velocity Dispersion by Orbit Crossing_ , _Phys. Rev._ D80 (2009) 043504 [0809.4606].
  * [113] M. Schmittfull, M. Simonović, M. M. Ivanov, O. H. E. Philcox and M. Zaldarriaga, _Modeling Galaxies in Redshift Space at the Field Level_ , _JCAP_ 05 (2021) 059 [2012.03334].
  * [114] G. Cabass, M. M. Ivanov, O. H. E. Philcox, M. Simonović and M. Zaldarriaga, _Constraints on Single-Field Inflation from the BOSS Galaxy Survey_ , 2201.07238.
  * [115] G. Cabass, M. M. Ivanov, O. H. E. Philcox, M. Simonović and M. Zaldarriaga, _Constraints on Multi-Field Inflation from the BOSS Galaxy Survey_ , 2204.01781.
  * [116] L. Senatore, _Bias in the Effective Field Theory of Large Scale Structures_ , _JCAP_ 1511 (2015) 007 [1406.7843].
  * [117] V. Assassi, D. Baumann, D. Green and M. Zaldarriaga, _Renormalized Halo Bias_ , _JCAP_ 1408 (2014) 056 [1402.5916].
  * [118] M. Mirbabayi, F. Schmidt and M. Zaldarriaga, _Biased Tracers and Time Evolution_ , _JCAP_ 1507 (2015) 030 [1412.5169].
  * [119] T. Lazeyras and F. Schmidt, _Beyond LIMD bias: a measurement of the complete set of third-order halo bias parameters_ , _JCAP_ 1809 (2018) 008 [1712.07531].
  * [120] L. Senatore and M. Zaldarriaga, _Redshift Space Distortions in the Effective Field Theory of Large Scale Structures_ , 1409.1225.
  * [121] M. Lewandowski, L. Senatore, F. Prada, C. Zhao and C.-H. Chuang, _EFT of large scale structures in redshift space_ , _Phys. Rev. D_ 97 (2018) 063526 [1512.06831].
  * [122] N. Kaiser, _Clustering in real space and in redshift space_ , _Mon. Not. Roy. Astron. Soc._ 227 (1987) 1.
  * [123] J. J. M. Carrasco, S. Foreman, D. Green and L. Senatore, _The 2-loop matter power spectrum and the IR-safe integrand_ , _JCAP_ 07 (2014) 056 [1304.4946].
  * [124] A. Perko, L. Senatore, E. Jennings and R. H. Wechsler, _Biased Tracers in Redshift Space in the EFT of Large-Scale Structure_ , 1610.09321.
  * [125] L. Senatore and M. Zaldarriaga, _The IR-resummed Effective Field Theory of Large Scale Structures_ , _JCAP_ 1502 (2015) 013 [1404.5954].
  * [126] Z. Vlah, U. Seljak, M. Y. Chu and Y. Feng, _Perturbation theory, effective field theory, and oscillations in the power spectrum_ , _JCAP_ 1603 (2016) 057 [1509.02120].
  * [127] T. Baldauf, M. Mirbabayi, M. Simonović and M. Zaldarriaga, _Equivalence Principle and the Baryon Acoustic Peak_ , _Phys. Rev._ D92 (2015) 043514 [1504.04366].
  * [128] D. Blas, M. Garny, M. M. Ivanov and S. Sibiryakov, _Time-Sliced Perturbation Theory for Large Scale Structure I: General Formalism_ , _JCAP_ 1607 (2016) 052 [1512.05807].
  * [129] D. Blas, M. Garny, M. M. Ivanov and S. Sibiryakov, _Time-Sliced Perturbation Theory II: Baryon Acoustic Oscillations and Infrared Resummation_ , _JCAP_ 1607 (2016) 028 [1605.02149].
  * [130] M. M. Ivanov and S. Sibiryakov, _Infrared Resummation for Biased Tracers in Redshift Space_ , _JCAP_ 1807 (2018) 053 [1804.05080].
  * [131] A. Vasudevan, M. M. Ivanov, S. Sibiryakov and J. Lesgourgues, _Time-sliced perturbation theory with primordial non-Gaussianity and effects of large bulk flows on inflationary oscillating features_ , _JCAP_ 09 (2019) 037 [1906.08697].
  * [132] J. E. McEwen, X. Fang, C. M. Hirata and J. A. Blazek, _FAST-PT: a novel algorithm to calculate convolution integrals in cosmological perturbation theory_ , _JCAP_ 1609 (2016) 015 [1603.04826].
  * [133] X. Fang, J. A. Blazek, J. E. McEwen and C. M. Hirata, _FAST-PT II: an algorithm to calculate convolution integrals of general tensor quantities in cosmological perturbation theory_ , _JCAP_ 1702 (2017) 030 [1609.05978].
  * [134] S.-F. Chen, Z. Vlah and M. White, _Consistent Modeling of Velocity Statistics and Redshift-Space Distortions in One-Loop Perturbation Theory_ , _JCAP_ 07 (2020) 062 [2005.00523].
  * [135] D. Blas, M. Garny and T. Konstandin, _On the non-linear scale of cosmological perturbation theory_ , _JCAP_ 09 (2013) 024 [1304.1546].
  * [136] D. Wadekar and R. Scoccimarro, _The Galaxy Power Spectrum Multipoles Covariance in Perturbation Theory_ , 1910.02914.
  * [137] D. Wadekar, M. M. Ivanov and R. Scoccimarro, _Cosmological constraints from BOSS with analytic covariance matrices_ , _Phys. Rev. D_ 102 (2020) 123521 [2009.00622].
  * [138] O. H. E. Philcox, M. M. Ivanov, M. Zaldarriaga, M. Simonovic and M. Schmittfull, _Fewer Mocks and Less Noise: Reducing the Dimensionality of Cosmological Observables with Subspace Projections_ , _Phys. Rev. D_ 103 (2021) 043508 [2009.03311].
  * [139] T. Baldauf, M. Mirbabayi, M. Simonović and M. Zaldarriaga, _LSS constraints with controlled theoretical uncertainties_ , 1602.00674.
  * [140] A. Chudaykin and M. M. Ivanov, _Measuring neutrino masses with large-scale structure: Euclid forecast with controlled theoretical error_ , _JCAP_ 11 (2019) 034 [1907.06666].
  * [141] A. Chudaykin, M. M. Ivanov and M. Simonović, _Optimizing large-scale structure data analysis with the theoretical error likelihood_ , _Phys. Rev. D_ 103 (2021) 043525 [2009.10724].
  * [142] M. M. Ivanov, O. H. E. Philcox, T. Nishimichi, M. Simonović, M. Takada and M. Zaldarriaga, _Precision analysis of the redshift-space galaxy bispectrum_ , _Phys. Rev. D_ 105 (2022) 063512 [2110.10161].
  * [143] T. Nishimichi, G. D’Amico, M. M. Ivanov, L. Senatore, M. Simonović, M. Takada et al., _Blinded challenge for precision cosmology with large-scale structure: results from effective field theory for the redshift-space galaxy power spectrum_ , _Phys. Rev. D_ 102 (2020) 123541 [2003.08277].
  * [144] M. M. Ivanov, O. H. E. Philcox, M. Simonović, M. Zaldarriaga, T. Nischimichi and M. Takada, _Cosmological constraints without nonlinear redshift-space distortions_ , _Phys. Rev. D_ 105 (2022) 043531 [2110.00006].
  * [145] B. Audren, J. Lesgourgues, K. Benabed and S. Prunet, _Conservative Constraints on Early Cosmology: an illustration of the Monte Python cosmological parameter inference code_ , _JCAP_ 1302 (2013) 001 [1210.7183].
  * [146] T. Brinckmann and J. Lesgourgues, _MontePython 3: boosted MCMC sampler and other features_ , _Phys. Dark Univ._ 24 (2019) 100260 [1804.07261].
  * [147] A. Lewis, _GetDist: a Python package for analysing Monte Carlo samples_ , 1910.13970.
  * [148] L. Mercolli and E. Pajer, _On the velocity in the Effective Field Theory of Large Scale Structures_ , _JCAP_ 03 (2014) 006 [1307.3220].
  * [149] S. L. Lumsden, A. F. Heavens and J. A. Peacock, _The clustering of peaks in a random Gaussian field_ , _MNRAS_ 238 (1989) 293.
  * [150] A. Barreira, T. Lazeyras and F. Schmidt, _Galaxy bias from forward models: linear and second-order bias of IllustrisTNG galaxies_ , 2105.02876.
  * [151] T. Lazeyras, A. Barreira and F. Schmidt, _Assembly bias in quadratic bias parameters of dark matter halos from forward modeling_ , _JCAP_ 10 (2021) 063 [2106.14713].
  * [152] G. Cabass, M. M. Ivanov, O. H. E. Philcox, M. Simonovic and M. Zaldarriaga, _Constraining Single-Field Inflation with MegaMapper_ , 2211.14899.
  * [153] O. H. E. Philcox, M. M. Ivanov, G. Cabass, M. Simonović, M. Zaldarriaga and T. Nishimichi, _Cosmology with the redshift-space galaxy bispectrum monopole at one-loop order_ , _Phys. Rev. D_ 106 (2022) 043530 [2206.02800].
  * [154] G. D’Amico, Y. Donath, M. Lewandowski, L. Senatore and P. Zhang, _The one-loop bispectrum of galaxies in redshift space from the Effective Field Theory of Large-Scale Structure_ , 2211.17130.
  * [155] S. S. Tie, D. H. Weinberg, P. Martini, W. Zhu, S. Peirani, T. Suarez et al., _UV background fluctuations and three-point correlations in the large-scale clustering of the Lyman $\alpha$ forest_, _Mon. Not. Roy. Astron. Soc._ 487 (2019) 5346 [1905.02208].
  * [156] T.-T. Fang and M. J. White, _Probing the statistics of the temperature - density relation of the IGM_ , _Astrophys. J. Lett._ 606 (2004) L9 [astro-ph/0312280].
  * [157] A. Font-Ribera et al., _The large-scale Quasar-Lyman \alpha\ Forest Cross-Correlation from BOSS_ , _JCAP_ 05 (2013) 018 [1303.1937].
  * [158] R. A. Porto, L. Senatore and M. Zaldarriaga, _The Lagrangian-space Effective Field Theory of Large Scale Structures_ , _JCAP_ 1405 (2014) 022 [1311.2168].
  * [159] Z. Vlah, M. White and A. Aviles, _A Lagrangian effective field theory_ , _JCAP_ 09 (2015) 014 [1506.05264].
  * [160] Z. Vlah, E. Castorina and M. White, _The Gaussian streaming model and convolution Lagrangian effective field theory_ , _JCAP_ 12 (2016) 007 [1609.02908].
  * [161] Z. Vlah and M. White, _Exploring redshift-space distortions in large-scale structure_ , _JCAP_ 1903 (2019) 007 [1812.02775].
  * [162] S.-F. Chen, Z. Vlah, E. Castorina and M. White, _Redshift-Space Distortions in Lagrangian Perturbation Theory_ , _JCAP_ 03 (2021) 100 [2012.04636].


# A COMPARATIVE ANALYSIS OF LATENT REGRESSOR LOSSES FOR SINGING VOICE
CONVERSION

###### Abstract

Previous research has shown that established techniques for spoken voice
conversion (VC) do not perform as well when applied to singing voice
conversion (SVC). We propose an alternative loss component in a loss function
that is otherwise well-established among VC tasks, which has been shown to
improve our model’s SVC performance. We first trained a singer identity
embedding (SIE) network on mel-spectrograms of singer recordings to produce
singer-specific variance encodings using contrastive learning. We subsequently
trained a well-known autoencoder framework (AutoVC) conditioned on these SIEs,
and measured differences in SVC performance when using different latent
regressor loss components. We found that using this loss w.r.t. SIEs leads to
better performance than w.r.t. bottleneck embeddings, where converted audio is
more natural and specific towards target singers. The inclusion of this loss
component has the advantage of explicitly forcing the network to reconstruct
with timbral similarity, and also negates the effect of poor disentanglement
in AutoVC’s bottleneck embeddings. We demonstrate peculiar diversity between
computational and human evaluations on singer-converted audio clips, which
highlights the necessity of both. We also propose a pitch-matching mechanism
between source and target singers to ensure these evaluations are not
influenced by differences in pitch register.

##  1 Introduction

As domains like speech and computer vision produce more convincing and novel
probabilistic machine learning-based transformations, the inevitable interest
of the entertainment industry has propelled further research in more artistic-
relevant domains such as singing. The ability to switch between multiple
singers without the target singer needing to re-record songs will be
revolutionary to the music industry in the amount of time and money it saves,
not to mention the artistic avenues this opens for up for composers and music
producers. This is made possible by neural networks that can achieve singing
voice conversion (SVC), which is the task of changing the perceived identity
of a singer in an audio recording. The majority of voice conversion research
has focused on the speech domain, while the singing domain has only gained
attention in recent years.

##  2 Approaches to Voice Conversion

To achieve voice conversion, a typical system comprises an encoder that
produces vocalist-independent embeddings, usually representing linguistic
content from input data. A decoder then combines these embeddings with the
speaker-specific timbre representations on which it was conditioned to
resynthesise the data in its voice-converted state.

The conditioning factor of speaker timbre can be represented in a one-hot
encoding format. However, this is inherently only able to train a network for
conversions to the finite number of vocalists seen in the training data [1, 2,
3]. More recent research is focused on zero-shot conversions, where systems
are able to take unseen examples as both the source and target signal. To
achieve this, the one-hot conditioning vectors are replaced with embeddings
containing variances that are vocalist-specific, without explicitly
representing vocalist labels [4, 5, 6]. The use of these voice identity
embeddings allows networks to train as universal background models, where they
are able to take a variety of more generalised voice recordings and learn from
these before being subjected to more particular downstream tasks [7, 8, 9].
Other conditioning factors to disentangle timbre include loudness [1, 7],
phonetic (usually from pretrained linguistic networks) [1, 7, 10, 11, 12, 13],
and pitch-related features either as one-hot encodings or continuous data [14,
7, 13, 12, 6, 15].

Voice identity embeddings are often averaged across multiple speaker
utterances in order to get a generalised representation of how a given speaker
sounds in different conditions. [6] discuss how this issue forces the voice
conversion models to rely on a finite number of static embeddings which
assists it to perform better voice conversion, but generalises more poorly to
unseen target embeddings. They demonstrated higher performance when using an
additional network, which takes F0 information, live-generated and speaker-
averaged embeddings to produce a new adjusted embedding. [13] proposed a U-net
to model with instance normalisation modules [16] after each downsampling
block to produce a hierarchical speaker embedding at multiple granularities.

Conditioning with voice identity embeddings does not however, imply perfect
disentanglement from the linguistic information. Methods towards improving
disentanglement in these systems include fine-tuning the size of the
bottleneck to have a capacity capable of retaining only linguistic content [4,
6], or applying a vocalist-classifier to the bottleneck and using its negative
loss as a regularisation component in the autoencoder loss function (which
have also been used on converted data for training and evaluation) [3].

However even with these considerations in place, autoencoders have still been
known to produce blurry spectra. Appending either post-processing nets [4, 17,
18, 19], or the generator of a trained GAN [3] to predict a spectrogram
representing the residual missing detail has been shown to improve outputs and
produce spectrograms of higher definition.

Variational autoencoders (VAEs) alone have been used for voice conversion [2],
while VAEs combined GAN frameworks during the training phase have been shown
to perform timbre conversion between voice and other instruments [20]. GANs
have also been applied for voice conversion where the generator is a fully
convolutional network [21], VAE [22, 23] encoder-decoder framework with cycle-
consistency [24], or U-net [25] framework.

Audio generative networks often rely on a pixel-wise comparison for
reconstruction loss. While this type of loss has been successfully and
consistently implemented across this field of research, other more
perceptually abstract feature comparisons can also be used as loss functions
to provide higher level characteristic comparisons between original and
synthesised data [26]. These are often generated by specific feature
extractors such as encoders for pitch, phonemes, singer identity embeddings,
as well as feature-matching between intermittent layers’ activations in
generative networks [27, 28, 29].

When feature-matching is implemented between original and reconstructed data
in the latent space, this can be referred to as the latent regressor loss or
cycle-consistency [4, 30, 7, 31, 32]. In the majority of literature, cycle-
consistency more often refers to the use of back-translation [33] in
generative networks to complete a two-step conversion cycle of source to
target to source [34, 35, 24, 32]. As the implementation in [4] describes
something more similar to a latent regressor loss[36], we will herein use this
terminology. Latent loss provides the additional intuition that generative
networks must rely on the relevant latent space embeddings in addition to
other inputs. Loss functions that penalise silences and attribute heavier
weightings towards spectral peaks are also used, as well as multi-resolution
losses [12]. Latent losses w.r.t. autoencoder bottleneck [4, 30, 6] and
speaker [37, 38] embeddings have been used in speaker conversion models of
varying frameworks. Recent work [39] demonstrated that training without a
bottleneck latent loss accelerated model convergence for the task of singing
technique conversion, producing audio with more detail and less audible
artefacts.

##  3 Experiments

###  3.1 Singing Voice Conversion System

Due to its influence on subsequent voice conversion research [6, 14, 40, 41,
7, 14], we have adopted the AutoVC architecture and explored adaptations to
its loss functions. A flowchart of this system and these adaptations is
provided for reference in Fig. 1. It consists of an autoencoder, where the
encoder and decoder are conditioned on embeddings that represent variances
specific to a given singer. These singer identity embeddings (SIEs) come from
a pretrained network and can be conceptually summarised as representing
singer-specific timbral qualities. For convenience, we will refer to this
autoencoder section and the pretrained encoder as the SVC and SIE networks
respectively (for greater detail on this architecture, readers can refer to
the original AutoVC paper [4]).

The input and output data of the network is in a mel-spectrogram format. It is
trained using the same input data for both SIE and SVC networks. The input
features used are 128 frames of 80 dimensional log mel-spectrograms of singer
recordings, generated from 16kHz audio with an FFT size of 1024 and a hop size
of 256 (frame duration of 16ms). After training, conversions can be achieved
by feeding the SVC encoder mel-spectrograms of a source singer, while the SIE
network is fed those of a target singer. As the SVC decoder is being
conditioned with SIEs, it is forced to use these vectors to reconstruct the
input data, provided that the bottleneck dimensions are not large enough to
encode them. For this reason, a calibration process of the bottleneck capacity
is required in order to disentangle timbre from the input data adequately.
However, this empirical process can be time-consuming, qualitatively ambiguous
and inherently causes a trade-off between good SVC and audio quality.

Our implementation of AutoVC incorporates the L1 loss to quantify
reconstruction, and a secondary weighted loss which represents the difference
between the SVC encoder embeddings for original data and reconstructed data,
acting as a bottleneck latent regressor loss (BN-LR). To follow up with
inconsistencies regarding the effect of BN-LR loss [39], we investigate
whether the exclusion of this loss would be beneficial in the context of SVC.
As an alternative solution, we also investigate how an SIE latent regressor
(SIE-LR) loss would affect SVC, as we believe this may improve performance
without the need for bottleneck capacity calibration. To summarise, the
objective functions we will be comparing in our experiments include the L1
loss for: just reconstruction; reconstruction and bottleneck latent loss; and
reconstruction and SIE latent loss, as defined in Equations 1, 2 and 3
respectively.

$L_{\mathit{RECON}}=L_{1}(\hat{X},X)$ (1)

$L_{\mathit{BN-LR}}=L_{1}(E_{\mathit{SVC}}(\hat{X}),E_{\mathit{SVC}}(X))$ (2)

$L_{\mathit{SIE-LR}}=L_{1}(E_{\mathit{SIE}}(\hat{X}),E_{\mathit{SIE}}(X))$ (3)

Figure 1: Diagram of the encoder/decoder components in the AutoVC network,
with a secondary cycle partition illustrating how encodings for the
reconstructed data are obtained. Reconstruction, bottleneck regressor loss
(BN-LR) and singer identity embedding regressor loss (SIE-LR) losses are shown
with dotted connectors. $X$ represents a mel-spectrogram encompassing
entangled timbre and residual content, while $R$ (red) and $S$ (blue)
represent disentangled bottleneck embeddings and SIE embeddings respectively.

As we are attempting to disentangle timbre from the input signal, we will
refer to the bottleneck embeddings as the residual information, which includes
attributes independent to singer identity such as pitch contours, singing
techniques, phonetics and other unaccounted variances. In the speech domain,
there is a consideration of whether accent is included in timbral perception.
However this is usually outside the scope of SVC research, likely due to very
little accent diversity within datasets.

###  3.2 Datasets

We choose to use the Digital Archive of Musical Performances (DAMP) Intonation
dataset [42] for this experiment. It is a subset of the singing data supplied
by a publicly available karaoke app designed by
Smule111https://ccrma.stanford.edu/damp/. It consists of 4702 unprocessed
recordings of 3556 singers, created in acoustically untreated environments
primarily with mobile phone devices. This means there is inherently a
considerable amount of background noise such as faintly heard backing tracks,
ambience and other miscellaneous sound events. It also consists of many non-
singing segments due to singers waiting for their backing tracks’ instrumental
sections to conclude. We remove these segments to the best of our abilities
using an empirically chosen volume threshold to detect vocal activity and
removing chunks that remain below this threshold.

###  3.3 Training and Synthesis

To train the SIE network, we use a 3-layer LSTM network, trained using the
GE2E loss objective [5]. This works as a self-supervised model, using a
contrastive learning loss to encourage randomly selected recording clips of
same singers to cluster in the output embedding’s latent space, while
remaining salient from clusters of different singers. This was trained on the
DAMP dataset with a batch size of 80 (8 singers * 10 utterance instances),
with early stopping set to 40 iterations. The network was trained for a total
of 314k iterations. The combination of the autoencoder and SIE network as
described in Section 3.1 allows us to perform SVC. We preserved 80% of it as
training data, while the remainder was split between validation and test
subsets. We train an AutoVC model with a reconstruction loss only (RECON);
reconstruction with bottleneck latent regressor loss (RECON+BN-LR); and
reconstruction with SIE latent regressor loss (RECON+SIE-LR). These models are
trained for a total of 500k iterations, a batch size of 2, and the ADAM
optimizer with a learning rate of $10^{-4}$.

As this experiment investigates the effects of loss components on SVC, we are
not making claims about SOTA results. We can therefore afford to reduce our
dataset to 75% of its original size (keeping subset ratios the same) as used
for SIE training, in the interest of saving on computation time. We reduced
this further by using precalculated SIEs for each singer by running the SIE
network sequentially across windowed chunks of all data for each singer. We
then took the mean of these embeddings, resulting in an average SIE
representation for each singer (as done to determine the GE2E loss used for
SIE training [5]). The embeddings were encapsulated as a lookup table to be
used during SVC training and synthesis to save on computational time.

In previous work on singing voice conversion [7], a similar architecture was
trained with pitch-conditioning embeddings. During the conversion, pitch
contours from the source data were transposed by an octave to match the octave
closest to the average range of the target singer. We consider this to be a
forceful application of pitch shifting that does not take the source singer’s
pitch range into consideration or how timbre can change based on pitch - but
in the context of some SVC methods we appreciate the necessity to do so.
However, for complete authenticity in SVC, we chose not to apply octave
shifts. Instead we applied a custom pitch-matching algorithm that chooses a
target singer provided that there is an example of them singing roughly in the
same range as the given source singer clip. This also ensures that evaluations
of the network’s performance can compare the speaker identities of two audio
clips without being affected by changes in pitch register.

###  3.4 Listening Study

To evaluate our model, we use a combination of objective metrics and
subjective human evaluations. Relying on computational metrics alone can
otherwise be misleading, as some methods of optimising a network may neglect
important aspects of data structures. This can ultimately lead to poor
reconstruction or conversions when evaluated by human listeners. We therefore
prepared a listening study where participants were asked to evaluate the
naturalness of the voice in audio segments synthesised by the three
aforementioned models under the four different source-target gender conditions
(M-M, M-F, F-M, F-F). 23 participants ranging from age 24 to 53 were recruited
from MIR communities, and were asked to rate how similar converted voices were
to target voices using a 5-point scale, as well as how natural the converted
voice sounded.

To synthesise the audio from mel-spectrograms, we used a basic Wavenet vocoder
provided by the authors of [4]222https://github.com/auspicious3000/autovc
which was reportedly pretrained on the VCTK dataset [43] and conditioned on
mel-spectrograms for spectrogram-to-audio conversion tasks. We expect that
this will induce some degree of audio degredation, which should be accounted
for. To do this, we converted the audio of four additional singer clips to
mel-spectrograms which were resynthesized back (without passing through our
SVC network) to audio using the Wavenet vocoder. These examples, along with 16
other voice-converted audio clips where evaluated by participants for audio
quality and conversion performance. Each listener was given a unique set of
audio clips, generated to satisfy each of the experimental conditions. A mean
opinion score (MOS) across participants was then calculated for each
condition.

####  3.4.1 Results

The top two graphs in Fig. 2 show the MOS results for different conditions,
grouped categorically by colour. Note that the MOS naturalness rating for
original audio clips was 3.72, which can be considered as the approximate
upperbound ceiling of perceptual evaluation due to the resynthesis process.

In previous work on singing technique conversion using similar methods [44],
no significant correlation was found between similarity and naturalness.
However, in this experiment a correlation between the two rating types is
noticeably visible, and a Pearson test between the two ratings resulted in
r=0.83, p $<$ 0.002. Due to the strength of this positive correlation, we must
be careful not to assume that participant’s perception of successful voice
conversion are disentangled from their perception of naturalness. We
hypothesise that while participants were asked to rate how confident they were
that the reference recording and the voice-converted recordings were made by
the same singer, lower similarity ratings may be caused by one of either of
these two circumstances: artefacts caused by the SVC process in the audio file
led to a lack of clarity in the timbre; or the timbre of the converted voice
itself was indeed not well matched to the target voice.

To assist in investigating conversion performance in a purely disentangled
manner, we took the cosine similarities between SIEs of converted audio and
SIEs of the target singer. These are shown in the bottom chart of Fig. 2. The
cosine similarity scores reinforce the perceptual similarity scores for the
different model conditions. We can see that compared to perceptual
similarities, cosine similarities are inverted for source-target gender
conditions and present male-to-male conversions with the highest similarities
(while human perception rates female-to-female as most similar). This
inversion between measurements suggests that human listeners may be assigning
more importance to higher frequencies over lower frequencies when assessing
timbral similarities between voices, giving them a bias towards females due to
the increased amount of high-frequency energy. Without analysing the two
similarity metrics comparatively, we would have concluded that female related
conversions were perceived to be more similar because males have a wider pitch
and timbral range due to their ability to move between chest and falsetto
voice, and therefore their voices are harder to model. This is a clear example
of why having both subjective and objective evaluation methods are vital to
assessing models’ performances.

From these graphs, we can deduce that the RECON-BN-LR condition severely
hindered the model’s capabilities of producing natural voices in comparison to
the RECON and RECON-SIE-LR conditions. This can be reasoned by the fact that
adding a loss component of equal weighting can slow down a network’s rate of
convergence if it has more objectives to consider optimising for. However it
is interesting to note that even though the RECON-SIE-LR condition is similar
to RECON-BN-LR in nature, we see no similar statistically significant drop in
performance when compared to RECON. This demonstrates that ensuring the
decoder prioritises the use of the conditioning SIEs by retaining this
information in the output leads towards significantly better voice conversion
and naturalness than the use of the bottleneck’s residual information.

Figure 2: MOS results for participants’ evaluation of naturalness (top) and
similarity (middle), and cosine similarity scores (bottom) grouped by colour
for different conditional levels.

###  3.5 Quantitative Metrics

To analyse the amount of disentanglement between the residual information and
SIEs, we trained a classification layer on our trained AutoVC networks’
bottleneck vectors using 20 singers from our DAMP test subset. Fig. 3 shows
the accuracy results for each model. Accuracy results for RECON (pink)
RECON+SIE-LR (green), and RECON+BN-LR (grey) were approximately 45%, 35% and
23% respectively.

Based on the accuracy of RECON, it is clear that while a bottleneck of
dimension 256 (16 timesteps by 16 frequency bins) might produce adequate
disentanglement for the VCTK speech dataset as reported in [4], this is not
the case for the DAMP singing dataset. This means that there is a considerable
amount of SIE information still entangled in the bottleneck.

Applying BN-LR loss allowed the network to achieve the lowest classification
accuracy, which implies that the network’s encoder encodes minimal singer
identity information and therefore achieves maximal disentanglement between
singer identity and residual information. This is likely due to the fact that
residual content is prioritised as the information that needs to be stored in
the bottleneck due to the conditioning SIEs already providing singer identity
information. SIEs may also vary between original and reconstructed
representations due to the use of averaged SIEs, which is ignored in the
RECON+BN-LR condition.

RECON+SIE-LR loss produces an accuracy 10% less than RECON. This reduction is
likely because the network is relying more heavily on the decoder utilising
the conditioning SIE. It does not however ensure that the encoder avoids
encoding singer-identity information, which is why accuracy is still higher
for this model than RECON+BN-LR. The RECON+SIE-LR condition had however been
shown to achieve the best singer identity conversion, as seen in Fig. 2, which
demonstrates that this loss component offers the added advantage of being
robust against poor disentanglement in the bottleneck. This shows that
utilising the SIE-LR loss avoids the necessity of manually calibrating the
bottleneck’s capacity.

Figure 3: Accuracy contours for models RECON (red), RECON+SIE-LR (green) and
RECON+BN-LR (grey).

##  4 Conclusion

We challenged the use of a latent regressor loss on bottleneck embeddings, and
confirmed that in an SIE-conditioned autoencoder system, using a latent
regressor loss w.r.t. singer identity embeddings rather than bottleneck
embeddings led towards voice-converted audio that was more natural and
successfully converted, according to both human perception and computational
metrics. The advantage of including this loss ensures a more explicit
requirement for good conversion, and also requires no calibration of the
bottleneck dimensions. While including an SIE latent regressor loss has not
been shown to perform better than excluding it, we propose that this
precautionary loss may be more advantageous when training an SVC network for
longer on larger or more complex datasets which will be addressed in future
work. We also proposed the use of a pitch-matching algorithm to ensure
evaluations are not misled by diversity in pitch registers between source and
target singer recordings (a consideration that is surprisingly absent from
previous SVC literature). Objective and subjective ratings for similarity were
inverted across gender conditions, which demonstrates the importance of using
both methods for evaluations as it is important to understand how a network
models data, while considering that the ultimate evaluation relies on human
perception. Finally, we present our results for singing voice conversion
online333trebolium.github.io/autosvc_sie-lr_audio, via a model that utilised
reconstruction with SIE-LR losses and was trained on the entirety of the DAMP
dataset for 1 million training steps.

Acknowledgments
This research is funded by the EPSRC and AHRC Centre for Doctoral Training in
Media and Arts Technology at Queen Mary University of London (EP/L01632X/1).

## References

  * [1] S. Liu, Y. Cao, N. Hu, D. Su, and H. Meng, “FastSVC: Fast Cross-Domain Singing Voice Conversion with Feature-Wise Linear Modulation,” in _Proceedings of 2021 IEEE International Conference on Multimedia and Expo (ICME)_ , Jul. 2021, pp. 1–6.
  * [2] Y.-J. Luo, C.-C. Hsu, K. Agres, and D. Herremans, “Singing Voice Conversion with Disentangled Representations of Singer and Vocal Technique Using Variational Autoencoders,” in _Proceedings of 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_ , May 2020, pp. 3277–3281.
  * [3] J.-c. Chou, C.-c. Yeh, H.-y. Lee, and L.-s. Lee, “Multi-target Voice Conversion without Parallel Data by Adversarially Learning Disentangled Audio Representations,” in _Interspeech 2018_. ISCA, 2018, pp. 501–505.
  * [4] K. Qian, Y. Zhang, S. Chang, X. Yang, and M. Hasegawa-Johnson, “AutoVC: Zero-Shot Voice Style Transfer with Only Autoencoder Loss,” in _Proceedings of the 36th International Conference on Machine Learning_ , vol. 97. PMLR, 2019, pp. 5210–5219.
  * [5] L. Wan, Q. Wang, A. Papir, and I. L. Moreno, “Generalized End-to-End Loss for Speaker Verification,” in _Proceedings of 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_ , Apr. 2018, pp. 4879–4883.
  * [6] Z. Tan, J. Wei, J. Xu, Y. He, and W. Lu, “Zero-Shot Voice Conversion with Adjusted Speaker Embeddings and Simple Acoustic Features,” in _Proceedings of 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_ , Jun. 2021, pp. 5964–5968.
  * [7] S. Nercessian, “Zero-shot Singing Voice Conversion,” in _Proceedings of ISMIR_ , 2020, pp. 70–76.
  * [8] J. Bonada and M. Blaauw, “Semi-Supervised Learning for Singing Synthesis Timbre,” in _Proceedings of 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_ , Jun. 2021, pp. 7083–7087.
  * [9] T. Hasan and J. H. Hansen, “A Study on Universal Background Model Training in Speaker Verification,” _IEEE Transactions on Audio, Speech, and Language Processing_ , vol. 19, no. 7, pp. 1890–1899, Sep. 2011.
  * [10] L. Sun, K. Li, H. Wang, S. Kang, and H. Meng, “Phonetic posteriorgrams for many-to-one voice conversion without parallel data training,” in _2016 IEEE International Conference on Multimedia and Expo (ICME)_ , Jul. 2016, pp. 1–6.
  * [11] R. Skerry-Ryan, E. Battenberg, Y. Xiao, Y. Wang, D. Stanton, J. Shor, R. J. Weiss, R. Clark, and R. A. Saurous, “Towards end-to-end prosody transfer for expressive speech synthesis with Tacotron,” in _Proceedings of Machine Learning Research_ , vol. 80, 2018, pp. 4693–4702.
  * [12] A. Polyak, L. Wolf, Y. Adi, and Y. Taigman, “Unsupervised Cross-Domain Singing Voice Conversion,” in _Interspeech 2020_. ISCA, Oct. 2020, pp. 801–805.
  * [13] X. Li, S. Liu, and Y. Shan, “A Hierarchical Speaker Representation Framework for One-shot Singing Voice Conversion,” in _Interspeech 2022_. ISCA, Sep. 2022, pp. 4307–4311.
  * [14] K. Qian, Z. Jin, M. Hasegawa-Johnson, and G. J. Mysore, “F0-Consistent Many-To-Many Non-Parallel Voice Conversion Via Conditional Autoencoder,” in _Proceedings of 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_ , May 2020, pp. 6284–6288.
  * [15] F. Fang, J. Yamagishi, I. Echizen, and J. Lorenzo-Trueba, “High-Quality Nonparallel Voice Conversion Based on Cycle-Consistent Adversarial Network,” in _Proceedings of 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_ , Apr. 2018, pp. 5279–5283.
  * [16] D. Ulyanov, A. Vedaldi, and V. Lempitsky, “Instance Normalization: The Missing Ingredient for Fast Stylization,” _arXiv preprint arXiv:1607.08022_ , Nov. 2017.
  * [17] Y. Wang, R. Skerry-Ryan, D. Stanton, Y. Wu, R. J. Weiss, N. Jaitly, Z. Yang, Y. Xiao, Z. Chen, S. Bengio, Q. Le, Y. Agiomyrgiannakis, R. Clark, and R. A. Saurous, “Tacotron: Towards End-to-End Speech Synthesis,” in _Interspeech 2017_. ISCA, Aug. 2017, pp. 4006–4010.
  * [18] J. Shen, R. Pang, R. J. Weiss, M. Schuster, N. Jaitly, Z. Yang, Z. Chen, Y. Zhang, Y. Wang, R. Skerrv-Ryan, R. A. Saurous, Y. Agiomvrgiannakis, and Y. Wu, “Natural TTS Synthesis by Conditioning Wavenet on Mel Spectrogram Predictions,” in _Proceedings of 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_ , Apr. 2018, pp. 4779–4783.
  * [19] H. Bai, R. Zheng, J. Chen, M. Ma, X. Li, and L. Huang, “A3T: Alignment-Aware Acoustic and Text Pretraining for Speech Synthesis and Editing,” in _Proceedings of the 39th International Conference on Machine Learning_. PMLR, Jun. 2022, pp. 1399–1411.
  * [20] A. Caillon and P. Esling, “RAVE: A variational autoencoder for fast and high-quality neural audio synthesis,” _arXiv preprint arXiv:2111.05011_ , Dec. 2021.
  * [21] M. Pasini, “MelGAN-VC: Voice Conversion and Audio Style Transfer on arbitrarily long samples using Spectrograms,” _arXiv preprint_ , Oct. 2019.
  * [22] J. Lu, K. Zhou, B. Sisman, and H. Li, “VAW-GAN for Singing Voice Conversion with Non-parallel Training Data,” in _2020 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC)_ , Dec. 2020, pp. 514–519.
  * [23] C.-C. Hsu, H.-T. Hwang, Y.-C. Wu, Y. Tsao, and H.-M. Wang, “Voice Conversion from Unaligned Corpora Using Variational Autoencoding Wasserstein Generative Adversarial Networks,” in _Interspeech 2017_. ISCA, Aug. 2017, pp. 3364–3368.
  * [24] T. Kaneko and H. Kameoka, “Parallel-Data-Free Voice Conversion Using Cycle-Consistent Adversarial Networks,” _arXiv preprint arXiv:1711.11293_ , Nov. 2017.
  * [25] P. Chandna, M. Blaauw, J. Bonada, and E. Gómez, “WGANSing: A Multi-Voice Singing Voice Synthesizer Based on the Wasserstein-GAN,” in _2019 27th European Signal Processing Conference (EUSIPCO)_ , Sep. 2019, pp. 1–5.
  * [26] J. Johnson, A. Alahi, and L. Fei-Fei, “Perceptual Losses for Real-Time Style Transfer and Super-Resolution,” in _Computer Vision – ECCV 2016_ , ser. Lecture Notes in Computer Science, B. Leibe, J. Matas, N. Sebe, and M. Welling, Eds. Cham: Springer International Publishing, 2016, pp. 694–711.
  * [27] Y. Mroueh, T. Sercu, and V. Goel, “McGan: Mean and Covariance Feature Matching GAN,” in _Proceedings of the 34th International Conference on Machine Learning_. PMLR, Jul. 2017, pp. 2527–2535.
  * [28] T. Salimans, I. Goodfellow, W. Zaremba, V. Cheung, A. Radford, and X. Chen, “Improved techniques for training GANs,” _Advances in Neural Infrmation Processing Systems_ , vol. 29, pp. 2234–2242, 2016.
  * [29] A. B. L. Larsen, S. K. Sønderby, H. Larochelle, and O. Winther, “Autoencoding beyond pixels using a learned similarity metric,” in _Proceedings of the 33rd International Conference on Machine Learning_. PMLR, Jun. 2016, pp. 1558–1566.
  * [30] Y. Jia, Y. Zhang, R. Weiss, Q. Wang, J. Shen, F. Ren, z. Chen, P. Nguyen, R. Pang, I. Lopez Moreno, and Y. Wu, “Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesis,” in _Advances in Neural Information Processing Systems_ , vol. 31. Curran Associates, Inc., 2018.
  * [31] J. H. Lee, H.-S. Choi, and K. Lee, “Audio query-based music source separation,” _arXiv preprint arXiv:1908.06593_ , p. 8, 2019.
  * [32] J.-Y. Zhu, T. Park, P. Isola, and A. A. Efros, “Unpaired Image-to-Image Translation Using Cycle-Consistent Adversarial Networks,” in _2017 IEEE International Conference on Computer Vision (ICCV)_. Venice: IEEE, Oct. 2017, pp. 2242–2251.
  * [33] R. Sennrich, B. Haddow, and A. Birch, “Improving Neural Machine Translation Models with Monolingual Data,” in _Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_. Berlin, Germany: Association for Computational Linguistics, Aug. 2016, pp. 86–96.
  * [34] M. Amodio and S. Krishnaswamy, “TraVeLGAN: Image-To-Image Translation by Transformation Vector Learning,” in _2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_ , Jun. 2019, pp. 8975–8984.
  * [35] T. Zhou, P. Krahenbuhl, M. Aubry, Q. Huang, and A. A. Efros, “Learning Dense Correspondence via 3D-Guided Cycle Consistency,” in _2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_. Las Vegas, NV, USA: IEEE, Jun. 2016, pp. 117–126.
  * [36] J. Donahue, P. Krähenbühl, and T. Darrell, “Adversarial Feature Learning,” in _International Conference on Learning Representations_ , 2017.
  * [37] H. Du, X. Tian, L. Xie, and H. Li, “Optimizing Voice Conversion Network with Cycle Consistency Loss of Speaker Identity,” in _2021 IEEE Spoken Language Technology Workshop (SLT)_ , Jan. 2021, pp. 507–513.
  * [38] S. Liu, J. Zhong, L. Sun, X. Wu, X. Liu, and H. Meng, “Voice Conversion Across Arbitrary Speakers Based on a Single Target-Speaker Utterance,” in _Interspeech 2018_. ISCA, Sep. 2018, pp. 496–500.
  * [39] B. O’Connor, S. Dixon, and G. Fazekas, “Zero-shot Singing Technique Conversion,” in _Proc. of the 15th International Symposium on CMMR_ , Tokyo, 2021, pp. 235–244.
  * [40] D.-Y. Wu and H.-y. Lee, “One-Shot Voice Conversion by Vector Quantization,” in _Proceedings of 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_ , May 2020, pp. 7734–7738.
  * [41] P. Chandna, M. Blaauw, J. Bonada, and E. Gómez, “Content Based Singing Voice Extraction from a Musical Mixture,” in _Proceedings of 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_ , May 2020, pp. 781–785.
  * [42] S. Wager, G. Tzanetakis, S. Sullivan, C.-i. Wang, J. Shimmin, M. Kim, and P. Cook, “Intonation: A Dataset of Quality Vocal Performances Refined by Spectral Clustering on Pitch Congruence,” in _Proceedings of 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_ , May 2019, pp. 476–480.
  * [43] C. Veaux, J. Yamagishi, and K. MacDonald, “Superseded CSTR VCTK corpus: English multi-speaker corpus for CSTR voice cloning toolkit,” University of Edinburgh, Centre for Speech Technology Research (CSTR), Tech. Rep., 2016.
  * [44] B. O’Connor, S. Dixon, and G. Fazekas, “An Exploratory Study on Perceptual Spaces of the Singing Voice,” in _Proceedings of 2020 Joint AI Conference on Music Creativity_ , vol. 1, Stockholm, Sweden, Oct. 2020.


# Implicit Methods with Reduced Memory for Thermal Radiative Transfer

Dmitriy Y. Anistratov  Joseph M. Coale  Department of Nuclear Engineering,
North Carolina State University Raleigh, NC<EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract

This paper presents approximation methods for time-dependent thermal radiative
transfer problems in high energy density physics. It is based on the
multilevel quasidiffusion method defined by the high-order radiative transfer
equation (RTE) and the low-order quasidiffusion (aka VEF) equations for the
moments of the specific intensity. A large part of data storage in TRT
problems between time steps is determined by the dimensionality of grid
functions of the radiation intensity. The approximate implicit methods with
reduced memory for the time-dependent Boltzmann equation are applied to the
high-order RTE, discretized in time with the backward Euler (BE) scheme. The
high-dimensional intensity from the previous time level in the BE scheme is
approximated by means of the low-rank proper orthogonal decomposition (POD).
Another version of the presented method applies the POD to the remainder term
of $P_{2}$ expansion of the intensity. The accuracy of the solution of the
approximate implicit methods depends of the rank of the POD. The proposed
methods enable one to reduce storage requirements in time dependent problems.
Numerical results of a Fleck-Cummings TRT test problem are presented.

###### keywords:

high-energy density physics , Boltzmann equation , radiative transfer ,
implicit schemes , memory reduction , proper orthogonal decomposition ,
multilevel methods

##  1 Introduction

We consider the thermal radiative transfer (TRT) problem in 1D slab geometry
that is defined by the time-dependent radiative transfer equation (RTE)

$\frac{1}{c}\frac{\partial I_{g}}{\partial t}(x,\mu,t)+\mu\frac{\partial
I_{g}}{\partial
x}(x,\mu,t)+\varkappa_{g}(T)I_{g}(x,\mu,t)=\varkappa_{g}(T)B_{g}(T)\,,$ (1)
$x\in[0,X]\,,\quad\mu\in[-1,1]\,,\quad g\in\mathbb{N}(G)\,,\quad t\geq
t_{0}\,,$
$\left.I_{g}\right|_{\stackrel{{\scriptstyle\mu>0}}{{x=0}}}=I_{g}^{in+}\,,\quad\left.I_{g}\right|_{\stackrel{{\scriptstyle\mu<0}}{{x=X}}}=I_{g}^{in-}\,,\quad\left.I_{g}\right|_{t=t_{0}}=I_{g}^{0}\,,$
(2)

and the material energy balance (MEB) equation

$\frac{\partial\varepsilon(T)}{\partial
t}=\sum_{g=1}^{G}\varkappa_{g}(T)\Big{(}\int_{-1}^{1}I_{g}(x,\mu,t)d\mu-2B_{g}(T)\Big{)}\,,\quad\left.T\right|_{t=t_{0}}=T_{0}\,,$
(3)

where $I_{g}$ is the group specific photon intensity; $x$ is the spatial
position; $\mu$ is the direction cosine of particle motion; $g$ is the index
of photon frequency group; $\mathbb{N}(G)=\\{1,\ldots,G\\}$; $t$ is time;
$\varkappa_{g}$ is the group opacity; $T$ is the material temperature;
$\varepsilon$ is the material energy density; $B_{g}$ is the group Planck
black-body distribution function.

The solution of the multigroup RTE in general geometry depends on 7
independent variables. Temporal discretization schemes for the RTE involve the
discrete solution at the previous time level. This requires storing in memory
6-dimensional grid functions that approximate the transport solution on a
given mesh in the phase space. There are different approaches for developing
approximate methods for time-dependent transport problems that reduce memory
requirements [1, 2, 3, 4, 5]. The $\alpha$-approximation of the intensity in
time reduces the RTE to a transport equation of steady-state form with a
modified opacity [1]. This approximation assumes that the intensity varies
exponentially over each time interval. The approximate rate of change in time
can be obtained by means of the solution of low-order moment equations. As
such, the $\alpha$-approximation rids one of the need to store the high-
dimensional solution from the previous time level [1]. This approximation
method for the time-dependent RTE demonstrated good accuracy in TRT problems
[6, 7]. Analysis showed that there are some limitations for the RTE in the
$\alpha$-approximation [4].

Recently, approximate implicit methods with reduced memory for the time-
dependent Boltzmann transport equation have been proposed [5]. They use the
modified backward Euler (MBE) time integration scheme that applies the proper
orthogonal decomposition (POD) of the transport solution from the previous
time step to compress the data and reduce memory requirements [8, 9, 10]. The
accuracy of the method depends on the order of the low-rank POD of the
discrete transport solution. The error decreases as rank increases. In this
paper, we apply the MBE scheme within the framework of the multilevel
quasidiffusion (MLQD) method for solving TRT problems [6, 11, 12].

The reminder of the paper is organized as follows. In Sec. 2, the MLQD method
with approximate implicit scheme is formulated. In Sec. 3, we present
different approximations of the specific intensity by means of the POD. The
numerical results are presented in Sec. 4. We conclude with a discussion in
Sec. 5.

##  2  The MLQD Method with Approximate Implicit Scheme for the High-Order
Problem

###  2.1 MLQD Equations and Discretization

The MLQD method is defined by a system of equations consisting of

  1. 1.

the multigroup high-order RTE (Eq. (1))

$\frac{1}{c}\frac{\partial I_{g}}{\partial t}+\mu\frac{\partial
I_{g}}{\partial x}+\varkappa_{g}I_{g}=\varkappa_{g}B_{g}\,,$ (4)

  2. 2.

the multigroup low-order quasidiffusion (aka VEF) equations for the group
radiation energy density and flux [11, 13]

$\frac{\partial E_{g}}{\partial t}+\frac{\partial F_{g}}{\partial
x}+c\varkappa_{g}E_{g}=2\varkappa_{g}B_{g}\,,$ (5a) $\frac{1}{c}\frac{\partial
F_{g}}{\partial t}+c\frac{\partial(f_{g}E_{g})}{\partial
x}+\varkappa_{g}F_{g}=0\,,$ (5b) where
$f_{g}=\frac{\int_{-1}^{1}\mu^{2}I_{g}d\mu}{\int_{-1}^{1}I_{g}d\mu}$ (5c)

is the group QD (Eddington) factor,

  3. 3.

the effective grey low-order quasidiffusion (LOQD) equations for the total
radiation energy density and fluxes

$\frac{\partial E}{\partial t}+\frac{\partial F}{\partial
x}+c\bar{\varkappa}_{E}E=c\bar{\varkappa}_{B}a_{R}T^{4}\,,$ (6a)
$\frac{1}{c}\frac{\partial F}{\partial
t}+c\frac{\partial(\bar{f}_{E}E)}{\partial
x}+\bar{\varkappa}_{|\\!F\\!|}F+\bar{\eta}E=0\,,$ (6b)

where the spectrum averaged opacities and factors are defined by

$\bar{\alpha}_{H}=\frac{\sum_{g=1}^{G}\alpha_{g}H_{g}}{\sum_{g=1}^{G}H_{g}}\,,\quad\bar{\eta}=\frac{\sum_{g=1}^{G}(\varkappa_{g}-\bar{\varkappa}_{|F|})F_{g}}{\sum_{g=1}^{G}E_{g}}\,,$
(7)

  4. 4.

the MEB equation (3) in grey form

$\frac{\partial\varepsilon(T)}{\partial
t}=c\big{(}\bar{\varkappa}_{E}E-\bar{\varkappa}_{B}a_{R}T^{4}\big{)}\,.$ (8)

We discretize the equations of the MLQD method by the backward Euler (BE) time
integration scheme. This yields the semi-discrete RTE at the $n$-th time level
given by

$\frac{1}{c\Delta t^{n}}\big{(}I_{g}^{n}-I_{g}^{n-1}\big{)}+\mu\frac{\partial
I_{g}^{n}}{\partial x}+\varkappa_{g}^{n}I_{g}^{n}=Q_{g}^{n}\,,$ (9)

where $\Delta t^{n}=t^{n}-t^{n-1}$ is the $n$-th time step,
$Q_{g}^{n}=\varkappa_{g}(T^{n})B_{g}(T^{n})$. The high-order equation (9) is
discretized in space by the step characteristic (SC) scheme. The multigroup
LOQD equations discretized in time by the BE scheme have the following form:

$\frac{1}{\Delta t^{n}}\big{(}E_{g}^{n}-E_{g}^{n-1}\big{)}+\frac{\partial
F_{g}^{n}}{\partial x}+c\varkappa_{g}^{n}E_{g}^{n}=2Q_{g}^{n}\,,$ (10a)
$\frac{1}{c\Delta
t^{n}}\big{(}F_{g}^{n}-F_{g}^{n-1}\big{)}+c\frac{\partial(f_{g}^{n}E_{g}^{n})}{\partial
x}+\varkappa_{g}^{n}F_{g}^{n}=0\,,$ (10b)
$f_{g}^{n}=\frac{\int_{-1}^{1}\mu^{2}I_{g}^{n}d\mu}{\int_{-1}^{1}I_{g}^{n}d\mu}\,.$
(11)

The grey LOQD and MEB equations approximated with the BE scheme are defined by

$\frac{1}{\Delta t^{n}}\big{(}E^{n}-E^{n-1}\big{)}+\frac{\partial
F^{n}}{\partial
x}+c\bar{\varkappa}_{E}^{n}E^{n}=c\bar{\varkappa}_{B}^{n}a_{R}(T^{n})^{4}\,,$
(12a) $\frac{1}{c\Delta
t^{n}}\big{(}F^{n}-F^{n-1}\big{)}+c\frac{\partial(\bar{f}_{E}^{n}E^{n})}{\partial
x}+\bar{\varkappa}_{|\\!F\\!|}^{n}F^{n}+\bar{\eta}^{n}E^{n}=0\,.$ (12b)
$\frac{1}{\Delta
t^{n}}\big{(}\varepsilon(T^{n})-\varepsilon(T^{n-1})\big{)}=c\big{(}\bar{\varkappa}_{E}^{n}E^{n}-\bar{\varkappa}_{B}^{n}a_{R}(T^{n})^{4}\big{)}\,.$
(13)

The multigroup LOQD equations are discretized in space by a second-order
finite volume (FV) method. The spatial discretization of the grey LOQD
equations is algebraically consistent with the discretized multigroup LOQD
equations [7]. We refer to the described method as the MLQD method with BE-SC
scheme.

###  2.2 Approximate Implicit Method for the RTE

In the approximate implicit scheme, the multigroup RTE (4) is discretized by
the MBE time integration scheme given by [5]

$\frac{1}{c\Delta
t^{n}}\big{(}I_{g}^{n}-\hat{I}_{g}^{n-1}\big{)}+\mu\frac{\partial
I_{g}^{n}}{\partial x}+\varkappa_{g}^{n}I_{g}^{n}=Q_{g}^{n}\,,$ (14)

where the grid functions of group intensity $\hat{I}_{g}^{n-1}$ are
approximated by the low-rank POD of the solution $I_{g}^{n-1}$ computed at the
time step $n-1$. The SC scheme for the high-order equation (14) is formulated
for the cell-edge $\big{(}I_{g\,m\,j+1/2}^{n}\big{)}$ and cell-average
$\big{(}I_{g\,m\,j}^{n}\big{)}$ angular fluxes by means of the detailed
particle balance equation and weighted auxiliary relation

${\frac{\Delta x_{j}}{c\Delta
t^{n}}}\big{(}I_{g\,m\,j}^{n}-\hat{I}_{g\,m\,j}^{n-1}\big{)}+\mu_{m}\big{(}I_{g\,m\,j+1/2}^{n}-I_{g\,m\,j-1/2}^{n}\big{)}+\varkappa_{g\,j}^{n}I_{g\,m\,j}^{n}\Delta
x_{j}=Q_{g\,j}^{n}\Delta x_{j}\,,$ (15a)
$I_{g\,m\,j}^{n}=\gamma_{g\,m\,j}^{n}I_{g\,m\,j-1/2}^{n}+(1-\gamma_{g\,m\,j}^{n})I_{g\,m\,j+1/2}^{n}\,,$
(15b)
$\gamma_{g\,m\,j}^{n}=\frac{1}{\tau_{g\,m\,j}^{n}}-\frac{1}{e^{\tau_{g\,m\,j}^{n}}-1}\,,\quad\tau_{g\,m\,j}^{n}=\frac{1}{\mu_{m}}\Big{(}\varkappa_{g\,j}^{n}+(c\Delta
t^{n})^{-1}\Big{)}\Delta x_{j}\,,$ (15c)

where $m\in\mathbb{N}(M)$ is the index of angular direction,
$j\in\mathbb{N}(J)$ is the index of the spatial interval, $\Delta x_{j}$ is
the width of the $j$-th cell. We refer to the discretized RTE (15) as the MBE-
SC scheme.

##  3  Approximation of the Specific Intensity

###  3.1 POD of the Intensity

The MBE-SC scheme (Eqs. (15)) needs to store the cell-average intensity
$I_{g\,m\,j}^{n}$. In each photon frequency group, it is a 2D discrete grid
function of $j$ and $m$. We interpret it in a matrix form defined by
$A_{I}=[\boldsymbol{I}_{1}\ldots\boldsymbol{I}_{M}]$
$\big{(}A_{I}\in\mathbb{R}^{J\times M}\big{)}$, where the columns are given by
$\boldsymbol{I}_{m}=[I_{\,m\,1}\ldots I_{m\,J}]^{T}$
$\big{(}\boldsymbol{I}_{m}\in\mathbb{R}^{J}\big{)}$. Here we omitted group and
time indices for the sake of brevity. We approximate the grid function of the
group cell-average intensity by the low–rank POD [9, 10]. The reduced singular
value decomposition (SVD) of $A_{I}$ has the form:

$A_{I}=U_{I}\Lambda_{I}V_{I}^{T}\,.$ (16)

$\Lambda_{I}=\text{diag}(\lambda_{1}\ldots\lambda_{d})\ \in\mathbb{R}^{d\times
d}$ is the diagonal matrix of singular values, where

$d=\min(J,M)$ (17)

is the rank of $A_{I}$. $U_{I}=[\mathbf{u}_{1}\ldots\mathbf{u}_{d}]$ and
$V_{I}=[\mathbf{v}_{1}\ldots\mathbf{v}_{d}]$ are the matrices of left and
right singular vectors, respectively, where
$\mathbf{u}_{\ell}\in\mathbb{R}^{J}$ and $\mathbf{v}_{\ell}\in\mathbb{R}^{M}$.
The approximate group intensity
$\boldsymbol{\hat{I}}_{m}=[\hat{I}_{m\,1}\ldots\hat{I}_{m\,J}]^{T}$ is defined
by the low-rank POD of $A_{I}$ given by

$\hat{A}_{I}^{r}=\sum_{\ell=1}^{r}\lambda_{\ell}\mathbf{u}_{\ell}\otimes(\mathbf{v}_{\ell})^{T}\,,\quad
r<d\,,\quad\mbox{where}\quad\hat{A}_{I}^{r}=[\boldsymbol{\hat{I}}_{1}\ldots\boldsymbol{\hat{I}}_{M}]\,.$
(18)

This is the optimal approximation of the matrix $A_{I}$ in the 2-norm [10,
14]. The low-rank approximation (18) requires storage of the first $r$
singular values and associated left and right singular vectors. Thus, this
approximation leads to memory allocation of a data set with the number of
elements $r(J+M+1)$ in each group. The rank can be chosen according to various
criteria.

###  3.2 POD of the Remainder Term

We cast the intensity as its $P_{2}$ approximation and the remainder term
defined by

$\Delta
I_{m\,j}=I_{m\,j}-\frac{1}{2}\Big{(}\tilde{\phi}_{j}+3\mu_{m}\tilde{F}_{j}+\frac{5}{4}\big{(}3\mu_{m}^{2}-1\big{)}\big{(}3f_{j}-1\big{)}\tilde{\phi}_{j}\Big{)}\,,$
(19)

where the $P_{2}$ expansion coefficients are calculated by the solution of the
high-order RTE, namely,

$\mathcal{\tilde{\phi}}_{j}=\sum_{m=1}^{M}I_{m\,j}w_{m}\,,\quad\tilde{F}_{j}=\sum_{m=1}^{M}\mu_{m}I_{m\,j}w_{m}\,,\quad
f_{j}=\frac{\sum_{m=1}^{M}\mu_{m}^{2}I_{m\,j}w_{m}}{\sum_{m=1}^{M}I_{m\,j}^{n}w_{m}}\,.$
(20)

The discrete 2D function $\Delta I_{m\,j}$ is treated as a matrix defined by
$\Delta A_{I}=[\Delta\boldsymbol{I}_{1}\ldots\Delta\boldsymbol{I}_{M}]$, where
$\Delta\boldsymbol{I}_{m}=[\Delta I_{m\,1}\ldots\Delta I_{m\,J}]^{T}$. Its POD
is given by

$\Delta A_{I}=U_{I}^{\prime}\Lambda_{I}^{\prime}(V_{I}^{\prime})^{T}\,.$ (21)

where
$\Lambda_{I}^{\prime}=\text{diag}(\lambda_{1}^{\prime}\ldots\lambda_{d}^{\prime})\
\in\mathbb{R}^{d\times d}$,
$U_{I}^{\prime}=[\mathbf{u}_{1}^{\prime}\ldots\mathbf{u}_{d}^{\prime}]$,
$V_{I}^{\prime}=[\mathbf{v}_{1}^{\prime}\ldots\mathbf{v}_{d}^{\prime}]$,
$\mathbf{u}_{\ell}^{\prime}\in\mathbb{R}^{J}$, and
$\mathbf{v}_{\ell}^{\prime}\in\mathbb{R}^{M}$. We apply the low-rank POD

$\Delta\hat{A}_{I}^{r}=[\Delta\boldsymbol{\hat{I}}_{1}\ldots\Delta\boldsymbol{\hat{I}}_{M}]=\sum_{k=1}^{r}\lambda_{k}^{\prime}\mathbf{u}_{k}^{\prime}\otimes(\mathbf{v}_{k}^{\prime})^{T}\,,\
\ r<d\,,\ \
\Delta\boldsymbol{\hat{I}}_{m}=[\Delta\hat{I}_{m\,1},\ldots,\Delta\hat{I}_{m\,J}]^{T}$
(22)

to define approximate intensities as the sum of its $P_{2}$ approximation and
the POD of the remainder term

$\hat{I}_{m\,j}=\frac{1}{2}\Big{(}\tilde{\phi}_{j}+3\mu_{m}\tilde{F}_{j}+\frac{5}{4}\big{(}3\mu_{m}^{2}-1\big{)}\big{(}3f_{j}^{n}-1\big{)}\tilde{\phi}_{j}\Big{)}+\Delta\hat{I}_{m\,j}\,.$
(23)

This approximation needs to store in memory $r(J+M+1)+2J$ elements that
includes (i) $r(J+M+1)$ elements for the remainder term and (ii) $2J$ elements
for vectors of two angular moments $\boldsymbol{\tilde{\phi}}$ and
$\boldsymbol{\tilde{F}}$.

##  4  Numerical Results

We present numerical results of the Fleck-Cummings (F-C) test [15]. The
spatial domain ($0\leq x\leq 6$) contains one material. The spectral opacity
of the material is given by
$\varkappa_{\nu}=\frac{27}{(h\nu)^{3}}\big{(}1-e^{-\frac{h\nu}{kT}}\big{)}$.
There is incoming radiation with black-body spectrum $B_{\nu}$ at temperature
$kT_{in}=1$ keV at the left boundary. The right boundary is vacuum. The
initial temperature of the slab is $kT_{0}=1$ eV. At $t=0$ the radiation
intensity in the slab has the black-body spectrum at $T_{0}$. The material
energy density is $\varepsilon=c_{\nu}T$, where
$c_{\nu}=0.5917a_{R}T_{in}^{3}$. The problem is solved over the time interval
$0\leq t\leq 6$ ns. The time step size is $\Delta t~{}=2\times 10^{-2}$ ns.
The uniform spatial mesh consists of $J=100$ cells. The angular mesh has 8
discrete directions ($M=8$). The double $S_{4}$ Gauss-Legendre quadrature set
is used. We define $G=17$ energy groups. The parameters of convergence
criteria for temperature and energy density are
$\epsilon_{T}=\epsilon_{E}=10^{-12}$, respectively.

(a) $\frac{||T_{h}-T_{h}^{r}||_{\infty}}{||T_{h}||_{\infty}}$.

(b) $\frac{||E_{h}-E_{h}^{r}||_{\infty}}{||E_{h}||_{\infty}}$.

Figure 1:  Relative error in $\infty$-norm of the solution of the MLQD method
with the MBE-SC scheme and POD of the intensity compared to the discrete
solution on the corresponding grid in phase space and time.

(a) $\frac{||T_{h}-T_{h}^{r}||_{\infty}}{||T_{h}||_{\infty}}$.

(b) $\frac{||E_{h}-E_{h}^{r}||_{\infty}}{||E_{h}||_{\infty}}$.

Figure 2:  Relative error in $\infty$-norm of the solution of the MLQD method
with the MBE-SC scheme and POD of the remainder term compared to the discrete
solution on the corresponding grid in phase space and time.

The discrete solution of the MLQD method with the MBE-SC scheme, namely, the
total radiation energy density $E^{r}_{h}$ and temperature $T^{r}_{h}$ of the
approximate implicit method with the rank $r$ POD is compared to the discrete
solution $T_{h}$ and $E_{h}$ of the MLQD method with the BE-SC scheme on the
corresponding grid in the phase space and time. The numerical results of the
method with the POD of the intensity of the rank $r$ in all groups are
presented in Figure 2. The plots show the relative error in reproducing the
discrete solution in $\infty$-norm, namely,
$\frac{||T_{h}-T_{h}^{r}||_{\infty}}{||T_{h}||_{\infty}}$ and
$\frac{||E_{h}-E_{h}^{r}||_{\infty}}{||E_{h}||_{\infty}}$ for the complete
range of $r$. The results obtained with the MBE-SC scheme using the POD of the
rank $r$ of the remainder term in each group are shown in Figure 2. In this
test, the full rank $d$ (Eq. (17)) equals 8. The results with the full-rank
POD ($r=8$) of both methods illustrate that they accurately reproduce the
discrete solution of the MLQD method with the SC scheme on the given grid as
expected. In case $r=5,6,7$ the solution of the method with the POD of the
remainder term has very small error. This is due to explicit accounting for
the first three Legendre moments of the intensity (Eq. (23)). The singular
eigenvalues $\lambda_{\ell}^{\prime}$ for $\ell=5,6,7$ in groups are very
small. In this test problem, the method with POD of the remainder term is
predominantly more accurate than to the method with POD of the intensity for
the given rank $r$. However, it uses more data for the rank $r$. Figure 3
shows the ratio between errors of the method with the POD of the remainder
term (POD-RT) and the one with the POD of the intensity (POD-I).

(a) $\frac{||T_{h}-T_{h}^{r}||_{\infty}^{\text{POD-
RT}}}{||T_{h}-T_{h}^{r}||_{\infty}^{\text{POD-I}}}$.

(b) $\frac{||E_{h}-E_{h}^{r}||_{\infty}^{\text{POD-
RT}}}{||E_{h}-E_{h}^{r}||_{\infty}^{\text{POD-I}}}$.

Figure 3:  The error of the method with POD of the remainder term (POD-RT)
over the error of the method with POD of intensity (POD-I).

The gains in memory allocation depend on both the number of spatial cells $J$
and angular directions $M$ and hence are problem specific. For the phase-space
grid used in the test, the size of the data set stored by this MLQD method
with the RTE discretized the BE-SC scheme at the end of each time step is
$D=G(J\times M+2\times J+1)+2\times J+1=17218$. This includes the data for (i)
the multigroup RTE, (ii) the multigroup and grey LOQD equations, and (iii) the
MEB equation. Table 1 shows the percentage reduction of required data storage
sizes of the MLQD method with each of the two versions of the MBE-SC scheme
compared to that of the MLQD method with the BE-SC scheme. Negative values
indicate an increase in storage compared to the BE-SC scheme. In this test,
the method with POD of intensities shows gains in memory for all ranks, i.e.
$r=1,\ldots,7$. The method with POD of the remainder term reduces memory
allocation for $r=1,\ldots,5$.

Table 1:  Reduction [%] in memory storage of previous step data of the MLQD method with the MBE-SC scheme ($J$=100, $M$=8). Rank ($r$) | 1 | 2 | 3 | 4 | 5 | 6 | 7
---|---|---|---|---|---|---|---
POD-I | 68.2 | 57.5 | 46.7 | 35.9 | 25.2 | 14.4 | 3.7
POD-RT | 48.5 | 37.7 | 27.0 | 16.2 | 5.4 | -5.3 | -16.1

Figures 5 and 5 present the results of spatial mesh refinement for the fixed
time step size $\Delta t$ = 2$\times$10-2 ns. They show the relative error of
E in $\infty$-norm for uniform meshes with $\Delta x$ = 0.24, 0.12, 0.06, 0.03
cm. The number of degrees of freedom of the discrete intensity increases with
refinement of spatial mesh. The results show that the change in the relative
error decreases with refinement. The factor of change on fine meshes
approaches one. This indicates that the error due to low-rank POD of data
representing intensities tends to a limit as $\Delta x\to 0$ for the fixed
time step $\Delta t$. Figures 7 and 7 present the relative error of E in
$\infty$-norm for the numerical solution computed with refined time steps
($\Delta t$ = 4$\times$10-2, 2$\times$10-2, 10-2, 5$\times$10-3 ns) on the
spatial mesh with $\Delta x$ = 6$\times$10-2 cm. These results show increase
in the relative error in reproducing the discrete solution on the given grids.
More analysis is needed to study properties of the methods.

(a) $\frac{||E_{h}-E_{h}^{r}||_{\infty}}{||E_{h}||_{\infty}}$ at $t=0.4$ ns

(b) $\frac{||E_{h}-E_{h}^{r}||_{\infty}}{||E_{h}||_{\infty}}$ at $t=1$ ns

(c) $\frac{||E_{h}-E_{h}^{r}||_{\infty}}{||E_{h}||_{\infty}}$ at $t=6$ ns

Figure 4: Results of refinement of spatial mesh for the MLQD method with the
MBE-SC scheme and POD of the intensity for $\Delta t=2\times 10^{-2}$.

(a) $\frac{||E_{h}-E_{h}^{r}||_{\infty}}{||E_{h}||_{\infty}}$ at $t=0.4$ ns

(b) $\frac{||E_{h}-E_{h}^{r}||_{\infty}}{||E_{h}||_{\infty}}$ at $t=1$ ns

(c) $\frac{||E_{h}-E_{h}^{r}||_{\infty}}{||E_{h}||_{\infty}}$ at $t=6$ ns

Figure 5: Results of spatial mesh refinement for the MLQD method with the MBE-
SC scheme and POD of the remainder term for $\Delta t=2\times 10^{-2}$ ns.

(a) $\frac{||E_{h}-E_{h}^{r}||_{\infty}}{||E_{h}||_{\infty}}$ at $t=0.4$ ns

(b) $\frac{||E_{h}-E_{h}^{r}||_{\infty}}{||E_{h}||_{\infty}}$ at $t=1$ ns

(c) $\frac{||E_{h}-E_{h}^{r}||_{\infty}}{||E_{h}||_{\infty}}$ at $t=6$ ns

Figure 6: Results of time step refinement the MLQD method with the MBE-SC
scheme and POD of the intensity $\Delta x=6\times 10^{-2}$ cm.

(a) $\frac{||E_{h}-E_{h}^{r}||_{\infty}}{||E_{h}||_{\infty}}$ at $t=0.4$ ns

(b) $\frac{||E_{h}-E_{h}^{r}||_{\infty}}{||E_{h}||_{\infty}}$ at $t=1$ ns

(c) $\frac{||E_{h}-E_{h}^{r}||_{\infty}}{||E_{h}||_{\infty}}$ at $t=6$ ns

Figure 7: Results of time step refinement the MLQD method with the MBE-SC
scheme and POD of the remainder term for $\Delta x=6\times 10^{-2}$ cm.

##  5  Conclusions

This paper presented implicit methods with approximate time evolution operator
in the high-order Boltzmann equation and reduced memory for TRT problems. The
obtained results showed that the proposed methods reproduce the numerical
solution of the underlying discretization method on the given phase-space grid
with various degrees of accuracy while reducing storage of data between time
steps. The accuracy depends on the rank of the POD of the data representing
intensity from the previous time level. It is possible to achieve accuracy
that is good for practical routine simulations and significantly reduce memory
usage. There are extra computational costs due to calculations of the POD of
intensities. The proposed approximate implicit methods are intended for
computer architectures on which one can take advantage of extra computations
for reduction of memory storage. The proposed approach can be applied to
various time integration methods and different kind of transport problems.

## Acknowledgements

This research project is funded by the Department of Defense, Defense Threat
Reduction Agency, grant number HDTRA1-18-1-0042. The content of the
information does not necessarily reflect the position or the policy of the
federal government, and no official endorsement should be inferred.

## References

  * [1] V. Ya. Gol’din, G. V. Danilova, B. N. Chetverushkin, Approximate method for solving time-dependent kinetic equation, in: Computational Methods in Transport Theory, Atomizdat, Moscow, 1969, pp. 50–57, (in Russian).
  * [2] A. Matsekh , L. Chacon, H. Park, G. Chen, Machine learning for memory reduction in the implicit monte carlo simulations of thermal radiative transfer, Tech. Rep. LA-UR-18-25444, Los Alamos National Laboratory (2018).
  * [3] Z. Peng, R. G. McClarren, M. Frank, A low-rank method for time-dependent transport calculations, in: Proc. of Int. Conf. on Math. and Comp., M&C 2019, Portland, OR, USA, 2019, pp. 957–965.
  * [4] P. Ghassemi, D. Y. Anistratov, An approximation method for time-dependent problems in high energy density thermal radiative transfer, Journal of Computational and Theoretical Transport 41 (2020) 31–50.
  * [5] D. Y. Anistratov, Implicit methods with reduced memory for time-dependent boltzmann transport equation, Transactions of American Nuclear Society 122 (2020) 367–370.
  * [6] D. Y. Anistratov, E. N. Aristova, V. Y. Gol’din, A nonlinear method for solving problems of radiation transfer in a physical system, Mathematical Modeling 8 (1996) 3–28, in Russian.
  * [7] D. Y. Anistratov, Stability analysis of a multilevel quasidiffusion method for thermal radiative transfer problems, Journal of Computational Physics 376 (2019) 186–209.
  * [8] L. Sirovich, Turbulence and the dynamics of coherent structures. parts i-iii, Quarterly of Applied Mathematics XLV (1987) 561–590.
  * [9] G. Berkooz, P. Holmes, J. L. Lumley, The proper orthogonal decomposition in the analysis of turbulent flows, Annual Review of Fluid Mechanics 25 (1993) 539–575.
  * [10] K. Kunisch, S. Volkwein, Galerkin proper orthogonal decomposition methods for a general equation in fluid dynamics, SIAM J. Numer. Anal 40 (2002) 492–515.
  * [11] V. Ya. Gol’din, A quasi-diffusion method of solving the kinetic equation, USSR Comp. Math. and Math. Phys. 4 (1964) 136–149.
  * [12] V. Ya. Gol’din, B. N. Chetverushkin, Methods of solving one-dimensional problems of radiation gas dynamics, USSR Comp. Math. and Math. Phys. 12 (1972) 177–189.
  * [13] L. H. Auer, D. Mihalas, On the use of variable Eddington factors in non-LTE stellar atmospheres computations, Monthly Notices of the Royal Astronomical Society 149 (1970) 65–74.
  * [14] I. Ipsen, Numerical Matrix Analysis, SIAM, Philadelphia, PA, 2009.
  * [15] J. A. Fleck, J. D. Cummings, An implicit monte carlo scheme for calculating time and frequency dependent nonlinear radiation transport, Journal of Computational Physics 8 (1971) 313–342.