for some constant $\Delta>0$. The above inequality is understood as partial
order defined by the convex cone of positive semi-definite matrices (see
Definition 2.20). Note that in (A.1) we neglect the error of off-diagonal
elements for convenience. Including these error terms would clearly make the
spurious correlation larger.

From (1.5) it follows that the cost created by the spurious correlation of a
greedy strategy $\hat{u}$ is given by,

SpurCor $\displaystyle=J(\hat{u};G^{\star})-J(\hat{u};\hat{G})$ (A.2)
$\displaystyle=\mathbb{E}\left[\sum_{i=1}^{M}\sum_{j=1}^{i}(G^{\star}_{i,j}-\hat{G}_{i,j})\hat{u}_{t_{j}}\hat{u}_{t_{i}}\right]$
$\displaystyle={\mathbb{E}}\left[\langle(G^{\star}-\hat{G})\hat{u},\hat{u}\rangle\right]$
$\displaystyle\geq\Delta{\color[rgb]{0,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@gray@stroke{0}\pgfsys@color<EMAIL_ADDRESS>
Note however that underestimating the propagator $G^{\star}$ will lead to
underestimation of price impact costs and will result in a faster execution
strategy and hence create additional trading costs. This monotonicity relation
is demonstrated in Figure 4 of [2], where it is shown that smaller propagators
allow for faster execution rate. Next, we derive a lower bound for the
spurious correlation contribution in (A.2), by considering a special case that
will help to simplify the computations and keep this example tractable. We
assume that the price impact is temporary, i.e.,
$\hat{G}=\hat{\kappa}\mathbb{I}_{M}$, and $G^{\star}=\kappa\mathbb{I}_{M}$,
for some constants $0<\hat{\kappa}<\kappa$. Note that in (A.2) we have
$\Delta=\kappa-\hat{\kappa}$. We further make the assumption that the
unobserved asset price has the following dynamics,

$P_{t}=\mu t+\sigma W_{t},$

where $W_{t}$ is a Brownian motion and $\mu,\sigma>0$ are constants.
Specifically $\mu$ is a trend that can be regarded as a deterministic signal.
In this case one can solve the continuous time analog of the execution problem
(2.4) (see [12, Exercise E.6.3, Chapter 6.9]) to get,

$u_{t}^{\star}=\frac{x_{0}}{T}+\frac{\mu}{4\kappa}(T-2t),\quad 0\leq t\leq T.$

In [14, Table 2 , Section 4.3], the price impact coefficient was estimated
from proprietary dataset of real transactions executed by a large investment
bank, where it was found that $\kappa\approx 10^{-10}$. Since in a bullish
market a daily stock return of $2\%$ is quite common we have $\mu T\approx
0.02$. Therefore, if we wish to execute $x_{0}=100$ stocks within one day of
$8$ hours of trading and the basic time unit is seconds, we have $T=28800$ and
it is clear that $\frac{x_{0}}{T}\ll\frac{\mu}{2\hat{\kappa}}(T-2t)$, except
for a small interval $T/2\pm O(\kappa/\mu)$. Outside this interval we have,

$\frac{\hat{u}_{t}}{u_{t}^{\star}}=\frac{\frac{x_{0}}{T}+\frac{\mu}{2\hat{\kappa}}(T-2t)}{\frac{x_{0}}{T}+\frac{\mu}{2\kappa}(T-2t)}\approx\frac{\kappa}{\hat{\kappa}}.$
(A.3)

Hence from (A.3) and (A.2) it follows that

SpurCor
$\displaystyle={\mathbb{E}}\left[\langle(G^{\star}-\hat{G})\hat{u},\hat{u}\rangle\right]$
$\displaystyle\geq(\kappa-\hat{\kappa})\int_{0}^{T}(\hat{u}_{t})^{2}\mathrm{d}t$
$\displaystyle\approx(\kappa-\hat{\kappa})\left(\frac{\kappa}{\hat{\kappa}}\right)^{2}\int_{0}^{T}(u^{\star}_{t})^{2}\mathrm{d}t,$
(A.4)

where we have neglected the aforementioned small interval in which
$\hat{u}_{t}/u_{t}^{\star}\approx 1$ with a smaller order contribution.

We learn from (A.4) that underestimating the price impact has an amplifying
effect of order $(\kappa/\hat{\kappa})^{2}>1$ on the transaction costs when a
greedy strategy is implemented.

In fact in this simple case we can compare the pessimistic strategy $u^{(1)}$
to the greedy strategy $\hat{u}$. Recall that,

$J(u;\kappa\mathbb{I}_{M})=\kappa\int_{0}^{T}u^{2}_{t}\mathrm{d}t+\mu\int_{0}^{T}u_{t}\mathrm{d}t,$

so the (optimal) uncertainty quantifier in Definition 1.1 takes the form
$\Gamma(u)=\Delta\int_{0}^{T}u_{t}^{2}\mathrm{d}t$ and the pessimistic cost
functional in (1.6) is

$J(u;\hat{\kappa}\mathbb{I}_{M})=(\hat{\kappa}+\Delta)\int_{0}^{T}u^{2}_{t}\mathrm{d}t+\mu\int_{0}^{T}u_{t}\mathrm{d}t.$

The minimiser $u^{(1)}$ of $J(u;\hat{\kappa}\mathbb{I})$ is given by

$u_{t}^{(1)}=\frac{x_{0}}{T}+\frac{\mu}{4(\hat{\kappa}+\Delta)}(T-2t),\quad
0\leq t\leq T.$

Since $\hat{\kappa}+\Delta=\kappa$ we get $u^{(1)}=u^{\star}$ so the
pessimistic strategy outperforms the greedy strategy $\hat{u}$ due to the
optimality of $u^{\star}$ with respect to $J(\cdot;\kappa\mathbb{I}_{M})$.
Note that in practice we do not know $\kappa$ (or $\Delta$). Therefore, we
need to choose a $\Gamma$ which may be not be as sharp as in this example. In
Section 3 we present further examples for spurious correlation costs in the
presence of trading signals and more general propagators.

We recall that $u^{(1)}$ denotes the optimal strategy with respect to the
pessimistic cost functional (1.6). In the following simple example, we show
how our choice of $\ell_{1}(V_{N,\lambda},\cdot)$ in (1.8) as a
$\delta$-uncertainty quantifier (see (1.6)) helps to keep the pessimistic
optimal strategy $u^{(1)}$ close to trajectories which are frequently observed
in the dataset.

######  Example A.2 (Effect of $\ell_{1}$-penalty function on order
execution).

The role of the penalty $\ell_{1}$ in (1.8) is to discourage the strategies to
visit out-of-distribution actions. We will illustrate this effect in the
following simple example. Consider a dataset of $N$ buy metaorders, such that
each order is executed over two even time bins, that is $M=2$. The dataset
contains the following strategies $u^{(i)}=(u^{(i)}_{1},u_{2}^{(i)})^{\top}$,
for $i=1,\ldots,N$. It is well known that in buy execution problems without
signals we should have

$u^{(i)}_{1}\gg u^{(i)}_{2}\geq 0,\quad\textrm{for all }i=1,\ldots,N,$ (A.5)

due to risk aversion terms and inventory constraints (see e.g., [12, Chapter
6.5, Figure 6.2]). For our example we choose for simplicity $\lambda=1$ in
$V_{N,\lambda}$ and hence

$V_{N,1}=\sum_{i=1}^{N}u^{(i)}(u^{(i)})^{\top}+\mathbb{I}_{M}=\begin{pmatrix}\sum_{i=1}^{N}(u_{1}^{(i)})^{2}+1&\sum_{i=1}^{N}u_{1}^{(i)}u_{2}^{(i)}\\\
\sum_{i=1}^{N}u_{1}^{(i)}u_{2}^{(i)}&\sum_{i=1}^{N}(u_{2}^{(i)})^{2}+1\end{pmatrix}.$

It follows that we can write,

$V_{N,1}=\begin{pmatrix}a_{1}&a_{2}\\\
a_{2}&a_{3}\end{pmatrix},\quad\textrm{for some }0<a_{3}<a_{2}<a_{1},$

where from the fact that $a_{3}<a_{2}$ follows from $u^{(i)}_{1}\gg
u^{(i)}_{2}$. Our assumptions also imply $a_{1}a_{3}>a_{2}^{2}$. Hence,
$V_{N,1}$ is invertible and we have,

$V^{-1}_{N,1}=\frac{1}{a_{1}a_{3}-a_{2}^{2}}\begin{pmatrix}a_{3}&-a_{2}\\\
-a_{2}&a_{1}\end{pmatrix}=:\begin{pmatrix}b_{1}&b_{2}\\\
b_{2}&b_{3}\end{pmatrix}\quad\textrm{ where }b_{2}<0<b_{1}<b_{3}.$

Now let $v=(v_{1},v_{2})^{\top}$ and consider

$\ell^{2}_{1}(v,V^{-1}_{N,1})=v^{\top}V^{-1}_{N,1}v=b_{1}v_{1}^{2}+2b_{2}v_{1}v_{2}+b_{3}v_{2}^{2}.$

Since according to our assumption we expect $b_{1}\ll b_{3}$, it follows that
$\ell_{1}$ will penalize strategies $v$ where $v_{1}\ll v_{2}$, which are not
covered by the dataset (see (A.5)).

##  Appendix B Volterra kernel estimation for a noisy dataset

In this example, we construct the trading dataset as follows: the strategies
$u_{t_{i}}^{(n)}$ for each $n\in\\{1,\ldots,N\\}$ and $i\in\\{1,\ldots,M\\}$
are i.i.d. samples from a normal distribution with mean 50 and standard
deviation $9$. We consider the propagator
$G^{\star}_{i,j}=\frac{\kappa}{(t_{i}-t_{j}+1)^{\beta}}$ $i\geq j$, with
$\kappa=0.01$ and $\beta=0.4$, and apply (2.16) to estimate it using observed
price trajectories. For the estimation procedure, we set $\lambda=10^{-3}$,
$N=252$, $M=78$. Numerical results are presented in Figure 4, which indicate
that (2.16) recovers the true propagator reasonably well.

Figure 4:  The exact propagator $G^{\star}$ (left), the estimated propagator
(right) and the relative error (bottom) of the Volterra estimator by observing
noisy trading strategies.

## References

  * Abbasi-Yadkori et al. [2011] Y. Abbasi-Yadkori, D. Pál, and C. Szepesvári.  Improved algorithms for linear stochastic bandits.  _Advances in neural information processing systems_ , 24, 2011.
  * Abi Jaber and Neuman [2022] E. Abi Jaber and E. Neuman.  Optimal liquidation with signals: the general propagator case.  _arXiv:2211.00447_ , 2022.
  * Abi Jaber et al. [2023] E. Abi Jaber, E. Neuman, and M. Voss.  Equilibrium in functional stochastic games with mean-field interaction.  _arXiv:2306.05433_ , 2023.
  * Alfonsi and Schied [2013] A. Alfonsi and A. Schied.  Capacitary measures for completely monotone kernels via singular control.  _SIAM Journal on Control and Optimization_ , 51(2):1758–1780, 2013.  doi: 10.1137/120862223.  URL https://doi.org/10.1137/120862223.
  * Alfonsi et al. [2012] A. Alfonsi, A. Schied, and A. Slynko.  Order book resilience, price manipulation, and the positive portfolio problem.  _SIAM Journal on Financial Mathematics_ , 3(1):511–533, 2012.  ISSN 1945-497X.  doi: 10.1137/110822098.  URL http://dx.doi.org/10.1137/110822098.
  * Almgren and Chriss [1999] R. Almgren and N. Chriss.  Value under liquidation.  _Risk_ , 12:61–63, 1999.
  * Almgren and Chriss [2000] R. Almgren and N. Chriss.  Optimal execution of portfolio transactions.  _Journal of Risk_ , 3(2):5–39, 2000.
  * Bonnans and Shapiro [2013] J. F. Bonnans and A. Shapiro.  _Perturbation analysis of optimization problems_.  Springer Series in Operations Research and Financial Engineering. Springer Science & Business Media, 1 edition, 2013.
  * Bouchaud et al. [2004] J.-P. Bouchaud, Y. Gefen, M. Potters, and M. Wyart.  Fluctuations and response in financial markets: the subtle nature of ‘random’ price changes.  _Quantitative finance_ , 4(2):176–190, 2004.
  * Bouchaud et al. [2018] J.-P. Bouchaud, J. Bonart, J. Donier, and M. Gould.  _Trades, Quotes and Prices: Financial Markets Under the Microscope_.  Cambridge University Press, 2018.  doi: 10.1017/9781316659335.
  * Brigo et al. [2022] D. Brigo, F. Graceffa, and E. Neuman.  Price impact on term structure.  _Quantitative Finance_ , 22(1):171–195, 01 2022\.  doi: 10.1080/14697688.2021.1983201.  URL https://doi.org/10.1080/14697688.2021.1983201.
  * Cartea et al. [2015] Á. Cartea, S. Jaimungal, and J. Penalva.  _Algorithmic and High-Frequency Trading (Mathematics, Finance and Risk)_.  Cambridge University Press, 1 edition, Oct. 2015.  ISBN 1107091144.  URL http://www.amazon.com/exec/obidos/redirect?tag=citeulike07-20&path=ASIN/1107091144.
  * Chang et al. [2021] J. Chang, M. Uehara, D. Sreenivas, R. Kidambi, and W. Sun.  Mitigating covariate shift in imitation learning via offline data with partial coverage.  _Advances in Neural Information Processing Systems_ , 34:965–979, 2021.
  * Collin-Dufresne et al. [2020] P. Collin-Dufresne, K. Daniel, and M. Saglam.  Liquidity regimes and optimal dynamic asset allocation.  _Journal of Financial Economics_ , 136(2):379–406, 2020.
  * Cont et al. [2023] R. Cont, E. Neuman, and A. Micheli.  Fast and slow optimal trading with exogenous information.  _arXiv:2210.01901_ , 2023.
  * Davies [2019] G. Davies.  The great bull market reaches its 10th birthday.  Available at https://www.ft.com/content/4f941406-5157-11e9-b401-8d9ef1626294., 2019.
  * Diamond and Boyd [2016] S. Diamond and S. Boyd.  CVXPY: A Python-embedded modeling language for convex optimization.  _Journal of Machine Learning Research_ , 17(83):1–5, 2016.
  * Forde et al. [2022] M. Forde, L. Sánchez-Betancourt, and B. Smith.  Optimal trade execution for Gaussian signals with power-law resilience.  _Quantitative Finance_ , 22(3):585–596, 03 2022\.  doi: 10.1080/14697688.2021.1950919.  URL https://doi.org/10.1080/14697688.2021.1950919.
  * Fruth et al. [2014] A. Fruth, T. Schöneborn, and M. Urusov.  Optimal trade execution and price manipulation in order books with time-varying liquidity.  _Mathematical Finance_ , 24:651–695, 2014.  URL http://ssrn.com/paper=1925808.
  * Gatheral [2010] J. Gatheral.  No-dynamic-arbitrage and market impact.  _Quantitative Finance_ , 10(7):749–759, 08 2010\.  doi: 10.1080/14697680903373692.  URL https://doi.org/10.1080/14697680903373692.
  * Gatheral et al. [2011] J. Gatheral, A. Schied, and A. Slynko.  Exponential resilience and decay of market impact.  In F. Abergel, B. Chakrabarti, A. Chakraborti, and M. Mitra, editors, _Econophysics of Order-driven Markets_ , pages 225–236. Springer-Verlag, 2011\.
  * Gatheral et al. [2012] J. Gatheral, A. Schied, and A. Slynko.  Transient linear price impact and Fredholm integral equations.  _Mathematical Finance_ , 22(3):445–474, 2012\.  doi: https://doi.org/10.1111/j.1467-9965.2011.00478.x.  URL https://onlinelibrary.wiley.com/doi/abs/10.1111/j.1467-9965.2011.00478.x.
  * Gohberg et al. [1990] I. Gohberg, S. Goldberg, and M. A. Kaashoek.  _Classes of linear operators_ , volume 49.  Birkhauser, 1990.
  * Jegadeesh [1990] N. Jegadeesh.  Evidence of predictable behavior of security returns.  _The Journal of Finance_ , 45(3):881–898, 1990\.  ISSN 00221082, 15406261.  URL http://www.jstor.org/stable/2328797.
  * Jin et al. [2021] Y. Jin, Z. Yang, and Z. Wang.  Is pessimism provably efficient for offline RL?  In _International Conference on Machine Learning_ , pages 5084–5096. PMLR, 2021.
  * Kidambi et al. [2020] R. Kidambi, A. Rajeswaran, P. Netrapalli, and T. Joachims.  Morel: Model-based offline reinforcement learning.  In H. Larochelle, M. Ranzato, R. Hadsell, M. Balcan, and H. Lin, editors, _Advances in Neural Information Processing Systems_ , volume 33, pages 21810–21823. Curran Associates, Inc., 2020.  URL https://proceedings.neurips.cc/paper_files/paper/2020/file/f7efa4f864ae9b88d43527f4b14f750f-Paper.pdf.
  * Lehalle and Neuman [2019] C. A. Lehalle and E. Neuman.  Incorporating signals into optimal trading.  _Finance and Stochastics_ , 23(2):275–311, 2019\.  doi: 10.1007/s00780-019-00382-7.  URL https://doi.org/10.1007/s00780-019-00382-7.
  * Li et al. [2021] J. Li, C. Tang, M. Tomizuka, and W. Zhan.  Dealing with the unknown: Pessimistic offline reinforcement learning.  _arXiv:2111.05440_ , 2021.
  * Madhavan [2003] A. Madhavan.  The Russell reconstitution effect.  _Financial Analysts Journal_ , 59(4):51–64, 2003\.  doi: 10.2469/faj.v59.n4.2545.  URL https://doi.org/10.2469/faj.v59.n4.2545.
  * Micheli and Neuman [2022] A. Micheli and E. Neuman.  Evidence of crowding on Russell 3000 reconstitution events.  _Market Microstructure and Liquidity_ , 2022.  doi: 10.1142/S2382626620500094.  URL https://doi.org/10.1142/S2382626620500094.
  * Micheli et al. [2023] A. Micheli, J. Muhle-Karbe, and E. Neuman.  Closed-loop Nash competition for liquidity.  _To appear in Mathematical Finance_ , 2023.  doi: https://doi.org/10.1111/mafi.12409.  URL https://onlinelibrary.wiley.com/doi/abs/10.1111/mafi.12409.
  * Neuman and Voss [2022] E. Neuman and M. Voss.  Optimal signal-adaptive trading with temporary and transient price impact.  _SIAM Journal on Financial Mathematics_ , 13(2):551–575, 2022.  doi: 10.1137/20M1375486.  URL https://doi.org/10.1137/20M1375486.
  * Neuman and Voss [2023] E. Neuman and M. Voss.  Trading with the crowd.  _Mathematical Finance_ , 33(3):548–617, 2023\.  doi: https://doi.org/10.1111/mafi.12390.  URL https://onlinelibrary.wiley.com/doi/abs/10.1111/mafi.12390.
  * Neuman and Zhang [2023] E. Neuman and Y. Zhang.  Statistical learning with sublinear regret of propagator models.  _arXiv preprint arXiv:2301.05157_ , 2023.
  * Obizhaeva and Wang [2013] A. A. Obizhaeva and J. Wang.  Optimal trading strategy and supply/demand dynamics.  _Journal of Financial Markets_ , 16(1):1 – 32, 2013.  ISSN 1386-4181.  doi: http://dx.doi.org/10.1016/j.finmar.2012.09.001.  URL http://www.sciencedirect.com/science/article/pii/S1386418112000328.
  * Randewich [2019] N. Randewich.  Wall Street’s oldest-ever bull market turns 10 years old.  Available at https://uk.reuters.com/article/usa-stocks-bull/rpt-wall-streets-oldest-ever-bull-market-turns-10-years-old-idUKL1N20V1RJ, 2019\.
  * Staff [2019] W. S. J. Staff.  Inside a Decadelong Bull Run.  Available at https://www.wsj.com/articles/inside-a-decade-long-bull-run-11552041001, 2019\.
  * Tóth et al. [2017] B. Tóth, Z. Eisler, and J. P. Bouchaud.  The short-term price impact of trades is universal.  _Market Microstructure and Liquidity_ , 03(02):1850002, 2017.  doi: 10.1142/S2382626618500028.  URL https://doi.org/10.1142/S2382626618500028.
  * Uehara and Sun [2021] M. Uehara and W. Sun.  Pessimistic model-based offline reinforcement learning under partial coverage.  _arXiv:2107.06226_ , 2021.
  * Vodret et al. [2022] M. Vodret, I. Mastromatteo, B. Tóth, and M. Benzaquen.  Do fundamentals shape the price response? A critical assessment of linear impact models.  _Quantitative Finance_ , 22(12):2139–2150, 2022\.  doi: 10.1080/14697688.2022.2114376.  URL https://doi.org/10.1080/14697688.2022.2114376.