# A maximal inequality for local empirical processes under weak dependence

Luis A. F. Alvarez  Alvarez: Post-doctoral researcher, Fundação Getúlio
Vargas.  and  Cristine C. X. Pinto  Pinto: Full professor, Insper.

(Date: June 2023)

###### Abstract.

We introduce a maximal inequality for a local empirical process under strongly
mixing data. Local empirical processes are defined as the (local) averages
$\frac{1}{nh}\sum_{i=1}^{n}\mathbf{1}\\{x-h\leq X_{i}\leq x+h\\}f(Z_{i})$,
where $f$ belongs to a class of functions, $x\in\mathbb{R}$ and $h>0$ is a
bandwidth. Our nonasymptotic bounds control estimation error uniformly over
the function class, evaluation point $x$ and bandwidth $h$. They are also
general enough to accomodate function classes whose complexity increases with
$n$. As an application, we apply our bounds to function classes that exhibit
polynomial decay in their uniform covering numbers. When specialized to the
problem of kernel density estimation, our bounds reveal that, under weak
dependence with exponential decay, these estimators achieve the same (up to a
logarithmic factor) sharp uniform-in-bandwidth rates derived in the iid
setting by Einmahl and Mason (2005).

##  1\. Introduction

In this paper, we introduce a maximal inequality for a local empirical process
under exponential decay of the sample $\alpha$-mixing coefficients. We provide
a nonasymptotic bound on an Orlicz norm of the uniform estimation error of the
local-average process $\frac{1}{nh}\sum_{i=1}^{n}\mathbf{1}\\{x-h\leq
X_{i}\leq x+h\\}f(Z_{i})$, where uniformity holds simultaneously over the
evaluation point $x\in\mathbb{R}$, bandwidth $h\in[a_{n},b_{n})$, and function
$f\in\mathcal{F}$, with $a_{n}\leq b_{n}$ being positive constants and
$\mathcal{F}$ a function class. The nonasymptotic nature of our results allows
for function classes whose complexity, as measured by their uniform covering
numbers, increases with $n$. The latter is especially useful for applications
in high-dimensional statistics (see Belloni et al. (2017) for an example). We
also discuss how to extend our results to multidimensional $x$ and
subgeometric decay of mixing coefficients.

We then apply our results to functional classes $\mathcal{F}$ that exhibit
polynomial decay in their uniform covering numbers. This class is particulary
interesting, as polynomial decay in uniform entropy is ensured by a finite VC
dimension (van der Vaart and Wellner, 1996, Theorem 2.6.7). When specialized
to the problem of kernel density estimation, our results show that, under
exponential decay in the mixing coefficients, kernel estimators achieve the
same, up to a logarithmic factor, sharp uniform-in-bandwidth rates obtained by
Einmahl and Mason (2005) in the iid setting.

To the best of our knowledge, the closest paper to ours is Escanciano (2020),
who provides uniform-in-bandwidth asymptotic rates for local empirical
processes under stationarity, a $\beta$-mixing condition and polynomial decay
in bracketing entropy. There are some important differences between his
approach and ours, though. First, Escanciano’s analysis is asymptotic, thus
not applicable to function classes of increasing complexity. Second, while the
author uses bracketing entropy to control complexity, our analysis relies on
uniform covering numbers. Finally, his main result (Theorem 2.1) leads to a
rate of $a_{n}^{-1/2}$ over the function classes encompassed by his setting.
When compared to our application in Section 3, we see that our resulting rates
are logarithmic in $a_{n}$, thus offering a great improvement over his results
in kernel-type problems.

The remainder of this paper is organized as follows. Section 2 introduces our
main result. Section 3 applies it to function classes with uniform entropy
displaying polynomial decay. Section 4 concludes.

##  2\. A maximal inequality for local empirical processes

We present our main result in the theorem below. In what follows, we define,
for some $p\geq 0$, $\psi_{p}(x)\coloneqq\exp(-x^{p})-1$; and, for a random
variable $Z$, the Orlicz norm with respect to $\psi_{p}$ is given by:

$\lVert
Z\rVert_{\psi_{p}}\coloneqq\inf\left\\{C>0:\mathbb{E}\left[\psi_{p}\left(\frac{|Z|}{C}\right)\right]<\infty\right\\}\,.$

Moreover, for a sequence $\\{Y_{i}\\}_{i=1}^{n}$ of random variables defined
on a common probability space $(\Omega,\Sigma,\mathbb{P})$, we define the
$\alpha$-mixing coefficients as, for each $k\in\mathbb{N}$.

$\alpha(k)\coloneqq\sup_{t\in\mathbb{N}}\sup_{A\in\sigma(Y_{1},\ldots,Y_{t}),B\in\sigma(Y_{t+k},Y_{t+k+1},\ldots)}|\mathbb{P}[A\cap
B]-\mathbb{P}[A]\mathbb{P}[B]|\,.$

######  Theorem 1.

Let $\\{(X_{i},Z_{i})\\}_{i\in\mathbb{N}}$ be a sequence of random variables
defined on a common probability space $(\Omega,\Sigma,\mathbb{P})$, with
mixing coefficients satisfying, for some $c>0$, $\alpha(i)\leq\exp(-2ci)$
$\forall i\in\mathbb{N}$. Suppose that each $X_{i}$ is real valued, with
common distribution fucntion $G$ that admits a bounded Lebesgue density $g$.
Suppose that each $Z_{i}$ takes values on a measurable space
$(\mathcal{Z},\Sigma)$. Let $\mathcal{F}$ be a pointwise measurable class of
real-valued functions with domain $\mathcal{Z}$. Define:

$S_{n}(x,f;h)\coloneqq\frac{1}{\sqrt{nh}}\sum_{i=1}^{n}\mathbf{1}\\{x-h\leq
X_{i}\leq x+h\\}f(Z_{i})\,,\quad
u\in[0,1],f\in\mathcal{F},h\in[a_{n},b_{n})\,,$

where $0<a_{n}\leq b_{n}$ are positive constants, with $\frac{1}{\lVert
g\rVert_{\infty}n}\leq b_{n}<\frac{1}{\lVert g\rVert_{\infty}}$.

Suppose that the class of functions $\mathcal{F}$ admits a measurable and
bounded envelope $\kappa$. Assume that, for some $q\in(1,\infty]$,
$\omega(\epsilon)\coloneqq\sup_{P\in\mathcal{P}(\mathcal{Z})}\mathcal{N}(2\epsilon\lVert\kappa\rVert_{\infty},\mathcal{F},\lVert\cdot\rVert_{q,P})$,
is finite for every $\epsilon>0$, where $\lVert
f\rVert_{P,q}=(P|f|^{q})^{1/q}$ and the supremum is taken over all probability
measures on $(\mathcal{Z},\Sigma)$. Then, for every $n\geq 2$:

$\left\lVert\sup_{x\in\mathbb{R},h\in[a_{n},b_{n}),f\in\mathcal{F}}|S_{n}(x,f;h)-\mathbb{E}[S_{n}(x,f;h)]|\right\rVert_{\psi_{1}}\leq
C\frac{\lVert\kappa\rVert_{\infty}}{\sqrt{na_{n}}}+\frac{D\lVert\kappa\rVert_{\infty}}{\sqrt{na_{n}}}\sum_{l=\lceil-\log_{2}(\lVert
g\rVert_{\infty}b_{n})\rceil}^{\lceil\log_{2}(n)\rceil}\inf_{\delta\geq
0}\psi_{l}(\delta)\,,$ (1)

where $C>0$ is an absolute consant; the constant $D>0$ depends solely on $c$;
and:

$\displaystyle\psi_{l}(\delta)=\left(\log(n)^{2}[(l+1)\lor\log(\omega(\delta))]+\sqrt{\frac{ln}{2^{l}}\lor
1}\sqrt{[(l+1)\lor\log(\omega(\delta))]}\right)+$ (2) $\displaystyle
n\delta\left[\left(n^{-1}\log(n)^{2}(l+1)+\sqrt{\frac{l}{2^{l}n}\lor\frac{1}{n^{2}}}\sqrt{(l+1)}\right)^{\frac{{q-1}}{q}}+\frac{1}{2^{\frac{(q-1)}{q}(l+1)}}\right]\,.$

The proof of Theorem 1 is deferred to Appendix A. It relies on a chaining
argument due to Rio (2017, Proposition 7.2), which we couple with the
Bernstein inequality of Merlevède et al. (2009) to help achieve control of the
error of the localizing function $\mathbf{1}\\{x\leq X\leq x+h\\}$ uniformly
over the bandwidth. Note that our bound is nonasymptotic, with the constant
$D$ depending solely on the decay parameter $c$. Our bound also depends on
minimising the functions $\psi_{l}(\delta)$. This minimisation involves
finding a “small” $\delta$ that is able to ensure appropriate control of the
uniform entropy $\omega(\delta)$. In the next section, we show that, for
classes with polynomial decay in their uniform entropy, a useul upper bound to
this infimum can be computed.

######  Remark 1 (Extension to nid $X$).

It is possible to extend our results to the setting where the distribution of
$X_{i}$ varies with $i$. In this case, one has to adapt Lemma 1 in Appendix A
to allow for a different localization parameter $K_{i}$ for every $i$. Since
this brings clutter to the proofs and does not reveal any new insights, for
ease of exposition, we focus on the case where the $X_{i}$ are identically
distributed. Observe that, in the statement of Theorem 1, we do not require
the $Z_{i}$ to be identically distributed.

######  Remark 2 (Extension to multidimensional $X$).

It is possible to extend our results to the setting where $X$ is
multidimensional by relying on Lemma 7.2 of Rio (2017). This result decomposes
the difference between a $d$-dimensional cdf at two points as a sum of $d$
differences of the $d$ marginal cdfs at different evaluation points. Using
this result, we may extend Lemma 1 in Appendix A to the multivariate setting,
and use it to prove a multivariate version of Theorem 1.

######  Remark 3 (Extension to unbounded envelopes and subgeometric decay).

The statement of Theorem 1 requires the function class to display bounded
envelope; and mixing coefficients to exhibit exponential (geometric) decay.
These assumptions can be relaxed by replacing the Bernstein inequality of
Merlevède et al. (2009) with a result due to Merlevède et al. (2010). In this
case, a tradeoff between the tail behaviour of the envelope function and the
decay of mixing coefficients emerges. Consequently, we achieve control over
the uniform estimation error under a different Orlicz norm.

##  3\. Application: function classes with polynomial decay

In this section, we apply our results to classes of functions that exhibit
polynomial decay in their covering numbers. Specifically, in the setting of
Theorem 1, we take $q=2$ and consider a sequence of bounded function classes
$\\{\mathcal{F}_{n}\\}_{n\in\mathbb{N}}$ with corresponding envelope
$\kappa_{n}$, whose uniform entropy numbers $\omega_{n}$ satisfy:

$\omega_{n}(\delta)\leq
C_{n}\left(\frac{1}{\delta}\right)^{v_{n}}\,,\quad\forall\delta\in(0,1],$

for positive constants $C_{n}>0$,$v_{n}>0$, $n\in\mathbb{N}$. We also assume
the envelopes to be uniformly bounded, with
$\sup_{n\in\mathbb{N}}\lVert\kappa_{n}\rVert_{\infty}\leq\phi<\infty$. Suppose
that $na_{n}\to\infty$. In this case, we may find a constant $G>0$ such that,
for every $n$ above a certain threshold, $\lceil-\log_{2}(\lVert
g\rVert_{\infty}b_{n})\rceil\leq l\leq\lceil\log_{2}(n)\rceil$; and
$\delta<\exp\left(\frac{1}{v_{n}}(\log(C_{n})-(l+1))\right)$:

$\displaystyle\psi_{l}(\delta)\leq
G\log(n)^{2}(\log(C_{n})-v_{n}\log(\delta))+G\sqrt{\frac{ln}{2^{l}}}\sqrt{(\log(C_{n})-v_{n}\log(\delta))}+Gn\delta\frac{\log(n)^{3/2}}{2^{(1/2)(l+1)}}\,,$
(3)

Consider the choice:

$\delta_{l}=\frac{C_{n}^{1/v_{n}}}{2^{((1/v_{n}))(l+1)}\sqrt{n}(\lVert g\rVert
b_{n})^{1/v_{n}}\log(n)^{3/2}}\,.$

Assuming that:

$\frac{\log(n)^{3}\log\log n}{\sqrt{Ta_{t}}}\to 0\,.$

In this case, we obtain the rate:

$\displaystyle\left\lVert\sup_{x\in\mathbb{R},h\in[a_{n},b_{n}),f\in\mathcal{F}}|S_{n}(x,f;h)-\mathbb{E}[S_{n}(x,f;h)]|\right\rVert_{\psi_{1}}=$
(4) $\displaystyle
O\left(\sqrt{\frac{b_{n}}{a_{n}}}(C_{n}^{1/v_{n}}\lor\sqrt{-\log(\lVert
g\rVert b_{n})v_{n}\log(n)}\lor-\log(\lVert g\rVert b_{n}))\right)\,.$

Our rate explicitly accomodates for function classes of increasing complexity.
If the sequence of functions has finite (albeit possibly increasing) VC
dimension, then Theorem 2.6.7 of van der Vaart and Wellner (1996) shows that
the $v_{n}$ and $C_{n}$ may be taken such that $C_{n}^{1/v_{n}}$ is bounded.
In this case, the complexity of the function class only affects the rate
through the $\sqrt{v_{n}}$ term, where $v_{n}$ scales linearly with the VC
dimension.

Finally, we consider the problem of kernel density estimation. In this case,
the function class is the same for every $n\in\mathbb{N}$. If the bandwidths
are taken such that $a_{n}\asymp b_{n}$, then our rate simplifies to:

$\sqrt{-\log(a_{n})}\sqrt{\log(n)\lor-\log(a_{n})}\,.$

In contrast, Remark 2 in Einmahl and Mason (2005) shows that, in the iid
setting, one can achieve the rate:

$\sqrt{\log\log(n)\lor-\log(a_{n})}\,,$

which coincides with our rate except for logarithmic factors. Note that, if we
consider, as it is usually done in practice, polynomial bandwidths, i.e
$a_{n}=Cn^{-\alpha}$ for $\alpha>0$, then our rate simplifies to $\log(n)$,
whereas Einmahl and Mason’s collapses to $\sqrt{\log(n)}$.

##  4\. Concluding remarks

In this paper, we introduced a maximal inequality for the uniform estimation
error of a local empirical process under strongly mixing data, where
uniformity holds simultaneously over the function class, bandwidth and
evaluation point. Our nonasymptotic bounds accomodate function classes with
increasing complexity, which is a useful feature for “high-dimensional”
statistical analyses. As an application, we computed our bounds to function
classes that exhibit polynomial decay in their uniform entropy. When
specialized to the kernel density estimation problem, these results show that
our bound leads to the same optimal rates derived by Einmahl and Mason (2005)
in the iid setting.

More generally, we view our results as a first step in the development of
rigorous uniform inference tools in local estimation problems under weak
dependence and data-driven bandwidths. Specifically, one may combine our
results with couplings in the weakly dependent setting (e.g. Cattaneo et al.,
2022) to devise test statistics that control size uniformly over the
evaluation point $x$. An example is the construction of uniform-in-$x$
confidence bands for local polynomial quantile regression estimators with time
series data. We intend to study such procedures in future research.

##  Appendix A Proof of Theorem 1

We first prove a preliminary result for uniform random variables.

######  Lemma 1.

Let $\\{(U_{i},Z_{i})\\}_{i\in\mathbb{N}}$ be a sequence of random variables
defined on a common probability space $(\Omega,\Sigma,\mathbb{P})$, with
mixing coefficients satisfying, for some $c>0$, $\alpha(i)\leq\exp(-2ci)$
$\forall i\in\mathbb{N}$. Suppose that each $U_{i}$ is uniformly distributed
on $[0,1]$, and that each $Z_{i}$ takes values on a measurable space
$(\mathcal{Z},\Sigma)$. Let $\mathcal{F}$ be a nonnegative pointwise
measurable class of real-valued functions with domain $\mathcal{Z}$. Define:

$Z_{n}(u,f)\coloneqq\frac{1}{\sqrt{n}}\left(\sum_{i=1}^{n}\mathbf{1}\\{U_{i}\leq
u\\}f(Z_{i})-\mathbb{E}[\mathbf{1}\\{U_{i}\leq u\\}f(Z_{i})]\right)\,,\quad
u\in[0,1],f\in\mathcal{F}\,.$

Suppose that the class of functions $\mathcal{F}$ admits a measurable and
bounded envelope $\kappa$. Assume that, for some $q\in(1,\infty]$,
$\omega(\epsilon)\coloneqq\sup_{P\in\mathcal{P}(\mathcal{Z})}\mathcal{N}(2\epsilon\lVert\kappa\rVert_{\infty},\mathcal{F},\lVert\cdot\rVert_{q,P})$,
is finite for every $\epsilon>0$, where $\lVert
f\rVert_{P,q}=(P|f|^{q})^{1/q}$ and the supremum is taken over all probability
measures on $(\mathcal{Z},\Sigma)$. Then, for every $n\geq 2$ and
$K\in\mathbb{N}$ such that $K\leq\lceil\log_{2}(n)\rceil$:

$\left\lVert\sup_{u\in[0,1],f\in\mathcal{F}}|Z_{n}(u,f)-Z_{n}(\Pi_{K}(u),f)|\right\rVert_{\psi_{1}}\leq\frac{\lVert\kappa\rVert_{\infty}}{\sqrt{n}}+D\lVert\kappa\rVert_{\infty}\sum_{l=K+1}^{\lceil\log_{2}(n)\rceil}\inf_{\delta\geq
0}\psi_{l}(\delta)\,,$

where the constant $D>0$ depends solely on $c$; for $J\in\mathbb{N}$,
$\Pi_{J}(u)=\frac{\lfloor 2^{J}u\rfloor}{2^{J}}$; and:

$\displaystyle\psi_{l}(\delta)=\left(n^{-1/2}\log(n)^{2}[(l+1)\lor\log(\omega(\delta))]+\sqrt{\frac{l}{2^{l}}\lor\frac{1}{n}}\sqrt{[(l+1)\lor\log(\omega(\delta))]}\right)+$
$\displaystyle\sqrt{n}\delta\left[\left(n^{-1}\log(n)^{2}(l+1)+\sqrt{\frac{l}{2^{l}n}\lor\frac{1}{n^{2}}}\sqrt{(l+1)}\right)^{\frac{{q-1}}{q}}+\frac{1}{2^{\frac{(q-1)}{q}(l+1)}}\right]\,.$

###### Proof.

We begin by adapting the chaining argument in the proof of Proposition 7.2. of
Rio (2017). Fix $f\in\mathcal{F}$. Observe that for $L\geq K$, we may write:

$\sup_{u\in[0,1]}|Z_{n}(u,f)-Z_{n}(\Pi_{K}(u),f)|\leq\sum_{l=K+1}^{L}\Delta_{l}(f)+\Delta^{*}_{L}(f)\,,$

where
$\Delta_{l}(f)=\max_{k\in\\{1,\ldots,2^{l}\\}}|Z_{n}((k-1)/2^{l},f)-Z_{n}(k/2^{l},f)|$;
and $\Delta^{*}_{L}(f)=\sup_{u\in[0,1]}|Z_{n}(\Pi_{L}(u),f)-Z_{n}(\Pi(u),f)|$.
Observe that:

$-\lVert\kappa\rVert_{\infty}\frac{\sqrt{n}}{2^{L}}\leq
Z_{n}(\Pi(u))-Z_{n}(\Pi_{L}(u))\leq\Delta_{L}(f)+\lVert\kappa\rVert_{\infty}\frac{\sqrt{n}}{2^{L}}\,.$

From the above, we obtain that:

$\sup_{u\in[0,1]}|Z_{n}(u,f)-Z_{n}(\Pi_{K}(u),f)|\leq
2\sum_{l=K+1}^{L}\Delta_{l}(f)+\lVert\kappa\rVert_{\infty}\frac{\sqrt{n}}{2^{L}}\,,$

and, by taking $L=\lceil\log_{2}(n)\rceil$, we arrive at:

$\displaystyle\sup_{u\in[0,1]}|Z_{n}(u,f)-Z_{n}(\Pi_{K}(u),f)|\leq
2\sum_{l=K+1}^{\lceil\log_{2}(n)\rceil}\Delta_{l}(f)+\lVert\kappa\rVert_{\infty}\frac{1}{\sqrt{n}}\,.$
(5)

The previous argument holds for a fixed $f$. We achieve uniformity as follows.
For a given sequence $\\{\delta_{l}\\}_{l=K+1}^{\lceil\log_{2}(n)\rceil}$, we
may write, for each $l$:

$\displaystyle\sup_{f\in\mathcal{F}}\Delta_{l}(f)\leq\max_{f\in
T_{l}}\max_{k\in\\{1,\ldots,2^{l}\\}}|Z_{n}((k-1)/2^{l},f)-Z_{n}(k/2^{l},f)|$
(6) $\displaystyle+\sup_{f\in\mathcal{F}:\lVert
f-f^{\prime}\rVert_{q,\hat{\mathbb{P}}_{n}}\lor\lVert
f-f^{\prime}\rVert_{q,\bar{\mathbb{P}}_{n}}\leq\delta_{l}}\max_{k\in\\{1,\ldots,2^{l}\\}}|Z_{n}((k-1)/2^{l},f)-Z_{n}(k/2^{l},f)-(Z_{n}((k-1)/2^{l},f^{\prime})-Z_{n}(k/2^{l},f^{\prime}))|\,,$

with $T_{l}$ being, simultaneously, a
$2\delta_{l}\lVert\kappa\rVert_{\infty}$-cover of $\mathcal{F}$ with respect
to $\lVert\cdot\rVert_{q,\bar{\mathbb{P}}_{n}}$ and
$\lVert\cdot\rVert_{q,\hat{\mathbb{P}}_{n}}$, where
$\lVert\kappa\rVert_{q,\bar{\mathbb{P}}_{n}}=\left(\frac{1}{n}\sum_{i=1}^{n}\mathbb{E}[|f(Z_{i})|^{q}]\right)^{1/q}$;
and $\hat{\mathbb{P}}_{n}$ denotes the empirical measure
$\frac{1}{n}\sum_{i=1}^{n}\delta_{Z_{i}}$. Observe that, by the assumption in
the statement of the lemma, $|T_{l}|\leq\omega(\delta_{l})^{2}$; and that the
set $T_{l}$ is random. By filling the $T_{l}$ with constant functions equal to
zero whenever $|T_{l}|<\omega(\delta_{l})^{2}$, we may assume, without loss of
generality, that $|T_{l}|=\omega(\delta_{l})^{2}$ and we may thus order the
random elements in
$T_{l}=\\{f_{1,l},f_{2,l},\ldots,f_{\omega(\delta_{l})^{2},l}\\}$. We thus
consider $T_{l}$ to be a set of $\omega(\delta_{l})^{2}$ random functions.

We will deal with each term on the right-hand side of (6) separately. Consider
the first term. For a given $l\in\mathbb{N}$ and some
$k\in\\{1,\ldots,2^{l}\\}$ and $f\in T_{l}$, define
$Y_{i,l}(k,f)=\frac{1}{\sqrt{n}}f(Z_{i})\mathbf{1}\\{(k-1)/2^{l}\leq U_{i}\leq
k/2^{l}\\}$. Rio’s covariance inequality (Rio, 2017, Theorem 1.1) yields:

$\sum_{s\geq
i}^{\infty}|\mathbb{C}(Y_{i,l}(j,f),Y_{s,l}(j,f))|\leq\frac{4\lVert\kappa\rVert_{\infty}^{2}}{n}\sum_{s=0}^{\infty}\alpha(s)\land\frac{1}{2^{l}}\leq\frac{4\lVert\kappa\rVert_{\infty}^{2}}{n}\left(\frac{l\log(2)}{c2}\frac{1}{2^{l}}+\frac{1}{2^{l-1}}\frac{1}{1-\exp(-2c)}\right)\eqqcolon
v_{l}^{2}n^{-1}\lVert\kappa\rVert^{2}\,.$

It then follows by Theorem 2 of Merlevède et al. (2009) that, fo a constant
$C$ depending only on $c$, we have that, for any $n\geq 2$ and every
$l\in\mathbb{N}$, $k\in\\{1,2,\ldots,2^{l}\\}$ and $f\in T_{l}$:

$\displaystyle\mathbb{P}[|Z_{n}((k-1)/2^{l},f)-Z_{n}(k/2^{l},f)|\geq
x]\leq\exp\left(\frac{Cx^{2}}{v^{2}_{l}\lVert\kappa\rVert_{\infty}^{2}+\lVert\kappa\rVert^{2}_{\infty}n^{-1}+x\lVert\kappa\rVert_{\infty}n^{-1/2}\log(n)^{2}}\right),\quad\forall
x\geq 0\,.$

By Lemma 2.2.10 of van der Vaart and Wellner (1996), there is a constant $K$
depending only on $c$ such that, for every $l\in\mathbb{N}$ and $n\geq 2$:

$\displaystyle\left\lVert\max_{f\in
T_{l}}\max_{k\in\\{1,\ldots,2^{l}\\}}|Z_{n}((k-1)/2^{l},f)-Z_{n}(k/2^{l},f)|\right\rVert_{\psi_{1}}\leq$
(7) $\displaystyle
K\lVert\kappa\rVert_{\infty}(n^{-1/2}\log(n)^{2}\log(1+2^{l}\omega(\delta_{l})^{2})+\sqrt{v_{l}^{2}+n^{-1}}\sqrt{\log(1+2^{l}\omega(\delta_{l})^{2})})\,.$

Next, we deal with the second term in (6). A simple argument reveals that:

$\displaystyle\max_{k\in\\{1,\ldots,2^{l}\\}}\sup_{f\in\mathcal{F}:\lVert
f-f^{\prime}\rVert_{q,\hat{\mathbb{P}}_{n}}\lor\lVert
f-f^{\prime}\rVert_{q,\bar{\mathbb{P}}_{n}}\leq\delta_{l}}|Z_{n}((k-1)/2^{l},f)-Z_{n}(k/2^{l},f)-(Z_{n}((k-1)/2^{l},f^{\prime})-Z_{n}(k/2^{l},f^{\prime}))|\leq$
$\displaystyle
2\lVert\kappa\rVert_{\infty}\delta_{l}n^{(2-q)/2q}\max_{k\in\\{1,\ldots
2^{l}\\}}\left|\sum_{i=1}^{n}\mathbf{1}\\{(k-1)/2^{l}\leq U_{i}\leq
k/2^{l}\\}\right|^{\frac{q-1}{q}}+2\lVert\kappa\rVert_{\infty}\delta_{l}\frac{\sqrt{n}}{2^{\left(\frac{q-1}{q}\right)l}}\,,$

from which it follows that:

$\displaystyle\left\lVert\max_{k\in\\{1,\ldots,2^{l}\\}}\sup_{f\in\mathcal{F}:\lVert
f-f^{\prime}\rVert_{q,\hat{\mathbb{P}}_{n}}\lor\lVert
f-f^{\prime}\rVert_{q,\bar{\mathbb{P}}_{n}}\leq\delta_{l}}|Z_{n}((k-1)/2^{l},f)-Z_{n}(k/2^{l},f)-(Z_{n}((k-1)/2^{l},f^{\prime})-Z_{n}(k/2^{l},f^{\prime}))|\right\rVert_{\psi_{1}}\leq$
(8) $\displaystyle
2S\lVert\kappa\rVert_{\infty}\delta_{l}n^{1/2q}\left\lVert\max_{k\in\\{1,\ldots
2^{l}\\}}\left|\frac{1}{\sqrt{n}}\sum_{i=1}^{n}(\mathbf{1}\\{(k-1)/2^{l}\leq
U_{i}\leq
k/2^{l}\\}-1/2^{l})\right|\right\rVert_{\psi_{1}}^{\frac{q-1}{q}}+S4\lVert\kappa\rVert_{\infty}\delta_{l}\frac{\sqrt{n}}{2^{\left(\frac{q-1}{q}\right)l}}\leq$
$\displaystyle
2S\delta_{l}\lVert\kappa\rVert_{\infty}\sqrt{n}\left(K(n^{-1}\log(n)^{2}\log(1+2^{l})+\sqrt{n^{-1}(v_{l}^{2}+n^{-1})}\sqrt{\log(1+2^{l})})^{\frac{q-1}{q}}+\frac{2}{2^{\left(\frac{q-1}{q}\right)l}}\right)\,,$

where $S>0$ is an absolute constant, and the last inequality follows from
replicating the preceding argument replacing $T_{l}$ with a constant function
equal to one. Using that
$\log(1+2^{l}\omega(\delta_{l})^{2})\leq\log(2^{l+1}\omega(\delta_{l})^{2})=(l+1)\log(2)+2\log(\omega(\delta_{l}))$
and $v^{2}_{l}\leq M\frac{l}{2^{l}}$ for a constant $M$ depending only on $c$,
we can simplify both (7) and (8). Then, plugging these back onto (6),
optimising, and then plugging back onto (5); we obtain the desired result. ∎

We are now ready to prove the main result. We begin by proving the result for
a nonnegative class of functions with bounded envelope. Start by noticing
that:

$\displaystyle|S(x,f;h)-\mathbb{E}[S(x,f;h)]|\leq\left|\frac{1}{\sqrt{nh}}\sum_{i=1}^{n}(\mathbf{1}\\{x\leq
X_{i}\leq x+h\\}f(Z_{i})-\mathbb{E}[\mathbf{1}\\{x\leq X_{i}\leq
x+h\\}f(Z_{i})])\right|+$ (9)
$\displaystyle\left|\frac{1}{\sqrt{nh}}\sum_{i=1}^{n}(\mathbf{1}\\{x-h\leq
X_{i}\leq x\\}f(Z_{i})-\mathbb{E}[\mathbf{1}\\{x-h\leq X_{i}\leq
x\\}f(Z_{i})])\right|+$
$\displaystyle\left|\frac{1}{\sqrt{nh}}\sum_{i=1}^{n}\mathbf{1}\\{X_{i}=x\\}\right|\,.$

We deal with each term separately. Consider the first term. Observe that,
since each $X_{i}$ is continuously distributed, we have that:

$\displaystyle\left|\frac{1}{\sqrt{nh}}\sum_{i=1}^{n}(\mathbf{1}\\{x\leq
X_{i}\leq x+h\\}f(Z_{i})-\mathbb{E}[\mathbf{1}\\{x\leq X_{i}\leq
x+h\\}f(Z_{i})])\right|=$ (10)
$\displaystyle\left|\frac{1}{\sqrt{nh}}\sum_{i=1}^{n}(\mathbf{1}\\{G(x)<U_{i}\leq
G(x+h)\\}f(Z_{i})-\mathbb{E}[\mathbf{1}\\{G(x)<U_{i}\leq
G(x+h)\\}f(Z_{i})])\right|+$
$\displaystyle\left|\frac{1}{\sqrt{nh}}\sum_{i=1}^{n}\mathbf{1}\\{U_{i}=G(x)\\}f(Z_{i})\right|\,,$

where each $U_{i}\coloneqq G(X_{i})$ is uniformly distributed. The last term
on the right hand side of (10) is, with probability one, bounded above by
$\frac{\lVert g\rVert_{\infty}}{\sqrt{a_{n}n}}$ uniformly over $h$, $f$ and
$x$. Next, note that, for any $x\in\mathbb{R}$ and $h>0$, trivially:

$G(x+h)-G(x)\leq\lVert g\rVert_{\infty}h\,.$

Combining this fact with continuity of $G$ reveals that, for each
$x\in\mathbb{R}$, the set:

$\Xi_{x}=\\{G(x+h)-G(x):0\leq h<b_{n}\\}\,,$

is an interval contained in

$[0,\lVert g\rVert b_{n})\,.$

Therefore, setting $K=\lfloor-\log_{2}(\lVert g\rVert_{\infty}b_{n})\rfloor$
yields that:

$\displaystyle\sup_{x\in\mathbb{R}}\sup_{a_{n}\leq
h<b_{n}}\sup_{f\in\mathcal{F}}\left|\frac{1}{\sqrt{nh}}\sum_{i=1}^{n}(\mathbf{1}\\{x\leq
X_{i}\leq x+h\\}f(Z_{i})-\mathbb{E}[\mathbf{1}\\{x\leq X_{i}\leq
x+h\\}f(Z_{i})])\right|\leq$
$\displaystyle\frac{1}{\sqrt{a_{n}}}\sup_{u\in[0,1]}\sup_{s\in[0,{2}^{-K})}\sup_{f\in\mathcal{F}}\left|\frac{1}{\sqrt{n}}\sum_{i=1}^{n}(\mathbf{1}\\{u<U_{i}\leq
u+s\\}f(Z_{i})-\mathbb{E}[\mathbf{1}\\{u<U_{i}\leq
u+s\\}f(Z_{i})])\right|+\frac{\lVert g\rVert_{\infty}}{\sqrt{a_{n}n}}\,.$

But we then note that:

$\displaystyle\sup_{u\in[0,1]}\sup_{s\in[0,{2}^{-K})}\sup_{f\in\mathcal{F}}\left|\frac{1}{\sqrt{n}}\sum_{i=1}^{n}(\mathbf{1}\\{u<U_{i}\leq
u+s\\}f(Z_{i})-\mathbb{E}[\mathbf{1}\\{u<U_{i}\leq
u+s\\}f(Z_{i})])\right|\leq$ (11) $\displaystyle\sup_{k\in\\{1,2,\ldots
2^{K}\\}}\sup_{s\in[0,{2}^{-K})}\sup_{f\in\mathcal{F}}\left|\frac{1}{\sqrt{n}}\sum_{i=1}^{n}\left(\mathbf{1}\left\\{\frac{(k-1)}{2^{K}}<U_{i}\leq\frac{(k-1)}{2^{K}}+s\right\\}f(Z_{i})-\mathbb{E}\left[\mathbf{1}\left\\{\frac{(k-1)}{2^{K}}<U_{i}\leq\frac{(k-1)}{2^{K}}+s\right\\}f(Z_{i})\right]\right)\right|+$
$\displaystyle\sup_{k\in\\{1,2,\ldots
2^{K}\\}}\sup_{s\in[0,{2}^{-K})}\sup_{f\in\mathcal{F}}\left|\frac{1}{\sqrt{n}}\sum_{i=1}^{n}\left(\mathbf{1}\left\\{\frac{(k-1)}{2^{K}}-s<U_{i}\leq\frac{(k-1)}{2^{K}}\right\\}f(Z_{i})-\mathbb{E}\left[\mathbf{1}\left\\{\frac{(k-1)}{2^{K}}-s<U_{i}\leq\frac{(k-1)}{2^{K}}\right\\}f(Z_{i})\right]\right)\right|\,.$

and, since both $U_{i}$ and $1-U_{i}$ are uniformly distributed, Lemma 1
provides control over the Orlicz norm of both terms on the right-hand side of
(11). An analogous argument enables controls of the uniform error of the
second term in (9). Finally, as for the third term in (9), we note that its
uniform error is bounded above by
$\frac{\lVert\kappa\rVert_{\infty}}{\sqrt{na_{n}}}$. Applying Lemma 1 and
combining the resulting terms proves Theorem 1 for nonnegative classes.

To extend the result to general bounded classes, consider the constant
function $\underline{f}(z)=-\lVert\kappa\rVert_{\infty}$ for all
$z\in\mathcal{Z}$. Next, decompose the function class as
$\mathcal{F}=(\mathcal{F}-\underline{f})+\\{\underline{f}_{n}\\}$. The class
$(\mathcal{F}-\underline{f})$ is nonnegative, admits bounded envelope
$2\lVert\kappa\rVert$ and same covering numbers as $\mathcal{F}$. Therefore,
the previous result applies to it. The second class of functions is a
singleton and $\underline{f}_{n}$ has constant sign; therefore the previous
result also applies to it, with $w_{n}(\delta)=1$ and hence $\delta=0$.
Combining with the bound on $(\mathcal{F}-\underline{f}_{n})$ yields the
desired result.

## References

  * Belloni et al. (2017) Belloni, A., V. Chernozhukov, I. Fernández-Val, and C. Hansen (2017).  Program evaluation and causal inference with high-dimensional data.  Econometrica 85(1), 233–298.
  * Cattaneo et al. (2022) Cattaneo, M. D., R. P. Masini, and W. G. Underwood (2022).  Yurinskii’s coupling for martingales.
  * Einmahl and Mason (2005) Einmahl, U. and D. M. Mason (2005).  Uniform in bandwidth consistency of kernel-type function estimators.  The Annals of Statistics 33(3), 1380 – 1403.
  * Escanciano (2020) Escanciano, J. C. (2020).  Uniform rates for kernel estimators of weakly dependent data.
  * Merlevède et al. (2009) Merlevède, F., M. Peligrad, and E. Rio (2009).  Bernstein inequality and moderate deviations under strong mixing conditions.  In Institute of Mathematical Statistics Collections, pp. 273–292. Institute of Mathematical Statistics.
  * Merlevède et al. (2010) Merlevède, F., M. Peligrad, and E. Rio (2010, June).  A bernstein type inequality and moderate deviations for weakly dependent sequences.  Probability Theory and Related Fields 151(3-4), 435–474.
  * Rio (2017) Rio, E. (2017).  Asymptotic Theory of Weakly Dependent Random Processes.  Springer Berlin Heidelberg.
  * van der Vaart and Wellner (1996) van der Vaart, A. W. and J. A. Wellner (1996).  Weak Convergence and Empirical Processes.  Springer New York.