where the residual is $o_{p}\left(n^{-1}\right)+o_{p}\left(\sqrt{V_{\tau
n}^{*}/n}\right)+o_{p}\left(V_{\tau
n}^{*}/\sqrt{n}\right)+O_{p}(n^{-3/2})+o_{p}(n^{-1}\sqrt{V_{\tau n}^{*}})$.
The fourth and fifth terms of the residual are $o_{p}(n^{-1})$ and
$o_{p}(n^{-1/2}\sqrt{V_{\tau n}^{*}})$, and hence asymptotically negligible.
The leading term in (53) is $O_{p}\left(\max\left\\{1/n,\sqrt{V_{\tau
n}^{*}/n}\right\\}\right)$.

∎

  * Proof of Lemma 3.

Let $\varphi_{i}=\varphi(Y_{i},D_{i},X_{i},\eta)$ be a realization of the
influence function in (14) and compute $\mathbb{V}[\varphi_{i}\mid
D_{i}=d,X_{i}=x]=4(\tau(x)-\tau_{av})^{2}\left[\frac{d\sigma_{1}^{2}(x)}{p(x)^{2}}+\frac{(1-d)\sigma_{0}^{2}(x)}{(1-p(x))^{2}}\right]$
and $\mathbb{E}[\varphi_{i}\mid D_{i}=d,X_{i}=x]=(\tau(x)-\tau_{av})^{2}$.

By the law of iterated expectations $\mathbb{E}[\varphi_{i}\mid
X_{i}=x]=(\tau(x)-\tau_{av})^{2}$. By applying the law of total variance
recursively, $\mathbb{V}(\varphi_{i})=\mathbb{V}(\mathbb{E}[\varphi_{i}\mid
X_{i}])+\mathbb{E}[\mathbb{E}[\mathbb{V}(\varphi_{i}\mid D_{i},X_{i})\mid
X]]+\mathbb{E}[\mathbb{V}(\mathbb{E}[\varphi_{i}\mid D_{i},X_{i}]\mid
X_{i})]$. This produces,
$\mathbb{V}(\varphi_{i})=\mathbb{V}((\tau(X_{i})-\tau_{av})^{2})+4\mathbb{E}\left[(\tau(X_{i})-\tau_{av})^{2}\left(\frac{\sigma_{1}^{2}(X_{i})}{p(X_{i})}+\frac{\sigma_{0}^{2}(X_{i})}{(1-p(X_{i}))}\right)\right]$.
∎

  * Proof of Corollary 2.

$\mathbb{V}((\tau(X_{i})-\tau_{av})^{2})\leq\mathbb{E}[(\tau(X_{i})-\tau_{av})^{4}]\leq\kappa^{2}V_{\tau}$.
By the Cauchy-Schwarz inequality, the term
$\mathbb{E}\left[(\tau(X_{i})-\tau_{av})^{2}\left(\frac{\sigma_{1}^{2}(X_{i})}{p(X_{i})}+\frac{\sigma_{0}^{2}(X_{i})}{(1-p(X_{i}))}\right)\right]$
is bounded by $\kappa
V_{\tau}\sqrt{\mathbb{E}\left[\left(\frac{\sigma_{1}^{2}(X_{i})}{p(X_{i})}+\frac{\sigma_{0}^{2}(X_{i})}{(1-p(X_{i}))}\right)^{2}\right]}$.
∎

  * Proof of Lemma 4.

Let $\mathcal{Q}(\theta)$ be the Jacobian of the least squares problem,
defined in (18). Substituting
$\widehat{W}(X_{i},D_{i})^{\prime}e_{4}=(D_{i}-p(X_{i}))\widehat{S}(X_{i})$
and $\lambda(X_{i})=[p(X_{i})(1-p(X_{i}))]^{-1}$ into the definition,
$\beta_{2}\mathcal{Q}(\theta)e_{4}=\frac{1}{n}\sum_{i=1}^{n}\frac{[D_{i}-p(X_{i})]}{p(X_{i})(1-p(X_{i}))}\beta_{2}\widehat{S}(X_{i})(Y_{i}-\widehat{W}(X_{i},D_{i}))$.
By the definition of the nuisance functions in $\tilde{\eta}_{\theta}(x)$ in
(19),
$\widehat{W}(X_{i},D_{i})^{\prime}\theta=\tilde{\mu}_{0,\theta}(X_{i})+D_{i}\tilde{\tau}_{\theta}(X_{i})$,
$\tilde{\tau}_{\theta}(x)=\beta_{1}+\beta_{2}\widehat{S}(X_{i})$, and
$\tilde{\tau}_{\theta}=\beta_{1}+\left[\frac{1}{n}\sum_{i=1}^{n}\widehat{S}(X_{i})\right]\beta_{2}$.
Furthermore, since $\frac{1}{n}\sum_{i=1}^{n}\widehat{S}(X_{i})=0$, then
$\beta_{2}\widehat{S}(X_{i})=\tilde{\tau}_{\theta}(X_{i})-\tilde{\tau}_{av,\theta}$
and

$\beta_{2}\mathcal{Q}(\theta)e_{4}=\frac{1}{n}\sum_{i=1}^{n}\frac{[D_{i}-p(X_{i})]}{p(X_{i})(1-p(X_{i}))}\beta_{2}\widehat{S}(X_{i})(Y_{i}-\tilde{\mu}_{0,\theta}(X_{i})-D_{i}\tilde{\tau}_{\theta}(X_{i})).$

Notice that $D_{i}-p(X_{i})=D_{i}(1-p(X_{i}))+p(X_{i})(1-D_{i})$. This implies
that
$\frac{D_{i}-p(X_{i})}{p(X_{i})}=\frac{D_{i}}{p(X_{i})}-\frac{(1-D_{i})}{1-p(X_{i})}$.
The next step is to substitute the estimated parameters. Then
$\frac{1}{n}\sum_{i=1}^{n}\varphi(Y_{i},D_{i},X_{i},\tilde{\eta}_{\widehat{\theta}_{n}})=\frac{1}{n}\sum_{i=1}^{n}(\tilde{\tau}_{\widehat{\theta}_{n}}(X_{i})-\tilde{\tau}_{av,\widehat{\theta}_{n}})^{2}+\widehat{\beta}_{2n}\mathcal{Q}(\widehat{\theta}_{n})e_{4}$.
The first term simplifies to
$\widehat{\beta}_{2n}^{2}\left[\frac{1}{n}\sum_{i=1}^{n}\widehat{S}(X_{i})^{2}\right]=\widehat{\beta}_{2n}^{2}\widehat{V}_{xn}$.
Finally, $\mathcal{Q}(\widehat{\theta}_{n})=0$ implies that the second term is
zero.

∎

  * Proof of Lemma 5.

The parameter
$\theta=(\alpha_{1},\alpha_{2},\beta_{1},\beta_{2})\in\Theta^{*}$ solves

$\mathbb{E}\left[\lambda(X)W(X,D)Y\right]-\mathbb{E}\left[\lambda(X)W(X,D)W(X,D)^{\prime}\right]\theta=0_{4\times
1}.$ (54)

If there exists a $\theta\in\mathbb{R}^{4}$ such that $\mathbb{E}[Y\mid
D=d,X=x]=\mu_{d}(x)=W(x,d)^{\prime}\theta$, then by the law of iterated
expectations
$\mathbb{E}\left[\lambda(X)W(X,D)Y\right]=\mathbb{E}[\lambda(X)W(X,D)W(X,D)^{\prime}\theta]$.
Such a $\theta$ would automatically satisfy (54). Now suppose that
$S(x)=\tau(x)-\tau_{av}$, $M(x)=\mu_{0}(x)+p(x)\tau(x)$, and
$\theta=(0,1,\tau_{av},1)$. For this choice,
$W(x,d)^{\prime}\theta=0+(\mu_{0}(x)+p(x)\tau(x))+(d-p(x))\tau_{av}+(d-p(x))(\tau(x)-\tau_{av})$.
This simplifies to $W(x,d)^{\prime}\theta=\mu_{0}(x)+d\tau(x)=\mu_{d}(x)$ and
$\theta$ solves (54).

∎

  * Proof of Theorem 3.(i).

Define a vector $\uline{Y}_{nk}\in\mathbb{R}^{n_{k}}$ with the outcomes in
fold $\mathcal{I}_{nk}$, $\uline{W}_{nk}$ be an $n_{k}\times 4$ matrix of
regressors, $\uline{U}_{nk}\in\mathbb{R}^{n_{k}}$ be a vector of errors, and
$\Lambda_{nk}$ be a $n_{k}\times n_{k}$ diagonal matrix with entries
$\\{\lambda(X_{i})\\}$. Let $\uline{\widehat{W}}_{nk}$ be an $n_{k}\times 4$
matrix of generated regressors defined in (22), and let $\Pi_{nk}$ be a
$4\times 4$ diagonal matrix with entries $(1,1,1,V_{xnk}^{-1/2})$. Let
$\theta_{nk}^{*}:=(\alpha_{1nk},\alpha_{2nk},\beta_{1nk},\sqrt{V}_{xnk}\beta_{2nk})$,
and define an infeasible estimator
$\widetilde{\theta}_{nk}^{*}:=\Pi_{nk}^{-1}\widetilde{\theta}_{nk}:=\left(\Pi_{nk}^{\prime}\uline{W}_{nk}^{\prime}\Lambda_{nk}\uline{W}_{nk}\Pi_{nk}/n_{k}\right)^{-1}\left(\Pi_{nk}^{\prime}\uline{W}_{nk}^{\prime}\Lambda_{nk}Y_{nk}/n_{k}\right)$.

Define
$Q_{ww,nk}:=\mathbb{E}[\lambda(X_{i})W_{i}W_{i}^{\prime}\mid\mathcal{I}_{-nk}]$.
By Lemma 11,

$\Pi_{nk}^{\prime}Q_{ww,nk}\Pi_{nk}=\begin{pmatrix}\mathbb{E}[\lambda(X_{i})\mid\mathcal{I}_{-nk}]&\mathbb{E}[\lambda(X_{i})M_{-k}(X_{i})\mid\mathcal{I}_{-nk}]&0&0\\\
\mathbb{E}[\lambda(X_{i})M_{-k}(X_{i})\mid\mathcal{I}_{-nk}]&\mathbb{E}[\lambda(X_{i})M_{-k}(X_{i})^{2}\mid\mathcal{I}_{-nk}]&0&0\\\
0&0&1&0\\\ 0&0&0&1\\\ \end{pmatrix}.$ (55)

By Assumptions 1.(ii), 3.(i) and (ii), $\Pi_{nk}^{\prime}Q_{ww,nk}\Pi_{nk}$
has bounded eigenvalues. Algebraically,
$\frac{\Pi_{nk}^{\prime}\uline{W}_{nk}^{\prime}\Lambda_{nk}\uline{W}_{nk}\Pi_{nk}}{n_{k}}=\frac{1}{n_{k}}\sum_{i\in\mathcal{I}_{nk}}\Pi_{nk}^{\prime}\lambda(X_{i})W_{i}W_{i}^{\prime}\Pi_{nk}$.
Since the terms are conditionally i.i.d, it converges to (55). By Assumptions
3.(iii) and (iv) $\lambda(X_{i})\Pi_{nk}W_{i}U_{i}$ is bounded in the $L_{2}$
norm, and
$\sqrt{n_{k}}(\widehat{\theta}_{nk}^{*}-\theta_{nk})=(\Pi_{nk}^{\prime}Q_{ww,nk}\Pi_{nk})^{-1}\frac{\Pi_{nk}^{\prime}\uline{W}_{nk}^{\prime}\Lambda_{nk}\uline{U}_{nk}}{\sqrt{n_{k}}}+o_{p}(1)$.
Also,
$V_{xnk}^{-1}\widehat{V}_{xnk}-1=\frac{1}{n_{k}}\sum_{i\in\mathcal{I}_{nk}}[V_{xnk}^{-1}S_{-k}(X_{i})^{2}-1]-\left[\frac{1}{n_{k}}\sum_{i\in\mathcal{I}_{nk}}V_{xnk}^{-1/2}S_{-k}(X_{i})\right]^{2}$.
The second term is $O_{p}(n^{-1})$ because the summand has mean zero and unit
variance.

$A_{i}:=\begin{pmatrix}\lambda(X_{i})\Pi_{nk}W_{i}U_{i}\\\
V_{xnk}^{-1}S_{-k}(X_{i})-1\end{pmatrix},\quad
J_{nk}^{*}:=\begin{bmatrix}(\Pi_{nk}^{\prime}Q_{ww,nk}\Pi_{nk})^{-1}&\uline{0}_{4}\\\
\uline{0}_{4}^{\prime}&1\end{bmatrix}.$

By (55), $(\Pi_{nk}^{\prime}Q_{ww,nk}\Pi_{nk})^{-1}$ has a block-diagonal
structure, with a one in the bottom-right cell, and hence
$\Upsilon[J_{nk}^{*}]^{-1}=\Upsilon$. This means that

$\displaystyle\sqrt{n_{k}}\Upsilon\begin{pmatrix}\widetilde{\theta}_{n}^{*}-\theta_{n}^{*}\\\
V_{xnk}^{-1}\widehat{V}_{xnk}-1\end{pmatrix}$
$\displaystyle=\Upsilon\left[J_{nk}^{*}\right]^{-1}\left[\frac{1}{\sqrt{n}_{k}}\sum_{i\in\mathcal{I}_{nk}}A_{i}\right]+o_{p}(1)$
(56)
$\displaystyle=\frac{1}{\sqrt{n_{k}}}\sum_{i\in\mathcal{I}_{nk}}\begin{pmatrix}V_{xnk}^{-1/2}\lambda(X_{i})(D_{i}-p(X_{i}))S_{-k}(X_{i})U_{i}\\\
V_{xnk}^{-1/2}S_{-k}(X_{i})^{2}-1\end{pmatrix}+o_{p}(1).$

Define
$\widehat{\tau}_{nk,av}:=\tfrac{1}{n_{k}}\sum_{i\in\mathcal{I}_{nk}}\widehat{\tau}_{-nk}(X_{i})$
and
$\tau_{nk,av}:=\mathbb{E}[\widehat{\tau}_{-nk}(X_{i})\mid\mathcal{I}_{-nk}]$.
Algebraically, $\uline{\widehat{W}}_{nk}=\uline{W}_{nk}(I_{4\times
4}+\Delta_{nk})$, where
$\Delta_{nk}=[\uline{0}_{4},\uline{0}_{4},\uline{0}_{4},e_{3}(\widehat{\tau}_{nk,av}-\tau_{nk,av})]$
and $e_{3}=[0,0,1,0]^{\prime}$. The bias in centering the fourth column of
$\widehat{W}_{nk}$ is a scalar times the third regressor. Substituting
$\uline{\widehat{W}}_{nk}=\uline{W}_{nk}\Pi_{nk}\Pi_{nk}^{-1}(I+\Delta_{nk})$,

$\widehat{\theta}_{nk}^{*}=\left(\Pi_{nk}^{-1}(I+\Delta_{nk})\Pi_{nk}\right)^{-1}\left(\Pi_{nk}^{\prime}\uline{W}_{nk}^{\prime}\Lambda_{nk}\uline{W}_{nk}\Pi_{nk}\right)^{-1}\left(\Pi_{nk}^{\prime}\uline{W}_{nk}^{\prime}\Lambda_{nk}\uline{Y}_{nk}\right),$
(57)

which simplifies to
$\widehat{\theta}_{nk}^{*}=\left(\Pi_{nk}^{-1}(I+\Delta_{nk})\Pi_{nk}\right)^{-1}\tilde{\theta}_{nk}^{*}$.
Since $\Delta_{nk}\Delta_{nk}=0_{4\times 4}$, then
$\left(\Pi_{nk}^{-1}(I+\Delta_{nk})\Pi_{nk}\right)^{-1}=\left(I+V_{xnk}^{-1/2}\Delta_{nk}\right)^{-1}=I-V_{xnk}^{-1/2}\Delta_{nk}$,
and

$\begin{pmatrix}\widehat{\beta}_{2n}^{*}\\\
V_{xnk}^{-1}\widehat{V}_{xnk}\end{pmatrix}:=\Upsilon\begin{pmatrix}\left(\Pi_{nk}^{-1}\widetilde{\Delta}_{nk}\Pi_{nk}\right)^{-1}&\uline{0}_{4\times
1}\\\ \uline{0}_{1\times
4}&1\end{pmatrix}\begin{pmatrix}\widetilde{\theta}_{nk}^{*}\\\
V_{xnk}^{-1}\widehat{V}_{xnk}\end{pmatrix}=\Upsilon\begin{pmatrix}\widetilde{\theta}_{nk}^{*}\\\
V_{xnk}^{-1}\widehat{V}_{xnk}\end{pmatrix}.$ (58)

The second equality follows from the fact that the selection matrix $\Upsilon$
only extracts the fourth and fifth rows of the vector. We can plug-in (58) and
the identity
$\Upsilon[\theta_{nk}^{*^{\prime}},1]^{\prime}=[\beta_{2nk}^{*},1]^{\prime}$
into (56). By Assumptions 4 and 5, the observations are conditionally i.i.d.
given $\mathcal{I}_{-nk}$, and $\Omega_{nk}$ has bounded eigenvalues. Then by
the Lindeberg-Feller CLT,

$\sqrt{n_{k}}\Omega_{nk}^{-1/2}\begin{pmatrix}\widehat{\beta}_{2nk}^{*}-\beta_{2nk}^{*}\\\
V_{xnk}^{-1}\widehat{V}_{xnk}-1\end{pmatrix}\mid\mathcal{I}_{-nk}\to^{d}\mathcal{N}(0_{2\times
1},I_{2\times 2}).$ (59)

∎

  * Proof of Theorem 3.(ii).

Let $\tilde{\Delta}_{nk}=I+\Delta_{nk}$, where the non-zero term in
$\Delta_{nk}$ is an average with mean zero and variance $V_{xnk}$. Under the
assumptions of part (a), $\widehat{\Pi}_{nk}\Pi_{nk}=I+o_{p}(1)$ and
$\Pi_{nk}\tilde{\Delta}_{nk}\Pi_{nk}^{-1}=I+V_{xnk}^{-1/2}\Delta_{nk}=I+o_{p}(1)$.
Decomposing
$\widehat{\Pi}_{nk}\uline{\widehat{W}}_{nk}^{\prime}\Lambda_{nk}\uline{\widehat{W}}_{nk}\widehat{\Pi}_{nk}$
in a similar way to (57),

$(\widehat{\Pi}_{nk}\tilde{\Delta}_{nk}^{\prime}\Pi_{nk}^{-1})(\Pi_{nk}^{\prime}\uline{W}_{nk}^{\prime}\Lambda_{nk}\uline{W}_{nk}\Pi_{nk})(\Pi_{nk}^{-1}\tilde{\Delta}_{nk}\widehat{\Pi}_{nk})=\Pi_{nk}^{\prime}Q_{ww,nk}\Pi_{nk}+o_{p}(1)$

. Hence $\widehat{J}_{nk}=J_{nk}^{*}+o_{p}(1)$. Substituting
$\uline{\widehat{W}}_{nk}$, into the upper-left block of $\widehat{H}_{nk}$,

$(\widehat{\Pi}_{nk}\tilde{\Delta}_{nk}^{\prime}\Pi_{nk}^{-1})\left[\frac{1}{n_{k}}\sum_{i\in\mathcal{I}_{-nk}}\widehat{U}_{i}^{2}\lambda(X_{i})^{2}\Pi_{nk}^{\prime}W_{i}W_{i}^{\prime}\Pi_{nk}\right](\Pi_{nk}^{-1}\tilde{\Delta}_{nk}\widehat{\Pi}_{nk}).$
(60)

As before the outer terms are $I+o_{p}(1)$. For the inner terms we apply (57),
$\widehat{U}_{i}=Y_{i}-\widehat{W}_{i}^{\prime}\widehat{\theta}_{nk}=Y_{i}-W_{i}^{\prime}\tilde{\Delta}_{nk}\widehat{\theta}_{nk}=Y_{i}-W_{i}^{\prime}\Pi_{nk}\Pi_{nk}^{-1}\tilde{\Delta}_{nk}\Pi_{nk}\Pi_{nk}^{-1}\widehat{\theta}_{nk}=Y_{i}-W_{i}^{\prime}\Pi_{nk}^{\prime}\widehat{\theta}_{nk}^{*}=U_{i}-W_{i}^{\prime}\Pi_{nk}(\widehat{\theta}_{nk}^{*}-\theta_{nk}^{*})$.
The inner term of (60) is
$\mathbb{E}[U_{i}^{2}\lambda(X_{i})^{2}\Pi_{nk}^{\prime}W_{i}W_{i}^{\prime}\Pi_{nk}\mid\mathcal{I}_{-nk}]+o_{p}(1)$
under Assumption 1.(ii) (overlap) and 3 (bounds on moments of $U_{i}$ and
$\Pi_{nk}W_{i}$). In particular, 3.(iv) ensures that
$\mathbb{E}[\|\Pi_{nk}W_{i}\|^{4}\mid\mathcal{I}_{-nk}]$ is uniformly bounded.

Define
$\bar{S}_{nk}:=\frac{1}{n_{k}}\sum_{i\in\mathcal{I}_{-nk}}S_{-k}(X_{i})$ and
$\widehat{S}_{-k}(X_{i})=\widehat{\tau}_{-k}(X_{i})-\frac{1}{n_{k}}\sum_{i\in\mathcal{I}_{-nk}}\widehat{\tau}_{-k}(X_{i})$.
Adding/subtracting the mean,
$\widehat{S}_{-k}(X_{i})=S_{-k}(X_{i})-\bar{S}_{nk}$. Algebraically,
$\frac{1}{n_{k}}\sum_{i\in\mathcal{I}_{nk}}\widehat{T}_{i}^{2}=\widehat{V}_{xnk}^{-2}\left(\frac{1}{n_{k}}\sum_{i\in\mathcal{I}_{nk}}[\widehat{S}_{-k}(X_{i})^{2}-\widehat{V}_{xnk}]^{2}\right)=\widehat{V}_{xnk}^{-2}\left(\frac{1}{n_{k}}\sum_{i\in\mathcal{I}_{nk}}\widehat{S}_{-k}(X_{i})^{4}-\widehat{V}_{xnk}^{2}\right)$.
This simplifies to
$\widehat{V}_{xnk}^{-2}\left(\frac{1}{n_{k}}\sum_{i\in\mathcal{I}_{nk}}[S_{-k}(X_{i})-\bar{S}_{nk}]^{4}\right)-1$.
By a binomial expansion,

$\displaystyle\frac{1}{n_{k}}\sum_{i\in\mathcal{I}_{nk}}\frac{S_{-k}(X_{i})^{4}}{\widehat{V}_{xnk}^{2}}$
$\displaystyle\quad+\underbrace{\left(\frac{V_{xnk}}{\widehat{V}_{xnk}}\right)^{2}}_{\to^{p}1}\sum_{\ell=1}^{4}{4\choose\ell}(-1)^{\ell}\underbrace{\frac{\bar{S}_{nk}^{\ell}}{V_{xnk}^{\ell/2}}}_{o_{p}(1)}\underbrace{\left[\frac{1}{n_{k}}\sum_{i\in\mathcal{I}_{nk}}V_{xnk}^{2-\ell/2}S_{-k}(X_{i})^{4-\ell}\right]}_{O_{p}(1)}-1.$

By Assumption 3.(iv), the bound on $V_{xnk}$, and the moment bounds for
$V_{xnk}^{-1/2}S_{-k}(X_{i})$, then for $\ell\in\\{1,2,3,4\\}$,
$\left[\frac{1}{n_{k}}\sum_{i\in\mathcal{I}_{nk}}V_{xnk}^{2-\ell/2}S_{-k}(X_{i})^{4-\ell}\right]$
is $O_{p}(1)$. By the weak law of large number,
$V_{xnk}^{-1/2}\bar{S}_{nk}=o_{p}(1)$ and by (59),
$\widehat{V}_{xnk}/V_{xnk}\to^{p}1$. Hence
$\frac{1}{n_{k}}\sum_{i\in\mathcal{I}_{nk}}\widehat{T}_{i}^{2}=\mathbb{E}[V_{xnk}^{-1}S_{-k}(X_{i})^{4}\mid\mathcal{I}_{-nk}]-1+o_{p}(1)=\mathbb{V}(V_{xnk}^{-1/2}S_{-k}(X_{i})^{2})+o_{p}(1)$.
This shows that the diagonals of $\widehat{H}_{nk}$ converge to their
population analogs. The proof of convergence for the off-diagonals is similar.
By the form of $Q_{ww,nk}$ in (55), $\Upsilon J_{nk}^{-1}=\Upsilon$. Hence
$\Upsilon\widehat{J}_{nk}^{-1}\widehat{H}_{nk}\widehat{J}_{nk}\Upsilon^{\prime}=\Upsilon
J_{nk}^{{}^{*}-1}H_{nk}J_{nk}^{*}\Upsilon^{\prime}+o_{p}(1)=\Upsilon
H_{nk}\Upsilon^{\prime}+o_{p}(1)=\Omega_{nk}+o_{p}(1)$.

∎

  * Proof of Lemma 4.

We start by proving that
$\|V_{\tau}(\gamma)-V_{\tau}^{*}(\gamma,\mathcal{I}_{-nk})\|\leq V_{\tau}$.
Since the second term of (33) is always non-negative,
$V_{\tau}^{*}(\gamma,\mathcal{I}_{-nk})\leq V_{\tau}(\gamma)$. Also, since
$\beta_{1}=\tau_{\gamma,av}$ and $\beta_{2}=0$ are part of the feasible set,
then the second term is at most
$V_{\tau}(\gamma)=\mathbb{E}_{\gamma}[(\tau_{\gamma}(X)-\tau_{\gamma,av})^{2}]$.
This shows that $V_{\tau}^{*}(\gamma,\mathcal{I}_{-nk})\geq 0$.

To prove the second bound, we examine the solution to the best linear
projection. Since the folds are independent, conditioning on
$\mathcal{I}_{-nk}$ is only important to be able to handle $S_{-k}(x)$ as a
deterministic function. For ease of exposition, let $F$ denote the conditional
distribution of $\gamma$ given $\mathcal{I}_{-nk}$, and let
$\|g\|_{F,2}=\sqrt{\mathbb{E}_{F}[\|g(X)\|^{2}]}$ denote the $L_{2}$ norm.
Since $\mathbb{E}_{F}[S_{-k}(X)]=0$, the optimal solution is
$\beta_{1F}^{*}=\tau_{F,av}$. When $\mathbb{V}_{F}(S_{-k}(X))=0$, the optimal
$\beta_{2F}^{*}$ is indeterminate, and
$V_{\tau}^{*}(\gamma,\mathcal{I}_{-nk})=0$. Otherwise,

$(\beta_{2F}^{*}-1)=\frac{Cov_{F}(S_{-k}(X),\tau_{F}(X))}{\mathbb{V}_{F}(S_{-k}(X))}-1=\frac{Cov_{F}(S_{-k}(X),\tau_{F}(X)-S_{-k}(X))}{\mathbb{V}_{F}(S_{-k}(X))}.$

By the Cauchy-Schwarz inequality,
$(\beta_{2F}^{*}-1)^{2}\mathbb{V}_{F}(S_{-k}(X))\leq\mathbb{E}_{F}[(\tau_{\gamma}(X)-\tau_{F,av}-S_{-k}(X))^{2}]$.
Rearranging (33) and substituting the optimum:

$\displaystyle
V_{\tau}-V_{\tau}^{*}=\mathbb{E}_{F}[(\tau_{\gamma}(X)-\tau_{av}-S_{-k}(X)-(\beta_{2}^{*}-1)S_{-k}(X))^{2}].$

By the triangle inequality for the $L_{2}$ norm, and the above Cauchy-Schwartz
bound,

$V_{\tau}(\gamma)-V_{\tau}^{*}(\gamma,\mathcal{I}_{-nk})\leq\left[2\|\tau_{F}(X)-\tau_{F,av}-S_{-k}(X)\|_{F,2}\right]^{2}.$
(61)

Substituting
$S_{-k}(x)=\widehat{\tau}_{-k}(x)-\mathbb{E}_{F}[\widehat{\tau}_{-k}(X)]$,
(61) can be rewritten as
$4\|(\tau_{\gamma}(X)-\widehat{\tau}_{-k}(X))-(\mathbb{E}_{F}[\tau_{\gamma}(X)]-\mathbb{E}_{F}[\widehat{\tau}_{-k}(X)])\|^{2}\leq
16\|\widehat{\tau}_{-k}-\tau_{\gamma}\|_{F,2}^{2}$. By the law of iterated
expectations
$\omega(\gamma):=\mathbb{E}_{\gamma}[\|\widehat{\tau}_{-k}(X)-\tau_{\gamma}(X)\|^{2}]=\mathbb{E}_{\gamma}[\|\widehat{\tau}_{-k}-\tau_{\gamma}\|_{F,2}^{2}]$.
Combining the two bounds and applying Jensen’s inequality,

$\mathbb{E}_{\gamma}[|V_{\tau}(\gamma)-V_{\tau}^{*}(\gamma,\mathcal{I}_{-nk})|]\leq\mathbb{E}_{\gamma}[\min\\{16\|\widehat{\tau}_{-k}-\tau\|_{F,2}^{2},V_{\tau}(\gamma)\\}]\leq\min\\{16\omega(\gamma),V_{\tau}(\gamma)\\}.$

∎

  * Proof of Theorem 5 .

Decompose $\Delta_{nk}=\Delta_{nk1}+\Delta_{nk2}$, where
$\Delta_{nk1}:=V_{\tau}(\gamma_{n})-V_{\tau}^{*}(\gamma_{n},\mathcal{I}_{-nk})$,
and $\Delta_{nk2}:=\widehat{V}_{\tau
nk}-V_{\tau}^{*}(\gamma_{n},\mathcal{I}_{-nk})$. First, by Lemma 4 and
Assumption 6,
$\mathbb{E}_{\gamma_{n}}[\|\Delta_{nk1}\|]\leq\min\\{16\omega(\gamma_{n})^{2},V_{\tau}(\gamma_{n})\\}=o(n_{k}^{-1/2})$.
By Markov’s inequality, $\Delta_{nk1}=o_{p}(n_{k}^{-1/2})$. Second,
conditional on a sequence $\\{\mathcal{I}_{-nk}\\}_{n=1}^{\infty}$, by Theorem
3 and Lemma 2,
$\|\Delta_{nk2}\|=O_{p}\left(\max\left\\{\frac{1}{n_{k}},\sqrt{V_{\tau}^{*}(\gamma_{n},\mathcal{I}_{-nk})}/\sqrt{n_{k}}\right\\}\right)$.
Since $V_{\tau}^{*}(\gamma_{n},\mathcal{I}_{-nk})\leq
V_{\tau}(\gamma_{n})=o(1)$, then almost surely, for all $\varepsilon>0$,
$\psi_{n,\varepsilon}(\mathcal{I}_{-nk}):=\mathbb{P}_{\gamma_{n}}(n_{k}^{1/2}\|\Delta_{nk2}\|>\varepsilon\mid\mathcal{I}_{-nk})\to
0$. By iterated expectations and the bounded convergence theorem,
$\lim_{n_{k}\to\infty}\mathbb{P}_{\gamma_{n}}(n_{k}^{1/2}\|\Delta_{nk2}\|<\varepsilon)=\lim_{n_{k}\to\infty}\mathbb{E}_{\gamma_{n}}[\psi_{n,\varepsilon}(\mathcal{I}_{-nk})]=\mathbb{E}_{\gamma_{n}}[\lim_{n_{k}\to\infty}\psi_{n,\varepsilon}(\mathcal{I}_{-nk})]=0$.

If, in addition, $n_{k}^{1/2+\rho}V_{\tau}(\gamma_{n})\to 0$ for
$\rho\in[0,1/2)$, then $n_{k}^{1/2+\rho}\Delta_{nk1}=o_{p}(1)$ (the second
bound dominates). Since $n_{k}^{\rho}V_{\tau}(\gamma_{n})\to 0$ as well and
$\rho<1/2$, then conditional on $\mathcal{I}_{-nk}$,
$n_{k}^{1/2+\rho}\Delta_{nk2}=o_{p}(1)$. We can apply the bounded convergence
theorem once again to show that
$\mathbb{P}_{\gamma_{n}}(n_{k}^{1/2+\rho}\|\Delta_{nk2}\|<\varepsilon)=o(1)$.

∎

  * Proof of Theorem 6.

Consider a sequence of distributions $\\{\gamma_{n}\\}_{n=1}^{\infty}$, with
associated nuisance functions
$(\tau_{n}(x),\mu_{0n}(x),p_{n}(x),\tau_{n,av})$.

Case 1 (Near Homogeneity): $V_{\tau}(\gamma_{n})\to 0$. By the triangle
inequality,

$\displaystyle\begin{split}\sqrt{n_{k}}\left\|\widehat{V}_{\tau
nk}-\frac{1}{n_{k}}\sum_{i\in\mathcal{I}_{nk}}\varphi_{i}\right\|&\quad\leq\underbrace{\sqrt{n_{k}}\left\|\widehat{V}_{\tau
nk}-V_{\tau}(\gamma_{n})\right\|}_{\xi_{n1}}+\underbrace{\sqrt{n_{k}}\left\|\frac{1}{n_{k}}\sum_{i\in\mathcal{I}_{nk}}\varphi_{i}-V_{\tau}(\gamma_{n})\right\|}_{\xi_{n2}}.\end{split}$
(62)

By Theorem 5, $\xi_{n1}=o_{p}(1)$. Since the $\varphi_{i}$ are i.i.d with mean
$V_{\tau}(\gamma_{n})$, then
$\mathbb{E}_{\gamma_{n}}[\xi_{n2}^{2}]=\mathbb{V}_{\gamma_{n}}(\varphi_{i})$.
Let $\sigma_{dn}^{2}(x)=\mathbb{V}_{\gamma_{n}}(Y_{i}\mid X_{i}=x,D_{i}=d)$
and $p_{n}(x)=\mathbb{P}_{\gamma_{n}}(D_{i}\mid X_{i}=x)$. By Lemma 3 and
Corollary 2,
$\mathbb{V}_{\gamma_{n}}(\varphi_{i})\leq\kappa^{2}V_{\tau}(\gamma_{n})^{2}+4\kappa
V_{\tau}(\gamma_{n})\sqrt{\mathbb{E}_{\gamma_{n}}\left[\left(\frac{\sigma_{1n}^{2}(X_{i})}{p_{n}(X_{i})}+\frac{\sigma_{0n}^{2}(X_{i})}{1-p_{n}(X_{i})}\right)\right]}$.
Since $V_{\tau}(\gamma_{n})\to 0$ as $n\to\infty$, then
$\mathbb{E}[\xi_{n2}^{2}]\to 0$ and hence $\xi_{n2}=o_{p}(1)$.

Case 2 (Strong Heterogeneity): $V_{\tau}(\gamma_{n})\to
V_{\tau}\in(0,\infty)$. For fixed $(x,d)$, the cross-fitted regressors are
$\widehat{W}(x,d)=[1,\widehat{\mu}_{0,-k}(x)+p_{n}(x)\widehat{\tau}_{-k}(x),(d-p_{n}(x)),(d-p(x))\widehat{S}_{-k}(x)]$.
Let $e_{\ell}$ be a $4\times 1$ vector with a one in the $\ell^{th}$
coordinate and zero otherwise. Analogous to (19), define the regression
adjusted nuisance functions as

$\displaystyle\widehat{\eta}_{-k,\widehat{\theta}_{nk}}(x)$
$\displaystyle=[e_{1}(\widehat{W}(x,1)-\widehat{W}(x,0))^{\prime}+e_{2}\widehat{W}(x,0)^{\prime}]\widehat{\theta}_{nk}+e_{3}p_{n}(x)+e_{4}\widehat{\tau}_{nk,av}.$
(63)

By applying Lemma 4 to the subsample in $i\in\mathcal{I}_{nk}$,

$\widehat{V}_{\tau
nk}=\frac{1}{n_{k}}\sum_{i\in\mathcal{I}_{k}}\varphi(Y_{i},D_{i},X_{i},\widehat{\eta}_{-k,\widehat{\theta}_{nk}}),$

where
$\eta_{-k,\widehat{\theta}_{nk},r}(x):=\eta(x)+r(\eta_{-k,\widehat{\theta}_{nk}}(x)-\eta(x))$.
The true nuisance function $\eta$ depends on the distribution indexed by $n$,
but we drop the subscript to simplify the notation. By a second-order term in
the Taylor expansion around $r=0$, for some $\tilde{r}\in(0,1)$,

$\displaystyle\begin{split}&\sqrt{n_{k}}(\widehat{V}_{\tau
nk}-V_{\tau}(\gamma_{n}))=\frac{1}{\sqrt{n_{k}}}\sum_{i\in\mathcal{I}_{nk}}[\varphi(Y_{i},D_{i},X_{i},\eta)-V_{\tau}(\gamma_{n})]\\\
&\qquad+\frac{1}{\sqrt{n_{k}}}\sum_{i\in\mathcal{I}_{nk}}\frac{\partial\varphi(Y_{i},D_{i},X_{i},\eta)}{\partial\eta^{\prime}}(\eta_{-k,\widehat{\theta}_{nk}}(X_{i})-\eta(X_{i}))\\\
&\qquad+\frac{1}{n_{k}}\sum_{i=1}^{n}\sqrt{n_{k}}(\eta_{-k,\widehat{\theta}_{nk}}(X_{i})-\eta(X_{i}))^{\prime}\frac{\partial^{2}\varphi(Y_{i},D_{i},X_{i},\eta_{-k,\widehat{\theta}_{nk},\tilde{r}})}{\partial\eta^{\prime}}(\eta_{-k,\widehat{\theta}_{nk}}(X_{i})-\eta(X_{i})).\end{split}$
(64)

Our ultimate goal is to show that the second and third terms of the expansion
are $o_{p}(1)$. To keep the notation concise, let
$\varphi_{i}(\eta):=\varphi(Y_{i},D_{i},X_{i},\eta)$. One of the main
challenges is that the nuisance functions are estimated in multiple steps,
combining information from the $\mathcal{I}_{nk}$ and $\mathcal{I}_{-nk}$
subsamples. The key is to decompose these different sources of uncertainty.
Define
$(\widehat{\lambda}_{nk}-\lambda_{nk}):=[(\widehat{\theta}_{nk}-\theta_{n}),\widehat{\tau}_{nk,av}(\widehat{\theta}_{nk}-\theta_{n}),(\widehat{\tau}_{nk,av}-\tau_{n,av})]^{\prime}$,
where $\theta_{n}=(0,1,\tau_{n,av},1)$. By Lemma 14, there exist matrices
$\Psi_{-nk}(x)$ and $\Delta_{-nk}(x)$ that are
$(x,\mathcal{I}_{-nk})$-measurable, such that for all $x\in\mathcal{X}$,

$\widehat{\eta}_{-k,\widehat{\theta}_{nk}}(x)-\eta(x)=\Psi_{-nk}(x)(\widehat{\lambda}_{nk}-\lambda_{nk})+\Delta_{-nk}(x).$
(65)

Let $e_{\ell}$ be a $4\times 1$ elementary basis vector. Lemma 14 shows that
$e_{3}^{\prime}[\widehat{\eta}_{-k,\widehat{\theta}_{nk}}(x)-\eta(x)]=0$ and
$e_{3}^{\prime}\Delta_{-nk}(x)=0$ because the experimental propensity scores
are known. Lemma 14 also shows that $e_{4}^{\prime}\Delta_{-nk}(x)=0$ by
construction. Substituting (65),

$\displaystyle\frac{1}{\sqrt{n_{k}}}\sum_{i\in\mathcal{I}_{nk}}\frac{\partial\varphi_{i}(\eta)}{\partial\eta^{\prime}}(\eta_{-k,\widehat{\theta}_{nk}}(X_{i})-\eta(X_{i}))=\frac{1}{\sqrt{n_{k}}}\sum_{i\in\mathcal{I}_{nk}}\sum_{\ell=1}^{4}\frac{\partial\varphi_{i}(\eta)}{\partial\eta_{\ell}}e_{\ell}^{\prime}(\eta_{-k,\widehat{\theta}_{nk}}(X_{i})-\eta(X_{i}))$
$\displaystyle=\underbrace{\left[\frac{1}{\sqrt{n_{k}}}\sum_{i\in\mathcal{I}_{nk}}\sum_{\ell\neq
3}\frac{\partial\varphi_{i}(\eta)}{\partial\eta_{\ell}}e_{\ell}^{\prime}\Psi_{-nk}(X_{i})\right]}_{\Lambda_{1nk}}(\widehat{\lambda}_{nk}-\lambda_{nk})+\underbrace{\left[\frac{1}{\sqrt{n_{k}}}\sum_{i\in\mathcal{I}_{nk}}\sum_{\ell=1}^{2}\frac{\partial\varphi_{i}(\eta)}{\partial\eta_{\ell}}e_{\ell}^{\prime}\Delta_{-nk}(X_{i})\right]}_{\Lambda_{2nk}}.$

Lemma 15 implies that $\widehat{\lambda}_{nk}-\lambda_{nk}=o_{p}(1)$.

(i) Prove that $\Lambda_{1nk}=O_{p}(1)$: By Lemma 13.(a) the conditional mean
of
$\frac{\partial\varphi_{i}(\eta)}{\partial(\eta_{1},\eta_{2},\eta_{4})^{\prime}}$
given $(x,\mathcal{I}_{-nk})$ is $[0,0,-2(\tau(x)-\tau_{n,av})]$ and by Lemma
14, $e_{4}^{\prime}\Psi_{-nk}=c^{\prime}$. Also,
$\mathbb{E}_{\gamma_{n}}[\tau_{n}(X_{i})\mid\mathcal{I}_{-nk}]=\mathbb{E}_{\gamma_{n}}[\tau_{n}(X_{i})]=\tau_{n,av}$
by fold independence. By the law of iterated expectations,
$\mathbb{E}_{\gamma_{n}}\left[\frac{\partial\varphi_{i}(\eta)}{\partial\eta_{\ell}}e_{\ell}^{\prime}\Psi_{-nk}(X_{i})\mid\mathcal{I}_{-nk}\right]=0$
for $\ell\in\\{1,2,4\\}$. By Assumption 7,
$\mathbb{E}_{\gamma_{n}}[\|\eta(X_{i})\|^{4}]$ and
$\mathbb{E}_{\gamma_{n}}\left[\left\|U_{i}\right\|^{4}\right]$ are uniformly
bounded by a constant $C<\infty$. By Assumption 1.(ii), $p_{n}(x)$ is
contained in $[\delta,1-\delta]$. Applying Lemma 13.(b),
$\mathbb{E}_{\gamma_{n}}\left[\left\|\partial\varphi_{i}(\eta)/\partial\eta_{\ell}\right\|^{4}\right]^{1/4}\leq(16/\delta)C<\infty$.
By Lemma 14.(d), and the triangle inequality,
$\mathbb{E}_{\gamma_{n}}\left[\left\|e_{\ell}^{\prime}\Psi_{-nk}(X_{i})\right\|^{4}\right]^{1/4}\leq
C[1+\mathbb{E}_{\gamma_{n}}[\|\widehat{\eta}_{-k}(X_{i})\|^{4}]^{1/4}]$. By
the bound in Assumption 7.(iii) and the Cauchy-Schwartz inequality,
$\mathbb{E}_{\gamma_{n}}\left[\|\frac{\partial\varphi_{i}(\eta)}{\partial\eta_{\ell}}e_{\ell}^{\prime}\Psi_{-nk}(X_{i})\|\right]<\infty$.
We can combine these moment bounds to apply Lemma 12.(a), hence
$\Lambda_{1nk}=O_{p}(1)$.

(ii) Prove that $\Lambda_{2nk}=o_{p}(1)$: By Lemma 13.(a),
$\mathbb{E}_{\gamma_{n}}\left[\frac{\partial\varphi_{i}(\eta)}{\partial\eta_{\ell}}e_{\ell}^{\prime}\Delta_{-nk}(X_{i})\mid\mathcal{I}_{-nk}\right]=0$
for $\ell\in\\{1,2\\}$. Using similar arguments
$\mathbb{E}_{\gamma_{n}}\left[\left\|\partial\varphi_{i}(\eta)/\partial\eta_{\ell}\right\|^{4}\right]<C$.
By Lemma 14.(b), $\|\Delta_{nk}(x)\|\leq
C\times\|\widehat{\eta}_{-k}(x)-\eta(x)\|$. By Assumption 7.(ii),
$\mathbb{E}_{\gamma_{n}}\left[\|\widehat{\eta}_{-k}(x)-\eta(x)\|^{4}\right]=o(1)$,
which means that
$\mathbb{E}_{\gamma_{n}}\left[\left\|\frac{\partial\varphi_{i}(\eta)}{\partial\eta_{\ell}}e_{\ell}^{\prime}\Psi_{-nk}(X_{i})\right\|^{2}\right]$
is $o(1)$. By Lemma 12.(b), $\Lambda_{2nk}=o_{p}(1)$.

Let $\Xi=[e_{1},e_{2},e_{4}]$ be a $4\times 3$ matrix such that $\|\Xi\|\leq
1$. Let $\Lambda_{3nk}$ be the second-order on the right-hand side of (64).
Since the propensity score is known,

$\displaystyle\Lambda_{3nk}:=$
$\displaystyle\frac{1}{n_{k}}\sum_{i\in\mathcal{I}_{nk}}\sqrt{n_{k}}(\eta_{-k,\widehat{\theta}_{nk}}(X_{i})-\eta(X_{i}))^{\prime}\Xi\frac{\partial^{2}\varphi_{i}(\eta_{-k,\widehat{\theta}_{nk},\tilde{r}})}{\partial(\eta_{1},\eta_{2},\eta_{4})\partial(\eta_{1},\eta_{2},\eta_{4})^{\prime}}\Xi^{\prime}(\eta_{-k,\widehat{\theta}_{nk}}(X_{i})-\eta(X_{i})).$

(iii) Prove that $\Lambda_{3nk}=o_{p}(1)$: By Lemma 13.(d),
$\left\|\frac{\partial^{2}\varphi_{i}(\eta_{-k,\widehat{\theta}_{nk},\tilde{r}})}{\partial(\eta_{1},\eta_{2},\eta_{4})\partial(\eta_{1},\eta_{2},\eta_{4})^{\prime}}\right\|\leq
18\times\sqrt{3}/\delta$. For scalars, $a,b\in\mathbb{R}$, $(a+b)^{2}\leq
4(a^{2}+b^{2})$. By (65) and the triangle inequality, $\|\Lambda_{3nk}\|\leq
4\times\frac{18\sqrt{3}}{\delta}\left(\sqrt{n_{k}}\|\widehat{\lambda}_{nk}-\lambda_{nk}\|^{2}\left[\frac{1}{n_{k}}\sum_{i\in\mathcal{I}_{nk}}\|\Psi_{-nk}(X_{i})\|^{2}\right]+\left[\frac{1}{n_{k}}\sum_{i\in\mathcal{I}_{nk}}\sqrt{n_{k}}\|\Delta_{-nk}(X_{i})\|^{2}\right]\right)$.
By Lemma 15,
$\sqrt{n_{k}}\|\widehat{\lambda}_{nk}-\lambda_{nk}\|^{2}=o_{p}(1)$. By Lemma
14.(d), $\|\Psi_{-nk}(x)\|\leq C[1+2\|\widehat{\eta}_{-k}(x)\|]$. By the
moment bound in Assumption 7.(i) and Markov’s inequality, then
$\frac{1}{n_{k}}\sum_{i\in\mathcal{I}_{nk}}\|\Psi_{-nk}(X_{i})\|^{2}=O_{p}(1)$.
By Lemma 14.(e), $\|\Delta_{-nk}(x)\|\leq
C\times\|\widehat{\eta}_{-k}(x)-\eta(x)\|$. By Assumption 7.(iii),
$\sqrt{n_{k}}\mathbb{E}_{\gamma_{n}}[\|\widehat{\eta}_{-k}(X_{i})-\eta(X_{i})\|^{2}]=o(1)$,
then by Lemma 12.(c),
$\frac{1}{n_{k}}\sum_{i\in\mathcal{I}_{nk}}\sqrt{n_{k}}\|\Delta_{-nk}(X_{i})\|^{2}=o_{p}(1)$.
Combining these results, $\Lambda_{3nk}=o_{p}(1)$.

∎

  * Proof of Theorem 7.

Let
$\rho(\gamma,\mathcal{I}_{-nk})=\mathbb{P}_{\gamma}(V_{\tau}^{*}(\gamma,\mathcal{I}_{-nk})\in\widehat{CI}_{\alpha
nk}\mid\mathcal{I}_{-nk})$ denote the conditional probability that the pseudo-
VCATE is contained in the confidence interval. Let $\mathcal{F}(\gamma)$ be
the support of $\mathcal{I}_{-nk}$, and let
$\mathcal{F}(\gamma,t)\subseteq\mathcal{F}(\gamma)$. Almost surely,

$\displaystyle\inf_{\mathcal{I}_{-nk}\in\mathcal{F}(\gamma)}\rho(\gamma,\mathcal{I}_{-nk})$
$\displaystyle\leq\mathbb{E}_{\gamma}[\rho(\gamma,\mathcal{I}_{-nk})]\leq\sup_{\mathcal{I}_{-nk}\in\mathcal{F}(\gamma,t)}\rho(\gamma,\mathcal{I}_{-nk})+\mathbb{P}_{\gamma}(\mathcal{I}_{-nk}\notin\mathcal{F}(\gamma,t)).$
(66)

The left inequality considers the worst-case coverage. The right inequality
applies the law of iterated expectations by the event
$\mathcal{I}_{-nk}\in\mathcal{F}^{*}(\gamma,t)$, then bounds
$\mathbb{P}_{\gamma}(\mathcal{I}_{-nk}\in\mathcal{F}(\gamma,t))\leq 1$ and
$\rho(\gamma,\mathcal{I}_{-nk})\leq 1$ to simplify the expressions. Applying
limits,

$\underset{n\to\infty}{\lim\inf}\inf_{\gamma\in\Gamma}\inf_{\mathcal{I}_{-nk}\in\mathcal{F}(\gamma)}\rho(\gamma,\mathcal{I}_{-nk})\leq\underset{n\to\infty}{\lim\inf}\inf_{\gamma\in\Gamma}\mathbb{P}_{\gamma}\left(V_{\tau}^{*}(\gamma,\mathcal{I}_{-nk})\in\widehat{CI}_{\alpha
nk}\right),$ (67)

and an analogous result for the upper bound. The data in $\mathcal{I}_{-nk}$
only affects fold $k$ through the estimated nuisance functions
$\widehat{\eta}_{-k}(x)$. To prove uniform coverage we will derive the bounds
for a class of distributions where the $(Y_{i},D_{i},X_{i})$ in fold $k$ is
distributed as $\gamma$ and the plug-in nuisance functions are deterministic.
We will define $\mathcal{F}(\gamma,t)$ as the set where
$|V_{\tau}^{*}(\gamma,\mathcal{I}_{-nk})\Omega_{nk,12}|\leq t$ for some fixed
$t>0$.

Let $\\{\gamma_{n}\\}_{n=1}^{\infty}$ and
$\\{\mathcal{I}_{-nk}\\}_{n=1}^{\infty}$ denote a sequence of distributions
and data realizations, respectively. Theorem 3 verifies that under Assumptions
1, 3, 4, and 5, the conditional CDFs
$\\{F_{\gamma_{n}\mid\mathcal{I}_{-nk}}\\}_{n=1}^{\infty}$ satisfy the high-
level Assumption 2, almost surely. Furthermore, $\widehat{CI}_{\alpha nk}$
satisfies the form in (11), substituting $(\widehat{V}_{\tau
nk},\widehat{\Omega}_{nk})$. Therefore, Lemma 10.(a) shows that the uniform
lower bound on the asymptotic coverage probability is $(1-\alpha)$. If
Assumption 8 also holds, then for all $t>0$,
$\underset{n\to\infty}{\lim\sup}\sup_{\gamma\in\Gamma}\mathbb{P}_{\gamma}(\mathcal{I}_{-nk}\notin\mathcal{F}^{*}(\gamma,t))=0$.
Hence by Lemma 10.(ii), the upper bound is $(1-\alpha)$ plus a term that can
be made arbitrarily small by choosing $t$ close to zero.

∎

  * Proof of Lemma 6.

Consider an arbitrary sequence of distributions
$\\{\gamma_{n}\\}_{n=1}^{\infty}\in\Gamma^{\infty}$ and a convergent
subsequence where $V_{\tau}(\gamma_{n_{\ell}})\to V_{\tau}$. Since
$V_{\tau}^{*}(\gamma_{n_{\ell}},\mathcal{I}_{-n_{\ell}k})\leq V_{\tau
n_{\ell}}$ and $\Omega_{n_{\ell}k}$ has bounded eigenvalues, if $V_{\tau}=0$
then
$\lim_{\ell\to\infty}\mathbb{P}_{\gamma_{n_{\ell}}}(\sqrt{V_{\tau}^{*}(\gamma_{n_{\ell}},\mathcal{I}_{-n_{\ell}k})}|\Omega_{n_{\ell}k,12}|>t)=0$.
Now suppose that $V_{\tau}>0$. By Assumption 6, the nuisance functions
converge to their true value. Then $\widehat{\tau}_{-k}(x)$ converges point-
wise to $\tau(x)$, and by the moment bound in 3.(v) and the dominated
convergence theorem,
$V_{xn_{\ell}k}=\mathbb{V}_{\gamma_{n_{\ell}}}(\widehat{\tau}_{-k}(X)\mid\mathcal{I}_{-n_{\ell}k})=V_{\tau}(\gamma_{n_{\ell}})+o_{p}(1)\geq
V_{\tau}+o_{p}(1)$. By (28),

$\Omega_{n_{\ell}k}=\mathbb{V}_{\gamma_{n_{\ell}}}\left(\begin{bmatrix}\lambda_{n_{\ell}}(X_{i})(D_{i}-p_{n_{\ell}}(x_{i}))V_{xn_{\ell}k}^{-1/2}S_{-k}(X_{i})U_{i}\\\
V_{xn_{\ell}k}^{-1}S_{-k}(X_{i})^{2}\end{bmatrix}\mid\mathcal{I}_{-n_{\ell}k}\right),$
(68)

where $U_{in_{\ell}}=Y_{i}-W_{i}^{\prime}\theta_{n_{\ell}k}$. By Assumption 7,
for fixed $\\{D_{i}=d,X_{i}=x\\}$, $W_{i}^{\prime}$ point-wise converges to
$W_{i}^{*^{\prime}}:=[1,\mu_{0n_{\ell}}(x)+p_{n}(x)\tau_{n_{\ell}}(x),(d-p_{n_{\ell}}(x)),(d-p_{n_{\ell}}(x))\tau_{n_{\ell}}(x)]$.
By Lemma 15,
$\theta_{n_{\ell}k}\to\theta_{n_{\ell}}:=[0,1,\tau_{n_{\ell},av},1]$. By Lemma
5, $W_{i}^{*^{\prime}}\theta_{n_{\ell}}=\mu_{d,n_{\ell}}(x)$, and hence
$U_{in_{\ell}}^{*}:=Y_{i}-W_{i}^{*^{\prime}}\theta_{n_{\ell}}$. Since
$\Omega_{n_{\ell}k}$ is almost surely bounded by Assumption 3 over random
partitions $\mathcal{I}_{-n_{\ell}k}$, then applying the dominated convergence
theorem,

$\Omega_{n_{\ell}k}=\mathbb{V}_{\gamma_{n_{\ell}}}\left(\begin{bmatrix}\lambda_{n_{\ell}}(X_{i})(D_{i}-p_{n_{\ell}}(x_{i}))V_{\tau}^{-1/2}(\tau_{n_{\ell}}(X_{i})-\tau_{n_{\ell},av})U_{in_{\ell}}^{*}\\\
V_{\tau}^{-1}(\tau_{n_{\ell}}(X_{i})-\tau_{n_{\ell},av})^{2}\end{bmatrix}\mid\mathcal{I}_{-n_{\ell}k}\right)+o_{p}(1),$
(69)

Since $\mathbb{E}_{\gamma_{n}}[U_{in_{\ell}}^{*}\mid
D_{i}=d,X_{i}=x,\mathcal{I}_{-n_{\ell}k}]=0$ and the second component of (69)
only depends on $X_{i}$. By iterated expectations,
$\Omega_{n_{\ell}k,12}=o_{p}(1)$ and the limiting probability is
$\lim_{\ell\to\infty}\mathbb{P}_{\gamma_{n_{\ell}}}(\sqrt{V_{\tau}^{*}(\gamma_{n_{\ell}},\mathcal{I}_{-n_{\ell}k})}|\Omega_{n_{\ell}k,12}|>t)=0$.
Hence we verified Assumption $B^{*}$ in Andrews et al. (2020). Uniform
consistency follows from their Corollary 2.1.

∎

  * Proof of Theorem 8.

We break-down the proof into cases.

Case (i): When $V_{\tau}(\gamma)=0$, then
$V_{\tau}(\gamma,\mathcal{I}_{-nk})=0$ almost surely. Therefore,
$n_{k}(\widehat{V}_{\tau
nk}-V_{\tau}^{*}(\gamma_{n},\mathcal{I}_{-nk}))=n_{k}(\widehat{V}_{\tau
nk}^{*}-V_{\tau}(\gamma))$. Define $\rho(\gamma,\mathcal{I}_{-nk})$ as in
Theorem 7. Then
$\underset{n\to\infty}{\lim\inf}\inf_{\mathcal{I}_{-nk}\in\mathcal{F}(\gamma)}\rho(\gamma,\mathcal{I}_{-nk})\leq\underset{n\to\infty}{\lim\inf}\mathbb{P}_{\gamma}\left(V_{\tau}^{*}(\gamma,\mathcal{I}_{-nk})\in\widehat{CI}_{\alpha
nk}\right)$. We can prove an analogous upper bound. This implies that we only
need to derive coverage bounds under sequences of conditional distributions
where $V_{\tau}^{*}(\gamma_{n},\mathcal{I}_{-nk})=0$. Exact coverage follows
from the proof of the near-homogeneity case in Lemma 10.

Case (ii) When $V_{\tau}(\gamma)>0$. By Assumption 6,
$\sqrt{n_{k}}\omega(\gamma)^{2}\to 0$ as $n_{k}\to\infty$. Then by Lemma 4,
$\sqrt{n_{k}}|V_{\tau}(\gamma)-V_{\tau}^{*}(\gamma,\mathcal{I}_{-nk})|=o_{p}(1)$,
and by the continuous mapping theorem,
$\sqrt{V_{\tau}^{*}(\gamma,\mathcal{I}_{-nk})}\to^{p}\sqrt{V_{\tau}(\gamma)}>0$.
By (69) in the proof of Lemma 6 and for fixed $\gamma_{n}=\gamma$,
$\Omega_{nk}=\Omega+o_{p}(1)$, where $\Omega$ is a population covariance
matrix that does not depend on $\mathcal{I}_{-nk}$. By applying similar limits
to the mild heterogeneity case in Lemma 10 we can show that
$(\widehat{V}_{\tau nk}-V_{\tau}(\gamma))$ is $\sqrt{n_{k}}$ asymptotically
equivalent to an empirical process indexed by the oracle $V_{\tau}(\gamma)$.
We obtain exact coverage due to Lemma 6. ∎

  * Proof of Corollary 3.

By the first part of Theorem 7,

$\underset{n\to\infty}{\lim\sup}\sup_{\gamma\in\Gamma}\mathbb{P}_{\gamma}\left(V_{\tau}^{*}(\gamma,\mathcal{I}_{-nk})\notin\widehat{CI}_{\alpha
nk}\right)=1-\underset{n\to\infty}{\lim\inf}\inf_{\gamma\in\Gamma}\mathbb{P}_{\gamma}\left(V_{\tau}^{*}(\gamma,\mathcal{I}_{-nk})\in\widehat{CI}_{\alpha
nk}\right)\leq\alpha.$

The result follows taking limits on either side of the inequality in (38). ∎

  * Proof of Theorem 9 .

By the definition in (32),

$V_{\tau}(\gamma)\geq\inf\widehat{CI}_{\alpha
n}^{\text{multifold}}\iff\frac{1}{K}\sum_{k=1}^{K}\mathbbm{1}\left\\{V_{\tau}(\gamma)\geq\inf\widehat{CI}_{\alpha
nk}\right\\}\geq\frac{1}{2}.$

By negating the statement, computing expectations, and applying Markov’s
inquality,

$\displaystyle\mathbb{P}_{\gamma}(V_{\tau}(\gamma)<\inf\widehat{CI}_{\alpha
n}^{\text{multifold}})$
$\displaystyle=\mathbb{E}_{\gamma}\left[\mathbbm{1}\left\\{\left(\frac{1}{K}\sum_{k}\mathbbm{1}\\{V_{\tau}(\gamma)<\inf\widehat{CI}_{\frac{\alpha}{2}nk}\\}\right)>1/2\right\\}\right]$
$\displaystyle\leq
2\frac{1}{K}\sum_{k}\mathbb{E}_{\gamma}[\mathbbm{1}\\{V_{\tau}(\gamma)<\inf\widehat{CI}_{\frac{\alpha}{2}nk}\\}]$
$\displaystyle\leq
2\mathbb{P}_{\gamma}\left(V_{\tau}(\gamma)<\inf\widehat{CI}_{\frac{\alpha}{2}nk}\right)\leq
2\mathbb{P}_{\gamma}\left(V_{\tau}(\gamma,\mathcal{I}_{-nk}^{*})\notin\widehat{CI}_{\frac{\alpha}{2}nk}\right)$

The last line follows from the fact that the folds are split at random and the
inequality in (38) holds almost surely. By Theorem 7 the asymptotic size is
uniformly bounded by $2(\alpha/2)=\alpha$. By construction,
$\mathbb{P}_{\gamma}(V_{\tau}(\gamma)\notin\widehat{CI}_{\alpha
n}^{\text{multifold}})=\mathbb{P}_{\gamma}(V_{\tau}(\gamma)<\inf\widehat{CI}_{\alpha
n}^{\text{multifold}})+\mathbb{P}_{\gamma}(V_{\tau}(\gamma)>\sup\widehat{CI}_{\alpha
n}^{\text{multifold}})$. Applying similar arguments as before,
$\mathbb{P}_{\gamma}(V_{\tau}(\gamma)\notin\inf\widehat{CI}_{\alpha
n}^{\text{multifold}})\leq
2\mathbb{P}_{\gamma}\left(V_{\tau}(\gamma)\notin\widehat{CI}_{\frac{\alpha}{2}nk}\right)$.
If Assumptions 6, and 7 also hold, then Theorem 8 implies that the right-hand
side is point-wise bounded by $\alpha$.

∎

  * Proof of Lemma 7.

Under the null hypothesis,
$G(n_{k},0,\widehat{\Omega}_{nk},Z,\zeta)=(e_{1}^{\prime}\widehat{\Omega}_{nk}^{1/2}Z)^{2}/n_{k}$,
where $\widehat{\Omega}_{nk,11}^{1/2}Z$, where $Z\sim\mathcal{N}(0,1)$. The
adjusted critical values are
$q_{\alpha/2}(n_{k},0,\widehat{\Omega}_{nk},\zeta)=0$ and
$q_{1-\alpha/2}(n_{k},0,\widehat{\Omega}_{nk},\zeta)=\widehat{\Omega}_{nk,11}z_{1-\alpha}^{2}/n_{k}$.
Then $0\in\widehat{CI}_{\alpha nk}$ if and only if $0\leq\widehat{V}_{\tau
nk}-0\leq\widehat{\Omega}_{nk,11}z_{1-\alpha}^{2}/n_{k}$. Following similar
steps to the “near homogeneity” regime in Theorem 7, $n_{k}(\widehat{V}_{\tau
nk}-V_{\tau}^{*}(\gamma_{n},\mathcal{I}_{-nk}))=(\Omega_{\infty,11}^{1/2}Z_{nk}+\sqrt{v})^{2}-v+o_{p}(1)$,
where $Z_{nk}\sim\mathcal{N}(0,1)$. Consequently, $n_{k}\widehat{V}_{\tau
nk}=(\Omega_{\infty,11}^{1/2}Z_{nk}+\sqrt{v})^{2}+o_{p}(1)$. Then

$\displaystyle\mathbb{P}_{\gamma_{n}}(0\in\widehat{CI}_{\alpha
nk}\mid\mathcal{I}_{-nk})$ $\displaystyle=\mathbb{P}_{\gamma_{n}}\left(0\leq
n_{k}\widehat{V}_{\tau nk}\leq\widehat{\Omega}_{nk,11}z_{1-\alpha}^{2}\right)$
$\displaystyle=\mathbb{P}_{\gamma_{n}}\left(0\leq(\Omega_{\infty,11}^{1/2}Z_{nk}+\sqrt{v})^{2}\leq\Omega_{\infty,11}z_{1-\alpha}^{2}\right)+o(1)$
$\displaystyle=\mathbb{P}_{\gamma_{n}}\left(-\frac{\sqrt{v}}{\sqrt{\Omega_{\infty,11}}}\leq
Z_{nk}+o_{p}(1)\leq
z_{1-\alpha}-\frac{\sqrt{v}}{\sqrt{\Omega_{\infty,11}}}\right)+o(1).$

and hence $\lim_{n\to\infty}\mathbb{P}_{\gamma_{n}}(0\in\widehat{CI}_{\alpha
nk}\mid\mathcal{I}_{-nk})=\Phi\left(z_{1-\alpha}-\sqrt{v}/\sqrt{\Omega_{\infty,11}}\right)-\Phi\left(-\sqrt{v}/\sqrt{\Omega_{\infty,11}}\right)$.
The final result is obtained by
$1-\mathbb{P}_{\gamma_{n}}(0\in\widehat{CI}_{\alpha
nk}\mid\mathcal{I}_{-nk})$. ∎

  * Proof of Lemma 8.

The first part of the proof is identical to that of Theorem 7 in terms of
setting up the problem, and defining a sequence the conditional distributions
$\\{F_{\gamma_{n}\mid\mathcal{I}_{-nk}}\\}_{n=1}^{\infty}$. By equation (39)
this sequence satisfies Assumption 2 almost surely with an effective sample
size $\tilde{n}=n/r_{n}$ and a particular choice of covariance estimator. To
complete the proof, we develop a modified version of Lemma 10 to prove uniform
coverage under cluster dependence. Consider a sequence of distributions
$\\{\gamma_{n}\\}_{n=1}^{\infty}\in\Gamma^{\infty}$ and a subsequence
$\\{n_{\ell}\\}_{\ell=1}^{\infty}$, where
$\frac{n_{\ell}}{r_{n_{\ell}}}V_{\tau n_{\ell}}^{*}\to^{p}v\in[0,\infty)$,
$V_{\tau n_{\ell}}^{*}\to^{p}0$, and $\Omega_{n_{\ell}}\to\Omega$ as
$n_{\ell}\to\infty$. Applying Lemma 2 , and factorizing terms as in Lemma 10,

$\frac{n_{\ell}}{r_{n_{\ell}}}(\widehat{V}_{\tau n_{\ell}}^{*}-V_{\tau
n_{\ell}}^{*})=(e_{1}^{\prime}\Omega^{1/2}Z_{n_{\ell}}+\sqrt{v})^{2}-v+o_{p}(1).$
(70)

Now we need to show that the quantiles of the empirical process converge to
the same limit. The quantity $\frac{n_{\ell}}{r_{n_{\ell}}}G(n_{\ell},V_{\tau
n_{\ell}}^{*},\widehat{\Omega}_{n_{\ell}},Z)$ is equal to

$n_{\ell}\frac{(e_{1}^{\prime}r_{n_{\ell}}^{-1/2}\widehat{\Omega}_{n_{\ell}}^{1/2}Z)^{2}}{n_{\ell}}+2\sqrt{\frac{n_{\ell}V_{\tau
n_{\ell}}^{*}}{r_{n_{\ell}}}}(e_{1}^{\prime}r_{n_{\ell}}^{-1/2}\widehat{\Omega}_{n_{\ell}}^{1/2}Z)+\sqrt{\frac{n_{\ell}V_{\tau
n_{\ell}}^{*}}{r_{n_{\ell}}}}\sqrt{V_{\tau
n_{\ell}}^{*}}(e_{2}^{\prime}r_{n_{\ell}}^{-1/2}\widehat{\Omega}_{n_{\ell}}^{1/2}Z).$
(71)

In the first term the $n_{\ell}$ components cancel out and
$r_{n_{\ell}}^{-1}\widehat{\Omega}_{n_{\ell}}\to^{p}\Omega$. Similarly, since
$\frac{n_{\ell}}{r_{n_{\ell}}}V_{\tau n_{\ell}}^{*}\to^{p}v$ the second term
of (71) converges to $2\sqrt{v}e_{1}^{\prime}\Omega^{1/2}Z$, and a suitable
factorization with the first and second the expression in (70). Since
$n_{\ell}/r_{n_{\ell}}\to\infty$ by assumption, then $V_{\tau n_{\ell}}^{*}\to
0$ and the third term of (71) is $o_{p}(1)$. Therefore the estimated quantiles
are consistent. Proving consistency of the quantiles for the mild
hereterogeneity proceeds analogously. Once we prove that the quantiles are
asymptotically correct, the rest of the proof is the same as in Lemma 10. ∎

  * Proof of Theorem 9 .

The first part of the proof is identical to that of Theorem 7 in terms of
setting up the problem. In this case the sequence of conditional distributions
$\\{F_{\gamma_{n}\mid\mathcal{I}_{-nk}}\\}_{n=1}^{\infty}$ only satisfies
Assumption 2 for subsequences where $V_{xnk}=0$. Instead, I will modify the
first part of the proof of Lemma 10.(i) for a class of regression-adjusted CIs
with possible degeneracy. Consider a sequence of distributions
$\\{\gamma_{n}\\}_{n=1}^{\infty}\in\Gamma^{\infty}$ and let $h_{n}:=(nV_{\tau
n}^{*},V_{\tau n}^{*},vec(\Omega_{n}),\zeta_{n},V_{xn})$ be a sequence of
parameters where $V_{xn}$ is the variance of $S_{-k}(X_{i})$. Our goal is to
show that for $h\in\mathcal{H}$ and all subsequences
$\\{n_{\ell}\\}_{\ell=1}^{\infty}$ where $h_{n_{\ell}}\to h\in\mathcal{H}$,

$\lim_{n_{\ell}\to\infty}\mathbb{P}_{\gamma_{n_{\ell}}}\left(V_{\tau
n_{\ell}}^{*}\in\widehat{CI}_{\alpha nk}^{0}\right)\geq 1-\alpha.$

Partition the subsequences in such a way that $h_{n_{\ell}}$ either has
$V_{xn_{\ell}}=0$ or $V_{xn_{\ell}}>0$. When $V_{xn_{\ell}}=0$, then
$\widehat{V}_{\tau n_{\ell}}^{*}$ is exactly degenerate and the confidence
interval (40) covers the pseudo-VCATE with probability one. For the sequences
where $V_{xn_{\ell}}>0$, we can apply the remaining cases from Lemma 10 which
have coverage $1-\alpha$. This satisfies Assumption B in Andrews et al. (2020)
and we can prove uniform conservative coverage of the pseudo-VCATE applying
their Corollary 2.1

We prove point-wise coverage by cases. When $V_{\tau}(\gamma)=0$, then
$V_{\tau}^{*}(\gamma,\mathcal{I}_{-nk})=0$ almost surely. By applying the
result above and the near homogeneity case in Theorem 7 we find that coverage
is at least $(1-\alpha)$. Now consider the case where $V_{\tau}(\gamma)$ is
bounded away from zero. By Assumption 6, the nuisance functions converge to
their true value. Then $\widehat{\tau}_{-k}(x)$ converges point-wise to
$\tau(x)$, and by the moment bound in 3.(v) and the dominated convergence
theorem,
$\mathbb{V}_{\gamma_{n}}(\widehat{\tau}_{-k}(X))=V_{\tau}(\gamma)+o(1)$, which
is bounded away from zero. Then we can apply the mild heterogeneity results in
Theorem 7 to prove (43). The proof of (44) is identical to that of Theorem 9,
substituting the confidence intervals $\widehat{CI}_{\alpha nk}^{0}$ instead
of $\widehat{CI}_{\alpha nk}$. Point-wise coverage of the fold-specific
confidence intervals holds by (43).

∎

  * Proof of Corollary 4.

The proof of (45) and (46) follows a similar structure to Corollary 3 and
Theorem 9, respectively. In each case the only thing that changes is that we
apply the uniformity result for degenerate CIs in (42) (Lemma 9), rather than
the non-degenerate uniformity result in Theorem 7.

∎

##  Appendix C Supporting Lemmas and Proofs

######  Lemma 10.

Suppose that $\Gamma$ is a set of distributions constrained in such a way that
Assumption 2 holds. Let
$V_{\tau}^{*}(\gamma)=\beta_{2}(\gamma)^{2}V_{x}(\gamma)$ and $\Omega(\gamma)$
be the pseudo-VCATE and covariance matrix associated with $\gamma$,
respectively. If $\widehat{CI}_{\alpha n}$ is a confidence interval obtained
by substituting $(\widehat{V}_{\tau n},\widehat{\Omega}_{n})$ into (11), then

  1. (i)

$1-\alpha\leq\underset{n\to\infty}{\lim\inf}\inf_{\gamma\in\Gamma}\mathbb{P}_{\gamma}\left(V_{\tau}^{*}(\gamma)\in\widehat{CI}_{\alpha
nk}\right)$

  2. (ii)

If in addition, $\Omega_{12}(\gamma)V_{\tau}(\gamma)\leq t$, then
$\underset{n\to\infty}{\lim\sup}\sup_{\gamma\in\Gamma}\mathbb{P}_{\gamma}\left(V_{\tau}^{*}(\gamma,\mathcal{I}_{-nk})\in\widehat{CI}_{\alpha
nk}\right)\leq(1-\alpha)+\tilde{\alpha}(t)$, where $\tilde{\alpha}(t)\geq 0$
and $\lim_{t\to 0}\tilde{\alpha}(t)=0$.

  * Proof.

Let $\\{\gamma_{n}\in\Gamma:n\in\mathbb{N}\\}$ denote a sequence of
distributions. Our goal is to verify that the confidence interval satisfies
the assumptions of Corollary 2.1(c) in Andrews et al. (2020). Define a
sequence of parameters, $h_{n}:=(nV_{\tau n}^{*},V_{\tau
n}^{*},vec(\Omega_{n}),\zeta_{n})$. By Assumption 2, each element is contained
in $\mathcal{H}$, a subset of the extended Euclidean space in which
$\Omega_{n}$ is positive-definite with bounded eigenvalues. The quantity
$nV_{\tau n}^{*}$ is positive but unbounded, and can converge to $+\infty$.
Assumption $B$ in Andrews et al. (2020) is stated in terms of subsequences and
the first step is to write the problem in this way. We show that for
$h\in\mathcal{H}$ and all subsequences $\\{n_{\ell}\\}_{\ell=1}^{\infty}$
where $h_{n_{\ell}}\to h\in\mathcal{H}$,

$1-\alpha\leq\lim_{n_{\ell}\to\infty}\mathbb{P}_{\gamma_{n_{\ell}}}\left(V_{\tau
n_{\ell}}^{*}\in\widehat{CI}_{\alpha
n}\right)\leq(1-\alpha)+\tilde{\alpha}(t),\qquad\tilde{\alpha}(t)\geq
0,\alpha\in[0,1]$

We break down the proof by cases. (a) Near homogeneity case: Suppose that
$n_{\ell}V_{\tau n_{\ell}}^{*}\to^{p}v\in[0,\infty)$, $V_{\tau
n_{\ell}}^{*}\to^{p}0$, $\zeta_{n_{\ell}}\to\zeta^{*}\in\\{0,1\\}$ and
$\Omega_{n_{\ell}}\to\Omega$ as $n\to\infty$, where $\Omega$ is positive-
definite. For this case, $\sqrt{n_{\ell}}V_{\tau n_{\ell}}^{*}=o(1)$. By
applying Lemma 2,

$\displaystyle n_{\ell}(\widehat{V}_{\tau n_{\ell}}^{*}-V_{\tau
n_{\ell}}^{*})$
$\displaystyle=(e_{1}^{\prime}\Omega^{1/2}Z_{n_{\ell}})^{2}+2\zeta^{*}\sqrt{v}e_{1}^{\prime}\Omega^{1/2}Z_{n_{\ell}}+o_{p}(1)$
$\displaystyle=(e_{1}^{\prime}\Omega^{1/2}Z_{n_{\ell}}+\zeta^{*}\sqrt{v})^{2}-v+o_{p}(1).$

Let $Z\sim\mathcal{N}(0,I_{2\times 2})$ (independent of $n_{\ell}$). Since
$\widehat{\Omega}_{n_{\ell}}=\Omega+o_{p}(1)$, then the estimated empirical
process at $V_{\tau n_{\ell}}^{*}$ has the same limiting distribution as the
estimator. For a fixed $\zeta\in\\{-1,1\\}$ (that may differ from
$\zeta^{*}$),

$\displaystyle n_{\ell}G(n,V_{\tau
n_{\ell}}^{*},\widehat{\Omega}_{n_{\ell}},Z,\zeta)$
$\displaystyle=(e_{1}^{\prime}\Omega^{1/2}Z+\zeta\sqrt{v})^{2}-v+o_{p}(1).$

Define the limiting CDF as
$H_{\Omega,v,\zeta}(\tilde{v}):=\mathbb{P}_{\gamma_{n_{\ell}}}\left((e_{1}^{\prime}\Omega^{1/2}Z+\zeta\sqrt{v})^{2}-v\leq\tilde{v}\right)$,
where $Z\sim\mathcal{N}(0,I_{2\times 2})$. Since $Z$ has mean zero,
$H_{\Omega,v,\zeta=1}(\tilde{v})=H_{\Omega,v,\zeta=-1}(\tilde{v})=H_{\Omega,\nu}(\tilde{v})$,
which does not depend on $\zeta$. Since $\Omega$ is positive-definite, the
function $H_{\Omega,v}(\tilde{v})$ is continuous and strictly increasing. Let
$\widetilde{F}_{n_{\ell},V_{\tau
n_{\ell}}^{*},\widehat{\Omega}_{n\ell},\zeta_{n_{\ell}}}(\tilde{v})=F_{n_{\ell},V_{\tau
n_{\ell}}^{*},\widehat{\Omega}_{n_{\ell}},\zeta_{n_{\ell}}}(\tilde{v}/n_{\ell})\to
H_{\Omega,v}(\tilde{v})$. Since $H_{\Omega,v}$ is continuous, then (Lehmann,
1999, Theorem 2.6.1) implies that this convergence is uniform in $\tilde{v}$,
and since the limiting CDF is strictly increasing,

$F_{n_{\ell},V_{\tau}^{*},\widehat{\Omega}_{n},\zeta}(\widehat{V}_{\tau
n_{\ell}}-V_{\tau}^{*})=\tilde{F}_{n_{\ell},V_{\tau
n_{\ell}}^{*},\widehat{\Omega}_{n},\zeta}(n_{\ell}(\widehat{V}_{\tau
n_{\ell}}-V_{\tau}^{*}))\to^{d}U_{n_{\ell}},$

for all $\zeta\in\\{-1,1\\}$ where $U_{n_{\ell}}\sim\sim Uniform[0,1]$. The
test statistic converges to a fixed distribution regardless of the choice of
$\zeta$. Similarly, $q_{\alpha/2}(n_{\ell},V_{\tau
n_{\ell}}^{*},\Omega,\zeta)\to^{p}\min\\{\alpha/2,H_{\Omega,v}(0)\\}$. Define
a random variable,
$\widehat{R}_{n_{\ell},\zeta}:=F_{n_{\ell},V_{\tau}^{*},\widehat{\Omega}_{n},\zeta}(\widehat{V}_{\tau
n_{\ell}}-V_{\tau n_{\ell}}^{*})-q_{\alpha/2}(n_{\ell},V_{\tau
n_{\ell}}^{*},\widehat{\Omega}_{n_{\ell}},\zeta)\to^{d}U_{n_{\ell}}-\min\\{\alpha/2,H_{\Omega,v}(0)\\}$.
By definition, $V_{\tau n_{\ell}}^{*}\in\widehat{CI}_{\alpha
n}\iff\bigcup_{\zeta\in\\{-1,1\\}}\left\\{\widehat{R}_{n_{\ell},\zeta}\in[0,1-\alpha]\right\\}.$
As $n_{\ell}\to\infty$,

$\displaystyle\begin{split}\mathbb{P}_{\gamma_{n_{\ell}}}\left(V_{\tau
n_{\ell}}^{*}\in\widehat{CI}_{\alpha
nk}\right)&\geq\max_{\zeta\in\\{-1,1\\}}\mathbb{P}_{\gamma_{n_{\ell}}}\left(\widehat{R}_{n_{\ell},\zeta}\in[0,1-\alpha]\right)\geq\mathbb{P}_{\gamma_{n_{\ell}}}\left(\widehat{R}_{n_{\ell},\zeta_{n_{\ell}}}\in[0,1-\alpha]\right)\\\
&=\mathbb{P}_{\gamma_{n_{\ell}}}\left(0\leq
U_{n_{\ell}}-\min\\{\alpha/2,H_{\Omega,v}(0)\\}\leq 1-\alpha\right)+o(1)\\\
&=(1-\alpha)+o(1).\end{split}$

Since the limiting distribution doesn’t depend on $\zeta$, we can apply the
continuous mapping theorem to show that
$\widehat{R}_{n_{\ell}}^{\max}:=\min_{\zeta\in\\{-1,1\\}}\widehat{R}_{n_{\ell},\zeta}$
and
$\widehat{R}_{n_{\ell}}^{\min}:=\max_{\zeta\in\\{-1,1\\}}\widehat{R}_{n_{\ell},\zeta}$
both converge to the same limit,
$U_{n_{\ell}}-\min\\{\alpha/2,H_{\Omega,v}(0)\\}$. As $n_{\ell}\to\infty$,

$\displaystyle\begin{split}\mathbb{P}_{\gamma_{n_{\ell}}k}\left(V_{\tau
n_{\ell}}^{*}\in\widehat{CI}_{\alpha
nk}\right)&\leq\mathbb{P}_{\gamma_{n_{\ell}}}\left(0\leq\widehat{R}_{n_{\ell}}^{\max},\widehat{R}_{n_{\ell}}^{\min}\leq
1-\alpha\right)=(1-\alpha)+o(1).\end{split}$ (72)

For this class of subsequences the confidence interval produces exact
coverage.

Mild heterogeneity case: Suppose that $n_{\ell}V_{\tau n_{\ell}}^{*}\to\infty$
and $V_{\tau n_{\ell}}^{*}\to V_{\tau}^{*}$, where $V_{\tau}\in[0,\infty)$,
$\zeta_{n_{\ell}}\to\zeta^{*}$, and $\Omega_{n_{\ell}}\to\Omega$. Then we can
rescale by $\sqrt{n_{\ell}/V_{\tau n_{\ell}}^{*}}$.

$\displaystyle\sqrt{\frac{n_{\ell}}{V_{\tau n_{\ell}}^{*}}}(\widehat{V}_{\tau
n_{\ell}}^{*}-V_{\tau n_{\ell}}^{*})$
$\displaystyle=\frac{(e_{1}^{\prime}\Omega_{n_{\ell}}^{1/2}Z_{n_{\ell}})^{2}}{\sqrt{n_{\ell}V_{\tau
n_{\ell}}^{*}}}+2\zeta_{n_{\ell}}e_{1}^{\prime}(\Omega_{n_{\ell}}^{1/2}Z_{n_{\ell}})+\sqrt{V_{\tau
n_{\ell}}^{*}}(e_{2}^{\prime}\Omega_{n_{\ell}}^{1/2}Z_{n_{\ell}})+o_{p}(1)$
$\displaystyle=2\zeta^{*}(e_{1}^{\prime}\Omega^{1/2}Z_{n_{\ell}})+\sqrt{V_{\tau}^{*}}(e_{2}^{\prime}\Omega^{1/2}Z_{n_{\ell}})+o_{p}(1).$

The limiting distribution is normal. For convenience, we write this as
$\sqrt{n_{\ell}/V_{\tau n_{\ell}}^{*}}(\widehat{V}_{\tau n_{\ell}}^{*}-V_{\tau
n_{\ell}}^{*})=\sigma(\zeta^{*})\tilde{Z}_{n_{\ell}}+o_{p}(1)$, where
$\tilde{Z}_{n_{\ell}}\sim\mathcal{N}(0,1)$ and
$\sigma(\zeta)^{2}:=\Omega_{11}+V_{\tau}^{*}\Omega_{22}+\zeta\sqrt{V_{\tau}^{*}}\Omega_{12}$
for $\zeta\in\\{1,-1\\}$. Since the norm of
$[1,\zeta\sqrt{V_{\tau}^{*}}]^{\prime}$ is larger than one, it follows that
$\sigma(\zeta)^{2}\geq\lambda_{\min}$, where $\lambda_{\min}$ is the smallest
eigenvalue of $\Omega$. In this case, the limiting CDF is
$H_{\Omega,V_{\tau}^{*},\zeta}(\tilde{v}):=\Phi(\tilde{v}/\sigma(\zeta))$
where $\Phi(\cdot)$ is the CDF of a standard normal. Let
$z_{\alpha}=\Phi^{-1}(\alpha)$ denote the $\alpha-$quantile, and $\phi(\cdot)$
the marginal of a standard normal.

$\displaystyle\begin{split}\mathbb{P}_{\gamma_{n_{\ell}}}(\widehat{R}_{n_{\ell},\zeta}\in[0,1-\alpha])&=\mathbb{P}\left(-\alpha/2\leq\Phi\left(\frac{\sigma(\zeta^{*})}{\sigma(\zeta)}\tilde{Z}_{n_{\ell}}\right)\leq
1-\alpha/2\right)+o(1)\\\
&=\underbrace{\Phi\left(\frac{\sigma(\zeta^{*})}{\sigma(\zeta)}z_{1-\alpha/2}\right)-\Phi\left(\frac{\sigma(\zeta^{*})}{\sigma(\zeta)}z_{-\alpha/2}\right)}_{\kappa(\sigma(\zeta),\sigma(\zeta^{*}))}+o(1).\end{split}$
(73)

To obtain the lower bound,$\mathbb{P}_{\gamma_{n_{\ell}}}\left(V_{\tau
n_{\ell}}^{*}\in\widehat{CI}_{\alpha
nk}\right)\geq\mathbb{P}_{\gamma_{n_{\ell}}}\left(\widehat{R}_{n_{\ell},\zeta^{*}}\in[0,1-\alpha]\right)=(1-\alpha)+o(1)$.
To obtain the upper bound, I write down a Taylor expansion. There is a
$\tilde{\sigma}\geq\sqrt{\lambda_{\min}}$ between $\sigma(\zeta^{*})$ and
$\sigma(\zeta)$ such that

$\displaystyle\|\kappa(\sigma_{\zeta},\sigma_{\zeta^{*}})-(1-\alpha)\|$
$\displaystyle\leq\left\|\left[\phi\left(\frac{\tilde{\sigma}z_{1-\alpha/2}}{\sigma(\zeta)}\right)z_{1-\alpha/2}-\phi\left(\frac{\tilde{\sigma}z_{-\alpha/2}}{\sigma(\zeta)}\right)z_{-\alpha/2}\right]\frac{[\sigma(\zeta^{*})-\sigma(\zeta)]}{\sigma(\zeta)}\right\|$
$\displaystyle\leq\frac{|z_{-\alpha/2}|}{\sqrt{2\pi\lambda_{min}}}\|\sigma(\zeta^{*})-\sigma(\zeta)\|.$

By another Taylor expansion,
$\|\sigma(\zeta^{*})-\sigma(\zeta)\|\leq\frac{1}{2\sqrt{\lambda_{\min}}}\|\sigma(\zeta^{*})^{2}-\sigma(\zeta)^{2}\|$.
If in addition, $\|\sqrt{V_{\tau}*}\Omega_{12}\|\leq t$, then
$\|\sigma(\zeta^{*})-\sigma(\zeta)\|\leq t$ and we can define a non-negative
function $\tilde{\alpha}(t)=t\|z_{-\alpha/2}\|/(2\lambda_{\min}\sqrt{2\pi})$,
which satisfies $\lim_{t\to 0}\tilde{\alpha}(t)=0$. Therefore we have bounded
the coverage over an exhaustive class of subsequences of distributions. This
satisfies Assumption $B$ in Andrews et al. (2020). By their Corollary 2.1,

$1-\alpha\leq\underset{n\to\infty}{\lim\inf}\inf_{\gamma\in\Gamma}\mathbb{P}_{\gamma}\left(V_{\tau}^{*}(\gamma)\in\widehat{CI}_{\alpha
n}\right)=\underset{n\to\infty}{\lim\sup}\sup_{\gamma\in\Gamma}\mathbb{P}_{\gamma}\left(V_{\tau}^{*}(\gamma)\in\widehat{CI}_{\alpha
n}\right)\leq 1-\alpha+\tilde{\alpha}(t).$

∎

######  Lemma 11.

Let $(S_{-k}(X_{i}),M_{-k}(X_{i}),W_{i})$ and $\lambda(X_{i})$ be the set of
regressors and weights, respectively, that were defined in (20). Define
$Q_{ww,nk}:=\mathbb{E}[\lambda(X_{i})W_{i}W_{i}^{\prime}\mid\mathcal{I}_{-nk}]$
and let $\Pi_{nk}$ be a $4\times 4$ diagonal matrix with entries
$(1,1,1,V_{xnk}^{-1/2})$. If $(X_{i},D_{i})$ are independent of the data in
$\mathcal{I}_{-nk}$, then $\Pi_{nk}^{\prime}Q_{ww,nk}\Pi_{nk}$ has the form in
(55).

  * Proof of Lemma 11.

Let $V_{i}=(D_{i}-p(X_{i}))$. By definition,
$\lambda(X_{i})W_{i}W_{i}^{\prime}$ is

$\displaystyle\lambda(X_{i})\begin{pmatrix}1&M_{-k}(X_{i})&V_{i}&V_{i}S_{-k}(X_{i})\\\
M_{-k}(X_{i})&M_{-k}(X_{i})^{2}&M_{-k}(X_{i})V_{i}&M_{-k}(X_{i})V_{i}S_{-k}(X_{i})\\\
V_{i}&M_{-k}(X_{i})V_{i}&V_{i}^{2}&V_{i}^{2}S_{-k}(X_{i})\\\
V_{i}S_{-k}(X_{i})&M_{-k}(X_{i})V_{i}S_{-k}(X_{i})&V_{i}^{2}S_{-k}(X_{i})&V_{i}^{2}S_{-k}(X_{i})^{2}\end{pmatrix}.$
(74)

Since $(X_{i},D_{i})$ are independent of the data in $\mathcal{I}_{-nk}$, then
$\mathbb{E}[V_{i}\mid X_{i}=x,\mathcal{I}_{-nk}]$ does not depend on
$\mathcal{I}_{-nk}$ and is equal to $\mathbb{E}[V_{i}\mid
X_{i}=x]=\mathbb{E}[D_{i}\mid X=x]-p(x)=0$. By a similar reasoning,
$\mathbb{E}[V_{i}^{2}\mid
X_{i}=x,\mathcal{I}_{-nk}]=p(X_{i})(1-p(X_{i}))=\lambda(X_{i})^{-1}$. Using
both conditional moment results, we can show that
$\mathbb{E}[\lambda(X_{i})W_{i}W_{i}^{\prime}\mid X_{i},\mathcal{I}_{-nk}]$ is
equal to

$\displaystyle
Q_{ww,nk}=\mathbb{E}\left[\left.\begin{matrix}\lambda(X_{i})&\lambda(X_{i})M_{-k}(X_{i})&0&0\\\
\lambda(X_{i})M_{-k}(X_{i})&\lambda(X_{i})M_{-k}(X_{i})^{2}&0&0\\\
0&0&1&S_{-k}(X_{i})\\\ 0&0&S_{-k}(X_{i})&S_{-k}(X_{i})^{2}\\\
\end{matrix}\quad\right|\mathcal{I}_{-nk}\right].$

We substitute $\mathbb{E}[S_{-k}(X_{i})\mid\mathcal{I}_{-nk}]=0$ and
$\mathbb{E}[S_{-k}(X_{i})^{2}\mid\mathcal{I}_{-nk}]=V_{xnk}$. Finally,
$\Pi_{nk}^{\prime}Q_{ww,nk}\Pi_{nk}$ only normalizes the lower right corner to
1.

∎

######  Lemma 12 (Convergence of cross-fitted sums).

Consider a sequence of distributions $\\{\gamma_{n}\\}_{n=1}^{\infty}$ over a
collection of random matrices
$\\{Z_{i1},\ldots,Z_{iL}\\}_{i\in\mathcal{I}_{nk}}$ where $L$ is a finite
constant, $Z_{i\ell}\in\mathbb{R}^{M}\times\mathbb{R}^{B}$. Define
$\widehat{\zeta}_{nk}:=\sum_{i\in\mathcal{I}_{-nk}}\sum_{\ell=1}^{L}Z_{i\ell}$.
If for all $\ell\in\\{1,\ldots,L\\}$, (i) the observations are i.i.d.
conditional on $\mathcal{I}_{-nk}$, (ii)
$\mathbb{E}_{\gamma_{n}}[Z_{i\ell}\mid\mathcal{I}_{-nk}]=\uline{0}_{M\times
B}$, and (iii) $\mathbb{E}_{\gamma_{n}}[\|Z_{i\ell}\|^{2}]$ has a uniform
upper bound and $\gamma_{n}$, then as $n_{k}\to\infty$, (a)
$n_{k}^{-1/2}\widehat{\zeta}_{nk}=O_{p}(1)$, (b) If in addition,
$\mathbb{E}_{\gamma_{n}}[\|Z_{i\ell}\|^{2}]=o(1)$, then
$n_{k}^{-1/2}\widehat{\zeta}_{nk}=o_{p}(1)$, (c) Now suppose (ii) and (iii) do
not necessarily hold, but instead
$n_{k}^{r}\mathbb{E}_{\gamma_{n}}[\|Z_{i\ell}\|]=o(1)$ for all $\ell$ and some
$r>0$, then $n_{k}^{r-1}\widehat{\zeta}_{nk}=o_{p}(1)$.

  * Proof.

Since $M,L,B$ are all finite, it suffices to consider $\widehat{\zeta}_{nk\ell
mb}:=\sum_{i\in\mathcal{I}_{-nk}}Z_{i\ell mb}$, where $Z_{i\ell mb}$ is the
coordinate $(m,b)$ of $Z_{i\ell}$. By the law of iterated expectations,
$R_{nk\ell mb}:=\mathbb{E}_{\gamma_{n}}[(\widehat{\zeta}_{nk\ell
mb}-\mathbb{E}_{\gamma_{n}}[\widehat{\zeta}_{nk\ell
mb}\mid\mathcal{I}_{-nk}])^{2}]$ is equal to
$\mathbb{E}_{\gamma_{n}}[\mathbb{E}_{\gamma_{n}}[(\widehat{\zeta}_{nk\ell
mb}-\mathbb{E}_{\gamma_{n}}[\widehat{\zeta}_{nk\ell
mb}\mid\mathcal{I}_{-nk}])^{2}\mid\mathcal{I}_{-nk}]]$. Substituting the
definition of conditional variance,

$R_{nk\ell
mb}=\mathbb{E}_{\gamma_{n}}\left[\mathbb{V}_{\gamma_{n}}[\widehat{\zeta}_{nk\ell
mb}\mid\mathcal{I}_{-nk}]+\mathbb{E}_{\gamma_{n}}[\widehat{\zeta}_{nk\ell
mb}\mid\mathcal{I}_{-nk}]^{2}\right].$ (75)

The first term of (75) is $O(n_{k}^{-1})$. By Assumption (i),
$\widehat{\zeta}_{nk\ell mb}$ is a sum of $n_{k}$ variables that are i.i.d.
conditional on $\mathcal{I}_{-nk}$, and hence
$\mathbb{V}_{\gamma_{n}}[\widehat{\zeta}_{nk\ell
mb}\mid\mathcal{I}_{-nk}]=n_{k}\mathbb{V}_{\gamma_{n}}\left(Z_{i\ell
mb}\mid\mathcal{I}_{nk}\right)=n_{k}\mathbb{E}_{\gamma_{n}}\left[Z_{i\ell
mb}^{2}\mid\mathcal{I}_{nk}\right]$. By the law of iterated expectations,
$\mathbb{E}_{\gamma_{n}}\left[\mathbb{V}_{\gamma_{n}}[\widehat{\zeta}_{nk\ell
mb}\mid\mathcal{I}_{-nk}]\right]=n_{k}\mathbb{E}_{\gamma_{n}}\left[Z_{i\ell
mb}^{2}\right]$. The second term of (75) is zero by Assumption (ii).

To prove (a), we apply Chebyshev’s inequality
$\mathbb{P}(n_{k}^{-1}\widehat{\zeta}_{nk\ell
mb}>t)\leq\mathbb{E}_{\gamma_{n}}[Z_{i\ell mb}^{2}]/t^{2}$ for some $t>0$.
This shows that $n_{k}^{-1/2}\widehat{\zeta}_{nk\ell mb}=O_{p}(1)$. To prove
part (b), we use the condition that $\mathbb{E}_{\gamma_{n}}[Z_{i\ell
mb}^{2}]=o(1)$ to show that $n_{k}^{-1/2}\widehat{\zeta}_{nk\ell
mb}=o_{p}(1)$. To prove (c), we apply the triangle inequality,
$\|n_{k}^{r-1}\widehat{\zeta}_{nk\ell mb}\|\leq
n_{k}^{r-1}\sum_{i\in\mathcal{I}_{-nk}}\|Z_{i\ell mb}\|$. By Markov’s
inequality, $\mathbb{P}(\|n_{k}^{r-1}\widehat{\zeta}_{nk\ell mb}\|>t)\leq
n_{k}^{r}\mathbb{E}[\|Z_{i\ell mb}\|]/t=o(1)$, hence
$n_{k}^{r-1}\widehat{\zeta}_{nk\ell mb}=o_{p}(1)$.

∎

######  Lemma 13 (Derivatives of Influence Function).

Let $\varphi_{i}(\eta):=\varphi(Y_{i},D_{i},X_{i},\eta)$, for
$i\in\mathcal{I}_{nk}$, and $U_{i}=Y_{i}-\mathbb{E}[Y_{i}\mid D_{i},X_{i}]$.
Suppose that $\\{Y_{i},D_{i},X_{i}\\}_{i\in\mathcal{I}_{nk}}$ is independent
of $\\{Y_{i},D_{i},X_{i}\\}_{i\in\mathcal{I}_{-nk}}$ for all
$k\in\\{1,\ldots,K\\}$, and consider a set of $\eta\in\mathcal{T}$ where the
propensity score is bounded in $[\delta,1-\delta]$ for $\delta\in(0,1/2]$.
Then (a)
$\mathbb{E}\left[\frac{\partial\varphi_{i}(\eta)}{\partial(\eta_{1},\eta_{2},\eta_{4})^{\prime}}\mid
X=x,\mathcal{I}_{-nk}\right]=[0,0,-2(\tau(x)-\tau_{av})]$ almost surely, (b)
$\mathbb{E}\left[\left\|\partial\varphi_{i}(\eta)/\partial\eta_{\ell}\right\|^{4}\right]^{1/4}\leq(8/\delta)(\mathbb{E}[\|\eta(X_{i})\|^{4}]^{1/4}+\mathbb{E}\left[\left\|U_{i}\right\|^{4}\right]^{1/4})$,
and (c)
$\left\|\frac{\partial^{2}\varphi_{i}(\tilde{\eta})}{\partial(\eta_{1},\eta_{2},\eta_{4})\partial(\eta_{1},\eta_{2},\eta_{4})^{\prime}}\right\|\leq
18\times\sqrt{3}/\delta$ almost surely, for $\tilde{\eta}\in\mathcal{T}$.

  * Proof.

Part (a): The jacobian of $\varphi_{i}(\eta)$,
$\frac{\partial\varphi_{i}(\eta)}{\partial(\eta_{1},\eta_{2},\eta_{4})}$ is

$\displaystyle\begin{pmatrix}2(\tau(X_{i})-\tau_{av})\left[1-\frac{D_{i}}{p(X_{i})}\right]+2\left[\frac{D_{i}(Y_{i}-\mu_{0}(X_{i})-D_{i}\tau(X_{i}))}{p(X_{i})}-\frac{(1-D_{i})(Y_{i}-\mu_{0}(X_{i}))}{1-p(X_{i})}\right]\\\
2(\tau(X_{i})-\tau_{av})\left[-\frac{D_{i}}{p(X_{i})}+\frac{1-D_{i}}{1-p(X_{i})}\right]\\\
-2(\tau(X_{i})-\tau_{av})-2\left[\frac{D_{i}(Y_{i}-\mu_{0}(X_{i})-D_{i}\tau(X_{i}))}{p(X_{i})}-\frac{(1-D_{i})(Y_{i}-\mu_{0}(X_{i}))}{1-p(X_{i})}\right]\end{pmatrix}$

Part (a): Substituting $\mathbb{E}[D_{i}\mid X_{i}=x,\mathcal{I}_{-nl}]=p(x)$
and $\mathbb{E}[D_{i}Y_{i}\mid
X=x,\mathcal{I}_{-nk}]=p(x)(\mu_{0}(x)+\tau(x))$, then
$\mathbb{E}\left[\partial\varphi_{i}(\eta)/\partial(\eta_{1},\eta_{2},\eta_{4})^{\prime}\mid
X=x,\mathcal{I}_{-nk}\right]=[0,0,-2(\tau(x)-\tau)]^{\prime}$.

Part (b): By construction, $U_{i}=Y_{i}-\mu_{0}(X_{i})-D_{i}\tau(X_{i})$. Then
the jacobian simplifies to
$\partial\varphi_{i}(\eta)/\partial(\eta_{1},\eta_{2},\eta_{4})^{\prime}=G_{1i}+G_{2i}$,
where
$G_{1i}:=2(\tau(X_{i})-\tau_{av})\times[(1-D_{i}/p(X_{i})),(-D_{i}/p(X_{i})+(1-D)/(1-p(X_{i}))),-1]$
and $G_{2i}:=2(D_{i}/p(X_{i})-(1-D_{i})/(1-p(X_{i})))\times[U_{i},0,-U_{i}]$.
Since $p(x)\in[\delta,1-\delta]$,
$\|G_{1i}\|\leq(4/\delta)\|\tau(X_{i})-\tau_{av}\|\leq(8/\delta)\|\eta(x)\|$
and $\|G_{2i}\|\leq(8/\delta)\|U_{i}\|$. Therefore, by the triangle
inequality,
$\mathbb{E}\left[\left\|\partial\varphi_{i}(\eta)/\partial\eta_{\ell}\right\|^{4}\right]^{1/4}\leq(8/\delta)(\mathbb{E}[\|\eta(X_{i})\|^{4}]^{1/4}+\mathbb{E}\left[\left\|U_{i}\right\|^{4}\right])$.

Part (c): The hessian of $\varphi_{i}(\tilde{\eta})$, which I denote by
$H(\tilde{\eta})$, is symmetric and

$\displaystyle\frac{\partial^{2}\varphi_{i}(\tilde{\eta})}{\partial(\eta_{1},\eta_{2},\eta_{4})\partial(\eta_{1},\eta_{2},\eta_{4})^{\prime}}=\begin{pmatrix}2\left[1-\frac{D_{i}}{\tilde{p}(X_{i})}\right]-2\frac{D_{i}}{\tilde{p}(X_{i})}&\cdot&\cdot\\\
2\left[-\frac{D_{i}}{\tilde{p}(X_{i})}+\frac{1-D_{i}}{1-\tilde{p}(X_{i})}\right]&0&\cdot\\\
2\left[1-\frac{D_{i}}{\tilde{p}(X_{i})}\right]&-2\left[\frac{D_{i}}{\tilde{p}(X_{i})}-\frac{(1-D_{i})}{1-\tilde{p}(X_{i})}\right]&-2\end{pmatrix}$

Since $\tilde{p}(x)\in[\delta,1-\delta]$, then $\|D_{i}/\tilde{p}(X_{i})\|\leq
1/\delta$, $\|1-D_{i}/\tilde{p}(X_{i})\|\leq(1+1/\delta)\leq 2/\delta$, and
$\|D_{i}/\tilde{p}(X_{i})-(1-D_{i})/(1-\tilde{p}(X_{i}))\|\leq 2/\delta$. This
means that each of the entries of $H(\tilde{\eta})$ is bounded by $6/\delta$.
By Lemma 16, $\|H(\tilde{\eta})\|\leq
3\times(6/\delta)\times\sqrt{3}=18\times\sqrt{3}/\delta$.

∎

######  Lemma 14 (Decomposition of Nuisance Functions).

Define $\widehat{\eta}_{-k,\widehat{\theta}_{nk}}(x)$ as in (63),
$\theta_{n}:=(0,1,\tau_{n,av},1)$,
$(\widehat{\lambda}_{nk}-\lambda_{nk}):=[(\widehat{\theta}_{nk}-\theta_{n}),\widehat{\tau}_{nk,av}(\widehat{\theta}_{nk}-\theta_{n}),(\widehat{\tau}_{nk,av}-\tau_{n,av})]^{\prime}$,
and let $\\{e_{j}\\}_{j=1}^{4}$ be $4\times 1$ vectors with 1 in the $j^{th}$
coordinate and zero otherwise. Then there exist
$(x,\mathcal{I}_{-nk})-$measurable matrices $\Psi_{-nk}(x)$,
$\Delta_{-nk}(x)$, such that

$\widehat{\eta}_{-k,\widehat{\theta}_{nk}}(x)-\eta(x)=\Psi_{-nk}(x)(\widehat{\lambda}_{nk}-\lambda_{nk})+\Delta_{-nk}(x),$
(76)

and for some constant $C<\infty$, (a)
$e_{3}^{\prime}[\widehat{\eta}_{-k,\widehat{\theta}_{nk}}(x)-\eta(x)]=0$, (b)
$e_{3}^{\prime}\Delta_{-nk}(x)=e_{4}^{\prime}\Delta_{-nk}(x)=0$, (c)
$e_{4}^{\prime}\Psi_{-nk}=c$, for $c\in\mathbb{R}^{9}$, (d)
$\|\Psi_{-nk}(x)\|\leq C\times\left[1+2\|\widehat{\eta}_{-k}(x)\|\right]$
a.s., (e) $\|\Delta_{-nk}(x)\|\leq C\times\|\widehat{\eta}_{-k}(x)-\eta(x)\|$
a.s.

  * Proof of Lemma 14.

Define
$\widehat{B}(x):=[e_{1}(\widehat{W}(x,1)-\widehat{W}(x,0))^{\prime}+e_{2}\widehat{W}(x,0)^{\prime}]$.

$\displaystyle\widehat{\eta}_{-k,\widehat{\theta}_{nk}}(x)^{\prime}$
$\displaystyle=\widehat{B}(x)+e_{3}p_{n}(x)+e_{4}\widehat{\tau}_{nk,av},$
$\displaystyle\widehat{\eta}_{-k,\theta_{n}}(x)$
$\displaystyle=\widehat{B}(x)\theta_{n}+e_{3}p_{n}(x)+e_{4}\widehat{\tau}_{nk,av},$
$\displaystyle\eta(x)$
$\displaystyle=e_{1}\tau_{n}(x)+e_{2}\mu_{0n}(x)+e_{3}p_{n}(x)+e_{4}\tau_{n,av}.$

The estimation error can be decomposed as

$\widehat{\eta}_{-k,\widehat{\theta}_{nk}}(x)-\eta(x)=\widehat{B}(x)(\widehat{\theta}_{nk}-\theta_{n})+e_{4}[\widehat{\tau}_{nk,av}-\tau_{n,av}]+[\widehat{B}(x)^{\prime}\theta_{n}-e_{1}\tau_{n}(x)+e_{2}\mu_{0n}(x)].$
(77)

By definition
$\widehat{B}(x)=\Psi_{1,-nk}(x)+\widehat{\tau}_{nk,av}\Psi_{2,-nk}(x)$, with
auxiliary matrices
$\Psi_{1,-nk}:=e_{1}[0,0,1,\widehat{\tau}_{-k}(x)]+e_{2}[1,M_{-k}(X),-p_{n}(x),-p_{n}(x)\widehat{\tau}_{-k}(x)]$
and $\Psi_{2,-nk}:=e_{1}[0,0,0,-1]+e_{2}[0,0,0,p_{n}(x)]$. Substituting the
parameter $\theta_{n}:=(0,1,\tau_{n,av},1)$ and grouping common terms,
$\widehat{B}(x)^{\prime}\theta_{n}-e_{1}\tau_{n}(x)-e_{2}\mu_{0n}(x)=e_{1}[(\widehat{\tau}_{-k}(x)-\tau_{n}(x))+(\tau_{n,av}-\widehat{\tau}_{nk,av})]+e_{2}[M_{-k}(x)-\mu_{0n}(x)-p_{n}(x)\widehat{\tau}_{-k}(x)+p_{n}(x)(\widehat{\tau}_{nk,av}-\tau_{n,av})]$.
We can simplify the second term of this expression by substituting
$M_{-k}(x)=\widehat{\mu}_{0,-k}(x)+p_{n}(x)\widehat{\tau}_{-k}(x)$, which
produces
$e_{2}[(\widehat{\mu}_{0n}(x)-\mu_{0n}(x))+p_{n}(x)(\widehat{\tau}_{nk,av}-\tau_{n,av})]$.
Consequently, the second and third terms of (77) can be written as
$\Psi_{3,-nk}(x)(\widehat{\tau}_{nk,av}-\tau_{n,av})+\Delta_{-nk}(x)$, where
$\Psi_{3,-nk}(x):=-e_{1}+e_{2}p_{n}(x)+e_{4}$ and
$\Delta_{-nk}(x):=e_{1}(\tau_{-k}(x)-\tau_{n}(x))+e_{2}(\widehat{\mu}_{-k}(x)-\mu_{0n}(x))$.

Define $\Psi_{-nk}(x):=[\Psi_{1,-nl}(x),\Psi_{2,-nk}(x),\Psi_{3,-nk}(x)]$ and
the parameter error as
$(\widehat{\lambda}_{nk}-\lambda_{nk}):=[(\widehat{\theta}_{nk}-\theta_{n})^{\prime},\widehat{\tau}_{nk,av}(\widehat{\theta}_{nk}-\theta_{n})^{\prime},(\widehat{\tau}_{nk,av}-\tau_{n,av})]^{\prime}$.
Combining the results,

$\widehat{\eta}_{-k,\widehat{\theta}_{nk}}(x)-\eta(x)=\Psi_{-nk}(x)(\widehat{\lambda}_{nk}-\lambda_{nk})+\Delta_{-nk}(x).$

Measurability with respect to $(x,\mathcal{I}_{-nk})$ can be verified by
inspection. Property (a) follows from the fact that the propensity score is
known, and (b) because $\Psi_{-nk}(x)$ and $\Delta_{-nk}(x)$ depend on vectors
$e_{1},e_{2}$, which are orthogonal to $e_{3},e_{4}$. Property (c) follows by
the fact that $e_{4}^{\prime}\Psi_{-nk}=[\uline{0}_{1\times 8},1]$. To prove
part (d), we apply Lemma 16 to show that $\Psi_{-nk}(x)$ is bounded by
$9\sqrt{4}$ times the largest absolute value of the matrix. Since
$p_{n}(x)\leq 1$, then the largest value is bounded by
$1+\|\widehat{\mu}_{0,-k}(x)\|+\|\widehat{\tau}_{-k}(x)\|$ which is less than
$1+2\|\widehat{\eta}_{-k}(x)\|$. This means that $\|\Psi_{-nk}(x)\|\leq
12\times\sqrt{4}\times\left[1+2\|\widehat{\eta}_{-k}(x)\|\right]$. To prove,
part (e) we once again apply Lemma 16. The quantity $\Delta_{-nk}(x)$ is a
$4\times 1$ vector, whose individual entries are bounded by
$\|\widehat{\mu}_{0,-k}-\mu_{0n}(x)\|+\|\widehat{\tau}_{-k}(x)-\tau_{n}(x)\|$,
which is weakly less than $2\|\widehat{\eta}_{-k}(x)-\eta(x)\|$.

∎

######  Lemma 15 (Convergence of regression parameters).

Consider a sequence of distributions $\\{\gamma_{n}\\}_{n=1}^{\infty}$ that
satisfy Assumptions 1, 3, 4, 5, 6, and 7, and that $V_{\tau n}\to V_{\tau}>0$.
Define oracle regressors,
$W_{i}^{*}:=[1,\mu_{0n}(X_{i})+p_{n}(X_{i})\tau_{n}(X_{i}),(D_{i}-p_{n}(x_{i})),(D_{i}-p_{n}(X_{i}))(\tau_{n}(X_{i})-\tau_{av,n})]^{\prime}$
and
$\theta_{n}:=\mathbb{E}_{\gamma_{n}}[\lambda_{n}(X_{i})W_{i}^{*}W_{i}^{*^{\prime}}]^{-1}\mathbb{E}_{\gamma_{n}}[\lambda_{n}(X_{i})W_{i}^{*}Y_{i}]$.
Then (a) $\widehat{\tau}_{nk,av}-\tau_{n,av}=o_{p}(n_{k}^{-1/4})$, (b)
$\theta_{n}=(0,1,\tau_{n,av},1)^{\prime}$, and (c)
$\widehat{\theta}_{nk}-\theta_{n}=o_{p}(n_{k}^{-1/4})$.

  * Proof.

Part (a): Decompose,
$n_{k}^{1/4}(\widehat{\tau}_{nk,av}-\tau_{av,n})=n_{k}^{-1/4}\left[\frac{1}{\sqrt{n_{k}}}\sum_{i\in\mathcal{I}_{nk}}\tau_{n}(X_{i})-\tau_{n,av}\right]+n_{k}^{1/4-1}\sum_{i\in\mathcal{I}_{nk}}[\widehat{\tau}_{-k}(X_{i})-\tau_{n}(X_{i})]$.
The first term is a centered random variable that is $o_{p}(1)$. By Assumption
7.(iv), and the Cauchy-Schwartz inequality,
$\mathbb{E}_{\gamma_{n}}[n_{k}^{1/4}\|\widehat{\tau}_{-k}(X_{i})-\tau_{n}(X_{i})]\|\leq\mathbb{E}_{\gamma_{n}}[n_{k}^{1/2}\|\widehat{\eta}_{-k}(X_{i})-\eta(X_{i})\|^{2}]^{1/2}\to
o(1)$. By applying Lemma 12.(c),
$n_{k}^{1/4}(\widehat{\tau}_{nk,av}-\tau_{av,n})=o_{p}(1)$.

Part (b): For given $\\{X_{i}=x,D_{i}=d\\}$,
$W_{i}^{*^{\prime}}(0,1,\tau_{n_{a}v},1)^{\prime}=\mu_{d}(x)$. Therefore, by
applying Lemma 5, $\theta_{n}=(0,1,\tau_{n,av},1)^{\prime}$.

Parts (c): Define
$\widehat{Q}_{ww}:=\frac{1}{n_{k}}\sum_{i\in\mathcal{I}_{nk}}\lambda_{n}(X_{i})\widehat{W}_{i}\widehat{W}_{i}^{\prime}$,
$Q_{ww}:=\mathbb{E}_{\gamma_{n}}[\lambda_{n}(X_{i})W_{i}^{*}W_{i}^{*^{\prime}}]$,
$M_{n}(x)=\mu_{0n}(x)+p_{n}(x)\tau_{n}(x)$. Following similar derivations to
Lemma 11,

$Q_{ww}=\begin{pmatrix}\mathbb{E}_{\gamma_{n}}[\lambda(X_{i})]&\mathbb{E}_{\gamma_{n}}[\lambda(X_{i})(\mu_{n}(X_{i}))]&0&0\\\
\mathbb{E}_{\gamma_{n}}[\lambda(X_{i})(\mu_{n}(X_{i}))]&\mathbb{E}_{\gamma_{n}}[\lambda(X_{i})(\mu_{n}(X_{i}))^{2}]&0&0\\\
0&0&1&0\\\ 0&0&0&V_{\tau n}\\\ \end{pmatrix}.$ (78)

The upper left block has bounded eigenvalues by Assumption 3.(i) and $V_{\tau
n}$ is asymptotically bounded. Therefore $Q_{ww}$ is positive definite with
bounded eigenvalues. Furthermore, $\widehat{Q}_{ww}-Q_{ww}$ can be decomposed
as:

$\left[\frac{1}{n_{k}}\sum_{i\in\mathcal{I}_{nk}}\lambda_{n}(X_{i})W_{i}^{*}W_{i}^{*^{\prime}}-\mathbb{E}_{\gamma_{n}}[\lambda_{n}(X_{i})W_{i}^{*}W_{i}^{*^{\prime}}]\right]+\left[\frac{1}{n_{k}}\sum_{i\in\mathcal{I}_{nk}}\lambda_{n}(X_{i})(\widehat{W_{i}}\widehat{W}_{i}^{\prime}-\widehat{W_{i}}^{*}\widehat{W}_{i}^{*^{\prime}})\right].$
(79)

The first term of (79) is an average of mean-zero random variables and bounded
variance, then it is $O_{p}(n_{k}^{-1/2})=o_{p}(n_{k}^{-1/4})$. To prove that
it has bounded variance, apply Lemma 16, then
$\|W_{i}^{*}\|\leq\sqrt{4}(1+\|\mu_{0n}(X_{i})\|+\|\tau_{n}(X_{i})\|)\leq\sqrt{4}(1+2\|\eta(X_{i})\|)$.
Since $p_{n}(x)\in[\delta,1-\delta]$ and
$\lambda_{n}(X_{i})=[p_{n}(X_{i})(1-p_{n}(X_{i}))]^{-1}$, then
$\mathbb{E}_{\gamma_{n}}[\|\lambda_{n}(X_{i})W_{i}^{*}\|^{2}]^{1/2}\leq(1/\delta^{2})\mathbb{E}_{\gamma_{n}}[\|W_{i}^{*}\|^{4}]^{1/4}\leq(\sqrt{4}/\delta^{2})(1+2\mathbb{E}_{\gamma_{n}}[\|\eta(X_{i})\|^{4}]^{1/4})$,
which is bounded by Assumption 7.(ii).

To bound the second term of (79), we apply the triangle inequality,
$\|\lambda_{n}(X_{i})(\widehat{W}_{i}\widehat{W}_{i}^{\prime}-W_{i}^{*}W_{i}^{*^{\prime}})\|\leq(1/\delta^{2})\widehat{\phi}_{i}$,
where $\widehat{\phi}_{i}:=2\|W_{i}^{*^{\prime}}\|\
\|\widehat{\zeta}_{i}\|+\|\widehat{\zeta}_{i}\|^{2}$ and
$\widehat{\zeta}_{i}:=\widehat{W}_{i}-W_{i}^{*}$. Our goal is to show that
$\frac{1}{n_{k}}\sum_{i\in\mathcal{I}_{-nk}}\widehat{\phi}_{i}=o_{p}(n_{k}^{-1/4})$.
Let $e_{\ell}$ be a $4\times 1$ vector with one in the $\ell^{th}$ entry and
zero otherwise. We can further decompose
$\widehat{\zeta}_{i}=\widehat{\Delta}_{nk}\widehat{\zeta}_{-nk,1}(X_{i})+\widehat{\zeta}_{-nk,2}(X_{i})$,
into components that map into our assumptions:
$\widehat{\Delta}_{nk}:=(\widehat{\tau}_{nk,av}-\tau_{n,av})$,
$\widehat{\zeta}_{-nk,1}(X_{i}):=e_{4}(D_{i}-p_{n}(X_{i}))$ and
$\widehat{\zeta}_{-nk,2}(X_{i}):=e_{1}(M_{-k}(X_{i})-M_{n}(X_{i}))+e_{2}(D_{i}-p_{n}(X_{i}))(\widehat{\tau}_{-k}(X_{i})-\tau_{n}(X_{i}))$.
By construction, $\|\zeta_{-nk,1}(X_{i})\|\leq 1$. Applying the triangle
inequality and grouping terms,
$\widehat{\phi}_{i}\leq\|\widehat{\Delta}_{nk}\|^{2}+\|\widehat{\Delta}_{nk}\|\widehat{\phi}_{i1}+\widehat{\phi}_{i2}$,
where $\widehat{\phi}_{i1}:=(2\|W_{i}^{*}\|+2\|\widehat{\zeta}_{-nk,2}\|)$,
and $\widehat{\phi}_{i2}:=(2\|W_{i}^{*}\|\
\|\widehat{\zeta}_{-nk,2}(X_{i})\|+\|\widehat{\zeta}_{-nk,2}(X_{i})\|^{2})$.
By Assumption 7.(iv),
$\mathbb{E}_{\gamma_{n}}[\|\widehat{\zeta}_{-nk,2}(X_{i})\|^{2}]^{1/2}\leq
3\mathbb{E}_{\gamma_{n}}[\|\widehat{\eta}_{-k}(X_{i})-\eta(X_{i})\|^{2}]^{1/2}=o(n_{k}^{-1/4})$.
Therefore, by the Cauchy-Schwarz inequality,
$\mathbb{E}_{\gamma_{n}}[\widehat{\phi}_{i\ell}^{2}]^{1/2}=o(n_{k}^{-1/4})$
for $\ell\in\\{1,2\\}$. Then by Lemma 12.(c),
$\left[\frac{1}{n_{k}}\sum_{i\in\mathcal{I}_{-nk}}\widehat{\phi}_{i_{\ell}}\right]=o_{p}(n_{k}^{-1/4})$
for $\ell\in\\{1,2\\}$. Combining terms,

$\left\|\frac{n_{k}^{1/4}}{n_{k}}\sum_{i\in\mathcal{I}_{-nk}}\lambda_{n}(X_{i})(\widehat{W_{i}}\widehat{W}_{i}^{\prime}-\widehat{W_{i}}^{*}\widehat{W}_{i}^{*^{\prime}})\right\|\leq\frac{1}{\delta^{2}}\sum_{\ell=0}^{2}n_{k}^{1/4}\|\widehat{\Delta}_{nk}\|^{\ell-2}\left[\frac{1}{n_{k}}\sum_{i\in\mathcal{I}_{-nk}}\widehat{\phi}_{i_{\ell}}\right]=o_{p}(1).$

Define
$\widehat{Q}_{wy}:=\frac{1}{n_{k}}\sum_{i\in\mathcal{I}_{nk}}\lambda_{n}(X_{i})\widehat{W}_{i}Y_{i}$
and $Q_{wy}:=\mathbb{E}_{\gamma_{n}}[\lambda_{n}(X_{i})W_{i}^{*}Y_{i}]$. We
can apply similar arguments as above to show that
$\widehat{Q}_{wy}-Q_{wy}=o_{p}(n_{k}^{-1/4})$.

Substituting the definition, $\theta_{n}:=Q_{ww}^{-1}Q_{wy}$ and rearranging
terms,
$n_{k}^{1/4}(\widehat{\theta}_{nk}-\theta_{n})=n_{k}^{1/4}(\widehat{Q}_{ww}^{-1}\widehat{Q}_{wy}-\theta_{nk})=\widehat{Q}_{ww}^{-1}n_{k}^{1/4}(\widehat{Q}_{wy}-Q_{wy})-n_{k}^{1/4}(Q_{ww}^{-1}Q_{wy}-\widehat{Q}_{ww}^{-1}Q_{wy})$.
The first term is $o_{p}(n_{k}^{-1/4})$. The second term can be rewritten as
$n_{k}^{1/4}(Q_{ww}^{-1}-\widehat{Q}_{ww}^{-1})Q_{wy}=o_{p}(n_{k}^{-1/4})$. To
prove this, note that
$\|Q_{ww}^{-1}-\widehat{Q}_{ww}^{-1}\|=\|Q_{ww}^{-1}(\widehat{Q}_{ww}-Q_{ww})\widehat{Q}_{ww}^{-1}\|\leq\|Q_{ww}^{-1}\|\
\|\widehat{Q}_{ww}-Q_{ww}\|\ \|\widehat{Q}_{ww}^{-1}\|$. The right-hand side
is $o_{p}(n_{k}^{-1/4})$ since $Q_{ww}$ has eigenvalues bounded away from zero
and $\widehat{Q}_{ww}$ converges to its true value at rate $n_{k}^{1/4}$.
Combining the results produces
$n_{k}^{1/4}(\widehat{\theta}_{nk}-\theta_{n})=o_{p}(n_{k}^{-1/4})$. ∎

######  Lemma 16 (Bound on Operator Norm).

Let $H$ be an $M\times L$ matrix and let
$\|H\|=\sup_{\\{z\in\mathbb{R}^{L}:\|z\|=1\\}}\|Hz\|$ be corresponding matrix
operator norm. The absolute value of the individual entries of $H$ is bounded
a constant $C$. Then $\|H\|\leq LC\sqrt{M}$.

  * Proof of Lemma 16.

Let $H_{m}$ be the $m^{th}$ row and $H_{m\ell}$ be the $(m,\ell)$ entry. Then
$\|H\|=\sup_{\\{z\in\mathbb{R}^{L}:\|z\|=1\\}}\sqrt{\sum_{m=1}^{M}\left[H_{m}z\right]^{2}}=\sup_{\\{z\in\mathbb{R}^{L}:\|z\|=1\\}}\sqrt{\sum_{m=1}^{M}\left[\sum_{\ell=1}^{L}H_{m\ell}z_{\ell}\right]^{2}}$.
Since $|z_{\ell}|\leq 1$ and $\|H_{m\ell}\|\leq C$,
$\|H\|=\sup_{\\{z\in\mathbb{R}^{L}:\|z\|=1\\}}\sqrt{\sum_{m=1}^{M}\left[\sum_{\ell=1}^{L}\left|H_{m\ell}\right|\
\left|z_{\ell}\right|\right]^{2}}\leq CL\sqrt{M}$. ∎