argument to that given in the first bullet point in Case 2 of (c,
$\Leftarrow$), there exists an increasing convex $g$ defined on $E_{\nu}$ that
agrees $\nu$-almost everywhere with $f$; recall that $E_{\nu}$ contains
$M_{\nu}^{R}$ if and only if $\nu(\\{M_{\nu}^{R}\\})>0$. Since
$\mathcal{F}_{\nu}^{m}=\\{[f]_{\nu}:f\in\mathcal{F}^{m}\\}\subseteq\mathcal{L}^{2}(\nu)$
by (a), we deduce that

$\operatorname{Cl}\mathcal{F}_{\nu}^{m}\subseteq\\{[f]_{\nu}:f\in
L^{2}(\nu)\text{ and}\left.f\right|_{E_{\nu}}\\!\text{ is convex and
increasing}\\}.$

For the reverse inclusion, we split into the two cases considered in (c,
$\Rightarrow$) above.

Case 1 – $\nu(\\{M_{\nu}^{R}\\})=0$: Here, we have $M_{\nu}^{R}\notin
E_{\nu}$. For a fixed $f\in L^{2}(\nu)$ such that $\left.f\right|_{E_{\nu}}$
is convex and increasing, we claim that there exists a sequence
$(f_{n})_{n=1}^{\infty}$ in $\mathcal{F}^{m}$ such that
$\|f_{n}-f\|_{L^{2}(\nu)}\to 0$. Indeed, fix a sequence
$(x_{n}\in\operatorname{supp}\nu\setminus\\{M_{\nu}^{R}\\}:n\in\mathbb{N})$
such that $x_{n}\nearrow M_{\nu}^{R}$, and for each $n$, observe that since
$\left.f\right|_{E_{\nu}}$ has a finite and non-negative subgradient at
$x_{n}$, there exists an increasing convex $f_{n}\in\mathcal{F}^{m}$ such that
$f_{n}=f$ on $E_{\nu}\cap[0,x_{n}]$, $f_{n}$ is linear on $[x_{n},1]$ and
$f_{n}\leq f$ on $E_{\nu}$. Thus, since $\nu(E_{\nu})=1$, $\inf_{E_{\nu}}f\leq
f_{n}\leq\sup_{E_{\nu}}f$ on $E_{\nu}$ for all $n$ and $f_{n}\to f$ pointwise
on $E_{\nu}$, it follows by the dominated convergence theorem that
$\|f_{n}-f\|_{L^{2}(\nu)}^{2}=\int_{E_{\nu}}\,\lvert f_{n}-f\rvert^{2}\to 0$,
as required.

Case 2 – $\nu(\\{M_{\nu}^{R}\\})>0$: Note that $M_{\nu}^{R}<m$ and
$M_{\nu}^{R}\in E_{\nu}$ in this case. As before, take any $f\in L^{2}(\nu)$
such that $\left.f\right|_{E_{\nu}}$ is convex and increasing, and fix a
sequence
$(x_{n}\in\operatorname{supp}\nu\setminus\\{M_{\nu}^{R}\\}:n\in\mathbb{N})$
such that $x_{n}\nearrow M_{\nu}^{R}$. For each $n$, let $f_{n}$ be the
(unique) function that satisfies $f_{n}=f$ on $[0,x_{n}]\cup\\{M_{\nu}^{R}\\}$
and is linear on $[x_{n},1]$. Then $f_{n}\in\mathcal{F}^{m}$ for all $n$ by
the convexity of $f$, and $f_{n}\to f$ pointwise on $E_{\nu}$. Thus, since
$\inf_{E_{\nu}}f\leq f_{n}\leq f(M_{\nu}^{R})<\infty$ on $E_{\nu}$ for all
$n$, we can once again apply the dominated convergence theorem to deduce that
$\|f_{n}-f\|_{L^{2}(\nu)}^{2}=\int_{E_{\nu}}\,\lvert f_{n}-f\rvert^{2}\to 0$.
This shows that
$\operatorname{Cl}\mathcal{F}_{\nu}^{m}\supseteq\\{[f]_{\nu}:f\in
L^{2}(\nu)\text{ and}\left.f\right|_{E_{\nu}}\\!\text{ is convex and
increasing}\\}$ in this case.

Straightforward modifications of the arguments above yield the analogous
conclusion when $\operatorname{supp}\nu\subseteq[m,1]$. This completes the
proof. ∎

###### Proof of Corollary S10.

(a) For $f\in L^{2}(P^{X})$, it is immediate from (S34) that
$f\in\psi_{m}^{*}(P)$ if and only if $[f]_{P^{X}}\in\mathcal{L}^{2}(P^{X})$ is
the projection of $[f_{P}]_{P^{X}}$ onto
$\operatorname{Cl}\mathcal{F}_{P^{X}}^{m}$, which is a closed, convex subset
of the Hilbert space $\mathcal{L}^{2}(P^{X})$ by Proposition S9.

(b) This follows directly from the definition of $\psi_{m}^{*}(P)$ and the
fact that every $f\in\mathcal{F}^{m}$ is bounded on $[0,1]$.

(c) Since
$\operatorname{Cl}\mathcal{F}_{P^{X}}^{m}=\operatorname{Cl}\mathcal{F}_{P^{X}}^{\tilde{m}}$
by Proposition S9(b), we have $\psi_{m}^{*}(P)=\psi_{\tilde{m}}^{*}(P)$ by
definition, and $L_{m}^{*}(P)=L_{\tilde{m}}^{*}(P)$ by the observation after
(S34). If there exists $f\in\psi_{m}^{0}(P)$, then setting
$\tilde{f}(x):=f(M_{L}\vee x\wedge M_{R})$ for $x\in[0,1]$ with
$M_{L}:=\min(\operatorname{supp}P^{X})$ and
$M_{R}:=\max(\operatorname{supp}P^{X})$, we have
$\tilde{f}\in\psi_{\tilde{m}}^{0}(P)$.

(d) If condition (i) holds, then
$\mathcal{F}_{P^{X}}^{m}=\operatorname{Cl}\mathcal{F}_{P^{X}}^{m}$ by
Proposition S9(c). Thus, there exists $f^{*}\in\mathcal{F}^{m}$ such that
$\psi^{*}(P)=[f^{*}]_{P^{X}}$ by (a) above, whence $f^{*}\in\psi_{m}^{0}(P)$.

Suppose now that condition (ii) holds, in which case $m=\tilde{m}$ and there
exist a regression function $f_{P}$ for $P$ and $b,\varepsilon>0$ such that
$\lvert f_{P}\rvert\leq b$ on $(m-\varepsilon,m+\varepsilon)$. We may assume
without loss of generality that $m=\max(\operatorname{supp}P^{X})$; the case
$m=\min(\operatorname{supp}P^{X})$ is similar. Suppose for a contradiction
that $\psi_{m}^{0}(P)=\emptyset$, i.e. that
$\psi^{*}(P)\in\bigl{(}\operatorname{Cl}\mathcal{F}_{P^{X}}^{m}\bigr{)}\setminus\mathcal{F}_{P^{X}}^{m}$.
In view of condition (i) in Proposition S9(c), this can only happen if
$P^{X}(\\{m\\})=0$. By Proposition S9(d), we can write
$\psi_{m}^{*}(P)=[f]_{P^{X}}$ for some $f\in L^{2}(P^{X})$ that is convex and
increasing on ${\operatorname{Int}(\operatorname{csupp}P^{X})}$. Since
$[f]_{P^{X}}=\psi_{m}^{*}(P)\notin\mathcal{F}_{P^{X}}^{m}$, the function
$\left.f\right|_{\operatorname{Int}(\operatorname{csupp}P^{X})}$ cannot be
extended to an element of $\mathcal{F}^{m}$, so we must have $f(x)\to\infty$
as $x\nearrow m$.

Therefore, we can find
$m^{\prime}\in\operatorname{Int}(\operatorname{csupp}P^{X})\cap(m-\varepsilon,m)$
such that $f(x)>b$ for all $x\in[m^{\prime},m)$. Since
$\left.f\right|_{\operatorname{Int}(\operatorname{csupp}P^{X})}$ has a finite
and non-negative subgradient at $m^{\prime}$, there exists an increasing
convex $\tilde{f}\in\mathcal{F}^{m}$ such that $\tilde{f}=f$ on
$\operatorname{Int}(\operatorname{csupp}P^{X})\cap[0,m^{\prime}]$, $\tilde{f}$
is linear on $[m^{\prime},1]$ and $f_{P}\leq b<\tilde{f}\leq f$ on
$[m^{\prime},m)$. But this means that
$\|\tilde{f}-f_{P}\|_{L^{2}(P^{X})}<\|f-f_{P}\|_{L^{2}(P^{X})}$, so
$L_{m}^{*}(P)\leq L(\tilde{f},P)<L(f,P)$ by (S34), contradicting the fact that
$f\in\psi_{m}^{*}(P)$. Thus, $\psi_{m}^{*}(P)\in\mathcal{F}_{P^{X}}^{m}$,
whence $\psi_{m}^{0}(P)\neq\emptyset$.

(e) If $f,g\in\psi_{m}^{0}(P)$, then $f\sim_{P^{X}}g$ by (a), so $f=g$ on some
dense subset $S\subseteq\operatorname{supp}P^{X}$; see the first paragraph of
the proof of Proposition S9(c). It follows that $f=g$ on
$(\operatorname{supp}P^{X})\setminus\\{m\\}$, a set on which both $f,g$ are
continuous. If $f,g$ are both continuous on $[0,1]$, then $f=g$ on
$\operatorname{supp}P^{X}$ by the same argument. If in addition
$P^{X}(\\{m\\})>0$, then clearly $f(m)=g(m)$.

(f) The forward implication was established in (e). For the converse, suppose
that $m\in\operatorname{supp}P^{X}$ and there is some $f\in\psi_{m}^{0}(P)$
that is discontinuous at $m$, so that $\lim_{x\nearrow m}f(x)<\lim_{x\searrow
m}f(x)$. If $P^{X}(\\{m\\})=0$, then any $\psi_{m}^{0}(P)$ contains any
$\tilde{f}\colon[0,1]\to\mathbb{R}$ such that $\tilde{f}=f$ on
$[0,1]\setminus\\{m\\}$ and $\lim_{x\nearrow
m}f(x)\leq\tilde{f}(m)\leq\lim_{x\searrow m}f(x)$, so the elements of
$\psi_{m}^{0}(P)$ do not all agree at $m\in\operatorname{supp}P^{X}$.

(g) Suppose that $\psi_{m}^{0}(P)$ contains a continuous function
$h\in\mathcal{F}^{m}$. For any other $f\in\psi_{m}^{0}(P)$, we know from (e)
that $f=h$ on $(\operatorname{supp}P^{X})\setminus\\{m\\}$. In view of the
continuity of $h$ at $m$ and the assumption that $\operatorname{supp}P^{X}$
has non-empty intersection with both $(m-\varepsilon,m)$ and
$(m,m+\varepsilon)$ for all $\varepsilon>0$, this forces $f(m)=h(m)$, so $f=h$
on $\operatorname{supp}P^{X}$ and $f$ is continuous. ∎

The proof of Proposition S12 relies on the following three key lemmas. Let the
marginal distribution $P^{X}$ on $[0,1]$ be as in Proposition S12, and define
$M_{L}:=\min(\operatorname{supp}P^{X})$,
$M_{R}:=\max(\operatorname{supp}P^{X})$ and
$C:=[M_{L},M_{R}]=\operatorname{csupp}P^{X}$.

######  Lemma S26.

Fix $x\in\operatorname{Int}C$ and $\ell\in[0,\infty)$. Let $(P_{n})$ be a
sequence in $\mathcal{P}$ that converges weakly to some $P\in\mathcal{P}$.
Then there exists $B\equiv B(x,\ell,P)<\infty$ such that for any sequence of
increasing functions $f_{n}\colon[0,1]\to\mathbb{R}$ with
$\limsup_{n\to\infty}L(f_{n},P_{n})\leq\ell$ for all $n$, we have
$\limsup_{n\to\infty}\,\lvert f_{n}(x)\rvert<B$.

###### Proof.

Since $x\in\operatorname{Int}C$, we have $P^{X}([0,x))\wedge P^{X}((x,1])>0$.
Let $(f_{n})$ be as above. Then for each $n$, note that since $f_{n}$ is
increasing, we have

$L(f_{n},P_{n})\geq\int_{[x,1]\times\mathbb{R}}\,\bigl{(}y-f_{n}(x)\bigr{)}^{2}\,dP_{n}(x,y)\geq\left(\frac{f_{n}^{+}(x)}{2}\right)^{2}P_{n}\left([x,1]\times\left(-\infty,\frac{f_{n}^{+}(x)}{2}\right]\right);$
(S69)

indeed, if $f_{n}(x)\leq 0$, then (S69) holds trivially, and if $f_{n}(x)>0$,
then for all $x^{\prime}\in[x,1]$ and $y^{\prime}\leq f_{n}(x)/2$, we have
$f_{n}(x^{\prime})-y^{\prime}\geq f_{n}(x)-y^{\prime}\geq f_{n}(x)/2>0$.

Now let $(f_{n_{k}})$ be a subsequence such that
$f_{n_{k}}^{+}(x)\to\limsup_{n\to\infty}f_{n}^{+}(x)=:2s$ as $k\to\infty$.
Since $P_{n}\overset{d}{\to}P$, an application of the portmanteau lemma (van
der Vaart, 1998, Lemma 2.2) shows that

$\liminf_{k\to\infty}\,\left(\frac{f_{n_{k}}^{+}(x)}{2}\right)^{2}\,P_{n_{k}}\\!\left((x,1]\times\left(-\infty,\frac{f_{n_{k}}^{+}(x)}{2}\right)\right)\geq
s^{2}\,P\bigl{(}(x,1]\times(-\infty,s)\bigr{)}.$

It follows from this and (S69) that
$s^{2}\,P\bigl{(}(x,1]\times(-\infty,s)\bigr{)}\leq\limsup_{n\to\infty}L(f_{n},P_{n})\leq\ell$.
Since $P\bigl{(}(x,1]\times(-\infty,b)\bigr{)}\to P^{X}((x,1])>0$ as
$b\to\infty$, we can therefore find $B^{\prime}\equiv
B^{\prime}(x,\ell,P)<\infty$ such that
$\limsup_{n\to\infty}\,f_{n}^{+}(x)=2s<B^{\prime}$ for any sequence $(f_{n})$
satisfying the conditions of Lemma S26. An analogous argument yields the same
conclusion for $\limsup_{n\to\infty}\,f_{n}^{-}(x)$. ∎

For sequences of S-shaped functions, the conclusion of Lemma S26 can be
strengthened.

######  Lemma S27.

Fix $\ell\in[0,\infty)$ and let $(m_{n})$ be a sequence in $[0,1]$ that
converges to some fixed $m_{0}\in\operatorname{Int}C$. Then under the
hypotheses of Lemma S26, there exists
$\tilde{B}\equiv\tilde{B}(m_{0},\ell,P)<\infty$ such that for any sequence
$(f_{n})$ with $f_{n}\in\mathcal{F}^{m_{n}}$ for all $n$ and
$\limsup_{n\to\infty}L(f_{n},P_{n})\leq\ell$, we have
$\limsup_{n\to\infty}\sup_{x\in[0,1]}\,\lvert f_{n}(x)\rvert<\tilde{B}$.

###### Proof.

Since $m_{n}\to m_{0}\in\operatorname{Int}C$, we can find
$a_{L},a_{R}\in\operatorname{Int}C$ such that $0<a_{L}<m_{0}<a_{R}<1$, so that
for all sufficiently large $n\in\mathbb{N}$, we have $f_{n}(a_{L})\leq
f_{n}(m_{n})\leq f_{n}(a_{R})$ and

$\frac{m_{n}f_{n}(a_{L})-a_{L}f_{n}(m_{n})}{m_{n}-a_{L}}\leq f_{n}(0)\leq
f_{n}(x)\leq
f_{n}(1)\leq\frac{(1-m_{n})f_{n}(m_{n})-(1-a_{R})f_{n}(a_{L})}{a_{R}-m_{n}}$
(S70)

for all $x\in[0,1]$. Since $\limsup_{n\to\infty}L(f_{n},P_{n})\leq\ell$, we
have $\limsup_{n\to\infty}\,\lvert f_{n}(x)\rvert\leq B(x,\ell,P)<\infty$ for
$x\in\\{a_{L},a_{R}\\}$ by Lemma S26, so the result follows from (S70) and the
fact that $m_{n}\to m_{0}$. ∎

Another important consequence of Lemma S26 is the following. Recall that
$\tilde{m}_{0}=\operatorname*{argmin}_{x\in C}\,\lvert x-m_{0}\rvert$.

######  Lemma S28.

Under the hypotheses of Proposition S12, let $(f_{n})_{n=1}^{\infty}$ be any
sequence with $f_{n}\in\mathcal{F}^{m_{n}}$ and
$\limsup_{n\to\infty}L(f_{n},P_{n})<\infty$ for all $n$. Then for every
subsequence $(g_{k})_{k=1}^{\infty}\equiv(f_{n_{k}})_{k=1}^{\infty}$, there is
a further subsequence $(g_{k_{\ell}})$ and a function $g$ defined on $[0,1]$
with the following properties:

  1. (i)

$g$ is increasing on $C\setminus\\{\tilde{m}_{0}\\}$, convex on
$C\cap[0,\tilde{m}_{0})$ and concave on $C\cap(\tilde{m}_{0},1]$.

  2. (ii)

$g_{k_{\ell}}\to g$ pointwise on $C$ and uniformly on closed subsets of
$C\setminus\\{\tilde{m}_{0}\\}$. In particular, if $z_{\ell}\to z\in
C\setminus\\{\tilde{m}_{0}\\}$, then $g_{k_{\ell}}(z_{\ell})\to g(z)$.

  3. (iii)

$g(x)\in\mathbb{R}$ for all $x\in C\setminus\\{\tilde{m}_{0}\\}$, and
$g(\tilde{m}_{0})\in(-\infty,\infty]$ if $\tilde{m}_{0}>M_{L}$ and
$g(\tilde{m}_{0})\in[-\infty,\infty)$ if $\tilde{m}_{0}<M_{R}$.

  4. (iv)

$g\in\mathcal{F}^{m_{0}}$ if $m_{0}\in\operatorname{Int}C$ and
$[g]_{P^{X}}\in\operatorname{Cl}\mathcal{F}_{P^{X}}^{m_{0}}\subseteq\mathcal{L}^{2}(P^{X})$
if $P^{X}(\\{\tilde{m}_{0}\\})=0$.

  5. (v)

Let $Q_{\ell}:=P_{n_{k_{\ell}}}$ for each $\ell$. If
$P^{X}(\\{\tilde{m}_{0}\\})=0$, then
$\liminf_{\ell\to\infty}L(g_{k_{\ell}},Q_{\ell})\geq L(g,P)\geq
L_{m_{0}}^{*}(P)$.

  6. (vi)

If $P_{n}=P$ for all $n$, then we can ensure that the conclusions of (iii) and
(iv) hold in all cases, even when the assumptions on $\tilde{m}_{0}$ are
dropped.

###### Proof.

Fix a subsequence $(g_{k})_{k=1}^{\infty}\equiv(f_{n_{k}})_{k=1}^{\infty}$. In
the setting of Proposition S12, we have $m_{n_{k}}\to m_{0}$, so if $0\leq
z<m_{0}<w\leq 1$, then all but finitely many of the functions $g_{k}$ are
convex on $[0,z]$ and concave on $[w,1]$. Since
$\\{g_{k}(x)\\}_{k=1}^{\infty}$ is bounded for all $x\in\operatorname{Int}C$
by Lemma S26, it follows from Rockafellar (1997, Theorem 10.9) and Lemma S29
that whenever $0\leq z<m_{0}<w\leq 1$, there is a subsequence of $(g_{k})$
that converges pointwise on $C\setminus(z,w)$. By considering sequences
$z_{n}\nearrow m_{0}$ and $w_{n}\searrow m_{0}$ with $z_{n}<m_{0}<w_{n}$ for
all $n$, we deduce by a diagonal argument that $(g_{k})$ has a subsequence
that converges pointwise to some $g\colon
C\setminus\\{\tilde{m}_{0}\\}\to\mathbb{R}$ on $C\setminus\\{\tilde{m}_{0}\\}$
and uniformly on closed subsets of $C\setminus\\{\tilde{m}_{0}\\}$.

To extend $g$ to $[0,1]\setminus\\{\tilde{m}_{0}\\}$, we set $g(x)=g(M_{L})$
for all $x\in[0,M_{L})$ and $g(x)=g(M_{R})$ for all $x\in(M_{R},1]$ if
$m_{0}\in\operatorname{Int}C$, and otherwise set $g(x)=0$ for all
$x\in[0,1]\setminus C$ if $m_{0}\notin\operatorname{Int}C$. Finally, by
extracting a further subsequence $(g_{k_{\ell}})$ if necessary, we can ensure
that $g_{k_{\ell}}(\tilde{m}_{0})$ converges to some $L\in[-\infty,\infty]$ as
$\ell\to\infty$, and we extend $g$ to $[0,1]$ by setting $g(\tilde{m}_{0})=L$.

(i) This follows from the construction of $g$ in the first paragraph. Note
also that if $m_{0}\notin\operatorname{Int}C$, then $g$ is increasing on $C$,
and $g$ is either concave or convex on $C$, depending on whether
$\tilde{m}_{0}=M_{L}$ or $\tilde{m}_{0}=M_{R}$ respectively.

(ii) By the continuity of $g$ on $C\setminus\\{\tilde{m}_{0}\\}$ and the
uniform convergence established above, we deduce that if $z_{\ell}\to z\in
C\setminus\\{\tilde{m}_{0}\\}$, then

$\lvert g_{k_{\ell}}(z_{\ell})-g(z)\rvert\leq\lvert
g_{k_{\ell}}(z_{\ell})-g(z_{\ell})\rvert+\lvert g(z_{\ell})-g(z)\rvert\to 0.$
(S71)

(iii) If $\tilde{m}_{0}>M_{L}$, then
$L=\lim_{\ell\to\infty}g_{k_{\ell}}(\tilde{m}_{0})\geq\lim_{x\nearrow\,\tilde{m}_{0}}\,\lim_{\ell\to\infty}g_{k_{\ell}}(x)=\lim_{x\nearrow\,\tilde{m}_{0}}g(x)$,
so $L\in(-\infty,\infty]$. Similarly if $\tilde{m}_{0}<M_{R}$, then
$L\leq\lim_{x\searrow\,\tilde{m}_{0}}g(x)$, whence $L\in[-\infty,\infty)$.

(iv) If $m_{0}\in\operatorname{Int}C$, then $g\in\mathcal{F}^{m_{0}}$ by
construction. Suppose now that $P^{X}(\\{\tilde{m}_{0}\\})=0$. Since
$Q_{\ell}\overset{d}{\to}P$ under the hypotheses of Proposition S12,
Skorokhod’s representation theorem (e.g. van der Vaart, 1998, Theorem 2.19)
guarantees the existence of random vectors
$(X,Y),(X_{1},Y_{1}),(X_{2},Y_{2}),\dotsc$ defined on a common probability
space such that $(X,Y)\sim P$, $(X_{\ell},Y_{\ell})\sim Q_{\ell}$ for all
$\ell$ and $(X_{\ell},Y_{\ell})\to(X,Y)$ almost surely. Then it follows from
(S71) that $g_{k_{\ell}}(X_{\ell})\to g(X)$ almost surely on the event
$\\{X=\tilde{m}_{0}\\}$, which has probability 1 since
$P^{X}(\\{\tilde{m}_{0}\\})=0$ by assumption. An application of Fatou’s lemma
shows that

$\displaystyle\infty>\limsup_{n\to\infty}\,L(f_{n},P_{n})\geq\liminf_{\ell\to\infty}\,L(g_{k_{\ell}},Q_{\ell})$
$\displaystyle=\liminf_{\ell\to\infty}\,\mathbb{E}\bigl{(}\\{Y_{\ell}-g_{k_{\ell}}(X_{\ell})\\}^{2}\bigr{)}$
$\displaystyle\geq\mathbb{E}\Bigl{(}\liminf_{\ell\to\infty}\,\\{Y_{\ell}-g_{k_{\ell}}(X_{\ell})\\}^{2}\Bigr{)}$
$\displaystyle=\mathbb{E}\bigl{(}\\{Y-g(X)\\}^{2}\bigr{)}=L(g,P).$ (S72)

Thus,
$\mathbb{E}\bigl{(}g(X)^{2}\bigr{)}^{1/2}\leq\mathbb{E}\bigl{(}\\{Y-g(X)\\}^{2}\bigr{)}^{1/2}+\mathbb{E}(Y^{2})^{1/2}<\infty$,
so $g\in L^{2}(P^{X})$. By Proposition S9(d) and the proof of (i) above, it
follows that
$[g]_{P^{X}}\in\operatorname{Cl}\mathcal{F}_{P^{X}}^{m_{0}}\subseteq\mathcal{L}^{2}(P^{X})$.

(v) Since $L(h_{n},P)\to L(h,P)$ whenever $\|h_{n}-h\|_{L^{2}(P^{X})}\to 0$,
it follows from (S72) and the conclusion of (iv) that
$\liminf_{\ell\to\infty}L(g_{k_{\ell}},Q_{\ell})\geq L(g,P)\geq
L_{m_{0}}^{*}(P)$, as required.

(vi) If $P_{n}=P$ for all $n$, then we can modify the argument leading up to
(S72) as follows: since $g_{k_{\ell}}\to g$ pointwise on $C$ and $P^{X}(C)=1$,
it follows that if $(X,Y)\sim P$, then $g_{k_{\ell}}(X)\to g(X)$ (almost
surely) and $\infty>\liminf_{\ell\to\infty}L(g_{k_{\ell}},P)\geq L(g,P)$, as
in (S72). Thus, $g\in L^{2}(P^{X})$ and
$[g]_{P^{X}}\in\operatorname{Cl}\mathcal{F}_{P^{X}}^{m_{0}}\subseteq\mathcal{L}^{2}(P^{X})$
as in the proof of (iv). In particular, when
$\tilde{m}_{0}\notin\operatorname{Int}C$ and $P^{X}(\\{\tilde{m}_{0}\\})>0$,
we must have $g(\tilde{m}_{0})\in\mathbb{R}$ since $g\in L^{2}(P^{X})$. In
this case, if $m_{0}\in C$, then $g\sim_{P^{X}}h$ for some
$h\in\mathcal{F}^{m_{0}}$, and otherwise if $m_{0}\notin C$, then
$[g]_{P^{X}}\in\operatorname{Cl}\mathcal{F}_{P^{X}}^{m_{0}}$ by Proposition
S9(d). We conclude as before that
$\liminf_{\ell\to\infty}L(g_{k_{\ell}},P)\geq L(g,P)\geq L_{m_{0}}^{*}(P)$ in
all cases, so the proof of Lemma S28 is complete. ∎

###### Proof of Proposition S12.

(a) Fix $\varepsilon>0$. By Proposition S9(a) and the observation in the
paragraph after (S34), there exists a continuous $h_{0}\in\mathcal{F}^{m_{0}}$
such that $L_{m_{0}}^{*}(P)\leq L(h_{0},P)\leq L_{m_{0}}^{*}(P)+\varepsilon$.
Now for $\eta\in[-m_{0},1-m_{0}]$, define $h_{\eta}\colon[0,1]\to\mathbb{R}$
by $h_{\eta}(x):=h_{0}(0\vee(x-\eta)\wedge 1)$. Then
$h_{\eta}\in\mathcal{F}^{m_{0}+\eta}$ and $\sup_{x\in[0,1]}\,\lvert
h_{\eta}(x)\rvert\leq\lvert h_{0}(0)\rvert\vee\lvert h_{0}(1)\rvert=:B$ for
all $\eta\in[-m_{0},1-m_{0}]$. In addition, it follows from Lemma S30 that
$h_{\eta}\to h_{0}$ uniformly on $[0,1]$ as $\eta\to 0$. For each $n$, let
$\eta_{n}:=m_{n}-m_{0}$, so that $h_{\eta_{n}}\in\mathcal{F}^{m_{n}}$ and

$L_{m_{n}}^{*}(P_{n})\leq
L(h_{\eta_{n}},P_{n})=L(h_{0},P_{n})+\bigl{(}L(h_{\eta_{n}},P_{n})-L(h_{0},P_{n})\bigr{)}$

by the definition of $L_{m_{n}}^{*}(P_{n})$. Observe that

$\displaystyle\lvert L(h_{\eta_{n}},P_{n})-L(h_{0},P_{n})\rvert$
$\displaystyle\leq\int_{[0,1]\times\mathbb{R}}\;\bigl{|}\bigl{(}y-h_{\eta_{n}}(x)\bigr{)}^{2}-\bigl{(}y-h_{0}(x)\bigr{)}^{2}\bigr{|}\,dP_{n}(x,y)$
$\displaystyle\leq\int_{[0,1]\times\mathbb{R}}\;\lvert
h_{0}(x)-h_{\eta_{n}}(x)\rvert\,\bigl{(}2\lvert y\rvert+\lvert
h_{0}(x)\rvert+\lvert h_{\eta_{n}}(x)\rvert\bigl{)}\,dP_{n}(x,y)$
$\displaystyle\leq\sup_{x\in[0,1]}\,\lvert
h_{0}(x)-h_{\eta_{n}}(x)\rvert\,\int_{[0,1]\times\mathbb{R}}\,2(\lvert
y\rvert+B)\,dP_{n}(x,y).$

Moreover, $W_{2}(P_{n},P)\to 0$ by assumption, so
$\int_{[0,1]\times\mathbb{R}}\|w\|^{2}\,dP_{n}(w)\to\int_{[0,1]\times\mathbb{R}}\|w\|^{2}\,dP(w)$.
Thus, since $\bigl{(}y-h_{0}(x)\bigr{)}^{2}\leq 2(y^{2}+B^{2})$ and $\lvert
y\rvert\leq(1+y^{2})/2$ for all $(x,y)\in[0,1]\times\mathbb{R}$, we deduce
using Lemma S31 and the continuity of $h_{0}$ that

$\displaystyle
L(h_{0},P_{n})=\int_{[0,1]\times\mathbb{R}}\,\bigl{(}y-h_{0}(x)\bigr{)}^{2}\,dP_{n}(x,y)$
$\displaystyle\to\int_{[0,1]\times\mathbb{R}}\,\bigl{(}y-h_{0}(x)\bigr{)}^{2}\,dP(x,y)=L(h_{0},P)$
$\displaystyle\text{and\quad}\int_{[0,1]\times\mathbb{R}}\,(\lvert
y\rvert+B)\,dP_{n}(x,y)$
$\displaystyle\to\int_{[0,1]\times\mathbb{R}}\,(\lvert
y\rvert+B)\,dP(x,y)<\infty$

as $n\to\infty$. Since $h_{\eta_{n}}\to h_{0}$ uniformly on $[0,1]$ as
$n\to\infty$, it follows from the above that

$\limsup_{n\to\infty}L_{m_{n}}^{*}(P_{n})\leq\lim_{n\to\infty}L(h_{0},P_{n})=L(h_{0},P)\leq
L_{m_{0}}^{*}(P)+\varepsilon.$

Since $\varepsilon>0$ was arbitrary, this yields (a).

The proofs of (b)–(i) are based on the key Lemmas S26, S27 and S28 above.

(b) Fix a deterministic, non-negative sequence $(\delta_{n})$ with
$\delta_{n}\to 0$ and let $(f_{n})$ be any sequence with
$f_{n}\in\psi_{m_{n}}^{\delta_{n}}(P_{n})$ for all $n$, so that
$L(f_{n},P_{n})\leq L_{m_{n}}^{*}(P_{n})+\delta_{n}$ for all $n$. Then
$\limsup_{n\to\infty}L(f_{n},P_{n})=\limsup_{n\to\infty}L_{m_{n}}^{*}(P_{n})\leq
L_{m_{0}}^{*}(P)<\infty$ by (a), so $(f_{n})$ satisfies the hypotheses of
Lemma S28. In particular, Lemma S28(v) applies to every subsequence of
$(f_{n})$ when $P^{X}(\\{\tilde{m}_{0}\\})=0$, in which case

$\liminf_{n\to\infty}\,L_{m_{n}}^{*}(P_{n})=\liminf_{n\to\infty}\,L(f_{n},P_{n})\geq
L_{m_{0}}^{*}(P).$ (S73)

(c) If in addition $P_{n}=P$ for all $n$, then Lemma S28(vi) applies to every
subsequence of $(f_{n})$, regardless of whether or not
$P^{X}(\\{\tilde{m}_{0}\\})=0$, so (S73) holds in all cases. Together with
part (a) above, this establishes (c) and the fact that $L^{*}(P)$ and
$\mathcal{I}^{*}(P)$ are well-defined.

(d) If $P^{X}(\\{m\\})=0$ for all $m\in[0,1]$, then (a) and (b) imply that for
any sequence $(m_{n}^{\prime})$ in $[0,1]$ converging to some
$m^{\prime}\in[0,1]$, we have
$\lim_{n\to\infty}L_{m_{n}^{\prime}}^{*}(P_{n})=L_{m^{\prime}}^{*}(P)$. In
other words, the functions $m\mapsto L_{m}^{*}(P_{n})$ converge continuously
to $m\mapsto L_{m}^{*}(P)$ on $[0,1]$ in the sense of Remmert (1991, Chapter
3.1.5). Since continuous convergence is equivalent to uniform convergence on
the compact space $[0,1]$ (e.g. Remmert, 1991, pages 98–99), the first part of
(d) follows. This in turn implies the second assertion that
$L^{*}(P_{n})=\min_{m\in[0,1]}L_{m}^{*}(P_{n})\to\min_{m\in[0,1]}L_{m}^{*}(P)=L^{*}(P)$
as $n\to\infty$.

(e) Let $(m_{n}^{\prime})$ be any sequence in $[0,1]$ with
$m_{n}^{\prime}\in\mathcal{I}^{\delta_{n}}(P_{n})$ for all $n$. For each
subsequence of $(m_{n}^{\prime})$, we may extract a further subsequence
$(m_{n_{k}}^{\prime})$ that converges to some $m^{\prime}\in[0,1]$. For each
$k$, we have $L^{*}(P_{n_{k}})\leq L_{m_{n_{k}}^{\prime}}^{*}(P_{n_{k}})\leq
L^{*}(P_{n_{k}})+\delta_{n_{k}}$ by the definition of
$\mathcal{I}^{\delta_{n_{k}}}(P_{n_{k}})$. Thus, if $P^{X}(\\{m\\})=0$ for all
$m\in[0,1]$, then for any $m^{*}\in\mathcal{I}^{*}(P)$, we deduce from (a) and
(b) that

$L^{*}(P)\leq
L_{m^{\prime}}^{*}(P)=\lim_{k\to\infty}L_{m_{n_{k}}^{\prime}}^{*}(P_{n_{k}})=\lim_{k\to\infty}L^{*}(P_{n_{k}})\leq\lim_{k\to\infty}L_{m^{*}}^{*}(P_{n_{k}})=L_{m^{*}}^{*}(P)=L^{*}(P),$

so $m^{\prime}\in\mathcal{I}^{*}(P)$ and $L^{*}(P_{n_{k}})\to L^{*}(P)$ as
$k\to\infty$. This shows that every subsequence of $(m_{n}^{\prime})$ has a
further subsequence that converges to an element of $\mathcal{I}^{*}(P)$.
Since $(m_{n}^{\prime})$ was arbitrary, this implies (e). Similarly, every
subsequence of $\bigl{(}L^{*}(P_{n})\bigr{)}$ has a further subsequence that
converges to $L^{*}(P)$; this is another way to obtain the second part of (d).

For (f)–(i), fix any sequence $(f_{n})$ with
$f_{n}\in\psi_{m_{n}}^{\delta_{n}}(P_{n})$ for all $n$. If
$P^{X}(\\{\tilde{m}_{0}\\})=0$, then for any subsequence
$(g_{k})_{k=1}^{\infty}\equiv(f_{n_{k}})_{k=1}^{\infty}$ of $(f_{n})$, we can
find a further subsequence $(g_{k_{\ell}})$ and a function $g$ on $[0,1]$
satisfying conditions (i)–(v) in Lemma S28. In particular, setting
$m_{\ell}^{\prime}:=m_{n_{k_{\ell}}}$ and $Q_{\ell}:=P_{n_{k_{\ell}}}$, we
deduce from (a) and Lemma S28(v) that

$L(g,P)\leq\liminf_{\ell\to\infty}\,L(g_{k_{\ell}},Q_{\ell})\leq\limsup_{\ell\to\infty}\,L(g_{k_{\ell}},Q_{\ell})=\limsup_{\ell\to\infty}\,L_{m_{\ell}^{\prime}}^{*}(Q_{\ell})\leq
L_{m_{0}}^{*}(P),$

where the equality above follows from the fact that
$L(g_{k_{\ell}},Q_{\ell})\leq
L_{m_{\ell}^{\prime}}^{*}(Q_{\ell})+\delta_{m_{k_{\ell}}}$ for all $\ell$. We
conclude from Lemma S28(iv) and Corollary S10(c) that
$g\in\psi_{\tilde{m}_{0}}^{*}(P)=\psi_{m_{0}}^{*}(P)$.

(f) For each closed set
$A\subseteq(\operatorname{supp}P^{X})\setminus\\{\tilde{m}_{0}\\}$, Lemma
S28(ii) asserts that $g_{k_{\ell}}\to g\in\psi_{m_{0}}^{*}(P)$ uniformly on
$A$. Thus, every subsequence of $(f_{n})$ has a further subsequence that
converges uniformly on $A$ to an element of $\psi_{m_{0}}^{*}(P)$, and by
Corollary S10(a), all elements of $\psi_{m_{0}}^{*}(P)$ agree $P^{X}$-almost
everywhere on $A$. Since $(f_{n})$ was arbitrary, (f) follows.

(g) If $\psi_{\tilde{m}_{0}}^{0}(P)\neq\emptyset$, then by Corollary S10(a),
there exists $f^{*}\in\psi_{\tilde{m}_{0}}^{0}(P)$ such that
$\psi_{\tilde{m}_{0}}^{*}(P)=[f^{*}]_{P^{X}}$, so $g\sim_{P^{X}}f^{*}$ in the
argument before (f). Thus, in view of Lemma S28(i), we may assume that
$g\in\mathcal{F}^{\tilde{m}_{0}}$, so that $g\in\psi_{\tilde{m}_{0}}^{0}(P)$.
By applying Lemma S28(ii) as above, we deduce that for any closed set
$A\subseteq(\operatorname{supp}P^{X})\setminus\\{\tilde{m}_{0}\\}$, every
subsequence of $(f_{n})$ has a further subsequence that converges uniformly on
$A$ to an element of $\psi_{\tilde{m}_{0}}^{0}(P)$. All functions in
$\psi_{\tilde{m}_{0}}^{0}(P)$ agree on $A$ by Corollary S10(e), and $(f_{n})$
was arbitrary, so (g) holds.

Suppose in addition that
$m_{0}\in\operatorname{Int}(\operatorname{csupp}P^{X})$ and
$P^{X}(\\{m_{0}\\})=0$. Then in the argument before (f), we can insist that
$g\in\mathcal{F}^{m_{0}}$ in view of Lemma S28(iv), so that
$g\in\psi_{m_{0}}^{0}(P)$.

(h) For fixed $q\in[1,\infty)$, Lemma S27 implies that there exists
$\tilde{B}<\infty$ such that $\lvert g_{k_{\ell}}-g\rvert^{q}\leq
2^{q-1}(\tilde{B}^{q}+\lvert g\rvert^{q})$ on $[0,1]$ for all sufficiently
large $\ell$, so $\|g_{k_{\ell}}-g\|_{L^{q}(P^{X})}^{q}=\int_{[0,1]}\,\lvert
g_{k_{\ell}}-g\rvert^{q}\,dP^{X}\to 0$ by the dominated convergence theorem.
In summary, every subsequence of
$\bigl{(}[f_{n}]_{P^{X}}\bigr{)}_{n=1}^{\infty}$ has a further subsequence
that converges in $\mathcal{L}^{q}(P^{X})$ to $\psi_{m_{0}}^{*}(P)$, so the
entire sequence converges in $\mathcal{L}^{q}(P^{X})$ to
$\psi_{m_{0}}^{*}(P)$, as required.

(i) Under the hypotheses of (i), all elements of $\psi_{m_{0}}^{0}(P)$ are
continuous by Corollary S10(f), so we can apply Lemmas S28(ii) and S30 to
obtain the stronger conclusion that every subsequence of $(f_{n})$ has a
further subsequence that converges uniformly on $\operatorname{csupp}P^{X}$ to
some function in $\psi_{m_{0}}^{0}(P)$. Since elements of
$\psi_{m_{0}}^{0}(P)$ agree on $\operatorname{supp}P^{X}$ by assumption and
$(f_{n})$ was arbitrary, (i) holds. ∎

###### Proof of Corollary S14.

(a) By definition, we have
$\psi^{0}(P)=\\{f\in\mathcal{F}:L(f,P)=L^{*}(P)\\}=\bigcup_{m\in\mathcal{I}^{*}(P)}\psi_{m}^{0}(P)$.
If $\psi_{m}^{0}(P)\neq\emptyset$ for some
$m\in\mathcal{I}^{*}(P)\setminus\operatorname{csupp}P^{X}$, then
$\tilde{m}=\operatorname*{argmin}_{x\in\operatorname{csupp}P^{X}}\lvert
x-m\rvert$ satisfies
$\tilde{m}\in\mathcal{I}^{*}(P)\cap\operatorname{csupp}P^{X}$ and
$\psi_{\tilde{m}}^{0}(P)\neq\emptyset$ by Corollary S10(c). The result now
follows from Corollary S10(d).

For (b)–(d), fix a sequence $(f_{n})$ with $f_{n}\in\psi^{\delta_{n}}(P_{n})$
for all $n$, so that there exists a sequence $(m_{n})$ in $[0,1]$ with
$m_{n}\in\mathcal{I}^{\delta_{n}}(P_{n})$ and
$f_{n}\in\psi_{m_{n}}^{\delta_{n}}(P_{n})$ for all $n$. By assumption, we have
$P^{X}(\\{m\\})=0$ for all $m\in[0,1]$, so for each subsequence of $(f_{n})$,
Proposition S12(e) ensures the existence of a further subsequence
$(f_{n_{k}})$ such that $m_{n_{k}}\to m^{*}$ for some
$m^{*}\in\mathcal{I}^{*}(P)$. Let
$\tilde{m}^{*}:=\operatorname*{argmin}_{x\in\operatorname{csupp}P^{X}}\,\lvert
m^{*}-x\rvert$. Then $L_{\tilde{m}^{*}}^{*}(P)=L_{m^{*}}^{*}(P)=L^{*}(P)$ by
Corollary S10(c), so $\tilde{m}^{*}\in\mathcal{I}^{*}(P)$. We are now in a
position to apply Proposition S12(f)–(i).

(b) Fix a closed set
$A\subseteq(\operatorname{supp}P^{X})\setminus\mathcal{I}^{*}(P)\subseteq(\operatorname{supp}P^{X})\setminus\\{\tilde{m}^{*}\\}$.
For any $f^{*}\in\psi_{m^{*}}^{*}(P)\subseteq\psi^{*}(P)$, Proposition S12(f)
implies that $\|f_{n_{k}}-f^{*}\|_{L^{\infty}(A,P^{X})}\to 0$. Thus, every
subsequence of $(f_{n})$ has a further subsequence that converges in
$\|{\cdot}\|_{L^{\infty}(A,P^{X})}$ to an element of $\psi^{*}(P)$. Since
$(f_{n})$ was arbitrary, this yields (b).

(c) Fix a closed set
$A\subseteq(\operatorname{supp}P^{X})\setminus\tilde{\mathcal{I}}^{*}(P)$.

  * •

If $\tilde{m}^{*}\notin\tilde{\mathcal{I}}^{*}(P)$, then
$\tilde{m}^{*}=m^{*}\in\operatorname{Int}(\operatorname{csupp}P^{X})$ and all
elements of $\psi_{\tilde{m}^{*}}^{0}(P)=\psi_{m^{*}}^{0}(P)\neq\emptyset$ are
continuous, so for any $f^{*}\in\psi_{m^{*}}^{0}(P)\subseteq\psi^{0}(P)$,
Proposition S12(i) implies that $f_{n_{k}}\to f^{*}$ uniformly on
$\operatorname{supp}P^{X}\supseteq A$.

  * •

If $\tilde{m}^{*}\in\tilde{\mathcal{I}}^{*}(P)$, then we still have
$\tilde{m}^{*}\in\mathcal{I}^{*}(P)\cap\operatorname{csupp}P^{X}$, so
$\psi_{\tilde{m}^{*}}^{0}(P)\neq\emptyset$ by assumption. Thus, for any
$f^{*}\in\psi_{\tilde{m}^{*}}^{0}(P)\subseteq\psi^{0}(P)$, Proposition S12(g)
implies that $f_{n_{k}}\to f^{*}$ uniformly on
$(\operatorname{supp}P^{X})\setminus\\{\tilde{m}^{*}\\}\supseteq A$.

Thus, every subsequence of $(f_{n})$ has a further subsequence that converges
uniformly on $A$ to an element of $\psi^{0}(P)$. Since $(f_{n})$ was
arbitrary, (c) follows.

(d) Here,
$m^{*}\in\mathcal{I}^{*}(P)\subseteq\operatorname{Int}(\operatorname{csupp}P^{X})$
by assumption, so for any $f^{*}\in\psi_{m^{*}}^{0}(P)\subseteq\psi^{*}(P)$,
Proposition S12(h) implies that $\|f_{n_{k}}-f^{*}\|_{L^{q}(P^{X})}\to 0$ for
any $q\in[1,\infty)$. By the same reasoning as in (b, c), we obtain (d). ∎

###### Proof of Proposition S16.

In the definition of $P_{0}$, we have
$\mathbb{E}(f_{0}(X)+\xi\,|\,X)=f_{0}(X)$ since $\xi$ is independent of $X$
and has mean 0 by (ii), so $f_{0}$ is a regression function for $P_{0}$ (in
the sense of (S34) in Section 2). It now follows from condition (iii) and
Corollary S10(d) that $\psi_{m}^{0}(P_{0})\neq\emptyset$ for all
$m\in\operatorname{csupp}P_{0}^{X}$, so in particular
$\psi^{0}(P_{0})\neq\emptyset$.

Writing $(D_{n})$ for any one of the sequences of random variables in (a)–(d),
we aim to prove that $D_{n}\overset{p}{\to}0$, or equivalently that every
subsequence $(D_{n_{k}})$ has a further subsequence that converges almost
surely to 0. If it can be shown that
$W_{2}(\mathbb{P}_{n},P_{0})\overset{p}{\to}0$, then we can take
$(D_{n_{k_{\ell}}})$ to be a subsequence of $(D_{n_{k}})$ such that
$W_{2}(\mathbb{P}_{n_{k_{\ell}}},P_{0})\to 0$ almost surely. Working on an
event $\Omega_{0}$ of probability 1 on which this convergence takes place, we
deduce directly from the relevant parts of Proposition S12 or Corollary S14
that $D_{n_{k_{\ell}}}\\!\to 0$ on $\Omega_{0}$; note that assertions (a, b)
follow from Proposition S12(d, e) and assertions (c, d) follow from Corollary
S14(c, d).

To complete the proof, we must therefore verify that
$W_{2}(\mathbb{P}_{n},P_{0})\overset{p}{\to}0$ under conditions (i)–(iii). It
suffices to show that

$\text{($\ast$)}\;\;\mathbb{P}_{n}\overset{d}{\to}P_{0}\text{ almost
surely}\qquad\text{and}\qquad\text{($\ast\ast$)}\;\;\int_{\mathbb{R}^{2}}\|w\|^{2}\,d\mathbb{P}_{n}(w)\overset{p}{\to}\int_{\mathbb{R}^{2}}\|w\|^{2}\,dP_{0}(w),$

since then we can argue along subsequences of $(\mathbb{P}_{n})$ as in the
previous paragraph.

($\ast$) Let
$\tilde{\mathbb{P}}_{n}:=n^{-1}\sum_{i=1}^{n}\delta_{(x_{ni},\,\xi_{ni})}$ for
each $n$ and define $\tilde{P}_{0}:=P_{0}^{X}\otimes P_{\xi}$. Defining the
map $F_{0}\colon(x,z)\mapsto(x,f_{0}(x)+z)$ on $\mathbb{R}^{2}$, we therefore
have $\mathbb{P}_{n}=\tilde{\mathbb{P}}_{n}\circ F_{0}^{-1}$ for each $n$ and
$P_{0}=\tilde{P}_{0}\circ F_{0}^{-1}$. The desired convergence statement
($\ast$) follows from the following two claims.

######  Claim 3.

$\tilde{\mathbb{P}}_{n}\overset{d}{\to}\tilde{P}_{0}$ almost surely.

###### Proof of Claim.

By Billingsley (1999, Theorem 2.3), the countable set
$\mathcal{R}:=\\{[a_{1},b_{1}]\times[a_{2},b_{2}]:a_{j}\leq b_{j}\text{ and
}a_{j},b_{j}\in\mathbb{Q}\text{ for }j=1,2\\}$ is a convergence-determining
class in the sense of Billingsley (1999, page 18), so it suffices to show that
$\tilde{\mathbb{P}}_{n}(R)\to\tilde{P}_{0}(R)$ almost surely for each
$R\in\mathcal{R}$. To this end, fix any $I_{1}\times I_{2}\in\mathcal{R}$,
where $I_{j}=[a_{j},b_{j}]$ is an interval with rational endpoints $a_{j}\leq
b_{j}$ for $j=1,2$. By condition (i),
$\mathbb{P}_{n}^{X}\overset{d}{\to}P_{0}^{X}$ and
$P_{0}^{X}(\\{a_{1},b_{1}\\})=0$, so $\mathbb{P}_{n}^{X}(I_{1})\to
P_{0}^{X}(I_{1})$ as $n\to\infty$. For each $n$, defining
$r_{n}:=\sum_{i=1}^{n}\mathbbm{1}_{\\{x_{ni}\in
I_{1}\\}}=n\mathbb{P}_{n}^{X}(I_{1})$, we can write
$\tilde{\mathbb{P}}_{n}(I_{1}\times I_{2})=\mathbb{P}_{n}^{X}(I_{1})\,T_{n}$,
where

$T_{n}:=\frac{\sum_{i=1}^{n}\mathbbm{1}_{\\{x_{ni}\in
I_{1}\\}}\mathbbm{1}_{\\{\xi_{ni}\in I_{2}\\}}}{r_{n}\vee
1}\sim\frac{1}{r_{n}\vee
1}\operatorname{Bin}\bigl{(}r_{n},P_{\xi}(I_{2})\bigr{)}.$

If $P_{0}^{X}(I_{1})=0$, then certainly $\tilde{\mathbb{P}}_{n}(I_{1}\times
I_{2})=\mathbb{P}_{n}^{X}(I_{1})\,T_{n}\to
0=P_{0}^{X}(I_{1})P_{\xi}(I_{2})=\tilde{P}_{0}(I_{1}\times I_{2})$. On the
other hand, if $P_{0}^{X}(I_{1})>0$, then
$r_{n}=n\bigl{(}1+o(1)\bigr{)}P_{0}^{X}(I_{1})\to\infty$, so for all $t>0$, we
have $\sum_{n=1}^{\infty}\mathbb{P}(\lvert
T_{n}-P_{\xi}(I_{2})\rvert>t)<\infty$ by Hoeffding’s inequality (or some other
suitable exponential tail bound for binomial random variables; see van der
Vaart and Wellner (1996, Appendix A.6.1) for example). Thus, by the first
Borel–Cantelli lemma, $T_{n}\to P_{\xi}(I_{2})$ almost surely, so
$\tilde{\mathbb{P}}_{n}(I_{1}\times I_{2})=\mathbb{P}_{n}^{X}(I_{1})\,T_{n}\to
P_{0}^{X}(I_{1})P_{\xi}(I_{2})=\tilde{P}_{0}(I_{1}\times I_{2})$ almost surely
as $n\to\infty$, as required. ∎

######  Claim 4.

If $(Q_{n})$ is any sequence of probability measures such that
$Q_{n}\overset{d}{\to}\tilde{P}_{0}$, then $Q_{n}\circ
F_{0}^{-1}\overset{d}{\to}\tilde{P}_{0}\circ F_{0}^{-1}=P_{0}$ under condition
(iii).

###### Proof of Claim.

It follows from Skorokhod’s representation theorem (e.g. van der Vaart, 1998,
Theorem 2.19) that there exist random vectors
$(\tilde{X},\tilde{Z}),(\tilde{X}_{1},\tilde{Z}_{1}),(\tilde{X}_{2},\tilde{Z}_{2}),\dotsc$
defined on a common probability space such that
$(\tilde{X},\tilde{Z})\sim\tilde{P}_{0}$, $(\tilde{X}_{n},\tilde{Z}_{n})\sim
Q_{n}$ for all $n$ and $(\tilde{X}_{n},\tilde{Z}_{n})\to(\tilde{X},\tilde{Z})$
almost surely.

Since $\tilde{X}\sim P_{0}^{X}$ and $f_{0}$ is continuous $P_{0}^{X}$-almost
everywhere under condition (iii), we have $f_{0}(\tilde{X}_{n})\to
f_{0}(\tilde{X})$ almost surely, so
$F_{0}(\tilde{X}_{n},\tilde{Z}_{n})=(\tilde{X}_{n},f_{0}(\tilde{X}_{n})+\tilde{Z}_{n})\to(\tilde{X},f_{0}(\tilde{X})+\tilde{Z})=F_{0}(\tilde{X},\tilde{Z})$
almost surely. Thus, the distribution $Q_{n}\circ F_{0}^{-1}$ of
$F_{0}(\tilde{X}_{n},\tilde{Z}_{n})$ converges weakly to the distribution
$\tilde{P}_{0}\circ F_{0}^{-1}=P_{0}$ of $F_{0}(\tilde{X},\tilde{Z})$, as
required. ∎

($\ast\ast$) Let $X\sim P_{0}^{X}$ and $\xi\sim P_{\xi}$ be independent, so
that $\bigl{(}X,f_{0}(X)+\xi\bigr{)}\sim P_{0}$. We have

$\int_{\mathbb{R}^{2}}\|w\|^{2}\,d\mathbb{P}_{n}(w)=\frac{1}{n}\sum_{i=1}^{n}(x_{ni}^{2}+Y_{ni}^{2})=\frac{1}{n}\sum_{i=1}^{n}\bigl{(}x_{ni}^{2}+f_{0}^{2}(x_{ni})+2f_{0}(x_{ni})\,\xi_{ni}+\xi_{ni}^{2}\bigr{)}$
(S74)

and now consider each of the summands on the right-hand side. Since
$(\mathbb{P}_{n}^{X})$ is a sequence of probability measures on the compact
set $[0,1]$ satisfying $\mathbb{P}_{n}^{X}\overset{d}{\to}P_{0}^{X}$, we have
$W_{2}(\mathbb{P}_{n}^{X},P_{0}^{X})\to 0$, so

$\frac{1}{n}\sum_{i=1}^{n}x_{ni}^{2}=\int_{[0,1]}x^{2}\,d\mathbb{P}_{n}^{X}(x)\to\int_{[0,1]}x^{2}\,dP_{0}^{X}(x)=\mathbb{E}(X^{2}).$
(S75)

In addition, we can apply condition (iii) and argue as in the proof of the
first Claim to see that $\mathbb{P}_{n}^{X}\circ
f_{0}^{-1}\overset{d}{\to}P_{0}^{X}\circ f_{0}^{-1}$. Since $f_{0}$ is bounded
on $[0,1]$ by (iii), these probability measures are also supported on some
compact set, so in fact $W_{2}\bigl{(}\mathbb{P}_{n}^{X}\circ
f_{0}^{-1},P_{0}^{X}\circ f_{0}^{-1}\bigr{)}\to 0$, whence

$\frac{1}{n}\sum_{i=1}^{n}f_{0}^{2}(x_{ni})=\int_{\mathbb{R}}x^{2}\,d\bigl{(}\mathbb{P}_{n}^{X}\circ
f_{0}^{-1}\bigr{)}(x)=\int_{\mathbb{R}}x^{2}\,d\bigl{(}\mathbb{P}_{0}^{X}\circ
f_{0}^{-1}\bigr{)}(x)=\mathbb{E}\bigl{(}f_{0}^{2}(X)\bigr{)}.$ (S76)

For all $t>0$, it follows from (S76), condition (ii) and Chebsyhev’s
inequality that

$\mathbb{P}\biggl{(}\frac{1}{n}\,\biggl{|}\sum_{i=1}^{n}f_{0}(x_{ni})\,\xi_{ni}\biggr{|}>t\biggr{)}\leq\frac{1}{(nt)^{2}}\operatorname{Var}\biggl{(}\sum_{i=1}^{n}f_{0}(x_{ni})\,\xi_{ni}\biggr{)}=\frac{1}{(nt)^{2}}\sum_{i=1}^{n}f_{0}^{2}(x_{ni})\operatorname{Var}(\xi)\to
0,$ (S77)

so
$n^{-1}\sum_{i=1}^{n}f_{0}(x_{ni})\,\xi_{ni}\overset{p}{\to}0=\mathbb{E}\bigl{(}f_{0}(X)\xi\bigr{)}$
by the independence of $X$ and $\xi$. Finally, by condition (ii),
$\xi_{n1},\dotsc,\xi_{nn}\overset{\mathrm{iid}}{\sim}P_{\xi}$ for each $n$, so
$n^{-1}\sum_{i=1}^{n}\xi_{ni}^{2}\overset{p}{\to}\mathbb{E}(\xi^{2})$ by the
weak law of large numbers. Together with (S74), (S75), (S76) and (S77), this
implies that

$\textstyle\int_{\mathbb{R}^{2}}\|w\|^{2}\,d\mathbb{P}_{n}(w)\overset{p}{\to}\mathbb{E}(X^{2})+\mathbb{E}\bigl{(}f_{0}^{2}(X)\bigr{)}+2\mathbb{E}\bigl{(}f_{0}(X)\xi\bigr{)}+\mathbb{E}(\xi^{2})=\int_{\mathbb{R}^{2}}\|w\|^{2}\,dP_{0}(w),$

so ($\ast\ast$) holds. ∎

###### Proof of Proposition 2.

The results for $(\tilde{g}_{n})=(\tilde{f}_{n})$ follow from Proposition S16.
When $(\tilde{g}_{n})=(\hat{f}_{n}^{m_{0}})$, we again have
$W_{2}(\mathbb{P}_{n},P_{0})\overset{p}{\to}0$ under conditions (i)–(iii) in
Proposition S16, so assertions (a, b, c, d) follow from Proposition S12(e, g,
i, h) in this case. ∎

##  S7 Auxiliary results and examples for Section S3

The proofs in Section S6 make use of two straightforward results on the
convergence of sequences of S-shaped functions.

######  Lemma S29.

Suppose that $(f_{n})_{n=1}^{\infty}$ is a sequence of increasing convex
functions on $[0,1)$ such that $\lim_{n\to\infty}f_{n}(x)$ exists for all
$x\in(0,1)$. Then $\lim_{n\to\infty}f_{n}(0)$ exists and the function
$f\colon[0,1)\to\mathbb{R}$ defined by $f(x):=\lim_{n\to\infty}f_{n}(x)$ is
convex and increasing. Moreover, $f_{n}\to f$ uniformly on $[0,w]$ for every
$w\in[0,1)$.

###### Proof.

Since $\left.f_{n}\right|_{(0,1)}$ is convex and increasing for all $n$, the
same is true of the pointwise limit $\left.f\right|_{(0,1)}$. Thus,
$l:=\lim_{t\,\searrow\,0}f(t)$ exists and is finite, and we now show that
$f_{n}(0)\to l$ as $n\to\infty$. Since each $f_{n}$ is convex on $[0,1)$, we
have $f_{n}(0)\geq 2f_{n}(t)-f_{n}(2t)$ for all $t>0$ and $n\in\mathbb{N}$, so

$\liminf_{n\to\infty}f_{n}(0)\geq\lim_{t\,\searrow\,0}\,\lim_{n\to\infty}\,\bigl{(}2f_{n}(t)-f_{n}(2t)\bigr{)}=\lim_{t\,\searrow\,0}\,\bigl{(}2f(t)-f(2t)\bigr{)}=l.$

On the other hand, $f_{n}$ is increasing on $[0,1)$ for all $n$, so

$\limsup_{n\to\infty}f_{n}(0)\leq\lim_{t\,\searrow\,0}\,\lim_{n\to\infty}f_{n}(t)=\lim_{t\,\searrow\,0}f(t)=l,$

which means that $f(0)=l$, as required. Consequently, $f$ is convex and
increasing on $[0,1)$. For the final assertion of the lemma, we extend
$f,f_{1},f_{2},\dotsc$ to increasing convex functions on $(-\infty,1)$ by
setting $f(x)=f(0)$ and $f_{n}(x)=f_{n}(0)$ for all $x<0$ and
$n\in\mathbb{N}$. It follows from what we have already shown that $f_{n}\to f$
pointwise on $(-\infty,1)$, so in fact $f_{n}\to f$ uniformly on compact
subsets of $(-\infty,1)$ by Rockafellar (1997, Theorem 10.8). This yields the
desired conclusion. ∎

######  Lemma S30.

If $m\in(0,1)$ and $(f_{n})_{n=1}^{\infty}$ is a sequence of functions in
$\mathcal{F}^{m}$ that converges pointwise on $[0,1]\setminus\\{m\\}$ to some
continuous $f\in\mathcal{F}^{m}$, then $f_{n}\to f$ uniformly on $[0,1]$.

###### Proof.

First note that $f_{n}\to f$ pointwise on $[0,1]$. Indeed,

$f(z)=\lim_{n\to\infty}f_{n}(z)\leq\liminf_{n\to\infty}f_{n}(m)\leq\limsup_{n\to\infty}f_{n}(m)\leq\lim_{n\to\infty}f_{n}(w)=f(w)$

whenever $z<m<w$, and since
$\lim_{z\nearrow\,m}f(z)=f(m)=\lim_{w\searrow\,m}f(w)$ by continuity, it
follows that $f_{n}(m)\to f(m)$. We now show that $f_{n}\to f$ uniformly on
$[0,1]$ by a standard argument: fix $\varepsilon>0$ and note that since $f$ is
continuous and increasing on $[0,1]$, we can find
$0=z_{0}<z_{1}<\dotsc<z_{k-1}<z_{k}=1$ such that
$f(z_{i})-f(z_{i-1})<\varepsilon$ for all $i\in[k]$. Since each $f_{n}$ is
increasing, we see that if $x\in[z_{i-1},z_{i}]$, then

$f_{n}(z_{i-1})-f(z_{i-1})-\varepsilon<f_{n}(z_{i-1})-f(z_{i})\leq
f_{n}(x)-f(x)\leq f_{n}(z_{i})-f(z_{i-1})<f_{n}(z_{i})-f(z_{i})+\varepsilon.$

Since $f_{n}\to f$ pointwise, $\limsup_{n\to\infty}\sup_{x\in[0,1]}\,\lvert
f_{n}(x)-f(x)\rvert<\lim_{n\to\infty}\max_{1\leq i\leq k}\,\lvert
f_{n}(z_{i})-f(z_{i})\rvert+\varepsilon=\varepsilon$. This holds for all
$\varepsilon>0$, so the result follows. ∎

The weak convergence result below is stated as Lemma 4.5 in Dümbgen et al.
(2011) and proved here for completeness.

######  Lemma S31.

Let $P,P_{1},P_{2},\dotsc$ be probability measures on $\mathbb{R}^{d}$ such
that $P_{n}\overset{d}{\to}P$. If $h$ is a non-negative, continuous function
on $\mathbb{R}^{d}$ such that
$\int_{\mathbb{R}^{d}}h\,dP_{n}\to\int_{\mathbb{R}^{d}}h\,dP<\infty$, then
$\int_{\mathbb{R}^{d}}f\,dP_{n}\to\int_{\mathbb{R}^{d}}f\,dP$ for any
continuous function $f\colon\mathbb{R}^{d}\to\mathbb{R}^{k}$ such that
$\|f\|/(1+h)$ is bounded.

###### Proof.

We can restrict attention to the case $k=1$ since the component functions can
be considered separately when $k>1$. Let $C>0$ be such that $\lvert
f\rvert\leq C(1+h)$ pointwise. For a Borel measure $Q$ on $\mathbb{R}^{d}$ and
a $Q$-integrable function $h$, we write $Q(h)$ as shorthand for
$\int_{\mathbb{R}^{d}}h\,dQ$.

Since $P_{n}\overset{d}{\to}P$, we have $\liminf_{n\to\infty}P_{n}(g)\geq
P(g)$ for all non-negative, continuous $g\colon\mathbb{R}^{d}\to\mathbb{R}$ by
the portmanteau lemma (e.g. van der Vaart, 1998, Lemma 2.2(iv)). Thus, since
$f^{+}:=f\vee 0$ and $C(1+h)-f^{+}$ are non-negative and continuous, we have
$\liminf_{n\to\infty}P_{n}(f^{+})\geq P(f^{+})$ and
$\liminf_{n\to\infty}P_{n}\bigl{(}C(1+h)-f^{+}\bigr{)}\geq
P\bigl{(}C(1+h)-f^{+}\bigr{)}$. Moreover, $P_{n}\bigl{(}C(1+h)\bigr{)}\to
P\bigl{(}C(1+h)\bigr{)}$ by assumption, so in fact $P_{n}(f^{+})\to P(f^{+})$.
A similar argument shows that $P_{n}(f^{-})\to P(f^{-})$, where
$f^{-}:=(-f)\vee 0$, so we indeed have $P_{n}(f)\to P(f)$. ∎

We conclude this subsection with a series of related examples which illustrate
that some of the assertions of Proposition S12 and Corollary S14 do not hold
in general if the associated technical conditions are not satisfied.

######  Example S32.

We first consider situations where either $P^{X}(\\{\tilde{m}_{0}\\})>0$ or
$m_{0}\notin\operatorname{Int}(\operatorname{csupp}P^{X})$. In each of the
following, we construct $P\in\mathcal{P}$ and a sequence $(P_{n})$ in
$\mathcal{P}$ with $W_{2}(P_{n},P)\to 0$, where $P=(1-\eta)Q+\eta\tilde{Q}$
and $P_{n}=(1-\eta_{n})Q_{n}+\eta_{n}\tilde{Q}_{n}$ for suitable
$Q,Q_{n},\tilde{Q}_{n}\in\mathcal{P}$ and $\eta,\eta_{n}\in[0,1]$. For
$w\in\mathbb{R}^{d}$ with $d\in\mathbb{N}$, we write $\delta_{w}$ for a point
mass at $w$.

  1. (a)

Fix $m_{0}\in(0,1]$ and $m\in(0,m_{0}]$. Let
$Q_{n}:=\frac{1}{2}\delta_{(0,0)}+\frac{1}{2}\delta_{(m(1-1/n),\,0)}$,
$Q:=\frac{1}{2}\delta_{(0,0)}+\frac{1}{2}\delta_{(m,0)}$,
$\tilde{Q}_{n}=\tilde{Q}:=\delta_{(m,1)}$ and $\eta_{n}=\eta:=1/3$ for all
$n$. Then
$P_{n}=\frac{1}{3}\delta_{(0,0)}+\frac{1}{3}\delta_{(m(1-1/n),\,0)}+\frac{1}{3}\delta_{(m,1)}\overset{d}{\to}\frac{1}{3}\delta_{(0,0)}+\frac{1}{3}\delta_{(m,0)}+\frac{1}{3}\delta_{(m,1)}=P$
and $P^{X}=\frac{1}{3}\delta_{0}+\frac{2}{3}\delta_{m}$, so
$m_{0}\notin\operatorname{Int}(\operatorname{csupp}P^{X})$,
$\mathcal{I}^{*}(P)=[0,1]\not\subseteq\operatorname{Int}(\operatorname{csupp}P^{X})$,
$\tilde{m}_{0}=m$ and $P^{X}(\\{\tilde{m}_{0}\\})>0$. Since
$P,P_{1},P_{2},\dotsc$ are supported on the compact set $[0,1]^{2}$, we
automatically have $W_{2}(P_{n},P)\to 0$.

We claim that Propositions S12(b, d, h) and Corollary S14(d) do not apply
here. Indeed, for each $n$, we have $f_{n}(m)=1$ for all
$f_{n}\in\psi_{m_{0}}^{0}(P_{n})=\psi^{0}(P_{n})$ and
$L^{*}(P_{n})=L_{m_{0}}^{*}(P_{n})=0$, whereas $f^{*}(m)=1/2$ for all
$f^{*}\in\psi_{m_{0}}^{*}(P)=\psi^{*}(P)$ and
$L^{*}(P)=L_{m_{0}}^{*}(P)=1/6>0=\lim_{n\to\infty}L_{m_{0}}^{*}(P_{n})$. Thus,
if $f_{n}\in\psi_{m_{0}}^{0}(P_{n})$ for all $n$, then $\lvert
f_{n}(m)-f^{*}(m)\rvert=1/2$ for all $n$ and $f^{*}\in\psi^{*}(P)$, so
$\inf_{f^{*}\in\psi^{*}(P)}\|f_{n}-f^{*}\|_{L^{q}(P^{X})}\nrightarrow 0$ for
$q\in[1,\infty)$.

  2. (b)

This is a variant of (a) with
$m=m_{0}\in\operatorname{Int}(\operatorname{csupp}P^{X})=(0,1)$ but
$P^{X}(\\{m_{0}\\})>0$. Let $Q_{n},Q$ and $\eta_{n},\eta$ be as in (a) but
instead define
$\tilde{Q}_{n}=\tilde{Q}:=\frac{1}{2}\delta_{(m_{0},1)}+\frac{1}{2}\delta_{(1,1)}$.
Then
$P^{X}=\frac{1}{3}\delta_{0}+\frac{1}{3}\delta_{m_{0}}+\frac{1}{3}\delta_{1}$
and all the deductions in the second paragraph of (a) remain valid here.

  3. (c)

Fix $m_{0}\in(0,1]$ and $m\in(0,m_{0}]$. For $n\in\mathbb{N}$, let $Q,Q_{n}$
be the uniform distributions on $\\{(x,0):0\leq x\leq m\\}$ and
$\\{(x,0):0\leq x\leq m(1-1/n)\\}$ respectively. Moreover, let $\tilde{Q}:=Q$,
$\eta:=0$, $\tilde{Q}_{n}:=\delta_{(m,n^{2})}$ and $\eta_{n}:=n^{-5}$ for all
$n$. Then $P_{n}=(1-\eta_{n})Q_{n}+\eta_{n}\tilde{Q}_{n}$ converges in $W_{2}$
to $P=Q$. Indeed, for any continuous $f\colon\mathbb{R}^{2}\to\mathbb{R}$ such
that $\lvert f(x,y)\rvert\leq 1+x^{2}+y^{2}$ for all $(x,y)\in\mathbb{R}^{2}$,
we have $\eta_{n}f(m,n^{2})=O(1/n)$, so
$\int_{\mathbb{R}^{2}}f\,dP_{n}=(1-\eta_{n})\int_{0}^{m(1-1/n)}f(x,0)\,dx+\eta_{n}f(m,n^{2})\to\int_{0}^{m}f(x,0)\,dx=\int_{\mathbb{R}^{2}}f\,dP$
as $n\to\infty$.

Here, $\tilde{m}_{0}=m$ and $P^{X}(\\{m^{\prime}\\})=0$ for all
$m^{\prime}\in[0,1]$, but
$m_{0}\notin\operatorname{Int}(\operatorname{csupp}P^{X})$,
$\mathcal{I}^{*}(P)=[0,1]\not\subseteq\operatorname{Int}(\operatorname{csupp}P^{X})$,
and Proposition S12(h) and Corollary S14(d) do not hold. Indeed, for each $n$,
let $f_{n}\in\psi_{m_{0}}^{0}(P_{n})=\psi^{0}(P_{n})$ be such that $f_{n}=0$
on $[0,m(1-1/n)]$, $f_{n}(m)=n^{2}$ and $f_{n}$ is linear on $[m(1-1/n),1]$.
Since $f^{*}=0$ $P^{X}$-almost everywhere on $[0,m]$ for all
$f^{*}\in\psi_{m_{0}}^{*}(P)=\psi^{*}(P)$, we have
$\inf_{f^{*}\in\psi^{*}(P)}\|f_{n}-f^{*}\|_{L^{q}(P^{X})}=\inf_{f^{*}\in\psi^{*}(P)}\|f_{n}\|_{L^{q}(P^{X})}\geq\inf_{f^{*}\in\psi^{*}(P)}\|f_{n}\|_{L^{1}(P^{X})}=n\to\infty$
for all $q\in[1,\infty)$.

In Proposition S12(i), the assumption that $\psi_{m_{0}}^{0}(P)\neq\emptyset$
and all elements of $\psi_{m_{0}}^{0}(P)$ agree on $\operatorname{supp}P^{X}$
is clearly necessary for the conclusion to hold (not least when $P_{n}=P$ for
all $n$). When this condition is not satisfied, some elements of
$\psi_{m_{0}}^{0}(P)$ are discontinuous by Corollary S10(e). If in addition
$m_{0}\in\operatorname{Int}(\operatorname{csupp}P^{X})\cap\operatorname{supp}P^{X}$
and $P^{X}(\\{m_{0}\\})=0$, then Proposition S12(i) fails. This is
demonstrated by the next example, which is a modification of Example S32(b).

######  Example S33.

Fix $m\in(0,1)$ and for $n\in\mathbb{N}$, let $f_{n}\colon[0,1]\to[0,1]$ be
such that $f_{n}=0$ on $[0,m(1-1/n)]$, $f_{n}=1$ on $[m,1]$, and $f_{n}$ is
continuous and linear on $[m(1-1/n),m]$. For each $n$, let $P_{n}$ be the
distribution supported on $\\{(x,f_{n}(x)):x\in[0,1]\\}$ for which the
corresponding marginal distribution $P_{n}^{X}$ is the uniform distribution on
$D:=[0,m]\cup[m^{\prime},1]$ for some $m^{\prime}\in[m,1)$. Then $(P_{n})$
converges in $W_{2}$ to the uniform distribution $P$ on
$\\{(x,\mathbbm{1}_{[m,1]}(x)):x\in D\\}$.

Not all elements of $\psi^{0}(P)=\psi_{m}^{0}(P)$ agree at
$m\in\operatorname{supp}P^{X}$; for example when $m^{\prime}=m$ and $D=[0,1]$,
the functions in $\psi_{m}^{0}(P)$ agree with $\mathbbm{1}_{[m,1]}$ on
$[0,1]\setminus\\{m\\}$ but can take any value in $[0,1]$ at $m$. We have
$P^{X}(\\{m\\})=0$ and $m\in\operatorname{Int}(\operatorname{csupp}P^{X})$, so
the other conditions of Proposition S12(i) are satisfied, but
$\psi^{0}(P_{n})=\psi_{m}^{0}(P_{n})=\\{f_{n}\\}$ for all $n$ and $(f_{n})$
does not have a uniform limit on $[0,m]\subseteq\operatorname{supp}P^{X}$.

In Example S33, note in addition that while all the conditions of Corollary
S14(c) are met, the associated convergence statement cannot be extended to
$A=\operatorname{supp}P^{X}$. In general, Proposition S12(g) also does not
hold with $A=\operatorname{supp}P^{X}$ under the stated conditions
$P^{X}(\\{\tilde{m}_{0}\\})=0$ and $\psi_{\tilde{m}_{0}}^{0}(P)\neq\emptyset$.
To see this, we can instead take $D=[0,m]$ in Example S33 and fix
$m_{0}\in(m,1]$, so that $\tilde{m}_{0}=m$, $P^{X}(\\{\tilde{m}_{0}\\})=0$ and
$0\in\psi_{\tilde{m}_{0}}^{0}(P)$. Then there is a sequence $(g_{n})$ with
$g_{n}\in\psi_{m_{0}}^{0}(P_{n})$ for all $n$ but no uniform limit on
$\operatorname{supp}P^{X}=[0,m]$; for example, let $g_{n}$ be such that
$g_{n}=0$ on $[0,m(1-1/n)]$, $g_{n}(m)=1$ and $g_{n}$ is linear on
$[m(1-1/n),1]$.

## References

  * Amelunxen et al. (2014) Amelunxen, D., Lotz, M., McCoy, M. B. and Tropp, J. A. (2014). Living on the edge: phase transition in convex programs with random data.  _Inf. Inference_ , 3, 224–294.
  * Archontoulis and Miguez (2015) Archontoulis, S. V. and Miguez, F. E. (2015). Nonlinear regression models and applications in agricultural research.  _Agronomy J._ , 107, 786–798.
  * Balabdaoui et al. (2011) Balabdaoui, F., Jankowski, H., Pavlides, M., Seregin, A., Wellner, J. A. (2011). On the Grenander estimator at zero.  _Statist. Sinica_ , 21, 873–899.
  * Balász et al. (2015) Balász, G., Gyögy, A. and Szepesvári, C. (2015). Near-optimal max-affine estimators for convex regression.  _Proc. Mach. Learn. Res._ , 38, 56–64.
  * Baraud and Birgé (2016) Baraud, Y. and Birgé, L. (2016). Rates of convergence of rho-estimators for sets of densities satisfying shape constraints.  _Stoch. Proc. Appl._ , 12, 3888–3912.
  * Barber and Samworth (2021) Barber, R. F. and Samworth, R. J. (2021). Local continuity of log-concave projection, with applications to estimation under model misspecification.  _Bernoulli_ , to appear.
  * Barlow et al (1972) Barlow, R. E., Bartholomew, D. J., Bremner, J. M. and Brunk, H. D. (1972). _Statistical Inference under Order Restrictions_.  Wiley, New York.
  * Bellec (2018) Bellec, P. C. (2018). Sharp oracle inequalities for least squares estimators in shape restricted regression.  _Ann. Statist._ , 46, 745–780.
  * Billingsley (1999) Billingsley, P. (1999). Convergence of Probability Measures, 2nd edition.  Wiley, New York.
  * Cao et al. (2019) Cao, L., Shi, P.-J., Li, L. and Chen, G. (2019). A new flexible sigmoidal growth model.  _Symmetry_ , 11, 204.
  * Chatterjee (2014) Chatterjee, S. (2014). A new perspective on least squares under convex constraint.  _Ann. Statist._ , 42, 2340–2381.
  * Chatterjee (2016) Chatterjee, S. (2016). An improved global risk bound in concave regression.  _Electron. J. Stat._ , 10, 1608–1629.
  * Chatterjee et al. (2015) Chatterjee, S., Guntuboyina, A. and Sen, B. (2015). On risk bounds in isotonic and other shape restricted regression problems.  _Ann. Statist._ , 43, 1774–1800.
  * Chatterjee and Lafferty (2019) Chatterjee, S. and Lafferty, J. (2019). Adaptive risk bounds in unimodal regression.  _Bernoulli_ , 25, 1–25.
  * Chen and Samworth (2014) Chen, Y. and Samworth, R. J. (2014). scar: shape-constrained additive regression: a maximum likelihood approach. R package version 0.2-1. Available at https://CRAN.R-project.org/package=scar.
  * Chen and Samworth (2016) Chen, Y. and Samworth, R. J. (2016). Generalized additive and index models with shape constraints.  _J. Roy. Statist. Soc., Ser. B_ , 78, 729–754.
  * Christopoulos (2016) Christopoulos, D.T. (2016). On the efficient identification of an inflection point.  _International Journal of Mathematics and Scientific Computing_ , 6, 13–20.
  * Christopoulos (2019) Christopoulos, D.T. (2019). inflection: finds the inflection point of a curve. R package version 1.3.5. Available at https://cran.r-project.org/web/packages/inflection.
  * Cule et al. (2010) Cule, M., Samworth, R. and Stewart, M. (2010). Maximum likelihood estimation of a multi-dimensional log-concave density. _J. Roy. Statist. Soc., Ser. B. (with discussion)_ , 72, 545–607.
  * Deng et al. (2020) Deng, H., Han, Q. and Sen, B. (2020). Inference for local parameters in convexity constrained models.  Available at https://arxiv.org/abs/2006.10264.
  * Dümbgen et al. (2007) Dümbgen, L., Hüsler, A. and Rufibach, K. (2007). Active set and EM algorithms for log-concave densities based on complete and censored data.  Available at https://arxiv.org/abs/0707.4643v4.
  * Dümbgen et al. (2011) Dümbgen, L., Samworth, R. and Schuhmacher, D. (2011). Approximation by log-concave distributions, with applications to regression.  _Ann. Statist._ , 39, 702–730.
  * Edner et al. (1989) Edner, H., Faris, G. W., Sunesson, A., Svanberg, S. (1989). Atmospheric atomic mercury monitoring using differential absorption lidar techniques.  _Appl. Opt._ , 28, 921–930.
  * Edner et al. (1992) Edner, H., Ragnarson, P., Svanberg, S., Wallinder, E., Deliso, A., Ferrara, R. and Maserti, B. E. (1992). Differential absorption lidar mapping of atmospheric atomic mercury in Italian geothermal fields.  _J. Geophys. Res._ , 97, 3779–3786.
  * Feng et al. (2021) Feng, O. Y., Guntuboyina, A., Kim, A. K. H. and Samworth, R. J. (2021). Adaptation in multivariate log-concave density estimation.  _Ann. Statist._ , 49, 129–153.
  * Feng et al. (2021) Feng, O. Y., Chen, Y., Han, Q., Carroll, R. J. and Samworth, R. J. (2021). Sshaped: Estimation of an S-shaped function. R package version 0.99. Available at https://CRAN.R-project.org/package=Sshaped.
  * Fraser and Massam (1989) Fraser, D. A. S. and Massam, H. (1989). A mixed primal-dual bases algorithm for regression under inequality constraints. Application to concave regression.  _Scand. J. Statist._ , 16, 65–74.
  * Frisch (1964) Frisch, R. (1964). _Theory of Production_.  Springer Science & Business Media.
  * Gibbs (2000) Gibbs, M. N. (2000). Variational Gaussian process classifiers.  _IEEE Trans. Neural Networks_ , 11, 1458–1464.
  * Giné and Nickl (2016) Giné, E. and Nickl, R. (2016). _Mathematical Foundations of Infinite-Dimensional Statistical Models_.  Cambridge University Press, Cambridge.
  * Ghosal and Sen (2017) Ghosal, P. and Sen, B. (2017). On univariate convex regression.  _Sankhya Ser A_ , 79, 215–253.
  * Ginsberg (1974) Ginsberg, W. (1974). The multiplant firm with increasing returns to scale.  _J. Econ. Theory_ , 9, 283–292.
  * Groeneboom (1996) Groeneboom, P. (1996). Inverse problems in statistics. In _Proceedings of the St. Flour Summer School in Probability_.  _Lecture Notes in Math._ , 1648, 67–164.
  * Groeneboom and Jongbloed (2014) Groeneboom, P. and Jongbloed, G. (2014). _Nonparametric Estimation under Shape Constraints_.  Cambridge University Press, Cambridge.
  * Groeneboom et al. (2008) Groeneboom, P., Jongbloed, G. and Wellner, J. A. (2008). The support reduction algorithm for computing non-parametric function estimates in mixture models.  _Scand. J. Statist._ , 35, 385–399.
  * Guntuboyina and Sen (2013) Guntuboyina, A. and Sen, B. (2013). Global risk bounds and adaptation in univariate convex regression.  _Probab. Theory and Related Fields_ , 163, 379–411.
  * Guntuboyina and Sen (2018) Guntuboyina, A. and Sen, B. (2018). Nonparametric shape-restricted regression.  _Statist. Sci._ , 33, 568–594.
  * Han (2021) Han, Q. (2021). Set structured global empirical risk minimizers are rate optimal in general dimensions  _Ann. Statist._ , to appear.
  * Han and Kato (2021) Han, Q. and Kato, K. (2021). Berry–Esseen bounds for Chernoff-type non-standard asymptotics in isotonic regression  _Ann. Appl. Probab._ , to appear.
  * Han et al. (2019) Han, Q., Wang, T., Chatterjee, S. and Samworth, R. J. (2019). Isotonic regression in general dimensions.  _Ann. Statist._ , 47, 2440–2471.
  * Han and Wellner (2016) Han, Q. and Wellner, J. A. (2016). Multivariate convex regression: global risk bounds and adaptation.  Available at https://arxiv.org/abs/1601.06844.
  * Holst et al. (1996) Holst, U., Hössjer, O., Björklund, C., Ragnarson, P., and Edner, H. (1996). Locally weighted least squares kernel regression and statistical evaluation of LIDAR measurements.  _Environmetrics_ , 7, 401–416.
  * Jarne et al. (2007) Jarne, G., Sanchez-Choliz, J. and Fatas-Villafranca, F. (2007). “S-shaped” curves in economic growth. A theoretical contribution and an application.  _Evol. Inst. Econ. Rev._ , 3, 239–259.
  * Kachouie and Schwartzman (2013) Kachouie, N. N. and Schwartzman, A. (2013). Non-parametric estimation of a single inflection point in noisy observed signal. _J. Electr. Electron. Syst._ , 2.
  * Kim et al. (2018) Kim, A. K. H., Guntuboyina, A. and Samworth, R. J. (2018). Adaptation in log-concave density estimation.  _Ann. Statist._ , 46, 2279–2306.
  * Kulikov and Lopuhaä (2006) Kulikov, V. N. and Lopuhaä, H. P. (2006). The behavior of the NPMLE of a decreasing density near the boundaries of the support. _Ann. Statist._ , 34, 742–768.
  * Kur et al. (2020) Kur, G., Gao, F., Guntuboyina, A. and Sen. B. (2020). Convex regression in multidimensions: suboptimality of least squares estimators.  Available at https://arxiv.org/abs/2006.02044.
  * Lee et al. (2020) Lee, U., Carroll, R. J., Marder, K., Wang, Y. and Garcia, T. P. (2020). Estimating disease onset from change points of markers measured with error.  _Biostatistics_ , kxz068.
  * Liao and Meyer (2016) Liao, X. and Meyer, M. (2016). ShapeChange: Change-point estimation using shape-restricted splines.  R package version 1.4. Available at https://CRAN.R-project.org/package=ShapeChange.
  * Liao and Meyer (2017) Liao, X. and Meyer, M. (2017). Change-point estimation using shape-restricted regression splines.  _J. Stat. Plan. Inference_ , 188, 8–21.
  * Meyer (1999) Meyer, M. (1999). An extension of the mixed primal-dual bases algorithm to the case of more constraints than dimensions.  _J. Statist. Plann. Inference_ , 81, 13–31.
  * Moreau (1962) Moreau, J. J. (1962). Décomposition orthogonale d’un espace hilbertien selon deux cônes mutuellement polaires.  _C. R. Acad. Sci._ , 255, 238–240.
  * Nocedal and Wright (2006) Nocedal, J., and Wright, S. J. (2006). _Numerical Optimization_ , 2nd edition.  Springer–Verlag, New York.
  * Pananjady and Samworth (2021) Pananjady, A. and Samworth, R. J. (2021). Isotonic regression with unknown permutations: statistics, computation, and adaptation.  _Ann. Statist._ , to appear.
  * Pya and Wood (2015) Pya, N. and Wood, S. N. (2015). scam: Shape Constrained Additive Models.  R package version 1.2–11. Available at https://CRAN.R-project.org/package=scam.
  * Remmert (1991) Remmert, R. (1991). Theory of Complex Functions.  Springer–Verlag, New York.
  * Rockafellar (1997) Rockafellar, R. T. (1997). Convex Analysis.  Princeton University Press, Princeton.
  * Rudin (1987) Rudin, W. (1987). Real and Complex Analysis, 3rd edition.  McGraw–Hill, New York.
  * Ruppert et al. (1995) Ruppert, D., Sheather, S. J. and Wand, M. P. (1995). An effective bandwidth selector for local least squares regression.  _J. Amer. Statist. Assoc._ , 90, 1257–1270.
  * Ruppert et al. (2003) Ruppert, D., Wand, M. P. and Carroll, R. J. (2003). Semiparametric Regression.  Cambridge University Press, Cambridge.
  * Samworth (2018) Samworth, R. J. (2018). Recent progress in log-concave density estimation.  _Statist. Sci._ , 33, 493–509.
  * Schneider (2014) Schneider, R. (2014). Convex Bodies: The Brunn–Minkowski Theory, 2nd edition.  Cambridge University Press, Cambridge.
  * Seijo and Sen (2011) Seijo, E. and Sen, B. (2011). Nonparametric least squares estimation of a multivariate convex regression function.  _Ann. Statist._ , 39, 1633–1657.
  * Shoung and Zhang (2001) Shoung, J.-M. and Zhang, C. H. (2001). Least squares estimators of the mode of a unimodal regression function.  _Ann. Statist._ , 29, 648–665.
  * Smith (2010) Smith, J. O. (2010). _Physical Audio Signal Processing_. W3K Publishing.
  * Stout (2008) Stout, Q. F. (2008). Unimodal regression via prefix isotonic regression.  _Comput. Statist. Data Anal._ , 53, 289–297.
  * Tarde (1903) Tarde, G. (1903). _The Laws of Imitation_. H. Holt & Co, New York.
  * van der Vaart (1998) van der Vaart, A. W. (1998). Asymptotic Statistics.  Cambridge University Press, Cambridge.
  * van der Vaart and Wellner (1996) van der Vaart, A. W. and Wellner, J. A. (1996). Weak Convergence and Empirical Processes.  Springer–Verlag, New York.
  * van Genuchten and Gupta (1993) van Genuchten, M. Th. and Gupta, S. K. (1993). A reassessment of the crop tolerance response function.  _J. Indian Soc. Soil Sci._ , 41, 730–737.
  * Wand (2018) Wand, M. (2018). SemiPar: Semiparametic Regression.  R package version 1.0–4.2. Available at https://CRAN.R-project.org/package=SemiPar.
  * Yagi et al. (2019) Yagi, D., Chen, Y., Johnson, A. L. and Morita, H. (2019). An axiomatic nonparametric production function estimator: modeling production in Japan’s cardboard industry. Available at https://arxiv.org/abs/1906.08359.
  * Yagi et al. (2020) Yagi, D., Chen, Y., Johnson, A. L. and Kuosmanen, T. (2020). Shape-constrained kernel-weighted least squares: estimating production functions for Chilean manufacturing industries. _J. Bus. Econ. Statist._ , 38, 43–54.
  * Zeidi (1993) Zeidi, B. (1993). Analysis of growth equations. _Forest Science_ , 39, 594–616.