Functional Measurement Error in Functional Regression

Sneha Jadhav, Shuangge Ma

Yale University

> Abstract: Measurement error is an important problem that has not been very
> well studied in the context of Functional Data Analysis. To the best of our
> knowledge, there are no existing methods that address the presence of
> functional measurement errors in generalized functional linear models. A
> framework is proposed for estimating the slope function in the presence of
> measurement error in the generalized functional linear model with a scalar
> response. This work extends the conditional-score method to the case when
> both the measurement error and the independent variables lie in an infinite
> dimensional space. Asymptotic results are obtained for the proposed estimate
> and its behavior is studied via simulations, when the response is continuous
> or binary. It’s performance on real data is demonstrated through a
> simulation study based on the Canadian Weather data-set, where errors are
> introduced in the data-set and it is observed that the proposed estimate
> indeed performs better than a naive estimate that ignores the measurement
> error.
>
> Key words and phrases: Measurement Error, Error in Variables, Functional
> Data Analysis, Generalized Functional Linear Models.

##  1 Introduction

Measurement error in multivariate data is a well-studied problem, and
consequently there are multiple ways to address it Carroll et al. (2004). This
problem arises in many diverse fields like nutrition, environmental studies
and so on Carroll & Raymond (1998). It is well established that ignoring this
error can lead to several problems like bias in the estimation of regression
parameters. For a detailed discussion on the repercussions of ignoring
measurement error, refer to Carroll et al. (2004). Measurement error also
arises in functional data where a large number of repeated measurements for
variables are available. It is only natural to assume that if there is error
in the data, it is present at all of the repeated measurements i.e. the
measurement error is a functional variable. In the Functional Data Analysis
literature, there are limited tools available to handle measurement errors.
Most of the available literature assumes measurement error at the discrete
points at which the functions are observed (Yao, Müller & Wang, 1998; Cardot,
Crambes et al. 2007 ; James, 2002; Crambes, Kneip & Sarda, 2009; Goldsmith et
al., 2011 ,Goldsmith, Wand & Crainiceanu, 2011). Specifically, denote the
observed values of the function $X(\cdot)$ at a grid of points $a\leq
t_{1}<...<t_{m}\leq b$ by $X_{1},...,X_{m}$. All of these works assume
$X_{k}=X(t_{k})+e_{k},\,k=1,...,m$ and the errors $e_{k}$ are indepedent or
uncorrelated. This assumption on errors is very restrictive, and the
asymptotics as well as the performance of these methods depends on the
validity of these assumptions. A recent work Cai (2015), allows for
correlation between the measurement errors however, imposes parametric
structures on the covariance. To the best of our knowledge, there is only one
work that considers the measurement error to be functional in nature
(Chakraborty & Panaretos, 2017). The measurement error model considered is
$W(\cdot)=X(\cdot)+U(\cdot),$ where $U(\cdot)$ is a measurement error
stochastic process. In order to avoid identifiability issues, certain
conditions are imposed. It is assumed that the measurement error process is at
a much finer scale than the true covariate. This is achieved by imposing the
following two conditions: 1) there exists $\delta>0$, such that
$cov(U(t),U(s))=0$ if $|s-t|>\delta$ and 2) the covariance operator of
$X(\cdot)$ is analytic on an open set containing $|s-t|\leq\delta$. Estimation
of the covariance function of the error process under these conditions, is
given in Descary & Panaretos (2016). The assumptions are more general than
those made previously but still quite restrictive. Moreover, none of the
methods can accommodate measurement error process with correlations in a
regression model with a binary response.

To fill the knowledge gap, we develop a framework that will allow for
measurement error with a more general correlation structure and response that
is binary or has a normal distribution. This proposed framework is based on
the conditional-score method proposed in Stefanski & Carroll (1987). We use
the Karhunen-Loéve expansion to obtain estimating equations similar to those
in Stefanski & Carroll (1987). However, in our case the number of parameters
diverge, making this framework and involved asymptotics non-trivial. The
simulations presented later, demonstrates how measurement errors lead to an
in-correct estimate of the slope function.

##  2 Methods

Given covariate $X(t),\,t\in L^{2}=L^{2}[a,b],$ assume that $Y$ has the
distribution $f_{Y}\\{y;\theta_{1},x(\cdot)\\}$ with respect to a dominating
measure $h$ given by:

$\displaystyle\text{exp}\left[\dfrac{y\\{\tilde{\beta}_{0}+\int_{a}^{b}\tilde{\beta}(t)x(t)d(t)\\}-b\\{\tilde{\beta}_{0}+\int_{a}^{b}\tilde{\beta}(t)x(t)dt\\}}{a(\tilde{\phi})}+c(y,\tilde{\phi})\right].$
(2.1)

All integrals hereafter are taken over $[a,b]$ and ,
$\theta_{1}=(\tilde{\beta}_{0},\tilde{\beta}(\cdot),\tilde{\phi})$. Refer to
Stefanski and Carroll (1987) for details on all the distributions that are
included in the above model.

Let $(\rho_{k})_{k=1}^{\infty}$ be orthonormal basis functions in the $L^{2}$
space. Using basis expansion, we obtain

$\displaystyle\int\tilde{\beta}(t)X(t)dt=\sum\limits_{k=1}^{\infty}X_{k}\tilde{\beta}_{k},$

where $X_{k}=\int X(t)\rho_{k}(t)dt$,
$\tilde{\beta}_{k}=\int\tilde{\beta}(t)\rho_{k}(t)dt,k\geq 1$. Thus, model
(2.1) has infinitely many parameters. We address this issue of infinite
dimension with a truncation strategy. Denote
$\tilde{\beta}=(\tilde{\beta}_{1},...,\tilde{\beta}_{p_{n}})^{{}^{\prime}}$
and $\tilde{\theta}=(\tilde{\beta}_{0},\tilde{\beta},\tilde{\phi})$. Instead
of the model (2.1), we work with the following sequence of models with
increasing dimension $p_{n}$:

$\displaystyle f_{Y}(y;\tilde{\theta},x)$
$\displaystyle=\text{exp}\left\\{\dfrac{y\left(\tilde{\beta}_{0}+\sum\limits_{k=1}^{p_{n}}x_{k}\tilde{\beta}_{k}\right)-b\left(\tilde{\beta}_{0}+\sum\limits_{k=1}^{p_{n}}x_{k}\tilde{\beta}_{k}\right)}{a(\tilde{\phi})}+c(y,\tilde{\phi})\right\\},$
(2.2)

where $p_{n}\to\infty$ as $n\to\infty$. Due to measurement error, we observe
surrogate variable $W(\cdot)$ as opposed to the true covariate $X(\cdot)$ .
Assume

$\displaystyle W(\cdot)=X(\cdot)+U(\cdot),$ (2.3)

where $U(\cdot)$ is a Gaussian process with mean function $0$ and covariance
function $K(\cdot,\cdot)$. Take $(\rho_{i})_{i=1}^{\infty}$ to be the basis
constructed from the eigenfunctions of the integral operator $\mathcal{K}$
associated with the covariance function $K(\cdot,\cdot)$. Denote $W_{k}=\int
W(t)\rho_{k}(t)dt$ and $U_{k}=\int U(t)\rho_{k}(t)dt$. This yields the
following measurement error set-up for (2.2):

$W_{k}=X_{k}+U_{k},\,1\leq k\leq p_{n},$

where $U_{k}$’s are independent and $U_{k}\sim N(0,\lambda_{k})$ with
$\lambda_{k}$ being the eigenvalue associated with the $k^{th}$ eigenfunction
of $\mathcal{K}$. This set-up is similar to that in Stefanski and Carroll
(1987) where the authors proposed the conditional-score based sufficiency
estimator for generalized linear models. For a sample of size $n$, assume that
$X_{i}(\cdot),\,i=1,...,n$ are deterministic and denote
$\mathbf{X}_{i}=(X_{i1},...,X_{ip_{n}})^{{}^{\prime}}$ and
$\mathbf{W}_{i}=(W_{i1},...,W_{ip_{n}})^{{}^{\prime}}$. Let
$\widetilde{W}_{i}^{(l)}(\cdot)$ denote the $l^{th}$ replicate of the function
$W_{i}(\cdot)$ and
$\widetilde{W}_{ik}^{(l)}=\int\rho_{k}(t)\widetilde{W}_{i}^{(l)}(t)dt$. Let
$\widetilde{\mathbf{W}}_{i}^{(l)}=(\widetilde{W}_{i1}^{(l)},...,\widetilde{W}_{ip_{n}}^{(l)})^{\prime}.$
The likelihood of
$\widetilde{W}_{i}=(\widetilde{\mathbf{W}}_{i}^{(1)},...,\widetilde{\mathbf{W}}_{i}^{(m)})^{\prime}$
is

$\displaystyle
f_{\widetilde{W}_{i}}(\widetilde{w}_{i},\tilde{\theta},\mathbf{x}_{i})$
$\displaystyle=\prod\limits_{j=1}^{m}\dfrac{(2\pi)^{-p_{n}/2}}{|\Omega_{1}|}\text{exp}\left\\{\dfrac{-1}{2}(\widetilde{\mathbf{w}}_{i}^{(j)}-\mathbf{x}_{i})^{T}(\widetilde{\mathbf{w}}_{i}^{(j)}-\mathbf{x}_{i})\right\\},$
(2.4)

where, $\Omega_{1}=\text{diag}(\lambda_{1},...,\lambda_{p_{n}})$ is the
unknown covariance matrix of the measurement error vector and $m$ denotes the
number of replicates. Without loss of generality we assume that $m=1$ and
$\Omega=\Omega_{1}/a(\tilde{\phi})$ is known. In practice we use the estimate
of $\Omega_{1}$ and $a(\tilde{\phi})$ that are given subsequently. The
estimate of $\Omega_{1}$ is obtained from the estimate of the covariance
function $K(\cdot,\cdot)$. We can use the method in Chakraborty & Panaretos
(2017) to estimate this covariance function. In this case, the covariance
function has to satisfy the two assumptions mentioned in the previous section.
If not, then additional data like replicates is required so that the
covariance function can be estimated without these assumptions. Thus, there is
always trade-off between additional data and assumptions in the measurement
error framework. Most methods related to functional data assume that there is
no measurement error, a few that relax this assumption make very strong
assumptions regarding its structure. We relax these stringent assumptions in
presence of replicates. Thus, if there is any reason to suspect measurement
error, a sound strategy is to collect additional required information right at
the outset instead of relying on unverifiable assumptions regarding
measurement error. Let $Y,\,\mathbf{W}$ denote random variables with the same
distribution as $Y_{i},\,\mathbf{W}_{i},\,i=1,...,n$ respectively. We refer to
the variable $Y$ as the response variable. For now, we drop the subscript $i$
and use it when necessary. We assume that $\mathbf{W}$ has no information on
$Y$ other than what is contained in $\mathbf{X}$ that is,

$\displaystyle
f_{Y,\mathbf{W}}(y,\mathbf{w};\tilde{\theta},\mathbf{x})=f_{Y}(y;\tilde{\theta},\mathbf{x})f_{\mathbf{W}}(\mathbf{w},\tilde{\theta},\mathbf{x}).$

Let
$\mathbf{\Delta}(\beta)=\mathbf{W}+Y\Omega\beta=(\Delta_{1}(\beta),...,\Delta_{p_{n}}(\beta))^{{}^{\prime}},\,\mathbf{\Delta}(\tilde{\beta})=\mathbf{\Delta}=(\Delta_{1},...,\Delta_{p_{n}})^{{}^{\prime}}$.
The distribution
$f_{Y|\mathbf{\Delta}}(y|\mathbf{\Delta}=\mathbf{\delta};\tilde{\theta})$ of
$Y|\mathbf{\Delta}$ is given by

$\displaystyle\text{exp}\left[y\tilde{\eta}-\dfrac{1}{2}y^{2}\tilde{\beta}^{{}^{\prime}}\Omega\tilde{\beta}/a(\tilde{\phi})+c(y,\tilde{\phi})-\text{log}\left\\{S(\tilde{\eta},\tilde{\beta},\tilde{\phi})\right\\}\right],$
(2.5)

where
$S(\tilde{\eta},\tilde{\beta},\tilde{\phi})=\displaystyle\int\text{exp}\left\\{y\tilde{\eta}-\dfrac{1}{2}y^{2}\tilde{\beta}^{{}^{\prime}}\Omega\tilde{\beta}/a(\tilde{\phi})+c(y,\tilde{\phi})\right\\}dh(y)$,
$\tilde{\eta}=(\tilde{\beta}_{0}+\mathbf{\delta}^{{}^{\prime}}\tilde{\beta})/a(\tilde{\phi})$.
This distribution belongs to the exponential family. Let
$\theta=(\beta_{0},\beta,\phi)^{{}^{\prime}},\,\beta=(\beta_{1},...,\beta_{p_{n}})^{{}^{\prime}},\\\
\Psi(y,\mathbf{w},\theta)=(\partial/\partial\theta)log\,f_{Y/\mathbf{\Delta}}(y|\mathbf{\delta};\theta),\,\mathbf{\delta}(\beta)=\mathbf{w}+y\Omega\beta$.
We use the following estimating equation to estimate $\theta$:

$\displaystyle\sum_{i=1}^{n}\Psi(Y_{i},\mathbf{W}_{i},\theta)=\mathbf{0},$
(2.6)

where $\Psi(Y_{i},\mathbf{W}_{i},\theta)$ is,

$\displaystyle\begin{bmatrix}\left[Y_{i}-E(Y_{i}|\Delta(\beta)=\mathbf{\delta}_{i}(\beta))\right]/a\left(\phi\right)\\\
\left[Y_{i}-E(Y_{i}|\mathbf{\Delta}_{i}(\beta)=\mathbf{\delta}_{i}(\beta))\right]\dfrac{\mathbf{\delta}_{i}(\beta)}{a(\phi)}-\left[y_{i}^{2}-E(Y_{i}^{2}|\mathbf{\Delta}_{i}(\beta)=\mathbf{\delta}_{i}(\beta))\right]\dfrac{\Omega\beta}{a(\phi)}\\\
r(Y_{i},\mathbf{W}_{i},\theta)-E(r(Y_{i},\mathbf{W}_{i},\theta)|\mathbf{\Delta}_{i}(\beta)=\mathbf{\delta}_{i}(\beta))\end{bmatrix},$

$r(Y_{i},\mathbf{W}_{i},\theta)=\dfrac{\partial
c(Y_{i},\theta)}{\partial\phi}-Y_{i}\dfrac{\beta_{0}+\mathbf{\delta}_{i}(\beta)^{{}^{\prime}}\beta}{a^{2}(\phi)}a^{\prime}(\phi)+Y_{i}^{2}\dfrac{\beta^{{}^{\prime}}\Omega\beta}{2a^{2}(\phi)}a^{\prime}(\phi),\,a^{{}^{\prime}}(x)$
denotes the derivative of function $a$ and $\mathbf{0}$ here and hence onwards
denotes a column vector containing $0$’s of appropriate dimension. Note that
the estimating equation (2.6) is unbiased for $\theta$ and the corresponding
estimator does not maximize the conditional likelihood. We study this
estimator in greater details when $Y$ is binary and gaussian.

###  2.1 Binary Response

When $Y$ is binary, the truncated model is
$P_{\tilde{\theta}}(Y=1|\textbf{X})=F(\tilde{\beta}_{0}+\textbf{X}^{{}^{\prime}}\tilde{\beta}),\,F(t)=1/(1+e^{-t})$.
For this case, it is easy to see that (2.5) gives
$P_{\tilde{\theta}}(Y=1|\Delta=\delta)=F\\{\tilde{\beta}_{0}+(\delta-0.5\Omega\tilde{\beta})^{{}^{\prime}}\tilde{\beta}\\}$.
Combining this with (2.6), we get the following estimating equations

$\displaystyle\sum_{i=1}^{n}\Psi(Y_{i},\mathbf{W}_{i},\theta)=\sum_{i=1}^{n}[Y_{i}-F[\beta_{0}+\\{\delta_{i}(\beta)-0.5\Omega\beta\\}^{{}^{\prime}}\beta]\begin{bmatrix}1\\\
\delta_{i}(\beta)-\Omega\beta\end{bmatrix}=\begin{bmatrix}0\\\
\textbf{0}\end{bmatrix}.$

Let

$\displaystyle\beta_{c}=(\beta_{0},\beta)^{{}^{\prime}},\,X_{c}=(1,\textbf{X}^{{}^{\prime}})^{{}^{\prime}},\,W_{c}=(1,\textbf{W}^{{}^{\prime}})^{{}^{\prime}},\,U_{c}=(0,U^{{}^{\prime}}),\,$
$\displaystyle\Omega_{c}=\begin{bmatrix}0,\textbf{0}^{\prime}\\\
\textbf{0},\Omega\end{bmatrix},\,\delta_{c}(\beta_{c})=W_{c}+Y\Omega_{c}\beta_{c}.$

Using these notations we can rewrite
$U(\beta_{c})=\sum_{i=1}^{n}\Psi(Y_{i},\mathbf{W}_{i},\theta)=\textbf{0},$ as

$\displaystyle
U(\beta_{c})=\sum_{i=1}^{n}[Y_{i}-F\\{\delta_{ci}(\beta_{c})^{{}^{\prime}}\beta_{c}-0.5\beta_{c}^{{}^{\prime}}\Omega_{c}\beta_{c}\\}]\begin{bmatrix}\delta_{ci}(\beta_{c})-\Omega_{c}\beta_{c}\end{bmatrix}=\textbf{0}.$
(2.7)

We now state the assumptions needed to show the existence and consistency of
the solution to this set of equations. We denote the Frobenius norm for a
matrix and euclidean norm for vectors by $\|\cdot\|$ , and a positive constant
by $c$.

######  Assumption 1.

Assume that $\int\tilde{\beta}^{2}(t)dt<\infty$. Then
$\|\tilde{\beta}\|<m_{1}<\infty.$

######  Assumption 2.

Assume that $\int X_{i}^{2}(t)dt<m_{2}<\infty.$

######  Assumption 3.

Let $\|\Omega\|<\infty$.

######  Assumption 4.

Let $p_{n}^{4}/n\rightarrow 0$.

######  Assumption 5.

Let $B=\lambda_{min}\left(\sum_{i=1}^{n}W_{ic}W^{\prime}_{ic}/n\right)$. Then

$\lambda_{max}(\Omega)\text{exp}(\lambda_{max}(\Omega)m_{1})\leq
B\dfrac{\text{exp}\left(\underset{1\leq i\leq
n}{inf}W_{ic}^{{}^{\prime}}\tilde{\beta}-\underset{1\leq i\leq
n}{sup}W_{ic}^{{}^{\prime}}\tilde{\beta}\right)}{1+\text{exp}\left(\underset{1\leq
i\leq n}{inf}W_{ic}^{{}^{\prime}}\tilde{\beta}\right)}\quad{a.s.}$

In the rest of the paper supremum and infimum are taken over $i=1,...,n$ when
not indicated explicitly. Without loss of generality we assume that the
function $\bar{X}=\sum_{i=1}^{n}n^{-1}X_{i}(t)=0$. This can be achieved by
replacing $X_{i}(\cdot)$ by $X_{i}^{c}(\cdot)=X_{i}(\cdot)-\bar{X}(\cdot)$ and
adjusting the intercept in the (2.1). The corresponding adjusted measurement
error model is $W_{i}^{c}(\cdot)=X_{i}^{c}(\cdot)+U_{i}(\cdot)$, where
$W_{i}^{c}(\cdot)=W_{i}(\cdot)-\sum_{i=1}^{n}n^{-1}E\\{W_{i}(t)\\}$. Let
$W_{ik}^{c}=\int W_{i}^{c}(t)\rho_{k}(t)dt,\,X_{ik}^{c}=\int
X_{i}^{c}(t)\rho_{k}(t)dt$ and
$\overline{W}_{k}=\sum_{i=1}^{n}n^{-1}E(W_{ik}).$ The adjusted error model
leads to $W_{ik}^{c}=X_{ik}^{c}+U_{ik}$, $W_{ik}^{c}=W_{ik}-\overline{W}_{k}.$
Thus, we need to replace the observed variables $W_{ik}$ by
$W_{ik}-\overline{W}_{k}$ for $k=1,...,p_{n}$. Using law of large numbers for
independent variables we can obtain a consistent estimate of
$\overline{W}_{k}$ as $\sum_{i=1}^{n}n^{-1}W_{ik}$ .

The following theorem is sufficient to prove weak consistency and existence of
the estimator. This follows from Theorem 6.3.4 of Ortega and Rheinboldt
(1970).

######  Theorem 1.

For all $\epsilon>0$, there exists a constant $\zeta>0$ such that for
sufficiently large $n$,

$P\left(\underset{||\beta_{c}-\tilde{\beta}_{c}||=\zeta\sqrt{p_{n}/n}}{\sup}(\beta_{c}-\tilde{\beta_{c}})^{\prime}U(\beta_{c})<0\right)\geq
1-\epsilon.$

Proof Using Taylors Theorem,

$\displaystyle(\beta_{c}-\tilde{\beta}_{c})^{{}^{\prime}}U(\beta_{c})$
$\displaystyle=(\beta_{c}-\tilde{\beta}_{c})^{{}^{\prime}}U(\tilde{\beta}_{c})+(\beta_{c}-\tilde{\beta}_{c})^{{}^{\prime}}J(\beta_{c}^{*})(\beta_{c}-\tilde{\beta}_{c})$
$\displaystyle=A_{1}+A_{2},$

where $\beta_{c}^{*}$ lies between $\beta_{c}$ and $\tilde{\beta}_{c}$ i.e.
$max(\|\beta_{c}^{*}-\beta_{c}\|,\|\beta_{c}^{*}-\tilde{\beta}_{c})\|\leq\|\beta_{c}-\tilde{\beta}_{c}\|$.
From Proposition 1 we get

$\displaystyle\underset{||\beta_{c}-\tilde{\beta}_{c}||=\zeta\sqrt{p_{n}/n}}{\sup}A_{1}=\zeta
O_{p}(\sqrt{n})\sqrt{p_{n}/n}=\zeta O_{p}(\sqrt{p_{n}}).$

From Proposition 3 and Proposition 4,

$\displaystyle\underset{||\beta_{c}-\tilde{\beta}_{c}||=\zeta\sqrt{p_{n}/n}}{\sup}A_{2}$
$\displaystyle=(\beta_{c}-\tilde{\beta}_{c})^{{}^{\prime}}\\{J(\beta_{c}^{*})-J(\tilde{\beta_{c}})\\}(\beta_{c}-\tilde{\beta}_{c})$
$\displaystyle\quad+(\beta_{c}-\tilde{\beta}_{c})^{{}^{\prime}}J(\tilde{\beta}_{c})(\beta_{c}-\tilde{\beta}_{c})$
$\displaystyle\leq\zeta^{2}p_{n}^{2}/\sqrt{n}-c\zeta^{2}p_{n}+\zeta^{2}o_{p}(p_{n})$
$\displaystyle\leq\zeta^{2}o(1)-c\zeta^{2}p_{n}+\zeta^{2}o_{p}(p_{n}).$

Thus,
$\underset{||\beta_{c}-\tilde{\beta}_{c}||=\zeta\sqrt{p_{n}/n}}{\sup}(\beta_{c}-\tilde{\beta_{c}})^{\prime}U(\beta_{c})$
is asymptotically dominated by $-c\zeta^{2}p_{n}$ and the result is proved.

###  2.2 Gaussian Response

We now consider the case where $Y$ has normal distribution. In this case,
$\Psi(Y_{i},\mathbf{W}_{i},\theta)$ in equations (2.6) can be written as

$\displaystyle\Psi(Y_{i},\mathbf{W}_{i},\theta)=\begin{bmatrix}\dfrac{1}{\sigma^{2}}(Y_{i}-\mu_{i})\\\
\dfrac{\Omega\beta}{1+\beta^{\prime}\Omega\beta}-\dfrac{1}{\sigma^{2}}\left[(Y_{i}-\mu_{i})^{2}\Omega\beta-(Y_{i}-\mu_{i})\\{\mathbf{\delta}_{i}(\beta)-2\mu_{i}\Omega\beta\\}\right]\\\
\dfrac{-1}{2\sigma^{2}}+\dfrac{(Y_{i}-\mu_{i})^{2}(1+\beta^{\prime}\Omega\beta)}{2\sigma^{4}}\end{bmatrix}.$
(2.8)

Denote identity matrix by $I$. Let
$\mathbf{\Delta}_{i}^{*}(\beta)=(I+\Omega\beta\beta^{\prime})^{-1}\\{\mathbf{\Delta}_{i}(\beta)-\beta_{0}\Omega\beta\\}$
and
$\mu_{i}=(\beta_{0}+\beta^{\prime}\mathbf{\delta}_{i}(\beta))/(1+\beta^{\prime}\Omega\beta)$.
Consider the following equations:

$\displaystyle\sum\limits_{i=1}^{n}(\mathbf{\Delta}_{i}^{*}(\beta)Y_{i}-\mathbf{\Delta}_{i}^{*}(\beta)\beta_{0}-\mathbf{\Delta}_{i}^{*}(\beta)\beta^{\prime}\mathbf{\Delta}_{i}^{*}(\beta))=0,$
(2.9)
$\displaystyle\sum\limits_{i=1}^{n}(Y_{i}-\beta_{0}-\beta^{\prime}\mathbf{\Delta}_{i}^{*}(\beta))=\textbf{0},$
$\displaystyle\sigma^{2}=\dfrac{1+\beta^{\prime}\Omega\beta}{n}\sum\limits_{i=1}^{n}(Y_{i}-\mu_{i})^{2}.$

The solution to equations (2.9) is also a solution to equations (2.8). Note
that the above equations are non-linear. Let
$\mathbf{W}^{c}=\mathbf{W}-\mathbf{\overline{W}},Y^{c}=Y-\overline{Y}$, where
$\overline{Y}$ and $\overline{W}$ denote the average of $Y_{1},...,Y_{n}$ and
$\mathbf{W}_{1},...,\mathbf{W}_{n}$ respectively. Then, the solution to
equations (2.9) is also a solution to the following equations:

$\displaystyle
U(\beta)=-\sum\limits_{i,j=1}^{n}\mathbf{W}_{i}^{c}Y_{i}^{c}\beta^{T}\Omega\beta+\sum\limits_{i,j=1}^{n}Y^{c2}_{i}\Omega\beta-\sum\limits_{i,j=1}^{n}\mathbf{W}_{i}^{c}\mathbf{W}_{i}^{cT}\beta+\sum\limits_{i,j=1}^{n}\mathbf{W}_{i}^{c}Y_{i}^{c}=\textbf{0}$
$\displaystyle\beta_{0}=\overline{Y}-\beta\overline{\mathbf{W}},\sigma^{2}=\dfrac{1+\beta^{T}\Omega\beta}{n}\sum\limits_{i=1}^{n}(Y_{i}-\mu_{i})^{2}.$
(2.10)

In addition to Assumptions 1, 2, 3, we need the following assumptions to show
that the equations (2.10) has a solution $\hat{\beta}$ and that this solution
is consistent.

######  Assumption 6.

Assume
$\lambda_{max}(\Omega_{1})\leq\lambda_{min}\left(\sum\limits_{i=1}^{n}\mathbf{X}_{i}^{c}\mathbf{X}_{i}^{c^{\prime}}/n\right)+\lambda_{min}(\Omega_{1})$.

######  Assumption 7.

Let $p_{n}^{3}/n\to 0.$

######  Theorem 2.

For all $\epsilon>0$, there exists a constant $\zeta>0$ such that for a
sufficiently large $n$,

$P\left(\underset{||\beta-\tilde{\beta}||=\zeta\sqrt{p_{n}/n}}{\sup}(\beta-\tilde{\beta})^{T}U(\beta)<0\right)\geq
1-\epsilon.$

Proof Let $\beta^{*}$ be such that
$\text{max}(\|\beta^{*}-\tilde{\beta}\|,\|\beta^{*}-\beta\|)\leq\|\beta-\tilde{\beta}\|$.
Using Taylor’s expansion we obtain,

$\displaystyle(\beta-\tilde{\beta})^{T}U(\beta)$
$\displaystyle=(\beta-\tilde{\beta})^{T}U(\tilde{\beta})+(\beta-\tilde{\beta})^{T}J(\beta^{*})(\beta-\tilde{\beta})$
$\displaystyle=(\beta-\tilde{\beta})^{T}U(\tilde{\beta})+(\beta-\tilde{\beta})^{T}\\{J(\beta^{*})-J(\tilde{\beta})\\}(\beta-\tilde{\beta})+(\beta-\tilde{\beta})^{T}J(\tilde{\beta})(\beta-\tilde{\beta}).$

From Proposition 5, 6, 7,

$\displaystyle\underset{||\beta-\tilde{\beta}||=\zeta\sqrt{p_{n}/n}}{\sup}(\beta-\tilde{\beta})^{T}U(\beta)\leq
O_{p}(\sqrt{p_{n}})\zeta+\zeta^{2}\dfrac{p_{n}^{1.5}}{\sqrt{n}}-c\zeta^{2}p_{n}+\zeta^{2}o_{p}(p_{n}).$

Using Assumption 7 and with an appropriate choice $\zeta$ the theorem holds.

##  3 Simulations

In this section, we compare performance of the proposed estimator with
alternatives in the presence of measurement errors with various covariance
structures and sample sizes. We first consider a case with Gaussian scalar
response and then a binary one. In the following, $P(k)$ denotes a Poisson
distribution with parameter $k$, $N(a,b)$ denotes normal distribution with
mean $a$ and variance $b$.

True covariate function $X_{i}(t),t\in[0,1],i=1,...,n$ is generated using the
Fourier basis as

$X_{i}(t)=\sum_{k=1}^{p}\varepsilon_{ik}\rho_{k}(t),\,i=1,...,n,\,p=2[n^{(1/5)}],\varepsilon_{ik}\sim
P(2).$

We study the effect of measurement error with two covariance structures. In
Setting $1$, we generate a centered Gaussian Process with covariance function
$K(s,t)=\sigma_{1}\text{exp}\\{-(s-t)^{2}/(2l^{2})\\}$. This is a squared
exponential function where the covariance depends on the distance between the
points. We vary the value of $l$ which controls the range of dependence. In
Setting $2$, we use covariance function of the Brownian Bridge:
$K(s,t)=\sigma_{2}\\{\text{min}(s,t)-st\\}$. The parameters
$\sigma_{1},\,\sigma_{2}$ are introduced to control the level of noise. For
$\sigma_{2}=1$, we obtain the Brownian Bridge. The observed covariate is
$W_{i}(t)=X_{i}(t)+U_{i}(t),\,i=1,...,n$. The effect function
$\tilde{\beta}(\cdot)$ is generated using the Fourier basis as
$\tilde{\beta}(t)=\sum_{k=1}^{p}\beta_{k}\rho_{k}(t),$ where
$\beta_{k}=k^{-1}.$ The Gaussian response is generated as $Y_{i}\sim N(\int
X_{i}(t)\tilde{\beta}(t)dt,1)$. For this case, $\sigma_{1}=5$ in Setting 1.
The binary response is generated from a Binomial distribution with
$P(Y_{i}=1)=\text{exp}\\{\text{max}_{t_{k}}X_{i}(t_{k})\\}/[1+\text{exp}\\{\text{max}_{t_{k}}X_{i}(t_{k})\\}]$.
For this case, we consider $\sigma_{1}=2$ in Setting 1.

To implement our proposed method, we first need to estimate the error
covariance structure. For this, we use 50 replicates of the function
$W_{i}(\cdot),i=1,...,n$ denoted by
$\widetilde{W}_{ij}(\cdot),\,i=1,...,n,\,j=1,...,50$. Let
$\widetilde{W}_{i.}(\cdot)$ denote the mean function of
$\widetilde{W}_{i}(\cdot)$. The estimate of the covariance function is
considered to be

$\hat{K}(s,t)=\dfrac{\sum\limits_{i=1}^{n}\sum\limits_{j=1}^{50}(\widetilde{W}_{ij}(t)-\widetilde{W}_{i.}(t))(\widetilde{W}_{ij}(s)-\widetilde{W}_{i.}(s))}{n(50-1)}.$

We next consider the problem of selecting the number of components $p_{n}$ in
the model. Cross-validation, which is a popular method to determine $p_{n}$ is
biased in the presence of measurement error (Datta and Zou, 2017). Including a
large number of components reduces the loss of information, on the other hand
in the presence of measurement error, adds to the total measurement error in
the model. Moreover, we use the Newton Raphson algorithm to solve the function
$U(\beta_{c})=0$ which involves inverting the derivative of $U(\beta)$. From
the simulations, we observed that selecting $p_{n}\leq p$ helps to avoid
singularity issues. Thus, ideally we should use $p_{n}=p$, however $p$ is
unknown in practice. We choose $p_{n}$ as the threshold beyond which the
proportion of variation explained by the first $p_{n}$ components levels off
as indicated in Figure 1. We observe that, method can lead to an accurate
determination of $p$ when the measurement error is not too large.

Figure 1: Selection of $p_{n}$ via the proportion of variance explained.

We use the Newton Raphson algorithm to obtain the solution to equations (2.7)
and (2.10). Note that these equations have multiple solutions. It is not clear
which solution to choose unlike in maximization or minimization problems. To
address this, we use the naive estimator as the initial value for the Newton
Raphson algorithm. This leads to an accurate estimate if the naive estimate is
close to the true value, in other words if the measurement error is small.
This naive estimate, $\hat{\beta}^{naive}(\cdot)$ is obtained by the
functional regression of $Y$ on $W(\cdot)$ as proposed by Müller and
Stadtmüller (2005). Let
$\hat{\beta}^{naive}_{k}=\int\hat{\beta}^{naive}(t)\hat{\rho}_{k}(t)dt.$ We
denote the basis functions formed by the eigenfunctions of the integral
operator associated with the estimated covariance function
$\hat{K}(\cdot,\cdot)$ as $\hat{\rho}_{k}(\cdot),\,k\geq 1$. Then,
$\hat{\beta}^{naive}=(\hat{\beta}^{naive}_{1},...,\hat{\beta}^{naive}_{p_{n}})$
is used as the initial value. The solution to this algorithm is the corrected
estimate denoted by
$\hat{\beta}^{cor}=(\hat{\beta}^{cor}_{1},...,\hat{\beta}^{cor}_{p_{n}})$. The
corrected estimate $\hat{\beta}^{cor}(\cdot)$ is
$\hat{\beta}^{cor}(\cdot)=\sum\limits_{k=1}^{p_{n}}\hat{\rho}_{k}(\cdot)\hat{\beta}^{cor}_{k}$.
The estimation errors are calculated as
$E_{n}=\int(\tilde{\beta}(t)-\hat{\beta}^{naive}(t))^{2}dt$ and
$E_{co}=\int(\tilde{\beta}(t)-\hat{\beta}^{cor}(t))^{2}dt$.

In the Gaussian case, we perform comparison with the PACE method proposed by
Yao, Müller and Wang (1998) and the calibration based estimate proposed by
Chakraborty and Panaretos (2017). PACE was implemented using MATLAB packages.
Covariance estimation for calibration method was implemented with the help of
the code provided by the authors. The error corresponding to PACE and
calibration estimates are denoted by $E_{p}$ and $E_{ca}$ respectively. Tables
1-3 report average errors based on $200$ repetitions. The values reported in
the column $p_{n}$ denote the averages of the number of components selected.

Table 1: Mean Error for Gaussian Response in Setting 1.

Sample $n=1000$ $n=3000$ $n=6000$ $l$ $0.050$ $0.080$ $0.100$ $0.050$ $0.080$
$0.100$ $0.050$ $0.080$ $0.100$ $p_{n}$ 8 8 8 10 10 10 12 12 12 $E_{n}$ 0.090
0.158 0.207 0.085 0.160 0.202 0.084 0.156 0.202 $E_{co}$ 0.027 0.052 0.071
0.024 0.047 0.060 0.024 0.045 0.056 $E_{p}$ 1.600 8.140 18.41 0.991 6.861
2.333 0.656 1.573 2.956 $E_{ca}$ 0.093 0.170 0.220 0.091 0.171 0.216 0.092
0.167 0.217

Table 2: Mean Error for Gaussian Response in Setting 2.

Sample $n=1000$ $n=3000$ $n=6000$ $\sigma_{2}$ $1.00$ $2.00$ $3.00$ $1.00$
$2.00$ $3.00$ $1.00$ $2.00$ $3.00$ $p_{n}$ 8 8 8 10 10 10 12 12 12 $E_{n}$
0.006 0.013 0.033 0.003 0.010 0.030 0.003 0.009 0.031 $E_{co}$ 0.004 0.004
0.006 0.001 0.002 0.002 0.001 0.001 0.001 $E_{p}$ 37.44 31.95 26.71 0.005
2.477 3.591 0.005 0.012 0.031 $E_{ca}$ 0.009 0.017 0.040 0.006 0.014 0.037
0.005 0.014 0.036

Table 3: Mean Error for Binary Response in Setting 1.

Sample $n=1000$ $n=3000$ $n=6000$ $l$ $0.050$ $0.080$ $0.100$ $0.050$ $0.080$
$0.100$ $0.050$ $0.080$ $0.100$ $E_{n}$ 1.209 1.151 1.110 1.587 1.579 1.573
1.581 1.575 1.572 $E_{co}$ 0.153 0.151 0.167 0.082 0.121 0.120 0.071 0.100
0.110

Table 4: Mean Error for Binary Response in Setting 2.

Sample $n=1000$ $n=3000$ $n=6000$ $\sigma_{2}$ $1.00$ $2.00$ $3.00$ $1.00$
$2.00$ $3.00$ $1.00$ $2.00$ $3.00$ $E_{n}$ 1.314 1.301 1.230 1.587 1.592 1.589
1.587 1.587 1.583 $E_{co}$ 0.184 0.190 0.208 0.072 0.085 0.101 0.040 0.061
0.070

From Tables 1–4, we observe that the error of the proposed corrected estimate,
highlighted in italics, is lower than all the other alternatives. From the
Table 1, we observe that the error of the PACE estimator increases with $l$.
PACE assumes that measurement error occurs at only discrete realizations of
the function. The calibration estimator does allow the error to be functional
in nature but requires the covariance structure to exist on a very small
interval. That is, it imposes a banded structure on the covariance function of
the error process. This explains the deteriorating performance of both these
methods with an increasing value of $l$. The naive estimator ignores
measurement error, and accordingly it’s performance decreases with increasing
$l$. A similar trend observed in the performance of the proposed corrected
estimate, can be explained by the fact that the naive estimate is used as the
initial value in the Newton Raphson algorithm. From Table 2, we observe that
the performance of all the methods deteriorate as the noise ($\sigma_{2}$)
increases. Table 3 and 4 exhibit similar trends.

###  3.1 Canadian weather data

We now perform a simulation study based on the Canadian weather data. This
data consists of daily temperature measurements obtained from 35 Canadian
weather stations for a period of one year. It also contains total annual
rainfall, on a log scale at each of these stations. A sample of three curves
from are displayed in Figure 2b. Denote the log annual precipitation and
curves obtained from smoothing the temperature data by
$(Y_{i},X_{i}(\cdot)),\,i=1,...,35$. Details on the smoothing method can be
found in Ramsay and Silverman (2004). With the development in technologies and
the low variability in the data, it is reasonable to assume that this data is
free from measurement error. That is, the observed data is indeed the true
covariate and error is added to it to obtain the contaminated covariate
$W_{i}(\cdot)=X_{i}(\cdot)+U_{i}(\cdot)$, where $U_{i}(\cdot),\,i=1,...n$ are
indepedent and identically distributed as a centered Gaussian Process with the
squared exponential covariance function
$K(s,t)=5\text{exp}(-(s-t)^{2}/(2*0.5^{2}))$. The slope function
$\tilde{\beta}(\cdot)$ referred to as the true slope, is obtained via the
regression model $Y_{i}=\int X(t)\beta(t)dt+\varepsilon_{i}$. It is assumed
that the temperature has a linear effect on the log precipitation, there are
no other covariates present in the model and that the slope function is indeed
accurate. From Figure 2a, we can see that the corrected estimate is closer to
$\tilde{\beta}(\cdot)$ than the naive one. The errors are
$E_{n}=0.52,\,E_{co}=0.01$. Thus, in the presence of measurement error, the
corrected estimate offers a marked improvement in the estimates.

Figure 2: a: Plots of the true, naive and corrected slope function estimate.
b: Plots of temperature at 3 of the stations.

##  4 Conclusion

We propose a mechanism to account for functional measurement error in
functional regression models. Moreover, it is the first attempt in which the
measurement error is functional and the response is binary. It allows us to
relax several assumptions made on the covariance structure of measurement
error in the presence of replicates. This can also serve as a guideline for
collecting additional data when there is a possibility of data contamination.
Simulations clearly indicate the competitive performance of our method over
several alternatives, especially in the Gaussian case. There is a wide scope
for future work in this area of measurement error in Functional Data Analysis.
Most existing methods including the one proposed here, assume the classical
linear measurement error model and involve parametric regression models.
Models can be investigated that allow for different measurement error models
particularly in the non-parametric regression framework.

##  5 Appendix

###  5.1 Appendix A

######  Proposition 1.

Let
$\tilde{\beta}_{c}=(\tilde{\beta_{0}},...,\tilde{\beta}_{p_{n}})^{{}^{\prime}}$.
Then,

$\|U(\tilde{\beta}_{c})\|=O_{p}(\sqrt{n}).$

###### Proof.

Let $\lambda_{0}=0$. Then,

$\displaystyle U(\tilde{\beta}_{c})$
$\displaystyle=\sum_{i=1}^{n}(\\{Y_{i}-E(Y_{i}|\delta_{ci})\\}(\delta_{ci}-\Omega_{c}\tilde{\beta}_{c})$
$\displaystyle\|U(\tilde{\beta}_{c})\|^{2}$
$\displaystyle=\sum_{l=0}^{p_{n}}\left\\{\sum_{i=1}^{n}(Y_{i}-E(Y_{i}|\delta_{ci}))(\delta_{ci}^{(l)}-\lambda_{l}\tilde{\beta}_{l})\right\\}^{2}$
$\displaystyle=$
$\displaystyle\sum_{l=0}^{p_{n}}\sum_{i=1}^{n}\left[\\{Y_{i}-E(Y_{i}|\delta_{ci})\\}(\delta_{ci}^{(l)}-\lambda_{l}\tilde{\beta}_{l})\right]^{2}$
$\displaystyle+\sum_{l=0}^{p_{n}}\sum_{i_{1}\neq
i_{2}=1}^{n}\\{Y_{i_{1}}-E(Y_{i_{1}}|\delta_{ci_{1}})\\}(\delta_{ci_{1}}^{(l)}-\lambda_{l}\tilde{\beta}_{l})(Y_{i_{2}}-E(Y_{i_{2}}|\delta_{ci_{2}}))(\delta_{ci_{2}}^{(l)}-\lambda_{l}\tilde{\beta}_{l})$
$\displaystyle=A_{1}+A_{2},$ $\displaystyle A_{1}$
$\displaystyle=\sum_{l=0}^{p_{n}}\sum_{i=1}^{n}\left\\{(Y_{i}-E(Y_{i}|\delta_{ci}))(\delta_{ci}^{(l)}-\lambda_{l}\tilde{\beta}_{l})\right\\}^{2}$
$\displaystyle A_{1}$
$\displaystyle=\sum_{i=1}^{n}(Y_{i}-E(Y_{i}|\delta_{ci}))^{2}\sum_{l=0}^{p_{n}}(\delta_{ci}^{(l)}-\lambda_{l}\tilde{\beta}_{l})^{2}$
$\displaystyle A_{1}$
$\displaystyle=\sum_{i=1}^{n}\\{Y_{i}-E(Y_{i}|\delta_{ci})\\}^{2}\|\delta_{ci}-\Omega_{c}\tilde{\beta}_{c}\|^{2}.$

From Assumption 1, 2, 3,

$\displaystyle\|\delta_{ci}-\Omega_{c}\tilde{\beta}_{c}\|^{2}$
$\displaystyle=\|W_{ci}+Y_{i}\Omega_{c}\tilde{\beta}_{c}-\Omega_{c}\tilde{\beta}_{c}\|^{2}$
$\displaystyle\leq\|X_{ci}\|^{2}+\|U_{ic}\|^{2}+\|\Omega_{c}\|^{2}\|\tilde{\beta}_{c}\|^{2}\leq
c+\sum_{l=1}^{p_{n}}U_{il}^{2}.$

Recall that $u_{il}\sim N(0,\lambda_{l})$. Thus,

$\displaystyle E(A_{1})$ $\displaystyle\leq
c\sum_{i=1}^{n}E[\\{Y_{i}-E(Y_{i}|\delta_{ci})\\}^{2}]\sum_{l=1}^{p_{n}}EU_{il}^{2}$
$\displaystyle\leq\sum_{i=1}^{n}E[\\{Y_{i}-E(Y_{i}|\delta_{ci})\\}^{2}]\sum_{l=1}^{p_{n}}\lambda_{l}=O(n).$

We now examine $A_{2}$ which is given as

$\displaystyle\sum_{l=0}^{p_{n}}\sum_{i_{1}\neq
i_{2}=1}^{n}\\{Y_{i_{1}}-E(Y_{i_{1}}|\delta_{ci_{1}})\\}(\delta_{ci_{1}}^{(l)}-\lambda_{l}\beta_{l})(Y_{i_{2}}-E(Y_{i_{2}}|\delta_{ci_{2}}))(\delta_{ci_{2}}^{(l)}-\lambda_{l}\beta_{l})$
$\displaystyle=\sum_{i_{1}\neq
i_{2}=1}^{n}\\{Y_{i_{1}}-E(Y_{i_{1}}|\delta_{ci_{1}})\\}(Y_{i_{2}}-E(Y_{i_{2}}|\delta_{ci_{2}}))\sum_{l=0}^{p_{n}}(\delta_{ci_{1}}^{(l)}-\lambda_{l}\beta_{l})(\delta_{ci_{2}}^{(l)}-\lambda_{l}\beta_{l}).$

Thus, we obtain $E(A_{2})=0$ and $E(\|U(\beta_{c})\|^{2})=O(n)$ implying
$\|U(\beta_{c})\|=O_{p}(\sqrt{n}).$

∎

######  Proposition 2.

Let $J(\beta_{c})=\dfrac{\partial U(\beta_{c})}{\partial\beta_{c}}$ and
$t_{i}(\beta_{c})=W_{ci}^{{}^{\prime}}\beta_{c}+(Y_{i}-0.5)\beta_{c}^{{}^{\prime}}\Omega\beta_{c}$.
Then

$\displaystyle J(\beta_{c})$
$\displaystyle=-\sum_{i=1}^{n}F(t_{i}(\beta_{c}))(Y_{i}-1)\Omega_{c}-\sum_{i=1}^{n}F(t_{i}(\beta_{c}))(1-F(t_{i}(\beta_{c})))(W_{ic}W_{ic}^{{}^{\prime}}+(Y_{i}-1)\Omega_{c}\beta_{c}W_{ic}^{{}^{\prime}})$
$\displaystyle\quad-\sum_{i=1}^{n}F(t_{i}(\beta_{c}))(1-F(t_{i}(\beta_{c})))(2(Y_{i}-0.5)W_{ic}(\Omega_{c}\beta_{c})^{{}^{\prime}}+2(Y_{i}-1)(Y_{i}-0.5)\Omega_{c}\beta_{c}(\Omega_{c}\beta_{c})^{{}^{\prime}})$

###### Proof.

$\displaystyle U(\beta_{c})$
$\displaystyle=\sum_{i=1}^{n}(Y_{i}-F(\delta_{ci}^{{}^{\prime}}\beta_{c}-0.5\beta_{c}^{{}^{\prime}}\Omega_{c}\beta_{c}))\begin{bmatrix}\delta_{ci}-\Omega_{c}\beta_{c}\end{bmatrix}$
$\displaystyle=\sum_{i=1}^{n}(Y_{i}-F(W_{ci}^{{}^{\prime}}\beta_{c}+(Y_{i}-0.5)\beta_{c}^{{}^{\prime}}\Omega_{c}\beta_{c}))\begin{bmatrix}W_{ci}+(Y_{i}-1)\Omega_{c}\beta_{c}\end{bmatrix}$
$\displaystyle=\sum_{i=1}^{n}Y_{i}W_{ci}+Y_{i}(Y_{i}-1)\Omega_{c}\beta_{c}-F(W_{ci}^{{}^{\prime}}\beta_{c}+(Y_{i}-0.5)\beta_{c}^{{}^{\prime}}\Omega_{c}\beta_{c})\begin{bmatrix}W_{ci}+(Y_{i}-1)\Omega_{c}\beta_{c}\end{bmatrix}$

Taking derivatives

$\displaystyle\dfrac{\partial U(\beta_{c})}{\partial\beta_{c}}$
$\displaystyle=-\sum_{i=1}^{n}F(t_{i}(\beta_{c}))(Y_{i}-1)\Omega_{c}-\sum_{i=1}^{n}F(t_{i}(\beta_{c}))(1-F(t_{i}(\beta_{c})))(W_{ic}W_{ic}^{{}^{\prime}}+(Y_{i}-1)\Omega_{c}\beta_{c}W_{ic}^{{}^{\prime}})$
$\displaystyle\quad-\sum_{i=1}^{n}F(t_{i}(\beta_{c}))(1-F(t_{i}(\beta_{c})))(2(Y_{i}-0.5)W_{ic}(\Omega_{c}\beta_{c})^{{}^{\prime}}+2(Y_{i}-1)(Y_{i}-0.5)\Omega_{c}\beta_{c}(\Omega_{c}\beta_{c})^{{}^{\prime}})$

∎

######  Proposition 3.

$\underset{\|\beta_{c}-\tilde{\beta}_{c}\|=\zeta\sqrt{p_{n}/n}}{\sup}\underset{\|b\|=1}{\sup}|b^{{}^{\prime}}(J(\beta_{c})-J(\tilde{\beta}_{c}))b|\leq\sqrt{n}p_{n}$

###### Proof.

Consider,

$\displaystyle|b^{{}^{\prime}}\sum_{i=1}^{n}(F(t_{i}(\beta_{c}))-F(t_{i}(\tilde{\beta}_{c})))(y_{i}-1)\Omega_{c}b|\leq\sum_{i=1}^{n}|(F(t_{i}(\beta_{c}))-F(t_{i}(\tilde{\beta}_{c})))||b^{{}^{\prime}}\Omega_{c}b|$

Using the fact that $F(\cdot)$ is a bounded continuous function with bounded
derivatives along with Taylors integral remainder theorem we obtain

$|F(t_{i}(\beta_{c}))-F(t_{i}(\tilde{\beta}_{c}))|<c|t_{i}(\beta_{c})-t_{i}(\tilde{\beta}_{c})|\leq\|W_{ci}\|\|\beta_{c}-\tilde{\beta}_{c}\|+0.5\|\beta_{c}^{{}^{\prime}}\Omega_{c}\beta_{c}^{{}^{\prime}}-\tilde{\beta}_{c}^{{}^{\prime}}\Omega_{c}\tilde{\beta}_{c}^{{}^{\prime}}\|.$

Thus from Assumptions 1, 2 and 3,

$\displaystyle|b^{{}^{\prime}}\sum_{i=1}^{n}(F(t_{i}(\beta_{c}))-F(t_{i}(\tilde{\beta}_{c})))(y_{i}-1)\Omega_{c}b|$
$\displaystyle\leq\sum_{i=1}^{n}\left(\|W_{ci}\|\|\beta_{c}-\tilde{\beta}_{c}\|+0.5\|\beta_{c}^{{}^{\prime}}\Omega_{c}\beta_{c}^{{}^{\prime}}-\tilde{\beta}_{c}^{{}^{\prime}}\Omega_{c}\tilde{\beta}_{c}^{{}^{\prime}}\|\right)|b^{{}^{\prime}}\Omega_{c}b|$
$\displaystyle\leq|b^{{}^{\prime}}\Omega_{c}b|\|\beta_{c}-\tilde{\beta}_{c}\|\sum_{i=1}^{n}\left(\|W_{ci}\|+c\|\Omega\|\|\tilde{\beta}_{c}\|\right).$

Taking supremum we obtain,

$\displaystyle\underset{\|\beta_{c}-\tilde{\beta}_{c}\|=\zeta\sqrt{p_{n}/n}}{\sup}\underset{\|b\|=1}{\sup}b^{{}^{\prime}}|\sum_{i=1}^{n}(F(t_{i}(\beta_{c}))-F(t_{i}(\tilde{\beta}_{c})))(y_{i}-1)\Omega_{c}b|\leq
O_{p}(\sqrt{n}p_{n}).$ (5.1)

Consider,

$|\sum_{i=1}^{n}(F(t_{i}(\beta_{c}))(1-F(t_{i}(\beta_{c})))-F(t_{i}(\tilde{\beta}_{c}))(1-F(t_{i}(\tilde{\beta}_{c}))))b^{{}^{\prime}}W_{ic}W_{ic}^{{}^{\prime}}b|$

By using the boundedness of $F(t_{i}(\beta_{c}))(1-F(t_{i}(\beta_{c})))$ we
obtain

$\displaystyle|\sum_{i=1}^{n}\left(F(t_{i}(\beta_{c}))(1-F(t_{i}(\beta_{c})))-F(t_{i}(\tilde{\beta}_{c}))(1-F(t_{i}(\tilde{\beta}_{c})))\right)b^{{}^{\prime}}W_{ic}W_{ic}^{{}^{\prime}}b|^{2}$
$\displaystyle=n^{2}\left(\sum_{i=1}^{n}\dfrac{F(t_{i}(\beta_{c}))(1-F(t_{i}(\beta_{c})))-F(t_{i}(\tilde{\beta}_{c}))(1-F(t_{i}(\tilde{\beta}_{c})))}{\sqrt{n}}\dfrac{b^{{}^{\prime}}X_{ic}X_{ic}^{{}^{\prime}}b+b^{{}^{\prime}}U_{ic}U_{ic}^{{}^{\prime}}b}{\sqrt{n}}\right)^{2}$
$\displaystyle\leq
4cn^{2}\sum_{i=1}^{n}\dfrac{(\|W_{ci}\|^{2}\|\beta_{c}-\tilde{\beta}_{c}\|^{2}+0.5\|\beta_{c}^{{}^{\prime}}\Omega_{c}\beta_{c}^{{}^{\prime}}-\tilde{\beta}_{c}^{{}^{\prime}}\Omega_{c}\tilde{\beta}_{c}^{{}^{\prime}}\|^{2})}{n}\sum_{i=1}^{n}\dfrac{(b^{{}^{\prime}}X_{ic}X_{ic}^{{}^{\prime}}b)^{2}+(b^{{}^{\prime}}U_{ic}U_{ic}^{{}^{\prime}}b)^{2}}{n}$
$\displaystyle\leq
4cn^{2}\sum_{i=1}^{n}\dfrac{(\|W_{ci}\|^{2}\|\beta_{c}-\tilde{\beta}_{c}\|^{2}+0.5\|\beta_{c}^{{}^{\prime}}\Omega_{c}\beta_{c}^{{}^{\prime}}-\tilde{\beta}_{c}^{{}^{\prime}}\Omega_{c}\tilde{\beta}_{c}^{{}^{\prime}}\|^{2})}{n}\left(\sum_{i=1}^{n}\dfrac{(b^{{}^{\prime}}X_{ic}X_{ic}^{{}^{\prime}}b)^{2}}{n}+E(b^{{}^{\prime}}U_{1c}U_{1c}^{{}^{\prime}}b)^{2}\right)$
$\displaystyle+\text{smaller order terms}$

Using Assumption 3,

$\displaystyle
E(b^{{}^{\prime}}u_{1c}u_{1c}^{{}^{\prime}}b)^{2}=E(\|b^{{}^{\prime}}U_{1c}\|^{4})\leq\|b\|^{4}E(\|U_{1c}\|^{4})$
$\displaystyle
E(\|U_{1c}\|^{4})=E(\sum_{l=1}^{p_{n}}U_{1cl}^{2})^{2}=\sum_{l=1}^{p_{n}}EU_{1cl}^{4}+\sum_{l=1}^{p_{n}}\lambda_{l}\sum_{l=1}^{p_{n}}\lambda_{l}=O(p_{n})$

Thus,

$\displaystyle|\sum_{i=1}^{n}\left(F(t_{i}(\beta_{c}))(1-F(t_{i}(\beta_{c})))-F(t_{i}(\tilde{\beta}_{c}))(1-F(t_{i}(\tilde{\beta}_{c})))\right)b^{{}^{\prime}}W_{ic}W_{ic}^{{}^{\prime}}b|^{2}$
$\displaystyle\leq\|\beta_{c}-\tilde{\beta}_{c}\|^{2}n^{2}O_{p}(p_{n})$

Taking supremum,

$\displaystyle\underset{\|\beta_{c}-\tilde{\beta}_{c}\|=\zeta\sqrt{p_{n}/n}}{\sup}\underset{\|b\|=1}{\sup}b^{{}^{\prime}}|\sum_{i=1}^{n}\left(F(t_{i}(\beta_{c}))(1-F(t_{i}(\beta_{c})))-F(t_{i}(\tilde{\beta}_{c}))(1-F(t_{i}(\tilde{\beta}_{c})))\right)b^{{}^{\prime}}W_{ic}W_{ic}^{{}^{\prime}}b|$
$\displaystyle\leq\zeta
O_{p}(\dfrac{\sqrt{p_{n}}}{\sqrt{n}}n\sqrt{p_{n}})=O_{p}(\sqrt{n}p_{n}).$

Similarly we can show that the other involved terms are of order
$\sqrt{n}p_{n}$ thus yielding the result. ∎

######  Proposition 4.

$\underset{\|\beta_{c}-\tilde{\beta}_{c}\|=\zeta\sqrt{p_{n}/n}}{sup}(\beta_{c}-\tilde{\beta}_{c})^{{}^{\prime}}J(\tilde{\beta}_{c})(\beta_{c}-\tilde{\beta}_{c})<-c\zeta^{2}p_{n}+\zeta^{2}o(p_{n})$

###### Proof.

Let $a_{c}=\beta_{c}-\tilde{\beta}_{c},\,a=\beta-\tilde{\beta}$

$\displaystyle a_{c}^{{}^{\prime}}J(\tilde{\beta}_{c})a$
$\displaystyle=a_{c}^{{}^{\prime}}\sum_{i=1}^{n}F(t_{i}(\tilde{\beta}_{c}))(1-Y_{i})\Omega_{c}a_{c}$
$\displaystyle-
a_{c}^{{}^{\prime}}\sum_{i=1}^{n}F(t_{i}(\tilde{\beta}_{c}))(1-F(t_{i})(\tilde{\beta}_{c}))W_{ic}W_{ic}^{{}^{\prime}})a_{c}$
$\displaystyle-
a_{c}^{{}^{\prime}}\sum_{i=1}^{n}F(t_{i}(\tilde{\beta}_{c}))(1-F(t_{i}(\tilde{\beta}_{c})))(Y_{i}-1)\Omega_{c}\beta_{c}W_{ic}^{{}^{\prime}}a_{c}$
$\displaystyle-
a_{c}^{{}^{\prime}}\sum_{i=1}^{n}F(t_{i}(\tilde{\beta}_{c}))(1-F(t_{i}(\tilde{\beta}_{c})))(Y_{i}-0.5)2W_{ic}(\Omega_{c}\beta_{c})^{{}^{\prime}}a_{c}$
$\displaystyle-
a_{c}\sum_{i=1}^{n}F(t_{i}(\tilde{\beta}_{c}))(1-F(t_{i}(\tilde{\beta}_{c})))2(Y_{i}-1)(Y_{i}-0.5)\Omega_{c}\beta_{c}(\Omega_{c}\beta_{c})^{{}^{\prime}}a_{c}$
$\displaystyle=A_{1}+A_{2}+A_{3}+A_{4}+A_{5}$

We study the orders of each of these terms. Consider,

$\displaystyle A_{3}$
$\displaystyle=\sum_{i=1}^{n}F(t_{i}(\tilde{\beta}_{c}))(1-F(t_{i}(\tilde{\beta}_{c})))(1-Y_{i})a_{c}^{{}^{\prime}}\Omega_{c}\beta_{c}W_{ic}^{{}^{\prime}}a_{c}$
$\displaystyle\leq\underset{i}{sup}F(t_{i}(\tilde{\beta}_{c}))(1-F(t_{i}(\tilde{\beta}_{c})))a^{{}^{\prime}}\Omega\tilde{\beta}n\sum_{i=1}^{n}\dfrac{X_{i}^{{}^{\prime}}+U_{i}^{{}^{\prime}}}{n}a$
$\displaystyle=\zeta^{2}o_{p}(p_{n})$ (5.2)

In a similar way we can show that $A_{4}=\zeta^{2}o_{p}(p_{n})$. Consider,

$\displaystyle
A_{1}=a_{c}^{{}^{\prime}}\sum_{i=1}^{n}F(t_{i}(\tilde{\beta}_{c}))(1-Y_{i})\Omega_{c}a_{c}$
$\displaystyle\leq\underset{1\leq i\leq
n}{sup}F(t_{i}(\tilde{\beta}_{c}))a_{c}^{{}^{\prime}}\Omega_{c}a_{c}n\sum_{i=1}^{n}(1-Y_{i})/n$
$\displaystyle\leq\underset{1\leq i\leq
n}{sup}F(t_{i}(\tilde{\beta}_{c}))a_{c}^{{}^{\prime}}\Omega_{c}a_{c}n\sum_{i=1}^{n}(1-F(X_{ic}^{{}^{\prime}}\tilde{\beta}_{c}))/n$
$\displaystyle\quad+\text{smaller order term}$
$\displaystyle\leq\underset{1\leq i\leq
n}{sup}F(t_{i}(\tilde{\beta}_{c}))(1-\underset{1\leq i\leq
n}{inf}F(X_{ic}^{{}^{\prime}}\tilde{\beta}_{c}))a_{c}^{{}^{\prime}}\Omega_{c}a_{c}n$

Note that the function $F(t)=1/(1+e^{-t})$ is a non-decreasing function and
$\Omega_{c}$ is positive semidefinite. Using

$\underset{1\leq i\leq n}{sup}t_{i}(\tilde{\beta}_{c})=\underset{1\leq i\leq
n}{sup}W_{ci}^{{}^{\prime}}\tilde{\beta}_{c}+(Y_{i}-0.5)\tilde{\beta}_{c}^{{}^{\prime}}\Omega\tilde{\beta}_{c}\leq\underset{1\leq
i\leq
n}{sup}W_{ci}^{{}^{\prime}}\tilde{\beta}_{c}+0.5\tilde{\beta}_{c}^{{}^{\prime}}\Omega\tilde{\beta}_{c}=t_{1},$
$\underset{1\leq i\leq n}{inf}t_{i}(\tilde{\beta}_{c})=\underset{1\leq i\leq
n}{inf}W_{ci}^{{}^{\prime}}\tilde{\beta}_{c}+(Y_{i}-0.5)\tilde{\beta}_{c}^{{}^{\prime}}\Omega\tilde{\beta}_{c}\geq\underset{1\leq
i\leq
n}{inf}W_{ci}^{{}^{\prime}}\tilde{\beta}_{c}-0.5\tilde{\beta}_{c}^{{}^{\prime}}\Omega\tilde{\beta}_{c}=t_{2},$

we obtain, $\underset{1\leq i\leq n}{sup}F(t_{i}(\tilde{\beta}_{c}))\leq
F\left(\underset{1\leq i\leq
n}{sup}t_{i}(\tilde{\beta}_{c})\right)\leq\dfrac{e^{t_{1}}}{1+e^{t_{1}}}$ and
$\dfrac{e^{t_{2}}}{1+e^{t_{2}}}=F(t_{2})\leq\underset{1\leq i\leq
n}{inf}F(t_{i}(\tilde{\beta}_{c})).$

Let $t_{3}=\underset{1\leq i\leq
n}{inf}X_{ic}^{{}^{\prime}}\tilde{\beta}_{c},\,t_{4}=\underset{1\leq i\leq
n}{sup}X_{ic}^{{}^{\prime}}\tilde{\beta}_{c}$.

$\displaystyle
A_{1}=a_{c}^{{}^{\prime}}\sum_{i=1}^{n}F(t_{i}(\tilde{\beta}_{c}))(1-Y_{i})\Omega_{c}a_{c}$
$\displaystyle\leq\dfrac{e^{t_{1}}}{(1+e^{t_{1}})(1+e^{t_{3}})}\lambda_{max}(\Omega)\zeta^{2}p_{n}$

$\displaystyle A_{4}$
$\displaystyle=-a_{c}\sum_{i=1}^{n}F(t_{i}(\tilde{\beta}_{c}))(1-F(t_{i}(\tilde{\beta}_{c})))2(Y_{i}-1)(Y_{i}-0.5)\Omega_{c}\beta_{c}(\Omega_{c}\beta_{c})^{{}^{\prime}}a_{c}$
$\displaystyle<0$ (5.3)

$\displaystyle A_{2}$
$\displaystyle=-a_{c}^{{}^{\prime}}\sum_{i=1}^{n}F(t_{i}(\tilde{\beta}_{c}))(1-F(t_{i})(\tilde{\beta}_{c}))(W_{ic}W_{ic}^{{}^{\prime}})a_{c}$
$\displaystyle\leq-\dfrac{e^{t_{2}}}{(1+e^{t_{2}})(1+e^{t_{1}})}na^{{}^{\prime}}\lambda_{min}\left(\sum_{i=1}^{n}\dfrac{W_{ic}W_{ic}^{{}^{\prime}}}{n}\right)a$
$\displaystyle\quad+\text{smaller order term}$ $\displaystyle A_{1}+A_{2}$
$\displaystyle\leq\dfrac{e^{t_{1}}}{(1+e^{t_{1}})(1+e^{t_{3}})}\lambda_{max}(\Omega)\zeta^{2}p_{n}$
$\displaystyle-\dfrac{e^{t_{2}}}{(1+e^{t_{2}})(1+e^{t_{1}})}\lambda_{min}\left(\sum_{i=1}^{n}\dfrac{W_{ic}W_{ic}^{{}^{\prime}}}{n}\right)\zeta^{2}p_{n}$
$\displaystyle=\dfrac{\zeta^{2}p_{n}}{1+e^{t_{1}}}\left(\dfrac{e^{t_{1}}}{1+e^{t_{3}}}\lambda_{max}(\Omega)-\dfrac{e^{t_{2}}}{1+e^{t_{2}}}\lambda_{min}\left(\sum_{i=1}^{n}\dfrac{W_{ic}W_{ic}^{{}^{\prime}}}{n}\right)\right)$

Let
$\lambda_{min}\left(\sum_{i=1}^{n}\dfrac{W_{ic}W_{ic}^{{}^{\prime}}}{n}\right)=B$.
From Assumption 5

$\displaystyle\lambda_{max}(\Omega)\text{exp}(\lambda_{max}(\Omega)m)$
$\displaystyle\leq
B\dfrac{\text{exp}(infW_{ic}^{{}^{\prime}}\tilde{\beta}-supW_{ic}^{{}^{\prime}}\tilde{\beta})}{1+\text{exp}(infW_{ic}^{{}^{\prime}}\tilde{\beta})}$
$\displaystyle\lambda_{max}(\Omega)$ $\displaystyle\leq
B\dfrac{\text{exp}(infW_{ic}^{{}^{\prime}}\tilde{\beta}-supw_{ic}^{{}^{\prime}}\tilde{\beta}-\lambda_{max}(\Omega)\|\tilde{\beta}\|^{2})}{1+\text{exp}(infW_{ic}^{{}^{\prime}}\tilde{\beta})}$
$\displaystyle\leq
B\dfrac{\text{exp}(infW_{ic}^{{}^{\prime}}\tilde{\beta}-supW_{ic}^{{}^{\prime}}\tilde{\beta}-\lambda_{max}(\Omega)\|\tilde{\beta}\|^{2})}{1+\text{exp}(infW_{ic}^{{}^{\prime}}\tilde{\beta}-0.5\lambda_{min}(\Omega)\|\tilde{\beta}\|^{2})}$
$\displaystyle\leq
B\dfrac{\text{exp}(infW_{ic}^{{}^{\prime}}\tilde{\beta}-supW_{ic}^{{}^{\prime}}\tilde{\beta}-\tilde{\beta}^{{}^{\prime}}\Omega\tilde{\beta})}{1+\text{exp}(infW_{ic}^{{}^{\prime}}\tilde{\beta}-0.5\tilde{\beta}^{{}^{\prime}}\Omega\tilde{\beta})}$
$\displaystyle\leq
B\dfrac{\text{exp}(infW_{ic}^{{}^{\prime}}\tilde{\beta}-0.5\tilde{\beta}^{{}^{\prime}}\Omega\tilde{\beta})}{1+\text{exp}(infW_{ic}^{{}^{\prime}}\tilde{\beta}-0.5\tilde{\beta}^{{}^{\prime}}\Omega\tilde{\beta})}\dfrac{1+\text{exp}(infX_{ic}^{{}^{\prime}}\tilde{\beta})}{\text{exp}(supW_{ic}^{{}^{\prime}}\tilde{\beta}+0.5\tilde{\beta}^{{}^{\prime}}\Omega\tilde{\beta})}$
$\displaystyle\lambda_{max}(\Omega)\dfrac{\text{exp}(supW_{ic}^{{}^{\prime}}\tilde{\beta}+0.5\tilde{\beta}^{{}^{\prime}}\Omega\tilde{\beta})}{1+\text{exp}(infW_{ic}^{{}^{\prime}}\tilde{\beta})}$
$\displaystyle\leq
B\dfrac{\text{exp}(infW_{ic}^{{}^{\prime}}\tilde{\beta}-0.5\tilde{\beta}^{{}^{\prime}}\Omega\tilde{\beta})}{1+\text{exp}(infW_{ic}^{{}^{\prime}}\tilde{\beta}-0.5\tilde{\beta}^{{}^{\prime}}\Omega\tilde{\beta})}$

Thus, $A_{1}+A_{2}\leq-c\zeta^{2}p_{n}$ and along with (5.2),(5.3) we get the
result.

∎

###  5.2 Appendix B

######  Proposition 5.

$\|U(\tilde{\beta})\|=O_{p}(\sqrt{n}).$

###### Proof.

$\displaystyle\|U(\tilde{\beta})\|$
$\displaystyle\leq\|\sum\limits_{i=1}^{n}\mathbf{W}_{i}^{c}Y_{i}^{c}\tilde{\beta}^{{}^{\prime}}\Omega\tilde{\beta}\|+\|\sum\limits_{i=1}^{n}Y^{c2}_{i}\Omega\tilde{\beta}\|+\|\sum\limits_{i=1}^{n}\mathbf{W}_{i}^{c}\mathbf{W}_{i}^{c^{\prime}}\tilde{\beta}\|+\|\sum\limits_{i=1}^{n}\mathbf{W}_{i}^{c}Y_{i}^{c}\|$
$\displaystyle=A_{1}+A_{2}+A_{3}+A_{4}$

We now examine each of these terms. Using Assumption 1, 3,

$\displaystyle
A_{1}^{2}=\|\sum\limits_{i=1}^{n}\mathbf{W}_{i}^{c}Y_{i}^{c}\tilde{\beta}^{{}^{\prime}}\Omega\tilde{\beta}\|^{2}\leq
c\left(\sum\limits_{i=1}^{n}\|\mathbf{W}_{i}^{c}\|\|Y_{i}^{c}\|\right)^{2}\leq
c\sum\limits_{i=1}^{n}\|\mathbf{W}_{i}^{c}\|^{2}\sum\limits_{i=1}^{n}\|\mathbf{Y}_{i}^{c}\|^{2}$
$\displaystyle\sum\limits_{i=1}^{n}E\|\mathbf{W}_{i}^{c}\|^{2}=\sum\limits_{i=1}^{n}E\|\mathbf{W}_{i}-\overline{\mathbf{W}}\|^{2}\leq\sum\limits_{i=1}^{n}E\|\mathbf{W}_{i}\|^{2}+2n\|\overline{\mathbf{X}}\|^{2}+2n\|\overline{\mathbf{U}}\|^{2}.$

Consider,

$\displaystyle\sum\limits_{i=1}^{n}E\|\mathbf{W}_{i}\|^{2}\leq\sum\limits_{i=1}^{n}\|\mathbf{X}_{i}\|^{2}+E\|\mathbf{U}_{i}\|^{2}\leq
cn+\sum\limits_{i=1}^{n}\sum\limits_{l=1}^{p_{n}}E(u_{il}^{2})=cn+\sum\limits_{i=1}^{n}\sum\limits_{l=1}^{p_{n}}\lambda_{l}=O(n)$
$\displaystyle
nE\|\overline{\mathbf{U}}\|^{2}=nE\|\sum\limits_{i=1}^{n}\dfrac{\mathbf{U}_{i}}{n}\|^{2}=n^{-1}E\left(\sum\limits_{l=1}^{p_{n}}\left(\sum\limits_{i=1}^{n}u_{il}\right)^{2}\right)=n^{-1}\sum\limits_{l=1}^{p_{n}}\sum\limits_{i=1}^{n}Eu_{il}^{2}+0=O(1).$

Using the fact that $\|\overline{\mathbf{X}}\|=O(1)$ and
$2\sum\limits_{i=1}^{n}\|\mathbf{Y}_{i}^{c}\|^{2}=O_{p}(n)$ we get
$A_{1}=O_{p}(\sqrt{n})$. Similarly we can prove that the other terms are of
the same order thus proving the result. ∎

Let $J(\beta)=\left(\dfrac{\partial U(\beta)}{\partial\beta}\right)$.

######  Proposition 6.

$\underset{\|\beta-\tilde{\beta}\|=\zeta\sqrt{p_{n}/n}}{\sup}\underset{\|b\|=1}{\sup}|b^{{}^{\prime}}(J(\beta)-J(\tilde{\beta})b|\leq
O_{p}(\sqrt{np_{n}})$

###### Proof.

We can easily show that

$J(\beta)=-\sum\limits_{i=1}^{n}\Omega(\mathbf{W^{c}}_{i}^{\prime}\beta)Y_{i}^{c}-\Omega\beta\mathbf{W}_{i}^{c^{\prime}}Y_{i}^{c}+\Omega
Y_{i}^{c2}-\mathbf{W}_{i}^{c}\mathbf{W}_{i}^{c^{\prime}}.$

Then,

$\displaystyle b^{{}^{\prime}}(J(\beta)-J(\tilde{\beta}))b$
$\displaystyle=b^{{}^{\prime}}\left(\sum\limits_{i=1}^{n}-2\Omega(\mathbf{W}_{i}^{c^{\prime}}\beta)Y_{i}^{c}+\Omega
Y_{i}^{c2}-\mathbf{W}_{i}^{c}\mathbf{W}_{i}^{c^{\prime}}\right)b$
$\displaystyle\quad-b^{{}^{\prime}}\left(\sum\limits_{i=1}^{n}-2\Omega(\mathbf{W}_{i}^{c^{\prime}}\tilde{\beta})Y_{i}^{c}+\Omega
Y_{i}^{c2}-\mathbf{W}_{i}^{c}\mathbf{W}_{i}^{c^{\prime}}\right)b$
$\displaystyle=b^{{}^{\prime}}\left(\sum\limits_{i=1}^{n}2\Omega(\mathbf{W}_{i}^{c^{\prime}}\tilde{\beta})Y_{i}^{c}-2\Omega(\mathbf{W}_{i}^{c^{\prime}}\beta)Y_{i}^{c}\right)b$
$\displaystyle=b^{{}^{\prime}}\left(\sum\limits_{i=1}^{n}2Y_{i}^{c}\Omega\mathbf{W}_{i}^{c^{\prime}}(\tilde{\beta}-\beta)\right)b$
$\displaystyle|b^{{}^{\prime}}(J(\beta)-J(\tilde{\beta}))b|$
$\displaystyle\leq\sum\limits_{i=1}^{n}|b^{{}^{\prime}}2Y_{i}^{c}\Omega\mathbf{W}_{i}^{c^{\prime}}(\tilde{\beta}-\beta)b|$
$\displaystyle\leq
2\|\Omega\|\|\tilde{\beta}-\beta\|\|b\|^{2}\sum\limits_{i=1}^{n}\|Y_{i}^{c}\|\|\mathbf{W}_{i}^{c^{\prime}}\|.$

Taking supremum

$\displaystyle\underset{\|\beta-\tilde{\beta}\|=\zeta\sqrt{p_{n}/n}}{\sup}\underset{\|b\|=1}{\sup}|b^{{}^{\prime}}(J(\beta)-J(\tilde{\beta}))b|\leq
O_{p}(\sqrt{np_{n}}).$

∎

######  Proposition 7.

$\underset{\|\beta-\tilde{\beta}\|=\zeta\sqrt{p_{n}/n}}{sup}(\beta-\tilde{\beta})^{{}^{\prime}}J(\tilde{\beta})(\beta-\tilde{\beta})<-c\zeta^{2}p_{n}+\zeta^{2}o_{p}(p_{n})$

###### Proof.

Let $b=\beta-\tilde{\beta}$. Consider,

$\displaystyle
J(\tilde{\beta})=-2\sum\limits_{i=1}^{n}\Omega(\mathbf{W^{c}}_{i}^{\prime}\tilde{\beta})Y_{i}^{c}+\Omega
Y_{i}^{c2}-\mathbf{W}_{i}^{c}\mathbf{W}_{i}^{c^{\prime}}$

We evaluate each term.

$\displaystyle-2\sum\limits_{i=1}^{n}\Omega(\mathbf{W^{c}}_{i}^{\prime}\tilde{\beta})Y_{i}^{c}$
$\displaystyle=-2\Omega\tilde{\beta}^{{}^{\prime}}\sum\limits_{i=1}^{n}Y_{i}^{c}(\mathbf{X}_{i}^{c}+\mathbf{U}_{i}^{c})$
$\displaystyle=-2\Omega\tilde{\beta}^{{}^{\prime}}n\sum\limits_{i=1}^{n}\dfrac{Y_{i}^{c}\mathbf{X}_{i}^{c}}{n}-2\Omega\tilde{\beta}^{{}^{\prime}}n\sum\limits_{i=1}^{n}\dfrac{Y_{i}^{c}\mathbf{U}_{i}^{c}}{n}$
$\displaystyle=-2\Omega\tilde{\beta}^{{}^{\prime}}n\sum\limits_{i=1}^{n}\dfrac{(\mathbf{X}_{i}^{c^{\prime}}\tilde{\beta})^{2}}{n}-2\Omega\tilde{\beta}^{{}^{\prime}}nE(Y_{1}^{c}\mathbf{U}_{1}^{c})+\text{smaller
order terms}$
$\displaystyle=-2\Omega\tilde{\beta}^{{}^{\prime}}n\sum\limits_{i=1}^{n}\dfrac{\tilde{\beta}^{{}^{\prime}}\mathbf{X}_{i}^{c}\mathbf{X}_{i}^{c^{\prime}}\tilde{\beta}}{n}-2\Omega
co_{p}(n)$
$\displaystyle-b^{{}^{\prime}}2\sum\limits_{i=1}^{n}\Omega(\mathbf{W^{c}}_{i}^{\prime}\tilde{\beta})Y_{i}^{c}b$
$\displaystyle\leq-c\zeta^{2}p_{n}-\zeta^{2}o_{p}(p_{n})$

Next consider,

$\displaystyle b^{{}^{\prime}}\sum\limits_{i=1}^{n}\Omega Y_{i}^{c2}b$
$\displaystyle=b^{{}^{\prime}}\Omega
b\sum\limits_{i=1}^{n}Y_{i}^{c2}=b^{{}^{\prime}}\Omega
bn\text{var}(Y_{1})+\text{smaller order
terms}\leq\zeta^{2}p_{n}\lambda_{max}(\Omega_{1}).$

Finally consider,

$\displaystyle-n\sum\limits_{i=1}^{n}\dfrac{\mathbf{W}_{i}^{c}\mathbf{W}_{i}^{c^{\prime}}}{n}$
$\displaystyle=-n\sum\limits_{i=1}^{n}\dfrac{(\mathbf{X}_{i}^{c}+\mathbf{U}_{i}^{c})(\mathbf{X}_{i}^{c^{\prime}}+\mathbf{U}_{i}^{c^{\prime}})}{n}$
$\displaystyle=-n\sum\limits_{i=1}^{n}\dfrac{\mathbf{X}_{i}^{c}\mathbf{X}_{i}^{c^{\prime}}}{n}-n\dfrac{\mathbf{X}_{i}^{c}\mathbf{U}_{i}^{c^{\prime}}}{n}-n\dfrac{\mathbf{U}_{i}^{c}\mathbf{X}_{i}^{c^{\prime}}}{n}-n\dfrac{\mathbf{U}_{i}^{c}\mathbf{U}_{i}^{c^{\prime}}}{n}$
$\displaystyle=-n\sum\limits_{i=1}^{n}\dfrac{\mathbf{X}_{i}^{c}\mathbf{X}_{i}^{c^{\prime}}}{n}-\Omega_{1}+\text{smaller
order terms}$ $\displaystyle-
nb^{{}^{\prime}}\sum\limits_{i=1}^{n}\dfrac{\mathbf{W}_{i}^{c}\mathbf{W}_{i}^{c^{\prime}}}{n}b$
$\displaystyle=-nb^{{}^{\prime}}\sum\limits_{i=1}^{n}\dfrac{\mathbf{X}_{i}^{c}\mathbf{X}_{i}^{c^{\prime}}}{n}b-b^{{}^{\prime}}\Omega_{1}b+\text{smaller
order terms}$
$\displaystyle\leq-\zeta^{2}p_{n}\lambda_{min}\left(\sum\limits_{i=1}^{n}\dfrac{\mathbf{X}_{i}^{c}\mathbf{X}_{i}^{c^{\prime}}}{n}\right)-\zeta^{2}p_{n}\lambda_{min}(\Omega_{1}).$

From Assumption 6, we obtain he desired result. ∎

## References

  * Carroll et al. (2004) Carroll, Raymond, J., Ruppert, David, Crainiceanu, Ciprian, M., Stefanski, Leonard, A. (2006).  Measurement Error in Nonlinear Models: A Modern Perspective. 2nd Edition.  Chapman and Hall/CRC.
  * Carroll and Raymond (1998) Carroll and Raymond, J. (1998).  Measurement error in epidemiologic studies .  Encyclopedia of biostatistics.
  * Yao, Müller and Wang (1998) Yao, F., Müller, H. and Wang, J. (2005).  Functional data analysis for sparse longitudinal data.  Journal of the American Statistical Association., 100, 577–590.
  * Cardot et al. (2007) Cardot, H., Crambes, C., Kneip, A. and Sarda, P. (2007).  Smoothing splines estimators in functional linear regression with errors-in-variables.  Computational statistics & data analysis., 51, 4832–4848.
  * James (2002) James G. (2002).  Generalized linear models with functional predictors.  Journal of the Royal Statistical Society: Series B (Statistical Methodology)., 64, 411–432.
  * Crambes, Kneip and Sarda (2009) Crambes, C., Kneip, A. and Sarda, P. (2009).  Smoothing splines estimators for functional linear regression.  The Annals of Statistics., 37, 35–72.
  * Goldsmith et al. (2011) Goldsmith, J., Bobb, J., Crainiceanu, C. M., Caffo, B. and Reich, D. (2011).  Penalized functional regression.  Journal of Computational and Graphical Statistics., 20, 830–851.
  * Goldsmith, Wand and Crainiceanu (2011) Goldsmith, J., Wand, M. P. and Crainiceanu, C. (2011).  Functional regression via variational Bayes.  Electronic journal of statistics., 5, 572.
  * Cai (2015) Cai, X. (2015).  Methods for handling measurement error and sources of variation in functional data models.
  * Chakraborty and Panaretos (2017) Chakraborty, A. and Panaretos, V.M. (2017).  Regression with genuinely functional errors-in-covariates.  arXiv preprint arXiv:1712.04290..
  * Descary and Panaretos (2016) Descary, M.H. and Panaretos, V. M. (2016).  Functional data analysis by matrix completion.  arXiv preprint arXiv:1609.00834..
  * Stefanski and Carroll (1987) Stefanski, L. A. and Carroll, R. J. (1987).  Conditional scores and optimal scores for generalized linear measurement-error models.  Biometrika., 74, 703–716.
  * Ortega and Rheinboldt (1970) Ortega, J. M. and Rheinboldt, W. C. (1970).  Iterative solution of nonlinear equations in several variables.  Academic Press, San Diego.
  * Datta and Zou (2017) Datta, A. and Zou, H. (2017).  Cocolasso for high-dimensional error-in-variables regression.  The Annals of Statistics., 45, 2400–2426.
  * Müller and Stadtmüller (2005) Müller, H. G. and Stadtmüller, U. (2005).  Generalized functional linear models.  The Annals of Statistics., 3, 774–805.
  * Ramsay and Silverman (2004) Ramsay, J. O. and Silverman B. W. (2005).  Functional Data Analysis.  Springer-Verlag, New York.

Yale University E-mail<EMAIL_ADDRESS>
Yale University E-mail<EMAIL_ADDRESS>