# Easy Maximum Empirical Likelihood Estimation of Linear Functionals Of A
Probability Measure With Infinitely Many Constraints

Shan Wang<EMAIL_ADDRESS>[ Hanxiang<EMAIL_ADDRESS>[  University of San Francisco
Department of Mathematics and Statistics
San Francisco, CA 94117, USA
<EMAIL_ADDRESS>Indiana University Purdue University Indianapolis
Department of Mathematical Sciences
Indianapolis, IN 46202-3267, USA
<EMAIL_ADDRESS>
###### Abstract

In this article, we construct semiparametrically efficient estimators of
linear functionals of a probability measure in the presence of side
information using an easy empirical likelihood approach. We use estimated
constraint functions and allow the number of constraints to grow with the
sample size. Considered are three cases of information which can be
characterized by infinitely many constraints: (1) the marginal distributions
are known, (2) the marginals are unknown but identical, and (3) distributional
symmetry. An improved spatial depth function is defined and its asymptotic
properties are studied. Simulation results on efficiency gain are reported.

Empirical likelihood; Infinitely many constraints; Maximum empirical
likelihood estimator; Semiparametric efficiency; Spatial median,

62G05; ,

62G20, 62H11,

###### keywords:

###### keywords:

[class=AMS]

and th1Corresponding author

##  1 Introduction

Suppose that $Z_{1},\dots,Z_{n}$ are independent and identically distributed
(i.i.d.) random variables with a common distribution $Q$ taking values in a
measurable space ${\mathcal{Z}}$. In this article, we are interested in
efficient estimation of the linear functional
$\mbox{\boldmath$\theta$\unboldmath}=\int\mbox{\boldmath$\psi$\unboldmath}\,dQ$
of $Q$ for some square-integrable function $\psi$ from ${\mathcal{Z}}$ to
${\mathcal{R}}^{r}$ when side information is available through a vector
function (constraint) ${\mathbf{u}}$ which satisfies

  * (C)

${\mathbf{u}}$ is measurable from ${\mathcal{Z}}$ to ${\mathcal{R}}^{m}$ such
that $\int{\mathbf{u}}\,dQ=0$ and the variance-covariance matrix
$\int{\mathbf{u}}{\mathbf{u}}^{\top}\,dQ$ is nonsingular.

The commonly used sample mean
$\bar{}\mbox{\boldmath$\psi$\unboldmath}=\frac{1}{n}\sum_{j=1}^{n}\mbox{\boldmath$\psi$\unboldmath}(Z_{j})$
of
$\mbox{\boldmath$\theta$\unboldmath}=E(\mbox{\boldmath$\psi$\unboldmath}(Z))$
does not use the information, and is not efficient in the sense of least
dispersed regular estimators, see e.g. Bickel, Klaassen, Ritov and Wellner
(1993). Based on the criterion of maximum empirical likelihood, an improved
estimator which utilizes the information is

$\mbox{\boldmath$\tilde{}\mbox{\boldmath$\theta$\unboldmath}$\unboldmath}=\frac{1}{n}\sum_{j=1}^{n}\frac{\mbox{\boldmath$\psi$\unboldmath}(Z_{j})}{1+{\mathbf{u}}(Z_{j})^{\top}\mbox{\boldmath$\tilde{\zeta}$\unboldmath}},$
(1.1)

where $\tilde{\zeta}$ is the solution to the equation

$\sum_{j=1}^{n}\frac{{\mathbf{u}}(Z_{j})}{1+{\mathbf{u}}(Z_{j})^{\top}\mbox{\boldmath$\zeta$\unboldmath}}=0.$
(1.2)

We shall refer to $\tilde{}\mbox{\boldmath$\theta$\unboldmath}$ as the _EL-
weighted_ estimator.

There is an extensive amount of literature on the empirical likelihood testing
of hypothesis, see e.g. Owen (1988, 2001). Soon it was used to construct point
estimators. Qin and Lawless (1994) studied maximum empirical likelihood
estimators (MELE) and showed in Corollary 2 that MELE are fully efficient. As
a special case of MELE, estimators of the preceding easy form were studied in
Zhang (1995, 1997) in M-estimation and quantile processes in the presence of
auxiliary information (side information). For a _fixed_ number $m$ of _known_
constraint functions, the asymptotic normality (ASN) and efficiency of MELE
were established.

Hjort, McKeague and Van Keilegom (2009) extended the scope of the empirical
likelihood testing hypothesis, and developed a general theory for constraints
with nuisance parameters and considered the case with infinitely many
constraints. Peng and Schick (2013) generalized the empirical likelihood
testing to allow for the number of constraints to grow with the sample size
and for the constraints to use estimated criteria functions. Peng and Tan
(2018) expanded the results of the latter to U-statistics based general
estimating equations with side information.

Parente and Smith (2011) studied generalized empirical likelihood estimators
for irregular constraints. Peng and Schick (2018) presented a theory of
maximum empirical likelihood estimation and empirical likelihood ratio testing
with irregular and estimated constraint functions. Wang and Peng (2022) used
the easy EL-weighted approach to construct improved estimators of linear
functionals of a probability measure when side information is available.
Motivated by nuisance parameters common in semiparametric models and the
infinite dimension of such models, they studied the use of estimated functions
for growing number of constraints with the sample size. They applied the
results to improve estimation efficiency in the structural equation models.

We shall rely the results of Wang and Peng (2022) to construct efficient
estimators of linear functionals of a probability measure for a few cases of
side information which is determined by infinitely many constraints. Bickel,
Ritov and Wellner (1991) characterized efficient estimation of $E(h(X;Y))$ for
known $h$ when the marginal distributions of $X$ and of $Y$ are _known_ , and
construct an efficient estimator based on the criterion of minimum chisquare-
type objective function. Peng and Schick (2005) calculated the information
lower bound when the marginal distributions are unknown but identical, and
constructed an efficient estimator based on the criterion of least squares
objective. Peng and Schick (2018) constructed empirical likelihood tests of
stochastic independence and distributional symmetry. Each of independence,
symmetry, known or equal marginal distributions is equivalent to infinitely
many equations (constraints), and can be used to improve estimation
efficiency. Here we construct the EL-weighted estimators and demonstrate the
semiparametric efficiency. Note the simple analytic form of our estimators,
and the property of easy incorporation of side information to improve
efficiency.

The efficiency criteria used are that of a least dispersed regular estimator
or that of a locally asymptotic minimax estimator, and are based on the
convolution theorems and on the lower bounds of the local asymptotic risk in
LAN and LAMN families, see the monograph by Bickel, et al. (1993) among
others.

In what follows, we will summarize some results from Wang and Peng (2022) for
the convenience of our use. Meanwhile, we provide the proof of the
semiparametric effiency. In many semiparametric models, the constraint vector
function ${\mathbf{u}}=(u_{1},...,u_{m})^{\top}$ is usually unknown and must
be estimated by some measurable function
$\hat{\mathbf{u}}=(\hat{u}_{1},...,\hat{u}_{m})^{\top}$. Using it, we now work
with the EL-weights,

$\hat{\pi}_{j}=\frac{1}{n}\frac{1}{1+\hat{\mathbf{u}}(Z_{j})^{\top}\mbox{\boldmath$\hat{\zeta}$\unboldmath}},\quad
j=1,\dots,n,$ (1.3)

where $\hat{\zeta}$ solves Eqt (1.2) with ${\mathbf{u}}=\hat{\mathbf{u}}$. A
natural estimate $\hat{}\mbox{\boldmath$\theta$\unboldmath}$ of $\theta$ now
is

$\mbox{\boldmath$\hat{}\mbox{\boldmath$\theta$\unboldmath}$\unboldmath}=\sum_{j=1}^{n}\hat{\pi}_{j}\mbox{\boldmath$\psi$\unboldmath}(Z_{j})=\frac{1}{n}\sum_{j=1}^{n}\frac{\mbox{\boldmath$\psi$\unboldmath}(Z_{j})}{1+\hat{\mathbf{u}}(Z_{j})^{\top}\mbox{\boldmath$\hat{\zeta}$\unboldmath}}.$
(1.4)

We now allow the number of constraints to depend on the sample size $n$,
$m=m_{n}$, and tend to infinity slowly with $n$. To stress the dependence,
write

${\mathbf{u}}_{n}=(u_{1},\dots,u_{m_{n}})^{\top},\quad\hat{\mathbf{u}}_{n}=(\hat{u}_{1},\dots,\hat{u}_{m_{n}})^{\top},$

and
$\mbox{\boldmath$\tilde{}\mbox{\boldmath$\theta$\unboldmath}$\unboldmath}_{n}=\mbox{\boldmath$\tilde{}\mbox{\boldmath$\theta$\unboldmath}$\unboldmath}$,
$\mbox{\boldmath$\hat{}\mbox{\boldmath$\theta$\unboldmath}$\unboldmath}_{n}=\mbox{\boldmath$\hat{}\mbox{\boldmath$\theta$\unboldmath}$\unboldmath}$
for the corresponding estimators of $\theta$, that is,

$\mbox{\boldmath$\tilde{}\mbox{\boldmath$\theta$\unboldmath}$\unboldmath}_{n}=\frac{1}{n}\sum_{j=1}^{n}\frac{\mbox{\boldmath$\psi$\unboldmath}(Z_{j})}{1+{\mathbf{u}}_{n}(Z_{j})^{\top}\tilde{}\mbox{\boldmath$\zeta$\unboldmath}_{n}}\quad\mbox{and}\quad\mbox{\boldmath$\hat{}\mbox{\boldmath$\theta$\unboldmath}$\unboldmath}_{n}=\frac{1}{n}\sum_{j=1}^{n}\frac{\mbox{\boldmath$\psi$\unboldmath}(Z_{j})}{1+\hat{\mathbf{u}}_{n}(Z_{j})^{\top}\hat{}\mbox{\boldmath$\zeta$\unboldmath}_{n}},$
(1.5)

where $\tilde{}\mbox{\boldmath$\zeta$\unboldmath}_{n}$ and
$\hat{}\mbox{\boldmath$\zeta$\unboldmath}_{n}$ solve Eqt (1.2) with
${\mathbf{u}}=\tilde{\mathbf{u}}_{n}$ and ${\mathbf{u}}=\hat{\mathbf{u}}_{n}$,
respectively,.

The ASN of
$\mbox{\boldmath$\tilde{}\mbox{\boldmath$\theta$\unboldmath}$\unboldmath}_{n}$
and
$\mbox{\boldmath$\hat{}\mbox{\boldmath$\theta$\unboldmath}$\unboldmath}_{n}$
are, respectively, given in Theorems 3 and 4 of Wang and Peng (2022), and we
now prove the semiparametric efficiency of
$\mbox{\boldmath$\tilde{}\mbox{\boldmath$\theta$\unboldmath}$\unboldmath}_{n}$
and quote Theorem 4 in the Appendix for convenience of our use. For
${\mathbf{a}}\in{\mathcal{R}}^{m}$, write $\|{\mathbf{a}}\|$ the euclidean
norm. For ${\mathbf{a}},{\mathbf{b}}\in{\mathcal{R}}^{m}$, write
${\mathbf{a}}\otimes{\mathbf{b}}$ the kronecker product. Let
$L_{2}^{m}(Q)=\left\\{{\mathbf{f}}=(f_{1},\dots,f_{m})^{\top}:\int\|{\mathbf{f}}\|^{2}\,dQ<\infty\right\\}$,
and let $L_{2,0}^{m}(Q)=\left\\{{\mathbf{f}}\in
L_{2}^{m}(Q):\int{\mathbf{f}}\,dQ=0\right\\}$. For ${\mathbf{f}}\in
L_{2}^{m}(Q)$, write
$\bar{\mathbf{f}}=n^{-1}\sum_{j=1}^{n}{\mathbf{f}}(Z_{j})$ the sample average
of ${\mathbf{f}}(Z_{1}),\dots,{\mathbf{f}}(Z_{n})$, and $[{\mathbf{f}}]$ the
closed linear span of the components $f_{1},\dots,f_{r}$ in $L_{2}(Q)$. Let
$Z$ be an i.i.d. copy of $Z_{1}$. Denote by $[{\mathbf{u}}_{\infty}]$ the
closed linear span of ${\mathbf{u}}_{\infty}=(u_{1},u_{2},\dots)$ in
$L_{2,0}(Q)$. Set

${\mathbf{W}}_{n}=\mathop{\rm
Var}\nolimits({\mathbf{u}}_{n}(Z)),\quad\bar{\mathbf{W}}_{n}=\frac{1}{n}\sum_{j=1}^{n}({\mathbf{u}}_{n}{\mathbf{u}}_{n}^{\top})(Z_{j}),\quad\hat{\mathbf{W}}_{n}=\frac{1}{n}\sum_{j=1}^{n}(\hat{\mathbf{u}}_{n}\hat{\mathbf{u}}_{n}^{\top})(Z_{j}).$

Following Peng and Schick (2013), a sequence ${\mathbf{W}}_{n}$ of
$m_{n}\times m_{n}$ dispersion matrices is said to be _regular_ if

$0<\inf_{n}\inf_{\|{\mathbf{u}}\|=1}{\mathbf{u}}^{\top}{\mathbf{W}}_{n}{\mathbf{u}}\leq\sup_{n}\sup_{\|{\mathbf{u}}\|=1}{\mathbf{u}}^{\top}{\mathbf{W}}_{n}{\mathbf{u}}<\infty.$

######  Theorem 1.1.

Suppose that ${\mathbf{u}}_{n}$ satisfies (C) for each $m=m_{n}$ such that

$\max_{1\leq j\leq n}\|{\mathbf{u}}_{n}(Z_{j})\|=o_{p}(m_{n}^{-3/2}n^{1/2}),$
(1.6)

the sequence of $m_{n}\times m_{n}$ dispersion matrices ${\mathbf{W}}_{n}$ is
regular and satisfies

$|\bar{\mathbf{W}}_{n}-{\mathbf{W}}_{n}|_{o}=o_{p}(m_{n}^{-1}),$ (1.7)
$\frac{1}{n}\sum_{j=1}^{n}\left(\mbox{\boldmath$\psi$\unboldmath}(Z_{j})\otimes{\mathbf{u}}_{n}(Z_{j})-E\big{(}\mbox{\boldmath$\psi$\unboldmath}(Z_{j})\otimes{\mathbf{u}}_{n}(Z_{j})\big{)}\right)=o_{p}(m_{n}^{-1/2}).$
(1.8)

Then
$\mbox{\boldmath$\tilde{}\mbox{\boldmath$\theta$\unboldmath}$\unboldmath}_{n}$
is semiparametrically efficient as $m_{n}\to\infty$. Moreover,

$\sqrt{n}(\mbox{\boldmath$\tilde{}\mbox{\boldmath$\theta$\unboldmath}$\unboldmath}_{n}-\mbox{\boldmath$\theta$\unboldmath}){\Longrightarrow}\mathscr{N}(0,\varSigma_{0}),$

where $\varSigma_{0}=\mathop{\rm
Var}\nolimits(\mbox{\boldmath$\psi$\unboldmath}(Z))-\mathop{\rm
Var}\nolimits(\mbox{\boldmath$\varphi$\unboldmath}_{0}(Z))$ with
$\mbox{\boldmath$\varphi$\unboldmath}_{0}=\Pi(\mbox{\boldmath$\psi$\unboldmath}|[{\mathbf{u}}_{\infty}])$.

Proof. We only need to show the efficiency. It suffices to prove that the
orthonormal complement ${\mathcal{T}}=[{\mathbf{u}}_{\infty}]^{\perp}$ in
$L_{2,0}(Q)$ is the tangent space. To this end, let $Q_{t}:|t|\leq t_{0}$ with
$Q_{0}=Q$ be a regular parametric submodel with the score function $a$. By
(C),

$\int u\,dQ_{t}=0,\quad u\in[{\mathbf{u}}_{\infty}].$

Differentiating both sides of the equality with respect to $t$ at $t=0$ yields

$\int ua\,dQ=0,\quad u\in[{\mathbf{u}}_{\infty}].$

This shows $a\in{\mathcal{T}}$. For any bounded $a\in{\mathcal{T}}$, consider
$q_{t}=dQ_{t}/dQ=1+at,|t|\leq t_{0}$ for sufficient small $t_{0}$. It is clear
that $q_{t}$ is a density and the submodel with the density has the score
function $a$ which satisfies $\int ua\,dQ=0$. Since bounded functions in
${\mathcal{T}}$ are dense, it follows that the above conclusion holds for any
$a\in{\mathcal{T}}$. This shows ${\mathcal{T}}$ is the tangent space.
$\hfill\square$

The article is organized as follows. In Section 2, the EL-weighted spatial
depth function is constructed, and its ASN and efficiency are established in
the presence of distributional symmetry. The ASN and efficiency of the EL-
weighted estimators of linear functionals are proved when the marginal
distribution functions are known in Section 3, and when the marginal
distributions are unknown but equal in Section 4. The simulation results are
reported in Section 5. Section Appendix contains Theorem 4 of Wang and Peng
(2022).

##  2 The EL-weighted spatial median

In this section, we introduce the EL-weighted spatial depth function, exhibit
efficiency and give the asymptotic normality.

The statistical depth functions provide a center-outward ordering of a point
in ${\mathcal{R}}^{p}$ with respect to a distribution. High depth values
correspond to centrality while low values to “outlyingness”. Depth functions
possess robustness property, and can be used to define multivariate medians,
which are robust location estimators. Common depth functions include the Tukey
depth (halfspace depth), the simplicial depth, the projection depth, and the
spatial depth. Here we shall use the easy EL-approach to constructing improved
depths, and illustrate it with the spatial depth. The (population) spatial
depth function $D({\mathbf{x}})$ with respect to a distribution $F$ is defined
as

$D({\mathbf{x}})=1-\|E\big{(}\mathbb{S}({\mathbf{x}}-{\mathbf{X}})\big{)}\|,\quad{\mathbf{x}}\in{\mathcal{R}}^{p},$

where $\mathbb{S}({\mathbf{x}})={\mathbf{x}}/\|{\mathbf{x}}\|$ if
${\mathbf{x}}\neq 0$ ($\mathbb{S}(0)=0$) is the spatial sign function and
${\mathbf{X}}$ has the distribution function (DF) $F({\mathbf{x}})$, denoted
by ${\mathbf{X}}\sim F({\mathbf{x}})$. The depth function $D({\mathbf{x}})$
can be estimated by the sample depth function given by

$D_{n}({\mathbf{x}})=1-\Big{\|}\frac{1}{n}\sum_{i=1}^{n}\mathbb{S}_{\mathbf{x}}({\mathbf{X}}_{i})\Big{\|}.$

where
$\mathbb{S}_{\mathbf{x}}({\mathbf{t}})=\mathbb{S}({\mathbf{t}}-{\mathbf{x}})$.
The sample spatial median $\mathbf{m}_{n}$ is defined as the value which
maximizes the depth function, that is,

$\mathbf{m}_{n}=\arg\max_{{\mathbf{x}}\in{\mathcal{R}}^{p}}D_{n}({\mathbf{x}})=\arg\min_{{\mathbf{x}}\in{\mathcal{R}}^{p}}\Big{\|}\frac{1}{n}\sum_{i=1}^{n}\mathbb{S}_{\mathbf{x}}({\mathbf{X}}_{i})\Big{\|}.$

Suppose that there is available additional information that can be expressed
by a constraint function ${\mathbf{u}}$. While the sample depth
$D_{n}({\mathbf{x}})$ does not utilize the information, the _EL-weighted depth
function_ $\widetilde{D}_{n}({\mathbf{x}})$ makes use of it and is defined by

$\widetilde{D}_{n}({\mathbf{x}})=1-\Big{\|}\frac{1}{n}\sum_{i=1}^{n}\frac{\mathbb{S}_{\mathbf{x}}({\mathbf{X}}_{i})}{1+{\mathbf{u}}({\mathbf{X}}_{i})^{\top}\mbox{\boldmath$\tilde{\zeta}$\unboldmath}}\Big{\|},\quad{\mathbf{x}}\in{\mathcal{R}}^{p},$
(2.1)

where $\tilde{\zeta}$ is the solution to the equation

$\sum_{j=1}^{n}\frac{{\mathbf{u}}({\mathbf{X}}_{j})}{1+{\mathbf{u}}({\mathbf{X}}_{j})^{\top}\mbox{\boldmath$\zeta$\unboldmath}}=0.$
(2.2)

The EL-weighted spatial median $\widetilde{\mathbf{m}}$ is defined as the
value which maximizes the EL-weighted depth function, that is,

$\widetilde{\mathbf{m}}=\arg\max_{{\mathbf{x}}\in{\mathcal{R}}^{p}}\widetilde{D}_{n}({\mathbf{x}})=\arg\min_{{\mathbf{x}}\in{\mathcal{R}}^{p}}\Big{\|}\frac{1}{n}\sum_{i=1}^{n}\frac{\mathbb{S}({\mathbf{x}}-{\mathbf{X}}_{i})}{1+{\mathbf{u}}({\mathbf{X}}_{i})^{\top}\mbox{\boldmath$\tilde{\zeta}$\unboldmath}}\Big{\|}.$
(2.3)

The EL-weighted estimator of
$\mbox{\boldmath$\theta$\unboldmath}({\mathbf{x}})=E(\mathbb{S}_{\mathbf{x}}({\mathbf{X}}))$
is given by

$\mbox{\boldmath$\tilde{}\mbox{\boldmath$\theta$\unboldmath}$\unboldmath}({\mathbf{x}})=\frac{1}{n}\sum_{i=1}^{n}\frac{\mathbb{S}_{\mathbf{x}}({\mathbf{X}}_{i})}{1+{\mathbf{u}}({\mathbf{X}}_{i})^{\top}\mbox{\boldmath$\tilde{\zeta}$\unboldmath}},\quad{\mathbf{x}}\in{\mathbf{R}}^{p}.$
(2.4)

######  Remark 2.1.

The sample spatial $D_{n}({\mathbf{x}})$ is robust with the breakdown point
$1/2$. The EL-weighted $\widetilde{D}_{n}({\mathbf{x}})$ improves efficiency
but reduces robustness resulted from the zero value of the EL-weights. One can
robustify $\widetilde{D}_{n}({\mathbf{x}})$ by truncating the EL-weights from
below by a fixed constant. Truncation is commonly used in the inverse
probability weighing method. Obviously, truncation leads to certain loss of
efficiency.

Known marginal medians. In our simulation study, we looked at the side
information that the bivariate random vector
${\mathbf{X}}=(X_{1},X_{2})^{\top}$ has _known_ marginal medians $m_{10}$ and
$m_{20}$. That is, the componentwise median $(m_{10},m_{20})^{\top}$ is known.
In this case, ${\mathbf{u}}(x_{1},x_{2})=(\mathbf{1}[x_{1}\leq
m_{10}]-1/2,\mathbf{1}[x_{2}\leq m_{20}]-1/2)^{\top}$. We are motivated as
follows. It is well known that the spatial median is a better location
estimator than the componentwise median because the former takes into account
the correlation of the components while the latter ignores it, see Chen, Dang,
Peng and Bart (2009). We are interested in how much information is lost when
the componentwise median is used by looking at how much efficiency of the EL-
weighted spatial median $\tilde{\mathbf{m}}$ (when the marginal medians are
known) gains over the sample spatial median (when the marginal medians are
unknown).

Growing number of constraints. Suppose that there exists some constant vector
${\mathbf{c}}$ such that $T={\mathbf{c}}^{\top}{\mathbf{X}}$ is _symmetric_
about some known value $\tau_{0}$. Let
$\varepsilon_{j}={\mathbf{c}}^{\top}{\mathbf{X}}_{j}-\tau_{0},j=1,\ldots,n$.
Then ${\varepsilon}_{j}$’s are i.i.d. random variables which are symmetric
about zero. Let ${\varepsilon}$ be an i.i.d. copy of ${\varepsilon}_{j}$’s,
and let $F$ be the distribution function of ${\varepsilon}$. Let
$L_{2,0}(F,\mathrm{odd})$ be the subspace of $L_{2,0}(F)$ consisting of the
odd functions. Symmetry of $\varepsilon$ about $0$ implies

$E(a(\varepsilon))=0,\quad a\in L_{2,0}(F,\mathrm{odd}).$

Let $s_{k}(t)=\sin(k\pi t),t\in[-1,1],k=1,2,...$ be the orthonormal
trigonometric basis. Define $G(t)=2F(t)-1,t\in{\mathcal{R}}$. Then $G(t)$ is
an odd function in $L_{2,0}(F,\mathrm{odd})$, and $s_{k}(G(t)),k=1,2,...$ form
a basis of the space.

In this case, the constraints are
${\mathbf{u}}_{n}({\mathbf{X}}_{j})=(s_{1}(G(\varepsilon_{j})),...,s_{m_{n}}(G(\varepsilon_{j})))^{\top}$,
where we allow $m_{n}$ to grow to infinity slowly with $n$. The EL-weighted
depth function is calculated by (2.1) with ${\mathbf{u}}={\mathbf{u}}_{n}$ and
$\mbox{\boldmath$\tilde{\zeta}$\unboldmath}=\mbox{\boldmath$\tilde{\zeta}$\unboldmath}_{n}$
which solves Eqt (2.2) with ${\mathbf{u}}={\mathbf{u}}_{n}$. The EL-weighted
estimator of
$\mbox{\boldmath$\theta$\unboldmath}({\mathbf{x}})=E(\mathbb{S}_{\mathbf{x}}({\mathbf{X}}))$
then is

$\mbox{\boldmath$\tilde{}\mbox{\boldmath$\theta$\unboldmath}$\unboldmath}_{n}({\mathbf{x}})=\frac{1}{n}\sum_{i=1}^{n}\frac{\mathbb{S}_{\mathbf{x}}({\mathbf{X}}_{i})}{1+{\mathbf{u}}({\mathbf{X}}_{i})^{\top}\mbox{\boldmath$\tilde{\zeta}$\unboldmath}_{n}},\quad{\mathbf{x}}\in{\mathbf{R}}^{p}.$
(2.5)

######  Theorem 2.1.

Suppose that $F$ is continuous. Then for arbitrary but fixed
${\mathbf{x}}\in{\mathbf{R}}^{p}$, as $m_{n}\to\infty$ such that
$m_{n}^{4}/n\to 0$,
$\mbox{\boldmath$\tilde{}\mbox{\boldmath$\theta$\unboldmath}$\unboldmath}_{n}({\mathbf{x}})$
in (2.5) satisfies

$\mbox{\boldmath$\tilde{}\mbox{\boldmath$\theta$\unboldmath}$\unboldmath}_{n}({\mathbf{x}})=\bar{\mathbb{S}}_{\mathbf{x}}-\bar{}\mbox{\boldmath$\varphi$\unboldmath}_{{\mathbf{x}}0}+o_{p}(n^{-1/2}),$

where
$\mbox{\boldmath$\varphi$\unboldmath}_{{\mathbf{x}}0}=\Pi(\mathbb{S}_{\mathbf{x}}({\mathbf{X}})|L_{2,0}(F,\mathrm{odd}))$
is the projection of $\mathbb{S}_{\mathbf{x}}({\mathbf{X}})$ onto
$L_{2,0}(F,\mathrm{odd})$. As a consequence, if
$\varSigma_{0}({\mathbf{x}})=\mathop{\rm
Var}\nolimits(\mathbb{S}_{\mathbf{x}}({\mathbf{X}}))-\mathop{\rm
Var}\nolimits(\mbox{\boldmath$\varphi$\unboldmath}_{{\mathbf{x}}0}({\mathbf{X}}))$
is nonsingular,

$\sqrt{n}(\mbox{\boldmath$\tilde{}\mbox{\boldmath$\theta$\unboldmath}$\unboldmath}_{n}({\mathbf{x}})-\mbox{\boldmath$\theta$\unboldmath}({\mathbf{x}})){\Longrightarrow}\mathscr{N}(0,\varSigma_{0}({\mathbf{x}})).$

Proof of Theorem 2.1. We shall apply Theorem 1.1 to prove the result. Since
${\mathbf{W}}_{n}=E(({\mathbf{u}}{\mathbf{u}}^{\top})({\mathbf{X}}))={\mathbf{I}}_{m_{n}}$
is the identity matrix, it follows that (C) holds and ${\mathbf{W}}_{n}$ is
regular. As $\|{\mathbf{u}}_{n}({\mathbf{X}}_{j})\|\leq m_{n}^{1/2}$ for each
$j$ and $m_{n}^{4}/n=o(1)$, (1.6) is satisfied, while (1.7) holds in view of
the inequalities

$nE(|\bar{\mathbf{W}}_{n}-{\mathbf{W}}_{n}|_{o}^{2})\leq
E(\|{\mathbf{u}}_{n}({\mathbf{X}})\|^{4})\leq m_{n}^{2}.$

Let ${\mathbf{K}}_{n}$ be the left hand side of (1.8). Then (1.8) follows from

$nE(\|{\mathbf{K}}_{n}\|^{2})\leq
E(\|\mathbb{S}_{\mathbf{x}}({\mathbf{X}})\otimes{\mathbf{u}}_{n}({\mathbf{X}})\|^{2})\leq{m_{n}}E(\|\mathbb{S}_{\mathbf{x}}({\mathbf{X}})\|^{2})={m_{n}}.$

We now apply Theorem 1.1 to complete the proof. $\hfill\square$

Efficiency gain and ASN for $\tilde{\mathbf{m}}$. By the properties of
empirical likelihood, one concludes that $\widetilde{D}_{n}({\mathbf{x}})$ is
a valid depth function at least for large $n$ as all
$1+{\mathbf{u}}({\mathbf{X}}_{i})^{\top}\mbox{\boldmath$\tilde{\zeta}$\unboldmath}>0$.
Fix ${\mathbf{x}}\in{\mathcal{R}}^{p}$, let ${\mathbf{P}}_{{\mathbf{x}}}$ be
the projection of $\mathbb{S}_{\mathbf{x}}({\mathbf{X}})$ onto the closed
linear span $[{\mathbf{u}}_{\infty}]=L_{2,0}(F,odd)$. Then
$\varSigma_{0}({\mathbf{x}})=\mathop{\rm
Var}\nolimits(\mathbb{S}_{\mathbf{x}}({\mathbf{X}}))-\mathop{\rm
Var}\nolimits({\mathbf{P}}_{{\mathbf{x}}}({\mathbf{X}}))$. Clearly,

$\mathop{\rm
Var}\nolimits({\mathbf{P}}_{{\mathbf{x}}}({\mathbf{X}}))=E\big{(}\mathbb{S}_{\mathbf{x}}({\mathbf{X}})v\otimes{\mathbf{u}}({\mathbf{X}})^{\top}\big{)}{\mathbf{W}}^{-1}E\big{(}\mathbb{S}_{\mathbf{x}}({\mathbf{X}})\otimes{\mathbf{u}}({\mathbf{X}})\big{)}.$
(2.6)

Let
$\mathbb{S}_{2}({\mathbf{x}})=\mathbb{S}\big{(}E(\mathbb{S}_{\mathbf{x}}({\mathbf{X}}))\big{)}$.
If
${\mathbf{W}}_{0}({\mathbf{x}}):=\mathbb{S}_{2}({\mathbf{x}}){\mathbf{V}}_{0}({\mathbf{x}})\mathbb{S}_{2}({\mathbf{x}})^{\top}$
is nonsingular, then by Theorem 2.1 for fixed ${\mathbf{x}}\in{\mathbf{R}}$,

$\sqrt{n}(\widetilde{D}_{n}({\mathbf{x}})-D({\mathbf{x}})){\Longrightarrow}\mathscr{N}(0,{\mathbf{W}}_{0}({\mathbf{x}})).$

Note that the sample depth $D_{n}({\mathbf{x}})$ satisfies

$\sqrt{n}(D_{n}({\mathbf{x}})-D({\mathbf{x}})){\Longrightarrow}\mathscr{N}(0,{\mathbf{W}}({\mathbf{x}})),$

where ${\mathbf{W}}({\mathbf{x}})=\mathbb{S}_{2}({\mathbf{x}})\mathop{\rm
Var}\nolimits(\mathbb{S}_{\mathbf{x}}({\mathbf{X}}))\mathbb{S}_{2}({\mathbf{x}})^{\top}$.
Thus the reduction of the asymptotic variance-covariance of the EL-weighted
depth $\widetilde{D}_{n}({\mathbf{x}})$ is

$\mathbb{S}_{2}({\mathbf{x}})\mathop{\rm
Var}\nolimits({\mathbf{P}}_{{\mathbf{x}}}({\mathbf{X}}))\mathbb{S}_{2}({\mathbf{x}})^{\top}.$

We now use the Delta method to drive the ASN of the EL-weighted spatial median
$\tilde{\mathbf{m}}$. To this end, we need some results from Chaudhuri (1992)
in the case of $m=1$ for which the spatial median corresponds to his
multivariate Hodges-Lehmann type location estimate. The following is his
Assumption 3.1.

  * (PC)

${\mathbf{X}}_{1}$, ${\dots}$, ${\mathbf{X}}_{n}$ are i.i.d random vectors in
${\mathcal{R}}^{d}$ with an absolutely continuous (with respect to the
Lebesgue measure) distribution having a density $f$ that is bounded on every
bounded subset of ${\mathcal{R}}^{d}$.

Assume (PC) and $d\geq 2$. Let
${\mathbf{H}}({\mathbf{x}})=\|{\mathbf{x}}\|^{-1}({\mathbf{I}}_{d}-{\mathbf{x}}{\mathbf{x}}^{\top}/\|{\mathbf{x}}\|^{2})$
if ${\mathbf{x}}\neq 0$ and ${\mathbf{H}}(0)=0$. Note that
$\mathbb{S}({\mathbf{x}})$ and ${\mathbf{H}}({\mathbf{x}})$ are the first and
second order partial derivatives of $\|{\mathbf{x}}\|$. Under (PC), the
underlying distribution is absolutely continuous with respect to the Lebesgue
measure on ${\mathcal{R}}^{p}(d\geq 2)$, hence the (population) spatial median
$\mathbf{m}_{0}$ uniquely exists and satisfies the equation
$E(\mathbb{S}(\mathbf{m}_{0}-{\mathbf{X}}))=0$. The spatial median
$\mathbf{m}_{n}$ satisfies

$\sum_{i=1}^{n}\mathbb{S}(\mathbf{m}_{n}-{\mathbf{X}}_{i})=0.$

Let
${\mathbf{J}}=E\big{(}(\mathbb{S}\mathbb{S}^{\top})({\mathbf{m}}_{0}-{\mathbf{X}})\big{)}$
and ${\mathbf{K}}=E\big{(}{\mathbf{H}}(\mathbf{m}_{0}-{\mathbf{X}})\big{)}$.
Chaudhuri (1992) showed in his Theorem 3.3 and its corollary that if (PC)
holds then the matrices ${\mathbf{J}}$ and ${\mathbf{K}}$ are positive
definite and $\mathbf{m}_{n}$ satisfies

$\sqrt{n}(\mathbf{m}_{n}-\mathbf{m}_{0}){\Longrightarrow}\mathscr{N}(0,\,{\mathbf{K}}^{-1}{\mathbf{J}}{\mathbf{K}}^{-\top}).$

Note that the EL-weighted spatial median $\widetilde{\mathbf{m}}_{n}$
satisfies the equation,

$\sum_{i=1}^{n}\frac{\mathbb{S}(\mathbf{m}-{\mathbf{X}}_{i})}{1+{\mathbf{u}}({\mathbf{X}}_{i})^{\top}\mbox{\boldmath$\tilde{\zeta}$\unboldmath}}=0.$

Using the Delta method, we derive, with
${\mathbf{V}}_{0}(\mathbf{m}_{0})={\mathbf{J}}-\mathop{\rm
Var}\nolimits({\mathbf{P}}_{\mathbf{m}_{0}}({\mathbf{X}}))$,

$\sqrt{n}(\widetilde{\mathbf{m}}_{n}-\mathbf{m}_{0}){\Longrightarrow}\mathscr{N}(0,\,{\mathbf{K}}^{-1}{\mathbf{V}}_{0}(\mathbf{m}_{0}){\mathbf{K}}^{-\top}),$

where $\mathop{\rm Var}\nolimits({\mathbf{P}}_{\mathbf{m}_{0}}({\mathbf{X}}))$
is calculated by (2.6).

Growing number of estimated constraints. For unknown $F(x)$, we estimate it by
the symmetrized empirical distribution function,

${\mathbb{F}}(x)=\frac{1}{n}\sum_{j=1}^{n}\frac{\mathbf{1}[\varepsilon_{j}\leq
x]+\mathbf{1}[-\varepsilon_{j}\leq x]}{2},\quad x\in{\mathcal{R}}.$

Let ${\mathbb{G}}(x)=2{\mathbb{F}}(x)-1$. We thus obtain computable functions
$s_{j}({\mathbb{G}}(x))$. Write ${\mathbf{u}}_{n}$ for ${\mathbf{u}}$, and
estimate it by
$\hat{\mathbf{u}}_{n}(x)=(s_{1}({\mathbb{G}}(x)),...,s_{m_{n}}({\mathbb{G}}(x)))^{\top}$.
The EL-weighted estimator of
$\mbox{\boldmath$\theta$\unboldmath}({\mathbf{x}})=E(\mathbb{S}_{\mathbf{x}}(X))$
is now given by

$\mbox{\boldmath$\hat{}\mbox{\boldmath$\theta$\unboldmath}$\unboldmath}_{n}({\mathbf{x}})=\frac{1}{n}\sum_{i=1}^{n}\frac{\mathbb{S}_{\mathbf{x}}({\mathbf{X}}_{i})}{1+\hat{\mathbf{u}}_{n}({\mathbf{X}}_{i})^{\top}\mbox{\boldmath$\hat{\zeta}$\unboldmath}_{n}},\quad{\mathbf{x}}\in{\mathcal{R}}^{p},$
(2.7)

where $\mbox{\boldmath$\hat{\zeta}$\unboldmath}_{n}$ solves Eqt (2.2) with
${\mathbf{u}}=\hat{\mathbf{u}}_{n}$. We have

######  Theorem 2.2.

Suppose that $F$ is continuous. Then
$\mbox{\boldmath$\hat{}\mbox{\boldmath$\theta$\unboldmath}$\unboldmath}_{n}$
defined in (2.7) satisfies the conclusions of Theorem 2.1 as $m_{n}\to\infty$
such that $m_{n}^{6}/n\to 0$.

Proof of Theorem 2.2. We shall use Theorem 6.1 of Wang and Peng (2022) for the
proof. First, (C) is satisfied with ${\mathbf{W}}_{n}$ regular as
${\mathbf{W}}_{n}=I_{m_{n}}$. Next, (6.1) follows from
$\|\hat{\mathbf{u}}_{n}(Z_{j})\|^{2}\leq m_{n}$ and $m_{n}^{4}/n=o(1)$. Let

$\hat{\mathbf{W}}_{n}=\frac{1}{n}\sum_{j=1}^{n}(\hat{\mathbf{u}}_{n}\hat{\mathbf{u}}_{n}^{\top})({\mathbf{X}}_{j}),\quad\bar{\mathbf{W}}_{n}=\frac{1}{n}\sum_{j=1}^{n}({\mathbf{u}}_{n}{\mathbf{u}}_{n}^{\top})({\mathbf{X}}_{j})^{\top}.$

Then $\bar{\mathbf{W}}_{n}-{\mathbf{W}}_{n}=o_{p}(m_{n}^{-1})$ follows from
$m_{n}^{4}/n=o(1)$ and

$nE(|\bar{\mathbf{W}}_{n}-{\mathbf{W}}_{n}|_{o}^{2})\leq
E(\|{\mathbf{u}}_{n}({\mathbf{Z}}_{1})\|^{4})\leq m_{n}^{2}.$

Let
$D_{n}=n^{-1}\sum_{j=1}^{n}\|\hat{\mathbf{u}}_{n}({\mathbf{X}}_{j})-{\mathbf{u}}_{n}({\mathbf{X}}_{j})\|^{2}$.
It is easy to see

$|\hat{\mathbf{W}}_{n}-\bar{\mathbf{W}}_{n}|_{o}\leq
D_{n}+2|\bar{\mathbf{W}}_{n}|_{o}^{1/2}D_{n}^{1/2}.$

Thus (6.2) follows from $D_{n}=o_{p}(m_{n}^{-2})$ to be shown next. To this
end, let $\mathbf{s}_{n}=(s_{1},...,s_{m_{n}})^{\top}$. Then
$\|\mathbf{s}_{n}(t)\|\leq m_{n}^{1/2}$. One verifies
$\|\mbox{\boldmath$\psi$\unboldmath}_{n}^{\prime}(t)\|\leq am_{n}^{3/2}$ for
some constant $a$. Therefore, $D_{n}=o_{p}(m_{n}^{-2})$ follows from
$D_{n}=O_{p}(m^{3}_{n}/n)$ and $m_{n}^{5}/n=o(1)$, in view of

$\frac{1}{n}\sum_{j=1}^{n}\|\mathbf{s}_{n}({\mathbb{G}}({\mathbf{X}}_{j}))-\mathbf{s}_{n}(G({\mathbf{X}}_{j}))\|^{2}\leq
am_{n}^{3}\sup_{t\in{\mathcal{R}}}|{\mathbb{G}}(t)-G(t)|^{2}=O_{p}(m^{3}_{n}/n).$

Denoting
$\mbox{\boldmath$\psi$\unboldmath}({\mathbf{y}})=\mathbb{S}_{\mathbf{x}}({\mathbf{y}})$,
we break

$\frac{1}{n}\sum_{j=1}^{n}\Big{(}\mbox{\boldmath$\psi$\unboldmath}({\mathbf{X}}_{j})\otimes\hat{\mathbf{u}}_{n}({\mathbf{X}}_{j})-E\big{(}\mbox{\boldmath$\psi$\unboldmath}({\mathbf{X}}_{j})\otimes{\mathbf{u}}_{n}({\mathbf{X}}_{j}))\Big{)}={\mathbf{J}}_{n}+{\mathbf{K}}_{n},\quad\textrm{where}$
$\displaystyle{\mathbf{J}}_{n}=\frac{1}{n}\sum_{j=1}^{n}\mbox{\boldmath$\psi$\unboldmath}({\mathbf{X}}_{j})\otimes\big{(}\hat{\mathbf{u}}_{n}({\mathbf{X}}_{j})-{\mathbf{u}}_{n}({\mathbf{X}}_{j})\big{)},$
$\displaystyle{\mathbf{K}}_{n}=\frac{1}{n}\sum_{j=1}^{n}\Big{(}\mbox{\boldmath$\psi$\unboldmath}({\mathbf{X}}_{j})\otimes{\mathbf{u}}_{n}({\mathbf{X}}_{j})-E\big{(}\mbox{\boldmath$\psi$\unboldmath}({\mathbf{X}}_{j})\otimes{\mathbf{u}}_{n}({\mathbf{X}}_{j})\big{)}\Big{)}.$

By Cauchy inequality,

$\displaystyle E(\|{\mathbf{J}}_{n}\|^{2})$ $\displaystyle\leq
E(\|\mbox{\boldmath$\psi$\unboldmath}({\mathbf{X}}_{1})\|^{2})\frac{1}{n}\sum_{j=1}^{n}E(\|\hat{\mathbf{u}}_{n}({\mathbf{X}}_{j})-{\mathbf{u}}_{n}({\mathbf{X}}_{j})\|^{2})$
$\displaystyle=E(D_{n})=O(m_{n}^{3}/n)=o(m_{n}^{-1}),$

as $m_{n}^{4}/n=o(1)$. We now bound the variance by the second moment to get

$E(\|{\mathbf{K}}_{n}\|^{2})\leq\frac{1}{n}E(\|\mbox{\boldmath$\psi$\unboldmath}({\mathbf{X}}_{1})\otimes{\mathbf{u}}_{n}({\mathbf{X}}_{1})\|^{2})\leq
4\frac{m_{n}}{n}=o(m_{n}^{-1})$

as $m_{n}^{2}/n=o(1)$. Taken together we prove (6.3) – (6.4).

We now show that (6.5) holds with ${\mathbf{v}}_{n}={\mathbf{u}}_{n}$. To this
end, using Taylor expansion we write

$\frac{1}{n}\sum_{j=1}^{n}(\hat{\mathbf{u}}_{n}({\mathbf{X}}_{j})-{\mathbf{u}}_{n}({\mathbf{X}}_{j}))={\mathbf{L}}_{n}+\mathbf{m}_{n},\quad\textrm{where}$
${\mathbf{L}}_{n}=\frac{1}{n}\sum_{j=1}^{n}\mbox{\boldmath$\psi$\unboldmath}_{n}^{\prime}(G(\varepsilon_{j}))\big{(}{\mathbb{G}}(\varepsilon_{j})-G(\varepsilon_{j})\big{)},\,\mathbf{m}_{n}=\frac{1}{n}\sum_{j=1}^{n}\mbox{\boldmath$\psi$\unboldmath}_{n}^{\prime\prime}(G_{nj}^{*})\big{(}{\mathbb{G}}(\varepsilon_{j})-G(\varepsilon_{j})\big{)}^{2},$

where $G_{nj}^{*}$ lies in between ${\mathbb{G}}(\varepsilon_{j})$ and
$G(\varepsilon_{j})$. It thus follows

$\displaystyle E\big{(}\|{\mathbf{L}}_{n}\|^{2}\big{)}$
$\displaystyle\leq\frac{1}{n}E\Big{(}\|\mbox{\boldmath$\psi$\unboldmath}_{n}^{\prime}(G(\varepsilon_{1}))\|^{2}\big{(}{\mathbb{G}}(\varepsilon_{j})-G(\varepsilon_{j})\big{)}^{2}\Big{)}$
$\displaystyle\leq
a\frac{m_{n}^{3}}{n}\sup_{t\in{\mathcal{R}}}\big{(}{\mathbb{G}}(t)-G(t)\big{)}^{2}$
$\displaystyle=O_{p}(m_{n}^{3}/n^{2})=o_{p}((m_{n}n)^{-1})$

as $m_{n}^{4}/n=o(1)$. This shows ${\mathbf{L}}_{n}=o_{p}((m_{n}n)^{-1/2})$.
One has
$\|\mbox{\boldmath$\psi$\unboldmath}^{\prime\prime}(t)\|=O_{p}(m_{n}^{5/2})$.
Using this, we get

$\|\mathbf{m}_{n}\|\leq
O(m_{n}^{5/2})\sup_{t\in{\mathcal{R}}}|{\mathbb{G}}(t)-G(t)|^{2}=O_{p}(m_{n}^{5/2}/n)=o_{p}((m_{n}n)^{-1/2})$

as $m_{n}^{6}/n=o(1)$. This yields $\mathbf{m}_{n}=o_{p}((m_{n}n)^{-1/2})$.
Taken together the desired (6.5) follows. We now apply Theorem 6.1 to finish
the proof. $\hfill\square$

##  3 Efficient estimation of linear functionals with known marginals

Suppose that there is available the information that the marginal
distributions $F$ and $G$ of $Q$ are _known_. This can be characterized by

$\displaystyle\int c(x)\,dQ(x,y)=\int c(x)\,dF(x)=0,\quad c\in L_{2,0}(F),$
$\displaystyle\int d(y)\,dQ(x,y)=\int d(y)\,dG(y)=0,\quad d\in L_{2,0}(G).$

Bickel, et al. (1991) and Peng and Schick (2002) constructed efficient
estimators of the linear functional $\theta=\int\psi\,dQ$, and proved the ASN
under the assumption,

  1. (K)

There exists $\rho>0$ such that for arbitrary measurable sets $A$ and $B$,

$P(X\in A,Y\in B)\geq\rho F(A)G(B).$

Bickel, et al. (1991) showed that the project of $\psi\in L_{2}(Q)$ onto the
sum space $L_{2,0}(F)+L_{2,0}(G)$ uniquely exists. They demonstrated that the
asymptotic variance of the efficient estimator $\tilde{\theta}$ of $\theta$
can be substantially less than that of the empirical estimator
$n^{-1}\sum_{j=1}^{n}\psi(X_{j},Y_{j})$. For example, they showed that the
empirical DF $n^{-1}\sum_{j=1}^{n}\mathbf{1}[X_{j}\leq 1/2,Y_{j}\leq 1/2]$ of
$\theta=P(X\leq 1/2,Y\leq 1/2)$ (taking $\psi_{s,t}(x,y)=\mathbf{1}[x\leq
s,y\leq t]$) has three times the asymptotic variance of the efficient
estimator $\tilde{\theta}$ of $\theta$ in the case that $F$ and $G$ are
uniform distributions over $[0,1]$ and $X,Y$ are independent

Here we propose an efficient estimator based on maximum empirical likelihood.
Employing a basis $\left\\{c_{k}\right\\}$ of $L_{2,0}(F)$ and
$\left\\{d_{k}\right\\}$ of $L_{2,0}(G)$, we can reduce the uncountably many
characterizing equations to countably many ones,

$\int c_{k}(x)\,dF(x)=0,\quad\int d_{k}(y)\,dG(y)=0,\quad k=1,2,\dots.$ (3.1)

Suppose that $F$ and $G$ are continuous. This allows us to take
$c_{k}=b_{k}(F)$ and $d_{k}=b_{k}(G)$, where $b_{k}(t)$ are the trigonometric
basis,

$b_{k}(t)=\sqrt{2}\cos(k\pi t),\quad t\in[0,1],k=1,2,\dots.$ (3.2)

That is, $\left\\{c_{k}\right\\}$ and $\left\\{d_{k}\right\\}$ are bases of
$L_{2,0}(F)$ and $L_{2,0}(G)$, respectively. Using the first $2m_{n}$ terms as
constraints, the EL-weighted estimator of $\theta$ is

$\hat{\theta}_{n}=\frac{1}{n}\sum_{j=1}^{n}\frac{\psi({\mathbf{Z}}_{j})}{1+\mbox{\boldmath$\zeta$\unboldmath}_{n}^{\top}{\mathbf{u}}_{n}({\mathbf{Z}}_{j})},$
(3.3)

where
${\mathbf{u}}_{n}(x,y)=({\mathbf{b}}_{n}(F(x))^{\top},{\mathbf{b}}_{n}(G(y))^{\top})^{\top}$
with ${\mathbf{b}}_{n}=(b_{1},...,b_{m_{n}})^{\top}$. Using Theorem 1.1, we
prove

######  Theorem 3.1.

Suppose that $F$ and $G$ are continuous. Assume (K). Then, as $m_{n}\to\infty$
such that $m_{n}^{4}/n\to 0$,

$\hat{\theta}_{n}=\bar{\psi}-\bar{\varphi}_{0}+o_{p}(n^{-1/2}),$

where $\varphi_{0}$ is the projection of $\psi$ onto the sum space
$L_{2,0}(F)+L_{2,0}(G)$. Hence,

$\sqrt{n}(\hat{\theta}_{n}-\theta){\Longrightarrow}\mathscr{N}(0,\Sigma),$

where $\Sigma=\mathop{\rm Var}\nolimits(\psi({\mathbf{Z}}))-\mathop{\rm
Var}\nolimits(\varphi_{0}({\mathbf{Z}}))$.

######  Remark 3.1.

By Bickel, et al. (1991) (pp. 1328–29), the estimator $\tilde{\theta}_{n}$ in
(3.3) of $\theta=\int\psi(x,y)\,dQ(x,y)$ is semiparametrically efficient.

Proof of Theorem 3.1. We shall rely on Theorem 1.1. Since
$\|{\mathbf{u}}_{n}\|\leq 2\sqrt{m_{n}}$ and $m_{n}^{4}/n=o(1)$, it follows
that (1.6) holds. Thus

$nE(|\bar{\mathbf{W}}_{n}-{\mathbf{W}}_{n}|_{o}^{2})\leq
E(\|{\mathbf{u}}_{n}({\mathbf{Z}})\|^{4})\leq 16m_{n}^{2}=o_{p}(m_{n}^{-2})$

as $m_{n}^{4}/n=o(1)$. This shows (1.7). Let

${\mathbf{K}}_{n}=\frac{1}{n}\sum_{j=1}^{n}\Big{(}\psi({\mathbf{Z}}_{j})\otimes{\mathbf{u}}_{n}({\mathbf{Z}}_{j})-E\big{(}\psi({\mathbf{Z}}_{j})\otimes{\mathbf{u}}_{n}({\mathbf{Z}}_{j})\big{)}\Big{)}.$
(3.4)

It follows from $m_{n}^{2}/n=o(1)$ that (1.8) holds in view of

$E(\|{\mathbf{K}}_{n}\|^{2})\leq\frac{1}{n}E(\|\psi({\mathbf{Z}}_{1})\otimes{\mathbf{u}}_{n}({\mathbf{Z}}_{1})\|^{2})\leq
4\frac{m_{n}}{n}E(|\psi({\mathbf{Z}}_{1})|^{2})=o(m_{n}^{-1}).$

We are now left to prove the regularity of ${\mathbf{W}}_{n}$. Since
${\mathbf{b}}_{n}$ are the first $m_{n}$ terms of the orthonormal basis
$\left\\{b_{k}\right\\}$, it follows that
$E({\mathbf{b}}_{n}(F(X)){\mathbf{b}}_{n}(F(X))^{\top})={\mathbf{I}}_{m_{n}}$.
The same holds for $G$. Let
${\mathbf{C}}_{n}=E({\mathbf{b}}_{n}(F(X)){\mathbf{b}}_{n}(G(Y))^{\top})$.
Then ${\mathbf{W}}_{n}$ is the $2m_{n}\times 2m_{n}$ dispersion matrix whose
(1,1)- and (2, 2)-blocks are equal to ${\mathbb{I}}_{m_{n}}$ and the
(1,2)-block equal to ${\mathbf{C}}_{n}$. For
${\mathbf{s}},{\mathbf{t}}\in{\mathcal{R}}^{m_{n}}$ with
$\|{\mathbf{s}}\|^{2}+\|{\mathbf{t}}\|^{2}=1$, set
${\mathbf{r}}=({\mathbf{s}}^{\top},{\mathbf{t}}^{\top})^{\top}$. We have

${\mathbf{r}}^{\top}{\mathbf{W}}_{n}{\mathbf{r}}=\|{\mathbf{s}}\|^{2}+\|{\mathbf{t}}\|^{2}+2{\mathbf{s}}^{\top}{\mathbf{C}}_{n}{\mathbf{t}}.$
(3.5)

By Cauchy inequality,

$\displaystyle({\mathbf{s}}^{\top}{\mathbf{C}}_{n}{\mathbf{t}})^{2}$
$\displaystyle\leq{\mathbf{s}}^{\top}E({\mathbf{b}}_{n}(F(X)){\mathbf{b}}_{n}(F(X))^{\top}){\mathbf{s}}\,{\mathbf{t}}^{\top}E({\mathbf{b}}_{n}(G(Y)){\mathbf{b}}_{n}(G(Y))^{\top}){\mathbf{t}}$
$\displaystyle=\|{\mathbf{s}}\|^{2}\|{\mathbf{t}}\|^{2}\leq 1.$

It thus follows from (3.5) that
${\mathbf{r}}^{\top}{\mathbf{W}}_{n}{\mathbf{r}}\leq 4$ uniformly in $n$ and
the above ${\mathbf{r}}$. For $a\in L_{2,0}(F)$ and $b\in L_{2,0}(G)$, (K)
implies

$\displaystyle\int(a(x)-b(y))^{2}\,dQ(x,y)$
$\displaystyle\geq\rho\int(a(x)-b(y))^{2}dF(x)dG(y)$
$\displaystyle=\rho\big{(}\int a^{2}\,dF+\int b^{2}\,dG\big{)}.$

Thus

$2\int ab\,dQ\leq(1-\rho)\big{(}\int a^{2}\,dF+\int b^{2}\,dG\big{)}.$

Replacing $a$ with $-a$ yields

$2\int ab\,dQ\geq-(1-\rho)\big{(}\int a^{2}\,dF+\int b^{2}\,dG\big{)}$

Taking $a={\mathbf{s}}^{\top}{\mathbf{b}}_{n}(F)$ and
$b={\mathbf{b}}_{n}(G)^{\top}{\mathbf{t}}$ and noticing

$\int a^{2}\,dF=\|{\mathbf{s}}\|^{2},\quad\int
b^{2}\,dG=\|{\mathbf{t}}\|^{2},$

we derive

$2{\mathbf{s}}^{\top}{\mathbf{C}}_{n}{\mathbf{t}}=2\int{\mathbf{s}}^{\top}{\mathbf{b}}_{n}(F(x)){\mathbf{b}}_{n}(G(y))^{\top}{\mathbf{t}}\,dQ(x,y)\geq-(1-\rho)(\|{\mathbf{s}}\|^{2}+\|{\mathbf{t}}\|^{2}).$

By (3.5), we thus arrive at

${\mathbf{r}}^{\top}{\mathbf{W}}_{n}{\mathbf{r}}\geq\|{\mathbf{s}}\|^{2}+\|{\mathbf{t}}\|^{2}-(1-\rho)(\|{\mathbf{s}}\|^{2}+\|{\mathbf{t}}\|^{2})=\rho>0.$

Taken together we prove the regularity of ${\mathbf{W}}_{n}$, and apply
Theorem 1.1 to complete the proof. $\hfill\square$

##  4 Efficient estimation of linear functionals with equal marginals

Suppose that the marginal distributions $F$ and $G$ of $X$ and $Y$ are _equal
but unknown_. This is equivalent to the assertion that

$E(a_{k}(X)-a_{k}(Y))=0,\quad k=1,2,\dots,$ (4.1)

where $\left\\{a_{k}\right\\}$ is an orthonormal basis of $L_{2,0}(H)$ with
$H=(F+G)/2$. Assume that $F$ and $G$ are continuous. This allows us take
$a_{k}(x)=b_{k}(H(x))$ under the assumption $F=G=H$, where
$\left\\{b_{k}\right\\}$ is the trigonometric basis in (3.2). As $H$ is
unknown, we estimate it by the pooled empirical distribution function,

${\mathbb{H}}(x)=\frac{1}{n}\sum_{j=1}^{n}\frac{1}{2}(\mathbf{1}[X_{j}\leq
x]+\mathbf{1}[Y_{j}\leq x]),\quad x\in{\mathcal{R}}.$

This gives us computable functions $b_{k}({\mathbb{H}}(x))$. Let
${\mathbf{u}}_{n}(x,y)={\mathbf{b}}_{n}(H(x))-{\mathbf{b}}_{n}(H(y)),x,y\in{\mathcal{R}}$.
This is unknown and can be estimated by
$\hat{\mathbf{u}}_{n}(x,y)={\mathbf{b}}_{n}({\mathbb{H}}(x))-{\mathbf{b}}_{n}({\mathbb{H}}(y))$.
Using the first $m_{n}$ terms as constraints, the EL-weighted estimator of
$\theta=E(\psi(X,Y))$ is given by

$\hat{\theta}_{n}=\frac{1}{n}\sum_{j=1}^{n}\frac{\psi(X_{j},Y_{j})}{1+\mbox{\boldmath$\hat{\zeta}$\unboldmath}_{n}^{\top}\hat{\mathbf{u}}_{n}(X_{j},Y_{j})},$
(4.2)

where $\mbox{\boldmath$\hat{\zeta}$\unboldmath}_{n}$ is the solution to Eqt
(1.2) with ${\mathbf{u}}=\hat{\mathbf{u}}_{n}$.

Peng and Schick (2005) constructed efficient estimators of linear functionals
of a bivariate distribution with equal marginals under the condition,

$\inf_{a\in A}E[(a(X)-a(Y))^{2}]>0,$ (4.3)

where ${\mathbb{A}}=\\{a\in L_{2,0}(H):\int a^{2}\,dH=1\\}$ is the unit sphere
in $L_{2,0}(H)$. They exhibited that the asymptotic variance of an efficient
estimator of $\theta$ is about 1/3 of that of the empirical estimator or
smaller.

Applying Theorem 6.1, we show that $\hat{\theta}_{n}$ is efficient.

######  Theorem 4.1.

Suppose that the distribution functions $F$ and $G$ are equal and continuous.
Assume (4.3). Then, as $m_{n}\to\infty$ such that $m_{n}^{6}/n\to 0$,
$\hat{\theta}_{n}$ given in (4.2) satisfies

$\hat{\theta}_{n}=\bar{\psi}-\bar{\varphi}+o_{p}(n^{-1/2}),$

where $\varphi$ is the projection of $\psi$ onto ${\mathbb{A}}$. Thus

$\sqrt{n}(\hat{\theta}_{n}-\theta){\Longrightarrow}\mathscr{N}(0,\Sigma),$

where $\Sigma=\mathop{\rm Var}\nolimits(\psi)-\mathop{\rm
Var}\nolimits(\varphi)$.

######  Remark 4.1.

By Theorem 3 of Peng and Schick (2005), the estimator $\hat{\theta}_{n}$ given
in (4.2) of $\theta=\int\psi(x,y)\,dQ(x,y)$ is semiparametrically efficient.

Proof of Theorem 4.1. We shall apply Theorem 6.1. Recalling the trigonometric
basis $\left\\{b_{k}\right\\}$ in (3.2), one readily verifies that
${\mathbf{b}}_{n}=(b_{1},\ldots,b_{m_{n}})^{\top}$ has the properties,

$\|{\mathbf{b}}_{n}\|\leq({2m_{n}})^{1/2},\quad\|{\mathbf{b}}_{n}^{\prime}\|\leq\sqrt{2}\pi
m_{n}^{3/2},\quad\|{\mathbf{b}}_{n}^{\prime\prime}\|\leq\sqrt{2}\pi^{2}m_{n}^{5/2},$
(4.4)

where ${\mathbf{b}}_{n}^{\prime}$ and ${\mathbf{b}}_{n}^{\prime\prime}$ denote
the first and second order derivatives of ${\mathbf{b}}$.

Recalling
${\mathbf{u}}_{n}(x,y)={\mathbf{b}}_{n}(H(x))-{\mathbf{b}}_{n}(H(y))$ and
$\hat{\mathbf{u}}_{n}(x,y)={\mathbf{b}}_{n}({\mathbb{H}}(x))-{\mathbf{b}}_{n}({\mathbb{H}}(y))$,
one gets by the first inequality in (4.4) that

$\|{\mathbf{u}}_{n}\|\leq
2\sqrt{2}\sqrt{m_{n}},\quad\|\hat{\mathbf{u}}_{n}\|\leq
2\sqrt{2}\sqrt{m_{n}}.$ (4.5)

Hence (6.1) holds as $m^{4}/n=o(1)$. Noting
${\mathbf{W}}_{n}=E({\mathbf{u}}_{n}({\mathbf{Z}}){\mathbf{u}}_{n}({\mathbf{Z}})^{\top})$,
one has by (4.3) that

$\mbox{\boldmath$\lambda$\unboldmath}^{\top}{\mathbf{W}}_{n}\mbox{\boldmath$\lambda$\unboldmath}=E\big{(}(\mbox{\boldmath$\lambda$\unboldmath}^{\top}{\mathbf{b}}_{n}(H(X))-\mbox{\boldmath$\lambda$\unboldmath}^{\top}{\mathbf{b}}_{n}(H(Y)))^{2}\big{)}\geq\inf_{a\in
A}E\big{(}(a(X)-a(Y))^{2}\big{)}>0$

uniformly in $n$ and $\|\mbox{\boldmath$\lambda$\unboldmath}\|=1$ as both
$\mbox{\boldmath$\lambda$\unboldmath}^{\top}{\mathbf{b}}_{n}(H(X))$ and
$\mbox{\boldmath$\lambda$\unboldmath}^{\top}{\mathbf{b}}_{n}(H(Y))$ live in
$A$. Moreover,

$\mbox{\boldmath$\lambda$\unboldmath}^{\top}{\mathbf{W}}_{n}\lambda\leq
4E\big{(}(\mbox{\boldmath$\lambda$\unboldmath}^{\top}{\mathbf{b}}_{n}(H(X)))^{2}\big{)}=4.$

Thus ${\mathbf{W}}_{n}$ is regular. Let

$\hat{\mathbf{W}}_{n}=\frac{1}{n}\sum_{j=1}^{n}\hat{\mathbf{u}}_{n}({\mathbf{Z}}_{j})\hat{\mathbf{u}}_{n}({\mathbf{Z}}_{j})^{\top},\quad\bar{\mathbf{W}}_{n}=\frac{1}{n}\sum_{j=1}^{n}{\mathbf{u}}_{n}({\mathbf{Z}}_{j}){\mathbf{u}}_{n}({\mathbf{Z}}_{j})^{\top}.$

Then by the first equality in (4.5),

$nE(|\bar{\mathbf{W}}_{n}-{\mathbf{W}}_{n}|_{o}^{2})\leq
E(|{\mathbf{u}}_{n}({\mathbf{Z}}_{1})|^{4})\leq 64m_{n}^{2}.$

Hence $\bar{\mathbf{W}}_{n}-{\mathbf{W}}_{n}=o_{p}(m_{n}^{-1})$ as
$m_{n}^{4}/n=o(1)$. It can be seen

$|\bar{\mathbf{W}}_{n}-{\mathbf{W}}_{n}|_{o}\leq
D_{n}+2|\bar{\mathbf{W}}_{n}|_{o}^{1/2}D_{n}^{1/2},$

where
$D_{n}=n^{-1}\sum_{j=1}^{n}\|\hat{\mathbf{u}}_{n}({\mathbf{Z}}_{j})-{\mathbf{u}}_{n}({\mathbf{Z}}_{j})\|^{2}$.
Thus (6.2) is implied by

$D_{n}=o_{p}(m_{n}^{-2}).$ (4.6)

Using the second inequality in (4.4), we derive

$\frac{1}{n}\sum_{j=1}^{n}|{\mathbf{b}}_{n}({\mathbb{H}}({\mathbf{Z}}_{j}))-{\mathbf{b}}_{n}(H({\mathbf{Z}}_{j}))|^{2}\leq
2\pi^{2}m_{n}^{3}\sup_{t\in{\mathcal{R}}}|{\mathbb{H}}(t)-H(t)|=O_{p}(m^{3}_{n}/n).$

Hence $D_{n}=O_{p}(m^{3}_{n}/n)$ and (4.6) holds as $m_{n}^{5}/n=o(1)$. We
break

$\frac{1}{n}\sum_{j=1}^{n}\Big{(}\psi({\mathbf{Z}}_{j})\otimes\hat{\mathbf{u}}_{n}({\mathbf{Z}}_{j})-E\big{(}\psi({\mathbf{Z}}_{j})\otimes{\mathbf{u}}_{n}({\mathbf{Z}}_{j}))\Big{)}=J_{n}+K_{n},$

where

$\displaystyle{\mathbf{J}}_{n}=\frac{1}{n}\sum_{j=1}^{n}\psi({\mathbf{Z}}_{j})\otimes\big{(}\hat{\mathbf{u}}_{n}({\mathbf{Z}}_{j})-{\mathbf{u}}_{n}({\mathbf{Z}}_{j})\big{)},$
$\displaystyle{\mathbf{K}}_{n}=\frac{1}{n}\sum_{j=1}^{n}\Big{(}\psi({\mathbf{Z}}_{j})\otimes{\mathbf{u}}_{n}({\mathbf{Z}}_{j})-E\big{(}\psi({\mathbf{Z}}_{j})\otimes{\mathbf{u}}_{n}({\mathbf{Z}}_{j})\big{)}\Big{)}.$

By Cauchy inequality,

$\displaystyle E(\|{\mathbf{J}}_{n}\|^{2})$ $\displaystyle\leq
E(|\psi({\mathbf{Z}}_{1})|^{2})\frac{1}{n}\sum_{j=1}^{n}E(\|\hat{\mathbf{u}}_{n}({\mathbf{Z}}_{j})-{\mathbf{u}}_{n}({\mathbf{Z}}_{j})\|^{2})=E(\|{\mathbf{J}}_{n}\|^{2})E(D_{n})$
$\displaystyle=O(m_{n}^{3}/n)=o(m_{n}^{-1})$

where the last equality holds as $m_{n}^{4}/n=o(1)$. We now bound the variance
by the second moment and by the first equality in (4.5) to get

$E(\|{\mathbf{K}}_{n}\|^{2})\leq\frac{1}{n}E(|\psi(Z_{1})\otimes{\mathbf{u}}_{n}(Z_{1})|^{2})\leq
8\frac{m_{n}}{n}E(|\psi(Z_{1})|^{2})=o(m_{n}^{-1})$

as $m_{n}^{2}/n=o(1)$. Taken together (6.3) follows. We now show (6.5) holds
with ${\mathbf{v}}_{n}={\mathbf{u}}_{n}$. Using Taylor’s expansion, we write

$\frac{1}{n}\sum_{j=1}^{n}\big{(}{\mathbf{b}}_{n}({\mathbb{H}}(X_{j}))-{\mathbf{b}}_{n}(H(X_{j}))\big{)}={\mathbf{L}}_{n}+\mathbf{M}_{n},$

where

$\displaystyle{\mathbf{L}}_{n}=\frac{1}{n}\sum_{j=1}^{n}{\mathbf{b}}_{n}^{\prime}(H(X_{j}))\big{(}{\mathbb{H}}(X_{j})-H(X_{j})\big{)},$
$\displaystyle\mathbf{M}_{n}=\frac{1}{n}\sum_{j=1}^{n}{\mathbf{b}}_{n}^{\prime\prime}(H_{nj}^{*})\big{(}{\mathbb{H}}(X_{j})-H(X_{j})\big{)}^{2},$

where $H_{nj}^{*}$ lies in between ${\mathbb{H}}(X_{j})$ and $H(X_{j})$. Using
the second inequality in (4.4), we get

$\displaystyle E\big{(}\|{\mathbf{L}}_{n}\|^{2}\big{)}$
$\displaystyle\leq\frac{1}{n}E\Big{(}\|{\mathbf{b}}_{n}^{\prime}(H(X_{1}))\|^{2}\big{(}{\mathbb{H}}(X_{1})-H(X_{1})\big{)}^{2}\Big{)}$
$\displaystyle\leq
2\pi^{2}\frac{m_{n}^{3}}{n}\sup_{t\in{\mathcal{R}}}\big{(}{\mathbb{H}}(X_{1})-H(X_{1})\big{)}^{2}$
$\displaystyle=O_{p}(m_{n}^{3}/n^{2})=o_{p}((m_{n}n)^{-1})$

as $m_{n}^{4}/n=o(1)$. This shows ${\mathbf{L}}_{n}=o_{p}((m_{n}n)^{-1/2})$.
Using the third inequality in (4.4), one has as $m_{n}^{6}/n=o(1)$ that

$\|\mathbf{M}_{n}\|\leq\sqrt{2}\pi^{2}m_{n}^{5/2}\sup_{t\in{\mathcal{R}}}|{\mathbb{H}}(t)-H(t)|^{2}=O_{p}(m_{n}^{5/2}/n)=o_{p}((m_{n}n)^{-1/2}).$

This yields $\mathbf{M}_{n}=o_{p}((m_{n}n)^{-1/2})$. Taken together one proves
(6.5). This and (4.6) imply (6.4) as $m_{n}^{4}/n=o(1)$. Clearly,
${\mathbf{U}}_{n}={\mathbf{I}}_{m_{n}}$ satisfies
$|{\mathbf{U}}_{n}|_{o}=1=O(1)$. Peng and Schick (2005) showed that the
projection of any $h\in L_{2}(Q)$ onto ${\mathbb{A}}$ uniquely exists under
the assumption (4.3). Moreover, it is clear that
$b_{k}(H(x))-b_{k}(H(y)),k=1,2,\dots$ is a basis of ${\mathbb{A}}$, so that
$[{\mathbf{u}}_{\infty}]={\mathbb{A}}$. We now apply Theorem 6.1 to complete
the proof. $\hfill\square$

##  5 Simulations

We ran a simulation study to compare the efficiency of the EL-weighted spatial
median $\widetilde{\mathbf{m}}_{n}$ with the sample spatial median
$\mathbf{m}_{n}$ in the presence of a variety of side information. Reported on
Tables 1–5 are the maximum eigenvalues of the asymptotic variance-covariance
matrices and their ratios. Random samples were generated from 2- and 3-
dimensional Cauchy distributions, Student $t(3)$ with 3 degrees of freedom
(df), the copula distributions (see the details in the Appendix) and the
asymmetric Laplace for sample sizes $n=50,100,200,500$. Based on repetitions
$M=2000$, we calculated the averages of the maximum eigenvalues $\lambda$ and
$\tilde{\lambda}$ (i.e. the spectral norms) of the asymptotic variance-
covariance matrices of $\mathbf{m}_{n}$ and $\widetilde{\mathbf{m}}_{n}$, and
the ratio $\tilde{\lambda}/\lambda$. A ratio less than one indicates a
reduction in the norm of the variance-covariance matrix of the EL-weighted
spatial median from that of the sample spatial median.

For Table 1, the side information is that the componentwise medians are known.
For Tables 2–5, the information is that one marginal is symmetric about the
origin ($m=1,3,5$ constraints considered), for which we looked at both known
and unknown marginal (estimated by the symmetrizied EDF).

Observe that for the case of known componentwise medians, the efficiency gain
of the EL-weighted spatial median over the sample spatial median exceeded 80%;
for the case of known or estimated symmetric marginal, the efficiency gain is
more than 30%. All the ratios considered are substantially smaller than one,
indicating substantial efficiency gains of the EL-weighted spatial depth over
the sample depth. The simulation results indicated that the componentwise
median is less efficient than the spatial median but not that much for the
case considered.

##  6 Declaration of interest statement

The authors report there are no competing interests to declare.

## References

  * [1] Bickel, P. J., Ritov, Y. and Wellner, J.A. (1991). Efficient estimation of linear functionals of a probability measure $P$ with known marginal distributions. _Ann. Statist._ 19: 1316–1346.
  * [2] Bickel, P.J., Klaassen, C.A.J., Ritov, Y. and Wellner, J.A. (1993). Efficient and Adaptive Estimation in Semiparametric Models. Johns Hopkins Univ. Press, Baltimore.
  * [3] Chaudhuri, P. (1992). Multivariate location estimation using extension of R-estimates through U-statistics type approach. Ann. Statist. 20: 897 – 916.
  * [4] Chen Y., Dang, X., Peng, H. and Bart, Jr., H.L. (2009). Outlier Detection with the Kernelized Spatial Depth Function. IEEE Transactions on Pattern Analysis and Machine Intelligence 31: 288 – 305.
  * [5] Hjort, N.L., McKeague, I.W. and Van Keilegom, I. (2009). Extending the scope of empirical likelihood. Ann. Statist. 37: 1079–1111.
  * [6] Owen, A. (1988). Empirical likelihood ratio confidence intervals for a single functional. Biometrika 75: 237–249.
  * [7] Owen, A. (2001). Empirical Likelihood. Chapman & Hall/CRC, London.
  * [8] Parente, P. M. D. C. and Smith, R. J. (2011). GEL methods for nonsmooth moment indicators. Econometric Theory 27: 74–113. MR2771012
  * [9] Peng, H. and Schick, A. (2002). On efficient estimation of linear functionals of a bivariate distribution with known marginals. Statistics & Probability Letters 59, 83–91.
  * [10] Peng, H. and Schick, A. (2005). Efficient estimation of linear functionals of a bivariate distribution with equal, but unknown marginals: the least-squares approach. J. Multiv. Anal. 95, 385–409.
  * [11] Peng, H. and Schick, A. (2013). An empirical likelihood approach of goodness of fit testing. Bernoulli 19: 954–981
  * [12] Peng, H. and Schick, A. (2018). Maximum empirical likelihood estimation and related topics. Electron J. Statist. 12: 2962-2994.
  * [13] Peng, H. and Tan, F. (2018). Jackknife empirical likelihood goodness-of-fit tests for U-statistics based general estimating equations. Bernoulli 24 (1): 449–464. DOI: 10.3150/16-BEJ884.
  * [14] Qin, J. and Lawless, J. (1994). Empirical likelihood and general estimating equations. Ann. Statist. 22: 30–325.
  * [15] Wang, S. and Peng, H. (2022). Improving Efficiency in Structural Equation Models By An Easy Empirical Likelihood Approach. Unpublished manuscript. The preprint can be found at `https://iupui.edu/~hpeng/preprints_hp.html.`
  * [16] Zhang, B. (1995). M-estimation and quantile estimation in the presence of auxiliary information. J. Statist. Plann. Infer. 44, 77 – 94.
  * [17] Zhang, B. (1997). Quantile processes in the presence of auxiliary information. Ann. Inst. Statist. Math. 49, 35 – 55.

## Appendix

The details of the coupla distributions. The 2-dimensional copula distribution
has $N(0,1)$ and $t(3)$ marginals with correlation coefficient $0.5$. The
3-dimensional copula has two $N(0,1)$ marginals with correlation coefficient
$0.5$ and one $t(3)$ marginal which is correlated with each of $N(0,1)$ with
correlation $0.1$. The copula has the joint cumulative distribution function
with the uniform marginals, where each uniform marginal is defined by applying
the probability integral transform on the cumulative distribution functions of
two $N(0,1)$ and one $t(3)$, respectively.

We cite Theorem 4 of Wang and Peng (2022) below for convenience.

######  Theorem 6.1.

Suppose ${\mathbf{u}}_{n}=(u_{1},\dots,u_{m_{n}})^{\top}$ satisfies (C) for
each $m=m_{n}$. Let $\hat{\mathbf{u}}_{n}$ be an estimator of
${\mathbf{u}}_{n}$ such that

$\max_{1\leq j\leq
n}\|\hat{\mathbf{u}}_{n}(Z_{j})\|=o_{p}(m_{n}^{-3/2}n^{1/2}),$ (6.1)
$|\hat{\mathbf{W}}_{n}-{\mathbf{W}}_{n}|_{o}=o_{p}(m_{n}^{-1})$ (6.2)

for which the $m_{n}\times m_{n}$ dispersion matrices ${\mathbf{W}}_{n}$ is
regular,

$\frac{1}{n}\sum_{j=1}^{n}\left(\mbox{\boldmath$\psi$\unboldmath}(Z_{j})\otimes\hat{\mathbf{u}}_{n}(Z_{j})-E\big{(}\mbox{\boldmath$\psi$\unboldmath}(Z_{j})\otimes\hat{\mathbf{u}}_{n}(Z_{j})\big{)}\right)=o_{p}(m_{n}^{-1/2}),$
(6.3)

there exists some measurable function ${\mathbf{v}}_{n}$ from ${\mathcal{Z}}$
into ${\mathcal{R}}^{m_{n}}$ such that (C) is met for every $m=m_{n}$, the
dispersion matrix
${\mathbf{U}}_{n}={\mathbf{W}}_{n}^{-1/2}\int{\mathbf{v}}_{n}{\mathbf{v}}_{n}^{\top}\,dQ{\mathbf{W}}_{n}^{-\top/2}$
satisfies ${\mathbf{U}}_{n}=O(1)$,

$\frac{1}{n}\sum_{j=1}^{n}E\left(\|\hat{\mathbf{u}}_{n}(Z_{j})-{\mathbf{v}}_{n}(Z_{j})\|^{2}\right)=o(m_{n}^{-1}),\quad\textrm{and}$
(6.4)
$\frac{1}{n}\sum_{j=1}^{n}\hat{\mathbf{u}}_{n}(Z_{j})=\frac{1}{n}\sum_{j=1}^{n}{\mathbf{v}}_{n}(Z_{j})+o_{p}(m_{n}^{-1/2}n^{-1/2}).$
(6.5)

Then $\hat{}\mbox{\boldmath$\theta$\unboldmath}$ satisfies, as $m_{n}$ tends
to infinity, the stochastic expansion,

$\mbox{\boldmath$\hat{}\mbox{\boldmath$\theta$\unboldmath}$\unboldmath}_{n}=\bar{}\mbox{\boldmath$\psi$\unboldmath}-\bar{}\mbox{\boldmath$\varphi$\unboldmath}+o_{p}(n^{-1/2}),$
(6.6)

where
$\mbox{\boldmath$\varphi$\unboldmath}=\Pi(\mbox{\boldmath$\psi$\unboldmath}|[{\mathbf{v}}_{\infty}])$
is the projection of $\psi$ onto the closed linear span
$[{\mathbf{v}}_{\infty}]$. Thus

$\sqrt{n}(\mbox{\boldmath$\hat{}\mbox{\boldmath$\theta$\unboldmath}$\unboldmath}_{n}-\mbox{\boldmath$\theta$\unboldmath}){\Longrightarrow}\mathscr{N}(0,\varSigma).$

where $\varSigma=\mathop{\rm
Var}\nolimits(\mbox{\boldmath$\psi$\unboldmath}(Z))-\mathop{\rm
Var}\nolimits(\mbox{\boldmath$\varphi$\unboldmath}(Z))$.

Table 1:  Simulated maximal eigenvalues $\tilde{\lambda}$ and $\lambda$ of the
variance-covariance matrices of the EL-weighted and sample spatial medians
with data generated from a few distributions in the presence of known
componentwise medians $(0,0)$ and $(0,0,0)$. Cauchy
---
| dim=2 | dim=3
$n$ | $\tilde{\lambda}$ | $\lambda$ | $\tilde{\lambda}/\lambda$ | $\tilde{\lambda}$ | $\lambda$ | $\tilde{\lambda}/\lambda$
50 | 0.0641 | 0.0094 | 0.1463 | 0.0830 | 0.0111 | 0.1339
100 | 0.0330 | 0.0040 | 0.1209 | 0.0379 | 0.0048 | 0.1260
200 | 0.0157 | 0.0018 | 0.1162 | 0.0207 | 0.0024 | 0.1153
500 | 0.0060 | 0.0007 | 0.1174 | 0.0078 | 0.0009 | 0.1181
Student $t$ (df=3)
| dim=2 | dim=3
$n$ | $\tilde{\lambda}$ | $\lambda$ | $\tilde{\lambda}/\lambda$ | $\tilde{\lambda}$ | $\lambda$ | $\tilde{\lambda}/\lambda$
50 | 0.0432 | 0.0064 | 0.1477 | 0.0609 | 0.0083 | 0.1363
100 | 0.0218 | 0.0029 | 0.1322 | 0.0281 | 0.0037 | 0.1329
200 | 0.0119 | 0.0014 | 0.1161 | 0.0145 | 0.0017 | 0.1198
500 | 0.0046 | 0.0005 | 0.1096 | 0.0055 | 0.0007 | 0.1257
Copula distribution with marginals $N(0,1)$ & $t(3)$
| dim=2 | dim=3
$n$ | $\tilde{\lambda}$ | $\lambda$ | $\tilde{\lambda}/\lambda$ | $\tilde{\lambda}$ | $\lambda$ | $\tilde{\lambda}/\lambda$
50 | 0.0523 | 0.0051 | 0.0972 | 0.0542 | 0.0082 | 0.1515
100 | 0.0262 | 0.0021 | 0.0790 | 0.0278 | 0.0036 | 0.1285
200 | 0.0131 | 0.0009 | 0.0715 | 0.0135 | 0.0017 | 0.1291
500 | 0.0054 | 0.0004 | 0.0679 | 0.0055 | 0.0007 | 0.1235
Asymmetric Laplace
| dim=2 | dim=3
$n$ | $\tilde{\lambda}$ | $\lambda$ | $\tilde{\lambda}/\lambda$ | $\tilde{\lambda}$ | $\lambda$ | $\tilde{\lambda}/\lambda$
50 | 0.0153 | 0.0021 | 0.1360 | 0.0191 | 0.0024 | 0.1248
100 | 0.0072 | 0.0009 | 0.1213 | 0.0090 | 0.0011 | 0.1209
200 | 0.0035 | 0.0004 | 0.1141 | 0.0043 | 0.0005 | 0.1155
500 | 0.0013 | 0.0001 | 0.1139 | 0.0017 | 0.0002 | 0.1072

Table 2:  Same as Table 1 except for data generated from $3$-dimensional
Cauchy in the presence of one marginal distribution symmetric about the
origin.  One marginal known
---
| $m=1$ | $m=3$ | $m=5$
$n$ | $\tilde{\lambda}$ | $\lambda$ | $\tilde{\lambda}/\lambda$ | $\tilde{\lambda}$ | $\lambda$ | $\tilde{\lambda}/\lambda$ | $\tilde{\lambda}$ | $\lambda$ | $\tilde{\lambda}/\lambda$
50 | 0.0783 | 0.0499 | 0.6366 | 0.0842 | 0.0527 | 0.6261 | 0.0773 | 0.0525 | 0.6790
100 | 0.0399 | 0.0250 | 0.6277 | 0.0381 | 0.0228 | 0.5989 | 0.0382 | 0.0237 | 0.6213
200 | 0.0189 | 0.0119 | 0.6268 | 0.0195 | 0.0116 | 0.5976 | 0.0190 | 0.0116 | 0.6093
500 | 0.0074 | 0.0046 | 0.6184 | 0.0082 | 0.0045 | 0.5530 | 0.0075 | 0.0045 | 0.6012
One marginal unknown
| $m=1$ | $m=3$ | $m=5$
$n$ | $\tilde{\lambda}$ | $\lambda$ | $\tilde{\lambda}/\lambda$ | $\tilde{\lambda}$ | $\lambda$ | $\tilde{\lambda}/\lambda$ | $\tilde{\lambda}$ | $\lambda$ | $\tilde{\lambda}/\lambda$
50 | 0.0776 | 0.0518 | 0.6679 | 0.0853 | 0.0533 | 0.6249 | 0.0778 | 0.0532 | 0.6841
100 | 0.0385 | 0.0234 | 0.6082 | 0.0404 | 0.0243 | 0.6016 | 0.0406 | 0.0241 | 0.5931
200 | 0.0204 | 0.0117 | 0.5720 | 0.0203 | 0.0122 | 0.6020 | 0.0197 | 0.0109 | 0.5538
500 | 0.0073 | 0.0045 | 0.6087 | 0.0079 | 0.0049 | 0.6158 | 0.0078 | 0.0044 | 0.5595

Table 3:  Same as Table 2 except for data generated from $t(3)$ One marginal
known
---
| $m=1$ | $m=3$ | $m=5$
$n$ | $\tilde{\lambda}$ | $\lambda$ | $\tilde{\lambda}/\lambda$ | $\tilde{\lambda}$ | $\lambda$ | $\tilde{\lambda}/\lambda$ | $\tilde{\lambda}$ | $\lambda$ | $\tilde{\lambda}/\lambda$
50 | 0.0581 | 0.0384 | 0.6602 | 0.0576 | 0.0357 | 0.6199 | 0.0602 | 0.0385 | 0.6403
100 | 0.0292 | 0.0177 | 0.6069 | 0.0262 | 0.0171 | 0.6519 | 0.0274 | 0.0169 | 0.6178
200 | 0.0149 | 0.0092 | 0.6204 | 0.0142 | 0.0088 | 0.6209 | 0.0136 | 0.0085 | 0.6239
500 | 0.0055 | 0.0036 | 0.6453 | 0.0057 | 0.0034 | 0.5973 | 0.0056 | 0.0033 | 0.5830
One marginal unknown
| $m=1$ | $m=3$ | $m=5$
$n$ | $\tilde{\lambda}$ | $\lambda$ | $\tilde{\lambda}/\lambda$ | $\tilde{\lambda}$ | $\lambda$ | $\tilde{\lambda}/\lambda$ | $\tilde{\lambda}$ | $\lambda$ | $\tilde{\lambda}/\lambda$
50 | 0.0554 | 0.0359 | 0.6481 | 0.0561 | 0.0352 | 0.6278 | 0.0575 | 0.0373 | 0.6477
100 | 0.0291 | 0.0184 | 0.6314 | 0.0288 | 0.0171 | 0.5921 | 0.0286 | 0.0174 | 0.6072
200 | 0.0150 | 0.0096 | 0.6410 | 0.0136 | 0.0087 | 0.6344 | 0.0141 | 0.0086 | 0.6053
500 | 0.0056 | 0.0034 | 0.6112 | 0.0057 | 0.0033 | 0.5732 | 0.0057 | 0.0032 | 0.5622

Table 4:  Same as Table 2 except for data generated from the copula with two
$N(0,1)$ and one $t(3)$ marginals. One marginal known
---
| $m=1$ | $m=3$ | $m=5$
$n$ | $\tilde{\lambda}$ | $\lambda$ | $\tilde{\lambda}/\lambda$ | $\tilde{\lambda}$ | $\lambda$ | $\tilde{\lambda}/\lambda$ | $\tilde{\lambda}$ | $\lambda$ | $\tilde{\lambda}/\lambda$
50 | 0.0545 | 0.0366 | 0.6713 | 0.0541 | 0.0357 | 0.6601 | 0.0527 | 0.0373 | 0.7066
100 | 0.0273 | 0.0181 | 0.6626 | 0.0265 | 0.0183 | 0.6921 | 0.0269 | 0.0174 | 0.6470
200 | 0.0132 | 0.0088 | 0.6665 | 0.0142 | 0.0085 | 0.6005 | 0.0137 | 0.0082 | 0.5995
500 | 0.0054 | 0.0039 | 0.7113 | 0.0053 | 0.0033 | 0.6208 | 0.0053 | 0.0033 | 0.6162
One marginal unknown
| $m=1$ | $m=3$ | $m=5$
$n$ | $\tilde{\lambda}$ | $\lambda$ | $\tilde{\lambda}/\lambda$ | $\tilde{\lambda}$ | $\lambda$ | $\tilde{\lambda}/\lambda$ | $\tilde{\lambda}$ | $\lambda$ | $\tilde{\lambda}/\lambda$
50 | 0.0562 | 0.0385 | 0.6841 | 0.0519 | 0.0348 | 0.6698 | 0.0543 | 0.0364 | 0.6707
100 | 0.0275 | 0.0193 | 0.7021 | 0.0272 | 0.0172 | 0.6321 | 0.0267 | 0.0172 | 0.6430
200 | 0.0127 | 0.0089 | 0.7012 | 0.0131 | 0.0085 | 0.6495 | 0.0129 | 0.0082 | 0.6324
500 | 0.0054 | 0.0035 | 0.6427 | 0.0055 | 0.0036 | 0.6451 | 0.0052 | 0.0032 | 0.6187

Table 5:  Same as Table 2 except for data generated from $3$-dimensional
Asymmetric Laplace Distribution. One marginal known
---
| $m=1$ | $m=3$ | $m=5$
$n$ | $\tilde{\lambda}$ | $\lambda$ | $\tilde{\lambda}/\lambda$ | $\tilde{\lambda}$ | $\lambda$ | $\tilde{\lambda}/\lambda$ | $\tilde{\lambda}$ | $\lambda$ | $\tilde{\lambda}/\lambda$
50 | 0.0189 | 0.0113 | 0.6006 | 0.0190 | 0.0114 | 0.6004 | 0.0181 | 0.0115 | 0.6335
100 | 0.0090 | 0.0053 | 0.5845 | 0.0093 | 0.0051 | 0.5464 | 0.0089 | 0.0051 | 0.5760
200 | 0.0044 | 0.0026 | 0.5962 | 0.0042 | 0.0023 | 0.5547 | 0.0042 | 0.0023 | 0.5549
500 | 0.0016 | 0.0009 | 0.5867 | 0.0016 | 0.0009 | 0.5627 | 0.0016 | 0.0009 | 0.5525
One marginal unknown
| $m=1$ | $m=3$ | $m=5$
$n$ | $\tilde{\lambda}$ | $\lambda$ | $\tilde{\lambda}/\lambda$ | $\tilde{\lambda}$ | $\lambda$ | $\tilde{\lambda}/\lambda$ | $\tilde{\lambda}$ | $\lambda$ | $\tilde{\lambda}/\lambda$
50 | 0.0178 | 0.0118 | 0.6656 | 0.0196 | 0.0118 | 0.6023 | 0.0186 | 0.0130 | 0.6966
100 | 0.0085 | 0.0056 | 0.6565 | 0.0087 | 0.0055 | 0.6346 | 0.0083 | 0.0051 | 0.6162
200 | 0.0042 | 0.0027 | 0.6504 | 0.0043 | 0.0025 | 0.5757 | 0.0044 | 0.0025 | 0.5792
500 | 0.0017 | 0.0011 | 0.6342 | 0.0017 | 0.0010 | 0.5748 | 0.0017 | 0.0010 | 0.5539


# A Deep Learning Approach to Video Anomaly Detection using Convolutional
Autoencoders

1st Gopikrishna Pavuluri  Computer Science and Engineering
University of Texas at Arlington
<EMAIL_ADDRESS>2ndGayathri Annem  Computer Science and Engineering
University of Texas at Arlington
<EMAIL_ADDRESS>
###### Abstract

In this research we propose a deep learning approach for detecting anomalies
in videos using convolutional autoencoder and decoder neural networks on the
UCSD dataset. Our method utilizes a convolutional autoencoder to learn the
spatiotemporal patterns of normal videos and then compares each frame of a
test video to this learned representation. We evaluated our approach on the
UCSD dataset and achieved an overall accuracy of 99.35% on the Ped1 dataset
and 99.77% on the Ped2 dataset, demonstrating the effectiveness of our method
for detecting anomalies in surveillance videos. The results show that our
method outperforms other state-of-the-art methods, and it can be used in real-
world applications for video anomaly detection.

###### Index Terms:

Video anomaly detection,Convolutional autoencoder,UCSD dataset,Anomaly
detection

##  I Introduction

Anomaly detection is an essential task in various domains, such as
surveillance, healthcare, and industrial control systems. In recent years,
deep learning-based approaches have shown promising results for anomaly
detection in video data. Among various deep learning models, autoencoders have
been widely used for anomaly detection due to their ability to learn a
compressed representation of the input data. However,traditional methods for
video anomaly detection rely on hand-crafted features and heuristics, which
limits their effectiveness in handling complex and diverse scenarios.

To address this limitation, we proposed and evaluated a Convolutional
Autoencoder-based approach for anomaly detection in video data. The proposed
approach takes a sequence of video frames as input and reconstructs the frames
using learned representation. By comparing the reconstructed frames with the
original frames, the model can detect any discrepancies that indicate the
presence of an anomaly in the video sequence. We conducted experiments on the
UCSD Pedestrian dataset and evaluated the performance of the proposed approach
using train, test, and validation dataset. Our results demonstrated that the
proposed approach outperforms existing state-of-the-art methods for anomaly
detection in video data.

The rest of this paper is organized as follows: in Section II, we review
related work in video anomaly detection. In Section III, We describe the
proposed methodology in detail, including the architecture of the convolution
autoencoder and decoder neural networks. In Section IV, we present
experimental results and discuss the performance of my approach using metrics.
Finally, we conclude our paper and discuss future work in Section VI.

##  II Related Work

Video anomaly detection has been an active area of research in computer vision
and pattern recognition in recent years. Early methods for anomaly detection
in videos relied on hand-crafted features and statistical models. For
instance, previous researchers proposed a method that used Gaussian mixture
models (GMM) to detect anomalies in videos [1]. Similarly, past researchers
demonstrated the utilization of of local spatio-temporal features and Bayesian
networks for anomaly detection in crowded scenes [2]. Although these methods
achieved good results in specific scenarios, they lacked generalization
capabilities and were sensitive to the choice of features.

Deep learning-based methods have shown promising results for video anomaly
detection due to their ability to learn high-level representations of data.
One of the most popular deep learning models for anomaly detection is the
autoencoder. Autoencoders are unsupervised neural networks that can learn
compressed representations of data by minimizing the reconstruction error
between the input and output [3]. To detect anomalies, a threshold can be set
on the reconstruction error. An input sample with a high reconstruction error
is more likely to be an anomaly than a sample with a low reconstruction error.

Convolutional autoencoders have been used in recent years for video anomaly
detection due to their ability to learn spatiotemporal patterns in video data
[4]. One of the earliest works that used convolutional autoencoders for
anomaly detection in videos used a deep architecture that combined a
convolutional autoencoder and a two-stream convolutional neural network (CNN)
for video anomaly detection [5]. Another study used a spatiotemporal
autoencoder for detecting anomalies in videos of complex scenes [6]. In this
research paper, we build upon these works and propose a novel deep learning
approach that utilizes convolutional autoencoder and decoder neural networks
for video anomaly detection on the UCSD Pedestrian dataset.

##  III Problem Statement

Video anomaly detection is a critical task in video surveillance, as it can
help identify abnormal events and prevent potential security threats. However,
traditional methods for anomaly detection in videos have limitations in terms
of their accuracy and robustness, particularly when dealing with complex
scenes and large datasets. Deep learning-based approaches, and in particular,
convolutional autoencoder and decoder neural networks, have shown promising
results for video anomaly detection due to their ability to learn high-level
representations of spatiotemporal patterns in video data. In this paper, we
propose a novel deep learning approach that utilizes a convolutional
autoencoder and decoder neural network for video anomaly detection on the UCSD
Pedestrian dataset. Our method builds upon previous works that have used
autoencoder-based models for anomaly detection in videos, and addresses some
of their limitations by incorporating a decoder network that can generate
reconstructed video frames.

To evaluate the performance of our proposed approach, we conducted experiments
on the UCSD Pedestrian dataset and used train,validation and test dataset to
measure the accuracy of the model. Our results demonstrated that the proposed
approach outperforms existing state-of-the-art methods for anomaly detection
in video data, achieving high accuracy in detecting anomalous events.

To implement our proposed approach, we designed and trained a convolutional
autoencoder and decoder neural network architecture. The encoder network
consists of several convolutional layers that extract spatiotemporal features
from video frames, followed by fully connected layers that encode the features
into a compressed representation. The decoder network takes the encoded
representation and generates reconstructed video frames. We trained the
network using a binary cross entropy loss, which encourages the network to
learn to reconstruct normal video frames accurately while detecting anomalous
ones.

In summary, the main objective of our research is to develop a deep learning-
based approach that can accurately detect anomalies in video data,
particularly in complex scenes and large datasets, using a novel convolutional
autoencoder and decoder neural network architecture. The proposed approach has
the potential to enhance the effectiveness and efficiency of video
surveillance systems and can contribute to improving public safety and
security.

##  IV Problem Solution

To address the challenges of video anomaly detection, we propose a novel
approach that utilizes convolutional autoencoder and decoder neural networks
on the UCSD Pedestrian dataset [7]. The architecture of our proposed model
consists of an encoder network and a decoder network, where the encoder
network extracts spatiotemporal features from video frames and encodes them
into a compressed representation, while the decoder network generates
reconstructed video frames from the encoded representation.

We trained our model using the Adam optimizer [8] and binary cross-entropy
loss function [9], which has been shown to be effective for anomaly detection
in video data. We also used a combination of reconstruction loss and anomaly
detection loss during training, which encourages the network to learn to
reconstruct normal video frames accurately while detecting anomalous ones.

###  IV-A Encoder Network

The encoder network,as shown in ‘Fig. 1‘ consists of several convolutional
layers with increasing filter sizes to extract higher-level features, followed
by a fully connected layer that encodes the features into a compressed
representation of size 32.The architecture details are presented in Table 1.

Figure 1: High level overview of Encoder Architecture[16] TABLE I: Encoder Architecture for Frame Dimensionality Reduction Layer | Input Size | Output Size | Activation Function
---|---|---|---
Input | $256\times 256\times 1$ | $256\times 256\times 1$ | N/A
Conv2D | $256\times 256\times 1$ | $128\times 128\times 16$ | ReLU
Conv2D | $128\times 128\times 16$ | $64\times 64\times 32$ | ReLU
Conv2D | $64\times 64\times 32$ | $32\times 32\times 64$ | ReLU
Conv2D | $32\times 32\times 64$ | $16\times 16\times 128$ | ReLU
Flatten | $16\times 16\times 128$ | $32768$ | N/A
Dense | $32768$ | $32$ | Sigmoid

###  IV-B Decoder Network

The decoder network then takes the compressed representation from encoder and
generates reconstructed video frames using several transposed convolutional
layers that gradually increase the resolution of the image.The architecture
details are presented in Table II.

TABLE II: Decoder Architecture for Frame Reconstruction Layer | Input Size | Output Size | Activation Function
---|---|---|---
Input | $32$ | $32$ | N/A
Dense | $32$ | $32768$ | ReLU
Reshape | $32768$ | $16\times 16\times 128$ | N/A
Conv2DTranspose | $16\times 16\times 128$ | $32\times 32\times 64$ | ReLU
Conv2DTranspose | $32\times 32\times 64$ | $64\times 64\times 32$ | ReLU
Conv2DTranspose | $64\times 64\times 32$ | $128\times 128\times 16$ | ReLU
Conv2DTranspose | $128\times 128\times 16$ | $256\times 256\times 1$ | Sigmoid

##  V Experiment Result

We conducted the experiments using the TensorFlow deep learning framework in
the Google Colab environment[11] and trained the proposed CNN autoencoder
network for 10 epochs on the NVIDIA A100-SXM GPU. The training accuracy
achieved was 99.78%, and the validation accuracy was 99.77%, indicating that
the model is not overfitting.

To evaluate the performance of the model, we used a separate test set from the
UCSD dataset and achieved an accuracy of 99.35%. The results demonstrate the
effectiveness of the proposed approach for detecting anomalies in video data.

The binary crossentropy loss function used in the experiments can be
represented by the following equation:

$\mathcal{L}(y,\hat{y})=-\frac{1}{N}\sum_{i=1}^{N}[y_{i}\log(\hat{y}_{i})+(1-y_{i})\log(1-\hat{y}_{i})],$
(1)

where $y$ represents the ground truth labels, $\hat{y}$ represents the
predicted labels, and $N$ is the number of samples in the batch.

able IV shows the detailed results of the experiments:

Epochs | Training Accuracy | Validation Accuracy
---|---|---
1 | 99.68 | 99.77
2 | 99.78 | 99.77
3 | 99.78 | 99.77
4 | 99.78 | 99.77
5 | 99.78 | 99.77
6 | 99.78 | 99.77
7 | 99.78 | 99.77
8 | 99.78 | 99.77
9 | 99.78 | 99.77
10 | 99.78 | 99.77
TABLE III: Training and Validation Accuracy for CNN Autoencoder on UCSD Video
Dataset

In order to visualize the training and validation accuracy of the proposed
approach, We plotted the accuracy of the model on the training and validation
dataset over the 10 epochs of training using the TensorFlow framework [10]. As
can be seen from Figure 2, both the training and validation accuracy quickly
reach a value of 99.78% and 99.77% and remain constant for the rest of the
training process.

Figure 2: Existing state of art method comparison. Method | Ped1 | Ped2
---|---|---
Adam[12] | 77.1 | -
SF[13] | 67.5 | 55.6
MPPCA[7] | 66.8 | 69.3
MPPCA+SF[7] | 74.2 | 61.3
HOFME [14] | 72.7 | 87.5
Spatial AE[15] | 89.9 | 87.4
Our Approach | 99.78* | 99.77*
TABLE IV: Training and Validation Accuracy for CNN Autoencoder on UCSD Video
Dataset

Based on the evaluation of the proposed approach on the separate test set from
the UCSD dataset, the achieved accuracy of 99.35% demonstrates the high
performance of the proposed CNN autoencoder network for detecting anomalies in
video data. The results indicate that the network has successfully learned the
characteristics of normal and abnormal behavior patterns in the dataset. The
high accuracy achieved can lead to the use of the proposed approach in various
real-world applications such as video surveillance systems, anomaly detection
in medical imaging, and quality control in manufacturing. However, further
evaluation on larger and more diverse datasets is necessary to assess the
generalizability and robustness of the proposed approach.

##  VI Conclusion

Based on the experimental results, it can be concluded that the proposed CNN
autoencoder network is an effective approach for anomaly detection in video
data. The model achieved high accuracy on both the training and validation
sets, as well as on a separate test set.

In future work, We plan to investigate the effectiveness of the proposed
approach on larger and more complex video datasets. Additionally, we will
explore the use of different architectures, such as variational autoencoders
and optical flow for anomaly detection in video data.

Overall, this research provides a promising approach for anomaly detection in
video data, with potential for further improvement and application in real-
world scenarios.

##

## References

  * [1] P. Turaga, R. Chellappa, and V. Subrahmanian, ”Machine recognition of human activities: A survey,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 18, no. 11, pp. 1473-1488, 2008.
  * [2] M. Harville and L. Davis, ”Crowd behavior recognition for video surveillance,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2003, pp. 19-25.
  * [3] I. Goodfellow, Y. Bengio, and A. Courville, Deep Learning, MIT Press, 2016.
  * [4] H. Lu et al., ”Abnormal event detection using deep learning in video surveillance,” in Proceedings of the 23rd ACM international conference on Multimedia, 2015, pp. 507-516.
  * [5] S. Sabokrou, M. Fayyaz, M. Fathy, and R. Klette, ”Deep-cascade: Cascading 3d deep neural networks for fast anomaly detection and localization in crowded scenes,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018, pp. 3468-3476.
  * [6] K. Simonyan and A. Zisserman, ”Two-stream convolutional networks for action recognition in videos,” in Proceedings of the Advances in Neural Information Processing Systems, 2014, pp. 568-576.
  * [7] V. Mahadevan, W. Li, V. Bhalodia, and N. Vasconcelos, “Anomaly detection in crowded scenes,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2010, pp. 1975-1981.
  * [8] D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” arXiv preprint arXiv:1412.6980, 2014.
  * [9] M. Sabokrou, M. Fathy, M. J. Hoseini, and K. Kiani, “Adversarially learned one-class classifier for novelty detection,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018, pp. 3379-3388.
  * [10] TensorFlow, “TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems,” https://www.tensorflow.org/, 2015.
  * [11] S. J. Ha, J. Lee, J. Kim, and J. Kim, “Google Colaboratory,” Journal of Open Source Software, vol. 4, no. 43, p. 1686, 2019.
  * [12] Adam, A., Rivlin, E., Shimshoni, I., Reinitz, D.: Robust real-time unusual event detection using multiple fixed-location monitors. IEEE Transactions on Pattern Analysis and Machine Intelligence 30(3), 555–560 (2008)
  * [13] Mehran, R., Oyama, A., Shah, M.: Abnormal crowd behavior detection using social force model. In: 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, CVPR Workshops 2009\. pp. 935–942 (2009)
  * [14] ] Wang, T., Snoussi, H.: Histograms of optical flow orientation for abnormal events detection. In: IEEE International Workshop on Performance Evaluation of Tracking and Surveillance, PETS. pp. 45–52 (2013)
  * [15] Chong, Y.S., Tay, Y.H. (2017). Abnormal Event Detection in Videos Using Spatiotemporal Autoencoder. In: Cong, F., Leung, A., Wei, Q. (eds) Advances in Neural Networks - ISNN 2017. ISNN 2017. Lecture Notes in Computer Science(), vol 10262. Springer, Cham
  * [16] LeNail, (2019). NN-SVG: Publication-Ready Neural Network Architecture Schematics. Journal of Open Source Software, 4(33), 747, https://doi.org/10.21105/joss.00747