# An Isoperimetric inequality for the first Steklov-Dirichlet Laplacian
eigenvalue of convex sets with a spherical hole

Nunzia Gavitone, Gloria Paoli, Gianpaolo Piscitelli, Rossano Sannipoli
Dipartimento di Matematica e Applicazioni “R. Caccioppoli”, Università degli
studi di Napoli Federico II
Via Cintia, Complesso Universitario Monte S. Angelo, 80126 Napoli, Italy.
<EMAIL_ADDRESS>Dipartimento di Matematica e Applicazioni “R.
Caccioppoli”, Università degli studi di Napoli Federico II
Via Cintia, Complesso Universitario Monte S. Angelo, 80126 Napoli, Italy.
<EMAIL_ADDRESS>Dipartimento di Ingegneria Elettrica e dell’Informazione
“M. Scarano”, Università degli Studi di Cassino e del Lazio Meridionale
Via G. Di Biasio n. 43, 03043 Cassino (FR), Italy.
<EMAIL_ADDRESS>Dipartimento di Matematica e Applicazioni “R.
Caccioppoli”, Università degli studi di Napoli Federico II
Via Cintia, Complesso Universitario Monte S. Angelo, 80126 Napoli, Italy.
<EMAIL_ADDRESS>
###### Abstract.

AbstractAbstract

###### Abstract.

In this paper we prove the existence of a maximum for the first Steklov-
Dirichlet Laplacian eigenvalue in the class of convex sets with a fixed
spherical hole under volume constraint. More precisely, if
$\Omega=\Omega_{0}\setminus\overline{B}_{R_{1}}$, where $B_{R_{1}}$ is the
ball centered at the origin with radius $R_{1}>0$ and
$\Omega_{0}\subset\mathbb{R}^{n}$, $n\geq 2$, is an open bounded and convex
set such that $B_{R_{1}}\Subset\Omega_{0}$, then the first Steklov-Dirichlet
Laplacian eigenvalue $\sigma_{1}(\Omega)$ has a maximum when $R_{1}$ and the
measure of $\Omega$ are fixed. Moreover, if $\Omega_{0}$ is contained in a
suitable ball, we prove that the spherical shell is the maximum.

MSC 2020: 28A75, 35J25, 35P15.
Keywords: Laplacian eigenvalue, Steklov-Dirichlet boundary conditions,
isoperimetric inequality.

AbstractAbstract

##  1\. Introduction and main results

Let $\Omega_{0}\subset\mathbb{R}^{n}$, $n\geq 2$, be an open, bounded,
connected set with Lipschitz boundary such that $B_{R_{1}}\Subset\Omega_{0}$,
where $B_{R_{1}}$ is the open ball of radius $R_{1}>0$ centered at the origin
and let us set $\Omega:=\Omega_{0}\setminus\overline{B_{R_{1}}}$. The first
Steklov-Dirichlet Laplacian eigenvalue of $\Omega$ is defined by

$\sigma_{1}(\Omega)=\min_{\begin{subarray}{c}v\in H^{1}_{\partial
B_{R_{1}}}(\Omega)\\\ v\not\equiv
0\end{subarray}}\dfrac{\displaystyle\int_{\Omega}|Dv|^{2}\;dx}{\displaystyle\int_{\partial\Omega_{0}}v^{2}\;d\mathcal{H}^{n-1}},$
(1.1)

where $H^{1}_{\partial B_{R_{1}}}(\Omega)$ is the set of Sobolev functions on
$\Omega$ vanishing on $\partial B_{R_{1}}$ (see Section 2 for the precise
definition). Denoting by $\nu$ the outer unit normal to $\partial\Omega_{0}$,
any minimizer of (1.1) satisfies the following problem

$\begin{cases}\Delta u=0&\mbox{in}\ \Omega\vspace{0.2cm}\\\ \dfrac{\partial
u}{\partial\nu}=\sigma(\Omega)u&\mbox{on}\ \partial\Omega_{0}\vspace{0.2cm}\\\
u=0&\mbox{on}\ \partial B_{R_{1}},\end{cases}$ (1.2)

with $\sigma(\Omega)=\sigma_{1}(\Omega)$. The spectrum of (1.2) is discrete
and the sequence of eigenvalues can be ordered (see for instance [A, D, P])

$0<\sigma_{1}(\Omega)\leq\sigma_{2}(\Omega)\leq...\ .$

Furthermore, the first eigenvalue is simple and the corresponding
eigenfunctions have constant sign (see e.g. [D, PPS]). The aim of this paper
is to prove an isoperimetric inequality for $\sigma_{1}(\Omega)$. When
$R_{1}=0$, (1.2) is the classical Steklov-Laplacian eigenvalue problem. In
this case, Weinstock in [W1, W2] proved an isoperimetric inequality for the
first non-trivial Steklov eigenvalue in two dimensions. More precisely, he
showed that among all simply connected sets of the plane with prescribed
perimeter, the disc maximizes the first non-trivial Steklov-Laplacian
eigenvalue. In [BFNT] the authors proved that Weinstock inequality holds true
in any dimension of the space, provided they restrict to the class of convex
sets with fixed perimeter. In [B], it is proved that the ball is still a
maximizer for the first non-trivial Steklov eigenvalue among all bounded open
sets with Lipschitz boundary of $\mathbb{R}^{n}$, $n\geq 2$, with fixed
volume. Stability and instability results are also studied (for instance we
refer to [BDPR, BN, GLMPT]). When we consider a spherical hole with
homegeneous Dirichlet boundary condition, that is $R_{1}>0$, the Steklov-
Dirichlet eigenvalue problem for the Laplacian (1.2) is substantially
different. The study of an eigenvalue problem on sets with a spherical hole is
actually a topic of interest and problem (1.2) has been considered by several
authors (see for instance [D, F, HLS, HP, PPS, SV]). In [PPS] it is proved
that the first eigenvalue $\sigma_{1}(\Omega)$, as defined in (1.1), is
bounded from above when both the volume of $\Omega$ and the radius of the
inner ball are fixed. Moreover the authors prove that the spherical shell is a
local maximizer. The aim of this paper is twofold. Firstly, we prove the
existence of a maximum for $\sigma_{1}(\Omega)$ in the class of sets
$\Omega=\Omega_{0}\setminus\overline{B}_{R_{1}}$, where
$\Omega_{0}\subset\mathbb{R}^{n}$, $n\geq 2$, is an open bounded and convex
set containing $B_{R_{1}}$, when $R_{1}$ and the measure of $\Omega$ are
fixed. We stress that, when $\Omega_{0}=B_{R_{2}}(x_{0})$ is a ball centered
at $x_{0}$ with radius $R_{2}>R_{1}$, in [F, SV] it is proved that
$\sigma_{1}(\Omega)$ achieves the maximum when $\Omega$ is the spherical
shell, that is when the two balls are concentric. Our second goal is to prove
that this is also true for a suitable class of annular sets. More precisely
our main result is the following.

######  Theorem 1.1.

Let $R_{1}>0$, $\Omega_{0}\subset\mathbb{R}^{n}$ be an open, bounded and
convex set, $n\geq 2$, such that $B_{R_{1}}\Subset\Omega_{0}\subseteq
B_{\bar{R}}$, where $B_{\bar{R}}$ is the ball centered at the origin with
radius $\bar{R}$ given by

$\bar{R}=\begin{cases}R_{1}e^{\sqrt{2}}&\text{ if }n=2\\\
R_{1}\bigg{[}\frac{(n-1)+(n-2)\sqrt{2(n-1)}}{n-1}\bigg{]}^{\frac{1}{n-2}}&\text{
if }n\geq 3.\end{cases}$ (1.3)

Then, denoted by $\Omega=\Omega_{0}\setminus\overline{B}_{R_{1}}$, the
following inequality holds

$\sigma_{1}(\Omega)\leq\sigma_{1}(A_{R_{1},R_{2}}),$ (1.4)

where $A_{R_{1},R_{2}}$ is the spherical shell of radii $R_{1}<R_{2}$ having
the same volume as $\Omega$.

The outline of the paper is the following. In the next Section we set the
notation and collect some basic results for the Steklov-Dirichlet eigenvalue
problem that will be needed in the sequel. In Section 3 we state and prove the
existence result. In Section 4, by using a suitable weighted isoperimetric
inequality, we prove that the spherical shell is a maximum for
$\sigma_{1}(\Omega)$ when both the volume of $\Omega$ and the radius of the
inner ball are fixed. Eventually, in Section 5, we discuss on a maximization
problem for $\sigma_{1}(\Omega)$ under a perimeter constraint.

##  2\. Background and preliminaries results

###  2.1. Notations and basic facts

Throughout this paper, we denote by $B_{R}(x_{0})$ the ball centered at
$x_{0}\in\mathbb{R}^{n}$ with radius $R>0$, by $B_{R}$ the ball centered at
the origin with radius $R$ and by $B$, $\mathbb{S}^{n-1}$ and $\omega_{n}$
respectively the unit ball of $\mathbb{R}^{n}$ centered at the origin, its
boundary and its volume. Let $R_{1},R_{2}$ be such that $0<R_{1}<R_{2}$, the
spherical shell will be denoted as follows:

$A_{R_{1},R_{2}}=\\{x\in\mathbb{R}^{n}\colon R_{1}<|x|<R_{2}\\}.$

Moreover, the $(n-1)$-dimensional Haussdorf measure in $\mathbb{R}^{n}$ will
be denoted by $\mathcal{H}^{n-1}$. The Euclidean product in $\mathbb{R}^{n}$
is denoted by $(\cdot,\cdot)$. Let $D\subseteq\mathbb{R}^{n}$ be an open
bounded set and let $E\subseteq\mathbb{R}^{n}$ be a measurable set. For the
sake of completeness, we recall here the definition of the perimeter of $E$ in
$D$ (see for instance [AFP, M]), that is

$P(E;D)=\sup\left\\{\int_{E}{\rm div}\varphi\>dx:\;\varphi\in
C^{\infty}_{c}(D;\mathbb{R}^{n}),\;||\varphi||_{\infty}\leq 1\right\\}.$

The perimeter of $E$ in $\mathbb{R}^{n}$ will be denoted by $P(E)$ and, if
$P(E)<\infty$, we say that $E$ is a set of finite perimeter. Moreover, if $E$
has Lipschitz boundary, it holds

$P(E)=\mathcal{H}^{n-1}(\partial E).$

The Lebesgue measure of a measurable set $E\subset\mathbb{R}^{n}$ will be
denoted by $V(E)$. If $E$ is a convex set of $\mathbb{R}^{n}$, the following
inequality holds (see for instance [DPBG, Sc])

$\rho(E)\leq\dfrac{nV(E)}{P(E)},$ (2.1)

where

$\rho(E)=\sup_{x\in E}\;\inf_{y\in\partial E}|x-y|$ (2.2)

is the inradius of $E$. The diameter of $E$ is defined as

$\operatorname{diam}(E)=\sup_{x,y\in E}|x-y|.$

Finally, we recall the definition of Hausdorff distance between two non-empty
compact sets $E,F\subset\mathbb{R}^{n}$, that is (see for instance [Sc])

$\delta_{\mathcal{H}}(E,F)=\inf\\{\varepsilon>0\ :\ E\subset
F+B_{\varepsilon},\ F\subset E+B_{\varepsilon}\\}.$

Note that, if $E,F$ are both convex sets, then
$\delta_{\mathcal{H}}(E,F)=\delta_{\mathcal{H}}(\partial E,\partial F)$. Let
$\\{E_{k}\\}_{k\in\mathbb{N}}$ be a sequence of non-empty compact subsets of
$\mathbb{R}^{n}$, we say that $E_{k}$ convergences to $E$ in the Hausdorff
sense and we denote

$E_{k}\stackrel{{\scriptstyle\mathcal{H}}}{{\longrightarrow}}E$

if and only if $\delta_{\mathcal{H}}(E_{k},E)\to 0$ as $k\to\infty$. Moreover,
we say that $\\{E_{k}\\}_{k\in\mathbb{N}}$ converges in measure to $E$, and we
write $E_{k}\rightarrow E$, if $\chi_{E_{k}}\rightarrow\chi_{E}$ in
$L^{1}(\mathbb{R}^{n})$, where $\chi_{E}$ and $\chi_{E_{k}}$ are the
characteristic functions of $E$ and $E_{k}$ respectively. In what follows we
recall some properties of the convex bodies, i.e., compact convex sets without
empty interior. We refer to [Sc] for further properties and the details. Let
$K\subset\mathbb{R}^{n}$ be a bounded convex body. The support function
$h_{K}$ of $K$ is defined as follows

$h_{K}\colon\mathbb{S}^{n-1}\to\mathbb{R},\quad h_{K}(\theta)=\sup_{y\in
K}(\theta,y).$

If the origin belongs to $K$ then $h_{K}$ is non-negative and
$h_{K}(\theta)\leq\text{diam}(K)$ for every $\theta\in\mathbb{S}^{n-1}$. Let
$K\subset\mathbb{R}^{n}$ be a bounded convex body such that the origin is an
interior point of $K$. The radial function of $K$ is defined as follows

$\rho_{K}(x)=\sup\\{\lambda\geq 0\colon\lambda x\in K\\},\quad
x\in\mathbb{S}^{n-1},$ (2.3)

and it is a Lipschitz function. The radial map is

$r_{K}\colon\mathbb{S}^{n-1}\to\partial K,\quad r_{K}(x)=x\rho_{K}(x).$ (2.4)

Then

$\partial K=\\{x\,\rho_{K}(x),x\in\mathbb{S}^{n-1}\\}.$ (2.5)

Moreover if $f\colon\partial K\to\mathbb{R}$ is $(n-1)-$integrable the
following formula for the change of variable given by the radial map holds:

$\int_{\partial
K}f\,d\mathcal{H}^{n-1}=\int_{\mathbb{S}^{n-1}}f(r_{K}(x))\displaystyle\frac{\rho_{K}(x)}{h_{K}(\nu_{K}(r_{K}(x)))}\,d\mathcal{H}^{n-1},$
(2.6)

where $\nu_{K}(r_{K}(x))$ is the outer unit normal to $\partial K$ at the
point $r_{K}(x)=x\rho_{K}(x)$. Being (see for example [Sc])

$\nu_{K}(r_{K}(x))=\displaystyle\frac{x\rho_{K}(x)-\nabla_{\tau}\rho_{K}(x)}{\sqrt{(\rho_{K}(x))^{2}+|\nabla_{\tau}\rho_{K}(x)|^{2}}},$

equivalently we have

$\int_{\partial
K}f\,d\mathcal{H}^{n-1}=\int_{\mathbb{S}^{n-1}}f(r_{K}(x))(\rho_{K}(x))^{n-1}\sqrt{1+\left(\frac{\nabla_{\tau}\rho_{K}(x)}{\rho_{K}(x)}\right)^{2}}\,d\mathcal{H}^{n-1}.$

The following result holds (see for instance [Ch], [HLYZ], [Sc]).

######  Lemma 2.1.

Let $K_{n}$ and $K$ be bounded convex bodies containing the origin for any
$n\in\mathbb{N}$ and such that $K_{n}\to K$ in the Hausdorff sense. For any
$n\in\\{0,1,2,...\\}$, let $h_{K_{n}},\rho_{K_{n}},r_{K_{n}}$ be the support
function, the radial function and the radial map of $K_{n}$, respectively.
Then the following statements hold

  * (i)

Let $h_{K}$ be the support function of $K$ then

$\sup_{\theta\in\mathbb{S}^{n-1}}|h_{K_{n}}(\theta)-h_{K}(\theta)|\to 0.$

  * (ii)

Let $\rho_{K}$ the radial function of $K$ then

$\sup_{x\in\mathbb{S}^{n-1}}|\rho_{K_{n}}(x)-\rho_{K}(x)|\to 0.$

  * (iii)

Let $x\in\partial K$ and $x_{n}\in\partial K_{n}$, $n\in\mathbb{N}$, points
where $\nu_{K}(x)$ and $\nu_{K_{n}}(x_{n})$ are well defined and such that

$\lim_{n\to\infty}x_{n}=x.$

Then

$\lim_{n\to\infty}\nu_{K_{n}}(x_{n})=\nu_{K}(x).$

By (2.6), Lemma 2.1 and the Lebesgue’s convergence Theorem we immediately get

######  Theorem 2.2.

Let $K_{n}$ and $K$ be bounded convex bodies containing the origin for any
$n\in\mathbb{N}$ and such that $K_{n}\to K$ in the Hausdorff sense. Let

$f_{n}\colon\partial K_{n}\to\mathbb{R},\quad f\colon\partial K\to\mathbb{R}$

be $\mathcal{H}^{n-1}$ measurable functions such that

  * (i)

there exists $C>0$ such that

$\|f\|_{L^{\infty}(\partial K)}\leq C,\quad\|f_{n}\|_{L^{\infty}(\partial
K_{n})}\leq C,\,\,\forall n\in\mathbb{N}$

  * (ii)

if $x_{n}\in\partial K_{n}$ is such that $\lim_{n\to\infty}x_{n}=x\in\partial
K$, $f_{n}$ is defined in $x_{n}$ and

$\lim_{n\to\infty}f_{n}(x_{n})=f(x).$

Then

$\lim_{n\to\infty}\displaystyle\int_{\partial
K_{n}}f_{n}(x)\,d\mathcal{H}^{n-1}=\displaystyle\int_{\partial
K}f(x)\,d\mathcal{H}^{n-1}.$

###  2.2. The Steklov-Dirichlet eigenvalue problem

Let $R_{1}>0$ and $\Omega_{0}\subset\mathbb{R}^{n}$ be an open bounded
connected set with Lipschitz boundary and such that
$B_{R_{1}}\Subset\Omega_{0}$, that means
$\overline{B}_{R_{1}}\subset\Omega_{0}$. Let us set
$\Omega:=\Omega_{0}\setminus\overline{B}_{R_{1}}$. We denote the set of
Sobolev functions on $\Omega$ vanishing on $\partial B_{R_{1}}$ by
$H^{1}_{\partial B_{R_{1}}}(\Omega)$, that is (see [ET]) the closure in
$H^{1}(\Omega)$ of the following set

$C^{\infty}_{\partial B_{R_{1}}}(\Omega):=\\{u_{|\Omega}\ |\ u\in
C_{0}^{\infty}(\mathbb{R}^{n}),\ \operatorname{spt}(u)\cap\partial
B_{R_{1}}=\emptyset\\}.$

Let us consider the following Steklov-Dirichlet eigenvalue problem in $\Omega$
for the Laplace operator

$\begin{cases}\Delta u=0&\mbox{in}\ \Omega\vspace{0.2cm}\\\ \dfrac{\partial
u}{\partial\nu}=\sigma(\Omega)u&\mbox{on}\ \partial\Omega_{0}\vspace{0.2cm}\\\
u=0&\mbox{on}\ \partial B_{R_{1}},\end{cases}$ (2.7)

where $\nu$ is the outer unit normal to $\partial\Omega_{0}$. The spectrum of
(2.7) is discrete and the eigenvalue can be ordered as follows

$0<\sigma_{1}(\Omega)\leq\sigma_{2}(\Omega)\leq...\ .$

In [PPS] the authors study the first eigenvalue of (2.7), which has the
following variational characterization

$\sigma_{1}(\Omega)=\min_{\begin{subarray}{c}v\in H^{1}_{\partial
B_{R_{1}}}(\Omega)\\\ v\not\equiv
0\end{subarray}}\dfrac{\displaystyle\int_{\Omega}|Dv|^{2}\;dx}{\displaystyle\int_{\partial\Omega_{0}}v^{2}\;d\mathcal{H}^{n-1}},$
(2.8)

and they also prove the following result.

######  Proposition 2.3.

Let $R_{1}>0$ and $\Omega_{0}\subset\mathbb{R}^{n}$ be an open bounded
connected set with Lipschitz boundary and such that
$B_{R_{1}}\Subset\Omega_{0}$ and let
$\Omega:=\Omega_{0}\setminus\overline{B}_{R_{1}}$. There exists a function
$u\in H^{1}_{\partial B_{R_{1}}}(\Omega)$ which achieves the maximum in (2.8)
and it is a weak solution to the problem (2.7). Moreover $\sigma_{1}(\Omega)$
is simple and the first eigenfunctions have constant sign in $\Omega$.

In [PPS] the authors also prove the following upper bound

$\sigma_{1}(\Omega)\leq C(n,R_{1},V(\Omega))V^{\frac{1}{n}}(\Omega),$ (2.9)

where

$C(n,R_{1},V(\Omega))=\dfrac{2}{n\omega_{n}^{\frac{1}{n}}\left(\left(\dfrac{V(\Omega)}{2\omega_{n}}+R_{1}^{n}\right)^{\frac{1}{n}}-R_{1}\right)^{2}}.$

As a consequence, the first Steklov-Dirichlet eigenvalue remains bounded from
above, when the volume of $\Omega$ is fixed. Obviously $\sigma_{1}(\Omega)$ is
bounded also when we fix the perimeter of $\Omega$, that is equivalent to fix
the perimeter of $\Omega_{0}$, instead of the volume. Indeed by (2.9) and the
isoperimetric inequality, we can deduce the following upper bound

$\sigma_{1}(\Omega)\leq\dfrac{2V^{\frac{1}{n}}(\Omega)}{n\omega_{n}^{\frac{1}{n}}\left(\left(\dfrac{V(\Omega)}{2\omega_{n}}+R_{1}^{n}\right)^{\frac{1}{n}}-R_{1}\right)^{2}}\leq
C(n)\dfrac{P^{\frac{1}{n-1}}(\Omega_{0})}{R_{1}^{2}},$ (2.10)

where $C(n)$ is a positive constants that depends only on the dimension
$n$.Finally, we recall the following scaling property for
$\sigma_{1}(\Omega)$:

$\sigma_{1}(t\Omega)=\frac{1}{t}\sigma_{1}(\Omega),\quad\forall t>0.$

###  2.3. The Steklov-Dirichlet eigenvalue problem in the spherical shell

In this Section we recall some known results about the first Steklov-Dirichlet
eigenvalue of the spherical shell. In this case, in [SV] the authors find the
explicit expression of the first eigenfunction.

######  Proposition 2.4.

Let $A_{R_{1},R_{2}}$ be the spherical shell with radii $R_{2}>R_{1}>0$. The
first eigenfunction associated to $\sigma_{1}(A_{R_{1},R_{2}})$, is a radially
symmetric, positive, strictly increasing function and it is given by

$w(r)=\begin{cases}\ln r-\ln R_{1}&{\rm for}\;\;n=2\vspace{0.1cm}\\\
\left(\dfrac{1}{R_{1}^{n-2}}-\dfrac{1}{r^{n-2}}\right)&{\rm for}\;\;n\geq
3\vspace{0.1cm},\end{cases}$ (2.11)

with $r=|x|$. The corresponding first Steklov-Dirichlet eigenvalue can be
computed and it is the following

$\sigma_{1}(A_{R_{1},R_{2}})=\begin{cases}\frac{1}{R_{2}\log\left(\frac{R_{2}}{R_{1}}\right)}&{\rm
for}\;\;n=2\vspace{0.1cm}\\\
\frac{n-2}{R_{2}\left[\left(\frac{R_{2}}{R_{1}}\right)^{n-2}-1\right]}&{\rm
for}\;\;n\geq 3\vspace{0.1cm}.\\\ \end{cases}$ (2.12)

######  Remark 2.5.

We point out that by (2.12), we have that $\sigma_{1}(A_{R_{1},R_{2}})$ is
increasing with respect to the internal radius $R_{1}$, that is

$\sigma_{1}(A_{R_{1},R_{2}})<\sigma_{1}(A_{r_{1},R_{2}}),\quad\text{if
}r_{1}>R_{1}$

Moreover it holds

$\lim_{R_{1}\to 0}\sigma_{1}(A_{R_{1},R_{2}})=0,$ (2.13)

that is $\sigma_{1}(A_{R_{1},R_{2}})$ tends to the first trivial Steklov
eigenvalue of the Laplacian for $R_{1}$ which goes to zero.

##  3\. The existence result

Inequality (2.9) ensures that the Steklov-Dirichlet eigenvalue
$\sigma_{1}(\Omega)$, defined in (1.1), is bounded from above if the volume of
$\Omega$ is fixed. In this section we prove the existence of a maximizer among
convex sets with fixed internal ball and fixed volume. Let $R_{1}>0$ and
$\omega>0$, then by $\mathcal{A}_{R_{1}}(\omega)$ we will denote the class of
convex sets having measure $\omega$ and containing the ball $B_{R_{1}}$, that
is

$\mathcal{A}_{R_{1}}(\omega):=\big{\\{}D=K\setminus\overline{B}_{R_{1}},\>\
K\subseteq\mathbb{R}^{n}\,\,\text{open, bounded, convex}:\ B_{R_{1}}\Subset
K,\ V(D)=\omega\big{\\}}.$

The main theorem of this section is the following existence result.

######  Theorem 3.1.

Let $\omega>0$ be fixed. There exists a set $E\in\mathcal{A}_{R_{1}}(\omega)$,
such that

$\max_{D\in\mathcal{A}_{R_{1}}(\omega)}\sigma_{1}(D)=\sigma_{1}(E).$

###### Proof.

The upper bound (2.9) implies that there exists $M>0$ such that

$\sup_{D\in\mathcal{A}_{R_{1}}(\omega)}\sigma_{1}(D)=M<+\infty.$

Hence, there exists a sequence
$\\{E_{k}\\}_{k\in\mathbb{N}}\subseteq\mathcal{A}_{R_{1}}(\omega)$ such that

$\lim_{k\to\infty}\sigma_{1}(E_{k})=M.$

In order to show the desired result, we need to prove the existence of a set
$E\in\mathcal{A}_{R_{1}}(\omega)$ such that
$E_{k}\stackrel{{\scriptstyle\mathcal{H}}}{{\longrightarrow}}E$ with
$\sigma_{1}(E)=M$. Firstly we prove that, up to a subsequence,
$\\{E_{k}\\}_{k\in\mathbb{N}}$ converges to a certain
$E\in\mathcal{A}_{R_{1}}(\omega)$ in the Hausdorff metric. Being
$\\{E_{k}\\}_{k\in\mathbb{N}}\subseteq\mathcal{A}_{R_{1}}(\omega)$ then, for
every $k\in\mathbb{N}$ there exists a convex set $E_{0,k}$, such that
$B_{R_{1}}\Subset E_{0,k}$,

$E_{k}=E_{0,k}\setminus\overline{B}_{R_{1}}$

and

$\omega_{0}:=V(E_{0,k})=\omega+\omega_{n}R_{1}^{n}.$

By the Blaschke selection Theorem and the continuity of the volume functional
with respect to the Hausdorff measure (see [Sc] as a reference), it is enough
to show that $\\{E_{0,k}\\}_{k\in\mathbb{N}}$ is equibounded. We proceed by
contradiction assuming that

$\lim_{k\to+\infty}{\rm diam}(E_{0,k})=+\infty.$ (3.1)

Inequality (2.1) gives

$\rho(E_{0,k})\leq\dfrac{nV(E_{0,k})}{P(E_{0,k})},$ (3.2)

where $\rho(E_{0,k})$ is the inradius of $E_{0,k}$ defined in (2.2). The
assumption (3.1) implies that the right-hand side in (3.2) tends to $0$ as
$k\to+\infty$, being $V(E_{0,k})$ fixed. Therefore, by (3.2), we have

$\lim_{k\to+\infty}\rho(E_{0,k})=0,$

which is in contradiction with

$0<R_{1}<\rho(E_{0,k}).$

Hence, the equiboundeness is proved and then $\\{E_{k}\\}_{k\in\mathbb{N}}$
converges up to a subsequence to a set $E\in\mathcal{A}_{R_{1}}(\omega)$ in
the Hausdorff metric. Hence, by the definition of
$\mathcal{A}_{R_{1}}(\omega)$, there exists an open bounded convex set $E_{0}$
such that $E=E_{0}\setminus\overline{B}_{R_{1}}$. In order to complete the
proof, we will prove that

$M=\lim_{k}\sigma_{1}(E_{k})\leq\sigma_{1}(E).$ (3.3)

Let $u\in H^{1}_{\partial B_{R_{1}}}(E)$ be the first positive eigenfunction
associated to $\sigma_{1}(E)$, such that

$\int_{\partial E_{0}}u^{2}\;d\mathcal{H}^{n-1}=1.$

Hence, we have

$\sigma_{1}(E)=\displaystyle\int_{E}|\nabla u|^{2}\;dx.$

By the extension theorem (see for instance [C, St] for Lipschitz domains), we
can extend $u$ in $\mathbb{R}^{n}$ obtaining a function $\tilde{u}\in
H^{1}_{\partial B_{R_{1}}}\left(\mathbb{R}^{n}\right)$ such that
$\tilde{u}=u$, a.e. in $E$, and

$\|\tilde{u}\|_{H^{1}_{\partial B_{R_{1}}}\left(\mathbb{R}^{n}\right)}\leq
c(n)\|u\|_{H^{1}_{\partial B_{R_{1}}}\left(E\right)},$

for some positive constant $c=c(n)$. For every $k\in\mathbb{N}$ we define
$u_{k}$ as the restriction of $\tilde{u}$ in $E_{k}$. Using $u_{k}$ as a test
function for $\sigma_{1}(E_{k})$, we have

$\sigma_{1}(E_{k})\leq\dfrac{\displaystyle\int_{E_{k}}|\nabla\tilde{u}|^{2}\;dx}{\displaystyle\int_{\partial
E_{0,k}}\tilde{u}^{2}\;d\mathcal{H}^{n-1}}.$ (3.4)

In order to get (3.3), we prove that the right-hand side in (3.4) converges to
$\sigma_{1}(E)$. We observe that

$\displaystyle\int_{E_{k}}|\nabla\tilde{u}|^{2}\;dx-\displaystyle\int_{E}|\nabla\tilde{u}|^{2}\;dx=\displaystyle\int_{\mathbb{R}^{n}}\left(\chi_{E_{k}}\;-\chi_{E}\right)|\nabla\tilde{u}|^{2}\;dx\to
0,$ (3.5)

since $E_{k}\to E$ in the Hausdorff metric and by the dominate convergence
theorem. In order to conclude the proof we have to prove

$\int_{\partial E_{0,k}}\tilde{u}^{2}\,d\mathcal{H}^{n-1}\to\int_{\partial
E_{0}}u^{2}\,d\mathcal{H}^{n-1}=1.$ (3.6)

The equiboundedness of the sequence $\\{E_{0,k}\\}_{k\in\mathbb{N}}$
guarantees the existence of a ball $B_{R}$ centered at the origin with radius
$R>0$ such that $E_{0,k}\subset B_{R}$, for every $k\in\mathbb{N}$. Extending
$\tilde{u}$ to zero in $B_{R_{1}}$ and by using an approximation argument, we
can suppose that $\tilde{u}\in C^{\infty}(B_{R})$. Then (3.6) follows by
Theorem 2.2. Finally, passing to the limit in (3.4), by (3.5) and (3.6), we
get (3.3), that is

$M\leq\sigma_{1}(E)$

and, consequently, we can conclude that

$\sigma_{1}(E)=M,$

obtaining the desired claim. ∎

##  4\. Proof of the main result

In this section we give the proof of the main result. The idea is to take as
test function in the quotient (2.8) the eigenfunction of the spherical shell
with the same measure as $\Omega$. Before giving the proof, we need a
preliminary result.

######  Lemma 4.1.

Let $R_{1}>0$ and let $f$ be the function defined in $]0,+\infty[$ as

$f(t)=\begin{cases}\log^{2}\left(\frac{\sqrt{t}}{R_{1}}\right)\sqrt{t}&n=2\\\
\big{(}\frac{1}{R_{1}^{n-2}}-\frac{1}{t^{\frac{n-2}{n}}}\big{)}t^{\frac{n-1}{n}}&n\geq
3.\end{cases}$

Then, $f$ is convex for every $\alpha_{-}(n)R_{1}^{n}\leq
t\leq\alpha_{+}(n)R_{1}^{n}$, where

$\alpha_{\pm}(n)=\begin{cases}e^{\pm 2\sqrt{2}}&n=2\\\
\bigg{[}\frac{(n-1)\pm(n-2)\sqrt{2(n-1)}}{n-1}\bigg{]}^{\frac{n}{n-2}}&n\geq
3.\end{cases}$

###### Proof.

Let us begin with the bidimensional case. After an easy computation one can
see that

$f^{\prime\prime}(t)=\frac{2-\log^{2}(\sqrt{t}/R_{1})}{4t\sqrt{t}},$

which gives immediately the conclusion. Now let us consider $n\geq 3$. After
some computations the second derivative of the function is the following

$f^{\prime\prime}(t)=t^{\frac{3}{n}-3}\bigg{[}\frac{R_{1}^{4-2n}}{n}\bigg{(}\frac{1}{n}-1\bigg{)}t^{2-\frac{4}{n}}+\frac{2R_{1}^{2-n}}{n}\bigg{(}1-\frac{1}{n}\bigg{)}t^{1-\frac{2}{n}}+\bigg{(}\frac{3}{n}-2\bigg{)}\bigg{(}\frac{3}{n}-1\bigg{)}\bigg{]}.$

If we call $y=t^{1-\frac{2}{n}}$, the previous function is non-negative if and
only if

$g(y)=\frac{R_{1}^{4-2n}}{n}\bigg{(}\frac{1}{n}-1\bigg{)}y^{2}+\frac{2R_{1}^{2-n}}{n}\bigg{(}1-\frac{1}{n}\bigg{)}y+\bigg{(}\frac{3}{n}-2\bigg{)}\bigg{(}\frac{3}{n}-1\bigg{)}\geq
0.$

It is not difficult to check that the zeros of $g(y)$ are

$y_{\pm}=R_{1}^{n-2}\,\frac{n-1\pm(n-2)\sqrt{2(n-1)}}{n-1}.$

Being $y_{-}=0$ for $n=3$ and $y_{-}<0$ for every $n\geq 4$ it must be
$y_{-}\leq y\leq y_{+}$, which concludes the proof. ∎

Now we can prove the main result.

###### Proof of the Theorem 1.1.

Let us consider the fundamental solution $w$, given in (2.11), as a test
function in (2.8). Then,

$\sigma_{1}(\Omega)\leq\displaystyle\frac{\displaystyle\int_{\Omega}|\nabla
w|^{2}\,dx}{\displaystyle\int_{\partial\Omega_{0}}w^{2}\,d\mathcal{H}^{n-1}}.$

In order to prove the result we will show that

$\frac{\displaystyle\int_{\Omega}|\nabla
w|^{2}\,dx}{\displaystyle\int_{\partial\Omega_{0}}w^{2}\,d\mathcal{H}^{n-1}}\leq\frac{\displaystyle\int_{A_{R_{1},R_{2}}}|\nabla
w|^{2}\,dx}{\displaystyle\int_{\partial
B_{R_{2}}}w^{2}\,d\mathcal{H}^{n-1}}=\sigma_{1}(A_{R_{1},R_{2}}).$ (4.1)

Since $|\nabla w|^{2}$ is a non-negative symmetric decreasing function for any
$n\geq 2$, by the Hardy-Littlewood inequality we have

$\begin{split}\int_{\Omega}|\nabla w|^{2}\ dx&=\int_{\Omega_{0}}|\nabla
w|^{2}\ dx-\int_{B_{R_{1}}}|\nabla w|^{2}\ dx\\\ &\leq\int_{B_{R_{2}}}|\nabla
w|^{2}\ dx-\int_{B_{R_{1}}}|\nabla w|^{2}\ dx=\int_{A_{R_{1},R_{2}}}|\nabla
w|^{2}\ dx.\end{split}$ (4.2)

Hence, it remains to prove the following inequality

$\int_{\partial\Omega_{0}}w^{2}\,d\mathcal{H}^{n-1}\geq\int_{\partial
B_{R_{2}}}w^{2}\,d\mathcal{H}^{n-1}.$ (4.3)

Let $\rho_{0}$ be the radial function of $\Omega_{0}$ defined in (2.3). By
(2.5), $\partial\Omega_{0}$ can be represented as follows

$\partial\Omega_{0}=\\{x\,\rho_{0}(x),x\in\mathbb{S}^{n-1}\\},$

with $R_{1}<\rho_{0}(\theta)\leq\bar{R}$ and $\bar{R}$ defined in (1.3).
Firstly, let us consider the case $n=2$. If we denote by
$z(\theta)=\rho_{0}^{2}(\theta)$, being $V(\Omega_{0})=V(B_{R_{2}})$, it holds

$R_{2}=\displaystyle\sqrt{\frac{1}{2\pi}\int_{0}^{2\pi}z(\theta)d\theta}.$
(4.4)

Moreover, we get

$\begin{split}\int_{\partial\Omega_{0}}w^{2}\,ds=&\int_{\partial\Omega_{0}}\left(\log(|x|-\log
R_{1})\right)^{2}\
ds=\int_{0}^{2\pi}\log^{2}\left(\frac{\rho_{0}(\theta)}{R_{1}}\right)\rho_{0}(\theta)\sqrt{1+\left(\frac{\rho_{0}^{\prime}(\theta)}{\rho_{0}(\theta)}\right)^{2}}\
d\theta\\\
&\geq\int_{0}^{2\pi}\log^{2}\left(\frac{\rho_{0}(\theta)}{R_{1}}\right)\rho_{0}(\theta)\
d\theta=\int_{0}^{2\pi}\log^{2}\left(\frac{\sqrt{z(\theta)}}{R_{1}}\right)\sqrt{z(\theta)}\
d\theta\\\ &\geq
2\pi\log^{2}\left(\frac{1}{R_{1}}\sqrt{\frac{\int_{0}^{2\pi}z(\theta)d\theta}{2\pi}}\right)\sqrt{\frac{\int_{0}^{2\pi}z(\theta)d\theta}{2\pi}}=\\\
&=2\pi R_{2}\log^{2}\left(\frac{R_{2}}{R_{1}}\right)=\int_{\partial
B_{R_{2}}}w^{2}\,ds,\end{split}$ (4.5)

where, since $\rho_{0}(\theta)\leq\bar{R}$, the last inequality follows by
(4.1) and by Jensen’s inequality. This conclude the proof of (4.3) in the
bidimensional case. Now, let us consider the case $n\geq 3$ and we proceed in
a similar way. Moreover since

$V(\Omega_{0})=\frac{1}{n}\int_{\mathbb{S}^{n-1}}\rho_{0}^{n}(x)\,d\mathcal{H}^{n-1}$

and being $V(\Omega_{0})=V(B_{R_{2}})$, it holds

$R_{2}=\displaystyle\left(\frac{1}{n\omega_{n}}\int_{\mathbb{S}^{n-1}}z(x)\,d\mathcal{H}^{n-1}\right)^{\frac{1}{n}},$
(4.6)

where $z(x)=\rho_{0}^{n}(x)$. Then, we have

$\begin{split}\int_{\partial\Omega_{0}}w^{2}\,d\mathcal{H}^{n-1}=&\int_{\partial\Omega_{0}}\bigg{(}\frac{1}{R_{1}^{n-2}}-\frac{1}{|x|^{n-2}}\bigg{)}^{2}\,d\mathcal{H}^{n-1}\\\
&=\int_{\mathbb{S}^{n-1}}\bigg{(}\frac{1}{R_{1}^{n-2}}-\frac{1}{(\rho_{0}(x))^{n-2}}\bigg{)}^{2}(\rho_{0}(x))^{n-1}\sqrt{1+\left(\frac{\nabla_{\tau}\rho_{0}(x)}{\rho_{0}(x)}\right)^{2}}\,d\mathcal{H}^{n-1}\\\
&\geq\int_{\mathbb{S}^{n-1}}\bigg{(}\frac{1}{R_{1}^{n-2}}-\frac{1}{(z(x))^{\frac{n-2}{n}}}\bigg{)}^{2}(z(x))^{\frac{n-1}{n}}\,d\mathcal{H}^{n-1}\\\
&\geq
n\omega_{n}\bigg{[}\frac{1}{R_{1}^{n-2}}-\frac{n\omega_{n}}{\left(\int_{\mathbb{S}^{n-1}}z(x)\,d\mathcal{H}^{n-1}\right)^{\frac{n-2}{n}}}\bigg{]}^{2}\bigg{(}\frac{\int_{\mathbb{S}^{n-1}}z(x)\,d\mathcal{H}^{n-1}}{n\omega_{n}}\bigg{)}^{\frac{n-1}{n}}\\\
&=n\omega_{n}\bigg{(}\frac{1}{R_{1}^{n-2}}-\frac{1}{R_{2}^{n-2}}\bigg{)}^{2}R_{2}^{n-1}=\int_{\partial
B_{R_{2}}}w^{2}\,d\mathcal{H}^{n-1}.\end{split}$

where last inequality follows by Lemma 4.1 and by Jensen’s inequality, being
$\rho_{0}(x)\leq\bar{R}$. This gives (4.3) for $n\geq 3$ and concludes the
proof. ∎

##  5\. Some remarks about the perimeter constraint

The estimate (2.10) states that the first Steklov-Dirichlet eigenvalue is
bounded from above also when we keep the outer perimeter and the inner radius
fixed. So, it is natural to investigate if there exists a set which maximizes
$\sigma_{1}(\Omega)$ in the following class

$\mathcal{B}_{R_{1}}(\kappa):=\big{\\{}D=K\setminus\overline{B}_{R_{1}}\ ,\
K\subset\mathbb{R}^{n},\,\,\text{open, convex}:\ B_{R_{1}}\Subset K,\
P(K)=\kappa\big{\\}},$

where $R_{1}>0$ and $\kappa>n\omega_{n}R_{1}^{n-1}.$ Arguing as Theorem 3.1,
we obtain the following existence result under a perimeter constraint.

######  Theorem 5.1.

Let $\kappa>n\omega_{n}R_{1}^{n-1}$ be fixed. There exists a set
$\Omega\in\mathcal{B}_{R_{1}}(\kappa)$ such that

$\sup_{D\in\mathcal{B}_{R_{1}}(\kappa)}\sigma_{1}(D)=\sigma_{1}(\Omega).$

######  Remark 5.2.

We stress that inequality (4.2) continues to hold true even if we fix the
perimeter of $\Omega_{0}$. Indeed the isoperimetric inequality ensures that
the ball $B_{R_{2}}$ centered at the origin and having the same measure than
$\Omega_{0}$ is contained in the ball centered at the origin and having the
same perimeter than $\Omega_{0}$. On the other hand we cannot prove, instead,
the inequality (4.3) under the perimeter constraint in order to obtain that
the spherical shell is still a maximum for $\sigma_{1}(\Omega)$. Indeed, if we
proceed as in the proof of Theorem 1.1, for instance in the planar case,
equation (4.4) has to be replaced by the following inequality:

$2\pi
R_{2}=P(B_{R_{2}})=P(\Omega_{0})=\int_{0}^{2\pi}\rho_{0}(\theta)\sqrt{1+\left(\frac{\rho_{0}^{\prime}(\theta)}{\rho_{0}(\theta)}\right)^{2}}\,d\theta\geq\int_{0}^{2\pi}\rho_{0}(\theta)\,d\theta.$
(5.1)

Then, in the last step of (LABEL:last), after using Jensen’s inequality, we do
not obtain the first Steklov-Dirichlet eigenvalue of the spherical shell,
since (5.1) is not an equality. In support of this fact, we give the following
numerical counterexample obtained by using ™WxMaxima. We consider
$R_{1}=10^{-5}$ and $\Omega_{0}$ an ellipse with the same perimeter as
$A_{R_{1},1}$. Let $a$ and $b$ the semi-axes of the ellipse. In order to
compute the integral over the ellipse, we used the formula
$P(\Omega_{0})=2\pi\sqrt{\frac{{a^{2}+b^{2}}}{2}}$, which is an approximation
by excess for the perimeter of the ellipse. Here we have chosen $b=1.1$. We
obtain

$D(\Omega_{0})\approx 832,820208>828,919156\approx D(A_{R_{1},1}),$

where $D(\Omega_{0})=\int_{\partial\Omega_{0}}w^{2}ds$ and $w$ is the
fundamental solution defined in (2.11). This means that we cannot study
separately the numerator and denominator terms to obtain inequality (1.4)
under perimeter constraint.

## Acknowledgements

This work has been partially supported by the MiUR-PRIN 2017 grant
“Qualitative and quantitative aspects of nonlinear PDEs”, by GNAMPA of INdAM,
by the MiUR-Dipartimenti di Eccellenza 2018-2022 grant “Sistemi distribuiti
intelligenti”of Dipartimento di Ingegneria Elettrica e dell’Informazione “M.
Scarano”, by the MiSE grant “SUMMa: Smart Urban Mobility Management.

## References

  * [A] M. S. Agranovich, On a mixed Poincaré-Steklov type spectral problem in a Lipschitz domain. Russ. J. Math. Phys. 13.3 (2006): 239-244.
  * [AFP] L. Ambrosio, N. Fusco, D. Pallara, Functions of bounded variation and free discontinuity problems, The Clarendon Press, Oxford University Press, New York (2000).
  * [BDPR] L. Brasco, G. De Philippis, B. Ruffini, Spectral optimization for the Stekloff-Laplacian: the stability issue. J. Funct. Anal. 262.11 (2012): 4675-4710.
  * [B] F. Brock, An isoperimetric inequality for eigenvalues of the Stekloff problem. ZAMM Z. Angew. Math. Mech. 81.1 (2001): 69- 71\.
  * [BFNT] D. Bucur, V. Ferone, C. Nitsch, C. Trombetti, C. Weinstock inequality in higher dimensions. Journal of Differential Geometry (to appear): 1-16.
  * [BN] D. Bucur, M. Nahon Stability and instability issues of the Weinstock inequality. Trans. Amer. Math. Soc., 374 (2021): 2201-2223.
  * [C] A. Calderon, Lebesgue Spaces Of Differentiable Functions And Distributions Partial Differential Equations: Proc. Symp. Pure Math. 4 (1961): 33-49.
  * [Ch] Z. Chen, The Lp Minkowski problem for $q$-capacity. Proceedings of the Royal Society of Edinburgh Section A: Mathematics (to appear): 1-31.
  * [DPBG] F. Della Pietra, G. di Blasio, N. Gavitone, Sharp estimates on the first Dirichlet eigenvalue of nonlinear elliptic operators via maximum principle. Advances in Nonlinear Analysis 9.1 (2018): 278-291.
  * [D] B. Dittmar, Eigenvalue problems and conformal mapping, Handbook of complex analysis: geometric function theory. Vol. 2, Elsevier Sci. B. V., Amsterdam (2005): 669-686.
  * [E] L.C. Evans, Partial Differential Equations, American Mathematical Society, Providence, R.I., (2010).
  * [ET] M. Egert, P. Tolksdorf, Characterizations of Sobolev functions that vanish on a part of the boundary. Discrete Contin. Dyn. Syst. Ser. S 10.4 (2017): 729-743.
  * [F] I. Ftouhi, Where to place a spherical obstacle so as to maximize the first Steklov eigenvalue. Hal (2019): 1-19.
  * [GLMPT] N. Gavitone, D. A. La Manna, G. Paoli, L. Trani, A quantitative Weinstock inequality for convex sets. Calc. Var. Partial Differential Equations 59.1 (2020): 20 pp.
  * [HP] J. Hersch, L. E. Payne, Extremal principles and isoperimetric inequalities for some mixed problems of Stekloff’s type. Zeitschrift für angewandte Mathematik und Physik ZAMP 19.5 (1968): 802-817.
  * [HLS] J. Hong, M. Lim, D.-H. Seo, Shape monotonicity of the first Steklov- Dirichlet eigenvalue on eccentric annuli. ArXiv (2020): 1-29.
  * [HLYZ] Y. Huang, E. Lutwak, D. Yang, G. Zhang, Geometric measures in the dual Brunn-Minkowski theory and their associated Minkowski problems. Acta Mathematica, 216.2 (2016): 325-388.
  * [M] F. Maggi, Sets of Finite Perimeter and Geometric Variational Problems: An Introduction to Geometric Measure Theory, Cambridge Studies in Advanced Mathematics (2012).
  * [P] B. V. E. Pal’tsev, Mixed problems with non-homogeneous boundary conditions in Lipschitz domains for second-order elliptic equations with a parameter. Sbornik: Mathematics, 187.4 (1996): 525.
  * [PPS] G. Paoli, G. Piscitelli, R. Sannipoli, A stability result for the Steklov Laplacian Eigenvalue Problem with a spherical obstacle. Comm. Pure Appl. Anal. 20.1 (2021): 145-158.
  * [SV] G. Santhanam, S. Verma, On eigenvalue problems related to the laplacian in a class of doubly connected domains. Monatshefte für Mathematik 193 (2020): 879-899.
  * [Sc] R. Schneider, Convex bodies: the Brunn-Minkowski theory. Second expanded edition, Encyclopedia of Mathematics and its Applications 151, Cambridge University Press, Cambridge (2014).
  * [St] E. M. Stein, Singular integrals and differentiability properties of functions (Vol. 2). Princeton university press (1970).
  * [W1] R. Weinstock, Inequalities for a Classical Eigenvalue Problem. Department of Math., Stanford Univ., Tech. Rep. 37 (1954).
  * [W2] R. Weinstock, Inequalities for a classical eigenvalue problem. J. Rational Mech. Anal. 3 (1954): 745-753.


# E-CIR: Event-Enhanced Continuous Intensity Recovery

Chen Song Qixing Huang Chandrajit Bajaj
The University of Texas at Austin
{song, bajaj<EMAIL_ADDRESS>
###### Abstract

A camera begins to sense light the moment we press the shutter button. During
the exposure interval, relative motion between the scene and the camera causes
motion blur, a common undesirable visual artifact. This paper presents E-CIR,
which converts a blurry image into a sharp video represented as a parametric
function from time to intensity. E-CIR leverages events as an auxiliary input.
We discuss how to exploit the temporal event structure to construct the
parametric bases. We demonstrate how to train a deep learning model to predict
the function coefficients. To improve the appearance consistency, we further
introduce a refinement module to propagate visual features among consecutive
frames. Compared to state-of-the-art event-enhanced deblurring approaches,
E-CIR generates smoother and more realistic results. The implementation of
E-CIR is available at https://github.com/chensong1995/E-CIR.

##  1 Introduction

The shutter speed, or the length of the exposure interval, controls how much
light reaches the image sensor from the environment. If the exposure interval
is too short, the camera only has the time to capture very few photons.
Consequently, the resulting image is not only unilluminated but also lacks
fine details. On the other hand, if the exposure interval is too long, the
relative motion between the scene and the camera may potentially be very
significant. The resulting image is then the temporal average of a moving
trajectory, causing blurry artifacts. Traditionally, it is presumed that any
motion during the exposure interval, including both camera shake and subject
movement, is unwanted and should therefore be removed. Over the past several
decades, researchers have studied extensively how to convert a blurry image
into a sharp one [33, 6, 13, 10, 17, 11, 35, 5, 43, 42, 28, 1, 14, 15]. It is
only until recently when several works that reconstruct the complete motion
trajectory have received profound attention [9, 29]. These works introduce
algorithms that convert a blurry image into a sharp video describing the exact
movement that causes the blurry artifact.

Figure 1: Problem Description. In this imaginary scene, we place a white
square along the edge of a black disk. The image taken by the conventional
camera is blurry because the disk rotates at a fast speed. It is as if the
perimeter of the disk somehow grows into a gray collar. During the exposure
interval, the event sensor produces a spiral of events. Our approach takes the
blurry frame and the events as input and produces a sharp video sequence as
output. The output video explains the motion blur by entailing the complete
motion trajectory of the rotating disk.

Sharp video reconstruction is an ill-posed problem because there are
infinitely many motion trajectories whose temporal averages correspond to the
same blurry frame. To compensate for the ambiguity, previous works [26, 25, 8,
21, 40, 44, 36, 7, 41] exploit event data as an auxiliary input, which
provides additional information during the exposure interval at a finer
temporal resolution, as shown in Figure 1. Even with the event input,
difficult challenges remain. The events fail to capture the complete motion
information. The video reconstruction quality is determined not only by the
appearance of each individual frame but also the temporal smoothness. The
immense density of events creates another obstacle for effective and efficient
processing. The success of video deblurring depends on how the blurry image,
the events, and priors about video sequences are integrated together. This
calls for suitable video representations and prediction algorithms.

This paper makes fundamental contributions in video representations and
methodologies for recovering accurate and temporally consistent videos.
Specifically, we propose a continuous video representation whose coefficients
are highly interpretable and easy to learn, due to their strong correlation to
the events. For every pixel $(x,y)$, we represent its intensity as a
parametric polynomial function $\mathbf{L}_{xy}(t)$, allowing us to render the
sharp image at any given timestamp $t$ during the exposure interval. We show
how to choose the polynomial bases such that the derivative of
$\mathbf{L}_{xy}(t)$ interpolates the significant intensity changes. We also
demonstrate how to train a deep neural network that regresses the polynomial
coefficients. Instead of processing the video as a volume and implicitly
encoding motions in convolutional filters, our approach explicitly asks the
model to elaborate the motions that have already been described by the events.
To further polish the frame quality, we introduce a refinement module that
propagates the visual features among consecutive frames, which can be trained
in an end-to-end manner with the rest of the model. The proposed regress-and-
refine paradigm nicely combines the strength of recurrent modules for
enforcing temporal smoothness and the strength of regression for drifting
avoidance.

We quantitatively evaluate our method on the synthetic REDS dataset [22]. In
terms of reconstruction quality, E-CIR achieves an MSE of 0.114, representing
a 37.4% improvement from state-of-the-art algorithms. We also present a
qualitative evaluation on the real captures provided by Pan et al. [26].
Compared with baseline approaches, our method is less noisy, more realistic,
and temporally smoother.

In summary, our key contributions are:

  1. 1.

We represent a video by per-pixel parametric polynomials. We discuss why this
representation integrates easily with the event mechanism by showing the
parallelism between function derivatives and events.

  2. 2.

From a blurry image and its associated events in the exposure interval, we
demonstrate how to use a deep learning model to predict a sharp video
represented by the proposed parametric polynomials.

  3. 3.

To overcome the limitations of the polynomial representation, we discuss how
to formulate a refinement objective and encourage the temporal propagation of
sharp visual features.

  4. 4.

We provide source code and documentation for converting the original REDS
dataset into the event format. This clears the vagueness of the evaluation
dataset in previous works and establishes an open-source benchmark for future
comparisons.

##  2 Related Work

###  2.1 Event-Enhanced Deblurring

First commercialized in 2006 [19], event cameras are an emerging type of
vision sensor that models the environment evolution as intensity changes and
represents the scene as events. Each event is a 4-tuple $(x,y,t,p)$ that
contains the location, time, and polarity of an intensity change. This simple
representation allows event cameras to support a fast data rate (up to 1 MHz),
orders of magnitude higher than the frame rate of conventional cameras. The
density of events during the exposure interval provides valuable motion
information to explain the blurred image.

Pan et al. propose the Event-based Double Integral (EDI) model [26, 25] that
analytically reconstructs a high frame-rate sharp video from a blurry frame
and its associated events. Jiang et al. [8] formulate a Maximum-a-Posteriori
problem and solve for the latent sharp images under the Markov assumption with
the help of deep neural networks. Lin et al. [21] believe it is inadequate to
calculate the intensity residual between sharp and blurry frames directly from
the event threshold and propose to predict the intensity residual using deep
learning. Meanwhile, the structural similarity between the EDI model and the
blur kernel formulation has inspired Wang et al. [40] to represent sharp
images as sparse codes in a learnable dictionary and optimize them using an
iterative network. Shang et al. [36] assume that the input sequence contains a
mixture of blurry and sharp frames and propose to wrap sharp frames to deblur
the blurry frame. Zhang et al. [44] emphasize the temporal correlation among
consecutive frames and design a multi-patch convolutional LSTM to exploit such
correlation. Han et al. [7] extend this idea by modeling the intensity
residual between neighboring sharp frames. Xu et al. [41] also identify the
importance of temporal correlation and propose to utilize the optical flow
estimation instead.

Closely related to deblurring, event-enhanced frame interpolation has also
attracted increasing attention [39, 24]. While both tasks aim at constructing
a high frame-rate video, frame interpolation methods typically assume the
input frames are free of motion blur. Several works have attempted to
reconstruct a high frame-rate video directly from the events without the
conventional frame input [31, 32, 4] as well. However, these event-only
methods are less robust than their dual-input counterparts [44, 21].

###  2.2 Video Representation

To the best of our knowledge, most existing works in computer vision process
videos as discrete collections of frames. The only exception is Vid-ODE [27],
which represent videos by continuous latent states. The latent state can be
evaluated at any given timestamp, allowing the video to be rendered with an
infinitely high frame rate.

With the help of per-pixel parametric polynomials, our proposed representation
also supports infinitely high rate rendering and enjoys two additional
advantages. First, the polynomial bases are chosen to closely mimic the event
mechanism, which makes the algorithm robust to domain differences between
synthetic training data and real testing data. Second, the polynomial
coefficients are more interpretable than the latent code hidden inside a deep
network. This allows humans to easily explain and debug the model.

##  3 Preliminaries

###  3.1 Event Camera Model

Let $\mathbf{L}_{xy}(t)$ be the latent intensity of pixel $(x,y)$ at time $t$.
In the natural logarithmic space, the temporal contrast between
$t_{\text{ref}}$ and $t$ is given by [19]:

$\Delta
ln[\mathbf{L}_{xy}(t)]=ln[\mathbf{L}_{xy}(t)]-ln[\mathbf{L}_{xy}(t_{\text{ref}})]$
(1)

where $t_{\text{ref}}$ denotes the timestamp of the last event associated with
pixel $(x,y)$. The magnitude of $\Delta ln[\mathbf{L}_{xy}(t)]$ determines
whether the hardware produces an event. Let $(x,y,t,p)$ denote an event, where
$p\in\\{-1,+1\\}$ is the polarity of the intensity change [19]:

$p=\begin{cases}+1&\Delta ln[\mathbf{L}_{xy}(t)]\geq c^{+}\\\ 0\text{ (no
event)}&c^{-}<\Delta ln[\mathbf{L}_{xy}(t)]<c^{+}\\\ -1&\Delta
ln[\mathbf{L}_{xy}(t)]\leq c^{-}\end{cases}$ (2)

Here, $c^{+}$ and $c^{-}$ are thresholds controlling the sensitivity of
positive and negative events, respectively. It is commonly assumed that
$c^{+}$ and $c^{-}$ are stochastic variables [20, 30].

During an exposure interval $[-\frac{T}{2},\frac{T}{2}]$ with length $T$, let
$\mathbf{B}_{h\times w}=\\{\mathbf{B}_{xy}\\}$ be the blurry output from the
conventional camera. The relation between the blurry frame and the latent
frames is given by temporal averaging [30, 26]:

$\mathbf{B}_{xy}=\frac{1}{T}\int_{-\frac{T}{2}}^{\frac{T}{2}}\mathbf{L}_{xy}(t)dt$
(3)

###  3.2 Task Description

The input to the task has two components:

  1. 1.

The blurry intensity $\mathbf{B}_{xy}$ for all pixels $(x,y)$;

  2. 2.

A collection of events during the exposure interval
$\\{e_{i}=(x_{i},y_{i},t_{i},p_{i})|-\frac{T}{2}\leq t_{i}\leq\frac{T}{2}\\}$.

Given an arbitrary timestamp $t\in[-\frac{T}{2},\frac{T}{2}]$, the goal of the
task is to construct the corresponding latent frame $\mathbf{L}_{h\times
w}(t)=\\{\mathbf{L}_{xy}(t)\\}$.

###  3.3 Challenges

Intensity reconstruction is a highly ill-posed problem because there are
infinitely many motion trajectories whose temporal averages correspond to the
same blurry frame. Specifically, we face three challenges:

First, the events fails to capture complete motion information during the
exposure interval. Equation (2) states that when an event happens, the
magnitude of $\Delta ln[\mathbf{L}_{xy}(t)]$ is greater than the event
threshold. It remains unclear exactly how much $\Delta ln[\mathbf{L}_{xy}(t)]$
exceeds the threshold. Imagine a scene with two edges moving in the same
pattern. The first edge has a strong contrast to the background and generates
significant intensity changes for its movement. The second edge has a weak
contrast to the background and generates small intensity changes. Suppose
these two sets of intensity changes both exceed the event threshold. This
means the number of events generated by the camera is determined only by the
edge length. These two edges will yield the same number of events, as long as
their lengths are equal, even though the absolute change in intensity of the
first edge is several times higher than that of the second edge.

Second, the reconstruction quality is determined not only by the appearance of
each individual frame but also the temporal smoothness. A naive model that
independently optimizes each latent frame’s quality may lead to unrealistic
motion trajectories and cause frequent jitter. We refer readers to the
supplementary animations for a demonstration of how some of our baseline
approaches fail to solve this issue.

Third, the event format is incompatible with popular deep learning models. One
possible remedy is to aggregate the events into a histogram.This method
ignores the event camera model and trains a network as if the inputs are
merely some unexplained features. It remains unclear how to properly instill
human knowledge about the bio-inspired event mechanism, such as the
correlation between an event and an intensity change, into the model design.

While previous works fail to address some or all of these challenges, the next
section discusses how the proposed E-CIR handles them effectively.

##  4 Method

###  4.1 Parametric Intensity Function

(a) |  (b)
---|---

Figure 2: Keypoint Selection Algorithm. (a) Imagine there are $n=5$ different
events scattered across the exposure interval represented by blue circles. We
first sample $n$ evenly spaced pivots from the exposure interval
$[-\frac{T}{2},\frac{T}{2}]$ (the orange crosses). We then shift the pivots to
their closest events and obtain $n$ keypoints (green triangles). This
selection scheme not only caters to the temporal event structure but also
provides support to regions not covered by events. (b) At each keypoint,
exactly one of the Lagrange bases has a value of 1, while all other Lagrange
bases have a value of 0. We use a neural network to predict the value of
$\frac{d\mathbf{L}_{xy}}{dt}(t)$ at these $n$ keypoints. Their interpolation
gives a degree-$(n-1)$ polynomial, which is used to recover the primitive
intensity signal $\mathbf{L}_{xy}(t)$ through indefinite integral. Under the
Lagrangian representation, the polynomial coefficients coincide with predicted
derivatives. Figure 3: The overall pipeline. We use the U-Net [34] model to
regress the polynomial coefficients. The network takes three inputs: the
blurry frame $\mathbf{B}$, the keypoint timestamps $\mathbf{K}$, and the event
histogram $\mathbf{E}$. The network then outputs the intensity derivatives
$\frac{d\mathbf{L}}{dt}$. Given an arbitrary timestamp $t$, the integrator
follows Equation (7) and calculates the initial frame reconstruction
$\mathbf{\hat{L}}(t)$. The refinement module takes $\mathbf{\hat{L}}(t)$ and
$\mathbf{E}$ as input and outputs the final frame reconstruction
$\mathbf{L}(t)$. In this figure, $m,n,d,h,w$ represent the number of histogram
bins, the number of keypoints, the number of frames in the output video, the
frame height, and the frame width, respectively.

For each pixel $(x,y)$, we propose to approximate the function
$\mathbf{L}_{xy}(t)$ as a continuously differentiable mapping from the time
domain to the intensity domain: $[-\frac{T}{2},\frac{T}{2}]\rightarrow[0,1]$.
Inspired by the Taylor’s theorem, we parameterize the mapping as a degree-$n$
polynomial. Let $\alpha_{0},\alpha_{1},\alpha_{2},\cdots,\alpha_{n}$ be the
$n+1$ polynomial coefficients. The simplest parameterization uses the standard
polynomial bases:

$\mathbf{L}_{xy}(t)=\alpha_{0}+\alpha_{1}t+\alpha_{2}t^{2}+\cdots+\alpha_{n}t^{n}$
(4)

Most real-life videos do not have frequent oscillations. This means a
realistic intensity curve does not have very high-order derivative
information. For a small change in $t$, the variation in $\mathbf{L}_{xy}(t)$
should not be too big. The fact that
$|\alpha_{0}|\approx|\alpha_{1}|>|\alpha_{2}|>\cdots\gg|\alpha_{n}|$ suggests
that the standard representation is prone to numerical issues since high-order
coefficients are expected to be very close to zero.

The temporal derivative of $\mathbf{L}_{xy}(t)$ reveals how the intensity
changes across time. The events associated with pixel $(x,y)$ provide a set of
timestamps where the intensity change considerably. The derivatives at these
timestamps are expected to have significant magnitudes. The key idea of our
proposed parameterization is to interpolate the temporal derivative of the
intensity signal at event timestamps. The number of events associated with
each pixel is different, presenting a challenge to efficient computation. To
address this issue, we extract a fixed number of $n$ keypoints for each pixel,
regardless of how many events the pixel initially possesses. The details of
our keypoint extraction algorithm are presented in Figure 2(a). This algorithm
ensures the selected keypoints are in correspondence to the event timestamps
and as distant to each other as possible. The use of the uniform pivots
further establishes spatial consistency in the keypoint choices among
different pixels. Let the set of $n$ keypoints for pixel $(x,y)$ be:

$\mathcal{K}_{xy}=\\{(t_{i},\frac{d\mathbf{L}_{xy}}{dt}(t_{i}))|1\leq i\leq
n\\}$ (5)

where $-\frac{T}{2}\leq t_{1}<t_{2}<\dots<t_{n}\leq\frac{T}{2}$. As shown in
Figure 2(b), we parameterize the intensity derivative
$\frac{d\mathbf{L}_{xy}}{dt}(t)$ as the polynomial interpolation of these $n$
keypoints:

$\frac{d\mathbf{L}_{xy}}{dt}(t)=\sum_{i=1}^{n}\frac{d\mathbf{L}_{xy}}{dt}(t_{i})\cdot\mathbf{\beta}_{xyi}(t)$
(6)

Here, $\mathbf{\beta}_{xyi}(t),1\leq i\leq n$ are the Lagrange bases of degree
$n-1$:

$\displaystyle\mathbf{\beta}_{xyi}(t)$
$\displaystyle=\frac{(t-t_{1})\cdots(t-t_{i-1})(t-t_{i+1})\cdots(t-t_{n})}{(t_{i}-t_{1})\cdots(t_{i}-t_{i-1})(t_{i}-t_{i+1})\cdots(t_{i}-t_{n})}$

The Lagrange bases have the characteristic that
$\mathbf{\beta}_{xyi}(t_{i})=1$ and $\forall j\neq
i,\mathbf{\beta}_{xyj}(t_{i})=0$. This ensures the continuous function
$\frac{d\mathbf{L}_{xy}}{dt}(t)$ passes through the $n$ discrete keypoints
$(t_{i},\frac{d\mathbf{L}_{xy}}{dt}(t_{i}))$, where $1\leq i\leq n$.

We can then recover the primitive intensity signal $\mathbf{L}_{xy}(t)$ from
its derivative $\frac{d\mathbf{L}_{xy}}{dt}(t)$ by taking the indefinite
integral:

$\mathbf{L}_{xy}(t)=\int\frac{d\mathbf{L}_{xy}}{dt}(t)dt+a_{xy}$ (7)

where $a_{xy}$ is a constant that can be solved from Equation (3).

Compared to the conventional frame-based representation, the main advantage of
the proposed polynomial representation is that it closely mimics the event
mechanism. We leverage event timestamps to construct the polynomial bases and
allow the polynomial coefficients to be interpreted as the intensity changes
that trigger the input events. The regression target of our model, the
polynomial coefficients, is therefore highly correlated to the input events.
While the input events characterize the locations of the edge features, the
output polynomial coefficients reveal exactly how significant the edges are.
Section 5.4 presents an empirical verification of the advantage of our
representation.

###  4.2 Prediction Pipeline

We illustrate the overall prediction pipeline in Figure 3, which consists of
the initialization stage and the refinement stage. The initialization stage
regresses the polynomial coefficients, evaluates predicted parametric
functions, and obtains coarse video reconstruction. The refinement stage
further polishes the details in the initial reconstruction by learning and
enforcing motion priors. This methodology nicely combines the strength of
motion priors for recovering temporally smooth videos and the strength of
regressing the volumetric output to avoid drifting.

Initialization: Polynomial Coefficient Regression. We assemble the $n$
keypoint timestamps associated with each pixel into an $n\times h\times w$
tensor $\mathbf{K}$, where $h\times w$ is the spatial frame resolution.
Following eSL-Net [40], we voxelize the event stream by creating an $m\times
h\times w$ histogram tensor $\mathbf{E}$, where $m=40$ is the number of
temporal bins. We adopt the U-Net [34] architecture as the backbone prediction
network. As shown in Figure 3, the U-Net takes the blurry frame, the
keypoints, and the events as input and regresses the polynomial coefficients
for $\mathbf{L}_{xy}(t)$ represented under the Lagrange bases of its
derivative. Let $\mathbf{t}=\\{t_{i}\\}$ be the $d$-dimensional vector
collecting all timestamps of interest. With the help of Equation (7), the
predicted coefficients allow us to reconstruct $d$ initial frames
$\\{\mathbf{\hat{L}}(t_{i})\\}$. At training time, $d=14$ is set to the number
of available ground-truth latent frames. At inference time, $d$ can be an
arbitrary positive integer.

Algorithm 1 Refinement

Input: Initial reconstruction: $\\{\mathbf{\hat{L}}(t_{i})|1\leq i\leq d\\}$
Input: All events: $\\{(x,y,t,p)|-\frac{T}{2}\leq t\leq\frac{T}{2}\\}$
Output: Final reconstruction: $\\{\mathbf{L}(t_{i})|1\leq i\leq d\\}$

1:loop $I_{max}$ iterations

2: for $i=1\textbf{ to }d-1$ do $\triangleright$ Residual Prediction

3: $\mathbf{E}\leftarrow$ Voxelize(events from $t_{i}$ to $t_{i+1})$

4: if i == 1 then

5: $\mathbf{R}_{i}\leftarrow
g_{\theta_{1}}^{\mathbf{R}}(\mathbf{E},\mathbf{\hat{L}}(t_{i}),\mathbf{\hat{L}}(t_{i+1}),$

6: $\nabla\mathbf{\hat{L}}(t_{i}),\nabla\mathbf{\hat{L}}(t_{i+1}))$

7: else

8: $\mathbf{R}_{i}\leftarrow
g_{\theta_{2}}^{\mathbf{R}}(\mathbf{R}_{i-1},\mathbf{E},\mathbf{\hat{L}}(t_{i}),\mathbf{\hat{L}}(t_{i+1}),$

9: $\nabla\mathbf{\hat{L}}(t_{i}),\nabla\mathbf{\hat{L}}(t_{i+1}))$

10: end if

11: end for

12: for $i=1\textbf{ to }d$ do $\triangleright$ Apply Updates

13: $\mathbf{A_{i}}\leftarrow g_{\phi}^{\mathbf{A}}(\mathbf{\hat{L}}(t_{i}))$

14: $\mathbf{D_{i}}\leftarrow\frac{\partial
f(\mathbf{R}_{1},\cdots,\mathbf{R}_{d-1},\mathbf{\hat{L}}(t_{1}),\cdots,\mathbf{\hat{L}}(t_{d}),\mathbf{L}(t_{1}),\cdots,\mathbf{L}(t_{d}))}{\partial\mathbf{L}(t_{i})}$

15:
$\mathbf{\hat{L}}(t_{i})\leftarrow\mathbf{\hat{L}}(t_{i})-\mathbf{A_{i}}\odot\mathbf{D_{i}}$

16: end for

17:end loop

18:for $i=1\textbf{ to }d$ do $\triangleright$ Final Polishing

19: $\mathbf{L}(t_{i})\leftarrow
g_{\gamma}^{\mathbf{L}}(\mathbf{\hat{L}}(t_{i}))$

20:end for

Refinement: Temporal Feature Propagation. When played as a video, the
initialization results show an accurate motion reconstruction. Different parts
of the scene move according to their respectively trajectories. Nonetheless,
we observe that the initial reconstruction occasionally fails to recover
temporally consistent features. This initial temporal inconsistency is
expected and addressed by the refinement.

There are two factors that contribute to the temporal inconsistency. First,
the polynomial function is a continuous signal that smooths sharp features and
makes them visually blurry. Second, visual features may only move during a
small part of the exposure interval. When the feature is actively moving, the
reconstruction is usually sharp because there are input events in the spatial
neighborhood depicting the edge locations. When the feature is not moving,
however, the reconstruction becomes blurry due to the lack of associated
events and difficulty for volumetric filters used in regression to capture
motion priors that are critical for deblurring.

In the refinement stage, we solve the first issue by optimizing the frames
independent of polynomial formulation. We solve the second issue by
encouraging visual features to propagate between consecutive frames (i.e.,
enforcing motion priors), a popular technique used extensively in video
synthesis via optical flows [41, 3], residuals [7, 18], or deformable
convolutional kernels [37, 38]. Details of our refinement process are
presented as Algorithm 1. Specifically, we use a recurrent network to predict
the residual $\mathbf{R}_{i}$ between adjacent frames $\mathbf{L}(t_{i})$ and
$\mathbf{L}(t_{i+1})$. The inputs to the network include the previous residual
$\mathbf{R}_{i-1}$ between frames $\mathbf{L}(t_{i-1})$ and
$\mathbf{L}(t_{i})$, the events from $t_{i}$ to $t_{i+1}$, the initial
reconstructions $\mathbf{\hat{L}}(t_{i})$ and $\mathbf{\hat{L}}(t_{i+1})$, as
well as their spatial gradients $\nabla\mathbf{\hat{L}}(t_{i})$ and
$\nabla\mathbf{\hat{L}}(t_{i+1})$. Algorithm 1 refers this recurrent network
as $g_{\theta_{1}}^{\mathbf{R}}$ (for the residual prediction between the
first two frames) and $g_{\theta_{2}}^{\mathbf{R}}$ (for the rest of
residuals).

The recurrent architecture allows the residual to be gradually updated
according to the relevant events and intensity reconstruction. We augment the
inputs to include
$\nabla\mathbf{\hat{L}}(t_{i})=(\frac{d\mathbf{\hat{L}}(t_{i})}{dx},\frac{d\mathbf{\hat{L}}(t_{i})}{dy})$
and
$\nabla\mathbf{\hat{L}}(t_{i+1})=(\frac{d\mathbf{\hat{L}}(t_{i+1})}{dx},\frac{d\mathbf{\hat{L}}(t_{i+1})}{dy})$.
This is because both the spatial gradients and the temporal residuals are
highly related to the edge features.

Consider the objective function in Equation (8), where
$\mathcal{L}(\cdot,\cdot)$ represents the distance between two matrices.

$f=\sum_{i=1}^{d-1}\mathcal{L}(\mathbf{L}(t_{i})+\mathbf{R}_{i},\mathbf{L}(t_{i+1}))+\lambda\sum_{i=1}^{d}\mathcal{L}(\mathbf{L}(t_{i}),\mathbf{\hat{L}}(t_{i}))$
(8)

The free variables are the refined frames $\mathbf{L}(t_{i})$’s. The first
objective term ensures the refinement output follows the residual flow. The
second term discourages the refinement from deviating too far from the
initialization. The trade-off parameter $\lambda$ balances these two terms. We
expect both terms to have small residuals throughout the optimization process
and the final result to be in local proximity to the initial frames. This
leads us to choose the L2-distance as $\mathcal{L}(\cdot,\cdot)$ for its
numerical stability. As shown in Algorithm 1, we apply gradient descent to
update the refinement result for $I_{max}$ iterations. Different pixels may
have different step sizes for the update, which are predicted by a
convolutional network referred to as $g_{\phi}^{\mathbf{A}}$. After the
gradient descent terminates, we use another convolutional network
$g_{\gamma}^{\mathbf{L}}$ to perform final polishing on each individual frame.

###  4.3 Training Objective

Derivative Loss. We use $\mathcal{L}_{\text{d}}$ to supervise the output
polynomial coefficients directly in the derivative domain:

$\mathcal{L}_{\text{d}}=|(\frac{d\mathbf{L}}{dt})_{\text{gt}}-(\frac{d\mathbf{L}}{dt})_{\text{pred}}|_{1}$
(9)

Primitive Loss. We first recover the primitive intensity signal from Equation
(7) and then use $\mathcal{L}_{\text{p}}$ to supervise the polynomial
coefficients indirectly in the primitive domain:

$\mathcal{L}_{\text{p}}=|\mathbf{L}_{\text{gt}}-\mathbf{L}_{\text{pred}}|_{1}$
(10)

Refinement Loss. We use $\mathcal{L}_{\text{ref}}$ to supervise the refinement
output:

$\mathcal{L}_{\text{ref}}=\sum_{t}|\mathbf{L}_{\text{gt}}(t)-\mathbf{L}_{\text{pred}}(t)|_{1}$
(11)

Residual Loss. We use $\mathcal{L}_{\text{res}}$ to supervise the residual
prediction in the refinement stage. Note that $\mathcal{L}_{\text{res}}$ uses
weighted L1-norm with $\rho=5$ because the intensity residual between
consecutive frames is sparse.

$\mathcal{L}_{\text{res}}=\sum_{i}(exp(\rho\cdot|\mathbf{R}_{i_{\text{gt}}}|_{1})\odot|\mathbf{R}_{i_{\text{gt}}}-\mathbf{R}_{i_{\text{pred}}}|_{1})$
(12)

Total Objective. The final training objective is the weighted sum of the
losses introduced above:

$\mathcal{L}_{\text{total}}=\lambda_{\text{d}}\mathcal{L}_{\text{d}}+\lambda_{\text{p}}\mathcal{L}_{\text{p}}+\lambda_{\text{ref}}\mathcal{L}_{\text{ref}}+\lambda_{\text{res}}\mathcal{L}_{\text{res}}$
(13)

where
$\lambda_{\text{d}},\lambda_{\text{p}},\lambda_{\text{ref}},\lambda_{\text{res}}$
are constant trade-off factors. The U-Net and the prediction networks in the
refinement stage
($g_{\theta_{1}}^{\mathbf{R}},g_{\theta_{2}}^{\mathbf{R}},g_{\phi}^{\mathbf{A}},g_{\gamma}^{\mathbf{L}}$)
are trained in an end-to-end manner.

##  5 Evaluation

###  5.1 Datasets

REDS [22] is a standard deblurring benchmark dataset designed for conventional
cameras. The dataset contains 240 training videos and 30 validation videos and
is publicly available under the CC BY 4.0 license. These videos are captured
at 120 fps and are sharp and clear. We use the frame interpolation algorithm
[23] to further increase the frame rate to 960 fps. After that, we convert the
videos to grayscale and resize the frames to 240$\times$180, consistent with
the DAVIS240111DAVIS240 is a popular camera that records grayscale
conventional frames and events simultaneously. [2] sensor resolution. We apply
the ESIM [30] simulator to generate events and synthesize blurry frames
according to Equation (3) with an exposure interval of 120 milliseconds. We
use least-squares to fit polynomial coefficients to the resized 960 fps sharp
frames during each exposure interval. These coefficients are used to supervise
network training. This process is an effort to reproduce the synthetic dataset
described by Wang et al. [40], who have released the model weights after
training but have not disclosed data processing or training scripts.

###  5.2 Baseline Approaches

Methods |  MSE $\downarrow$ |  PSNR $\uparrow$ |  SSIM $\uparrow$
---|---|---|---
EDI [26] | 0.182 | 21.663 | 0.664
eSL-Net [40] (official) | 0.203 | 20.640 | 0.601
eSL-Net [40] (re-trained) | 0.201 | 20.748 | 0.646
Ours | 0.114 | 25.531 | 0.819
Table 1: Quantitative evaluation on the REDS [22] dataset.

For quantitative evaluation, we compare our model with eSL-Net [40] using both
the official weights released by Wang et al. and the weights re-trained on our
synthetic data. We take EDI [26] as an additional baseline model. At the
submission time, they are the only two approaches with open-source
implementation available online.

For qualitative evaluation, we also visualize the results on real event
captures provided by Pan et al. [26]. The blurriness of real captures comes
from the physical sensor instead of simulated temporal averaging. Therefore,
there is a lack of “ground-truth” sharp images, and we are unable to
numerically evaluate the performance of each algorithm.

Input |  Ground Truth |  EDI [26] |  eSL-Net [40] |  Ours
---|---|---|---|---

Figure 4: Qualitative visualization on the REDS [22] dataset.

Input |  EDI [26] |  eSL-Net [40] |  Ours
---|---|---|---

Figure 5: Qualitative visualization on real captures by Pan et al. [26]. Row | Input Sources | Video Format | Stages | Performance Metrics
---|---|---|---|---
Frame | Events | Poly. | Frame | Initialization | Refinement | MSE $\downarrow$ | PSNR $\uparrow$ | SSIM $\uparrow$
1 | ✗ | ✓ | ✓ | ✗ | ✓ | ✗ | 0.134 | 24.210 | 0.767
2 | ✓ | ✗ | ✓ | ✗ | ✓ | ✗ | 0.180 | 21.721 | 0.654
3 | ✓ | ✓ | ✓ | ✗ | ✓ | ✗ | 0.125 | 24.807 | 0.787
4 | ✓ | ✓ | ✗ | ✓ | ✓ | ✗ | 0.136 | 23.504 | 0.723
5 | ✓ | ✓ | ✓ | ✗ | ✓ | ✓ | 0.114 | 25.531 | 0.819
Table 2: On the REDS [22] dataset, we use ablation studies to demonstrate the
importance of using a dual-stream input, the power of the polynomial
representation, and the strength of the refinement module.

###  5.3 Training Details

The trade-off parameters for the derivative ($\mathcal{L}_{\text{d}}$),
primitive ($\mathcal{L}_{\text{p}}$), refinement ($\mathcal{L}_{\text{ref}}$),
and residual ($\mathcal{L}_{\text{res}}$) losses are 1, 10, 10, and 0.5,
respectively. We use the Adam [12] optimizer with a batch size of 96 and train
the network for 50 epochs. The initial learning rate is 0.0001 and is halved
at the end of the 20${}^{\text{th}}$ and the 40${}^{\text{th}}$ epochs. The
degree of the polynomial functions we use is $n=10$.

###  5.4 Analysis of Results

Baseline Comparison. We present a quantitative evaluation of our approach in
Table 1. Compared to baseline models, our method obtains lower MSE, higher
PSNR, and higher SSIM. Specifically, the proposed E-CIR improves the current
state-of-the-art algorithm by 37.4% in MSE, 17.8% in PSNR, and 23.3% in SSIM.

Qualitatively, we compare E-CIR with baseline models in Figure 4 and Figure 5.
Visually, our results are not only sharper but also less noisy than the EDI
[25] reconstruction. The blurriness of EDI can be explained by their use of
total variation as an image quality prior, which penalizes spatial edge
features. By contrast, our method implicitly learns a deep prior from the
sharp images in the training dataset. In terms of the noises, EDI assumes the
event threshold can precisely characterize all intensity changes. As discussed
in Section 3.3, this assumption is inaccurate and susceptible to artifacts.
Compared to eSL-Net [40], our model does not over exaggerate visual features.
As shown in the second row of Figure 4, the eSL-Net output contains a
distorted patch in the bottom-right corner, an overemphasis of the floor tile.
The excessive amount of noise makes eSL-Net unfavorable in the quantitative
evaluation, even though subjectively, it reconstructs sharper frames than EDI.
We also invite readers to watch our supplementary animations. These animations
demonstrate that our method generates temporally smooth reconstruction, while
the baseline approaches suffer from trajectory discontinuity to varying
degrees.

Ablation Study. The input to our model has two components: the blurry frame
and the events. Prior works have approached the video reconstruction problem
using both the blurry frame alone [9, 29] and the events alone [31, 32, 4]. We
begin our ablation study by examining the benefit of using a dual-stream
input. As shown the first three rows in Table 2, removing either input from
the pipeline leads to noticeable performance degradation. For example, the MSE
of the dual-input model is 6.7% lower than the event-only model and 30.6%
lower than the frame-only model. This suggests that conventional frames and
events are complementary to each other, and our proposed E-CIR is able to take
collaborative advantage of the combined information.

To examine if the polynomial video representation is indeed superior to the
traditional frame-based representation, we use the same U-Net architecture to
regress the $d=14$ ground-truth sharp frames directly. The comparison between
the third and fourth rows in Table 2 shows that our proposed polynomial
representation outperforms the frame-based baseline by 8.1% in MSE. This
result demonstrates power of explicit derivative modeling in event data.

Finally, we examine the effectiveness of refinement. Between the third and the
fifth row in Table 2, we observe that the refinement module improves the
initialization by 8.8% in MSE. The supplementary material includes an animated
comparison between the initialization and refinement frames. While the
initialization results successfully recovers the motion trajectory, the visual
features are occasionally not sharp enough. The refinement algorithm is able
to sharpen the initial results by encouraging high-quality visual features to
propagate among consecutive frames.

##  6 Limitations

We point out that standard image quality metrics have a negative bias towards
approaches that generate sharp but noisy results, such as eSL-Net [40].
Ideally, we would like to separate the visual signal and the noise and measure
them independently. Practically, we have to resort to distance-based metrics
without the disentanglement. Readers are encouraged to compare our approach
with the baselines visually while using the quantitative evaluation as a
reference.

##  7 Conclusion

This paper introduces E-CIR, a novel event-enhanced deblurring approach that
represents the intensity signal as a continuous parametric function.
Experiments show that E-CIR outperforms current state-of-the-art models in
reconstruction quality. In the future, we plan to extend the approach and
integrate spatial continuity into the formulation. Another possible direction
is to explore probabilistic inference since each blurry image corresponds to
more than one possible realistic motion trajectory.

##  8 Acknowledgement

We would like to sincerely appreciate Jiajia Yu (RPI), Jiacheng Chen (SFU),
Zhenpei Yang (UT Austin), Bo Sun (UT Austin), Siming Yan (UT Austin), Haitao
Yang (UT Austin), Jiaru Song (UT Austin), Yan He (Rice), and Ao Li (Yext) for
their constructive criticism and valuable suggestions. Additionally, Qixing
Huang acknowledges the support from NSF Career IIS-2047677 and NSF HDR
TRIPODS1934932. Chandrajit Bajaj acknowledges the support from the NIH
(DK129979), in part from the Peter O’Donnell Foundation, and in part from a
grant from the Army Research Office accomplished under Cooperative Agreement
Number W911NF-19-2-0333.

##  9 Supplementary Material

The source code of E-CIR, scripts and step-by-step instructions to converting
the REDS [22] dataset to the event format are available at
https://github.com/chensong1995/E-CIR. Additional animations can be viewed at
http://songc.me/ecir/visualization.html. The static version of these
additional results are presented in the remaining pages of this .pdf file.

##  10 Potential Negative Societal Impact

In our experiments, we train E-CIR using three Tesla V100-SXM2-32GB GPUs for
approximately 100 hours. The total emission is estimated to be 38.88 kgCO2eq,
equivalent to 157.2 km driven by an average car. This estimation is conducted
using the Machine Learning Impact calculator presented in [16]. To mitigate
repetitive labor and negative environmental impact in future research, we have
released our open-source implementation together with trained network weights.

Input |  Ground Truth |  EDI [26] |  eSL-Net [40] |  Ours
---|---|---|---|---

Figure 6: Qualitative visualization on the REDS [22] dataset.

Input |  Ground Truth |  EDI [26] |  eSL-Net [40] |  Ours
---|---|---|---|---

Figure 7: Qualitative visualization on the REDS [22] dataset.

Input |  Ground Truth |  EDI [26] |  eSL-Net [40] |  Ours
---|---|---|---|---

Figure 8: Qualitative visualization on the REDS [22] dataset.

Input |  EDI [26] |  eSL-Net [40] |  Ours
---|---|---|---

Figure 9: Qualitative visualization on real captures by Pan et al. [26].

## References

  * [1] S Derin Babacan, Rafael Molina, Minh N Do, and Aggelos K Katsaggelos.  Bayesian blind deconvolution with general sparse image priors.  In European conference on computer vision, pages 341–355. Springer, 2012.
  * [2] Christian Brandli, Raphael Berner, Minhao Yang, Shih-Chii Liu, and Tobi Delbruck.  A 240 × 180 130 db 3 µs latency global shutter spatiotemporal vision sensor.  IEEE Journal of Solid-State Circuits, 49(10):2333–2341, 2014.
  * [3] Jose Caballero, Christian Ledig, Andrew Aitken, Alejandro Acosta, Johannes Totz, Zehan Wang, and Wenzhe Shi.  Real-time video super-resolution with spatio-temporal networks and motion compensation.  In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 4778–4787, 2017.
  * [4] Pablo Rodrigo Gantier Cadena, Yeqiang Qian, Chunxiang Wang, and Ming Yang.  Spade-e2vid: Spatially-adaptive denormalization for event-based video reconstruction.  IEEE Transactions on Image Processing, 30:2488–2500, 2021.
  * [5] Rob Fergus, Barun Singh, Aaron Hertzmann, Sam T Roweis, and William T Freeman.  Removing camera shake from a single photograph.  In ACM SIGGRAPH 2006 Papers, pages 787–794, 2006.
  * [6] DA Fish, AM Brinicombe, ER Pike, and JG Walker.  Blind deconvolution by means of the richardson–lucy algorithm.  JOSA A, 12(1):58–65, 1995.
  * [7] Jin Han, Yixin Yang, Chu Zhou, Chao Xu, and Boxin Shi.  Evintsr-net: Event guided multiple latent frames reconstruction and super-resolution.  In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 4882–4891, 2021.
  * [8] Zhe Jiang, Yu Zhang, Dongqing Zou, Jimmy Ren, Jiancheng Lv, and Yebin Liu.  Learning event-based motion deblurring.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3320–3329, 2020.
  * [9] Meiguang Jin, Givi Meishvili, and Paolo Favaro.  Learning to extract a video sequence from a single motion-blurred image.  In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 6334–6342, 2018.
  * [10] Neel Joshi, C Lawrence Zitnick, Richard Szeliski, and David J Kriegman.  Image deblurring and denoising using color priors.  In 2009 IEEE Conference on Computer Vision and Pattern Recognition, pages 1550–1557. IEEE, 2009.
  * [11] Sang Ku Kim, Sang Rae Park, and Joon Ki Paik.  Simultaneous out-of-focus blur estimation and restoration for digital auto-focusing system.  IEEE Transactions on Consumer Electronics, 44(3):1071–1075, 1998\.
  * [12] Diederik P Kingma and Jimmy Ba.  Adam: A method for stochastic optimization.  arXiv preprint arXiv:1412.6980, 2014.
  * [13] Dilip Krishnan and Rob Fergus.  Fast image deconvolution using hyper-laplacian priors.  Advances in neural information processing systems, 22:1033–1041, 2009.
  * [14] Orest Kupyn, Volodymyr Budzan, Mykola Mykhailych, Dmytro Mishkin, and Jiří Matas.  Deblurgan: Blind motion deblurring using conditional adversarial networks.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 8183–8192, 2018.
  * [15] Orest Kupyn, Tetiana Martyniuk, Junru Wu, and Zhangyang Wang.  Deblurgan-v2: Deblurring (orders-of-magnitude) faster and better.  In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 8878–8887, 2019.
  * [16] Alexandre Lacoste, Alexandra Luccioni, Victor Schmidt, and Thomas Dandres.  Quantifying the carbon emissions of machine learning.  arXiv preprint arXiv:1910.09700, 2019.
  * [17] Anat Levin, Rob Fergus, Frédo Durand, and William T Freeman.  Image and depth from a conventional camera with a coded aperture.  ACM transactions on graphics (TOG), 26(3):70–es, 2007.
  * [18] Sheng Li, Fengxiang He, Bo Du, Lefei Zhang, Yonghao Xu, and Dacheng Tao.  Fast spatio-temporal residual network for video super-resolution.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10522–10531, 2019.
  * [19] P. Lichtsteiner, C. Posch, and T. Delbruck.  A 128 x 128 120db 30mw asynchronous vision sensor that responds to relative intensity change.  In 2006 IEEE International Solid State Circuits Conference - Digest of Technical Papers, pages 2060–2069, 2006.
  * [20] Patrick Lichtsteiner, Christoph Posch, and Tobi Delbruck.  A 128$\times$ 128 120 db 15 $\mu$s latency asynchronous temporal contrast vision sensor.  IEEE Journal of Solid-State Circuits, 43(2):566–576, 2008.
  * [21] Songnan Lin, Jiawei Zhang, Jinshan Pan, Zhe Jiang, Dongqing Zou, Yongtian Wang, Jing Chen, and Jimmy Ren.  Learning event-driven video deblurring and interpolation.  In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part VIII 16, pages 695–710. Springer, 2020.
  * [22] Seungjun Nah, Sungyong Baik, Seokil Hong, Gyeongsik Moon, Sanghyun Son, Radu Timofte, and Kyoung Mu Lee.  Ntire 2019 challenge on video deblurring and super-resolution: Dataset and study.  In CVPR Workshops, June 2019.
  * [23] Simon Niklaus, Long Mai, and Feng Liu.  Video frame interpolation via adaptive separable convolution.  In Proceedings of the IEEE International Conference on Computer Vision, pages 261–270, 2017.
  * [24] Genady Paikin, Yotam Ater, Roy Shaul, and Evgeny Soloveichik.  Efi-net: Video frame interpolation from fusion of events and frames.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, pages 1291–1301, June 2021.
  * [25] Liyuan Pan, Richard Hartley, Cedric Scheerlinck, Miaomiao Liu, Xin Yu, and Yuchao Dai.  High frame rate video reconstruction based on an event camera.  IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020\.
  * [26] Liyuan Pan, Cedric Scheerlinck, Xin Yu, Richard Hartley, Miaomiao Liu, and Yuchao Dai.  Bringing a blurry frame alive at high frame-rate with an event camera.  In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 6820–6829, 2019.
  * [27] Sunghyun Park, Kangyeol Kim, Junsoo Lee, Jaegul Choo, Joonseok Lee, Sookyung Kim, and Edward Choi.  Vid-ode: Continuous-time video generation with neural ordinary differential equation.  arXiv preprint arXiv:2010.08188, 2020.
  * [28] Daniele Perrone and Paolo Favaro.  Total variation blind deconvolution: The devil is in the details.  In 2014 IEEE Conference on Computer Vision and Pattern Recognition, pages 2909–2916, 2014.
  * [29] Kuldeep Purohit, Anshul Shah, and AN Rajagopalan.  Bringing alive blurred moments.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6830–6839, 2019.
  * [30] Henri Rebecq, Daniel Gehrig, and Davide Scaramuzza.  ESIM: an open event camera simulator.  Conf. on Robotics Learning (CoRL), Oct. 2018.
  * [31] Henri Rebecq, René Ranftl, Vladlen Koltun, and Davide Scaramuzza.  Events-to-video: Bringing modern computer vision to event cameras.  IEEE Conf. Comput. Vis. Pattern Recog. (CVPR), 2019.
  * [32] Henri Rebecq, René Ranftl, Vladlen Koltun, and Davide Scaramuzza.  High speed and high dynamic range video with an event camera.  IEEE Trans. Pattern Anal. Mach. Intell. (T-PAMI), 2019.
  * [33] William Hadley Richardson.  Bayesian-based iterative method of image restoration.  JoSA, 62(1):55–59, 1972.
  * [34] Olaf Ronneberger, Philipp Fischer, and Thomas Brox.  U-net: Convolutional networks for biomedical image segmentation.  In International Conference on Medical image computing and computer-assisted intervention, pages 234–241. Springer, 2015.
  * [35] Qi Shan, Jiaya Jia, and Aseem Agarwala.  High-quality motion deblurring from a single image.  Acm transactions on graphics (tog), 27(3):1–10, 2008.
  * [36] Wei Shang, Dongwei Ren, Dongqing Zou, Jimmy S Ren, Ping Luo, and Wangmeng Zuo.  Bringing events into video deblurring with non-consecutively blurry frames.  In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 4531–4540, 2021.
  * [37] Yapeng Tian, Yulun Zhang, Yun Fu, and Chenliang Xu.  Tdan: Temporally deformable alignment network for video super-resolution.  arXiv preprint arXiv:1812.02898, 2018.
  * [38] Yapeng Tian, Yulun Zhang, Yun Fu, and Chenliang Xu.  Tdan: Temporally-deformable alignment network for video super-resolution.  In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2020.
  * [39] Stepan Tulyakov, Daniel Gehrig, Stamatios Georgoulis, Julius Erbach, Mathias Gehrig, Yuanyou Li, and Davide Scaramuzza.  Time lens: Event-based video frame interpolation.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16155–16164, 2021.
  * [40] Bishan Wang, Jingwei He, Lei Yu, Gui-Song Xia, and Wen Yang.  Event enhanced high-quality image recovery.  In European Conference on Computer Vision. Springer, 2020.
  * [41] Fang Xu, Lei Yu, Bishan Wang, Wen Yang, Gui-Song Xia, Xu Jia, Zhendong Qiao, and Jianzhuang Liu.  Motion deblurring with real events.  In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 2583–2592, 2021.
  * [42] Li Xu and Jiaya Jia.  Two-phase kernel estimation for robust motion deblurring.  In European conference on computer vision, pages 157–170. Springer, 2010.
  * [43] Li Xu, Shicheng Zheng, and Jiaya Jia.  Unnatural l0 sparse representation for natural image deblurring.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1107–1114, 2013.
  * [44] Limeng Zhang, Hongguang Zhang, Chenyang Zhu, Shasha Guo, Jihua Chen, and Lei Wang.  Fine-grained video deblurring with event camera.  In International Conference on Multimedia Modeling, pages 352–364. Springer, 2021.